2024-03-08T23:29:15.883 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-08T23:29:15.883 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315 --name rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi --owner scheduled_rishabh@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/orig.config.yaml 2024-03-08T23:29:15.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T23:29:16.038 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315 branch: rishabh-2024mar8 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7588315' kernel: client: sha1: distro flavor: default kdb: true sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-2024mar8 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 7d63482aa89628d733e298984a762389b625c761 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 7d63482aa89628d733e298984a762389b625c761 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2024mar8 sha1: 7d63482aa89628d733e298984a762389b625c761 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1249 sha1: 7d63482aa89628d733e298984a762389b625c761 sleep_before_teardown: 0 subset: 481/512 suite: fs suite_branch: rishabh-2024mar8 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7d63482aa89628d733e298984a762389b625c761 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKeRhSrqUtiBx0tIulRULBWf1qe+zwxyeRi5klsKSZkdMBEZs/XtFC9X3HHMlugG1+oRPCHTjsiwCVgWNxMqb2A= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFg0qYPoLRelSD2a2DmIYAAnA9i6+e1DVoyu9tiDKDKrY7RKGwofNxjRi5QJqKNR5JuKkk+rDs8fXTcjBwAo1W8= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH4CG47NywenjtTdSmMJsbctV7eNJoUg0vtQhLUDv+tYVoGdahAsLLd1ncxmLzSVY1cYMZ8hjEzaehZk1EU3ZI8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-08_19:54:47 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-08T23:29:16.038 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa; will attempt to use it 2024-03-08T23:29:16.039 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks 2024-03-08T23:29:16.039 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-08T23:29:16.040 INFO:teuthology.task.internal:Checking packages... 2024-03-08T23:29:16.062 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7d63482aa89628d733e298984a762389b625c761' 2024-03-08T23:29:16.063 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-08T23:29:16.063 INFO:teuthology.packaging:ref: None 2024-03-08T23:29:16.063 INFO:teuthology.packaging:tag: None 2024-03-08T23:29:16.063 INFO:teuthology.packaging:branch: rishabh-2024mar8 2024-03-08T23:29:16.063 INFO:teuthology.packaging:sha1: 7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:29:16.063 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-2024mar8 2024-03-08T23:29:16.278 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1965.g7d63482a 2024-03-08T23:29:16.280 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-08T23:29:16.286 INFO:teuthology.task.internal:no buildpackages task found 2024-03-08T23:29:16.287 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-08T23:29:16.347 INFO:teuthology.task.internal:Saving configuration 2024-03-08T23:29:16.370 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-08T23:29:16.377 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T23:29:16.403 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 23:24:19.797767', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKeRhSrqUtiBx0tIulRULBWf1qe+zwxyeRi5klsKSZkdMBEZs/XtFC9X3HHMlugG1+oRPCHTjsiwCVgWNxMqb2A='} 2024-03-08T23:29:16.426 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 23:24:19.794913', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFg0qYPoLRelSD2a2DmIYAAnA9i6+e1DVoyu9tiDKDKrY7RKGwofNxjRi5QJqKNR5JuKkk+rDs8fXTcjBwAo1W8='} 2024-03-08T23:29:16.449 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 23:24:19.796982', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH4CG47NywenjtTdSmMJsbctV7eNJoUg0vtQhLUDv+tYVoGdahAsLLd1ncxmLzSVY1cYMZ8hjEzaehZk1EU3ZI8='} 2024-03-08T23:29:16.449 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-08T23:29:16.456 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-08T23:29:16.456 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-08T23:29:16.456 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-08T23:29:16.456 INFO:teuthology.run_tasks:Running task console_log... 2024-03-08T23:29:16.569 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3073b260d0>, signals=[15]) 2024-03-08T23:29:16.569 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-08T23:29:16.575 INFO:teuthology.task.internal:Opening connections... 2024-03-08T23:29:16.575 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-03-08T23:29:16.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:29:16.684 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-03-08T23:29:16.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:29:16.787 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-03-08T23:29:16.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:29:16.873 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-08T23:29:16.880 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-03-08T23:29:16.899 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-03-08T23:29:16.899 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T23:29:16.957 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-03-08T23:29:16.958 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-03-08T23:29:16.958 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T23:29:16.958 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-03-08T23:29:16.958 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T23:29:16.958 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T23:29:16.958 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T23:29:16.959 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-03-08T23:29:16.979 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-03-08T23:29:16.996 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-03-08T23:29:16.997 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-03-08T23:29:17.057 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-03-08T23:29:17.057 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-03-08T23:29:17.057 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T23:29:17.058 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T23:29:17.058 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-03-08T23:29:17.079 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-03-08T23:29:17.097 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-03-08T23:29:17.097 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-03-08T23:29:17.155 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T23:29:17.156 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T23:29:17.157 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-03-08T23:29:17.179 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-08T23:29:17.197 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-08T23:29:17.213 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-08T23:29:17.213 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-03-08T23:29:17.218 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-03-08T23:29:17.221 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-03-08T23:29:17.238 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-08T23:29:17.246 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-08T23:29:17.247 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-03-08T23:29:17.275 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-03-08T23:29:17.282 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-03-08T23:29:17.306 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-08T23:29:17.383 INFO:teuthology.run_tasks:Running task kernel... 2024-03-08T23:29:17.400 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412'} 2024-03-08T23:29:17.400 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-08T23:29:17.400 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-08T23:29:17.401 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T23:29:17.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T23:29:17.418 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-03-08T23:29:17.476 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-361.el9.x86_64 2024-03-08T23:29:17.476 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.14.0-361.el9.x86_64 2024-03-08T23:29:17.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2024-03-08T23:29:19.581 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-08T23:29:21.073 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-08T23:29:22.930 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-03-08T23:29:27.423 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-03-08T23:29:31.473 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-08T23:29:32.924 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 15 kB 00:00 2024-03-08T23:29:33.903 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-08T23:29:39.201 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 6.0 kB/s | 1.7 kB 00:00 2024-03-08T23:29:41.504 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-08T23:29:41.504 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-08T23:29:41.602 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-08T23:29:41.613 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-08T23:29:41.614 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:29:41.614 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-03-08T23:29:41.614 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-03-08T23:29:41.614 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-03-08T23:29:41.614 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:29:41.614 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 100 M 2024-03-08T23:29:41.615 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 123 M 2024-03-08T23:29:41.615 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-03-08T23:29:56.600 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 411 kB/s | 6.0 MB 00:14 2024-03-08T23:30:26.922 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 472 kB/s | 21 MB 00:45 2024-03-08T23:31:09.749 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 468 kB/s | 33 MB 01:13 2024-03-08T23:31:17.665 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 419 kB/s | 39 MB 01:35 2024-03-08T23:31:17.665 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-03-08T23:31:17.665 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.0 MB/s | 100 MB 01:36 2024-03-08T23:31:18.583 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-03-08T23:31:19.033 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-03-08T23:31:19.033 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-03-08T23:31:20.439 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-03-08T23:31:20.439 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-03-08T23:31:23.323 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-03-08T23:31:23.996 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-08T23:31:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T23:31:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T23:31:27.237 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T23:31:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T23:31:32.457 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T23:31:37.578 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T23:32:08.335 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T23:32:08.336 INFO:teuthology.orchestra.run.smithi022.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T23:32:08.336 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:32:08.421 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-08T23:32:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T23:32:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-08T23:32:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T23:32:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:32:09.325 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-03-08T23:32:09.834 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T23:32:10.025 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T23:32:10.025 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T23:32:10.025 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-08T23:32:10.026 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-08T23:32:10.026 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-08T23:32:10.026 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-08T23:32:10.026 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-03-08T23:32:10.026 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T23:32:10.152 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T23:32:10.153 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T23:32:10.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 68.el9 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8264254 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-03-08T23:32:10.200 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T23:32:10.201 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T23:32:10.202 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-03-08T23:32:10.202 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:32:10.202 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T23:32:10.203 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T23:32:10.203 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 5.14.0-427.el9.x86_64 2024-03-08T23:32:10.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T23:32:10.745 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-03-08T23:32:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T23:32:12.190 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-03-08T23:32:12.192 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-03-08T23:32:12.209 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.JGi2Do9SkK 2024-03-08T23:32:12.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JGi2Do9SkK 2024-03-08T23:32:12.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.JGi2Do9SkK 2024-03-08T23:32:12.437 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.JGi2Do9SkK is 6KB 2024-03-08T23:32:12.486 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.JGi2Do9SkK 2024-03-08T23:32:12.501 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T23:32:12.570 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T23:32:12.570 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T23:32:12.570 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T23:32:12.570 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-08T23:32:12.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-08T23:32:12.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-03-08T23:32:42.745 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-08T23:32:42.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-08T23:32:42.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:33:01.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-03-08T23:33:10.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-08T23:33:10.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:34:10.095 DEBUG:teuthology.orchestra.remote:timed out 2024-03-08T23:34:22.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-08T23:34:22.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:34:22.459 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-03-08T23:34:22.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-08T23:34:22.849 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-08T23:34:22.850 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-03-08T23:34:22.866 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-427.el9.x86_64 2024-03-08T23:34:22.866 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-08T23:34:22.866 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T23:34:22.866 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T23:34:23.867 DEBUG:teuthology.parallel:result is None 2024-03-08T23:34:23.868 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-08T23:34:23.875 INFO:teuthology.task.internal:Creating test directory... 2024-03-08T23:34:23.876 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T23:34:23.878 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T23:34:23.881 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T23:34:23.899 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-08T23:34:23.905 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-08T23:34:23.918 INFO:teuthology.task.internal:Creating archive directory... 2024-03-08T23:34:23.918 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T23:34:23.940 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T23:34:23.942 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T23:34:23.982 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-08T23:34:23.988 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-08T23:34:23.989 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T23:34:24.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T23:34:24.021 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T23:34:24.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T23:34:24.037 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T23:34:24.040 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T23:34:24.063 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T23:34:24.119 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T23:34:24.124 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T23:34:24.135 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T23:34:24.149 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T23:34:24.155 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T23:34:24.158 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T23:34:24.159 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-08T23:34:24.167 INFO:teuthology.task.internal:Configuring sudo... 2024-03-08T23:34:24.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T23:34:24.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T23:34:24.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T23:34:24.203 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-08T23:34:24.212 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-08T23:34:24.212 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T23:34:24.243 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T23:34:24.246 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T23:34:24.262 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T23:34:24.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T23:34:24.395 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T23:34:24.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T23:34:24.527 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:34:24.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T23:34:24.597 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T23:34:24.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T23:34:24.696 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T23:34:24.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T23:34:24.826 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:34:24.827 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T23:34:24.898 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T23:34:24.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T23:34:25.005 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T23:34:25.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T23:34:25.093 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:34:25.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T23:34:25.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-03-08T23:34:25.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-03-08T23:34:25.166 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-03-08T23:34:25.207 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T23:34:25.209 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T23:34:25.249 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T23:34:25.774 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-08T23:34:25.782 INFO:teuthology.task.internal:Starting timer... 2024-03-08T23:34:25.782 INFO:teuthology.run_tasks:Running task pcp... 2024-03-08T23:34:25.792 INFO:teuthology.run_tasks:Running task selinux... 2024-03-08T23:34:25.801 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-08T23:34:25.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-03-08T23:34:25.886 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-03-08T23:34:25.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-03-08T23:34:25.941 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-03-08T23:34:25.943 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-03-08T23:34:25.990 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-03-08T23:34:25.991 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-08T23:34:25.992 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-03-08T23:34:26.012 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-03-08T23:34:26.013 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-03-08T23:34:26.035 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-03-08T23:34:26.036 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-03-08T23:34:26.058 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-03-08T23:34:26.058 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-03-08T23:34:26.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T23:34:26.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T23:34:26.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T23:34:26.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T23:34:26.119 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T23:34:26.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T23:34:26.147 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-08T23:34:26.147 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-03-08T23:34:26.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-03-08T23:34:26.213 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-03-08T23:34:26.244 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-08T23:34:26.254 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-08T23:34:26.271 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-08T23:34:26.273 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi062.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-08T23:39:58.644 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-03-08T23:39:58.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-08T23:39:58.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:39:58.736 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-03-08T23:39:58.826 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-03-08T23:39:58.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-08T23:39:58.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:39:58.929 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-03-08T23:39:59.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-08T23:39:59.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-03-08T23:39:59.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T23:39:59.093 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-03-08T23:39:59.166 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-03-08T23:39:59.167 INFO:teuthology.run_tasks:Running task clock... 2024-03-08T23:39:59.177 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-08T23:39:59.177 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T23:39:59.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T23:39:59.180 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T23:39:59.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T23:39:59.182 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T23:39:59.182 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T23:39:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T23:39:59.217 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T23:39:59.231 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T23:39:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T23:39:59.242 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T23:39:59.259 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T23:39:59.262 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-03-08T23:39:59.275 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-03-08T23:39:59.277 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-03-08T23:39:59.291 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-03-08T23:39:59.292 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-03-08T23:39:59.294 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T23:39:59.305 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-03-08T23:39:59.311 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T23:39:59.312 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T23:39:59.321 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T23:39:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T23:39:59.338 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T23:39:59.375 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-03-08T23:39:59.378 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T23:39:59.378 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-03-08T23:39:59.378 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.378 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.379 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.379 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.380 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-03-08T23:39:59.382 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-03-08T23:39:59.382 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T23:39:59.383 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-03-08T23:39:59.383 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.383 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.383 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.383 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.385 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T23:39:59.385 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-03-08T23:39:59.385 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.385 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.385 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.385 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T23:39:59.386 INFO:teuthology.run_tasks:Running task install... 2024-03-08T23:39:59.395 DEBUG:teuthology.task.install:project ceph 2024-03-08T23:39:59.395 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7d63482aa89628d733e298984a762389b625c761'}} 2024-03-08T23:39:59.395 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7d63482aa89628d733e298984a762389b625c761'} 2024-03-08T23:39:59.395 INFO:teuthology.task.install:Using flavor: default 2024-03-08T23:39:59.401 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-08T23:39:59.401 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-08T23:39:59.401 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'tag': None, 'wait_for_package': False} 2024-03-08T23:39:59.401 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:39:59.403 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'tag': None, 'wait_for_package': False} 2024-03-08T23:39:59.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:39:59.404 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'tag': None, 'wait_for_package': False} 2024-03-08T23:39:59.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:39:59.550 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/ 2024-03-08T23:39:59.551 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1965.g7d63482a 2024-03-08T23:39:59.618 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/ 2024-03-08T23:39:59.619 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1965.g7d63482a 2024-03-08T23:39:59.635 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/ 2024-03-08T23:39:59.635 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1965.g7d63482a 2024-03-08T23:39:59.667 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T23:39:59.667 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:39:59.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T23:39:59.695 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T23:39:59.696 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d63482aa89628d733e298984a762389b625c761/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T23:39:59.730 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T23:39:59.730 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:39:59.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T23:39:59.740 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T23:39:59.740 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:39:59.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T23:39:59.763 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T23:39:59.763 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d63482aa89628d733e298984a762389b625c761/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T23:39:59.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T23:39:59.768 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T23:39:59.768 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d63482aa89628d733e298984a762389b625c761/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T23:39:59.834 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T23:39:59.836 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T23:39:59.845 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T23:39:59.911 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-03-08T23:39:59.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-03-08T23:39:59.916 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T23:39:59.921 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T23:39:59.951 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-03-08T23:39:59.952 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-03-08T23:39:59.982 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-03-08T23:39:59.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-03-08T23:40:00.156 INFO:teuthology.orchestra.run.smithi022.stdout:74 files removed 2024-03-08T23:40:00.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T23:40:00.201 INFO:teuthology.orchestra.run.smithi062.stdout:74 files removed 2024-03-08T23:40:00.221 INFO:teuthology.orchestra.run.smithi192.stdout:74 files removed 2024-03-08T23:40:00.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T23:40:00.247 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T23:40:00.918 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 200 kB/s | 82 kB 00:00 2024-03-08T23:40:00.949 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 198 kB/s | 82 kB 00:00 2024-03-08T23:40:00.998 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 179 kB/s | 82 kB 00:00 2024-03-08T23:40:01.452 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-03-08T23:40:01.475 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-03-08T23:40:01.481 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 23 kB/s | 10 kB 00:00 2024-03-08T23:40:01.828 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-03-08T23:40:01.834 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-03-08T23:40:01.857 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-03-08T23:40:02.293 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-08T23:40:02.328 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-08T23:40:02.638 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 8.7 MB/s | 6.7 MB 00:00 2024-03-08T23:40:03.668 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-03-08T23:40:03.778 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-08T23:40:04.180 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-03-08T23:40:04.702 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-08T23:40:05.337 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-03-08T23:40:07.738 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-08T23:40:08.420 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 7.3 MB/s | 19 MB 00:02 2024-03-08T23:40:12.940 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.1 MB 00:01 2024-03-08T23:40:13.783 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 841 kB/s | 8.0 MB 00:09 2024-03-08T23:40:14.348 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 15 kB 00:00 2024-03-08T23:40:16.100 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-08T23:40:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 775 kB/s | 6.1 MB 00:08 2024-03-08T23:40:21.200 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-08T23:40:22.612 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2024-03-08T23:40:23.539 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-03-08T23:40:23.815 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T23:40:23.815 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T23:40:23.830 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T23:40:23.831 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T23:40:23.832 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T23:40:23.833 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T23:40:23.836 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T23:40:23.839 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T23:40:23.840 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T23:40:23.925 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-03-08T23:40:23.937 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 6.6 k 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.4 M 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 889 k 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 145 k 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 1.5 M 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 139 k 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 1.8 M 2024-03-08T23:40:23.938 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 7.4 M 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 49 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 9.8 M 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 47 M 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 264 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 748 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 220 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 27 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 18 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 31 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 745 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 127 k 2024-03-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 162 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 320 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 298 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 99 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 85 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 171 k 2024-03-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.4 M 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 20 M 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 34 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 2.1 M 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 248 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 4.7 M 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 17 M 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 16 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 25 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T23:40:23.943 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 163 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 501 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.1 M 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi022.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 45 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 131 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T23:40:23.947 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T23:40:23.948 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T23:40:23.949 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-03-08T23:40:23.950 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================ 2024-03-08T23:40:23.951 INFO:teuthology.orchestra.run.smithi022.stdout:Install 150 Packages 2024-03-08T23:40:23.951 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 24 Packages 2024-03-08T23:40:23.951 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:40:23.951 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 223 M 2024-03-08T23:40:23.951 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-03-08T23:40:24.801 INFO:teuthology.orchestra.run.smithi022.stdout:(1/174): ceph-19.0.0-1965.g7d63482a.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-03-08T23:40:25.542 INFO:teuthology.orchestra.run.smithi022.stdout:(2/174): ceph-fuse-19.0.0-1965.g7d63482a.el9.x8 1.2 MB/s | 889 kB 00:00 2024-03-08T23:40:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:(3/174): ceph-immutable-object-cache-19.0.0-196 1.3 MB/s | 145 kB 00:00 2024-03-08T23:40:26.309 INFO:teuthology.orchestra.run.smithi022.stdout:(4/174): ceph-mds-19.0.0-1965.g7d63482a.el9.x86 3.2 MB/s | 2.1 MB 00:00 2024-03-08T23:40:26.585 INFO:teuthology.orchestra.run.smithi022.stdout:(5/174): ceph-mgr-19.0.0-1965.g7d63482a.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-03-08T23:40:26.777 INFO:teuthology.orchestra.run.smithi022.stdout:(6/174): ceph-base-19.0.0-1965.g7d63482a.el9.x8 2.4 MB/s | 5.4 MB 00:02 2024-03-08T23:40:27.352 INFO:teuthology.orchestra.run.smithi022.stdout:(7/174): ceph-mon-19.0.0-1965.g7d63482a.el9.x86 6.1 MB/s | 4.7 MB 00:00 2024-03-08T23:40:28.468 INFO:teuthology.orchestra.run.smithi022.stdout:(8/174): ceph-radosgw-19.0.0-1965.g7d63482a.el9 8.8 MB/s | 9.8 MB 00:01 2024-03-08T23:40:28.527 INFO:teuthology.orchestra.run.smithi022.stdout:(9/174): ceph-selinux-19.0.0-1965.g7d63482a.el9 433 kB/s | 25 kB 00:00 2024-03-08T23:40:28.781 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-03-08T23:40:28.836 INFO:teuthology.orchestra.run.smithi022.stdout:(10/174): ceph-osd-19.0.0-1965.g7d63482a.el9.x8 8.2 MB/s | 17 MB 00:02 2024-03-08T23:40:28.903 INFO:teuthology.orchestra.run.smithi022.stdout:(11/174): cephfs-mirror-19.0.0-1965.g7d63482a.e 3.2 MB/s | 220 kB 00:00 2024-03-08T23:40:28.986 INFO:teuthology.orchestra.run.smithi022.stdout:(12/174): cephfs-shell-19.0.0-1965.g7d63482a.el 325 kB/s | 27 kB 00:00 2024-03-08T23:40:29.070 INFO:teuthology.orchestra.run.smithi022.stdout:(13/174): libcephfs-devel-19.0.0-1965.g7d63482a 374 kB/s | 31 kB 00:00 2024-03-08T23:40:29.245 INFO:teuthology.orchestra.run.smithi022.stdout:(14/174): libcephfs2-19.0.0-1965.g7d63482a.el9. 4.2 MB/s | 745 kB 00:00 2024-03-08T23:40:29.329 INFO:teuthology.orchestra.run.smithi022.stdout:(15/174): libcephsqlite-19.0.0-1965.g7d63482a.e 1.9 MB/s | 163 kB 00:00 2024-03-08T23:40:29.404 INFO:teuthology.orchestra.run.smithi022.stdout:(16/174): librados-devel-19.0.0-1965.g7d63482a. 1.7 MB/s | 127 kB 00:00 2024-03-08T23:40:29.504 INFO:teuthology.orchestra.run.smithi022.stdout:(17/174): libradosstriper1-19.0.0-1965.g7d63482 4.9 MB/s | 501 kB 00:00 2024-03-08T23:40:30.263 INFO:teuthology.orchestra.run.smithi022.stdout:(18/174): ceph-common-19.0.0-1965.g7d63482a.el9 3.5 MB/s | 20 MB 00:05 2024-03-08T23:40:30.330 INFO:teuthology.orchestra.run.smithi022.stdout:(19/174): librgw2-19.0.0-1965.g7d63482a.el9.x86 6.2 MB/s | 5.1 MB 00:00 2024-03-08T23:40:30.347 INFO:teuthology.orchestra.run.smithi022.stdout:(20/174): python3-ceph-argparse-19.0.0-1965.g7d 542 kB/s | 45 kB 00:00 2024-03-08T23:40:30.397 INFO:teuthology.orchestra.run.smithi022.stdout:(21/174): python3-ceph-common-19.0.0-1965.g7d63 1.9 MB/s | 131 kB 00:00 2024-03-08T23:40:30.438 INFO:teuthology.orchestra.run.smithi022.stdout:(22/174): python3-cephfs-19.0.0-1965.g7d63482a. 1.7 MB/s | 162 kB 00:00 2024-03-08T23:40:30.481 INFO:teuthology.orchestra.run.smithi022.stdout:(23/174): python3-rados-19.0.0-1965.g7d63482a.e 3.7 MB/s | 320 kB 00:00 2024-03-08T23:40:30.548 INFO:teuthology.orchestra.run.smithi022.stdout:(24/174): python3-rbd-19.0.0-1965.g7d63482a.el9 2.7 MB/s | 298 kB 00:00 2024-03-08T23:40:30.573 INFO:teuthology.orchestra.run.smithi022.stdout:(25/174): python3-rgw-19.0.0-1965.g7d63482a.el9 1.1 MB/s | 99 kB 00:00 2024-03-08T23:40:30.650 INFO:teuthology.orchestra.run.smithi022.stdout:(26/174): rbd-fuse-19.0.0-1965.g7d63482a.el9.x8 835 kB/s | 85 kB 00:00 2024-03-08T23:40:30.765 INFO:teuthology.orchestra.run.smithi022.stdout:(27/174): rbd-nbd-19.0.0-1965.g7d63482a.el9.x86 1.5 MB/s | 171 kB 00:00 2024-03-08T23:40:30.874 INFO:teuthology.orchestra.run.smithi022.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1965.g 313 kB/s | 34 kB 00:00 2024-03-08T23:40:30.966 INFO:teuthology.orchestra.run.smithi022.stdout:(29/174): rbd-mirror-19.0.0-1965.g7d63482a.el9. 7.9 MB/s | 3.1 MB 00:00 2024-03-08T23:40:30.991 INFO:teuthology.orchestra.run.smithi022.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1965.g7d63482 1.2 MB/s | 139 kB 00:00 2024-03-08T23:40:31.329 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T23:40:31.330 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T23:40:31.345 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T23:40:31.346 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T23:40:31.347 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T23:40:31.348 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T23:40:31.350 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T23:40:31.353 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T23:40:31.354 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T23:40:31.443 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 6.6 k 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.4 M 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 889 k 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 145 k 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 1.5 M 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 139 k 2024-03-08T23:40:31.460 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 1.8 M 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 7.4 M 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 49 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 9.8 M 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 47 M 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 264 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 748 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 220 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 27 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 18 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 31 k 2024-03-08T23:40:31.461 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 745 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 127 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 162 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 320 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 298 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 99 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 85 k 2024-03-08T23:40:31.462 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 171 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T23:40:31.463 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.4 M 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T23:40:31.464 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 20 M 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 34 k 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 2.1 M 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 248 k 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 4.7 M 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 17 M 2024-03-08T23:40:31.465 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 16 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 25 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 163 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T23:40:31.466 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 501 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.1 M 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T23:40:31.467 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 45 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 131 k 2024-03-08T23:40:31.468 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T23:40:31.469 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T23:40:31.470 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T23:40:31.471 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T23:40:31.472 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout:Install 150 Packages 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 24 Packages 2024-03-08T23:40:31.473 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:40:31.474 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 223 M 2024-03-08T23:40:31.474 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-03-08T23:40:31.491 INFO:teuthology.orchestra.run.smithi022.stdout:(31/174): ceph-test-19.0.0-1965.g7d63482a.el9.x 16 MB/s | 47 MB 00:02 2024-03-08T23:40:31.533 INFO:teuthology.orchestra.run.smithi022.stdout:(32/174): ceph-mgr-dashboard-19.0.0-1965.g7d634 3.2 MB/s | 1.8 MB 00:00 2024-03-08T23:40:31.558 INFO:teuthology.orchestra.run.smithi022.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1965.g7d 3.6 MB/s | 248 kB 00:00 2024-03-08T23:40:31.600 INFO:teuthology.orchestra.run.smithi022.stdout:(34/174): ceph-mgr-rook-19.0.0-1965.g7d63482a.e 738 kB/s | 49 kB 00:00 2024-03-08T23:40:31.626 INFO:teuthology.orchestra.run.smithi022.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1965.g7 246 kB/s | 16 kB 00:00 2024-03-08T23:40:31.676 INFO:teuthology.orchestra.run.smithi022.stdout:(36/174): ceph-volume-19.0.0-1965.g7d63482a.el9 3.4 MB/s | 264 kB 00:00 2024-03-08T23:40:31.759 INFO:teuthology.orchestra.run.smithi022.stdout:(37/174): cephadm-19.0.0-1965.g7d63482a.el9.noa 5.5 MB/s | 748 kB 00:00 2024-03-08T23:40:31.785 INFO:teuthology.orchestra.run.smithi022.stdout:(38/174): cephfs-top-19.0.0-1965.g7d63482a.el9. 168 kB/s | 18 kB 00:00 2024-03-08T23:40:31.960 INFO:teuthology.orchestra.run.smithi022.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-03-08T23:40:32.002 INFO:teuthology.orchestra.run.smithi022.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 920 kB 00:00 2024-03-08T23:40:32.018 INFO:teuthology.orchestra.run.smithi022.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-08T23:40:32.044 INFO:teuthology.orchestra.run.smithi022.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-08T23:40:32.069 INFO:teuthology.orchestra.run.smithi022.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-03-08T23:40:32.119 INFO:teuthology.orchestra.run.smithi022.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 349 kB/s | 17 kB 00:00 2024-03-08T23:40:32.178 INFO:teuthology.orchestra.run.smithi022.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-03-08T23:40:32.195 INFO:teuthology.orchestra.run.smithi022.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 136 kB/s | 10 kB 00:00 2024-03-08T23:40:32.212 INFO:teuthology.orchestra.run.smithi022.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-03-08T23:40:32.229 INFO:teuthology.orchestra.run.smithi022.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-08T23:40:32.266 INFO:teuthology.orchestra.run.smithi062.stdout:(1/174): ceph-19.0.0-1965.g7d63482a.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-03-08T23:40:32.271 INFO:teuthology.orchestra.run.smithi022.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-08T23:40:32.296 INFO:teuthology.orchestra.run.smithi022.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-03-08T23:40:32.321 INFO:teuthology.orchestra.run.smithi022.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 46 kB 00:00 2024-03-08T23:40:32.347 INFO:teuthology.orchestra.run.smithi022.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-08T23:40:32.372 INFO:teuthology.orchestra.run.smithi022.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-08T23:40:32.406 INFO:teuthology.orchestra.run.smithi022.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-08T23:40:32.431 INFO:teuthology.orchestra.run.smithi022.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 586 kB/s | 34 kB 00:00 2024-03-08T23:40:32.540 INFO:teuthology.orchestra.run.smithi022.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-08T23:40:32.582 INFO:teuthology.orchestra.run.smithi022.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-03-08T23:40:32.607 INFO:teuthology.orchestra.run.smithi022.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-08T23:40:32.632 INFO:teuthology.orchestra.run.smithi022.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-08T23:40:32.724 INFO:teuthology.orchestra.run.smithi022.stdout:(60/174): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-03-08T23:40:32.808 INFO:teuthology.orchestra.run.smithi022.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-03-08T23:40:32.841 INFO:teuthology.orchestra.run.smithi022.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-03-08T23:40:32.849 INFO:teuthology.orchestra.run.smithi062.stdout:(2/174): ceph-fuse-19.0.0-1965.g7d63482a.el9.x8 1.5 MB/s | 889 kB 00:00 2024-03-08T23:40:32.866 INFO:teuthology.orchestra.run.smithi022.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 558 kB/s | 78 kB 00:00 2024-03-08T23:40:32.975 INFO:teuthology.orchestra.run.smithi022.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-08T23:40:33.000 INFO:teuthology.orchestra.run.smithi022.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 187 kB/s | 24 kB 00:00 2024-03-08T23:40:33.024 INFO:teuthology.orchestra.run.smithi062.stdout:(3/174): ceph-immutable-object-cache-19.0.0-196 831 kB/s | 145 kB 00:00 2024-03-08T23:40:33.025 INFO:teuthology.orchestra.run.smithi022.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-08T23:40:33.051 INFO:teuthology.orchestra.run.smithi022.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-08T23:40:33.076 INFO:teuthology.orchestra.run.smithi022.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-08T23:40:33.176 INFO:teuthology.orchestra.run.smithi022.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-08T23:40:33.201 INFO:teuthology.orchestra.run.smithi022.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-08T23:40:33.226 INFO:teuthology.orchestra.run.smithi022.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-08T23:40:33.252 INFO:teuthology.orchestra.run.smithi022.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 921 kB/s | 69 kB 00:00 2024-03-08T23:40:33.277 INFO:teuthology.orchestra.run.smithi022.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 827 kB/s | 41 kB 00:00 2024-03-08T23:40:33.302 INFO:teuthology.orchestra.run.smithi022.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-08T23:40:33.327 INFO:teuthology.orchestra.run.smithi022.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-08T23:40:33.353 INFO:teuthology.orchestra.run.smithi022.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-03-08T23:40:33.378 INFO:teuthology.orchestra.run.smithi022.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-08T23:40:33.411 INFO:teuthology.orchestra.run.smithi022.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 511 kB/s | 43 kB 00:00 2024-03-08T23:40:33.754 INFO:teuthology.orchestra.run.smithi022.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-03-08T23:40:33.796 INFO:teuthology.orchestra.run.smithi022.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 101 kB/s | 39 kB 00:00 2024-03-08T23:40:33.821 INFO:teuthology.orchestra.run.smithi022.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 101 kB/s | 41 kB 00:00 2024-03-08T23:40:33.846 INFO:teuthology.orchestra.run.smithi022.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 300 kB 00:00 2024-03-08T23:40:33.871 INFO:teuthology.orchestra.run.smithi022.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-08T23:40:33.896 INFO:teuthology.orchestra.run.smithi022.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 840 kB/s | 63 kB 00:00 2024-03-08T23:40:33.921 INFO:teuthology.orchestra.run.smithi022.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 586 kB/s | 44 kB 00:00 2024-03-08T23:40:33.948 INFO:teuthology.orchestra.run.smithi022.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-08T23:40:33.972 INFO:teuthology.orchestra.run.smithi022.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-08T23:40:34.005 INFO:teuthology.orchestra.run.smithi022.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-08T23:40:34.031 INFO:teuthology.orchestra.run.smithi022.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-08T23:40:34.056 INFO:teuthology.orchestra.run.smithi022.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-08T23:40:34.081 INFO:teuthology.orchestra.run.smithi022.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-08T23:40:34.206 INFO:teuthology.orchestra.run.smithi022.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-08T23:40:34.232 INFO:teuthology.orchestra.run.smithi022.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-08T23:40:34.257 INFO:teuthology.orchestra.run.smithi022.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-08T23:40:34.291 INFO:teuthology.orchestra.run.smithi022.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-08T23:40:34.649 INFO:teuthology.orchestra.run.smithi062.stdout:(4/174): ceph-mds-19.0.0-1965.g7d63482a.el9.x86 1.3 MB/s | 2.1 MB 00:01 2024-03-08T23:40:34.725 INFO:teuthology.orchestra.run.smithi062.stdout:(5/174): ceph-base-19.0.0-1965.g7d63482a.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-03-08T23:40:34.857 INFO:teuthology.orchestra.run.smithi022.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-03-08T23:40:34.916 INFO:teuthology.orchestra.run.smithi022.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 39 kB/s | 25 kB 00:00 2024-03-08T23:40:35.725 INFO:teuthology.orchestra.run.smithi062.stdout:(6/174): ceph-mgr-19.0.0-1965.g7d63482a.el9.x86 1.4 MB/s | 1.5 MB 00:01 2024-03-08T23:40:35.849 INFO:teuthology.orchestra.run.smithi022.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 247 kB/s | 245 kB 00:00 2024-03-08T23:40:35.900 INFO:teuthology.orchestra.run.smithi062.stdout:(7/174): ceph-mon-19.0.0-1965.g7d63482a.el9.x86 4.0 MB/s | 4.7 MB 00:01 2024-03-08T23:40:35.916 INFO:teuthology.orchestra.run.smithi022.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 248 kB/s | 248 kB 00:00 2024-03-08T23:40:36.474 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 893 kB/s | 19 MB 00:21 2024-03-08T23:40:36.608 INFO:teuthology.orchestra.run.smithi022.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 50 kB/s | 35 kB 00:00 2024-03-08T23:40:36.825 INFO:teuthology.orchestra.run.smithi022.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 222 kB/s | 216 kB 00:00 2024-03-08T23:40:37.267 INFO:teuthology.orchestra.run.smithi022.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 68 kB/s | 44 kB 00:00 2024-03-08T23:40:37.350 INFO:teuthology.orchestra.run.smithi022.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-03-08T23:40:37.492 INFO:teuthology.orchestra.run.smithi022.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-03-08T23:40:37.584 INFO:teuthology.orchestra.run.smithi022.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 123 kB/s | 94 kB 00:00 2024-03-08T23:40:37.751 INFO:teuthology.orchestra.run.smithi022.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-03-08T23:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-03-08T23:40:37.802 INFO:teuthology.orchestra.run.smithi022.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 393 kB/s | 19 kB 00:00 2024-03-08T23:40:37.927 INFO:teuthology.orchestra.run.smithi022.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-03-08T23:40:37.952 INFO:teuthology.orchestra.run.smithi022.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 174 kB/s | 26 kB 00:00 2024-03-08T23:40:37.986 INFO:teuthology.orchestra.run.smithi022.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-03-08T23:40:38.012 INFO:teuthology.orchestra.run.smithi022.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-08T23:40:38.037 INFO:teuthology.orchestra.run.smithi022.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-03-08T23:40:38.079 INFO:teuthology.orchestra.run.smithi022.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-03-08T23:40:38.104 INFO:teuthology.orchestra.run.smithi022.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 8.2 MB/s | 548 kB 00:00 2024-03-08T23:40:38.129 INFO:teuthology.orchestra.run.smithi022.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 887 kB/s | 43 kB 00:00 2024-03-08T23:40:38.154 INFO:teuthology.orchestra.run.smithi022.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-03-08T23:40:38.180 INFO:teuthology.orchestra.run.smithi022.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-03-08T23:40:38.205 INFO:teuthology.orchestra.run.smithi022.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-03-08T23:40:38.230 INFO:teuthology.orchestra.run.smithi022.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-03-08T23:40:38.255 INFO:teuthology.orchestra.run.smithi022.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-03-08T23:40:38.289 INFO:teuthology.orchestra.run.smithi022.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 610 kB/s | 35 kB 00:00 2024-03-08T23:40:38.314 INFO:teuthology.orchestra.run.smithi022.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-08T23:40:38.373 INFO:teuthology.orchestra.run.smithi022.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 25 MB/s | 2.0 MB 00:00 2024-03-08T23:40:38.390 INFO:teuthology.orchestra.run.smithi022.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-08T23:40:38.415 INFO:teuthology.orchestra.run.smithi022.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-03-08T23:40:38.440 INFO:teuthology.orchestra.run.smithi022.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-03-08T23:40:38.466 INFO:teuthology.orchestra.run.smithi022.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-03-08T23:40:38.491 INFO:teuthology.orchestra.run.smithi022.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-03-08T23:40:38.516 INFO:teuthology.orchestra.run.smithi022.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-03-08T23:40:38.550 INFO:teuthology.orchestra.run.smithi022.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-03-08T23:40:38.575 INFO:teuthology.orchestra.run.smithi022.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 807 kB/s | 46 kB 00:00 2024-03-08T23:40:38.600 INFO:teuthology.orchestra.run.smithi022.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-08T23:40:38.608 INFO:teuthology.orchestra.run.smithi062.stdout:(8/174): ceph-radosgw-19.0.0-1965.g7d63482a.el9 3.6 MB/s | 9.8 MB 00:02 2024-03-08T23:40:38.617 INFO:teuthology.orchestra.run.smithi022.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-03-08T23:40:38.651 INFO:teuthology.orchestra.run.smithi022.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-08T23:40:38.668 INFO:teuthology.orchestra.run.smithi022.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-03-08T23:40:38.701 INFO:teuthology.orchestra.run.smithi022.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-08T23:40:38.717 INFO:teuthology.orchestra.run.smithi062.stdout:(9/174): ceph-selinux-19.0.0-1965.g7d63482a.el9 233 kB/s | 25 kB 00:00 2024-03-08T23:40:38.718 INFO:teuthology.orchestra.run.smithi022.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-03-08T23:40:38.735 INFO:teuthology.orchestra.run.smithi022.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 921 kB/s | 31 kB 00:00 2024-03-08T23:40:38.769 INFO:teuthology.orchestra.run.smithi022.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-08T23:40:38.794 INFO:teuthology.orchestra.run.smithi022.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-08T23:40:38.819 INFO:teuthology.orchestra.run.smithi022.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-03-08T23:40:38.845 INFO:teuthology.orchestra.run.smithi022.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-08T23:40:38.870 INFO:teuthology.orchestra.run.smithi022.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-08T23:40:38.895 INFO:teuthology.orchestra.run.smithi022.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-03-08T23:40:38.921 INFO:teuthology.orchestra.run.smithi022.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.4 MB/s | 427 kB 00:00 2024-03-08T23:40:38.946 INFO:teuthology.orchestra.run.smithi022.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 404 kB/s | 20 kB 00:00 2024-03-08T23:40:38.971 INFO:teuthology.orchestra.run.smithi022.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-03-08T23:40:39.021 INFO:teuthology.orchestra.run.smithi022.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-03-08T23:40:40.072 INFO:teuthology.orchestra.run.smithi022.stdout:(150/174): librados2-19.0.0-1965.g7d63482a.el9. 3.1 MB/s | 3.4 MB 00:01 2024-03-08T23:40:40.122 INFO:teuthology.orchestra.run.smithi022.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-08T23:40:40.384 INFO:teuthology.orchestra.run.smithi062.stdout:(10/174): ceph-common-19.0.0-1965.g7d63482a.el9 2.4 MB/s | 20 MB 00:08 2024-03-08T23:40:40.424 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-08T23:40:40.447 INFO:teuthology.orchestra.run.smithi022.stdout:(152/174): openssl-devel-3.0.7-27.el9.x86_64.rp 687 kB/s | 4.1 MB 00:06 2024-03-08T23:40:40.506 INFO:teuthology.orchestra.run.smithi022.stdout:(153/174): librbd1-19.0.0-1965.g7d63482a.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-03-08T23:40:40.534 INFO:teuthology.orchestra.run.smithi062.stdout:(11/174): cephfs-mirror-19.0.0-1965.g7d63482a.e 1.4 MB/s | 220 kB 00:00 2024-03-08T23:40:40.556 INFO:teuthology.orchestra.run.smithi022.stdout:(154/174): NetworkManager-1.46.0-1.el9.x86_64.r 5.3 MB/s | 2.3 MB 00:00 2024-03-08T23:40:40.598 INFO:teuthology.orchestra.run.smithi022.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-08T23:40:40.617 INFO:teuthology.orchestra.run.smithi062.stdout:(12/174): cephfs-shell-19.0.0-1965.g7d63482a.el 324 kB/s | 27 kB 00:00 2024-03-08T23:40:40.631 INFO:teuthology.orchestra.run.smithi022.stdout:(156/174): NetworkManager-team-1.46.0-1.el9.x86 338 kB/s | 42 kB 00:00 2024-03-08T23:40:40.657 INFO:teuthology.orchestra.run.smithi022.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-08T23:40:40.690 INFO:teuthology.orchestra.run.smithi022.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 625 kB/s | 36 kB 00:00 2024-03-08T23:40:40.727 INFO:teuthology.orchestra.run.smithi062.stdout:(13/174): libcephfs-devel-19.0.0-1965.g7d63482a 286 kB/s | 31 kB 00:00 2024-03-08T23:40:40.749 INFO:teuthology.orchestra.run.smithi022.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-08T23:40:40.782 INFO:teuthology.orchestra.run.smithi022.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-08T23:40:40.808 INFO:teuthology.orchestra.run.smithi022.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 394 kB/s | 23 kB 00:00 2024-03-08T23:40:40.825 INFO:teuthology.orchestra.run.smithi022.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-08T23:40:40.866 INFO:teuthology.orchestra.run.smithi022.stdout:(163/174): NetworkManager-libnm-1.46.0-1.el9.x8 4.3 MB/s | 1.8 MB 00:00 2024-03-08T23:40:41.033 INFO:teuthology.orchestra.run.smithi022.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 654 kB/s | 136 kB 00:00 2024-03-08T23:40:41.121 INFO:teuthology.orchestra.run.smithi062.stdout:(14/174): libcephfs2-19.0.0-1965.g7d63482a.el9. 1.9 MB/s | 745 kB 00:00 2024-03-08T23:40:41.126 INFO:teuthology.orchestra.run.smithi022.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 488 kB/s | 155 kB 00:00 2024-03-08T23:40:41.142 INFO:teuthology.orchestra.run.smithi022.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 262 kB/s | 29 kB 00:00 2024-03-08T23:40:41.160 INFO:teuthology.orchestra.run.smithi022.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 218 kB/s | 64 kB 00:00 2024-03-08T23:40:41.260 INFO:teuthology.orchestra.run.smithi062.stdout:(15/174): libcephsqlite-19.0.0-1965.g7d63482a.e 1.1 MB/s | 163 kB 00:00 2024-03-08T23:40:41.352 INFO:teuthology.orchestra.run.smithi062.stdout:(16/174): librados-devel-19.0.0-1965.g7d63482a. 1.4 MB/s | 127 kB 00:00 2024-03-08T23:40:41.393 INFO:teuthology.orchestra.run.smithi022.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.5 MB/s | 1.2 MB 00:00 2024-03-08T23:40:41.443 INFO:teuthology.orchestra.run.smithi022.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-03-08T23:40:41.477 INFO:teuthology.orchestra.run.smithi022.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 937 kB/s | 31 kB 00:00 2024-03-08T23:40:41.636 INFO:teuthology.orchestra.run.smithi062.stdout:(17/174): libradosstriper1-19.0.0-1965.g7d63482 1.7 MB/s | 501 kB 00:00 2024-03-08T23:40:41.727 INFO:teuthology.orchestra.run.smithi022.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.3 MB/s | 1.1 MB 00:00 2024-03-08T23:40:41.786 INFO:teuthology.orchestra.run.smithi022.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 2.3 MB 00:00 2024-03-08T23:40:41.892 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 15 kB 00:00 2024-03-08T23:40:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:(173/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.7 MB/s | 2.2 MB 00:00 2024-03-08T23:40:42.436 INFO:teuthology.orchestra.run.smithi022.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 105 kB/s | 74 kB 00:00 2024-03-08T23:40:42.443 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-03-08T23:40:42.443 INFO:teuthology.orchestra.run.smithi022.stdout:Total 12 MB/s | 223 MB 00:18 2024-03-08T23:40:42.932 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-03-08T23:40:43.266 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-03-08T23:40:43.266 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-03-08T23:40:43.878 INFO:teuthology.orchestra.run.smithi062.stdout:(18/174): librgw2-19.0.0-1965.g7d63482a.el9.x86 2.3 MB/s | 5.1 MB 00:02 2024-03-08T23:40:43.961 INFO:teuthology.orchestra.run.smithi062.stdout:(19/174): python3-ceph-argparse-19.0.0-1965.g7d 544 kB/s | 45 kB 00:00 2024-03-08T23:40:44.045 INFO:teuthology.orchestra.run.smithi062.stdout:(20/174): python3-ceph-common-19.0.0-1965.g7d63 1.5 MB/s | 131 kB 00:00 2024-03-08T23:40:44.237 INFO:teuthology.orchestra.run.smithi062.stdout:(21/174): python3-cephfs-19.0.0-1965.g7d63482a. 846 kB/s | 162 kB 00:00 2024-03-08T23:40:44.445 INFO:teuthology.orchestra.run.smithi062.stdout:(22/174): python3-rados-19.0.0-1965.g7d63482a.e 1.5 MB/s | 320 kB 00:00 2024-03-08T23:40:44.629 INFO:teuthology.orchestra.run.smithi062.stdout:(23/174): python3-rbd-19.0.0-1965.g7d63482a.el9 1.6 MB/s | 298 kB 00:00 2024-03-08T23:40:44.721 INFO:teuthology.orchestra.run.smithi062.stdout:(24/174): python3-rgw-19.0.0-1965.g7d63482a.el9 1.1 MB/s | 99 kB 00:00 2024-03-08T23:40:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:(25/174): rbd-fuse-19.0.0-1965.g7d63482a.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-08T23:40:45.377 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-03-08T23:40:45.377 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-03-08T23:40:45.646 INFO:teuthology.orchestra.run.smithi062.stdout:(26/174): ceph-osd-19.0.0-1965.g7d63482a.el9.x8 1.7 MB/s | 17 MB 00:09 2024-03-08T23:40:45.763 INFO:teuthology.orchestra.run.smithi062.stdout:(27/174): rbd-nbd-19.0.0-1965.g7d63482a.el9.x86 1.4 MB/s | 171 kB 00:00 2024-03-08T23:40:45.905 INFO:teuthology.orchestra.run.smithi062.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1965.g 240 kB/s | 34 kB 00:00 2024-03-08T23:40:46.031 INFO:teuthology.orchestra.run.smithi062.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1965.g7d63482 1.1 MB/s | 139 kB 00:00 2024-03-08T23:40:46.247 INFO:teuthology.orchestra.run.smithi062.stdout:(30/174): rbd-mirror-19.0.0-1965.g7d63482a.el9. 2.1 MB/s | 3.1 MB 00:01 2024-03-08T23:40:47.331 INFO:teuthology.orchestra.run.smithi062.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1965.g7d634 1.4 MB/s | 1.8 MB 00:01 2024-03-08T23:40:47.565 INFO:teuthology.orchestra.run.smithi062.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1965.g7d 1.0 MB/s | 248 kB 00:00 2024-03-08T23:40:47.640 INFO:teuthology.orchestra.run.smithi062.stdout:(33/174): ceph-mgr-rook-19.0.0-1965.g7d63482a.e 660 kB/s | 49 kB 00:00 2024-03-08T23:40:47.748 INFO:teuthology.orchestra.run.smithi062.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1965.g7 152 kB/s | 16 kB 00:00 2024-03-08T23:40:48.032 INFO:teuthology.orchestra.run.smithi062.stdout:(35/174): ceph-volume-19.0.0-1965.g7d63482a.el9 931 kB/s | 264 kB 00:00 2024-03-08T23:40:48.582 INFO:teuthology.orchestra.run.smithi062.stdout:(36/174): cephadm-19.0.0-1965.g7d63482a.el9.noa 1.3 MB/s | 748 kB 00:00 2024-03-08T23:40:48.658 INFO:teuthology.orchestra.run.smithi062.stdout:(37/174): cephfs-top-19.0.0-1965.g7d63482a.el9. 244 kB/s | 18 kB 00:00 2024-03-08T23:40:48.925 INFO:teuthology.orchestra.run.smithi062.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-08T23:40:49.025 INFO:teuthology.orchestra.run.smithi062.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 9.0 MB/s | 920 kB 00:00 2024-03-08T23:40:49.100 INFO:teuthology.orchestra.run.smithi062.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-03-08T23:40:49.170 INFO:teuthology.orchestra.run.smithi062.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-08T23:40:49.201 INFO:teuthology.orchestra.run.smithi062.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 946 kB/s | 32 kB 00:00 2024-03-08T23:40:49.410 INFO:teuthology.orchestra.run.smithi062.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-08T23:40:49.452 INFO:teuthology.orchestra.run.smithi062.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 411 kB/s | 17 kB 00:00 2024-03-08T23:40:49.493 INFO:teuthology.orchestra.run.smithi062.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 242 kB/s | 10 kB 00:00 2024-03-08T23:40:49.544 INFO:teuthology.orchestra.run.smithi062.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-08T23:40:49.594 INFO:teuthology.orchestra.run.smithi062.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-03-08T23:40:49.661 INFO:teuthology.orchestra.run.smithi062.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-08T23:40:49.728 INFO:teuthology.orchestra.run.smithi062.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-03-08T23:40:49.778 INFO:teuthology.orchestra.run.smithi062.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 930 kB/s | 46 kB 00:00 2024-03-08T23:40:49.837 INFO:teuthology.orchestra.run.smithi062.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-08T23:40:49.887 INFO:teuthology.orchestra.run.smithi062.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-08T23:40:49.971 INFO:teuthology.orchestra.run.smithi062.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2024-03-08T23:40:50.096 INFO:teuthology.orchestra.run.smithi062.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 273 kB/s | 34 kB 00:00 2024-03-08T23:40:50.272 INFO:teuthology.orchestra.run.smithi062.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-08T23:40:50.364 INFO:teuthology.orchestra.run.smithi062.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-03-08T23:40:50.514 INFO:teuthology.orchestra.run.smithi062.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 5.3 MB/s | 811 kB 00:00 2024-03-08T23:40:50.564 INFO:teuthology.orchestra.run.smithi062.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 2.6 MB/s | 134 kB 00:00 2024-03-08T23:40:50.789 INFO:teuthology.orchestra.run.smithi062.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-03-08T23:40:50.898 INFO:teuthology.orchestra.run.smithi062.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-03-08T23:40:50.990 INFO:teuthology.orchestra.run.smithi062.stdout:(61/174): ceph-mgr-diskprediction-local-19.0.0- 1.6 MB/s | 7.4 MB 00:04 2024-03-08T23:40:51.015 INFO:teuthology.orchestra.run.smithi062.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 665 kB/s | 78 kB 00:00 2024-03-08T23:40:51.224 INFO:teuthology.orchestra.run.smithi062.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-08T23:40:51.250 INFO:teuthology.orchestra.run.smithi062.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 104 kB/s | 24 kB 00:00 2024-03-08T23:40:51.275 INFO:teuthology.orchestra.run.smithi062.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 357 kB/s | 18 kB 00:00 2024-03-08T23:40:51.301 INFO:teuthology.orchestra.run.smithi062.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-08T23:40:51.326 INFO:teuthology.orchestra.run.smithi062.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-08T23:40:51.527 INFO:teuthology.orchestra.run.smithi062.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-08T23:40:51.552 INFO:teuthology.orchestra.run.smithi062.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 966 kB/s | 217 kB 00:00 2024-03-08T23:40:51.577 INFO:teuthology.orchestra.run.smithi062.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-08T23:40:51.602 INFO:teuthology.orchestra.run.smithi062.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-08T23:40:51.628 INFO:teuthology.orchestra.run.smithi062.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 820 kB/s | 41 kB 00:00 2024-03-08T23:40:51.661 INFO:teuthology.orchestra.run.smithi062.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-08T23:40:51.687 INFO:teuthology.orchestra.run.smithi062.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-08T23:40:51.721 INFO:teuthology.orchestra.run.smithi062.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2024-03-08T23:40:51.737 INFO:teuthology.orchestra.run.smithi062.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 937 kB/s | 47 kB 00:00 2024-03-08T23:40:51.771 INFO:teuthology.orchestra.run.smithi062.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-03-08T23:40:52.205 INFO:teuthology.orchestra.run.smithi062.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 89 kB/s | 39 kB 00:00 2024-03-08T23:40:52.489 INFO:teuthology.orchestra.run.smithi062.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-03-08T23:40:52.514 INFO:teuthology.orchestra.run.smithi062.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 133 kB/s | 41 kB 00:00 2024-03-08T23:40:52.539 INFO:teuthology.orchestra.run.smithi062.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-08T23:40:52.564 INFO:teuthology.orchestra.run.smithi062.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-08T23:40:52.589 INFO:teuthology.orchestra.run.smithi062.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-08T23:40:52.615 INFO:teuthology.orchestra.run.smithi062.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-03-08T23:40:52.642 INFO:teuthology.orchestra.run.smithi062.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-08T23:40:52.674 INFO:teuthology.orchestra.run.smithi062.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-08T23:40:52.724 INFO:teuthology.orchestra.run.smithi062.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-08T23:40:52.749 INFO:teuthology.orchestra.run.smithi062.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-08T23:40:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-08T23:40:52.788 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-03-08T23:40:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-08T23:40:52.873 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T23:40:52.900 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T23:40:52.929 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T23:40:52.939 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T23:40:52.988 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T23:40:53.033 INFO:teuthology.orchestra.run.smithi062.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 749 kB/s | 187 kB 00:00 2024-03-08T23:40:53.042 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T23:40:53.083 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T23:40:53.108 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T23:40:53.150 INFO:teuthology.orchestra.run.smithi062.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-03-08T23:40:53.157 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T23:40:53.201 INFO:teuthology.orchestra.run.smithi062.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-03-08T23:40:53.243 INFO:teuthology.orchestra.run.smithi062.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-03-08T23:40:53.268 INFO:teuthology.orchestra.run.smithi062.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-03-08T23:40:53.293 INFO:teuthology.orchestra.run.smithi062.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-03-08T23:40:53.344 INFO:teuthology.orchestra.run.smithi062.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-03-08T23:40:53.402 INFO:teuthology.orchestra.run.smithi062.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-03-08T23:40:53.452 INFO:teuthology.orchestra.run.smithi062.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-08T23:40:53.499 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T23:40:53.528 INFO:teuthology.orchestra.run.smithi062.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 464 kB/s | 35 kB 00:00 2024-03-08T23:40:53.551 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:40:53.578 INFO:teuthology.orchestra.run.smithi062.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 892 kB/s | 44 kB 00:00 2024-03-08T23:40:53.637 INFO:teuthology.orchestra.run.smithi062.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-08T23:40:53.667 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:40:53.703 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:40:53.745 INFO:teuthology.orchestra.run.smithi062.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-03-08T23:40:53.781 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:40:53.819 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T23:40:53.867 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T23:40:54.050 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T23:40:54.097 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T23:40:54.125 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:40:54.213 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:40:54.229 INFO:teuthology.orchestra.run.smithi062.stdout:(104/174): ceph-test-19.0.0-1965.g7d63482a.el9. 3.1 MB/s | 47 MB 00:15 2024-03-08T23:40:54.242 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T23:40:54.296 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T23:40:54.296 INFO:teuthology.orchestra.run.smithi062.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.0 MB/s | 4.1 MB 00:01 2024-03-08T23:40:54.338 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T23:40:54.360 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T23:40:54.376 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T23:40:54.455 INFO:teuthology.orchestra.run.smithi062.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-03-08T23:40:54.535 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T23:40:54.546 INFO:teuthology.orchestra.run.smithi062.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-03-08T23:40:54.572 INFO:teuthology.orchestra.run.smithi062.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-08T23:40:54.597 INFO:teuthology.orchestra.run.smithi062.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-03-08T23:40:54.647 INFO:teuthology.orchestra.run.smithi062.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-03-08T23:40:54.659 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T23:40:54.715 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T23:40:54.722 INFO:teuthology.orchestra.run.smithi062.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-03-08T23:40:54.748 INFO:teuthology.orchestra.run.smithi062.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 322 kB/s | 49 kB 00:00 2024-03-08T23:40:54.815 INFO:teuthology.orchestra.run.smithi062.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-03-08T23:40:54.882 INFO:teuthology.orchestra.run.smithi062.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-03-08T23:40:54.889 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T23:40:54.941 INFO:teuthology.orchestra.run.smithi062.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 747 kB/s | 43 kB 00:00 2024-03-08T23:40:54.977 INFO:teuthology.orchestra.run.smithi062.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-03-08T23:40:55.010 INFO:teuthology.orchestra.run.smithi062.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 463 kB/s | 32 kB 00:00 2024-03-08T23:40:55.017 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T23:40:55.044 INFO:teuthology.orchestra.run.smithi062.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-03-08T23:40:55.086 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T23:40:55.086 INFO:teuthology.orchestra.run.smithi062.stdout:(119/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 838 kB 00:00 2024-03-08T23:40:55.111 INFO:teuthology.orchestra.run.smithi062.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-08T23:40:55.137 INFO:teuthology.orchestra.run.smithi062.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-08T23:40:55.162 INFO:teuthology.orchestra.run.smithi062.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-03-08T23:40:55.187 INFO:teuthology.orchestra.run.smithi062.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-03-08T23:40:55.196 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:40:55.212 INFO:teuthology.orchestra.run.smithi062.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-08T23:40:55.244 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:40:55.246 INFO:teuthology.orchestra.run.smithi062.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-08T23:40:55.288 INFO:teuthology.orchestra.run.smithi062.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-03-08T23:40:55.322 INFO:teuthology.orchestra.run.smithi062.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-03-08T23:40:55.381 INFO:teuthology.orchestra.run.smithi062.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-03-08T23:40:55.398 INFO:teuthology.orchestra.run.smithi062.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 212 kB/s | 23 kB 00:00 2024-03-08T23:40:55.414 INFO:teuthology.orchestra.run.smithi062.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-03-08T23:40:55.439 INFO:teuthology.orchestra.run.smithi062.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-03-08T23:40:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:40:55.482 INFO:teuthology.orchestra.run.smithi062.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-03-08T23:40:55.507 INFO:teuthology.orchestra.run.smithi062.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-03-08T23:40:55.540 INFO:teuthology.orchestra.run.smithi062.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-03-08T23:40:55.565 INFO:teuthology.orchestra.run.smithi062.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-08T23:40:55.579 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T23:40:55.591 INFO:teuthology.orchestra.run.smithi062.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-08T23:40:55.616 INFO:teuthology.orchestra.run.smithi062.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-03-08T23:40:55.641 INFO:teuthology.orchestra.run.smithi062.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-08T23:40:55.667 INFO:teuthology.orchestra.run.smithi062.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-03-08T23:40:55.673 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T23:40:55.692 INFO:teuthology.orchestra.run.smithi062.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-03-08T23:40:55.708 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T23:40:55.717 INFO:teuthology.orchestra.run.smithi062.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-08T23:40:55.735 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T23:40:55.743 INFO:teuthology.orchestra.run.smithi062.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-03-08T23:40:55.768 INFO:teuthology.orchestra.run.smithi062.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-03-08T23:40:55.770 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T23:40:55.793 INFO:teuthology.orchestra.run.smithi062.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-03-08T23:40:55.818 INFO:teuthology.orchestra.run.smithi062.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-08T23:40:55.844 INFO:teuthology.orchestra.run.smithi062.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-08T23:40:55.846 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T23:40:55.869 INFO:teuthology.orchestra.run.smithi062.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-03-08T23:40:55.894 INFO:teuthology.orchestra.run.smithi062.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-08T23:40:55.920 INFO:teuthology.orchestra.run.smithi062.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-08T23:40:56.003 INFO:teuthology.orchestra.run.smithi062.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-03-08T23:40:56.024 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T23:40:56.062 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T23:40:56.112 INFO:teuthology.orchestra.run.smithi062.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 321 kB 00:00 2024-03-08T23:40:56.128 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T23:40:56.177 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T23:40:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T23:40:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T23:40:56.464 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T23:40:56.671 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T23:40:56.711 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 39/198 2024-03-08T23:40:56.712 INFO:teuthology.orchestra.run.smithi062.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 3.8 MB/s | 2.3 MB 00:00 2024-03-08T23:40:56.763 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 40/198 2024-03-08T23:40:56.791 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 41/198 2024-03-08T23:40:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T23:40:56.906 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T23:40:56.992 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T23:40:57.138 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T23:40:57.192 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T23:40:57.254 INFO:teuthology.orchestra.run.smithi062.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.4 MB/s | 1.8 MB 00:00 2024-03-08T23:40:57.296 INFO:teuthology.orchestra.run.smithi062.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-08T23:40:57.371 INFO:teuthology.orchestra.run.smithi062.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 3.2 MB/s | 248 kB 00:00 2024-03-08T23:40:57.413 INFO:teuthology.orchestra.run.smithi062.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-08T23:40:57.463 INFO:teuthology.orchestra.run.smithi062.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 729 kB/s | 36 kB 00:00 2024-03-08T23:40:57.556 INFO:teuthology.orchestra.run.smithi062.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.1 MB/s | 192 kB 00:00 2024-03-08T23:40:57.648 INFO:teuthology.orchestra.run.smithi062.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-08T23:40:57.689 INFO:teuthology.orchestra.run.smithi062.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-03-08T23:40:57.731 INFO:teuthology.orchestra.run.smithi062.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-08T23:40:57.763 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T23:40:57.790 INFO:teuthology.orchestra.run.smithi062.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-08T23:40:57.865 INFO:teuthology.orchestra.run.smithi062.stdout:(163/174): librbd1-19.0.0-1965.g7d63482a.el9.x8 1.6 MB/s | 3.1 MB 00:01 2024-03-08T23:40:57.915 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T23:40:57.915 INFO:teuthology.orchestra.run.smithi062.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-08T23:40:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 848 kB/s | 64 kB 00:00 2024-03-08T23:40:57.966 INFO:teuthology.orchestra.run.smithi062.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-03-08T23:40:58.023 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T23:40:58.081 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi062.stdout:(167/174): librados2-19.0.0-1965.g7d63482a.el9. 1.5 MB/s | 3.4 MB 00:02 2024-03-08T23:40:58.090 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T23:40:58.122 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T23:40:58.179 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T23:40:58.217 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 53/198 2024-03-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi062.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.4 MB/s | 2.2 MB 00:00 2024-03-08T23:40:58.292 INFO:teuthology.orchestra.run.smithi062.stdout:(169/174): openssl-3.0.7-27.el9.x86_64.rpm 3.5 MB/s | 1.2 MB 00:00 2024-03-08T23:40:58.334 INFO:teuthology.orchestra.run.smithi062.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 749 kB/s | 31 kB 00:00 2024-03-08T23:40:58.367 INFO:teuthology.orchestra.run.smithi062.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-03-08T23:40:58.418 INFO:teuthology.orchestra.run.smithi062.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.5 MB/s | 74 kB 00:00 2024-03-08T23:40:58.518 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T23:40:58.651 INFO:teuthology.orchestra.run.smithi062.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.0 MB/s | 2.3 MB 00:00 2024-03-08T23:40:58.685 INFO:teuthology.orchestra.run.smithi062.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.1 MB/s | 1.1 MB 00:00 2024-03-08T23:40:58.689 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-03-08T23:40:58.689 INFO:teuthology.orchestra.run.smithi062.stdout:Total 8.2 MB/s | 223 MB 00:27 2024-03-08T23:40:58.759 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T23:40:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T23:40:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T23:40:59.087 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T23:40:59.139 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T23:40:59.196 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T23:40:59.208 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-03-08T23:40:59.271 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T23:40:59.307 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T23:40:59.343 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T23:40:59.391 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T23:40:59.426 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T23:40:59.470 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T23:40:59.506 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-03-08T23:40:59.507 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-03-08T23:40:59.512 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T23:40:59.653 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T23:40:59.692 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T23:40:59.722 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T23:40:59.764 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T23:40:59.862 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T23:40:59.906 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T23:40:59.946 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T23:40:59.971 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T23:40:59.996 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T23:41:00.074 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T23:41:00.201 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T23:41:00.312 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T23:41:00.355 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T23:41:00.413 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T23:41:00.486 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T23:41:00.679 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T23:41:00.726 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T23:41:00.878 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T23:41:00.928 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T23:41:00.972 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T23:41:01.002 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T23:41:01.031 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T23:41:01.066 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T23:41:01.105 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T23:41:01.144 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T23:41:01.174 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T23:41:01.254 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T23:41:01.285 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T23:41:01.318 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T23:41:01.360 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T23:41:01.401 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T23:41:01.436 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-03-08T23:41:01.437 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-03-08T23:41:01.480 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T23:41:01.570 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T23:41:01.617 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T23:41:01.648 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T23:41:01.697 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T23:41:01.807 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T23:41:01.860 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T23:41:02.640 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T23:41:02.688 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T23:41:02.722 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T23:41:02.772 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T23:41:02.866 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T23:41:02.899 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T23:41:03.084 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T23:41:03.113 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T23:41:03.192 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T23:41:03.892 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T23:41:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T23:41:05.315 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T23:41:05.396 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T23:41:05.430 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T23:41:05.460 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T23:41:05.488 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T23:41:05.566 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 893 kB/s | 21 MB 00:23 2024-03-08T23:41:05.817 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T23:41:05.843 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T23:41:05.934 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T23:41:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 125/198 2024-03-08T23:41:06.002 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T23:41:06.481 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T23:41:06.521 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T23:41:06.575 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T23:41:06.606 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 129/198 2024-03-08T23:41:08.456 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-03-08T23:41:08.589 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T23:41:08.615 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T23:41:08.654 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T23:41:08.664 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T23:41:08.704 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T23:41:08.758 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T23:41:08.791 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T23:41:08.817 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T23:41:08.844 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:41:08.859 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T23:41:08.877 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:41:09.214 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T23:41:09.259 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:41:09.383 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:41:09.410 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:41:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:41:09.485 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:41:09.497 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T23:41:09.514 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T23:41:09.550 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T23:41:09.590 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T23:41:09.590 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T23:41:09.590 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T23:41:09.590 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T23:41:09.739 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T23:41:09.788 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T23:41:09.815 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:41:09.904 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:41:09.933 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T23:41:09.980 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T23:41:10.020 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T23:41:10.071 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T23:41:10.087 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T23:41:10.286 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T23:41:10.316 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T23:41:10.350 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T23:41:10.449 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-08T23:41:10.457 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T23:41:10.560 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T23:41:10.620 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T23:41:10.690 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:41:10.738 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:41:10.965 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:41:11.071 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T23:41:11.166 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T23:41:11.201 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T23:41:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T23:41:11.263 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T23:41:11.340 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T23:41:11.392 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T23:41:11.430 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T23:41:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T23:41:11.529 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T23:41:11.573 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T23:41:11.605 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T23:41:11.632 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T23:41:11.695 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T23:41:11.729 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 39/198 2024-03-08T23:41:11.773 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 40/198 2024-03-08T23:41:11.809 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 41/198 2024-03-08T23:41:11.880 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T23:41:11.925 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T23:41:12.000 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T23:41:12.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T23:41:12.186 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T23:41:12.739 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T23:41:12.815 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T23:41:12.892 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T23:41:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T23:41:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T23:41:12.933 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T23:41:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T23:41:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T23:41:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T23:41:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T23:41:12.941 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T23:41:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T23:41:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T23:41:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T23:41:12.973 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T23:41:13.014 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T23:41:13.025 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 6.6 k 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.4 M 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 889 k 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 145 k 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 1.5 M 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 139 k 2024-03-08T23:41:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 1.8 M 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 7.4 M 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 49 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 9.8 M 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 47 M 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 264 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 748 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 220 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 27 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 18 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T23:41:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 31 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 745 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 127 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 162 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 320 k 2024-03-08T23:41:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 298 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 99 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 85 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 171 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T23:41:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.4 M 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T23:41:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 20 M 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 34 k 2024-03-08T23:41:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 2.1 M 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 248 k 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 4.7 M 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 17 M 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 16 k 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 25 k 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T23:41:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 163 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 501 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T23:41:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.1 M 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T23:41:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 45 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 131 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T23:41:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T23:41:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T23:41:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T23:41:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T23:41:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-03-08T23:41:13.059 INFO:teuthology.orchestra.run.smithi192.stdout:Install 150 Packages 2024-03-08T23:41:13.060 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 24 Packages 2024-03-08T23:41:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:41:13.060 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 53/198 2024-03-08T23:41:13.061 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 223 M 2024-03-08T23:41:13.061 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-03-08T23:41:13.081 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T23:41:13.315 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T23:41:13.526 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T23:41:13.592 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T23:41:13.637 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T23:41:13.796 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T23:41:13.847 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T23:41:13.904 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T23:41:13.978 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T23:41:14.014 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T23:41:14.051 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T23:41:14.100 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T23:41:14.133 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T23:41:14.178 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T23:41:14.219 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T23:41:14.317 INFO:teuthology.orchestra.run.smithi192.stdout:(1/174): ceph-19.0.0-1965.g7d63482a.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-08T23:41:14.362 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T23:41:14.417 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T23:41:14.447 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T23:41:14.506 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T23:41:14.602 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T23:41:14.638 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T23:41:14.678 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T23:41:14.704 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T23:41:14.728 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T23:41:14.794 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T23:41:14.867 INFO:teuthology.orchestra.run.smithi192.stdout:(2/174): ceph-fuse-19.0.0-1965.g7d63482a.el9.x8 1.6 MB/s | 889 kB 00:00 2024-03-08T23:41:14.926 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T23:41:14.976 INFO:teuthology.orchestra.run.smithi192.stdout:(3/174): ceph-base-19.0.0-1965.g7d63482a.el9.x8 5.0 MB/s | 5.4 MB 00:01 2024-03-08T23:41:14.993 INFO:teuthology.orchestra.run.smithi192.stdout:(4/174): ceph-immutable-object-cache-19.0.0-196 1.1 MB/s | 145 kB 00:00 2024-03-08T23:41:15.035 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T23:41:15.078 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T23:41:15.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T23:41:15.200 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T23:41:15.293 INFO:teuthology.orchestra.run.smithi192.stdout:(5/174): ceph-mds-19.0.0-1965.g7d63482a.el9.x86 6.7 MB/s | 2.1 MB 00:00 2024-03-08T23:41:15.369 INFO:teuthology.orchestra.run.smithi192.stdout:(6/174): ceph-mgr-19.0.0-1965.g7d63482a.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-03-08T23:41:15.404 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T23:41:15.457 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T23:41:15.604 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T23:41:15.638 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T23:41:15.670 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T23:41:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T23:41:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T23:41:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T23:41:15.802 INFO:teuthology.orchestra.run.smithi192.stdout:(7/174): ceph-mon-19.0.0-1965.g7d63482a.el9.x86 9.2 MB/s | 4.7 MB 00:00 2024-03-08T23:41:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T23:41:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T23:41:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T23:41:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T23:41:15.991 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T23:41:16.024 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T23:41:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T23:41:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T23:41:16.186 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T23:41:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T23:41:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T23:41:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T23:41:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T23:41:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T23:41:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T23:41:16.677 INFO:teuthology.orchestra.run.smithi192.stdout:(8/174): ceph-common-19.0.0-1965.g7d63482a.el9. 7.2 MB/s | 20 MB 00:02 2024-03-08T23:41:16.794 INFO:teuthology.orchestra.run.smithi192.stdout:(9/174): ceph-radosgw-19.0.0-1965.g7d63482a.el9 9.9 MB/s | 9.8 MB 00:00 2024-03-08T23:41:16.819 INFO:teuthology.orchestra.run.smithi192.stdout:(10/174): ceph-selinux-19.0.0-1965.g7d63482a.el 178 kB/s | 25 kB 00:00 2024-03-08T23:41:16.936 INFO:teuthology.orchestra.run.smithi192.stdout:(11/174): cephfs-mirror-19.0.0-1965.g7d63482a.e 1.8 MB/s | 220 kB 00:00 2024-03-08T23:41:17.036 INFO:teuthology.orchestra.run.smithi192.stdout:(12/174): cephfs-shell-19.0.0-1965.g7d63482a.el 270 kB/s | 27 kB 00:00 2024-03-08T23:41:17.170 INFO:teuthology.orchestra.run.smithi192.stdout:(13/174): libcephfs-devel-19.0.0-1965.g7d63482a 233 kB/s | 31 kB 00:00 2024-03-08T23:41:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T23:41:17.362 INFO:teuthology.orchestra.run.smithi192.stdout:(14/174): libcephfs2-19.0.0-1965.g7d63482a.el9. 3.8 MB/s | 745 kB 00:00 2024-03-08T23:41:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T23:41:17.446 INFO:teuthology.orchestra.run.smithi192.stdout:(15/174): libcephsqlite-19.0.0-1965.g7d63482a.e 1.9 MB/s | 163 kB 00:00 2024-03-08T23:41:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T23:41:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T23:41:17.571 INFO:teuthology.orchestra.run.smithi192.stdout:(16/174): librados-devel-19.0.0-1965.g7d63482a. 1.0 MB/s | 127 kB 00:00 2024-03-08T23:41:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T23:41:17.705 INFO:teuthology.orchestra.run.smithi192.stdout:(17/174): libradosstriper1-19.0.0-1965.g7d63482 3.7 MB/s | 501 kB 00:00 2024-03-08T23:41:17.762 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T23:41:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T23:41:17.976 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T23:41:18.005 INFO:teuthology.orchestra.run.smithi192.stdout:(18/174): ceph-osd-19.0.0-1965.g7d63482a.el9.x8 6.4 MB/s | 17 MB 00:02 2024-03-08T23:41:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T23:41:18.097 INFO:teuthology.orchestra.run.smithi192.stdout:(19/174): python3-ceph-argparse-19.0.0-1965.g7d 493 kB/s | 45 kB 00:00 2024-03-08T23:41:18.197 INFO:teuthology.orchestra.run.smithi192.stdout:(20/174): python3-ceph-common-19.0.0-1965.g7d63 1.3 MB/s | 131 kB 00:00 2024-03-08T23:41:18.323 INFO:teuthology.orchestra.run.smithi192.stdout:(21/174): librgw2-19.0.0-1965.g7d63482a.el9.x86 8.3 MB/s | 5.1 MB 00:00 2024-03-08T23:41:18.348 INFO:teuthology.orchestra.run.smithi192.stdout:(22/174): python3-cephfs-19.0.0-1965.g7d63482a. 1.1 MB/s | 162 kB 00:00 2024-03-08T23:41:18.399 INFO:teuthology.orchestra.run.smithi192.stdout:(23/174): python3-rados-19.0.0-1965.g7d63482a.e 4.2 MB/s | 320 kB 00:00 2024-03-08T23:41:18.449 INFO:teuthology.orchestra.run.smithi192.stdout:(24/174): python3-rbd-19.0.0-1965.g7d63482a.el9 2.9 MB/s | 298 kB 00:00 2024-03-08T23:41:18.474 INFO:teuthology.orchestra.run.smithi192.stdout:(25/174): python3-rgw-19.0.0-1965.g7d63482a.el9 1.3 MB/s | 99 kB 00:00 2024-03-08T23:41:18.533 INFO:teuthology.orchestra.run.smithi192.stdout:(26/174): rbd-fuse-19.0.0-1965.g7d63482a.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-08T23:41:18.650 INFO:teuthology.orchestra.run.smithi192.stdout:(27/174): rbd-nbd-19.0.0-1965.g7d63482a.el9.x86 1.4 MB/s | 171 kB 00:00 2024-03-08T23:41:18.733 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T23:41:18.758 INFO:teuthology.orchestra.run.smithi192.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1965.g 313 kB/s | 34 kB 00:00 2024-03-08T23:41:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:(29/174): rbd-mirror-19.0.0-1965.g7d63482a.el9. 8.4 MB/s | 3.1 MB 00:00 2024-03-08T23:41:18.853 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T23:41:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1965.g7d63482 1.2 MB/s | 139 kB 00:00 2024-03-08T23:41:19.084 INFO:teuthology.orchestra.run.smithi192.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1965.g7d634 7.4 MB/s | 1.8 MB 00:00 2024-03-08T23:41:19.159 INFO:teuthology.orchestra.run.smithi192.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1965.g7d 3.2 MB/s | 248 kB 00:00 2024-03-08T23:41:19.226 INFO:teuthology.orchestra.run.smithi192.stdout:(33/174): ceph-mgr-rook-19.0.0-1965.g7d63482a.e 739 kB/s | 49 kB 00:00 2024-03-08T23:41:19.293 INFO:teuthology.orchestra.run.smithi192.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1965.g7 247 kB/s | 16 kB 00:00 2024-03-08T23:41:19.377 INFO:teuthology.orchestra.run.smithi192.stdout:(35/174): ceph-volume-19.0.0-1965.g7d63482a.el9 3.1 MB/s | 264 kB 00:00 2024-03-08T23:41:19.477 INFO:teuthology.orchestra.run.smithi192.stdout:(36/174): cephadm-19.0.0-1965.g7d63482a.el9.noa 7.3 MB/s | 748 kB 00:00 2024-03-08T23:41:19.544 INFO:teuthology.orchestra.run.smithi192.stdout:(37/174): cephfs-top-19.0.0-1965.g7d63482a.el9. 273 kB/s | 18 kB 00:00 2024-03-08T23:41:19.945 INFO:teuthology.orchestra.run.smithi192.stdout:(38/174): ceph-test-19.0.0-1965.g7d63482a.el9.x 15 MB/s | 47 MB 00:03 2024-03-08T23:41:20.095 INFO:teuthology.orchestra.run.smithi192.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-03-08T23:41:20.149 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T23:41:20.162 INFO:teuthology.orchestra.run.smithi192.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 662 kB 00:00 2024-03-08T23:41:20.187 INFO:teuthology.orchestra.run.smithi192.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-03-08T23:41:20.212 INFO:teuthology.orchestra.run.smithi192.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 920 kB 00:00 2024-03-08T23:41:20.238 INFO:teuthology.orchestra.run.smithi192.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-03-08T23:41:20.262 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T23:41:20.263 INFO:teuthology.orchestra.run.smithi192.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 423 kB/s | 32 kB 00:00 2024-03-08T23:41:20.288 INFO:teuthology.orchestra.run.smithi192.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 342 kB/s | 17 kB 00:00 2024-03-08T23:41:20.308 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T23:41:20.339 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T23:41:20.355 INFO:teuthology.orchestra.run.smithi192.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2024-03-08T23:41:20.366 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T23:41:20.380 INFO:teuthology.orchestra.run.smithi192.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 86 kB/s | 10 kB 00:00 2024-03-08T23:41:20.414 INFO:teuthology.orchestra.run.smithi192.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-03-08T23:41:20.439 INFO:teuthology.orchestra.run.smithi192.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 755 kB/s | 44 kB 00:00 2024-03-08T23:41:20.465 INFO:teuthology.orchestra.run.smithi192.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-08T23:41:20.490 INFO:teuthology.orchestra.run.smithi192.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-08T23:41:20.515 INFO:teuthology.orchestra.run.smithi192.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 46 kB 00:00 2024-03-08T23:41:20.540 INFO:teuthology.orchestra.run.smithi192.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-08T23:41:20.566 INFO:teuthology.orchestra.run.smithi192.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-08T23:41:20.591 INFO:teuthology.orchestra.run.smithi192.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 689 kB/s | 34 kB 00:00 2024-03-08T23:41:20.616 INFO:teuthology.orchestra.run.smithi192.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-08T23:41:20.695 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T23:41:20.700 INFO:teuthology.orchestra.run.smithi192.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-03-08T23:41:20.721 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T23:41:20.742 INFO:teuthology.orchestra.run.smithi192.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-03-08T23:41:20.775 INFO:teuthology.orchestra.run.smithi192.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.1 MB/s | 811 kB 00:00 2024-03-08T23:41:20.800 INFO:teuthology.orchestra.run.smithi192.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-08T23:41:20.812 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T23:41:20.842 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 125/198 2024-03-08T23:41:20.843 INFO:teuthology.orchestra.run.smithi192.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-08T23:41:20.879 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T23:41:20.943 INFO:teuthology.orchestra.run.smithi192.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-08T23:41:20.968 INFO:teuthology.orchestra.run.smithi192.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 464 kB/s | 78 kB 00:00 2024-03-08T23:41:21.052 INFO:teuthology.orchestra.run.smithi192.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-08T23:41:21.069 INFO:teuthology.orchestra.run.smithi192.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 198 kB/s | 24 kB 00:00 2024-03-08T23:41:21.085 INFO:teuthology.orchestra.run.smithi192.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 158 kB/s | 18 kB 00:00 2024-03-08T23:41:21.111 INFO:teuthology.orchestra.run.smithi192.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-08T23:41:21.127 INFO:teuthology.orchestra.run.smithi192.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 757 kB/s | 32 kB 00:00 2024-03-08T23:41:21.203 INFO:teuthology.orchestra.run.smithi192.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 4.8 MB 00:00 2024-03-08T23:41:21.228 INFO:teuthology.orchestra.run.smithi192.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 217 kB 00:00 2024-03-08T23:41:21.253 INFO:teuthology.orchestra.run.smithi192.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-08T23:41:21.279 INFO:teuthology.orchestra.run.smithi192.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 921 kB/s | 69 kB 00:00 2024-03-08T23:41:21.304 INFO:teuthology.orchestra.run.smithi192.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 820 kB/s | 41 kB 00:00 2024-03-08T23:41:21.329 INFO:teuthology.orchestra.run.smithi192.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-08T23:41:21.354 INFO:teuthology.orchestra.run.smithi192.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-08T23:41:21.359 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T23:41:21.380 INFO:teuthology.orchestra.run.smithi192.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-03-08T23:41:21.399 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T23:41:21.405 INFO:teuthology.orchestra.run.smithi192.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-08T23:41:21.428 INFO:teuthology.orchestra.run.smithi192.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 586 kB/s | 43 kB 00:00 2024-03-08T23:41:21.429 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T23:41:21.429 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-03-08T23:41:21.429 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-03-08T23:41:21.429 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-03-08T23:41:21.429 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-03-08T23:41:21.429 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-03-08T23:41:21.430 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-03-08T23:41:21.430 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-03-08T23:41:21.430 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:21.470 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T23:41:21.492 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 129/198 2024-03-08T23:41:21.602 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T23:41:21.617 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T23:41:21.617 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:21.618 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T23:41:21.618 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:21.712 INFO:teuthology.orchestra.run.smithi192.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-08T23:41:21.728 INFO:teuthology.orchestra.run.smithi192.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 130 kB/s | 39 kB 00:00 2024-03-08T23:41:21.745 INFO:teuthology.orchestra.run.smithi192.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 130 kB/s | 41 kB 00:00 2024-03-08T23:41:21.770 INFO:teuthology.orchestra.run.smithi192.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-08T23:41:21.796 INFO:teuthology.orchestra.run.smithi192.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-08T23:41:21.821 INFO:teuthology.orchestra.run.smithi192.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 833 kB/s | 63 kB 00:00 2024-03-08T23:41:21.838 INFO:teuthology.orchestra.run.smithi192.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 652 kB/s | 44 kB 00:00 2024-03-08T23:41:21.863 INFO:teuthology.orchestra.run.smithi192.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-08T23:41:21.881 INFO:teuthology.orchestra.run.smithi192.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-08T23:41:21.914 INFO:teuthology.orchestra.run.smithi192.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-08T23:41:21.939 INFO:teuthology.orchestra.run.smithi192.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-08T23:41:21.965 INFO:teuthology.orchestra.run.smithi192.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-08T23:41:21.981 INFO:teuthology.orchestra.run.smithi192.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-08T23:41:22.849 INFO:teuthology.orchestra.run.smithi192.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 206 kB/s | 187 kB 00:00 2024-03-08T23:41:22.916 INFO:teuthology.orchestra.run.smithi192.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 209 kB/s | 195 kB 00:00 2024-03-08T23:41:22.991 INFO:teuthology.orchestra.run.smithi192.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 288 kB/s | 21 kB 00:00 2024-03-08T23:41:23.033 INFO:teuthology.orchestra.run.smithi192.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 611 kB/s | 25 kB 00:00 2024-03-08T23:41:23.157 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T23:41:23.182 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T23:41:23.182 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:23.182 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T23:41:23.182 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T23:41:23.183 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T23:41:23.183 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:23.399 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T23:41:23.421 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T23:41:23.422 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:23.422 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T23:41:23.422 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T23:41:23.422 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T23:41:23.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:23.625 INFO:teuthology.orchestra.run.smithi192.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 161 kB/s | 125 kB 00:00 2024-03-08T23:41:23.658 INFO:teuthology.orchestra.run.smithi192.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 479 kB/s | 803 kB 00:01 2024-03-08T23:41:23.743 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:41:23.780 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:41:23.784 INFO:teuthology.orchestra.run.smithi192.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-03-08T23:41:23.809 INFO:teuthology.orchestra.run.smithi192.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-03-08T23:41:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:(100/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.9 MB/s | 4.1 MB 00:00 2024-03-08T23:41:23.883 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T23:41:23.901 INFO:teuthology.orchestra.run.smithi192.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 377 kB/s | 35 kB 00:00 2024-03-08T23:41:23.906 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T23:41:23.906 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:23.906 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T23:41:23.906 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T23:41:23.906 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T23:41:23.906 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:23.927 INFO:teuthology.orchestra.run.smithi192.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-03-08T23:41:23.952 INFO:teuthology.orchestra.run.smithi192.stdout:(103/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.3 MB/s | 216 kB 00:00 2024-03-08T23:41:23.977 INFO:teuthology.orchestra.run.smithi192.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-03-08T23:41:24.002 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T23:41:24.061 INFO:teuthology.orchestra.run.smithi192.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-03-08T23:41:24.211 INFO:teuthology.orchestra.run.smithi192.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-03-08T23:41:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:41:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T23:41:24.438 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T23:41:24.456 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T23:41:24.457 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T23:41:24.457 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T23:41:24.457 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:24.461 INFO:teuthology.orchestra.run.smithi192.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-03-08T23:41:24.467 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T23:41:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T23:41:24.500 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 139/198 2024-03-08T23:41:24.561 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 140/198 2024-03-08T23:41:24.586 INFO:teuthology.orchestra.run.smithi192.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 588 kB/s | 309 kB 00:00 2024-03-08T23:41:24.590 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T23:41:24.620 INFO:teuthology.orchestra.run.smithi192.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 853 kB/s | 548 kB 00:00 2024-03-08T23:41:24.687 INFO:teuthology.orchestra.run.smithi192.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-03-08T23:41:24.721 INFO:teuthology.orchestra.run.smithi192.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-03-08T23:41:24.796 INFO:teuthology.orchestra.run.smithi192.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-03-08T23:41:24.836 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T23:41:24.838 INFO:teuthology.orchestra.run.smithi192.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-08T23:41:24.874 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T23:41:25.005 INFO:teuthology.orchestra.run.smithi192.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-03-08T23:41:25.030 INFO:teuthology.orchestra.run.smithi192.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-03-08T23:41:25.114 INFO:teuthology.orchestra.run.smithi192.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-03-08T23:41:25.131 INFO:teuthology.orchestra.run.smithi192.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-03-08T23:41:25.215 INFO:teuthology.orchestra.run.smithi192.stdout:(118/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-03-08T23:41:25.231 INFO:teuthology.orchestra.run.smithi192.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-03-08T23:41:25.257 INFO:teuthology.orchestra.run.smithi192.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-08T23:41:25.340 INFO:teuthology.orchestra.run.smithi192.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-08T23:41:25.365 INFO:teuthology.orchestra.run.smithi192.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-03-08T23:41:25.391 INFO:teuthology.orchestra.run.smithi192.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-03-08T23:41:25.441 INFO:teuthology.orchestra.run.smithi192.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-08T23:41:25.491 INFO:teuthology.orchestra.run.smithi192.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-03-08T23:41:25.542 INFO:teuthology.orchestra.run.smithi192.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-03-08T23:41:25.609 INFO:teuthology.orchestra.run.smithi192.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-03-08T23:41:25.634 INFO:teuthology.orchestra.run.smithi192.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 124 kB/s | 18 kB 00:00 2024-03-08T23:41:25.660 INFO:teuthology.orchestra.run.smithi192.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-03-08T23:41:25.710 INFO:teuthology.orchestra.run.smithi192.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-03-08T23:41:25.735 INFO:teuthology.orchestra.run.smithi192.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-03-08T23:41:25.786 INFO:teuthology.orchestra.run.smithi192.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-03-08T23:41:25.811 INFO:teuthology.orchestra.run.smithi192.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 460 kB/s | 46 kB 00:00 2024-03-08T23:41:25.848 INFO:teuthology.orchestra.run.smithi192.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 699 kB/s | 79 kB 00:00 2024-03-08T23:41:25.873 INFO:teuthology.orchestra.run.smithi192.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 659 kB/s | 58 kB 00:00 2024-03-08T23:41:25.890 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T23:41:25.915 INFO:teuthology.orchestra.run.smithi192.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-08T23:41:25.924 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T23:41:25.940 INFO:teuthology.orchestra.run.smithi192.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-08T23:41:25.974 INFO:teuthology.orchestra.run.smithi192.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-03-08T23:41:26.008 INFO:teuthology.orchestra.run.smithi192.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-03-08T23:41:26.029 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T23:41:26.033 INFO:teuthology.orchestra.run.smithi192.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-03-08T23:41:26.075 INFO:teuthology.orchestra.run.smithi192.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-08T23:41:26.109 INFO:teuthology.orchestra.run.smithi192.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-03-08T23:41:26.134 INFO:teuthology.orchestra.run.smithi192.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-03-08T23:41:26.176 INFO:teuthology.orchestra.run.smithi192.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-03-08T23:41:26.203 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 144/198 2024-03-08T23:41:26.209 INFO:teuthology.orchestra.run.smithi192.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-08T23:41:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T23:41:26.243 INFO:teuthology.orchestra.run.smithi192.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-03-08T23:41:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T23:41:26.258 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:26.258 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T23:41:26.258 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T23:41:26.258 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T23:41:26.258 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:26.310 INFO:teuthology.orchestra.run.smithi192.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-03-08T23:41:26.310 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T23:41:26.322 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T23:41:26.335 INFO:teuthology.orchestra.run.smithi192.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-03-08T23:41:26.360 INFO:teuthology.orchestra.run.smithi192.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-08T23:41:26.371 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 147/198 2024-03-08T23:41:26.444 INFO:teuthology.orchestra.run.smithi192.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-03-08T23:41:26.509 INFO:teuthology.orchestra.run.smithi192.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.9 MB/s | 321 kB 00:00 2024-03-08T23:41:26.686 INFO:teuthology.orchestra.run.smithi192.stdout:(152/174): librbd1-19.0.0-1965.g7d63482a.el9.x8 9.6 MB/s | 3.1 MB 00:00 2024-03-08T23:41:26.878 INFO:teuthology.orchestra.run.smithi192.stdout:(153/174): librados2-19.0.0-1965.g7d63482a.el9. 6.2 MB/s | 3.4 MB 00:00 2024-03-08T23:41:27.278 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T23:41:27.304 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T23:41:27.327 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T23:41:27.327 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:27.327 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T23:41:27.327 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T23:41:27.328 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T23:41:27.328 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:27.372 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T23:41:27.394 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T23:41:27.394 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:27.394 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T23:41:27.394 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:27.439 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:41:27.461 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:41:27.461 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:27.461 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T23:41:27.462 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T23:41:27.462 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T23:41:27.462 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:27.620 INFO:teuthology.orchestra.run.smithi192.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 57 kB/s | 42 kB 00:00 2024-03-08T23:41:27.798 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T23:41:27.821 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T23:41:27.822 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:27.822 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T23:41:27.822 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T23:41:27.822 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T23:41:27.822 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:28.587 INFO:teuthology.orchestra.run.smithi192.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 257 kB/s | 248 kB 00:00 2024-03-08T23:41:29.278 INFO:teuthology.orchestra.run.smithi192.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 103 kB/s | 71 kB 00:00 2024-03-08T23:41:29.937 INFO:teuthology.orchestra.run.smithi192.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 573 kB/s | 1.8 MB 00:03 2024-03-08T23:41:29.962 INFO:teuthology.orchestra.run.smithi192.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 53 kB/s | 36 kB 00:00 2024-03-08T23:41:30.188 INFO:teuthology.orchestra.run.smithi192.stdout:(159/174): NetworkManager-1.46.0-1.el9.x86_64.r 637 kB/s | 2.3 MB 00:03 2024-03-08T23:41:30.821 INFO:teuthology.orchestra.run.smithi192.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 218 kB/s | 192 kB 00:00 2024-03-08T23:41:30.846 INFO:teuthology.orchestra.run.smithi192.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 38 kB/s | 23 kB 00:00 2024-03-08T23:41:30.963 INFO:teuthology.orchestra.run.smithi192.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 255 kB/s | 256 kB 00:01 2024-03-08T23:41:31.580 INFO:teuthology.orchestra.run.smithi192.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 143 kB/s | 108 kB 00:00 2024-03-08T23:41:31.689 INFO:teuthology.orchestra.run.smithi192.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 184 kB/s | 155 kB 00:00 2024-03-08T23:41:31.797 INFO:teuthology.orchestra.run.smithi192.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 163 kB/s | 136 kB 00:00 2024-03-08T23:41:32.264 INFO:teuthology.orchestra.run.smithi192.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 93 kB/s | 64 kB 00:00 2024-03-08T23:41:32.289 INFO:teuthology.orchestra.run.smithi192.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 48 kB/s | 29 kB 00:00 2024-03-08T23:41:32.520 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T23:41:32.570 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 154/198 2024-03-08T23:41:32.599 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 155/198 2024-03-08T23:41:32.626 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T23:41:32.857 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 157/198 2024-03-08T23:41:32.980 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T23:41:33.026 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 159/198 2024-03-08T23:41:33.078 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T23:41:33.150 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T23:41:33.185 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T23:41:33.217 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 163/198 2024-03-08T23:41:33.258 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T23:41:33.288 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T23:41:33.353 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T23:41:33.419 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T23:41:33.461 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T23:41:34.078 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:41:34.098 INFO:teuthology.orchestra.run.smithi192.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 539 kB/s | 1.2 MB 00:02 2024-03-08T23:41:34.109 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:41:34.129 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:41:35.039 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 170/198 2024-03-08T23:41:35.112 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T23:41:35.155 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T23:41:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T23:41:35.274 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T23:41:35.299 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T23:41:35.325 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T23:41:35.351 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T23:41:35.351 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T23:41:35.439 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T23:41:35.448 INFO:teuthology.orchestra.run.smithi192.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 345 kB/s | 465 kB 00:01 2024-03-08T23:41:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T23:41:35.469 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T23:41:35.554 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T23:41:35.586 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T23:41:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T23:41:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T23:41:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T23:41:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T23:41:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T23:41:35.768 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T23:41:35.793 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T23:41:35.818 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T23:41:35.845 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T23:41:35.873 INFO:teuthology.orchestra.run.smithi192.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 612 kB/s | 2.2 MB 00:03 2024-03-08T23:41:35.877 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T23:41:35.884 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:41:35.884 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:41:36.064 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T23:41:36.065 INFO:teuthology.orchestra.run.smithi192.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 619 kB/s | 2.3 MB 00:03 2024-03-08T23:41:36.090 INFO:teuthology.orchestra.run.smithi192.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 48 kB/s | 31 kB 00:00 2024-03-08T23:41:36.146 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T23:41:36.165 INFO:teuthology.orchestra.run.smithi192.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 751 kB/s | 74 kB 00:00 2024-03-08T23:41:36.171 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T23:41:36.196 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T23:41:36.221 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T23:41:36.247 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T23:41:36.317 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-03-08T23:41:36.583 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:36.754 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T23:41:36.770 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T23:41:36.771 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:36.771 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T23:41:36.771 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:38.040 INFO:teuthology.orchestra.run.smithi192.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 507 kB/s | 1.1 MB 00:02 2024-03-08T23:41:38.045 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-03-08T23:41:38.045 INFO:teuthology.orchestra.run.smithi192.stdout:Total 8.9 MB/s | 223 MB 00:24 2024-03-08T23:41:38.309 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T23:41:38.336 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T23:41:38.336 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:38.336 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T23:41:38.336 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T23:41:38.336 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T23:41:38.336 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:38.503 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-03-08T23:41:38.568 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T23:41:38.592 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T23:41:38.592 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:38.592 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T23:41:38.592 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T23:41:38.593 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T23:41:38.593 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:38.772 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-03-08T23:41:38.773 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-03-08T23:41:39.052 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T23:41:39.075 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T23:41:39.075 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:39.075 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T23:41:39.076 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T23:41:39.076 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T23:41:39.076 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:39.170 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T23:41:39.514 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T23:41:39.543 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T23:41:39.576 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 139/198 2024-03-08T23:41:39.637 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 140/198 2024-03-08T23:41:39.658 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T23:41:39.903 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T23:41:39.942 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T23:41:40.567 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-03-08T23:41:40.568 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-03-08T23:41:40.957 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T23:41:40.984 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T23:41:41.091 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T23:41:41.126 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 1/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 3/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 4/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 5/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 6/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 7/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 8/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 9/198 2024-03-08T23:41:41.127 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 13/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 14/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 15/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 17/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 18/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 19/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 20/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 21/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 22/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 23/198 2024-03-08T23:41:41.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 24/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 25/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 26/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 27/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 28/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 29/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 30/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 31/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 32/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 33/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 34/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 35/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 36/198 2024-03-08T23:41:41.129 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 37/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 38/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 39/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T23:41:41.130 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T23:41:41.131 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T23:41:41.132 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T23:41:41.133 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T23:41:41.134 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T23:41:41.135 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T23:41:41.136 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T23:41:41.137 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T23:41:41.138 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T23:41:41.139 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T23:41:41.140 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T23:41:41.141 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T23:41:41.142 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T23:41:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 144/198 2024-03-08T23:41:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T23:41:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T23:41:41.311 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:41.311 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T23:41:41.311 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T23:41:41.311 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T23:41:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T23:41:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T23:41:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 147/198 2024-03-08T23:41:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T23:41:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T23:41:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T23:41:42.388 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:42.388 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T23:41:42.388 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T23:41:42.388 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T23:41:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T23:41:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T23:41:42.455 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:42.455 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T23:41:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:41:42.530 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:41:42.530 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:42.531 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T23:41:42.531 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T23:41:42.531 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T23:41:42.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T23:41:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T23:41:42.873 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:41:42.873 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T23:41:42.873 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T23:41:42.873 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T23:41:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:44.206 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T23:41:44.206 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:44.206 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-03-08T23:41:44.206 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:44.206 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.207 INFO:teuthology.orchestra.run.smithi022.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.208 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.209 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T23:41:44.210 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: jq-1.6-16.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T23:41:44.211 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T23:41:44.212 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T23:41:44.213 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T23:41:44.214 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T23:41:44.215 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T23:41:44.216 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T23:41:44.217 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T23:41:44.218 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T23:41:44.219 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T23:41:44.219 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T23:41:44.219 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T23:41:44.219 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T23:41:44.219 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:41:44.219 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-03-08T23:41:44.676 DEBUG:teuthology.parallel:result is None 2024-03-08T23:41:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-03-08T23:41:47.552 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T23:41:47.553 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T23:41:47.586 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T23:41:47.595 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 154/198 2024-03-08T23:41:47.625 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 155/198 2024-03-08T23:41:47.626 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T23:41:47.634 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T23:41:47.644 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T23:41:47.674 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T23:41:47.730 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T23:41:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T23:41:47.796 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T23:41:47.834 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 157/198 2024-03-08T23:41:47.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T23:41:47.949 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T23:41:47.993 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 159/198 2024-03-08T23:41:48.031 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T23:41:48.092 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T23:41:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T23:41:48.158 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T23:41:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 163/198 2024-03-08T23:41:48.192 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T23:41:48.222 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T23:41:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:41:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T23:41:48.342 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T23:41:48.342 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:41:48.372 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T23:41:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:41:48.488 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:41:48.519 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T23:41:48.558 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T23:41:48.744 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T23:41:48.800 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T23:41:48.828 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:41:48.914 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:41:48.938 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T23:41:48.994 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:41:49.001 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T23:41:49.037 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:41:49.049 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T23:41:49.057 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:41:49.072 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T23:41:49.088 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T23:41:49.505 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T23:41:49.546 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T23:41:49.594 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T23:41:49.692 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T23:41:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T23:41:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T23:41:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:41:49.951 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 170/198 2024-03-08T23:41:49.966 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:41:50.032 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T23:41:50.073 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T23:41:50.156 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T23:41:50.172 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T23:41:50.185 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T23:41:50.210 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T23:41:50.236 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T23:41:50.262 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T23:41:50.262 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T23:41:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T23:41:50.359 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T23:41:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T23:41:50.396 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T23:41:50.396 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T23:41:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T23:41:50.430 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T23:41:50.466 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T23:41:50.481 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T23:41:50.507 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T23:41:50.541 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T23:41:50.541 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T23:41:50.566 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T23:41:50.591 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T23:41:50.602 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T23:41:50.624 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T23:41:50.641 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T23:41:50.663 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T23:41:50.696 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T23:41:50.705 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T23:41:50.721 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T23:41:50.739 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T23:41:50.770 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T23:41:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T23:41:50.797 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T23:41:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T23:41:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T23:41:50.829 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:41:50.829 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:41:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T23:41:50.920 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T23:41:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 39/198 2024-03-08T23:41:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 40/198 2024-03-08T23:41:51.016 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:41:51.017 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T23:41:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 41/198 2024-03-08T23:41:51.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T23:41:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T23:41:51.132 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T23:41:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T23:41:51.157 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T23:41:51.191 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T23:41:51.216 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T23:41:51.217 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T23:41:51.283 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T23:41:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T23:41:51.395 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T23:41:51.917 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T23:41:51.990 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T23:41:52.088 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T23:41:52.127 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T23:41:52.135 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T23:41:52.167 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T23:41:52.216 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T23:41:52.254 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 53/198 2024-03-08T23:41:52.284 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T23:41:52.514 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T23:41:52.674 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T23:41:52.728 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T23:41:52.772 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T23:41:52.895 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T23:41:52.950 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T23:41:53.004 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T23:41:53.068 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T23:41:53.100 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T23:41:53.145 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T23:41:53.192 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T23:41:53.228 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T23:41:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T23:41:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T23:41:53.446 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T23:41:53.494 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T23:41:53.523 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T23:41:53.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T23:41:53.658 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T23:41:53.706 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T23:41:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T23:41:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T23:41:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T23:41:53.877 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T23:41:54.014 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T23:41:54.112 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T23:41:54.155 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T23:41:54.196 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T23:41:54.250 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T23:41:54.434 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T23:41:54.476 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T23:41:54.603 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T23:41:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T23:41:54.681 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T23:41:54.712 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T23:41:54.740 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T23:41:54.791 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T23:41:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T23:41:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T23:41:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T23:41:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T23:41:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T23:41:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T23:41:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T23:41:55.103 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T23:41:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T23:41:55.282 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T23:41:55.325 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T23:41:55.357 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T23:41:55.404 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T23:41:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T23:41:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T23:41:56.140 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 1/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 3/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 4/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 5/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 6/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 7/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 8/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 9/198 2024-03-08T23:41:56.141 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 13/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 14/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 15/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 17/198 2024-03-08T23:41:56.142 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 18/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 19/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 20/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 21/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 22/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 23/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 24/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 25/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 26/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 27/198 2024-03-08T23:41:56.143 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 28/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 29/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 30/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 31/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 32/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 33/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 34/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 35/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 36/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 37/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 38/198 2024-03-08T23:41:56.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 39/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T23:41:56.145 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T23:41:56.146 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T23:41:56.147 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T23:41:56.148 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T23:41:56.149 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T23:41:56.150 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T23:41:56.151 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T23:41:56.152 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T23:41:56.153 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T23:41:56.154 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T23:41:56.155 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T23:41:56.156 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T23:41:56.157 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T23:41:56.158 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T23:41:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T23:41:56.160 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T23:41:56.277 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T23:41:56.322 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T23:41:56.356 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T23:41:56.404 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T23:41:56.490 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T23:41:56.531 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T23:41:56.711 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T23:41:56.746 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T23:41:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T23:41:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T23:41:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T23:41:58.771 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T23:41:58.846 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T23:41:58.877 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T23:41:58.908 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T23:41:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T23:41:59.192 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T23:41:59.192 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:59.192 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-03-08T23:41:59.192 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:59.192 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T23:41:59.194 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.195 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T23:41:59.196 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-16.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T23:41:59.197 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T23:41:59.198 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T23:41:59.199 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T23:41:59.200 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T23:41:59.201 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T23:41:59.202 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T23:41:59.203 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T23:41:59.204 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T23:41:59.205 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T23:41:59.205 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-08T23:41:59.205 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-03-08T23:41:59.247 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T23:41:59.274 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T23:41:59.389 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T23:41:59.420 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 125/198 2024-03-08T23:41:59.457 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T23:41:59.633 DEBUG:teuthology.parallel:result is None 2024-03-08T23:41:59.910 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T23:41:59.943 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T23:42:00.014 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T23:42:00.037 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 129/198 2024-03-08T23:42:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:42:02.216 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:42:02.754 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T23:42:02.787 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T23:42:02.867 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T23:42:02.867 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T23:42:02.867 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T23:42:02.867 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:02.896 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T23:42:14.863 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T23:42:14.863 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-03-08T23:42:14.863 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-03-08T23:42:14.863 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-03-08T23:42:14.863 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-03-08T23:42:14.863 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-03-08T23:42:14.863 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-03-08T23:42:14.864 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-03-08T23:42:14.864 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T23:42:15.042 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T23:42:15.042 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:15.042 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T23:42:15.042 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:16.530 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T23:42:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T23:42:16.555 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:16.555 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T23:42:16.555 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T23:42:16.555 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T23:42:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T23:42:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T23:42:16.786 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:16.786 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T23:42:16.786 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T23:42:16.786 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T23:42:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T23:42:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T23:42:17.245 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:17.246 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T23:42:17.246 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T23:42:17.246 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T23:42:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:17.333 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T23:42:17.725 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T23:42:17.781 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T23:42:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 139/198 2024-03-08T23:42:17.873 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 140/198 2024-03-08T23:42:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T23:42:18.128 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T23:42:18.163 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T23:42:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T23:42:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T23:42:19.247 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T23:42:19.409 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 144/198 2024-03-08T23:42:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T23:42:19.463 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T23:42:19.463 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:19.463 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T23:42:19.464 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T23:42:19.464 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T23:42:19.464 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:19.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T23:42:19.519 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T23:42:19.564 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 147/198 2024-03-08T23:42:20.428 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T23:42:20.468 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T23:42:20.490 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T23:42:20.491 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:20.491 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T23:42:20.491 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T23:42:20.491 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T23:42:20.491 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:20.536 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T23:42:20.557 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T23:42:20.557 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:20.557 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T23:42:20.557 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:20.611 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:42:20.633 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:42:20.633 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:20.633 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T23:42:20.633 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T23:42:20.633 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T23:42:20.633 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:20.937 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T23:42:20.959 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T23:42:20.959 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T23:42:20.959 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T23:42:20.959 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T23:42:20.959 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T23:42:20.959 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:25.391 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T23:42:25.432 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 154/198 2024-03-08T23:42:25.463 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 155/198 2024-03-08T23:42:25.490 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T23:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 157/198 2024-03-08T23:42:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T23:42:25.821 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 159/198 2024-03-08T23:42:25.860 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T23:42:25.913 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T23:42:25.956 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T23:42:25.980 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 163/198 2024-03-08T23:42:26.032 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T23:42:26.058 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T23:42:26.112 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T23:42:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T23:42:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T23:42:26.794 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:42:26.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:42:26.857 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T23:42:27.673 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 170/198 2024-03-08T23:42:27.758 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T23:42:27.799 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T23:42:27.880 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T23:42:27.920 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T23:42:27.945 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T23:42:27.970 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T23:42:27.996 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T23:42:27.997 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T23:42:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T23:42:28.122 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T23:42:28.122 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T23:42:28.190 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T23:42:28.217 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T23:42:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T23:42:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T23:42:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T23:42:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T23:42:28.372 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T23:42:28.405 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T23:42:28.430 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T23:42:28.455 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T23:42:28.480 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T23:42:28.506 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T23:42:28.512 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:42:28.512 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:42:28.711 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T23:42:28.712 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T23:42:28.792 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T23:42:28.817 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T23:42:28.841 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T23:42:28.874 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T23:42:28.901 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T23:42:28.965 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 1/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 3/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 4/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 5/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 6/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 7/198 2024-03-08T23:42:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 8/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 9/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 13/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 14/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 15/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 17/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 18/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 19/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 20/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 21/198 2024-03-08T23:42:33.603 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 22/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 23/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 24/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 25/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 26/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 27/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 28/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 29/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 30/198 2024-03-08T23:42:33.604 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 31/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 32/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 33/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 34/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 35/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 36/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 37/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 38/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 39/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T23:42:33.621 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T23:42:33.622 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T23:42:33.623 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T23:42:33.624 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T23:42:33.625 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T23:42:33.626 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T23:42:33.627 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T23:42:33.628 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T23:42:33.629 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T23:42:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T23:42:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T23:42:33.632 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T23:42:33.633 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T23:42:33.633 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T23:42:33.633 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T23:42:33.633 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T23:42:36.660 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.661 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T23:42:36.662 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.663 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T23:42:36.664 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-16.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.665 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T23:42:36.666 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T23:42:36.667 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T23:42:36.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T23:42:36.669 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T23:42:36.670 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T23:42:36.671 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T23:42:36.672 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.673 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:42:36.674 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-03-08T23:42:37.282 DEBUG:teuthology.parallel:result is None 2024-03-08T23:42:37.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:42:37.426 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T23:42:37.451 INFO:teuthology.orchestra.run.smithi022.stdout:19.0.0-1965.g7d63482a.el9 2024-03-08T23:42:37.452 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1965.g7d63482a.el9 2024-03-08T23:42:37.452 INFO:teuthology.task.install:The correct ceph version 19.0.0-1965.g7d63482a is installed. 2024-03-08T23:42:37.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:42:37.600 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T23:42:37.626 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-1965.g7d63482a.el9 2024-03-08T23:42:37.627 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1965.g7d63482a.el9 2024-03-08T23:42:37.627 INFO:teuthology.task.install:The correct ceph version 19.0.0-1965.g7d63482a is installed. 2024-03-08T23:42:37.629 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:42:37.766 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T23:42:37.792 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-1965.g7d63482a.el9 2024-03-08T23:42:37.792 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1965.g7d63482a.el9 2024-03-08T23:42:37.792 INFO:teuthology.task.install:The correct ceph version 19.0.0-1965.g7d63482a is installed. 2024-03-08T23:42:37.794 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-08T23:42:37.794 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:42:37.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T23:42:37.828 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:42:37.828 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T23:42:37.868 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:42:37.868 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T23:42:37.903 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-08T23:42:37.903 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:42:37.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T23:42:37.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T23:42:38.004 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:42:38.004 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T23:42:38.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T23:42:38.115 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:42:38.115 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T23:42:38.146 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T23:42:38.217 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-08T23:42:38.217 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:42:38.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T23:42:38.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T23:42:38.313 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:42:38.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T23:42:38.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T23:42:38.415 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:42:38.415 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T23:42:38.446 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T23:42:38.517 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-08T23:42:38.517 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:42:38.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T23:42:38.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T23:42:38.611 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:42:38.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T23:42:38.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T23:42:38.714 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:42:38.714 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T23:42:38.745 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T23:42:38.815 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-08T23:42:38.912 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'subvols': {'create': 2}} 2024-03-08T23:42:38.912 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:42:38.912 INFO:tasks.cephadm:Cluster fsid is 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:42:38.912 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-08T23:42:38.913 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.22', 'mon.b': '172.21.15.62', 'mon.c': '172.21.15.192'} 2024-03-08T23:42:38.913 INFO:tasks.cephadm:First mon is mon.a on smithi022 2024-03-08T23:42:38.913 INFO:tasks.cephadm:First mgr is x 2024-03-08T23:42:38.913 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-08T23:42:38.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-03-08T23:42:38.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-03-08T23:42:38.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-03-08T23:42:39.003 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-08T23:42:39.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:42:39.217 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77443/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1965-g7d63482a', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1965.g7d63482a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 17:09:11.137784', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2024mar8', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-08T23:42:39.354 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref rishabh-2024mar8, sha1 7d63482aa89628d733e298984a762389b625c761 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:42:39.356 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm 2024-03-08T23:42:39.356 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:40.348 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 767402 Mar 8 23:42 /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:40.348 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:41.187 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 767402 Mar 8 23:42 /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:41.187 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:42.046 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 767402 Mar 8 23:42 /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:42.047 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:42.065 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:42.084 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-08T23:42:42.119 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 on all hosts... 2024-03-08T23:42:42.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 pull 2024-03-08T23:42:42.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 pull 2024-03-08T23:42:42.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 pull 2024-03-08T23:42:42.871 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T23:42:43.030 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T23:42:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T23:43:17.320 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-08T23:43:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T23:43:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249", 2024-03-08T23:43:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-03-08T23:43:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd" 2024-03-08T23:43:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-03-08T23:43:17.321 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-08T23:43:30.084 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:43:30.084 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T23:43:30.084 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249", 2024-03-08T23:43:30.084 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-03-08T23:43:30.084 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd" 2024-03-08T23:43:30.084 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-08T23:43:30.084 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:43:31.537 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-08T23:43:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T23:43:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249", 2024-03-08T23:43:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-03-08T23:43:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd" 2024-03-08T23:43:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-03-08T23:43:31.538 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-08T23:43:31.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-03-08T23:43:31.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-03-08T23:43:31.640 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-03-08T23:43:31.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-03-08T23:43:31.702 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-03-08T23:43:31.737 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-03-08T23:43:31.764 INFO:tasks.cephadm:Writing seed config... 2024-03-08T23:43:31.765 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-08T23:43:31.766 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-08T23:43:31.767 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-08T23:43:31.768 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-08T23:43:31.769 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-08T23:43:31.770 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-08T23:43:31.771 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:43:31.771 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-08T23:43:31.789 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8c5a3016-dda5-11ee-95c6-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-08T23:43:31.789 DEBUG:teuthology.orchestra.run.smithi022:mon.a> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a.service 2024-03-08T23:43:31.832 DEBUG:teuthology.orchestra.run.smithi022:mgr.x> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.x.service 2024-03-08T23:43:31.875 INFO:tasks.cephadm:Bootstrapping... 2024-03-08T23:43:31.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 -v bootstrap --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.22 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-08T23:43:32.076 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-03-08T23:43:32.076 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761', '-v', 'bootstrap', '--fsid', '8c5a3016-dda5-11ee-95c6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.22', '--skip-admin-label'] 2024-03-08T23:43:32.102 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-08T23:43:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-08T23:43:32.102 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2024-03-08T23:43:32.125 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-08T23:43:32.125 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2024-03-08T23:43:32.125 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2024-03-08T23:43:32.132 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-08T23:43:32.132 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-08T23:43:32.138 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-08T23:43:32.139 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-08T23:43:32.144 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-03-08T23:43:32.150 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-03-08T23:43:32.150 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-03-08T23:43:32.150 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2024-03-08T23:43:32.173 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-08T23:43:32.173 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-08T23:43:32.173 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2024-03-08T23:43:32.173 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2024-03-08T23:43:32.179 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-08T23:43:32.179 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-08T23:43:32.185 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-08T23:43:32.185 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-08T23:43:32.191 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-03-08T23:43:32.197 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-03-08T23:43:32.198 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-03-08T23:43:32.198 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2024-03-08T23:43:32.198 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:43:32.198 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 139919253196224 on /run/cephadm/8c5a3016-dda5-11ee-95c6-87774f69a715.lock 2024-03-08T23:43:32.198 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 139919253196224 acquired on /run/cephadm/8c5a3016-dda5-11ee-95c6-87774f69a715.lock 2024-03-08T23:43:32.198 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 3300 ... 2024-03-08T23:43:32.199 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2024-03-08T23:43:32.199 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:3300, 172.21.15.22:6789], mon addrv is [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-03-08T23:43:32.202 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.22 metric 100 2024-03-08T23:43:32.202 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2024-03-08T23:43:32.204 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-08T23:43:32.205 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-08T23:43:32.205 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-08T23:43:32.207 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-08T23:43:32.207 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-08T23:43:32.207 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-08T23:43:32.207 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-08T23:43:32.207 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-03-08T23:43:32.207 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-08T23:43:32.208 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-03-08T23:43:32.208 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-03-08T23:43:32.208 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-08T23:43:32.208 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-08T23:43:32.209 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T23:43:32.645 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 2024-03-08T23:43:32.645 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T23:43:32.645 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2024-03-08T23:43:32.645 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:7a785ebef89f89c1f7e1e5df571ab58da14c499794e7964e586765517f59740d 2024-03-08T23:43:32.645 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-08T23:43:32.645 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 2024-03-08T23:43:32.645 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-08T23:43:33.882 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T23:43:33.882 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T23:43:33.882 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2024-03-08T23:43:34.828 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2024-03-08T23:43:34.829 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2024-03-08T23:43:35.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQCnoutlVmUSCRAA7IsZGBpGN/09WJjFDQkUbA== 2024-03-08T23:43:36.664 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQCooutlfyDDARAA3LeMjRHYmYGt6s4Ei9juRg== 2024-03-08T23:43:37.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQCooutl8my2ORAAqcFSPBls2A6YnHmuQemlyA== 2024-03-08T23:43:37.538 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2024-03-08T23:43:38.402 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for a [v2:172.21.15.22:3300,v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:43:38.403 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2024-03-08T23:43:39.492 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.a on 2024-03-08T23:43:39.693 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-08T23:43:39.945 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-08T23:43:40.143 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715.target. 2024-03-08T23:43:40.143 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715.target. 2024-03-08T23:43:40.391 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a 2024-03-08T23:43:40.392 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a.service: Unit ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a.service not loaded. 2024-03-08T23:43:40.603 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715.target.wants/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@.service. 2024-03-08T23:43:41.163 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T23:43:41.164 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-08T23:43:41.170 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T23:43:41.170 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-08T23:43:41.170 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-08T23:43:41.170 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-03-08T23:43:41.171 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2024-03-08T23:43:41.171 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2024-03-08T23:43:41.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:41 smithi022 systemd[1]: Started Ceph mon.a for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:43:42.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:42 smithi022 ceph-mon[31526]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-08T23:43:42.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:42 smithi022 ceph-mon[31526]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:43:42.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:42 smithi022 ceph-mon[31526]: fsmap 2024-03-08T23:43:42.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:42 smithi022 ceph-mon[31526]: osdmap e1: 0 total, 0 up, 0 in 2024-03-08T23:43:42.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:42 smithi022 ceph-mon[31526]: mgrmap e1: no daemons active 2024-03-08T23:43:42.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:42 smithi022 ceph-mon[31526]: from='client.? 172.21.15.22:0/4249620200' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.570459s) 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-08T23:43:42.301 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.630+0000 7ff59c2e6700 1 Processor -- start 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.630+0000 7ff59c2e6700 1 -- start start 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.630+0000 7ff59c2e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff594106aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.630+0000 7ff59c2e6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594107070 con 0x7ff594106680 2024-03-08T23:43:42.302 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff59a082700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff594106aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff59a082700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff594106aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43860/0 (socket says 172.21.15.22:43860) 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff59a082700 1 -- 172.21.15.22:0/1821209653 learned_addr learned my addr 172.21.15.22:0/1821209653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff59a082700 1 -- 172.21.15.22:0/1821209653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594107880 con 0x7ff594106680 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff59a082700 1 --2- 172.21.15.22:0/1821209653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff594106aa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0098a0 tx=0x7ff58c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b17cd83c30dbaaa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff599080700 1 -- 172.21.15.22:0/1821209653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff58c004160 con 0x7ff594106680 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff599080700 1 -- 172.21.15.22:0/1821209653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c004300 con 0x7ff594106680 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.631+0000 7ff599080700 1 -- 172.21.15.22:0/1821209653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff58c0036a0 con 0x7ff594106680 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.632+0000 7ff59c2e6700 1 -- 172.21.15.22:0/1821209653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 msgr2=0x7ff594106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.632+0000 7ff59c2e6700 1 --2- 172.21.15.22:0/1821209653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff594106aa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0098a0 tx=0x7ff58c009bb0 comp rx=0 tx=0).stop 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.632+0000 7ff59c2e6700 1 -- 172.21.15.22:0/1821209653 shutdown_connections 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.632+0000 7ff59c2e6700 1 --2- 172.21.15.22:0/1821209653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff594106aa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.632+0000 7ff59c2e6700 1 -- 172.21.15.22:0/1821209653 >> 172.21.15.22:0/1821209653 conn(0x7ff594101cc0 msgr2=0x7ff5941040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:42.303 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.632+0000 7ff59c2e6700 1 -- 172.21.15.22:0/1821209653 shutdown_connections 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.632+0000 7ff59c2e6700 1 -- 172.21.15.22:0/1821209653 wait complete. 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59c2e6700 1 Processor -- start 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59c2e6700 1 -- start start 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59c2e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff59407aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59c2e6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594107db0 con 0x7ff594106680 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59a082700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff59407aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59a082700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff59407aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43876/0 (socket says 172.21.15.22:43876) 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59a082700 1 -- 172.21.15.22:0/4249620200 learned_addr learned my addr 172.21.15.22:0/4249620200 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59a082700 1 -- 172.21.15.22:0/4249620200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59419dba0 con 0x7ff594106680 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59a082700 1 --2- 172.21.15.22:0/4249620200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff59407aad0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff58c000c00 tx=0x7ff58c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff5937fe700 1 -- 172.21.15.22:0/4249620200 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff58c003db0 con 0x7ff594106680 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff5937fe700 1 -- 172.21.15.22:0/4249620200 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c011630 con 0x7ff594106680 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59419dd70 con 0x7ff594106680 2024-03-08T23:43:42.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.633+0000 7ff5937fe700 1 -- 172.21.15.22:0/4249620200 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff58c011950 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.634+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59419e190 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.634+0000 7ff5937fe700 1 -- 172.21.15.22:0/4249620200 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff58c01a460 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.634+0000 7ff5937fe700 1 -- 172.21.15.22:0/4249620200 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff58c01b780 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.634+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594068eb0 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.637+0000 7ff5937fe700 1 -- 172.21.15.22:0/4249620200 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ff58c031020 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.712+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff5940008d0 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.712+0000 7ff5937fe700 1 -- 172.21.15.22:0/4249620200 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7ff58c01ba20 con 0x7ff594106680 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.713+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 msgr2=0x7ff59407aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.713+0000 7ff59c2e6700 1 --2- 172.21.15.22:0/4249620200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff59407aad0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff58c000c00 tx=0x7ff58c004750 comp rx=0 tx=0).stop 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.713+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 shutdown_connections 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.713+0000 7ff59c2e6700 1 --2- 172.21.15.22:0/4249620200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff594106680 0x7ff59407aad0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.713+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 >> 172.21.15.22:0/4249620200 conn(0x7ff594101cc0 msgr2=0x7ff5941909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:42.305 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.713+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 shutdown_connections 2024-03-08T23:43:42.306 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:41.713+0000 7ff59c2e6700 1 -- 172.21.15.22:0/4249620200 wait complete. 2024-03-08T23:43:42.306 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2024-03-08T23:43:42.306 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2024-03-08T23:43:43.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:43 smithi022 ceph-mon[31526]: from='client.? 172.21.15.22:0/4010128368' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-08T23:43:43.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:43 smithi022 ceph-mon[31526]: from='client.? 172.21.15.22:0/4010128368' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-08T23:43:43.284 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-08T23:43:43.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.673+0000 7f77d78c9700 1 Processor -- start 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.673+0000 7f77d78c9700 1 -- start start 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.673+0000 7f77d78c9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d0107c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.673+0000 7f77d78c9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0108230 con 0x7f77d0107840 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.673+0000 7f77d5665700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d0107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.673+0000 7f77d5665700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d0107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43884/0 (socket says 172.21.15.22:43884) 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.673+0000 7f77d5665700 1 -- 172.21.15.22:0/3390863669 learned_addr learned my addr 172.21.15.22:0/3390863669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77d5665700 1 -- 172.21.15.22:0/3390863669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d01089f0 con 0x7f77d0107840 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77d5665700 1 --2- 172.21.15.22:0/3390863669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d0107c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0098a0 tx=0x7f77cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40aabfdd118ad5d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77c7fff700 1 -- 172.21.15.22:0/3390863669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77cc004160 con 0x7f77d0107840 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77c7fff700 1 -- 172.21.15.22:0/3390863669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc004300 con 0x7f77d0107840 2024-03-08T23:43:43.286 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77c7fff700 1 -- 172.21.15.22:0/3390863669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77cc0036a0 con 0x7f77d0107840 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77d78c9700 1 -- 172.21.15.22:0/3390863669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 msgr2=0x7f77d0107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77d78c9700 1 --2- 172.21.15.22:0/3390863669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d0107c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0098a0 tx=0x7f77cc009bb0 comp rx=0 tx=0).stop 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77d78c9700 1 -- 172.21.15.22:0/3390863669 shutdown_connections 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.674+0000 7f77d78c9700 1 --2- 172.21.15.22:0/3390863669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d0107c60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.675+0000 7f77d78c9700 1 -- 172.21.15.22:0/3390863669 >> 172.21.15.22:0/3390863669 conn(0x7f77d0102e00 msgr2=0x7f77d0105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.675+0000 7f77d78c9700 1 -- 172.21.15.22:0/3390863669 shutdown_connections 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.675+0000 7f77d78c9700 1 -- 172.21.15.22:0/3390863669 wait complete. 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.675+0000 7f77d78c9700 1 Processor -- start 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.675+0000 7f77d78c9700 1 -- start start 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.675+0000 7f77d78c9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.675+0000 7f77d78c9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0108f20 con 0x7f77d0107840 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77d5665700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:43.287 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77d5665700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43886/0 (socket says 172.21.15.22:43886) 2024-03-08T23:43:43.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77d5665700 1 -- 172.21.15.22:0/4010128368 learned_addr learned my addr 172.21.15.22:0/4010128368 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:43.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77d5665700 1 -- 172.21.15.22:0/4010128368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d019b430 con 0x7f77d0107840 2024-03-08T23:43:43.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77d5665700 1 --2- 172.21.15.22:0/4010128368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d019aef0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f77cc000c00 tx=0x7f77cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:43.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77cc003db0 con 0x7f77d0107840 2024-03-08T23:43:43.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc011630 con 0x7f77d0107840 2024-03-08T23:43:43.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d019b6c0 con 0x7f77d0107840 2024-03-08T23:43:43.288 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77cc011950 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d019e3b0 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f77cc01a460 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.676+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f77cc01b780 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.677+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d0068eb0 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.679+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f77cc031020 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.750+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f77d019bba0 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.754+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f77d019bba0 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.754+0000 7f77c67fc700 1 -- 172.21.15.22:0/4010128368 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77cc01a8e0 con 0x7f77d0107840 2024-03-08T23:43:43.289 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.755+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 msgr2=0x7f77d019aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:43.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.755+0000 7f77d78c9700 1 --2- 172.21.15.22:0/4010128368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d019aef0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f77cc000c00 tx=0x7f77cc004750 comp rx=0 tx=0).stop 2024-03-08T23:43:43.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.755+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 shutdown_connections 2024-03-08T23:43:43.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.755+0000 7f77d78c9700 1 --2- 172.21.15.22:0/4010128368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d0107840 0x7f77d019aef0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:43.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.755+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 >> 172.21.15.22:0/4010128368 conn(0x7f77d0102e00 msgr2=0x7f77d0191b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:43.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.755+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 shutdown_connections 2024-03-08T23:43:43.290 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:42.755+0000 7f77d78c9700 1 -- 172.21.15.22:0/4010128368 wait complete. 2024-03-08T23:43:43.291 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2024-03-08T23:43:44.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:44 smithi022 ceph-mon[31526]: from='client.? 172.21.15.22:0/347514469' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.715+0000 7f8cb0871700 1 Processor -- start 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.715+0000 7f8cb0871700 1 -- start start 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.716+0000 7f8cb0871700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac107c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.716+0000 7f8cb0871700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac108230 con 0x7f8cac107840 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.716+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.716+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43892/0 (socket says 172.21.15.22:43892) 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.716+0000 7f8caa59c700 1 -- 172.21.15.22:0/1894711151 learned_addr learned my addr 172.21.15.22:0/1894711151 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:46.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.716+0000 7f8caa59c700 1 -- 172.21.15.22:0/1894711151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac1089f0 con 0x7f8cac107840 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.716+0000 7f8caa59c700 1 --2- 172.21.15.22:0/1894711151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac107c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098a0 tx=0x7f8c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb984e0ca9fb0af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8ca959a700 1 -- 172.21.15.22:0/1894711151 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c004160 con 0x7f8cac107840 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8ca959a700 1 -- 172.21.15.22:0/1894711151 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c9c004300 con 0x7f8cac107840 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8ca959a700 1 -- 172.21.15.22:0/1894711151 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c011b50 con 0x7f8cac107840 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8cb0871700 1 -- 172.21.15.22:0/1894711151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 msgr2=0x7f8cac107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8cb0871700 1 --2- 172.21.15.22:0/1894711151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac107c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098a0 tx=0x7f8c9c009bb0 comp rx=0 tx=0).stop 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8cb0871700 1 -- 172.21.15.22:0/1894711151 shutdown_connections 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8cb0871700 1 --2- 172.21.15.22:0/1894711151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac107c60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8cb0871700 1 -- 172.21.15.22:0/1894711151 >> 172.21.15.22:0/1894711151 conn(0x7f8cac102e00 msgr2=0x7f8cac105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8cb0871700 1 -- 172.21.15.22:0/1894711151 shutdown_connections 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.717+0000 7f8cb0871700 1 -- 172.21.15.22:0/1894711151 wait complete. 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.718+0000 7f8cb0871700 1 Processor -- start 2024-03-08T23:43:46.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.718+0000 7f8cb0871700 1 -- start start 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.718+0000 7f8cb0871700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.718+0000 7f8cb0871700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac10bda0 con 0x7f8cac107840 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.718+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.718+0000 7f8caa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43908/0 (socket says 172.21.15.22:43908) 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.718+0000 7f8caa59c700 1 -- 172.21.15.22:0/347514469 learned_addr learned my addr 172.21.15.22:0/347514469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8caa59c700 1 -- 172.21.15.22:0/347514469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac19bcb0 con 0x7f8cac107840 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8caa59c700 1 --2- 172.21.15.22:0/347514469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac19b770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009fb0 tx=0x7f8c9c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8ca37fe700 1 -- 172.21.15.22:0/347514469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c0038b0 con 0x7f8cac107840 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8ca37fe700 1 -- 172.21.15.22:0/347514469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c9c011e50 con 0x7f8cac107840 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac19bf40 con 0x7f8cac107840 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8ca37fe700 1 -- 172.21.15.22:0/347514469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c019550 con 0x7f8cac107840 2024-03-08T23:43:46.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac19ec30 con 0x7f8cac107840 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8ca37fe700 1 -- 172.21.15.22:0/347514469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8c9c019b90 con 0x7f8cac107840 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.719+0000 7f8ca37fe700 1 -- 172.21.15.22:0/347514469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8c9c024730 con 0x7f8cac107840 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.720+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac068eb0 con 0x7f8cac107840 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.722+0000 7f8ca37fe700 1 -- 172.21.15.22:0/347514469 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f8c9c003a50 con 0x7f8cac107840 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.793+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8cac19ee00 con 0x7f8cac107840 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.793+0000 7f8ca37fe700 1 -- 172.21.15.22:0/347514469 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f8c9c024930 con 0x7f8cac107840 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.794+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 msgr2=0x7f8cac19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.794+0000 7f8cb0871700 1 --2- 172.21.15.22:0/347514469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac19b770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009fb0 tx=0x7f8c9c011bf0 comp rx=0 tx=0).stop 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.794+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 shutdown_connections 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.794+0000 7f8cb0871700 1 --2- 172.21.15.22:0/347514469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cac107840 0x7f8cac19b770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.794+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 >> 172.21.15.22:0/347514469 conn(0x7f8cac102e00 msgr2=0x7f8cac103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:46.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.794+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 shutdown_connections 2024-03-08T23:43:46.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:43.794+0000 7f8cb0871700 1 -- 172.21.15.22:0/347514469 wait complete. 2024-03-08T23:43:46.319 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2024-03-08T23:43:46.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:46 smithi022 systemd[1]: Stopping Ceph mon.a for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:43:47.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a[31521]: 2024-03-08T23:43:47.443+0000 7f261bde8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T23:43:47.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a[31521]: 2024-03-08T23:43:47.443+0000 7f261bde8700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-08T23:43:47.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:47 smithi022 podman[31772]: 2024-03-08 23:43:47.475146904 +0000 UTC m=+0.993356073 container died 6fe6802930583cb627d938c22e92dffa719d15b2de4ddbcdab5cdc8fbdd772a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-03-08T23:43:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:50 smithi022 podman[31772]: 2024-03-08 23:43:50.997227655 +0000 UTC m=+4.515436821 container cleanup 6fe6802930583cb627d938c22e92dffa719d15b2de4ddbcdab5cdc8fbdd772a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-08T23:43:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:50 smithi022 bash[31772]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a 2024-03-08T23:43:53.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 podman[31785]: 2024-03-08 23:43:53.055450895 +0000 UTC m=+5.580437475 container remove 6fe6802930583cb627d938c22e92dffa719d15b2de4ddbcdab5cdc8fbdd772a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-08T23:43:53.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-08T23:43:53.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 systemd[1]: Stopped Ceph mon.a for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:43:53.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 systemd[1]: Starting Ceph mon.a for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:43:53.745 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-08T23:43:53.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 podman[31875]: 2024-03-08 23:43:53.498425465 +0000 UTC m=+0.111110132 container create 6c0c3ed4de50525bd6007c4b9cf987d119edf28a0ff2400f4590441d21e0515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T23:43:53.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 podman[31875]: 2024-03-08 23:43:53.404975938 +0000 UTC m=+0.017660604 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 podman[31875]: 2024-03-08 23:43:53.667213509 +0000 UTC m=+0.279898176 container init 6c0c3ed4de50525bd6007c4b9cf987d119edf28a0ff2400f4590441d21e0515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 podman[31875]: 2024-03-08 23:43:53.669998971 +0000 UTC m=+0.282683640 container start 6c0c3ed4de50525bd6007c4b9cf987d119edf28a0ff2400f4590441d21e0515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: set uid:gid to 167:167 (ceph:ceph) 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev), process ceph-mon, pid 2 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: pidfile_write: ignore empty --pid-file 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: load: jerasure load: lrc 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: RocksDB version: 7.9.2 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Git sha 0 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Compile date 2024-03-08 15:06:51 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: DB SUMMARY 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: DB Session ID: 35IP9GBP1QZEMM3QGDQ4 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: CURRENT file: CURRENT 2024-03-08T23:43:53.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: IDENTITY file: IDENTITY 2024-03-08T23:43:53.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-08T23:43:53.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-08T23:43:53.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100294 ; 2024-03-08T23:43:53.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.error_if_exists: 0 2024-03-08T23:43:53.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.create_if_missing: 0 2024-03-08T23:43:53.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.paranoid_checks: 1 2024-03-08T23:43:53.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-08T23:43:53.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-08T23:43:53.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.env: 0x565150441b20 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.fs: PosixFileSystem 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.info_log: 0x5651522cd8e0 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_file_opening_threads: 16 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.statistics: (nil) 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.use_fsync: 0 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_log_file_size: 0 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.keep_log_file_num: 1000 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.recycle_log_file_num: 0 2024-03-08T23:43:53.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.allow_fallocate: 1 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.allow_mmap_reads: 0 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.allow_mmap_writes: 0 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.use_direct_reads: 0 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.create_missing_column_families: 0 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.db_log_dir: 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.wal_dir: 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-08T23:43:53.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.advise_random_on_open: 1 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.db_write_buffer_size: 0 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.write_buffer_manager: 0x5651522a1400 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.rate_limiter: (nil) 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.wal_recovery_mode: 2 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.enable_thread_tracking: 0 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.enable_pipelined_write: 0 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.unordered_write: 0 2024-03-08T23:43:53.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.row_cache: None 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.wal_filter: None 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.allow_ingest_behind: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.two_write_queues: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.manual_wal_flush: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.wal_compression: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.atomic_flush: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-08T23:43:53.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.log_readahead_size: 0 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.best_efforts_recovery: 0 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.allow_data_in_errors: 0 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.db_host_id: __hostname__ 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_background_jobs: 2 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_background_compactions: -1 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_subcompactions: 1 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-08T23:43:53.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_total_wal_size: 0 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_open_files: -1 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bytes_per_sync: 0 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_readahead_size: 0 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_background_flushes: -1 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Compression algorithms supported: 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kZSTD supported: 0 2024-03-08T23:43:53.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kXpressCompression supported: 0 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kLZ4HCCompression supported: 1 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kZlibCompression supported: 1 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kSnappyCompression supported: 1 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kLZ4Compression supported: 1 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: kBZip2Compression supported: 0 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.merge_operator: 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_filter: None 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_filter_factory: None 2024-03-08T23:43:53.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.sst_partitioner_factory: None 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5651522cd9e0) 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_type: 0 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_index_type: 0 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_shortening: 1 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: checksum: 4 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: no_block_cache: 0 2024-03-08T23:43:53.777 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache: 0x5651522d3090 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_options: 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: capacity : 536870912 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_shard_bits : 4 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: strict_capacity_limit : 0 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_compressed: (nil) 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: persistent_cache: (nil) 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size: 4096 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size_deviation: 10 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_restart_interval: 16 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_block_restart_interval: 1 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: metadata_block_size: 4096 2024-03-08T23:43:53.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: partition_filters: 0 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: use_delta_encoding: 1 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: filter_policy: bloomfilter 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: whole_key_filtering: 1 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: verify_compression: 0 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: format_version: 5 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_index_compression: 1 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_align: 0 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: max_auto_readahead_size: 262144 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: prepopulate_block_cache: 0 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: initial_auto_readahead_size: 8192 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.write_buffer_size: 33554432 2024-03-08T23:43:53.779 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_write_buffer_number: 2 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression: NoCompression 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression: Disabled 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.prefix_extractor: nullptr 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.num_levels: 7 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-08T23:43:53.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.level: 32767 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.strategy: 0 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.enabled: false 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-08T23:43:53.781 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.target_file_size_base: 67108864 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-08T23:43:53.782 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.arena_block_size: 1048576 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.disable_auto_compactions: 0 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-08T23:43:53.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.table_properties_collectors: 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.inplace_update_support: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.bloom_locality: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.max_successive_merges: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.paranoid_file_checks: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.force_consistency_checks: 1 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.report_bg_io_stats: 0 2024-03-08T23:43:53.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.ttl: 2592000 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.enable_blob_files: false 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.min_blob_size: 0 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.blob_file_size: 268435456 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.blob_file_starting_level: 0 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-08T23:43:53.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a0b4535c-521e-4afc-ac40-d0062531f8e0 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709941433698525, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709941433699482, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95428, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93418, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13914, "raw_average_key_size": 50, "raw_value_size": 85851, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709941433, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a0b4535c-521e-4afc-ac40-d0062531f8e0", "db_session_id": "35IP9GBP1QZEMM3QGDQ4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709941433699564, "job": 1, "event": "recovery_finished"} 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5651523c6000 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: DB pointer 0x5651523b0000 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-08T23:43:53.786 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** DB Stats ** 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 95.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T23:43:53.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 95.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-08T23:43:53.788 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 30.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 30.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache BinnedLRUCache@0x5651522d3090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: starting mon.a rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???) e1 preinit fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???).mds e1 new map 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???).mds e1 print_map 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: e1 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-08T23:43:53.789 INFO:journalctl@ceph.mon.a.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout: legacy client fscid: -1 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout: No filesystems configured 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 bash[31875]: 6c0c3ed4de50525bd6007c4b9cf987d119edf28a0ff2400f4590441d21e0515b 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 systemd[1]: Started Ceph mon.a for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:43:53.790 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: fsmap 2024-03-08T23:43:53.791 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: osdmap e1: 0 total, 0 up, 0 in 2024-03-08T23:43:53.791 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:53 smithi022 ceph-mon[31894]: mgrmap e1: no daemons active 2024-03-08T23:43:54.778 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.165+0000 7f16c6e6e700 1 Processor -- start 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.165+0000 7f16c6e6e700 1 -- start start 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.165+0000 7f16c6e6e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c0105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.165+0000 7f16c6e6e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0106450 con 0x7f16c0105a60 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.165+0000 7f16c4c0a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c0105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.165+0000 7f16c4c0a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c0105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52902/0 (socket says 172.21.15.22:52902) 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.165+0000 7f16c4c0a700 1 -- 172.21.15.22:0/1469842817 learned_addr learned my addr 172.21.15.22:0/1469842817 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16c4c0a700 1 -- 172.21.15.22:0/1469842817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c01089f0 con 0x7f16c0105a60 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16c4c0a700 1 --2- 172.21.15.22:0/1469842817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c0105e80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f16b40098a0 tx=0x7f16b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac12f103eed174da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16bf7fe700 1 -- 172.21.15.22:0/1469842817 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b4004160 con 0x7f16c0105a60 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16bf7fe700 1 -- 172.21.15.22:0/1469842817 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b4004300 con 0x7f16c0105a60 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16bf7fe700 1 -- 172.21.15.22:0/1469842817 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b4011b50 con 0x7f16c0105a60 2024-03-08T23:43:54.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16c6e6e700 1 -- 172.21.15.22:0/1469842817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 msgr2=0x7f16c0105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16c6e6e700 1 --2- 172.21.15.22:0/1469842817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c0105e80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f16b40098a0 tx=0x7f16b4009bb0 comp rx=0 tx=0).stop 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16c6e6e700 1 -- 172.21.15.22:0/1469842817 shutdown_connections 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16c6e6e700 1 --2- 172.21.15.22:0/1469842817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c0105e80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.166+0000 7f16c6e6e700 1 -- 172.21.15.22:0/1469842817 >> 172.21.15.22:0/1469842817 conn(0x7f16c0101390 msgr2=0x7f16c01037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c6e6e700 1 -- 172.21.15.22:0/1469842817 shutdown_connections 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c6e6e700 1 -- 172.21.15.22:0/1469842817 wait complete. 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c6e6e700 1 Processor -- start 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c6e6e700 1 -- start start 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c6e6e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c6e6e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c010bda0 con 0x7f16c0105a60 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c4c0a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c4c0a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52918/0 (socket says 172.21.15.22:52918) 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.167+0000 7f16c4c0a700 1 -- 172.21.15.22:0/2107240732 learned_addr learned my addr 172.21.15.22:0/2107240732 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16c4c0a700 1 -- 172.21.15.22:0/2107240732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c019deb0 con 0x7f16c0105a60 2024-03-08T23:43:54.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16c4c0a700 1 --2- 172.21.15.22:0/2107240732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c019d970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f16b4011670 tx=0x7f16b4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16bdffb700 1 -- 172.21.15.22:0/2107240732 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b40039c0 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16bdffb700 1 -- 172.21.15.22:0/2107240732 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b4011e50 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c019e140 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16bdffb700 1 -- 172.21.15.22:0/2107240732 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b401a5c0 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c01a0e30 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16bdffb700 1 -- 172.21.15.22:0/2107240732 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f16b401ac00 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.168+0000 7f16bdffb700 1 -- 172.21.15.22:0/2107240732 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f16b4019700 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.169+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0063410 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.171+0000 7f16bdffb700 1 -- 172.21.15.22:0/2107240732 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f16b4003b60 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.241+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f16c01a1000 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.242+0000 7f16bdffb700 1 -- 172.21.15.22:0/2107240732 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f16b4019900 con 0x7f16c0105a60 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.243+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 msgr2=0x7f16c019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.243+0000 7f16c6e6e700 1 --2- 172.21.15.22:0/2107240732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c019d970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f16b4011670 tx=0x7f16b4011b60 comp rx=0 tx=0).stop 2024-03-08T23:43:54.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.243+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 shutdown_connections 2024-03-08T23:43:54.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.243+0000 7f16c6e6e700 1 --2- 172.21.15.22:0/2107240732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c0105a60 0x7f16c019d970 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:54.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.243+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 >> 172.21.15.22:0/2107240732 conn(0x7f16c0101390 msgr2=0x7f16c0101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:54.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.243+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 shutdown_connections 2024-03-08T23:43:54.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:54.243+0000 7f16c6e6e700 1 -- 172.21.15.22:0/2107240732 wait complete. 2024-03-08T23:43:54.782 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-08T23:43:54.804 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-08T23:43:54.804 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2024-03-08T23:43:54.804 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-08T23:43:54.805 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-08T23:43:55.031 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.x 2024-03-08T23:43:55.032 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.x.service: Unit ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.x.service not loaded. 2024-03-08T23:43:55.190 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715.target.wants/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@.service. 2024-03-08T23:43:55.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:55 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2107240732' entity='client.admin' 2024-03-08T23:43:55.488 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:55 smithi022 podman[32108]: 2024-03-08 23:43:55.328757872 +0000 UTC m=+0.017464413 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:43:55.763 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T23:43:55.763 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-08T23:43:55.771 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T23:43:55.771 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-08T23:43:55.771 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-08T23:43:55.771 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-03-08T23:43:55.780 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T23:43:55.780 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-08T23:43:55.787 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T23:43:55.788 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-08T23:43:55.788 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-08T23:43:55.788 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-08T23:43:55.788 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2024-03-08T23:43:55.788 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2024-03-08T23:43:55.912 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:55 smithi022 podman[32108]: 2024-03-08 23:43:55.652719462 +0000 UTC m=+0.341425990 container init a551c7f49a0d85e91168d17d13c8a58552d118edc947d693e1188bbf37d03628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-08T23:43:55.912 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:55 smithi022 podman[32108]: 2024-03-08 23:43:55.655041226 +0000 UTC m=+0.343747756 container start a551c7f49a0d85e91168d17d13c8a58552d118edc947d693e1188bbf37d03628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T23:43:55.912 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:55 smithi022 bash[32108]: a551c7f49a0d85e91168d17d13c8a58552d118edc947d693e1188bbf37d03628 2024-03-08T23:43:55.912 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:55 smithi022 systemd[1]: Started Ceph mgr.x for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:43:56.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:43:56 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/868697074' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T23:43:56.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:56 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:56.228+0000 7fe0effeb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:43:56.488 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:56 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:56.326+0000 7fe0effeb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:43:56.878 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:43:56.878 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:43:56.878 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:43:56.878 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T23:43:56.878 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T23:43:56.878 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T23:43:56.878 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T23:43:56.879 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:43:56.880 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T23:43:56.880 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:43:56.880 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T23:43:56.880 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T23:43:56.880 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T23:43:56.888 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T23:43:56.888 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T23:43:56.888 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T23:43:56.888 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:43:56.888 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:43:56.889 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:43:56.890 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T23:43:41.144667+0000", 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.240+0000 7fa19e904700 1 Processor -- start 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.240+0000 7fa19e904700 1 -- start start 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.240+0000 7fa19e904700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa198107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.240+0000 7fa19e904700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198108180 con 0x7fa198107790 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.240+0000 7fa197fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa198107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.240+0000 7fa197fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa198107bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52952/0 (socket says 172.21.15.22:52952) 2024-03-08T23:43:56.891 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.240+0000 7fa197fff700 1 -- 172.21.15.22:0/3039372608 learned_addr learned my addr 172.21.15.22:0/3039372608 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.241+0000 7fa197fff700 1 -- 172.21.15.22:0/3039372608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198108940 con 0x7fa198107790 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.241+0000 7fa197fff700 1 --2- 172.21.15.22:0/3039372608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa198107bb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa1880098a0 tx=0x7fa188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a479a9464ccc7622 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.241+0000 7fa196ffd700 1 -- 172.21.15.22:0/3039372608 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188004160 con 0x7fa198107790 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.241+0000 7fa196ffd700 1 -- 172.21.15.22:0/3039372608 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa188004300 con 0x7fa198107790 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.241+0000 7fa196ffd700 1 -- 172.21.15.22:0/3039372608 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188011b50 con 0x7fa198107790 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 -- 172.21.15.22:0/3039372608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 msgr2=0x7fa198107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 --2- 172.21.15.22:0/3039372608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa198107bb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa1880098a0 tx=0x7fa188009bb0 comp rx=0 tx=0).stop 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 -- 172.21.15.22:0/3039372608 shutdown_connections 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 --2- 172.21.15.22:0/3039372608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa198107bb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 -- 172.21.15.22:0/3039372608 >> 172.21.15.22:0/3039372608 conn(0x7fa198102d70 msgr2=0x7fa198105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 -- 172.21.15.22:0/3039372608 shutdown_connections 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 -- 172.21.15.22:0/3039372608 wait complete. 2024-03-08T23:43:56.892 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.242+0000 7fa19e904700 1 Processor -- start 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa19e904700 1 -- start start 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa19e904700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa19807cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa19e904700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19810bcf0 con 0x7fa198107790 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa197fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa19807cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa197fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa19807cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52964/0 (socket says 172.21.15.22:52964) 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa197fff700 1 -- 172.21.15.22:0/868697074 learned_addr learned my addr 172.21.15.22:0/868697074 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa197fff700 1 -- 172.21.15.22:0/868697074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19807d0d0 con 0x7fa198107790 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.243+0000 7fa197fff700 1 --2- 172.21.15.22:0/868697074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa19807cb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa188011850 tx=0x7fa188011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.244+0000 7fa1957fa700 1 -- 172.21.15.22:0/868697074 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa188003af0 con 0x7fa198107790 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.244+0000 7fa1957fa700 1 -- 172.21.15.22:0/868697074 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa188020070 con 0x7fa198107790 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.244+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19807b2c0 con 0x7fa198107790 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.244+0000 7fa1957fa700 1 -- 172.21.15.22:0/868697074 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1880195a0 con 0x7fa198107790 2024-03-08T23:43:56.893 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.244+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19807b800 con 0x7fa198107790 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.244+0000 7fa1957fa700 1 -- 172.21.15.22:0/868697074 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa188019c30 con 0x7fa198107790 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.244+0000 7fa1957fa700 1 -- 172.21.15.22:0/868697074 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa188011e00 con 0x7fa198107790 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.245+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198068eb0 con 0x7fa198107790 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.247+0000 7fa1957fa700 1 -- 172.21.15.22:0/868697074 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fa188003c90 con 0x7fa198107790 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.321+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa19807bd30 con 0x7fa198107790 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.322+0000 7fa1957fa700 1 -- 172.21.15.22:0/868697074 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fa18801a430 con 0x7fa198107790 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.322+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 msgr2=0x7fa19807cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.322+0000 7fa19e904700 1 --2- 172.21.15.22:0/868697074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa19807cb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa188011850 tx=0x7fa188011c10 comp rx=0 tx=0).stop 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.323+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 shutdown_connections 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.323+0000 7fa19e904700 1 --2- 172.21.15.22:0/868697074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa198107790 0x7fa19807cb90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.323+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 >> 172.21.15.22:0/868697074 conn(0x7fa198102d70 msgr2=0x7fa198103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.323+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 shutdown_connections 2024-03-08T23:43:56.894 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:56.323+0000 7fa19e904700 1 -- 172.21.15.22:0/868697074 wait complete. 2024-03-08T23:43:56.895 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2024-03-08T23:43:57.237 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.001+0000 7fe0effeb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:43:57.237 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.095+0000 7fe0effeb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:43:57.689 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.239+0000 7fe0effeb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:43:57.690 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.337+0000 7fe0effeb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:43:57.690 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.422+0000 7fe0effeb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:43:57.690 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.691+0000 7fe0effeb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:43:57.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.803+0000 7fe0effeb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:43:57.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.891+0000 7fe0effeb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:43:57.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:57.988+0000 7fe0effeb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:43:58.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:58 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:58.167+0000 7fe0effeb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:43:59.307 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:59 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:59.173+0000 7fe0effeb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:43:59.307 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:59 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:59.308+0000 7fe0effeb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:43:59.941 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:59 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:59.648+0000 7fe0effeb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:43:59.942 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:59 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:59.742+0000 7fe0effeb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:44:00.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:00 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2368128320' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T23:44:00.237 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:43:59 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:43:59.946+0000 7fe0effeb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:44:00.237 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:00.143+0000 7fe0effeb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:44:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:44:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:44:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:44:00.588 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T23:44:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T23:44:00.590 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T23:44:00.599 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T23:44:00.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:00.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T23:43:41.144667+0000", 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.984+0000 7f293bb44700 1 Processor -- start 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.984+0000 7f293bb44700 1 -- start start 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.984+0000 7f293bb44700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934106a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.984+0000 7f293bb44700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2934107040 con 0x7f2934106650 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.984+0000 7f29398e0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934106a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:00.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.984+0000 7f29398e0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934106a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52980/0 (socket says 172.21.15.22:52980) 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.984+0000 7f29398e0700 1 -- 172.21.15.22:0/3932966640 learned_addr learned my addr 172.21.15.22:0/3932966640 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.985+0000 7f29398e0700 1 -- 172.21.15.22:0/3932966640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934107850 con 0x7f2934106650 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.985+0000 7f29398e0700 1 --2- 172.21.15.22:0/3932966640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934106a70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f293000bb40 tx=0x7f293000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78e6f8bf243abf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.985+0000 7f29388de700 1 -- 172.21.15.22:0/3932966640 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f293000dbe0 con 0x7f2934106650 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.985+0000 7f29388de700 1 -- 172.21.15.22:0/3932966640 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29300044f0 con 0x7f2934106650 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.985+0000 7f29388de700 1 -- 172.21.15.22:0/3932966640 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2930010470 con 0x7f2934106650 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 -- 172.21.15.22:0/3932966640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 msgr2=0x7f2934106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 --2- 172.21.15.22:0/3932966640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934106a70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f293000bb40 tx=0x7f293000be50 comp rx=0 tx=0).stop 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 -- 172.21.15.22:0/3932966640 shutdown_connections 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 --2- 172.21.15.22:0/3932966640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934106a70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 -- 172.21.15.22:0/3932966640 >> 172.21.15.22:0/3932966640 conn(0x7f2934101c30 msgr2=0x7f2934104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 -- 172.21.15.22:0/3932966640 shutdown_connections 2024-03-08T23:44:00.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 -- 172.21.15.22:0/3932966640 wait complete. 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 Processor -- start 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 -- start start 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.986+0000 7f293bb44700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2934079ff0 con 0x7f2934106650 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f29398e0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f29398e0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52982/0 (socket says 172.21.15.22:52982) 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f29398e0700 1 -- 172.21.15.22:0/2368128320 learned_addr learned my addr 172.21.15.22:0/2368128320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f29398e0700 1 -- 172.21.15.22:0/2368128320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934199a40 con 0x7f2934106650 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f29398e0700 1 --2- 172.21.15.22:0/2368128320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934199500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f293000b340 tx=0x7f29300040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f292affd700 1 -- 172.21.15.22:0/2368128320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2930027070 con 0x7f2934106650 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f292affd700 1 -- 172.21.15.22:0/2368128320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2930010be0 con 0x7f2934106650 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2934199cd0 con 0x7f2934106650 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293419c9c0 con 0x7f2934106650 2024-03-08T23:44:00.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f292affd700 1 -- 172.21.15.22:0/2368128320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2930022c40 con 0x7f2934106650 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f292affd700 1 -- 172.21.15.22:0/2368128320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f293002a070 con 0x7f2934106650 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.987+0000 7f292affd700 1 -- 172.21.15.22:0/2368128320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f293001f070 con 0x7f2934106650 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.988+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2918005360 con 0x7f2934106650 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:43:59.990+0000 7f292affd700 1 -- 172.21.15.22:0/2368128320 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f2930031030 con 0x7f2934106650 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.064+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2918005920 con 0x7f2934106650 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.065+0000 7f292affd700 1 -- 172.21.15.22:0/2368128320 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f2930015070 con 0x7f2934106650 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.065+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 msgr2=0x7f2934199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.065+0000 7f293bb44700 1 --2- 172.21.15.22:0/2368128320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934199500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f293000b340 tx=0x7f29300040b0 comp rx=0 tx=0).stop 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.065+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 shutdown_connections 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.065+0000 7f293bb44700 1 --2- 172.21.15.22:0/2368128320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2934106650 0x7f2934199500 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.065+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 >> 172.21.15.22:0/2368128320 conn(0x7f2934101c30 msgr2=0x7f293418fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.066+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 shutdown_connections 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:00.066+0000 7f293bb44700 1 -- 172.21.15.22:0/2368128320 wait complete. 2024-03-08T23:44:00.605 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2024-03-08T23:44:00.699 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:00.438+0000 7fe0effeb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:44:00.700 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:00.701+0000 7fe0effeb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:44:00.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:00.943+0000 7fe0effeb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:44:02.237 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:01 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:01.778+0000 7fe0effeb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:44:02.685 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:02.361+0000 7fe0effeb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:44:02.685 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:02.462+0000 7fe0effeb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:44:02.685 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:02.686+0000 7fe0effeb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:44:02.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: Activating manager daemon x 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: mgrmap e2: x(active, starting, since 0.00285618s) 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: Manager daemon x is now available 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' 2024-03-08T23:44:02.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' 2024-03-08T23:44:02.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:02 smithi022 ceph-mon[31894]: from='mgr.14100 172.21.15.22:0/1180885193' entity='mgr.x' 2024-03-08T23:44:03.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:03 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1726468337' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T23:44:03.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:03 smithi022 ceph-mon[31894]: mgrmap e3: x(active, since 1.00669s) 2024-03-08T23:44:04.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-08T23:44:04.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T23:44:04.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T23:44:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T23:44:04.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T23:43:41.144667+0000", 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.088+0000 7ffb8791a700 1 Processor -- start 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.088+0000 7ffb8791a700 1 -- start start 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.088+0000 7ffb8791a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8007b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.088+0000 7ffb8791a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8007b840 con 0x7ffb8007ce50 2024-03-08T23:44:04.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.088+0000 7ffb856b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8007b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.088+0000 7ffb856b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8007b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36038/0 (socket says 172.21.15.22:36038) 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.088+0000 7ffb856b6700 1 -- 172.21.15.22:0/2518322521 learned_addr learned my addr 172.21.15.22:0/2518322521 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.089+0000 7ffb856b6700 1 -- 172.21.15.22:0/2518322521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8007b9c0 con 0x7ffb8007ce50 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.089+0000 7ffb856b6700 1 --2- 172.21.15.22:0/2518322521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8007b300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecec471364b3e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.089+0000 7ffb77fff700 1 -- 172.21.15.22:0/2518322521 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb7c004160 con 0x7ffb8007ce50 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.089+0000 7ffb77fff700 1 -- 172.21.15.22:0/2518322521 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb7c004300 con 0x7ffb8007ce50 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.089+0000 7ffb77fff700 1 -- 172.21.15.22:0/2518322521 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb7c011b50 con 0x7ffb8007ce50 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.089+0000 7ffb8791a700 1 -- 172.21.15.22:0/2518322521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 msgr2=0x7ffb8007b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.089+0000 7ffb8791a700 1 --2- 172.21.15.22:0/2518322521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8007b300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 -- 172.21.15.22:0/2518322521 shutdown_connections 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 --2- 172.21.15.22:0/2518322521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8007b300 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 -- 172.21.15.22:0/2518322521 >> 172.21.15.22:0/2518322521 conn(0x7ffb80102d70 msgr2=0x7ffb80105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 -- 172.21.15.22:0/2518322521 shutdown_connections 2024-03-08T23:44:04.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 -- 172.21.15.22:0/2518322521 wait complete. 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 Processor -- start 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 -- start start 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.090+0000 7ffb8791a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8010dd60 con 0x7ffb8007ce50 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb856b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb856b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36046/0 (socket says 172.21.15.22:36046) 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb856b6700 1 -- 172.21.15.22:0/1726468337 learned_addr learned my addr 172.21.15.22:0/1726468337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb856b6700 1 -- 172.21.15.22:0/1726468337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8019dea0 con 0x7ffb8007ce50 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb856b6700 1 --2- 172.21.15.22:0/1726468337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8019d960 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c011b60 tx=0x7ffb7c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb767fc700 1 -- 172.21.15.22:0/1726468337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb7c003970 con 0x7ffb8007ce50 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb767fc700 1 -- 172.21.15.22:0/1726468337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb7c003b10 con 0x7ffb8007ce50 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8019e130 con 0x7ffb8007ce50 2024-03-08T23:44:04.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb767fc700 1 -- 172.21.15.22:0/1726468337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb7c024610 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.091+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb801a0e20 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.092+0000 7ffb767fc700 1 -- 172.21.15.22:0/1726468337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 2) v1 ==== 45862+0+0 (secure 0 0 0) 0x7ffb7c0248c0 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.092+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb80068eb0 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.092+0000 7ffb767fc700 1 -- 172.21.15.22:0/1726468337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb7c011e00 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.094+0000 7ffb767fc700 1 -- 172.21.15.22:0/1726468337 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ffb7c024c10 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.167+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffb801a0ff0 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.167+0000 7ffb767fc700 1 -- 172.21.15.22:0/1726468337 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ffb801a0ff0 con 0x7ffb8007ce50 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.168+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 msgr2=0x7ffb8019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.168+0000 7ffb8791a700 1 --2- 172.21.15.22:0/1726468337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8019d960 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c011b60 tx=0x7ffb7c011c40 comp rx=0 tx=0).stop 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.168+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 shutdown_connections 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.168+0000 7ffb8791a700 1 --2- 172.21.15.22:0/1726468337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb8007ce50 0x7ffb8019d960 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.168+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 >> 172.21.15.22:0/1726468337 conn(0x7ffb80102d70 msgr2=0x7ffb80103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.168+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 shutdown_connections 2024-03-08T23:44:04.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:03.168+0000 7ffb8791a700 1 -- 172.21.15.22:0/1726468337 wait complete. 2024-03-08T23:44:04.156 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (3/15)... 2024-03-08T23:44:06.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:05 smithi022 ceph-mon[31894]: mgrmap e4: x(active, since 2s) 2024-03-08T23:44:07.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:06 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2656347774' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T23:44:07.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:44:07.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:44:07.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:44:07.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T23:44:07.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:07.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T23:44:07.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T23:44:07.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T23:43:41.144667+0000", 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T23:44:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.599+0000 7feb0c6a7700 1 Processor -- start 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.599+0000 7feb0c6a7700 1 -- start start 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.599+0000 7feb0c6a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb04107c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.599+0000 7feb0c6a7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04108230 con 0x7feb04107840 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.599+0000 7feb0a443700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb04107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.599+0000 7feb0a443700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb04107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36062/0 (socket says 172.21.15.22:36062) 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.599+0000 7feb0a443700 1 -- 172.21.15.22:0/1284030929 learned_addr learned my addr 172.21.15.22:0/1284030929 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.600+0000 7feb0a443700 1 -- 172.21.15.22:0/1284030929 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb041089f0 con 0x7feb04107840 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.600+0000 7feb0a443700 1 --2- 172.21.15.22:0/1284030929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb04107c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feaf40098a0 tx=0x7feaf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206e446fcce57b4b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.600+0000 7feb09441700 1 -- 172.21.15.22:0/1284030929 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4004160 con 0x7feb04107840 2024-03-08T23:44:07.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.600+0000 7feb09441700 1 -- 172.21.15.22:0/1284030929 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaf4004300 con 0x7feb04107840 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.600+0000 7feb09441700 1 -- 172.21.15.22:0/1284030929 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4011b50 con 0x7feb04107840 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 -- 172.21.15.22:0/1284030929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 msgr2=0x7feb04107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 --2- 172.21.15.22:0/1284030929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb04107c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feaf40098a0 tx=0x7feaf4009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 -- 172.21.15.22:0/1284030929 shutdown_connections 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 --2- 172.21.15.22:0/1284030929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb04107c60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 -- 172.21.15.22:0/1284030929 >> 172.21.15.22:0/1284030929 conn(0x7feb04102e00 msgr2=0x7feb04105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 -- 172.21.15.22:0/1284030929 shutdown_connections 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 -- 172.21.15.22:0/1284030929 wait complete. 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 Processor -- start 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.601+0000 7feb0c6a7700 1 -- start start 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0c6a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb0419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0c6a7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0410bda0 con 0x7feb04107840 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0a443700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb0419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:07.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0a443700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb0419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36068/0 (socket says 172.21.15.22:36068) 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0a443700 1 -- 172.21.15.22:0/2656347774 learned_addr learned my addr 172.21.15.22:0/2656347774 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0a443700 1 -- 172.21.15.22:0/2656347774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0419dec0 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0a443700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb0419d980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009fb0 tx=0x7feaf4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feafb7fe700 1 -- 172.21.15.22:0/2656347774 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf40038b0 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0419e150 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feafb7fe700 1 -- 172.21.15.22:0/2656347774 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaf4011e50 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feafb7fe700 1 -- 172.21.15.22:0/2656347774 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4019550 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.602+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb041a0e40 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.603+0000 7feafb7fe700 1 -- 172.21.15.22:0/2656347774 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7feaf4019850 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.603+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb04068eb0 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.603+0000 7feafb7fe700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7feaf0038e80 0x7feaf003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.603+0000 7feafb7fe700 1 -- 172.21.15.22:0/2656347774 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feaf404d210 con 0x7feb04107840 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.603+0000 7feb09c42700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7feaf0038e80 0x7feaf003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:07.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.604+0000 7feb09c42700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7feaf0038e80 0x7feaf003b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb00006fd0 tx=0x7feb00006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.609+0000 7feafb7fe700 1 -- 172.21.15.22:0/2656347774 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7feaf400f2e0 con 0x7feb04107840 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.894+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7feb041a1010 con 0x7feb04107840 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.894+0000 7feafb7fe700 1 -- 172.21.15.22:0/2656347774 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7feb041a1010 con 0x7feb04107840 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7feaf0038e80 msgr2=0x7feaf003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7feaf0038e80 0x7feaf003b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb00006fd0 tx=0x7feb00006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 msgr2=0x7feb0419d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb0419d980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009fb0 tx=0x7feaf4011bf0 comp rx=0 tx=0).stop 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 shutdown_connections 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7feaf0038e80 0x7feaf003b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 --2- 172.21.15.22:0/2656347774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb04107840 0x7feb0419d980 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:07.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 >> 172.21.15.22:0/2656347774 conn(0x7feb04102e00 msgr2=0x7feb041038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:07.487 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 shutdown_connections 2024-03-08T23:44:07.487 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:06.896+0000 7feb0c6a7700 1 -- 172.21.15.22:0/2656347774 wait complete. 2024-03-08T23:44:07.487 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2024-03-08T23:44:08.437 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2644296940' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-08T23:44:08.705 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.871+0000 7f5a0a1b4700 1 Processor -- start 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.871+0000 7f5a0a1b4700 1 -- start start 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.871+0000 7f5a0a1b4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:08.706 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.871+0000 7f5a0a1b4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0407aeb0 con 0x7f5a041081a0 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36072/0 (socket says 172.21.15.22:36072) 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a037fe700 1 -- 172.21.15.22:0/1238538494 learned_addr learned my addr 172.21.15.22:0/1238538494 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a037fe700 1 -- 172.21.15.22:0/1238538494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0410aad0 con 0x7f5a041081a0 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a037fe700 1 --2- 172.21.15.22:0/1238538494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0410a590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59f40098a0 tx=0x7f59f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48d94e4a421c489 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a027fc700 1 -- 172.21.15.22:0/1238538494 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59f4004160 con 0x7f5a041081a0 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a027fc700 1 -- 172.21.15.22:0/1238538494 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59f4004300 con 0x7f5a041081a0 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.872+0000 7f5a027fc700 1 -- 172.21.15.22:0/1238538494 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59f4011b50 con 0x7f5a041081a0 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.873+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/1238538494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 msgr2=0x7f5a0410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.873+0000 7f5a0a1b4700 1 --2- 172.21.15.22:0/1238538494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0410a590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59f40098a0 tx=0x7f59f4009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.873+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/1238538494 shutdown_connections 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.873+0000 7f5a0a1b4700 1 --2- 172.21.15.22:0/1238538494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0410a590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.873+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/1238538494 >> 172.21.15.22:0/1238538494 conn(0x7f5a04101c30 msgr2=0x7f5a04104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:08.707 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.873+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/1238538494 shutdown_connections 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.873+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/1238538494 wait complete. 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a0a1b4700 1 Processor -- start 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a0a1b4700 1 -- start start 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a0a1b4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a0a1b4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0410ba80 con 0x7f5a041081a0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36076/0 (socket says 172.21.15.22:36076) 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a037fe700 1 -- 172.21.15.22:0/2644296940 learned_addr learned my addr 172.21.15.22:0/2644296940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a037fe700 1 -- 172.21.15.22:0/2644296940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0419bba0 con 0x7f5a041081a0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.874+0000 7f5a037fe700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0419b660 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59f4011670 tx=0x7f59f4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.875+0000 7f5a00ff9700 1 -- 172.21.15.22:0/2644296940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59f40039c0 con 0x7f5a041081a0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.875+0000 7f5a00ff9700 1 -- 172.21.15.22:0/2644296940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59f4011e50 con 0x7f5a041081a0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.875+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0419be90 con 0x7f5a041081a0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.875+0000 7f5a00ff9700 1 -- 172.21.15.22:0/2644296940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59f401a5c0 con 0x7f5a041081a0 2024-03-08T23:44:08.708 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.875+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0419eb20 con 0x7f5a041081a0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.875+0000 7f5a00ff9700 1 -- 172.21.15.22:0/2644296940 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f59f401a8c0 con 0x7f5a041081a0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.876+0000 7f5a00ff9700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f59ec038e80 0x7f59ec03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.876+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a04068eb0 con 0x7f5a041081a0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.876+0000 7f5a00ff9700 1 -- 172.21.15.22:0/2644296940 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f59f404d0b0 con 0x7f5a041081a0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.876+0000 7f5a02ffd700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f59ec038e80 0x7f59ec03b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.876+0000 7f5a02ffd700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f59ec038e80 0x7f59ec03b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006fd0 tx=0x7f59f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:07.881+0000 7f5a00ff9700 1 -- 172.21.15.22:0/2644296940 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f59f4019c70 con 0x7f5a041081a0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.087+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5a0419ecf0 con 0x7f5a041081a0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.088+0000 7f5a00ff9700 1 -- 172.21.15.22:0/2644296940 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f59f4003b60 con 0x7f5a041081a0 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f59ec038e80 msgr2=0x7f59ec03b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f59ec038e80 0x7f59ec03b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006fd0 tx=0x7f59f8006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 msgr2=0x7f5a0419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0419b660 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59f4011670 tx=0x7f59f4011b60 comp rx=0 tx=0).stop 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 shutdown_connections 2024-03-08T23:44:08.709 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f59ec038e80 0x7f59ec03b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:08.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 --2- 172.21.15.22:0/2644296940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a041081a0 0x7f5a0419b660 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:08.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 >> 172.21.15.22:0/2644296940 conn(0x7f5a04101c30 msgr2=0x7f5a041027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:08.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 shutdown_connections 2024-03-08T23:44:08.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:08.090+0000 7f5a0a1b4700 1 -- 172.21.15.22:0/2644296940 wait complete. 2024-03-08T23:44:08.710 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2024-03-08T23:44:09.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:09 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2846822807' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-08T23:44:09.737 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:09 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: ignoring --setuser ceph since I am not root 2024-03-08T23:44:09.737 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:09 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: ignoring --setgroup ceph since I am not root 2024-03-08T23:44:10.056 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:10.019+0000 7fe5ead50200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:44:10.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99fd41700 1 Processor -- start 2024-03-08T23:44:10.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99fd41700 1 -- start start 2024-03-08T23:44:10.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99fd41700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd998107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:10.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99fd41700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998108190 con 0x7fd9981077a0 2024-03-08T23:44:10.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99dadd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd998107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:10.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99dadd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd998107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36090/0 (socket says 172.21.15.22:36090) 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99dadd700 1 -- 172.21.15.22:0/1166044181 learned_addr learned my addr 172.21.15.22:0/1166044181 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.131+0000 7fd99dadd700 1 -- 172.21.15.22:0/1166044181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd998108950 con 0x7fd9981077a0 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99dadd700 1 --2- 172.21.15.22:0/1166044181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd998107bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098a0 tx=0x7fd988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142f5a3990dbd77a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99cadb700 1 -- 172.21.15.22:0/1166044181 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd988004160 con 0x7fd9981077a0 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99cadb700 1 -- 172.21.15.22:0/1166044181 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd988004300 con 0x7fd9981077a0 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99cadb700 1 -- 172.21.15.22:0/1166044181 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd988011b50 con 0x7fd9981077a0 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99fd41700 1 -- 172.21.15.22:0/1166044181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 msgr2=0x7fd998107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99fd41700 1 --2- 172.21.15.22:0/1166044181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd998107bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098a0 tx=0x7fd988009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99fd41700 1 -- 172.21.15.22:0/1166044181 shutdown_connections 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99fd41700 1 --2- 172.21.15.22:0/1166044181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd998107bc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.132+0000 7fd99fd41700 1 -- 172.21.15.22:0/1166044181 >> 172.21.15.22:0/1166044181 conn(0x7fd998102da0 msgr2=0x7fd9981051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99fd41700 1 -- 172.21.15.22:0/1166044181 shutdown_connections 2024-03-08T23:44:10.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99fd41700 1 -- 172.21.15.22:0/1166044181 wait complete. 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99fd41700 1 Processor -- start 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99fd41700 1 -- start start 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99fd41700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd99807cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99fd41700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99810bd00 con 0x7fd9981077a0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99dadd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd99807cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99dadd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd99807cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36100/0 (socket says 172.21.15.22:36100) 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.133+0000 7fd99dadd700 1 -- 172.21.15.22:0/2846822807 learned_addr learned my addr 172.21.15.22:0/2846822807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.134+0000 7fd99dadd700 1 -- 172.21.15.22:0/2846822807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99807d0d0 con 0x7fd9981077a0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.134+0000 7fd99dadd700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd99807cb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd988000c00 tx=0x7fd988011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.134+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd988003b20 con 0x7fd9981077a0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.134+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd988003cc0 con 0x7fd9981077a0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.134+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99807b2c0 con 0x7fd9981077a0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.134+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd988024910 con 0x7fd9981077a0 2024-03-08T23:44:10.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.134+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99807b800 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.135+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7fd988022020 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.135+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd998063410 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.135+0000 7fd98effd700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7fd984038f20 0x7fd98403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.135+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd988011e00 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.135+0000 7fd99d2dc700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7fd984038f20 0x7fd98403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.136+0000 7fd99d2dc700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7fd984038f20 0x7fd98403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd994006fd0 tx=0x7fd994006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.140+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd988016260 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.398+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd99807ba80 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.449+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd98804c8a0 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.451+0000 7fd98effd700 1 -- 172.21.15.22:0/2846822807 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd98801a430 con 0x7fd9981077a0 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7fd984038f20 msgr2=0x7fd98403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7fd984038f20 0x7fd98403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd994006fd0 tx=0x7fd994006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 msgr2=0x7fd99807cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:10.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd99807cb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd988000c00 tx=0x7fd988011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:10.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 shutdown_connections 2024-03-08T23:44:10.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7fd984038f20 0x7fd98403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:10.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 --2- 172.21.15.22:0/2846822807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9981077a0 0x7fd99807cb90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:10.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 >> 172.21.15.22:0/2846822807 conn(0x7fd998102da0 msgr2=0x7fd9981038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:10.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 shutdown_connections 2024-03-08T23:44:10.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:09.452+0000 7fd99fd41700 1 -- 172.21.15.22:0/2846822807 wait complete. 2024-03-08T23:44:10.309 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:10.117+0000 7fe5ead50200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:44:10.585 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:10 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2846822807' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-08T23:44:10.586 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:10 smithi022 ceph-mon[31894]: mgrmap e5: x(active, since 6s) 2024-03-08T23:44:10.847 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:10.800+0000 7fe5ead50200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:44:11.133 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:10.907+0000 7fe5ead50200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:44:11.133 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:11.065+0000 7fe5ead50200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:44:11.400 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:11.164+0000 7fe5ead50200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:44:11.400 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:11.249+0000 7fe5ead50200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:44:11.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:44:11.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-08T23:44:11.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-08T23:44:11.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f544d655700 1 Processor -- start 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f544d655700 1 -- start start 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f544d655700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f5448106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f544d655700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544807ae50 con 0x7f5448105ff0 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f5446ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f5448106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f5446ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f5448106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43666/0 (socket says 172.21.15.22:43666) 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f5446ffd700 1 -- 172.21.15.22:0/4261005834 learned_addr learned my addr 172.21.15.22:0/4261005834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.525+0000 7f5446ffd700 1 -- 172.21.15.22:0/4261005834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448106950 con 0x7f5448105ff0 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.526+0000 7f5446ffd700 1 --2- 172.21.15.22:0/4261005834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f5448106410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54380098a0 tx=0x7f5438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5eb07c33b23a62 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.526+0000 7f5445ffb700 1 -- 172.21.15.22:0/4261005834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5438004160 con 0x7f5448105ff0 2024-03-08T23:44:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.526+0000 7f5445ffb700 1 -- 172.21.15.22:0/4261005834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5438004300 con 0x7f5448105ff0 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.526+0000 7f5445ffb700 1 -- 172.21.15.22:0/4261005834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5438011b50 con 0x7f5448105ff0 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 -- 172.21.15.22:0/4261005834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 msgr2=0x7f5448106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 --2- 172.21.15.22:0/4261005834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f5448106410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54380098a0 tx=0x7f5438009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 -- 172.21.15.22:0/4261005834 shutdown_connections 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 --2- 172.21.15.22:0/4261005834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f5448106410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 -- 172.21.15.22:0/4261005834 >> 172.21.15.22:0/4261005834 conn(0x7f5448101c30 msgr2=0x7f5448104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 -- 172.21.15.22:0/4261005834 shutdown_connections 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 -- 172.21.15.22:0/4261005834 wait complete. 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.527+0000 7f544d655700 1 Processor -- start 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f544d655700 1 -- start start 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f544d655700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f544819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f544d655700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54481079e0 con 0x7f5448105ff0 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f5446ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f544819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f5446ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f544819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43668/0 (socket says 172.21.15.22:43668) 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f5446ffd700 1 -- 172.21.15.22:0/2114844298 learned_addr learned my addr 172.21.15.22:0/2114844298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f5446ffd700 1 -- 172.21.15.22:0/2114844298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544819de20 con 0x7f5448105ff0 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.528+0000 7f5446ffd700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f544819d8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5438011b60 tx=0x7f5438011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.529+0000 7f542ffff700 1 -- 172.21.15.22:0/2114844298 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5438003b20 con 0x7f5448105ff0 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.529+0000 7f542ffff700 1 -- 172.21.15.22:0/2114844298 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5438003cc0 con 0x7f5448105ff0 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.529+0000 7f542ffff700 1 -- 172.21.15.22:0/2114844298 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5438024840 con 0x7f5448105ff0 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.529+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544819e0b0 con 0x7f5448105ff0 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.529+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54481a0da0 con 0x7f5448105ff0 2024-03-08T23:44:11.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.530+0000 7f542ffff700 1 -- 172.21.15.22:0/2114844298 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f5438022020 con 0x7f5448105ff0 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.530+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5448068eb0 con 0x7f5448105ff0 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.530+0000 7f542ffff700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 0x7f543003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.530+0000 7f542ffff700 1 -- 172.21.15.22:0/2114844298 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5438011e00 con 0x7f5448105ff0 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.530+0000 7f54467fc700 1 -- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 msgr2=0x7f543003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3384641534 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.530+0000 7f54467fc700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 0x7f543003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.535+0000 7f542ffff700 1 -- 172.21.15.22:0/2114844298 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f543800ba30 con 0x7f5448105ff0 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.731+0000 7f54467fc700 1 -- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 msgr2=0x7f543003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3384641534 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.731+0000 7f54467fc700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 0x7f543003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.799+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f54480008d0 con 0x7f5448105ff0 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.800+0000 7f542ffff700 1 -- 172.21.15.22:0/2114844298 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f543800bc90 con 0x7f5448105ff0 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 msgr2=0x7f543003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 0x7f543003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:11.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 msgr2=0x7f544819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f544819d8e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5438011b60 tx=0x7f5438011c40 comp rx=0 tx=0).stop 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 shutdown_connections 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f5430038f70 0x7f543003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 --2- 172.21.15.22:0/2114844298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5448105ff0 0x7f544819d8e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 >> 172.21.15.22:0/2114844298 conn(0x7f5448101c30 msgr2=0x7f5448194550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 shutdown_connections 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:10.802+0000 7f544d655700 1 -- 172.21.15.22:0/2114844298 wait complete. 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-03-08T23:44:11.414 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2024-03-08T23:44:11.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2114844298' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-08T23:44:11.660 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:11.523+0000 7fe5ead50200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:44:11.660 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:11.636+0000 7fe5ead50200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:44:11.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:11.728+0000 7fe5ead50200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:44:11.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:11.826+0000 7fe5ead50200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:44:12.479 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:12 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:12.007+0000 7fe5ead50200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:44:13.280 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:12 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:12.989+0000 7fe5ead50200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:44:13.280 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:13.123+0000 7fe5ead50200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:44:13.557 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:13.465+0000 7fe5ead50200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:44:13.557 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:13.558+0000 7fe5ead50200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:44:13.943 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:13.756+0000 7fe5ead50200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:44:13.944 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:13.945+0000 7fe5ead50200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:44:14.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:14.236+0000 7fe5ead50200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:44:14.881 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:14.498+0000 7fe5ead50200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:44:14.881 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:14.740+0000 7fe5ead50200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:44:15.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:15 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:15.565+0000 7fe5ead50200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:44:16.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:16.178+0000 7fe5ead50200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:44:16.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:16.279+0000 7fe5ead50200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:44:16.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:16.502+0000 7fe5ead50200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:44:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: Active manager daemon x restarted 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: Activating manager daemon x 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: osdmap e2: 0 total, 0 up, 0 in 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: mgrmap e6: x(active, starting, since 0.00344393s) 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: Manager daemon x is now available 2024-03-08T23:44:16.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:16.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:16.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:16.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:16 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:17.830 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:17 smithi022 ceph-mon[31894]: Found migration_current of "None". Setting to last migration. 2024-03-08T23:44:17.831 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:17 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:17.831 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:17 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T23:44:17.831 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:17 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T23:44:17.831 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:17 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:17.831 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:17 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:17.831 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:17 smithi022 ceph-mon[31894]: mgrmap e7: x(active, since 1.0056s) 2024-03-08T23:44:18.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f98a2511700 1 Processor -- start 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f98a2511700 1 -- start start 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f98a2511700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f98a2511700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c107050 con 0x7f989c106660 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f989bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f989bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43678/0 (socket says 172.21.15.22:43678) 2024-03-08T23:44:18.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f989bfff700 1 -- 172.21.15.22:0/1459546135 learned_addr learned my addr 172.21.15.22:0/1459546135 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.874+0000 7f989bfff700 1 -- 172.21.15.22:0/1459546135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c107860 con 0x7f989c106660 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.875+0000 7f989bfff700 1 --2- 172.21.15.22:0/1459546135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c106a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098a0 tx=0x7f988c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5788cabc5140a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.875+0000 7f989affd700 1 -- 172.21.15.22:0/1459546135 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f988c004160 con 0x7f989c106660 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.875+0000 7f989affd700 1 -- 172.21.15.22:0/1459546135 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f988c004300 con 0x7f989c106660 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.875+0000 7f989affd700 1 -- 172.21.15.22:0/1459546135 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f988c011b50 con 0x7f989c106660 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.875+0000 7f98a2511700 1 -- 172.21.15.22:0/1459546135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 msgr2=0x7f989c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.875+0000 7f98a2511700 1 --2- 172.21.15.22:0/1459546135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c106a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098a0 tx=0x7f988c009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 -- 172.21.15.22:0/1459546135 shutdown_connections 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 --2- 172.21.15.22:0/1459546135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c106a80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 -- 172.21.15.22:0/1459546135 >> 172.21.15.22:0/1459546135 conn(0x7f989c101c80 msgr2=0x7f989c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 -- 172.21.15.22:0/1459546135 shutdown_connections 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 -- 172.21.15.22:0/1459546135 wait complete. 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 Processor -- start 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 -- start start 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c191810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.876+0000 7f98a2511700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c10ac10 con 0x7f989c106660 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f989bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c191810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f989bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c191810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43690/0 (socket says 172.21.15.22:43690) 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f989bfff700 1 -- 172.21.15.22:0/2183380888 learned_addr learned my addr 172.21.15.22:0/2183380888 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f989bfff700 1 -- 172.21.15.22:0/2183380888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c191d50 con 0x7f989c106660 2024-03-08T23:44:18.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f989bfff700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c191810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f988c009fb0 tx=0x7f988c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f988c0038b0 con 0x7f989c106660 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f988c011e50 con 0x7f989c106660 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f988c019550 con 0x7f989c106660 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c18ff40 con 0x7f989c106660 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.877+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c190480 con 0x7f989c106660 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.878+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f988c003a50 con 0x7f989c106660 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.878+0000 7f98997fa700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.878+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 --> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f988403baa0 con 0x7f9884038ed0 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.879+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f988c04d010 con 0x7f989c106660 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.879+0000 7f989b7fe700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 msgr2=0x7f988403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3384641534 2024-03-08T23:44:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:11.879+0000 7f989b7fe700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:12.080+0000 7f989b7fe700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 msgr2=0x7f988403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3384641534 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:12.080+0000 7f989b7fe700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:12.481+0000 7f989b7fe700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 msgr2=0x7f988403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3384641534 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:12.481+0000 7f989b7fe700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:13.282+0000 7f989b7fe700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 msgr2=0x7f988403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3384641534 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:13.282+0000 7f989b7fe700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:14.883+0000 7f989b7fe700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 msgr2=0x7f988403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3384641534 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:14.883+0000 7f989b7fe700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:16.517+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7f988c019850 con 0x7f989c106660 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:16.517+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 msgr2=0x7f988403b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:16.517+0000 7f98997fa700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:18.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.518+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7f988c04f0f0 con 0x7f989c106660 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.518+0000 7f98997fa700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f988403c8e0 0x7f988403ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.518+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f988c001d30 con 0x7f988403c8e0 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.518+0000 7f989b7fe700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f988403c8e0 0x7f988403ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.519+0000 7f989b7fe700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f988403c8e0 0x7f988403ecd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9890003a10 tx=0x7f9890009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.520+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f988c001d30 con 0x7f988403c8e0 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.526+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f989c068eb0 con 0x7f988403c8e0 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.526+0000 7f98997fa700 1 -- 172.21.15.22:0/2183380888 <== mgr.14120 v2:172.21.15.22:6800/4028157579 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f989c068eb0 con 0x7f988403c8e0 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f988403c8e0 msgr2=0x7f988403ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f988403c8e0 0x7f988403ecd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9890003a10 tx=0x7f9890009290 comp rx=0 tx=0).stop 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 msgr2=0x7f989c191810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:18.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c191810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f988c009fb0 tx=0x7f988c011bf0 comp rx=0 tx=0).stop 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 shutdown_connections 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f988403c8e0 0x7f988403ecd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f989c106660 0x7f989c191810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 --2- 172.21.15.22:0/2183380888 >> [v2:172.21.15.22:6800/3384641534,v1:172.21.15.22:6801/3384641534] conn(0x7f9884038ed0 0x7f988403b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 >> 172.21.15.22:0/2183380888 conn(0x7f989c101c80 msgr2=0x7f989c102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 shutdown_connections 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:17.527+0000 7f98a2511700 1 -- 172.21.15.22:0/2183380888 wait complete. 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2024-03-08T23:44:18.123 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2024-03-08T23:44:18.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:18 smithi022 ceph-mon[31894]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-08T23:44:18.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:18 smithi022 ceph-mon[31894]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-08T23:44:19.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f69662700 1 Processor -- start 2024-03-08T23:44:19.380 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f69662700 1 -- start start 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f69662700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f6410a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f69662700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6407ae70 con 0x7f2f64108230 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f62ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f6410a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f62ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f6410a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43742/0 (socket says 172.21.15.22:43742) 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f62ffd700 1 -- 172.21.15.22:0/1413114659 learned_addr learned my addr 172.21.15.22:0/1413114659 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.561+0000 7f2f62ffd700 1 -- 172.21.15.22:0/1413114659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6410ab60 con 0x7f2f64108230 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.562+0000 7f2f62ffd700 1 --2- 172.21.15.22:0/1413114659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f6410a620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3658da112d46b89b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.562+0000 7f2f61ffb700 1 -- 172.21.15.22:0/1413114659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f54004160 con 0x7f2f64108230 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.562+0000 7f2f61ffb700 1 -- 172.21.15.22:0/1413114659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f54004300 con 0x7f2f64108230 2024-03-08T23:44:19.381 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.562+0000 7f2f61ffb700 1 -- 172.21.15.22:0/1413114659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f54011b50 con 0x7f2f64108230 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.562+0000 7f2f69662700 1 -- 172.21.15.22:0/1413114659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 msgr2=0x7f2f6410a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.562+0000 7f2f69662700 1 --2- 172.21.15.22:0/1413114659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f6410a620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 -- 172.21.15.22:0/1413114659 shutdown_connections 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 --2- 172.21.15.22:0/1413114659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f6410a620 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 -- 172.21.15.22:0/1413114659 >> 172.21.15.22:0/1413114659 conn(0x7f2f64101c40 msgr2=0x7f2f64104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 -- 172.21.15.22:0/1413114659 shutdown_connections 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 -- 172.21.15.22:0/1413114659 wait complete. 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 Processor -- start 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 -- start start 2024-03-08T23:44:19.382 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.563+0000 7f2f69662700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f64107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f69662700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6410bb30 con 0x7f2f64108230 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f62ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f64107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f62ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f64107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43752/0 (socket says 172.21.15.22:43752) 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f62ffd700 1 -- 172.21.15.22:0/4129572961 learned_addr learned my addr 172.21.15.22:0/4129572961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f62ffd700 1 -- 172.21.15.22:0/4129572961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64107ff0 con 0x7f2f64108230 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f62ffd700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f64107ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f54011670 tx=0x7f2f54011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f540039c0 con 0x7f2f64108230 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f641061e0 con 0x7f2f64108230 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f54011e50 con 0x7f2f64108230 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f5401a5c0 con 0x7f2f64108230 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.564+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f64106720 con 0x7f2f64108230 2024-03-08T23:44:19.383 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.565+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7f2f5401a8c0 con 0x7f2f64108230 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.565+0000 7f2f4bfff700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f2f4c038dc0 0x7f2f4c03b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.565+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2f5404d1d0 con 0x7f2f64108230 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.565+0000 7f2f627fc700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f2f4c038dc0 0x7f2f4c03b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.566+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f64063410 con 0x7f2f64108230 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.566+0000 7f2f627fc700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f2f4c038dc0 0x7f2f4c03b280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f58006fd0 tx=0x7f2f58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.571+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2f54019c70 con 0x7f2f64108230 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.575+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f2f5402d910 con 0x7f2f64108230 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.806+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f64106e40 con 0x7f2f4c038dc0 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.811+0000 7f2f4bfff700 1 -- 172.21.15.22:0/4129572961 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2f64106e40 con 0x7f2f4c038dc0 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f2f4c038dc0 msgr2=0x7f2f4c03b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:19.384 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f2f4c038dc0 0x7f2f4c03b280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f58006fd0 tx=0x7f2f58006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 msgr2=0x7f2f64107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f64107ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f54011670 tx=0x7f2f54011b60 comp rx=0 tx=0).stop 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 shutdown_connections 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f2f4c038dc0 0x7f2f4c03b280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 --2- 172.21.15.22:0/4129572961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f64108230 0x7f2f64107ab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 >> 172.21.15.22:0/4129572961 conn(0x7f2f64101c40 msgr2=0x7f2f64102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 shutdown_connections 2024-03-08T23:44:19.385 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:18.813+0000 7f2f69662700 1 -- 172.21.15.22:0/4129572961 wait complete. 2024-03-08T23:44:19.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: mgrmap e8: x(active, since 2s) 2024-03-08T23:44:19.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:19.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:19.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:19.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:18] ENGINE Bus STARTING 2024-03-08T23:44:19.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:18] ENGINE Serving on http://172.21.15.22:8765 2024-03-08T23:44:19.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:19] ENGINE Serving on https://172.21.15.22:7150 2024-03-08T23:44:19.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:19] ENGINE Bus STARTED 2024-03-08T23:44:19.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:19 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:20.594 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-08T23:44:20.594 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.779+0000 7faad62c3700 1 Processor -- start 2024-03-08T23:44:20.594 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.779+0000 7faad62c3700 1 -- start start 2024-03-08T23:44:20.594 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faad62c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad0107c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:20.594 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faad62c3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad0108230 con 0x7faad0107840 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faacffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad0107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faacffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad0107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43764/0 (socket says 172.21.15.22:43764) 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faacffff700 1 -- 172.21.15.22:0/375986189 learned_addr learned my addr 172.21.15.22:0/375986189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faacffff700 1 -- 172.21.15.22:0/375986189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad01089f0 con 0x7faad0107840 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faacffff700 1 --2- 172.21.15.22:0/375986189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad0107c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faac00098a0 tx=0x7faac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693e91ccf0b3c075 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faaceffd700 1 -- 172.21.15.22:0/375986189 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faac0004160 con 0x7faad0107840 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faaceffd700 1 -- 172.21.15.22:0/375986189 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faac0004300 con 0x7faad0107840 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.780+0000 7faaceffd700 1 -- 172.21.15.22:0/375986189 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faac0011b50 con 0x7faad0107840 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.781+0000 7faad62c3700 1 -- 172.21.15.22:0/375986189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 msgr2=0x7faad0107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.781+0000 7faad62c3700 1 --2- 172.21.15.22:0/375986189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad0107c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faac00098a0 tx=0x7faac0009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.781+0000 7faad62c3700 1 -- 172.21.15.22:0/375986189 shutdown_connections 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.781+0000 7faad62c3700 1 --2- 172.21.15.22:0/375986189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad0107c60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.781+0000 7faad62c3700 1 -- 172.21.15.22:0/375986189 >> 172.21.15.22:0/375986189 conn(0x7faad0102e00 msgr2=0x7faad0105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.781+0000 7faad62c3700 1 -- 172.21.15.22:0/375986189 shutdown_connections 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.781+0000 7faad62c3700 1 -- 172.21.15.22:0/375986189 wait complete. 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faad62c3700 1 Processor -- start 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faad62c3700 1 -- start start 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faad62c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faad62c3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad010bda0 con 0x7faad0107840 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faacffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faacffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43772/0 (socket says 172.21.15.22:43772) 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faacffff700 1 -- 172.21.15.22:0/83625398 learned_addr learned my addr 172.21.15.22:0/83625398 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.782+0000 7faacffff700 1 -- 172.21.15.22:0/83625398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad019bcb0 con 0x7faad0107840 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.783+0000 7faacffff700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad019b770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faac0011850 tx=0x7faac0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.783+0000 7faacd7fa700 1 -- 172.21.15.22:0/83625398 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faac0003af0 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.783+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad019bf40 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.783+0000 7faacd7fa700 1 -- 172.21.15.22:0/83625398 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faac0020070 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.783+0000 7faacd7fa700 1 -- 172.21.15.22:0/83625398 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faac00195a0 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.783+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad019ec30 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.784+0000 7faacd7fa700 1 -- 172.21.15.22:0/83625398 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7faac0019850 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.784+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad0068eb0 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.784+0000 7faacd7fa700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7faab8038f70 0x7faab803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.784+0000 7faacd7fa700 1 -- 172.21.15.22:0/83625398 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faac0011e00 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.784+0000 7faacf7fe700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7faab8038f70 0x7faab803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.784+0000 7faacf7fe700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7faab8038f70 0x7faab803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faac4006fd0 tx=0x7faac4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.789+0000 7faacd7fa700 1 -- 172.21.15.22:0/83625398 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7faac0003c90 con 0x7faad0107840 2024-03-08T23:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.992+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7faad01040a0 con 0x7faab8038f70 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.993+0000 7faacd7fa700 1 -- 172.21.15.22:0/83625398 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7faad01040a0 con 0x7faab8038f70 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7faab8038f70 msgr2=0x7faab803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7faab8038f70 0x7faab803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faac4006fd0 tx=0x7faac4006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 msgr2=0x7faad019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad019b770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faac0011850 tx=0x7faac0011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 shutdown_connections 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7faab8038f70 0x7faab803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 --2- 172.21.15.22:0/83625398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faad0107840 0x7faad019b770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 >> 172.21.15.22:0/83625398 conn(0x7faad0102e00 msgr2=0x7faad0103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 shutdown_connections 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:19.995+0000 7faad62c3700 1 -- 172.21.15.22:0/83625398 wait complete. 2024-03-08T23:44:20.598 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2024-03-08T23:44:20.829 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:20 smithi022 ceph-mon[31894]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:21.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-mon[31894]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:21.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-mon[31894]: Generating ssh key... 2024-03-08T23:44:21.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:21.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:21.834 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: Generating public/private rsa key pair. 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: Your identification has been saved in /tmp/tmpknhc535l/key. 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: Your public key has been saved in /tmp/tmpknhc535l/key.pub. 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: The key fingerprint is: 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: SHA256:U+8w+e4YZl/Ht4ujTPAkD/uoX2z5gY7jUxGFjSc8znw ceph-8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: The key's randomart image is: 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: +---[RSA 3072]----+ 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | . =. | 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | B o | 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | = = | 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | . B E | 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | S++.+ | 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | .X*o . | 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | .=Xo.. +| 2024-03-08T23:44:21.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | =X=.oo.o| 2024-03-08T23:44:21.836 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: | .+=+*=o.o.| 2024-03-08T23:44:21.836 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: +----[SHA256]-----+ 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.045+0000 7fed7e17f700 1 Processor -- start 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.045+0000 7fed7e17f700 1 -- start start 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.045+0000 7fed7e17f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed78106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.045+0000 7fed7e17f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7807ae50 con 0x7fed78105ff0 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.045+0000 7fed777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed78106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.046+0000 7fed777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed78106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35932/0 (socket says 172.21.15.22:35932) 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.046+0000 7fed777fe700 1 -- 172.21.15.22:0/4139995711 learned_addr learned my addr 172.21.15.22:0/4139995711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:22.084 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.046+0000 7fed777fe700 1 -- 172.21.15.22:0/4139995711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed78106950 con 0x7fed78105ff0 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.046+0000 7fed777fe700 1 --2- 172.21.15.22:0/4139995711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed78106410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed680098a0 tx=0x7fed68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237a66e264080b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.046+0000 7fed767fc700 1 -- 172.21.15.22:0/4139995711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed68004160 con 0x7fed78105ff0 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.046+0000 7fed767fc700 1 -- 172.21.15.22:0/4139995711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed68004300 con 0x7fed78105ff0 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.046+0000 7fed767fc700 1 -- 172.21.15.22:0/4139995711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed68011b50 con 0x7fed78105ff0 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.047+0000 7fed7e17f700 1 -- 172.21.15.22:0/4139995711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 msgr2=0x7fed78106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.047+0000 7fed7e17f700 1 --2- 172.21.15.22:0/4139995711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed78106410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed680098a0 tx=0x7fed68009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.047+0000 7fed7e17f700 1 -- 172.21.15.22:0/4139995711 shutdown_connections 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.047+0000 7fed7e17f700 1 --2- 172.21.15.22:0/4139995711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed78106410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.047+0000 7fed7e17f700 1 -- 172.21.15.22:0/4139995711 >> 172.21.15.22:0/4139995711 conn(0x7fed78101c30 msgr2=0x7fed78104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.047+0000 7fed7e17f700 1 -- 172.21.15.22:0/4139995711 shutdown_connections 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.047+0000 7fed7e17f700 1 -- 172.21.15.22:0/4139995711 wait complete. 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed7e17f700 1 Processor -- start 2024-03-08T23:44:22.085 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed7e17f700 1 -- start start 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed7e17f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed7819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed7e17f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781079e0 con 0x7fed78105ff0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed7819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed7819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35942/0 (socket says 172.21.15.22:35942) 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed777fe700 1 -- 172.21.15.22:0/2305945138 learned_addr learned my addr 172.21.15.22:0/2305945138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.048+0000 7fed777fe700 1 -- 172.21.15.22:0/2305945138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7819de20 con 0x7fed78105ff0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.049+0000 7fed777fe700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed7819d8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed68011670 tx=0x7fed68011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.049+0000 7fed74ff9700 1 -- 172.21.15.22:0/2305945138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed680039c0 con 0x7fed78105ff0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.049+0000 7fed74ff9700 1 -- 172.21.15.22:0/2305945138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed68011e50 con 0x7fed78105ff0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.049+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7819e0b0 con 0x7fed78105ff0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.049+0000 7fed74ff9700 1 -- 172.21.15.22:0/2305945138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed6801a5c0 con 0x7fed78105ff0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.049+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed781a0da0 con 0x7fed78105ff0 2024-03-08T23:44:22.086 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.050+0000 7fed74ff9700 1 -- 172.21.15.22:0/2305945138 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fed68024430 con 0x7fed78105ff0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.050+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed78068eb0 con 0x7fed78105ff0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.050+0000 7fed74ff9700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7fed60038ed0 0x7fed6003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.050+0000 7fed74ff9700 1 -- 172.21.15.22:0/2305945138 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed6804d310 con 0x7fed78105ff0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.050+0000 7fed76ffd700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7fed60038ed0 0x7fed6003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.050+0000 7fed76ffd700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7fed60038ed0 0x7fed6003b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006fd0 tx=0x7fed6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.055+0000 7fed74ff9700 1 -- 172.21.15.22:0/2305945138 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fed6802d910 con 0x7fed78105ff0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.263+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fed780008d0 con 0x7fed60038ed0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.533+0000 7fed74ff9700 1 -- 172.21.15.22:0/2305945138 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fed780008d0 con 0x7fed60038ed0 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7fed60038ed0 msgr2=0x7fed6003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7fed60038ed0 0x7fed6003b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006fd0 tx=0x7fed6c006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 msgr2=0x7fed7819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed7819d8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fed68011670 tx=0x7fed68011b60 comp rx=0 tx=0).stop 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 shutdown_connections 2024-03-08T23:44:22.087 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7fed60038ed0 0x7fed6003b390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:22.088 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 --2- 172.21.15.22:0/2305945138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed78105ff0 0x7fed7819d8e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:22.088 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.535+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 >> 172.21.15.22:0/2305945138 conn(0x7fed78101c30 msgr2=0x7fed78194550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:22.088 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.536+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 shutdown_connections 2024-03-08T23:44:22.088 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:21.536+0000 7fed7e17f700 1 -- 172.21.15.22:0/2305945138 wait complete. 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC85usSZKjWMuxIhqOs3KZl0lHDRrFUJeLlIoSTeAwuJY1cw+YeRwl023yTbheHJT3OibrxirzVYjLiQUByPjfr8/FFYBwLpl5+SupJ6u5Hw8dr2DE5V93T/MG55eIYx4MWQaisZMNxzapnscnvd5njzx28KZnX+IYDFGPEa5DSoq0axRd2nRLEr6pKmGLYVHxtcnGu5QkbiTRMt6RGY41GSgrSJQOhwbPlJTDzeuKvyoKKi7TFIrGgb78wtKfd197nd1JQmDRkr/oIYJDmvAdg6oCPxfTjIhOBbDygGj4poO4cMp3LOKjB2BoxqA/KU77qT/4dAErLAbj6pEtZPVGYb1M9UBaiQdUepcoUCmn0IlJOlm3yhjFmEp8kc3Q4GLZFhSVA66gwu46TeNSf8QpqPU+ImeiOyrlzNaR79HH3Dznxjs13RmCRrM9D4gbvPqkmSNaFvqLu0R6Nxa1kCtxZ1R0FeLCzKhZWQ8v2aXWbhyE+bhYf7CtnBltBvod4I9s= ceph-8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d12fd1700 1 Processor -- start 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d12fd1700 1 -- start start 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d12fd1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c107c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d12fd1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c108230 con 0x7f8d0c107840 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d10d6d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d10d6d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35946/0 (socket says 172.21.15.22:35946) 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d10d6d700 1 -- 172.21.15.22:0/2194036537 learned_addr learned my addr 172.21.15.22:0/2194036537 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.455+0000 7f8d10d6d700 1 -- 172.21.15.22:0/2194036537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c1089f0 con 0x7f8d0c107840 2024-03-08T23:44:23.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d10d6d700 1 --2- 172.21.15.22:0/2194036537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c107c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d000098a0 tx=0x7f8d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57e6f15e967c9d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d0b7fe700 1 -- 172.21.15.22:0/2194036537 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00004160 con 0x7f8d0c107840 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d0b7fe700 1 -- 172.21.15.22:0/2194036537 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d00004300 con 0x7f8d0c107840 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d0b7fe700 1 -- 172.21.15.22:0/2194036537 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00011b50 con 0x7f8d0c107840 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d12fd1700 1 -- 172.21.15.22:0/2194036537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 msgr2=0x7f8d0c107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d12fd1700 1 --2- 172.21.15.22:0/2194036537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c107c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d000098a0 tx=0x7f8d00009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d12fd1700 1 -- 172.21.15.22:0/2194036537 shutdown_connections 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d12fd1700 1 --2- 172.21.15.22:0/2194036537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c107c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.456+0000 7f8d12fd1700 1 -- 172.21.15.22:0/2194036537 >> 172.21.15.22:0/2194036537 conn(0x7f8d0c102e00 msgr2=0x7f8d0c105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d12fd1700 1 -- 172.21.15.22:0/2194036537 shutdown_connections 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d12fd1700 1 -- 172.21.15.22:0/2194036537 wait complete. 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d12fd1700 1 Processor -- start 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d12fd1700 1 -- start start 2024-03-08T23:44:23.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d12fd1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d12fd1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c10bda0 con 0x7f8d0c107840 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d10d6d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d10d6d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35956/0 (socket says 172.21.15.22:35956) 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.457+0000 7f8d10d6d700 1 -- 172.21.15.22:0/1183032 learned_addr learned my addr 172.21.15.22:0/1183032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.458+0000 7f8d10d6d700 1 -- 172.21.15.22:0/1183032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c19bcb0 con 0x7f8d0c107840 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.458+0000 7f8d10d6d700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c19b770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d00011850 tx=0x7f8d00011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.458+0000 7f8d09ffb700 1 -- 172.21.15.22:0/1183032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00003af0 con 0x7f8d0c107840 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.458+0000 7f8d09ffb700 1 -- 172.21.15.22:0/1183032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d00020070 con 0x7f8d0c107840 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.458+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c19bf40 con 0x7f8d0c107840 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.458+0000 7f8d09ffb700 1 -- 172.21.15.22:0/1183032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d000195a0 con 0x7f8d0c107840 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.458+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c19ec30 con 0x7f8d0c107840 2024-03-08T23:44:23.217 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.459+0000 7f8d09ffb700 1 -- 172.21.15.22:0/1183032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f8d00019850 con 0x7f8d0c107840 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.459+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c068eb0 con 0x7f8d0c107840 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.459+0000 7f8d09ffb700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f8cfc038f70 0x7f8cfc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.459+0000 7f8d09ffb700 1 -- 172.21.15.22:0/1183032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8d00011e00 con 0x7f8d0c107840 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.459+0000 7f8d0bfff700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f8cfc038f70 0x7f8cfc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.460+0000 7f8d0bfff700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f8cfc038f70 0x7f8cfc03b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006fd0 tx=0x7f8cf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.464+0000 7f8d09ffb700 1 -- 172.21.15.22:0/1183032 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8d00003c90 con 0x7f8d0c107840 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.670+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d0c1040a0 con 0x7f8cfc038f70 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.670+0000 7f8d09ffb700 1 -- 172.21.15.22:0/1183032 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f8d0c1040a0 con 0x7f8cfc038f70 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f8cfc038f70 msgr2=0x7f8cfc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f8cfc038f70 0x7f8cfc03b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006fd0 tx=0x7f8cf8006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 msgr2=0x7f8d0c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c19b770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d00011850 tx=0x7f8d00011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:23.218 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 shutdown_connections 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f8cfc038f70 0x7f8cfc03b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 --2- 172.21.15.22:0/1183032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d0c107840 0x7f8d0c19b770 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.672+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 >> 172.21.15.22:0/1183032 conn(0x7f8d0c102e00 msgr2=0x7f8d0c103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.673+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 shutdown_connections 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:22.673+0000 7f8d12fd1700 1 -- 172.21.15.22:0/1183032 wait complete. 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2024-03-08T23:44:23.219 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2024-03-08T23:44:23.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:23 smithi022 ceph-mon[31894]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:24.701 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:24 smithi022 ceph-mon[31894]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:25 smithi022 ceph-mon[31894]: Deploying cephadm binary to smithi022 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f524e2be700 1 Processor -- start 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f524e2be700 1 -- start start 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f524e2be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f5248107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f524e2be700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5248108190 con 0x7f52481077a0 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f5247fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f5248107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f5247fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f5248107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35960/0 (socket says 172.21.15.22:35960) 2024-03-08T23:44:28.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f5247fff700 1 -- 172.21.15.22:0/4094860277 learned_addr learned my addr 172.21.15.22:0/4094860277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f5247fff700 1 -- 172.21.15.22:0/4094860277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248108950 con 0x7f52481077a0 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.625+0000 7f5247fff700 1 --2- 172.21.15.22:0/4094860277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f5248107bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f52380098a0 tx=0x7f5238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd353f5b90d385ee server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f5246ffd700 1 -- 172.21.15.22:0/4094860277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5238004160 con 0x7f52481077a0 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f5246ffd700 1 -- 172.21.15.22:0/4094860277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5238004300 con 0x7f52481077a0 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f5246ffd700 1 -- 172.21.15.22:0/4094860277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5238011b50 con 0x7f52481077a0 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f524e2be700 1 -- 172.21.15.22:0/4094860277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 msgr2=0x7f5248107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f524e2be700 1 --2- 172.21.15.22:0/4094860277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f5248107bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f52380098a0 tx=0x7f5238009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f524e2be700 1 -- 172.21.15.22:0/4094860277 shutdown_connections 2024-03-08T23:44:28.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f524e2be700 1 --2- 172.21.15.22:0/4094860277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f5248107bc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f524e2be700 1 -- 172.21.15.22:0/4094860277 >> 172.21.15.22:0/4094860277 conn(0x7f5248102da0 msgr2=0x7f52481051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f524e2be700 1 -- 172.21.15.22:0/4094860277 shutdown_connections 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.626+0000 7f524e2be700 1 -- 172.21.15.22:0/4094860277 wait complete. 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f524e2be700 1 Processor -- start 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f524e2be700 1 -- start start 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f524e2be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f524807cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f524e2be700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524810bd00 con 0x7f52481077a0 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f5247fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f524807cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f5247fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f524807cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35976/0 (socket says 172.21.15.22:35976) 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f5247fff700 1 -- 172.21.15.22:0/3462061850 learned_addr learned my addr 172.21.15.22:0/3462061850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.627+0000 7f5247fff700 1 -- 172.21.15.22:0/3462061850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524807d0d0 con 0x7f52481077a0 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.628+0000 7f5247fff700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f524807cb90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5238000c00 tx=0x7f5238011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:28.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.628+0000 7f52457fa700 1 -- 172.21.15.22:0/3462061850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5238003b20 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.628+0000 7f52457fa700 1 -- 172.21.15.22:0/3462061850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5238003cc0 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.628+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524807b2c0 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.628+0000 7f52457fa700 1 -- 172.21.15.22:0/3462061850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5238024910 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.628+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524807b800 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.629+0000 7f52457fa700 1 -- 172.21.15.22:0/3462061850 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f5238022020 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.629+0000 7f52457fa700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f5230038f70 0x7f523003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.629+0000 7f52457fa700 1 -- 172.21.15.22:0/3462061850 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5238011e00 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.629+0000 7f52477fe700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f5230038f70 0x7f523003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.629+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5248063410 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.629+0000 7f52477fe700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f5230038f70 0x7f523003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f523c006fd0 tx=0x7f523c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.634+0000 7f52457fa700 1 -- 172.21.15.22:0/3462061850 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f52380172b0 con 0x7f52481077a0 2024-03-08T23:44:28.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:23.840+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}) v1 -- 0x7f524807bf30 con 0x7f5230038f70 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.215+0000 7f52457fa700 1 -- 172.21.15.22:0/3462061850 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f524807bf30 con 0x7f5230038f70 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f5230038f70 msgr2=0x7f523003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f5230038f70 0x7f523003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f523c006fd0 tx=0x7f523c006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 msgr2=0x7f524807cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f524807cb90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5238000c00 tx=0x7f5238011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 shutdown_connections 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f5230038f70 0x7f523003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 --2- 172.21.15.22:0/3462061850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52481077a0 0x7f524807cb90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 >> 172.21.15.22:0/3462061850 conn(0x7f5248102da0 msgr2=0x7f52481038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:28.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 shutdown_connections 2024-03-08T23:44:28.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:28.218+0000 7f524e2be700 1 -- 172.21.15.22:0/3462061850 wait complete. 2024-03-08T23:44:28.786 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mon service... 2024-03-08T23:44:29.449 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:29 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:29.449 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:29 smithi022 ceph-mon[31894]: Added host smithi022 2024-03-08T23:44:29.449 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:29 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:30.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:30 smithi022 ceph-mon[31894]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:30.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:30 smithi022 ceph-mon[31894]: Saving service mon spec with placement count:5 2024-03-08T23:44:30.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:30 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:31.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-08T23:44:31.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0d0591b700 1 Processor -- start 2024-03-08T23:44:31.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0d0591b700 1 -- start start 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0d0591b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d00106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0d0591b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00107030 con 0x7f0d00106640 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0cfeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d00106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0cfeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d00106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35978/0 (socket says 172.21.15.22:35978) 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0cfeffd700 1 -- 172.21.15.22:0/3755045782 learned_addr learned my addr 172.21.15.22:0/3755045782 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0cfeffd700 1 -- 172.21.15.22:0/3755045782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00107840 con 0x7f0d00106640 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.223+0000 7f0cfeffd700 1 --2- 172.21.15.22:0/3755045782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d00106a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00098a0 tx=0x7f0cf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66303d7639a31a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0cfdffb700 1 -- 172.21.15.22:0/3755045782 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf0004160 con 0x7f0d00106640 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0cfdffb700 1 -- 172.21.15.22:0/3755045782 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cf0004300 con 0x7f0d00106640 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0cfdffb700 1 -- 172.21.15.22:0/3755045782 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf0011b50 con 0x7f0d00106640 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0d0591b700 1 -- 172.21.15.22:0/3755045782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 msgr2=0x7f0d00106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0d0591b700 1 --2- 172.21.15.22:0/3755045782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d00106a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00098a0 tx=0x7f0cf0009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:31.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0d0591b700 1 -- 172.21.15.22:0/3755045782 shutdown_connections 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0d0591b700 1 --2- 172.21.15.22:0/3755045782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d00106a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0d0591b700 1 -- 172.21.15.22:0/3755045782 >> 172.21.15.22:0/3755045782 conn(0x7f0d00101cc0 msgr2=0x7f0d001040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0d0591b700 1 -- 172.21.15.22:0/3755045782 shutdown_connections 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.224+0000 7f0d0591b700 1 -- 172.21.15.22:0/3755045782 wait complete. 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.225+0000 7f0d0591b700 1 Processor -- start 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.225+0000 7f0d0591b700 1 -- start start 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.225+0000 7f0d0591b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d001917a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.225+0000 7f0d0591b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0010abf0 con 0x7f0d00106640 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.225+0000 7f0cfeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d001917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.225+0000 7f0cfeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d001917a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35990/0 (socket says 172.21.15.22:35990) 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.225+0000 7f0cfeffd700 1 -- 172.21.15.22:0/1355342799 learned_addr learned my addr 172.21.15.22:0/1355342799 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.226+0000 7f0cfeffd700 1 -- 172.21.15.22:0/1355342799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00191ce0 con 0x7f0d00106640 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.226+0000 7f0cfeffd700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d001917a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0000c00 tx=0x7f0cf0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:31.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.226+0000 7f0d04919700 1 -- 172.21.15.22:0/1355342799 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf0003b20 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.226+0000 7f0d04919700 1 -- 172.21.15.22:0/1355342799 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cf0003cc0 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.226+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0018ff20 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.226+0000 7f0d04919700 1 -- 172.21.15.22:0/1355342799 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf0024910 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.226+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d00190400 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.227+0000 7f0d04919700 1 -- 172.21.15.22:0/1355342799 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f0cf0022020 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.227+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d00063410 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.227+0000 7f0d04919700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f0ce8038f70 0x7f0ce803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.227+0000 7f0d04919700 1 -- 172.21.15.22:0/1355342799 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0cf0011e00 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.227+0000 7f0cfe7fc700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f0ce8038f70 0x7f0ce803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.228+0000 7f0cfe7fc700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f0ce8038f70 0x7f0ce803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006fd0 tx=0x7f0cf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.232+0000 7f0d04919700 1 -- 172.21.15.22:0/1355342799 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0cf001acd0 con 0x7f0d00106640 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.450+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0d001955b0 con 0x7f0ce8038f70 2024-03-08T23:44:31.030 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.455+0000 7f0d04919700 1 -- 172.21.15.22:0/1355342799 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0d001955b0 con 0x7f0ce8038f70 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f0ce8038f70 msgr2=0x7f0ce803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f0ce8038f70 0x7f0ce803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006fd0 tx=0x7f0cf4006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 msgr2=0x7f0d001917a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d001917a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0000c00 tx=0x7f0cf0011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 shutdown_connections 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f0ce8038f70 0x7f0ce803b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 --2- 172.21.15.22:0/1355342799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d00106640 0x7f0d001917a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.457+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 >> 172.21.15.22:0/1355342799 conn(0x7f0d00101cc0 msgr2=0x7f0d00102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.458+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 shutdown_connections 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:29.458+0000 7f0d0591b700 1 -- 172.21.15.22:0/1355342799 wait complete. 2024-03-08T23:44:31.031 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mgr service... 2024-03-08T23:44:31.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:31 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:31.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:31 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:32.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-08T23:44:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.504+0000 7f01dc160700 1 Processor -- start 2024-03-08T23:44:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.504+0000 7f01dc160700 1 -- start start 2024-03-08T23:44:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.504+0000 7f01dc160700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d410a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.504+0000 7f01dc160700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d407aed0 con 0x7f01d4108200 2024-03-08T23:44:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d9efc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d410a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d9efc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d410a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51094/0 (socket says 172.21.15.22:51094) 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d9efc700 1 -- 172.21.15.22:0/233925610 learned_addr learned my addr 172.21.15.22:0/233925610 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d9efc700 1 -- 172.21.15.22:0/233925610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d410ab30 con 0x7f01d4108200 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d9efc700 1 --2- 172.21.15.22:0/233925610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d410a5f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01c40098a0 tx=0x7f01c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49f6e302b7c2685 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d8efa700 1 -- 172.21.15.22:0/233925610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c4004160 con 0x7f01d4108200 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d8efa700 1 -- 172.21.15.22:0/233925610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01c4004300 con 0x7f01d4108200 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.505+0000 7f01d8efa700 1 -- 172.21.15.22:0/233925610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c4011b20 con 0x7f01d4108200 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.506+0000 7f01dc160700 1 -- 172.21.15.22:0/233925610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 msgr2=0x7f01d410a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.506+0000 7f01dc160700 1 --2- 172.21.15.22:0/233925610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d410a5f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01c40098a0 tx=0x7f01c4009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.506+0000 7f01dc160700 1 -- 172.21.15.22:0/233925610 shutdown_connections 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.506+0000 7f01dc160700 1 --2- 172.21.15.22:0/233925610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d410a5f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.506+0000 7f01dc160700 1 -- 172.21.15.22:0/233925610 >> 172.21.15.22:0/233925610 conn(0x7f01d4101c70 msgr2=0x7f01d4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.506+0000 7f01dc160700 1 -- 172.21.15.22:0/233925610 shutdown_connections 2024-03-08T23:44:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.506+0000 7f01dc160700 1 -- 172.21.15.22:0/233925610 wait complete. 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01dc160700 1 Processor -- start 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01dc160700 1 -- start start 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01dc160700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01dc160700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d410bba0 con 0x7f01d4108200 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01d9efc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01d9efc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d419b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51110/0 (socket says 172.21.15.22:51110) 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01d9efc700 1 -- 172.21.15.22:0/339548018 learned_addr learned my addr 172.21.15.22:0/339548018 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01d9efc700 1 -- 172.21.15.22:0/339548018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d419bc30 con 0x7f01d4108200 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.507+0000 7f01d9efc700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d419b6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01c40117e0 tx=0x7f01c4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.508+0000 7f01caffd700 1 -- 172.21.15.22:0/339548018 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c4003ba0 con 0x7f01d4108200 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.508+0000 7f01caffd700 1 -- 172.21.15.22:0/339548018 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01c401a910 con 0x7f01d4108200 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.508+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d419bec0 con 0x7f01d4108200 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.508+0000 7f01caffd700 1 -- 172.21.15.22:0/339548018 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c40248f0 con 0x7f01d4108200 2024-03-08T23:44:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.508+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d419c320 con 0x7f01d4108200 2024-03-08T23:44:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.508+0000 7f01caffd700 1 -- 172.21.15.22:0/339548018 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f01c4022020 con 0x7f01d4108200 2024-03-08T23:44:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.509+0000 7f01caffd700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f01c0038f70 0x7f01c003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.509+0000 7f01caffd700 1 -- 172.21.15.22:0/339548018 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f01c404d680 con 0x7f01d4108200 2024-03-08T23:44:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.509+0000 7f01d96fb700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f01c0038f70 0x7f01c003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.509+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d4063410 con 0x7f01d4108200 2024-03-08T23:44:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.509+0000 7f01d96fb700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f01c0038f70 0x7f01c003b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01d0006fd0 tx=0x7f01d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.514+0000 7f01caffd700 1 -- 172.21.15.22:0/339548018 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f01c40172b0 con 0x7f01d4108200 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.730+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f01d4102e50 con 0x7f01c0038f70 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.735+0000 7f01caffd700 1 -- 172.21.15.22:0/339548018 <== mgr.14120 v2:172.21.15.22:6800/4028157579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f01d4102e50 con 0x7f01c0038f70 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f01c0038f70 msgr2=0x7f01c003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f01c0038f70 0x7f01c003b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01d0006fd0 tx=0x7f01d0006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 msgr2=0x7f01d419b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d419b6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01c40117e0 tx=0x7f01c4011ba0 comp rx=0 tx=0).stop 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 shutdown_connections 2024-03-08T23:44:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f01c0038f70 0x7f01c003b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 --2- 172.21.15.22:0/339548018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d4108200 0x7f01d419b6f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 >> 172.21.15.22:0/339548018 conn(0x7f01d4101c70 msgr2=0x7f01d4102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 shutdown_connections 2024-03-08T23:44:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:31.737+0000 7f01dc160700 1 -- 172.21.15.22:0/339548018 wait complete. 2024-03-08T23:44:32.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:32 smithi022 ceph-mon[31894]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:32.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:32 smithi022 ceph-mon[31894]: Saving service mgr spec with placement count:2 2024-03-08T23:44:32.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:32 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:33.926 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.642+0000 7ffbac5d0700 1 Processor -- start 2024-03-08T23:44:33.926 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.642+0000 7ffbac5d0700 1 -- start start 2024-03-08T23:44:33.926 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.643+0000 7ffbac5d0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba410a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:33.926 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.643+0000 7ffbac5d0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba407ae90 con 0x7ffba4108230 2024-03-08T23:44:33.926 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.643+0000 7ffbaa36c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba410a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:33.926 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.643+0000 7ffbaa36c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba410a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51122/0 (socket says 172.21.15.22:51122) 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.643+0000 7ffbaa36c700 1 -- 172.21.15.22:0/2049921393 learned_addr learned my addr 172.21.15.22:0/2049921393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.643+0000 7ffbaa36c700 1 -- 172.21.15.22:0/2049921393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba410ab60 con 0x7ffba4108230 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.643+0000 7ffbaa36c700 1 --2- 172.21.15.22:0/2049921393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba410a620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098a0 tx=0x7ffba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764da8f0efe7ea42 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffba936a700 1 -- 172.21.15.22:0/2049921393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffba0004160 con 0x7ffba4108230 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffba936a700 1 -- 172.21.15.22:0/2049921393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba0004300 con 0x7ffba4108230 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffba936a700 1 -- 172.21.15.22:0/2049921393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffba0011b20 con 0x7ffba4108230 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2049921393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 msgr2=0x7ffba410a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffbac5d0700 1 --2- 172.21.15.22:0/2049921393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba410a620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098a0 tx=0x7ffba0009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2049921393 shutdown_connections 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffbac5d0700 1 --2- 172.21.15.22:0/2049921393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba410a620 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2049921393 >> 172.21.15.22:0/2049921393 conn(0x7ffba4101cc0 msgr2=0x7ffba41040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.644+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2049921393 shutdown_connections 2024-03-08T23:44:33.927 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2049921393 wait complete. 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbac5d0700 1 Processor -- start 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbac5d0700 1 -- start start 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbac5d0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbac5d0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba410bbc0 con 0x7ffba4108230 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbaa36c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbaa36c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51136/0 (socket says 172.21.15.22:51136) 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.645+0000 7ffbaa36c700 1 -- 172.21.15.22:0/2311128583 learned_addr learned my addr 172.21.15.22:0/2311128583 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.646+0000 7ffbaa36c700 1 -- 172.21.15.22:0/2311128583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba419bbd0 con 0x7ffba4108230 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.646+0000 7ffbaa36c700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba419b690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffba00116a0 tx=0x7ffba0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.646+0000 7ffb9b7fe700 1 -- 172.21.15.22:0/2311128583 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffba0003a40 con 0x7ffba4108230 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.646+0000 7ffb9b7fe700 1 -- 172.21.15.22:0/2311128583 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba0003be0 con 0x7ffba4108230 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.646+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba419be60 con 0x7ffba4108230 2024-03-08T23:44:33.928 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.646+0000 7ffb9b7fe700 1 -- 172.21.15.22:0/2311128583 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffba001a750 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.646+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba419c2c0 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.647+0000 7ffb9b7fe700 1 -- 172.21.15.22:0/2311128583 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7ffba001aa80 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.647+0000 7ffb9b7fe700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7ffb90038f20 0x7ffb9003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.647+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba4068eb0 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.647+0000 7ffb9b7fe700 1 -- 172.21.15.22:0/2311128583 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffba004d3e0 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.647+0000 7ffba9b6b700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7ffb90038f20 0x7ffb9003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.648+0000 7ffba9b6b700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7ffb90038f20 0x7ffb9003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006fd0 tx=0x7ffb94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.654+0000 7ffb9b7fe700 1 -- 172.21.15.22:0/2311128583 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ffba0011e20 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.876+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ffba419ec10 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.879+0000 7ffb9b7fe700 1 -- 172.21.15.22:0/2311128583 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ffba000ba30 con 0x7ffba4108230 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7ffb90038f20 msgr2=0x7ffb9003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7ffb90038f20 0x7ffb9003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006fd0 tx=0x7ffb94006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 msgr2=0x7ffba419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:33.929 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba419b690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffba00116a0 tx=0x7ffba0011ba0 comp rx=0 tx=0).stop 2024-03-08T23:44:33.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 shutdown_connections 2024-03-08T23:44:33.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7ffb90038f20 0x7ffb9003b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:33.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 --2- 172.21.15.22:0/2311128583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffba4108230 0x7ffba419b690 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:33.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 >> 172.21.15.22:0/2311128583 conn(0x7ffba4101cc0 msgr2=0x7ffba4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:33.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 shutdown_connections 2024-03-08T23:44:33.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:32.881+0000 7ffbac5d0700 1 -- 172.21.15.22:0/2311128583 wait complete. 2024-03-08T23:44:33.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:33 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2311128583' entity='client.admin' 2024-03-08T23:44:33.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:33 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:35.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.412+0000 7f05c9bd2700 1 Processor -- start 2024-03-08T23:44:35.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.412+0000 7f05c9bd2700 1 -- start start 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.412+0000 7f05c9bd2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c4105e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.412+0000 7f05c9bd2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4106440 con 0x7f05c4105a50 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c4105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c4105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51144/0 (socket says 172.21.15.22:51144) 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c37fe700 1 -- 172.21.15.22:0/924797646 learned_addr learned my addr 172.21.15.22:0/924797646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c37fe700 1 -- 172.21.15.22:0/924797646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c4108a80 con 0x7f05c4105a50 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c37fe700 1 --2- 172.21.15.22:0/924797646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c4105e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05b80098a0 tx=0x7f05b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b433b1528b2d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c27fc700 1 -- 172.21.15.22:0/924797646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05b8004160 con 0x7f05c4105a50 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c27fc700 1 -- 172.21.15.22:0/924797646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b8004300 con 0x7f05c4105a50 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.413+0000 7f05c27fc700 1 -- 172.21.15.22:0/924797646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05b8011b20 con 0x7f05c4105a50 2024-03-08T23:44:35.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.414+0000 7f05c9bd2700 1 -- 172.21.15.22:0/924797646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 msgr2=0x7f05c4105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.414+0000 7f05c9bd2700 1 --2- 172.21.15.22:0/924797646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c4105e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05b80098a0 tx=0x7f05b8009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.414+0000 7f05c9bd2700 1 -- 172.21.15.22:0/924797646 shutdown_connections 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.414+0000 7f05c9bd2700 1 --2- 172.21.15.22:0/924797646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c4105e70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.414+0000 7f05c9bd2700 1 -- 172.21.15.22:0/924797646 >> 172.21.15.22:0/924797646 conn(0x7f05c41013e0 msgr2=0x7f05c4103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.414+0000 7f05c9bd2700 1 -- 172.21.15.22:0/924797646 shutdown_connections 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.414+0000 7f05c9bd2700 1 -- 172.21.15.22:0/924797646 wait complete. 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c9bd2700 1 Processor -- start 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c9bd2700 1 -- start start 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c9bd2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c9bd2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c410bcb0 con 0x7f05c4105a50 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c419b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51152/0 (socket says 172.21.15.22:51152) 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c37fe700 1 -- 172.21.15.22:0/1631717436 learned_addr learned my addr 172.21.15.22:0/1631717436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:35.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.415+0000 7f05c37fe700 1 -- 172.21.15.22:0/1631717436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c419bc60 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.416+0000 7f05c37fe700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c419b720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05b80117e0 tx=0x7f05b8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.416+0000 7f05c0ff9700 1 -- 172.21.15.22:0/1631717436 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05b8003bf0 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.416+0000 7f05c0ff9700 1 -- 172.21.15.22:0/1631717436 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b801a910 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.416+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c419bef0 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.416+0000 7f05c0ff9700 1 -- 172.21.15.22:0/1631717436 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05b80249a0 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.416+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c419c350 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.417+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c4063410 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.417+0000 7f05c0ff9700 1 -- 172.21.15.22:0/1631717436 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f05b8022020 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.417+0000 7f05c0ff9700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f05ac038f20 0x7f05ac03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.417+0000 7f05c0ff9700 1 -- 172.21.15.22:0/1631717436 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f05b804d150 con 0x7f05c4105a50 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.417+0000 7f05c2ffd700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f05ac038f20 0x7f05ac03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:35.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.418+0000 7f05c2ffd700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f05ac038f20 0x7f05ac03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006fd0 tx=0x7f05b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.422+0000 7f05c0ff9700 1 -- 172.21.15.22:0/1631717436 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f05b8011e20 con 0x7f05c4105a50 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.635+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f05c419ebe0 con 0x7f05c4105a50 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.638+0000 7f05c0ff9700 1 -- 172.21.15.22:0/1631717436 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f05b800ba00 con 0x7f05c4105a50 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f05ac038f20 msgr2=0x7f05ac03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f05ac038f20 0x7f05ac03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006fd0 tx=0x7f05b4006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 msgr2=0x7f05c419b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c419b720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05b80117e0 tx=0x7f05b8011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 shutdown_connections 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f05ac038f20 0x7f05ac03b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 --2- 172.21.15.22:0/1631717436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05c4105a50 0x7f05c419b720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.640+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 >> 172.21.15.22:0/1631717436 conn(0x7f05c41013e0 msgr2=0x7f05c4101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.641+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 shutdown_connections 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:34.641+0000 7f05c9bd2700 1 -- 172.21.15.22:0/1631717436 wait complete. 2024-03-08T23:44:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2024-03-08T23:44:35.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:35 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1631717436' entity='client.admin' 2024-03-08T23:44:35.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:35 smithi022 ceph-mon[31894]: from='mgr.14120 172.21.15.22:0/2945284751' entity='mgr.x' 2024-03-08T23:44:36.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:36 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2275615812' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-08T23:44:36.953 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: ignoring --setuser ceph since I am not root 2024-03-08T23:44:36.953 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: ignoring --setgroup ceph since I am not root 2024-03-08T23:44:37.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98bdb54700 1 Processor -- start 2024-03-08T23:44:37.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98bdb54700 1 -- start start 2024-03-08T23:44:37.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98bdb54700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b8108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:37.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98bdb54700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b81085e0 con 0x7f98b8107bf0 2024-03-08T23:44:37.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98b77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b8108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98b77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b8108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51166/0 (socket says 172.21.15.22:51166) 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98b77fe700 1 -- 172.21.15.22:0/908534856 learned_addr learned my addr 172.21.15.22:0/908534856 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.624+0000 7f98b77fe700 1 -- 172.21.15.22:0/908534856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8108760 con 0x7f98b8107bf0 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98b77fe700 1 --2- 172.21.15.22:0/908534856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b8108010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98a80098a0 tx=0x7f98a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15def02c67588a73 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98b67fc700 1 -- 172.21.15.22:0/908534856 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98a8004160 con 0x7f98b8107bf0 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98b67fc700 1 -- 172.21.15.22:0/908534856 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98a8004300 con 0x7f98b8107bf0 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98b67fc700 1 -- 172.21.15.22:0/908534856 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98a8011b20 con 0x7f98b8107bf0 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98bdb54700 1 -- 172.21.15.22:0/908534856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 msgr2=0x7f98b8108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98bdb54700 1 --2- 172.21.15.22:0/908534856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b8108010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98a80098a0 tx=0x7f98a8009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98bdb54700 1 -- 172.21.15.22:0/908534856 shutdown_connections 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98bdb54700 1 --2- 172.21.15.22:0/908534856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b8108010 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.625+0000 7f98bdb54700 1 -- 172.21.15.22:0/908534856 >> 172.21.15.22:0/908534856 conn(0x7f98b81013e0 msgr2=0x7f98b8103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:37.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.626+0000 7f98bdb54700 1 -- 172.21.15.22:0/908534856 shutdown_connections 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.626+0000 7f98bdb54700 1 -- 172.21.15.22:0/908534856 wait complete. 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.626+0000 7f98bdb54700 1 Processor -- start 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.626+0000 7f98bdb54700 1 -- start start 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98bdb54700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98bdb54700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b810ab70 con 0x7f98b8107bf0 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98b77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98b77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b819b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51172/0 (socket says 172.21.15.22:51172) 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98b77fe700 1 -- 172.21.15.22:0/2275615812 learned_addr learned my addr 172.21.15.22:0/2275615812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98b77fe700 1 -- 172.21.15.22:0/2275615812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b819bc60 con 0x7f98b8107bf0 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98b77fe700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b819b720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98a80117e0 tx=0x7f98a8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:37.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98a8003bf0 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98a801a910 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b819bef0 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.627+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b819c350 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.628+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98a8024970 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.628+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f98a8022020 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.628+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a4005360 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.629+0000 7f98b4ff9700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f98a0038f70 0x7f98a003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.629+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f98a804d710 con 0x7f98b8107bf0 2024-03-08T23:44:37.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.629+0000 7f98b6ffd700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f98a0038f70 0x7f98a003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.629+0000 7f98b6ffd700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f98a0038f70 0x7f98a003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98ac006fd0 tx=0x7f98ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.634+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f98a80172b0 con 0x7f98b8107bf0 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:35.908+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f98a4005650 con 0x7f98b8107bf0 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.640+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f98a800b5a0 con 0x7f98b8107bf0 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.642+0000 7f98b4ff9700 1 -- 172.21.15.22:0/2275615812 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f98a8018aa0 con 0x7f98b8107bf0 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.643+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f98a0038f70 msgr2=0x7f98a003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.643+0000 7f98bdb54700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f98a0038f70 0x7f98a003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98ac006fd0 tx=0x7f98ac006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.643+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 msgr2=0x7f98b819b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.643+0000 7f98bdb54700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b819b720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98a80117e0 tx=0x7f98a8011ba0 comp rx=0 tx=0).stop 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.644+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 shutdown_connections 2024-03-08T23:44:37.215 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.644+0000 7f98bdb54700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f98a0038f70 0x7f98a003b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:37.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.644+0000 7f98bdb54700 1 --2- 172.21.15.22:0/2275615812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98b8107bf0 0x7f98b819b720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:37.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.644+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 >> 172.21.15.22:0/2275615812 conn(0x7f98b81013e0 msgr2=0x7f98b8102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:37.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.644+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 shutdown_connections 2024-03-08T23:44:37.216 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:36.644+0000 7f98bdb54700 1 -- 172.21.15.22:0/2275615812 wait complete. 2024-03-08T23:44:37.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:37 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:37.231+0000 7f77333df200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:44:37.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:37 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:37.328+0000 7f77333df200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:44:37.847 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:37 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2275615812' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-08T23:44:37.847 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:37 smithi022 ceph-mon[31894]: mgrmap e9: x(active, since 20s) 2024-03-08T23:44:38.201 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.003+0000 7f77333df200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:44:38.201 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.098+0000 7f77333df200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:44:38.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.244+0000 7f77333df200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:44:38.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.342+0000 7f77333df200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:44:38.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.427+0000 7f77333df200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:44:38.536 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.640+0000 7f672617e700 1 Processor -- start 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.640+0000 7f672617e700 1 -- start start 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.640+0000 7f672617e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f6720107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.640+0000 7f672617e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720108180 con 0x7f6720107790 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.641+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f6720107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.641+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f6720107bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51204/0 (socket says 172.21.15.22:51204) 2024-03-08T23:44:38.537 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.641+0000 7f671f7fe700 1 -- 172.21.15.22:0/1956772877 learned_addr learned my addr 172.21.15.22:0/1956772877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.641+0000 7f671f7fe700 1 -- 172.21.15.22:0/1956772877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6720108940 con 0x7f6720107790 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.641+0000 7f671f7fe700 1 --2- 172.21.15.22:0/1956772877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f6720107bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67100098a0 tx=0x7f6710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afc21bd72276f43 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.643+0000 7f671e7fc700 1 -- 172.21.15.22:0/1956772877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6710004160 con 0x7f6720107790 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.643+0000 7f671e7fc700 1 -- 172.21.15.22:0/1956772877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6710004300 con 0x7f6720107790 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.643+0000 7f671e7fc700 1 -- 172.21.15.22:0/1956772877 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6710011b20 con 0x7f6720107790 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.644+0000 7f672617e700 1 -- 172.21.15.22:0/1956772877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 msgr2=0x7f6720107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.644+0000 7f672617e700 1 --2- 172.21.15.22:0/1956772877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f6720107bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67100098a0 tx=0x7f6710009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.644+0000 7f672617e700 1 -- 172.21.15.22:0/1956772877 shutdown_connections 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.644+0000 7f672617e700 1 --2- 172.21.15.22:0/1956772877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f6720107bb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.644+0000 7f672617e700 1 -- 172.21.15.22:0/1956772877 >> 172.21.15.22:0/1956772877 conn(0x7f6720102d70 msgr2=0x7f6720105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.645+0000 7f672617e700 1 -- 172.21.15.22:0/1956772877 shutdown_connections 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.645+0000 7f672617e700 1 -- 172.21.15.22:0/1956772877 wait complete. 2024-03-08T23:44:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.645+0000 7f672617e700 1 Processor -- start 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f672617e700 1 -- start start 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f672617e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f672019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f672617e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672010bcf0 con 0x7f6720107790 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f672019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f672019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51212/0 (socket says 172.21.15.22:51212) 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f671f7fe700 1 -- 172.21.15.22:0/880598254 learned_addr learned my addr 172.21.15.22:0/880598254 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f671f7fe700 1 -- 172.21.15.22:0/880598254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672019bc50 con 0x7f6720107790 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f671f7fe700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f672019b710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6710000c00 tx=0x7f6710011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f671cff9700 1 -- 172.21.15.22:0/880598254 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6710003b60 con 0x7f6720107790 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f671cff9700 1 -- 172.21.15.22:0/880598254 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f671001a910 con 0x7f6720107790 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.646+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672019bee0 con 0x7f6720107790 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.647+0000 7f671cff9700 1 -- 172.21.15.22:0/880598254 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6710024930 con 0x7f6720107790 2024-03-08T23:44:38.539 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.647+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672019c340 con 0x7f6720107790 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.648+0000 7f671cff9700 1 -- 172.21.15.22:0/880598254 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f6710022020 con 0x7f6720107790 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.648+0000 7f671cff9700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 0x7f670803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.648+0000 7f671cff9700 1 -- 172.21.15.22:0/880598254 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f671004d0d0 con 0x7f6720107790 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.648+0000 7f671effd700 1 -- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 msgr2=0x7f670803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4028157579 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.648+0000 7f671effd700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 0x7f670803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.648+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6720068eb0 con 0x7f6720107790 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.653+0000 7f671cff9700 1 -- 172.21.15.22:0/880598254 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f67100172b0 con 0x7f6720107790 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.849+0000 7f671effd700 1 -- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 msgr2=0x7f670803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4028157579 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.849+0000 7f671effd700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 0x7f670803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.915+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f672019f000 con 0x7f6720107790 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.916+0000 7f671cff9700 1 -- 172.21.15.22:0/880598254 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f671004ca20 con 0x7f6720107790 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 msgr2=0x7f670803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:44:38.540 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 0x7f670803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 msgr2=0x7f672019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f672019b710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6710000c00 tx=0x7f6710011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 shutdown_connections 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f6708039020 0x7f670803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 --2- 172.21.15.22:0/880598254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6720107790 0x7f672019b710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 >> 172.21.15.22:0/880598254 conn(0x7f6720102d70 msgr2=0x7f67201038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 shutdown_connections 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:37.918+0000 7f672617e700 1 -- 172.21.15.22:0/880598254 wait complete. 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-03-08T23:44:38.541 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2024-03-08T23:44:38.815 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/880598254' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-08T23:44:38.816 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.702+0000 7f77333df200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:44:38.816 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.817+0000 7f77333df200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:44:39.127 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:38.910+0000 7f77333df200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:44:39.127 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:39.008+0000 7f77333df200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:44:39.487 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:39.187+0000 7f77333df200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:44:40.323 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:40.187+0000 7f77333df200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:44:40.324 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:40.325+0000 7f77333df200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:44:40.964 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:40.670+0000 7f77333df200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:44:40.964 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:40.764+0000 7f77333df200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:44:40.965 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:40.966+0000 7f77333df200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:44:41.237 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:41.158+0000 7f77333df200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:44:41.709 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:41.449+0000 7f77333df200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:44:41.709 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:41.710+0000 7f77333df200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:44:41.987 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:41.955+0000 7f77333df200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:44:43.237 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:42.785+0000 7f77333df200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:44:43.691 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:43.368+0000 7f77333df200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:44:43.691 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:43.469+0000 7f77333df200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:44:43.691 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:44:43.692+0000 7f77333df200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:44:43.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: Active manager daemon x restarted 2024-03-08T23:44:43.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: Activating manager daemon x 2024-03-08T23:44:43.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: osdmap e3: 0 total, 0 up, 0 in 2024-03-08T23:44:43.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: mgrmap e10: x(active, starting, since 0.00309939s) 2024-03-08T23:44:43.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:44:43.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T23:44:43.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T23:44:43.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T23:44:43.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T23:44:43.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: Manager daemon x is now available 2024-03-08T23:44:43.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:43.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:44.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T23:44:44.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T23:44:44.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:44 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:44] ENGINE Bus STARTING 2024-03-08T23:44:44.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:44 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:44] ENGINE Serving on http://172.21.15.22:8765 2024-03-08T23:44:44.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:44 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:44] ENGINE Serving on https://172.21.15.22:7150 2024-03-08T23:44:44.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:44 smithi022 ceph-mon[31894]: [08/Mar/2024:23:44:44] ENGINE Bus STARTED 2024-03-08T23:44:44.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:44 smithi022 ceph-mon[31894]: mgrmap e11: x(active, since 1.00609s) 2024-03-08T23:44:45.343 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.923+0000 7f20c4781700 1 Processor -- start 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.923+0000 7f20c4781700 1 -- start start 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.923+0000 7f20c4781700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc108070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.923+0000 7f20c4781700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc108640 con 0x7f20bc107c50 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c251d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c251d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51224/0 (socket says 172.21.15.22:51224) 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c251d700 1 -- 172.21.15.22:0/2736725503 learned_addr learned my addr 172.21.15.22:0/2736725503 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c251d700 1 -- 172.21.15.22:0/2736725503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc1068b0 con 0x7f20bc107c50 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c251d700 1 --2- 172.21.15.22:0/2736725503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc108070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20b800bb40 tx=0x7f20b800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978ddec56803dc97 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c151b700 1 -- 172.21.15.22:0/2736725503 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20b800dbe0 con 0x7f20bc107c50 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c151b700 1 -- 172.21.15.22:0/2736725503 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20b8004510 con 0x7f20bc107c50 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.924+0000 7f20c151b700 1 -- 172.21.15.22:0/2736725503 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20b8010470 con 0x7f20bc107c50 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.925+0000 7f20c4781700 1 -- 172.21.15.22:0/2736725503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 msgr2=0x7f20bc108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.925+0000 7f20c4781700 1 --2- 172.21.15.22:0/2736725503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc108070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20b800bb40 tx=0x7f20b800be50 comp rx=0 tx=0).stop 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.925+0000 7f20c4781700 1 -- 172.21.15.22:0/2736725503 shutdown_connections 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.925+0000 7f20c4781700 1 --2- 172.21.15.22:0/2736725503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc108070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.925+0000 7f20c4781700 1 -- 172.21.15.22:0/2736725503 >> 172.21.15.22:0/2736725503 conn(0x7f20bc101420 msgr2=0x7f20bc103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.925+0000 7f20c4781700 1 -- 172.21.15.22:0/2736725503 shutdown_connections 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.925+0000 7f20c4781700 1 -- 172.21.15.22:0/2736725503 wait complete. 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c4781700 1 Processor -- start 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c4781700 1 -- start start 2024-03-08T23:44:45.345 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c4781700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c4781700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc1057a0 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c251d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c251d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51240/0 (socket says 172.21.15.22:51240) 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c251d700 1 -- 172.21.15.22:0/304987420 learned_addr learned my addr 172.21.15.22:0/304987420 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c251d700 1 -- 172.21.15.22:0/304987420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc199aa0 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20c251d700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc199560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f20b8003bb0 tx=0x7f20b8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20b8010680 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.926+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20b801d070 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.927+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20bc199d30 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.927+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20b8021bb0 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.927+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20bc19ca20 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.928+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f20b801f030 con 0x7f20bc107c50 2024-03-08T23:44:45.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.928+0000 7f20b37fe700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.928+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 --> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20a803ff70 con 0x7f20a803d3a0 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.928+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f20b804d200 con 0x7f20bc107c50 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.928+0000 7f20c1d1c700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 msgr2=0x7f20a803f860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4028157579 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:38.928+0000 7f20c1d1c700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:39.129+0000 7f20c1d1c700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 msgr2=0x7f20a803f860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4028157579 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:39.129+0000 7f20c1d1c700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:39.530+0000 7f20c1d1c700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 msgr2=0x7f20a803f860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4028157579 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:39.530+0000 7f20c1d1c700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:40.331+0000 7f20c1d1c700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 msgr2=0x7f20a803f860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4028157579 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:40.331+0000 7f20c1d1c700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:41.932+0000 7f20c1d1c700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 msgr2=0x7f20a803f860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/4028157579 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:41.932+0000 7f20c1d1c700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-08T23:44:45.347 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:43.708+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f20b8019720 con 0x7f20bc107c50 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:43.708+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 msgr2=0x7f20a803f860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:43.708+0000 7f20b37fe700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.709+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f20b8018c70 con 0x7f20bc107c50 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.709+0000 7f20b37fe700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20a8040fc0 0x7f20a80433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.709+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20a803ff70 con 0x7f20a8040fc0 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.709+0000 7f20c1d1c700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20a8040fc0 0x7f20a80433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.710+0000 7f20c1d1c700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20a8040fc0 0x7f20a80433b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20ac003a10 tx=0x7f20ac009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.710+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f20a803ff70 con 0x7f20a8040fc0 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.717+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f20bc068eb0 con 0x7f20a8040fc0 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20b37fe700 1 -- 172.21.15.22:0/304987420 <== mgr.14152 v2:172.21.15.22:6800/2813827512 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f20bc068eb0 con 0x7f20a8040fc0 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20a8040fc0 msgr2=0x7f20a80433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20a8040fc0 0x7f20a80433b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20ac003a10 tx=0x7f20ac009270 comp rx=0 tx=0).stop 2024-03-08T23:44:45.348 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 msgr2=0x7f20bc199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc199560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f20b8003bb0 tx=0x7f20b8004290 comp rx=0 tx=0).stop 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 shutdown_connections 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20a8040fc0 0x7f20a80433b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20bc107c50 0x7f20bc199560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 --2- 172.21.15.22:0/304987420 >> [v2:172.21.15.22:6800/4028157579,v1:172.21.15.22:6801/4028157579] conn(0x7f20a803d3a0 0x7f20a803f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 >> 172.21.15.22:0/304987420 conn(0x7f20bc101420 msgr2=0x7f20bc18fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 shutdown_connections 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:44.718+0000 7f20c4781700 1 -- 172.21.15.22:0/304987420 wait complete. 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2024-03-08T23:44:45.349 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2024-03-08T23:44:46.098 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:45 smithi022 ceph-mon[31894]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-08T23:44:46.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:45 smithi022 ceph-mon[31894]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-08T23:44:46.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:46.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:46 smithi022 ceph-mon[31894]: mgrmap e12: x(active, since 2s) 2024-03-08T23:44:46.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:46 smithi022 ceph-mon[31894]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:46.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:46 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:46.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:46 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:47.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-08T23:44:47.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.100+0000 7f1e09770700 1 Processor -- start 2024-03-08T23:44:47.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.100+0000 7f1e09770700 1 -- start start 2024-03-08T23:44:47.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.100+0000 7f1e09770700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e0410a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:47.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.100+0000 7f1e09770700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0407aed0 con 0x7f1e041081f0 2024-03-08T23:44:47.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.101+0000 7f1e02ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e0410a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.101+0000 7f1e02ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e0410a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50620/0 (socket says 172.21.15.22:50620) 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.101+0000 7f1e02ffd700 1 -- 172.21.15.22:0/1882792054 learned_addr learned my addr 172.21.15.22:0/1882792054 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.101+0000 7f1e02ffd700 1 -- 172.21.15.22:0/1882792054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0410ab20 con 0x7f1e041081f0 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.101+0000 7f1e02ffd700 1 --2- 172.21.15.22:0/1882792054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e0410a5e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009e80 tx=0x7f1df400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a74bda942cea76a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.101+0000 7f1e01ffb700 1 -- 172.21.15.22:0/1882792054 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df4017070 con 0x7f1e041081f0 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.101+0000 7f1e01ffb700 1 -- 172.21.15.22:0/1882792054 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df400b8c0 con 0x7f1e041081f0 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e01ffb700 1 -- 172.21.15.22:0/1882792054 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df40045c0 con 0x7f1e041081f0 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e09770700 1 -- 172.21.15.22:0/1882792054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 msgr2=0x7f1e0410a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e09770700 1 --2- 172.21.15.22:0/1882792054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e0410a5e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009e80 tx=0x7f1df400b330 comp rx=0 tx=0).stop 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e09770700 1 -- 172.21.15.22:0/1882792054 shutdown_connections 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e09770700 1 --2- 172.21.15.22:0/1882792054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e0410a5e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:47.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e09770700 1 -- 172.21.15.22:0/1882792054 >> 172.21.15.22:0/1882792054 conn(0x7f1e04101cc0 msgr2=0x7f1e041040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e09770700 1 -- 172.21.15.22:0/1882792054 shutdown_connections 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.102+0000 7f1e09770700 1 -- 172.21.15.22:0/1882792054 wait complete. 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.103+0000 7f1e09770700 1 Processor -- start 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.103+0000 7f1e09770700 1 -- start start 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.103+0000 7f1e09770700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e04107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.103+0000 7f1e09770700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0410bb90 con 0x7f1e041081f0 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.103+0000 7f1e02ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e04107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.103+0000 7f1e02ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e04107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50626/0 (socket says 172.21.15.22:50626) 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.103+0000 7f1e02ffd700 1 -- 172.21.15.22:0/2791044016 learned_addr learned my addr 172.21.15.22:0/2791044016 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.104+0000 7f1e02ffd700 1 -- 172.21.15.22:0/2791044016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04107fb0 con 0x7f1e041081f0 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.104+0000 7f1e02ffd700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e04107a70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1df4004350 tx=0x7f1df400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.104+0000 7f1dfbfff700 1 -- 172.21.15.22:0/2791044016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df4020020 con 0x7f1e041081f0 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.104+0000 7f1dfbfff700 1 -- 172.21.15.22:0/2791044016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df4003d90 con 0x7f1e041081f0 2024-03-08T23:44:47.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.104+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e041061a0 con 0x7f1e041081f0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.104+0000 7f1dfbfff700 1 -- 172.21.15.22:0/2791044016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df4017040 con 0x7f1e041081f0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.104+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e041066e0 con 0x7f1e041081f0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.105+0000 7f1dfbfff700 1 -- 172.21.15.22:0/2791044016 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f1df4023020 con 0x7f1e041081f0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.105+0000 7f1dfbfff700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df0038f80 0x7f1df003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.105+0000 7f1dfbfff700 1 -- 172.21.15.22:0/2791044016 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1df400f050 con 0x7f1e041081f0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.105+0000 7f1e027fc700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df0038f80 0x7f1df003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.105+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04068eb0 con 0x7f1e041081f0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.106+0000 7f1e027fc700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df0038f80 0x7f1df003b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006fd0 tx=0x7f1dec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.110+0000 7f1dfbfff700 1 -- 172.21.15.22:0/2791044016 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1df402a020 con 0x7f1e041081f0 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.326+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e04106d70 con 0x7f1df0038f80 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.556+0000 7f1dfbfff700 1 -- 172.21.15.22:0/2791044016 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f1e04106d70 con 0x7f1df0038f80 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df0038f80 msgr2=0x7f1df003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:47.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df0038f80 0x7f1df003b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006fd0 tx=0x7f1dec006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 msgr2=0x7f1e04107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e04107a70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1df4004350 tx=0x7f1df400bd50 comp rx=0 tx=0).stop 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 shutdown_connections 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df0038f80 0x7f1df003b440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 --2- 172.21.15.22:0/2791044016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e041081f0 0x7f1e04107a70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 >> 172.21.15.22:0/2791044016 conn(0x7f1e04101cc0 msgr2=0x7f1e041027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 shutdown_connections 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:46.558+0000 7f1e09770700 1 -- 172.21.15.22:0/2791044016 wait complete. 2024-03-08T23:44:47.154 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2024-03-08T23:44:48.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$t4JbSg07DVLe01.RBLPq8OomskmVcCY0SIGvtxdvah5/TBwNxTBTG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709941488, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-08T23:44:48.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.572+0000 7f9a6fdd0700 1 Processor -- start 2024-03-08T23:44:48.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.572+0000 7f9a6fdd0700 1 -- start start 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.572+0000 7f9a6fdd0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6810a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.572+0000 7f9a6fdd0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6807aed0 con 0x7f9a681081f0 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.572+0000 7f9a6db6c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6810a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.573+0000 7f9a6db6c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6810a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50630/0 (socket says 172.21.15.22:50630) 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.573+0000 7f9a6db6c700 1 -- 172.21.15.22:0/3558902014 learned_addr learned my addr 172.21.15.22:0/3558902014 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.573+0000 7f9a6db6c700 1 -- 172.21.15.22:0/3558902014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6810ab20 con 0x7f9a681081f0 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.573+0000 7f9a6db6c700 1 --2- 172.21.15.22:0/3558902014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6810a5e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a600098a0 tx=0x7f9a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6c979954fba4d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.573+0000 7f9a6cb6a700 1 -- 172.21.15.22:0/3558902014 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a60004160 con 0x7f9a681081f0 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.573+0000 7f9a6cb6a700 1 -- 172.21.15.22:0/3558902014 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a60004300 con 0x7f9a681081f0 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.573+0000 7f9a6cb6a700 1 -- 172.21.15.22:0/3558902014 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a60011b20 con 0x7f9a681081f0 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/3558902014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 msgr2=0x7f9a6810a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 --2- 172.21.15.22:0/3558902014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6810a5e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a600098a0 tx=0x7f9a60009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:48.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/3558902014 shutdown_connections 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 --2- 172.21.15.22:0/3558902014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6810a5e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/3558902014 >> 172.21.15.22:0/3558902014 conn(0x7f9a68101cc0 msgr2=0x7f9a681040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/3558902014 shutdown_connections 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/3558902014 wait complete. 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 Processor -- start 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.574+0000 7f9a6fdd0700 1 -- start start 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6fdd0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6fdd0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6810bb90 con 0x7f9a681081f0 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6db6c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6db6c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50638/0 (socket says 172.21.15.22:50638) 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6db6c700 1 -- 172.21.15.22:0/1465027713 learned_addr learned my addr 172.21.15.22:0/1465027713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6db6c700 1 -- 172.21.15.22:0/1465027713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6819bbf0 con 0x7f9a681081f0 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6db6c700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6819b6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a600116a0 tx=0x7f9a60011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:48.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a66ffd700 1 -- 172.21.15.22:0/1465027713 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a60003a40 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6819be80 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a66ffd700 1 -- 172.21.15.22:0/1465027713 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a60003be0 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a66ffd700 1 -- 172.21.15.22:0/1465027713 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a6001a750 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.575+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6819c2e0 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.576+0000 7f9a66ffd700 1 -- 172.21.15.22:0/1465027713 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f9a60024430 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.577+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a68068eb0 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.577+0000 7f9a66ffd700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9a50038f20 0x7f9a5003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.577+0000 7f9a66ffd700 1 -- 172.21.15.22:0/1465027713 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a6004d660 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.577+0000 7f9a6d36b700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9a50038f20 0x7f9a5003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.577+0000 7f9a6d36b700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9a50038f20 0x7f9a5003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c006fd0 tx=0x7f9a5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.582+0000 7f9a66ffd700 1 -- 172.21.15.22:0/1465027713 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9a60011e20 con 0x7f9a681081f0 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:47.813+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9a6819ed10 con 0x7f9a50038f20 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.061+0000 7f9a66ffd700 1 -- 172.21.15.22:0/1465027713 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f9a6819ed10 con 0x7f9a50038f20 2024-03-08T23:44:48.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9a50038f20 msgr2=0x7f9a5003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9a50038f20 0x7f9a5003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c006fd0 tx=0x7f9a5c006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 msgr2=0x7f9a6819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6819b6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a600116a0 tx=0x7f9a60011ba0 comp rx=0 tx=0).stop 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 shutdown_connections 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9a50038f20 0x7f9a5003b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 --2- 172.21.15.22:0/1465027713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a681081f0 0x7f9a6819b6b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 >> 172.21.15.22:0/1465027713 conn(0x7f9a68101cc0 msgr2=0x7f9a68102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 shutdown_connections 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:48.063+0000 7f9a6fdd0700 1 -- 172.21.15.22:0/1465027713 wait complete. 2024-03-08T23:44:48.609 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2024-03-08T23:44:48.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:48 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:48.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:48 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:49.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:49 smithi022 ceph-mon[31894]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:44:49.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:49 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/514090270' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-08T23:44:49.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2024-03-08T23:44:49.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.094+0000 7fb6cb6d0700 1 Processor -- start 2024-03-08T23:44:49.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.094+0000 7fb6cb6d0700 1 -- start start 2024-03-08T23:44:49.971 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6cb6d0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c4106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6cb6d0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c4107080 con 0x7fb6c4106690 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6c946c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c4106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6c946c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c4106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50654/0 (socket says 172.21.15.22:50654) 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6c946c700 1 -- 172.21.15.22:0/1910071175 learned_addr learned my addr 172.21.15.22:0/1910071175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6c946c700 1 -- 172.21.15.22:0/1910071175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c4107890 con 0x7fb6c4106690 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6c946c700 1 --2- 172.21.15.22:0/1910071175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c4106ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098a0 tx=0x7fb6c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787ce1f83bbd3f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6bbfff700 1 -- 172.21.15.22:0/1910071175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0004160 con 0x7fb6c4106690 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6bbfff700 1 -- 172.21.15.22:0/1910071175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6c0004300 con 0x7fb6c4106690 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.095+0000 7fb6bbfff700 1 -- 172.21.15.22:0/1910071175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0011b20 con 0x7fb6c4106690 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.096+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/1910071175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 msgr2=0x7fb6c4106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.096+0000 7fb6cb6d0700 1 --2- 172.21.15.22:0/1910071175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c4106ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098a0 tx=0x7fb6c0009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.096+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/1910071175 shutdown_connections 2024-03-08T23:44:49.972 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.096+0000 7fb6cb6d0700 1 --2- 172.21.15.22:0/1910071175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c4106ab0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.096+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/1910071175 >> 172.21.15.22:0/1910071175 conn(0x7fb6c4101c70 msgr2=0x7fb6c4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.096+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/1910071175 shutdown_connections 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.096+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/1910071175 wait complete. 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6cb6d0700 1 Processor -- start 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6cb6d0700 1 -- start start 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6cb6d0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6cb6d0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c410ac40 con 0x7fb6c4106690 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6c946c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6c946c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50666/0 (socket says 172.21.15.22:50666) 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6c946c700 1 -- 172.21.15.22:0/514090270 learned_addr learned my addr 172.21.15.22:0/514090270 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6c946c700 1 -- 172.21.15.22:0/514090270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c419bc70 con 0x7fb6c4106690 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6c946c700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c419b730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00117e0 tx=0x7fb6c0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.097+0000 7fb6ba7fc700 1 -- 172.21.15.22:0/514090270 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0003bf0 con 0x7fb6c4106690 2024-03-08T23:44:49.973 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.098+0000 7fb6ba7fc700 1 -- 172.21.15.22:0/514090270 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6c001a910 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.098+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c419bf00 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.098+0000 7fb6ba7fc700 1 -- 172.21.15.22:0/514090270 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0024970 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.098+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c419c360 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.098+0000 7fb6ba7fc700 1 -- 172.21.15.22:0/514090270 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fb6c0022020 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.098+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c4063410 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.099+0000 7fb6ba7fc700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6b0038f70 0x7fb6b003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.099+0000 7fb6ba7fc700 1 -- 172.21.15.22:0/514090270 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb6c004d900 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.099+0000 7fb6c8c6b700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6b0038f70 0x7fb6b003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.099+0000 7fb6c8c6b700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6b0038f70 0x7fb6b003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4006fd0 tx=0x7fb6b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.104+0000 7fb6ba7fc700 1 -- 172.21.15.22:0/514090270 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb6c001a430 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.319+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb6c40629f0 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.320+0000 7fb6ba7fc700 1 -- 172.21.15.22:0/514090270 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb6c002cc80 con 0x7fb6c4106690 2024-03-08T23:44:49.974 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6b0038f70 msgr2=0x7fb6b003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6b0038f70 0x7fb6b003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4006fd0 tx=0x7fb6b4006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 msgr2=0x7fb6c419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c419b730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00117e0 tx=0x7fb6c0011ba0 comp rx=0 tx=0).stop 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 shutdown_connections 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6b0038f70 0x7fb6b003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 --2- 172.21.15.22:0/514090270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6c4106690 0x7fb6c419b730 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 >> 172.21.15.22:0/514090270 conn(0x7fb6c4101c70 msgr2=0x7fb6c4102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 shutdown_connections 2024-03-08T23:44:49.975 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:49.322+0000 7fb6cb6d0700 1 -- 172.21.15.22:0/514090270 wait complete. 2024-03-08T23:44:49.978 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T23:44:49.978 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-03-08T23:44:49.985 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T23:44:49.985 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-03-08T23:44:49.985 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-03-08T23:44:49.985 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-08T23:44:49.987 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2024-03-08T23:44:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2024-03-08T23:44:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2024-03-08T23:44:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: Password: dzans3vvne 2024-03-08T23:44:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:49.988 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config directory 2024-03-08T23:44:49.988 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling autotune for osd_memory_target 2024-03-08T23:44:51.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:51 smithi022 ceph-mon[31894]: mgrmap e13: x(active, since 6s) 2024-03-08T23:44:52.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.288+0000 7f1cc3e6a700 1 Processor -- start 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.288+0000 7f1cc3e6a700 1 -- start start 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.288+0000 7f1cc3e6a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.288+0000 7f1cc3e6a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc1081a0 con 0x7f1cbc1077b0 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc1c06700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc1c06700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc107bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53458/0 (socket says 172.21.15.22:53458) 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc1c06700 1 -- 172.21.15.22:0/1729076108 learned_addr learned my addr 172.21.15.22:0/1729076108 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc1c06700 1 -- 172.21.15.22:0/1729076108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc108960 con 0x7f1cbc1077b0 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc1c06700 1 --2- 172.21.15.22:0/1729076108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc107bd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009e80 tx=0x7f1cac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaedc7ef9637a15 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc0c04700 1 -- 172.21.15.22:0/1729076108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cac017070 con 0x7f1cbc1077b0 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc0c04700 1 -- 172.21.15.22:0/1729076108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cac00b8c0 con 0x7f1cbc1077b0 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.289+0000 7f1cc0c04700 1 -- 172.21.15.22:0/1729076108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cac0045c0 con 0x7f1cbc1077b0 2024-03-08T23:44:52.250 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.290+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/1729076108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 msgr2=0x7f1cbc107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.290+0000 7f1cc3e6a700 1 --2- 172.21.15.22:0/1729076108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc107bd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009e80 tx=0x7f1cac00b330 comp rx=0 tx=0).stop 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.290+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/1729076108 shutdown_connections 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.290+0000 7f1cc3e6a700 1 --2- 172.21.15.22:0/1729076108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc107bd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.290+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/1729076108 >> 172.21.15.22:0/1729076108 conn(0x7f1cbc102d70 msgr2=0x7f1cbc105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.290+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/1729076108 shutdown_connections 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.290+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/1729076108 wait complete. 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc3e6a700 1 Processor -- start 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc3e6a700 1 -- start start 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc3e6a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc3e6a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc10bd10 con 0x7f1cbc1077b0 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc1c06700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc1c06700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53462/0 (socket says 172.21.15.22:53462) 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc1c06700 1 -- 172.21.15.22:0/277070174 learned_addr learned my addr 172.21.15.22:0/277070174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:52.251 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.291+0000 7f1cc1c06700 1 -- 172.21.15.22:0/277070174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc07d0d0 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.292+0000 7f1cc1c06700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc07cb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00bca0 tx=0x7f1cac00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.292+0000 7f1cbaffd700 1 -- 172.21.15.22:0/277070174 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cac020020 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.292+0000 7f1cbaffd700 1 -- 172.21.15.22:0/277070174 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cac003d90 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.292+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc07b2c0 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.292+0000 7f1cbaffd700 1 -- 172.21.15.22:0/277070174 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cac017040 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.292+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc07b800 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.293+0000 7f1cbaffd700 1 -- 172.21.15.22:0/277070174 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f1cac023020 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.293+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc063410 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.293+0000 7f1cbaffd700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1ca4038fc0 0x7f1ca403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.293+0000 7f1cbaffd700 1 -- 172.21.15.22:0/277070174 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1cac00f050 con 0x7f1cbc1077b0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.293+0000 7f1cc1405700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1ca4038fc0 0x7f1ca403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.293+0000 7f1cc1405700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1ca4038fc0 0x7f1ca403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:52.252 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.298+0000 7f1cbaffd700 1 -- 172.21.15.22:0/277070174 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1cac02a020 con 0x7f1cbc1077b0 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.499+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f1cbc07c120 con 0x7f1cbc1077b0 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.500+0000 7f1cbaffd700 1 -- 172.21.15.22:0/277070174 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f1cac02d350 con 0x7f1cbc1077b0 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1ca4038fc0 msgr2=0x7f1ca403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1ca4038fc0 0x7f1ca403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 msgr2=0x7f1cbc07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc07cb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00bca0 tx=0x7f1cac00bd80 comp rx=0 tx=0).stop 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 shutdown_connections 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1ca4038fc0 0x7f1ca403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 --2- 172.21.15.22:0/277070174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cbc1077b0 0x7f1cbc07cb90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 >> 172.21.15.22:0/277070174 conn(0x7f1cbc102d70 msgr2=0x7f1cbc103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 shutdown_connections 2024-03-08T23:44:52.253 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:51.502+0000 7f1cc3e6a700 1 -- 172.21.15.22:0/277070174 wait complete. 2024-03-08T23:44:53.506 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fe1808700 1 Processor -- start 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fe1808700 1 -- start start 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fe1808700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fe1808700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc108170 con 0x7f4fdc107780 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fdaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fdaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53470/0 (socket says 172.21.15.22:53470) 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fdaffd700 1 -- 172.21.15.22:0/3332438794 learned_addr learned my addr 172.21.15.22:0/3332438794 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fdaffd700 1 -- 172.21.15.22:0/3332438794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc108930 con 0x7f4fdc107780 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.663+0000 7f4fdaffd700 1 --2- 172.21.15.22:0/3332438794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc107ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0098a0 tx=0x7f4fcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b894aa016dfbfbc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fd9ffb700 1 -- 172.21.15.22:0/3332438794 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc004160 con 0x7f4fdc107780 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fd9ffb700 1 -- 172.21.15.22:0/3332438794 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fcc004300 con 0x7f4fdc107780 2024-03-08T23:44:53.507 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fd9ffb700 1 -- 172.21.15.22:0/3332438794 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc011b20 con 0x7f4fdc107780 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fe1808700 1 -- 172.21.15.22:0/3332438794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 msgr2=0x7f4fdc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fe1808700 1 --2- 172.21.15.22:0/3332438794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc107ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0098a0 tx=0x7f4fcc009bb0 comp rx=0 tx=0).stop 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fe1808700 1 -- 172.21.15.22:0/3332438794 shutdown_connections 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fe1808700 1 --2- 172.21.15.22:0/3332438794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc107ba0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fe1808700 1 -- 172.21.15.22:0/3332438794 >> 172.21.15.22:0/3332438794 conn(0x7f4fdc102da0 msgr2=0x7f4fdc1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fe1808700 1 -- 172.21.15.22:0/3332438794 shutdown_connections 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.664+0000 7f4fe1808700 1 -- 172.21.15.22:0/3332438794 wait complete. 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.665+0000 7f4fe1808700 1 Processor -- start 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.665+0000 7f4fe1808700 1 -- start start 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.665+0000 7f4fe1808700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.665+0000 7f4fe1808700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc10bce0 con 0x7f4fdc107780 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.665+0000 7f4fdaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:53.508 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.665+0000 7f4fdaffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53486/0 (socket says 172.21.15.22:53486) 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.665+0000 7f4fdaffd700 1 -- 172.21.15.22:0/2843138249 learned_addr learned my addr 172.21.15.22:0/2843138249 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.666+0000 7f4fdaffd700 1 -- 172.21.15.22:0/2843138249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc19bbe0 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.666+0000 7f4fdaffd700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc19b6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0117e0 tx=0x7f4fcc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.666+0000 7f4fe0806700 1 -- 172.21.15.22:0/2843138249 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc003bf0 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.666+0000 7f4fe0806700 1 -- 172.21.15.22:0/2843138249 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fcc01a910 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.666+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc19be70 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.666+0000 7f4fe0806700 1 -- 172.21.15.22:0/2843138249 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc0249a0 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.666+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc19c2d0 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.667+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc063410 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.667+0000 7f4fe0806700 1 -- 172.21.15.22:0/2843138249 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f4fcc022020 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.667+0000 7f4fe0806700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fc4039020 0x7f4fc403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.667+0000 7f4fe0806700 1 -- 172.21.15.22:0/2843138249 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4fcc04d320 con 0x7f4fdc107780 2024-03-08T23:44:53.509 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.667+0000 7f4fda7fc700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fc4039020 0x7f4fc403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.668+0000 7f4fda7fc700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fc4039020 0x7f4fc403b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0006fd0 tx=0x7f4fd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.672+0000 7f4fe0806700 1 -- 172.21.15.22:0/2843138249 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4fcc011e20 con 0x7f4fdc107780 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.986+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f4fdc19ed60 con 0x7f4fdc107780 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.988+0000 7f4fe0806700 1 -- 172.21.15.22:0/2843138249 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4fcc004300 con 0x7f4fdc107780 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fc4039020 msgr2=0x7f4fc403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fc4039020 0x7f4fc403b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0006fd0 tx=0x7f4fd0006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 msgr2=0x7f4fdc19b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc19b6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0117e0 tx=0x7f4fcc011c10 comp rx=0 tx=0).stop 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 shutdown_connections 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fc4039020 0x7f4fc403b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 --2- 172.21.15.22:0/2843138249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc107780 0x7f4fdc19b6a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 >> 172.21.15.22:0/2843138249 conn(0x7f4fdc102da0 msgr2=0x7f4fdc103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:53.510 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 shutdown_connections 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2024-03-08T23:44:52.991+0000 7f4fe1808700 1 -- 172.21.15.22:0/2843138249 wait complete. 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.511 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2024-03-08T23:44:53.512 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.512 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-08T23:44:53.512 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:44:53.512 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2024-03-08T23:44:53.543 INFO:tasks.cephadm:Fetching config... 2024-03-08T23:44:53.544 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:44:53.544 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-08T23:44:53.560 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-08T23:44:53.560 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:44:53.560 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-08T23:44:53.615 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-08T23:44:53.615 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:44:53.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-08T23:44:53.682 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-08T23:44:53.682 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:44:53.682 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-08T23:44:53.736 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-08T23:44:53.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC85usSZKjWMuxIhqOs3KZl0lHDRrFUJeLlIoSTeAwuJY1cw+YeRwl023yTbheHJT3OibrxirzVYjLiQUByPjfr8/FFYBwLpl5+SupJ6u5Hw8dr2DE5V93T/MG55eIYx4MWQaisZMNxzapnscnvd5njzx28KZnX+IYDFGPEa5DSoq0axRd2nRLEr6pKmGLYVHxtcnGu5QkbiTRMt6RGY41GSgrSJQOhwbPlJTDzeuKvyoKKi7TFIrGgb78wtKfd197nd1JQmDRkr/oIYJDmvAdg6oCPxfTjIhOBbDygGj4poO4cMp3LOKjB2BoxqA/KU77qT/4dAErLAbj6pEtZPVGYb1M9UBaiQdUepcoUCmn0IlJOlm3yhjFmEp8kc3Q4GLZFhSVA66gwu46TeNSf8QpqPU+ImeiOyrlzNaR79HH3Dznxjs13RmCRrM9D4gbvPqkmSNaFvqLu0R6Nxa1kCtxZ1R0FeLCzKhZWQ8v2aXWbhyE+bhYf7CtnBltBvod4I9s= ceph-8c5a3016-dda5-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-08T23:44:53.816 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC85usSZKjWMuxIhqOs3KZl0lHDRrFUJeLlIoSTeAwuJY1cw+YeRwl023yTbheHJT3OibrxirzVYjLiQUByPjfr8/FFYBwLpl5+SupJ6u5Hw8dr2DE5V93T/MG55eIYx4MWQaisZMNxzapnscnvd5njzx28KZnX+IYDFGPEa5DSoq0axRd2nRLEr6pKmGLYVHxtcnGu5QkbiTRMt6RGY41GSgrSJQOhwbPlJTDzeuKvyoKKi7TFIrGgb78wtKfd197nd1JQmDRkr/oIYJDmvAdg6oCPxfTjIhOBbDygGj4poO4cMp3LOKjB2BoxqA/KU77qT/4dAErLAbj6pEtZPVGYb1M9UBaiQdUepcoUCmn0IlJOlm3yhjFmEp8kc3Q4GLZFhSVA66gwu46TeNSf8QpqPU+ImeiOyrlzNaR79HH3Dznxjs13RmCRrM9D4gbvPqkmSNaFvqLu0R6Nxa1kCtxZ1R0FeLCzKhZWQ8v2aXWbhyE+bhYf7CtnBltBvod4I9s= ceph-8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:44:53.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC85usSZKjWMuxIhqOs3KZl0lHDRrFUJeLlIoSTeAwuJY1cw+YeRwl023yTbheHJT3OibrxirzVYjLiQUByPjfr8/FFYBwLpl5+SupJ6u5Hw8dr2DE5V93T/MG55eIYx4MWQaisZMNxzapnscnvd5njzx28KZnX+IYDFGPEa5DSoq0axRd2nRLEr6pKmGLYVHxtcnGu5QkbiTRMt6RGY41GSgrSJQOhwbPlJTDzeuKvyoKKi7TFIrGgb78wtKfd197nd1JQmDRkr/oIYJDmvAdg6oCPxfTjIhOBbDygGj4poO4cMp3LOKjB2BoxqA/KU77qT/4dAErLAbj6pEtZPVGYb1M9UBaiQdUepcoUCmn0IlJOlm3yhjFmEp8kc3Q4GLZFhSVA66gwu46TeNSf8QpqPU+ImeiOyrlzNaR79HH3Dznxjs13RmCRrM9D4gbvPqkmSNaFvqLu0R6Nxa1kCtxZ1R0FeLCzKhZWQ8v2aXWbhyE+bhYf7CtnBltBvod4I9s= ceph-8c5a3016-dda5-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-08T23:44:53.884 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC85usSZKjWMuxIhqOs3KZl0lHDRrFUJeLlIoSTeAwuJY1cw+YeRwl023yTbheHJT3OibrxirzVYjLiQUByPjfr8/FFYBwLpl5+SupJ6u5Hw8dr2DE5V93T/MG55eIYx4MWQaisZMNxzapnscnvd5njzx28KZnX+IYDFGPEa5DSoq0axRd2nRLEr6pKmGLYVHxtcnGu5QkbiTRMt6RGY41GSgrSJQOhwbPlJTDzeuKvyoKKi7TFIrGgb78wtKfd197nd1JQmDRkr/oIYJDmvAdg6oCPxfTjIhOBbDygGj4poO4cMp3LOKjB2BoxqA/KU77qT/4dAErLAbj6pEtZPVGYb1M9UBaiQdUepcoUCmn0IlJOlm3yhjFmEp8kc3Q4GLZFhSVA66gwu46TeNSf8QpqPU+ImeiOyrlzNaR79HH3Dznxjs13RmCRrM9D4gbvPqkmSNaFvqLu0R6Nxa1kCtxZ1R0FeLCzKhZWQ8v2aXWbhyE+bhYf7CtnBltBvod4I9s= ceph-8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:44:53.902 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC85usSZKjWMuxIhqOs3KZl0lHDRrFUJeLlIoSTeAwuJY1cw+YeRwl023yTbheHJT3OibrxirzVYjLiQUByPjfr8/FFYBwLpl5+SupJ6u5Hw8dr2DE5V93T/MG55eIYx4MWQaisZMNxzapnscnvd5njzx28KZnX+IYDFGPEa5DSoq0axRd2nRLEr6pKmGLYVHxtcnGu5QkbiTRMt6RGY41GSgrSJQOhwbPlJTDzeuKvyoKKi7TFIrGgb78wtKfd197nd1JQmDRkr/oIYJDmvAdg6oCPxfTjIhOBbDygGj4poO4cMp3LOKjB2BoxqA/KU77qT/4dAErLAbj6pEtZPVGYb1M9UBaiQdUepcoUCmn0IlJOlm3yhjFmEp8kc3Q4GLZFhSVA66gwu46TeNSf8QpqPU+ImeiOyrlzNaR79HH3Dznxjs13RmCRrM9D4gbvPqkmSNaFvqLu0R6Nxa1kCtxZ1R0FeLCzKhZWQ8v2aXWbhyE+bhYf7CtnBltBvod4I9s= ceph-8c5a3016-dda5-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-08T23:44:53.950 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC85usSZKjWMuxIhqOs3KZl0lHDRrFUJeLlIoSTeAwuJY1cw+YeRwl023yTbheHJT3OibrxirzVYjLiQUByPjfr8/FFYBwLpl5+SupJ6u5Hw8dr2DE5V93T/MG55eIYx4MWQaisZMNxzapnscnvd5njzx28KZnX+IYDFGPEa5DSoq0axRd2nRLEr6pKmGLYVHxtcnGu5QkbiTRMt6RGY41GSgrSJQOhwbPlJTDzeuKvyoKKi7TFIrGgb78wtKfd197nd1JQmDRkr/oIYJDmvAdg6oCPxfTjIhOBbDygGj4poO4cMp3LOKjB2BoxqA/KU77qT/4dAErLAbj6pEtZPVGYb1M9UBaiQdUepcoUCmn0IlJOlm3yhjFmEp8kc3Q4GLZFhSVA66gwu46TeNSf8QpqPU+ImeiOyrlzNaR79HH3Dznxjs13RmCRrM9D4gbvPqkmSNaFvqLu0R6Nxa1kCtxZ1R0FeLCzKhZWQ8v2aXWbhyE+bhYf7CtnBltBvod4I9s= ceph-8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:44:53.964 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-08T23:44:54.204 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:44:54.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:53 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2843138249' entity='client.admin' 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.411+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3626797940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 msgr2=0x7f48f0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.411+0000 7f48f5d7c700 1 --2- 172.21.15.22:0/3626797940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f0100b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f48e000b0a0 tx=0x7f48e000b3b0 comp rx=0 tx=0).stop 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.411+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3626797940 shutdown_connections 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.411+0000 7f48f5d7c700 1 --2- 172.21.15.22:0/3626797940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f0100b30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.411+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3626797940 >> 172.21.15.22:0/3626797940 conn(0x7f48f00fbcb0 msgr2=0x7f48f00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.411+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3626797940 shutdown_connections 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.411+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3626797940 wait complete. 2024-03-08T23:44:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48f5d7c700 1 Processor -- start 2024-03-08T23:44:56.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48f5d7c700 1 -- start start 2024-03-08T23:44:56.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48f5d7c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:56.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48f5d7c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f01067a0 con 0x7f48f0100710 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48ef7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48ef7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f006a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53498/0 (socket says 172.21.15.22:53498) 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48ef7fe700 1 -- 172.21.15.22:0/3700472579 learned_addr learned my addr 172.21.15.22:0/3700472579 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.412+0000 7f48ef7fe700 1 -- 172.21.15.22:0/3700472579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0199bf0 con 0x7f48f0100710 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.413+0000 7f48ef7fe700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f006a4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48e0007890 tx=0x7f48e0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.413+0000 7f48ecff9700 1 -- 172.21.15.22:0/3700472579 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48e0017040 con 0x7f48f0100710 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.413+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f0199dc0 con 0x7f48f0100710 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.413+0000 7f48ecff9700 1 -- 172.21.15.22:0/3700472579 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48e000bd50 con 0x7f48f0100710 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.413+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f019a160 con 0x7f48f0100710 2024-03-08T23:44:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.413+0000 7f48ecff9700 1 -- 172.21.15.22:0/3700472579 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48e001d7a0 con 0x7f48f0100710 2024-03-08T23:44:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.414+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f0106f20 con 0x7f48f0100710 2024-03-08T23:44:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.414+0000 7f48ecff9700 1 -- 172.21.15.22:0/3700472579 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f48e0024020 con 0x7f48f0100710 2024-03-08T23:44:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.414+0000 7f48ecff9700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48dc039070 0x7f48dc03b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.414+0000 7f48ecff9700 1 -- 172.21.15.22:0/3700472579 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f48e0022020 con 0x7f48f0100710 2024-03-08T23:44:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.414+0000 7f48eeffd700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48dc039070 0x7f48dc03b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.415+0000 7f48eeffd700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48dc039070 0x7f48dc03b530 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:56.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.419+0000 7f48ecff9700 1 -- 172.21.15.22:0/3700472579 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f48e002b020 con 0x7f48f0100710 2024-03-08T23:44:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.624+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f48f002da40 con 0x7f48f0100710 2024-03-08T23:44:56.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.626+0000 7f48ecff9700 1 -- 172.21.15.22:0/3700472579 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f48e0007e20 con 0x7f48f0100710 2024-03-08T23:44:56.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.628+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48dc039070 msgr2=0x7f48dc03b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:56.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.628+0000 7f48f5d7c700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48dc039070 0x7f48dc03b530 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.628+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 msgr2=0x7f48f006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.628+0000 7f48f5d7c700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f006a4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48e0007890 tx=0x7f48e0009790 comp rx=0 tx=0).stop 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.629+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 shutdown_connections 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.629+0000 7f48f5d7c700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48dc039070 0x7f48dc03b530 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.629+0000 7f48f5d7c700 1 --2- 172.21.15.22:0/3700472579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48f0100710 0x7f48f006a4b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.629+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 >> 172.21.15.22:0/3700472579 conn(0x7f48f00fbcb0 msgr2=0x7f48f00fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.629+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 shutdown_connections 2024-03-08T23:44:56.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:56.629+0000 7f48f5d7c700 1 -- 172.21.15.22:0/3700472579 wait complete. 2024-03-08T23:44:56.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:56.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:56.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:44:56.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:56.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:56.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:56.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3700472579' entity='client.admin' 2024-03-08T23:44:56.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:44:56.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:44:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:44:57.237 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-08T23:44:57.237 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-08T23:44:57.908 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.441+0000 7fd494c4d700 1 -- 172.21.15.22:0/923586877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 msgr2=0x7fd490101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.441+0000 7fd494c4d700 1 --2- 172.21.15.22:0/923586877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490101c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd480009910 tx=0x7fd480009c20 comp rx=0 tx=0).stop 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.442+0000 7fd494c4d700 1 -- 172.21.15.22:0/923586877 shutdown_connections 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.442+0000 7fd494c4d700 1 --2- 172.21.15.22:0/923586877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490101c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.442+0000 7fd494c4d700 1 -- 172.21.15.22:0/923586877 >> 172.21.15.22:0/923586877 conn(0x7fd4900fcdd0 msgr2=0x7fd4900ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.442+0000 7fd494c4d700 1 -- 172.21.15.22:0/923586877 shutdown_connections 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.442+0000 7fd494c4d700 1 -- 172.21.15.22:0/923586877 wait complete. 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.442+0000 7fd494c4d700 1 Processor -- start 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.442+0000 7fd494c4d700 1 -- start start 2024-03-08T23:44:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd494c4d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd494c4d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4901078c0 con 0x7fd490101830 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd48e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd48e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490075a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53504/0 (socket says 172.21.15.22:53504) 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd48e59c700 1 -- 172.21.15.22:0/1033910881 learned_addr learned my addr 172.21.15.22:0/1033910881 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd48e59c700 1 -- 172.21.15.22:0/1033910881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490075f90 con 0x7fd490101830 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd48e59c700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490075a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd480006010 tx=0x7fd480005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd47f7fe700 1 -- 172.21.15.22:0/1033910881 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd480027070 con 0x7fd490101830 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4900741d0 con 0x7fd490101830 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd47f7fe700 1 -- 172.21.15.22:0/1033910881 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd48000be40 con 0x7fd490101830 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd47f7fe700 1 -- 172.21.15.22:0/1033910881 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd48000f460 con 0x7fd490101830 2024-03-08T23:44:59.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.443+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4900746b0 con 0x7fd490101830 2024-03-08T23:44:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.444+0000 7fd47f7fe700 1 -- 172.21.15.22:0/1033910881 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fd48002a070 con 0x7fd490101830 2024-03-08T23:44:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.444+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd490108040 con 0x7fd490101830 2024-03-08T23:44:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.444+0000 7fd47f7fe700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd478039020 0x7fd47803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:44:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.444+0000 7fd47f7fe700 1 -- 172.21.15.22:0/1033910881 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd48001e030 con 0x7fd490101830 2024-03-08T23:44:59.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.444+0000 7fd48dd9b700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd478039020 0x7fd47803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:44:59.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.445+0000 7fd48dd9b700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd478039020 0x7fd47803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd484006fd0 tx=0x7fd484006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:44:59.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.449+0000 7fd47f7fe700 1 -- 172.21.15.22:0/1033910881 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd480013070 con 0x7fd490101830 2024-03-08T23:44:59.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.658+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4901902a0 con 0x7fd478039020 2024-03-08T23:44:59.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.661+0000 7fd47f7fe700 1 -- 172.21.15.22:0/1033910881 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd4901902a0 con 0x7fd478039020 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.663+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd478039020 msgr2=0x7fd47803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.663+0000 7fd494c4d700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd478039020 0x7fd47803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd484006fd0 tx=0x7fd484006e40 comp rx=0 tx=0).stop 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.663+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 msgr2=0x7fd490075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.663+0000 7fd494c4d700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490075a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd480006010 tx=0x7fd480005050 comp rx=0 tx=0).stop 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.664+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 shutdown_connections 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.664+0000 7fd494c4d700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd478039020 0x7fd47803b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.664+0000 7fd494c4d700 1 --2- 172.21.15.22:0/1033910881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd490101830 0x7fd490075a50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:44:59.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.664+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 >> 172.21.15.22:0/1033910881 conn(0x7fd4900fcdd0 msgr2=0x7fd4900fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:44:59.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.664+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 shutdown_connections 2024-03-08T23:44:59.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:44:59.664+0000 7fd494c4d700 1 -- 172.21.15.22:0/1033910881 wait complete. 2024-03-08T23:45:00.412 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi062 2024-03-08T23:45:00.412 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:45:00.412 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.conf 2024-03-08T23:45:00.432 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:45:00.432 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:00.489 INFO:tasks.cephadm:Adding host smithi062 to orchestrator... 2024-03-08T23:45:00.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch host add smithi062 2024-03-08T23:45:00.728 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:00.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:00.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:00.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:00.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:00.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:00.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-08T23:45:00.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:00.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:00.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:01.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:01 smithi022 ceph-mon[31894]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:45:01.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:01 smithi022 ceph-mon[31894]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:01.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:01 smithi022 ceph-mon[31894]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:45:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.284+0000 7f86f56c7700 1 -- 172.21.15.22:0/1624266402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 msgr2=0x7f86f0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.284+0000 7f86f56c7700 1 --2- 172.21.15.22:0/1624266402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0107ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009910 tx=0x7f86e0009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.284+0000 7f86f56c7700 1 -- 172.21.15.22:0/1624266402 shutdown_connections 2024-03-08T23:45:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.284+0000 7f86f56c7700 1 --2- 172.21.15.22:0/1624266402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0107ba0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.284+0000 7f86f56c7700 1 -- 172.21.15.22:0/1624266402 >> 172.21.15.22:0/1624266402 conn(0x7f86f00fd2f0 msgr2=0x7f86f00ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.284+0000 7f86f56c7700 1 -- 172.21.15.22:0/1624266402 shutdown_connections 2024-03-08T23:45:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.284+0000 7f86f56c7700 1 -- 172.21.15.22:0/1624266402 wait complete. 2024-03-08T23:45:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.285+0000 7f86f56c7700 1 Processor -- start 2024-03-08T23:45:02.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.285+0000 7f86f56c7700 1 -- start start 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.285+0000 7f86f56c7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.285+0000 7f86f56c7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f010be00 con 0x7f86f01077c0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86eeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86eeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51618/0 (socket says 172.21.15.22:51618) 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86eeffd700 1 -- 172.21.15.22:0/934805761 learned_addr learned my addr 172.21.15.22:0/934805761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86eeffd700 1 -- 172.21.15.22:0/934805761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0197840 con 0x7f86f01077c0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86eeffd700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0197300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86e0005850 tx=0x7f86e0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86d7fff700 1 -- 172.21.15.22:0/934805761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e0026070 con 0x7f86f01077c0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86d7fff700 1 -- 172.21.15.22:0/934805761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e000bbd0 con 0x7f86f01077c0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f0197b30 con 0x7f86f01077c0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86d7fff700 1 -- 172.21.15.22:0/934805761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e0003970 con 0x7f86f01077c0 2024-03-08T23:45:02.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.286+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f019bbd0 con 0x7f86f01077c0 2024-03-08T23:45:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.287+0000 7f86d7fff700 1 -- 172.21.15.22:0/934805761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f86e0029070 con 0x7f86f01077c0 2024-03-08T23:45:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.287+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f0102f00 con 0x7f86f01077c0 2024-03-08T23:45:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.287+0000 7f86d7fff700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f86d8038fc0 0x7f86d803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.287+0000 7f86d7fff700 1 -- 172.21.15.22:0/934805761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f86e001e070 con 0x7f86f01077c0 2024-03-08T23:45:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.287+0000 7f86ee7fc700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f86d8038fc0 0x7f86d803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:02.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.288+0000 7f86ee7fc700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f86d8038fc0 0x7f86d803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f86e4006fd0 tx=0x7f86e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:02.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.292+0000 7f86d7fff700 1 -- 172.21.15.22:0/934805761 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f86e0013070 con 0x7f86f01077c0 2024-03-08T23:45:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:02.505+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}) v1 -- 0x7f86f0198660 con 0x7f86d8038fc0 2024-03-08T23:45:03.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:02 smithi022 ceph-mon[31894]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:04.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:03 smithi022 ceph-mon[31894]: Deploying cephadm binary to smithi062 2024-03-08T23:45:05.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:04 smithi022 ceph-mon[31894]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:06.899 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi062' with addr '172.21.15.62' 2024-03-08T23:45:06.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.901+0000 7f86d7fff700 1 -- 172.21.15.22:0/934805761 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f86f0198660 con 0x7f86d8038fc0 2024-03-08T23:45:06.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f86d8038fc0 msgr2=0x7f86d803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f86d8038fc0 0x7f86d803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f86e4006fd0 tx=0x7f86e4006e40 comp rx=0 tx=0).stop 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 msgr2=0x7f86f0197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0197300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86e0005850 tx=0x7f86e0005030 comp rx=0 tx=0).stop 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 shutdown_connections 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f86d8038fc0 0x7f86d803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 --2- 172.21.15.22:0/934805761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86f01077c0 0x7f86f0197300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 >> 172.21.15.22:0/934805761 conn(0x7f86f00fd2f0 msgr2=0x7f86f0109330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 shutdown_connections 2024-03-08T23:45:06.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:06.904+0000 7f86f56c7700 1 -- 172.21.15.22:0/934805761 wait complete. 2024-03-08T23:45:07.038 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:06 smithi022 ceph-mon[31894]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:07.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-08T23:45:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:08.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:08.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:07 smithi022 ceph-mon[31894]: Added host smithi062 2024-03-08T23:45:08.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:08.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:09.140 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:08 smithi022 ceph-mon[31894]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:09.140 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:08 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:09.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 -- 172.21.15.22:0/2888158190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 msgr2=0x7fad38106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:09.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 --2- 172.21.15.22:0/2888158190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38106a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad28009910 tx=0x7fad28009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:09.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 -- 172.21.15.22:0/2888158190 shutdown_connections 2024-03-08T23:45:09.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 --2- 172.21.15.22:0/2888158190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38106a80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:09.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 -- 172.21.15.22:0/2888158190 >> 172.21.15.22:0/2888158190 conn(0x7fad380fc190 msgr2=0x7fad380fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:09.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 -- 172.21.15.22:0/2888158190 shutdown_connections 2024-03-08T23:45:09.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 -- 172.21.15.22:0/2888158190 wait complete. 2024-03-08T23:45:09.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 Processor -- start 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.188+0000 7fad3e8ce700 1 -- start start 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad3e8ce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad3e8ce700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3810ace0 con 0x7fad381066a0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51634/0 (socket says 172.21.15.22:51634) 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad37fff700 1 -- 172.21.15.22:0/941366703 learned_addr learned my addr 172.21.15.22:0/941366703 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad37fff700 1 -- 172.21.15.22:0/941366703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad38199a70 con 0x7fad381066a0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad37fff700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38199530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad28005850 tx=0x7fad28005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad357fa700 1 -- 172.21.15.22:0/941366703 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad28026070 con 0x7fad381066a0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad357fa700 1 -- 172.21.15.22:0/941366703 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad2800bbd0 con 0x7fad381066a0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad38199d60 con 0x7fad381066a0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad357fa700 1 -- 172.21.15.22:0/941366703 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad28003970 con 0x7fad381066a0 2024-03-08T23:45:09.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.189+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3819de00 con 0x7fad381066a0 2024-03-08T23:45:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.190+0000 7fad357fa700 1 -- 172.21.15.22:0/941366703 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fad28029070 con 0x7fad381066a0 2024-03-08T23:45:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.190+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad38101de0 con 0x7fad381066a0 2024-03-08T23:45:09.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.190+0000 7fad357fa700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fad20039020 0x7fad2003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:09.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.190+0000 7fad357fa700 1 -- 172.21.15.22:0/941366703 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad2801e070 con 0x7fad381066a0 2024-03-08T23:45:09.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.191+0000 7fad377fe700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fad20039020 0x7fad2003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:09.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.191+0000 7fad377fe700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fad20039020 0x7fad2003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad2c006fd0 tx=0x7fad2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.195+0000 7fad357fa700 1 -- 172.21.15.22:0/941366703 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fad28013070 con 0x7fad381066a0 2024-03-08T23:45:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.407+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad3819a890 con 0x7fad20039020 2024-03-08T23:45:09.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.408+0000 7fad357fa700 1 -- 172.21.15.22:0/941366703 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fad3819a890 con 0x7fad20039020 2024-03-08T23:45:09.406 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:45:09.406 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}] 2024-03-08T23:45:09.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fad20039020 msgr2=0x7fad2003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:09.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fad20039020 0x7fad2003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad2c006fd0 tx=0x7fad2c006e40 comp rx=0 tx=0).stop 2024-03-08T23:45:09.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 msgr2=0x7fad38199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38199530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad28005850 tx=0x7fad28005030 comp rx=0 tx=0).stop 2024-03-08T23:45:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 shutdown_connections 2024-03-08T23:45:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fad20039020 0x7fad2003b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 --2- 172.21.15.22:0/941366703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad381066a0 0x7fad38199530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 >> 172.21.15.22:0/941366703 conn(0x7fad380fc190 msgr2=0x7fad38069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 shutdown_connections 2024-03-08T23:45:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:09.410+0000 7fad3e8ce700 1 -- 172.21.15.22:0/941366703 wait complete. 2024-03-08T23:45:10.030 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-03-08T23:45:10.030 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:45:10.030 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-03-08T23:45:10.049 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:45:10.049 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:10.105 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-03-08T23:45:10.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch host add smithi192 2024-03-08T23:45:10.349 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:10.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:10 smithi022 ceph-mon[31894]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:45:10.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:10 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:11.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:11 smithi022 ceph-mon[31894]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.868+0000 7face48e0700 1 -- 172.21.15.22:0/394897260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 msgr2=0x7face0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.868+0000 7face48e0700 1 --2- 172.21.15.22:0/394897260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0107b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7facd000b0a0 tx=0x7facd000b3b0 comp rx=0 tx=0).stop 2024-03-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.868+0000 7face48e0700 1 -- 172.21.15.22:0/394897260 shutdown_connections 2024-03-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.868+0000 7face48e0700 1 --2- 172.21.15.22:0/394897260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0107b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.868+0000 7face48e0700 1 -- 172.21.15.22:0/394897260 >> 172.21.15.22:0/394897260 conn(0x7face00fd320 msgr2=0x7face00ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.868+0000 7face48e0700 1 -- 172.21.15.22:0/394897260 shutdown_connections 2024-03-08T23:45:11.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.868+0000 7face48e0700 1 -- 172.21.15.22:0/394897260 wait complete. 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7face48e0700 1 Processor -- start 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7face48e0700 1 -- start start 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7face48e0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7face48e0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face010bdf0 con 0x7face01077b0 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7facde59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7facde59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37678/0 (socket says 172.21.15.22:37678) 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7facde59c700 1 -- 172.21.15.22:0/1284644468 learned_addr learned my addr 172.21.15.22:0/1284644468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7facde59c700 1 -- 172.21.15.22:0/1284644468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0197770 con 0x7face01077b0 2024-03-08T23:45:11.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7facde59c700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0197230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facd00093e0 tx=0x7facd0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7facd77fe700 1 -- 172.21.15.22:0/1284644468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facd0017040 con 0x7face01077b0 2024-03-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7facd77fe700 1 -- 172.21.15.22:0/1284644468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facd0022070 con 0x7face01077b0 2024-03-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.869+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face0197a00 con 0x7face01077b0 2024-03-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.870+0000 7facd77fe700 1 -- 172.21.15.22:0/1284644468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facd001d440 con 0x7face01077b0 2024-03-08T23:45:11.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.870+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face019bb00 con 0x7face01077b0 2024-03-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.870+0000 7facd77fe700 1 -- 172.21.15.22:0/1284644468 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7facd0004000 con 0x7face01077b0 2024-03-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.871+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face0102ef0 con 0x7face01077b0 2024-03-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.871+0000 7facd77fe700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faccc039070 0x7faccc03b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.871+0000 7facd77fe700 1 -- 172.21.15.22:0/1284644468 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facd000f050 con 0x7face01077b0 2024-03-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.871+0000 7facddd9b700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faccc039070 0x7faccc03b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:11.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.871+0000 7facddd9b700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faccc039070 0x7faccc03b530 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7facc8006fd0 tx=0x7facc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:11.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:11.876+0000 7facd77fe700 1 -- 172.21.15.22:0/1284644468 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7facd0007e20 con 0x7face01077b0 2024-03-08T23:45:12.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:12.086+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7face0198590 con 0x7faccc039070 2024-03-08T23:45:13.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:13.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:13.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: Deploying cephadm binary to smithi192 2024-03-08T23:45:13.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:13.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:13.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:13.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:13.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:45:13.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:13.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:14.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:14 smithi022 ceph-mon[31894]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-08T23:45:14.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:14 smithi022 ceph-mon[31894]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:45:15.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:15 smithi022 ceph-mon[31894]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.328+0000 7facd77fe700 1 -- 172.21.15.22:0/1284644468 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7face0198590 con 0x7faccc039070 2024-03-08T23:45:16.326 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faccc039070 msgr2=0x7faccc03b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faccc039070 0x7faccc03b530 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7facc8006fd0 tx=0x7facc8006e40 comp rx=0 tx=0).stop 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 msgr2=0x7face0197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0197230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facd00093e0 tx=0x7facd0003ce0 comp rx=0 tx=0).stop 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 shutdown_connections 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faccc039070 0x7faccc03b530 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 --2- 172.21.15.22:0/1284644468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7face01077b0 0x7face0197230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 >> 172.21.15.22:0/1284644468 conn(0x7face00fd320 msgr2=0x7face01092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 shutdown_connections 2024-03-08T23:45:16.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:16.330+0000 7face48e0700 1 -- 172.21.15.22:0/1284644468 wait complete. 2024-03-08T23:45:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:16 smithi022 ceph-mon[31894]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:17.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-08T23:45:17.347 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:17.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:17 smithi022 ceph-mon[31894]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:17.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:17 smithi022 ceph-mon[31894]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:45:17.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:17 smithi022 ceph-mon[31894]: Added host smithi192 2024-03-08T23:45:17.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:17.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:18.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.982+0000 7fe9a16b1700 1 -- 172.21.15.22:0/2863592358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 msgr2=0x7fe99c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:18.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.982+0000 7fe9a16b1700 1 --2- 172.21.15.22:0/2863592358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c107ba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009910 tx=0x7fe98c009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:18.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.982+0000 7fe9a16b1700 1 -- 172.21.15.22:0/2863592358 shutdown_connections 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.982+0000 7fe9a16b1700 1 --2- 172.21.15.22:0/2863592358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c107ba0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.982+0000 7fe9a16b1700 1 -- 172.21.15.22:0/2863592358 >> 172.21.15.22:0/2863592358 conn(0x7fe99c0fd2f0 msgr2=0x7fe99c0ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.982+0000 7fe9a16b1700 1 -- 172.21.15.22:0/2863592358 shutdown_connections 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.982+0000 7fe9a16b1700 1 -- 172.21.15.22:0/2863592358 wait complete. 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.983+0000 7fe9a16b1700 1 Processor -- start 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.983+0000 7fe9a16b1700 1 -- start start 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.983+0000 7fe9a16b1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.983+0000 7fe9a16b1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c10be00 con 0x7fe99c1077c0 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.983+0000 7fe99affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.983+0000 7fe99affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c0740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37698/0 (socket says 172.21.15.22:37698) 2024-03-08T23:45:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.983+0000 7fe99affd700 1 -- 172.21.15.22:0/3553550545 learned_addr learned my addr 172.21.15.22:0/3553550545 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.984+0000 7fe99affd700 1 -- 172.21.15.22:0/3553550545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c075fd0 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.984+0000 7fe99affd700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c0740e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005850 tx=0x7fe98c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.984+0000 7fe983fff700 1 -- 172.21.15.22:0/3553550545 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c026070 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.984+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c074680 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.984+0000 7fe983fff700 1 -- 172.21.15.22:0/3553550545 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe98c00bc30 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.984+0000 7fe983fff700 1 -- 172.21.15.22:0/3553550545 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c003a20 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.984+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c074b40 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.985+0000 7fe983fff700 1 -- 172.21.15.22:0/3553550545 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fe98c029070 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.985+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c102f00 con 0x7fe99c1077c0 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.985+0000 7fe983fff700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe984038fc0 0x7fe98403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.985+0000 7fe983fff700 1 -- 172.21.15.22:0/3553550545 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe98c01e070 con 0x7fe99c1077c0 2024-03-08T23:45:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.985+0000 7fe99a7fc700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe984038fc0 0x7fe98403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.986+0000 7fe99a7fc700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe984038fc0 0x7fe98403b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe990006fd0 tx=0x7fe990006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:18.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:18.990+0000 7fe983fff700 1 -- 172.21.15.22:0/3553550545 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe98c013070 con 0x7fe99c1077c0 2024-03-08T23:45:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.197+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe99c075680 con 0x7fe984038fc0 2024-03-08T23:45:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.198+0000 7fe983fff700 1 -- 172.21.15.22:0/3553550545 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7fe99c075680 con 0x7fe984038fc0 2024-03-08T23:45:19.196 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:45:19.197 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-03-08T23:45:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe984038fc0 msgr2=0x7fe98403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe984038fc0 0x7fe98403b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe990006fd0 tx=0x7fe990006e40 comp rx=0 tx=0).stop 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 msgr2=0x7fe99c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c0740e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005850 tx=0x7fe98c005030 comp rx=0 tx=0).stop 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 shutdown_connections 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe984038fc0 0x7fe98403b480 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 --2- 172.21.15.22:0/3553550545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99c1077c0 0x7fe99c0740e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.200+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 >> 172.21.15.22:0/3553550545 conn(0x7fe99c0fd2f0 msgr2=0x7fe99c109370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.201+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 shutdown_connections 2024-03-08T23:45:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:19.201+0000 7fe9a16b1700 1 -- 172.21.15.22:0/3553550545 wait complete. 2024-03-08T23:45:19.869 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:19 smithi022 ceph-mon[31894]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:19.869 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:19 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:19.900 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-08T23:45:19.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd crush tunables default 2024-03-08T23:45:20.142 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:20 smithi022 ceph-mon[31894]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:45:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.592+0000 7f3f3e98b700 1 -- 172.21.15.22:0/2742930903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 msgr2=0x7f3f38107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.592+0000 7f3f3e98b700 1 --2- 172.21.15.22:0/2742930903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f38107ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009910 tx=0x7f3f28009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.592+0000 7f3f3e98b700 1 -- 172.21.15.22:0/2742930903 shutdown_connections 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.592+0000 7f3f3e98b700 1 --2- 172.21.15.22:0/2742930903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f38107ba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.592+0000 7f3f3e98b700 1 -- 172.21.15.22:0/2742930903 >> 172.21.15.22:0/2742930903 conn(0x7f3f380fd2f0 msgr2=0x7f3f380ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.592+0000 7f3f3e98b700 1 -- 172.21.15.22:0/2742930903 shutdown_connections 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.592+0000 7f3f3e98b700 1 -- 172.21.15.22:0/2742930903 wait complete. 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.593+0000 7f3f3e98b700 1 Processor -- start 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.593+0000 7f3f3e98b700 1 -- start start 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.593+0000 7f3f3e98b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f380740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.593+0000 7f3f3e98b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3810be00 con 0x7f3f381077c0 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f380740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f37fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f380740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40738/0 (socket says 172.21.15.22:40738) 2024-03-08T23:45:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f37fff700 1 -- 172.21.15.22:0/352134092 learned_addr learned my addr 172.21.15.22:0/352134092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f37fff700 1 -- 172.21.15.22:0/352134092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f38075fd0 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f37fff700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f380740e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f28006010 tx=0x7f3f28005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f357fa700 1 -- 172.21.15.22:0/352134092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f28026070 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f357fa700 1 -- 172.21.15.22:0/352134092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f2800bc30 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f357fa700 1 -- 172.21.15.22:0/352134092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f280039d0 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f38074680 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.594+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f38074b40 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.595+0000 7f3f357fa700 1 -- 172.21.15.22:0/352134092 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f3f28029070 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.595+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f38102f00 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.595+0000 7f3f357fa700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3f20039020 0x7f3f2003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.595+0000 7f3f357fa700 1 -- 172.21.15.22:0/352134092 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3f2801e070 con 0x7f3f381077c0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.595+0000 7f3f377fe700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3f20039020 0x7f3f2003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.596+0000 7f3f377fe700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3f20039020 0x7f3f2003b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006fd0 tx=0x7f3f2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:21.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.600+0000 7f3f357fa700 1 -- 172.21.15.22:0/352134092 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3f28013070 con 0x7f3f381077c0 2024-03-08T23:45:21.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:21.810+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3f38063410 con 0x7f3f381077c0 2024-03-08T23:45:21.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:21 smithi022 ceph-mon[31894]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:22.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.640+0000 7f3f357fa700 1 -- 172.21.15.22:0/352134092 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3f2802bd20 con 0x7f3f381077c0 2024-03-08T23:45:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3f20039020 msgr2=0x7f3f2003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3f20039020 0x7f3f2003b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006fd0 tx=0x7f3f2c006e40 comp rx=0 tx=0).stop 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 msgr2=0x7f3f380740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f380740e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f28006010 tx=0x7f3f28005030 comp rx=0 tx=0).stop 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 shutdown_connections 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3f20039020 0x7f3f2003b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 --2- 172.21.15.22:0/352134092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f381077c0 0x7f3f380740e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 >> 172.21.15.22:0/352134092 conn(0x7f3f380fd2f0 msgr2=0x7f3f38109330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 shutdown_connections 2024-03-08T23:45:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:22.643+0000 7f3f3e98b700 1 -- 172.21.15.22:0/352134092 wait complete. 2024-03-08T23:45:22.643 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/352134092' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:22.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:23.410 INFO:tasks.cephadm:Adding mon.a on smithi022 2024-03-08T23:45:23.410 INFO:tasks.cephadm:Adding mon.b on smithi062 2024-03-08T23:45:23.410 INFO:tasks.cephadm:Adding mon.c on smithi192 2024-03-08T23:45:23.410 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch apply mon '3;smithi022:172.21.15.22=a;smithi062:172.21.15.62=b;smithi192:172.21.15.192=c' 2024-03-08T23:45:23.635 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:45:23.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:23.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-08T23:45:23.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:45:23.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:23.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/352134092' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-08T23:45:23.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:23.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:45:23.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:23.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:23.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:25.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.102+0000 7f4157847700 1 -- 172.21.15.192:0/496693615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 msgr2=0x7f4150106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:25.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.102+0000 7f4157847700 1 --2- 172.21.15.192:0/496693615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f4150106a90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f414c009910 tx=0x7f414c009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:25.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.102+0000 7f4157847700 1 -- 172.21.15.192:0/496693615 shutdown_connections 2024-03-08T23:45:25.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.102+0000 7f4157847700 1 --2- 172.21.15.192:0/496693615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f4150106a90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:25.103 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.102+0000 7f4157847700 1 -- 172.21.15.192:0/496693615 >> 172.21.15.192:0/496693615 conn(0x7f4150077320 msgr2=0x7f41500fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.102+0000 7f4157847700 1 -- 172.21.15.192:0/496693615 shutdown_connections 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.102+0000 7f4157847700 1 -- 172.21.15.192:0/496693615 wait complete. 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.103+0000 7f4157847700 1 Processor -- start 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.103+0000 7f4157847700 1 -- start start 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.103+0000 7f4157847700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f41501972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.103+0000 7f4157847700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415010acf0 con 0x7f41501066b0 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41555e3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f41501972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41555e3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f41501972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.192:36774/0 (socket says 172.21.15.192:36774) 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41555e3700 1 -- 172.21.15.192:0/892958658 learned_addr learned my addr 172.21.15.192:0/892958658 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41555e3700 1 -- 172.21.15.192:0/892958658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4150197820 con 0x7f41501066b0 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41555e3700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f41501972e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f414c005850 tx=0x7f414c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41467fc700 1 -- 172.21.15.192:0/892958658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f414c026070 con 0x7f41501066b0 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4150197b10 con 0x7f41501066b0 2024-03-08T23:45:25.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41467fc700 1 -- 172.21.15.192:0/892958658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f414c00bbd0 con 0x7f41501066b0 2024-03-08T23:45:25.105 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f41467fc700 1 -- 172.21.15.192:0/892958658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f414c003970 con 0x7f41501066b0 2024-03-08T23:45:25.105 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.104+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415019bbb0 con 0x7f41501066b0 2024-03-08T23:45:25.105 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.105+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4150101df0 con 0x7f41501066b0 2024-03-08T23:45:25.105 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.105+0000 7f41467fc700 1 -- 172.21.15.192:0/892958658 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f414c029070 con 0x7f41501066b0 2024-03-08T23:45:25.105 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.105+0000 7f41467fc700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f413c038fc0 0x7f413c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:25.105 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.105+0000 7f41467fc700 1 -- 172.21.15.192:0/892958658 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f414c01e070 con 0x7f41501066b0 2024-03-08T23:45:25.105 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.106+0000 7f4154de2700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f413c038fc0 0x7f413c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:25.106 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.106+0000 7f4154de2700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f413c038fc0 0x7f413c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4140006fd0 tx=0x7f4140006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:25.108 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.111+0000 7f41467fc700 1 -- 172.21.15.192:0/892958658 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f414c013070 con 0x7f41501066b0 2024-03-08T23:45:25.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.314+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi062:172.21.15.62=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f4150198640 con 0x7f413c038fc0 2024-03-08T23:45:25.318 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.321+0000 7f41467fc700 1 -- 172.21.15.192:0/892958658 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4150198640 con 0x7f413c038fc0 2024-03-08T23:45:25.319 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-03-08T23:45:25.321 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f413c038fc0 msgr2=0x7f413c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:25.321 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f413c038fc0 0x7f413c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4140006fd0 tx=0x7f4140006e40 comp rx=0 tx=0).stop 2024-03-08T23:45:25.321 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 msgr2=0x7f41501972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:25.321 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f41501972e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f414c005850 tx=0x7f414c005030 comp rx=0 tx=0).stop 2024-03-08T23:45:25.321 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 shutdown_connections 2024-03-08T23:45:25.322 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f413c038fc0 0x7f413c03b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:25.322 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 --2- 172.21.15.192:0/892958658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41501066b0 0x7f41501972e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:25.322 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 >> 172.21.15.192:0/892958658 conn(0x7f4150077320 msgr2=0x7f41500691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:25.322 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 shutdown_connections 2024-03-08T23:45:25.322 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:25.323+0000 7f4157847700 1 -- 172.21.15.192:0/892958658 wait complete. 2024-03-08T23:45:25.936 DEBUG:teuthology.orchestra.run.smithi062:mon.b> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.b.service 2024-03-08T23:45:25.939 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.c.service 2024-03-08T23:45:25.941 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T23:45:25.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:25.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:26.634 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:45:26.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:26 smithi022 ceph-mon[31894]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi062:172.21.15.62=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:26.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:26 smithi022 ceph-mon[31894]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi062:172.21.15.62=b;smithi192:172.21.15.192=c;count:3 2024-03-08T23:45:26.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:26 smithi022 ceph-mon[31894]: Deploying daemon mon.c on smithi192 2024-03-08T23:45:27.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:27 smithi022 ceph-mon[31894]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.672+0000 7fd00e080700 1 -- 172.21.15.192:0/1846393423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 msgr2=0x7fd008107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.672+0000 7fd00e080700 1 --2- 172.21.15.192:0/1846393423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd008107b90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009910 tx=0x7fcff8009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.672+0000 7fd00e080700 1 -- 172.21.15.192:0/1846393423 shutdown_connections 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.672+0000 7fd00e080700 1 --2- 172.21.15.192:0/1846393423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd008107b90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.672+0000 7fd00e080700 1 -- 172.21.15.192:0/1846393423 >> 172.21.15.192:0/1846393423 conn(0x7fd0080fd320 msgr2=0x7fd0080ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.672+0000 7fd00e080700 1 -- 172.21.15.192:0/1846393423 shutdown_connections 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.672+0000 7fd00e080700 1 -- 172.21.15.192:0/1846393423 wait complete. 2024-03-08T23:45:28.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.673+0000 7fd00e080700 1 Processor -- start 2024-03-08T23:45:28.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.673+0000 7fd00e080700 1 -- start start 2024-03-08T23:45:28.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.673+0000 7fd00e080700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd0081994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:28.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.673+0000 7fd00e080700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00810bdf0 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd0077fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd0081994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd0077fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd0081994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.192:42954/0 (socket says 172.21.15.192:42954) 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd0077fe700 1 -- 172.21.15.192:0/2089601946 learned_addr learned my addr 172.21.15.192:0/2089601946 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd0077fe700 1 -- 172.21.15.192:0/2089601946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0081999f0 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd0077fe700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd0081994b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcff8005850 tx=0x7fcff8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd004ff9700 1 -- 172.21.15.192:0/2089601946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff8026070 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd008199ce0 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd004ff9700 1 -- 172.21.15.192:0/2089601946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff800bbd0 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd004ff9700 1 -- 172.21.15.192:0/2089601946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff8003970 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.674+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00819dd80 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.675+0000 7fd004ff9700 1 -- 172.21.15.192:0/2089601946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fcff8029070 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.675+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd008102ef0 con 0x7fd0081077b0 2024-03-08T23:45:28.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.675+0000 7fd004ff9700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcff0038fc0 0x7fcff003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:28.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.675+0000 7fd004ff9700 1 -- 172.21.15.192:0/2089601946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcff801e070 con 0x7fd0081077b0 2024-03-08T23:45:28.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.676+0000 7fd006ffd700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcff0038fc0 0x7fcff003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:28.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.676+0000 7fd006ffd700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcff0038fc0 0x7fcff003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcffc006fd0 tx=0x7fcffc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:28.678 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.680+0000 7fd004ff9700 1 -- 172.21.15.192:0/2089601946 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fcff8013070 con 0x7fd0081077b0 2024-03-08T23:45:28.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.963+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd00819a810 con 0x7fd0081077b0 2024-03-08T23:45:28.962 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.964+0000 7fd004ff9700 1 -- 172.21.15.192:0/2089601946 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7fcff8030070 con 0x7fd0081077b0 2024-03-08T23:45:28.962 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:45:28.963 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","modified":"2024-03-08T23:43:37.891079Z","created":"2024-03-08T23:43:37.891079Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-08T23:45:28.964 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcff0038fc0 msgr2=0x7fcff003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:28.964 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcff0038fc0 0x7fcff003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcffc006fd0 tx=0x7fcffc006e40 comp rx=0 tx=0).stop 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 msgr2=0x7fd0081994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd0081994b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcff8005850 tx=0x7fcff8005030 comp rx=0 tx=0).stop 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 shutdown_connections 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcff0038fc0 0x7fcff003b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 --2- 172.21.15.192:0/2089601946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0081077b0 0x7fd0081994b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 >> 172.21.15.192:0/2089601946 conn(0x7fd0080fd320 msgr2=0x7fd008075ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 shutdown_connections 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:28.966+0000 7fd00e080700 1 -- 172.21.15.192:0/2089601946 wait complete. 2024-03-08T23:45:28.965 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-03-08T23:45:29.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:29 smithi022 ceph-mon[31894]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:29.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:29 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/2089601946' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T23:45:31.109 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T23:45:31.110 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T23:45:31.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:30 smithi192 systemd[1]: Started Ceph mon.c for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:45:31.342 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:45:35.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.997+0000 7fa88e700700 1 -- 172.21.15.192:0/331498041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 msgr2=0x7fa870005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:35.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.997+0000 7fa88e700700 1 --2- 172.21.15.192:0/331498041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa870005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa87c005fa0 tx=0x7fa87c00f4a0 comp rx=0 tx=0).stop 2024-03-08T23:45:35.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.997+0000 7fa88e700700 1 -- 172.21.15.192:0/331498041 shutdown_connections 2024-03-08T23:45:35.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.997+0000 7fa88e700700 1 --2- 172.21.15.192:0/331498041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa870005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:35.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.997+0000 7fa88e700700 1 -- 172.21.15.192:0/331498041 >> 172.21.15.192:0/331498041 conn(0x7fa8880fc280 msgr2=0x7fa8880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:35.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.997+0000 7fa88e700700 1 -- 172.21.15.192:0/331498041 shutdown_connections 2024-03-08T23:45:35.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.997+0000 7fa88e700700 1 -- 172.21.15.192:0/331498041 wait complete. 2024-03-08T23:45:35.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.998+0000 7fa88e700700 1 Processor -- start 2024-03-08T23:45:35.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.998+0000 7fa88e700700 1 -- start start 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.998+0000 7fa88e700700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa888197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.998+0000 7fa88e700700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 0x7fa88819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.998+0000 7fa88e700700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888108830 con 0x7fa888106790 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.998+0000 7fa88e700700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8881086b0 con 0x7fa888197950 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa8877fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 0x7fa88819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa887fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa888197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa8877fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 0x7fa88819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57014/0 (socket says 172.21.15.192:57014) 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa887fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa888197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.192:52660/0 (socket says 172.21.15.192:52660) 2024-03-08T23:45:35.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa8877fe700 1 -- 172.21.15.192:0/1070204542 learned_addr learned my addr 172.21.15.192:0/1070204542 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa8877fe700 1 -- 172.21.15.192:0/1070204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 msgr2=0x7fa88819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa8877fe700 1 -- 172.21.15.192:0/1070204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 msgr2=0x7fa88819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa8877fe700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 0x7fa88819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa8877fe700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 0x7fa88819bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa887fff700 1 -- 172.21.15.192:0/1070204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 msgr2=0x7fa88819bc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa887fff700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 0x7fa88819bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa887fff700 1 -- 172.21.15.192:0/1070204542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888197f90 con 0x7fa888106790 2024-03-08T23:45:35.999 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:35.999+0000 7fa887fff700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa888197410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000c00 tx=0x7fa87c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:36.000 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.002+0000 7fa8857fa700 1 -- 172.21.15.192:0/1070204542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa87c0054c0 con 0x7fa888106790 2024-03-08T23:45:36.000 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.002+0000 7fa8857fa700 1 -- 172.21.15.192:0/1070204542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa87c021040 con 0x7fa888106790 2024-03-08T23:45:36.000 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.002+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88819c1d0 con 0x7fa888106790 2024-03-08T23:45:36.000 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.002+0000 7fa8857fa700 1 -- 172.21.15.192:0/1070204542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa87c012e10 con 0x7fa888106790 2024-03-08T23:45:36.000 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.002+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88819c730 con 0x7fa888106790 2024-03-08T23:45:36.001 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.003+0000 7fa8857fa700 1 -- 172.21.15.192:0/1070204542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fa87c02e020 con 0x7fa888106790 2024-03-08T23:45:36.001 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.003+0000 7fa8857fa700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa874038d60 0x7fa87403b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:36.002 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.003+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa888101ed0 con 0x7fa888106790 2024-03-08T23:45:36.002 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.004+0000 7fa8857fa700 1 -- 172.21.15.192:0/1070204542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa87c02c020 con 0x7fa888106790 2024-03-08T23:45:36.002 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.004+0000 7fa8877fe700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa874038d60 0x7fa87403b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:36.002 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.004+0000 7fa8877fe700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa874038d60 0x7fa87403b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa878009180 tx=0x7fa878008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:36.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.009+0000 7fa8857fa700 1 -- 172.21.15.192:0/1070204542 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa87c010630 con 0x7fa888106790 2024-03-08T23:45:36.282 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.284+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa88819c9c0 con 0x7fa888106790 2024-03-08T23:45:36.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:45:36.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: mon.a calling monitor election 2024-03-08T23:45:36.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:36.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: mon.c calling monitor election 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: fsmap 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: mgrmap e13: x(active, since 52s) 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: overall HEALTH_OK 2024-03-08T23:45:36.283 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:36.284 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:36.284 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:36.284 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:36.284 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:36.284 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.284+0000 7fa8857fa700 1 -- 172.21.15.192:0/1070204542 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fa87c0150d0 con 0x7fa888106790 2024-03-08T23:45:36.284 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:45:36.284 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","modified":"2024-03-08T23:45:30.903382Z","created":"2024-03-08T23:43:37.891079Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-08T23:45:36.285 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa874038d60 msgr2=0x7fa87403b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:36.285 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa874038d60 0x7fa87403b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa878009180 tx=0x7fa878008040 comp rx=0 tx=0).stop 2024-03-08T23:45:36.285 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 msgr2=0x7fa888197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa888197410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa87c000c00 tx=0x7fa87c004760 comp rx=0 tx=0).stop 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 shutdown_connections 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa874038d60 0x7fa87403b220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa888106790 0x7fa888197410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 --2- 172.21.15.192:0/1070204542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa888197950 0x7fa88819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 >> 172.21.15.192:0/1070204542 conn(0x7fa8880fc280 msgr2=0x7fa88806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.286+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 shutdown_connections 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:36.287+0000 7fa88e700700 1 -- 172.21.15.192:0/1070204542 wait complete. 2024-03-08T23:45:36.286 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: mon.a calling monitor election 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: mon.c calling monitor election 2024-03-08T23:45:36.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: fsmap 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: mgrmap e13: x(active, since 52s) 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: overall HEALTH_OK 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:36.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:37.275 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:37 smithi192 ceph-mon[35995]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:37.275 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:37 smithi192 ceph-mon[35995]: Deploying daemon mon.b on smithi062 2024-03-08T23:45:37.275 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:37 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/1070204542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T23:45:37.275 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:37 smithi022 ceph-mon[31894]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:37 smithi022 ceph-mon[31894]: Deploying daemon mon.b on smithi062 2024-03-08T23:45:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:37 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/1070204542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T23:45:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:37.747 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T23:45:37.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T23:45:37.981 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:45:39.257 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:39 smithi192 ceph-mon[35995]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:39.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:39 smithi022 ceph-mon[31894]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.472+0000 7f3890118700 1 -- 172.21.15.192:0/2675194937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 msgr2=0x7f3888107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.472+0000 7f3890118700 1 --2- 172.21.15.192:0/2675194937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888107ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3878009910 tx=0x7f3878009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.472+0000 7f3890118700 1 -- 172.21.15.192:0/2675194937 shutdown_connections 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.472+0000 7f3890118700 1 --2- 172.21.15.192:0/2675194937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888107ca0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.472+0000 7f3890118700 1 -- 172.21.15.192:0/2675194937 >> 172.21.15.192:0/2675194937 conn(0x7f38880fd3b0 msgr2=0x7f38880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.472+0000 7f3890118700 1 -- 172.21.15.192:0/2675194937 shutdown_connections 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.472+0000 7f3890118700 1 -- 172.21.15.192:0/2675194937 wait complete. 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f3890118700 1 Processor -- start 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f3890118700 1 -- start start 2024-03-08T23:45:39.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f3890118700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f3890118700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 0x7f388819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f3890118700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388810bf00 con 0x7f38881078c0 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f3890118700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388810bd80 con 0x7f3888197aa0 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f388deb4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f388d6b3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 0x7f388819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f388deb4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.192:52678/0 (socket says 172.21.15.192:52678) 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.473+0000 7f388deb4700 1 -- 172.21.15.192:0/36870934 learned_addr learned my addr 172.21.15.192:0/36870934 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388d6b3700 1 -- 172.21.15.192:0/36870934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 msgr2=0x7f388819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388d6b3700 1 -- 172.21.15.192:0/36870934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 msgr2=0x7f388819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388d6b3700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 0x7f388819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388d6b3700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 0x7f388819be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:45:39.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388deb4700 1 -- 172.21.15.192:0/36870934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 msgr2=0x7f388819be30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388deb4700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 0x7f388819be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388deb4700 1 -- 172.21.15.192:0/36870934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38881980e0 con 0x7f38881078c0 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f388deb4700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888197560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3878005b40 tx=0x7f3878005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f387effd700 1 -- 172.21.15.192:0/36870934 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f3878005dc0 con 0x7f38881078c0 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f387effd700 1 -- 172.21.15.192:0/36870934 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f387800be40 con 0x7f38881078c0 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f387effd700 1 -- 172.21.15.192:0/36870934 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f387800f460 con 0x7f38881078c0 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388819c370 con 0x7f38881078c0 2024-03-08T23:45:39.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.474+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388819c8d0 con 0x7f38881078c0 2024-03-08T23:45:39.476 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.475+0000 7f387effd700 1 -- 172.21.15.192:0/36870934 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f387802a070 con 0x7f38881078c0 2024-03-08T23:45:39.476 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.475+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3888103000 con 0x7f38881078c0 2024-03-08T23:45:39.476 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.475+0000 7f387effd700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3874038d60 0x7f387403b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:39.476 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.475+0000 7f387effd700 1 -- 172.21.15.192:0/36870934 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f387801e030 con 0x7f38881078c0 2024-03-08T23:45:39.476 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.476+0000 7f388d6b3700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3874038d60 0x7f387403b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:39.476 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.476+0000 7f388d6b3700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3874038d60 0x7f387403b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3884009850 tx=0x7f3884008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:39.478 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.481+0000 7f387effd700 1 -- 172.21.15.192:0/36870934 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3878013070 con 0x7f38881078c0 2024-03-08T23:45:39.755 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.756+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f388819cb80 con 0x7f38881078c0 2024-03-08T23:45:39.755 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.757+0000 7f387effd700 1 -- 172.21.15.192:0/36870934 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f3878031020 con 0x7f38881078c0 2024-03-08T23:45:39.756 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:45:39.756 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","modified":"2024-03-08T23:45:30.903382Z","created":"2024-03-08T23:43:37.891079Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3874038d60 msgr2=0x7f387403b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3874038d60 0x7f387403b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3884009850 tx=0x7f3884008040 comp rx=0 tx=0).stop 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 msgr2=0x7f3888197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888197560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3878005b40 tx=0x7f3878005050 comp rx=0 tx=0).stop 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 shutdown_connections 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3874038d60 0x7f387403b220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38881078c0 0x7f3888197560 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 --2- 172.21.15.192:0/36870934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3888197aa0 0x7f388819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 >> 172.21.15.192:0/36870934 conn(0x7f38880fd3b0 msgr2=0x7f38881033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 shutdown_connections 2024-03-08T23:45:39.757 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:39.759+0000 7f3890118700 1 -- 172.21.15.192:0/36870934 wait complete. 2024-03-08T23:45:39.758 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-03-08T23:45:39.777 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 podman[36110]: 2024-03-08 23:45:39.756087861 +0000 UTC m=+0.355729478 container start 18fb8795a3875eb2e06a95d576156ce87425adfe3c3ab8e6b09feb229e8aac24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: set uid:gid to 167:167 (ceph:ceph) 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev), process ceph-mon, pid 2 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pidfile_write: ignore empty --pid-file 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: load: jerasure load: lrc 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: RocksDB version: 7.9.2 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Git sha 0 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Compile date 2024-03-08 15:06:51 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: DB SUMMARY 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: DB Session ID: J30971F0YXDAB8KCAROI 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: CURRENT file: CURRENT 2024-03-08T23:45:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: IDENTITY file: IDENTITY 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.error_if_exists: 0 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.create_if_missing: 0 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.paranoid_checks: 1 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.env: 0x563e988c1b20 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.fs: PosixFileSystem 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.info_log: 0x563e998038e0 2024-03-08T23:45:40.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_file_opening_threads: 16 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.statistics: (nil) 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.use_fsync: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_log_file_size: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.keep_log_file_num: 1000 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.recycle_log_file_num: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.allow_fallocate: 1 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.allow_mmap_reads: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.allow_mmap_writes: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.use_direct_reads: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.create_missing_column_families: 0 2024-03-08T23:45:40.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.db_log_dir: 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.wal_dir: 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.advise_random_on_open: 1 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.db_write_buffer_size: 0 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.write_buffer_manager: 0x563e997d7400 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-08T23:45:40.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.rate_limiter: (nil) 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.wal_recovery_mode: 2 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.enable_thread_tracking: 0 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.enable_pipelined_write: 0 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.unordered_write: 0 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.row_cache: None 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.wal_filter: None 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-08T23:45:40.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.allow_ingest_behind: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.two_write_queues: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.manual_wal_flush: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.wal_compression: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.atomic_flush: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.log_readahead_size: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.best_efforts_recovery: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.allow_data_in_errors: 0 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.db_host_id: __hostname__ 2024-03-08T23:45:40.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-08T23:45:40.121 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_background_jobs: 2 2024-03-08T23:45:40.121 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_background_compactions: -1 2024-03-08T23:45:40.121 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_subcompactions: 1 2024-03-08T23:45:40.121 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-08T23:45:40.121 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_total_wal_size: 0 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_open_files: -1 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bytes_per_sync: 0 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_readahead_size: 0 2024-03-08T23:45:40.122 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_background_flushes: -1 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Compression algorithms supported: 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kZSTD supported: 0 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kXpressCompression supported: 0 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kLZ4HCCompression supported: 1 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kZlibCompression supported: 1 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kSnappyCompression supported: 1 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kLZ4Compression supported: 1 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: kBZip2Compression supported: 0 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-08T23:45:40.123 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.merge_operator: 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_filter: None 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_filter_factory: None 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.sst_partitioner_factory: None 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563e998039e0) 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_type: 0 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout: data_block_index_type: 0 2024-03-08T23:45:40.124 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_shortening: 1 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: checksum: 4 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: no_block_cache: 0 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache: 0x563e99809090 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_options: 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: capacity : 536870912 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: num_shard_bits : 4 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: strict_capacity_limit : 0 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_compressed: (nil) 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: persistent_cache: (nil) 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_size: 4096 2024-03-08T23:45:40.125 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_size_deviation: 10 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_restart_interval: 16 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_block_restart_interval: 1 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: metadata_block_size: 4096 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: partition_filters: 0 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: use_delta_encoding: 1 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: filter_policy: bloomfilter 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: whole_key_filtering: 1 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: verify_compression: 0 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: format_version: 5 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: enable_index_compression: 1 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_align: 0 2024-03-08T23:45:40.126 INFO:journalctl@ceph.mon.b.smithi062.stdout: max_auto_readahead_size: 262144 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout: prepopulate_block_cache: 0 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout: initial_auto_readahead_size: 8192 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.write_buffer_size: 33554432 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_write_buffer_number: 2 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression: NoCompression 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression: Disabled 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.prefix_extractor: nullptr 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.num_levels: 7 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-08T23:45:40.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.level: 32767 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.strategy: 0 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-08T23:45:40.128 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.enabled: false 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.target_file_size_base: 67108864 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-08T23:45:40.129 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.arena_block_size: 1048576 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.disable_auto_compactions: 0 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-08T23:45:40.130 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.table_properties_collectors: 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.inplace_update_support: 0 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-08T23:45:40.131 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.bloom_locality: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.max_successive_merges: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.paranoid_file_checks: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.force_consistency_checks: 1 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.report_bg_io_stats: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.ttl: 2592000 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.enable_blob_files: false 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.min_blob_size: 0 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.blob_file_size: 268435456 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-08T23:45:40.132 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.blob_file_starting_level: 0 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7f8e2b1-ce1b-4922-9aca-47914e344f5b 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709941539786799, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709941539787229, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709941539, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7f8e2b1-ce1b-4922-9aca-47914e344f5b", "db_session_id": "J30971F0YXDAB8KCAROI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709941539787320, "job": 1, "event": "recovery_finished"} 2024-03-08T23:45:40.133 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563e998fc000 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: DB pointer 0x563e998e6000 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** DB Stats ** 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T23:45:40.134 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-03-08T23:45:40.135 INFO:journalctl@ceph.mon.b.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: Block cache BinnedLRUCache@0x563e99809090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: using public_addr v2:172.21.15.62:0/0 -> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: starting mon.b rank -1 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:45:40.136 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(???) e0 preinit fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).mds e1 new map 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout: e1 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout: legacy client fscid: -1 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout: No filesystems configured 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-08T23:45:40.137 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Deploying cephadm binary to smithi192 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.138 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:45:40.139 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Added host smithi192 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/352134092' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:40.140 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/352134092' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:40.141 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi062:172.21.15.62=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi062:172.21.15.62=b;smithi192:172.21.15.192=c;count:3 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Deploying daemon mon.c on smithi192 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/2089601946' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.a calling monitor election 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.142 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.c calling monitor election 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: fsmap 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mgrmap e13: x(active, since 52s) 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: overall HEALTH_OK 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.143 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: Deploying daemon mon.b on smithi062 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/1070204542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 ceph-mon[36129]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 bash[36110]: 18fb8795a3875eb2e06a95d576156ce87425adfe3c3ab8e6b09feb229e8aac24 2024-03-08T23:45:40.144 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:39 smithi062 systemd[1]: Started Ceph mon.b for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:45:41.319 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T23:45:41.320 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T23:45:41.561 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:45:45.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:45.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: mon.c calling monitor election 2024-03-08T23:45:45.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:45:45.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: mon.a calling monitor election 2024-03-08T23:45:45.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: fsmap 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: mgrmap e13: x(active, since 61s) 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: mon.b (rank 2) addr [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] is down (out of quorum) 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:45.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: mon.c calling monitor election 2024-03-08T23:45:45.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:45:45.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: mon.a calling monitor election 2024-03-08T23:45:45.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: fsmap 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: mgrmap e13: x(active, since 61s) 2024-03-08T23:45:45.384 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-08T23:45:45.385 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-08T23:45:45.385 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-08T23:45:45.385 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: mon.b (rank 2) addr [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] is down (out of quorum) 2024-03-08T23:45:45.385 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.385 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.385 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.385 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:45.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.861+0000 7f6b531fb700 1 -- 172.21.15.192:0/3483642300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 msgr2=0x7f6b34005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:45.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.861+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3483642300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b34005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c1017f0 tx=0x7f6b44009e80 comp rx=0 tx=0).stop 2024-03-08T23:45:45.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.861+0000 7f6b531fb700 1 -- 172.21.15.192:0/3483642300 shutdown_connections 2024-03-08T23:45:45.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.861+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3483642300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b34005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.861+0000 7f6b531fb700 1 -- 172.21.15.192:0/3483642300 >> 172.21.15.192:0/3483642300 conn(0x7f6b4c0fd3b0 msgr2=0x7f6b4c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.861+0000 7f6b531fb700 1 -- 172.21.15.192:0/3483642300 shutdown_connections 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.861+0000 7f6b531fb700 1 -- 172.21.15.192:0/3483642300 wait complete. 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 Processor -- start 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 -- start start 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b4c197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 0x7f6b4c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 0x7f6b4c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c109910 con 0x7f6b340035c0 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c109790 con 0x7f6b4c197a50 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.862+0000 7f6b531fb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c109a90 con 0x7f6b4c19c320 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b4c197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:45.863 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b51798700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 0x7f6b4c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b4c197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.192:44382/0 (socket says 172.21.15.192:44382) 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 -- 172.21.15.192:0/3181121897 learned_addr learned my addr 172.21.15.192:0/3181121897 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b4bfff700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 0x7f6b4c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b51798700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 msgr2=0x7f6b4c19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b51798700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 msgr2=0x7f6b4c19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b51798700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 0x7f6b4c19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b4bfff700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 msgr2=0x7f6b4c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b51798700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 0x7f6b4c19c7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b4bfff700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 msgr2=0x7f6b4c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b4bfff700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 0x7f6b4c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b4bfff700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 0x7f6b4c19bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 msgr2=0x7f6b4c19c7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 0x7f6b4c19c7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 msgr2=0x7f6b4c19bde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:45:45.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 0x7f6b4c19bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:45.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 -- 172.21.15.192:0/3181121897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c074010 con 0x7f6b340035c0 2024-03-08T23:45:45.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.863+0000 7f6b50f97700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b4c197510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b44002230 tx=0x7f6b44017850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:45.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.864+0000 7f6b49ffb700 1 -- 172.21.15.192:0/3181121897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44025070 con 0x7f6b340035c0 2024-03-08T23:45:45.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.864+0000 7f6b49ffb700 1 -- 172.21.15.192:0/3181121897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b44021d40 con 0x7f6b340035c0 2024-03-08T23:45:45.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.864+0000 7f6b49ffb700 1 -- 172.21.15.192:0/3181121897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440059f0 con 0x7f6b340035c0 2024-03-08T23:45:45.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.864+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c0742a0 con 0x7f6b340035c0 2024-03-08T23:45:45.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.864+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c074790 con 0x7f6b340035c0 2024-03-08T23:45:45.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.865+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c063410 con 0x7f6b340035c0 2024-03-08T23:45:45.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.866+0000 7f6b49ffb700 1 -- 172.21.15.192:0/3181121897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f6b44028070 con 0x7f6b340035c0 2024-03-08T23:45:45.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.866+0000 7f6b49ffb700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6b38039450 0x7f6b3803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:45.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.866+0000 7f6b49ffb700 1 -- 172.21.15.192:0/3181121897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b4404e180 con 0x7f6b340035c0 2024-03-08T23:45:45.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.866+0000 7f6b4bfff700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6b38039450 0x7f6b3803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:45.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.866+0000 7f6b4bfff700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6b38039450 0x7f6b3803b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009850 tx=0x7f6b40008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:45.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:45.870+0000 7f6b49ffb700 1 -- 172.21.15.192:0/3181121897 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6b4402ae10 con 0x7f6b340035c0 2024-03-08T23:45:46.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:46.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:46.156 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.158+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6b4c198870 con 0x7f6b340035c0 2024-03-08T23:45:46.157 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.159+0000 7f6b49ffb700 1 -- 172.21.15.192:0/3181121897 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7f6b4c198870 con 0x7f6b340035c0 2024-03-08T23:45:46.157 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-08T23:45:46.157 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","modified":"2024-03-08T23:45:39.809699Z","created":"2024-03-08T23:43:37.891079Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-08T23:45:46.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6b38039450 msgr2=0x7f6b3803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:46.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6b38039450 0x7f6b3803b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009850 tx=0x7f6b40008040 comp rx=0 tx=0).stop 2024-03-08T23:45:46.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 msgr2=0x7f6b4c197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:46.159 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b4c197510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b44002230 tx=0x7f6b44017850 comp rx=0 tx=0).stop 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 shutdown_connections 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b4c19c320 0x7f6b4c19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6b38039450 0x7f6b3803b910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b340035c0 0x7f6b4c197510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 --2- 172.21.15.192:0/3181121897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b4c197a50 0x7f6b4c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 >> 172.21.15.192:0/3181121897 conn(0x7f6b4c0fd3b0 msgr2=0x7f6b4c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 shutdown_connections 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:46.161+0000 7f6b531fb700 1 -- 172.21.15.192:0/3181121897 wait complete. 2024-03-08T23:45:46.160 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2024-03-08T23:45:46.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:46.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:46.953 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-08T23:45:46.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph config generate-minimal-conf 2024-03-08T23:45:47.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:47.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: mon.c calling monitor election 2024-03-08T23:45:47.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:45:47.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:47.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: mon.a calling monitor election 2024-03-08T23:45:47.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:45:47.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:47.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:47.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:47.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:47.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:47.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: fsmap 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: mgrmap e13: x(active, since 61s) 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-08T23:45:47.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-08T23:45:47.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: mon.b (rank 2) addr [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] is down (out of quorum) 2024-03-08T23:45:47.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:47.117 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:47.199 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: mon.b calling monitor election 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: mon.b calling monitor election 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: mon.c calling monitor election 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: mon.a calling monitor election 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: fsmap 2024-03-08T23:45:47.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: mgrmap e13: x(active, since 63s) 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: Cluster is now healthy 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: overall HEALTH_OK 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:47.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: Reconfiguring mon.a (unknown last config time)... 2024-03-08T23:45:47.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:47.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:47.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:47.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: Reconfiguring daemon mon.a on smithi022 2024-03-08T23:45:47.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:48.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: mon.b calling monitor election 2024-03-08T23:45:48.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: mon.b calling monitor election 2024-03-08T23:45:48.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: mon.c calling monitor election 2024-03-08T23:45:48.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: mon.a calling monitor election 2024-03-08T23:45:48.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-08T23:45:48.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: fsmap 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: mgrmap e13: x(active, since 63s) 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: Cluster is now healthy 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: overall HEALTH_OK 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: Reconfiguring mon.a (unknown last config time)... 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: Reconfiguring daemon mon.a on smithi022 2024-03-08T23:45:48.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: mon.b calling monitor election 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: mon.b calling monitor election 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: mon.c calling monitor election 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: mon.a calling monitor election 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: fsmap 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: mgrmap e13: x(active, since 63s) 2024-03-08T23:45:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: Cluster is now healthy 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: overall HEALTH_OK 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: Reconfiguring mon.a (unknown last config time)... 2024-03-08T23:45:48.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:48.135 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:48.135 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:48.135 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: Reconfiguring daemon mon.a on smithi022 2024-03-08T23:45:48.135 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:45:49.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:48 smithi022 ceph-mon[31894]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:49.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:48 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:48 smithi062 ceph-mon[36129]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:48 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:48 smithi192 ceph-mon[35995]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:49.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:48 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 -- 172.21.15.22:0/1763166556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 msgr2=0x7fee60101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 --2- 172.21.15.22:0/1763166556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60101ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee50009990 tx=0x7fee50009ca0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 -- 172.21.15.22:0/1763166556 shutdown_connections 2024-03-08T23:45:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 --2- 172.21.15.22:0/1763166556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee601021e0 0x7fee6010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 --2- 172.21.15.22:0/1763166556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee60105d10 0x7fee601060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 --2- 172.21.15.22:0/1763166556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60101ca0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 -- 172.21.15.22:0/1763166556 >> 172.21.15.22:0/1763166556 conn(0x7fee60076010 msgr2=0x7fee60076420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 -- 172.21.15.22:0/1763166556 shutdown_connections 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.761+0000 7fee67a72700 1 -- 172.21.15.22:0/1763166556 wait complete. 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 Processor -- start 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 -- start start 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee601021e0 0x7fee60198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee60105d10 0x7fee6019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee601110e0 con 0x7fee60101820 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee60110f60 con 0x7fee601021e0 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee67a72700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee60111260 con 0x7fee60105d10 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee6580e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee6580e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33604/0 (socket says 172.21.15.22:33604) 2024-03-08T23:45:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee6580e700 1 -- 172.21.15.22:0/683390619 learned_addr learned my addr 172.21.15.22:0/683390619 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.762+0000 7fee6500d700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee601021e0 0x7fee60198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee6580e700 1 -- 172.21.15.22:0/683390619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee60105d10 msgr2=0x7fee6019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee6580e700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee60105d10 0x7fee6019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee6580e700 1 -- 172.21.15.22:0/683390619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee601021e0 msgr2=0x7fee60198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee6580e700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee601021e0 0x7fee60198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee6580e700 1 -- 172.21.15.22:0/683390619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6019d640 con 0x7fee60101820 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee6580e700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60198630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00e9a0 tx=0x7fee5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee56ffd700 1 -- 172.21.15.22:0/683390619 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00cbd0 con 0x7fee60101820 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee56ffd700 1 -- 172.21.15.22:0/683390619 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee5c00cd70 con 0x7fee60101820 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6019d930 con 0x7fee60101820 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee56ffd700 1 -- 172.21.15.22:0/683390619 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c018750 con 0x7fee60101820 2024-03-08T23:45:49.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.763+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee601a51b0 con 0x7fee60101820 2024-03-08T23:45:49.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.764+0000 7fee56ffd700 1 -- 172.21.15.22:0/683390619 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fee5c010430 con 0x7fee60101820 2024-03-08T23:45:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.764+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee60068aa0 con 0x7fee60101820 2024-03-08T23:45:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.764+0000 7fee56ffd700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fee4c039200 0x7fee4c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.764+0000 7fee56ffd700 1 -- 172.21.15.22:0/683390619 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee5c04c390 con 0x7fee60101820 2024-03-08T23:45:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.764+0000 7fee6500d700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fee4c039200 0x7fee4c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:49.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.765+0000 7fee6500d700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fee4c039200 0x7fee4c03b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee60076580 tx=0x7fee50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:49.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.769+0000 7fee56ffd700 1 -- 172.21.15.22:0/683390619 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fee5c020360 con 0x7fee60101820 2024-03-08T23:45:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.972+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fee601a5440 con 0x7fee60101820 2024-03-08T23:45:49.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.973+0000 7fee56ffd700 1 -- 172.21.15.22:0/683390619 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7fee5c010720 con 0x7fee60101820 2024-03-08T23:45:49.971 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:45:49.971 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-03-08T23:45:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:45:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-03-08T23:45:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fee4c039200 msgr2=0x7fee4c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fee4c039200 0x7fee4c03b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee60076580 tx=0x7fee50005fd0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 msgr2=0x7fee60198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:49.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60198630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00e9a0 tx=0x7fee5c00ecb0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 shutdown_connections 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee60105d10 0x7fee6019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fee4c039200 0x7fee4c03b6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee60101820 0x7fee60198630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 --2- 172.21.15.22:0/683390619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee601021e0 0x7fee60198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 >> 172.21.15.22:0/683390619 conn(0x7fee60076010 msgr2=0x7fee60100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 shutdown_connections 2024-03-08T23:45:49.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:45:49.975+0000 7fee67a72700 1 -- 172.21.15.22:0/683390619 wait complete. 2024-03-08T23:45:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: Reconfiguring mon.b (monmap changed)... 2024-03-08T23:45:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: Reconfiguring daemon mon.b on smithi062 2024-03-08T23:45:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:50 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/683390619' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:50.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:50.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:50.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: Reconfiguring mon.b (monmap changed)... 2024-03-08T23:45:50.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:50.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:50.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:50.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: Reconfiguring daemon mon.b on smithi062 2024-03-08T23:45:50.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:50 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/683390619' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: Reconfiguring mon.b (monmap changed)... 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: Reconfiguring daemon mon.b on smithi062 2024-03-08T23:45:50.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:50 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/683390619' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:50.675 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-08T23:45:50.675 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:45:50.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-03-08T23:45:50.704 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:45:50.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:50.771 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:45:50.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-03-08T23:45:50.806 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:45:50.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:50.878 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:45:50.878 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-03-08T23:45:50.914 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:45:50.914 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:45:50.982 INFO:tasks.cephadm:Adding mgr.x on smithi022 2024-03-08T23:45:50.982 INFO:tasks.cephadm:Adding mgr.y on smithi062 2024-03-08T23:45:50.982 INFO:tasks.cephadm:Adding mgr.z on smithi192 2024-03-08T23:45:50.982 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch apply mgr '3;smithi022=x;smithi062=y;smithi192=z' 2024-03-08T23:45:51.256 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:45:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:51 smithi062 ceph-mon[36129]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:51 smithi192 ceph-mon[35995]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:51.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:51 smithi022 ceph-mon[31894]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:51.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:51.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:51.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T23:45:51.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T23:45:51.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:52.418 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:52 smithi192 ceph-mon[35995]: Reconfiguring mon.c (monmap changed)... 2024-03-08T23:45:52.418 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:52 smithi192 ceph-mon[35995]: Reconfiguring daemon mon.c on smithi192 2024-03-08T23:45:52.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.608+0000 7feca14fc700 1 -- 172.21.15.192:0/1142499418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c101930 msgr2=0x7fec9c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:52.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.608+0000 7feca14fc700 1 --2- 172.21.15.192:0/1142499418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c101930 0x7fec9c101d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009910 tx=0x7fec8c009c20 comp rx=0 tx=0).stop 2024-03-08T23:45:52.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.608+0000 7feca14fc700 1 -- 172.21.15.192:0/1142499418 shutdown_connections 2024-03-08T23:45:52.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.608+0000 7feca14fc700 1 --2- 172.21.15.192:0/1142499418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c101930 0x7fec9c101d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:52.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.608+0000 7feca14fc700 1 -- 172.21.15.192:0/1142499418 >> 172.21.15.192:0/1142499418 conn(0x7fec9c0fced0 msgr2=0x7fec9c0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:52.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.608+0000 7feca14fc700 1 -- 172.21.15.192:0/1142499418 shutdown_connections 2024-03-08T23:45:52.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.608+0000 7feca14fc700 1 -- 172.21.15.192:0/1142499418 wait complete. 2024-03-08T23:45:52.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.609+0000 7feca14fc700 1 Processor -- start 2024-03-08T23:45:52.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.609+0000 7feca14fc700 1 -- start start 2024-03-08T23:45:52.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.609+0000 7feca14fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec9c101930 0x7fec9c075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.609+0000 7feca14fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9c0741d0 0x7fec9c074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.609+0000 7feca14fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c074b90 0x7fec9c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7feca14fc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c1079c0 con 0x7fec9c074b90 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7feca14fc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c107840 con 0x7fec9c101930 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7feca14fc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c107b40 con 0x7fec9c0741d0 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9c0741d0 0x7fec9c074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9c0741d0 0x7fec9c074650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:47740/0 (socket says 172.21.15.192:47740) 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 -- 172.21.15.192:0/459840325 learned_addr learned my addr 172.21.15.192:0/459840325 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec9b7fe700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c074b90 0x7fec9c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec9affd700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec9c101930 0x7fec9c075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 -- 172.21.15.192:0/459840325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec9c101930 msgr2=0x7fec9c075b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec9c101930 0x7fec9c075b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:52.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 -- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c074b90 msgr2=0x7fec9c1a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c074b90 0x7fec9c1a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 -- 172.21.15.192:0/459840325 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c1a1770 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec927fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9c0741d0 0x7fec9c074650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec8400b780 tx=0x7fec8400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec98ff9700 1 -- 172.21.15.192:0/459840325 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84004230 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c1a1a60 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec98ff9700 1 -- 172.21.15.192:0/459840325 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec840043d0 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7fec98ff9700 1 -- 172.21.15.192:0/459840325 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84011c90 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.610+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c1a2070 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.611+0000 7fec98ff9700 1 -- 172.21.15.192:0/459840325 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fec8401a400 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.611+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c063410 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.611+0000 7fec98ff9700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fec88039450 0x7fec8803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.611+0000 7fec98ff9700 1 -- 172.21.15.192:0/459840325 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fec8404bbe0 con 0x7fec9c0741d0 2024-03-08T23:45:52.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.612+0000 7fec9affd700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fec88039450 0x7fec8803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:45:52.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.612+0000 7fec9affd700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fec88039450 0x7fec8803b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b5c0 tx=0x7fec8c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:45:52.614 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.616+0000 7fec98ff9700 1 -- 172.21.15.192:0/459840325 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fec840036a0 con 0x7fec9c0741d0 2024-03-08T23:45:52.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:52 smithi022 ceph-mon[31894]: Reconfiguring mon.c (monmap changed)... 2024-03-08T23:45:52.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:52 smithi022 ceph-mon[31894]: Reconfiguring daemon mon.c on smithi192 2024-03-08T23:45:52.828 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.830+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi062=y;smithi192=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fec9c0622b0 con 0x7fec88039450 2024-03-08T23:45:52.835 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.837+0000 7fec98ff9700 1 -- 172.21.15.192:0/459840325 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fec9c0622b0 con 0x7fec88039450 2024-03-08T23:45:52.835 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fec88039450 msgr2=0x7fec8803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fec88039450 0x7fec8803b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b5c0 tx=0x7fec8c01a040 comp rx=0 tx=0).stop 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9c0741d0 msgr2=0x7fec9c074650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9c0741d0 0x7fec9c074650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec8400b780 tx=0x7fec8400ba90 comp rx=0 tx=0).stop 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 shutdown_connections 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fec88039450 0x7fec8803b910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec9c074b90 0x7fec9c1a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec9c101930 0x7fec9c075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 --2- 172.21.15.192:0/459840325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec9c0741d0 0x7fec9c074650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:45:52.837 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 >> 172.21.15.192:0/459840325 conn(0x7fec9c0fced0 msgr2=0x7fec9c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:45:52.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 shutdown_connections 2024-03-08T23:45:52.838 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:45:52.839+0000 7feca14fc700 1 -- 172.21.15.192:0/459840325 wait complete. 2024-03-08T23:45:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:52 smithi062 ceph-mon[36129]: Reconfiguring mon.c (monmap changed)... 2024-03-08T23:45:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:52 smithi062 ceph-mon[36129]: Reconfiguring daemon mon.c on smithi192 2024-03-08T23:45:53.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:53 smithi192 ceph-mon[35995]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:53.585 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:53 smithi062 ceph-mon[36129]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:53 smithi022 ceph-mon[31894]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:53.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:53.892 DEBUG:teuthology.orchestra.run.smithi062:mgr.y> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.y.service 2024-03-08T23:45:53.894 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.z.service 2024-03-08T23:45:53.897 INFO:tasks.cephadm:Deploying OSDs... 2024-03-08T23:45:53.897 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:45:53.897 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T23:45:53.916 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T23:45:53.916 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-03-08T23:45:53.973 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-08 23:44:57.004837007 +0000 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-08 23:38:18.237208587 +0000 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-08 23:38:18.237208587 +0000 2024-03-08T23:45:53.974 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-08 23:38:18.237208587 +0000 2024-03-08T23:45:53.974 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T23:45:54.043 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-08T23:45:54.043 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-08T23:45:54.043 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000120554 s, 4.2 MB/s 2024-03-08T23:45:54.045 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T23:45:54.102 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-08 23:44:57.031836649 +0000 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-08 23:38:18.580204015 +0000 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-08 23:38:18.580204015 +0000 2024-03-08T23:45:54.159 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-08 23:38:18.580204015 +0000 2024-03-08T23:45:54.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T23:45:54.226 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-08T23:45:54.226 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-08T23:45:54.226 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000113921 s, 4.5 MB/s 2024-03-08T23:45:54.228 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T23:45:54.285 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-08 23:44:57.057836305 +0000 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-08 23:38:18.902199722 +0000 2024-03-08T23:45:54.342 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-08 23:38:18.902199722 +0000 2024-03-08T23:45:54.343 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-08 23:38:18.902199722 +0000 2024-03-08T23:45:54.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T23:45:54.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:54 smithi192 ceph-mon[35995]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi062=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:54.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:54 smithi192 ceph-mon[35995]: Saving service mgr spec with placement smithi022=x;smithi062=y;smithi192=z;count:3 2024-03-08T23:45:54.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:54 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:54.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:54 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:54.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:54 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-08T23:45:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-08T23:45:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 9.6127e-05 s, 5.3 MB/s 2024-03-08T23:45:54.411 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T23:45:54.466 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-03-08T23:45:54.521 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-08 23:44:57.083835961 +0000 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-08 23:38:19.240195216 +0000 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-08 23:38:19.240195216 +0000 2024-03-08T23:45:54.522 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-08 23:38:19.240195216 +0000 2024-03-08T23:45:54.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T23:45:54.587 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-03-08T23:45:54.588 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-03-08T23:45:54.588 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 9.0894e-05 s, 5.6 MB/s 2024-03-08T23:45:54.589 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T23:45:54.644 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-08T23:45:54.644 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T23:45:54.663 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T23:45:54.663 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-08 23:45:39.218829445 +0000 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-08 23:38:46.513210825 +0000 2024-03-08T23:45:54.722 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-08 23:38:46.513210825 +0000 2024-03-08T23:45:54.723 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-08 23:38:46.513210825 +0000 2024-03-08T23:45:54.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T23:45:54.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:54 smithi022 ceph-mon[31894]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi062=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:54.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:54 smithi022 ceph-mon[31894]: Saving service mgr spec with placement smithi022=x;smithi062=y;smithi192=z;count:3 2024-03-08T23:45:54.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:54 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:54.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:54 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:54.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:54 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:54.792 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:54 smithi062 ceph-mon[36129]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi062=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:45:54.792 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:54 smithi062 ceph-mon[36129]: Saving service mgr spec with placement smithi022=x;smithi062=y;smithi192=z;count:3 2024-03-08T23:45:54.793 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:54 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:54.793 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:54 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:54.793 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:54 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:45:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-08T23:45:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-08T23:45:54.795 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000230863 s, 2.2 MB/s 2024-03-08T23:45:54.796 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T23:45:54.854 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-03-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:54.911 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-08 23:45:39.219829428 +0000 2024-03-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-08 23:38:46.858204970 +0000 2024-03-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-08 23:38:46.858204970 +0000 2024-03-08T23:45:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-08 23:38:46.858204970 +0000 2024-03-08T23:45:54.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T23:45:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-08T23:45:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-08T23:45:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00272015 s, 188 kB/s 2024-03-08T23:45:54.987 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T23:45:55.044 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-08 23:45:39.219829428 +0000 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-08 23:38:47.208199030 +0000 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-08 23:38:47.208199030 +0000 2024-03-08T23:45:55.101 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-08 23:38:47.208199030 +0000 2024-03-08T23:45:55.102 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T23:45:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-08T23:45:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-08T23:45:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00022139 s, 2.3 MB/s 2024-03-08T23:45:55.173 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T23:45:55.229 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-03-08T23:45:55.285 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T23:45:55.285 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:55.285 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-08T23:45:55.285 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:55.285 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:55.286 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-08 23:45:39.220829412 +0000 2024-03-08T23:45:55.286 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-08 23:38:47.554193158 +0000 2024-03-08T23:45:55.286 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-08 23:38:47.554193158 +0000 2024-03-08T23:45:55.286 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-08 23:38:47.554193158 +0000 2024-03-08T23:45:55.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T23:45:55.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-08T23:45:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-08T23:45:55.355 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000178593 s, 2.9 MB/s 2024-03-08T23:45:55.356 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T23:45:55.412 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-08T23:45:55.413 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T23:45:55.428 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T23:45:55.429 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-03-08T23:45:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T23:45:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:55.483 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-08T23:45:55.483 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:55.483 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:55.483 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-08 23:45:30.357698191 +0000 2024-03-08T23:45:55.484 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-08 23:39:36.626089380 +0000 2024-03-08T23:45:55.484 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-08 23:39:36.626089380 +0000 2024-03-08T23:45:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-08 23:39:36.626089380 +0000 2024-03-08T23:45:55.484 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T23:45:55.550 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-08T23:45:55.550 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-08T23:45:55.550 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000263121 s, 1.9 MB/s 2024-03-08T23:45:55.551 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:45:55.593 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:55.606 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-03-08T23:45:55.661 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T23:45:55.661 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:55.661 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-08T23:45:55.661 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:55.661 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:55.662 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-08 23:45:30.357698191 +0000 2024-03-08T23:45:55.662 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-08 23:39:36.979082477 +0000 2024-03-08T23:45:55.662 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-08 23:39:36.979082477 +0000 2024-03-08T23:45:55.662 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-08 23:39:36.979082477 +0000 2024-03-08T23:45:55.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T23:45:55.727 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-08T23:45:55.727 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-08T23:45:55.727 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000178552 s, 2.9 MB/s 2024-03-08T23:45:55.728 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T23:45:55.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:55.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:55.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:55.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:55.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:45:55.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T23:45:55.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:45:55.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:55.785 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-03-08T23:45:55.839 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-08 23:45:30.358698172 +0000 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-08 23:39:37.324075730 +0000 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-08 23:39:37.324075730 +0000 2024-03-08T23:45:55.840 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-08 23:39:37.324075730 +0000 2024-03-08T23:45:55.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:45:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:55.906 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-08T23:45:55.906 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-08T23:45:55.906 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000206817 s, 2.5 MB/s 2024-03-08T23:45:55.907 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T23:45:55.963 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-08 23:45:30.358698172 +0000 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-08 23:39:37.661069141 +0000 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-08 23:39:37.661069141 +0000 2024-03-08T23:45:56.019 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-08 23:39:37.661069141 +0000 2024-03-08T23:45:56.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T23:45:56.083 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-08T23:45:56.083 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-08T23:45:56.083 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000174595 s, 2.9 MB/s 2024-03-08T23:45:56.086 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T23:45:56.141 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/vg_nvme/lv_4... 2024-03-08T23:45:56.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-08T23:45:56.389 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:56 smithi022 ceph-mon[31894]: Deploying daemon mgr.z on smithi192 2024-03-08T23:45:56.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:56 smithi192 ceph-mon[35995]: Deploying daemon mgr.z on smithi192 2024-03-08T23:45:56.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:56 smithi062 ceph-mon[36129]: Deploying daemon mgr.z on smithi192 2024-03-08T23:45:57.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:57 smithi192 ceph-mon[35995]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:57.627 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:57 smithi192 ceph-mgr[38021]: -- 172.21.15.192:0/592206207 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55822a1645a0 con 0x55822a138c00 2024-03-08T23:45:57.627 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:57 smithi192 bash[38002]: 44bdf83cbba1eb68f6a986d2c1516afbe64e02f8ad0820f5d812c051db11f0e5 2024-03-08T23:45:57.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:57 smithi022 ceph-mon[31894]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:57.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:57 smithi062 ceph-mon[36129]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:57.882 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:57 smithi192 systemd[1]: Started Ceph mgr.z for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:45:58.225 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-08T23:45:58.225 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-08T23:45:58.225 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-08T23:45:58.225 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-08T23:45:58.225 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00828284 s, 1.3 GB/s 2024-03-08T23:45:58.225 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-08T23:45:58.382 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:57.967+0000 7f2fd8aac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:45:58.383 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:58.059+0000 7f2fd8aac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:45:58.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2024-03-08T23:45:58.807 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:58 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:58.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:58 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:58.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:58 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:58.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:58 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:45:58.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:58 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T23:45:58.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:58 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:45:58.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:58 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:58.692+0000 7f2fd8aac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:58.786+0000 7f2fd8aac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:58.928+0000 7f2fd8aac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:45:59.021 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T23:45:59.022 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:45:59.022 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:58 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:45:59.049 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:58 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:59.050 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:58 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:59.050 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:58 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:45:59.050 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:58 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:45:59.050 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:58 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T23:45:59.050 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:58 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:45:59.050 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:58 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:45:59.367 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:59.022+0000 7f2fd8aac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:45:59.368 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:59.104+0000 7f2fd8aac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:45:59.368 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:59.369+0000 7f2fd8aac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:45:59.632 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:59.478+0000 7f2fd8aac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:45:59.633 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:59.563+0000 7f2fd8aac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:45:59.961 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:59 smithi062 ceph-mon[36129]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:59.961 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:45:59 smithi062 ceph-mon[36129]: Deploying daemon mgr.y on smithi062 2024-03-08T23:45:59.961 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:45:59 smithi062 podman[37510]: 2024-03-08 23:45:59.867064727 +0000 UTC m=+0.018074169 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:45:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:59 smithi022 ceph-mon[31894]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:45:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:45:59 smithi022 ceph-mon[31894]: Deploying daemon mgr.y on smithi062 2024-03-08T23:46:00.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-mon[35995]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:00.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-mon[35995]: Deploying daemon mgr.y on smithi062 2024-03-08T23:46:00.133 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:59.657+0000 7f2fd8aac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:46:00.133 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:45:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:45:59.833+0000 7f2fd8aac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:46:00.261 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:00 smithi062 podman[37510]: 2024-03-08 23:46:00.181440479 +0000 UTC m=+0.332449933 container init 6914d87866581bc40829ae48c7214f31d713977b0fea3636e20a6cbaf7612357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-08T23:46:00.262 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:00 smithi062 podman[37510]: 2024-03-08 23:46:00.184202366 +0000 UTC m=+0.335211802 container start 6914d87866581bc40829ae48c7214f31d713977b0fea3636e20a6cbaf7612357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T23:46:00.262 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:00 smithi062 ceph-mgr[37529]: -- 172.21.15.62:0/420423390 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56166987a5a0 con 0x56166984ec00 2024-03-08T23:46:00.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:00 smithi062 bash[37510]: 6914d87866581bc40829ae48c7214f31d713977b0fea3636e20a6cbaf7612357 2024-03-08T23:46:00.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:00 smithi062 systemd[1]: Started Ceph mgr.y for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 -- 172.21.15.22:0/2255552593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 msgr2=0x7fa49c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/2255552593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c102220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00b0a0 tx=0x7fa48c00b3b0 comp rx=0 tx=0).stop 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 -- 172.21.15.22:0/2255552593 shutdown_connections 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/2255552593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa49c103920 0x7fa49c10a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/2255552593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c102220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/2255552593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa49c102f60 0x7fa49c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 -- 172.21.15.22:0/2255552593 >> 172.21.15.22:0/2255552593 conn(0x7fa49c0fd3c0 msgr2=0x7fa49c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 -- 172.21.15.22:0/2255552593 shutdown_connections 2024-03-08T23:46:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.628+0000 7fa4a2c82700 1 -- 172.21.15.22:0/2255552593 wait complete. 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 Processor -- start 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 -- start start 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa49c102f60 0x7fa49c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa49c103920 0x7fa49c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c10d9f0 con 0x7fa49c101e00 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c10d870 con 0x7fa49c102f60 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a2c82700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c10db70 con 0x7fa49c103920 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a0a1e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a0a1e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40432/0 (socket says 172.21.15.22:40432) 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a0a1e700 1 -- 172.21.15.22:0/918109004 learned_addr learned my addr 172.21.15.22:0/918109004 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa49bfff700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa49c102f60 0x7fa49c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:00.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.629+0000 7fa4a121f700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa49c103920 0x7fa49c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a0a1e700 1 -- 172.21.15.22:0/918109004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa49c103920 msgr2=0x7fa49c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a0a1e700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa49c103920 0x7fa49c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a0a1e700 1 -- 172.21.15.22:0/918109004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa49c102f60 msgr2=0x7fa49c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a0a1e700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa49c102f60 0x7fa49c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a0a1e700 1 -- 172.21.15.22:0/918109004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c19fc60 con 0x7fa49c101e00 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a0a1e700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c198000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa48c007d10 tx=0x7fa48c007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c021020 con 0x7fa49c101e00 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa48c01d950 con 0x7fa49c101e00 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c19fef0 con 0x7fa49c101e00 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0054f0 con 0x7fa49c101e00 2024-03-08T23:46:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.630+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1a0290 con 0x7fa49c101e00 2024-03-08T23:46:00.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.631+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fa48c024020 con 0x7fa49c101e00 2024-03-08T23:46:00.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.631+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c068aa0 con 0x7fa49c101e00 2024-03-08T23:46:00.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.631+0000 7fa499ffb700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa488039200 0x7fa48803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:00.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.631+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa48c00f020 con 0x7fa49c101e00 2024-03-08T23:46:00.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.631+0000 7fa49bfff700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa488039200 0x7fa48803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:00.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.632+0000 7fa49bfff700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa488039200 0x7fa48803b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa484004630 tx=0x7fa48400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:46:00.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.636+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa48c011be0 con 0x7fa49c101e00 2024-03-08T23:46:00.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:00.850+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa49c062220 con 0x7fa488039200 2024-03-08T23:46:01.113 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:00 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:00.761+0000 7f6b00fbf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:46:01.113 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:00 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:00.861+0000 7f6b00fbf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:46:01.132 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:00.819+0000 7f2fd8aac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:46:01.132 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:00.955+0000 7f2fd8aac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:46:01.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:01.559+0000 7f6b00fbf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:46:01.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:01.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:01.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:01.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:01.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:01.618 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:01.326+0000 7f2fd8aac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:01.420+0000 7f2fd8aac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:01.620+0000 7f2fd8aac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:01.619 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:01.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:01.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:01.882 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:01.809+0000 7f2fd8aac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:46:01.988 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:01.654+0000 7f6b00fbf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:46:01.988 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:01.805+0000 7f6b00fbf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:46:01.988 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:01.904+0000 7f6b00fbf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:46:01.988 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:01.990+0000 7f6b00fbf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:46:02.351 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:02.093+0000 7f2fd8aac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:46:02.351 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:02.353+0000 7f2fd8aac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.352 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:46:02.353 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:46:02.353 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:02.557 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:02.260+0000 7f6b00fbf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:46:02.557 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:02.373+0000 7f6b00fbf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:46:02.557 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:02.462+0000 7f6b00fbf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:46:02.557 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:02.559+0000 7f6b00fbf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:46:02.557 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:46:02.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:46:02.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:02.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:02.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T23:46:02.597 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:02.632 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:02.589+0000 7f2fd8aac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:46:02.863 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:02 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:02.739+0000 7f6b00fbf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:46:03.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:03 smithi062 ceph-mon[36129]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:03.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:03 smithi062 ceph-mon[36129]: Reconfiguring mgr.x (unknown last config time)... 2024-03-08T23:46:03.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:03 smithi062 ceph-mon[36129]: Reconfiguring daemon mgr.x on smithi022 2024-03-08T23:46:03.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:03 smithi192 ceph-mon[35995]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:03.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:03 smithi192 ceph-mon[35995]: Reconfiguring mgr.x (unknown last config time)... 2024-03-08T23:46:03.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:03 smithi192 ceph-mon[35995]: Reconfiguring daemon mgr.x on smithi022 2024-03-08T23:46:03.633 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:03.387+0000 7f2fd8aac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:46:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:03 smithi022 ceph-mon[31894]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:03 smithi022 ceph-mon[31894]: Reconfiguring mgr.x (unknown last config time)... 2024-03-08T23:46:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:03 smithi022 ceph-mon[31894]: Reconfiguring daemon mgr.x on smithi022 2024-03-08T23:46:04.113 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:03 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:03.767+0000 7f6b00fbf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:46:04.113 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:03 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:03.903+0000 7f6b00fbf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:46:04.271 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:03.960+0000 7f2fd8aac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:46:04.271 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:04.058+0000 7f2fd8aac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:46:04.271 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:46:04.273+0000 7f2fd8aac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:46:04.534 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:04.245+0000 7f6b00fbf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:46:04.534 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:04.337+0000 7f6b00fbf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:04.536+0000 7f6b00fbf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4224301161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bc68a9c-ec9a-4342-aa98-70c2835d2f06"}]: dispatch 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4224301161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bc68a9c-ec9a-4342-aa98-70c2835d2f06"}]': finished 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: osdmap e5: 1 total, 0 up, 1 in 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3467718405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: Standby manager daemon z started 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T23:46:04.535 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4224301161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bc68a9c-ec9a-4342-aa98-70c2835d2f06"}]: dispatch 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4224301161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bc68a9c-ec9a-4342-aa98-70c2835d2f06"}]': finished 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: osdmap e5: 1 total, 0 up, 1 in 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3467718405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: Standby manager daemon z started 2024-03-08T23:46:04.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T23:46:04.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:46:04.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T23:46:04.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:04 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:46:04.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4224301161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bc68a9c-ec9a-4342-aa98-70c2835d2f06"}]: dispatch 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4224301161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bc68a9c-ec9a-4342-aa98-70c2835d2f06"}]': finished 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: osdmap e5: 1 total, 0 up, 1 in 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3467718405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: Standby manager daemon z started 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T23:46:04.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:04 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/2293742169' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:46:04.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:04.778+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 14) v1 ==== 91698+0+0 (secure 0 0 0) 0x7fa48c01db00 con 0x7fa49c101e00 2024-03-08T23:46:04.863 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:04.725+0000 7f6b00fbf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:46:05.281 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:05 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:05.015+0000 7f6b00fbf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:46:05.281 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:05 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:05.283+0000 7f6b00fbf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:46:05.535 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:05 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:05.524+0000 7f6b00fbf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:46:05.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:05 smithi062 ceph-mon[36129]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:05.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:05 smithi062 ceph-mon[36129]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-08T23:46:05.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T23:46:05.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:05 smithi192 ceph-mon[35995]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:05.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:05 smithi192 ceph-mon[35995]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-08T23:46:05.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T23:46:05.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:05 smithi022 ceph-mon[31894]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:05 smithi022 ceph-mon[31894]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-08T23:46:05.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T23:46:06.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:06 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:06.359+0000 7f6b00fbf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:46:07.274 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:06 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:06.954+0000 7f6b00fbf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:46:07.274 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:07.055+0000 7f6b00fbf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:46:07.274 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:46:07.276+0000 7f6b00fbf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:46:07.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:07.549+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7fa48c01acc0 con 0x7fa49c101e00 2024-03-08T23:46:07.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:07 smithi022 ceph-mon[31894]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:07.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:07 smithi022 ceph-mon[31894]: Standby manager daemon y started 2024-03-08T23:46:07.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:07 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T23:46:07.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:07 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:46:07.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:07 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T23:46:07.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:07 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:46:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-mon[36129]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-mon[36129]: Standby manager daemon y started 2024-03-08T23:46:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T23:46:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:46:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T23:46:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:07 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:46:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:07 smithi192 ceph-mon[35995]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:07 smithi192 ceph-mon[35995]: Standby manager daemon y started 2024-03-08T23:46:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:07 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T23:46:07.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:07 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:46:07.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:07 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T23:46:07.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:07 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/603983010' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:46:08.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:08 smithi062 ceph-mon[36129]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-08T23:46:08.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:08 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T23:46:08.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:08 smithi192 ceph-mon[35995]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-08T23:46:08.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:08 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T23:46:08.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:08 smithi022 ceph-mon[31894]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-08T23:46:08.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:08 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T23:46:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:09 smithi062 ceph-mon[36129]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:09 smithi192 ceph-mon[35995]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:09.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:09.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:09.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:09 smithi022 ceph-mon[31894]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:09.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:09.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:09.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:09.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:09.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:11.598 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:11 smithi022 ceph-mon[31894]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:11.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:11 smithi062 ceph-mon[36129]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:11.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:11 smithi192 ceph-mon[35995]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-08T23:46:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:12 smithi062 ceph-mon[36129]: Deploying daemon osd.0 on smithi022 2024-03-08T23:46:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-08T23:46:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:12 smithi192 ceph-mon[35995]: Deploying daemon osd.0 on smithi022 2024-03-08T23:46:12.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-08T23:46:12.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:12.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:12 smithi022 ceph-mon[31894]: Deploying daemon osd.0 on smithi022 2024-03-08T23:46:13.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:13 smithi022 ceph-mon[31894]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:13 smithi062 ceph-mon[36129]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:13 smithi192 ceph-mon[35995]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:15 smithi062 ceph-mon[36129]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:15.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:15 smithi192 ceph-mon[35995]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:15.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:15 smithi022 ceph-mon[31894]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:17.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:17 smithi022 ceph-mon[31894]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:17.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:17.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:17.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:17 smithi062 ceph-mon[36129]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:17 smithi192 ceph-mon[35995]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:19.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:19 smithi062 ceph-mon[36129]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:19 smithi192 ceph-mon[35995]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:19 smithi022 ceph-mon[31894]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:20.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.714+0000 7fa499ffb700 1 -- 172.21.15.22:0/918109004 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa49c062220 con 0x7fa488039200 2024-03-08T23:46:20.713 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2024-03-08T23:46:20.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa488039200 msgr2=0x7fa48803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:20.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa488039200 0x7fa48803b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa484004630 tx=0x7fa48400b3c0 comp rx=0 tx=0).stop 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 msgr2=0x7fa49c198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c198000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa48c007d10 tx=0x7fa48c007d40 comp rx=0 tx=0).stop 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 shutdown_connections 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa49c103920 0x7fa49c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa488039200 0x7fa48803b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c101e00 0x7fa49c198000 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 --2- 172.21.15.22:0/918109004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa49c102f60 0x7fa49c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.716+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 >> 172.21.15.22:0/918109004 conn(0x7fa49c0fd3c0 msgr2=0x7fa49c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.717+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 shutdown_connections 2024-03-08T23:46:20.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:46:20.717+0000 7fa4a2c82700 1 -- 172.21.15.22:0/918109004 wait complete. 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:20.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:21.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:21.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:21.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:21.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:21.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:21.377 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.0.service 2024-03-08T23:46:21.420 INFO:tasks.cephadm:Deploying osd.1 on smithi062 with /dev/vg_nvme/lv_4... 2024-03-08T23:46:21.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-08T23:46:21.675 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:46:21.930 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:21 smithi022 ceph-mon[31894]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-08T23:46:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:21 smithi062 ceph-mon[36129]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-08T23:46:22.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:21 smithi192 ceph-mon[35995]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-08T23:46:22.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:22 smithi022 ceph-mon[31894]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:22.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:22 smithi022 ceph-mon[31894]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-08T23:46:22.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:22 smithi022 ceph-mon[31894]: osdmap e6: 1 total, 0 up, 1 in 2024-03-08T23:46:22.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:22 smithi022 ceph-mon[31894]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:46:22.864 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:22 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:23.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:22 smithi062 ceph-mon[36129]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:23.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:22 smithi062 ceph-mon[36129]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-08T23:46:23.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:22 smithi062 ceph-mon[36129]: osdmap e6: 1 total, 0 up, 1 in 2024-03-08T23:46:23.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:22 smithi062 ceph-mon[36129]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:46:23.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:22 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:23.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:22 smithi192 ceph-mon[35995]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:23.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:22 smithi192 ceph-mon[35995]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-08T23:46:23.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:22 smithi192 ceph-mon[35995]: osdmap e6: 1 total, 0 up, 1 in 2024-03-08T23:46:23.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:22 smithi192 ceph-mon[35995]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:46:23.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:22 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:23.695 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-08T23:46:23.695 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-08T23:46:23.695 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-08T23:46:23.695 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-08T23:46:23.695 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113598 s, 923 MB/s 2024-03-08T23:46:23.696 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-08T23:46:23.737 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:46:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0[46570]: 2024-03-08T23:46:23.393+0000 7efd13759700 -1 osd.0 0 waiting for initial osdmap 2024-03-08T23:46:23.737 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:46:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0[46570]: 2024-03-08T23:46:23.397+0000 7efd0b544700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:46:24.018 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:23 smithi062 ceph-mon[36129]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:46:24.018 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:23 smithi062 ceph-mon[36129]: osdmap e7: 1 total, 0 up, 1 in 2024-03-08T23:46:24.018 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.018 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.018 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:23 smithi062 ceph-mon[36129]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' 2024-03-08T23:46:24.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:23 smithi022 ceph-mon[31894]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:46:24.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:23 smithi022 ceph-mon[31894]: osdmap e7: 1 total, 0 up, 1 in 2024-03-08T23:46:24.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:23 smithi022 ceph-mon[31894]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' 2024-03-08T23:46:24.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:23 smithi192 ceph-mon[35995]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:46:24.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:23 smithi192 ceph-mon[35995]: osdmap e7: 1 total, 0 up, 1 in 2024-03-08T23:46:24.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:23 smithi192 ceph-mon[35995]: from='osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658]' entity='osd.0' 2024-03-08T23:46:24.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-03-08T23:46:24.554 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:46:24.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:46:24.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:46:24.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi022 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] boot 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: osdmap e8: 1 total, 1 up, 1 in 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi022 to 17093M 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi022 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] boot 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: osdmap e8: 1 total, 1 up, 1 in 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi022 to 17093M 2024-03-08T23:46:25.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:25.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:25.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi022 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: osd.0 [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] boot 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: osdmap e8: 1 total, 1 up, 1 in 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:46:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi022 to 17093M 2024-03-08T23:46:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:25.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 -- 172.21.15.62:0/3256540554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 msgr2=0x7f236c109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 --2- 172.21.15.62:0/3256540554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c109160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f236400b0d0 tx=0x7f236400b3e0 comp rx=0 tx=0).stop 2024-03-08T23:46:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 -- 172.21.15.62:0/3256540554 shutdown_connections 2024-03-08T23:46:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 --2- 172.21.15.62:0/3256540554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c109160 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 --2- 172.21.15.62:0/3256540554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c100d10 0x7f236c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 --2- 172.21.15.62:0/3256540554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f236c101f10 0x7f236c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 -- 172.21.15.62:0/3256540554 >> 172.21.15.62:0/3256540554 conn(0x7f236c0fc2b0 msgr2=0x7f236c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 -- 172.21.15.62:0/3256540554 shutdown_connections 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.062+0000 7f23722e6700 1 -- 172.21.15.62:0/3256540554 wait complete. 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 Processor -- start 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 -- start start 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f236c100d10 0x7f236c196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c101f10 0x7f236c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c10c900 con 0x7f236c101f10 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c10c780 con 0x7f236c1028b0 2024-03-08T23:46:26.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f23722e6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c10ca80 con 0x7f236c100d10 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46740/0 (socket says 172.21.15.62:46740) 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 -- 172.21.15.62:0/27337619 learned_addr learned my addr 172.21.15.62:0/27337619 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f236bfff700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f236c100d10 0x7f236c196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2363fff700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c101f10 0x7f236c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 -- 172.21.15.62:0/27337619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f236c100d10 msgr2=0x7f236c196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f236c100d10 0x7f236c196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 -- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c101f10 msgr2=0x7f236c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c101f10 0x7f236c197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.063+0000 7f2370883700 1 -- 172.21.15.62:0/27337619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c19eb70 con 0x7f236c1028b0 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.064+0000 7f2370883700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c19e4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2364006070 tx=0x7f2364007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:46:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.064+0000 7f2369ffb700 1 -- 172.21.15.62:0/27337619 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364021020 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.064+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c19ee00 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.064+0000 7f2369ffb700 1 -- 172.21.15.62:0/27337619 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f236401d950 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.064+0000 7f2369ffb700 1 -- 172.21.15.62:0/27337619 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364005450 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.064+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c19f260 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.065+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c068aa0 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.066+0000 7f2369ffb700 1 -- 172.21.15.62:0/27337619 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f2364024020 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.066+0000 7f2369ffb700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23580a2800 0x7f23580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.067+0000 7f2369ffb700 1 -- 172.21.15.62:0/27337619 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f23640ce3a0 con 0x7f236c1028b0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.067+0000 7f236bfff700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23580a2800 0x7f23580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:26.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.067+0000 7f236bfff700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23580a2800 0x7f23580a4cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f236c101d70 tx=0x7f235c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:46:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.071+0000 7f2369ffb700 1 -- 172.21.15.62:0/27337619 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f236402b020 con 0x7f236c1028b0 2024-03-08T23:46:26.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:26.286+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f236c062220 con 0x7f23580a2800 2024-03-08T23:46:27.089 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:26 smithi062 ceph-mon[36129]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:27.090 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:26 smithi062 ceph-mon[36129]: osdmap e9: 1 total, 1 up, 1 in 2024-03-08T23:46:27.090 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:26 smithi062 ceph-mon[36129]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:27.090 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:27.090 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:27.090 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:27.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:26 smithi192 ceph-mon[35995]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:27.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:26 smithi192 ceph-mon[35995]: osdmap e9: 1 total, 1 up, 1 in 2024-03-08T23:46:27.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:26 smithi192 ceph-mon[35995]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:27.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:27.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:27.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:27.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:26 smithi022 ceph-mon[31894]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:27.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:26 smithi022 ceph-mon[31894]: osdmap e9: 1 total, 1 up, 1 in 2024-03-08T23:46:27.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:26 smithi022 ceph-mon[31894]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:27.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:27.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:27.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:29.107 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:28 smithi062 ceph-mon[36129]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:29.107 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:28 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/4163463635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]: dispatch 2024-03-08T23:46:29.108 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:28 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]: dispatch 2024-03-08T23:46:29.108 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:28 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]': finished 2024-03-08T23:46:29.108 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:28 smithi062 ceph-mon[36129]: osdmap e10: 2 total, 1 up, 2 in 2024-03-08T23:46:29.108 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:28 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:29.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:28 smithi192 ceph-mon[35995]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:29.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:28 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/4163463635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]: dispatch 2024-03-08T23:46:29.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:28 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]: dispatch 2024-03-08T23:46:29.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:28 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]': finished 2024-03-08T23:46:29.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:28 smithi192 ceph-mon[35995]: osdmap e10: 2 total, 1 up, 2 in 2024-03-08T23:46:29.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:28 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:29.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:28 smithi022 ceph-mon[31894]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:29.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:28 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/4163463635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]: dispatch 2024-03-08T23:46:29.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:28 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]: dispatch 2024-03-08T23:46:29.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:28 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06bad818-44be-4c08-a916-5e7cf0548753"}]': finished 2024-03-08T23:46:29.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:28 smithi022 ceph-mon[31894]: osdmap e10: 2 total, 1 up, 2 in 2024-03-08T23:46:29.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:28 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:30.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:29 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/852249554' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:30.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:29 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/852249554' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:30.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:29 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/852249554' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:31.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:30 smithi062 ceph-mon[36129]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:31.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:30 smithi192 ceph-mon[35995]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:31.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:30 smithi022 ceph-mon[31894]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:33.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:32 smithi062 ceph-mon[36129]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:33.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:32 smithi192 ceph-mon[35995]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:33.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:32 smithi022 ceph-mon[31894]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:34.933 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:34 smithi062 ceph-mon[36129]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:34 smithi192 ceph-mon[35995]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:34 smithi022 ceph-mon[31894]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:37.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:36 smithi062 ceph-mon[36129]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:37.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:36 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-08T23:46:37.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:36 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:37.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:36 smithi062 ceph-mon[36129]: Deploying daemon osd.1 on smithi062 2024-03-08T23:46:37.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:36 smithi192 ceph-mon[35995]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:37.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-08T23:46:37.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:37.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:36 smithi192 ceph-mon[35995]: Deploying daemon osd.1 on smithi062 2024-03-08T23:46:37.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:36 smithi022 ceph-mon[31894]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:37.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-08T23:46:37.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:37.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:36 smithi022 ceph-mon[31894]: Deploying daemon osd.1 on smithi062 2024-03-08T23:46:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:38 smithi062 ceph-mon[36129]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:38 smithi192 ceph-mon[35995]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:39.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:38 smithi022 ceph-mon[31894]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:41.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:40 smithi192 ceph-mon[35995]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:41.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:40 smithi022 ceph-mon[31894]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:41.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:40 smithi062 ceph-mon[36129]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:42.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:42 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:42.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:42 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:42.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:42 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:42.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:42 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:42.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:42 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:42.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:42 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:42.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:42 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:42.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:42 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:42.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:42 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:43.218 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:43 smithi062 ceph-mon[36129]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:43.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:43 smithi192 ceph-mon[35995]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:43 smithi022 ceph-mon[31894]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.398+0000 7f2369ffb700 1 -- 172.21.15.62:0/27337619 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f236c062220 con 0x7f23580a2800 2024-03-08T23:46:44.397 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 1 on host 'smithi062' 2024-03-08T23:46:44.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23580a2800 msgr2=0x7f23580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:44.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23580a2800 0x7f23580a4cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f236c101d70 tx=0x7f235c00a380 comp rx=0 tx=0).stop 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 msgr2=0x7f236c19e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c19e4d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2364006070 tx=0x7f2364007d90 comp rx=0 tx=0).stop 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 shutdown_connections 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236c1028b0 0x7f236c19e4d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f236c100d10 0x7f236c196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23580a2800 0x7f23580a4cc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 --2- 172.21.15.62:0/27337619 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f236c101f10 0x7f236c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 >> 172.21.15.62:0/27337619 conn(0x7f236c0fc2b0 msgr2=0x7f236c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 shutdown_connections 2024-03-08T23:46:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:46:44.400+0000 7f23722e6700 1 -- 172.21.15.62:0/27337619 wait complete. 2024-03-08T23:46:44.944 DEBUG:teuthology.orchestra.run.smithi062:osd.1> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.1.service 2024-03-08T23:46:44.946 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2024-03-08T23:46:44.946 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-08T23:46:45.192 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:46:45.362 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:46:44 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1[40734]: 2024-03-08T23:46:44.996+0000 7f114e9dc7c0 -1 osd.1 0 log_to_monitors true 2024-03-08T23:46:45.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:45 smithi192 ceph-mon[35995]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:45.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:45 smithi192 ceph-mon[35995]: from='osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T23:46:45.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:45 smithi192 ceph-mon[35995]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T23:46:45.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.636 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:45 smithi062 ceph-mon[36129]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:45.636 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.636 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.636 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:45 smithi062 ceph-mon[36129]: from='osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T23:46:45.636 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:45 smithi062 ceph-mon[36129]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T23:46:45.636 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.636 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:45 smithi022 ceph-mon[31894]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:45.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:45 smithi022 ceph-mon[31894]: from='osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T23:46:45.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:45 smithi022 ceph-mon[31894]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T23:46:45.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:45.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:46.500 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:46 smithi192 ceph-mon[35995]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-08T23:46:46.500 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:46 smithi192 ceph-mon[35995]: from='osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:46:46.500 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:46 smithi192 ceph-mon[35995]: osdmap e11: 2 total, 1 up, 2 in 2024-03-08T23:46:46.501 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:46 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:46.501 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:46 smithi192 ceph-mon[35995]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:46:46.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:46 smithi062 ceph-mon[36129]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-08T23:46:46.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:46 smithi062 ceph-mon[36129]: from='osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:46:46.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:46 smithi062 ceph-mon[36129]: osdmap e11: 2 total, 1 up, 2 in 2024-03-08T23:46:46.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:46.734 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:46 smithi062 ceph-mon[36129]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:46:46.734 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:46:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1[40734]: 2024-03-08T23:46:46.385+0000 7f1144041700 -1 osd.1 0 waiting for initial osdmap 2024-03-08T23:46:46.734 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:46:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1[40734]: 2024-03-08T23:46:46.388+0000 7f113fe34700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:46:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:46 smithi022 ceph-mon[31894]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-08T23:46:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:46 smithi022 ceph-mon[31894]: from='osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:46:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:46 smithi022 ceph-mon[31894]: osdmap e11: 2 total, 1 up, 2 in 2024-03-08T23:46:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:46 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:46 smithi022 ceph-mon[31894]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:46:47.110 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-08T23:46:47.111 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-08T23:46:47.111 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-08T23:46:47.111 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-08T23:46:47.111 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112315 s, 934 MB/s 2024-03-08T23:46:47.111 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-08T23:46:47.716 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:47 smithi192 ceph-mon[35995]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:47.716 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:47 smithi192 ceph-mon[35995]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:46:47.716 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:47 smithi192 ceph-mon[35995]: osdmap e12: 2 total, 1 up, 2 in 2024-03-08T23:46:47.716 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:47.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:47 smithi022 ceph-mon[31894]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:47.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:47 smithi022 ceph-mon[31894]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:46:47.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:47 smithi022 ceph-mon[31894]: osdmap e12: 2 total, 1 up, 2 in 2024-03-08T23:46:47.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:47.738 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:47 smithi062 ceph-mon[36129]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:47.738 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:47 smithi062 ceph-mon[36129]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:46:47.738 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:47 smithi062 ceph-mon[36129]: osdmap e12: 2 total, 1 up, 2 in 2024-03-08T23:46:47.738 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:47.743 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-03-08T23:46:47.987 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:46:48.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:48 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:46:48.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:48 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:46:48.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:48 smithi192 ceph-mon[35995]: osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] boot 2024-03-08T23:46:48.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:48 smithi192 ceph-mon[35995]: osdmap e13: 2 total, 2 up, 2 in 2024-03-08T23:46:48.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:48 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:48.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:48 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:46:48.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:48 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:46:48.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:48 smithi022 ceph-mon[31894]: osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] boot 2024-03-08T23:46:48.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:48 smithi022 ceph-mon[31894]: osdmap e13: 2 total, 2 up, 2 in 2024-03-08T23:46:48.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:48 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:48.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:48 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:46:48.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:48 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:46:48.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:48 smithi062 ceph-mon[36129]: osd.1 [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] boot 2024-03-08T23:46:48.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:48 smithi062 ceph-mon[36129]: osdmap e13: 2 total, 2 up, 2 in 2024-03-08T23:46:48.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:48 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:46:49.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 -- 172.21.15.192:0/4135419075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104101f00 msgr2=0x7f1104102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 --2- 172.21.15.192:0/4135419075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104101f00 0x7f1104102360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009990 tx=0x7f10f8009ca0 comp rx=0 tx=0).stop 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 -- 172.21.15.192:0/4135419075 shutdown_connections 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 --2- 172.21.15.192:0/4135419075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11041028a0 0x7f1104109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 --2- 172.21.15.192:0/4135419075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104100d00 0x7f1104101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 --2- 172.21.15.192:0/4135419075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104101f00 0x7f1104102360 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 -- 172.21.15.192:0/4135419075 >> 172.21.15.192:0/4135419075 conn(0x7f11040fc2a0 msgr2=0x7f11040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 -- 172.21.15.192:0/4135419075 shutdown_connections 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 -- 172.21.15.192:0/4135419075 wait complete. 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 Processor -- start 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 -- start start 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.482+0000 7f110a77d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104100d00 0x7f110406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:49.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f110a77d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1104101f00 0x7f110419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f110a77d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11041028a0 0x7f110419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f110a77d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110410c8e0 con 0x7f1104101f00 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f110a77d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110410c760 con 0x7f11041028a0 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f110a77d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110410ca60 con 0x7f1104100d00 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104100d00 0x7f110406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104100d00 0x7f110406aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54034/0 (socket says 172.21.15.192:54034) 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 -- 172.21.15.192:0/1142501468 learned_addr learned my addr 172.21.15.192:0/1142501468 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1108d1a700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11041028a0 0x7f110419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f11037fe700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1104101f00 0x7f110419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 -- 172.21.15.192:0/1142501468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11041028a0 msgr2=0x7f110419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11041028a0 0x7f110419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 -- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1104101f00 msgr2=0x7f110419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1104101f00 0x7f110419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:46:49.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 -- 172.21.15.192:0/1142501468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110419fc50 con 0x7f1104100d00 2024-03-08T23:46:49.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f11037fe700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1104101f00 0x7f110419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:46:49.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f1103fff700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104100d00 0x7f110406aee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c8f0 tx=0x7f10f400cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:46:49.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f11017fa700 1 -- 172.21.15.192:0/1142501468 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4004220 con 0x7f1104100d00 2024-03-08T23:46:49.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f11017fa700 1 -- 172.21.15.192:0/1142501468 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10f40043c0 con 0x7f1104100d00 2024-03-08T23:46:49.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110419ff40 con 0x7f1104100d00 2024-03-08T23:46:49.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f11017fa700 1 -- 172.21.15.192:0/1142501468 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4003d40 con 0x7f1104100d00 2024-03-08T23:46:49.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.483+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11041a0340 con 0x7f1104100d00 2024-03-08T23:46:49.485 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.484+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110402d830 con 0x7f1104100d00 2024-03-08T23:46:49.485 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.485+0000 7f11017fa700 1 -- 172.21.15.192:0/1142501468 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f10f4007740 con 0x7f1104100d00 2024-03-08T23:46:49.485 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.486+0000 7f11017fa700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10ec0a2800 0x7f10ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:46:49.485 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.486+0000 7f11017fa700 1 -- 172.21.15.192:0/1142501468 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f10f40cd150 con 0x7f1104100d00 2024-03-08T23:46:49.485 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.486+0000 7f11037fe700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10ec0a2800 0x7f10ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:46:49.485 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.487+0000 7f11037fe700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10ec0a2800 0x7f10ec0a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b5c0 tx=0x7f10f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:46:49.487 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.489+0000 7f11017fa700 1 -- 172.21.15.192:0/1142501468 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f10f4014030 con 0x7f1104100d00 2024-03-08T23:46:49.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: osdmap e14: 2 total, 2 up, 2 in 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi062 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi062 to 17080M 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:49.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:49.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:49.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:49.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:49.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.688 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:46:49.690+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1104062220 con 0x7f10ec0a2800 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: osdmap e14: 2 total, 2 up, 2 in 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi062 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi062 to 17080M 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:49.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:49.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:49.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:49.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-08T23:46:49.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: osdmap e14: 2 total, 2 up, 2 in 2024-03-08T23:46:49.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi062 2024-03-08T23:46:49.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi062 to 17080M 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:46:49.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:46:50.595 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:50 smithi192 ceph-mon[35995]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:50.596 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:50.596 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:50.596 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:50.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:50 smithi022 ceph-mon[31894]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:50.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:50.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:50.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:50.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:50 smithi062 ceph-mon[36129]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:46:50.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:46:50.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:46:50.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:51.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:51 smithi192 ceph-mon[35995]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:51.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:51 smithi022 ceph-mon[31894]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:51.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:51 smithi062 ceph-mon[36129]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:52.683 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:52 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/1833822534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]: dispatch 2024-03-08T23:46:52.683 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:52 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]: dispatch 2024-03-08T23:46:52.683 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:52 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]': finished 2024-03-08T23:46:52.683 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:52 smithi192 ceph-mon[35995]: osdmap e15: 3 total, 2 up, 3 in 2024-03-08T23:46:52.683 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:52 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:46:52.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:52 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/1833822534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]: dispatch 2024-03-08T23:46:52.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:52 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]: dispatch 2024-03-08T23:46:52.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:52 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]': finished 2024-03-08T23:46:52.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:52 smithi022 ceph-mon[31894]: osdmap e15: 3 total, 2 up, 3 in 2024-03-08T23:46:52.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:52 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:46:52.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:52 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/1833822534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]: dispatch 2024-03-08T23:46:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:52 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]: dispatch 2024-03-08T23:46:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:52 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5674ba8-0f77-4cca-baa2-8d27058e2259"}]': finished 2024-03-08T23:46:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:52 smithi062 ceph-mon[36129]: osdmap e15: 3 total, 2 up, 3 in 2024-03-08T23:46:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:52 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:46:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:53 smithi022 ceph-mon[31894]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:53 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/1337116480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:53 smithi062 ceph-mon[36129]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:53 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/1337116480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:53 smithi192 ceph-mon[35995]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:53 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/1337116480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:46:55.665 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:55 smithi062 ceph-mon[36129]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:55.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:55 smithi192 ceph-mon[35995]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:55.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:55 smithi022 ceph-mon[31894]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:57.485 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:57 smithi192 ceph-mon[35995]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:57.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:57 smithi022 ceph-mon[31894]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:57.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:57 smithi062 ceph-mon[36129]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:59.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:59 smithi192 ceph-mon[35995]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:59.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-08T23:46:59.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:46:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:59.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:59 smithi022 ceph-mon[31894]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:59.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-08T23:46:59.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:46:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:46:59.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:59 smithi062 ceph-mon[36129]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:46:59.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-08T23:46:59.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:46:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:00.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:00 smithi192 ceph-mon[35995]: Deploying daemon osd.2 on smithi192 2024-03-08T23:47:00.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:00 smithi022 ceph-mon[31894]: Deploying daemon osd.2 on smithi192 2024-03-08T23:47:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:00 smithi062 ceph-mon[36129]: Deploying daemon osd.2 on smithi192 2024-03-08T23:47:01.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:01 smithi192 ceph-mon[35995]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:01.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:01 smithi022 ceph-mon[31894]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:01.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:01 smithi062 ceph-mon[36129]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:03 smithi022 ceph-mon[31894]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:03.851 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:03 smithi192 ceph-mon[35995]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:03 smithi062 ceph-mon[36129]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:05.977 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:05 smithi192 ceph-mon[35995]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:05.978 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:05.978 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:05.978 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:05.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:05 smithi022 ceph-mon[31894]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:05.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:05.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:05.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:06.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:05 smithi062 ceph-mon[36129]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:06.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:06.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:06.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:07.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:07 smithi022 ceph-mon[31894]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:07.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:07.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:07.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:07.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:07.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:08.036 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:07 smithi192 ceph-mon[35995]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:08.037 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:08.037 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:08.037 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:08.037 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:08.037 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:07 smithi062 ceph-mon[36129]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:08.403 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.405+0000 7f11017fa700 1 -- 172.21.15.192:0/1142501468 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1104062220 con 0x7f10ec0a2800 2024-03-08T23:47:08.403 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2024-03-08T23:47:08.405 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10ec0a2800 msgr2=0x7f10ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10ec0a2800 0x7f10ec0a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b5c0 tx=0x7f10f8005fb0 comp rx=0 tx=0).stop 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104100d00 msgr2=0x7f110406aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104100d00 0x7f110406aee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c8f0 tx=0x7f10f400cc00 comp rx=0 tx=0).stop 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 shutdown_connections 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11041028a0 0x7f110419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1104100d00 0x7f110406aee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10ec0a2800 0x7f10ec0a4cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 --2- 172.21.15.192:0/1142501468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1104101f00 0x7f110419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 >> 172.21.15.192:0/1142501468 conn(0x7f11040fc2a0 msgr2=0x7f1104105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 shutdown_connections 2024-03-08T23:47:08.406 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:47:08.408+0000 7f110a77d700 1 -- 172.21.15.192:0/1142501468 wait complete. 2024-03-08T23:47:09.084 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.2.service 2024-03-08T23:47:09.086 INFO:tasks.cephadm:Deploying osd.3 on smithi022 with /dev/vg_nvme/lv_3... 2024-03-08T23:47:09.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-08T23:47:09.336 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:47:09.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:09 smithi022 ceph-mon[31894]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:09.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:09.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:09.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:09.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:09 smithi022 ceph-mon[31894]: from='osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T23:47:09.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:09 smithi022 ceph-mon[31894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T23:47:09.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:09 smithi062 ceph-mon[36129]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:09.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:09.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:09.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:09.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:09 smithi062 ceph-mon[36129]: from='osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T23:47:09.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:09 smithi062 ceph-mon[36129]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T23:47:09.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:09 smithi192 ceph-mon[35995]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:09.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:09.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:09.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:09.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:09 smithi192 ceph-mon[35995]: from='osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T23:47:09.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:09 smithi192 ceph-mon[35995]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T23:47:10.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:10 smithi192 ceph-mon[35995]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-08T23:47:10.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:10 smithi192 ceph-mon[35995]: osdmap e16: 3 total, 2 up, 3 in 2024-03-08T23:47:10.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:10 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:10.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:10 smithi192 ceph-mon[35995]: from='osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:47:10.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:10 smithi192 ceph-mon[35995]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:47:10.633 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:10 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2[41283]: 2024-03-08T23:47:10.382+0000 7fd3b966a700 -1 osd.2 0 waiting for initial osdmap 2024-03-08T23:47:10.633 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:10 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2[41283]: 2024-03-08T23:47:10.386+0000 7fd3b545d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:47:10.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:10 smithi022 ceph-mon[31894]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-08T23:47:10.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:10 smithi022 ceph-mon[31894]: osdmap e16: 3 total, 2 up, 3 in 2024-03-08T23:47:10.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:10 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:10.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:10 smithi022 ceph-mon[31894]: from='osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:47:10.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:10 smithi022 ceph-mon[31894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:47:10.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:10 smithi062 ceph-mon[36129]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-08T23:47:10.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:10 smithi062 ceph-mon[36129]: osdmap e16: 3 total, 2 up, 3 in 2024-03-08T23:47:10.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:10 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:10.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:10 smithi062 ceph-mon[36129]: from='osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:47:10.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:10 smithi062 ceph-mon[36129]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:47:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-08T23:47:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-08T23:47:11.409 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-08T23:47:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-08T23:47:11.409 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0085133 s, 1.2 GB/s 2024-03-08T23:47:11.409 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-08T23:47:11.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:11 smithi192 ceph-mon[35995]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:11.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:11 smithi192 ceph-mon[35995]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:47:11.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:11 smithi192 ceph-mon[35995]: osdmap e17: 3 total, 2 up, 3 in 2024-03-08T23:47:11.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:11 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:11.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:11 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:11 smithi022 ceph-mon[31894]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:11 smithi022 ceph-mon[31894]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:47:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:11 smithi022 ceph-mon[31894]: osdmap e17: 3 total, 2 up, 3 in 2024-03-08T23:47:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:11 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:11 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:11.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:11 smithi062 ceph-mon[36129]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-08T23:47:11.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:11 smithi062 ceph-mon[36129]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:47:11.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:11 smithi062 ceph-mon[36129]: osdmap e17: 3 total, 2 up, 3 in 2024-03-08T23:47:11.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:11 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:11.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:11 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:12.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2024-03-08T23:47:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:47:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:47:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:47:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] boot 2024-03-08T23:47:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: osdmap e18: 3 total, 3 up, 3 in 2024-03-08T23:47:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:12.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-08T23:47:12.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:12.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:12.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:12.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:47:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:47:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] boot 2024-03-08T23:47:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: osdmap e18: 3 total, 3 up, 3 in 2024-03-08T23:47:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-08T23:47:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:12.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:12.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:12.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:47:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:47:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: osd.2 [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] boot 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: osdmap e18: 3 total, 3 up, 3 in 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:12.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:13.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:13 smithi022 ceph-mon[31894]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:13.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:13 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi192 2024-03-08T23:47:13.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:13 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi192 to 17080M 2024-03-08T23:47:13.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-08T23:47:13.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:13 smithi022 ceph-mon[31894]: osdmap e19: 3 total, 3 up, 3 in 2024-03-08T23:47:13.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-08T23:47:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:13 smithi062 ceph-mon[36129]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:13 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi192 2024-03-08T23:47:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:13 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi192 to 17080M 2024-03-08T23:47:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-08T23:47:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:13 smithi062 ceph-mon[36129]: osdmap e19: 3 total, 3 up, 3 in 2024-03-08T23:47:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-08T23:47:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:13 smithi192 ceph-mon[35995]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:13 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi192 2024-03-08T23:47:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:13 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi192 to 17080M 2024-03-08T23:47:13.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-08T23:47:13.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:13 smithi192 ceph-mon[35995]: osdmap e19: 3 total, 3 up, 3 in 2024-03-08T23:47:13.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-08T23:47:13.911 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:13 smithi022 sudo[51084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T23:47:13.911 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:13 smithi022 sudo[51084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:13.911 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:13 smithi022 sudo[51084]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:13.911 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:13 smithi022 sudo[51084]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1214372426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0107ba0 msgr2=0x7f9ac0109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1214372426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0107ba0 0x7f9ac0109f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800b0a0 tx=0x7f9ab800b3b0 comp rx=0 tx=0).stop 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1214372426 shutdown_connections 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1214372426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0107ba0 0x7f9ac0109f90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1214372426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ac0102940 0x7f9ac0104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1214372426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac0105270 0x7f9ac0107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1214372426 >> 172.21.15.22:0/1214372426 conn(0x7f9ac00fc2b0 msgr2=0x7f9ac00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1214372426 shutdown_connections 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.912+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1214372426 wait complete. 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 Processor -- start 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 -- start start 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0102940 0x7f9ac01022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac0105270 0x7f9ac01008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ac0107ba0 0x7f9ac0100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac010d9e0 con 0x7f9ac0102940 2024-03-08T23:47:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac010d860 con 0x7f9ac0105270 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.913+0000 7f9ac7a64700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac010db60 con 0x7f9ac0107ba0 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0102940 0x7f9ac01022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0102940 0x7f9ac01022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33198/0 (socket says 172.21.15.22:33198) 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 -- 172.21.15.22:0/1409696154 learned_addr learned my addr 172.21.15.22:0/1409696154 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac4fff700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac0105270 0x7f9ac01008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac6001700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ac0107ba0 0x7f9ac0100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 -- 172.21.15.22:0/1409696154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ac0107ba0 msgr2=0x7f9ac0100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ac0107ba0 0x7f9ac0100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 -- 172.21.15.22:0/1409696154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac0105270 msgr2=0x7f9ac01008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac0105270 0x7f9ac01008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 -- 172.21.15.22:0/1409696154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac01016e0 con 0x7f9ac0102940 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac5800700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0102940 0x7f9ac01022a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9c0 tx=0x7f9abc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00cbd0 con 0x7f9ac0102940 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac01a2150 con 0x7f9ac0102940 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9abc00ee60 con 0x7f9ac0102940 2024-03-08T23:47:13.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc005630 con 0x7f9ac0102940 2024-03-08T23:47:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.914+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac01a2610 con 0x7f9ac0102940 2024-03-08T23:47:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.916+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac0068aa0 con 0x7f9ac0102940 2024-03-08T23:47:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.916+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f9abc004750 con 0x7f9ac0102940 2024-03-08T23:47:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.917+0000 7f9ab67fc700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9aac0a2800 0x7f9aac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.917+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f9abc0cd800 con 0x7f9ac0102940 2024-03-08T23:47:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.917+0000 7f9ac4fff700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9aac0a2800 0x7f9aac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:13.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.918+0000 7f9ac4fff700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9aac0a2800 0x7f9aac0a4cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009c60 tx=0x7f9ab0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:47:13.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:13.922+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9abc09bd50 con 0x7f9ac0102940 2024-03-08T23:47:14.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:14.137+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac0062220 con 0x7f9aac0a2800 2024-03-08T23:47:14.236 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:14 smithi022 sudo[51109]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T23:47:14.236 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:14 smithi022 sudo[51109]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:14.236 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:14 smithi022 sudo[51109]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:14.236 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 08 23:47:14 smithi022 sudo[51109]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:14.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:14 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-08T23:47:14.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:14 smithi022 ceph-mon[31894]: osdmap e20: 3 total, 3 up, 3 in 2024-03-08T23:47:14.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:14 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:14.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:14 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:47:14.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:14 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:47:14.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:14 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:14.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:14 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-08T23:47:14.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:14 smithi062 ceph-mon[36129]: osdmap e20: 3 total, 3 up, 3 in 2024-03-08T23:47:14.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:14 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:14.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:14 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:47:14.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:14 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:47:14.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:14 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:14.864 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44895]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T23:47:14.864 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44895]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:14.864 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44895]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:14.864 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44895]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:14.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:14 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-08T23:47:14.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:14 smithi192 ceph-mon[35995]: osdmap e20: 3 total, 3 up, 3 in 2024-03-08T23:47:14.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:14 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:14.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:14 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:47:14.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:14 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:47:14.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:14 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:15.363 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44898]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T23:47:15.363 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44898]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:15.363 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44898]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:15.363 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 08 23:47:14 smithi062 sudo[44898]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:15.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:15.413+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9abc097cc0 con 0x7f9ac0102940 2024-03-08T23:47:15.471 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:15 smithi192 ceph-mon[35995]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:15.472 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:15 smithi192 ceph-mon[35995]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:47:15.472 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:15 smithi192 ceph-mon[35995]: osdmap e21: 3 total, 3 up, 3 in 2024-03-08T23:47:15.472 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:15 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:15.472 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45196]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T23:47:15.472 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45196]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:15.472 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45196]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:15.472 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45196]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:15.671 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 ceph-mon[31894]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:15.671 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 ceph-mon[31894]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:47:15.671 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 ceph-mon[31894]: osdmap e21: 3 total, 3 up, 3 in 2024-03-08T23:47:15.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:15 smithi062 ceph-mon[36129]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:15 smithi062 ceph-mon[36129]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:47:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:15 smithi062 ceph-mon[36129]: osdmap e21: 3 total, 3 up, 3 in 2024-03-08T23:47:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:15 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:15.882 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45199]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T23:47:15.882 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45199]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:15.882 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45199]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:15.882 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 08 23:47:15 smithi192 sudo[45199]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:15.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 sudo[51275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T23:47:15.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 sudo[51275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:15.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 sudo[51275]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:15.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:15 smithi022 sudo[51275]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:16.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 sudo[51283]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T23:47:16.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 sudo[51283]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:16.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 sudo[51283]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:16.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 sudo[51283]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:16.519 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45202]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T23:47:16.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45202]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:16.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45202]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:16.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45202]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:16.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:16.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:16.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45205]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T23:47:16.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:16.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45205]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:16.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 sudo[45205]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:16.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:16.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:17.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 sudo[44901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T23:47:17.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 sudo[44901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:17.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 sudo[44901]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:17.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:16 smithi062 sudo[44901]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:17.800 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3112181233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da0b079a-88ee-483c-b094-40c3cc5fdcd2"}]: dispatch 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3112181233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da0b079a-88ee-483c-b094-40c3cc5fdcd2"}]': finished 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: osdmap e22: 4 total, 3 up, 4 in 2024-03-08T23:47:17.801 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:17.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 sudo[44904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 sudo[44904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 sudo[44904]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 sudo[44904]: pam_unix(sudo:session): session closed for user root 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3112181233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da0b079a-88ee-483c-b094-40c3cc5fdcd2"}]: dispatch 2024-03-08T23:47:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3112181233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da0b079a-88ee-483c-b094-40c3cc5fdcd2"}]': finished 2024-03-08T23:47:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: osdmap e22: 4 total, 3 up, 4 in 2024-03-08T23:47:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3112181233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da0b079a-88ee-483c-b094-40c3cc5fdcd2"}]: dispatch 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3112181233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da0b079a-88ee-483c-b094-40c3cc5fdcd2"}]': finished 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: osdmap e22: 4 total, 3 up, 4 in 2024-03-08T23:47:17.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:18.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1699004142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:47:18.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1699004142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:47:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1699004142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:47:19.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:19 smithi062 ceph-mon[36129]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:19 smithi192 ceph-mon[35995]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:19 smithi022 ceph-mon[31894]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:21.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:21 smithi022 ceph-mon[31894]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:21.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:21 smithi062 ceph-mon[36129]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:21.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:21 smithi192 ceph-mon[35995]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:23.700 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:23 smithi022 ceph-mon[31894]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:23 smithi062 ceph-mon[36129]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:23 smithi192 ceph-mon[35995]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:25 smithi022 ceph-mon[31894]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-08T23:47:25.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:25 smithi062 ceph-mon[36129]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-08T23:47:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:25 smithi192 ceph-mon[35995]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-08T23:47:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:26.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:26 smithi022 ceph-mon[31894]: Deploying daemon osd.3 on smithi022 2024-03-08T23:47:26.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:26 smithi062 ceph-mon[36129]: Deploying daemon osd.3 on smithi022 2024-03-08T23:47:26.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:26 smithi192 ceph-mon[35995]: Deploying daemon osd.3 on smithi022 2024-03-08T23:47:27.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:27 smithi062 ceph-mon[36129]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:27.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:27 smithi192 ceph-mon[35995]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:27.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:27 smithi022 ceph-mon[31894]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:29.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:29 smithi022 ceph-mon[31894]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:29.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:29 smithi062 ceph-mon[36129]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:29.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:29 smithi192 ceph-mon[35995]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:31.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:31 smithi192 ceph-mon[35995]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:31.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:31.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:31.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:31.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:31 smithi022 ceph-mon[31894]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:31.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:31.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:31.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:31.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:31 smithi062 ceph-mon[36129]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:33.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:33 smithi192 ceph-mon[35995]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:33.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:33 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:33 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:33 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:33.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:33 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:33.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:33 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:33 smithi022 ceph-mon[31894]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:33 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:33 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:33 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:33 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:33.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:33 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:33 smithi062 ceph-mon[36129]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:33 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:33 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:33 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:33 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:33 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:34.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.003+0000 7f9ab67fc700 1 -- 172.21.15.22:0/1409696154 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9ac0062220 con 0x7f9aac0a2800 2024-03-08T23:47:34.001 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 3 on host 'smithi022' 2024-03-08T23:47:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9aac0a2800 msgr2=0x7f9aac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9aac0a2800 0x7f9aac0a4cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009c60 tx=0x7f9ab0009260 comp rx=0 tx=0).stop 2024-03-08T23:47:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0102940 msgr2=0x7f9ac01022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0102940 0x7f9ac01022a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9c0 tx=0x7f9abc00ecd0 comp rx=0 tx=0).stop 2024-03-08T23:47:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 shutdown_connections 2024-03-08T23:47:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ac0107ba0 0x7f9ac0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9aac0a2800 0x7f9aac0a4cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ac0102940 0x7f9ac01022a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:34.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 --2- 172.21.15.22:0/1409696154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac0105270 0x7f9ac01008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:34.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 >> 172.21.15.22:0/1409696154 conn(0x7f9ac00fc2b0 msgr2=0x7f9ac0103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:47:34.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 shutdown_connections 2024-03-08T23:47:34.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:47:34.006+0000 7f9ac7a64700 1 -- 172.21.15.22:0/1409696154 wait complete. 2024-03-08T23:47:34.687 DEBUG:teuthology.orchestra.run.smithi022:osd.3> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.3.service 2024-03-08T23:47:34.689 INFO:tasks.cephadm:Deploying osd.4 on smithi062 with /dev/vg_nvme/lv_3... 2024-03-08T23:47:34.690 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-08T23:47:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:47:35.068 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:34 smithi062 ceph-mon[36129]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:35.069 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:35.069 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:35.069 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:35.069 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:34 smithi062 ceph-mon[36129]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-08T23:47:35.206 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:34 smithi022 ceph-mon[31894]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:35.206 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:35.206 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:35.207 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:35.207 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:34 smithi022 ceph-mon[31894]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-08T23:47:35.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:34 smithi192 ceph-mon[35995]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:35.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:35.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:35.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:35.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:34 smithi192 ceph-mon[35995]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-08T23:47:36.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:35 smithi022 ceph-mon[31894]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-08T23:47:36.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:35 smithi022 ceph-mon[31894]: osdmap e23: 4 total, 3 up, 4 in 2024-03-08T23:47:36.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:35 smithi022 ceph-mon[31894]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:47:36.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:35 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:36.295 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:35 smithi062 ceph-mon[36129]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-08T23:47:36.295 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:35 smithi062 ceph-mon[36129]: osdmap e23: 4 total, 3 up, 4 in 2024-03-08T23:47:36.295 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:35 smithi062 ceph-mon[36129]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:47:36.296 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:35 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:36.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:35 smithi192 ceph-mon[35995]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-08T23:47:36.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:35 smithi192 ceph-mon[35995]: osdmap e23: 4 total, 3 up, 4 in 2024-03-08T23:47:36.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:35 smithi192 ceph-mon[35995]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:47:36.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:35 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:36.737 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3[53373]: 2024-03-08T23:47:36.542+0000 7ff90b787700 -1 osd.3 0 waiting for initial osdmap 2024-03-08T23:47:36.737 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3[53373]: 2024-03-08T23:47:36.548+0000 7ff903572700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:47:36.896 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-08T23:47:36.896 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-08T23:47:36.896 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-08T23:47:36.896 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-08T23:47:36.897 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111433 s, 941 MB/s 2024-03-08T23:47:36.897 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-08T23:47:37.219 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:36 smithi062 ceph-mon[36129]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:37.219 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:36 smithi062 ceph-mon[36129]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:47:37.219 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:36 smithi062 ceph-mon[36129]: osdmap e24: 4 total, 3 up, 4 in 2024-03-08T23:47:37.219 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:36 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:37.219 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:36 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:37.219 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:36 smithi062 ceph-mon[36129]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' 2024-03-08T23:47:37.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:36 smithi192 ceph-mon[35995]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:37.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:36 smithi192 ceph-mon[35995]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:47:37.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:36 smithi192 ceph-mon[35995]: osdmap e24: 4 total, 3 up, 4 in 2024-03-08T23:47:37.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:37.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:37.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:36 smithi192 ceph-mon[35995]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' 2024-03-08T23:47:37.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-mon[31894]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-08T23:47:37.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-mon[31894]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:47:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-mon[31894]: osdmap e24: 4 total, 3 up, 4 in 2024-03-08T23:47:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:37.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:36 smithi022 ceph-mon[31894]: from='osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159]' entity='osd.3' 2024-03-08T23:47:37.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-03-08T23:47:37.833 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:47:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:37 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:47:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:37 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:47:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:37 smithi062 ceph-mon[36129]: osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] boot 2024-03-08T23:47:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:37 smithi062 ceph-mon[36129]: osdmap e25: 4 total, 4 up, 4 in 2024-03-08T23:47:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:38.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:37 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:47:38.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:37 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:47:38.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:37 smithi192 ceph-mon[35995]: osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] boot 2024-03-08T23:47:38.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:37 smithi192 ceph-mon[35995]: osdmap e25: 4 total, 4 up, 4 in 2024-03-08T23:47:38.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:38.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:37 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:47:38.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:37 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:47:38.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:37 smithi022 ceph-mon[31894]: osd.3 [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] boot 2024-03-08T23:47:38.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:37 smithi022 ceph-mon[31894]: osdmap e25: 4 total, 4 up, 4 in 2024-03-08T23:47:38.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: osdmap e26: 4 total, 4 up, 4 in 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi022 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi022 to 8546M 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:39.260 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:39.261 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:39.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: osdmap e26: 4 total, 4 up, 4 in 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi022 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi022 to 8546M 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:39.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: osdmap e26: 4 total, 4 up, 4 in 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi022 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi022 to 8546M 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:39.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:39.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 -- 172.21.15.62:0/2921683735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 msgr2=0x7f5c90102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:39.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 --2- 172.21.15.62:0/2921683735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90102230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009990 tx=0x7f5c84009ca0 comp rx=0 tx=0).stop 2024-03-08T23:47:39.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 -- 172.21.15.62:0/2921683735 shutdown_connections 2024-03-08T23:47:39.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 --2- 172.21.15.62:0/2921683735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90103930 0x7f5c9010a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:39.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 --2- 172.21.15.62:0/2921683735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90102230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:39.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 --2- 172.21.15.62:0/2921683735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c90102f70 0x7f5c901033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:39.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 -- 172.21.15.62:0/2921683735 >> 172.21.15.62:0/2921683735 conn(0x7f5c900fd3f0 msgr2=0x7f5c900ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:47:39.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 -- 172.21.15.62:0/2921683735 shutdown_connections 2024-03-08T23:47:39.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.463+0000 7f5c95131700 1 -- 172.21.15.62:0/2921683735 wait complete. 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 Processor -- start 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 -- start start 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90102f70 0x7f5c900741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c90103930 0x7f5c90074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9010da60 con 0x7f5c90102f70 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9010d8e0 con 0x7f5c90101e10 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.464+0000 7f5c95131700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9010dbe0 con 0x7f5c90103930 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59126/0 (socket says 172.21.15.62:59126) 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 -- 172.21.15.62:0/3585969366 learned_addr learned my addr 172.21.15.62:0/3585969366 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-08T23:47:39.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8f59e700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c90103930 0x7f5c90074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8e59c700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90102f70 0x7f5c900741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 -- 172.21.15.62:0/3585969366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c90103930 msgr2=0x7f5c90074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c90103930 0x7f5c90074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 -- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90102f70 msgr2=0x7f5c900741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90102f70 0x7f5c900741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 -- 172.21.15.62:0/3585969366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c90074fc0 con 0x7f5c90101e10 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c8ed9d700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90075b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005b30 tx=0x7f5c84009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c77fff700 1 -- 172.21.15.62:0/3585969366 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84004830 con 0x7f5c90101e10 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c901a2180 con 0x7f5c90101e10 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c77fff700 1 -- 172.21.15.62:0/3585969366 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c84003c00 con 0x7f5c90101e10 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c77fff700 1 -- 172.21.15.62:0/3585969366 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84005330 con 0x7f5c90101e10 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.465+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c901a25f0 con 0x7f5c90101e10 2024-03-08T23:47:39.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.466+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c90068aa0 con 0x7f5c90101e10 2024-03-08T23:47:39.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.467+0000 7f5c77fff700 1 -- 172.21.15.62:0/3585969366 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5c84029060 con 0x7f5c90101e10 2024-03-08T23:47:39.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.468+0000 7f5c77fff700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5c780a2890 0x7f5c780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:47:39.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.468+0000 7f5c77fff700 1 -- 172.21.15.62:0/3585969366 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f5c840ce4e0 con 0x7f5c90101e10 2024-03-08T23:47:39.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.468+0000 7f5c8e59c700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5c780a2890 0x7f5c780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:47:39.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.469+0000 7f5c8e59c700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5c780a2890 0x7f5c780a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009a10 tx=0x7f5c80009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:47:39.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.473+0000 7f5c77fff700 1 -- 172.21.15.62:0/3585969366 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5c84013070 con 0x7f5c90101e10 2024-03-08T23:47:39.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:39.685+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c90062220 con 0x7f5c780a2890 2024-03-08T23:47:40.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:40 smithi062 ceph-mon[36129]: osdmap e27: 4 total, 4 up, 4 in 2024-03-08T23:47:40.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:40 smithi062 ceph-mon[36129]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:47:40.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:47:40.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:47:40.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:40 smithi192 ceph-mon[35995]: osdmap e27: 4 total, 4 up, 4 in 2024-03-08T23:47:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:40 smithi192 ceph-mon[35995]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:47:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:47:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:47:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:40.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:40 smithi022 ceph-mon[31894]: osdmap e27: 4 total, 4 up, 4 in 2024-03-08T23:47:40.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:40 smithi022 ceph-mon[31894]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:47:40.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:47:40.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:47:40.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:41.301 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:41 smithi062 ceph-mon[36129]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:41.301 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:41 smithi062 ceph-mon[36129]: osdmap e28: 4 total, 4 up, 4 in 2024-03-08T23:47:41.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:41 smithi192 ceph-mon[35995]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:41.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:41 smithi192 ceph-mon[35995]: osdmap e28: 4 total, 4 up, 4 in 2024-03-08T23:47:41.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:41 smithi022 ceph-mon[31894]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:41.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:41 smithi022 ceph-mon[31894]: osdmap e28: 4 total, 4 up, 4 in 2024-03-08T23:47:42.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:42 smithi062 ceph-mon[36129]: osdmap e29: 4 total, 4 up, 4 in 2024-03-08T23:47:42.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:42 smithi192 ceph-mon[35995]: osdmap e29: 4 total, 4 up, 4 in 2024-03-08T23:47:42.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:42 smithi022 ceph-mon[31894]: osdmap e29: 4 total, 4 up, 4 in 2024-03-08T23:47:43.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:43 smithi062 ceph-mon[36129]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:43.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:43 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/4293771440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]: dispatch 2024-03-08T23:47:43.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:43 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]: dispatch 2024-03-08T23:47:43.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:43 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]': finished 2024-03-08T23:47:43.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:43 smithi062 ceph-mon[36129]: osdmap e30: 5 total, 4 up, 5 in 2024-03-08T23:47:43.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:47:43.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:43 smithi192 ceph-mon[35995]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:43.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:43 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/4293771440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]: dispatch 2024-03-08T23:47:43.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:43 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]: dispatch 2024-03-08T23:47:43.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:43 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]': finished 2024-03-08T23:47:43.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:43 smithi192 ceph-mon[35995]: osdmap e30: 5 total, 4 up, 5 in 2024-03-08T23:47:43.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:47:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:43 smithi022 ceph-mon[31894]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:43 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/4293771440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]: dispatch 2024-03-08T23:47:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:43 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]: dispatch 2024-03-08T23:47:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:43 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b0bac61-2019-4ebf-8688-8ae47e9457ac"}]': finished 2024-03-08T23:47:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:43 smithi022 ceph-mon[31894]: osdmap e30: 5 total, 4 up, 5 in 2024-03-08T23:47:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:47:44.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:44 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/510137750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:47:44.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:44 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/510137750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:47:44.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:44 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/510137750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:47:45.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:45 smithi062 ceph-mon[36129]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-08T23:47:45.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:45 smithi192 ceph-mon[35995]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-08T23:47:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:45 smithi022 ceph-mon[31894]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-08T23:47:47.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:47 smithi062 ceph-mon[36129]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:47:47.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:47 smithi192 ceph-mon[35995]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:47:47.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:47 smithi022 ceph-mon[31894]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:47:49.258 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:49 smithi062 ceph-mon[36129]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-08T23:47:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:49 smithi192 ceph-mon[35995]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-08T23:47:49.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:49 smithi022 ceph-mon[31894]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-08T23:47:50.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-08T23:47:50.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:50 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:50.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-08T23:47:50.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:50 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:50.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-08T23:47:50.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:50 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:51.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:51 smithi062 ceph-mon[36129]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-08T23:47:51.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:51 smithi062 ceph-mon[36129]: Deploying daemon osd.4 on smithi062 2024-03-08T23:47:51.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:51 smithi192 ceph-mon[35995]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-08T23:47:51.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:51 smithi192 ceph-mon[35995]: Deploying daemon osd.4 on smithi062 2024-03-08T23:47:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:51 smithi022 ceph-mon[31894]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-08T23:47:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:51 smithi022 ceph-mon[31894]: Deploying daemon osd.4 on smithi062 2024-03-08T23:47:53.307 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:53 smithi062 ceph-mon[36129]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:47:53.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:53 smithi192 ceph-mon[35995]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:47:53.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:53 smithi022 ceph-mon[31894]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:47:55.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:55 smithi062 ceph-mon[36129]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-08T23:47:55.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:55 smithi192 ceph-mon[35995]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-08T23:47:55.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:55 smithi022 ceph-mon[31894]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-08T23:47:56.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:56.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:56.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:56 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:57.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:56 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:57.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:56 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:57.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:56 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:57.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:56 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:57.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:56 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:57.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:56 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:57.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:57 smithi062 ceph-mon[36129]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:57.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:57 smithi022 ceph-mon[31894]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:58.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:57 smithi192 ceph-mon[35995]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:59.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.453+0000 7f5c77fff700 1 -- 172.21.15.62:0/3585969366 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5c90062220 con 0x7f5c780a2890 2024-03-08T23:47:59.451 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 4 on host 'smithi062' 2024-03-08T23:47:59.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.455+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5c780a2890 msgr2=0x7f5c780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:59.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.455+0000 7f5c95131700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5c780a2890 0x7f5c780a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009a10 tx=0x7f5c80009190 comp rx=0 tx=0).stop 2024-03-08T23:47:59.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.455+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 msgr2=0x7f5c90075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:47:59.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.455+0000 7f5c95131700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90075b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005b30 tx=0x7f5c84009580 comp rx=0 tx=0).stop 2024-03-08T23:47:59.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.455+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 shutdown_connections 2024-03-08T23:47:59.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.455+0000 7f5c95131700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c90103930 0x7f5c90074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:59.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.456+0000 7f5c95131700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c90101e10 0x7f5c90075b80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:59.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.456+0000 7f5c95131700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5c780a2890 0x7f5c780a4d50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:59.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.456+0000 7f5c95131700 1 --2- 172.21.15.62:0/3585969366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90102f70 0x7f5c900741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:47:59.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.456+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 >> 172.21.15.62:0/3585969366 conn(0x7f5c900fd3f0 msgr2=0x7f5c90106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:47:59.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.456+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 shutdown_connections 2024-03-08T23:47:59.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:47:59.456+0000 7f5c95131700 1 -- 172.21.15.62:0/3585969366 wait complete. 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.767 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.768 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:59.768 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T23:47:59.768 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:47:59 smithi062 ceph-mon[36129]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T23:47:59.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:59.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T23:47:59.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:47:59 smithi192 ceph-mon[35995]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T23:47:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:47:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:47:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:47:59.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:47:59.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:47:59.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T23:47:59.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:47:59 smithi022 ceph-mon[31894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T23:48:00.073 DEBUG:teuthology.orchestra.run.smithi062:osd.4> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.4.service 2024-03-08T23:48:00.076 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2024-03-08T23:48:00.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-08T23:48:00.321 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:48:01.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-mon[36129]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:48:01.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-mon[36129]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-08T23:48:01.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-mon[36129]: osdmap e31: 5 total, 4 up, 5 in 2024-03-08T23:48:01.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:01.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-mon[36129]: from='osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:48:01.558 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-mon[36129]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:48:01.559 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-08T23:48:01.428+0000 7f87776cf700 -1 osd.4 0 waiting for initial osdmap 2024-03-08T23:48:01.559 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 08 23:48:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-08T23:48:01.434+0000 7f87734c2700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:48:01.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:01 smithi022 ceph-mon[31894]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:48:01.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:01 smithi022 ceph-mon[31894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-08T23:48:01.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:01 smithi022 ceph-mon[31894]: osdmap e31: 5 total, 4 up, 5 in 2024-03-08T23:48:01.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:01 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:01.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:01 smithi022 ceph-mon[31894]: from='osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:48:01.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:01 smithi022 ceph-mon[31894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:48:01.759 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:01 smithi192 ceph-mon[35995]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:48:01.759 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:01 smithi192 ceph-mon[35995]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-08T23:48:01.759 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:01 smithi192 ceph-mon[35995]: osdmap e31: 5 total, 4 up, 5 in 2024-03-08T23:48:01.759 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:01 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:01.759 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:01 smithi192 ceph-mon[35995]: from='osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:48:01.759 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:01 smithi192 ceph-mon[35995]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:48:02.374 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-08T23:48:02.375 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-08T23:48:02.375 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-08T23:48:02.375 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-08T23:48:02.375 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113547 s, 923 MB/s 2024-03-08T23:48:02.375 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-08T23:48:02.697 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:02 smithi192 ceph-mon[35995]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:48:02.698 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:02 smithi192 ceph-mon[35995]: osdmap e32: 5 total, 4 up, 5 in 2024-03-08T23:48:02.698 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:02.698 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:02.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:02 smithi022 ceph-mon[31894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:48:02.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:02 smithi022 ceph-mon[31894]: osdmap e32: 5 total, 4 up, 5 in 2024-03-08T23:48:02.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:02.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:02.772 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:02 smithi062 ceph-mon[36129]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:48:02.772 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:02 smithi062 ceph-mon[36129]: osdmap e32: 5 total, 4 up, 5 in 2024-03-08T23:48:02.773 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:02.773 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:03.001 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-03-08T23:48:03.250 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:48:03.626 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] boot 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: osdmap e33: 5 total, 5 up, 5 in 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:03.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:03.628 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:48:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:48:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:48:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] boot 2024-03-08T23:48:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: osdmap e33: 5 total, 5 up, 5 in 2024-03-08T23:48:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:03.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: osd.4 [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] boot 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: osdmap e33: 5 total, 5 up, 5 in 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:04.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:04 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi062 2024-03-08T23:48:04.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:04 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi062 to 8540M 2024-03-08T23:48:04.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:04 smithi022 ceph-mon[31894]: osdmap e34: 5 total, 5 up, 5 in 2024-03-08T23:48:04.742 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:04 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi062 2024-03-08T23:48:04.742 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:04 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi062 to 8540M 2024-03-08T23:48:04.742 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:04 smithi192 ceph-mon[35995]: osdmap e34: 5 total, 5 up, 5 in 2024-03-08T23:48:04.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:04 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi062 2024-03-08T23:48:04.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:04 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi062 to 8540M 2024-03-08T23:48:04.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:04 smithi062 ceph-mon[36129]: osdmap e34: 5 total, 5 up, 5 in 2024-03-08T23:48:04.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.954+0000 7fbf28e33700 1 -- 172.21.15.192:0/222283719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf241019d0 msgr2=0x7fbf24109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:04.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.954+0000 7fbf28e33700 1 --2- 172.21.15.192:0/222283719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf241019d0 0x7fbf24109290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b0d0 tx=0x7fbf1c00b3e0 comp rx=0 tx=0).stop 2024-03-08T23:48:04.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.954+0000 7fbf28e33700 1 -- 172.21.15.192:0/222283719 shutdown_connections 2024-03-08T23:48:04.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 --2- 172.21.15.192:0/222283719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf241019d0 0x7fbf24109290 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:04.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 --2- 172.21.15.192:0/222283719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf241022f0 0x7fbf24102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:04.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 --2- 172.21.15.192:0/222283719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf24101030 0x7fbf24101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:04.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 -- 172.21.15.192:0/222283719 >> 172.21.15.192:0/222283719 conn(0x7fbf240fba80 msgr2=0x7fbf240fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:48:04.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 -- 172.21.15.192:0/222283719 shutdown_connections 2024-03-08T23:48:04.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 -- 172.21.15.192:0/222283719 wait complete. 2024-03-08T23:48:04.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 Processor -- start 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 -- start start 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf24101030 0x7fbf24196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.955+0000 7fbf28e33700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf241019d0 0x7fbf241973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf28e33700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf241022f0 0x7fbf2419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf28e33700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2410c980 con 0x7fbf241019d0 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf28e33700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2410c800 con 0x7fbf241022f0 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf28e33700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2410cb00 con 0x7fbf24101030 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf24101030 0x7fbf24196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf24101030 0x7fbf24196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:47532/0 (socket says 172.21.15.192:47532) 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 -- 172.21.15.192:0/720864345 learned_addr learned my addr 172.21.15.192:0/720864345 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf22d9d700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf241022f0 0x7fbf2419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf21d9b700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf241019d0 0x7fbf241973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:04.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 -- 172.21.15.192:0/720864345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf241022f0 msgr2=0x7fbf2419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf241022f0 0x7fbf2419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 -- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf241019d0 msgr2=0x7fbf241973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf241019d0 0x7fbf241973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 -- 172.21.15.192:0/720864345 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2419eb00 con 0x7fbf24101030 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf22d9d700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf241022f0 0x7fbf2419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf2259c700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf24101030 0x7fbf24196ea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800d9b0 tx=0x7fbf1800dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf137fe700 1 -- 172.21.15.192:0/720864345 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18017030 con 0x7fbf24101030 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2419edf0 con 0x7fbf24101030 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf137fe700 1 -- 172.21.15.192:0/720864345 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf18004500 con 0x7fbf24101030 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf137fe700 1 -- 172.21.15.192:0/720864345 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18003aa0 con 0x7fbf24101030 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.956+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2419f250 con 0x7fbf24101030 2024-03-08T23:48:04.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.957+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24068aa0 con 0x7fbf24101030 2024-03-08T23:48:04.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.958+0000 7fbf137fe700 1 -- 172.21.15.192:0/720864345 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbf1801a070 con 0x7fbf24101030 2024-03-08T23:48:04.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.959+0000 7fbf137fe700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbf0c0a2890 0x7fbf0c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:04.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.959+0000 7fbf137fe700 1 -- 172.21.15.192:0/720864345 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fbf180cd740 con 0x7fbf24101030 2024-03-08T23:48:04.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.959+0000 7fbf21d9b700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbf0c0a2890 0x7fbf0c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:04.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.960+0000 7fbf21d9b700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbf0c0a2890 0x7fbf0c0a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbf14005fd0 tx=0x7fbf1400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:48:04.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:04.963+0000 7fbf137fe700 1 -- 172.21.15.192:0/720864345 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fbf1809b850 con 0x7fbf24101030 2024-03-08T23:48:05.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:05.167+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf24062220 con 0x7fbf0c0a2890 2024-03-08T23:48:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:05 smithi022 ceph-mon[31894]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:05.763 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:05 smithi192 ceph-mon[35995]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:05.763 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:05.763 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:05.763 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:05 smithi062 ceph-mon[36129]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:05.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:05.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:05.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:06.715 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:06 smithi192 ceph-mon[35995]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:06.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:06 smithi022 ceph-mon[31894]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:06 smithi062 ceph-mon[36129]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:07.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:07 smithi022 ceph-mon[31894]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:07.788 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:07 smithi062 ceph-mon[36129]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:07 smithi192 ceph-mon[35995]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:08.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:08 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/437613755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]: dispatch 2024-03-08T23:48:08.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:08 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]: dispatch 2024-03-08T23:48:08.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:08 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]': finished 2024-03-08T23:48:08.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:08 smithi192 ceph-mon[35995]: osdmap e35: 6 total, 5 up, 6 in 2024-03-08T23:48:08.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:08 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/437613755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]: dispatch 2024-03-08T23:48:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:08 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]: dispatch 2024-03-08T23:48:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:08 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]': finished 2024-03-08T23:48:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:08 smithi022 ceph-mon[31894]: osdmap e35: 6 total, 5 up, 6 in 2024-03-08T23:48:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:08 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:08.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:08 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/437613755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]: dispatch 2024-03-08T23:48:08.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:08 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]: dispatch 2024-03-08T23:48:08.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:08 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45fe7b13-24f3-4e73-984b-04ff364a0d6a"}]': finished 2024-03-08T23:48:08.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:08 smithi062 ceph-mon[36129]: osdmap e35: 6 total, 5 up, 6 in 2024-03-08T23:48:08.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:08 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:09 smithi022 ceph-mon[31894]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:09 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/371692870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:48:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:09 smithi062 ceph-mon[36129]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:09 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/371692870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:48:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:09 smithi192 ceph-mon[35995]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:09 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/371692870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:48:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:11 smithi022 ceph-mon[31894]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:11.847 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:11 smithi192 ceph-mon[35995]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:11.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:11 smithi062 ceph-mon[36129]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:13.713 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:13 smithi192 ceph-mon[35995]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:13.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:13 smithi022 ceph-mon[31894]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:13 smithi062 ceph-mon[36129]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:15.727 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:15 smithi192 ceph-mon[35995]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:15.727 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:15 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-08T23:48:15.727 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:15 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:15.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:15 smithi022 ceph-mon[31894]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:15.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:15 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-08T23:48:15.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:15 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:15 smithi062 ceph-mon[36129]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:15 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-08T23:48:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:15 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:16.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:16 smithi192 ceph-mon[35995]: Deploying daemon osd.5 on smithi192 2024-03-08T23:48:16.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:16 smithi022 ceph-mon[31894]: Deploying daemon osd.5 on smithi192 2024-03-08T23:48:16.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:16 smithi062 ceph-mon[36129]: Deploying daemon osd.5 on smithi192 2024-03-08T23:48:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:17 smithi192 ceph-mon[35995]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:17.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:17 smithi022 ceph-mon[31894]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:17.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:17 smithi062 ceph-mon[36129]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:19.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:19 smithi022 ceph-mon[31894]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:19 smithi062 ceph-mon[36129]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:19 smithi192 ceph-mon[35995]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:21.529 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:21 smithi192 ceph-mon[35995]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:21.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:21.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:21.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:21.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:21 smithi022 ceph-mon[31894]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:21.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:21.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:21.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:21.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:21 smithi062 ceph-mon[36129]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:21.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:21.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:21.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:23 smithi022 ceph-mon[31894]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:23 smithi062 ceph-mon[36129]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:23 smithi192 ceph-mon[35995]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:24.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.788+0000 7fbf137fe700 1 -- 172.21.15.192:0/720864345 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbf24062220 con 0x7fbf0c0a2890 2024-03-08T23:48:24.786 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbf0c0a2890 msgr2=0x7fbf0c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbf0c0a2890 0x7fbf0c0a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbf14005fd0 tx=0x7fbf1400a560 comp rx=0 tx=0).stop 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf24101030 msgr2=0x7fbf24196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf24101030 0x7fbf24196ea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800d9b0 tx=0x7fbf1800dd70 comp rx=0 tx=0).stop 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 shutdown_connections 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf241022f0 0x7fbf2419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf24101030 0x7fbf24196ea0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:24.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbf0c0a2890 0x7fbf0c0a4d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:24.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 --2- 172.21.15.192:0/720864345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf241019d0 0x7fbf241973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:24.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.790+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 >> 172.21.15.192:0/720864345 conn(0x7fbf240fba80 msgr2=0x7fbf24105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:48:24.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.791+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 shutdown_connections 2024-03-08T23:48:24.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:48:24.791+0000 7fbf28e33700 1 -- 172.21.15.192:0/720864345 wait complete. 2024-03-08T23:48:25.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:25.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:25.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:25.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T23:48:25.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T23:48:25.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T23:48:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T23:48:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:25.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:25.445 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.5.service 2024-03-08T23:48:25.447 INFO:tasks.cephadm:Deploying osd.6 on smithi022 with /dev/vg_nvme/lv_2... 2024-03-08T23:48:25.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-08T23:48:25.701 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:48:26.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:26 smithi062 ceph-mon[36129]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-08T23:48:26.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:26 smithi062 ceph-mon[36129]: osdmap e36: 6 total, 5 up, 6 in 2024-03-08T23:48:26.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:26.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:26 smithi062 ceph-mon[36129]: from='osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:48:26.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:26 smithi062 ceph-mon[36129]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:48:26.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:26 smithi192 ceph-mon[35995]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-08T23:48:26.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:26 smithi192 ceph-mon[35995]: osdmap e36: 6 total, 5 up, 6 in 2024-03-08T23:48:26.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:26.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:26 smithi192 ceph-mon[35995]: from='osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:48:26.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:26 smithi192 ceph-mon[35995]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:48:26.811 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 08 23:48:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-08T23:48:26.499+0000 7f1bb3d85700 -1 osd.5 0 waiting for initial osdmap 2024-03-08T23:48:26.811 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 08 23:48:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-08T23:48:26.506+0000 7f1bafb78700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:48:26.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:26 smithi022 ceph-mon[31894]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-08T23:48:26.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:26 smithi022 ceph-mon[31894]: osdmap e36: 6 total, 5 up, 6 in 2024-03-08T23:48:26.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:26.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:26 smithi022 ceph-mon[31894]: from='osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:48:26.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:26 smithi022 ceph-mon[31894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:48:27.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:27 smithi022 ceph-mon[31894]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:27.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:27 smithi022 ceph-mon[31894]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:48:27.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:27 smithi022 ceph-mon[31894]: osdmap e37: 6 total, 5 up, 6 in 2024-03-08T23:48:27.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:27 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:27.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:27 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:27.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:27 smithi192 ceph-mon[35995]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:27.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:27 smithi192 ceph-mon[35995]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:48:27.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:27 smithi192 ceph-mon[35995]: osdmap e37: 6 total, 5 up, 6 in 2024-03-08T23:48:27.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:27 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:27.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:27 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:27.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:27 smithi062 ceph-mon[36129]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-08T23:48:27.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:27 smithi062 ceph-mon[36129]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:48:27.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:27 smithi062 ceph-mon[36129]: osdmap e37: 6 total, 5 up, 6 in 2024-03-08T23:48:27.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:27 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:27.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:27 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-08T23:48:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-08T23:48:27.933 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-08T23:48:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-08T23:48:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0084828 s, 1.2 GB/s 2024-03-08T23:48:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-08T23:48:28.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:28 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:48:28.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:28 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:48:28.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:28 smithi022 ceph-mon[31894]: osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] boot 2024-03-08T23:48:28.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:28 smithi022 ceph-mon[31894]: osdmap e38: 6 total, 6 up, 6 in 2024-03-08T23:48:28.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:28 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:28.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2024-03-08T23:48:28.841 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:48:28.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:28 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:48:28.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:28 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:48:28.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:28 smithi062 ceph-mon[36129]: osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] boot 2024-03-08T23:48:28.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:28 smithi062 ceph-mon[36129]: osdmap e38: 6 total, 6 up, 6 in 2024-03-08T23:48:28.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:28 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:28.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:28 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:48:28.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:28 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:48:28.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:28 smithi192 ceph-mon[35995]: osd.5 [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] boot 2024-03-08T23:48:28.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:28 smithi192 ceph-mon[35995]: osdmap e38: 6 total, 6 up, 6 in 2024-03-08T23:48:28.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:28 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:48:29.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:29.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: osdmap e39: 6 total, 6 up, 6 in 2024-03-08T23:48:29.810 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi192 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi192 to 8540M 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:29.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: osdmap e39: 6 total, 6 up, 6 in 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi192 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi192 to 8540M 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:29.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: osdmap e39: 6 total, 6 up, 6 in 2024-03-08T23:48:29.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi192 2024-03-08T23:48:29.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:29.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:29.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi192 to 8540M 2024-03-08T23:48:29.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:29.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:29.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:29.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 -- 172.21.15.22:0/4128118818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08103920 msgr2=0x7f2a0810a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 --2- 172.21.15.22:0/4128118818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08103920 0x7f2a0810a240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b0a0 tx=0x7f2a0000b3b0 comp rx=0 tx=0).stop 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 -- 172.21.15.22:0/4128118818 shutdown_connections 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 --2- 172.21.15.22:0/4128118818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08103920 0x7f2a0810a240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 --2- 172.21.15.22:0/4128118818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a08101e00 0x7f2a08102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 --2- 172.21.15.22:0/4128118818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a08102f60 0x7f2a081033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 -- 172.21.15.22:0/4128118818 >> 172.21.15.22:0/4128118818 conn(0x7f2a080fd3c0 msgr2=0x7f2a080ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 -- 172.21.15.22:0/4128118818 shutdown_connections 2024-03-08T23:48:30.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.559+0000 7f2a0d224700 1 -- 172.21.15.22:0/4128118818 wait complete. 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 Processor -- start 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 -- start start 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08101e00 0x7f2a08075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a08102f60 0x7f2a080741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a08103920 0x7f2a08074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0810da50 con 0x7f2a08101e00 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0810d8d0 con 0x7f2a08102f60 2024-03-08T23:48:30.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.560+0000 7f2a0d224700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0810dbd0 con 0x7f2a08103920 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08101e00 0x7f2a08075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08101e00 0x7f2a08075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36002/0 (socket says 172.21.15.22:36002) 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 -- 172.21.15.22:0/429904536 learned_addr learned my addr 172.21.15.22:0/429904536 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a0659c700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a08102f60 0x7f2a080741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a0759e700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a08103920 0x7f2a08074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 -- 172.21.15.22:0/429904536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a08103920 msgr2=0x7f2a08074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a08103920 0x7f2a08074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 -- 172.21.15.22:0/429904536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a08102f60 msgr2=0x7f2a080741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a08102f60 0x7f2a080741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 -- 172.21.15.22:0/429904536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a08074fc0 con 0x7f2a08101e00 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a0759e700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a08103920 0x7f2a08074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a06d9d700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08101e00 0x7f2a08075b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29f800b530 tx=0x7f29f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:48:30.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f29fffff700 1 -- 172.21.15.22:0/429904536 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8015020 con 0x7f2a08101e00 2024-03-08T23:48:30.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f29fffff700 1 -- 172.21.15.22:0/429904536 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f8011d10 con 0x7f2a08101e00 2024-03-08T23:48:30.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f29fffff700 1 -- 172.21.15.22:0/429904536 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f800f2e0 con 0x7f2a08101e00 2024-03-08T23:48:30.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a081a20d0 con 0x7f2a08101e00 2024-03-08T23:48:30.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.561+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a081a2610 con 0x7f2a08101e00 2024-03-08T23:48:30.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.562+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a08068aa0 con 0x7f2a08101e00 2024-03-08T23:48:30.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.564+0000 7f29fffff700 1 -- 172.21.15.22:0/429904536 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f29f8018020 con 0x7f2a08101e00 2024-03-08T23:48:30.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.564+0000 7f29fffff700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f29f40a2890 0x7f29f40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:30.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.564+0000 7f2a0659c700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f29f40a2890 0x7f29f40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:30.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.564+0000 7f29fffff700 1 -- 172.21.15.22:0/429904536 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f29f80cda00 con 0x7f2a08101e00 2024-03-08T23:48:30.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.565+0000 7f2a0659c700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f29f40a2890 0x7f29f40a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f29f000ac50 tx=0x7f29f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.568+0000 7f29fffff700 1 -- 172.21.15.22:0/429904536 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f29f801f020 con 0x7f2a08101e00 2024-03-08T23:48:30.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:30.784+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a08062220 con 0x7f29f40a2890 2024-03-08T23:48:30.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:30 smithi022 ceph-mon[31894]: osdmap e40: 6 total, 6 up, 6 in 2024-03-08T23:48:30.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:30 smithi062 ceph-mon[36129]: osdmap e40: 6 total, 6 up, 6 in 2024-03-08T23:48:30.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:30 smithi192 ceph-mon[35995]: osdmap e40: 6 total, 6 up, 6 in 2024-03-08T23:48:31.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:31 smithi022 ceph-mon[31894]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:31.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:31 smithi022 ceph-mon[31894]: osdmap e41: 6 total, 6 up, 6 in 2024-03-08T23:48:31.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:31.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:31.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:31.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:31 smithi062 ceph-mon[36129]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:31 smithi062 ceph-mon[36129]: osdmap e41: 6 total, 6 up, 6 in 2024-03-08T23:48:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:31.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:31 smithi192 ceph-mon[35995]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:31.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:31 smithi192 ceph-mon[35995]: osdmap e41: 6 total, 6 up, 6 in 2024-03-08T23:48:31.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:31.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:31.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:32.704 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:32 smithi022 ceph-mon[31894]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:32.704 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:32 smithi022 ceph-mon[31894]: osdmap e42: 6 total, 6 up, 6 in 2024-03-08T23:48:32.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:32 smithi062 ceph-mon[36129]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:32.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:32 smithi062 ceph-mon[36129]: osdmap e42: 6 total, 6 up, 6 in 2024-03-08T23:48:32.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:32 smithi192 ceph-mon[35995]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:32.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:32 smithi192 ceph-mon[35995]: osdmap e42: 6 total, 6 up, 6 in 2024-03-08T23:48:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:33 smithi062 ceph-mon[36129]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:33.876 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:33 smithi022 ceph-mon[31894]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:33.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:33 smithi192 ceph-mon[35995]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:34.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:34 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/248690146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be6cb9f2-a8f1-410a-a997-76adcb1fa023"}]: dispatch 2024-03-08T23:48:34.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:34 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/248690146' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be6cb9f2-a8f1-410a-a997-76adcb1fa023"}]': finished 2024-03-08T23:48:34.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:34 smithi062 ceph-mon[36129]: osdmap e43: 7 total, 6 up, 7 in 2024-03-08T23:48:34.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:34.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:34 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/191400341' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:48:34.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:34 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/248690146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be6cb9f2-a8f1-410a-a997-76adcb1fa023"}]: dispatch 2024-03-08T23:48:34.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:34 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/248690146' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be6cb9f2-a8f1-410a-a997-76adcb1fa023"}]': finished 2024-03-08T23:48:34.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:34 smithi192 ceph-mon[35995]: osdmap e43: 7 total, 6 up, 7 in 2024-03-08T23:48:34.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:34.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:34 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/191400341' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:48:34.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:34 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/248690146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be6cb9f2-a8f1-410a-a997-76adcb1fa023"}]: dispatch 2024-03-08T23:48:34.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:34 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/248690146' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be6cb9f2-a8f1-410a-a997-76adcb1fa023"}]': finished 2024-03-08T23:48:34.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:34 smithi022 ceph-mon[31894]: osdmap e43: 7 total, 6 up, 7 in 2024-03-08T23:48:34.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:34.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:34 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/191400341' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:48:35.846 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:35 smithi062 ceph-mon[36129]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-08T23:48:35.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:35 smithi192 ceph-mon[35995]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-08T23:48:35.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:35 smithi022 ceph-mon[31894]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-08T23:48:37.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:37 smithi062 ceph-mon[36129]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:48:37.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:37 smithi192 ceph-mon[35995]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:48:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:37 smithi022 ceph-mon[31894]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:48:39.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:39 smithi062 ceph-mon[36129]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-08T23:48:39.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:39 smithi022 ceph-mon[31894]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-08T23:48:39.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:39 smithi192 ceph-mon[35995]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-08T23:48:41.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:41 smithi022 ceph-mon[31894]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-08T23:48:41.836 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-08T23:48:41.836 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:41.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:41 smithi062 ceph-mon[36129]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-08T23:48:41.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-08T23:48:41.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:41 smithi192 ceph-mon[35995]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-08T23:48:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-08T23:48:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:42.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:42 smithi022 ceph-mon[31894]: Deploying daemon osd.6 on smithi022 2024-03-08T23:48:42.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:42 smithi062 ceph-mon[36129]: Deploying daemon osd.6 on smithi022 2024-03-08T23:48:42.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:42 smithi192 ceph-mon[35995]: Deploying daemon osd.6 on smithi022 2024-03-08T23:48:43.791 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:43 smithi022 ceph-mon[31894]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:48:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:43 smithi062 ceph-mon[36129]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:48:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:43 smithi192 ceph-mon[35995]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:48:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:45 smithi062 ceph-mon[36129]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:45 smithi192 ceph-mon[35995]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:45.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:45 smithi022 ceph-mon[31894]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:47.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:47 smithi062 ceph-mon[36129]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:47.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:47.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:47.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:47.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:47 smithi192 ceph-mon[35995]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:47.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:47.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:47.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:47.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:47 smithi022 ceph-mon[31894]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:47.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:47.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:47.906 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:49.836 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:49 smithi022 ceph-mon[31894]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:49.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:49 smithi062 ceph-mon[36129]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:49.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:49 smithi192 ceph-mon[35995]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:51.198 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 6 on host 'smithi022' 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.198+0000 7f29fffff700 1 -- 172.21.15.22:0/429904536 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2a08062220 con 0x7f29f40a2890 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.200+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f29f40a2890 msgr2=0x7f29f40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.200+0000 7f2a0d224700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f29f40a2890 0x7f29f40a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f29f000ac50 tx=0x7f29f000a380 comp rx=0 tx=0).stop 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08101e00 msgr2=0x7f2a08075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08101e00 0x7f2a08075b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29f800b530 tx=0x7f29f800b840 comp rx=0 tx=0).stop 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 shutdown_connections 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a08103920 0x7f2a08074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:51.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f29f40a2890 0x7f29f40a4d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:51.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a08101e00 0x7f2a08075b80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:51.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 --2- 172.21.15.22:0/429904536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a08102f60 0x7f2a080741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:51.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 >> 172.21.15.22:0/429904536 conn(0x7f2a080fd3c0 msgr2=0x7f2a08106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:48:51.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 shutdown_connections 2024-03-08T23:48:51.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:48:51.201+0000 7f2a0d224700 1 -- 172.21.15.22:0/429904536 wait complete. 2024-03-08T23:48:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-08T23:48:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-08T23:48:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-08T23:48:51.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:51.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:48:51.822 DEBUG:teuthology.orchestra.run.smithi022:osd.6> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.6.service 2024-03-08T23:48:51.825 INFO:tasks.cephadm:Deploying osd.7 on smithi062 with /dev/vg_nvme/lv_2... 2024-03-08T23:48:51.825 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-08T23:48:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:48:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:52 smithi022 ceph-mon[31894]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-08T23:48:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:52 smithi022 ceph-mon[31894]: osdmap e44: 7 total, 6 up, 7 in 2024-03-08T23:48:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:52 smithi022 ceph-mon[31894]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:48:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:52 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:52.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:52 smithi062 ceph-mon[36129]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-08T23:48:52.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:52 smithi062 ceph-mon[36129]: osdmap e44: 7 total, 6 up, 7 in 2024-03-08T23:48:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:52 smithi062 ceph-mon[36129]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:48:52.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:52 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:52.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:52 smithi192 ceph-mon[35995]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-08T23:48:52.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:52 smithi192 ceph-mon[35995]: osdmap e44: 7 total, 6 up, 7 in 2024-03-08T23:48:52.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:52 smithi192 ceph-mon[35995]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:48:52.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:52 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:53.368 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-08T23:48:53.112+0000 7f02ecb22700 -1 osd.6 0 waiting for initial osdmap 2024-03-08T23:48:53.368 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-08T23:48:53.120+0000 7f02e490d700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:48:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-mon[31894]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-mon[31894]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:48:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-mon[31894]: osdmap e45: 7 total, 6 up, 7 in 2024-03-08T23:48:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:53 smithi022 ceph-mon[31894]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' 2024-03-08T23:48:53.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:53 smithi062 ceph-mon[36129]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:53.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:53 smithi062 ceph-mon[36129]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:48:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:53 smithi062 ceph-mon[36129]: osdmap e45: 7 total, 6 up, 7 in 2024-03-08T23:48:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:53 smithi062 ceph-mon[36129]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' 2024-03-08T23:48:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:53 smithi192 ceph-mon[35995]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:53 smithi192 ceph-mon[35995]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:48:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:53 smithi192 ceph-mon[35995]: osdmap e45: 7 total, 6 up, 7 in 2024-03-08T23:48:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:53 smithi192 ceph-mon[35995]: from='osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132]' entity='osd.6' 2024-03-08T23:48:54.346 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-08T23:48:54.346 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-08T23:48:54.346 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-08T23:48:54.346 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-08T23:48:54.346 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126137 s, 831 MB/s 2024-03-08T23:48:54.346 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-08T23:48:54.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:54 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:48:54.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:54 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:48:54.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:54 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:54.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:54 smithi062 ceph-mon[36129]: osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] boot 2024-03-08T23:48:54.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:54 smithi062 ceph-mon[36129]: osdmap e46: 7 total, 7 up, 7 in 2024-03-08T23:48:54.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:54 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:54.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:54 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:48:54.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:54 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:48:54.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:54 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:54.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:54 smithi192 ceph-mon[35995]: osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] boot 2024-03-08T23:48:54.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:54 smithi192 ceph-mon[35995]: osdmap e46: 7 total, 7 up, 7 in 2024-03-08T23:48:54.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:54 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:54.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:54 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:48:54.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:54 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:48:54.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:54 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:54.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:54 smithi022 ceph-mon[31894]: osd.6 [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] boot 2024-03-08T23:48:54.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:54 smithi022 ceph-mon[31894]: osdmap e46: 7 total, 7 up, 7 in 2024-03-08T23:48:54.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:54 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:48:54.996 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-03-08T23:48:55.248 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:48:55.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:55.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.651 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:55.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:55.652 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:55.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:55.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:55.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-08T23:48:55.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:48:55.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:55.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:55.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:48:55.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:48:56.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:56 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi022 2024-03-08T23:48:56.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:56 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi022 to 5697M 2024-03-08T23:48:56.697 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:56 smithi062 ceph-mon[36129]: osdmap e47: 7 total, 7 up, 7 in 2024-03-08T23:48:56.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:56 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi022 2024-03-08T23:48:56.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:56 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi022 to 5697M 2024-03-08T23:48:56.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:56 smithi192 ceph-mon[35995]: osdmap e47: 7 total, 7 up, 7 in 2024-03-08T23:48:56.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:56 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi022 2024-03-08T23:48:56.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:56 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi022 to 5697M 2024-03-08T23:48:56.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:56 smithi022 ceph-mon[31894]: osdmap e47: 7 total, 7 up, 7 in 2024-03-08T23:48:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.074+0000 7efe2bd72700 1 -- 172.21.15.62:0/3368713602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe241030b0 msgr2=0x7efe24103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.074+0000 7efe2bd72700 1 --2- 172.21.15.62:0/3368713602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe241030b0 0x7efe24103510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe14009990 tx=0x7efe14009ca0 comp rx=0 tx=0).stop 2024-03-08T23:48:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.074+0000 7efe2bd72700 1 -- 172.21.15.62:0/3368713602 shutdown_connections 2024-03-08T23:48:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.074+0000 7efe2bd72700 1 --2- 172.21.15.62:0/3368713602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe24103a50 0x7efe2410a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.074+0000 7efe2bd72700 1 --2- 172.21.15.62:0/3368713602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe241000d0 0x7efe241004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.074+0000 7efe2bd72700 1 --2- 172.21.15.62:0/3368713602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe241030b0 0x7efe24103510 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.074+0000 7efe2bd72700 1 -- 172.21.15.62:0/3368713602 >> 172.21.15.62:0/3368713602 conn(0x7efe240fb9f0 msgr2=0x7efe240fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 -- 172.21.15.62:0/3368713602 shutdown_connections 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 -- 172.21.15.62:0/3368713602 wait complete. 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 Processor -- start 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 -- start start 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe241000d0 0x7efe24197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241030b0 0x7efe24198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe24103a50 0x7efe2419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2410d9c0 con 0x7efe241030b0 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2410d840 con 0x7efe24103a50 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2bd72700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2410db40 con 0x7efe241000d0 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.075+0000 7efe2a30f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe24103a50 0x7efe2419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe24103a50 0x7efe2419f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43146/0 (socket says 172.21.15.62:43146) 2024-03-08T23:48:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 -- 172.21.15.62:0/762753904 learned_addr learned my addr 172.21.15.62:0/762753904 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2930d700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241030b0 0x7efe24198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe29b0e700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe241000d0 0x7efe24197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 -- 172.21.15.62:0/762753904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe241000d0 msgr2=0x7efe24197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe241000d0 0x7efe24197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 -- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241030b0 msgr2=0x7efe24198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241030b0 0x7efe24198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 -- 172.21.15.62:0/762753904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2419fbb0 con 0x7efe24103a50 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2a30f700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe24103a50 0x7efe2419f510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe1c007ee0 tx=0x7efe1c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe1affd700 1 -- 172.21.15.62:0/762753904 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c004bb0 con 0x7efe24103a50 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe1affd700 1 -- 172.21.15.62:0/762753904 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe1c004d50 con 0x7efe24103a50 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2419fea0 con 0x7efe24103a50 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe1affd700 1 -- 172.21.15.62:0/762753904 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c005990 con 0x7efe24103a50 2024-03-08T23:48:57.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.076+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe241a0300 con 0x7efe24103a50 2024-03-08T23:48:57.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.077+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe24068aa0 con 0x7efe24103a50 2024-03-08T23:48:57.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.079+0000 7efe1affd700 1 -- 172.21.15.62:0/762753904 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7efe1c01a020 con 0x7efe24103a50 2024-03-08T23:48:57.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.079+0000 7efe1affd700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efe100a2890 0x7efe100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:48:57.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.079+0000 7efe1affd700 1 -- 172.21.15.62:0/762753904 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7efe1c0cda30 con 0x7efe24103a50 2024-03-08T23:48:57.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.080+0000 7efe29b0e700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efe100a2890 0x7efe100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:48:57.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.080+0000 7efe29b0e700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efe100a2890 0x7efe100a4d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe2406a380 tx=0x7efe2000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:48:57.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.083+0000 7efe1affd700 1 -- 172.21.15.62:0/762753904 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7efe1c010070 con 0x7efe24103a50 2024-03-08T23:48:57.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:48:57.299+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7efe24062220 con 0x7efe100a2890 2024-03-08T23:48:57.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:57 smithi192 ceph-mon[35995]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:48:57.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:57 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:57.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:57 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:57.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:57 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:57.924 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:57 smithi062 ceph-mon[36129]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:48:57.924 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:57 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:57.924 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:57 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:57.924 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:57 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:57.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:57 smithi022 ceph-mon[31894]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:48:57.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:57 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:48:57.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:57 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:48:57.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:57 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:48:58.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:58 smithi062 ceph-mon[36129]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:58.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:58 smithi192 ceph-mon[35995]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:58.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:58 smithi022 ceph-mon[31894]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:48:59.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:48:59 smithi062 ceph-mon[36129]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:48:59.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:48:59 smithi192 ceph-mon[35995]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:48:59.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:48:59 smithi022 ceph-mon[31894]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:00 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/776533252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]: dispatch 2024-03-08T23:49:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:00 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]: dispatch 2024-03-08T23:49:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:00 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]': finished 2024-03-08T23:49:00.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:00 smithi062 ceph-mon[36129]: osdmap e48: 8 total, 7 up, 8 in 2024-03-08T23:49:00.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:00 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:00 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/776533252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]: dispatch 2024-03-08T23:49:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:00 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]: dispatch 2024-03-08T23:49:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:00 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]': finished 2024-03-08T23:49:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:00 smithi192 ceph-mon[35995]: osdmap e48: 8 total, 7 up, 8 in 2024-03-08T23:49:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:00 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:00.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:00 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/776533252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]: dispatch 2024-03-08T23:49:00.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:00 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]: dispatch 2024-03-08T23:49:00.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:00 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db83755e-a0a0-435c-963a-41f2eeb41266"}]': finished 2024-03-08T23:49:00.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:00 smithi022 ceph-mon[31894]: osdmap e48: 8 total, 7 up, 8 in 2024-03-08T23:49:00.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:01.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:01 smithi192 ceph-mon[35995]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:01.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:01 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/2875809074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:01.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:01 smithi022 ceph-mon[31894]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:01.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:01 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/2875809074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:02.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:01 smithi062 ceph-mon[36129]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:02.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:01 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/2875809074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:03 smithi192 ceph-mon[35995]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:03.927 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:03 smithi062 ceph-mon[36129]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:03.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:03 smithi022 ceph-mon[31894]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:05 smithi062 ceph-mon[36129]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:05.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:05 smithi192 ceph-mon[35995]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:05.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:05 smithi022 ceph-mon[31894]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:07.726 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:07 smithi062 ceph-mon[36129]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:07.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:07 smithi022 ceph-mon[31894]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:08.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:07 smithi192 ceph-mon[35995]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:08.645 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:08 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-08T23:49:08.646 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:08 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:08.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:08 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-08T23:49:08.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:08 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:09.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:08 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-08T23:49:09.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:08 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:09.752 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:09 smithi062 ceph-mon[36129]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:09.752 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:09 smithi062 ceph-mon[36129]: Deploying daemon osd.7 on smithi062 2024-03-08T23:49:09.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:09 smithi022 ceph-mon[31894]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:09.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:09 smithi022 ceph-mon[31894]: Deploying daemon osd.7 on smithi062 2024-03-08T23:49:10.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:09 smithi192 ceph-mon[35995]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:10.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:09 smithi192 ceph-mon[35995]: Deploying daemon osd.7 on smithi062 2024-03-08T23:49:11.753 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:11 smithi062 ceph-mon[36129]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:11.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:11 smithi022 ceph-mon[31894]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:12.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:11 smithi192 ceph-mon[35995]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:13.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:13 smithi062 ceph-mon[36129]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:13.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:13.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:13.899 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:13 smithi022 ceph-mon[31894]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:14.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:13 smithi192 ceph-mon[35995]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:14.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:14.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:14.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:15 smithi062 ceph-mon[36129]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:15.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:15 smithi022 ceph-mon[31894]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:16.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:15 smithi192 ceph-mon[35995]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:17.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.467+0000 7efe1affd700 1 -- 172.21.15.62:0/762753904 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efe24062220 con 0x7efe100a2890 2024-03-08T23:49:17.464 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 7 on host 'smithi062' 2024-03-08T23:49:17.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efe100a2890 msgr2=0x7efe100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:17.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efe100a2890 0x7efe100a4d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe2406a380 tx=0x7efe2000a3b0 comp rx=0 tx=0).stop 2024-03-08T23:49:17.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe24103a50 msgr2=0x7efe2419f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe24103a50 0x7efe2419f510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe1c007ee0 tx=0x7efe1c00d3b0 comp rx=0 tx=0).stop 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 shutdown_connections 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe24103a50 0x7efe2419f510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe241000d0 0x7efe24197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efe100a2890 0x7efe100a4d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 --2- 172.21.15.62:0/762753904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe241030b0 0x7efe24198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 >> 172.21.15.62:0/762753904 conn(0x7efe240fb9f0 msgr2=0x7efe24106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 shutdown_connections 2024-03-08T23:49:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:49:17.469+0000 7efe2bd72700 1 -- 172.21.15.62:0/762753904 wait complete. 2024-03-08T23:49:17.856 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.857 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:17.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T23:49:17.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T23:49:17.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:17.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:18.186 DEBUG:teuthology.orchestra.run.smithi062:osd.7> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.7.service 2024-03-08T23:49:18.189 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2024-03-08T23:49:18.189 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-08T23:49:18.435 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:49:18.676 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:18 smithi062 ceph-mon[36129]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-08T23:49:18.676 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:18 smithi062 ceph-mon[36129]: from='osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:49:18.676 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:18 smithi062 ceph-mon[36129]: osdmap e49: 8 total, 7 up, 8 in 2024-03-08T23:49:18.676 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:18 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:18.676 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:18 smithi062 ceph-mon[36129]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:49:18.676 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 08 23:49:18 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-08T23:49:18.660+0000 7f044480f700 -1 osd.7 0 waiting for initial osdmap 2024-03-08T23:49:18.676 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 08 23:49:18 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-08T23:49:18.674+0000 7f0440602700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:49:18.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:18 smithi192 ceph-mon[35995]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-08T23:49:18.841 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:18 smithi192 ceph-mon[35995]: from='osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:49:18.841 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:18 smithi192 ceph-mon[35995]: osdmap e49: 8 total, 7 up, 8 in 2024-03-08T23:49:18.841 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:18 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:18.841 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:18 smithi192 ceph-mon[35995]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:49:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:18 smithi022 ceph-mon[31894]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-08T23:49:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:18 smithi022 ceph-mon[31894]: from='osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:49:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:18 smithi022 ceph-mon[31894]: osdmap e49: 8 total, 7 up, 8 in 2024-03-08T23:49:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:18 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:18 smithi022 ceph-mon[31894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: osdmap e50: 8 total, 7 up, 8 in 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] boot 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: osdmap e51: 8 total, 8 up, 8 in 2024-03-08T23:49:19.760 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:19 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:49:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: osdmap e50: 8 total, 7 up, 8 in 2024-03-08T23:49:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] boot 2024-03-08T23:49:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: osdmap e51: 8 total, 8 up, 8 in 2024-03-08T23:49:19.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:19 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-08T23:49:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:49:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: osdmap e50: 8 total, 7 up, 8 in 2024-03-08T23:49:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: osd.7 [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] boot 2024-03-08T23:49:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: osdmap e51: 8 total, 8 up, 8 in 2024-03-08T23:49:19.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:19 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:49:20.480 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-08T23:49:20.480 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-08T23:49:20.480 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-08T23:49:20.480 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-08T23:49:20.480 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113202 s, 926 MB/s 2024-03-08T23:49:20.480 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-08T23:49:20.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:20 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:49:20.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:20 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:49:20.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:20 smithi062 ceph-mon[36129]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:20.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:20 smithi062 ceph-mon[36129]: osdmap e52: 8 total, 8 up, 8 in 2024-03-08T23:49:21.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:20 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:49:21.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:20 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:49:21.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:20 smithi192 ceph-mon[35995]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:21.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:20 smithi192 ceph-mon[35995]: osdmap e52: 8 total, 8 up, 8 in 2024-03-08T23:49:21.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:20 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:49:21.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:20 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:49:21.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:20 smithi022 ceph-mon[31894]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:21.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:20 smithi022 ceph-mon[31894]: osdmap e52: 8 total, 8 up, 8 in 2024-03-08T23:49:21.255 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-03-08T23:49:21.497 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:49:21.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: osdmap e53: 8 total, 8 up, 8 in 2024-03-08T23:49:21.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi062 2024-03-08T23:49:21.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:21.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:21.870 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:21.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:21.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:21.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi062 to 5693M 2024-03-08T23:49:21.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:21.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:21.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:21.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:21.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: osdmap e53: 8 total, 8 up, 8 in 2024-03-08T23:49:21.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi062 2024-03-08T23:49:21.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:21.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:21.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:21.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:21.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:21.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi062 to 5693M 2024-03-08T23:49:21.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:21.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:21.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:21.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:22.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: osdmap e53: 8 total, 8 up, 8 in 2024-03-08T23:49:22.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi062 2024-03-08T23:49:22.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi062 to 5693M 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:22.957 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:22 smithi192 ceph-mon[35995]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:22.957 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:22 smithi192 ceph-mon[35995]: osdmap e54: 8 total, 8 up, 8 in 2024-03-08T23:49:23.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:22 smithi062 ceph-mon[36129]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:23.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:22 smithi062 ceph-mon[36129]: osdmap e54: 8 total, 8 up, 8 in 2024-03-08T23:49:23.197 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 -- 172.21.15.192:0/1196834979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241028a0 msgr2=0x7f4a24104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:23.197 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 --2- 172.21.15.192:0/1196834979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241028a0 0x7f4a24104c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009990 tx=0x7f4a14009ca0 comp rx=0 tx=0).stop 2024-03-08T23:49:23.197 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 -- 172.21.15.192:0/1196834979 shutdown_connections 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 --2- 172.21.15.192:0/1196834979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24107b00 0x7f4a24109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 --2- 172.21.15.192:0/1196834979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241028a0 0x7f4a24104c90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 --2- 172.21.15.192:0/1196834979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a241051d0 0x7f4a241075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 -- 172.21.15.192:0/1196834979 >> 172.21.15.192:0/1196834979 conn(0x7f4a240fc2b0 msgr2=0x7f4a240fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 -- 172.21.15.192:0/1196834979 shutdown_connections 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.197+0000 7f4a29925700 1 -- 172.21.15.192:0/1196834979 wait complete. 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 Processor -- start 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 -- start start 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a241028a0 0x7f4a24102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241051d0 0x7f4a24100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24107b00 0x7f4a24100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2410d9e0 con 0x7f4a241028a0 2024-03-08T23:49:23.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2410d860 con 0x7f4a24107b00 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a29925700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2410db60 con 0x7f4a241051d0 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.198+0000 7f4a227fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241051d0 0x7f4a24100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241051d0 0x7f4a24100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38774/0 (socket says 172.21.15.192:38774) 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 -- 172.21.15.192:0/356429694 learned_addr learned my addr 172.21.15.192:0/356429694 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a22ffd700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a241028a0 0x7f4a24102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a237fe700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24107b00 0x7f4a24100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 -- 172.21.15.192:0/356429694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24107b00 msgr2=0x7f4a24100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24107b00 0x7f4a24100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 -- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a241028a0 msgr2=0x7f4a24102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a241028a0 0x7f4a24102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 -- 172.21.15.192:0/356429694 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a24101640 con 0x7f4a241051d0 2024-03-08T23:49:23.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a237fe700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24107b00 0x7f4a24100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:49:23.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a227fc700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241051d0 0x7f4a24100850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800d950 tx=0x7f4a1800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:49:23.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a28923700 1 -- 172.21.15.192:0/356429694 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18016070 con 0x7f4a241051d0 2024-03-08T23:49:23.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a28923700 1 -- 172.21.15.192:0/356429694 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a18009e20 con 0x7f4a241051d0 2024-03-08T23:49:23.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a241a20f0 con 0x7f4a241051d0 2024-03-08T23:49:23.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a28923700 1 -- 172.21.15.192:0/356429694 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800f620 con 0x7f4a241051d0 2024-03-08T23:49:23.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.199+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a241a2600 con 0x7f4a241051d0 2024-03-08T23:49:23.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.200+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a24068aa0 con 0x7f4a241051d0 2024-03-08T23:49:23.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.202+0000 7f4a28923700 1 -- 172.21.15.192:0/356429694 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4a18019070 con 0x7f4a241051d0 2024-03-08T23:49:23.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.202+0000 7f4a28923700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4a0c0a2890 0x7f4a0c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:23.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.203+0000 7f4a28923700 1 -- 172.21.15.192:0/356429694 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f4a180cdaf0 con 0x7f4a241051d0 2024-03-08T23:49:23.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.203+0000 7f4a22ffd700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4a0c0a2890 0x7f4a0c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:23.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.203+0000 7f4a22ffd700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4a0c0a2890 0x7f4a0c0a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400b5c0 tx=0x7f4a14003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:49:23.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.206+0000 7f4a28923700 1 -- 172.21.15.192:0/356429694 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4a1809b6f0 con 0x7f4a241051d0 2024-03-08T23:49:23.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:22 smithi022 ceph-mon[31894]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:23.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:22 smithi022 ceph-mon[31894]: osdmap e54: 8 total, 8 up, 8 in 2024-03-08T23:49:23.408 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:23.411+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a2402dca0 con 0x7f4a0c0a2890 2024-03-08T23:49:24.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:23 smithi192 ceph-mon[35995]: osdmap e55: 8 total, 8 up, 8 in 2024-03-08T23:49:24.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:23 smithi192 ceph-mon[35995]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:49:24.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:49:24.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:49:24.098 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:23 smithi062 ceph-mon[36129]: osdmap e55: 8 total, 8 up, 8 in 2024-03-08T23:49:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:23 smithi062 ceph-mon[36129]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:49:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:49:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:49:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:24.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:23 smithi022 ceph-mon[31894]: osdmap e55: 8 total, 8 up, 8 in 2024-03-08T23:49:24.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:23 smithi022 ceph-mon[31894]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:49:24.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:49:24.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:49:24.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:24.987 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:24 smithi192 ceph-mon[35995]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:25.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:24 smithi062 ceph-mon[36129]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:25.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:24 smithi022 ceph-mon[31894]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:27.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:26 smithi062 ceph-mon[36129]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:27.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:26 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/3923487761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]: dispatch 2024-03-08T23:49:27.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:26 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]: dispatch 2024-03-08T23:49:27.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:26 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]': finished 2024-03-08T23:49:27.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:26 smithi062 ceph-mon[36129]: osdmap e56: 9 total, 8 up, 9 in 2024-03-08T23:49:27.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:27.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:26 smithi192 ceph-mon[35995]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:27.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:26 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/3923487761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]: dispatch 2024-03-08T23:49:27.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:26 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]: dispatch 2024-03-08T23:49:27.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:26 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]': finished 2024-03-08T23:49:27.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:26 smithi192 ceph-mon[35995]: osdmap e56: 9 total, 8 up, 9 in 2024-03-08T23:49:27.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:27.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:26 smithi022 ceph-mon[31894]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:27.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:26 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/3923487761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]: dispatch 2024-03-08T23:49:27.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:26 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]: dispatch 2024-03-08T23:49:27.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:26 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "766c75f7-7820-4cb6-ac6b-ab7596735fc0"}]': finished 2024-03-08T23:49:27.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:26 smithi022 ceph-mon[31894]: osdmap e56: 9 total, 8 up, 9 in 2024-03-08T23:49:27.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:28.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:27 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/3956251021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:28.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:27 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/3956251021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:28.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:27 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/3956251021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:29.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:28 smithi062 ceph-mon[36129]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:29.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:28 smithi192 ceph-mon[35995]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:29.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:28 smithi022 ceph-mon[31894]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:31.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:30 smithi062 ceph-mon[36129]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:31.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:30 smithi192 ceph-mon[35995]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:31.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:30 smithi022 ceph-mon[31894]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:33.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:32 smithi062 ceph-mon[36129]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:33.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:32 smithi192 ceph-mon[35995]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:33.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:32 smithi022 ceph-mon[31894]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:33.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:33 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-08T23:49:33.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:33 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:33.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:33 smithi192 ceph-mon[35995]: Deploying daemon osd.8 on smithi192 2024-03-08T23:49:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:33 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-08T23:49:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:33 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:33 smithi062 ceph-mon[36129]: Deploying daemon osd.8 on smithi192 2024-03-08T23:49:34.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:33 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-08T23:49:34.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:33 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:34.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:33 smithi022 ceph-mon[31894]: Deploying daemon osd.8 on smithi192 2024-03-08T23:49:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:34 smithi062 ceph-mon[36129]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:34 smithi192 ceph-mon[35995]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:34 smithi022 ceph-mon[31894]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:36 smithi062 ceph-mon[36129]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:37.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:36 smithi192 ceph-mon[35995]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:37.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:36 smithi022 ceph-mon[31894]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:38 smithi062 ceph-mon[36129]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:38 smithi192 ceph-mon[35995]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:39.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:38 smithi022 ceph-mon[31894]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:40.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:40.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:40.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:40.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:40.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:40.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:40.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:40.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:40.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:41.702 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:41 smithi192 ceph-mon[35995]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:41.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:41 smithi022 ceph-mon[31894]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:41.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:41 smithi062 ceph-mon[36129]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:42.875 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2024-03-08T23:49:42.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.877+0000 7f4a28923700 1 -- 172.21.15.192:0/356429694 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a2402dca0 con 0x7f4a0c0a2890 2024-03-08T23:49:42.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.879+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4a0c0a2890 msgr2=0x7f4a0c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:42.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.879+0000 7f4a29925700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4a0c0a2890 0x7f4a0c0a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400b5c0 tx=0x7f4a14003960 comp rx=0 tx=0).stop 2024-03-08T23:49:42.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241051d0 msgr2=0x7f4a24100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:42.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241051d0 0x7f4a24100850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800d950 tx=0x7f4a1800dc60 comp rx=0 tx=0).stop 2024-03-08T23:49:42.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 shutdown_connections 2024-03-08T23:49:42.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a24107b00 0x7f4a24100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:42.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4a0c0a2890 0x7f4a0c0a4d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:42.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a241028a0 0x7f4a24102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:42.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 --2- 172.21.15.192:0/356429694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a241051d0 0x7f4a24100850 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:42.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 >> 172.21.15.192:0/356429694 conn(0x7f4a240fc2b0 msgr2=0x7f4a241031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:49:42.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 shutdown_connections 2024-03-08T23:49:42.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:49:42.880+0000 7f4a29925700 1 -- 172.21.15.192:0/356429694 wait complete. 2024-03-08T23:49:43.555 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:43 smithi192 ceph-mon[35995]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:43.556 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:43.556 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:43.556 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:43 smithi192 ceph-mon[35995]: from='osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T23:49:43.556 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:43 smithi192 ceph-mon[35995]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T23:49:43.582 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.8.service 2024-03-08T23:49:43.585 INFO:tasks.cephadm:Deploying osd.9 on smithi022 with /dev/vg_nvme/lv_1... 2024-03-08T23:49:43.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-08T23:49:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:43 smithi022 ceph-mon[31894]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:43 smithi022 ceph-mon[31894]: from='osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T23:49:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:43 smithi022 ceph-mon[31894]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T23:49:43.841 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:49:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:43 smithi062 ceph-mon[36129]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:43 smithi062 ceph-mon[36129]: from='osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T23:49:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:43 smithi062 ceph-mon[36129]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T23:49:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-mon[35995]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-08T23:49:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-mon[35995]: osdmap e57: 9 total, 8 up, 9 in 2024-03-08T23:49:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-mon[35995]: from='osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:49:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-mon[35995]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:49:44.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-08T23:49:44.875+0000 7fcc04c61700 -1 osd.8 0 waiting for initial osdmap 2024-03-08T23:49:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:44 smithi022 ceph-mon[31894]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-08T23:49:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:44 smithi022 ceph-mon[31894]: osdmap e57: 9 total, 8 up, 9 in 2024-03-08T23:49:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:44 smithi022 ceph-mon[31894]: from='osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:49:44.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:44 smithi022 ceph-mon[31894]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:44 smithi062 ceph-mon[36129]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-08T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:44 smithi062 ceph-mon[36129]: osdmap e57: 9 total, 8 up, 9 in 2024-03-08T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:44 smithi062 ceph-mon[36129]: from='osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:44 smithi062 ceph-mon[36129]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:49:45.313 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 08 23:49:44 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-08T23:49:44.887+0000 7fcc00a54700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:49:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:45 smithi192 ceph-mon[35995]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:45 smithi192 ceph-mon[35995]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:49:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:45 smithi192 ceph-mon[35995]: osdmap e58: 9 total, 8 up, 9 in 2024-03-08T23:49:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:45.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:45.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:45 smithi022 ceph-mon[31894]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:45.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:45 smithi022 ceph-mon[31894]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:49:45.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:45 smithi022 ceph-mon[31894]: osdmap e58: 9 total, 8 up, 9 in 2024-03-08T23:49:45.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:45.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:46.002 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-08T23:49:46.002 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-08T23:49:46.002 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-03-08T23:49:46.002 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-03-08T23:49:46.002 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00850599 s, 1.2 GB/s 2024-03-08T23:49:46.002 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-03-08T23:49:46.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:45 smithi062 ceph-mon[36129]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:46.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:45 smithi062 ceph-mon[36129]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:49:46.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:45 smithi062 ceph-mon[36129]: osdmap e58: 9 total, 8 up, 9 in 2024-03-08T23:49:46.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:46.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:46.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi022:vg_nvme/lv_1 2024-03-08T23:49:47.055 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:49:47.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:46 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:49:47.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:46 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:49:47.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:46 smithi192 ceph-mon[35995]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:47.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:46 smithi192 ceph-mon[35995]: osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] boot 2024-03-08T23:49:47.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:46 smithi192 ceph-mon[35995]: osdmap e59: 9 total, 9 up, 9 in 2024-03-08T23:49:47.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:46 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:47.176 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:46 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:49:47.176 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:46 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:49:47.176 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:46 smithi022 ceph-mon[31894]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:47.176 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:46 smithi022 ceph-mon[31894]: osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] boot 2024-03-08T23:49:47.176 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:46 smithi022 ceph-mon[31894]: osdmap e59: 9 total, 9 up, 9 in 2024-03-08T23:49:47.176 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:46 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:47.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:46 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:49:47.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:46 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:49:47.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:46 smithi062 ceph-mon[36129]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-03-08T23:49:47.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:46 smithi062 ceph-mon[36129]: osd.8 [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] boot 2024-03-08T23:49:47.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:46 smithi062 ceph-mon[36129]: osdmap e59: 9 total, 9 up, 9 in 2024-03-08T23:49:47.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:49:48.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: osdmap e60: 9 total, 9 up, 9 in 2024-03-08T23:49:48.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi192 2024-03-08T23:49:48.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi192 to 5693M 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:47 smithi192 ceph-mon[35995]: osdmap e61: 9 total, 9 up, 9 in 2024-03-08T23:49:48.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: osdmap e60: 9 total, 9 up, 9 in 2024-03-08T23:49:48.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi192 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi192 to 5693M 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:48.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.173 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:47 smithi022 ceph-mon[31894]: osdmap e61: 9 total, 9 up, 9 in 2024-03-08T23:49:48.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: osdmap e60: 9 total, 9 up, 9 in 2024-03-08T23:49:48.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi192 2024-03-08T23:49:48.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi192 to 5693M 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:49:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:47 smithi062 ceph-mon[36129]: osdmap e61: 9 total, 9 up, 9 in 2024-03-08T23:49:48.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 -- 172.21.15.22:0/3489596493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04103950 msgr2=0x7f9c0410a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 --2- 172.21.15.22:0/3489596493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04103950 0x7f9c0410a270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009990 tx=0x7f9bfc009ca0 comp rx=0 tx=0).stop 2024-03-08T23:49:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 -- 172.21.15.22:0/3489596493 shutdown_connections 2024-03-08T23:49:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 --2- 172.21.15.22:0/3489596493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04103950 0x7f9c0410a270 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 --2- 172.21.15.22:0/3489596493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c04101e30 0x7f9c04102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 --2- 172.21.15.22:0/3489596493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102f90 0x7f9c04103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:48.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 -- 172.21.15.22:0/3489596493 >> 172.21.15.22:0/3489596493 conn(0x7f9c040fd390 msgr2=0x7f9c040ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:49:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 -- 172.21.15.22:0/3489596493 shutdown_connections 2024-03-08T23:49:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.742+0000 7f9c08e40700 1 -- 172.21.15.22:0/3489596493 wait complete. 2024-03-08T23:49:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 Processor -- start 2024-03-08T23:49:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 -- start start 2024-03-08T23:49:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04101e30 0x7f9c04075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04102f90 0x7f9c040741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c04103950 0x7f9c04074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0410da20 con 0x7f9c04102f90 2024-03-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0410d8a0 con 0x7f9c04101e30 2024-03-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c08e40700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0410dba0 con 0x7f9c04103950 2024-03-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c01d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04102f90 0x7f9c040741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c01d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04102f90 0x7f9c040741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41990/0 (socket says 172.21.15.22:41990) 2024-03-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c01d9b700 1 -- 172.21.15.22:0/2484868459 learned_addr learned my addr 172.21.15.22:0/2484868459 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c0259c700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04101e30 0x7f9c04075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c02d9d700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c04103950 0x7f9c04074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c01d9b700 1 -- 172.21.15.22:0/2484868459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c04103950 msgr2=0x7f9c04074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c01d9b700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c04103950 0x7f9c04074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:48.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c01d9b700 1 -- 172.21.15.22:0/2484868459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04101e30 msgr2=0x7f9c04075b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.743+0000 7f9c01d9b700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04101e30 0x7f9c04075b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9c01d9b700 1 -- 172.21.15.22:0/2484868459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c04074fc0 con 0x7f9c04102f90 2024-03-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9c02d9d700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c04103950 0x7f9c04074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9c01d9b700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04102f90 0x7f9c040741d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c9b0 tx=0x7f9bec00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2484868459 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec007b00 con 0x7f9c04102f90 2024-03-08T23:49:48.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2484868459 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bec007ca0 con 0x7f9c04102f90 2024-03-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c041a20d0 con 0x7f9c04102f90 2024-03-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2484868459 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec018780 con 0x7f9c04102f90 2024-03-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.744+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c041a25e0 con 0x7f9c04102f90 2024-03-08T23:49:48.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.745+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c04068aa0 con 0x7f9c04102f90 2024-03-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.747+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2484868459 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9bec00f450 con 0x7f9c04102f90 2024-03-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.747+0000 7f9bfb7fe700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9bf00a2890 0x7f9bf00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.747+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2484868459 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f9bec0ce400 con 0x7f9c04102f90 2024-03-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.747+0000 7f9c0259c700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9bf00a2890 0x7f9bf00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.748+0000 7f9c0259c700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9bf00a2890 0x7f9bf00a4d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009d10 tx=0x7f9bf4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:49:48.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.751+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2484868459 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9bec014030 con 0x7f9c04102f90 2024-03-08T23:49:48.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:49:48.967+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c04062220 con 0x7f9bf00a2890 2024-03-08T23:49:49.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:49 smithi022 ceph-mon[31894]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:49:49.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:49 smithi022 ceph-mon[31894]: osdmap e62: 9 total, 9 up, 9 in 2024-03-08T23:49:49.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:49:49.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:49:49.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:49.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:49 smithi062 ceph-mon[36129]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:49:49.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:49 smithi062 ceph-mon[36129]: osdmap e62: 9 total, 9 up, 9 in 2024-03-08T23:49:49.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:49:49.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:49:49.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:49.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:49 smithi192 ceph-mon[35995]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:49:49.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:49 smithi192 ceph-mon[35995]: osdmap e62: 9 total, 9 up, 9 in 2024-03-08T23:49:49.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:49:49.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:49:49.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:49:50.699 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:50 smithi022 ceph-mon[31894]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:49:50.699 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:50 smithi022 ceph-mon[31894]: osdmap e63: 9 total, 9 up, 9 in 2024-03-08T23:49:50.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:50 smithi062 ceph-mon[36129]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:49:50.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:50 smithi062 ceph-mon[36129]: osdmap e63: 9 total, 9 up, 9 in 2024-03-08T23:49:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:50 smithi192 ceph-mon[35995]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:49:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:50 smithi192 ceph-mon[35995]: osdmap e63: 9 total, 9 up, 9 in 2024-03-08T23:49:51.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:51 smithi022 ceph-mon[31894]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-08T23:49:51.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:51 smithi062 ceph-mon[36129]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-08T23:49:51.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:51 smithi192 ceph-mon[35995]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-08T23:49:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:52 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2404938072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029fa555-d0bd-4863-9199-b610bec7e240"}]: dispatch 2024-03-08T23:49:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:52 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2404938072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029fa555-d0bd-4863-9199-b610bec7e240"}]': finished 2024-03-08T23:49:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:52 smithi022 ceph-mon[31894]: osdmap e64: 10 total, 9 up, 10 in 2024-03-08T23:49:52.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:52 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:49:52.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:52 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2404938072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029fa555-d0bd-4863-9199-b610bec7e240"}]: dispatch 2024-03-08T23:49:52.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:52 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2404938072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029fa555-d0bd-4863-9199-b610bec7e240"}]': finished 2024-03-08T23:49:52.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:52 smithi062 ceph-mon[36129]: osdmap e64: 10 total, 9 up, 10 in 2024-03-08T23:49:52.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:52 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:49:52.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:52 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2404938072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029fa555-d0bd-4863-9199-b610bec7e240"}]: dispatch 2024-03-08T23:49:52.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:52 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2404938072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029fa555-d0bd-4863-9199-b610bec7e240"}]': finished 2024-03-08T23:49:52.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:52 smithi192 ceph-mon[35995]: osdmap e64: 10 total, 9 up, 10 in 2024-03-08T23:49:52.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:52 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:49:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:53 smithi022 ceph-mon[31894]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-08T23:49:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:53 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3926662802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:53.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:53 smithi062 ceph-mon[36129]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-08T23:49:53.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:53 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3926662802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:53.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:53 smithi192 ceph-mon[35995]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-08T23:49:53.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:53 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3926662802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:49:55.664 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:55 smithi062 ceph-mon[36129]: pgmap v209: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-08T23:49:55.684 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:55 smithi192 ceph-mon[35995]: pgmap v209: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-08T23:49:55.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:55 smithi022 ceph-mon[31894]: pgmap v209: 1 pgs: 1 peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-08T23:49:57.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:57 smithi022 ceph-mon[31894]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-08T23:49:57.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:57 smithi062 ceph-mon[36129]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-08T23:49:57.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:57 smithi192 ceph-mon[35995]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-08T23:49:59.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:49:59 smithi022 ceph-mon[31894]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:49:59.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:49:59 smithi062 ceph-mon[36129]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:49:59.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:49:59 smithi192 ceph-mon[35995]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:00.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-08T23:50:00.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:00 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:00.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:00 smithi022 ceph-mon[31894]: Deploying daemon osd.9 on smithi022 2024-03-08T23:50:00.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:00 smithi022 ceph-mon[31894]: overall HEALTH_OK 2024-03-08T23:50:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:00 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-08T23:50:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:00 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:00 smithi062 ceph-mon[36129]: Deploying daemon osd.9 on smithi022 2024-03-08T23:50:00.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:00 smithi062 ceph-mon[36129]: overall HEALTH_OK 2024-03-08T23:50:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:00 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-08T23:50:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:00 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:00 smithi192 ceph-mon[35995]: Deploying daemon osd.9 on smithi022 2024-03-08T23:50:00.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:00 smithi192 ceph-mon[35995]: overall HEALTH_OK 2024-03-08T23:50:01.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:01 smithi022 ceph-mon[31894]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:01.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:01 smithi062 ceph-mon[36129]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:01.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:01 smithi192 ceph-mon[35995]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:03.698 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:03 smithi022 ceph-mon[31894]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:03 smithi062 ceph-mon[36129]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:03 smithi192 ceph-mon[35995]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:05 smithi022 ceph-mon[31894]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:05 smithi062 ceph-mon[36129]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:05.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:05 smithi192 ceph-mon[35995]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:05.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:05.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:05.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:07.601 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:07 smithi022 ceph-mon[31894]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:07.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:07 smithi062 ceph-mon[36129]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:07.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:07 smithi192 ceph-mon[35995]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:09.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.487+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2484868459 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9c04062220 con 0x7f9bf00a2890 2024-03-08T23:50:09.485 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 9 on host 'smithi022' 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9bf00a2890 msgr2=0x7f9bf00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9bf00a2890 0x7f9bf00a4d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009d10 tx=0x7f9bf4009040 comp rx=0 tx=0).stop 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04102f90 msgr2=0x7f9c040741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04102f90 0x7f9c040741d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c9b0 tx=0x7f9bec00cd70 comp rx=0 tx=0).stop 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 shutdown_connections 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c04103950 0x7f9c04074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04101e30 0x7f9c04075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:09.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9bf00a2890 0x7f9bf00a4d50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 --2- 172.21.15.22:0/2484868459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c04102f90 0x7f9c040741d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 >> 172.21.15.22:0/2484868459 conn(0x7f9c040fd390 msgr2=0x7f9c04106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:50:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 shutdown_connections 2024-03-08T23:50:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:50:09.492+0000 7f9c08e40700 1 -- 172.21.15.22:0/2484868459 wait complete. 2024-03-08T23:50:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:09 smithi022 ceph-mon[31894]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:09 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:09 smithi022 ceph-mon[31894]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-08T23:50:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:09 smithi062 ceph-mon[36129]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:09 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:09 smithi062 ceph-mon[36129]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-08T23:50:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:09 smithi192 ceph-mon[35995]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:09 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:09 smithi192 ceph-mon[35995]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-08T23:50:10.191 DEBUG:teuthology.orchestra.run.smithi022:osd.9> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.9.service 2024-03-08T23:50:10.194 INFO:tasks.cephadm:Deploying osd.10 on smithi062 with /dev/vg_nvme/lv_1... 2024-03-08T23:50:10.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-08T23:50:10.466 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:50:10.586 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:10 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:10.586 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:10 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:10.586 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:10 smithi062 ceph-mon[36129]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-08T23:50:10.586 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:10 smithi062 ceph-mon[36129]: osdmap e65: 10 total, 9 up, 10 in 2024-03-08T23:50:10.586 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:10 smithi062 ceph-mon[36129]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:50:10.586 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:10 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:10.587 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:10 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:10.721 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:10.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:10.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-mon[31894]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-08T23:50:10.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-mon[31894]: osdmap e65: 10 total, 9 up, 10 in 2024-03-08T23:50:10.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-mon[31894]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:50:10.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:10.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:10.722 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-08T23:50:10.474+0000 7fc656b9f700 -1 osd.9 0 waiting for initial osdmap 2024-03-08T23:50:10.722 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 08 23:50:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-08T23:50:10.486+0000 7fc652992700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:50:10.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:10 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:10.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:10 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:10.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:10 smithi192 ceph-mon[35995]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-08T23:50:10.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:10 smithi192 ceph-mon[35995]: osdmap e65: 10 total, 9 up, 10 in 2024-03-08T23:50:10.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:10 smithi192 ceph-mon[35995]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-03-08T23:50:10.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:10 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:10.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:10 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:11 smithi022 ceph-mon[31894]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:11 smithi022 ceph-mon[31894]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:50:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:11 smithi022 ceph-mon[31894]: osdmap e66: 10 total, 9 up, 10 in 2024-03-08T23:50:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:11 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:11.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:11 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:11.742 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:11 smithi062 ceph-mon[36129]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:11.742 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:11 smithi062 ceph-mon[36129]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:50:11.742 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:11 smithi062 ceph-mon[36129]: osdmap e66: 10 total, 9 up, 10 in 2024-03-08T23:50:11.742 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:11 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:11.742 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:11 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:11.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:11 smithi192 ceph-mon[35995]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-03-08T23:50:11.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:11 smithi192 ceph-mon[35995]: from='osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-03-08T23:50:11.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:11 smithi192 ceph-mon[35995]: osdmap e66: 10 total, 9 up, 10 in 2024-03-08T23:50:11.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:11 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:11.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:11 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:12.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:12 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:50:12.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:12 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:50:12.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:12 smithi062 ceph-mon[36129]: osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] boot 2024-03-08T23:50:12.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:12 smithi062 ceph-mon[36129]: osdmap e67: 10 total, 10 up, 10 in 2024-03-08T23:50:12.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:12 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:12.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:12 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:50:12.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:12 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:50:12.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:12 smithi022 ceph-mon[31894]: osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] boot 2024-03-08T23:50:12.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:12 smithi022 ceph-mon[31894]: osdmap e67: 10 total, 10 up, 10 in 2024-03-08T23:50:12.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:12 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-08T23:50:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-08T23:50:12.764 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-08T23:50:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-08T23:50:12.764 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110471 s, 949 MB/s 2024-03-08T23:50:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-08T23:50:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:12 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:50:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:12 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:50:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:12 smithi192 ceph-mon[35995]: osd.9 [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] boot 2024-03-08T23:50:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:12 smithi192 ceph-mon[35995]: osdmap e67: 10 total, 10 up, 10 in 2024-03-08T23:50:12.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:12 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:50:13.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-03-08T23:50:13.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:13.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: osdmap e68: 10 total, 10 up, 10 in 2024-03-08T23:50:13.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:13.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:13.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.779 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:13.779 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: osdmap e68: 10 total, 10 up, 10 in 2024-03-08T23:50:13.779 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.779 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.779 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.779 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.779 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.780 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.780 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.780 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:13.780 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:13.780 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: osdmap e68: 10 total, 10 up, 10 in 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:13.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:13.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:14.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:14 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi022 2024-03-08T23:50:14.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:14 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi022 to 4273M 2024-03-08T23:50:14.771 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:14 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi022 2024-03-08T23:50:14.771 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:14 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi022 to 4273M 2024-03-08T23:50:14.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:14 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi022 2024-03-08T23:50:14.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:14 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi022 to 4273M 2024-03-08T23:50:15.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 -- 172.21.15.62:0/1426304690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 msgr2=0x7f4164104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:15.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 --2- 172.21.15.62:0/1426304690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164104d10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4158009990 tx=0x7f4158009ca0 comp rx=0 tx=0).stop 2024-03-08T23:50:15.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 -- 172.21.15.62:0/1426304690 shutdown_connections 2024-03-08T23:50:15.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 --2- 172.21.15.62:0/1426304690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4164107b80 0x7f4164109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:15.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 --2- 172.21.15.62:0/1426304690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164104d10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:15.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 --2- 172.21.15.62:0/1426304690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164105250 0x7f4164107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:15.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 -- 172.21.15.62:0/1426304690 >> 172.21.15.62:0/1426304690 conn(0x7f41640fc2b0 msgr2=0x7f41640fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:50:15.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 -- 172.21.15.62:0/1426304690 shutdown_connections 2024-03-08T23:50:15.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 -- 172.21.15.62:0/1426304690 wait complete. 2024-03-08T23:50:15.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 Processor -- start 2024-03-08T23:50:15.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.582+0000 7f416b181700 1 -- start start 2024-03-08T23:50:15.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f416b181700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:15.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f416b181700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4164105250 0x7f4164198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:15.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f416b181700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164107b80 0x7f416419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:15.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f416b181700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416410da00 con 0x7f4164105250 2024-03-08T23:50:15.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f416b181700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416410d880 con 0x7f4164102920 2024-03-08T23:50:15.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f416b181700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416410db80 con 0x7f4164107b80 2024-03-08T23:50:15.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:15.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44942/0 (socket says 172.21.15.62:44942) 2024-03-08T23:50:15.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 -- 172.21.15.62:0/27633400 learned_addr learned my addr 172.21.15.62:0/27633400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-03-08T23:50:15.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4163fff700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4164105250 0x7f4164198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:15.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f416971e700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164107b80 0x7f416419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:15.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 -- 172.21.15.62:0/27633400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164107b80 msgr2=0x7f416419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:15.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164107b80 0x7f416419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:15.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 -- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4164105250 msgr2=0x7f4164198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:15.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4164105250 0x7f4164198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:15.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 -- 172.21.15.62:0/27633400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416419fc40 con 0x7f4164102920 2024-03-08T23:50:15.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4168f1d700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164197fe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41580098e0 tx=0x7f4158005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:50:15.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4161ffb700 1 -- 172.21.15.62:0/27633400 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158027070 con 0x7f4164102920 2024-03-08T23:50:15.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4161ffb700 1 -- 172.21.15.62:0/27633400 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4158003f40 con 0x7f4164102920 2024-03-08T23:50:15.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.583+0000 7f4161ffb700 1 -- 172.21.15.62:0/27633400 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158022e60 con 0x7f4164102920 2024-03-08T23:50:15.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.584+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416419fed0 con 0x7f4164102920 2024-03-08T23:50:15.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.584+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41641a0270 con 0x7f4164102920 2024-03-08T23:50:15.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.585+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4164068aa0 con 0x7f4164102920 2024-03-08T23:50:15.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.586+0000 7f4161ffb700 1 -- 172.21.15.62:0/27633400 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f415802a070 con 0x7f4164102920 2024-03-08T23:50:15.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.587+0000 7f4161ffb700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f41540a2890 0x7f41540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:15.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.587+0000 7f4161ffb700 1 -- 172.21.15.62:0/27633400 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f41580ceff0 con 0x7f4164102920 2024-03-08T23:50:15.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.587+0000 7f4163fff700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f41540a2890 0x7f41540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:15.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.587+0000 7f4163fff700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f41540a2890 0x7f41540a4d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4150004630 tx=0x7f415000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:50:15.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.590+0000 7f4161ffb700 1 -- 172.21.15.62:0/27633400 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4158013070 con 0x7f4164102920 2024-03-08T23:50:15.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:15.802+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4164062220 con 0x7f41540a2890 2024-03-08T23:50:15.800 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:15 smithi062 ceph-mon[36129]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:15 smithi192 ceph-mon[35995]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:15.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:15 smithi022 ceph-mon[31894]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:16.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:50:16.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:50:16.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:16 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:16.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:50:16.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:50:16.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:16 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:16.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:50:16.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:50:16.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:16 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:17.699 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:17 smithi062 ceph-mon[36129]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:17.699 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:17 smithi062 ceph-mon[36129]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:50:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:17 smithi192 ceph-mon[35995]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:17.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:17 smithi192 ceph-mon[35995]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:50:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:17 smithi022 ceph-mon[31894]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:17 smithi022 ceph-mon[31894]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:50:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:19 smithi062 ceph-mon[36129]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/2151976719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]: dispatch 2024-03-08T23:50:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:19 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]: dispatch 2024-03-08T23:50:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:19 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]': finished 2024-03-08T23:50:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:19 smithi062 ceph-mon[36129]: osdmap e69: 11 total, 10 up, 11 in 2024-03-08T23:50:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:19 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:19.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.62:0/1369878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:50:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:19 smithi192 ceph-mon[35995]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/2151976719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]: dispatch 2024-03-08T23:50:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:19 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]: dispatch 2024-03-08T23:50:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:19 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]': finished 2024-03-08T23:50:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:19 smithi192 ceph-mon[35995]: osdmap e69: 11 total, 10 up, 11 in 2024-03-08T23:50:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:19 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:19.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.62:0/1369878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:50:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:19 smithi022 ceph-mon[31894]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/2151976719' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]: dispatch 2024-03-08T23:50:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:19 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]: dispatch 2024-03-08T23:50:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:19 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0094504e-2371-485f-b601-d2e36f3062ae"}]': finished 2024-03-08T23:50:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:19 smithi022 ceph-mon[31894]: osdmap e69: 11 total, 10 up, 11 in 2024-03-08T23:50:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:19 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.62:0/1369878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:50:21.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:21 smithi062 ceph-mon[36129]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:21.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:21 smithi192 ceph-mon[35995]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:21.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:21 smithi022 ceph-mon[31894]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:23.843 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:23 smithi062 ceph-mon[36129]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:23 smithi192 ceph-mon[35995]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:23.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:23 smithi022 ceph-mon[31894]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:25.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:25 smithi062 ceph-mon[36129]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:25 smithi192 ceph-mon[35995]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:25.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:25 smithi022 ceph-mon[31894]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:26.812 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-08T23:50:26.813 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:26.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-08T23:50:26.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:26.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-08T23:50:26.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:27.846 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:27 smithi062 ceph-mon[36129]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:27.846 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:27 smithi062 ceph-mon[36129]: Deploying daemon osd.10 on smithi062 2024-03-08T23:50:27.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:27 smithi192 ceph-mon[35995]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:27.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:27 smithi192 ceph-mon[35995]: Deploying daemon osd.10 on smithi062 2024-03-08T23:50:27.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:27 smithi022 ceph-mon[31894]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:27.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:27 smithi022 ceph-mon[31894]: Deploying daemon osd.10 on smithi062 2024-03-08T23:50:29.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:29 smithi062 ceph-mon[36129]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:29.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:29 smithi192 ceph-mon[35995]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:29.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:29 smithi022 ceph-mon[31894]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:31.570 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:31 smithi062 ceph-mon[36129]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:31.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:31 smithi192 ceph-mon[35995]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:31.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:31 smithi022 ceph-mon[31894]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:33.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:32 smithi062 ceph-mon[36129]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:33.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:32 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:33.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:32 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:33.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:32 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:33.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:32 smithi192 ceph-mon[35995]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:33.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:32 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:33.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:32 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:33.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:32 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:33.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:32 smithi022 ceph-mon[31894]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:33.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:32 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:33.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:32 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:33.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:32 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:35.094 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:34 smithi062 ceph-mon[36129]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:35.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:34 smithi192 ceph-mon[35995]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:35.419 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 10 on host 'smithi062' 2024-03-08T23:50:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.421+0000 7f4161ffb700 1 -- 172.21.15.62:0/27633400 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4164062220 con 0x7f41540a2890 2024-03-08T23:50:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f41540a2890 msgr2=0x7f41540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f41540a2890 0x7f41540a4d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4150004630 tx=0x7f415000b3c0 comp rx=0 tx=0).stop 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 msgr2=0x7f4164197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164197fe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41580098e0 tx=0x7f4158005e80 comp rx=0 tx=0).stop 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 shutdown_connections 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164107b80 0x7f416419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4164102920 0x7f4164197fe0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f41540a2890 0x7f41540a4d50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 --2- 172.21.15.62:0/27633400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4164105250 0x7f4164198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 >> 172.21.15.62:0/27633400 conn(0x7f41640fc2b0 msgr2=0x7f4164103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 shutdown_connections 2024-03-08T23:50:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-03-08T23:50:35.423+0000 7f416b181700 1 -- 172.21.15.62:0/27633400 wait complete. 2024-03-08T23:50:35.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:34 smithi022 ceph-mon[31894]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:36.127 DEBUG:teuthology.orchestra.run.smithi062:osd.10> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.10.service 2024-03-08T23:50:36.129 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2024-03-08T23:50:36.129 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-08T23:50:36.375 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:50:36.496 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:36.496 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:36 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:36.496 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:36 smithi192 ceph-mon[35995]: from='osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T23:50:36.496 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:36 smithi192 ceph-mon[35995]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T23:50:36.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:36.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:36 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:36.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:36 smithi022 ceph-mon[31894]: from='osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T23:50:36.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:36 smithi022 ceph-mon[31894]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T23:50:36.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:36 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:36.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:36 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:36.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:36 smithi062 ceph-mon[36129]: from='osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T23:50:36.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:36 smithi062 ceph-mon[36129]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: from='osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: osdmap e70: 11 total, 10 up, 11 in 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:50:37.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:37.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-08T23:50:37.418+0000 7f88cb7fd700 -1 osd.10 0 waiting for initial osdmap 2024-03-08T23:50:37.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 08 23:50:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-08T23:50:37.429+0000 7f88c75f0700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:50:37.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:37.710 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-08T23:50:37.710 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: from='osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:50:37.710 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:37.710 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: osdmap e70: 11 total, 10 up, 11 in 2024-03-08T23:50:37.710 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:37.710 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:50:37.710 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:37.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:37.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-08T23:50:37.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: from='osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:50:37.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:37.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: osdmap e70: 11 total, 10 up, 11 in 2024-03-08T23:50:37.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:37.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-08T23:50:37.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:38.663 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:38 smithi192 ceph-mon[35995]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:50:38.663 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:38 smithi192 ceph-mon[35995]: osdmap e71: 11 total, 10 up, 11 in 2024-03-08T23:50:38.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:38.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:38 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:38.669 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:38 smithi062 ceph-mon[36129]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:50:38.670 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:38 smithi062 ceph-mon[36129]: osdmap e71: 11 total, 10 up, 11 in 2024-03-08T23:50:38.670 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:38.670 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:38 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:38.673 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-08T23:50:38.673 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-08T23:50:38.673 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-08T23:50:38.673 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-08T23:50:38.673 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117184 s, 895 MB/s 2024-03-08T23:50:38.673 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-08T23:50:38.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:38 smithi022 ceph-mon[31894]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-08T23:50:38.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:38 smithi022 ceph-mon[31894]: osdmap e71: 11 total, 10 up, 11 in 2024-03-08T23:50:38.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:38.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:38 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:39.471 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2024-03-08T23:50:39.712 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:50:39.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:50:39.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:50:39.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:39.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:39.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] boot 2024-03-08T23:50:39.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: osdmap e72: 11 total, 11 up, 11 in 2024-03-08T23:50:39.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:39.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:39 smithi022 ceph-mon[31894]: osdmap e73: 11 total, 11 up, 11 in 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] boot 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: osdmap e72: 11 total, 11 up, 11 in 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:39 smithi062 ceph-mon[36129]: osdmap e73: 11 total, 11 up, 11 in 2024-03-08T23:50:39.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:50:39.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:50:39.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-03-08T23:50:39.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:39.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: osd.10 [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] boot 2024-03-08T23:50:39.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: osdmap e72: 11 total, 11 up, 11 in 2024-03-08T23:50:39.833 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:50:39.834 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:39 smithi192 ceph-mon[35995]: osdmap e73: 11 total, 11 up, 11 in 2024-03-08T23:50:40.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:40.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: osdmap e74: 11 total, 11 up, 11 in 2024-03-08T23:50:40.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi062 2024-03-08T23:50:40.835 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi062 to 4270M 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:40.836 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:41.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:41.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: osdmap e74: 11 total, 11 up, 11 in 2024-03-08T23:50:41.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi062 2024-03-08T23:50:41.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi062 to 4270M 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: osdmap e74: 11 total, 11 up, 11 in 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi062 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi062 to 4270M 2024-03-08T23:50:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:41.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:41.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:41.388 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 -- 172.21.15.192:0/3526531382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 msgr2=0x7f1d7410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/3526531382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d7410c1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00b0a0 tx=0x7f1d6c00b3b0 comp rx=0 tx=0).stop 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 -- 172.21.15.192:0/3526531382 shutdown_connections 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/3526531382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d7410c1a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/3526531382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d74075d90 0x7f1d740741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/3526531382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d740747c0 0x7f1d74074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 -- 172.21.15.192:0/3526531382 >> 172.21.15.192:0/3526531382 conn(0x7f1d740fd3f0 msgr2=0x7f1d740ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 -- 172.21.15.192:0/3526531382 shutdown_connections 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.376+0000 7f1d79c8c700 1 -- 172.21.15.192:0/3526531382 wait complete. 2024-03-08T23:50:41.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 Processor -- start 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 -- start start 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d740747c0 0x7f1d7419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d74075d90 0x7f1d7419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d741a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7410fc50 con 0x7f1d74075d90 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7410fad0 con 0x7f1d740747c0 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d79c8c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7410fdd0 con 0x7f1d74109d30 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d73fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d741a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d73fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d741a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:36828/0 (socket says 172.21.15.192:36828) 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d73fff700 1 -- 172.21.15.192:0/1702795559 learned_addr learned my addr 172.21.15.192:0/1702795559 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d6affd700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d74075d90 0x7f1d7419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.377+0000 7f1d737fe700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d740747c0 0x7f1d7419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:41.390 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d73fff700 1 -- 172.21.15.192:0/1702795559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d740747c0 msgr2=0x7f1d7419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d73fff700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d740747c0 0x7f1d7419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d73fff700 1 -- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d74075d90 msgr2=0x7f1d7419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d73fff700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d74075d90 0x7f1d7419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d73fff700 1 -- 172.21.15.192:0/1702795559 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d741a1e00 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d737fe700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d740747c0 0x7f1d7419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d6affd700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d74075d90 0x7f1d7419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d73fff700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d741a1760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d010 tx=0x7f1d6c004e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d717fa700 1 -- 172.21.15.192:0/1702795559 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c021070 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d741a2090 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d717fa700 1 -- 172.21.15.192:0/1702795559 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d6c004020 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d717fa700 1 -- 172.21.15.192:0/1702795559 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c01d6b0 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.378+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d741a2590 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.379+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d74068aa0 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.380+0000 7f1d717fa700 1 -- 172.21.15.192:0/1702795559 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1d6c0041d0 con 0x7f1d74109d30 2024-03-08T23:50:41.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.381+0000 7f1d717fa700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1d600a2890 0x7f1d600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:50:41.392 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.381+0000 7f1d737fe700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1d600a2890 0x7f1d600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:50:41.392 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.381+0000 7f1d717fa700 1 -- 172.21.15.192:0/1702795559 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f1d6c00f020 con 0x7f1d74109d30 2024-03-08T23:50:41.392 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.381+0000 7f1d737fe700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1d600a2890 0x7f1d600a4d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d64007900 tx=0x7f1d64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:50:41.392 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.384+0000 7f1d717fa700 1 -- 172.21.15.192:0/1702795559 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1d6c02a020 con 0x7f1d74109d30 2024-03-08T23:50:41.583 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:50:41.586+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d74062220 con 0x7f1d600a2890 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: osdmap e75: 11 total, 11 up, 11 in 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:50:41.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:41 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: osdmap e75: 11 total, 11 up, 11 in 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:50:42.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:41 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: osdmap e75: 11 total, 11 up, 11 in 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T23:50:42.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:41 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:43.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:42 smithi062 ceph-mon[36129]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:43.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:42 smithi062 ceph-mon[36129]: osdmap e76: 11 total, 11 up, 11 in 2024-03-08T23:50:43.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:42 smithi192 ceph-mon[35995]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:43.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:42 smithi192 ceph-mon[35995]: osdmap e76: 11 total, 11 up, 11 in 2024-03-08T23:50:43.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:42 smithi022 ceph-mon[31894]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:43.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:42 smithi022 ceph-mon[31894]: osdmap e76: 11 total, 11 up, 11 in 2024-03-08T23:50:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:44 smithi062 ceph-mon[36129]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:44 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/2048022109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]: dispatch 2024-03-08T23:50:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:44 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]: dispatch 2024-03-08T23:50:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:44 smithi062 ceph-mon[36129]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]': finished 2024-03-08T23:50:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:44 smithi062 ceph-mon[36129]: osdmap e77: 12 total, 11 up, 12 in 2024-03-08T23:50:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:50:45.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:44 smithi192 ceph-mon[35995]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:45.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:44 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/2048022109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]: dispatch 2024-03-08T23:50:45.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:44 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]: dispatch 2024-03-08T23:50:45.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:44 smithi192 ceph-mon[35995]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]': finished 2024-03-08T23:50:45.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:44 smithi192 ceph-mon[35995]: osdmap e77: 12 total, 11 up, 12 in 2024-03-08T23:50:45.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:50:45.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:44 smithi022 ceph-mon[31894]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:45.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:44 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/2048022109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]: dispatch 2024-03-08T23:50:45.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:44 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]: dispatch 2024-03-08T23:50:45.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:44 smithi022 ceph-mon[31894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "182eae10-50a1-43f2-b776-13486e2939b3"}]': finished 2024-03-08T23:50:45.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:44 smithi022 ceph-mon[31894]: osdmap e77: 12 total, 11 up, 12 in 2024-03-08T23:50:45.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:50:46.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:45 smithi062 ceph-mon[36129]: from='client.? 172.21.15.192:0/2136985587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:50:46.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:45 smithi192 ceph-mon[35995]: from='client.? 172.21.15.192:0/2136985587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:50:46.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:45 smithi022 ceph-mon[31894]: from='client.? 172.21.15.192:0/2136985587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T23:50:47.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:46 smithi062 ceph-mon[36129]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:50:47.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:46 smithi192 ceph-mon[35995]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:50:47.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:46 smithi022 ceph-mon[31894]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T23:50:49.099 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:48 smithi192 ceph-mon[35995]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-08T23:50:49.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:48 smithi062 ceph-mon[36129]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-08T23:50:49.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:48 smithi022 ceph-mon[31894]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-08T23:50:51.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:50 smithi192 ceph-mon[35995]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T23:50:51.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:50 smithi062 ceph-mon[36129]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T23:50:51.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:50 smithi022 ceph-mon[31894]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T23:50:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:52 smithi192 ceph-mon[35995]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:50:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:52 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-08T23:50:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:52 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:52 smithi192 ceph-mon[35995]: Deploying daemon osd.11 on smithi192 2024-03-08T23:50:53.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:52 smithi062 ceph-mon[36129]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:50:53.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:52 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-08T23:50:53.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:52 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:53.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:52 smithi062 ceph-mon[36129]: Deploying daemon osd.11 on smithi192 2024-03-08T23:50:53.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:52 smithi022 ceph-mon[31894]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:50:53.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:52 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-08T23:50:53.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:52 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:50:53.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:52 smithi022 ceph-mon[31894]: Deploying daemon osd.11 on smithi192 2024-03-08T23:50:54.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:54 smithi192 ceph-mon[35995]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:50:55.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:54 smithi062 ceph-mon[36129]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:50:55.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:54 smithi022 ceph-mon[31894]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T23:50:57.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:56 smithi062 ceph-mon[36129]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-08T23:50:57.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:56 smithi192 ceph-mon[35995]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-08T23:50:57.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:56 smithi022 ceph-mon[31894]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-08T23:50:59.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:59 smithi062 ceph-mon[36129]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:59.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:59.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:59.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:50:59 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:59.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:59 smithi192 ceph-mon[35995]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:59.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:59.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:59.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:50:59 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:50:59.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:59 smithi022 ceph-mon[31894]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:50:59.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:59.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:50:59.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:50:59 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:01.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:01 smithi062 ceph-mon[36129]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:01.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:01 smithi192 ceph-mon[35995]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:01.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:01 smithi022 ceph-mon[31894]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:01.553 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2024-03-08T23:51:01.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.556+0000 7f1d717fa700 1 -- 172.21.15.192:0/1702795559 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1d74062220 con 0x7f1d600a2890 2024-03-08T23:51:01.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1d600a2890 msgr2=0x7f1d600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:01.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1d600a2890 0x7f1d600a4d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d64007900 tx=0x7f1d64008040 comp rx=0 tx=0).stop 2024-03-08T23:51:01.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 msgr2=0x7f1d741a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:01.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d741a1760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d010 tx=0x7f1d6c004e80 comp rx=0 tx=0).stop 2024-03-08T23:51:01.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 shutdown_connections 2024-03-08T23:51:01.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d74109d30 0x7f1d741a1760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:01.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d740747c0 0x7f1d7419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:01.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1d600a2890 0x7f1d600a4d50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:01.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 --2- 172.21.15.192:0/1702795559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d74075d90 0x7f1d7419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:01.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 >> 172.21.15.192:0/1702795559 conn(0x7f1d740fd3f0 msgr2=0x7f1d74105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:01.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 shutdown_connections 2024-03-08T23:51:01.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:01.559+0000 7f1d79c8c700 1 -- 172.21.15.192:0/1702795559 wait complete. 2024-03-08T23:51:02.231 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.11.service 2024-03-08T23:51:02.233 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-08T23:51:02.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-08T23:51:02.490 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:51:02.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:02 smithi062 ceph-mon[36129]: from='osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T23:51:02.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:02 smithi062 ceph-mon[36129]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T23:51:02.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:02 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:02 smithi022 ceph-mon[31894]: from='osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T23:51:02.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:02 smithi022 ceph-mon[31894]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T23:51:02.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:02 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:02 smithi192 ceph-mon[35995]: from='osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T23:51:02.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:02 smithi192 ceph-mon[35995]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T23:51:02.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:02.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:02 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:03 smithi062 ceph-mon[36129]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:03 smithi062 ceph-mon[36129]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-08T23:51:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:03 smithi062 ceph-mon[36129]: osdmap e78: 12 total, 11 up, 12 in 2024-03-08T23:51:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:03 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:03 smithi062 ceph-mon[36129]: from='osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:51:03.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:03 smithi062 ceph-mon[36129]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:51:03.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:03 smithi022 ceph-mon[31894]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:03.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:03 smithi022 ceph-mon[31894]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-08T23:51:03.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:03 smithi022 ceph-mon[31894]: osdmap e78: 12 total, 11 up, 12 in 2024-03-08T23:51:03.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:03 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:03.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:03 smithi022 ceph-mon[31894]: from='osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:51:03.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:03 smithi022 ceph-mon[31894]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:51:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-mon[35995]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-mon[35995]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-08T23:51:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-mon[35995]: osdmap e78: 12 total, 11 up, 12 in 2024-03-08T23:51:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-mon[35995]: from='osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:51:03.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-mon[35995]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-08T23:51:03.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-08T23:51:03.557+0000 7f3749c91700 -1 osd.11 0 waiting for initial osdmap 2024-03-08T23:51:03.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 08 23:51:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-08T23:51:03.571+0000 7f3745a84700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 -- 172.21.15.22:0/1933429494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4102800 msgr2=0x7f23a41090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 --2- 172.21.15.22:0/1933429494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4102800 0x7f23a41090b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f239400b0d0 tx=0x7f239400b3e0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 -- 172.21.15.22:0/1933429494 shutdown_connections 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 --2- 172.21.15.22:0/1933429494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4102800 0x7f23a41090b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 --2- 172.21.15.22:0/1933429494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4100c60 0x7f23a4101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 --2- 172.21.15.22:0/1933429494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23a4101e60 0x7f23a41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 -- 172.21.15.22:0/1933429494 >> 172.21.15.22:0/1933429494 conn(0x7f23a40fc200 msgr2=0x7f23a40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 -- 172.21.15.22:0/1933429494 shutdown_connections 2024-03-08T23:51:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 -- 172.21.15.22:0/1933429494 wait complete. 2024-03-08T23:51:04.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.415+0000 7f23a9802700 1 Processor -- start 2024-03-08T23:51:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a9802700 1 -- start start 2024-03-08T23:51:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a9802700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4100c60 0x7f23a4104bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a9802700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4101e60 0x7f23a4103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a9802700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23a4102800 0x7f23a4103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a9802700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a410c830 con 0x7f23a4100c60 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a9802700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a410c6b0 con 0x7f23a4102800 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a9802700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a410c9b0 con 0x7f23a4101e60 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4100c60 0x7f23a4104bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4100c60 0x7f23a4104bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37848/0 (socket says 172.21.15.22:37848) 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 -- 172.21.15.22:0/3051939834 learned_addr learned my addr 172.21.15.22:0/3051939834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a27fc700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4101e60 0x7f23a4103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a37fe700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23a4102800 0x7f23a4103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 -- 172.21.15.22:0/3051939834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4101e60 msgr2=0x7f23a4103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4101e60 0x7f23a4103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 -- 172.21.15.22:0/3051939834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23a4102800 msgr2=0x7f23a4103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:04.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23a4102800 0x7f23a4103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 -- 172.21.15.22:0/3051939834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4104030 con 0x7f23a4100c60 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a27fc700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4101e60 0x7f23a4103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.416+0000 7f23a2ffd700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4100c60 0x7f23a4104bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f239c00d980 tx=0x7f239c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.417+0000 7f23a8800700 1 -- 172.21.15.22:0/3051939834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c017030 con 0x7f23a4100c60 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.417+0000 7f23a8800700 1 -- 172.21.15.22:0/3051939834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f239c004500 con 0x7f23a4100c60 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.417+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a41a2090 con 0x7f23a4100c60 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.417+0000 7f23a8800700 1 -- 172.21.15.22:0/3051939834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c003aa0 con 0x7f23a4100c60 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.417+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a41a25a0 con 0x7f23a4100c60 2024-03-08T23:51:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.418+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a4068aa0 con 0x7f23a4100c60 2024-03-08T23:51:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.420+0000 7f23a8800700 1 -- 172.21.15.22:0/3051939834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f239c01a070 con 0x7f23a4100c60 2024-03-08T23:51:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.420+0000 7f23a8800700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23900a2890 0x7f23900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.420+0000 7f23a27fc700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23900a2890 0x7f23900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.420+0000 7f23a8800700 1 -- 172.21.15.22:0/3051939834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f239c0ce1f0 con 0x7f23a4100c60 2024-03-08T23:51:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.421+0000 7f23a27fc700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23900a2890 0x7f23900a4d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f238c005950 tx=0x7f238c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:04.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.424+0000 7f23a8800700 1 -- 172.21.15.22:0/3051939834 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f239c09b7b0 con 0x7f23a4100c60 2024-03-08T23:51:04.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.632+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f23a41a2980 con 0x7f23a4100c60 2024-03-08T23:51:04.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.632+0000 7f23a8800700 1 -- 172.21.15.22:0/3051939834 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f239c09b0b0 con 0x7f23a4100c60 2024-03-08T23:51:04.630 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23900a2890 msgr2=0x7f23900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23900a2890 0x7f23900a4d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f238c005950 tx=0x7f238c0058e0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4100c60 msgr2=0x7f23a4104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4100c60 0x7f23a4104bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f239c00d980 tx=0x7f239c00dc90 comp rx=0 tx=0).stop 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 shutdown_connections 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23a4102800 0x7f23a4103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f23900a2890 0x7f23900a4d50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f23a4100c60 0x7f23a4104bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 --2- 172.21.15.22:0/3051939834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4101e60 0x7f23a4103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 >> 172.21.15.22:0/3051939834 conn(0x7f23a40fc200 msgr2=0x7f23a40fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 shutdown_connections 2024-03-08T23:51:04.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:04.635+0000 7f23a9802700 1 -- 172.21.15.22:0/3051939834 wait complete. 2024-03-08T23:51:04.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:04 smithi022 ceph-mon[31894]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:51:04.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:04 smithi022 ceph-mon[31894]: osdmap e79: 12 total, 11 up, 12 in 2024-03-08T23:51:04.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:04 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:04.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:04 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:04.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:04 smithi062 ceph-mon[36129]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:51:04.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:04 smithi062 ceph-mon[36129]: osdmap e79: 12 total, 11 up, 12 in 2024-03-08T23:51:04.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:04 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:04.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:04 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:04.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:04 smithi192 ceph-mon[35995]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-08T23:51:04.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:04 smithi192 ceph-mon[35995]: osdmap e79: 12 total, 11 up, 12 in 2024-03-08T23:51:04.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:04 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:04.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:04 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:05.420 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1709941864,"num_in_osds":12,"osd_in_since":1709941844,"num_remapped_pgs":0} 2024-03-08T23:51:05.420 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-08T23:51:05.673 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:51:05.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: purged_snaps scrub starts 2024-03-08T23:51:05.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: purged_snaps scrub ok 2024-03-08T23:51:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] boot 2024-03-08T23:51:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: osdmap e80: 12 total, 12 up, 12 in 2024-03-08T23:51:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:05.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:05 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3051939834' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: purged_snaps scrub starts 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: purged_snaps scrub ok 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] boot 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: osdmap e80: 12 total, 12 up, 12 in 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:05.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:05 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3051939834' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: purged_snaps scrub starts 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: purged_snaps scrub ok 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: osd.11 [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] boot 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: osdmap e80: 12 total, 12 up, 12 in 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:51:05.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:05 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3051939834' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-08T23:51:06.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: osdmap e81: 12 total, 12 up, 12 in 2024-03-08T23:51:06.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:06.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:06.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:06 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: osdmap e81: 12 total, 12 up, 12 in 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:06.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:06.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:06 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:06.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: osdmap e81: 12 total, 12 up, 12 in 2024-03-08T23:51:06.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:06.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:06 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.361+0000 7f75ac11e700 1 -- 172.21.15.22:0/2257859486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406ae80 msgr2=0x7f75a410eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.361+0000 7f75ac11e700 1 --2- 172.21.15.22:0/2257859486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406ae80 0x7f75a410eca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f759c00b0d0 tx=0x7f759c00b3e0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 -- 172.21.15.22:0/2257859486 shutdown_connections 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 --2- 172.21.15.22:0/2257859486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406ae80 0x7f75a410eca0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 --2- 172.21.15.22:0/2257859486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a4107e60 0x7f75a4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 --2- 172.21.15.22:0/2257859486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75a406a4e0 0x7f75a406a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 -- 172.21.15.22:0/2257859486 >> 172.21.15.22:0/2257859486 conn(0x7f75a4076d00 msgr2=0x7f75a4077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 -- 172.21.15.22:0/2257859486 shutdown_connections 2024-03-08T23:51:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 -- 172.21.15.22:0/2257859486 wait complete. 2024-03-08T23:51:07.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 Processor -- start 2024-03-08T23:51:07.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.362+0000 7f75ac11e700 1 -- start start 2024-03-08T23:51:07.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75ac11e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406a4e0 0x7f75a41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:07.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75ac11e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a406ae80 0x7f75a4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:07.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75ac11e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75a4107e60 0x7f75a419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:07.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75ac11e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41120e0 con 0x7f75a406a4e0 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75ac11e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4111f60 con 0x7f75a406ae80 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75ac11e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4112260 con 0x7f75a4107e60 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406a4e0 0x7f75a41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406a4e0 0x7f75a41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37870/0 (socket says 172.21.15.22:37870) 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 -- 172.21.15.22:0/3890526566 learned_addr learned my addr 172.21.15.22:0/3890526566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a96b9700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a406ae80 0x7f75a4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75aa6bb700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75a4107e60 0x7f75a419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 -- 172.21.15.22:0/3890526566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75a4107e60 msgr2=0x7f75a419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75a4107e60 0x7f75a419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 -- 172.21.15.22:0/3890526566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a406ae80 msgr2=0x7f75a4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a406ae80 0x7f75a4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 -- 172.21.15.22:0/3890526566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a419d4e0 con 0x7f75a406a4e0 2024-03-08T23:51:07.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f75a9eba700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406a4e0 0x7f75a41984d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f75a000d950 tx=0x7f75a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f759affd700 1 -- 172.21.15.22:0/3890526566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0016070 con 0x7f75a406a4e0 2024-03-08T23:51:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.363+0000 7f759affd700 1 -- 172.21.15.22:0/3890526566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75a000fd70 con 0x7f75a406a4e0 2024-03-08T23:51:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.364+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a419d7d0 con 0x7f75a406a4e0 2024-03-08T23:51:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.364+0000 7f759affd700 1 -- 172.21.15.22:0/3890526566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000e610 con 0x7f75a406a4e0 2024-03-08T23:51:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.364+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a41a5050 con 0x7f75a406a4e0 2024-03-08T23:51:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.364+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a4103510 con 0x7f75a406a4e0 2024-03-08T23:51:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.366+0000 7f759affd700 1 -- 172.21.15.22:0/3890526566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f75a0019070 con 0x7f75a406a4e0 2024-03-08T23:51:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.367+0000 7f759affd700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f75900a2910 0x7f75900a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.367+0000 7f75a96b9700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f75900a2910 0x7f75900a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.367+0000 7f759affd700 1 -- 172.21.15.22:0/3890526566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75a00cdff0 con 0x7f75a406a4e0 2024-03-08T23:51:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.367+0000 7f75a96b9700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f75900a2910 0x7f75900a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f75a4077300 tx=0x7f759400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:07.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.371+0000 7f759affd700 1 -- 172.21.15.22:0/3890526566 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f75a009b4d0 con 0x7f75a406a4e0 2024-03-08T23:51:07.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.575+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75a4199830 con 0x7f75a406a4e0 2024-03-08T23:51:07.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.576+0000 7f759affd700 1 -- 172.21.15.22:0/3890526566 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f75a009add0 con 0x7f75a406a4e0 2024-03-08T23:51:07.573 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:07.573 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:05.560356+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:51:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f75900a2910 msgr2=0x7f75900a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f75900a2910 0x7f75900a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f75a4077300 tx=0x7f759400b3c0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406a4e0 msgr2=0x7f75a41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406a4e0 0x7f75a41984d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f75a000d950 tx=0x7f75a000dc60 comp rx=0 tx=0).stop 2024-03-08T23:51:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 shutdown_connections 2024-03-08T23:51:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75a4107e60 0x7f75a419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f75900a2910 0x7f75900a4dd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75a406a4e0 0x7f75a41984d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 --2- 172.21.15.22:0/3890526566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a406ae80 0x7f75a4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 >> 172.21.15.22:0/3890526566 conn(0x7f75a4076d00 msgr2=0x7f75a40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 shutdown_connections 2024-03-08T23:51:07.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:07.578+0000 7f75ac11e700 1 -- 172.21.15.22:0/3890526566 wait complete. 2024-03-08T23:51:07.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:07 smithi062 ceph-mon[36129]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:07.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:07 smithi062 ceph-mon[36129]: Detected new or changed devices on smithi192 2024-03-08T23:51:07.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:07 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi192 to 4270M 2024-03-08T23:51:07.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:07.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:07.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:07 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:07 smithi192 ceph-mon[35995]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:07 smithi192 ceph-mon[35995]: Detected new or changed devices on smithi192 2024-03-08T23:51:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:07 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi192 to 4270M 2024-03-08T23:51:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:07.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:07 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:07.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:07 smithi022 ceph-mon[31894]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:07.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:07 smithi022 ceph-mon[31894]: Detected new or changed devices on smithi192 2024-03-08T23:51:07.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:07 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi192 to 4270M 2024-03-08T23:51:07.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:07.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:07.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:07 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:08.336 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T23:51:08.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-08T23:51:08.582 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:51:08.836 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3890526566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:08.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:08 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3890526566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:08.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:08 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3890526566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:09 smithi062 ceph-mon[36129]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:09.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:09 smithi192 ceph-mon[35995]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:09.967 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:09 smithi022 ceph-mon[31894]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 -- 172.21.15.22:0/1751394531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c100ce0 msgr2=0x7fdf7c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 --2- 172.21.15.22:0/1751394531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c100ce0 0x7fdf7c101140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009990 tx=0x7fdf70009ca0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 -- 172.21.15.22:0/1751394531 shutdown_connections 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 --2- 172.21.15.22:0/1751394531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf7c101680 0x7fdf7c10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 --2- 172.21.15.22:0/1751394531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c106d20 0x7fdf7c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 --2- 172.21.15.22:0/1751394531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c100ce0 0x7fdf7c101140 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 -- 172.21.15.22:0/1751394531 >> 172.21.15.22:0/1751394531 conn(0x7fdf7c077c10 msgr2=0x7fdf7c078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 -- 172.21.15.22:0/1751394531 shutdown_connections 2024-03-08T23:51:10.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.328+0000 7fdf82510700 1 -- 172.21.15.22:0/1751394531 wait complete. 2024-03-08T23:51:10.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 Processor -- start 2024-03-08T23:51:10.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 -- start start 2024-03-08T23:51:10.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c100ce0 0x7fdf7c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c101680 0x7fdf7c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf7c106d20 0x7fdf7c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c111020 con 0x7fdf7c101680 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c110ea0 con 0x7fdf7c106d20 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf82510700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c1111a0 con 0x7fdf7c100ce0 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf7b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c101680 0x7fdf7c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf7b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c101680 0x7fdf7c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37896/0 (socket says 172.21.15.22:37896) 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf7b7fe700 1 -- 172.21.15.22:0/806731162 learned_addr learned my addr 172.21.15.22:0/806731162 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.329+0000 7fdf7bfff700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c100ce0 0x7fdf7c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf80aad700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf7c106d20 0x7fdf7c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf7b7fe700 1 -- 172.21.15.22:0/806731162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c100ce0 msgr2=0x7fdf7c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf7b7fe700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c100ce0 0x7fdf7c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf7b7fe700 1 -- 172.21.15.22:0/806731162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf7c106d20 msgr2=0x7fdf7c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:10.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf7b7fe700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf7c106d20 0x7fdf7c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf7b7fe700 1 -- 172.21.15.22:0/806731162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c19d5a0 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf7bfff700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c100ce0 0x7fdf7c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf80aad700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf7c106d20 0x7fdf7c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf7b7fe700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c101680 0x7fdf7c198ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000c00 tx=0x7fdf700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf797fa700 1 -- 172.21.15.22:0/806731162 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70027070 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf797fa700 1 -- 172.21.15.22:0/806731162 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf70004050 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c19d830 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf797fa700 1 -- 172.21.15.22:0/806731162 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000f460 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.330+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c1a5110 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.331+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7c068aa0 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.332+0000 7fdf797fa700 1 -- 172.21.15.22:0/806731162 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdf7002a070 con 0x7fdf7c101680 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.333+0000 7fdf797fa700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdf640a2910 0x7fdf640a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.333+0000 7fdf7bfff700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdf640a2910 0x7fdf640a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:10.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.333+0000 7fdf797fa700 1 -- 172.21.15.22:0/806731162 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf700cf4f0 con 0x7fdf7c101680 2024-03-08T23:51:10.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.333+0000 7fdf7bfff700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdf640a2910 0x7fdf640a4dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00bcf0 tx=0x7fdf6c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:10.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.337+0000 7fdf797fa700 1 -- 172.21.15.22:0/806731162 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdf7005e260 con 0x7fdf7c101680 2024-03-08T23:51:10.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.538+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf7c1a52e0 con 0x7fdf7c101680 2024-03-08T23:51:10.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.539+0000 7fdf797fa700 1 -- 172.21.15.22:0/806731162 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf70013070 con 0x7fdf7c101680 2024-03-08T23:51:10.535 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:51:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdf640a2910 msgr2=0x7fdf640a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdf640a2910 0x7fdf640a4dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00bcf0 tx=0x7fdf6c00b2c0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c101680 msgr2=0x7fdf7c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c101680 0x7fdf7c198ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000c00 tx=0x7fdf700047b0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 shutdown_connections 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf7c106d20 0x7fdf7c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c100ce0 0x7fdf7c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdf640a2910 0x7fdf640a4dd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 --2- 172.21.15.22:0/806731162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf7c101680 0x7fdf7c198ad0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 >> 172.21.15.22:0/806731162 conn(0x7fdf7c077c10 msgr2=0x7fdf7c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 shutdown_connections 2024-03-08T23:51:10.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:10.541+0000 7fdf82510700 1 -- 172.21.15.22:0/806731162 wait complete. 2024-03-08T23:51:10.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:10 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/806731162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:10.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:10 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/806731162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:10.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:10 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/806731162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:11.319 INFO:tasks.cephadm:Adding mds.a on smithi022 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.d on smithi022 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.g on smithi022 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.j on smithi022 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.b on smithi062 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.e on smithi062 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.h on smithi062 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.k on smithi062 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.c on smithi192 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.f on smithi192 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.i on smithi192 2024-03-08T23:51:11.320 INFO:tasks.cephadm:Adding mds.l on smithi192 2024-03-08T23:51:11.320 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph orch apply mds all '12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2024-03-08T23:51:11.570 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T23:51:11.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:11 smithi192 ceph-mon[35995]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:11.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:11 smithi062 ceph-mon[36129]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:11.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:11 smithi022 ceph-mon[31894]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 -- 172.21.15.192:0/1065366671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 msgr2=0x7fbb1010a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 --2- 172.21.15.192:0/1065366671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1010a240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800b0a0 tx=0x7fbb0800b3b0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 -- 172.21.15.192:0/1065366671 shutdown_connections 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 --2- 172.21.15.192:0/1065366671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1010a240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 --2- 172.21.15.192:0/1065366671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb10101e00 0x7fbb10102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 --2- 172.21.15.192:0/1065366671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10102f60 0x7fbb101033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 -- 172.21.15.192:0/1065366671 >> 172.21.15.192:0/1065366671 conn(0x7fbb100fd3c0 msgr2=0x7fbb100ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:13.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 -- 172.21.15.192:0/1065366671 shutdown_connections 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.244+0000 7fbb16feb700 1 -- 172.21.15.192:0/1065366671 wait complete. 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 Processor -- start 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 -- start start 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb10101e00 0x7fbb10197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10102f60 0x7fbb101983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1010d930 con 0x7fbb10102f60 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1010d7b0 con 0x7fbb10101e00 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb16feb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1010dab0 con 0x7fbb10103920 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:53312/0 (socket says 172.21.15.192:53312) 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 -- 172.21.15.192:0/2497895064 learned_addr learned my addr 172.21.15.192:0/2497895064 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-08T23:51:13.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb14d87700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb10101e00 0x7fbb10197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb0ffff700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10102f60 0x7fbb101983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 -- 172.21.15.192:0/2497895064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb10101e00 msgr2=0x7fbb10197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb10101e00 0x7fbb10197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 -- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10102f60 msgr2=0x7fbb101983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10102f60 0x7fbb101983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb15588700 1 -- 172.21.15.192:0/2497895064 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1019fac0 con 0x7fbb10103920 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.245+0000 7fbb0ffff700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10102f60 0x7fbb101983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.246+0000 7fbb15588700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1019f420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009cd0 tx=0x7fbb0800be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.246+0000 7fbb0dffb700 1 -- 172.21.15.192:0/2497895064 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08021070 con 0x7fbb10103920 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.246+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1019fd50 con 0x7fbb10103920 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.246+0000 7fbb0dffb700 1 -- 172.21.15.192:0/2497895064 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb0801d930 con 0x7fbb10103920 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.246+0000 7fbb0dffb700 1 -- 172.21.15.192:0/2497895064 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08005580 con 0x7fbb10103920 2024-03-08T23:51:13.245 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.246+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb101a01b0 con 0x7fbb10103920 2024-03-08T23:51:13.246 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.247+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb10068aa0 con 0x7fbb10103920 2024-03-08T23:51:13.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.248+0000 7fbb0dffb700 1 -- 172.21.15.192:0/2497895064 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbb08024070 con 0x7fbb10103920 2024-03-08T23:51:13.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.248+0000 7fbb0dffb700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbb000a2890 0x7fbb000a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:13.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.249+0000 7fbb0dffb700 1 -- 172.21.15.192:0/2497895064 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb0800f030 con 0x7fbb10103920 2024-03-08T23:51:13.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.249+0000 7fbb14d87700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbb000a2890 0x7fbb000a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:13.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.249+0000 7fbb14d87700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbb000a2890 0x7fbb000a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb04004630 tx=0x7fbb0400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:13.249 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.253+0000 7fbb0dffb700 1 -- 172.21.15.192:0/2497895064 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fbb0809d0a0 con 0x7fbb10103920 2024-03-08T23:51:13.458 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.461+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb10062320 con 0x7fbb000a2890 2024-03-08T23:51:13.466 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.469+0000 7fbb0dffb700 1 -- 172.21.15.192:0/2497895064 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fbb10062320 con 0x7fbb000a2890 2024-03-08T23:51:13.466 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2024-03-08T23:51:13.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.471+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbb000a2890 msgr2=0x7fbb000a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.471+0000 7fbb16feb700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbb000a2890 0x7fbb000a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb04004630 tx=0x7fbb0400b3c0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.471+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 msgr2=0x7fbb1019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.471+0000 7fbb16feb700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1019f420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009cd0 tx=0x7fbb0800be60 comp rx=0 tx=0).stop 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 shutdown_connections 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb10103920 0x7fbb1019f420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb10101e00 0x7fbb10197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbb000a2890 0x7fbb000a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 --2- 172.21.15.192:0/2497895064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb10102f60 0x7fbb101983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 >> 172.21.15.192:0/2497895064 conn(0x7fbb100fd3c0 msgr2=0x7fbb10106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 shutdown_connections 2024-03-08T23:51:13.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-08T23:51:13.472+0000 7fbb16feb700 1 -- 172.21.15.192:0/2497895064 wait complete. 2024-03-08T23:51:13.663 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:13.663 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:13.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:13.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:13.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:13.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:13.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:13.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:13.664 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:13 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:13.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:13 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:13 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:14.199 DEBUG:teuthology.orchestra.run.smithi022:mds.a> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.a.service 2024-03-08T23:51:14.202 DEBUG:teuthology.orchestra.run.smithi022:mds.d> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.d.service 2024-03-08T23:51:14.246 DEBUG:teuthology.orchestra.run.smithi022:mds.g> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.g.service 2024-03-08T23:51:14.290 DEBUG:teuthology.orchestra.run.smithi022:mds.j> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.j.service 2024-03-08T23:51:14.334 DEBUG:teuthology.orchestra.run.smithi062:mds.b> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.b.service 2024-03-08T23:51:14.336 DEBUG:teuthology.orchestra.run.smithi062:mds.e> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.e.service 2024-03-08T23:51:14.379 DEBUG:teuthology.orchestra.run.smithi062:mds.h> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.h.service 2024-03-08T23:51:14.426 DEBUG:teuthology.orchestra.run.smithi062:mds.k> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.k.service 2024-03-08T23:51:14.469 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.c.service 2024-03-08T23:51:14.471 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.f.service 2024-03-08T23:51:14.514 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.i.service 2024-03-08T23:51:14.557 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.l.service 2024-03-08T23:51:14.600 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-08T23:51:14.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 -- 172.21.15.22:0/555470563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 msgr2=0x7f4308101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 --2- 172.21.15.22:0/555470563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308101740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009900 tx=0x7f42fc02bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 -- 172.21.15.22:0/555470563 shutdown_connections 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 --2- 172.21.15.22:0/555470563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4308102ed0 0x7f4308109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 --2- 172.21.15.22:0/555470563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4308102510 0x7f4308102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 --2- 172.21.15.22:0/555470563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308101740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 -- 172.21.15.22:0/555470563 >> 172.21.15.22:0/555470563 conn(0x7f43080fc8d0 msgr2=0x7f43080fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 -- 172.21.15.22:0/555470563 shutdown_connections 2024-03-08T23:51:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.725+0000 7f430efb2640 1 -- 172.21.15.22:0/555470563 wait complete. 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 Processor -- start 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 -- start start 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4308102510 0x7f4308197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4308102ed0 0x7f430819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430810cfd0 con 0x7f4308101320 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430810ce50 con 0x7f4308102510 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430efb2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430810d150 con 0x7f4308102ed0 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48940/0 (socket says 172.21.15.22:48940) 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 -- 172.21.15.22:0/2511306641 learned_addr learned my addr 172.21.15.22:0/2511306641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f4307fff640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4308102510 0x7f4308197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430d54f640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4308102ed0 0x7f430819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 -- 172.21.15.22:0/2511306641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4308102ed0 msgr2=0x7f430819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4308102ed0 0x7f430819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 -- 172.21.15.22:0/2511306641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4308102510 msgr2=0x7f4308197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4308102510 0x7f4308197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.726+0000 7f430cd4e640 1 -- 172.21.15.22:0/2511306641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430819f240 con 0x7f4308101320 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f4307fff640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4308102510 0x7f4308197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f430d54f640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4308102ed0 0x7f430819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f430cd4e640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308197620 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f42fc000c00 tx=0x7f42fc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f4305ffb640 1 -- 172.21.15.22:0/2511306641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc043020 con 0x7f4308101320 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f4305ffb640 1 -- 172.21.15.22:0/2511306641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42fc0054e0 con 0x7f4308101320 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430819f4d0 con 0x7f4308101320 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f4305ffb640 1 -- 172.21.15.22:0/2511306641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc003e60 con 0x7f4308101320 2024-03-08T23:51:14.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.727+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430819fa20 con 0x7f4308101320 2024-03-08T23:51:14.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.728+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4308108640 con 0x7f4308101320 2024-03-08T23:51:14.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.729+0000 7f4305ffb640 1 -- 172.21.15.22:0/2511306641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f42fc046020 con 0x7f4308101320 2024-03-08T23:51:14.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.730+0000 7f4305ffb640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f42f80a2800 0x7f42f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:14.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.730+0000 7f4307fff640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f42f80a2800 0x7f42f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:14.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.730+0000 7f4305ffb640 1 -- 172.21.15.22:0/2511306641 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f42fc0f0d10 con 0x7f4308101320 2024-03-08T23:51:14.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.730+0000 7f4307fff640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f42f80a2800 0x7f42f80a4cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f42f40058c0 tx=0x7f42f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:14.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.733+0000 7f4305ffb640 1 -- 172.21.15.22:0/2511306641 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f42fc04d020 con 0x7f4308101320 2024-03-08T23:51:14.758 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:14 smithi192 ceph-mon[35995]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:51:14.758 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:14 smithi192 ceph-mon[35995]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-08T23:51:14.758 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:14 smithi192 ceph-mon[35995]: Deploying daemon mds.l on smithi192 2024-03-08T23:51:14.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:14 smithi062 ceph-mon[36129]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:51:14.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:14 smithi062 ceph-mon[36129]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-08T23:51:14.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:14 smithi062 ceph-mon[36129]: Deploying daemon mds.l on smithi192 2024-03-08T23:51:14.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.916+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4308061ef0 con 0x7f4308101320 2024-03-08T23:51:14.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.917+0000 7f4305ffb640 1 -- 172.21.15.22:0/2511306641 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f42fc0be1b0 con 0x7f4308101320 2024-03-08T23:51:14.914 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:14.914 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:05.560356+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:51:14.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f42f80a2800 msgr2=0x7f42f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f42f80a2800 0x7f42f80a4cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f42f40058c0 tx=0x7f42f4005850 comp rx=0 tx=0).stop 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 msgr2=0x7f4308197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308197620 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f42fc000c00 tx=0x7f42fc005dd0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 shutdown_connections 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4308102ed0 0x7f430819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4308102510 0x7f4308197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f42f80a2800 0x7f42f80a4cc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 --2- 172.21.15.22:0/2511306641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4308101320 0x7f4308197620 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 >> 172.21.15.22:0/2511306641 conn(0x7f43080fc8d0 msgr2=0x7f4308105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 shutdown_connections 2024-03-08T23:51:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:14.920+0000 7f430efb2640 1 -- 172.21.15.22:0/2511306641 wait complete. 2024-03-08T23:51:14.926 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:14 smithi022 ceph-mon[31894]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:51:14.926 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:14 smithi022 ceph-mon[31894]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-08T23:51:14.926 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:14 smithi022 ceph-mon[31894]: Deploying daemon mds.l on smithi192 2024-03-08T23:51:14.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T23:51:14.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:51:15.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 -- 172.21.15.22:0/464465893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18100200 msgr2=0x7f1e18100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 --2- 172.21.15.22:0/464465893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18100200 0x7f1e18100620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009900 tx=0x7f1e1402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:15.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 -- 172.21.15.22:0/464465893 shutdown_connections 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 --2- 172.21.15.22:0/464465893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e18101db0 0x7f1e18108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 --2- 172.21.15.22:0/464465893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e181013f0 0x7f1e18101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 --2- 172.21.15.22:0/464465893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18100200 0x7f1e18100620 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 -- 172.21.15.22:0/464465893 >> 172.21.15.22:0/464465893 conn(0x7f1e180fb790 msgr2=0x7f1e180fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 -- 172.21.15.22:0/464465893 shutdown_connections 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.037+0000 7f1e20559640 1 -- 172.21.15.22:0/464465893 wait complete. 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 Processor -- start 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 -- start start 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e18100200 0x7f1e181975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e181013f0 0x7f1e18197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18101db0 0x7f1e1819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1810be90 con 0x7f1e18101db0 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1810bd10 con 0x7f1e18100200 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e20559640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1810c010 con 0x7f1e181013f0 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18101db0 0x7f1e1819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18101db0 0x7f1e1819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48966/0 (socket says 172.21.15.22:48966) 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 -- 172.21.15.22:0/322969351 learned_addr learned my addr 172.21.15.22:0/322969351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1e2f5640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e18100200 0x7f1e181975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1daf4640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e181013f0 0x7f1e18197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 -- 172.21.15.22:0/322969351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e181013f0 msgr2=0x7f1e18197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e181013f0 0x7f1e18197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 -- 172.21.15.22:0/322969351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e18100200 msgr2=0x7f1e181975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e18100200 0x7f1e181975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 -- 172.21.15.22:0/322969351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1819f1d0 con 0x7f1e18101db0 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1e2f5640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e18100200 0x7f1e181975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1daf4640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e181013f0 0x7f1e18197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.038+0000 7f1e1eaf6640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18101db0 0x7f1e1819eb60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000eb10 tx=0x7f1e1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.039+0000 7f1e0f7fe640 1 -- 172.21.15.22:0/322969351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10004b20 con 0x7f1e18101db0 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.039+0000 7f1e0f7fe640 1 -- 172.21.15.22:0/322969351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e10004cc0 con 0x7f1e18101db0 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.039+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1819f4c0 con 0x7f1e18101db0 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.039+0000 7f1e0f7fe640 1 -- 172.21.15.22:0/322969351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10010760 con 0x7f1e18101db0 2024-03-08T23:51:15.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.039+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1819f9c0 con 0x7f1e18101db0 2024-03-08T23:51:15.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.039+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1819fc80 con 0x7f1e18101db0 2024-03-08T23:51:15.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.041+0000 7f1e0f7fe640 1 -- 172.21.15.22:0/322969351 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1e10005000 con 0x7f1e18101db0 2024-03-08T23:51:15.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.042+0000 7f1e0f7fe640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e040a2800 0x7f1e040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.042+0000 7f1e1e2f5640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e040a2800 0x7f1e040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.042+0000 7f1e0f7fe640 1 -- 172.21.15.22:0/322969351 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1e100cef60 con 0x7f1e18101db0 2024-03-08T23:51:15.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.042+0000 7f1e1e2f5640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e040a2800 0x7f1e040a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e1402c7b0 tx=0x7f1e14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:15.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.045+0000 7f1e0f7fe640 1 -- 172.21.15.22:0/322969351 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1e10053260 con 0x7f1e18101db0 2024-03-08T23:51:15.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.225+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e1819fe50 con 0x7f1e18101db0 2024-03-08T23:51:15.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.226+0000 7f1e0f7fe640 1 -- 172.21.15.22:0/322969351 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e1009c400 con 0x7f1e18101db0 2024-03-08T23:51:15.223 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e040a2800 msgr2=0x7f1e040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e040a2800 0x7f1e040a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e1402c7b0 tx=0x7f1e14005fd0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18101db0 msgr2=0x7f1e1819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18101db0 0x7f1e1819eb60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000eb10 tx=0x7f1e1000c2d0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 shutdown_connections 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e040a2800 0x7f1e040a4cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e18101db0 0x7f1e1819eb60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e181013f0 0x7f1e18197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 --2- 172.21.15.22:0/322969351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e18100200 0x7f1e181975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 >> 172.21.15.22:0/322969351 conn(0x7f1e180fb790 msgr2=0x7f1e180fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:15.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 shutdown_connections 2024-03-08T23:51:15.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.228+0000 7f1e20559640 1 -- 172.21.15.22:0/322969351 wait complete. 2024-03-08T23:51:15.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:15.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 -- 172.21.15.22:0/2738289593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 msgr2=0x7fe6ac1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2738289593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1006b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009ca0 tx=0x7fe6a402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 -- 172.21.15.22:0/2738289593 shutdown_connections 2024-03-08T23:51:15.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2738289593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6ac101e40 0x7fe6ac1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2738289593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ac101480 0x7fe6ac101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2738289593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1006b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 -- 172.21.15.22:0/2738289593 >> 172.21.15.22:0/2738289593 conn(0x7fe6ac0fb820 msgr2=0x7fe6ac0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 -- 172.21.15.22:0/2738289593 shutdown_connections 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.341+0000 7fe6b3685640 1 -- 172.21.15.22:0/2738289593 wait complete. 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 Processor -- start 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 -- start start 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6ac101480 0x7fe6ac197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ac101e40 0x7fe6ac19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac10beb0 con 0x7fe6ac100290 2024-03-08T23:51:15.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac10bd30 con 0x7fe6ac101e40 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac10c030 con 0x7fe6ac101480 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48994/0 (socket says 172.21.15.22:48994) 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 -- 172.21.15.22:0/2659361950 learned_addr learned my addr 172.21.15.22:0/2659361950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b0c20640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6ac101480 0x7fe6ac197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1c22640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ac101e40 0x7fe6ac19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 -- 172.21.15.22:0/2659361950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6ac101480 msgr2=0x7fe6ac197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6ac101480 0x7fe6ac197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 -- 172.21.15.22:0/2659361950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ac101e40 msgr2=0x7fe6ac19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ac101e40 0x7fe6ac19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 -- 172.21.15.22:0/2659361950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac19f300 con 0x7fe6ac100290 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b0c20640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6ac101480 0x7fe6ac197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:15.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1c22640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ac101e40 0x7fe6ac19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:15.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b1421640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1976e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4006010 tx=0x7fe6a4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:15.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6aa7fc640 1 -- 172.21.15.22:0/2659361950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4041020 con 0x7fe6ac100290 2024-03-08T23:51:15.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac19f590 con 0x7fe6ac100290 2024-03-08T23:51:15.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6aa7fc640 1 -- 172.21.15.22:0/2659361950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6a4034cd0 con 0x7fe6ac100290 2024-03-08T23:51:15.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6aa7fc640 1 -- 172.21.15.22:0/2659361950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4039040 con 0x7fe6ac100290 2024-03-08T23:51:15.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.342+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac19fa80 con 0x7fe6ac100290 2024-03-08T23:51:15.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.343+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac19fce0 con 0x7fe6ac100290 2024-03-08T23:51:15.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.344+0000 7fe6aa7fc640 1 -- 172.21.15.22:0/2659361950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe6a4044020 con 0x7fe6ac100290 2024-03-08T23:51:15.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.345+0000 7fe6aa7fc640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6940a2800 0x7fe6940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:15.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.345+0000 7fe6b0c20640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6940a2800 0x7fe6940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:15.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.345+0000 7fe6aa7fc640 1 -- 172.21.15.22:0/2659361950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe6a40f0560 con 0x7fe6ac100290 2024-03-08T23:51:15.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.345+0000 7fe6b0c20640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6940a2800 0x7fe6940a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005d00 tx=0x7fe6a0005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:15.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.349+0000 7fe6aa7fc640 1 -- 172.21.15.22:0/2659361950 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe6a404b020 con 0x7fe6ac100290 2024-03-08T23:51:15.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.575+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6ac19ffd0 con 0x7fe6ac100290 2024-03-08T23:51:15.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.575+0000 7fe6aa7fc640 1 -- 172.21.15.22:0/2659361950 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fe6a40bdb30 con 0x7fe6ac100290 2024-03-08T23:51:15.572 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:15.572 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-08T23:51:15.573 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 1 2024-03-08T23:51:15.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6940a2800 msgr2=0x7fe6940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6940a2800 0x7fe6940a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005d00 tx=0x7fe6a0005ed0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 msgr2=0x7fe6ac1976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1976e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4006010 tx=0x7fe6a4032f40 comp rx=0 tx=0).stop 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 shutdown_connections 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ac101e40 0x7fe6ac19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6ac101480 0x7fe6ac197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6940a2800 0x7fe6940a4cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 --2- 172.21.15.22:0/2659361950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6ac100290 0x7fe6ac1976e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 >> 172.21.15.22:0/2659361950 conn(0x7fe6ac0fb820 msgr2=0x7fe6ac0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 shutdown_connections 2024-03-08T23:51:15.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:15.578+0000 7fe6b3685640 1 -- 172.21.15.22:0/2659361950 wait complete. 2024-03-08T23:51:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:15 smithi062 ceph-mon[36129]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2511306641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/322969351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2659361950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:15 smithi192 ceph-mon[35995]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2511306641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/322969351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:15.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2659361950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:15.882 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:15 smithi192 systemd[1]: Starting Ceph mds.l for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:15.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:15 smithi022 ceph-mon[31894]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:15.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2511306641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:15.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/322969351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:15.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2659361950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:16.276 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 podman[66861]: 2024-03-08 23:51:15.911206763 +0000 UTC m=+0.017783849 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:16.276 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 podman[66861]: 2024-03-08 23:51:16.077251655 +0000 UTC m=+0.183828785 container create 7f6a1d705b91e9193cf60eec538054032089502963576afb27c42df82d937c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T23:51:16.276 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 podman[66861]: 2024-03-08 23:51:16.279723644 +0000 UTC m=+0.386300720 container init 7f6a1d705b91e9193cf60eec538054032089502963576afb27c42df82d937c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-03-08T23:51:16.631 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 podman[66861]: 2024-03-08 23:51:16.282778707 +0000 UTC m=+0.389355785 container start 7f6a1d705b91e9193cf60eec538054032089502963576afb27c42df82d937c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-03-08T23:51:16.632 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 ceph-mds[66880]: -- 172.21.15.192:0/3231219653 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e7277f3a40 con 0x55e7277c7000 2024-03-08T23:51:16.632 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l[66876]: starting mds.l at 2024-03-08T23:51:16.632 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 bash[66861]: 7f6a1d705b91e9193cf60eec538054032089502963576afb27c42df82d937c59 2024-03-08T23:51:16.632 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 08 23:51:16 smithi192 systemd[1]: Started Ceph mds.l for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6832/3599891094,v1:172.21.15.192:6833/3599891094] up:boot 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: fsmap 1 up:standby 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:17.427 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:17.428 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:17 smithi062 ceph-mon[36129]: Deploying daemon mds.k on smithi062 2024-03-08T23:51:17.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:17.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:17.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6832/3599891094,v1:172.21.15.192:6833/3599891094] up:boot 2024-03-08T23:51:17.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: fsmap 1 up:standby 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:17.615 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:17 smithi022 ceph-mon[31894]: Deploying daemon mds.k on smithi062 2024-03-08T23:51:17.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6832/3599891094,v1:172.21.15.192:6833/3599891094] up:boot 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: fsmap 1 up:standby 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:17 smithi192 ceph-mon[35995]: Deploying daemon mds.k on smithi062 2024-03-08T23:51:17.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 -- 172.21.15.22:0/2942684175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 msgr2=0x7f0570106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:17.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 --2- 172.21.15.22:0/2942684175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f0570106ab0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0564009900 tx=0x7f056402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:17.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 -- 172.21.15.22:0/2942684175 shutdown_connections 2024-03-08T23:51:17.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 --2- 172.21.15.22:0/2942684175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570106ff0 0x7f0570109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 --2- 172.21.15.22:0/2942684175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f0570106ab0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 --2- 172.21.15.22:0/2942684175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0570101d90 0x7f0570104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 -- 172.21.15.22:0/2942684175 >> 172.21.15.22:0/2942684175 conn(0x7f05700fb780 msgr2=0x7f05700fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 -- 172.21.15.22:0/2942684175 shutdown_connections 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.704+0000 7f05749b9640 1 -- 172.21.15.22:0/2942684175 wait complete. 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 Processor -- start 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 -- start start 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0570101d90 0x7f0570101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f05700ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570106ff0 0x7f0570100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057010cfa0 con 0x7f05701046c0 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057010ce20 con 0x7f0570106ff0 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f05749b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057010d120 con 0x7f0570101d90 2024-03-08T23:51:17.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f056dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f05700ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f056dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f05700ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49008/0 (socket says 172.21.15.22:49008) 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.705+0000 7f056dd9b640 1 -- 172.21.15.22:0/3940676978 learned_addr learned my addr 172.21.15.22:0/3940676978 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056ed9d640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570106ff0 0x7f0570100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056e59c640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0570101d90 0x7f0570101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056dd9b640 1 -- 172.21.15.22:0/3940676978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0570101d90 msgr2=0x7f0570101700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056dd9b640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0570101d90 0x7f0570101700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056dd9b640 1 -- 172.21.15.22:0/3940676978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570106ff0 msgr2=0x7f0570100280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056dd9b640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570106ff0 0x7f0570100280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056dd9b640 1 -- 172.21.15.22:0/3940676978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0570100b10 con 0x7f05701046c0 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056ed9d640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570106ff0 0x7f0570100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f056dd9b640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f05700ffd40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0564005ab0 tx=0x7f0564005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f055f7fe640 1 -- 172.21.15.22:0/3940676978 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564041070 con 0x7f05701046c0 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f055f7fe640 1 -- 172.21.15.22:0/3940676978 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05640054e0 con 0x7f05701046c0 2024-03-08T23:51:17.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05701a16b0 con 0x7f05701046c0 2024-03-08T23:51:17.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f055f7fe640 1 -- 172.21.15.22:0/3940676978 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564034440 con 0x7f05701046c0 2024-03-08T23:51:17.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.706+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05701a1c00 con 0x7f05701046c0 2024-03-08T23:51:17.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.707+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05700fd270 con 0x7f05701046c0 2024-03-08T23:51:17.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.708+0000 7f055f7fe640 1 -- 172.21.15.22:0/3940676978 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0564044070 con 0x7f05701046c0 2024-03-08T23:51:17.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.709+0000 7f055f7fe640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05580a2800 0x7f05580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:17.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.709+0000 7f056e59c640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05580a2800 0x7f05580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:17.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.709+0000 7f055f7fe640 1 -- 172.21.15.22:0/3940676978 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05640f02e0 con 0x7f05701046c0 2024-03-08T23:51:17.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.709+0000 7f056e59c640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05580a2800 0x7f05580a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0560005fd0 tx=0x7f0560005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:17.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.713+0000 7f055f7fe640 1 -- 172.21.15.22:0/3940676978 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f05640bd820 con 0x7f05701046c0 2024-03-08T23:51:17.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.936+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f057002ce70 con 0x7f05701046c0 2024-03-08T23:51:17.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.937+0000 7f055f7fe640 1 -- 172.21.15.22:0/3940676978 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f05640bd120 con 0x7f05701046c0 2024-03-08T23:51:17.934 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:17.934 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T23:51:17.934 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-03-08T23:51:17.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05580a2800 msgr2=0x7f05580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:17.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05580a2800 0x7f05580a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0560005fd0 tx=0x7f0560005850 comp rx=0 tx=0).stop 2024-03-08T23:51:17.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 msgr2=0x7f05700ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:17.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f05700ffd40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0564005ab0 tx=0x7f0564005dc0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 shutdown_connections 2024-03-08T23:51:17.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570106ff0 0x7f0570100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05580a2800 0x7f05580a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05701046c0 0x7f05700ffd40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 --2- 172.21.15.22:0/3940676978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0570101d90 0x7f0570101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:17.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 >> 172.21.15.22:0/3940676978 conn(0x7f05700fb780 msgr2=0x7f05701026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:17.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 shutdown_connections 2024-03-08T23:51:17.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:17.939+0000 7f05749b9640 1 -- 172.21.15.22:0/3940676978 wait complete. 2024-03-08T23:51:18.602 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3940676978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:18.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3940676978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:18.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3940676978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:18.862 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:18 smithi062 systemd[1]: Starting Ceph mds.k for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:19.262 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 podman[65664]: 2024-03-08 23:51:18.913416686 +0000 UTC m=+0.018165597 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:19.262 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 podman[65664]: 2024-03-08 23:51:19.038350718 +0000 UTC m=+0.143099624 container create d0630ecc7c86da24f20a87c2905be96aab8cb4e53c3729d3c1c046b018513d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-08T23:51:19.262 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 podman[65664]: 2024-03-08 23:51:19.266045698 +0000 UTC m=+0.370794606 container init d0630ecc7c86da24f20a87c2905be96aab8cb4e53c3729d3c1c046b018513d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k, RELEASE=rishabh-2024mar8-7d63482, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-08T23:51:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:19 smithi062 ceph-mon[36129]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:19.612 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 podman[65664]: 2024-03-08 23:51:19.269012329 +0000 UTC m=+0.373761227 container start d0630ecc7c86da24f20a87c2905be96aab8cb4e53c3729d3c1c046b018513d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T23:51:19.612 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 ceph-mds[65683]: -- 172.21.15.62:0/1863016493 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8f876fa40 con 0x55a8f8742c00 2024-03-08T23:51:19.612 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k[65679]: starting mds.k at 2024-03-08T23:51:19.612 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 bash[65664]: d0630ecc7c86da24f20a87c2905be96aab8cb4e53c3729d3c1c046b018513d08 2024-03-08T23:51:19.612 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 08 23:51:19 smithi062 systemd[1]: Started Ceph mds.k for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:19 smithi192 ceph-mon[35995]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:19 smithi022 ceph-mon[31894]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:19.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:20.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 -- 172.21.15.22:0/1161940145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 msgr2=0x7f0758102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 --2- 172.21.15.22:0/1161940145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758102990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f074c009900 tx=0x7f074c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 -- 172.21.15.22:0/1161940145 shutdown_connections 2024-03-08T23:51:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 --2- 172.21.15.22:0/1161940145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0758102ed0 0x7f0758109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 --2- 172.21.15.22:0/1161940145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758102990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 --2- 172.21.15.22:0/1161940145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0758101320 0x7f0758101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 -- 172.21.15.22:0/1161940145 >> 172.21.15.22:0/1161940145 conn(0x7f07580fc8d0 msgr2=0x7f07580fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 -- 172.21.15.22:0/1161940145 shutdown_connections 2024-03-08T23:51:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 -- 172.21.15.22:0/1161940145 wait complete. 2024-03-08T23:51:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 Processor -- start 2024-03-08T23:51:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.055+0000 7f075e937640 1 -- start start 2024-03-08T23:51:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075e937640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0758101320 0x7f0758197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075e937640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075e937640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0758102ed0 0x7f075819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075e937640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075810cfb0 con 0x7f0758102510 2024-03-08T23:51:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075e937640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075810ce30 con 0x7f0758101320 2024-03-08T23:51:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075e937640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075810d130 con 0x7f0758102ed0 2024-03-08T23:51:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49022/0 (socket says 172.21.15.22:49022) 2024-03-08T23:51:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 -- 172.21.15.22:0/692370393 learned_addr learned my addr 172.21.15.22:0/692370393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075ced4640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0758102ed0 0x7f075819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f0757fff640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0758101320 0x7f0758197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 -- 172.21.15.22:0/692370393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0758102ed0 msgr2=0x7f075819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0758102ed0 0x7f075819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 -- 172.21.15.22:0/692370393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0758101320 msgr2=0x7f0758197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0758101320 0x7f0758197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 -- 172.21.15.22:0/692370393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075819f220 con 0x7f0758102510 2024-03-08T23:51:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f075ced4640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0758102ed0 0x7f075819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07577fe640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758197b40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f074c009230 tx=0x7f074c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.056+0000 7f07557fa640 1 -- 172.21.15.22:0/692370393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c041070 con 0x7f0758102510 2024-03-08T23:51:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.057+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075819f4b0 con 0x7f0758102510 2024-03-08T23:51:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.057+0000 7f07557fa640 1 -- 172.21.15.22:0/692370393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f074c0054e0 con 0x7f0758102510 2024-03-08T23:51:20.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.057+0000 7f07557fa640 1 -- 172.21.15.22:0/692370393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c034440 con 0x7f0758102510 2024-03-08T23:51:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.057+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075819f9a0 con 0x7f0758102510 2024-03-08T23:51:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.058+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0758108570 con 0x7f0758102510 2024-03-08T23:51:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.058+0000 7f07557fa640 1 -- 172.21.15.22:0/692370393 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f074c044070 con 0x7f0758102510 2024-03-08T23:51:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.059+0000 7f07557fa640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f07400a2800 0x7f07400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:20.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.059+0000 7f0757fff640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f07400a2800 0x7f07400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:20.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.059+0000 7f07557fa640 1 -- 172.21.15.22:0/692370393 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f074c0f0370 con 0x7f0758102510 2024-03-08T23:51:20.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.060+0000 7f0757fff640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f07400a2800 0x7f07400a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07480058c0 tx=0x7f0748005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:20.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.064+0000 7f07557fa640 1 -- 172.21.15.22:0/692370393 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f074c0bd8b0 con 0x7f0758102510 2024-03-08T23:51:20.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.291+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f075802ce70 con 0x7f0758102510 2024-03-08T23:51:20.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.292+0000 7f07557fa640 1 -- 172.21.15.22:0/692370393 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7f074c0bd1b0 con 0x7f0758102510 2024-03-08T23:51:20.289 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:20.289 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T23:51:20.289 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f07400a2800 msgr2=0x7f07400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f07400a2800 0x7f07400a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07480058c0 tx=0x7f0748005850 comp rx=0 tx=0).stop 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 msgr2=0x7f0758197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758197b40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f074c009230 tx=0x7f074c005dd0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 shutdown_connections 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0758102ed0 0x7f075819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f07400a2800 0x7f07400a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0758102510 0x7f0758197b40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 --2- 172.21.15.22:0/692370393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0758101320 0x7f0758197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:20.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 >> 172.21.15.22:0/692370393 conn(0x7f07580fc8d0 msgr2=0x7f0758105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:20.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 shutdown_connections 2024-03-08T23:51:20.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:20.294+0000 7f075e937640 1 -- 172.21.15.22:0/692370393 wait complete. 2024-03-08T23:51:20.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6832/292631857,v1:172.21.15.62:6833/292631857] up:boot 2024-03-08T23:51:20.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: fsmap 2 up:standby 2024-03-08T23:51:20.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T23:51:20.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:20.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:20.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:20.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: Deploying daemon mds.j on smithi022 2024-03-08T23:51:20.488 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:20 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/692370393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:20.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6832/292631857,v1:172.21.15.62:6833/292631857] up:boot 2024-03-08T23:51:20.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: fsmap 2 up:standby 2024-03-08T23:51:20.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T23:51:20.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: Deploying daemon mds.j on smithi022 2024-03-08T23:51:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:20 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/692370393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:20.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6832/292631857,v1:172.21.15.62:6833/292631857] up:boot 2024-03-08T23:51:20.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: fsmap 2 up:standby 2024-03-08T23:51:20.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T23:51:20.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:20.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:20.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:20.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:20.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: Deploying daemon mds.j on smithi022 2024-03-08T23:51:20.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:20 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/692370393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:21.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:21 smithi022 ceph-mon[31894]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:21.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:21 smithi062 ceph-mon[36129]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:21.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:21 smithi192 ceph-mon[35995]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:21.986 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:21 smithi022 systemd[1]: Starting Ceph mds.j for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:22.303 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:22.304 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 podman[71872]: 2024-03-08 23:51:21.953248433 +0000 UTC m=+0.017932472 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:22.304 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 podman[71872]: 2024-03-08 23:51:22.062272591 +0000 UTC m=+0.126956618 container create 962b59335e90d2f7906ca3e5e693e6183659aaeaef57b8d1061976377a6db86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-08T23:51:22.304 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 podman[71872]: 2024-03-08 23:51:22.282181811 +0000 UTC m=+0.346865861 container init 962b59335e90d2f7906ca3e5e693e6183659aaeaef57b8d1061976377a6db86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2) 2024-03-08T23:51:22.304 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 podman[71872]: 2024-03-08 23:51:22.28470959 +0000 UTC m=+0.349393619 container start 962b59335e90d2f7906ca3e5e693e6183659aaeaef57b8d1061976377a6db86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307) 2024-03-08T23:51:22.304 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 ceph-mds[71891]: -- 172.21.15.22:0/399529535 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556707c0fa40 con 0x556707be2c00 2024-03-08T23:51:22.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.415+0000 7eff1ab59640 1 -- 172.21.15.22:0/14543625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14102590 msgr2=0x7eff14102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.415+0000 7eff1ab59640 1 --2- 172.21.15.22:0/14543625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14102590 0x7eff14102a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7eff00009930 tx=0x7eff0002bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- 172.21.15.22:0/14543625 shutdown_connections 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 --2- 172.21.15.22:0/14543625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14102f50 0x7eff141097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 --2- 172.21.15.22:0/14543625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff14102590 0x7eff14102a10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 --2- 172.21.15.22:0/14543625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff140ff5c0 0x7eff140ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- 172.21.15.22:0/14543625 >> 172.21.15.22:0/14543625 conn(0x7eff140faef0 msgr2=0x7eff140fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- 172.21.15.22:0/14543625 shutdown_connections 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- 172.21.15.22:0/14543625 wait complete. 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 Processor -- start 2024-03-08T23:51:22.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- start start 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff140ff5c0 0x7eff14101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff14102590 0x7eff14100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14102f50 0x7eff14100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1410cfb0 con 0x7eff140ff5c0 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1410ce30 con 0x7eff14102590 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.416+0000 7eff1ab59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1410d130 con 0x7eff14102f50 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff140ff5c0 0x7eff14101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff140ff5c0 0x7eff14101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57900/0 (socket says 172.21.15.22:57900) 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 -- 172.21.15.22:0/4123977800 learned_addr learned my addr 172.21.15.22:0/4123977800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff13fff640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff14102590 0x7eff14100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff190f6640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14102f50 0x7eff14100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 -- 172.21.15.22:0/4123977800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14102f50 msgr2=0x7eff14100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:22.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14102f50 0x7eff14100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 -- 172.21.15.22:0/4123977800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff14102590 msgr2=0x7eff14100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff14102590 0x7eff14100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 -- 172.21.15.22:0/4123977800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14101310 con 0x7eff140ff5c0 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff188f5640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff140ff5c0 0x7eff14101f00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff0800e9c0 tx=0x7eff0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff11ffb640 1 -- 172.21.15.22:0/4123977800 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0800cd80 con 0x7eff140ff5c0 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff11ffb640 1 -- 172.21.15.22:0/4123977800 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff080035f0 con 0x7eff140ff5c0 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff11ffb640 1 -- 172.21.15.22:0/4123977800 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08010690 con 0x7eff140ff5c0 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.417+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff141a1730 con 0x7eff140ff5c0 2024-03-08T23:51:22.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.418+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff141a1c50 con 0x7eff140ff5c0 2024-03-08T23:51:22.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.419+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff14067850 con 0x7eff140ff5c0 2024-03-08T23:51:22.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.419+0000 7eff11ffb640 1 -- 172.21.15.22:0/4123977800 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7eff080037a0 con 0x7eff140ff5c0 2024-03-08T23:51:22.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.420+0000 7eff11ffb640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff040a2800 0x7eff040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:22.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.420+0000 7eff13fff640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff040a2800 0x7eff040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:22.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.420+0000 7eff11ffb640 1 -- 172.21.15.22:0/4123977800 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff080ce9d0 con 0x7eff140ff5c0 2024-03-08T23:51:22.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.421+0000 7eff13fff640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff040a2800 0x7eff040a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7eff0002efd0 tx=0x7eff00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.425+0000 7eff11ffb640 1 -- 172.21.15.22:0/4123977800 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7eff0809bf10 con 0x7eff140ff5c0 2024-03-08T23:51:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.650+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff14108740 con 0x7eff140ff5c0 2024-03-08T23:51:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.651+0000 7eff11ffb640 1 -- 172.21.15.22:0/4123977800 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7eff14108740 con 0x7eff140ff5c0 2024-03-08T23:51:22.648 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:22.648 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T23:51:22.648 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 4 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff040a2800 msgr2=0x7eff040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff040a2800 0x7eff040a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7eff0002efd0 tx=0x7eff00000bc0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff140ff5c0 msgr2=0x7eff14101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff140ff5c0 0x7eff14101f00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff0800e9c0 tx=0x7eff0800ee80 comp rx=0 tx=0).stop 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 shutdown_connections 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14102f50 0x7eff14100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff14102590 0x7eff14100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff040a2800 0x7eff040a4cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 --2- 172.21.15.22:0/4123977800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff140ff5c0 0x7eff14101f00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 >> 172.21.15.22:0/4123977800 conn(0x7eff140faef0 msgr2=0x7eff14105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 shutdown_connections 2024-03-08T23:51:22.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:22.653+0000 7eff1ab59640 1 -- 172.21.15.22:0/4123977800 wait complete. 2024-03-08T23:51:22.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j[71887]: starting mds.j at 2024-03-08T23:51:22.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 bash[71872]: 962b59335e90d2f7906ca3e5e693e6183659aaeaef57b8d1061976377a6db86c 2024-03-08T23:51:22.660 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 08 23:51:22 smithi022 systemd[1]: Started Ceph mds.j for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:23.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.22:6834/1092846184,v1:172.21.15.22:6835/1092846184] up:boot 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: fsmap 3 up:standby 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: Deploying daemon mds.d on smithi022 2024-03-08T23:51:23.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:23 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4123977800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.22:6834/1092846184,v1:172.21.15.22:6835/1092846184] up:boot 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: fsmap 3 up:standby 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: Deploying daemon mds.d on smithi022 2024-03-08T23:51:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:23 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4123977800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:23.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:23.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.22:6834/1092846184,v1:172.21.15.22:6835/1092846184] up:boot 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: fsmap 3 up:standby 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: Deploying daemon mds.d on smithi022 2024-03-08T23:51:23.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:23 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4123977800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:24.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 -- 172.21.15.22:0/3675271567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 msgr2=0x7f0144074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 --2- 172.21.15.22:0/3675271567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144074130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f012c009900 tx=0x7f012c02bcd0 comp rx=0 tx=0).stop 2024-03-08T23:51:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 -- 172.21.15.22:0/3675271567 shutdown_connections 2024-03-08T23:51:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 --2- 172.21.15.22:0/3675271567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01441091d0 0x7f014410b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 --2- 172.21.15.22:0/3675271567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144074130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:24.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 --2- 172.21.15.22:0/3675271567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01440752b0 0x7f0144073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 -- 172.21.15.22:0/3675271567 >> 172.21.15.22:0/3675271567 conn(0x7f01440fc890 msgr2=0x7f01440fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 -- 172.21.15.22:0/3675271567 shutdown_connections 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.782+0000 7f0149589640 1 -- 172.21.15.22:0/3675271567 wait complete. 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 Processor -- start 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 -- start start 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01440752b0 0x7f0144199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01441091d0 0x7f01441a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014410f1c0 con 0x7f0144073cd0 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014410f040 con 0x7f01440752b0 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0149589640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014410f340 con 0x7f01441091d0 2024-03-08T23:51:24.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57906/0 (socket says 172.21.15.22:57906) 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 -- 172.21.15.22:0/3253133562 learned_addr learned my addr 172.21.15.22:0/3253133562 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f013bfff640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01440752b0 0x7f0144199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 -- 172.21.15.22:0/3253133562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01441091d0 msgr2=0x7f01441a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01441091d0 0x7f01441a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 -- 172.21.15.22:0/3253133562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01440752b0 msgr2=0x7f0144199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01440752b0 0x7f0144199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 -- 172.21.15.22:0/3253133562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01441a1480 con 0x7f0144073cd0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f013bfff640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01440752b0 0x7f0144199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0142ffd640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144199860 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f013400d770 tx=0x7f013400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0140ff9640 1 -- 172.21.15.22:0/3253133562 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134017070 con 0x7f0144073cd0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.783+0000 7f0140ff9640 1 -- 172.21.15.22:0/3253133562 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0134005500 con 0x7f0144073cd0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.784+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01441a1770 con 0x7f0144073cd0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.784+0000 7f0140ff9640 1 -- 172.21.15.22:0/3253133562 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134004e00 con 0x7f0144073cd0 2024-03-08T23:51:24.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.784+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01441a1c40 con 0x7f0144073cd0 2024-03-08T23:51:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.784+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0144107f90 con 0x7f0144073cd0 2024-03-08T23:51:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.786+0000 7f0140ff9640 1 -- 172.21.15.22:0/3253133562 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0134004fa0 con 0x7f0144073cd0 2024-03-08T23:51:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.786+0000 7f0140ff9640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f01300a2800 0x7f01300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:24.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.786+0000 7f013bfff640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f01300a2800 0x7f01300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.786+0000 7f0140ff9640 1 -- 172.21.15.22:0/3253133562 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01340ce4f0 con 0x7f0144073cd0 2024-03-08T23:51:24.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.787+0000 7f013bfff640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f01300a2800 0x7f01300a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f012c006010 tx=0x7f012c02c700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:24.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:24.790+0000 7f0140ff9640 1 -- 172.21.15.22:0/3253133562 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f013409ba30 con 0x7f0144073cd0 2024-03-08T23:51:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.011+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f014402ce70 con 0x7f0144073cd0 2024-03-08T23:51:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.012+0000 7f0140ff9640 1 -- 172.21.15.22:0/3253133562 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7f013409b330 con 0x7f0144073cd0 2024-03-08T23:51:25.008 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:25.008 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T23:51:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 4 2024-03-08T23:51:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f01300a2800 msgr2=0x7f01300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f01300a2800 0x7f01300a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f012c006010 tx=0x7f012c02c700 comp rx=0 tx=0).stop 2024-03-08T23:51:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 msgr2=0x7f0144199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144199860 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f013400d770 tx=0x7f013400dc30 comp rx=0 tx=0).stop 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 shutdown_connections 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01441091d0 0x7f01441a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01440752b0 0x7f0144199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f01300a2800 0x7f01300a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 --2- 172.21.15.22:0/3253133562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0144073cd0 0x7f0144199860 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.013+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 >> 172.21.15.22:0/3253133562 conn(0x7f01440fc890 msgr2=0x7f0144105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.014+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 shutdown_connections 2024-03-08T23:51:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:25.014+0000 7f0149589640 1 -- 172.21.15.22:0/3253133562 wait complete. 2024-03-08T23:51:25.435 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 08 23:51:25 smithi022 systemd[1]: Started Ceph mds.d for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3253133562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:25.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3253133562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:25.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:25.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3253133562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:26.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:26 smithi062 ceph-mon[36129]: Deploying daemon mds.e on smithi062 2024-03-08T23:51:26.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:26 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.22:6836/1255227444,v1:172.21.15.22:6837/1255227444] up:boot 2024-03-08T23:51:26.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:26 smithi062 ceph-mon[36129]: fsmap 4 up:standby 2024-03-08T23:51:26.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:26 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T23:51:26.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:26 smithi192 ceph-mon[35995]: Deploying daemon mds.e on smithi062 2024-03-08T23:51:26.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:26 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.22:6836/1255227444,v1:172.21.15.22:6837/1255227444] up:boot 2024-03-08T23:51:26.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:26 smithi192 ceph-mon[35995]: fsmap 4 up:standby 2024-03-08T23:51:26.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:26 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T23:51:26.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:26 smithi022 ceph-mon[31894]: Deploying daemon mds.e on smithi062 2024-03-08T23:51:26.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:26 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.22:6836/1255227444,v1:172.21.15.22:6837/1255227444] up:boot 2024-03-08T23:51:26.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:26 smithi022 ceph-mon[31894]: fsmap 4 up:standby 2024-03-08T23:51:26.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:26 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T23:51:27.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 -- 172.21.15.22:0/3480842221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102ed0 msgr2=0x7fed48109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 --2- 172.21.15.22:0/3480842221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102ed0 0x7fed48109760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fed38009ca0 tx=0x7fed3802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 -- 172.21.15.22:0/3480842221 shutdown_connections 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 --2- 172.21.15.22:0/3480842221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102ed0 0x7fed48109760 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 --2- 172.21.15.22:0/3480842221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed48102510 0x7fed48102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 --2- 172.21.15.22:0/3480842221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed48101320 0x7fed48101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 -- 172.21.15.22:0/3480842221 >> 172.21.15.22:0/3480842221 conn(0x7fed480fc8d0 msgr2=0x7fed480fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 -- 172.21.15.22:0/3480842221 shutdown_connections 2024-03-08T23:51:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.141+0000 7fed4fb69640 1 -- 172.21.15.22:0/3480842221 wait complete. 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 Processor -- start 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 -- start start 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed48101320 0x7fed481975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102510 0x7fed48197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed48102ed0 0x7fed4819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4810cfb0 con 0x7fed48102510 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4810ce30 con 0x7fed48101320 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4fb69640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4810d130 con 0x7fed48102ed0 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102510 0x7fed48197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102510 0x7fed48197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57960/0 (socket says 172.21.15.22:57960) 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 -- 172.21.15.22:0/130593175 learned_addr learned my addr 172.21.15.22:0/130593175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:27.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d905640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed48101320 0x7fed481975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4e106640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed48102ed0 0x7fed4819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 -- 172.21.15.22:0/130593175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed48102ed0 msgr2=0x7fed4819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed48102ed0 0x7fed4819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 -- 172.21.15.22:0/130593175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed48101320 msgr2=0x7fed481975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed48101320 0x7fed481975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d104640 1 -- 172.21.15.22:0/130593175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4819f1c0 con 0x7fed48102510 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.142+0000 7fed4d905640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed48101320 0x7fed481975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.143+0000 7fed4d104640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102510 0x7fed48197ae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00d770 tx=0x7fed3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.143+0000 7fed46ffd640 1 -- 172.21.15.22:0/130593175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c017070 con 0x7fed48102510 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.143+0000 7fed46ffd640 1 -- 172.21.15.22:0/130593175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed3c005500 con 0x7fed48102510 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.143+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4819f4b0 con 0x7fed48102510 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.143+0000 7fed46ffd640 1 -- 172.21.15.22:0/130593175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c004e00 con 0x7fed48102510 2024-03-08T23:51:27.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.143+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4819f9b0 con 0x7fed48102510 2024-03-08T23:51:27.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.144+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed481085e0 con 0x7fed48102510 2024-03-08T23:51:27.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.146+0000 7fed46ffd640 1 -- 172.21.15.22:0/130593175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fed3c004fa0 con 0x7fed48102510 2024-03-08T23:51:27.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.146+0000 7fed46ffd640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fed300a2800 0x7fed300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:27.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.146+0000 7fed4d905640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fed300a2800 0x7fed300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:27.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.146+0000 7fed46ffd640 1 -- 172.21.15.22:0/130593175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fed3c0ce4f0 con 0x7fed48102510 2024-03-08T23:51:27.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.147+0000 7fed4d905640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fed300a2800 0x7fed300a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fed48102370 tx=0x7fed40005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:27.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.150+0000 7fed46ffd640 1 -- 172.21.15.22:0/130593175 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fed3c09ba30 con 0x7fed48102510 2024-03-08T23:51:27.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.369+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed4802d0e0 con 0x7fed48102510 2024-03-08T23:51:27.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.370+0000 7fed46ffd640 1 -- 172.21.15.22:0/130593175 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7fed3c09b330 con 0x7fed48102510 2024-03-08T23:51:27.367 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:27.367 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T23:51:27.367 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 5 2024-03-08T23:51:27.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fed300a2800 msgr2=0x7fed300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:27.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fed300a2800 0x7fed300a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fed48102370 tx=0x7fed40005e70 comp rx=0 tx=0).stop 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102510 msgr2=0x7fed48197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102510 0x7fed48197ae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00d770 tx=0x7fed3c00dc30 comp rx=0 tx=0).stop 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 shutdown_connections 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed48102ed0 0x7fed4819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fed300a2800 0x7fed300a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed48102510 0x7fed48197ae0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 --2- 172.21.15.22:0/130593175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed48101320 0x7fed481975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 >> 172.21.15.22:0/130593175 conn(0x7fed480fc8d0 msgr2=0x7fed481057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 shutdown_connections 2024-03-08T23:51:27.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:27.372+0000 7fed4fb69640 1 -- 172.21.15.22:0/130593175 wait complete. 2024-03-08T23:51:27.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:27 smithi022 ceph-mon[31894]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:27.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:27 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/130593175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:27.782 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:27 smithi062 ceph-mon[36129]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:27.782 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:27 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/130593175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:27.782 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:27 smithi062 systemd[1]: Starting Ceph mds.e for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:27.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:27 smithi192 ceph-mon[35995]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:27.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:27 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/130593175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:28.050 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:27 smithi062 podman[65983]: 2024-03-08 23:51:27.688514995 +0000 UTC m=+0.018304502 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:28.050 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:27 smithi062 podman[65983]: 2024-03-08 23:51:27.7933926 +0000 UTC m=+0.123182089 container create 898377f6b379ccf14c6f5d26b267b0708986cd2e65937b7b397d38ad64795d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-08T23:51:28.050 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:28 smithi062 podman[65983]: 2024-03-08 23:51:28.054286141 +0000 UTC m=+0.384075649 container init 898377f6b379ccf14c6f5d26b267b0708986cd2e65937b7b397d38ad64795d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-08T23:51:28.362 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:28 smithi062 podman[65983]: 2024-03-08 23:51:28.057163317 +0000 UTC m=+0.386952811 container start 898377f6b379ccf14c6f5d26b267b0708986cd2e65937b7b397d38ad64795d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-08T23:51:28.362 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:28 smithi062 ceph-mds[66002]: -- 172.21.15.62:0/3061364807 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e3fcb55a40 con 0x55e3fcb28c00 2024-03-08T23:51:28.362 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e[65998]: starting mds.e at 2024-03-08T23:51:28.362 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:28 smithi062 bash[65983]: 898377f6b379ccf14c6f5d26b267b0708986cd2e65937b7b397d38ad64795d57 2024-03-08T23:51:28.362 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 08 23:51:28 smithi062 systemd[1]: Started Ceph mds.e for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:29.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:29.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: Deploying daemon mds.f on smithi192 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6834/1656345994,v1:172.21.15.62:6835/1656345994] up:boot 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: fsmap 5 up:standby 2024-03-08T23:51:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:29 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 -- 172.21.15.22:0/3610741972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528100290 msgr2=0x7f35281006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 --2- 172.21.15.22:0/3610741972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528100290 0x7f35281006b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3524009900 tx=0x7f352402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 -- 172.21.15.22:0/3610741972 shutdown_connections 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 --2- 172.21.15.22:0/3610741972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3528101e40 0x7f35281086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 --2- 172.21.15.22:0/3610741972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3528101480 0x7f3528101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 --2- 172.21.15.22:0/3610741972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528100290 0x7f35281006b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 -- 172.21.15.22:0/3610741972 >> 172.21.15.22:0/3610741972 conn(0x7f35280fb820 msgr2=0x7f35280fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 -- 172.21.15.22:0/3610741972 shutdown_connections 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.498+0000 7f3530012640 1 -- 172.21.15.22:0/3610741972 wait complete. 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 Processor -- start 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 -- start start 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3528100290 0x7f3528104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528101480 0x7f3528102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3528101e40 0x7f3528102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352810beb0 con 0x7f3528101480 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352810bd30 con 0x7f3528101e40 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f3530012640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352810c030 con 0x7f3528100290 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f352d5ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528101480 0x7f3528102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f352d5ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528101480 0x7f3528102860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57974/0 (socket says 172.21.15.22:57974) 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f352d5ad640 1 -- 172.21.15.22:0/153216667 learned_addr learned my addr 172.21.15.22:0/153216667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:29.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f352ddae640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3528100290 0x7f3528104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.499+0000 7f352e5af640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3528101e40 0x7f3528102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f352d5ad640 1 -- 172.21.15.22:0/153216667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3528100290 msgr2=0x7f3528104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f352d5ad640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3528100290 0x7f3528104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f352d5ad640 1 -- 172.21.15.22:0/153216667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3528101e40 msgr2=0x7f3528102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f352d5ad640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3528101e40 0x7f3528102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f352d5ad640 1 -- 172.21.15.22:0/153216667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528103630 con 0x7f3528101480 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f352d5ad640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528101480 0x7f3528102860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f351800c890 tx=0x7f351800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f351effd640 1 -- 172.21.15.22:0/153216667 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518004220 con 0x7f3528101480 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f351effd640 1 -- 172.21.15.22:0/153216667 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35180043c0 con 0x7f3528101480 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35281a1800 con 0x7f3528101480 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f351effd640 1 -- 172.21.15.22:0/153216667 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180049e0 con 0x7f3528101480 2024-03-08T23:51:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.500+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35281a1da0 con 0x7f3528101480 2024-03-08T23:51:29.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.501+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35281a2060 con 0x7f3528101480 2024-03-08T23:51:29.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.502+0000 7f351effd640 1 -- 172.21.15.22:0/153216667 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f35180079b0 con 0x7f3528101480 2024-03-08T23:51:29.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.503+0000 7f351effd640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f35140a2880 0x7f35140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:29.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.503+0000 7f352ddae640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f35140a2880 0x7f35140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:29.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.503+0000 7f351effd640 1 -- 172.21.15.22:0/153216667 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f35180ce640 con 0x7f3528101480 2024-03-08T23:51:29.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.503+0000 7f352ddae640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f35140a2880 0x7f35140a4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3524005ab0 tx=0x7f3524005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:29.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.507+0000 7f351effd640 1 -- 172.21.15.22:0/153216667 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3518014030 con 0x7f3528101480 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: Deploying daemon mds.f on smithi192 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6834/1656345994,v1:172.21.15.62:6835/1656345994] up:boot 2024-03-08T23:51:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: fsmap 5 up:standby 2024-03-08T23:51:29.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:29 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T23:51:29.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:29.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:29.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: Deploying daemon mds.f on smithi192 2024-03-08T23:51:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6834/1656345994,v1:172.21.15.62:6835/1656345994] up:boot 2024-03-08T23:51:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: fsmap 5 up:standby 2024-03-08T23:51:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:29 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T23:51:29.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.729+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35281a2230 con 0x7f3528101480 2024-03-08T23:51:29.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.729+0000 7f351effd640 1 -- 172.21.15.22:0/153216667 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f351809bc00 con 0x7f3528101480 2024-03-08T23:51:29.726 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:29.726 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-08T23:51:29.726 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 6 2024-03-08T23:51:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f35140a2880 msgr2=0x7f35140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f35140a2880 0x7f35140a4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3524005ab0 tx=0x7f3524005fd0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528101480 msgr2=0x7f3528102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528101480 0x7f3528102860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f351800c890 tx=0x7f351800cd50 comp rx=0 tx=0).stop 2024-03-08T23:51:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 shutdown_connections 2024-03-08T23:51:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3528101e40 0x7f3528102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f35140a2880 0x7f35140a4d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3528101480 0x7f3528102860 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 --2- 172.21.15.22:0/153216667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3528100290 0x7f3528104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 >> 172.21.15.22:0/153216667 conn(0x7f35280fb820 msgr2=0x7f35280fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 shutdown_connections 2024-03-08T23:51:29.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:29.731+0000 7f3530012640 1 -- 172.21.15.22:0/153216667 wait complete. 2024-03-08T23:51:30.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:30 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/153216667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:30.595 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:30 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/153216667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:30.595 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 systemd[1]: Starting Ceph mds.f for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:30.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:30 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/153216667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:30.855 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 podman[67186]: 2024-03-08 23:51:30.501783962 +0000 UTC m=+0.018221621 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:30.856 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 podman[67186]: 2024-03-08 23:51:30.623811846 +0000 UTC m=+0.140249505 container create f9a3eec91580d83d5dbd19e3ed94563bc02d8adc42984978c41eef0615eadf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-08T23:51:30.856 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 podman[67186]: 2024-03-08 23:51:30.859904191 +0000 UTC m=+0.376341860 container init f9a3eec91580d83d5dbd19e3ed94563bc02d8adc42984978c41eef0615eadf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-08T23:51:31.131 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 podman[67186]: 2024-03-08 23:51:30.862435243 +0000 UTC m=+0.378872899 container start f9a3eec91580d83d5dbd19e3ed94563bc02d8adc42984978c41eef0615eadf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T23:51:31.132 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 ceph-mds[67206]: -- 172.21.15.192:0/147208862 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5648c90ffa40 con 0x5648c90d2800 2024-03-08T23:51:31.132 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f[67202]: starting mds.f at 2024-03-08T23:51:31.132 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 bash[67186]: f9a3eec91580d83d5dbd19e3ed94563bc02d8adc42984978c41eef0615eadf0a 2024-03-08T23:51:31.132 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 08 23:51:30 smithi192 systemd[1]: Started Ceph mds.f for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:31.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:31 smithi022 ceph-mon[31894]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:31.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:31.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:31.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:31 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:31.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:31 smithi062 ceph-mon[36129]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:31.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:31.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:31.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:31 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:31.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:31 smithi192 ceph-mon[35995]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:31.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:31.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:31.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:31.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:31 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:31.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 -- 172.21.15.22:0/4095864673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 msgr2=0x7ff2ec1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/4095864673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec1086d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009ca0 tx=0x7ff2e402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 -- 172.21.15.22:0/4095864673 shutdown_connections 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/4095864673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec1086d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/4095864673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2ec101480 0x7ff2ec101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/4095864673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2ec100290 0x7ff2ec1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 -- 172.21.15.22:0/4095864673 >> 172.21.15.22:0/4095864673 conn(0x7ff2ec0fb820 msgr2=0x7ff2ec0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 -- 172.21.15.22:0/4095864673 shutdown_connections 2024-03-08T23:51:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.858+0000 7ff2f0be6640 1 -- 172.21.15.22:0/4095864673 wait complete. 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 Processor -- start 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 -- start start 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2ec100290 0x7ff2ec104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2ec101480 0x7ff2ec102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec10be80 con 0x7ff2ec101e40 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec10bd00 con 0x7ff2ec101480 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2f0be6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec10c000 con 0x7ff2ec100290 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ead9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ead9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53132/0 (socket says 172.21.15.22:53132) 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ead9d640 1 -- 172.21.15.22:0/2695308884 learned_addr learned my addr 172.21.15.22:0/2695308884 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ea59c640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2ec100290 0x7ff2ec104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2e9d9b640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2ec101480 0x7ff2ec102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ead9d640 1 -- 172.21.15.22:0/2695308884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2ec100290 msgr2=0x7ff2ec104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ead9d640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2ec100290 0x7ff2ec104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ead9d640 1 -- 172.21.15.22:0/2695308884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2ec101480 msgr2=0x7ff2ec102860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.859+0000 7ff2ead9d640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2ec101480 0x7ff2ec102860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.860+0000 7ff2ead9d640 1 -- 172.21.15.22:0/2695308884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec103630 con 0x7ff2ec101e40 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.860+0000 7ff2ead9d640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec102da0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4037040 tx=0x7ff2e4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.860+0000 7ff2db7fe640 1 -- 172.21.15.22:0/2695308884 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4041020 con 0x7ff2ec101e40 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.860+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec1a1620 con 0x7ff2ec101e40 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.860+0000 7ff2db7fe640 1 -- 172.21.15.22:0/2695308884 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e4034d90 con 0x7ff2ec101e40 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.860+0000 7ff2db7fe640 1 -- 172.21.15.22:0/2695308884 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4039040 con 0x7ff2ec101e40 2024-03-08T23:51:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.860+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec1a1b70 con 0x7ff2ec101e40 2024-03-08T23:51:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.861+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec1a1dd0 con 0x7ff2ec101e40 2024-03-08T23:51:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.862+0000 7ff2db7fe640 1 -- 172.21.15.22:0/2695308884 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff2e4044020 con 0x7ff2ec101e40 2024-03-08T23:51:31.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.862+0000 7ff2db7fe640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2d40a2800 0x7ff2d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.863+0000 7ff2ea59c640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2d40a2800 0x7ff2d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.863+0000 7ff2db7fe640 1 -- 172.21.15.22:0/2695308884 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff2e40f0530 con 0x7ff2ec101e40 2024-03-08T23:51:31.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.863+0000 7ff2ea59c640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2d40a2800 0x7ff2d40a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec1012e0 tx=0x7ff2e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:31.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:31.867+0000 7ff2db7fe640 1 -- 172.21.15.22:0/2695308884 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff2e404b020 con 0x7ff2ec101e40 2024-03-08T23:51:32.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.087+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2ec1a2160 con 0x7ff2ec101e40 2024-03-08T23:51:32.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.087+0000 7ff2db7fe640 1 -- 172.21.15.22:0/2695308884 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7ff2e40bd9d0 con 0x7ff2ec101e40 2024-03-08T23:51:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:32.085 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-08T23:51:32.085 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2024-03-08T23:51:32.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2d40a2800 msgr2=0x7ff2d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2d40a2800 0x7ff2d40a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec1012e0 tx=0x7ff2e0005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 msgr2=0x7ff2ec102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec102da0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4037040 tx=0x7ff2e4032f20 comp rx=0 tx=0).stop 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 shutdown_connections 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2d40a2800 0x7ff2d40a4cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff2ec101e40 0x7ff2ec102da0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2ec101480 0x7ff2ec102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:32.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 --2- 172.21.15.22:0/2695308884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2ec100290 0x7ff2ec104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:32.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.089+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 >> 172.21.15.22:0/2695308884 conn(0x7ff2ec0fb820 msgr2=0x7ff2ec0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:32.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.090+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 shutdown_connections 2024-03-08T23:51:32.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:32.090+0000 7ff2f0be6640 1 -- 172.21.15.22:0/2695308884 wait complete. 2024-03-08T23:51:32.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:32 smithi022 ceph-mon[31894]: Deploying daemon mds.c on smithi192 2024-03-08T23:51:32.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:32 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6834/1768340207,v1:172.21.15.192:6835/1768340207] up:boot 2024-03-08T23:51:32.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:32 smithi022 ceph-mon[31894]: fsmap 6 up:standby 2024-03-08T23:51:32.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:32 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T23:51:32.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:32 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2695308884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:32.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:32 smithi062 ceph-mon[36129]: Deploying daemon mds.c on smithi192 2024-03-08T23:51:32.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:32 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6834/1768340207,v1:172.21.15.192:6835/1768340207] up:boot 2024-03-08T23:51:32.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:32 smithi062 ceph-mon[36129]: fsmap 6 up:standby 2024-03-08T23:51:32.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:32 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T23:51:32.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:32 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2695308884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:32.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:32 smithi192 ceph-mon[35995]: Deploying daemon mds.c on smithi192 2024-03-08T23:51:32.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:32 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6834/1768340207,v1:172.21.15.192:6835/1768340207] up:boot 2024-03-08T23:51:32.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:32 smithi192 ceph-mon[35995]: fsmap 6 up:standby 2024-03-08T23:51:32.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:32 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T23:51:32.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:32 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2695308884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:33.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:33 smithi192 ceph-mon[35995]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:33.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:33 smithi022 ceph-mon[31894]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:33.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:33 smithi062 ceph-mon[36129]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:34.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:34.131 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 08 23:51:33 smithi192 systemd[1]: Started Ceph mds.c for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 -- 172.21.15.22:0/2594919707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 msgr2=0x7f70401086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 --2- 172.21.15.22:0/2594919707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401086d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7030009ca0 tx=0x7f703002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 -- 172.21.15.22:0/2594919707 shutdown_connections 2024-03-08T23:51:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 --2- 172.21.15.22:0/2594919707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401086d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 --2- 172.21.15.22:0/2594919707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7040101480 0x7f7040101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 --2- 172.21.15.22:0/2594919707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7040100290 0x7f70401006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 -- 172.21.15.22:0/2594919707 >> 172.21.15.22:0/2594919707 conn(0x7f70400fb820 msgr2=0x7f70400fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 -- 172.21.15.22:0/2594919707 shutdown_connections 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.216+0000 7f704853b640 1 -- 172.21.15.22:0/2594919707 wait complete. 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 Processor -- start 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 -- start start 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7040100290 0x7f704006a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7040101480 0x7f704019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704010beb0 con 0x7f7040101e40 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704010bd30 con 0x7f7040101480 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f704853b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704010c030 con 0x7f7040100290 2024-03-08T23:51:34.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53138/0 (socket says 172.21.15.22:53138) 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 -- 172.21.15.22:0/735830322 learned_addr learned my addr 172.21.15.22:0/735830322 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7045ad6640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7040101480 0x7f704019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f70462d7640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7040100290 0x7f704006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 -- 172.21.15.22:0/735830322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7040100290 msgr2=0x7f704006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7040100290 0x7f704006a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 -- 172.21.15.22:0/735830322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7040101480 msgr2=0x7f704019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7040101480 0x7f704019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 -- 172.21.15.22:0/735830322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70401a1410 con 0x7f7040101e40 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f7046ad8640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401a0c20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f703002c630 tx=0x7f7030032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.217+0000 7f703f7fe640 1 -- 172.21.15.22:0/735830322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030041020 con 0x7f7040101e40 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.218+0000 7f703f7fe640 1 -- 172.21.15.22:0/735830322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7030034cd0 con 0x7f7040101e40 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.218+0000 7f703f7fe640 1 -- 172.21.15.22:0/735830322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030039040 con 0x7f7040101e40 2024-03-08T23:51:34.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.218+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70401a1640 con 0x7f7040101e40 2024-03-08T23:51:34.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.218+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70401a1b30 con 0x7f7040101e40 2024-03-08T23:51:34.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.218+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70401075b0 con 0x7f7040101e40 2024-03-08T23:51:34.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.220+0000 7f703f7fe640 1 -- 172.21.15.22:0/735830322 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7030044020 con 0x7f7040101e40 2024-03-08T23:51:34.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.220+0000 7f703f7fe640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f70280a2830 0x7f70280a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.220+0000 7f703f7fe640 1 -- 172.21.15.22:0/735830322 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70300f0460 con 0x7f7040101e40 2024-03-08T23:51:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.220+0000 7f70462d7640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f70280a2830 0x7f70280a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.221+0000 7f70462d7640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f70280a2830 0x7f70280a4cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70401012e0 tx=0x7f7038005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.225+0000 7f703f7fe640 1 -- 172.21.15.22:0/735830322 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f703004b020 con 0x7f7040101e40 2024-03-08T23:51:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.446+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f704002d070 con 0x7f7040101e40 2024-03-08T23:51:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.446+0000 7f703f7fe640 1 -- 172.21.15.22:0/735830322 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f70300bda30 con 0x7f7040101e40 2024-03-08T23:51:34.443 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:34.443 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-08T23:51:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-03-08T23:51:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f70280a2830 msgr2=0x7f70280a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f70280a2830 0x7f70280a4cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70401012e0 tx=0x7f7038005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 msgr2=0x7f70401a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401a0c20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f703002c630 tx=0x7f7030032ea0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 shutdown_connections 2024-03-08T23:51:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f70280a2830 0x7f70280a4cf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7040101e40 0x7f70401a0c20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7040101480 0x7f704019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 --2- 172.21.15.22:0/735830322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7040100290 0x7f704006a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 >> 172.21.15.22:0/735830322 conn(0x7f70400fb820 msgr2=0x7f7040104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.448+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 shutdown_connections 2024-03-08T23:51:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:34.449+0000 7f704853b640 1 -- 172.21.15.22:0/735830322 wait complete. 2024-03-08T23:51:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:35.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:35.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: Deploying daemon mds.h on smithi062 2024-03-08T23:51:35.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] up:boot 2024-03-08T23:51:35.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: fsmap 7 up:standby 2024-03-08T23:51:35.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T23:51:35.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:34 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/735830322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:35.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:35.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: Deploying daemon mds.h on smithi062 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] up:boot 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: fsmap 7 up:standby 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T23:51:35.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:34 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/735830322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:35.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: Deploying daemon mds.h on smithi062 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] up:boot 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: fsmap 7 up:standby 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T23:51:35.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:34 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/735830322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:36.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:36.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.575+0000 7f910f137640 1 -- 172.21.15.22:0/1425821282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108101e10 msgr2=0x7f9108104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.575+0000 7f910f137640 1 --2- 172.21.15.22:0/1425821282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108101e10 0x7f9108104200 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9100009ca0 tx=0x7f910002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.575+0000 7f910f137640 1 -- 172.21.15.22:0/1425821282 shutdown_connections 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.575+0000 7f910f137640 1 --2- 172.21.15.22:0/1425821282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9108107070 0x7f9108109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.575+0000 7f910f137640 1 --2- 172.21.15.22:0/1425821282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9108104740 0x7f9108106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 --2- 172.21.15.22:0/1425821282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108101e10 0x7f9108104200 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 -- 172.21.15.22:0/1425821282 >> 172.21.15.22:0/1425821282 conn(0x7f91080fb7a0 msgr2=0x7f91080fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 -- 172.21.15.22:0/1425821282 shutdown_connections 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 -- 172.21.15.22:0/1425821282 wait complete. 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 Processor -- start 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 -- start start 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9108101e10 0x7f9108101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108104740 0x7f91080ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9108107070 0x7f9108100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910810cfc0 con 0x7f9108104740 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910810ce40 con 0x7f9108101e10 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f910f137640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910810d140 con 0x7f9108107070 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f9107fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108104740 0x7f91080ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f9107fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108104740 0x7f91080ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53152/0 (socket says 172.21.15.22:53152) 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.576+0000 7f9107fff640 1 -- 172.21.15.22:0/3845673547 learned_addr learned my addr 172.21.15.22:0/3845673547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f910ced3640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9108101e10 0x7f9108101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f910d6d4640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9108107070 0x7f9108100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9107fff640 1 -- 172.21.15.22:0/3845673547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9108107070 msgr2=0x7f9108100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9107fff640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9108107070 0x7f9108100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9107fff640 1 -- 172.21.15.22:0/3845673547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9108101e10 msgr2=0x7f9108101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9107fff640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9108101e10 0x7f9108101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9107fff640 1 -- 172.21.15.22:0/3845673547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9108100b90 con 0x7f9108104740 2024-03-08T23:51:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f910ced3640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9108101e10 0x7f9108101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9107fff640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108104740 0x7f91080ffdc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e9c0 tx=0x7f90f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9105ffb640 1 -- 172.21.15.22:0/3845673547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f800cd80 con 0x7f9108104740 2024-03-08T23:51:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9105ffb640 1 -- 172.21.15.22:0/3845673547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90f8005500 con 0x7f9108104740 2024-03-08T23:51:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91081a17a0 con 0x7f9108104740 2024-03-08T23:51:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f9105ffb640 1 -- 172.21.15.22:0/3845673547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8010690 con 0x7f9108104740 2024-03-08T23:51:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.577+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91081a1cf0 con 0x7f9108104740 2024-03-08T23:51:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.578+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91080fd290 con 0x7f9108104740 2024-03-08T23:51:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.579+0000 7f9105ffb640 1 -- 172.21.15.22:0/3845673547 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f90f80046c0 con 0x7f9108104740 2024-03-08T23:51:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.580+0000 7f9105ffb640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f90f40a2800 0x7f90f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.580+0000 7f910ced3640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f90f40a2800 0x7f90f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.580+0000 7f9105ffb640 1 -- 172.21.15.22:0/3845673547 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90f80ceae0 con 0x7f9108104740 2024-03-08T23:51:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.581+0000 7f910ced3640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f90f40a2800 0x7f90f40a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9100006010 tx=0x7f91000325b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.585+0000 7f9105ffb640 1 -- 172.21.15.22:0/3845673547 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f90f809c020 con 0x7f9108104740 2024-03-08T23:51:36.612 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:51:36 smithi062 podman[66290]: 2024-03-08 23:51:36.473650226 +0000 UTC m=+0.124295614 container create 12de8012086c79af9fc2f973552e1d1e0e718e1061a2ce1c633031345414ba07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T23:51:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.806+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f910802d020 con 0x7f9108104740 2024-03-08T23:51:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.807+0000 7f9105ffb640 1 -- 172.21.15.22:0/3845673547 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f90f809b920 con 0x7f9108104740 2024-03-08T23:51:36.803 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:36.803 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-08T23:51:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-03-08T23:51:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.808+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f90f40a2800 msgr2=0x7f90f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.808+0000 7f910f137640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f90f40a2800 0x7f90f40a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9100006010 tx=0x7f91000325b0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108104740 msgr2=0x7f91080ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108104740 0x7f91080ffdc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e9c0 tx=0x7f90f800ee80 comp rx=0 tx=0).stop 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 shutdown_connections 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9108107070 0x7f9108100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f90f40a2800 0x7f90f40a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9108104740 0x7f91080ffdc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 --2- 172.21.15.22:0/3845673547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9108101e10 0x7f9108101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 >> 172.21.15.22:0/3845673547 conn(0x7f91080fb7a0 msgr2=0x7f9108102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 shutdown_connections 2024-03-08T23:51:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:36.809+0000 7f910f137640 1 -- 172.21.15.22:0/3845673547 wait complete. 2024-03-08T23:51:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:36 smithi062 ceph-mon[36129]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:36 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3845673547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:37.112 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:51:36 smithi062 podman[66290]: 2024-03-08 23:51:36.751664792 +0000 UTC m=+0.402310195 container init 12de8012086c79af9fc2f973552e1d1e0e718e1061a2ce1c633031345414ba07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-08T23:51:37.112 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:51:36 smithi062 podman[66290]: 2024-03-08 23:51:36.754432815 +0000 UTC m=+0.405078207 container start 12de8012086c79af9fc2f973552e1d1e0e718e1061a2ce1c633031345414ba07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-08T23:51:37.112 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:51:36 smithi062 ceph-mds[66309]: -- 172.21.15.62:0/2239958455 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fca7799a40 con 0x55fca776cc00 2024-03-08T23:51:37.112 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:51:36 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: starting mds.h at 2024-03-08T23:51:37.112 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:51:36 smithi062 bash[66290]: 12de8012086c79af9fc2f973552e1d1e0e718e1061a2ce1c633031345414ba07 2024-03-08T23:51:37.113 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:51:36 smithi062 systemd[1]: Started Ceph mds.h for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:37.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:36 smithi192 ceph-mon[35995]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:37.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:36 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3845673547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:37.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:36 smithi022 ceph-mon[31894]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:37.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:36 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3845673547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6836/1631408985,v1:172.21.15.62:6837/1631408985] up:boot 2024-03-08T23:51:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: fsmap 8 up:standby 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:37 smithi062 ceph-mon[36129]: Deploying daemon mds.i on smithi192 2024-03-08T23:51:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6836/1631408985,v1:172.21.15.62:6837/1631408985] up:boot 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: fsmap 8 up:standby 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:37 smithi192 ceph-mon[35995]: Deploying daemon mds.i on smithi192 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6836/1631408985,v1:172.21.15.62:6837/1631408985] up:boot 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: fsmap 8 up:standby 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:37 smithi022 ceph-mon[31894]: Deploying daemon mds.i on smithi192 2024-03-08T23:51:38.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 -- 172.21.15.22:0/2556710139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690107070 msgr2=0x7f66901094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 --2- 172.21.15.22:0/2556710139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690107070 0x7f66901094c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6688009ca0 tx=0x7f668802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 -- 172.21.15.22:0/2556710139 shutdown_connections 2024-03-08T23:51:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 --2- 172.21.15.22:0/2556710139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690107070 0x7f66901094c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 --2- 172.21.15.22:0/2556710139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6690104740 0x7f6690106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 --2- 172.21.15.22:0/2556710139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6690101e10 0x7f6690104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 -- 172.21.15.22:0/2556710139 >> 172.21.15.22:0/2556710139 conn(0x7f66900fb820 msgr2=0x7f66900fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 -- 172.21.15.22:0/2556710139 shutdown_connections 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.935+0000 7f66970ea640 1 -- 172.21.15.22:0/2556710139 wait complete. 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 Processor -- start 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 -- start start 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6690101e10 0x7f6690101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690104740 0x7f66900ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6690107070 0x7f6690100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669010cfb0 con 0x7f6690104740 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669010ce30 con 0x7f6690101e10 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f66970ea640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669010d130 con 0x7f6690107070 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690104740 0x7f66900ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:38.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690104740 0x7f66900ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53156/0 (socket says 172.21.15.22:53156) 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 -- 172.21.15.22:0/3875716758 learned_addr learned my addr 172.21.15.22:0/3875716758 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f6695687640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6690107070 0x7f6690100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f6694e86640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6690101e10 0x7f6690101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 -- 172.21.15.22:0/3875716758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6690107070 msgr2=0x7f6690100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6690107070 0x7f6690100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 -- 172.21.15.22:0/3875716758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6690101e10 msgr2=0x7f6690101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6690101e10 0x7f6690101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f668ffff640 1 -- 172.21.15.22:0/3875716758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690100b90 con 0x7f6690104740 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.936+0000 7f6694e86640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6690101e10 0x7f6690101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.937+0000 7f668ffff640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690104740 0x7f66900ffdc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d770 tx=0x7f667c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.937+0000 7f668dffb640 1 -- 172.21.15.22:0/3875716758 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c017070 con 0x7f6690104740 2024-03-08T23:51:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.937+0000 7f668dffb640 1 -- 172.21.15.22:0/3875716758 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f667c009c30 con 0x7f6690104740 2024-03-08T23:51:38.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.937+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66901a16e0 con 0x7f6690104740 2024-03-08T23:51:38.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.937+0000 7f668dffb640 1 -- 172.21.15.22:0/3875716758 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00f620 con 0x7f6690104740 2024-03-08T23:51:38.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.937+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66901a1c30 con 0x7f6690104740 2024-03-08T23:51:38.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.938+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900fd310 con 0x7f6690104740 2024-03-08T23:51:38.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.939+0000 7f668dffb640 1 -- 172.21.15.22:0/3875716758 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f667c01a070 con 0x7f6690104740 2024-03-08T23:51:38.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.940+0000 7f668dffb640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f66800a2800 0x7f66800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:38.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.940+0000 7f6694e86640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f66800a2800 0x7f66800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:38.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.940+0000 7f668dffb640 1 -- 172.21.15.22:0/3875716758 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f667c0ce130 con 0x7f6690104740 2024-03-08T23:51:38.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.940+0000 7f6694e86640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f66800a2800 0x7f66800a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6684009b50 tx=0x7f6684005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:38.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:38.945+0000 7f668dffb640 1 -- 172.21.15.22:0/3875716758 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f667c009de0 con 0x7f6690104740 2024-03-08T23:51:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:38 smithi192 ceph-mon[35995]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:39.132 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 systemd[1]: Starting Ceph mds.i for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:39.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:38 smithi022 ceph-mon[31894]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:39.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.165+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f669002ce70 con 0x7f6690104740 2024-03-08T23:51:39.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.166+0000 7f668dffb640 1 -- 172.21.15.22:0/3875716758 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7f667c09b5d0 con 0x7f6690104740 2024-03-08T23:51:39.163 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:39.163 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-08T23:51:39.163 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f66800a2800 msgr2=0x7f66800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f66800a2800 0x7f66800a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6684009b50 tx=0x7f6684005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690104740 msgr2=0x7f66900ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690104740 0x7f66900ffdc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d770 tx=0x7f667c00dc30 comp rx=0 tx=0).stop 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 shutdown_connections 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6690107070 0x7f6690100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f66800a2800 0x7f66800a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6690104740 0x7f66900ffdc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 --2- 172.21.15.22:0/3875716758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6690101e10 0x7f6690101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 >> 172.21.15.22:0/3875716758 conn(0x7f66900fb820 msgr2=0x7f66901026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 shutdown_connections 2024-03-08T23:51:39.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:39.168+0000 7f66970ea640 1 -- 172.21.15.22:0/3875716758 wait complete. 2024-03-08T23:51:39.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:38 smithi062 ceph-mon[36129]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:39.520 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 podman[67808]: 2024-03-08 23:51:39.160053156 +0000 UTC m=+0.018502518 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:39.520 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 podman[67808]: 2024-03-08 23:51:39.26237772 +0000 UTC m=+0.120827115 container create 60a42d821339e8b1ce3bda27dcbd3a16830afc0788578e8d235d2bd4a1d06575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-03-08T23:51:39.520 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 podman[67808]: 2024-03-08 23:51:39.524278645 +0000 UTC m=+0.382728017 container init 60a42d821339e8b1ce3bda27dcbd3a16830afc0788578e8d235d2bd4a1d06575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-08T23:51:39.882 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 podman[67808]: 2024-03-08 23:51:39.526992341 +0000 UTC m=+0.385441696 container start 60a42d821339e8b1ce3bda27dcbd3a16830afc0788578e8d235d2bd4a1d06575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T23:51:39.882 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mds[67827]: -- 172.21.15.192:0/694766454 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b496bba40 con 0x559b4968f000 2024-03-08T23:51:39.882 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: starting mds.i at 2024-03-08T23:51:39.882 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 bash[67808]: 60a42d821339e8b1ce3bda27dcbd3a16830afc0788578e8d235d2bd4a1d06575 2024-03-08T23:51:39.882 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:39 smithi192 systemd[1]: Started Ceph mds.i for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:40.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3875716758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:40.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:40.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:40.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:40.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:39 smithi022 ceph-mon[31894]: Deploying daemon mds.g on smithi022 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3875716758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:39 smithi062 ceph-mon[36129]: Deploying daemon mds.g on smithi022 2024-03-08T23:51:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3875716758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:40.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:39 smithi192 ceph-mon[35995]: Deploying daemon mds.g on smithi022 2024-03-08T23:51:41.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:41.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:40 smithi022 ceph-mon[31894]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:41.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:40 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] up:boot 2024-03-08T23:51:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:40 smithi022 ceph-mon[31894]: fsmap 9 up:standby 2024-03-08T23:51:41.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:40 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T23:51:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 -- 172.21.15.22:0/3384123430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102ec0 msgr2=0x7fd2ac109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/3384123430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102ec0 0x7fd2ac109750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009ca0 tx=0x7fd2a402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 -- 172.21.15.22:0/3384123430 shutdown_connections 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/3384123430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102ec0 0x7fd2ac109750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/3384123430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2ac102500 0x7fd2ac102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/3384123430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac101310 0x7fd2ac101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 -- 172.21.15.22:0/3384123430 >> 172.21.15.22:0/3384123430 conn(0x7fd2ac0fc8a0 msgr2=0x7fd2ac0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 -- 172.21.15.22:0/3384123430 shutdown_connections 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.292+0000 7fd2b19b9640 1 -- 172.21.15.22:0/3384123430 wait complete. 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 Processor -- start 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 -- start start 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2ac101310 0x7fd2ac1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:41.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102500 0x7fd2ac197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac102ec0 0x7fd2ac19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac10cfe0 con 0x7fd2ac102500 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac10ce60 con 0x7fd2ac102ec0 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2b19b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac10d160 con 0x7fd2ac101310 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102500 0x7fd2ac197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102500 0x7fd2ac197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55906/0 (socket says 172.21.15.22:55906) 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 -- 172.21.15.22:0/1871286910 learned_addr learned my addr 172.21.15.22:0/1871286910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2ab7fe640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac102ec0 0x7fd2ac19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 -- 172.21.15.22:0/1871286910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2ac101310 msgr2=0x7fd2ac1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2aaffd640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2ac101310 0x7fd2ac1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2ac101310 0x7fd2ac1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 -- 172.21.15.22:0/1871286910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac102ec0 msgr2=0x7fd2ac19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac102ec0 0x7fd2ac19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.293+0000 7fd2a3fff640 1 -- 172.21.15.22:0/1871286910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac19f210 con 0x7fd2ac102500 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2ab7fe640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac102ec0 0x7fd2ac19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2aaffd640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2ac101310 0x7fd2ac1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2a3fff640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102500 0x7fd2ac197b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd29400b4a0 tx=0x7fd29400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2b09b7640 1 -- 172.21.15.22:0/1871286910 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294014070 con 0x7fd2ac102500 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2b09b7640 1 -- 172.21.15.22:0/1871286910 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd294010af0 con 0x7fd2ac102500 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac19f500 con 0x7fd2ac102500 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2b09b7640 1 -- 172.21.15.22:0/1871286910 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294017d10 con 0x7fd2ac102500 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.294+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac19fa50 con 0x7fd2ac102500 2024-03-08T23:51:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.295+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac108630 con 0x7fd2ac102500 2024-03-08T23:51:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.296+0000 7fd2b09b7640 1 -- 172.21.15.22:0/1871286910 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd294017070 con 0x7fd2ac102500 2024-03-08T23:51:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.296+0000 7fd2b09b7640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2980a2800 0x7fd2980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.297+0000 7fd2aaffd640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2980a2800 0x7fd2980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.297+0000 7fd2b09b7640 1 -- 172.21.15.22:0/1871286910 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2940cda80 con 0x7fd2ac102500 2024-03-08T23:51:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.297+0000 7fd2aaffd640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2980a2800 0x7fd2980a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac102360 tx=0x7fd29c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:41.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.301+0000 7fd2b09b7640 1 -- 172.21.15.22:0/1871286910 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd2940cfe20 con 0x7fd2ac102500 2024-03-08T23:51:41.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:40 smithi062 ceph-mon[36129]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:41.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:40 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] up:boot 2024-03-08T23:51:41.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:40 smithi062 ceph-mon[36129]: fsmap 9 up:standby 2024-03-08T23:51:41.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:40 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T23:51:41.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:40 smithi192 ceph-mon[35995]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:41.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:40 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] up:boot 2024-03-08T23:51:41.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:40 smithi192 ceph-mon[35995]: fsmap 9 up:standby 2024-03-08T23:51:41.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:40 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T23:51:41.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.523+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2ac02ce70 con 0x7fd2ac102500 2024-03-08T23:51:41.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.524+0000 7fd2b09b7640 1 -- 172.21.15.22:0/1871286910 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7fd29409af20 con 0x7fd2ac102500 2024-03-08T23:51:41.526 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:41.526 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-08T23:51:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 10 2024-03-08T23:51:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.531+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2980a2800 msgr2=0x7fd2980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.531+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2980a2800 0x7fd2980a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac102360 tx=0x7fd29c005d30 comp rx=0 tx=0).stop 2024-03-08T23:51:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.531+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102500 msgr2=0x7fd2ac197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:41.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.531+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102500 0x7fd2ac197b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd29400b4a0 tx=0x7fd29400b960 comp rx=0 tx=0).stop 2024-03-08T23:51:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 shutdown_connections 2024-03-08T23:51:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2ac102ec0 0x7fd2ac19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2980a2800 0x7fd2980a4cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac102500 0x7fd2ac197b30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 --2- 172.21.15.22:0/1871286910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2ac101310 0x7fd2ac1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 >> 172.21.15.22:0/1871286910 conn(0x7fd2ac0fc8a0 msgr2=0x7fd2ac105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 shutdown_connections 2024-03-08T23:51:41.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:41.532+0000 7fd2b19b9640 1 -- 172.21.15.22:0/1871286910 wait complete. 2024-03-08T23:51:41.965 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:41 smithi022 systemd[1]: Starting Ceph mds.g for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:42.222 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:41 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1871286910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:42.222 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 podman[72868]: 2024-03-08 23:51:41.987656469 +0000 UTC m=+0.017951999 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:42.222 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 podman[72868]: 2024-03-08 23:51:42.100928807 +0000 UTC m=+0.131224381 container create 56d084ddba189ea5ccdc032e8914c4dd80ff2499d4c1e6107eb7c346188fca92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-08T23:51:42.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:41 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1871286910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:42.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:41 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1871286910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:42.486 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 podman[72868]: 2024-03-08 23:51:42.312004209 +0000 UTC m=+0.342299737 container init 56d084ddba189ea5ccdc032e8914c4dd80ff2499d4c1e6107eb7c346188fca92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-08T23:51:42.486 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 podman[72868]: 2024-03-08 23:51:42.31483268 +0000 UTC m=+0.345128200 container start 56d084ddba189ea5ccdc032e8914c4dd80ff2499d4c1e6107eb7c346188fca92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-08T23:51:42.486 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 ceph-mds[72887]: -- 172.21.15.22:0/216557815 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b9c683a40 con 0x559b9c656c00 2024-03-08T23:51:42.486 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g[72883]: starting mds.g at 2024-03-08T23:51:42.486 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 bash[72868]: 56d084ddba189ea5ccdc032e8914c4dd80ff2499d4c1e6107eb7c346188fca92 2024-03-08T23:51:42.486 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 08 23:51:42 smithi022 systemd[1]: Started Ceph mds.g for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:43.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:43.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.650+0000 7ffab8e6f640 1 -- 172.21.15.22:0/1580485094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4100290 msgr2=0x7ffab41006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/1580485094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4100290 0x7ffab41006b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009900 tx=0x7ffaa402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 -- 172.21.15.22:0/1580485094 shutdown_connections 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/1580485094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffab4101e40 0x7ffab41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/1580485094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4101480 0x7ffab4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/1580485094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4100290 0x7ffab41006b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 -- 172.21.15.22:0/1580485094 >> 172.21.15.22:0/1580485094 conn(0x7ffab40fb820 msgr2=0x7ffab40fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 -- 172.21.15.22:0/1580485094 shutdown_connections 2024-03-08T23:51:43.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 -- 172.21.15.22:0/1580485094 wait complete. 2024-03-08T23:51:43.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 Processor -- start 2024-03-08T23:51:43.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.651+0000 7ffab8e6f640 1 -- start start 2024-03-08T23:51:43.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffab4100290 0x7ffab4197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:43.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4101480 0x7ffab4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:43.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4101e40 0x7ffab419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab410beb0 con 0x7ffab4101480 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab410bd30 con 0x7ffab4101e40 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab410c030 con 0x7ffab4100290 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4101480 0x7ffab4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4101480 0x7ffab4197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55948/0 (socket says 172.21.15.22:55948) 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 -- 172.21.15.22:0/665685640 learned_addr learned my addr 172.21.15.22:0/665685640 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab259c640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffab4100290 0x7ffab4197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab2d9d640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4101e40 0x7ffab419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 -- 172.21.15.22:0/665685640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffab4100290 msgr2=0x7ffab4197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffab4100290 0x7ffab4197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 -- 172.21.15.22:0/665685640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4101e40 msgr2=0x7ffab419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4101e40 0x7ffab419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 -- 172.21.15.22:0/665685640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab419f260 con 0x7ffab4101480 2024-03-08T23:51:43.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab1d9b640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4101480 0x7ffab4197bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800d950 tx=0x7ffaa800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:43.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffaa37fe640 1 -- 172.21.15.22:0/665685640 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8017070 con 0x7ffab4101480 2024-03-08T23:51:43.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab419f550 con 0x7ffab4101480 2024-03-08T23:51:43.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffaa37fe640 1 -- 172.21.15.22:0/665685640 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaa8009c30 con 0x7ffab4101480 2024-03-08T23:51:43.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffaa37fe640 1 -- 172.21.15.22:0/665685640 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800f6b0 con 0x7ffab4101480 2024-03-08T23:51:43.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.652+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab419faa0 con 0x7ffab4101480 2024-03-08T23:51:43.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.654+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab419fd00 con 0x7ffab4101480 2024-03-08T23:51:43.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.654+0000 7ffaa37fe640 1 -- 172.21.15.22:0/665685640 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffaa80109d0 con 0x7ffab4101480 2024-03-08T23:51:43.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.655+0000 7ffaa37fe640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ffa9c0a2800 0x7ffa9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:43.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.655+0000 7ffab259c640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ffa9c0a2800 0x7ffa9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:43.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.655+0000 7ffaa37fe640 1 -- 172.21.15.22:0/665685640 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffaa80ce220 con 0x7ffab4101480 2024-03-08T23:51:43.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.655+0000 7ffab259c640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ffa9c0a2800 0x7ffa9c0a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4000c00 tx=0x7ffaa4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:43.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.659+0000 7ffaa37fe640 1 -- 172.21.15.22:0/665685640 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ffaa8009de0 con 0x7ffab4101480 2024-03-08T23:51:43.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:43.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: Deploying daemon mds.a on smithi022 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.22:6838/1763775736,v1:172.21.15.22:6839/1763775736] up:boot 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: fsmap 10 up:standby 2024-03-08T23:51:43.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:43 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:43.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: Deploying daemon mds.a on smithi022 2024-03-08T23:51:43.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.22:6838/1763775736,v1:172.21.15.22:6839/1763775736] up:boot 2024-03-08T23:51:43.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: fsmap 10 up:standby 2024-03-08T23:51:43.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:43 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T23:51:43.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: Deploying daemon mds.a on smithi022 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.22:6838/1763775736,v1:172.21.15.22:6839/1763775736] up:boot 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: fsmap 10 up:standby 2024-03-08T23:51:43.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:43 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T23:51:43.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.889+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffab419fed0 con 0x7ffab4101480 2024-03-08T23:51:43.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.890+0000 7ffaa37fe640 1 -- 172.21.15.22:0/665685640 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7ffaa809b070 con 0x7ffab4101480 2024-03-08T23:51:43.886 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:43.886 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-08T23:51:43.887 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 11 2024-03-08T23:51:43.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ffa9c0a2800 msgr2=0x7ffa9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:43.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ffa9c0a2800 0x7ffa9c0a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4000c00 tx=0x7ffaa4005fd0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4101480 msgr2=0x7ffab4197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4101480 0x7ffab4197bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800d950 tx=0x7ffaa800de10 comp rx=0 tx=0).stop 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 shutdown_connections 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4101e40 0x7ffab419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ffa9c0a2800 0x7ffa9c0a4cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffab4101480 0x7ffab4197bd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 --2- 172.21.15.22:0/665685640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffab4100290 0x7ffab4197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 >> 172.21.15.22:0/665685640 conn(0x7ffab40fb820 msgr2=0x7ffab40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 shutdown_connections 2024-03-08T23:51:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:43.892+0000 7ffab8e6f640 1 -- 172.21.15.22:0/665685640 wait complete. 2024-03-08T23:51:44.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:44 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/665685640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:44.736 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:44 smithi022 systemd[1]: Starting Ceph mds.a for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:44 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/665685640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:44.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:44 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/665685640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:45.056 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:44 smithi022 podman[73216]: 2024-03-08 23:51:44.710144594 +0000 UTC m=+0.017516102 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:45.056 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:44 smithi022 podman[73216]: 2024-03-08 23:51:44.824166476 +0000 UTC m=+0.131538010 container create 8957338e3b2fbf3460a9ae78e95ae4469f0c13b3f7a9d30e8112a04d4c14f2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-08T23:51:45.056 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:45 smithi022 podman[73216]: 2024-03-08 23:51:45.060326494 +0000 UTC m=+0.367697983 container init 8957338e3b2fbf3460a9ae78e95ae4469f0c13b3f7a9d30e8112a04d4c14f2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T23:51:45.473 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:45 smithi022 podman[73216]: 2024-03-08 23:51:45.062972875 +0000 UTC m=+0.370344369 container start 8957338e3b2fbf3460a9ae78e95ae4469f0c13b3f7a9d30e8112a04d4c14f2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-08T23:51:45.473 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mds[73235]: -- 172.21.15.22:0/3936642249 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55615aaf9a40 con 0x55615aacc800 2024-03-08T23:51:45.473 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a[73231]: starting mds.a at 2024-03-08T23:51:45.473 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:45 smithi022 bash[73216]: 8957338e3b2fbf3460a9ae78e95ae4469f0c13b3f7a9d30e8112a04d4c14f2d7 2024-03-08T23:51:45.474 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 08 23:51:45 smithi022 systemd[1]: Started Ceph mds.a for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mon[31894]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:45.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:45.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:45 smithi062 ceph-mon[36129]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:45 smithi192 ceph-mon[35995]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T23:51:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T23:51:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:45.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 -- 172.21.15.22:0/1776273892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4441070d0 msgr2=0x7fc4441094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 --2- 172.21.15.22:0/1776273892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4441070d0 0x7fc4441094c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009ca0 tx=0x7fc43c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 -- 172.21.15.22:0/1776273892 shutdown_connections 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 --2- 172.21.15.22:0/1776273892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4441070d0 0x7fc4441094c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 --2- 172.21.15.22:0/1776273892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4441047a0 0x7fc444106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 --2- 172.21.15.22:0/1776273892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc444101e70 0x7fc444104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 -- 172.21.15.22:0/1776273892 >> 172.21.15.22:0/1776273892 conn(0x7fc4440fb820 msgr2=0x7fc4440fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 -- 172.21.15.22:0/1776273892 shutdown_connections 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.016+0000 7fc448bc4640 1 -- 172.21.15.22:0/1776273892 wait complete. 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 Processor -- start 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 -- start start 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc444101e70 0x7fc4441017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4441047a0 0x7fc4440ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4441070d0 0x7fc444100360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44410cfb0 con 0x7fc444101e70 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44410ce30 con 0x7fc4441047a0 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc448bc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44410d130 con 0x7fc4441070d0 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc44259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc444101e70 0x7fc4441017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc44259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc444101e70 0x7fc4441017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55986/0 (socket says 172.21.15.22:55986) 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc44259c640 1 -- 172.21.15.22:0/4209229043 learned_addr learned my addr 172.21.15.22:0/4209229043 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc441d9b640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4441047a0 0x7fc4440ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.017+0000 7fc442d9d640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4441070d0 0x7fc444100360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc44259c640 1 -- 172.21.15.22:0/4209229043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4441070d0 msgr2=0x7fc444100360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc44259c640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4441070d0 0x7fc444100360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc44259c640 1 -- 172.21.15.22:0/4209229043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4441047a0 msgr2=0x7fc4440ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc44259c640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4441047a0 0x7fc4440ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc44259c640 1 -- 172.21.15.22:0/4209229043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444100bf0 con 0x7fc444101e70 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc441d9b640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4441047a0 0x7fc4440ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc44259c640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc444101e70 0x7fc4441017e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc43400d770 tx=0x7fc43400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc4337fe640 1 -- 172.21.15.22:0/4209229043 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434017070 con 0x7fc444101e70 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4441a16f0 con 0x7fc444101e70 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc4337fe640 1 -- 172.21.15.22:0/4209229043 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc434005500 con 0x7fc444101e70 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc4337fe640 1 -- 172.21.15.22:0/4209229043 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434004e00 con 0x7fc444101e70 2024-03-08T23:51:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.018+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4441a1bf0 con 0x7fc444101e70 2024-03-08T23:51:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.019+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4440fd310 con 0x7fc444101e70 2024-03-08T23:51:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.021+0000 7fc4337fe640 1 -- 172.21.15.22:0/4209229043 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc434004fa0 con 0x7fc444101e70 2024-03-08T23:51:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.021+0000 7fc4337fe640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc42c0a2800 0x7fc42c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.021+0000 7fc441d9b640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc42c0a2800 0x7fc42c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.021+0000 7fc4337fe640 1 -- 172.21.15.22:0/4209229043 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4340ce440 con 0x7fc444101e70 2024-03-08T23:51:46.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.022+0000 7fc441d9b640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc42c0a2800 0x7fc42c0a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc444101550 tx=0x7fc438005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:46.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.025+0000 7fc4337fe640 1 -- 172.21.15.22:0/4209229043 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc43409b980 con 0x7fc444101e70 2024-03-08T23:51:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.245+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc44402ce70 con 0x7fc444101e70 2024-03-08T23:51:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.246+0000 7fc4337fe640 1 -- 172.21.15.22:0/4209229043 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9248 (secure 0 0 0) 0x7fc43409b280 con 0x7fc444101e70 2024-03-08T23:51:46.243 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:46.243 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-08T23:51:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 12 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc42c0a2800 msgr2=0x7fc42c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc42c0a2800 0x7fc42c0a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc444101550 tx=0x7fc438005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc444101e70 msgr2=0x7fc4441017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc444101e70 0x7fc4441017e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc43400d770 tx=0x7fc43400dc30 comp rx=0 tx=0).stop 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 shutdown_connections 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4441070d0 0x7fc444100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4441047a0 0x7fc4440ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc42c0a2800 0x7fc42c0a4cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 --2- 172.21.15.22:0/4209229043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc444101e70 0x7fc4441017e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 >> 172.21.15.22:0/4209229043 conn(0x7fc4440fb820 msgr2=0x7fc444102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 shutdown_connections 2024-03-08T23:51:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:46.248+0000 7fc448bc4640 1 -- 172.21.15.22:0/4209229043 wait complete. 2024-03-08T23:51:46.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:46 smithi022 ceph-mon[31894]: Deploying daemon mds.b on smithi062 2024-03-08T23:51:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:46 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.22:6840/2885682684,v1:172.21.15.22:6841/2885682684] up:boot 2024-03-08T23:51:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:46 smithi022 ceph-mon[31894]: fsmap 11 up:standby 2024-03-08T23:51:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:46 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T23:51:46.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:46 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4209229043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:46.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:46 smithi062 ceph-mon[36129]: Deploying daemon mds.b on smithi062 2024-03-08T23:51:46.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:46 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.22:6840/2885682684,v1:172.21.15.22:6841/2885682684] up:boot 2024-03-08T23:51:46.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:46 smithi062 ceph-mon[36129]: fsmap 11 up:standby 2024-03-08T23:51:46.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:46 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T23:51:46.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:46 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4209229043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:46 smithi192 ceph-mon[35995]: Deploying daemon mds.b on smithi062 2024-03-08T23:51:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:46 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.22:6840/2885682684,v1:172.21.15.22:6841/2885682684] up:boot 2024-03-08T23:51:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:46 smithi192 ceph-mon[35995]: fsmap 11 up:standby 2024-03-08T23:51:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:46 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T23:51:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:46 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4209229043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:47.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:47 smithi062 ceph-mon[36129]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:47.612 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:47 smithi062 systemd[1]: Starting Ceph mds.b for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:51:47.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:47 smithi022 ceph-mon[31894]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:47.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:47 smithi192 ceph-mon[35995]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:48.035 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:47 smithi062 podman[66599]: 2024-03-08 23:51:47.720354976 +0000 UTC m=+0.087660845 container create e0f4eae37122ade1691fac39b70c41e6d435863c6f11e7c19abe566993251058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T23:51:48.035 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:47 smithi062 podman[66599]: 2024-03-08 23:51:47.650850883 +0000 UTC m=+0.018156746 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:51:48.035 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:47 smithi062 podman[66599]: 2024-03-08 23:51:47.939505677 +0000 UTC m=+0.306811534 container init e0f4eae37122ade1691fac39b70c41e6d435863c6f11e7c19abe566993251058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T23:51:48.036 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:47 smithi062 podman[66599]: 2024-03-08 23:51:47.942132663 +0000 UTC m=+0.309438513 container start e0f4eae37122ade1691fac39b70c41e6d435863c6f11e7c19abe566993251058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-08T23:51:48.036 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:47 smithi062 ceph-mds[66619]: -- 172.21.15.62:0/277541019 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cacf51da40 con 0x55cacf4f1000 2024-03-08T23:51:48.036 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:47 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b[66615]: starting mds.b at 2024-03-08T23:51:48.036 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:48 smithi062 bash[66599]: e0f4eae37122ade1691fac39b70c41e6d435863c6f11e7c19abe566993251058 2024-03-08T23:51:48.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:48.361 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 08 23:51:48 smithi062 systemd[1]: Started Ceph mds.b for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:51:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 -- 172.21.15.22:0/731179154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884100210 msgr2=0x7fc884100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/731179154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884100210 0x7fc884100630 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc874009900 tx=0x7fc87402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 -- 172.21.15.22:0/731179154 shutdown_connections 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/731179154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc884101dc0 0x7fc884108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/731179154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc884101400 0x7fc884101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/731179154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884100210 0x7fc884100630 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 -- 172.21.15.22:0/731179154 >> 172.21.15.22:0/731179154 conn(0x7fc8840fb780 msgr2=0x7fc8840fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 -- 172.21.15.22:0/731179154 shutdown_connections 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 -- 172.21.15.22:0/731179154 wait complete. 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 Processor -- start 2024-03-08T23:51:48.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.370+0000 7fc88a4c0640 1 -- start start 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc88a4c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc884100210 0x7fc8841041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc88a4c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884101400 0x7fc8841027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc88a4c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc884101dc0 0x7fc884102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc88a4c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88410bea0 con 0x7fc884101400 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc88a4c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88410bd20 con 0x7fc884100210 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc88a4c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88410c020 con 0x7fc884101dc0 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884101400 0x7fc8841027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc883fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc884100210 0x7fc8841041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884101400 0x7fc8841027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56002/0 (socket says 172.21.15.22:56002) 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 -- 172.21.15.22:0/3140555699 learned_addr learned my addr 172.21.15.22:0/3140555699 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc888a5d640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc884101dc0 0x7fc884102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 -- 172.21.15.22:0/3140555699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc884101dc0 msgr2=0x7fc884102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc884101dc0 0x7fc884102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 -- 172.21.15.22:0/3140555699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc884100210 msgr2=0x7fc8841041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc884100210 0x7fc8841041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 -- 172.21.15.22:0/3140555699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8841035b0 con 0x7fc884101400 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc883fff640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc884100210 0x7fc8841041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.371+0000 7fc8837fe640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884101400 0x7fc8841027e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc87800e9f0 tx=0x7fc87800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.372+0000 7fc8817fa640 1 -- 172.21.15.22:0/3140555699 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87800cd80 con 0x7fc884101400 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.372+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8841a1740 con 0x7fc884101400 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.372+0000 7fc8817fa640 1 -- 172.21.15.22:0/3140555699 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc878010910 con 0x7fc884101400 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.372+0000 7fc8817fa640 1 -- 172.21.15.22:0/3140555699 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878018950 con 0x7fc884101400 2024-03-08T23:51:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.372+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8841a1ce0 con 0x7fc884101400 2024-03-08T23:51:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.373+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8841a1fa0 con 0x7fc884101400 2024-03-08T23:51:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.374+0000 7fc8817fa640 1 -- 172.21.15.22:0/3140555699 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc878010ac0 con 0x7fc884101400 2024-03-08T23:51:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.374+0000 7fc8817fa640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc86c0a2800 0x7fc86c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.374+0000 7fc883fff640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc86c0a2800 0x7fc86c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.374+0000 7fc8817fa640 1 -- 172.21.15.22:0/3140555699 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc8780ce980 con 0x7fc884101400 2024-03-08T23:51:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.375+0000 7fc883fff640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc86c0a2800 0x7fc86c0a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc874000c00 tx=0x7fc874005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.383+0000 7fc8817fa640 1 -- 172.21.15.22:0/3140555699 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc87809b870 con 0x7fc884101400 2024-03-08T23:51:48.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.606+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8841a2170 con 0x7fc884101400 2024-03-08T23:51:48.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.606+0000 7fc8817fa640 1 -- 172.21.15.22:0/3140555699 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10040 (secure 0 0 0) 0x7fc87809f910 con 0x7fc884101400 2024-03-08T23:51:48.603 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:48.604 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-08T23:51:48.604 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 13 2024-03-08T23:51:48.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.608+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc86c0a2800 msgr2=0x7fc86c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.608+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc86c0a2800 0x7fc86c0a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc874000c00 tx=0x7fc874005fd0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.608+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884101400 msgr2=0x7fc8841027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.608+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884101400 0x7fc8841027e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc87800e9f0 tx=0x7fc87800eeb0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 shutdown_connections 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc884101dc0 0x7fc884102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc86c0a2800 0x7fc86c0a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc884101400 0x7fc8841027e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 --2- 172.21.15.22:0/3140555699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc884100210 0x7fc8841041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 >> 172.21.15.22:0/3140555699 conn(0x7fc8840fb780 msgr2=0x7fc8840fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 shutdown_connections 2024-03-08T23:51:48.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.609+0000 7fc88a4c0640 1 -- 172.21.15.22:0/3140555699 wait complete. 2024-03-08T23:51:48.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:51:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.723+0000 7f8330340640 1 -- 172.21.15.22:0/513016779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101dc0 msgr2=0x7f8328108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.723+0000 7f8330340640 1 --2- 172.21.15.22:0/513016779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101dc0 0x7f8328108650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8320009ca0 tx=0x7f832002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- 172.21.15.22:0/513016779 shutdown_connections 2024-03-08T23:51:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 --2- 172.21.15.22:0/513016779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101dc0 0x7f8328108650 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 --2- 172.21.15.22:0/513016779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8328101400 0x7f8328101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 --2- 172.21.15.22:0/513016779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8328100210 0x7f8328100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- 172.21.15.22:0/513016779 >> 172.21.15.22:0/513016779 conn(0x7f83280fb780 msgr2=0x7f83280fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- 172.21.15.22:0/513016779 shutdown_connections 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- 172.21.15.22:0/513016779 wait complete. 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 Processor -- start 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- start start 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8328100210 0x7f8328196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101400 0x7f8328196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8328101dc0 0x7f832819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832810bea0 con 0x7f8328101400 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832810bd20 con 0x7f8328101dc0 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f8330340640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832810c020 con 0x7f8328100210 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f832d8db640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101400 0x7f8328196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f832d8db640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101400 0x7f8328196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56016/0 (socket says 172.21.15.22:56016) 2024-03-08T23:51:48.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.724+0000 7f832d8db640 1 -- 172.21.15.22:0/3934507523 learned_addr learned my addr 172.21.15.22:0/3934507523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832e0dc640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8328100210 0x7f8328196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832e8dd640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8328101dc0 0x7f832819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832d8db640 1 -- 172.21.15.22:0/3934507523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8328100210 msgr2=0x7f8328196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832d8db640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8328100210 0x7f8328196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832d8db640 1 -- 172.21.15.22:0/3934507523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8328101dc0 msgr2=0x7f832819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832d8db640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8328101dc0 0x7f832819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832d8db640 1 -- 172.21.15.22:0/3934507523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832819e150 con 0x7f8328101400 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832e0dc640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8328100210 0x7f8328196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f832d8db640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101400 0x7f8328196a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e9f0 tx=0x7f831c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f831b7fe640 1 -- 172.21.15.22:0/3934507523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00cd80 con 0x7f8328101400 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f831b7fe640 1 -- 172.21.15.22:0/3934507523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f831c005500 con 0x7f8328101400 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832819e440 con 0x7f8328101400 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f831b7fe640 1 -- 172.21.15.22:0/3934507523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c010690 con 0x7f8328101400 2024-03-08T23:51:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.725+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832819e990 con 0x7f8328101400 2024-03-08T23:51:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.726+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83281074d0 con 0x7f8328101400 2024-03-08T23:51:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.727+0000 7f831b7fe640 1 -- 172.21.15.22:0/3934507523 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f831c0046c0 con 0x7f8328101400 2024-03-08T23:51:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.728+0000 7f831b7fe640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f83140a2880 0x7f83140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.728+0000 7f832e0dc640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f83140a2880 0x7f83140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.728+0000 7f831b7fe640 1 -- 172.21.15.22:0/3934507523 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f831c0ce970 con 0x7f8328101400 2024-03-08T23:51:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.729+0000 7f832e0dc640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f83140a2880 0x7f83140a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8328101260 tx=0x7f8324005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.733+0000 7f831b7fe640 1 -- 172.21.15.22:0/3934507523 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f831c09beb0 con 0x7f8328101400 2024-03-08T23:51:48.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.934+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f832802d0c0 con 0x7f8328101400 2024-03-08T23:51:48.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.935+0000 7f831b7fe640 1 -- 172.21.15.22:0/3934507523 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f831c09b7b0 con 0x7f8328101400 2024-03-08T23:51:48.932 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:48.932 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:05.560356+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:51:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f83140a2880 msgr2=0x7f83140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f83140a2880 0x7f83140a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8328101260 tx=0x7f8324005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101400 msgr2=0x7f8328196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101400 0x7f8328196a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e9f0 tx=0x7f831c00edf0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 shutdown_connections 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8328101dc0 0x7f832819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f83140a2880 0x7f83140a4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8328101400 0x7f8328196a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 --2- 172.21.15.22:0/3934507523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8328100210 0x7f8328196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 >> 172.21.15.22:0/3934507523 conn(0x7f83280fb780 msgr2=0x7f83281046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:48.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.937+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 shutdown_connections 2024-03-08T23:51:48.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:48.938+0000 7f8330340640 1 -- 172.21.15.22:0/3934507523 wait complete. 2024-03-08T23:51:48.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T23:51:48.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/3332285583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 msgr2=0x7f8b38101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/3332285583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38101880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009900 tx=0x7f8b3402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/3332285583 shutdown_connections 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/3332285583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b38101dc0 0x7f8b38108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/3332285583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38101880 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/3332285583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b38100210 0x7f8b38100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/3332285583 >> 172.21.15.22:0/3332285583 conn(0x7f8b380fb7a0 msgr2=0x7f8b380fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/3332285583 shutdown_connections 2024-03-08T23:51:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/3332285583 wait complete. 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 Processor -- start 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- start start 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b38100210 0x7f8b38197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b38101dc0 0x7f8b3819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3810bee0 con 0x7f8b38101400 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3810bd60 con 0x7f8b38101dc0 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.056+0000 7f8b3f5bf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3810c060 con 0x7f8b38100210 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56042/0 (socket says 172.21.15.22:56042) 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 -- 172.21.15.22:0/546533105 learned_addr learned my addr 172.21.15.22:0/546533105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3db5c640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b38101dc0 0x7f8b3819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3d35b640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b38100210 0x7f8b38197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 -- 172.21.15.22:0/546533105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b38100210 msgr2=0x7f8b38197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b38100210 0x7f8b38197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 -- 172.21.15.22:0/546533105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b38101dc0 msgr2=0x7f8b3819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b38101dc0 0x7f8b3819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 -- 172.21.15.22:0/546533105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3819f340 con 0x7f8b38101400 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3cb5a640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38197cd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b3402ce70 tx=0x7f8b34005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b2a7fc640 1 -- 172.21.15.22:0/546533105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34041070 con 0x7f8b38101400 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3819f5d0 con 0x7f8b38101400 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b2a7fc640 1 -- 172.21.15.22:0/546533105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b340054e0 con 0x7f8b38101400 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b2a7fc640 1 -- 172.21.15.22:0/546533105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340346a0 con 0x7f8b38101400 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.057+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3819fb10 con 0x7f8b38101400 2024-03-08T23:51:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.058+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3819fd70 con 0x7f8b38101400 2024-03-08T23:51:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.059+0000 7f8b2a7fc640 1 -- 172.21.15.22:0/546533105 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8b34044070 con 0x7f8b38101400 2024-03-08T23:51:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.060+0000 7f8b2a7fc640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8b240a2800 0x7f8b240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.060+0000 7f8b3d35b640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8b240a2800 0x7f8b240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.060+0000 7f8b2a7fc640 1 -- 172.21.15.22:0/546533105 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b340f03f0 con 0x7f8b38101400 2024-03-08T23:51:49.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.060+0000 7f8b3d35b640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8b240a2800 0x7f8b240a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c005fd0 tx=0x7f8b2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:49.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.064+0000 7f8b2a7fc640 1 -- 172.21.15.22:0/546533105 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8b340bd890 con 0x7f8b38101400 2024-03-08T23:51:49.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:49.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] up:boot 2024-03-08T23:51:49.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: fsmap 12 up:standby 2024-03-08T23:51:49.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T23:51:49.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3140555699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.237 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:49 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3934507523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.248+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b381a0100 con 0x7f8b38101400 2024-03-08T23:51:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.249+0000 7f8b2a7fc640 1 -- 172.21.15.22:0/546533105 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b340bd190 con 0x7f8b38101400 2024-03-08T23:51:49.245 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:51:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8b240a2800 msgr2=0x7f8b240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8b240a2800 0x7f8b240a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c005fd0 tx=0x7f8b2c005850 comp rx=0 tx=0).stop 2024-03-08T23:51:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 msgr2=0x7f8b38197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38197cd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b3402ce70 tx=0x7f8b34005e40 comp rx=0 tx=0).stop 2024-03-08T23:51:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 shutdown_connections 2024-03-08T23:51:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b38101dc0 0x7f8b3819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8b240a2800 0x7f8b240a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b38101400 0x7f8b38197cd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 --2- 172.21.15.22:0/546533105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b38100210 0x7f8b38197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 >> 172.21.15.22:0/546533105 conn(0x7f8b380fb7a0 msgr2=0x7f8b380fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 shutdown_connections 2024-03-08T23:51:49.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.251+0000 7f8b3f5bf640 1 -- 172.21.15.22:0/546533105 wait complete. 2024-03-08T23:51:49.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T23:51:49.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:49.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] up:boot 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: fsmap 12 up:standby 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3140555699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:49 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3934507523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 -- 172.21.15.22:0/3806295010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 msgr2=0x7fc118100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 --2- 172.21.15.22:0/3806295010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118100630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc110009900 tx=0x7fc11002bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 -- 172.21.15.22:0/3806295010 shutdown_connections 2024-03-08T23:51:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 --2- 172.21.15.22:0/3806295010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc118101dc0 0x7fc118108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 --2- 172.21.15.22:0/3806295010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc118101400 0x7fc118101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 --2- 172.21.15.22:0/3806295010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118100630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 -- 172.21.15.22:0/3806295010 >> 172.21.15.22:0/3806295010 conn(0x7fc1180fb780 msgr2=0x7fc1180fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 -- 172.21.15.22:0/3806295010 shutdown_connections 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.368+0000 7fc11d019640 1 -- 172.21.15.22:0/3806295010 wait complete. 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 Processor -- start 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 -- start start 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc118101400 0x7fc11819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc118101dc0 0x7fc11819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11810bea0 con 0x7fc118100210 2024-03-08T23:51:49.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11810bd20 con 0x7fc118101dc0 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11d019640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11810c020 con 0x7fc118101400 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc116d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc116d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118068fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56070/0 (socket says 172.21.15.22:56070) 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc116d9d640 1 -- 172.21.15.22:0/4276256526 learned_addr learned my addr 172.21.15.22:0/4276256526 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11659c640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc118101400 0x7fc11819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.369+0000 7fc11759e640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc118101dc0 0x7fc11819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc116d9d640 1 -- 172.21.15.22:0/4276256526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc118101400 msgr2=0x7fc11819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc116d9d640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc118101400 0x7fc11819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc116d9d640 1 -- 172.21.15.22:0/4276256526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc118101dc0 msgr2=0x7fc11819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc116d9d640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc118101dc0 0x7fc11819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc116d9d640 1 -- 172.21.15.22:0/4276256526 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11819f250 con 0x7fc118100210 2024-03-08T23:51:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc116d9d640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118068fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc110000c00 tx=0x7fc110005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc10ffff640 1 -- 172.21.15.22:0/4276256526 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110041070 con 0x7fc118100210 2024-03-08T23:51:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11819f4e0 con 0x7fc118100210 2024-03-08T23:51:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc10ffff640 1 -- 172.21.15.22:0/4276256526 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1100044d0 con 0x7fc118100210 2024-03-08T23:51:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc10ffff640 1 -- 172.21.15.22:0/4276256526 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110034440 con 0x7fc118100210 2024-03-08T23:51:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.370+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11819f9d0 con 0x7fc118100210 2024-03-08T23:51:49.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.371+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc118107530 con 0x7fc118100210 2024-03-08T23:51:49.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.372+0000 7fc10ffff640 1 -- 172.21.15.22:0/4276256526 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc110044070 con 0x7fc118100210 2024-03-08T23:51:49.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.373+0000 7fc10ffff640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1040a2800 0x7fc1040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.373+0000 7fc11659c640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1040a2800 0x7fc1040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.373+0000 7fc10ffff640 1 -- 172.21.15.22:0/4276256526 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1100f0150 con 0x7fc118100210 2024-03-08T23:51:49.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.373+0000 7fc11659c640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1040a2800 0x7fc1040a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc118068d50 tx=0x7fc100005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:49.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.377+0000 7fc10ffff640 1 -- 172.21.15.22:0/4276256526 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc1100bcfa0 con 0x7fc118100210 2024-03-08T23:51:49.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:49.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] up:boot 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: fsmap 12 up:standby 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3140555699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:49.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:49 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3934507523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:49.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.563+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc118061ef0 con 0x7fc118100210 2024-03-08T23:51:49.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.564+0000 7fc10ffff640 1 -- 172.21.15.22:0/4276256526 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc1100c1040 con 0x7fc118100210 2024-03-08T23:51:49.561 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:49.561 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-03-08T23:51:49.561 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-08T23:51:49.561 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-03-08T23:51:49.561 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-03-08T23:51:49.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:51:49.561 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1040a2800 msgr2=0x7fc1040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1040a2800 0x7fc1040a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc118068d50 tx=0x7fc100005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 msgr2=0x7fc118068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118068fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc110000c00 tx=0x7fc110005d50 comp rx=0 tx=0).stop 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 shutdown_connections 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc118101dc0 0x7fc11819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc118101400 0x7fc11819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1040a2800 0x7fc1040a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 --2- 172.21.15.22:0/4276256526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc118100210 0x7fc118068fe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 >> 172.21.15.22:0/4276256526 conn(0x7fc1180fb780 msgr2=0x7fc118104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:49.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 shutdown_connections 2024-03-08T23:51:49.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.566+0000 7fc11d019640 1 -- 172.21.15.22:0/4276256526 wait complete. 2024-03-08T23:51:49.573 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T23:51:49.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 -- 172.21.15.22:0/2836276657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6481013f0 msgr2=0x7fe648101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 --2- 172.21.15.22:0/2836276657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6481013f0 0x7fe648101870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009900 tx=0x7fe63c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 -- 172.21.15.22:0/2836276657 shutdown_connections 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 --2- 172.21.15.22:0/2836276657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe648101db0 0x7fe648108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 --2- 172.21.15.22:0/2836276657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6481013f0 0x7fe648101870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 --2- 172.21.15.22:0/2836276657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 0x7fe648100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 -- 172.21.15.22:0/2836276657 >> 172.21.15.22:0/2836276657 conn(0x7fe6480fb790 msgr2=0x7fe6480fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 -- 172.21.15.22:0/2836276657 shutdown_connections 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 -- 172.21.15.22:0/2836276657 wait complete. 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 Processor -- start 2024-03-08T23:51:49.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.686+0000 7fe64e3df640 1 -- start start 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe64e3df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 0x7fe64806a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe64e3df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6481013f0 0x7fe64819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe64e3df640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe648101db0 0x7fe6481a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe64e3df640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64810bee0 con 0x7fe648101db0 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe64e3df640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64810bd60 con 0x7fe648100200 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe64e3df640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64810c060 con 0x7fe6481013f0 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe64c97c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe648101db0 0x7fe6481a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe647fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 0x7fe64806a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe6477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6481013f0 0x7fe64819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe647fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 0x7fe64806a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:37222/0 (socket says 172.21.15.22:37222) 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe647fff640 1 -- 172.21.15.22:0/481713776 learned_addr learned my addr 172.21.15.22:0/481713776 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe6477fe640 1 -- 172.21.15.22:0/481713776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 msgr2=0x7fe64806a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe6477fe640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 0x7fe64806a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe6477fe640 1 -- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe648101db0 msgr2=0x7fe6481a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe6477fe640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe648101db0 0x7fe6481a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe6477fe640 1 -- 172.21.15.22:0/481713776 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6481a14c0 con 0x7fe6481013f0 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe647fff640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 0x7fe64806a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.687+0000 7fe6477fe640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6481013f0 0x7fe64819e780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe63c036040 tx=0x7fe63c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.688+0000 7fe6457fa640 1 -- 172.21.15.22:0/481713776 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c043020 con 0x7fe6481013f0 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.688+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6481a16f0 con 0x7fe6481013f0 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.688+0000 7fe6457fa640 1 -- 172.21.15.22:0/481713776 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe63c0054e0 con 0x7fe6481013f0 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.688+0000 7fe6457fa640 1 -- 172.21.15.22:0/481713776 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c003c10 con 0x7fe6481013f0 2024-03-08T23:51:49.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.688+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6481a1c60 con 0x7fe6481013f0 2024-03-08T23:51:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.689+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe648107520 con 0x7fe6481013f0 2024-03-08T23:51:49.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.690+0000 7fe6457fa640 1 -- 172.21.15.22:0/481713776 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe63c046020 con 0x7fe6481013f0 2024-03-08T23:51:49.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.690+0000 7fe6457fa640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6300a2830 0x7fe6300a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:49.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.691+0000 7fe647fff640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6300a2830 0x7fe6300a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:49.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.691+0000 7fe6457fa640 1 -- 172.21.15.22:0/481713776 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe63c0f0c20 con 0x7fe6481013f0 2024-03-08T23:51:49.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.691+0000 7fe647fff640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6300a2830 0x7fe6300a4cf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6380058c0 tx=0x7fe638005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.695+0000 7fe6457fa640 1 -- 172.21.15.22:0/481713776 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe63c04d020 con 0x7fe6481013f0 2024-03-08T23:51:49.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:49.918+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe64802ce70 con 0x7fe6481013f0 2024-03-08T23:51:50.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.225+0000 7fe6457fa640 1 -- 172.21.15.22:0/481713776 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe63c0be0c0 con 0x7fe6481013f0 2024-03-08T23:51:50.222 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-03-08T23:51:50.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6300a2830 msgr2=0x7fe6300a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6300a2830 0x7fe6300a4cf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6380058c0 tx=0x7fe638005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6481013f0 msgr2=0x7fe64819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6481013f0 0x7fe64819e780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe63c036040 tx=0x7fe63c02c650 comp rx=0 tx=0).stop 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 shutdown_connections 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6300a2830 0x7fe6300a4cf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe648101db0 0x7fe6481a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6481013f0 0x7fe64819e780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 --2- 172.21.15.22:0/481713776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648100200 0x7fe64806a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 >> 172.21.15.22:0/481713776 conn(0x7fe6480fb790 msgr2=0x7fe6481046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 shutdown_connections 2024-03-08T23:51:50.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.230+0000 7fe64e3df640 1 -- 172.21.15.22:0/481713776 wait complete. 2024-03-08T23:51:50.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-08T23:51:50.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.353+0000 7f1712797640 1 -- 172.21.15.22:0/2992415747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1001f0 msgr2=0x7f170c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:50.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.353+0000 7f1712797640 1 --2- 172.21.15.22:0/2992415747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1001f0 0x7f170c100610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009900 tx=0x7f16fc02bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:50.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.353+0000 7f1712797640 1 -- 172.21.15.22:0/2992415747 shutdown_connections 2024-03-08T23:51:50.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.353+0000 7f1712797640 1 --2- 172.21.15.22:0/2992415747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f170c101da0 0x7f170c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.353+0000 7f1712797640 1 --2- 172.21.15.22:0/2992415747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f170c1013e0 0x7f170c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.353+0000 7f1712797640 1 --2- 172.21.15.22:0/2992415747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1001f0 0x7f170c100610 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.353+0000 7f1712797640 1 -- 172.21.15.22:0/2992415747 >> 172.21.15.22:0/2992415747 conn(0x7f170c0fb760 msgr2=0x7f170c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 -- 172.21.15.22:0/2992415747 shutdown_connections 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 -- 172.21.15.22:0/2992415747 wait complete. 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 Processor -- start 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 -- start start 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f170c1001f0 0x7f170c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1013e0 0x7f170c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f170c101da0 0x7f170c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c10be80 con 0x7f170c1013e0 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c10bd00 con 0x7f170c101da0 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f1712797640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c10c000 con 0x7f170c1001f0 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.354+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1013e0 0x7f170c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1013e0 0x7f170c197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56114/0 (socket says 172.21.15.22:56114) 2024-03-08T23:51:50.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 -- 172.21.15.22:0/733940097 learned_addr learned my addr 172.21.15.22:0/733940097 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170bfff640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f170c1001f0 0x7f170c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 -- 172.21.15.22:0/733940097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f170c1001f0 msgr2=0x7f170c197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f170c1001f0 0x7f170c197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f1710d34640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f170c101da0 0x7f170c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 -- 172.21.15.22:0/733940097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f170c101da0 msgr2=0x7f170c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f170c101da0 0x7f170c19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 -- 172.21.15.22:0/733940097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c19f250 con 0x7f170c1013e0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f170b7fe640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1013e0 0x7f170c197b70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f170000c970 tx=0x7f170000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f17097fa640 1 -- 172.21.15.22:0/733940097 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700007bb0 con 0x7f170c1013e0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c19f540 con 0x7f170c1013e0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f17097fa640 1 -- 172.21.15.22:0/733940097 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1700007d50 con 0x7f170c1013e0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f17097fa640 1 -- 172.21.15.22:0/733940097 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170000f450 con 0x7f170c1013e0 2024-03-08T23:51:50.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.355+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c19fa40 con 0x7f170c1013e0 2024-03-08T23:51:50.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.356+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c19fd00 con 0x7f170c1013e0 2024-03-08T23:51:50.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.358+0000 7f17097fa640 1 -- 172.21.15.22:0/733940097 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1700004fe0 con 0x7f170c1013e0 2024-03-08T23:51:50.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.358+0000 7f17097fa640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f16f40a2800 0x7f16f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:50.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.359+0000 7f17097fa640 1 -- 172.21.15.22:0/733940097 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f17000ce920 con 0x7f170c1013e0 2024-03-08T23:51:50.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.359+0000 7f170bfff640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f16f40a2800 0x7f16f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:50.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.359+0000 7f170bfff640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f16f40a2800 0x7f16f40a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000c00 tx=0x7f16fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:50.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.362+0000 7f17097fa640 1 -- 172.21.15.22:0/733940097 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1700014030 con 0x7f170c1013e0 2024-03-08T23:51:50.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:50 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/546533105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:50.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:50 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4276256526' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-08T23:51:50.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:50.549+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f170c1a0420 con 0x7f170c1013e0 2024-03-08T23:51:50.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:50 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/546533105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:50.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:50 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4276256526' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-08T23:51:50.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:50 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/546533105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:51:50.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:50 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4276256526' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-08T23:51:51.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.224+0000 7f17097fa640 1 -- 172.21.15.22:0/733940097 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f170009bcf0 con 0x7f170c1013e0 2024-03-08T23:51:51.220 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-03-08T23:51:51.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.231+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f16f40a2800 msgr2=0x7f16f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:51.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.231+0000 7f1712797640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f16f40a2800 0x7f16f40a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000c00 tx=0x7f16fc005fd0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.231+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1013e0 msgr2=0x7f170c197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:51.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.231+0000 7f1712797640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1013e0 0x7f170c197b70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f170000c970 tx=0x7f170000ce30 comp rx=0 tx=0).stop 2024-03-08T23:51:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.234+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 shutdown_connections 2024-03-08T23:51:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.234+0000 7f1712797640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f170c101da0 0x7f170c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.234+0000 7f1712797640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f16f40a2800 0x7f16f40a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.234+0000 7f1712797640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170c1013e0 0x7f170c197b70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.234+0000 7f1712797640 1 --2- 172.21.15.22:0/733940097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f170c1001f0 0x7f170c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.234+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 >> 172.21.15.22:0/733940097 conn(0x7f170c0fb760 msgr2=0x7f170c0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:51.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.239+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 shutdown_connections 2024-03-08T23:51:51.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.240+0000 7f1712797640 1 -- 172.21.15.22:0/733940097 wait complete. 2024-03-08T23:51:51.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 -- 172.21.15.22:0/1207265092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 msgr2=0x7f1480108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 --2- 172.21.15.22:0/1207265092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480108650 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1478009ca0 tx=0x7f147802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 -- 172.21.15.22:0/1207265092 shutdown_connections 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 --2- 172.21.15.22:0/1207265092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480108650 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 --2- 172.21.15.22:0/1207265092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1480101400 0x7f1480101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 --2- 172.21.15.22:0/1207265092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480100210 0x7f1480100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 -- 172.21.15.22:0/1207265092 >> 172.21.15.22:0/1207265092 conn(0x7f14800fb7a0 msgr2=0x7f14800fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 -- 172.21.15.22:0/1207265092 shutdown_connections 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.364+0000 7f14858e9640 1 -- 172.21.15.22:0/1207265092 wait complete. 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 Processor -- start 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 -- start start 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1480100210 0x7f1480104210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101400 0x7f1480102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148010bea0 con 0x7f1480101dc0 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148010bd20 con 0x7f1480101400 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f14858e9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148010c020 con 0x7f1480100210 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56262/0 (socket says 172.21.15.22:56262) 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 -- 172.21.15.22:0/3325723033 learned_addr learned my addr 172.21.15.22:0/3325723033 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147e7fc640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101400 0x7f1480102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147effd640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1480100210 0x7f1480104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 -- 172.21.15.22:0/3325723033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1480100210 msgr2=0x7f1480104210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1480100210 0x7f1480104210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 -- 172.21.15.22:0/3325723033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101400 msgr2=0x7f1480102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101400 0x7f1480102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.365+0000 7f147f7fe640 1 -- 172.21.15.22:0/3325723033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480103620 con 0x7f1480101dc0 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.366+0000 7f147f7fe640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480102d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f147802c630 tx=0x7f1478032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.366+0000 7f14848e7640 1 -- 172.21.15.22:0/3325723033 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478041020 con 0x7f1480101dc0 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.366+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14801a1660 con 0x7f1480101dc0 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.366+0000 7f14848e7640 1 -- 172.21.15.22:0/3325723033 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1478034cd0 con 0x7f1480101dc0 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.366+0000 7f14848e7640 1 -- 172.21.15.22:0/3325723033 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478039040 con 0x7f1480101dc0 2024-03-08T23:51:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.366+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14801a1bb0 con 0x7f1480101dc0 2024-03-08T23:51:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.367+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14801a1e10 con 0x7f1480101dc0 2024-03-08T23:51:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.368+0000 7f14848e7640 1 -- 172.21.15.22:0/3325723033 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1478044020 con 0x7f1480101dc0 2024-03-08T23:51:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.369+0000 7f14848e7640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f14680a2800 0x7f14680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.369+0000 7f147effd640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f14680a2800 0x7f14680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.369+0000 7f14848e7640 1 -- 172.21.15.22:0/3325723033 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f14780f0bc0 con 0x7f1480101dc0 2024-03-08T23:51:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.370+0000 7f147effd640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f14680a2800 0x7f14680a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1480101260 tx=0x7f1470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:51.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.373+0000 7f14848e7640 1 -- 172.21.15.22:0/3325723033 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f147804b020 con 0x7f1480101dc0 2024-03-08T23:51:51.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/481713776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: osdmap e82: 12 total, 12 up, 12 in 2024-03-08T23:51:51.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/733940097' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-08T23:51:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:51 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:51.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:51.608+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f14801a2080 con 0x7f1480101dc0 2024-03-08T23:51:51.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/481713776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: osdmap e82: 12 total, 12 up, 12 in 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/733940097' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:51.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:51 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:51.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:51.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/481713776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T23:51:51.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T23:51:51.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: osdmap e82: 12 total, 12 up, 12 in 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/733940097' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:51.636 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:51 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:51:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2024-03-08T23:51:52.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.234+0000 7f14848e7640 1 -- 172.21.15.22:0/3325723033 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f14780bde10 con 0x7f1480101dc0 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.238+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f14680a2800 msgr2=0x7f14680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.238+0000 7f14858e9640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f14680a2800 0x7f14680a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1480101260 tx=0x7f1470005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.238+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 msgr2=0x7f1480102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.238+0000 7f14858e9640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480102d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f147802c630 tx=0x7f1478032ea0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 shutdown_connections 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f14680a2800 0x7f14680a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480101dc0 0x7f1480102d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101400 0x7f1480102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 --2- 172.21.15.22:0/3325723033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1480100210 0x7f1480104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 >> 172.21.15.22:0/3325723033 conn(0x7f14800fb7a0 msgr2=0x7f14800698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 shutdown_connections 2024-03-08T23:51:52.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.239+0000 7f14858e9640 1 -- 172.21.15.22:0/3325723033 wait complete. 2024-03-08T23:51:52.255 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-08T23:51:52.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 -- 172.21.15.22:0/4021494706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 msgr2=0x7fbcc4101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 --2- 172.21.15.22:0/4021494706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4101860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009900 tx=0x7fbcb802bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 -- 172.21.15.22:0/4021494706 shutdown_connections 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 --2- 172.21.15.22:0/4021494706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc4101da0 0x7fbcc4108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 --2- 172.21.15.22:0/4021494706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4101860 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 --2- 172.21.15.22:0/4021494706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc41001f0 0x7fbcc4100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 -- 172.21.15.22:0/4021494706 >> 172.21.15.22:0/4021494706 conn(0x7fbcc40fb780 msgr2=0x7fbcc40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 -- 172.21.15.22:0/4021494706 shutdown_connections 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.379+0000 7fbcc9080640 1 -- 172.21.15.22:0/4021494706 wait complete. 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 Processor -- start 2024-03-08T23:51:52.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 -- start start 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc41001f0 0x7fbcc4196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4101da0 0x7fbcc419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc410be80 con 0x7fbcc41013e0 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc410bd00 con 0x7fbcc41001f0 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.380+0000 7fbcc9080640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc410c000 con 0x7fbcc4101da0 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56282/0 (socket says 172.21.15.22:56282) 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 -- 172.21.15.22:0/2121515948 learned_addr learned my addr 172.21.15.22:0/2121515948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc359e640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4101da0 0x7fbcc419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc2d9d640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc41001f0 0x7fbcc4196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 -- 172.21.15.22:0/2121515948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4101da0 msgr2=0x7fbcc419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4101da0 0x7fbcc419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 -- 172.21.15.22:0/2121515948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc41001f0 msgr2=0x7fbcc4196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc41001f0 0x7fbcc4196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 -- 172.21.15.22:0/2121515948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc419e130 con 0x7fbcc41013e0 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc2d9d640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc41001f0 0x7fbcc4196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc359e640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4101da0 0x7fbcc419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc259c640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4196a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009230 tx=0x7fbcb802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcabfff640 1 -- 172.21.15.22:0/2121515948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8041070 con 0x7fbcc41013e0 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc419e3c0 con 0x7fbcc41013e0 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcabfff640 1 -- 172.21.15.22:0/2121515948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb8004450 con 0x7fbcc41013e0 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc419e900 con 0x7fbcc41013e0 2024-03-08T23:51:52.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.381+0000 7fbcabfff640 1 -- 172.21.15.22:0/2121515948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8003e20 con 0x7fbcc41013e0 2024-03-08T23:51:52.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.383+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc41075b0 con 0x7fbcc41013e0 2024-03-08T23:51:52.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.384+0000 7fbcabfff640 1 -- 172.21.15.22:0/2121515948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbcb8044070 con 0x7fbcc41013e0 2024-03-08T23:51:52.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.385+0000 7fbcabfff640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbcac0a2880 0x7fbcac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:52.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.385+0000 7fbcc2d9d640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbcac0a2880 0x7fbcac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:52.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.385+0000 7fbcabfff640 1 -- 172.21.15.22:0/2121515948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbcb80f0560 con 0x7fbcc41013e0 2024-03-08T23:51:52.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.386+0000 7fbcc2d9d640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbcac0a2880 0x7fbcac0a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4101240 tx=0x7fbcb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:52.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.389+0000 7fbcabfff640 1 -- 172.21.15.22:0/2121515948 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fbcb80bd060 con 0x7fbcc41013e0 2024-03-08T23:51:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:52 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/733940097' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-08T23:51:52.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:52 smithi022 ceph-mon[31894]: osdmap e83: 12 total, 12 up, 12 in 2024-03-08T23:51:52.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:52 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3325723033' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-08T23:51:52.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:52 smithi022 ceph-mon[31894]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-08T23:51:52.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:52.594+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fbcc4061ef0 con 0x7fbcc41013e0 2024-03-08T23:51:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:52 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/733940097' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-08T23:51:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:52 smithi062 ceph-mon[36129]: osdmap e83: 12 total, 12 up, 12 in 2024-03-08T23:51:52.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:52 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3325723033' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-08T23:51:52.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:52 smithi062 ceph-mon[36129]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-08T23:51:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:52 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/733940097' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-08T23:51:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:52 smithi192 ceph-mon[35995]: osdmap e83: 12 total, 12 up, 12 in 2024-03-08T23:51:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:52 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3325723033' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-08T23:51:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:52 smithi192 ceph-mon[35995]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-08T23:51:53.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.227+0000 7fbcabfff640 1 -- 172.21.15.22:0/2121515948 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7fbcb80c1100 con 0x7fbcc41013e0 2024-03-08T23:51:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbcac0a2880 msgr2=0x7fbcac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbcac0a2880 0x7fbcac0a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4101240 tx=0x7fbcb4005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 msgr2=0x7fbcc4196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4196a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009230 tx=0x7fbcb802c650 comp rx=0 tx=0).stop 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 shutdown_connections 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4101da0 0x7fbcc419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbcac0a2880 0x7fbcac0a4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41013e0 0x7fbcc4196a50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 --2- 172.21.15.22:0/2121515948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc41001f0 0x7fbcc4196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 >> 172.21.15.22:0/2121515948 conn(0x7fbcc40fb780 msgr2=0x7fbcc4104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 shutdown_connections 2024-03-08T23:51:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.230+0000 7fbcc9080640 1 -- 172.21.15.22:0/2121515948 wait complete. 2024-03-08T23:51:53.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T23:51:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.369+0000 7ff3b8414640 1 -- 172.21.15.22:0/4232554721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 msgr2=0x7ff3b0101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.369+0000 7ff3b8414640 1 --2- 172.21.15.22:0/4232554721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0101880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009900 tx=0x7ff3a402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.369+0000 7ff3b8414640 1 -- 172.21.15.22:0/4232554721 shutdown_connections 2024-03-08T23:51:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.369+0000 7ff3b8414640 1 --2- 172.21.15.22:0/4232554721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b0101dc0 0x7ff3b0108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.369+0000 7ff3b8414640 1 --2- 172.21.15.22:0/4232554721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0101880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.369+0000 7ff3b8414640 1 --2- 172.21.15.22:0/4232554721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3b0100210 0x7ff3b0100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.369+0000 7ff3b8414640 1 -- 172.21.15.22:0/4232554721 >> 172.21.15.22:0/4232554721 conn(0x7ff3b00fb780 msgr2=0x7ff3b00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 -- 172.21.15.22:0/4232554721 shutdown_connections 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 -- 172.21.15.22:0/4232554721 wait complete. 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 Processor -- start 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 -- start start 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3b0100210 0x7ff3b0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b0101dc0 0x7ff3b019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b010bea0 con 0x7ff3b0101400 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b010bd20 con 0x7ff3b0101dc0 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b8414640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b010c020 con 0x7ff3b0100210 2024-03-08T23:51:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b59af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b59af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56302/0 (socket says 172.21.15.22:56302) 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b59af640 1 -- 172.21.15.22:0/1292892759 learned_addr learned my addr 172.21.15.22:0/1292892759 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.370+0000 7ff3b69b1640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b0101dc0 0x7ff3b019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b61b0640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3b0100210 0x7ff3b0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b59af640 1 -- 172.21.15.22:0/1292892759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3b0100210 msgr2=0x7ff3b0197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b59af640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3b0100210 0x7ff3b0197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b59af640 1 -- 172.21.15.22:0/1292892759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b0101dc0 msgr2=0x7ff3b019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b59af640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b0101dc0 0x7ff3b019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b59af640 1 -- 172.21.15.22:0/1292892759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b019f250 con 0x7ff3b0101400 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b59af640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0197b70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009230 tx=0x7ff3a4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3af7fe640 1 -- 172.21.15.22:0/1292892759 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4041070 con 0x7ff3b0101400 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b019f4e0 con 0x7ff3b0101400 2024-03-08T23:51:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3af7fe640 1 -- 172.21.15.22:0/1292892759 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3a40044d0 con 0x7ff3b0101400 2024-03-08T23:51:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3af7fe640 1 -- 172.21.15.22:0/1292892759 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4034440 con 0x7ff3b0101400 2024-03-08T23:51:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.371+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b019fa20 con 0x7ff3b0101400 2024-03-08T23:51:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.372+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b019fc80 con 0x7ff3b0101400 2024-03-08T23:51:53.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.373+0000 7ff3af7fe640 1 -- 172.21.15.22:0/1292892759 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff3a4044070 con 0x7ff3b0101400 2024-03-08T23:51:53.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.374+0000 7ff3af7fe640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff3980a2800 0x7ff3980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:53.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.374+0000 7ff3b61b0640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff3980a2800 0x7ff3980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:53.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.374+0000 7ff3af7fe640 1 -- 172.21.15.22:0/1292892759 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8277+0+0 (secure 0 0 0) 0x7ff3a40f0770 con 0x7ff3b0101400 2024-03-08T23:51:53.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.374+0000 7ff3b61b0640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff3980a2800 0x7ff3980a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8005d00 tx=0x7ff3a8005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.378+0000 7ff3af7fe640 1 -- 172.21.15.22:0/1292892759 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff3a40bd780 con 0x7ff3b0101400 2024-03-08T23:51:53.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:53.561+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff3b019fe50 con 0x7ff3b0101400 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: pgmap v287: 97 pgs: 12 creating+peering, 7 active+clean, 78 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: osdmap e84: 12 total, 12 up, 12 in 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3325723033' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2121515948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2121515948' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-08T23:51:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:53 smithi062 ceph-mon[36129]: osdmap e85: 12 total, 12 up, 12 in 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: pgmap v287: 97 pgs: 12 creating+peering, 7 active+clean, 78 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: osdmap e84: 12 total, 12 up, 12 in 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3325723033' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2121515948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:53.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:53.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2121515948' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-08T23:51:53.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:53 smithi192 ceph-mon[35995]: osdmap e85: 12 total, 12 up, 12 in 2024-03-08T23:51:53.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: pgmap v287: 97 pgs: 12 creating+peering, 7 active+clean, 78 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: osdmap e84: 12 total, 12 up, 12 in 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3325723033' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2121515948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:51:53.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:51:53.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:51:53.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2121515948' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-08T23:51:53.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:53 smithi022 ceph-mon[31894]: osdmap e85: 12 total, 12 up, 12 in 2024-03-08T23:51:54.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:54 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] up:active 2024-03-08T23:51:54.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:54 smithi062 ceph-mon[36129]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:54.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:54 smithi062 ceph-mon[36129]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:54.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:54 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1292892759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-08T23:51:54.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:54 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] up:active 2024-03-08T23:51:54.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:54 smithi192 ceph-mon[35995]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:54.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:54 smithi192 ceph-mon[35995]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:54.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:54 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1292892759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-08T23:51:54.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:54 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] up:active 2024-03-08T23:51:54.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:54 smithi022 ceph-mon[31894]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:54.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:54 smithi022 ceph-mon[31894]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:54.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:54 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1292892759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-08T23:51:55.244 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:51:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: 2024-03-08T23:51:54.967+0000 7f61e175e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T23:51:55.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.252+0000 7ff3af7fe640 1 -- 172.21.15.22:0/1292892759 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff3a40bd080 con 0x7ff3b0101400 2024-03-08T23:51:55.249 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' created 2024-03-08T23:51:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff3980a2800 msgr2=0x7ff3980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff3980a2800 0x7ff3980a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8005d00 tx=0x7ff3a8005c50 comp rx=0 tx=0).stop 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 msgr2=0x7ff3b0197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0197b70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009230 tx=0x7ff3a4005d50 comp rx=0 tx=0).stop 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 shutdown_connections 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b0101dc0 0x7ff3b019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff3980a2800 0x7ff3980a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3b0101400 0x7ff3b0197b70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 --2- 172.21.15.22:0/1292892759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3b0100210 0x7ff3b0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 >> 172.21.15.22:0/1292892759 conn(0x7ff3b00fb780 msgr2=0x7ff3b0068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 shutdown_connections 2024-03-08T23:51:55.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.256+0000 7ff3b8414640 1 -- 172.21.15.22:0/1292892759 wait complete. 2024-03-08T23:51:55.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-08T23:51:55.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.386+0000 7f5ae6b89640 1 -- 172.21.15.22:0/1656398884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 msgr2=0x7f5ae0108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:55.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.386+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/1656398884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae0108630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009ca0 tx=0x7f5ad802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.386+0000 7f5ae6b89640 1 -- 172.21.15.22:0/1656398884 shutdown_connections 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/1656398884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae0108630 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/1656398884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae01013e0 0x7f5ae0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/1656398884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ae01001f0 0x7f5ae0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 -- 172.21.15.22:0/1656398884 >> 172.21.15.22:0/1656398884 conn(0x7f5ae00fb780 msgr2=0x7f5ae00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 -- 172.21.15.22:0/1656398884 shutdown_connections 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 -- 172.21.15.22:0/1656398884 wait complete. 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 Processor -- start 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 -- start start 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ae01001f0 0x7f5ae010ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae01013e0 0x7f5ae01091b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae01096f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae010cff0 con 0x7f5ae0101da0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae010ce70 con 0x7f5ae01013e0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae6b89640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae010d170 con 0x7f5ae01001f0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae5126640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae01096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae5126640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae01096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56318/0 (socket says 172.21.15.22:56318) 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.387+0000 7f5ae5126640 1 -- 172.21.15.22:0/2494296905 learned_addr learned my addr 172.21.15.22:0/2494296905 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae4925640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ae01001f0 0x7f5ae010ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae5126640 1 -- 172.21.15.22:0/2494296905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ae01001f0 msgr2=0x7f5ae010ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae5126640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ae01001f0 0x7f5ae010ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae5126640 1 -- 172.21.15.22:0/2494296905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae01013e0 msgr2=0x7f5ae01091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5adffff640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae01013e0 0x7f5ae01091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae5126640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae01013e0 0x7f5ae01091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae5126640 1 -- 172.21.15.22:0/2494296905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0109f80 con 0x7f5ae0101da0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae5126640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae01096f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8037040 tx=0x7f5ad8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5addffb640 1 -- 172.21.15.22:0/2494296905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8039040 con 0x7f5ae0101da0 2024-03-08T23:51:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5addffb640 1 -- 172.21.15.22:0/2494296905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ad8044070 con 0x7f5ae0101da0 2024-03-08T23:51:55.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5addffb640 1 -- 172.21.15.22:0/2494296905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8004330 con 0x7f5ae0101da0 2024-03-08T23:51:55.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae01a1700 con 0x7f5ae0101da0 2024-03-08T23:51:55.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.388+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae01a1c50 con 0x7f5ae0101da0 2024-03-08T23:51:55.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.389+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae0107510 con 0x7f5ae0101da0 2024-03-08T23:51:55.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.391+0000 7f5addffb640 1 -- 172.21.15.22:0/2494296905 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5ad8003a00 con 0x7f5ae0101da0 2024-03-08T23:51:55.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.392+0000 7f5addffb640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5ad00a2800 0x7f5ad00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:55.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.392+0000 7f5ae4925640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5ad00a2800 0x7f5ad00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:55.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.392+0000 7f5addffb640 1 -- 172.21.15.22:0/2494296905 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f5ad80f1c40 con 0x7f5ae0101da0 2024-03-08T23:51:55.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.392+0000 7f5ae4925640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5ad00a2800 0x7f5ad00a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0101240 tx=0x7f5ad4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:55.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.395+0000 7f5addffb640 1 -- 172.21.15.22:0/2494296905 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5ad80bea60 con 0x7f5ae0101da0 2024-03-08T23:51:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:55.577+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f5ae0061ef0 con 0x7f5ae0101da0 2024-03-08T23:51:55.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:55 smithi062 ceph-mon[36129]: pgmap v290: 97 pgs: 20 creating+peering, 31 active+clean, 46 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-08T23:51:55.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:55 smithi062 ceph-mon[36129]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-08T23:51:55.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:55 smithi062 ceph-mon[36129]: Cluster is now healthy 2024-03-08T23:51:55.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:55 smithi062 ceph-mon[36129]: osdmap e86: 12 total, 12 up, 12 in 2024-03-08T23:51:55.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:55 smithi192 ceph-mon[35995]: pgmap v290: 97 pgs: 20 creating+peering, 31 active+clean, 46 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-08T23:51:55.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:55 smithi192 ceph-mon[35995]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-08T23:51:55.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:55 smithi192 ceph-mon[35995]: Cluster is now healthy 2024-03-08T23:51:55.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:55 smithi192 ceph-mon[35995]: osdmap e86: 12 total, 12 up, 12 in 2024-03-08T23:51:55.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:55 smithi022 ceph-mon[31894]: pgmap v290: 97 pgs: 20 creating+peering, 31 active+clean, 46 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-08T23:51:55.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:55 smithi022 ceph-mon[31894]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-08T23:51:55.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:55 smithi022 ceph-mon[31894]: Cluster is now healthy 2024-03-08T23:51:55.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:55 smithi022 ceph-mon[31894]: osdmap e86: 12 total, 12 up, 12 in 2024-03-08T23:51:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.258+0000 7f5addffb640 1 -- 172.21.15.22:0/2494296905 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5ad80be360 con 0x7f5ae0101da0 2024-03-08T23:51:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 4 allow_ec_overwrites to true 2024-03-08T23:51:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5ad00a2800 msgr2=0x7f5ad00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5ad00a2800 0x7f5ad00a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0101240 tx=0x7f5ad4008040 comp rx=0 tx=0).stop 2024-03-08T23:51:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 msgr2=0x7f5ae01096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae01096f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8037040 tx=0x7f5ad8032f90 comp rx=0 tx=0).stop 2024-03-08T23:51:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 shutdown_connections 2024-03-08T23:51:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5ad00a2800 0x7f5ad00a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ae0101da0 0x7f5ae01096f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae01013e0 0x7f5ae01091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 --2- 172.21.15.22:0/2494296905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ae01001f0 0x7f5ae010ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.262+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 >> 172.21.15.22:0/2494296905 conn(0x7f5ae00fb780 msgr2=0x7f5ae01046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.263+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 shutdown_connections 2024-03-08T23:51:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.263+0000 7f5ae6b89640 1 -- 172.21.15.22:0/2494296905 wait complete. 2024-03-08T23:51:56.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-08T23:51:56.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.396+0000 7f6d27812640 1 -- 172.21.15.22:0/1183462880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d20101d70 msgr2=0x7f6d20104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.396+0000 7f6d27812640 1 --2- 172.21.15.22:0/1183462880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d20101d70 0x7f6d20104160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009ca0 tx=0x7f6d1802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.396+0000 7f6d27812640 1 -- 172.21.15.22:0/1183462880 shutdown_connections 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.396+0000 7f6d27812640 1 --2- 172.21.15.22:0/1183462880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d20106fd0 0x7f6d20109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.396+0000 7f6d27812640 1 --2- 172.21.15.22:0/1183462880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d201046a0 0x7f6d20106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.396+0000 7f6d27812640 1 --2- 172.21.15.22:0/1183462880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d20101d70 0x7f6d20104160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.396+0000 7f6d27812640 1 -- 172.21.15.22:0/1183462880 >> 172.21.15.22:0/1183462880 conn(0x7f6d200fb780 msgr2=0x7f6d200fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 -- 172.21.15.22:0/1183462880 shutdown_connections 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 -- 172.21.15.22:0/1183462880 wait complete. 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 Processor -- start 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 -- start start 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d20101d70 0x7f6d20197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d201046a0 0x7f6d20197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:56.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d20106fd0 0x7f6d2019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2010cf80 con 0x7f6d201046a0 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2010ce00 con 0x7f6d20101d70 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d27812640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2010d100 con 0x7f6d20106fd0 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d24dad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d201046a0 0x7f6d20197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d24dad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d201046a0 0x7f6d20197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56332/0 (socket says 172.21.15.22:56332) 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d24dad640 1 -- 172.21.15.22:0/2656547445 learned_addr learned my addr 172.21.15.22:0/2656547445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.397+0000 7f6d25daf640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d20106fd0 0x7f6d2019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d255ae640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d20101d70 0x7f6d20197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d24dad640 1 -- 172.21.15.22:0/2656547445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d20106fd0 msgr2=0x7f6d2019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d24dad640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d20106fd0 0x7f6d2019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d24dad640 1 -- 172.21.15.22:0/2656547445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d20101d70 msgr2=0x7f6d20197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d24dad640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d20101d70 0x7f6d20197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d24dad640 1 -- 172.21.15.22:0/2656547445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2019f2b0 con 0x7f6d201046a0 2024-03-08T23:51:56.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d24dad640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d201046a0 0x7f6d20197bd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e9f0 tx=0x7f6d1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:56.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d1e7fc640 1 -- 172.21.15.22:0/2656547445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400ccc0 con 0x7f6d201046a0 2024-03-08T23:51:56.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2019f5a0 con 0x7f6d201046a0 2024-03-08T23:51:56.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d1e7fc640 1 -- 172.21.15.22:0/2656547445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d14005500 con 0x7f6d201046a0 2024-03-08T23:51:56.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d1e7fc640 1 -- 172.21.15.22:0/2656547445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14010690 con 0x7f6d201046a0 2024-03-08T23:51:56.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.398+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2019faa0 con 0x7f6d201046a0 2024-03-08T23:51:56.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.399+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d200fd270 con 0x7f6d201046a0 2024-03-08T23:51:56.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.400+0000 7f6d1e7fc640 1 -- 172.21.15.22:0/2656547445 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6d140046c0 con 0x7f6d201046a0 2024-03-08T23:51:56.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.401+0000 7f6d1e7fc640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6d080a2800 0x7f6d080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:56.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.401+0000 7f6d255ae640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6d080a2800 0x7f6d080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:56.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.401+0000 7f6d1e7fc640 1 -- 172.21.15.22:0/2656547445 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f6d140cf1d0 con 0x7f6d201046a0 2024-03-08T23:51:56.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.401+0000 7f6d255ae640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6d080a2800 0x7f6d080a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6d18006010 tx=0x7f6d18032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:56.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.404+0000 7f6d1e7fc640 1 -- 172.21.15.22:0/2656547445 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6d14053260 con 0x7f6d201046a0 2024-03-08T23:51:56.513 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:56 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1292892759' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-08T23:51:56.513 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:56 smithi062 ceph-mon[36129]: osdmap e87: 12 total, 12 up, 12 in 2024-03-08T23:51:56.513 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:56 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2494296905' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-08T23:51:56.526 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:56 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1292892759' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-08T23:51:56.527 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:56 smithi192 ceph-mon[35995]: osdmap e87: 12 total, 12 up, 12 in 2024-03-08T23:51:56.527 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:56 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2494296905' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-08T23:51:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:56.627+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f6d2002ce70 con 0x7f6d201046a0 2024-03-08T23:51:56.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:56 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1292892759' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-08T23:51:56.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:56 smithi022 ceph-mon[31894]: osdmap e87: 12 total, 12 up, 12 in 2024-03-08T23:51:56.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:56 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2494296905' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-08T23:51:57.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.260+0000 7f6d1e7fc640 1 -- 172.21.15.22:0/2656547445 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f6d1409c0f0 con 0x7f6d201046a0 2024-03-08T23:51:57.257 INFO:teuthology.orchestra.run.smithi022.stderr:added data pool 4 to fsmap 2024-03-08T23:51:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6d080a2800 msgr2=0x7f6d080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6d080a2800 0x7f6d080a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6d18006010 tx=0x7f6d18032590 comp rx=0 tx=0).stop 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d201046a0 msgr2=0x7f6d20197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d201046a0 0x7f6d20197bd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e9f0 tx=0x7f6d1400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 shutdown_connections 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d20106fd0 0x7f6d2019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6d080a2800 0x7f6d080a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d201046a0 0x7f6d20197bd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 --2- 172.21.15.22:0/2656547445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d20101d70 0x7f6d20197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 >> 172.21.15.22:0/2656547445 conn(0x7f6d200fb780 msgr2=0x7f6d20102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:57.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 shutdown_connections 2024-03-08T23:51:57.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.263+0000 7f6d27812640 1 -- 172.21.15.22:0/2656547445 wait complete. 2024-03-08T23:51:57.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 -- 172.21.15.22:0/1023113296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538100230 msgr2=0x7f4538100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 --2- 172.21.15.22:0/1023113296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538100230 0x7f4538100650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 -- 172.21.15.22:0/1023113296 shutdown_connections 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 --2- 172.21.15.22:0/1023113296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4538101de0 0x7f4538108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 --2- 172.21.15.22:0/1023113296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4538101420 0x7f45381018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 --2- 172.21.15.22:0/1023113296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538100230 0x7f4538100650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 -- 172.21.15.22:0/1023113296 >> 172.21.15.22:0/1023113296 conn(0x7f45380fb7a0 msgr2=0x7f45380fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 -- 172.21.15.22:0/1023113296 shutdown_connections 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 -- 172.21.15.22:0/1023113296 wait complete. 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 Processor -- start 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.386+0000 7f453e29a640 1 -- start start 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4538100230 0x7f4538197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538101420 0x7f4538197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4538101de0 0x7f453819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453810bec0 con 0x7f4538101420 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453810bd40 con 0x7f4538100230 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453810c040 con 0x7f4538101de0 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538101420 0x7f4538197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538101420 0x7f4538197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56356/0 (socket says 172.21.15.22:56356) 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 -- 172.21.15.22:0/284478175 learned_addr learned my addr 172.21.15.22:0/284478175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453c837640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4538101de0 0x7f453819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f4537fff640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4538100230 0x7f4538197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 -- 172.21.15.22:0/284478175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4538101de0 msgr2=0x7f453819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4538101de0 0x7f453819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 -- 172.21.15.22:0/284478175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4538100230 msgr2=0x7f4538197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4538100230 0x7f4538197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 -- 172.21.15.22:0/284478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453819f290 con 0x7f4538101420 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45377fe640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538101420 0x7f4538197bb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f452c00e9f0 tx=0x7f452c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:57.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45357fa640 1 -- 172.21.15.22:0/284478175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00cd80 con 0x7f4538101420 2024-03-08T23:51:57.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453819f580 con 0x7f4538101420 2024-03-08T23:51:57.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45357fa640 1 -- 172.21.15.22:0/284478175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f452c005500 con 0x7f4538101420 2024-03-08T23:51:57.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f45357fa640 1 -- 172.21.15.22:0/284478175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c010690 con 0x7f4538101420 2024-03-08T23:51:57.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.387+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453819fad0 con 0x7f4538101420 2024-03-08T23:51:57.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.389+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453819fd90 con 0x7f4538101420 2024-03-08T23:51:57.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.389+0000 7f45357fa640 1 -- 172.21.15.22:0/284478175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f452c0046c0 con 0x7f4538101420 2024-03-08T23:51:57.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.390+0000 7f45357fa640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f45200a2800 0x7f45200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.390+0000 7f4537fff640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f45200a2800 0x7f45200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.390+0000 7f45357fa640 1 -- 172.21.15.22:0/284478175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f452c0cf140 con 0x7f4538101420 2024-03-08T23:51:57.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.390+0000 7f4537fff640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f45200a2800 0x7f45200a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f452802c7b0 tx=0x7f4528005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:57.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.395+0000 7f45357fa640 1 -- 172.21.15.22:0/284478175 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f452c09bfa0 con 0x7f4538101420 2024-03-08T23:51:57.526 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:57 smithi062 ceph-mon[36129]: pgmap v293: 129 pgs: 23 creating+peering, 73 active+clean, 33 unknown; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-08T23:51:57.526 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:57 smithi062 ceph-mon[36129]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T23:51:57.527 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:57 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2494296905' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-08T23:51:57.527 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:57 smithi062 ceph-mon[36129]: osdmap e88: 12 total, 12 up, 12 in 2024-03-08T23:51:57.527 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:57 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2656547445' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-08T23:51:57.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.613+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f453819ff60 con 0x7f4538101420 2024-03-08T23:51:57.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.614+0000 7f45357fa640 1 -- 172.21.15.22:0/284478175 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11283 (secure 0 0 0) 0x7f452c09b8a0 con 0x7f4538101420 2024-03-08T23:51:57.611 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:57.611 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:51:56.628883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-08T23:51:57.611 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-03-08T23:51:57.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f45200a2800 msgr2=0x7f45200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f45200a2800 0x7f45200a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f452802c7b0 tx=0x7f4528005fd0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538101420 msgr2=0x7f4538197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538101420 0x7f4538197bb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f452c00e9f0 tx=0x7f452c00edf0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 shutdown_connections 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4538101de0 0x7f453819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f45200a2800 0x7f45200a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4538101420 0x7f4538197bb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 --2- 172.21.15.22:0/284478175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4538100230 0x7f4538197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 >> 172.21.15.22:0/284478175 conn(0x7f45380fb7a0 msgr2=0x7f45380fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 shutdown_connections 2024-03-08T23:51:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.616+0000 7f453e29a640 1 -- 172.21.15.22:0/284478175 wait complete. 2024-03-08T23:51:57.622 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:57 smithi022 ceph-mon[31894]: pgmap v293: 129 pgs: 23 creating+peering, 73 active+clean, 33 unknown; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-08T23:51:57.622 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:57 smithi022 ceph-mon[31894]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T23:51:57.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:57 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2494296905' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-08T23:51:57.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:57 smithi022 ceph-mon[31894]: osdmap e88: 12 total, 12 up, 12 in 2024-03-08T23:51:57.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:57 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2656547445' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-08T23:51:57.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:51:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:57 smithi192 ceph-mon[35995]: pgmap v293: 129 pgs: 23 creating+peering, 73 active+clean, 33 unknown; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-08T23:51:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:57 smithi192 ceph-mon[35995]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T23:51:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:57 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2494296905' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-08T23:51:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:57 smithi192 ceph-mon[35995]: osdmap e88: 12 total, 12 up, 12 in 2024-03-08T23:51:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:57 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2656547445' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-08T23:51:57.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 -- 172.21.15.22:0/2937416759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 msgr2=0x7f2038109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 --2- 172.21.15.22:0/2937416759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038109750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2030009ca0 tx=0x7f203002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 -- 172.21.15.22:0/2937416759 shutdown_connections 2024-03-08T23:51:57.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 --2- 172.21.15.22:0/2937416759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038109750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 --2- 172.21.15.22:0/2937416759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2038102500 0x7f2038102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 --2- 172.21.15.22:0/2937416759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101310 0x7f2038101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 -- 172.21.15.22:0/2937416759 >> 172.21.15.22:0/2937416759 conn(0x7f20380fc8a0 msgr2=0x7f20380fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:57.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 -- 172.21.15.22:0/2937416759 shutdown_connections 2024-03-08T23:51:57.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.730+0000 7f203ce43640 1 -- 172.21.15.22:0/2937416759 wait complete. 2024-03-08T23:51:57.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 Processor -- start 2024-03-08T23:51:57.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 -- start start 2024-03-08T23:51:57.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101310 0x7f20380750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2038102500 0x7f20380736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203810cfa0 con 0x7f2038102ec0 2024-03-08T23:51:57.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203810ce20 con 0x7f2038101310 2024-03-08T23:51:57.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203ce43640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203810d120 con 0x7f2038102500 2024-03-08T23:51:57.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56382/0 (socket says 172.21.15.22:56382) 2024-03-08T23:51:57.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 -- 172.21.15.22:0/2826191597 learned_addr learned my addr 172.21.15.22:0/2826191597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:57.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f203659c640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101310 0x7f20380750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f202ffff640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2038102500 0x7f20380736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 -- 172.21.15.22:0/2826191597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2038102500 msgr2=0x7f20380736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2038102500 0x7f20380736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 -- 172.21.15.22:0/2826191597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101310 msgr2=0x7f20380750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101310 0x7f20380750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 -- 172.21.15.22:0/2826191597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380744b0 con 0x7f2038102ec0 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.731+0000 7f2036d9d640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038073c20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f203002c700 tx=0x7f2030032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.732+0000 7f202f7fe640 1 -- 172.21.15.22:0/2826191597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030039040 con 0x7f2038102ec0 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.732+0000 7f202f7fe640 1 -- 172.21.15.22:0/2826191597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2030044070 con 0x7f2038102ec0 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.732+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20381a1640 con 0x7f2038102ec0 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.732+0000 7f202f7fe640 1 -- 172.21.15.22:0/2826191597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20300042a0 con 0x7f2038102ec0 2024-03-08T23:51:57.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.732+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20381a1b90 con 0x7f2038102ec0 2024-03-08T23:51:57.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.733+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2038108650 con 0x7f2038102ec0 2024-03-08T23:51:57.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.734+0000 7f202f7fe640 1 -- 172.21.15.22:0/2826191597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2030003970 con 0x7f2038102ec0 2024-03-08T23:51:57.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.734+0000 7f202f7fe640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20240a2800 0x7f20240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:57.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.734+0000 7f203659c640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20240a2800 0x7f20240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:57.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.734+0000 7f202f7fe640 1 -- 172.21.15.22:0/2826191597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f20300f1160 con 0x7f2038102ec0 2024-03-08T23:51:57.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.735+0000 7f203659c640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20240a2800 0x7f20240a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2028005fd0 tx=0x7f202800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:57.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.739+0000 7f202f7fe640 1 -- 172.21.15.22:0/2826191597 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f20300be050 con 0x7f2038102ec0 2024-03-08T23:51:57.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.922+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2038061ef0 con 0x7f2038102ec0 2024-03-08T23:51:57.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.923+0000 7f202f7fe640 1 -- 172.21.15.22:0/2826191597 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7f20300bd950 con 0x7f2038102ec0 2024-03-08T23:51:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:57.920 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:51:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20240a2800 msgr2=0x7f20240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20240a2800 0x7f20240a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2028005fd0 tx=0x7f202800b410 comp rx=0 tx=0).stop 2024-03-08T23:51:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 msgr2=0x7f2038073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038073c20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f203002c700 tx=0x7f2030032f20 comp rx=0 tx=0).stop 2024-03-08T23:51:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 shutdown_connections 2024-03-08T23:51:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f20240a2800 0x7f20240a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2038102ec0 0x7f2038073c20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2038102500 0x7f20380736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 --2- 172.21.15.22:0/2826191597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101310 0x7f20380750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 >> 172.21.15.22:0/2826191597 conn(0x7f20380fc8a0 msgr2=0x7f2038105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 shutdown_connections 2024-03-08T23:51:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:57.925+0000 7f203ce43640 1 -- 172.21.15.22:0/2826191597 wait complete. 2024-03-08T23:51:57.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:51:58.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 -- 172.21.15.22:0/2482517372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b8102910 msgr2=0x7f32b8102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 --2- 172.21.15.22:0/2482517372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b8102910 0x7f32b8102d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009900 tx=0x7f32b402bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:58.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 -- 172.21.15.22:0/2482517372 shutdown_connections 2024-03-08T23:51:58.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 --2- 172.21.15.22:0/2482517372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32b80ffd40 0x7f32b81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 --2- 172.21.15.22:0/2482517372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32b80ff380 0x7f32b80ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 --2- 172.21.15.22:0/2482517372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b8102910 0x7f32b8102d30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 -- 172.21.15.22:0/2482517372 >> 172.21.15.22:0/2482517372 conn(0x7f32b80faf20 msgr2=0x7f32b80fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 -- 172.21.15.22:0/2482517372 shutdown_connections 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.039+0000 7f32bfd59640 1 -- 172.21.15.22:0/2482517372 wait complete. 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 Processor -- start 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 -- start start 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32b80ff380 0x7f32b8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b80ffd40 0x7f32b8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32b8102910 0x7f32b819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810cff0 con 0x7f32b80ffd40 2024-03-08T23:51:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810ce70 con 0x7f32b80ff380 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810d170 con 0x7f32b8102910 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b80ffd40 0x7f32b8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b80ffd40 0x7f32b8197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56406/0 (socket says 172.21.15.22:56406) 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 -- 172.21.15.22:0/314289927 learned_addr learned my addr 172.21.15.22:0/314289927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bdaf5640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32b80ff380 0x7f32b8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32be2f6640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32b8102910 0x7f32b819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 -- 172.21.15.22:0/314289927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32b8102910 msgr2=0x7f32b819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32b8102910 0x7f32b819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 -- 172.21.15.22:0/314289927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32b80ff380 msgr2=0x7f32b8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32b80ff380 0x7f32b8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 -- 172.21.15.22:0/314289927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b819f250 con 0x7f32b80ffd40 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bd2f4640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b80ffd40 0x7f32b8197b70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d950 tx=0x7f32a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32aeffd640 1 -- 172.21.15.22:0/314289927 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8017070 con 0x7f32b80ffd40 2024-03-08T23:51:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.040+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b819f540 con 0x7f32b80ffd40 2024-03-08T23:51:58.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.041+0000 7f32aeffd640 1 -- 172.21.15.22:0/314289927 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32a8009c30 con 0x7f32b80ffd40 2024-03-08T23:51:58.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.041+0000 7f32aeffd640 1 -- 172.21.15.22:0/314289927 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a800f6b0 con 0x7f32b80ffd40 2024-03-08T23:51:58.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.041+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b819fae0 con 0x7f32b80ffd40 2024-03-08T23:51:58.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.041+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8108850 con 0x7f32b80ffd40 2024-03-08T23:51:58.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.043+0000 7f32aeffd640 1 -- 172.21.15.22:0/314289927 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f32a80109d0 con 0x7f32b80ffd40 2024-03-08T23:51:58.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.043+0000 7f32aeffd640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f32a40a2800 0x7f32a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.043+0000 7f32bdaf5640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f32a40a2800 0x7f32a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.043+0000 7f32aeffd640 1 -- 172.21.15.22:0/314289927 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f32a80ce930 con 0x7f32b80ffd40 2024-03-08T23:51:58.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.044+0000 7f32bdaf5640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f32a40a2800 0x7f32a40a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f32b402efd0 tx=0x7f32b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:58.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.047+0000 7f32aeffd640 1 -- 172.21.15.22:0/314289927 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f32a8009de0 con 0x7f32b80ffd40 2024-03-08T23:51:58.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.234+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32b8061ef0 con 0x7f32b80ffd40 2024-03-08T23:51:58.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.235+0000 7f32aeffd640 1 -- 172.21.15.22:0/314289927 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7f32a809b790 con 0x7f32b80ffd40 2024-03-08T23:51:58.232 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:58.232 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f32a40a2800 msgr2=0x7f32a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f32a40a2800 0x7f32a40a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f32b402efd0 tx=0x7f32b4000bc0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b80ffd40 msgr2=0x7f32b8197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b80ffd40 0x7f32b8197b70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d950 tx=0x7f32a800de10 comp rx=0 tx=0).stop 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 shutdown_connections 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32b8102910 0x7f32b819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f32a40a2800 0x7f32a40a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32b80ffd40 0x7f32b8197b70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 --2- 172.21.15.22:0/314289927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32b80ff380 0x7f32b8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 >> 172.21.15.22:0/314289927 conn(0x7f32b80faf20 msgr2=0x7f32b8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:58.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 shutdown_connections 2024-03-08T23:51:58.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.237+0000 7f32bfd59640 1 -- 172.21.15.22:0/314289927 wait complete. 2024-03-08T23:51:58.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 -- 172.21.15.22:0/2159999869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8101dc0 msgr2=0x7fe6f8108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 --2- 172.21.15.22:0/2159999869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8101dc0 0x7fe6f8108650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009ca0 tx=0x7fe6f002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 -- 172.21.15.22:0/2159999869 shutdown_connections 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 --2- 172.21.15.22:0/2159999869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8101dc0 0x7fe6f8108650 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 --2- 172.21.15.22:0/2159999869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6f8101400 0x7fe6f8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 --2- 172.21.15.22:0/2159999869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f8100210 0x7fe6f8100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 -- 172.21.15.22:0/2159999869 >> 172.21.15.22:0/2159999869 conn(0x7fe6f80fb780 msgr2=0x7fe6f80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 -- 172.21.15.22:0/2159999869 shutdown_connections 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.367+0000 7fe700516640 1 -- 172.21.15.22:0/2159999869 wait complete. 2024-03-08T23:51:58.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 Processor -- start 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 -- start start 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8100210 0x7fe6f8068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6f8101400 0x7fe6f819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f8101dc0 0x7fe6f819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f810bea0 con 0x7fe6f8100210 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f810bd20 con 0x7fe6f8101dc0 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe700516640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f810c020 con 0x7fe6f8101400 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8100210 0x7fe6f8068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8100210 0x7fe6f8068fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56424/0 (socket says 172.21.15.22:56424) 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 -- 172.21.15.22:0/3485139897 learned_addr learned my addr 172.21.15.22:0/3485139897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6feab3640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f8101dc0 0x7fe6f819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fdab1640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6f8101400 0x7fe6f819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 -- 172.21.15.22:0/3485139897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6f8101400 msgr2=0x7fe6f819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6f8101400 0x7fe6f819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 -- 172.21.15.22:0/3485139897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f8101dc0 msgr2=0x7fe6f819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f8101dc0 0x7fe6f819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 -- 172.21.15.22:0/3485139897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f819f1d0 con 0x7fe6f8100210 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6feab3640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f8101dc0 0x7fe6f819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fdab1640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6f8101400 0x7fe6f819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.368+0000 7fe6fe2b2640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8100210 0x7fe6f8068fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400e9c0 tx=0x7fe6f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.369+0000 7fe6ef7fe640 1 -- 172.21.15.22:0/3485139897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400cd80 con 0x7fe6f8100210 2024-03-08T23:51:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.369+0000 7fe6ef7fe640 1 -- 172.21.15.22:0/3485139897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f40035f0 con 0x7fe6f8100210 2024-03-08T23:51:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.369+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f819f4c0 con 0x7fe6f8100210 2024-03-08T23:51:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.369+0000 7fe6ef7fe640 1 -- 172.21.15.22:0/3485139897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4010690 con 0x7fe6f8100210 2024-03-08T23:51:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.369+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f819fa60 con 0x7fe6f8100210 2024-03-08T23:51:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.370+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f81074d0 con 0x7fe6f8100210 2024-03-08T23:51:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.371+0000 7fe6ef7fe640 1 -- 172.21.15.22:0/3485139897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe6f40037a0 con 0x7fe6f8100210 2024-03-08T23:51:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.371+0000 7fe6ef7fe640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6e40a2880 0x7fe6e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.371+0000 7fe6fdab1640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6e40a2880 0x7fe6e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.371+0000 7fe6ef7fe640 1 -- 172.21.15.22:0/3485139897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe6f40cf200 con 0x7fe6f8100210 2024-03-08T23:51:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.372+0000 7fe6fdab1640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6e40a2880 0x7fe6e40a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8068d50 tx=0x7fe6e8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.375+0000 7fe6ef7fe640 1 -- 172.21.15.22:0/3485139897 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe6f409c0f0 con 0x7fe6f8100210 2024-03-08T23:51:58.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.598+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6f802d0e0 con 0x7fe6f8100210 2024-03-08T23:51:58.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.599+0000 7fe6ef7fe640 1 -- 172.21.15.22:0/3485139897 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11283 (secure 0 0 0) 0x7fe6f409b9f0 con 0x7fe6f8100210 2024-03-08T23:51:58.595 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:58.595 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:51:56.628883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-08T23:51:58.596 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-03-08T23:51:58.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6e40a2880 msgr2=0x7fe6e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6e40a2880 0x7fe6e40a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8068d50 tx=0x7fe6e8005d30 comp rx=0 tx=0).stop 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8100210 msgr2=0x7fe6f8068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8100210 0x7fe6f8068fe0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400e9c0 tx=0x7fe6f400ee80 comp rx=0 tx=0).stop 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 shutdown_connections 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f8101dc0 0x7fe6f819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6f8101400 0x7fe6f819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe6e40a2880 0x7fe6e40a4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 --2- 172.21.15.22:0/3485139897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6f8100210 0x7fe6f8068fe0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 >> 172.21.15.22:0/3485139897 conn(0x7fe6f80fb780 msgr2=0x7fe6f81046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 shutdown_connections 2024-03-08T23:51:58.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.601+0000 7fe700516640 1 -- 172.21.15.22:0/3485139897 wait complete. 2024-03-08T23:51:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:58 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2656547445' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-08T23:51:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:58 smithi022 ceph-mon[31894]: osdmap e89: 12 total, 12 up, 12 in 2024-03-08T23:51:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:58 smithi022 ceph-mon[31894]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:58 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/284478175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:58 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2826191597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:58 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/314289927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.608 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:51:58.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:58 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2656547445' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-08T23:51:58.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:58 smithi062 ceph-mon[36129]: osdmap e89: 12 total, 12 up, 12 in 2024-03-08T23:51:58.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:58 smithi062 ceph-mon[36129]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:58.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:58 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/284478175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:58 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2826191597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:58 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/314289927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:58 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2656547445' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-08T23:51:58.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:58 smithi192 ceph-mon[35995]: osdmap e89: 12 total, 12 up, 12 in 2024-03-08T23:51:58.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:58 smithi192 ceph-mon[35995]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:51:58.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:58 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/284478175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:58 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2826191597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:58 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/314289927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 -- 172.21.15.22:0/4173744091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 msgr2=0x7f52bc102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 --2- 172.21.15.22:0/4173744091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc102980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009900 tx=0x7f52b002bc30 comp rx=0 tx=0).stop 2024-03-08T23:51:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 -- 172.21.15.22:0/4173744091 shutdown_connections 2024-03-08T23:51:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 --2- 172.21.15.22:0/4173744091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52bc102ec0 0x7f52bc109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 --2- 172.21.15.22:0/4173744091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc102980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 --2- 172.21.15.22:0/4173744091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52bc101310 0x7f52bc101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 -- 172.21.15.22:0/4173744091 >> 172.21.15.22:0/4173744091 conn(0x7f52bc0fc8a0 msgr2=0x7f52bc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 -- 172.21.15.22:0/4173744091 shutdown_connections 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.714+0000 7f52c27df640 1 -- 172.21.15.22:0/4173744091 wait complete. 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 Processor -- start 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 -- start start 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52bc101310 0x7f52bc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52bc102ec0 0x7f52bc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc10cfa0 con 0x7f52bc102500 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc10ce20 con 0x7f52bc101310 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c27df640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc10d120 con 0x7f52bc102ec0 2024-03-08T23:51:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56446/0 (socket says 172.21.15.22:56446) 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 -- 172.21.15.22:0/704401575 learned_addr learned my addr 172.21.15.22:0/704401575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bbfff640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52bc101310 0x7f52bc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c0d7c640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52bc102ec0 0x7f52bc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 -- 172.21.15.22:0/704401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52bc102ec0 msgr2=0x7f52bc19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52bc102ec0 0x7f52bc19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 -- 172.21.15.22:0/704401575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52bc101310 msgr2=0x7f52bc1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52bc101310 0x7f52bc1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bb7fe640 1 -- 172.21.15.22:0/704401575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc19f210 con 0x7f52bc102500 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52c0d7c640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52bc102ec0 0x7f52bc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.715+0000 7f52bbfff640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52bc101310 0x7f52bc1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.716+0000 7f52bb7fe640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc197b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009230 tx=0x7f52b0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.716+0000 7f52b97fa640 1 -- 172.21.15.22:0/704401575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0041070 con 0x7f52bc102500 2024-03-08T23:51:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.716+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc19f4a0 con 0x7f52bc102500 2024-03-08T23:51:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.716+0000 7f52b97fa640 1 -- 172.21.15.22:0/704401575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52b00044d0 con 0x7f52bc102500 2024-03-08T23:51:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.716+0000 7f52b97fa640 1 -- 172.21.15.22:0/704401575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0034440 con 0x7f52bc102500 2024-03-08T23:51:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.716+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc19f990 con 0x7f52bc102500 2024-03-08T23:51:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.717+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc108630 con 0x7f52bc102500 2024-03-08T23:51:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.718+0000 7f52b97fa640 1 -- 172.21.15.22:0/704401575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f52b0044070 con 0x7f52bc102500 2024-03-08T23:51:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.718+0000 7f52b97fa640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52a40a2800 0x7f52a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.719+0000 7f52bbfff640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52a40a2800 0x7f52a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.719+0000 7f52b97fa640 1 -- 172.21.15.22:0/704401575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f52b00f0920 con 0x7f52bc102500 2024-03-08T23:51:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.719+0000 7f52bbfff640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52a40a2800 0x7f52a40a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52bc102360 tx=0x7f52ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.723+0000 7f52b97fa640 1 -- 172.21.15.22:0/704401575 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f52b00bd780 con 0x7f52bc102500 2024-03-08T23:51:58.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.905+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52bc061ef0 con 0x7f52bc102500 2024-03-08T23:51:58.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.907+0000 7f52b97fa640 1 -- 172.21.15.22:0/704401575 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7f52b00bd080 con 0x7f52bc102500 2024-03-08T23:51:58.904 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:51:58.904 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:51:58.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52a40a2800 msgr2=0x7f52a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52a40a2800 0x7f52a40a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52bc102360 tx=0x7f52ac005f90 comp rx=0 tx=0).stop 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 msgr2=0x7f52bc197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc197b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009230 tx=0x7f52b0005d50 comp rx=0 tx=0).stop 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 shutdown_connections 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52bc102ec0 0x7f52bc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52a40a2800 0x7f52a40a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52bc102500 0x7f52bc197b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 --2- 172.21.15.22:0/704401575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52bc101310 0x7f52bc1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 >> 172.21.15.22:0/704401575 conn(0x7f52bc0fc8a0 msgr2=0x7f52bc105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 shutdown_connections 2024-03-08T23:51:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:58.909+0000 7f52c27df640 1 -- 172.21.15.22:0/704401575 wait complete. 2024-03-08T23:51:58.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T23:51:51.610292+0000', 'modified': '2024-03-08T23:51:56.628883+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.192:6839/1126404206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 1126404206}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 1126404206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-08T23:51:58.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T23:51:58.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 -- 172.21.15.22:0/637114868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 msgr2=0x7f0554109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 --2- 172.21.15.22:0/637114868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554109420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f054c009ca0 tx=0x7f054c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 -- 172.21.15.22:0/637114868 shutdown_connections 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 --2- 172.21.15.22:0/637114868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554109420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 --2- 172.21.15.22:0/637114868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05541046a0 0x7f0554106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 --2- 172.21.15.22:0/637114868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0554101d70 0x7f0554104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 -- 172.21.15.22:0/637114868 >> 172.21.15.22:0/637114868 conn(0x7f05540fb780 msgr2=0x7f05540fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 -- 172.21.15.22:0/637114868 shutdown_connections 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.024+0000 7f055a6fc640 1 -- 172.21.15.22:0/637114868 wait complete. 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 Processor -- start 2024-03-08T23:51:59.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 -- start start 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554101d70 0x7f05541016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05541046a0 0x7f05540ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554100260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055410cf80 con 0x7f0554106fd0 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055410ce00 con 0x7f0554101d70 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f055a6fc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055410d100 con 0x7f05541046a0 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f0558c99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554100260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f0558c99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554100260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56470/0 (socket says 172.21.15.22:56470) 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f0558c99640 1 -- 172.21.15.22:0/1190420729 learned_addr learned my addr 172.21.15.22:0/1190420729 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.025+0000 7f0553fff640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554101d70 0x7f05541016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f05537fe640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05541046a0 0x7f05540ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f0558c99640 1 -- 172.21.15.22:0/1190420729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05541046a0 msgr2=0x7f05540ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f0558c99640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05541046a0 0x7f05540ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f0558c99640 1 -- 172.21.15.22:0/1190420729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554101d70 msgr2=0x7f05541016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f0558c99640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554101d70 0x7f05541016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f0558c99640 1 -- 172.21.15.22:0/1190420729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554100af0 con 0x7f0554106fd0 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f0558c99640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554100260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c700 tx=0x7f054c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f05517fa640 1 -- 172.21.15.22:0/1190420729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c039040 con 0x7f0554106fd0 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f05517fa640 1 -- 172.21.15.22:0/1190420729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f054c044070 con 0x7f0554106fd0 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05541a16b0 con 0x7f0554106fd0 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f05517fa640 1 -- 172.21.15.22:0/1190420729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c0042a0 con 0x7f0554106fd0 2024-03-08T23:51:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.026+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05541a1c00 con 0x7f0554106fd0 2024-03-08T23:51:59.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f05517fa640 1 -- 172.21.15.22:0/1190420729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f054c003970 con 0x7f0554106fd0 2024-03-08T23:51:59.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 Processor -- start 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 -- start start 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82d0e0 0x56302b82d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82dad0 0x56302b8325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b8350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b7927c0 con 0x56302b82d0e0 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b73bb50 con 0x56302b832c40 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f055d3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b78e860 con 0x56302b82dad0 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f05537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82d0e0 0x56302b82d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f05537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82d0e0 0x56302b82d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56486/0 (socket says 172.21.15.22:56486) 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f05537fe640 1 -- 172.21.15.22:0/3695773323 learned_addr learned my addr 172.21.15.22:0/3695773323 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.029+0000 7f05517fa640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05400a2800 0x7f05400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f0553fff640 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82dad0 0x56302b8325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f0558c99640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05400a2800 0x7f05400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f05537fe640 1 -- 172.21.15.22:0/3695773323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82dad0 msgr2=0x56302b8325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f05537fe640 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82dad0 0x56302b8325d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f05537fe640 1 -- 172.21.15.22:0/3695773323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 msgr2=0x56302b8350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f05537fe640 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b8350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f05537fe640 1 -- 172.21.15.22:0/3695773323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56302b74f9a0 con 0x56302b82d0e0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f0553fff640 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82dad0 0x56302b8325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f05517fa640 1 -- 172.21.15.22:0/1190420729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f054c0f1270 con 0x7f0554106fd0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f05537fe640 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82d0e0 0x56302b82d500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f054400bd00 tx=0x7f0544011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1d3d9a4cfc48d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f054a7fc640 1 -- 172.21.15.22:0/3695773323 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054401a400 con 0x56302b82d0e0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f0558c99640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05400a2800 0x7f05400a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f054c04d000 tx=0x7f054c0a8040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f054a7fc640 1 -- 172.21.15.22:0/3695773323 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f054401aa80 con 0x56302b82d0e0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.030+0000 7f054a7fc640 1 -- 172.21.15.22:0/3695773323 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544012ad0 con 0x56302b82d0e0 2024-03-08T23:51:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 -- 172.21.15.22:0/3695773323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82d0e0 msgr2=0x56302b82d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82d0e0 0x56302b82d500 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f054400bd00 tx=0x7f0544011750 comp rx=0 tx=0).stop 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 -- 172.21.15.22:0/3695773323 shutdown_connections 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b8350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82dad0 0x56302b8325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3695773323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82d0e0 0x56302b82d500 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 -- 172.21.15.22:0/3695773323 >> 172.21.15.22:0/3695773323 conn(0x56302b7f5400 msgr2=0x56302b828a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 -- 172.21.15.22:0/3695773323 shutdown_connections 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 -- 172.21.15.22:0/3695773323 wait complete. 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 -- start start 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 Processor -- start 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 0x56302b837630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.031+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b837b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b83ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b835bf0 con 0x56302b82dad0 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b74ef90 con 0x56302b832c40 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b835d70 con 0x56302b82d0e0 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 10 client.0 _ll_register_callbacks cb 0x56302b78a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b837b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b837b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56490/0 (socket says 172.21.15.22:56490) 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 -- 172.21.15.22:0/1947228856 learned_addr learned my addr 172.21.15.22:0/1947228856 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f05537fe640 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b83ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 -- 172.21.15.22:0/1947228856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 msgr2=0x56302b837630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 0x56302b837630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 -- 172.21.15.22:0/1947228856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 msgr2=0x56302b83ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b83ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 -- 172.21.15.22:0/1947228856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56302b4a72d0 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f0553fff640 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b837b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f053c00b940 tx=0x7f053c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4ee624fa94d8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f05337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f05337fe640 1 -- 172.21.15.22:0/1947228856 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c005020 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f05337fe640 1 -- 172.21.15.22:0/1947228856 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f053c0051c0 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f054401aa80 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f05337fe640 1 -- 172.21.15.22:0/1947228856 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c0053c0 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 10 client.14766 fetch_fsmap learned FSMap version 17 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56302b5716b0 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.032+0000 7f05337fe640 1 -- 172.21.15.22:0/1947228856 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f054401aa80 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f05337fe640 1 -- 172.21.15.22:0/1947228856 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f053c0127b0 con 0x56302b82dad0 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f055d3b3740 10 client.14766 fetch_fsmap finished waiting for FSMap version 17 2024-03-08T23:51:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f055d3b3740 10 client.14766 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f055d3b3740 20 client.14766 populate_metadata read hostname 'smithi022' 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b844190 0x56302b843250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f055d3b3740 4 client.14766 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56302b79fd00 con 0x56302b844190 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f05537fe640 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b844190 0x56302b843250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f05337fe640 10 client.14766 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.033+0000 7f05537fe640 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b844190 0x56302b843250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a255e664e1bcba41 server_cookie=6521d4760b817c17 in_seq=0 out_seq=0 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f05337fe640 1 -- 172.21.15.22:0/1947228856 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56302b79fd00 con 0x56302b844190 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f05337fe640 10 client.14766 handle_command_reply: tid=0 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 client.14766 shutdown 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 msgr2=0x56302b837b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b837b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f053c00b940 tx=0x7f053c004610 comp rx=0 tx=0).stop 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 shutdown_connections 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b844190 0x56302b843250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b83ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b837b70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f05337fe640 0 client.14766 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:51:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1947228856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 0x56302b837630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.034+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 >> 172.21.15.22:0/1947228856 conn(0x56302b7f5400 msgr2=0x56302b827c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.035+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 shutdown_connections 2024-03-08T23:51:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.035+0000 7f055d3b3740 1 -- 172.21.15.22:0/1947228856 wait complete. 2024-03-08T23:51:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.035+0000 7f055d3b3740 20 client.14766 trim_cache size 0 max 16384 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 Processor -- start 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 -- start start 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 0x56302b828570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b828ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b83eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b8439b0 con 0x56302b82dad0 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b844010 con 0x56302b832c40 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f055d3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b844670 con 0x56302b82d0e0 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f0553fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b828ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f0553fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b828ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56498/0 (socket says 172.21.15.22:56498) 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.050+0000 7f0553fff640 1 -- 172.21.15.22:0/3418179858 learned_addr learned my addr 172.21.15.22:0/3418179858 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0553fff640 1 -- 172.21.15.22:0/3418179858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 msgr2=0x56302b828570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0553fff640 1 --2- 172.21.15.22:0/3418179858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 0x56302b828570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0553fff640 1 -- 172.21.15.22:0/3418179858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 msgr2=0x56302b83eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0553fff640 1 --2- 172.21.15.22:0/3418179858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b83eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0553fff640 1 -- 172.21.15.22:0/3418179858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56302b8297a0 con 0x56302b82dad0 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0553fff640 1 --2- 172.21.15.22:0/3418179858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b828ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f053c00b910 tx=0x7f053c004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba511cd95d386fad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0549ffb640 1 -- 172.21.15.22:0/3418179858 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c02e070 con 0x56302b82dad0 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0549ffb640 1 -- 172.21.15.22:0/3418179858 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f053c016070 con 0x56302b82dad0 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f0549ffb640 1 -- 172.21.15.22:0/3418179858 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c020a10 con 0x56302b82dad0 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f055d3b3740 1 -- 172.21.15.22:0/3418179858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 msgr2=0x56302b828ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.051+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3418179858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b828ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f053c00b910 tx=0x7f053c004110 comp rx=0 tx=0).stop 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- 172.21.15.22:0/3418179858 shutdown_connections 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3418179858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b832c40 0x56302b83eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3418179858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b828ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 --2- 172.21.15.22:0/3418179858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b82d0e0 0x56302b828570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- 172.21.15.22:0/3418179858 >> 172.21.15.22:0/3418179858 conn(0x56302b7f5400 msgr2=0x56302b82e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- 172.21.15.22:0/3418179858 shutdown_connections 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- 172.21.15.22:0/3418179858 wait complete. 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- start start 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 Processor -- start 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b82d0e0 0x56302b83b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b83beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b832c40 0x56302b899d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b845330 con 0x56302b82dad0 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b844cd0 con 0x56302b82d0e0 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b845990 con 0x56302b832c40 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f055d3b3740 10 client.0 _ll_register_callbacks cb 0x7f054400b4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f0553fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b83beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f0553fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b83beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56504/0 (socket says 172.21.15.22:56504) 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f0553fff640 1 -- 172.21.15.22:0/1268267546 learned_addr learned my addr 172.21.15.22:0/1268267546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f0553fff640 1 -- 172.21.15.22:0/1268267546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b832c40 msgr2=0x56302b899d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.052+0000 7f0553fff640 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b832c40 0x56302b899d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f0553fff640 1 -- 172.21.15.22:0/1268267546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b82d0e0 msgr2=0x56302b83b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f0553fff640 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b82d0e0 0x56302b83b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f0553fff640 1 -- 172.21.15.22:0/1268267546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56302b832990 con 0x56302b82dad0 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f0553fff640 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b83beb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f053c005d00 tx=0x7f053c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71efe49c3155a635 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f05337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f05337fe640 1 -- 172.21.15.22:0/1268267546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c02e040 con 0x56302b82dad0 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f05337fe640 1 -- 172.21.15.22:0/1268267546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f053c012650 con 0x56302b82dad0 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f053c016070 con 0x56302b82dad0 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f05337fe640 1 -- 172.21.15.22:0/1268267546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c012850 con 0x56302b82dad0 2024-03-08T23:51:59.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 10 client.14772 fetch_fsmap learned FSMap version 17 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56302b89b2c0 con 0x56302b82dad0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f05337fe640 1 -- 172.21.15.22:0/1268267546 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f053c016070 con 0x56302b82dad0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f05337fe640 1 -- 172.21.15.22:0/1268267546 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f053c029980 con 0x56302b82dad0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 10 client.14772 fetch_fsmap finished waiting for FSMap version 17 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 10 client.14772 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 20 client.14772 populate_metadata read hostname 'smithi022' 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b89f7a0 0x56302b89b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 4 client.14772 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.053+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56302b79a050 con 0x56302b89f7a0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f05537fe640 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b89f7a0 0x56302b89b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f05537fe640 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b89f7a0 0x56302b89b4f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30fb63684c89eb40 server_cookie=52fc0e6b783e894b in_seq=0 out_seq=0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f05337fe640 10 client.14772 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f05337fe640 1 -- 172.21.15.22:0/1268267546 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56302b79a050 con 0x56302b89f7a0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f05337fe640 10 client.14772 handle_command_reply: tid=0 2024-03-08T23:51:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 client.14772 shutdown 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 msgr2=0x56302b83beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b83beb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f053c005d00 tx=0x7f053c0039c0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 shutdown_connections 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56302b89f7a0 0x56302b89b4f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56302b832c40 0x56302b899d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302b82dad0 0x56302b83beb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 --2- 172.21.15.22:0/1268267546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56302b82d0e0 0x56302b83b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f05337fe640 0 client.14772 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 >> 172.21.15.22:0/1268267546 conn(0x56302b7f5400 msgr2=0x56302b828570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.054+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 shutdown_connections 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055d3b3740 1 -- 172.21.15.22:0/1268267546 wait complete. 2024-03-08T23:51:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055d3b3740 20 client.14772 trim_cache size 0 max 16384 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24496, 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-08T23:51:59.058 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 6.8170807230000001, 2024-03-08T23:51:59.059 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 17, 2024-03-08T23:51:59.059 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 89, 2024-03-08T23:51:59.059 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 88, 2024-03-08T23:51:59.059 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 19.502990787000002 2024-03-08T23:51:59.059 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05400a2800 msgr2=0x7f05400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05400a2800 0x7f05400a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f054c04d000 tx=0x7f054c0a8040 comp rx=0 tx=0).stop 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 msgr2=0x7f0554100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554100260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c700 tx=0x7f054c032f20 comp rx=0 tx=0).stop 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 shutdown_connections 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05400a2800 0x7f05400a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0554106fd0 0x7f0554100260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05541046a0 0x7f05540ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 --2- 172.21.15.22:0/1190420729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554101d70 0x7f05541016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 >> 172.21.15.22:0/1190420729 conn(0x7f05540fb780 msgr2=0x7f05541026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 shutdown_connections 2024-03-08T23:51:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:51:59.055+0000 7f055a6fc640 1 -- 172.21.15.22:0/1190420729 wait complete. 2024-03-08T23:51:59.062 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T23:51:59.063 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:51:59.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-08T23:51:59.303 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:51:59.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:59 smithi022 ceph-mon[31894]: pgmap v296: 129 pgs: 4 creating+peering, 111 active+clean, 14 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-08T23:51:59.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:59 smithi022 ceph-mon[31894]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-08T23:51:59.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:59 smithi022 ceph-mon[31894]: Cluster is now healthy 2024-03-08T23:51:59.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:59 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3485139897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:59.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:51:59 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/704401575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:59.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:59 smithi062 ceph-mon[36129]: pgmap v296: 129 pgs: 4 creating+peering, 111 active+clean, 14 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-08T23:51:59.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:59 smithi062 ceph-mon[36129]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-08T23:51:59.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:59 smithi062 ceph-mon[36129]: Cluster is now healthy 2024-03-08T23:51:59.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:59 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3485139897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:59.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:51:59 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/704401575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:51:59.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:59 smithi192 ceph-mon[35995]: pgmap v296: 129 pgs: 4 creating+peering, 111 active+clean, 14 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-08T23:51:59.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:59 smithi192 ceph-mon[35995]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-08T23:51:59.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:59 smithi192 ceph-mon[35995]: Cluster is now healthy 2024-03-08T23:51:59.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:59 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3485139897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:51:59.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:51:59 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/704401575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 -- 172.21.15.22:0/1983912352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c106d20 msgr2=0x7f9c9c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1983912352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c106d20 0x7f9c9c107100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009990 tx=0x7f9c90009ca0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 -- 172.21.15.22:0/1983912352 shutdown_connections 2024-03-08T23:52:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1983912352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c9c101680 0x7f9c9c10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1983912352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c106d20 0x7f9c9c107100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1983912352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c100ce0 0x7f9c9c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 -- 172.21.15.22:0/1983912352 >> 172.21.15.22:0/1983912352 conn(0x7f9c9c077c10 msgr2=0x7f9c9c078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 -- 172.21.15.22:0/1983912352 shutdown_connections 2024-03-08T23:52:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.123+0000 7f9ca1084700 1 -- 172.21.15.22:0/1983912352 wait complete. 2024-03-08T23:52:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 Processor -- start 2024-03-08T23:52:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 -- start start 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c100ce0 0x7f9c9c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c9c101680 0x7f9c9c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c106d20 0x7f9c9c1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c110f60 con 0x7f9c9c100ce0 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c110de0 con 0x7f9c9c106d20 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9ca1084700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c1110e0 con 0x7f9c9c101680 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c100ce0 0x7f9c9c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c100ce0 0x7f9c9c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52492/0 (socket says 172.21.15.22:52492) 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 -- 172.21.15.22:0/1300486764 learned_addr learned my addr 172.21.15.22:0/1300486764 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9a59c700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c9c101680 0x7f9c9c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:01.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9b59e700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c106d20 0x7f9c9c1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 -- 172.21.15.22:0/1300486764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c9c101680 msgr2=0x7f9c9c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c9c101680 0x7f9c9c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 -- 172.21.15.22:0/1300486764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c106d20 msgr2=0x7f9c9c1a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c106d20 0x7f9c9c1a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.124+0000 7f9c9ad9d700 1 -- 172.21.15.22:0/1300486764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c1a4940 con 0x7f9c9c100ce0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.125+0000 7f9c9b59e700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c106d20 0x7f9c9c1a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.125+0000 7f9c9ad9d700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c100ce0 0x7f9c9c199380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000c00 tx=0x7f9c900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.125+0000 7f9c83fff700 1 -- 172.21.15.22:0/1300486764 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90027070 con 0x7f9c9c100ce0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.125+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c1a4b10 con 0x7f9c9c100ce0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.125+0000 7f9c83fff700 1 -- 172.21.15.22:0/1300486764 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c90003780 con 0x7f9c9c100ce0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.125+0000 7f9c83fff700 1 -- 172.21.15.22:0/1300486764 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90022dd0 con 0x7f9c9c100ce0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.125+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c1a4fc0 con 0x7f9c9c100ce0 2024-03-08T23:52:01.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.126+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9c068aa0 con 0x7f9c9c100ce0 2024-03-08T23:52:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.127+0000 7f9c83fff700 1 -- 172.21.15.22:0/1300486764 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9c9002a070 con 0x7f9c9c100ce0 2024-03-08T23:52:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.128+0000 7f9c83fff700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c840a2890 0x7f9c840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.128+0000 7f9c9a59c700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c840a2890 0x7f9c840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:01.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.128+0000 7f9c83fff700 1 -- 172.21.15.22:0/1300486764 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9c900cf960 con 0x7f9c9c100ce0 2024-03-08T23:52:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.128+0000 7f9c9a59c700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c840a2890 0x7f9c840a4d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c078350 tx=0x7f9c8c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:01.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.132+0000 7f9c83fff700 1 -- 172.21.15.22:0/1300486764 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9c90013070 con 0x7f9c9c100ce0 2024-03-08T23:52:01.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.382+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c9c02d9f0 con 0x7f9c9c100ce0 2024-03-08T23:52:01.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.382+0000 7f9c83fff700 1 -- 172.21.15.22:0/1300486764 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9c9009c8e0 con 0x7f9c9c100ce0 2024-03-08T23:52:01.379 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.384+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c840a2890 msgr2=0x7f9c840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.384+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c840a2890 0x7f9c840a4d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c078350 tx=0x7f9c8c0073d0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.384+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c100ce0 msgr2=0x7f9c9c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.384+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c100ce0 0x7f9c9c199380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000c00 tx=0x7f9c900047b0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.384+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 shutdown_connections 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.385+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9c106d20 0x7f9c9c1a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.385+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c840a2890 0x7f9c840a4d50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.385+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9c100ce0 0x7f9c9c199380 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.385+0000 7f9ca1084700 1 --2- 172.21.15.22:0/1300486764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c9c101680 0x7f9c9c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.385+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 >> 172.21.15.22:0/1300486764 conn(0x7f9c9c077c10 msgr2=0x7f9c9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.385+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 shutdown_connections 2024-03-08T23:52:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:01.385+0000 7f9ca1084700 1 -- 172.21.15.22:0/1300486764 wait complete. 2024-03-08T23:52:01.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:01 smithi062 ceph-mon[36129]: pgmap v297: 129 pgs: 4 creating+peering, 123 active+clean, 2 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-08T23:52:01.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:01 smithi192 ceph-mon[35995]: pgmap v297: 129 pgs: 4 creating+peering, 123 active+clean, 2 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-08T23:52:01.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:01 smithi022 ceph-mon[31894]: pgmap v297: 129 pgs: 4 creating+peering, 123 active+clean, 2 unknown; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T23:52:02.183 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-08T23:52:02.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.184 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-03-08T23:52:02.200 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T23:52:02.200 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T23:52:02.200 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T23:52:02.200 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42105sec preferred_lft 42105sec 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-03-08T23:52:02.201 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-08T23:52:02.202 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T23:52:02.202 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-03-08T23:52:02.202 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-03-08T23:52:02.202 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T23:52:02.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.203 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:52:02.203 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:52:02.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-03-08T23:52:02.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-03-08T23:52:02.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-03-08T23:52:02.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T23:52:02.203 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:52:02.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.362 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T23:52:02.387 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-03-08T23:52:02.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.389 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-03-08T23:52:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-03-08T23:52:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T23:52:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T23:52:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T23:52:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T23:52:02.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.458 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:52:02.458 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:52:02.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T23:52:02.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T23:52:02.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T23:52:02.458 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:52:02.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:02 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1300486764' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T23:52:02.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:02 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1300486764' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T23:52:02.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:02 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1300486764' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T23:52:02.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.706 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-08T23:52:02.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.721 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-03-08T23:52:02.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.775 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:52:02.775 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:52:02.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T23:52:02.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-08T23:52:02.775 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:52:02.858 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-08T23:52:02.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:02.981 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:52:02.981 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:52:02.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-08T23:52:02.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T23:52:02.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-08T23:52:02.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-08T23:52:02.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-08T23:52:02.982 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:52:03.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:03.082 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:52:03.082 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:52:03.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-03-08T23:52:03.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T23:52:03.082 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:52:03.123 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-08T23:52:03.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T23:52:03.123 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:52:03.176 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-08T23:52:03.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T23:52:03.176 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:52:03.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-08T23:52:03.298 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-08T23:52:03.353 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T23:52:03.354 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysctl.service 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0d1ded0f84d7b9f41c28d3da9f176875de9ac5547285d986b6438f00b557f8ea/merged 2024-03-08T23:52:03.355 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/ef21de54ca120686765363467030e8b5f1b1b7329f07cd7e43e58c9ddf795f35/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/0 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/9f01b6765e8fb5905850c0362f876798d4556c7a762d5ece84455cd01ca61725/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/78cfc21f55aada3b3659d0480b6915c9d8080b25e26db6b4111c0e128a050c8c/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1e75a6bf0f05ff07aa24115ad3629b000e0fdc222e44467f7a15012f7b969afd/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/cc3059c8fef71b440e494cb1d6d3f1b68ac9dea51ee7f9ee1894d31dda593c22/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/b6aaf7ebac4df44743df1c6f4c1ea84e00ee2eed38601fa93b75e32b13a3e36c/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/d4b1538fae05a30772b14e745e8887e6bb5c8d374b3b00d8eddf32e6d24ecb87/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1a64a9a56a76b7c7d12c7792295091fac6429cf8e1c93e03cb816fef99955a22/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/ab69e7c73976489401f7024748523e77a7e574647e363ce12b52f689217c7977/merged 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T23:52:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T23:52:03.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:03.357 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-08T23:52:03.408 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-08T23:52:03.409 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-08T23:52:03.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2024-03-08T23:52:03.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:03 smithi022 ceph-mon[31894]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-08T23:52:03.476 DEBUG:teuthology.orchestra.run.smithi022:> cat /proc/self/mounts | awk '{print $2}' 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/proc 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/sys 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/dev 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/security 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/shm 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/pts 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/run 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/cgroup 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/pstore 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/bpf 2024-03-08T23:52:03.532 INFO:teuthology.orchestra.run.smithi022.stdout:/ 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/selinux 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/hugepages 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/mqueue 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/debug 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/tracing 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/fs/fuse/connections 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/sys/kernel/config 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-sysctl.service 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/nfs/rpc_pipefs 2024-03-08T23:52:03.533 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/1000 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/ceph 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/0d1ded0f84d7b9f41c28d3da9f176875de9ac5547285d986b6438f00b557f8ea/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/ef21de54ca120686765363467030e8b5f1b1b7329f07cd7e43e58c9ddf795f35/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/run/user/0 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/9f01b6765e8fb5905850c0362f876798d4556c7a762d5ece84455cd01ca61725/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/78cfc21f55aada3b3659d0480b6915c9d8080b25e26db6b4111c0e128a050c8c/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1e75a6bf0f05ff07aa24115ad3629b000e0fdc222e44467f7a15012f7b969afd/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/cc3059c8fef71b440e494cb1d6d3f1b68ac9dea51ee7f9ee1894d31dda593c22/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/b6aaf7ebac4df44743df1c6f4c1ea84e00ee2eed38601fa93b75e32b13a3e36c/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/d4b1538fae05a30772b14e745e8887e6bb5c8d374b3b00d8eddf32e6d24ecb87/merged 2024-03-08T23:52:03.534 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/1a64a9a56a76b7c7d12c7792295091fac6429cf8e1c93e03cb816fef99955a22/merged 2024-03-08T23:52:03.535 INFO:teuthology.orchestra.run.smithi022.stdout:/var/lib/containers/storage/overlay/ab69e7c73976489401f7024748523e77a7e574647e363ce12b52f689217c7977/merged 2024-03-08T23:52:03.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns 2024-03-08T23:52:03.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T23:52:03.535 INFO:teuthology.orchestra.run.smithi022.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T23:52:03.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:03.535 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2024-03-08T23:52:03.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:2024-03-08T23:52:03.544+0000 7fe53b248480 -1 init, newargv = 0x55bfe2075ac0 newargc=15 2024-03-08T23:52:03.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[75438]: starting ceph client 2024-03-08T23:52:03.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[75438]: starting fuse 2024-03-08T23:52:03.556 INFO:teuthology.orchestra.run.smithi022.stdout:119 2024-03-08T23:52:03.556 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-08T23:52:03.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2024-03-08T23:52:03.556 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2024-03-08T23:52:03.556 DEBUG:teuthology.orchestra.run.smithi022:> import re 2024-03-08T23:52:03.556 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-03-08T23:52:03.556 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-08T23:52:03.557 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2024-03-08T23:52:03.558 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2024-03-08T23:52:03.558 DEBUG:teuthology.orchestra.run.smithi022:> return f 2024-03-08T23:52:03.558 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-08T23:52:03.558 DEBUG:teuthology.orchestra.run.smithi022:> 2024-03-08T23:52:03.558 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.admin")) 2024-03-08T23:52:03.558 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-03-08T23:52:03.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:03 smithi062 ceph-mon[36129]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-08T23:52:03.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:03 smithi192 ceph-mon[35995]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-08T23:52:03.676 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.admin.75438.asok 2024-03-08T23:52:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T23:52:03.687 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75438.asok 2024-03-08T23:52:03.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:03.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75438.asok status 2024-03-08T23:52:03.825 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "7d63482aa89628d733e298984a762389b625c761", 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "admin", 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "75438", 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 14784, 2024-03-08T23:52:03.826 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 14784 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2318493702 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "192.168.0.1:0", 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2318493702 2024-03-08T23:52:03.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.14784 192.168.0.1:0/2318493702", 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "192.168.0.1:0/2318493702", 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 17, 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 89, 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2024-03-08T23:52:03.828 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:52:03.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:03.834 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2024-03-08T23:52:03.834 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:52:03.849 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2024-03-08T23:52:03.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:52:03.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:03.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:52:03.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:03.917 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-08T23:52:03.989 INFO:teuthology.orchestra.run.smithi022.stdout:# file: . 2024-03-08T23:52:03.990 INFO:teuthology.orchestra.run.smithi022.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-08T23:52:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:03.990 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:52:03.990 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T23:52:04.046 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.191+0000 7fc19e043640 1 -- 172.21.15.22:0/1256344574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198106ff0 msgr2=0x7fc198109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.191+0000 7fc19e043640 1 --2- 172.21.15.22:0/1256344574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198106ff0 0x7fc198109440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc190009900 tx=0x7fc19002bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- 172.21.15.22:0/1256344574 shutdown_connections 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 --2- 172.21.15.22:0/1256344574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198106ff0 0x7fc198109440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 --2- 172.21.15.22:0/1256344574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1981046c0 0x7fc198106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 --2- 172.21.15.22:0/1256344574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc198101d90 0x7fc198104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- 172.21.15.22:0/1256344574 >> 172.21.15.22:0/1256344574 conn(0x7fc1980fb780 msgr2=0x7fc1980fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- 172.21.15.22:0/1256344574 shutdown_connections 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- 172.21.15.22:0/1256344574 wait complete. 2024-03-08T23:52:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 Processor -- start 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- start start 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198101d90 0x7fc1981997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1981046c0 0x7fc198199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc198106ff0 0x7fc1981a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981010f0 con 0x7fc198101d90 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198100f70 con 0x7fc198106ff0 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.192+0000 7fc19e043640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198101270 con 0x7fc1981046c0 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198101d90 0x7fc1981997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198101d90 0x7fc1981997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52502/0 (socket says 172.21.15.22:52502) 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 -- 172.21.15.22:0/3436137545 learned_addr learned my addr 172.21.15.22:0/3436137545 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:04.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc196ffd640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1981046c0 0x7fc198199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc197fff640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc198106ff0 0x7fc1981a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 -- 172.21.15.22:0/3436137545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1981046c0 msgr2=0x7fc198199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1981046c0 0x7fc198199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 -- 172.21.15.22:0/3436137545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc198106ff0 msgr2=0x7fc1981a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc198106ff0 0x7fc1981a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 -- 172.21.15.22:0/3436137545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1981a13e0 con 0x7fc198101d90 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc197fff640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc198106ff0 0x7fc1981a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc1977fe640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198101d90 0x7fc1981997c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc18800ebc0 tx=0x7fc18800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc194ff9640 1 -- 172.21.15.22:0/3436137545 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188010630 con 0x7fc198101d90 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc194ff9640 1 -- 172.21.15.22:0/3436137545 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc188010cb0 con 0x7fc198101d90 2024-03-08T23:52:04.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1981a16d0 con 0x7fc198101d90 2024-03-08T23:52:04.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc194ff9640 1 -- 172.21.15.22:0/3436137545 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188018bd0 con 0x7fc198101d90 2024-03-08T23:52:04.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.193+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1981a1c20 con 0x7fc198101d90 2024-03-08T23:52:04.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.194+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1980fd270 con 0x7fc198101d90 2024-03-08T23:52:04.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.195+0000 7fc194ff9640 1 -- 172.21.15.22:0/3436137545 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc1880107d0 con 0x7fc198101d90 2024-03-08T23:52:04.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.196+0000 7fc194ff9640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1800a2800 0x7fc1800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.196+0000 7fc196ffd640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1800a2800 0x7fc1800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.196+0000 7fc194ff9640 1 -- 172.21.15.22:0/3436137545 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc1880cf5e0 con 0x7fc198101d90 2024-03-08T23:52:04.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.197+0000 7fc196ffd640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1800a2800 0x7fc1800a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005fd0 tx=0x7fc18c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:04.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.200+0000 7fc194ff9640 1 -- 172.21.15.22:0/3436137545 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc18809c440 con 0x7fc198101d90 2024-03-08T23:52:04.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.386+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc198061f20 con 0x7fc198101d90 2024-03-08T23:52:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.388+0000 7fc194ff9640 1 -- 172.21.15.22:0/3436137545 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7fc18809bd40 con 0x7fc198101d90 2024-03-08T23:52:04.385 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:04.385 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1800a2800 msgr2=0x7fc1800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1800a2800 0x7fc1800a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005fd0 tx=0x7fc18c005850 comp rx=0 tx=0).stop 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198101d90 msgr2=0x7fc1981997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198101d90 0x7fc1981997c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc18800ebc0 tx=0x7fc18800c6a0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 shutdown_connections 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc198106ff0 0x7fc1981a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1981046c0 0x7fc198199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc1800a2800 0x7fc1800a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 --2- 172.21.15.22:0/3436137545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc198101d90 0x7fc1981997c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 >> 172.21.15.22:0/3436137545 conn(0x7fc1980fb780 msgr2=0x7fc1981026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 shutdown_connections 2024-03-08T23:52:04.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.390+0000 7fc19e043640 1 -- 172.21.15.22:0/3436137545 wait complete. 2024-03-08T23:52:04.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:52:04.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:52:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 -- 172.21.15.22:0/2447640032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 msgr2=0x7f6c9c0ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 --2- 172.21.15.22:0/2447640032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c0ff9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009930 tx=0x7f6c8c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 -- 172.21.15.22:0/2447640032 shutdown_connections 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 --2- 172.21.15.22:0/2447640032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c9c102f60 0x7f6c9c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 --2- 172.21.15.22:0/2447640032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9c1025a0 0x7f6c9c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 --2- 172.21.15.22:0/2447640032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c0ff9f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 -- 172.21.15.22:0/2447640032 >> 172.21.15.22:0/2447640032 conn(0x7f6c9c0faf00 msgr2=0x7f6c9c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 -- 172.21.15.22:0/2447640032 shutdown_connections 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 -- 172.21.15.22:0/2447640032 wait complete. 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.508+0000 7f6ca0844640 1 Processor -- start 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 -- start start 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c9c1025a0 0x7f6c9c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9c102f60 0x7f6c9c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c10cfc0 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c10ce40 con 0x7f6c9c1025a0 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c10d140 con 0x7f6c9c102f60 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52508/0 (socket says 172.21.15.22:52508) 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 -- 172.21.15.22:0/1222631484 learned_addr learned my addr 172.21.15.22:0/1222631484 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9ad9d640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9c102f60 0x7f6c9c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c93fff640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c9c1025a0 0x7f6c9c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 -- 172.21.15.22:0/1222631484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9c102f60 msgr2=0x7f6c9c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9c102f60 0x7f6c9c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 -- 172.21.15.22:0/1222631484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c9c1025a0 msgr2=0x7f6c9c197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c9c1025a0 0x7f6c9c197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 -- 172.21.15.22:0/1222631484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c19f280 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c9a59c640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c197660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c02c7b0 tx=0x7f6c8c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c937fe640 1 -- 172.21.15.22:0/1222631484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c041070 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6c937fe640 1 -- 172.21.15.22:0/1222631484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c8c0054e0 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.509+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c19f510 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.510+0000 7f6c937fe640 1 -- 172.21.15.22:0/1222631484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c034440 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.510+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c19fa50 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.510+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c108750 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.512+0000 7f6c937fe640 1 -- 172.21.15.22:0/1222631484 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6c8c044070 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.512+0000 7f6c937fe640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6c880a2880 0x7f6c880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.512+0000 7f6c93fff640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6c880a2880 0x7f6c880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.512+0000 7f6c937fe640 1 -- 172.21.15.22:0/1222631484 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6c8c0f0cd0 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.512+0000 7f6c93fff640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6c880a2880 0x7f6c880a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c84005fd0 tx=0x7f6c84005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:04.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.516+0000 7f6c937fe640 1 -- 172.21.15.22:0/1222631484 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6c8c0bdb30 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.699+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c9c061ef0 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.699+0000 7f6c937fe640 1 -- 172.21.15.22:0/1222631484 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c8c0bd430 con 0x7f6c9c0ff5d0 2024-03-08T23:52:04.696 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:52:04.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6c880a2880 msgr2=0x7f6c880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6c880a2880 0x7f6c880a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c84005fd0 tx=0x7f6c84005850 comp rx=0 tx=0).stop 2024-03-08T23:52:04.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 msgr2=0x7f6c9c197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c197660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c02c7b0 tx=0x7f6c8c005dd0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 shutdown_connections 2024-03-08T23:52:04.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9c102f60 0x7f6c9c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c9c1025a0 0x7f6c9c197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f6c880a2880 0x7f6c880a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 --2- 172.21.15.22:0/1222631484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c9c0ff5d0 0x7f6c9c197660 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 >> 172.21.15.22:0/1222631484 conn(0x7f6c9c0faf00 msgr2=0x7f6c9c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:04.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 shutdown_connections 2024-03-08T23:52:04.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.701+0000 7f6ca0844640 1 -- 172.21.15.22:0/1222631484 wait complete. 2024-03-08T23:52:04.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:52:04.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 -- 172.21.15.22:0/1401010171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c101300 msgr2=0x7f3a7c101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 --2- 172.21.15.22:0/1401010171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c101300 0x7f3a7c101720 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009900 tx=0x7f3a6c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 -- 172.21.15.22:0/1401010171 shutdown_connections 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 --2- 172.21.15.22:0/1401010171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a7c102eb0 0x7f3a7c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 --2- 172.21.15.22:0/1401010171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 --2- 172.21.15.22:0/1401010171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c101300 0x7f3a7c101720 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 -- 172.21.15.22:0/1401010171 >> 172.21.15.22:0/1401010171 conn(0x7f3a7c0fc890 msgr2=0x7f3a7c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 -- 172.21.15.22:0/1401010171 shutdown_connections 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.817+0000 7f3a81c92640 1 -- 172.21.15.22:0/1401010171 wait complete. 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 Processor -- start 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 -- start start 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c101300 0x7f3a7c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a7c102eb0 0x7f3a7c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c10cf60 con 0x7f3a7c1024f0 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c10cde0 con 0x7f3a7c101300 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a81c92640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c10d0e0 con 0x7f3a7c102eb0 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a7affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a7affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52514/0 (socket says 172.21.15.22:52514) 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a7affd640 1 -- 172.21.15.22:0/2412357860 learned_addr learned my addr 172.21.15.22:0/2412357860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.818+0000 7f3a7b7fe640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c101300 0x7f3a7c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a7bfff640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a7c102eb0 0x7f3a7c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a7affd640 1 -- 172.21.15.22:0/2412357860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a7c102eb0 msgr2=0x7f3a7c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a7affd640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a7c102eb0 0x7f3a7c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a7affd640 1 -- 172.21.15.22:0/2412357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c101300 msgr2=0x7f3a7c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a7affd640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c101300 0x7f3a7c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a7affd640 1 -- 172.21.15.22:0/2412357860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c19f2a0 con 0x7f3a7c1024f0 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a7affd640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c197bc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000d950 tx=0x7f3a7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:04.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a78ff9640 1 -- 172.21.15.22:0/2412357860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70017070 con 0x7f3a7c1024f0 2024-03-08T23:52:04.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c19f590 con 0x7f3a7c1024f0 2024-03-08T23:52:04.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a78ff9640 1 -- 172.21.15.22:0/2412357860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a70009c30 con 0x7f3a7c1024f0 2024-03-08T23:52:04.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a78ff9640 1 -- 172.21.15.22:0/2412357860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000f6b0 con 0x7f3a7c1024f0 2024-03-08T23:52:04.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.819+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c19fae0 con 0x7f3a7c1024f0 2024-03-08T23:52:04.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.820+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c108740 con 0x7f3a7c1024f0 2024-03-08T23:52:04.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.821+0000 7f3a78ff9640 1 -- 172.21.15.22:0/2412357860 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3a700109d0 con 0x7f3a7c1024f0 2024-03-08T23:52:04.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.822+0000 7f3a78ff9640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3a640a2800 0x7f3a640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:04.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.822+0000 7f3a7b7fe640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3a640a2800 0x7f3a640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:04.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.822+0000 7f3a78ff9640 1 -- 172.21.15.22:0/2412357860 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3a700ce910 con 0x7f3a7c1024f0 2024-03-08T23:52:04.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.822+0000 7f3a7b7fe640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3a640a2800 0x7f3a640a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:04.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:04.826+0000 7f3a78ff9640 1 -- 172.21.15.22:0/2412357860 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3a70009de0 con 0x7f3a7c1024f0 2024-03-08T23:52:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.007+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a7c061ef0 con 0x7f3a7c1024f0 2024-03-08T23:52:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.007+0000 7f3a78ff9640 1 -- 172.21.15.22:0/2412357860 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a7009b770 con 0x7f3a7c1024f0 2024-03-08T23:52:05.004 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3a640a2800 msgr2=0x7f3a640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3a640a2800 0x7f3a640a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c005fd0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c1024f0 msgr2=0x7f3a7c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c197bc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000d950 tx=0x7f3a7000de10 comp rx=0 tx=0).stop 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 shutdown_connections 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a7c102eb0 0x7f3a7c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3a640a2800 0x7f3a640a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c197bc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 --2- 172.21.15.22:0/2412357860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a7c101300 0x7f3a7c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.009+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 >> 172.21.15.22:0/2412357860 conn(0x7f3a7c0fc890 msgr2=0x7f3a7c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.010+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 shutdown_connections 2024-03-08T23:52:05.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.010+0000 7f3a81c92640 1 -- 172.21.15.22:0/2412357860 wait complete. 2024-03-08T23:52:05.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 -- 172.21.15.22:0/3365633483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c101da0 msgr2=0x7ff26c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 --2- 172.21.15.22:0/3365633483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c101da0 0x7ff26c108630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff264009ca0 tx=0x7ff26402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 -- 172.21.15.22:0/3365633483 shutdown_connections 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 --2- 172.21.15.22:0/3365633483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c101da0 0x7ff26c108630 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 --2- 172.21.15.22:0/3365633483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c1013e0 0x7ff26c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 --2- 172.21.15.22:0/3365633483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff26c1001f0 0x7ff26c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 -- 172.21.15.22:0/3365633483 >> 172.21.15.22:0/3365633483 conn(0x7ff26c0fb780 msgr2=0x7ff26c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 -- 172.21.15.22:0/3365633483 shutdown_connections 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.125+0000 7ff272020640 1 -- 172.21.15.22:0/3365633483 wait complete. 2024-03-08T23:52:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 Processor -- start 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 -- start start 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c1001f0 0x7ff26c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c1013e0 0x7ff26c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff26c101da0 0x7ff26c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10be80 con 0x7ff26c1001f0 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10bd00 con 0x7ff26c101da0 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff272020640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10c000 con 0x7ff26c1013e0 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c1001f0 0x7ff26c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c1001f0 0x7ff26c197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52524/0 (socket says 172.21.15.22:52524) 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.126+0000 7ff26b7fe640 1 -- 172.21.15.22:0/1748636935 learned_addr learned my addr 172.21.15.22:0/1748636935 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26affd640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c1013e0 0x7ff26c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26bfff640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff26c101da0 0x7ff26c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26b7fe640 1 -- 172.21.15.22:0/1748636935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c1013e0 msgr2=0x7ff26c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26b7fe640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c1013e0 0x7ff26c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26b7fe640 1 -- 172.21.15.22:0/1748636935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff26c101da0 msgr2=0x7ff26c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26b7fe640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff26c101da0 0x7ff26c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26b7fe640 1 -- 172.21.15.22:0/1748636935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c19f220 con 0x7ff26c1001f0 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26affd640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c1013e0 0x7ff26c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff26b7fe640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c1001f0 0x7ff26c197600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d770 tx=0x7ff25c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff268ff9640 1 -- 172.21.15.22:0/1748636935 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c017070 con 0x7ff26c1001f0 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff268ff9640 1 -- 172.21.15.22:0/1748636935 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff25c009c30 con 0x7ff26c1001f0 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c19f510 con 0x7ff26c1001f0 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff268ff9640 1 -- 172.21.15.22:0/1748636935 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c003dd0 con 0x7ff26c1001f0 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.127+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c19fab0 con 0x7ff26c1001f0 2024-03-08T23:52:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.128+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c19fd10 con 0x7ff26c1001f0 2024-03-08T23:52:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.129+0000 7ff268ff9640 1 -- 172.21.15.22:0/1748636935 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff25c003fb0 con 0x7ff26c1001f0 2024-03-08T23:52:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.130+0000 7ff268ff9640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2580a2880 0x7ff2580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.130+0000 7ff26affd640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2580a2880 0x7ff2580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.130+0000 7ff268ff9640 1 -- 172.21.15.22:0/1748636935 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff25c0ce670 con 0x7ff26c1001f0 2024-03-08T23:52:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.130+0000 7ff26affd640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2580a2880 0x7ff2580a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff26c198b10 tx=0x7ff254005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:05.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.134+0000 7ff268ff9640 1 -- 172.21.15.22:0/1748636935 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff25c009de0 con 0x7ff26c1001f0 2024-03-08T23:52:05.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.319+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff26c19fee0 con 0x7ff26c1001f0 2024-03-08T23:52:05.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.319+0000 7ff268ff9640 1 -- 172.21.15.22:0/1748636935 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff25c09b4d0 con 0x7ff26c1001f0 2024-03-08T23:52:05.316 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2580a2880 msgr2=0x7ff2580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2580a2880 0x7ff2580a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff26c198b10 tx=0x7ff254005d30 comp rx=0 tx=0).stop 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c1001f0 msgr2=0x7ff26c197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c1001f0 0x7ff26c197600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d770 tx=0x7ff25c00dc30 comp rx=0 tx=0).stop 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 shutdown_connections 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff26c101da0 0x7ff26c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff26c1013e0 0x7ff26c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2580a2880 0x7ff2580a4d40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 --2- 172.21.15.22:0/1748636935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff26c1001f0 0x7ff26c197600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 >> 172.21.15.22:0/1748636935 conn(0x7ff26c0fb780 msgr2=0x7ff26c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:05.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 shutdown_connections 2024-03-08T23:52:05.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.321+0000 7ff272020640 1 -- 172.21.15.22:0/1748636935 wait complete. 2024-03-08T23:52:05.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:52:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 -- 172.21.15.22:0/93304394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 msgr2=0x7f2db0109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/93304394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0109440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009900 tx=0x7f2da802bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 -- 172.21.15.22:0/93304394 shutdown_connections 2024-03-08T23:52:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/93304394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0109440 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/93304394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db01046c0 0x7f2db0106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/93304394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db0101d90 0x7f2db0104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 -- 172.21.15.22:0/93304394 >> 172.21.15.22:0/93304394 conn(0x7f2db00fb7a0 msgr2=0x7f2db00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 -- 172.21.15.22:0/93304394 shutdown_connections 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.437+0000 7f2db5c2b640 1 -- 172.21.15.22:0/93304394 wait complete. 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 Processor -- start 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 -- start start 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db0101d90 0x7f2db0101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db01046c0 0x7f2db00ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db010d000 con 0x7f2db0106ff0 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db010ce80 con 0x7f2db0101d90 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2db5c2b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db010d180 con 0x7f2db01046c0 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2daffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2daffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0100280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52534/0 (socket says 172.21.15.22:52534) 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.438+0000 7f2daffff640 1 -- 172.21.15.22:0/385197765 learned_addr learned my addr 172.21.15.22:0/385197765 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daf7fe640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db0101d90 0x7f2db0101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daeffd640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db01046c0 0x7f2db00ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daffff640 1 -- 172.21.15.22:0/385197765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db01046c0 msgr2=0x7f2db00ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daffff640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db01046c0 0x7f2db00ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daffff640 1 -- 172.21.15.22:0/385197765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db0101d90 msgr2=0x7f2db0101700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daffff640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db0101d90 0x7f2db0101700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daffff640 1 -- 172.21.15.22:0/385197765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db0100b10 con 0x7f2db0106ff0 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2daffff640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0100280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2da8005ab0 tx=0x7f2da8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2dacff9640 1 -- 172.21.15.22:0/385197765 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8041070 con 0x7f2db0106ff0 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db01a1710 con 0x7f2db0106ff0 2024-03-08T23:52:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2dacff9640 1 -- 172.21.15.22:0/385197765 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2da80054e0 con 0x7f2db0106ff0 2024-03-08T23:52:05.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2dacff9640 1 -- 172.21.15.22:0/385197765 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80346a0 con 0x7f2db0106ff0 2024-03-08T23:52:05.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.439+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db01a1cb0 con 0x7f2db0106ff0 2024-03-08T23:52:05.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.440+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db00fd290 con 0x7f2db0106ff0 2024-03-08T23:52:05.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.441+0000 7f2dacff9640 1 -- 172.21.15.22:0/385197765 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2da8044070 con 0x7f2db0106ff0 2024-03-08T23:52:05.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.442+0000 7f2dacff9640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2d980a2800 0x7f2d980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.442+0000 7f2daf7fe640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2d980a2800 0x7f2d980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.442+0000 7f2dacff9640 1 -- 172.21.15.22:0/385197765 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2da80f1170 con 0x7f2db0106ff0 2024-03-08T23:52:05.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.442+0000 7f2daf7fe640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2d980a2800 0x7f2d980a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009f20 tx=0x7f2da00094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:05.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.445+0000 7f2dacff9640 1 -- 172.21.15.22:0/385197765 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2da80bdfd0 con 0x7f2db0106ff0 2024-03-08T23:52:05.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:05 smithi062 ceph-mon[36129]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-08T23:52:05.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:05 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3436137545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:05.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:05 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1222631484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:52:05.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:05 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2412357860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:52:05.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.628+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2db002ce70 con 0x7f2db0106ff0 2024-03-08T23:52:05.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.629+0000 7f2dacff9640 1 -- 172.21.15.22:0/385197765 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2da80bd8d0 con 0x7f2db0106ff0 2024-03-08T23:52:05.625 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:52:05.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2d980a2800 msgr2=0x7f2d980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2d980a2800 0x7f2d980a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009f20 tx=0x7f2da00094f0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 msgr2=0x7f2db0100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0100280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2da8005ab0 tx=0x7f2da8004270 comp rx=0 tx=0).stop 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 shutdown_connections 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2d980a2800 0x7f2d980a4cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2db0106ff0 0x7f2db0100280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db01046c0 0x7f2db00ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 --2- 172.21.15.22:0/385197765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2db0101d90 0x7f2db0101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 >> 172.21.15.22:0/385197765 conn(0x7f2db00fb7a0 msgr2=0x7f2db0102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 shutdown_connections 2024-03-08T23:52:05.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.631+0000 7f2db5c2b640 1 -- 172.21.15.22:0/385197765 wait complete. 2024-03-08T23:52:05.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:05 smithi192 ceph-mon[35995]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-08T23:52:05.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:05 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3436137545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:05.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:05 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1222631484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:52:05.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:05 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2412357860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:52:05.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:05 smithi022 ceph-mon[31894]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-08T23:52:05.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:05 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3436137545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:05.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:05 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1222631484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:52:05.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:05 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2412357860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:52:05.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T23:52:05.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 -- 172.21.15.22:0/846632581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc101da0 msgr2=0x7ff1dc108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/846632581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc101da0 0x7ff1dc108630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009ca0 tx=0x7ff1d402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 -- 172.21.15.22:0/846632581 shutdown_connections 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/846632581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc101da0 0x7ff1dc108630 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/846632581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1dc1013e0 0x7ff1dc101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/846632581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1dc1001f0 0x7ff1dc100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 -- 172.21.15.22:0/846632581 >> 172.21.15.22:0/846632581 conn(0x7ff1dc0fb780 msgr2=0x7ff1dc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 -- 172.21.15.22:0/846632581 shutdown_connections 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.745+0000 7ff1e1f53640 1 -- 172.21.15.22:0/846632581 wait complete. 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 Processor -- start 2024-03-08T23:52:05.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 -- start start 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1dc1001f0 0x7ff1dc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc1013e0 0x7ff1dc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1dc101da0 0x7ff1dc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc10cf90 con 0x7ff1dc1013e0 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc10ce10 con 0x7ff1dc101da0 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1e1f53640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc10d110 con 0x7ff1dc1001f0 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1daffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc1013e0 0x7ff1dc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1daffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc1013e0 0x7ff1dc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52552/0 (socket says 172.21.15.22:52552) 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1daffd640 1 -- 172.21.15.22:0/3465363512 learned_addr learned my addr 172.21.15.22:0/3465363512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1dbfff640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1dc101da0 0x7ff1dc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.746+0000 7ff1db7fe640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1dc1001f0 0x7ff1dc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1daffd640 1 -- 172.21.15.22:0/3465363512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1dc1001f0 msgr2=0x7ff1dc1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1daffd640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1dc1001f0 0x7ff1dc1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1daffd640 1 -- 172.21.15.22:0/3465363512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1dc101da0 msgr2=0x7ff1dc19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1daffd640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1dc101da0 0x7ff1dc19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1daffd640 1 -- 172.21.15.22:0/3465363512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc19f200 con 0x7ff1dc1013e0 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1daffd640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc1013e0 0x7ff1dc197b20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000d950 tx=0x7ff1d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1d8ff9640 1 -- 172.21.15.22:0/3465363512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0017070 con 0x7ff1dc1013e0 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1d8ff9640 1 -- 172.21.15.22:0/3465363512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1d0009c30 con 0x7ff1dc1013e0 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc19f4f0 con 0x7ff1dc1013e0 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1d8ff9640 1 -- 172.21.15.22:0/3465363512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000f6b0 con 0x7ff1dc1013e0 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.747+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc19fa40 con 0x7ff1dc1013e0 2024-03-08T23:52:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.748+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc107510 con 0x7ff1dc1013e0 2024-03-08T23:52:05.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.749+0000 7ff1d8ff9640 1 -- 172.21.15.22:0/3465363512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff1d00109d0 con 0x7ff1dc1013e0 2024-03-08T23:52:05.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.750+0000 7ff1d8ff9640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1c40a2800 0x7ff1c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:05.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.750+0000 7ff1db7fe640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1c40a2800 0x7ff1c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:05.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.750+0000 7ff1d8ff9640 1 -- 172.21.15.22:0/3465363512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff1d00ce900 con 0x7ff1dc1013e0 2024-03-08T23:52:05.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.750+0000 7ff1db7fe640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1c40a2800 0x7ff1c40a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc005fd0 tx=0x7ff1cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:05.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.754+0000 7ff1d8ff9640 1 -- 172.21.15.22:0/3465363512 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff1d0009de0 con 0x7ff1dc1013e0 2024-03-08T23:52:05.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.949+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff1dc061ef0 con 0x7ff1dc1013e0 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.950+0000 7ff1d8ff9640 1 -- 172.21.15.22:0/3465363512 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+590 (secure 0 0 0) 0x7ff1d009b0b0 con 0x7ff1dc1013e0 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3201080518 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3384641534 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/99443044 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1968010400 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3762004902 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3384641534 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:05.947 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/4028157579 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:05.948 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/4028157579 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:05.948 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1133673382 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:05.948 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2337825839 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:listed 10 entries 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1c40a2800 msgr2=0x7ff1c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1c40a2800 0x7ff1c40a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc005fd0 tx=0x7ff1cc00b390 comp rx=0 tx=0).stop 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc1013e0 msgr2=0x7ff1dc197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc1013e0 0x7ff1dc197b20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000d950 tx=0x7ff1d000de10 comp rx=0 tx=0).stop 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 shutdown_connections 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1dc101da0 0x7ff1dc19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1c40a2800 0x7ff1c40a4cc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1dc1013e0 0x7ff1dc197b20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 --2- 172.21.15.22:0/3465363512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1dc1001f0 0x7ff1dc1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:05.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 >> 172.21.15.22:0/3465363512 conn(0x7ff1dc0fb780 msgr2=0x7ff1dc1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:05.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 shutdown_connections 2024-03-08T23:52:05.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:05.952+0000 7ff1e1f53640 1 -- 172.21.15.22:0/3465363512 wait complete. 2024-03-08T23:52:05.959 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:52:05.959 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T23:52:05.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 -- 172.21.15.22:0/948902253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368100290 msgr2=0x7f13681006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 --2- 172.21.15.22:0/948902253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368100290 0x7f13681006b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1358009900 tx=0x7f135802bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 -- 172.21.15.22:0/948902253 shutdown_connections 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 --2- 172.21.15.22:0/948902253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1368101e40 0x7f13681086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 --2- 172.21.15.22:0/948902253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1368101480 0x7f1368101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 --2- 172.21.15.22:0/948902253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368100290 0x7f13681006b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 -- 172.21.15.22:0/948902253 >> 172.21.15.22:0/948902253 conn(0x7f13680fb820 msgr2=0x7f13680fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 -- 172.21.15.22:0/948902253 shutdown_connections 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.121+0000 7f136c91b640 1 -- 172.21.15.22:0/948902253 wait complete. 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 Processor -- start 2024-03-08T23:52:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 -- start start 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1368100290 0x7f1368197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368101480 0x7f1368197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1368101e40 0x7f136819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136810beb0 con 0x7f1368101480 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136810bd30 con 0x7f1368100290 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136c91b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136810c030 con 0x7f1368101e40 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f135ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368101480 0x7f1368197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f135ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368101480 0x7f1368197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52568/0 (socket says 172.21.15.22:52568) 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f135ffff640 1 -- 172.21.15.22:0/3837099625 learned_addr learned my addr 172.21.15.22:0/3837099625 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f136659c640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1368100290 0x7f1368197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f135ffff640 1 -- 172.21.15.22:0/3837099625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1368101e40 msgr2=0x7f136819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f135ffff640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1368101e40 0x7f136819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.122+0000 7f135ffff640 1 -- 172.21.15.22:0/3837099625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1368100290 msgr2=0x7f1368197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f135ffff640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1368100290 0x7f1368197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f135ffff640 1 -- 172.21.15.22:0/3837099625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136819f280 con 0x7f1368101480 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f136659c640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1368100290 0x7f1368197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f135ffff640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368101480 0x7f1368197ba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f135000d950 tx=0x7f135000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f135f7fe640 1 -- 172.21.15.22:0/3837099625 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1350017070 con 0x7f1368101480 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136819f570 con 0x7f1368101480 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f135f7fe640 1 -- 172.21.15.22:0/3837099625 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1350009c30 con 0x7f1368101480 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f135f7fe640 1 -- 172.21.15.22:0/3837099625 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135000f6b0 con 0x7f1368101480 2024-03-08T23:52:06.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.123+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136819fac0 con 0x7f1368101480 2024-03-08T23:52:06.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.124+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136819fd20 con 0x7f1368101480 2024-03-08T23:52:06.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.125+0000 7f135f7fe640 1 -- 172.21.15.22:0/3837099625 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f13500109d0 con 0x7f1368101480 2024-03-08T23:52:06.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.125+0000 7f135f7fe640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13540a2800 0x7f13540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.126+0000 7f136659c640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13540a2800 0x7f13540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.126+0000 7f135f7fe640 1 -- 172.21.15.22:0/3837099625 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f13500cea60 con 0x7f1368101480 2024-03-08T23:52:06.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.126+0000 7f136659c640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13540a2800 0x7f13540a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1358036040 tx=0x7f1358000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:06.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.130+0000 7f135f7fe640 1 -- 172.21.15.22:0/3837099625 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1350009de0 con 0x7f1368101480 2024-03-08T23:52:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.318+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13681a00b0 con 0x7f1368101480 2024-03-08T23:52:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.319+0000 7f135f7fe640 1 -- 172.21.15.22:0/3837099625 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7f135009b860 con 0x7f1368101480 2024-03-08T23:52:06.316 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:06.316 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13540a2800 msgr2=0x7f13540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13540a2800 0x7f13540a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1358036040 tx=0x7f1358000bc0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368101480 msgr2=0x7f1368197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368101480 0x7f1368197ba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f135000d950 tx=0x7f135000de10 comp rx=0 tx=0).stop 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 shutdown_connections 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1368101e40 0x7f136819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13540a2800 0x7f13540a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1368101480 0x7f1368197ba0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 --2- 172.21.15.22:0/3837099625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1368100290 0x7f1368197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 >> 172.21.15.22:0/3837099625 conn(0x7f13680fb820 msgr2=0x7f1368069920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 shutdown_connections 2024-03-08T23:52:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.322+0000 7f136c91b640 1 -- 172.21.15.22:0/3837099625 wait complete. 2024-03-08T23:52:06.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:52:06.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:52:06.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 -- 172.21.15.22:0/3811752472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 msgr2=0x7f481c102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 --2- 172.21.15.22:0/3811752472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c102970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f480c009900 tx=0x7f480c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:06.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 -- 172.21.15.22:0/3811752472 shutdown_connections 2024-03-08T23:52:06.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 --2- 172.21.15.22:0/3811752472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f481c102eb0 0x7f481c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 --2- 172.21.15.22:0/3811752472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c102970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 --2- 172.21.15.22:0/3811752472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c101300 0x7f481c101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 -- 172.21.15.22:0/3811752472 >> 172.21.15.22:0/3811752472 conn(0x7f481c0fc890 msgr2=0x7f481c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 -- 172.21.15.22:0/3811752472 shutdown_connections 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.438+0000 7f4823fab640 1 -- 172.21.15.22:0/3811752472 wait complete. 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 Processor -- start 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 -- start start 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c101300 0x7f481c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f481c102eb0 0x7f481c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c10cf90 con 0x7f481c1024f0 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c10ce10 con 0x7f481c101300 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4823fab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c10d110 con 0x7f481c102eb0 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52590/0 (socket says 172.21.15.22:52590) 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 -- 172.21.15.22:0/3674029025 learned_addr learned my addr 172.21.15.22:0/3674029025 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821d47640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c101300 0x7f481c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4822548640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f481c102eb0 0x7f481c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 -- 172.21.15.22:0/3674029025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f481c102eb0 msgr2=0x7f481c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f481c102eb0 0x7f481c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 -- 172.21.15.22:0/3674029025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c101300 msgr2=0x7f481c197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c101300 0x7f481c197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.439+0000 7f4821546640 1 -- 172.21.15.22:0/3674029025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c19f230 con 0x7f481c1024f0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.440+0000 7f4822548640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f481c102eb0 0x7f481c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.440+0000 7f4821546640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c197b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f480c009230 tx=0x7f480c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.440+0000 7f4812ffd640 1 -- 172.21.15.22:0/3674029025 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c041070 con 0x7f481c1024f0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.440+0000 7f4812ffd640 1 -- 172.21.15.22:0/3674029025 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f480c0054e0 con 0x7f481c1024f0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.440+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c19f4c0 con 0x7f481c1024f0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.440+0000 7f4812ffd640 1 -- 172.21.15.22:0/3674029025 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c034440 con 0x7f481c1024f0 2024-03-08T23:52:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.440+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c19f9b0 con 0x7f481c1024f0 2024-03-08T23:52:06.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.441+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c108620 con 0x7f481c1024f0 2024-03-08T23:52:06.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.442+0000 7f4812ffd640 1 -- 172.21.15.22:0/3674029025 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f480c044070 con 0x7f481c1024f0 2024-03-08T23:52:06.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.442+0000 7f4812ffd640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48080a2800 0x7f48080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.442+0000 7f4812ffd640 1 -- 172.21.15.22:0/3674029025 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f480c0f09a0 con 0x7f481c1024f0 2024-03-08T23:52:06.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.442+0000 7f4821d47640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48080a2800 0x7f48080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.443+0000 7f4821d47640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48080a2800 0x7f48080a4cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4818005fd0 tx=0x7f481800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:06.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.447+0000 7f4812ffd640 1 -- 172.21.15.22:0/3674029025 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f480c0bd800 con 0x7f481c1024f0 2024-03-08T23:52:06.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:06 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1748636935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:52:06.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:06 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/385197765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:52:06.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:06 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3465363512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T23:52:06.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.629+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f481c061ef0 con 0x7f481c1024f0 2024-03-08T23:52:06.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.630+0000 7f4812ffd640 1 -- 172.21.15.22:0/3674029025 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f480c0bd100 con 0x7f481c1024f0 2024-03-08T23:52:06.626 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:52:06.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48080a2800 msgr2=0x7f48080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48080a2800 0x7f48080a4cc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4818005fd0 tx=0x7f481800b410 comp rx=0 tx=0).stop 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 msgr2=0x7f481c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c197b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f480c009230 tx=0x7f480c005dd0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 shutdown_connections 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f481c102eb0 0x7f481c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f48080a2800 0x7f48080a4cc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f481c1024f0 0x7f481c197b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 --2- 172.21.15.22:0/3674029025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c101300 0x7f481c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 >> 172.21.15.22:0/3674029025 conn(0x7f481c0fc890 msgr2=0x7f481c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 shutdown_connections 2024-03-08T23:52:06.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.632+0000 7f4823fab640 1 -- 172.21.15.22:0/3674029025 wait complete. 2024-03-08T23:52:06.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:06 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1748636935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:52:06.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:06 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/385197765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:52:06.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:06 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3465363512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T23:52:06.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:06 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1748636935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:52:06.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:06 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/385197765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:52:06.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:06 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3465363512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T23:52:06.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:52:06.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 -- 172.21.15.22:0/2756387371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d0073cd0 msgr2=0x7f56d0074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 --2- 172.21.15.22:0/2756387371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d0073cd0 0x7f56d0074130 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009900 tx=0x7f56bc02bcd0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 -- 172.21.15.22:0/2756387371 shutdown_connections 2024-03-08T23:52:06.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 --2- 172.21.15.22:0/2756387371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d01091d0 0x7f56d010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 --2- 172.21.15.22:0/2756387371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d0073cd0 0x7f56d0074130 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 --2- 172.21.15.22:0/2756387371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d00752b0 0x7f56d0073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 -- 172.21.15.22:0/2756387371 >> 172.21.15.22:0/2756387371 conn(0x7f56d00fc890 msgr2=0x7f56d00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 -- 172.21.15.22:0/2756387371 shutdown_connections 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.749+0000 7f56d7227640 1 -- 172.21.15.22:0/2756387371 wait complete. 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 Processor -- start 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 -- start start 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d0073cd0 0x7f56d0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00752b0 0x7f56d0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d01091d0 0x7f56d01a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d010f1c0 con 0x7f56d01091d0 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d010f040 con 0x7f56d00752b0 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d7227640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d010f340 con 0x7f56d0073cd0 2024-03-08T23:52:06.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d01091d0 0x7f56d01a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d01091d0 0x7f56d01a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52614/0 (socket says 172.21.15.22:52614) 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 -- 172.21.15.22:0/2153329886 learned_addr learned my addr 172.21.15.22:0/2153329886 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56cffff640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00752b0 0x7f56d0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d4fc3640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d0073cd0 0x7f56d0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 -- 172.21.15.22:0/2153329886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d0073cd0 msgr2=0x7f56d0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d0073cd0 0x7f56d0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 -- 172.21.15.22:0/2153329886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00752b0 msgr2=0x7f56d0199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00752b0 0x7f56d0199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.750+0000 7f56d57c4640 1 -- 172.21.15.22:0/2153329886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d01a1430 con 0x7f56d01091d0 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.751+0000 7f56d57c4640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d01091d0 0x7f56d01a0dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f56c800eb10 tx=0x7f56c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.751+0000 7f56cdffb640 1 -- 172.21.15.22:0/2153329886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8004b20 con 0x7f56d01091d0 2024-03-08T23:52:06.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.751+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d01a1720 con 0x7f56d01091d0 2024-03-08T23:52:06.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.751+0000 7f56cdffb640 1 -- 172.21.15.22:0/2153329886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56c8004cc0 con 0x7f56d01091d0 2024-03-08T23:52:06.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.751+0000 7f56cdffb640 1 -- 172.21.15.22:0/2153329886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8010760 con 0x7f56d01091d0 2024-03-08T23:52:06.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.751+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d01a1bf0 con 0x7f56d01091d0 2024-03-08T23:52:06.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.752+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d0107f90 con 0x7f56d01091d0 2024-03-08T23:52:06.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.753+0000 7f56cdffb640 1 -- 172.21.15.22:0/2153329886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f56c8005020 con 0x7f56d01091d0 2024-03-08T23:52:06.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.753+0000 7f56cdffb640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f56c00a2800 0x7f56c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:06.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.753+0000 7f56cdffb640 1 -- 172.21.15.22:0/2153329886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f56c80cf210 con 0x7f56d01091d0 2024-03-08T23:52:06.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.753+0000 7f56d4fc3640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f56c00a2800 0x7f56c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:06.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.754+0000 7f56d4fc3640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f56c00a2800 0x7f56c00a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005fd0 tx=0x7f56c400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:06.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.758+0000 7f56cdffb640 1 -- 172.21.15.22:0/2153329886 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f56c809c070 con 0x7f56d01091d0 2024-03-08T23:52:06.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.940+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56d0061ef0 con 0x7f56d01091d0 2024-03-08T23:52:06.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.940+0000 7f56cdffb640 1 -- 172.21.15.22:0/2153329886 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56c809b970 con 0x7f56d01091d0 2024-03-08T23:52:06.937 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f56c00a2800 msgr2=0x7f56c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f56c00a2800 0x7f56c00a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005fd0 tx=0x7f56c400b390 comp rx=0 tx=0).stop 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d01091d0 msgr2=0x7f56d01a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d01091d0 0x7f56d01a0dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f56c800eb10 tx=0x7f56c800c2d0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 shutdown_connections 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f56c00a2800 0x7f56c00a4cc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56d01091d0 0x7f56d01a0dc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00752b0 0x7f56d0199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 --2- 172.21.15.22:0/2153329886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d0073cd0 0x7f56d0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 >> 172.21.15.22:0/2153329886 conn(0x7f56d00fc890 msgr2=0x7f56d0105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:06.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 shutdown_connections 2024-03-08T23:52:06.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:06.942+0000 7f56d7227640 1 -- 172.21.15.22:0/2153329886 wait complete. 2024-03-08T23:52:06.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:52:07.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 -- 172.21.15.22:0/2989346630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010101310 msgr2=0x7f0010101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 --2- 172.21.15.22:0/2989346630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010101310 0x7f0010101730 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0000009900 tx=0x7f000002bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 -- 172.21.15.22:0/2989346630 shutdown_connections 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 --2- 172.21.15.22:0/2989346630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010102ec0 0x7f0010109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 --2- 172.21.15.22:0/2989346630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0010102500 0x7f0010102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 --2- 172.21.15.22:0/2989346630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010101310 0x7f0010101730 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 -- 172.21.15.22:0/2989346630 >> 172.21.15.22:0/2989346630 conn(0x7f00100fc880 msgr2=0x7f00100fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 -- 172.21.15.22:0/2989346630 shutdown_connections 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 -- 172.21.15.22:0/2989346630 wait complete. 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 Processor -- start 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.060+0000 7f001586c640 1 -- start start 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010101310 0x7f0010197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010102500 0x7f0010197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0010102ec0 0x7f001019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001010cfa0 con 0x7f0010102500 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001010ce20 con 0x7f0010101310 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001010d120 con 0x7f0010102ec0 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010102500 0x7f0010197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010102500 0x7f0010197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52640/0 (socket says 172.21.15.22:52640) 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 -- 172.21.15.22:0/5532224 learned_addr learned my addr 172.21.15.22:0/5532224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f000effd640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010101310 0x7f0010197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f000f7fe640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0010102ec0 0x7f001019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 -- 172.21.15.22:0/5532224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0010102ec0 msgr2=0x7f001019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0010102ec0 0x7f001019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 -- 172.21.15.22:0/5532224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010101310 msgr2=0x7f0010197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010101310 0x7f0010197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 -- 172.21.15.22:0/5532224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001019f220 con 0x7f0010102500 2024-03-08T23:52:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f0007fff640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010102500 0x7f0010197b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efff800d770 tx=0x7efff800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001486a640 1 -- 172.21.15.22:0/5532224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8017070 con 0x7f0010102500 2024-03-08T23:52:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001486a640 1 -- 172.21.15.22:0/5532224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efff8005500 con 0x7f0010102500 2024-03-08T23:52:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001019f510 con 0x7f0010102500 2024-03-08T23:52:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001486a640 1 -- 172.21.15.22:0/5532224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8004e00 con 0x7f0010102500 2024-03-08T23:52:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.061+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001019fa10 con 0x7f0010102500 2024-03-08T23:52:07.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.063+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0010108630 con 0x7f0010102500 2024-03-08T23:52:07.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.064+0000 7f001486a640 1 -- 172.21.15.22:0/5532224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7efff8004fa0 con 0x7f0010102500 2024-03-08T23:52:07.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.064+0000 7f001486a640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efffc0a2800 0x7efffc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.064+0000 7f000effd640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efffc0a2800 0x7efffc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.064+0000 7f001486a640 1 -- 172.21.15.22:0/5532224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efff80ce630 con 0x7f0010102500 2024-03-08T23:52:07.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.065+0000 7f000effd640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efffc0a2800 0x7efffc0a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f0000005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:07.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.068+0000 7f001486a640 1 -- 172.21.15.22:0/5532224 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7efff809ae40 con 0x7f0010102500 2024-03-08T23:52:07.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.252+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0010061ef0 con 0x7f0010102500 2024-03-08T23:52:07.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.253+0000 7f001486a640 1 -- 172.21.15.22:0/5532224 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7efff809eee0 con 0x7f0010102500 2024-03-08T23:52:07.249 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:52:07.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efffc0a2800 msgr2=0x7efffc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efffc0a2800 0x7efffc0a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f0000005fd0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010102500 msgr2=0x7f0010197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010102500 0x7f0010197b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efff800d770 tx=0x7efff800dc30 comp rx=0 tx=0).stop 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 shutdown_connections 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0010102ec0 0x7f001019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7efffc0a2800 0x7efffc0a4cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0010102500 0x7f0010197b40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 --2- 172.21.15.22:0/5532224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010101310 0x7f0010197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 >> 172.21.15.22:0/5532224 conn(0x7f00100fc880 msgr2=0x7f0010105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 shutdown_connections 2024-03-08T23:52:07.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.255+0000 7f001586c640 1 -- 172.21.15.22:0/5532224 wait complete. 2024-03-08T23:52:07.262 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:52:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 -- 172.21.15.22:0/2047125188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c1024f0 msgr2=0x7ff78c102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 --2- 172.21.15.22:0/2047125188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c1024f0 0x7ff78c102970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff774009900 tx=0x7ff77402bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 -- 172.21.15.22:0/2047125188 shutdown_connections 2024-03-08T23:52:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 --2- 172.21.15.22:0/2047125188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff78c102eb0 0x7ff78c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 --2- 172.21.15.22:0/2047125188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c1024f0 0x7ff78c102970 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 --2- 172.21.15.22:0/2047125188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c101300 0x7ff78c101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 -- 172.21.15.22:0/2047125188 >> 172.21.15.22:0/2047125188 conn(0x7ff78c0fc890 msgr2=0x7ff78c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 -- 172.21.15.22:0/2047125188 shutdown_connections 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.371+0000 7ff791162640 1 -- 172.21.15.22:0/2047125188 wait complete. 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 Processor -- start 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 -- start start 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c101300 0x7ff78c0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff78c1024f0 0x7ff78c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c102eb0 0x7ff78c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c10cf90 con 0x7ff78c102eb0 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c10ce10 con 0x7ff78c1024f0 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff791162640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c10d110 con 0x7ff78c101300 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff78b59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c102eb0 0x7ff78c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff78b59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c102eb0 0x7ff78c073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52648/0 (socket says 172.21.15.22:52648) 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff78b59e640 1 -- 172.21.15.22:0/683451463 learned_addr learned my addr 172.21.15.22:0/683451463 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:07.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff78ad9d640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c101300 0x7ff78c0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.372+0000 7ff783fff640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff78c1024f0 0x7ff78c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff78b59e640 1 -- 172.21.15.22:0/683451463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c101300 msgr2=0x7ff78c0750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff78b59e640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c101300 0x7ff78c0750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff78b59e640 1 -- 172.21.15.22:0/683451463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff78c1024f0 msgr2=0x7ff78c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff78b59e640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff78c1024f0 0x7ff78c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff78b59e640 1 -- 172.21.15.22:0/683451463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0744b0 con 0x7ff78c102eb0 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff783fff640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff78c1024f0 0x7ff78c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff78ad9d640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c101300 0x7ff78c0750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff78b59e640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c102eb0 0x7ff78c073c20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff784009fd0 tx=0x7ff78400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff788d99640 1 -- 172.21.15.22:0/683451463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784017070 con 0x7ff78c102eb0 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff788d99640 1 -- 172.21.15.22:0/683451463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff784003e50 con 0x7ff78c102eb0 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c1a16e0 con 0x7ff78c102eb0 2024-03-08T23:52:07.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff788d99640 1 -- 172.21.15.22:0/683451463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7840106c0 con 0x7ff78c102eb0 2024-03-08T23:52:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.373+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c1a1be0 con 0x7ff78c102eb0 2024-03-08T23:52:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.374+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c1086a0 con 0x7ff78c102eb0 2024-03-08T23:52:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.375+0000 7ff788d99640 1 -- 172.21.15.22:0/683451463 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff784003970 con 0x7ff78c102eb0 2024-03-08T23:52:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.376+0000 7ff788d99640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff7780a2800 0x7ff7780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.376+0000 7ff78ad9d640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff7780a2800 0x7ff7780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.376+0000 7ff788d99640 1 -- 172.21.15.22:0/683451463 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff7840cea80 con 0x7ff78c102eb0 2024-03-08T23:52:07.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.376+0000 7ff78ad9d640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff7780a2800 0x7ff7780a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff78c102350 tx=0x7ff77c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:07.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.379+0000 7ff788d99640 1 -- 172.21.15.22:0/683451463 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff78409b880 con 0x7ff78c102eb0 2024-03-08T23:52:07.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.561+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff78c02ce70 con 0x7ff78c102eb0 2024-03-08T23:52:07.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.562+0000 7ff788d99640 1 -- 172.21.15.22:0/683451463 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff78409b180 con 0x7ff78c102eb0 2024-03-08T23:52:07.558 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:52:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff7780a2800 msgr2=0x7ff7780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff7780a2800 0x7ff7780a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff78c102350 tx=0x7ff77c005db0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c102eb0 msgr2=0x7ff78c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c102eb0 0x7ff78c073c20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff784009fd0 tx=0x7ff78400b260 comp rx=0 tx=0).stop 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 shutdown_connections 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff7780a2800 0x7ff7780a4cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff78c102eb0 0x7ff78c073c20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff78c1024f0 0x7ff78c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 --2- 172.21.15.22:0/683451463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c101300 0x7ff78c0750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 >> 172.21.15.22:0/683451463 conn(0x7ff78c0fc890 msgr2=0x7ff78c105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 shutdown_connections 2024-03-08T23:52:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.564+0000 7ff791162640 1 -- 172.21.15.22:0/683451463 wait complete. 2024-03-08T23:52:07.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:07 smithi022 ceph-mon[31894]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2024-03-08T23:52:07.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:07 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3837099625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:07.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:07 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3674029025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:07 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2153329886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:07 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/5532224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T23:52:07.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:07 smithi062 ceph-mon[36129]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2024-03-08T23:52:07.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:07 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3837099625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:07.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:07 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3674029025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:07 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2153329886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:07 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/5532224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:07 smithi192 ceph-mon[35995]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 1 op/s 2024-03-08T23:52:07.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:07 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3837099625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:07.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:07 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3674029025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:07 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2153329886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:07 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/5532224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:52:07.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 -- 172.21.15.22:0/3985652049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c102800 msgr2=0x7f496c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 --2- 172.21.15.22:0/3985652049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c102800 0x7f496c102c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f495c009900 tx=0x7f495c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:07.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 -- 172.21.15.22:0/3985652049 shutdown_connections 2024-03-08T23:52:07.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 --2- 172.21.15.22:0/3985652049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c0ffc30 0x7f496c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 --2- 172.21.15.22:0/3985652049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c0ff270 0x7f496c0ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 --2- 172.21.15.22:0/3985652049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c102800 0x7f496c102c20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 -- 172.21.15.22:0/3985652049 >> 172.21.15.22:0/3985652049 conn(0x7f496c0fadf0 msgr2=0x7f496c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 -- 172.21.15.22:0/3985652049 shutdown_connections 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.678+0000 7f497099d640 1 -- 172.21.15.22:0/3985652049 wait complete. 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 Processor -- start 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 -- start start 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ff270 0x7f496c101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c0ffc30 0x7f496c100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c102800 0x7f496c100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c10cee0 con 0x7f496c0ff270 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c10cd60 con 0x7f496c0ffc30 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f497099d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c10d060 con 0x7f496c102800 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f496b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ff270 0x7f496c101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f496b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ff270 0x7f496c101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52664/0 (socket says 172.21.15.22:52664) 2024-03-08T23:52:07.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f496b7fe640 1 -- 172.21.15.22:0/3429739226 learned_addr learned my addr 172.21.15.22:0/3429739226 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.679+0000 7f496bfff640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c102800 0x7f496c100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496affd640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c0ffc30 0x7f496c100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496b7fe640 1 -- 172.21.15.22:0/3429739226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c102800 msgr2=0x7f496c100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496b7fe640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c102800 0x7f496c100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496b7fe640 1 -- 172.21.15.22:0/3429739226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c0ffc30 msgr2=0x7f496c100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496b7fe640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c0ffc30 0x7f496c100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496b7fe640 1 -- 172.21.15.22:0/3429739226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c101310 con 0x7f496c0ff270 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496affd640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c0ffc30 0x7f496c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f496b7fe640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ff270 0x7f496c101f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f495c005ab0 tx=0x7f495c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f4968ff9640 1 -- 172.21.15.22:0/3429739226 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c041070 con 0x7f496c0ff270 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f4968ff9640 1 -- 172.21.15.22:0/3429739226 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f495c0054e0 con 0x7f496c0ff270 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c1a1590 con 0x7f496c0ff270 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f4968ff9640 1 -- 172.21.15.22:0/3429739226 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c034440 con 0x7f496c0ff270 2024-03-08T23:52:07.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.680+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c1a1ae0 con 0x7f496c0ff270 2024-03-08T23:52:07.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.681+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496c1086c0 con 0x7f496c0ff270 2024-03-08T23:52:07.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.682+0000 7f4968ff9640 1 -- 172.21.15.22:0/3429739226 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f495c044070 con 0x7f496c0ff270 2024-03-08T23:52:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.683+0000 7f4968ff9640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f49540a2800 0x7f49540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.683+0000 7f496affd640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f49540a2800 0x7f49540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.683+0000 7f4968ff9640 1 -- 172.21.15.22:0/3429739226 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f495c0f0910 con 0x7f496c0ff270 2024-03-08T23:52:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.683+0000 7f496affd640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f49540a2800 0x7f49540a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f496c101c70 tx=0x7f4960005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.687+0000 7f4968ff9640 1 -- 172.21.15.22:0/3429739226 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f495c0bd770 con 0x7f496c0ff270 2024-03-08T23:52:07.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.884+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f496c02d0b0 con 0x7f496c0ff270 2024-03-08T23:52:07.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.885+0000 7f4968ff9640 1 -- 172.21.15.22:0/3429739226 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+590 (secure 0 0 0) 0x7f495c0bd070 con 0x7f496c0ff270 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3201080518 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3384641534 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/99443044 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1968010400 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3762004902 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3384641534 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/4028157579 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/4028157579 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1133673382 2024-03-09T23:44:16.515766+0000 2024-03-08T23:52:07.882 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2337825839 2024-03-09T23:44:43.705970+0000 2024-03-08T23:52:07.883 INFO:teuthology.orchestra.run.smithi022.stderr:listed 10 entries 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f49540a2800 msgr2=0x7f49540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f49540a2800 0x7f49540a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f496c101c70 tx=0x7f4960005d30 comp rx=0 tx=0).stop 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ff270 msgr2=0x7f496c101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ff270 0x7f496c101f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f495c005ab0 tx=0x7f495c005dc0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 shutdown_connections 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c102800 0x7f496c100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c0ffc30 0x7f496c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f49540a2800 0x7f49540a4cc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 --2- 172.21.15.22:0/3429739226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ff270 0x7f496c101f00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:07.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 >> 172.21.15.22:0/3429739226 conn(0x7f496c0fadf0 msgr2=0x7f496c105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:07.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 shutdown_connections 2024-03-08T23:52:07.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:07.887+0000 7f497099d640 1 -- 172.21.15.22:0/3429739226 wait complete. 2024-03-08T23:52:07.894 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2024-03-08T23:52:07.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:07.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-08T23:52:07.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[75438]: fuse finished with error 0 and tester_r 0 2024-03-08T23:52:07.921 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T23:52:08.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:08 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/683451463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:52:08.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:08 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3429739226' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T23:52:08.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:08 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/683451463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:52:08.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:08 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3429739226' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T23:52:08.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/683451463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:52:08.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3429739226' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T23:52:09.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:09 smithi062 ceph-mon[36129]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-03-08T23:52:09.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:09 smithi192 ceph-mon[35995]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-03-08T23:52:09.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:09 smithi022 ceph-mon[31894]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-03-08T23:52:11.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:11 smithi062 ceph-mon[36129]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:11.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:11 smithi192 ceph-mon[35995]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:11.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:11 smithi022 ceph-mon[31894]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:13.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:13 smithi062 ceph-mon[36129]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:13.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:13 smithi192 ceph-mon[35995]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:13.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:13 smithi022 ceph-mon[31894]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:14.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-03-08T23:52:14.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:52:14.023 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-08T23:52:14.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/2912767577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 msgr2=0x7f0a94101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/2912767577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94101880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009900 tx=0x7f0a8802bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/2912767577 shutdown_connections 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/2912767577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a94101dc0 0x7f0a94108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/2912767577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94101880 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/2912767577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a94100210 0x7f0a94100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/2912767577 >> 172.21.15.22:0/2912767577 conn(0x7f0a940fb780 msgr2=0x7f0a940fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/2912767577 shutdown_connections 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.216+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/2912767577 wait complete. 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 Processor -- start 2024-03-08T23:52:14.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 -- start start 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a94100210 0x7f0a94197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a94101dc0 0x7f0a9419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9410cfb0 con 0x7f0a94101400 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9410ce30 con 0x7f0a94100210 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9bdf1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9410d130 con 0x7f0a94101dc0 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41698/0 (socket says 172.21.15.22:41698) 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 -- 172.21.15.22:0/925710176 learned_addr learned my addr 172.21.15.22:0/925710176 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a99b8d640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a94100210 0x7f0a94197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9a38e640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a94101dc0 0x7f0a9419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 -- 172.21.15.22:0/925710176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a94101dc0 msgr2=0x7f0a9419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a94101dc0 0x7f0a9419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 -- 172.21.15.22:0/925710176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a94100210 msgr2=0x7f0a94197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a94100210 0x7f0a94197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 -- 172.21.15.22:0/925710176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9419f250 con 0x7f0a94101400 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.217+0000 7f0a9938c640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94197b70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009230 tx=0x7f0a88005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.218+0000 7f0a92ffd640 1 -- 172.21.15.22:0/925710176 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88041070 con 0x7f0a94101400 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.218+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9419f4e0 con 0x7f0a94101400 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.218+0000 7f0a92ffd640 1 -- 172.21.15.22:0/925710176 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a880044d0 con 0x7f0a94101400 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.218+0000 7f0a92ffd640 1 -- 172.21.15.22:0/925710176 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88034440 con 0x7f0a94101400 2024-03-08T23:52:14.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.218+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9419f9d0 con 0x7f0a94101400 2024-03-08T23:52:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.219+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94107530 con 0x7f0a94101400 2024-03-08T23:52:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.220+0000 7f0a92ffd640 1 -- 172.21.15.22:0/925710176 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0a88044070 con 0x7f0a94101400 2024-03-08T23:52:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.220+0000 7f0a92ffd640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.221+0000 7f0a99b8d640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.221+0000 7f0a92ffd640 1 -- 172.21.15.22:0/925710176 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0a880f0930 con 0x7f0a94101400 2024-03-08T23:52:14.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.221+0000 7f0a99b8d640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a94101260 tx=0x7f0a8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:14.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.225+0000 7f0a92ffd640 1 -- 172.21.15.22:0/925710176 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0a880bd790 con 0x7f0a94101400 2024-03-08T23:52:14.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.410+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a94061ef0 con 0x7f0a94101400 2024-03-08T23:52:14.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.412+0000 7f0a92ffd640 1 -- 172.21.15.22:0/925710176 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7f0a880bd090 con 0x7f0a94101400 2024-03-08T23:52:14.408 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:14.409 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a7c0a2800 msgr2=0x7f0a7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a94101260 tx=0x7f0a8c005f90 comp rx=0 tx=0).stop 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 msgr2=0x7f0a94197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94197b70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009230 tx=0x7f0a88005d50 comp rx=0 tx=0).stop 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 shutdown_connections 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a94101dc0 0x7f0a9419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a7c0a2800 0x7f0a7c0a4cc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a94101400 0x7f0a94197b70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 --2- 172.21.15.22:0/925710176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a94100210 0x7f0a94197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 >> 172.21.15.22:0/925710176 conn(0x7f0a940fb780 msgr2=0x7f0a94104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 shutdown_connections 2024-03-08T23:52:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.414+0000 7f0a9bdf1640 1 -- 172.21.15.22:0/925710176 wait complete. 2024-03-08T23:52:14.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 -- 172.21.15.22:0/3590863787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101dc0 msgr2=0x7fca9c108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3590863787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101dc0 0x7fca9c108650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fca94009ca0 tx=0x7fca9402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 -- 172.21.15.22:0/3590863787 shutdown_connections 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3590863787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101dc0 0x7fca9c108650 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3590863787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca9c101400 0x7fca9c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3590863787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c100210 0x7fca9c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 -- 172.21.15.22:0/3590863787 >> 172.21.15.22:0/3590863787 conn(0x7fca9c0fb780 msgr2=0x7fca9c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:14.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 -- 172.21.15.22:0/3590863787 shutdown_connections 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.528+0000 7fcaa440c640 1 -- 172.21.15.22:0/3590863787 wait complete. 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 Processor -- start 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 -- start start 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca9c100210 0x7fca9c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101400 0x7fca9c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c101dc0 0x7fca9c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c10be70 con 0x7fca9c101400 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c10bcf0 con 0x7fca9c100210 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa440c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c10bff0 con 0x7fca9c101dc0 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101400 0x7fca9c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101400 0x7fca9c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41716/0 (socket says 172.21.15.22:41716) 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 -- 172.21.15.22:0/3152068349 learned_addr learned my addr 172.21.15.22:0/3152068349 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa21a8640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca9c100210 0x7fca9c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa29a9640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c101dc0 0x7fca9c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 -- 172.21.15.22:0/3152068349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c101dc0 msgr2=0x7fca9c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c101dc0 0x7fca9c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 -- 172.21.15.22:0/3152068349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca9c100210 msgr2=0x7fca9c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca9c100210 0x7fca9c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.529+0000 7fcaa19a7640 1 -- 172.21.15.22:0/3152068349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c19f1e0 con 0x7fca9c101400 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.530+0000 7fcaa19a7640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101400 0x7fca9c197b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e9f0 tx=0x7fca8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.530+0000 7fca937fe640 1 -- 172.21.15.22:0/3152068349 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00cd80 con 0x7fca9c101400 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.530+0000 7fca937fe640 1 -- 172.21.15.22:0/3152068349 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca8c010910 con 0x7fca9c101400 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.530+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c19f4d0 con 0x7fca9c101400 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.530+0000 7fca937fe640 1 -- 172.21.15.22:0/3152068349 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c018990 con 0x7fca9c101400 2024-03-08T23:52:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.530+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c19fa70 con 0x7fca9c101400 2024-03-08T23:52:14.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.531+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c19fcd0 con 0x7fca9c101400 2024-03-08T23:52:14.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.532+0000 7fca937fe640 1 -- 172.21.15.22:0/3152068349 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fca8c010ac0 con 0x7fca9c101400 2024-03-08T23:52:14.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.532+0000 7fca937fe640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca880a2880 0x7fca880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.533+0000 7fcaa21a8640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca880a2880 0x7fca880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.533+0000 7fca937fe640 1 -- 172.21.15.22:0/3152068349 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fca8c0cefd0 con 0x7fca9c101400 2024-03-08T23:52:14.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.533+0000 7fcaa21a8640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca880a2880 0x7fca880a4d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca98005fd0 tx=0x7fca9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:14.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.537+0000 7fca937fe640 1 -- 172.21.15.22:0/3152068349 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fca8c09bdd0 con 0x7fca9c101400 2024-03-08T23:52:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.722+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca9c1a0440 con 0x7fca9c101400 2024-03-08T23:52:14.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.724+0000 7fca937fe640 1 -- 172.21.15.22:0/3152068349 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7fca8c09b6d0 con 0x7fca9c101400 2024-03-08T23:52:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:14.721 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca880a2880 msgr2=0x7fca880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca880a2880 0x7fca880a4d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca98005fd0 tx=0x7fca9800b410 comp rx=0 tx=0).stop 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101400 msgr2=0x7fca9c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101400 0x7fca9c197b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e9f0 tx=0x7fca8c00eeb0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 shutdown_connections 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c101dc0 0x7fca9c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca880a2880 0x7fca880a4d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca9c101400 0x7fca9c197b00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 --2- 172.21.15.22:0/3152068349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca9c100210 0x7fca9c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 >> 172.21.15.22:0/3152068349 conn(0x7fca9c0fb780 msgr2=0x7fca9c068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 shutdown_connections 2024-03-08T23:52:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.726+0000 7fcaa440c640 1 -- 172.21.15.22:0/3152068349 wait complete. 2024-03-08T23:52:14.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T23:52:14.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 -- 172.21.15.22:0/1524633929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 msgr2=0x7f0db0104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1524633929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0104180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009900 tx=0x7f0da002bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 -- 172.21.15.22:0/1524633929 shutdown_connections 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1524633929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0db0106ff0 0x7f0db0109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1524633929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0db01046c0 0x7f0db0106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1524633929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0104180 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 -- 172.21.15.22:0/1524633929 >> 172.21.15.22:0/1524633929 conn(0x7f0db00fb7a0 msgr2=0x7f0db00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 -- 172.21.15.22:0/1524633929 shutdown_connections 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 -- 172.21.15.22:0/1524633929 wait complete. 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.843+0000 7f0db5b59640 1 Processor -- start 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0db5b59640 1 -- start start 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0db5b59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0db5b59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0db01046c0 0x7f0db0197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0db5b59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0db0106ff0 0x7f0db019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0db5b59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db010cfa0 con 0x7f0db0101d90 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0db5b59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db010ce20 con 0x7f0db0106ff0 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0db5b59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db010d120 con 0x7f0db01046c0 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41736/0 (socket says 172.21.15.22:41736) 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 -- 172.21.15.22:0/1326914008 learned_addr learned my addr 172.21.15.22:0/1326914008 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daffff640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0db0106ff0 0x7f0db019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daeffd640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0db01046c0 0x7f0db0197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 -- 172.21.15.22:0/1326914008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0db01046c0 msgr2=0x7f0db0197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0db01046c0 0x7f0db0197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 -- 172.21.15.22:0/1326914008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0db0106ff0 msgr2=0x7f0db019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0db0106ff0 0x7f0db019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:14.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 -- 172.21.15.22:0/1326914008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db019f240 con 0x7f0db0101d90 2024-03-08T23:52:14.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daffff640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0db0106ff0 0x7f0db019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:52:14.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0daf7fe640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0197620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009230 tx=0x7f0da0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:14.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.844+0000 7f0dacff9640 1 -- 172.21.15.22:0/1326914008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0041070 con 0x7f0db0101d90 2024-03-08T23:52:14.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.845+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db019f4d0 con 0x7f0db0101d90 2024-03-08T23:52:14.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.845+0000 7f0dacff9640 1 -- 172.21.15.22:0/1326914008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0da00044d0 con 0x7f0db0101d90 2024-03-08T23:52:14.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.845+0000 7f0dacff9640 1 -- 172.21.15.22:0/1326914008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0034440 con 0x7f0db0101d90 2024-03-08T23:52:14.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.845+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db019fa10 con 0x7f0db0101d90 2024-03-08T23:52:14.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.846+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db00fd290 con 0x7f0db0101d90 2024-03-08T23:52:14.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.847+0000 7f0dacff9640 1 -- 172.21.15.22:0/1326914008 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0da0044070 con 0x7f0db0101d90 2024-03-08T23:52:14.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.847+0000 7f0dacff9640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0d980a2800 0x7f0d980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:14.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.847+0000 7f0daeffd640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0d980a2800 0x7f0d980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:14.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.848+0000 7f0dacff9640 1 -- 172.21.15.22:0/1326914008 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0da00f0980 con 0x7f0db0101d90 2024-03-08T23:52:14.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.848+0000 7f0daeffd640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0d980a2800 0x7f0d980a4cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005d00 tx=0x7f0da4005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:14.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:14.852+0000 7f0dacff9640 1 -- 172.21.15.22:0/1326914008 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0da00bd7e0 con 0x7f0db0101d90 2024-03-08T23:52:15.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.074+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0db002ce70 con 0x7f0db0101d90 2024-03-08T23:52:15.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.308+0000 7f0dacff9640 1 -- 172.21.15.22:0/1326914008 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0da00bd0e0 con 0x7f0db0101d90 2024-03-08T23:52:15.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0d980a2800 msgr2=0x7f0d980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0d980a2800 0x7f0d980a4cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005d00 tx=0x7f0da4005c50 comp rx=0 tx=0).stop 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 msgr2=0x7f0db0197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0197620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009230 tx=0x7f0da0005d50 comp rx=0 tx=0).stop 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 shutdown_connections 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0db0106ff0 0x7f0db019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0db01046c0 0x7f0db0197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0d980a2800 0x7f0d980a4cc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 --2- 172.21.15.22:0/1326914008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db0101d90 0x7f0db0197620 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 >> 172.21.15.22:0/1326914008 conn(0x7f0db00fb7a0 msgr2=0x7f0db0102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 shutdown_connections 2024-03-08T23:52:15.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.311+0000 7f0db5b59640 1 -- 172.21.15.22:0/1326914008 wait complete. 2024-03-08T23:52:15.318 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-08T23:52:15.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-08T23:52:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 -- 172.21.15.22:0/2701586491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 msgr2=0x7f507c108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 --2- 172.21.15.22:0/2701586491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c108650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5064009900 tx=0x7f506402bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 -- 172.21.15.22:0/2701586491 shutdown_connections 2024-03-08T23:52:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 --2- 172.21.15.22:0/2701586491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c108650 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 --2- 172.21.15.22:0/2701586491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f507c101400 0x7f507c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 --2- 172.21.15.22:0/2701586491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f507c100210 0x7f507c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 -- 172.21.15.22:0/2701586491 >> 172.21.15.22:0/2701586491 conn(0x7f507c0fb780 msgr2=0x7f507c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.427+0000 7f50831e6640 1 -- 172.21.15.22:0/2701586491 shutdown_connections 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 -- 172.21.15.22:0/2701586491 wait complete. 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 Processor -- start 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 -- start start 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f507c100210 0x7f507c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f507c101400 0x7f507c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:15.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c10bee0 con 0x7f507c101dc0 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c10bd60 con 0x7f507c101400 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f50831e6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c10c060 con 0x7f507c100210 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f5081783640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f5081783640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41750/0 (socket says 172.21.15.22:41750) 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f5081783640 1 -- 172.21.15.22:0/4181973044 learned_addr learned my addr 172.21.15.22:0/4181973044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.428+0000 7f507bfff640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f507c101400 0x7f507c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5080f82640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f507c100210 0x7f507c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5081783640 1 -- 172.21.15.22:0/4181973044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f507c100210 msgr2=0x7f507c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5081783640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f507c100210 0x7f507c197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5081783640 1 -- 172.21.15.22:0/4181973044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f507c101400 msgr2=0x7f507c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5081783640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f507c101400 0x7f507c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5081783640 1 -- 172.21.15.22:0/4181973044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c19f2a0 con 0x7f507c101dc0 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5081783640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c19ec30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5064009230 tx=0x7f5064004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5079ffb640 1 -- 172.21.15.22:0/4181973044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064041070 con 0x7f507c101dc0 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c19f530 con 0x7f507c101dc0 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5079ffb640 1 -- 172.21.15.22:0/4181973044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50640054e0 con 0x7f507c101dc0 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f5079ffb640 1 -- 172.21.15.22:0/4181973044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064034440 con 0x7f507c101dc0 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.429+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c19fa70 con 0x7f507c101dc0 2024-03-08T23:52:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.430+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c19fcd0 con 0x7f507c101dc0 2024-03-08T23:52:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.431+0000 7f5079ffb640 1 -- 172.21.15.22:0/4181973044 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5064044070 con 0x7f507c101dc0 2024-03-08T23:52:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.432+0000 7f5079ffb640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f50680a2800 0x7f50680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.432+0000 7f5080f82640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f50680a2800 0x7f50680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.432+0000 7f5079ffb640 1 -- 172.21.15.22:0/4181973044 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f50640f0d60 con 0x7f507c101dc0 2024-03-08T23:52:15.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.433+0000 7f5080f82640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f50680a2800 0x7f50680a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5074005fd0 tx=0x7f5074005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:15.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.436+0000 7f5079ffb640 1 -- 172.21.15.22:0/4181973044 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f50640bdbc0 con 0x7f507c101dc0 2024-03-08T23:52:15.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:15 smithi062 ceph-mon[36129]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:15.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/925710176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:15.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3152068349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:15.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1326914008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-08T23:52:15.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:15 smithi192 ceph-mon[35995]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:15.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/925710176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:15.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3152068349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:15.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1326914008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-08T23:52:15.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:15.658+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f507c1a0060 con 0x7f507c101dc0 2024-03-08T23:52:15.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:15 smithi022 ceph-mon[31894]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:15.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/925710176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:15.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3152068349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:15.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1326914008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-08T23:52:16.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.314+0000 7f5079ffb640 1 -- 172.21.15.22:0/4181973044 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f50640bd4c0 con 0x7f507c101dc0 2024-03-08T23:52:16.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f50680a2800 msgr2=0x7f50680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:16.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f50680a2800 0x7f50680a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5074005fd0 tx=0x7f5074005850 comp rx=0 tx=0).stop 2024-03-08T23:52:16.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 msgr2=0x7f507c19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:16.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c19ec30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5064009230 tx=0x7f5064004ab0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 shutdown_connections 2024-03-08T23:52:16.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f50680a2800 0x7f50680a4cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f507c101dc0 0x7f507c19ec30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f507c101400 0x7f507c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 --2- 172.21.15.22:0/4181973044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f507c100210 0x7f507c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 >> 172.21.15.22:0/4181973044 conn(0x7f507c0fb780 msgr2=0x7f507c0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:16.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 shutdown_connections 2024-03-08T23:52:16.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.318+0000 7f50831e6640 1 -- 172.21.15.22:0/4181973044 wait complete. 2024-03-08T23:52:16.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-08T23:52:16.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 -- 172.21.15.22:0/2690236231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934103030 msgr2=0x7f69341098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:16.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 --2- 172.21.15.22:0/2690236231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934103030 0x7f69341098c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f692c009cd0 tx=0x7f692c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 -- 172.21.15.22:0/2690236231 shutdown_connections 2024-03-08T23:52:16.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 --2- 172.21.15.22:0/2690236231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934103030 0x7f69341098c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 --2- 172.21.15.22:0/2690236231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6934102670 0x7f6934102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 --2- 172.21.15.22:0/2690236231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ff600 0x7f69340ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 -- 172.21.15.22:0/2690236231 >> 172.21.15.22:0/2690236231 conn(0x7f69340faef0 msgr2=0x7f69340fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 -- 172.21.15.22:0/2690236231 shutdown_connections 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.443+0000 7f693b68c640 1 -- 172.21.15.22:0/2690236231 wait complete. 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 Processor -- start 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 -- start start 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69340ff600 0x7f6934197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934102670 0x7f6934197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934103030 0x7f693419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693410cfc0 con 0x7f6934102670 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693410ce40 con 0x7f6934103030 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f693b68c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693410d140 con 0x7f69340ff600 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934102670 0x7f6934197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934102670 0x7f6934197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41764/0 (socket says 172.21.15.22:41764) 2024-03-08T23:52:16.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 -- 172.21.15.22:0/2688049352 learned_addr learned my addr 172.21.15.22:0/2688049352 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6939c29640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934103030 0x7f693419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6939428640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69340ff600 0x7f6934197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 -- 172.21.15.22:0/2688049352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69340ff600 msgr2=0x7f6934197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69340ff600 0x7f6934197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 -- 172.21.15.22:0/2688049352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934103030 msgr2=0x7f693419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934103030 0x7f693419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 -- 172.21.15.22:0/2688049352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693419f160 con 0x7f6934102670 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6939c29640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934103030 0x7f693419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f6938c27640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934102670 0x7f6934197a80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f692400c920 tx=0x7f692400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.444+0000 7f692a7fc640 1 -- 172.21.15.22:0/2688049352 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924007c70 con 0x7f6934102670 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.445+0000 7f692a7fc640 1 -- 172.21.15.22:0/2688049352 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6924007e10 con 0x7f6934102670 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.445+0000 7f692a7fc640 1 -- 172.21.15.22:0/2688049352 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924018890 con 0x7f6934102670 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.445+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693419f450 con 0x7f6934102670 2024-03-08T23:52:16.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.445+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693419f9f0 con 0x7f6934102670 2024-03-08T23:52:16.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.446+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934108740 con 0x7f6934102670 2024-03-08T23:52:16.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.447+0000 7f692a7fc640 1 -- 172.21.15.22:0/2688049352 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6924018a70 con 0x7f6934102670 2024-03-08T23:52:16.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.447+0000 7f692a7fc640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f69200a2880 0x7f69200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:16.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.447+0000 7f6939428640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f69200a2880 0x7f69200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:16.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.447+0000 7f692a7fc640 1 -- 172.21.15.22:0/2688049352 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f69240cee20 con 0x7f6934102670 2024-03-08T23:52:16.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.448+0000 7f6939428640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f69200a2880 0x7f69200a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6930005fd0 tx=0x7f6930005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:16.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.451+0000 7f692a7fc640 1 -- 172.21.15.22:0/2688049352 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6924014030 con 0x7f6934102670 2024-03-08T23:52:16.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:16 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1326914008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-08T23:52:16.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:16 smithi062 ceph-mon[36129]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:52:16.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:16 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4181973044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-08T23:52:16.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:16 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1326914008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-08T23:52:16.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:16 smithi192 ceph-mon[35995]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:52:16.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:16 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4181973044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-08T23:52:16.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:16.671+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f693402ce70 con 0x7f6934102670 2024-03-08T23:52:16.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:16 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1326914008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-08T23:52:16.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:16 smithi022 ceph-mon[31894]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:52:16.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:16 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4181973044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-08T23:52:17.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.323+0000 7f692a7fc640 1 -- 172.21.15.22:0/2688049352 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f692409bc80 con 0x7f6934102670 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f69200a2880 msgr2=0x7f69200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f69200a2880 0x7f69200a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6930005fd0 tx=0x7f6930005850 comp rx=0 tx=0).stop 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934102670 msgr2=0x7f6934197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934102670 0x7f6934197a80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f692400c920 tx=0x7f692400cde0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 shutdown_connections 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934103030 0x7f693419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f69200a2880 0x7f69200a4d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6934102670 0x7f6934197a80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 --2- 172.21.15.22:0/2688049352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69340ff600 0x7f6934197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 >> 172.21.15.22:0/2688049352 conn(0x7f69340faef0 msgr2=0x7f6934105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 shutdown_connections 2024-03-08T23:52:17.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.326+0000 7f693b68c640 1 -- 172.21.15.22:0/2688049352 wait complete. 2024-03-08T23:52:17.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 -- 172.21.15.22:0/3277608548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c101c90 msgr2=0x7fa21c108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 --2- 172.21.15.22:0/3277608548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c101c90 0x7fa21c108520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa214009ca0 tx=0x7fa21402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 -- 172.21.15.22:0/3277608548 shutdown_connections 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 --2- 172.21.15.22:0/3277608548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c101c90 0x7fa21c108520 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 --2- 172.21.15.22:0/3277608548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa21c1012d0 0x7fa21c101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 --2- 172.21.15.22:0/3277608548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa21c1000e0 0x7fa21c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 -- 172.21.15.22:0/3277608548 >> 172.21.15.22:0/3277608548 conn(0x7fa21c0fb670 msgr2=0x7fa21c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:17.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 -- 172.21.15.22:0/3277608548 shutdown_connections 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 -- 172.21.15.22:0/3277608548 wait complete. 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.455+0000 7fa21b59e640 1 Processor -- start 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 -- start start 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa21c1000e0 0x7fa21c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c1012d0 0x7fa21c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa21c101c90 0x7fa21c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c10bd30 con 0x7fa21c1012d0 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c10bbb0 con 0x7fa21c101c90 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c10beb0 con 0x7fa21c1000e0 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c1012d0 0x7fa21c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c1012d0 0x7fa21c19c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41774/0 (socket says 172.21.15.22:41774) 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 -- 172.21.15.22:0/1875765044 learned_addr learned my addr 172.21.15.22:0/1875765044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21ad9d640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa21c101c90 0x7fa21c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:17.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21a59c640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa21c1000e0 0x7fa21c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 -- 172.21.15.22:0/1875765044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa21c1000e0 msgr2=0x7fa21c06a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa21c1000e0 0x7fa21c06a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 -- 172.21.15.22:0/1875765044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa21c101c90 msgr2=0x7fa21c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa21c101c90 0x7fa21c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 -- 172.21.15.22:0/1875765044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c19f060 con 0x7fa21c1012d0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21a59c640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa21c1000e0 0x7fa21c06a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa213fff640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c1012d0 0x7fa21c19c130 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa20400d950 tx=0x7fa20400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa2137fe640 1 -- 172.21.15.22:0/1875765044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204017070 con 0x7fa21c1012d0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa2137fe640 1 -- 172.21.15.22:0/1875765044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa204009c30 con 0x7fa21c1012d0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.456+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c19f2f0 con 0x7fa21c1012d0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.457+0000 7fa2137fe640 1 -- 172.21.15.22:0/1875765044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20400f6b0 con 0x7fa21c1012d0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.457+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c19f890 con 0x7fa21c1012d0 2024-03-08T23:52:17.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.458+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c107330 con 0x7fa21c1012d0 2024-03-08T23:52:17.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.459+0000 7fa2137fe640 1 -- 172.21.15.22:0/1875765044 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa2040109d0 con 0x7fa21c1012d0 2024-03-08T23:52:17.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.459+0000 7fa2137fe640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa2080a2880 0x7fa2080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:17.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.459+0000 7fa21a59c640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa2080a2880 0x7fa2080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:17.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.459+0000 7fa2137fe640 1 -- 172.21.15.22:0/1875765044 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa2040ce860 con 0x7fa21c1012d0 2024-03-08T23:52:17.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.460+0000 7fa21a59c640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa2080a2880 0x7fa2080a4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa21c101130 tx=0x7fa20c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:17.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.464+0000 7fa2137fe640 1 -- 172.21.15.22:0/1875765044 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa204009de0 con 0x7fa21c1012d0 2024-03-08T23:52:17.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:17.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-08T23:52:17.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4181973044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-08T23:52:17.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-08T23:52:17.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:52:17.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-08T23:52:17.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-08T23:52:17.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2688049352' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-08T23:52:17.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-08T23:52:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4181973044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-08T23:52:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-08T23:52:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:52:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-08T23:52:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-08T23:52:17.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2688049352' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-08T23:52:17.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:17.686+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fa21c02ce70 con 0x7fa21c1012d0 2024-03-08T23:52:17.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-08T23:52:17.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-08T23:52:17.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4181973044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-08T23:52:17.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-08T23:52:17.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-08T23:52:17.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-08T23:52:17.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-08T23:52:17.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2688049352' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-08T23:52:18.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.334+0000 7fa2137fe640 1 -- 172.21.15.22:0/1875765044 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fa20409b740 con 0x7fa21c1012d0 2024-03-08T23:52:18.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa2080a2880 msgr2=0x7fa2080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa2080a2880 0x7fa2080a4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa21c101130 tx=0x7fa20c005db0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c1012d0 msgr2=0x7fa21c19c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c1012d0 0x7fa21c19c130 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa20400d950 tx=0x7fa20400de10 comp rx=0 tx=0).stop 2024-03-08T23:52:18.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 shutdown_connections 2024-03-08T23:52:18.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa21c101c90 0x7fa21c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa2080a2880 0x7fa2080a4d40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa21c1012d0 0x7fa21c19c130 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 --2- 172.21.15.22:0/1875765044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa21c1000e0 0x7fa21c06a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 >> 172.21.15.22:0/1875765044 conn(0x7fa21c0fb670 msgr2=0x7fa21c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:18.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 shutdown_connections 2024-03-08T23:52:18.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.337+0000 7fa21b59e640 1 -- 172.21.15.22:0/1875765044 wait complete. 2024-03-08T23:52:18.348 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-08T23:52:18.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-08T23:52:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 -- 172.21.15.22:0/3393704279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258104730 msgr2=0x7fc258106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 --2- 172.21.15.22:0/3393704279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258104730 0x7fc258106b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc244009900 tx=0x7fc24402bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 -- 172.21.15.22:0/3393704279 shutdown_connections 2024-03-08T23:52:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 --2- 172.21.15.22:0/3393704279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc258107060 0x7fc258109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 --2- 172.21.15.22:0/3393704279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258104730 0x7fc258106b20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 --2- 172.21.15.22:0/3393704279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc258101e00 0x7fc2581041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 -- 172.21.15.22:0/3393704279 >> 172.21.15.22:0/3393704279 conn(0x7fc2580fb790 msgr2=0x7fc2580fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 -- 172.21.15.22:0/3393704279 shutdown_connections 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 -- 172.21.15.22:0/3393704279 wait complete. 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 Processor -- start 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.465+0000 7fc25ec80640 1 -- start start 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ec80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258101e00 0x7fc2581976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ec80640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc258104730 0x7fc258197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ec80640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc258107060 0x7fc25819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ec80640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25810d020 con 0x7fc258101e00 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ec80640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25810cea0 con 0x7fc258107060 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ec80640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25810d1a0 con 0x7fc258104730 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258101e00 0x7fc2581976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258101e00 0x7fc2581976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41786/0 (socket says 172.21.15.22:41786) 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 -- 172.21.15.22:0/1411427820 learned_addr learned my addr 172.21.15.22:0/1411427820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25d21d640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc258107060 0x7fc25819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc257fff640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc258104730 0x7fc258197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 -- 172.21.15.22:0/1411427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc258104730 msgr2=0x7fc258197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc258104730 0x7fc258197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 -- 172.21.15.22:0/1411427820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc258107060 msgr2=0x7fc25819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc258107060 0x7fc25819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 -- 172.21.15.22:0/1411427820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25819f2c0 con 0x7fc258101e00 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc257fff640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc258104730 0x7fc258197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25d21d640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc258107060 0x7fc25819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ca1c640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258101e00 0x7fc2581976a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00d950 tx=0x7fc24c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc255ffb640 1 -- 172.21.15.22:0/1411427820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c017070 con 0x7fc258101e00 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc255ffb640 1 -- 172.21.15.22:0/1411427820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc24c009c30 con 0x7fc258101e00 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25819f5b0 con 0x7fc258101e00 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.466+0000 7fc255ffb640 1 -- 172.21.15.22:0/1411427820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00f6b0 con 0x7fc258101e00 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.467+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25819fb00 con 0x7fc258101e00 2024-03-08T23:52:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.468+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2580fd280 con 0x7fc258101e00 2024-03-08T23:52:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.468+0000 7fc255ffb640 1 -- 172.21.15.22:0/1411427820 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc24c0109d0 con 0x7fc258101e00 2024-03-08T23:52:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.469+0000 7fc255ffb640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2480a2800 0x7fc2480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.469+0000 7fc255ffb640 1 -- 172.21.15.22:0/1411427820 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc24c0cea00 con 0x7fc258101e00 2024-03-08T23:52:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.469+0000 7fc257fff640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2480a2800 0x7fc2480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.470+0000 7fc257fff640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2480a2800 0x7fc2480a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2440096c0 tx=0x7fc244005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.474+0000 7fc255ffb640 1 -- 172.21.15.22:0/1411427820 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc24c009de0 con 0x7fc258101e00 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2688049352' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] up:active 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: Cluster is now healthy 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-08T23:52:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1875765044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-08T23:52:18.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2688049352' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] up:active 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: Cluster is now healthy 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-08T23:52:18.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1875765044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-08T23:52:18.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.684+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fc258061730 con 0x7fc2480a2800 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2688049352' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] up:active 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: Cluster is now healthy 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-08T23:52:18.685 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-08T23:52:18.686 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1875765044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-08T23:52:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.711+0000 7fc255ffb640 1 -- 172.21.15.22:0/1411427820 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc258061730 con 0x7fc2480a2800 2024-03-08T23:52:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2480a2800 msgr2=0x7fc2480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2480a2800 0x7fc2480a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2440096c0 tx=0x7fc244005fd0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258101e00 msgr2=0x7fc2581976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258101e00 0x7fc2581976a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00d950 tx=0x7fc24c00de10 comp rx=0 tx=0).stop 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 shutdown_connections 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc258107060 0x7fc25819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc258104730 0x7fc258197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2480a2800 0x7fc2480a4cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 --2- 172.21.15.22:0/1411427820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc258101e00 0x7fc2581976a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 >> 172.21.15.22:0/1411427820 conn(0x7fc2580fb790 msgr2=0x7fc258102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 shutdown_connections 2024-03-08T23:52:18.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.713+0000 7fc25ec80640 1 -- 172.21.15.22:0/1411427820 wait complete. 2024-03-08T23:52:18.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-08T23:52:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.831+0000 7ff305485640 1 -- 172.21.15.22:0/3505242379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 msgr2=0x7ff300102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.831+0000 7ff305485640 1 --2- 172.21.15.22:0/3505242379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300102a30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009930 tx=0x7ff2f402bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.831+0000 7ff305485640 1 -- 172.21.15.22:0/3505242379 shutdown_connections 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.831+0000 7ff305485640 1 --2- 172.21.15.22:0/3505242379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff300102f70 0x7ff300109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.831+0000 7ff305485640 1 --2- 172.21.15.22:0/3505242379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300102a30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.831+0000 7ff305485640 1 --2- 172.21.15.22:0/3505242379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3000ff5e0 0x7ff3000ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.831+0000 7ff305485640 1 -- 172.21.15.22:0/3505242379 >> 172.21.15.22:0/3505242379 conn(0x7ff3000faef0 msgr2=0x7ff3000fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 -- 172.21.15.22:0/3505242379 shutdown_connections 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 -- 172.21.15.22:0/3505242379 wait complete. 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 Processor -- start 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 -- start start 2024-03-08T23:52:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3000ff5e0 0x7ff300197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff300102f70 0x7ff30019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30010cfd0 con 0x7ff3001025b0 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30010ce50 con 0x7ff300102f70 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff305485640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30010d150 con 0x7ff3000ff5e0 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.832+0000 7ff2fe7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41810/0 (socket says 172.21.15.22:41810) 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 -- 172.21.15.22:0/1288332455 learned_addr learned my addr 172.21.15.22:0/1288332455 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2feffd640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3000ff5e0 0x7ff300197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2ff7fe640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff300102f70 0x7ff30019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 -- 172.21.15.22:0/1288332455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3000ff5e0 msgr2=0x7ff300197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3000ff5e0 0x7ff300197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 -- 172.21.15.22:0/1288332455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff300102f70 msgr2=0x7ff30019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff300102f70 0x7ff30019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 -- 172.21.15.22:0/1288332455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30019f270 con 0x7ff3001025b0 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2ff7fe640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff300102f70 0x7ff30019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2fe7fc640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300197b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2e7fff640 1 -- 172.21.15.22:0/1288332455 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4041070 con 0x7ff3001025b0 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30019f500 con 0x7ff3001025b0 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2e7fff640 1 -- 172.21.15.22:0/1288332455 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2f40054e0 con 0x7ff3001025b0 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff2e7fff640 1 -- 172.21.15.22:0/1288332455 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40346a0 con 0x7ff3001025b0 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.833+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30019fa40 con 0x7ff3001025b0 2024-03-08T23:52:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.834+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff300108800 con 0x7ff3001025b0 2024-03-08T23:52:18.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.835+0000 7ff2e7fff640 1 -- 172.21.15.22:0/1288332455 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff2f4044070 con 0x7ff3001025b0 2024-03-08T23:52:18.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.836+0000 7ff2e7fff640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2e80a2800 0x7ff2e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:18.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.836+0000 7ff2feffd640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2e80a2800 0x7ff2e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:18.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.836+0000 7ff2e7fff640 1 -- 172.21.15.22:0/1288332455 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff2f40f0930 con 0x7ff3001025b0 2024-03-08T23:52:18.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.836+0000 7ff2feffd640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2e80a2800 0x7ff2e80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff300069700 tx=0x7ff2f800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:18.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:18.840+0000 7ff2e7fff640 1 -- 172.21.15.22:0/1288332455 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff2f40bd790 con 0x7ff3001025b0 2024-03-08T23:52:18.985 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:52:18.688+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:52:18.985 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:52:18.688+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:52:18.986 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:52:18.688+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:52:18.986 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:52:18.688+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:52:18.986 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:52:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:52:18.688+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:52:19.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.052+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3000617c0 con 0x7ff2e80a2800 2024-03-08T23:52:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.074+0000 7ff2e7fff640 1 -- 172.21.15.22:0/1288332455 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3000617c0 con 0x7ff2e80a2800 2024-03-08T23:52:19.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2e80a2800 msgr2=0x7ff2e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2e80a2800 0x7ff2e80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff300069700 tx=0x7ff2f800b390 comp rx=0 tx=0).stop 2024-03-08T23:52:19.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 msgr2=0x7ff300197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300197b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4005dd0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 shutdown_connections 2024-03-08T23:52:19.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff300102f70 0x7ff30019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff2e80a2800 0x7ff2e80a4cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3001025b0 0x7ff300197b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 --2- 172.21.15.22:0/1288332455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3000ff5e0 0x7ff300197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.076+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 >> 172.21.15.22:0/1288332455 conn(0x7ff3000faef0 msgr2=0x7ff3001059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:19.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.077+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 shutdown_connections 2024-03-08T23:52:19.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.077+0000 7ff305485640 1 -- 172.21.15.22:0/1288332455 wait complete. 2024-03-08T23:52:19.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 -- 172.21.15.22:0/3683151685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe764102ea0 msgr2=0x7fe764109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/3683151685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe764102ea0 0x7fe764109730 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ca0 tx=0x7fe75c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 -- 172.21.15.22:0/3683151685 shutdown_connections 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/3683151685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe764102ea0 0x7fe764109730 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/3683151685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7641024e0 0x7fe764102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/3683151685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7641012f0 0x7fe764101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 -- 172.21.15.22:0/3683151685 >> 172.21.15.22:0/3683151685 conn(0x7fe7640fc880 msgr2=0x7fe7640feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 -- 172.21.15.22:0/3683151685 shutdown_connections 2024-03-08T23:52:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.194+0000 7fe76a2a7640 1 -- 172.21.15.22:0/3683151685 wait complete. 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 Processor -- start 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 -- start start 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7641012f0 0x7fe764197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7641024e0 0x7fe7641979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe764102ea0 0x7fe76419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76410cf80 con 0x7fe7641012f0 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76410ce00 con 0x7fe7641024e0 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe76a2a7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76410d100 con 0x7fe764102ea0 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7641012f0 0x7fe764197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7641012f0 0x7fe764197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41830/0 (socket says 172.21.15.22:41830) 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 -- 172.21.15.22:0/699963678 learned_addr learned my addr 172.21.15.22:0/699963678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe7637fe640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7641024e0 0x7fe7641979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe768844640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe764102ea0 0x7fe76419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 -- 172.21.15.22:0/699963678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe764102ea0 msgr2=0x7fe76419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe764102ea0 0x7fe76419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 -- 172.21.15.22:0/699963678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7641024e0 msgr2=0x7fe7641979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7641024e0 0x7fe7641979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.195+0000 7fe763fff640 1 -- 172.21.15.22:0/699963678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76419f0b0 con 0x7fe7641012f0 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.196+0000 7fe7637fe640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7641024e0 0x7fe7641979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.196+0000 7fe763fff640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7641012f0 0x7fe764197490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c970 tx=0x7fe75400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.196+0000 7fe7617fa640 1 -- 172.21.15.22:0/699963678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754007c10 con 0x7fe7641012f0 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.196+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76419f3a0 con 0x7fe7641012f0 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.196+0000 7fe7617fa640 1 -- 172.21.15.22:0/699963678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe754007db0 con 0x7fe7641012f0 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.196+0000 7fe7617fa640 1 -- 172.21.15.22:0/699963678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400f8a0 con 0x7fe7641012f0 2024-03-08T23:52:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.196+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76419f940 con 0x7fe7641012f0 2024-03-08T23:52:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.197+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe764108540 con 0x7fe7641012f0 2024-03-08T23:52:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.198+0000 7fe7617fa640 1 -- 172.21.15.22:0/699963678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe75400fb00 con 0x7fe7641012f0 2024-03-08T23:52:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.198+0000 7fe7617fa640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe7500a2800 0x7fe7500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.198+0000 7fe7617fa640 1 -- 172.21.15.22:0/699963678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe7540cf070 con 0x7fe7641012f0 2024-03-08T23:52:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.198+0000 7fe7637fe640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe7500a2800 0x7fe7500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.199+0000 7fe7637fe640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe7500a2800 0x7fe7500a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe7641989a0 tx=0x7fe74c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.203+0000 7fe7617fa640 1 -- 172.21.15.22:0/699963678 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe75409bf60 con 0x7fe7641012f0 2024-03-08T23:52:19.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.416+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe764061730 con 0x7fe7500a2800 2024-03-08T23:52:19.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.420+0000 7fe7617fa640 1 -- 172.21.15.22:0/699963678 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fe764061730 con 0x7fe7500a2800 2024-03-08T23:52:19.417 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_0/e0f7248c-9e10-4918-9259-99a5201c2d98 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe7500a2800 msgr2=0x7fe7500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe7500a2800 0x7fe7500a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe7641989a0 tx=0x7fe74c00a5c0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7641012f0 msgr2=0x7fe764197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7641012f0 0x7fe764197490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c970 tx=0x7fe75400ce30 comp rx=0 tx=0).stop 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 shutdown_connections 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe764102ea0 0x7fe76419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7641024e0 0x7fe7641979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe7500a2800 0x7fe7500a4cc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 --2- 172.21.15.22:0/699963678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe7641012f0 0x7fe764197490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 >> 172.21.15.22:0/699963678 conn(0x7fe7640fc880 msgr2=0x7fe764105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:19.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 shutdown_connections 2024-03-08T23:52:19.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.422+0000 7fe76a2a7640 1 -- 172.21.15.22:0/699963678 wait complete. 2024-03-08T23:52:19.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:19 smithi022 ceph-mon[31894]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-08T23:52:19.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1875765044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-08T23:52:19.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:19 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] up:active 2024-03-08T23:52:19.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:19 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-08T23:52:19.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:19 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:52:19.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:19 smithi022 ceph-mon[31894]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:19.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-08T23:52:19.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 -- 172.21.15.22:0/2991537865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101400 msgr2=0x7f1e20101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 --2- 172.21.15.22:0/2991537865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101400 0x7f1e20101880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e1002bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 -- 172.21.15.22:0/2991537865 shutdown_connections 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 --2- 172.21.15.22:0/2991537865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e20101dc0 0x7f1e20108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 --2- 172.21.15.22:0/2991537865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101400 0x7f1e20101880 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 --2- 172.21.15.22:0/2991537865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20100210 0x7f1e20100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 -- 172.21.15.22:0/2991537865 >> 172.21.15.22:0/2991537865 conn(0x7f1e200fb7a0 msgr2=0x7f1e200fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 -- 172.21.15.22:0/2991537865 shutdown_connections 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 -- 172.21.15.22:0/2991537865 wait complete. 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 Processor -- start 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.541+0000 7f1e28505640 1 -- start start 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20100210 0x7f1e20196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e20101400 0x7f1e20196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101dc0 0x7f1e2019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2010bea0 con 0x7f1e20101dc0 2024-03-08T23:52:19.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2010bd20 con 0x7f1e20100210 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2010c020 con 0x7f1e20101400 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101dc0 0x7f1e2019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101dc0 0x7f1e2019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41854/0 (socket says 172.21.15.22:41854) 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 -- 172.21.15.22:0/2260719519 learned_addr learned my addr 172.21.15.22:0/2260719519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e25aa0640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e20101400 0x7f1e20196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e262a1640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20100210 0x7f1e20196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 -- 172.21.15.22:0/2260719519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e20101400 msgr2=0x7f1e20196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e20101400 0x7f1e20196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 -- 172.21.15.22:0/2260719519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20100210 msgr2=0x7f1e20196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20100210 0x7f1e20196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 -- 172.21.15.22:0/2260719519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2019e120 con 0x7f1e20101dc0 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e262a1640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20100210 0x7f1e20196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e26aa2640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101dc0 0x7f1e2019dab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800eaa0 tx=0x7f1e1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:19.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e177fe640 1 -- 172.21.15.22:0/2260719519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800cdb0 con 0x7f1e20101dc0 2024-03-08T23:52:19.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2019e410 con 0x7f1e20101dc0 2024-03-08T23:52:19.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e177fe640 1 -- 172.21.15.22:0/2260719519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e18005020 con 0x7f1e20101dc0 2024-03-08T23:52:19.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e177fe640 1 -- 172.21.15.22:0/2260719519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e180116d0 con 0x7f1e20101dc0 2024-03-08T23:52:19.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.542+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2019e930 con 0x7f1e20101dc0 2024-03-08T23:52:19.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.543+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e20107600 con 0x7f1e20101dc0 2024-03-08T23:52:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.544+0000 7f1e177fe640 1 -- 172.21.15.22:0/2260719519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1e180054d0 con 0x7f1e20101dc0 2024-03-08T23:52:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.545+0000 7f1e177fe640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e0c0a2800 0x7f1e0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.545+0000 7f1e262a1640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e0c0a2800 0x7f1e0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.545+0000 7f1e177fe640 1 -- 172.21.15.22:0/2260719519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1e180cf320 con 0x7f1e20101dc0 2024-03-08T23:52:19.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.545+0000 7f1e262a1640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e0c0a2800 0x7f1e0c0a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e20101260 tx=0x7f1e1c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:19.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.548+0000 7f1e177fe640 1 -- 172.21.15.22:0/2260719519 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1e18016030 con 0x7f1e20101dc0 2024-03-08T23:52:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:19 smithi062 ceph-mon[36129]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-08T23:52:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1875765044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-08T23:52:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:19 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] up:active 2024-03-08T23:52:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:19 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-08T23:52:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:19 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:52:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:19 smithi062 ceph-mon[36129]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:19 smithi192 ceph-mon[35995]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-08T23:52:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1875765044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-08T23:52:19.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:19 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] up:active 2024-03-08T23:52:19.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:19 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-08T23:52:19.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:19 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:52:19.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:19 smithi192 ceph-mon[35995]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:19.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.761+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e20061730 con 0x7f1e0c0a2800 2024-03-08T23:52:19.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.783+0000 7f1e177fe640 1 -- 172.21.15.22:0/2260719519 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1e20061730 con 0x7f1e0c0a2800 2024-03-08T23:52:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e0c0a2800 msgr2=0x7f1e0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e0c0a2800 0x7f1e0c0a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e20101260 tx=0x7f1e1c005e70 comp rx=0 tx=0).stop 2024-03-08T23:52:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101dc0 msgr2=0x7f1e2019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101dc0 0x7f1e2019dab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800eaa0 tx=0x7f1e1800ef60 comp rx=0 tx=0).stop 2024-03-08T23:52:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 shutdown_connections 2024-03-08T23:52:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1e0c0a2800 0x7f1e0c0a4cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e20101dc0 0x7f1e2019dab0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e20101400 0x7f1e20196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 --2- 172.21.15.22:0/2260719519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20100210 0x7f1e20196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.785+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 >> 172.21.15.22:0/2260719519 conn(0x7f1e200fb7a0 msgr2=0x7f1e201047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.786+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 shutdown_connections 2024-03-08T23:52:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.786+0000 7f1e28505640 1 -- 172.21.15.22:0/2260719519 wait complete. 2024-03-08T23:52:19.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-08T23:52:19.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.903+0000 7f17bea85640 1 -- 172.21.15.22:0/2349675253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 msgr2=0x7f17b8100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.903+0000 7f17bea85640 1 --2- 172.21.15.22:0/2349675253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8100610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009ca0 tx=0x7f17b002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- 172.21.15.22:0/2349675253 shutdown_connections 2024-03-08T23:52:19.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 --2- 172.21.15.22:0/2349675253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b8101da0 0x7f17b8108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 --2- 172.21.15.22:0/2349675253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b81013e0 0x7f17b8101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 --2- 172.21.15.22:0/2349675253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8100610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- 172.21.15.22:0/2349675253 >> 172.21.15.22:0/2349675253 conn(0x7f17b80fb760 msgr2=0x7f17b80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- 172.21.15.22:0/2349675253 shutdown_connections 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- 172.21.15.22:0/2349675253 wait complete. 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 Processor -- start 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- start start 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81013e0 0x7f17b81027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b8101da0 0x7f17b8102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b810be50 con 0x7f17b81001f0 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b810bcd0 con 0x7f17b8101da0 2024-03-08T23:52:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bea85640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b810bfd0 con 0x7f17b81013e0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bc821640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bc821640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8104180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41870/0 (socket says 172.21.15.22:41870) 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.904+0000 7f17bc821640 1 -- 172.21.15.22:0/660402676 learned_addr learned my addr 172.21.15.22:0/660402676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17b7fff640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81013e0 0x7f17b81027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bc821640 1 -- 172.21.15.22:0/660402676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81013e0 msgr2=0x7f17b81027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bc821640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81013e0 0x7f17b81027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bc821640 1 -- 172.21.15.22:0/660402676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b8101da0 msgr2=0x7f17b8102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bc821640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b8101da0 0x7f17b8102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bc821640 1 -- 172.21.15.22:0/660402676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8103590 con 0x7f17b81001f0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bc821640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8104180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f17b00324c0 tx=0x7f17b0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17b5ffb640 1 -- 172.21.15.22:0/660402676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0041020 con 0x7f17b81001f0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17b5ffb640 1 -- 172.21.15.22:0/660402676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17b0034cd0 con 0x7f17b81001f0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b81a16e0 con 0x7f17b81001f0 2024-03-08T23:52:19.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17b5ffb640 1 -- 172.21.15.22:0/660402676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0039040 con 0x7f17b81001f0 2024-03-08T23:52:19.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.905+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b81a1c30 con 0x7f17b81001f0 2024-03-08T23:52:19.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.906+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b81a1f50 con 0x7f17b81001f0 2024-03-08T23:52:19.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.907+0000 7f17b5ffb640 1 -- 172.21.15.22:0/660402676 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f17b0044020 con 0x7f17b81001f0 2024-03-08T23:52:19.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.908+0000 7f17b5ffb640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17a00a2800 0x7f17a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:19.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.908+0000 7f17b7fff640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17a00a2800 0x7f17a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:19.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.908+0000 7f17b5ffb640 1 -- 172.21.15.22:0/660402676 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f17b00f0cc0 con 0x7f17b81001f0 2024-03-08T23:52:19.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.908+0000 7f17b7fff640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17a00a2800 0x7f17a00a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f17b8103ef0 tx=0x7f17ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:19.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:19.912+0000 7f17b5ffb640 1 -- 172.21.15.22:0/660402676 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f17b00f7080 con 0x7f17b81001f0 2024-03-08T23:52:20.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.123+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f17b80fd970 con 0x7f17a00a2800 2024-03-08T23:52:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.127+0000 7f17b5ffb640 1 -- 172.21.15.22:0/660402676 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f17b80fd970 con 0x7f17a00a2800 2024-03-08T23:52:20.123 INFO:teuthology.orchestra.run.smithi022.stdout:/volumes/qa/sv_1/d3c2f801-c4f0-4116-844e-51d32882c535 2024-03-08T23:52:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17a00a2800 msgr2=0x7f17a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17a00a2800 0x7f17a00a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f17b8103ef0 tx=0x7f17ac005f90 comp rx=0 tx=0).stop 2024-03-08T23:52:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 msgr2=0x7f17b8104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8104180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f17b00324c0 tx=0x7f17b0032f40 comp rx=0 tx=0).stop 2024-03-08T23:52:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 shutdown_connections 2024-03-08T23:52:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b8101da0 0x7f17b8102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81013e0 0x7f17b81027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17a00a2800 0x7f17a00a4cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 --2- 172.21.15.22:0/660402676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81001f0 0x7f17b8104180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 >> 172.21.15.22:0/660402676 conn(0x7f17b80fb760 msgr2=0x7f17b80fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 shutdown_connections 2024-03-08T23:52:20.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.129+0000 7f17bea85640 1 -- 172.21.15.22:0/660402676 wait complete. 2024-03-08T23:52:20.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:52:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 -- 172.21.15.22:0/4162166892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102510 msgr2=0x7f053c102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 --2- 172.21.15.22:0/4162166892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102510 0x7f053c102990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f052c009900 tx=0x7f052c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 -- 172.21.15.22:0/4162166892 shutdown_connections 2024-03-08T23:52:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 --2- 172.21.15.22:0/4162166892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c102ed0 0x7f053c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 --2- 172.21.15.22:0/4162166892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102510 0x7f053c102990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 --2- 172.21.15.22:0/4162166892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f053c101320 0x7f053c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 -- 172.21.15.22:0/4162166892 >> 172.21.15.22:0/4162166892 conn(0x7f053c0fc8d0 msgr2=0x7f053c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 -- 172.21.15.22:0/4162166892 shutdown_connections 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 -- 172.21.15.22:0/4162166892 wait complete. 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.242+0000 7f0543569640 1 Processor -- start 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 -- start start 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f053c101320 0x7f053c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c102510 0x7f053c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102ed0 0x7f053c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c10cfb0 con 0x7f053c102ed0 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c10ce30 con 0x7f053c101320 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c10d130 con 0x7f053c102510 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102ed0 0x7f053c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102ed0 0x7f053c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41882/0 (socket says 172.21.15.22:41882) 2024-03-08T23:52:20.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 -- 172.21.15.22:0/889740035 learned_addr learned my addr 172.21.15.22:0/889740035 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541305640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f053c101320 0x7f053c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0540b04640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c102510 0x7f053c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 -- 172.21.15.22:0/889740035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c102510 msgr2=0x7f053c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c102510 0x7f053c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 -- 172.21.15.22:0/889740035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f053c101320 msgr2=0x7f053c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f053c101320 0x7f053c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 -- 172.21.15.22:0/889740035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c19f1e0 con 0x7f053c102ed0 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541305640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f053c101320 0x7f053c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0540b04640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c102510 0x7f053c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0541b06640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102ed0 0x7f053c19eb70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f053400cce0 tx=0x7f053400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f05327fc640 1 -- 172.21.15.22:0/889740035 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400f450 con 0x7f053c102ed0 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f05327fc640 1 -- 172.21.15.22:0/889740035 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f053400fad0 con 0x7f053c102ed0 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c19f4d0 con 0x7f053c102ed0 2024-03-08T23:52:20.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f05327fc640 1 -- 172.21.15.22:0/889740035 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534018c10 con 0x7f053c102ed0 2024-03-08T23:52:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.243+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c19f9d0 con 0x7f053c102ed0 2024-03-08T23:52:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.244+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c1085e0 con 0x7f053c102ed0 2024-03-08T23:52:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.245+0000 7f05327fc640 1 -- 172.21.15.22:0/889740035 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f053400f5f0 con 0x7f053c102ed0 2024-03-08T23:52:20.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.246+0000 7f05327fc640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05280a2800 0x7f05280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.246+0000 7f0541305640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05280a2800 0x7f05280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.246+0000 7f05327fc640 1 -- 172.21.15.22:0/889740035 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f05340cf3a0 con 0x7f053c102ed0 2024-03-08T23:52:20.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.246+0000 7f0541305640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05280a2800 0x7f05280a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0538005d00 tx=0x7f0538005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:20.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.251+0000 7f05327fc640 1 -- 172.21.15.22:0/889740035 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0534014030 con 0x7f053c102ed0 2024-03-08T23:52:20.336 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 08 23:52:19 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: 2024-03-08T23:52:19.968+0000 7f61e175e700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-08T23:52:20.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.476+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f053c061ef0 con 0x7f053c102ed0 2024-03-08T23:52:20.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.477+0000 7f05327fc640 1 -- 172.21.15.22:0/889740035 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f053409c200 con 0x7f053c102ed0 2024-03-08T23:52:20.473 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:20.474 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:52:18.337548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:52:20.474 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-03-08T23:52:20.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05280a2800 msgr2=0x7f05280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05280a2800 0x7f05280a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0538005d00 tx=0x7f0538005ef0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102ed0 msgr2=0x7f053c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102ed0 0x7f053c19eb70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f053400cce0 tx=0x7f053400a620 comp rx=0 tx=0).stop 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 shutdown_connections 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f05280a2800 0x7f05280a4cc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f053c102ed0 0x7f053c19eb70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c102510 0x7f053c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 --2- 172.21.15.22:0/889740035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f053c101320 0x7f053c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 >> 172.21.15.22:0/889740035 conn(0x7f053c0fc8d0 msgr2=0x7f053c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:20.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 shutdown_connections 2024-03-08T23:52:20.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.479+0000 7f0543569640 1 -- 172.21.15.22:0/889740035 wait complete. 2024-03-08T23:52:20.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:20 smithi022 ceph-mon[31894]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:20 smithi022 ceph-mon[31894]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:20 smithi022 ceph-mon[31894]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:52:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 -- 172.21.15.22:0/2333814613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102ec0 msgr2=0x7ff95c109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 --2- 172.21.15.22:0/2333814613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102ec0 0x7ff95c109750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009900 tx=0x7ff94c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:52:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 -- 172.21.15.22:0/2333814613 shutdown_connections 2024-03-08T23:52:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 --2- 172.21.15.22:0/2333814613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102ec0 0x7ff95c109750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 --2- 172.21.15.22:0/2333814613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff95c102500 0x7ff95c102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 --2- 172.21.15.22:0/2333814613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff95c101310 0x7ff95c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 -- 172.21.15.22:0/2333814613 >> 172.21.15.22:0/2333814613 conn(0x7ff95c0fc8a0 msgr2=0x7ff95c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 -- 172.21.15.22:0/2333814613 shutdown_connections 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 -- 172.21.15.22:0/2333814613 wait complete. 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 Processor -- start 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.593+0000 7ff962b97640 1 -- start start 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff95c101310 0x7ff95c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102500 0x7ff95c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff95c102ec0 0x7ff95c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c10cfe0 con 0x7ff95c102500 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c10ce60 con 0x7ff95c102ec0 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c10d160 con 0x7ff95c101310 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102500 0x7ff95c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102500 0x7ff95c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45064/0 (socket says 172.21.15.22:45064) 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 -- 172.21.15.22:0/2240445123 learned_addr learned my addr 172.21.15.22:0/2240445123 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff961134640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff95c102ec0 0x7ff95c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff960933640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff95c101310 0x7ff95c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 -- 172.21.15.22:0/2240445123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff95c101310 msgr2=0x7ff95c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff95c101310 0x7ff95c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 -- 172.21.15.22:0/2240445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff95c102ec0 msgr2=0x7ff95c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff95c102ec0 0x7ff95c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 -- 172.21.15.22:0/2240445123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c19f210 con 0x7ff95c102500 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff961134640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff95c102ec0 0x7ff95c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff960933640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff95c101310 0x7ff95c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff95bfff640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102500 0x7ff95c197b30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff95000b4a0 tx=0x7ff95000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff959ffb640 1 -- 172.21.15.22:0/2240445123 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950014070 con 0x7ff95c102500 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff959ffb640 1 -- 172.21.15.22:0/2240445123 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff950010af0 con 0x7ff95c102500 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c19f500 con 0x7ff95c102500 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff959ffb640 1 -- 172.21.15.22:0/2240445123 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950017d10 con 0x7ff95c102500 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.594+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c19fa50 con 0x7ff95c102500 2024-03-08T23:52:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.595+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c108630 con 0x7ff95c102500 2024-03-08T23:52:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.596+0000 7ff959ffb640 1 -- 172.21.15.22:0/2240445123 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff950017070 con 0x7ff95c102500 2024-03-08T23:52:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.597+0000 7ff959ffb640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff9440a2800 0x7ff9440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.597+0000 7ff960933640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff9440a2800 0x7ff9440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.597+0000 7ff959ffb640 1 -- 172.21.15.22:0/2240445123 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff9500ce0c0 con 0x7ff95c102500 2024-03-08T23:52:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.597+0000 7ff960933640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff9440a2800 0x7ff9440a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff95c102360 tx=0x7ff954009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.601+0000 7ff959ffb640 1 -- 172.21.15.22:0/2240445123 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff95009af20 con 0x7ff95c102500 2024-03-08T23:52:20.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:20 smithi062 ceph-mon[36129]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:20 smithi062 ceph-mon[36129]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:20 smithi062 ceph-mon[36129]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:20 smithi192 ceph-mon[35995]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:20 smithi192 ceph-mon[35995]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:20 smithi192 ceph-mon[35995]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:20.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.787+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff95c061ef0 con 0x7ff95c102500 2024-03-08T23:52:20.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.789+0000 7ff959ffb640 1 -- 172.21.15.22:0/2240445123 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7ff95009a820 con 0x7ff95c102500 2024-03-08T23:52:20.785 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:20.786 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff9440a2800 msgr2=0x7ff9440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff9440a2800 0x7ff9440a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff95c102360 tx=0x7ff954009400 comp rx=0 tx=0).stop 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102500 msgr2=0x7ff95c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102500 0x7ff95c197b30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff95000b4a0 tx=0x7ff95000b960 comp rx=0 tx=0).stop 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 shutdown_connections 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff95c102ec0 0x7ff95c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff9440a2800 0x7ff9440a4cc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff95c102500 0x7ff95c197b30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 --2- 172.21.15.22:0/2240445123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff95c101310 0x7ff95c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 >> 172.21.15.22:0/2240445123 conn(0x7ff95c0fc8a0 msgr2=0x7ff95c105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:20.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 shutdown_connections 2024-03-08T23:52:20.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.791+0000 7ff962b97640 1 -- 172.21.15.22:0/2240445123 wait complete. 2024-03-08T23:52:20.799 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T23:52:20.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T23:52:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 -- 172.21.15.22:0/514760042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4102ec0 msgr2=0x7fc5f4109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 --2- 172.21.15.22:0/514760042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4102ec0 0x7fc5f4109750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009ca0 tx=0x7fc5ec02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 -- 172.21.15.22:0/514760042 shutdown_connections 2024-03-08T23:52:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 --2- 172.21.15.22:0/514760042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4102ec0 0x7fc5f4109750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 --2- 172.21.15.22:0/514760042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5f4102500 0x7fc5f4102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 --2- 172.21.15.22:0/514760042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f4101310 0x7fc5f4101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 -- 172.21.15.22:0/514760042 >> 172.21.15.22:0/514760042 conn(0x7fc5f40fc8a0 msgr2=0x7fc5f40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 -- 172.21.15.22:0/514760042 shutdown_connections 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.906+0000 7fc5faf29640 1 -- 172.21.15.22:0/514760042 wait complete. 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 Processor -- start 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 -- start start 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4101310 0x7fc5f41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5f4102500 0x7fc5f4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f4102ec0 0x7fc5f419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f410cfa0 con 0x7fc5f4101310 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f410ce20 con 0x7fc5f4102ec0 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5faf29640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f410d120 con 0x7fc5f4102500 2024-03-08T23:52:20.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4101310 0x7fc5f41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4101310 0x7fc5f41975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45082/0 (socket says 172.21.15.22:45082) 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 -- 172.21.15.22:0/4031531595 learned_addr learned my addr 172.21.15.22:0/4031531595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f94c6640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f4102ec0 0x7fc5f419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f3fff640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5f4102500 0x7fc5f4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 -- 172.21.15.22:0/4031531595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5f4102500 msgr2=0x7fc5f4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5f4102500 0x7fc5f4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 -- 172.21.15.22:0/4031531595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f4102ec0 msgr2=0x7fc5f419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f4102ec0 0x7fc5f419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.907+0000 7fc5f8cc5640 1 -- 172.21.15.22:0/4031531595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f419f1c0 con 0x7fc5f4101310 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.908+0000 7fc5f8cc5640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4101310 0x7fc5f41975a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800e9c0 tx=0x7fc5e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.908+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/4031531595 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800cd80 con 0x7fc5f4101310 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.908+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f419f4b0 con 0x7fc5f4101310 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.908+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/4031531595 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5e80035f0 con 0x7fc5f4101310 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.908+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/4031531595 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8010690 con 0x7fc5f4101310 2024-03-08T23:52:20.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.908+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f419fa50 con 0x7fc5f4101310 2024-03-08T23:52:20.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.909+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f4108630 con 0x7fc5f4101310 2024-03-08T23:52:20.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.910+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/4031531595 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc5e80037a0 con 0x7fc5f4101310 2024-03-08T23:52:20.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.910+0000 7fc5f1ffb640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc5e40a2800 0x7fc5e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:20.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.910+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/4031531595 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc5e80cf100 con 0x7fc5f4101310 2024-03-08T23:52:20.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.911+0000 7fc5f3fff640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc5e40a2800 0x7fc5e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:20.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.911+0000 7fc5f3fff640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc5e40a2800 0x7fc5e40a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005fd0 tx=0x7fc5e0009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:20.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:20.915+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/4031531595 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc5e809bf60 con 0x7fc5f4101310 2024-03-08T23:52:21.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.094+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5f4061730 con 0x7fc5e40a2800 2024-03-08T23:52:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.104+0000 7fc5f1ffb640 1 -- 172.21.15.22:0/4031531595 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340287 (secure 0 0 0) 0x7fc5f4061730 con 0x7fc5e40a2800 2024-03-08T23:52:21.105 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-08T23:52:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc5e40a2800 msgr2=0x7fc5e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc5e40a2800 0x7fc5e40a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005fd0 tx=0x7fc5e0009470 comp rx=0 tx=0).stop 2024-03-08T23:52:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4101310 msgr2=0x7fc5f41975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4101310 0x7fc5f41975a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800e9c0 tx=0x7fc5e800ee80 comp rx=0 tx=0).stop 2024-03-08T23:52:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 shutdown_connections 2024-03-08T23:52:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f4102ec0 0x7fc5f419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5f4102500 0x7fc5f4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc5e40a2800 0x7fc5e40a4cc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 --2- 172.21.15.22:0/4031531595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5f4101310 0x7fc5f41975a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 >> 172.21.15.22:0/4031531595 conn(0x7fc5f40fc8a0 msgr2=0x7fc5f4105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 shutdown_connections 2024-03-08T23:52:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.108+0000 7fc5faf29640 1 -- 172.21.15.22:0/4031531595 wait complete. 2024-03-08T23:52:21.115 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-03-08T23:52:19.792459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616520,"num_objects":48,"num_object_clones":0,"num_object_copies":145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":154,"num_read_kb":155,"num_write":286,"num_write_kb":2994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362608,"kb_used_data":13420,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579216,"statfs":{"total":1151940427776,"available":1151569117184,"internally_reserved":0,"allocated":13742080,"data_stored":7517076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21653,"num_objects":24,"num_object_clones":0,"num_object_copies":73,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":18,"num_read_kb":36,"num_write":58,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003774"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404049+0000","last_change":"2024-03-08T23:51:52.246270+0000","last_active":"2024-03-08T23:51:58.404049+0000","last_peered":"2024-03-08T23:51:58.404049+0000","last_clean":"2024-03-08T23:51:58.404049+0000","last_became_active":"2024-03-08T23:51:52.245321+0000","last_became_peered":"2024-03-08T23:51:52.245321+0000","last_unstale":"2024-03-08T23:51:58.404049+0000","last_undegraded":"2024-03-08T23:51:58.404049+0000","last_fullsized":"2024-03-08T23:51:58.404049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:39.993965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130853+0000","last_change":"2024-03-08T23:51:52.245433+0000","last_active":"2024-03-08T23:51:58.130853+0000","last_peered":"2024-03-08T23:51:58.130853+0000","last_clean":"2024-03-08T23:51:58.130853+0000","last_became_active":"2024-03-08T23:51:52.245106+0000","last_became_peered":"2024-03-08T23:51:52.245106+0000","last_unstale":"2024-03-08T23:51:58.130853+0000","last_undegraded":"2024-03-08T23:51:58.130853+0000","last_fullsized":"2024-03-08T23:51:58.130853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:02:20.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130821+0000","last_change":"2024-03-08T23:51:52.244112+0000","last_active":"2024-03-08T23:51:58.130821+0000","last_peered":"2024-03-08T23:51:58.130821+0000","last_clean":"2024-03-08T23:51:58.130821+0000","last_became_active":"2024-03-08T23:51:52.243783+0000","last_became_peered":"2024-03-08T23:51:52.243783+0000","last_unstale":"2024-03-08T23:51:58.130821+0000","last_undegraded":"2024-03-08T23:51:58.130821+0000","last_fullsized":"2024-03-08T23:51:58.130821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:48:43.719188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378367+0000","last_change":"2024-03-08T23:51:52.249375+0000","last_active":"2024-03-08T23:51:57.378367+0000","last_peered":"2024-03-08T23:51:57.378367+0000","last_clean":"2024-03-08T23:51:57.378367+0000","last_became_active":"2024-03-08T23:51:52.249045+0000","last_became_peered":"2024-03-08T23:51:52.249045+0000","last_unstale":"2024-03-08T23:51:57.378367+0000","last_undegraded":"2024-03-08T23:51:57.378367+0000","last_fullsized":"2024-03-08T23:51:57.378367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:43:33.788427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239576+0000","last_change":"2024-03-08T23:51:52.247091+0000","last_active":"2024-03-08T23:51:58.239576+0000","last_peered":"2024-03-08T23:51:58.239576+0000","last_clean":"2024-03-08T23:51:58.239576+0000","last_became_active":"2024-03-08T23:51:52.246761+0000","last_became_peered":"2024-03-08T23:51:52.246761+0000","last_unstale":"2024-03-08T23:51:58.239576+0000","last_undegraded":"2024-03-08T23:51:58.239576+0000","last_fullsized":"2024-03-08T23:51:58.239576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:46:46.810857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403691+0000","last_change":"2024-03-08T23:51:52.257558+0000","last_active":"2024-03-08T23:51:58.403691+0000","last_peered":"2024-03-08T23:51:58.403691+0000","last_clean":"2024-03-08T23:51:58.403691+0000","last_became_active":"2024-03-08T23:51:52.257116+0000","last_became_peered":"2024-03-08T23:51:52.257116+0000","last_unstale":"2024-03-08T23:51:58.403691+0000","last_undegraded":"2024-03-08T23:51:58.403691+0000","last_fullsized":"2024-03-08T23:51:58.403691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:13:05.325780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263915+0000","last_change":"2024-03-08T23:51:52.253624+0000","last_active":"2024-03-08T23:51:57.263915+0000","last_peered":"2024-03-08T23:51:57.263915+0000","last_clean":"2024-03-08T23:51:57.263915+0000","last_became_active":"2024-03-08T23:51:52.253284+0000","last_became_peered":"2024-03-08T23:51:52.253284+0000","last_unstale":"2024-03-08T23:51:57.263915+0000","last_undegraded":"2024-03-08T23:51:57.263915+0000","last_fullsized":"2024-03-08T23:51:57.263915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:09:33.789502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123350+0000","last_change":"2024-03-08T23:51:52.255321+0000","last_active":"2024-03-08T23:51:58.123350+0000","last_peered":"2024-03-08T23:51:58.123350+0000","last_clean":"2024-03-08T23:51:58.123350+0000","last_became_active":"2024-03-08T23:51:52.254975+0000","last_became_peered":"2024-03-08T23:51:52.254975+0000","last_unstale":"2024-03-08T23:51:58.123350+0000","last_undegraded":"2024-03-08T23:51:58.123350+0000","last_fullsized":"2024-03-08T23:51:58.123350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:02:56.788519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403983+0000","last_change":"2024-03-08T23:51:52.244623+0000","last_active":"2024-03-08T23:51:58.403983+0000","last_peered":"2024-03-08T23:51:58.403983+0000","last_clean":"2024-03-08T23:51:58.403983+0000","last_became_active":"2024-03-08T23:51:52.244162+0000","last_became_peered":"2024-03-08T23:51:52.244162+0000","last_unstale":"2024-03-08T23:51:58.403983+0000","last_undegraded":"2024-03-08T23:51:58.403983+0000","last_fullsized":"2024-03-08T23:51:58.403983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:30:43.050616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404325+0000","last_change":"2024-03-08T23:51:52.258006+0000","last_active":"2024-03-08T23:51:58.404325+0000","last_peered":"2024-03-08T23:51:58.404325+0000","last_clean":"2024-03-08T23:51:58.404325+0000","last_became_active":"2024-03-08T23:51:52.256770+0000","last_became_peered":"2024-03-08T23:51:52.256770+0000","last_unstale":"2024-03-08T23:51:58.404325+0000","last_undegraded":"2024-03-08T23:51:58.404325+0000","last_fullsized":"2024-03-08T23:51:58.404325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:18.584212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264671+0000","last_change":"2024-03-08T23:51:52.260530+0000","last_active":"2024-03-08T23:51:57.264671+0000","last_peered":"2024-03-08T23:51:57.264671+0000","last_clean":"2024-03-08T23:51:57.264671+0000","last_became_active":"2024-03-08T23:51:52.260193+0000","last_became_peered":"2024-03-08T23:51:52.260193+0000","last_unstale":"2024-03-08T23:51:57.264671+0000","last_undegraded":"2024-03-08T23:51:57.264671+0000","last_fullsized":"2024-03-08T23:51:57.264671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:02.549683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378265+0000","last_change":"2024-03-08T23:51:52.248679+0000","last_active":"2024-03-08T23:51:57.378265+0000","last_peered":"2024-03-08T23:51:57.378265+0000","last_clean":"2024-03-08T23:51:57.378265+0000","last_became_active":"2024-03-08T23:51:52.248358+0000","last_became_peered":"2024-03-08T23:51:52.248358+0000","last_unstale":"2024-03-08T23:51:57.378265+0000","last_undegraded":"2024-03-08T23:51:57.378265+0000","last_fullsized":"2024-03-08T23:51:57.378265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:26.680887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651313+0000","last_change":"2024-03-08T23:51:52.231797+0000","last_active":"2024-03-08T23:51:57.651313+0000","last_peered":"2024-03-08T23:51:57.651313+0000","last_clean":"2024-03-08T23:51:57.651313+0000","last_became_active":"2024-03-08T23:51:52.231425+0000","last_became_peered":"2024-03-08T23:51:52.231425+0000","last_unstale":"2024-03-08T23:51:57.651313+0000","last_undegraded":"2024-03-08T23:51:57.651313+0000","last_fullsized":"2024-03-08T23:51:57.651313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:51:30.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130452+0000","last_change":"2024-03-08T23:51:52.245394+0000","last_active":"2024-03-08T23:51:58.130452+0000","last_peered":"2024-03-08T23:51:58.130452+0000","last_clean":"2024-03-08T23:51:58.130452+0000","last_became_active":"2024-03-08T23:51:52.245038+0000","last_became_peered":"2024-03-08T23:51:52.245038+0000","last_unstale":"2024-03-08T23:51:58.130452+0000","last_undegraded":"2024-03-08T23:51:58.130452+0000","last_fullsized":"2024-03-08T23:51:58.130452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:15.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130510+0000","last_change":"2024-03-08T23:51:52.250861+0000","last_active":"2024-03-08T23:51:58.130510+0000","last_peered":"2024-03-08T23:51:58.130510+0000","last_clean":"2024-03-08T23:51:58.130510+0000","last_became_active":"2024-03-08T23:51:52.250548+0000","last_became_peered":"2024-03-08T23:51:52.250548+0000","last_unstale":"2024-03-08T23:51:58.130510+0000","last_undegraded":"2024-03-08T23:51:58.130510+0000","last_fullsized":"2024-03-08T23:51:58.130510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:00.280043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403190+0000","last_change":"2024-03-08T23:51:52.254996+0000","last_active":"2024-03-08T23:51:58.403190+0000","last_peered":"2024-03-08T23:51:58.403190+0000","last_clean":"2024-03-08T23:51:58.403190+0000","last_became_active":"2024-03-08T23:51:52.254634+0000","last_became_peered":"2024-03-08T23:51:52.254634+0000","last_unstale":"2024-03-08T23:51:58.403190+0000","last_undegraded":"2024-03-08T23:51:58.403190+0000","last_fullsized":"2024-03-08T23:51:58.403190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:15.449988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403921+0000","last_change":"2024-03-08T23:51:52.246558+0000","last_active":"2024-03-08T23:51:58.403921+0000","last_peered":"2024-03-08T23:51:58.403921+0000","last_clean":"2024-03-08T23:51:58.403921+0000","last_became_active":"2024-03-08T23:51:52.245741+0000","last_became_peered":"2024-03-08T23:51:52.245741+0000","last_unstale":"2024-03-08T23:51:58.403921+0000","last_undegraded":"2024-03-08T23:51:58.403921+0000","last_fullsized":"2024-03-08T23:51:58.403921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:36.737017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264263+0000","last_change":"2024-03-08T23:51:52.240111+0000","last_active":"2024-03-08T23:51:57.264263+0000","last_peered":"2024-03-08T23:51:57.264263+0000","last_clean":"2024-03-08T23:51:57.264263+0000","last_became_active":"2024-03-08T23:51:52.238961+0000","last_became_peered":"2024-03-08T23:51:52.238961+0000","last_unstale":"2024-03-08T23:51:57.264263+0000","last_undegraded":"2024-03-08T23:51:57.264263+0000","last_fullsized":"2024-03-08T23:51:57.264263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:03:55.313117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265808+0000","last_change":"2024-03-08T23:51:52.245979+0000","last_active":"2024-03-08T23:51:57.265808+0000","last_peered":"2024-03-08T23:51:57.265808+0000","last_clean":"2024-03-08T23:51:57.265808+0000","last_became_active":"2024-03-08T23:51:52.245666+0000","last_became_peered":"2024-03-08T23:51:52.245666+0000","last_unstale":"2024-03-08T23:51:57.265808+0000","last_undegraded":"2024-03-08T23:51:57.265808+0000","last_fullsized":"2024-03-08T23:51:57.265808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:20.789144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265553+0000","last_change":"2024-03-08T23:51:56.288161+0000","last_active":"2024-03-08T23:51:57.265553+0000","last_peered":"2024-03-08T23:51:57.265553+0000","last_clean":"2024-03-08T23:51:57.265553+0000","last_became_active":"2024-03-08T23:51:56.287778+0000","last_became_peered":"2024-03-08T23:51:56.287778+0000","last_unstale":"2024-03-08T23:51:57.265553+0000","last_undegraded":"2024-03-08T23:51:57.265553+0000","last_fullsized":"2024-03-08T23:51:57.265553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:12:59.636228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264583+0000","last_change":"2024-03-08T23:51:51.244911+0000","last_active":"2024-03-08T23:51:57.264583+0000","last_peered":"2024-03-08T23:51:57.264583+0000","last_clean":"2024-03-08T23:51:57.264583+0000","last_became_active":"2024-03-08T23:51:51.244561+0000","last_became_peered":"2024-03-08T23:51:51.244561+0000","last_unstale":"2024-03-08T23:51:57.264583+0000","last_undegraded":"2024-03-08T23:51:57.264583+0000","last_fullsized":"2024-03-08T23:51:57.264583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:58:45.908307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123954+0000","last_change":"2024-03-08T23:51:52.253854+0000","last_active":"2024-03-08T23:51:58.123954+0000","last_peered":"2024-03-08T23:51:58.123954+0000","last_clean":"2024-03-08T23:51:58.123954+0000","last_became_active":"2024-03-08T23:51:52.253538+0000","last_became_peered":"2024-03-08T23:51:52.253538+0000","last_unstale":"2024-03-08T23:51:58.123954+0000","last_undegraded":"2024-03-08T23:51:58.123954+0000","last_fullsized":"2024-03-08T23:51:58.123954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:33.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.377727+0000","last_change":"2024-03-08T23:51:56.283307+0000","last_active":"2024-03-08T23:51:57.377727+0000","last_peered":"2024-03-08T23:51:57.377727+0000","last_clean":"2024-03-08T23:51:57.377727+0000","last_became_active":"2024-03-08T23:51:56.282868+0000","last_became_peered":"2024-03-08T23:51:56.282868+0000","last_unstale":"2024-03-08T23:51:57.377727+0000","last_undegraded":"2024-03-08T23:51:57.377727+0000","last_fullsized":"2024-03-08T23:51:57.377727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:15.027743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.124107+0000","last_change":"2024-03-08T23:51:51.247368+0000","last_active":"2024-03-08T23:51:58.124107+0000","last_peered":"2024-03-08T23:51:58.124107+0000","last_clean":"2024-03-08T23:51:58.124107+0000","last_became_active":"2024-03-08T23:51:51.247023+0000","last_became_peered":"2024-03-08T23:51:51.247023+0000","last_unstale":"2024-03-08T23:51:58.124107+0000","last_undegraded":"2024-03-08T23:51:58.124107+0000","last_fullsized":"2024-03-08T23:51:58.124107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:28.311850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130889+0000","last_change":"2024-03-08T23:51:52.245805+0000","last_active":"2024-03-08T23:51:58.130889+0000","last_peered":"2024-03-08T23:51:58.130889+0000","last_clean":"2024-03-08T23:51:58.130889+0000","last_became_active":"2024-03-08T23:51:52.244455+0000","last_became_peered":"2024-03-08T23:51:52.244455+0000","last_unstale":"2024-03-08T23:51:58.130889+0000","last_undegraded":"2024-03-08T23:51:58.130889+0000","last_fullsized":"2024-03-08T23:51:58.130889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:11:17.809148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.377931+0000","last_change":"2024-03-08T23:51:56.286666+0000","last_active":"2024-03-08T23:51:57.377931+0000","last_peered":"2024-03-08T23:51:57.377931+0000","last_clean":"2024-03-08T23:51:57.377931+0000","last_became_active":"2024-03-08T23:51:56.285968+0000","last_became_peered":"2024-03-08T23:51:56.285968+0000","last_unstale":"2024-03-08T23:51:57.377931+0000","last_undegraded":"2024-03-08T23:51:57.377931+0000","last_fullsized":"2024-03-08T23:51:57.377931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:51.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.240154+0000","last_change":"2024-03-08T23:51:51.247684+0000","last_active":"2024-03-08T23:51:58.240154+0000","last_peered":"2024-03-08T23:51:58.240154+0000","last_clean":"2024-03-08T23:51:58.240154+0000","last_became_active":"2024-03-08T23:51:51.247374+0000","last_became_peered":"2024-03-08T23:51:51.247374+0000","last_unstale":"2024-03-08T23:51:58.240154+0000","last_undegraded":"2024-03-08T23:51:58.240154+0000","last_fullsized":"2024-03-08T23:51:58.240154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:20.425302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403526+0000","last_change":"2024-03-08T23:51:52.248702+0000","last_active":"2024-03-08T23:51:58.403526+0000","last_peered":"2024-03-08T23:51:58.403526+0000","last_clean":"2024-03-08T23:51:58.403526+0000","last_became_active":"2024-03-08T23:51:52.248388+0000","last_became_peered":"2024-03-08T23:51:52.248388+0000","last_unstale":"2024-03-08T23:51:58.403526+0000","last_undegraded":"2024-03-08T23:51:58.403526+0000","last_fullsized":"2024-03-08T23:51:58.403526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:27:12.251957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.377883+0000","last_change":"2024-03-08T23:51:56.286572+0000","last_active":"2024-03-08T23:51:57.377883+0000","last_peered":"2024-03-08T23:51:57.377883+0000","last_clean":"2024-03-08T23:51:57.377883+0000","last_became_active":"2024-03-08T23:51:56.286066+0000","last_became_peered":"2024-03-08T23:51:56.286066+0000","last_unstale":"2024-03-08T23:51:57.377883+0000","last_undegraded":"2024-03-08T23:51:57.377883+0000","last_fullsized":"2024-03-08T23:51:57.377883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:13.694615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239846+0000","last_change":"2024-03-08T23:51:51.244207+0000","last_active":"2024-03-08T23:51:58.239846+0000","last_peered":"2024-03-08T23:51:58.239846+0000","last_clean":"2024-03-08T23:51:58.239846+0000","last_became_active":"2024-03-08T23:51:51.241605+0000","last_became_peered":"2024-03-08T23:51:51.241605+0000","last_unstale":"2024-03-08T23:51:58.239846+0000","last_undegraded":"2024-03-08T23:51:58.239846+0000","last_fullsized":"2024-03-08T23:51:58.239846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:03:51.766681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278595+0000","last_change":"2024-03-08T23:51:52.251015+0000","last_active":"2024-03-08T23:51:57.278595+0000","last_peered":"2024-03-08T23:51:57.278595+0000","last_clean":"2024-03-08T23:51:57.278595+0000","last_became_active":"2024-03-08T23:51:52.250690+0000","last_became_peered":"2024-03-08T23:51:52.250690+0000","last_unstale":"2024-03-08T23:51:57.278595+0000","last_undegraded":"2024-03-08T23:51:57.278595+0000","last_fullsized":"2024-03-08T23:51:57.278595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:57:23.033172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265749+0000","last_change":"2024-03-08T23:51:56.281066+0000","last_active":"2024-03-08T23:51:57.265749+0000","last_peered":"2024-03-08T23:51:57.265749+0000","last_clean":"2024-03-08T23:51:57.265749+0000","last_became_active":"2024-03-08T23:51:56.280654+0000","last_became_peered":"2024-03-08T23:51:56.280654+0000","last_unstale":"2024-03-08T23:51:57.265749+0000","last_undegraded":"2024-03-08T23:51:57.265749+0000","last_fullsized":"2024-03-08T23:51:57.265749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:53.999115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.332528+0000","last_change":"2024-03-08T23:51:51.245947+0000","last_active":"2024-03-08T23:52:16.332528+0000","last_peered":"2024-03-08T23:52:16.332528+0000","last_clean":"2024-03-08T23:52:16.332528+0000","last_became_active":"2024-03-08T23:51:51.245505+0000","last_became_peered":"2024-03-08T23:51:51.245505+0000","last_unstale":"2024-03-08T23:52:16.332528+0000","last_undegraded":"2024-03-08T23:52:16.332528+0000","last_fullsized":"2024-03-08T23:52:16.332528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:20:44.692598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239595+0000","last_change":"2024-03-08T23:51:52.241590+0000","last_active":"2024-03-08T23:51:58.239595+0000","last_peered":"2024-03-08T23:51:58.239595+0000","last_clean":"2024-03-08T23:51:58.239595+0000","last_became_active":"2024-03-08T23:51:52.241296+0000","last_became_peered":"2024-03-08T23:51:52.241296+0000","last_unstale":"2024-03-08T23:51:58.239595+0000","last_undegraded":"2024-03-08T23:51:58.239595+0000","last_fullsized":"2024-03-08T23:51:58.239595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:52:48.042360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265748+0000","last_change":"2024-03-08T23:51:56.280897+0000","last_active":"2024-03-08T23:51:57.265748+0000","last_peered":"2024-03-08T23:51:57.265748+0000","last_clean":"2024-03-08T23:51:57.265748+0000","last_became_active":"2024-03-08T23:51:56.280511+0000","last_became_peered":"2024-03-08T23:51:56.280511+0000","last_unstale":"2024-03-08T23:51:57.265748+0000","last_undegraded":"2024-03-08T23:51:57.265748+0000","last_fullsized":"2024-03-08T23:51:57.265748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:12:26.963132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:17.342088+0000","last_change":"2024-03-08T23:51:51.241384+0000","last_active":"2024-03-08T23:52:17.342088+0000","last_peered":"2024-03-08T23:52:17.342088+0000","last_clean":"2024-03-08T23:52:17.342088+0000","last_became_active":"2024-03-08T23:51:51.241018+0000","last_became_peered":"2024-03-08T23:51:51.241018+0000","last_unstale":"2024-03-08T23:52:17.342088+0000","last_undegraded":"2024-03-08T23:52:17.342088+0000","last_fullsized":"2024-03-08T23:52:17.342088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:29:50.860663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278276+0000","last_change":"2024-03-08T23:51:52.250207+0000","last_active":"2024-03-08T23:51:57.278276+0000","last_peered":"2024-03-08T23:51:57.278276+0000","last_clean":"2024-03-08T23:51:57.278276+0000","last_became_active":"2024-03-08T23:51:52.249863+0000","last_became_peered":"2024-03-08T23:51:52.249863+0000","last_unstale":"2024-03-08T23:51:57.278276+0000","last_undegraded":"2024-03-08T23:51:57.278276+0000","last_fullsized":"2024-03-08T23:51:57.278276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:18:48.732419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403965+0000","last_change":"2024-03-08T23:51:56.282092+0000","last_active":"2024-03-08T23:51:58.403965+0000","last_peered":"2024-03-08T23:51:58.403965+0000","last_clean":"2024-03-08T23:51:58.403965+0000","last_became_active":"2024-03-08T23:51:56.281657+0000","last_became_peered":"2024-03-08T23:51:56.281657+0000","last_unstale":"2024-03-08T23:51:58.403965+0000","last_undegraded":"2024-03-08T23:51:58.403965+0000","last_fullsized":"2024-03-08T23:51:58.403965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:50.440520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:17.342820+0000","last_change":"2024-03-08T23:51:51.236816+0000","last_active":"2024-03-08T23:52:17.342820+0000","last_peered":"2024-03-08T23:52:17.342820+0000","last_clean":"2024-03-08T23:52:17.342820+0000","last_became_active":"2024-03-08T23:51:51.236485+0000","last_became_peered":"2024-03-08T23:51:51.236485+0000","last_unstale":"2024-03-08T23:52:17.342820+0000","last_undegraded":"2024-03-08T23:52:17.342820+0000","last_fullsized":"2024-03-08T23:52:17.342820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:27.300950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123476+0000","last_change":"2024-03-08T23:51:52.252146+0000","last_active":"2024-03-08T23:51:58.123476+0000","last_peered":"2024-03-08T23:51:58.123476+0000","last_clean":"2024-03-08T23:51:58.123476+0000","last_became_active":"2024-03-08T23:51:52.244201+0000","last_became_peered":"2024-03-08T23:51:52.244201+0000","last_unstale":"2024-03-08T23:51:58.123476+0000","last_undegraded":"2024-03-08T23:51:58.123476+0000","last_fullsized":"2024-03-08T23:51:58.123476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:07.647078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130945+0000","last_change":"2024-03-08T23:51:56.279727+0000","last_active":"2024-03-08T23:51:58.130945+0000","last_peered":"2024-03-08T23:51:58.130945+0000","last_clean":"2024-03-08T23:51:58.130945+0000","last_became_active":"2024-03-08T23:51:56.279184+0000","last_became_peered":"2024-03-08T23:51:56.279184+0000","last_unstale":"2024-03-08T23:51:58.130945+0000","last_undegraded":"2024-03-08T23:51:58.130945+0000","last_fullsized":"2024-03-08T23:51:58.130945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:21.804410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278148+0000","last_change":"2024-03-08T23:51:51.244812+0000","last_active":"2024-03-08T23:51:57.278148+0000","last_peered":"2024-03-08T23:51:57.278148+0000","last_clean":"2024-03-08T23:51:57.278148+0000","last_became_active":"2024-03-08T23:51:51.244513+0000","last_became_peered":"2024-03-08T23:51:51.244513+0000","last_unstale":"2024-03-08T23:51:57.278148+0000","last_undegraded":"2024-03-08T23:51:57.278148+0000","last_fullsized":"2024-03-08T23:51:57.278148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:56:07.235214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263775+0000","last_change":"2024-03-08T23:51:52.244566+0000","last_active":"2024-03-08T23:51:57.263775+0000","last_peered":"2024-03-08T23:51:57.263775+0000","last_clean":"2024-03-08T23:51:57.263775+0000","last_became_active":"2024-03-08T23:51:52.244250+0000","last_became_peered":"2024-03-08T23:51:52.244250+0000","last_unstale":"2024-03-08T23:51:57.263775+0000","last_undegraded":"2024-03-08T23:51:57.263775+0000","last_fullsized":"2024-03-08T23:51:57.263775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:55:50.596364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123631+0000","last_change":"2024-03-08T23:51:56.282487+0000","last_active":"2024-03-08T23:51:58.123631+0000","last_peered":"2024-03-08T23:51:58.123631+0000","last_clean":"2024-03-08T23:51:58.123631+0000","last_became_active":"2024-03-08T23:51:56.282104+0000","last_became_peered":"2024-03-08T23:51:56.282104+0000","last_unstale":"2024-03-08T23:51:58.123631+0000","last_undegraded":"2024-03-08T23:51:58.123631+0000","last_fullsized":"2024-03-08T23:51:58.123631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:23:23.787431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.345408+0000","last_change":"2024-03-08T23:51:51.236612+0000","last_active":"2024-03-08T23:52:18.345408+0000","last_peered":"2024-03-08T23:52:18.345408+0000","last_clean":"2024-03-08T23:52:18.345408+0000","last_became_active":"2024-03-08T23:51:51.236274+0000","last_became_peered":"2024-03-08T23:51:51.236274+0000","last_unstale":"2024-03-08T23:52:18.345408+0000","last_undegraded":"2024-03-08T23:52:18.345408+0000","last_fullsized":"2024-03-08T23:52:18.345408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:37.551870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404194+0000","last_change":"2024-03-08T23:51:52.244564+0000","last_active":"2024-03-08T23:51:58.404194+0000","last_peered":"2024-03-08T23:51:58.404194+0000","last_clean":"2024-03-08T23:51:58.404194+0000","last_became_active":"2024-03-08T23:51:52.243651+0000","last_became_peered":"2024-03-08T23:51:52.243651+0000","last_unstale":"2024-03-08T23:51:58.404194+0000","last_undegraded":"2024-03-08T23:51:58.404194+0000","last_fullsized":"2024-03-08T23:51:58.404194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:35:32.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263880+0000","last_change":"2024-03-08T23:51:56.279354+0000","last_active":"2024-03-08T23:51:57.263880+0000","last_peered":"2024-03-08T23:51:57.263880+0000","last_clean":"2024-03-08T23:51:57.263880+0000","last_became_active":"2024-03-08T23:51:56.278894+0000","last_became_peered":"2024-03-08T23:51:56.278894+0000","last_unstale":"2024-03-08T23:51:57.263880+0000","last_undegraded":"2024-03-08T23:51:57.263880+0000","last_fullsized":"2024-03-08T23:51:57.263880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:20:30.164542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.331291+0000","last_change":"2024-03-08T23:51:51.246417+0000","last_active":"2024-03-08T23:52:16.331291+0000","last_peered":"2024-03-08T23:52:16.331291+0000","last_clean":"2024-03-08T23:52:16.331291+0000","last_became_active":"2024-03-08T23:51:51.246058+0000","last_became_peered":"2024-03-08T23:51:51.246058+0000","last_unstale":"2024-03-08T23:52:16.331291+0000","last_undegraded":"2024-03-08T23:52:16.331291+0000","last_fullsized":"2024-03-08T23:52:16.331291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:48:53.569295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130924+0000","last_change":"2024-03-08T23:51:52.255357+0000","last_active":"2024-03-08T23:51:58.130924+0000","last_peered":"2024-03-08T23:51:58.130924+0000","last_clean":"2024-03-08T23:51:58.130924+0000","last_became_active":"2024-03-08T23:51:52.255044+0000","last_became_peered":"2024-03-08T23:51:52.255044+0000","last_unstale":"2024-03-08T23:51:58.130924+0000","last_undegraded":"2024-03-08T23:51:58.130924+0000","last_fullsized":"2024-03-08T23:51:58.130924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:13:40.710283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378179+0000","last_change":"2024-03-08T23:51:56.284282+0000","last_active":"2024-03-08T23:51:57.378179+0000","last_peered":"2024-03-08T23:51:57.378179+0000","last_clean":"2024-03-08T23:51:57.378179+0000","last_became_active":"2024-03-08T23:51:56.283863+0000","last_became_peered":"2024-03-08T23:51:56.283863+0000","last_unstale":"2024-03-08T23:51:57.378179+0000","last_undegraded":"2024-03-08T23:51:57.378179+0000","last_fullsized":"2024-03-08T23:51:57.378179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:59:21.972283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278376+0000","last_change":"2024-03-08T23:51:51.245199+0000","last_active":"2024-03-08T23:51:57.278376+0000","last_peered":"2024-03-08T23:51:57.278376+0000","last_clean":"2024-03-08T23:51:57.278376+0000","last_became_active":"2024-03-08T23:51:51.244906+0000","last_became_peered":"2024-03-08T23:51:51.244906+0000","last_unstale":"2024-03-08T23:51:57.278376+0000","last_undegraded":"2024-03-08T23:51:57.278376+0000","last_fullsized":"2024-03-08T23:51:57.278376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:54:39.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403693+0000","last_change":"2024-03-08T23:51:52.251209+0000","last_active":"2024-03-08T23:51:58.403693+0000","last_peered":"2024-03-08T23:51:58.403693+0000","last_clean":"2024-03-08T23:51:58.403693+0000","last_became_active":"2024-03-08T23:51:52.250392+0000","last_became_peered":"2024-03-08T23:51:52.250392+0000","last_unstale":"2024-03-08T23:51:58.403693+0000","last_undegraded":"2024-03-08T23:51:58.403693+0000","last_fullsized":"2024-03-08T23:51:58.403693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:56:17.811505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263768+0000","last_change":"2024-03-08T23:51:56.281154+0000","last_active":"2024-03-08T23:51:57.263768+0000","last_peered":"2024-03-08T23:51:57.263768+0000","last_clean":"2024-03-08T23:51:57.263768+0000","last_became_active":"2024-03-08T23:51:56.280736+0000","last_became_peered":"2024-03-08T23:51:56.280736+0000","last_unstale":"2024-03-08T23:51:57.263768+0000","last_undegraded":"2024-03-08T23:51:57.263768+0000","last_fullsized":"2024-03-08T23:51:57.263768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:01:49.909731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'19","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:19.349714+0000","last_change":"2024-03-08T23:51:51.239716+0000","last_active":"2024-03-08T23:52:19.349714+0000","last_peered":"2024-03-08T23:52:19.349714+0000","last_clean":"2024-03-08T23:52:19.349714+0000","last_became_active":"2024-03-08T23:51:51.239384+0000","last_became_peered":"2024-03-08T23:51:51.239384+0000","last_unstale":"2024-03-08T23:52:19.349714+0000","last_undegraded":"2024-03-08T23:52:19.349714+0000","last_fullsized":"2024-03-08T23:52:19.349714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:40.892723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239953+0000","last_change":"2024-03-08T23:51:52.248715+0000","last_active":"2024-03-08T23:51:58.239953+0000","last_peered":"2024-03-08T23:51:58.239953+0000","last_clean":"2024-03-08T23:51:58.239953+0000","last_became_active":"2024-03-08T23:51:52.248258+0000","last_became_peered":"2024-03-08T23:51:52.248258+0000","last_unstale":"2024-03-08T23:51:58.239953+0000","last_undegraded":"2024-03-08T23:51:58.239953+0000","last_fullsized":"2024-03-08T23:51:58.239953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:38.185469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278754+0000","last_change":"2024-03-08T23:51:56.278268+0000","last_active":"2024-03-08T23:51:57.278754+0000","last_peered":"2024-03-08T23:51:57.278754+0000","last_clean":"2024-03-08T23:51:57.278754+0000","last_became_active":"2024-03-08T23:51:56.277831+0000","last_became_peered":"2024-03-08T23:51:56.277831+0000","last_unstale":"2024-03-08T23:51:57.278754+0000","last_undegraded":"2024-03-08T23:51:57.278754+0000","last_fullsized":"2024-03-08T23:51:57.278754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:32:13.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.331231+0000","last_change":"2024-03-08T23:51:51.247553+0000","last_active":"2024-03-08T23:52:16.331231+0000","last_peered":"2024-03-08T23:52:16.331231+0000","last_clean":"2024-03-08T23:52:16.331231+0000","last_became_active":"2024-03-08T23:51:51.247225+0000","last_became_peered":"2024-03-08T23:51:51.247225+0000","last_unstale":"2024-03-08T23:52:16.331231+0000","last_undegraded":"2024-03-08T23:52:16.331231+0000","last_fullsized":"2024-03-08T23:52:16.331231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:53:29.846303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651115+0000","last_change":"2024-03-08T23:51:52.247733+0000","last_active":"2024-03-08T23:51:57.651115+0000","last_peered":"2024-03-08T23:51:57.651115+0000","last_clean":"2024-03-08T23:51:57.651115+0000","last_became_active":"2024-03-08T23:51:52.247435+0000","last_became_peered":"2024-03-08T23:51:52.247435+0000","last_unstale":"2024-03-08T23:51:57.651115+0000","last_undegraded":"2024-03-08T23:51:57.651115+0000","last_fullsized":"2024-03-08T23:51:57.651115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:46:12.739601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403746+0000","last_change":"2024-03-08T23:51:56.279865+0000","last_active":"2024-03-08T23:51:58.403746+0000","last_peered":"2024-03-08T23:51:58.403746+0000","last_clean":"2024-03-08T23:51:58.403746+0000","last_became_active":"2024-03-08T23:51:56.279446+0000","last_became_peered":"2024-03-08T23:51:56.279446+0000","last_unstale":"2024-03-08T23:51:58.403746+0000","last_undegraded":"2024-03-08T23:51:58.403746+0000","last_fullsized":"2024-03-08T23:51:58.403746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:07:33.672490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.332381+0000","last_change":"2024-03-08T23:51:51.246860+0000","last_active":"2024-03-08T23:52:16.332381+0000","last_peered":"2024-03-08T23:52:16.332381+0000","last_clean":"2024-03-08T23:52:16.332381+0000","last_became_active":"2024-03-08T23:51:51.246545+0000","last_became_peered":"2024-03-08T23:51:51.246545+0000","last_unstale":"2024-03-08T23:52:16.332381+0000","last_undegraded":"2024-03-08T23:52:16.332381+0000","last_fullsized":"2024-03-08T23:52:16.332381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:11:07.246657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264375+0000","last_change":"2024-03-08T23:51:52.247353+0000","last_active":"2024-03-08T23:51:57.264375+0000","last_peered":"2024-03-08T23:51:57.264375+0000","last_clean":"2024-03-08T23:51:57.264375+0000","last_became_active":"2024-03-08T23:51:52.247008+0000","last_became_peered":"2024-03-08T23:51:52.247008+0000","last_unstale":"2024-03-08T23:51:57.264375+0000","last_undegraded":"2024-03-08T23:51:57.264375+0000","last_fullsized":"2024-03-08T23:51:57.264375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:08:50.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264384+0000","last_change":"2024-03-08T23:51:56.284103+0000","last_active":"2024-03-08T23:51:57.264384+0000","last_peered":"2024-03-08T23:51:57.264384+0000","last_clean":"2024-03-08T23:51:57.264384+0000","last_became_active":"2024-03-08T23:51:56.283561+0000","last_became_peered":"2024-03-08T23:51:56.283561+0000","last_unstale":"2024-03-08T23:51:57.264384+0000","last_undegraded":"2024-03-08T23:51:57.264384+0000","last_fullsized":"2024-03-08T23:51:57.264384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:47:20.909292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.345862+0000","last_change":"2024-03-08T23:51:51.242483+0000","last_active":"2024-03-08T23:52:18.345862+0000","last_peered":"2024-03-08T23:52:18.345862+0000","last_clean":"2024-03-08T23:52:18.345862+0000","last_became_active":"2024-03-08T23:51:51.242153+0000","last_became_peered":"2024-03-08T23:51:51.242153+0000","last_unstale":"2024-03-08T23:52:18.345862+0000","last_undegraded":"2024-03-08T23:52:18.345862+0000","last_fullsized":"2024-03-08T23:52:18.345862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:44:07.728700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264068+0000","last_change":"2024-03-08T23:51:52.255411+0000","last_active":"2024-03-08T23:51:57.264068+0000","last_peered":"2024-03-08T23:51:57.264068+0000","last_clean":"2024-03-08T23:51:57.264068+0000","last_became_active":"2024-03-08T23:51:52.255048+0000","last_became_peered":"2024-03-08T23:51:52.255048+0000","last_unstale":"2024-03-08T23:51:57.264068+0000","last_undegraded":"2024-03-08T23:51:57.264068+0000","last_fullsized":"2024-03-08T23:51:57.264068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:11.836187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651515+0000","last_change":"2024-03-08T23:51:56.286405+0000","last_active":"2024-03-08T23:51:57.651515+0000","last_peered":"2024-03-08T23:51:57.651515+0000","last_clean":"2024-03-08T23:51:57.651515+0000","last_became_active":"2024-03-08T23:51:56.285996+0000","last_became_peered":"2024-03-08T23:51:56.285996+0000","last_unstale":"2024-03-08T23:51:57.651515+0000","last_undegraded":"2024-03-08T23:51:57.651515+0000","last_fullsized":"2024-03-08T23:51:57.651515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:48:30.928436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239452+0000","last_change":"2024-03-08T23:51:51.240146+0000","last_active":"2024-03-08T23:51:58.239452+0000","last_peered":"2024-03-08T23:51:58.239452+0000","last_clean":"2024-03-08T23:51:58.239452+0000","last_became_active":"2024-03-08T23:51:51.239795+0000","last_became_peered":"2024-03-08T23:51:51.239795+0000","last_unstale":"2024-03-08T23:51:58.239452+0000","last_undegraded":"2024-03-08T23:51:58.239452+0000","last_fullsized":"2024-03-08T23:51:58.239452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:55.409922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278114+0000","last_change":"2024-03-08T23:51:52.254701+0000","last_active":"2024-03-08T23:51:57.278114+0000","last_peered":"2024-03-08T23:51:57.278114+0000","last_clean":"2024-03-08T23:51:57.278114+0000","last_became_active":"2024-03-08T23:51:52.254391+0000","last_became_peered":"2024-03-08T23:51:52.254391+0000","last_unstale":"2024-03-08T23:51:57.278114+0000","last_undegraded":"2024-03-08T23:51:57.278114+0000","last_fullsized":"2024-03-08T23:51:57.278114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:11:12.258181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278244+0000","last_change":"2024-03-08T23:51:56.277878+0000","last_active":"2024-03-08T23:51:57.278244+0000","last_peered":"2024-03-08T23:51:57.278244+0000","last_clean":"2024-03-08T23:51:57.278244+0000","last_became_active":"2024-03-08T23:51:56.277493+0000","last_became_peered":"2024-03-08T23:51:56.277493+0000","last_unstale":"2024-03-08T23:51:57.278244+0000","last_undegraded":"2024-03-08T23:51:57.278244+0000","last_fullsized":"2024-03-08T23:51:57.278244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:44:16.968632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.240086+0000","last_change":"2024-03-08T23:51:51.241457+0000","last_active":"2024-03-08T23:51:58.240086+0000","last_peered":"2024-03-08T23:51:58.240086+0000","last_clean":"2024-03-08T23:51:58.240086+0000","last_became_active":"2024-03-08T23:51:51.241148+0000","last_became_peered":"2024-03-08T23:51:51.241148+0000","last_unstale":"2024-03-08T23:51:58.240086+0000","last_undegraded":"2024-03-08T23:51:58.240086+0000","last_fullsized":"2024-03-08T23:51:58.240086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:13:27.246265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403822+0000","last_change":"2024-03-08T23:51:52.248465+0000","last_active":"2024-03-08T23:51:58.403822+0000","last_peered":"2024-03-08T23:51:58.403822+0000","last_clean":"2024-03-08T23:51:58.403822+0000","last_became_active":"2024-03-08T23:51:52.248135+0000","last_became_peered":"2024-03-08T23:51:52.248135+0000","last_unstale":"2024-03-08T23:51:58.403822+0000","last_undegraded":"2024-03-08T23:51:58.403822+0000","last_fullsized":"2024-03-08T23:51:58.403822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:55:53.712958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239727+0000","last_change":"2024-03-08T23:51:56.276019+0000","last_active":"2024-03-08T23:51:58.239727+0000","last_peered":"2024-03-08T23:51:58.239727+0000","last_clean":"2024-03-08T23:51:58.239727+0000","last_became_active":"2024-03-08T23:51:56.275564+0000","last_became_peered":"2024-03-08T23:51:56.275564+0000","last_unstale":"2024-03-08T23:51:58.239727+0000","last_undegraded":"2024-03-08T23:51:58.239727+0000","last_fullsized":"2024-03-08T23:51:58.239727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:12:36.096470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263962+0000","last_change":"2024-03-08T23:51:51.241629+0000","last_active":"2024-03-08T23:51:57.263962+0000","last_peered":"2024-03-08T23:51:57.263962+0000","last_clean":"2024-03-08T23:51:57.263962+0000","last_became_active":"2024-03-08T23:51:51.241300+0000","last_became_peered":"2024-03-08T23:51:51.241300+0000","last_unstale":"2024-03-08T23:51:57.263962+0000","last_undegraded":"2024-03-08T23:51:57.263962+0000","last_fullsized":"2024-03-08T23:51:57.263962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:01:45.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404171+0000","last_change":"2024-03-08T23:51:52.246024+0000","last_active":"2024-03-08T23:51:58.404171+0000","last_peered":"2024-03-08T23:51:58.404171+0000","last_clean":"2024-03-08T23:51:58.404171+0000","last_became_active":"2024-03-08T23:51:52.245706+0000","last_became_peered":"2024-03-08T23:51:52.245706+0000","last_unstale":"2024-03-08T23:51:58.404171+0000","last_undegraded":"2024-03-08T23:51:58.404171+0000","last_fullsized":"2024-03-08T23:51:58.404171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:00:44.557269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651351+0000","last_change":"2024-03-08T23:51:56.285649+0000","last_active":"2024-03-08T23:51:57.651351+0000","last_peered":"2024-03-08T23:51:57.651351+0000","last_clean":"2024-03-08T23:51:57.651351+0000","last_became_active":"2024-03-08T23:51:56.282731+0000","last_became_peered":"2024-03-08T23:51:56.282731+0000","last_unstale":"2024-03-08T23:51:57.651351+0000","last_undegraded":"2024-03-08T23:51:57.651351+0000","last_fullsized":"2024-03-08T23:51:57.651351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:23:54.103637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264292+0000","last_change":"2024-03-08T23:51:51.249442+0000","last_active":"2024-03-08T23:51:57.264292+0000","last_peered":"2024-03-08T23:51:57.264292+0000","last_clean":"2024-03-08T23:51:57.264292+0000","last_became_active":"2024-03-08T23:51:51.249084+0000","last_became_peered":"2024-03-08T23:51:51.249084+0000","last_unstale":"2024-03-08T23:51:57.264292+0000","last_undegraded":"2024-03-08T23:51:57.264292+0000","last_fullsized":"2024-03-08T23:51:57.264292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:48:19.370095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404058+0000","last_change":"2024-03-08T23:51:52.257586+0000","last_active":"2024-03-08T23:51:58.404058+0000","last_peered":"2024-03-08T23:51:58.404058+0000","last_clean":"2024-03-08T23:51:58.404058+0000","last_became_active":"2024-03-08T23:51:52.253741+0000","last_became_peered":"2024-03-08T23:51:52.253741+0000","last_unstale":"2024-03-08T23:51:58.404058+0000","last_undegraded":"2024-03-08T23:51:58.404058+0000","last_fullsized":"2024-03-08T23:51:58.404058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:18.633166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651390+0000","last_change":"2024-03-08T23:51:56.285264+0000","last_active":"2024-03-08T23:51:57.651390+0000","last_peered":"2024-03-08T23:51:57.651390+0000","last_clean":"2024-03-08T23:51:57.651390+0000","last_became_active":"2024-03-08T23:51:56.284727+0000","last_became_peered":"2024-03-08T23:51:56.284727+0000","last_unstale":"2024-03-08T23:51:57.651390+0000","last_undegraded":"2024-03-08T23:51:57.651390+0000","last_fullsized":"2024-03-08T23:51:57.651390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:01:20.723010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278562+0000","last_change":"2024-03-08T23:51:52.251461+0000","last_active":"2024-03-08T23:51:57.278562+0000","last_peered":"2024-03-08T23:51:57.278562+0000","last_clean":"2024-03-08T23:51:57.278562+0000","last_became_active":"2024-03-08T23:51:52.250996+0000","last_became_peered":"2024-03-08T23:51:52.250996+0000","last_unstale":"2024-03-08T23:51:57.278562+0000","last_undegraded":"2024-03-08T23:51:57.278562+0000","last_fullsized":"2024-03-08T23:51:57.278562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:51.719142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.345280+0000","last_change":"2024-03-08T23:51:51.245698+0000","last_active":"2024-03-08T23:52:18.345280+0000","last_peered":"2024-03-08T23:52:18.345280+0000","last_clean":"2024-03-08T23:52:18.345280+0000","last_became_active":"2024-03-08T23:51:51.245389+0000","last_became_peered":"2024-03-08T23:51:51.245389+0000","last_unstale":"2024-03-08T23:52:18.345280+0000","last_undegraded":"2024-03-08T23:52:18.345280+0000","last_fullsized":"2024-03-08T23:52:18.345280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:14:08.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:59.677777+0000","last_change":"2024-03-08T23:51:56.291884+0000","last_active":"2024-03-08T23:51:59.677777+0000","last_peered":"2024-03-08T23:51:59.677777+0000","last_clean":"2024-03-08T23:51:59.677777+0000","last_became_active":"2024-03-08T23:51:56.291419+0000","last_became_peered":"2024-03-08T23:51:56.291419+0000","last_unstale":"2024-03-08T23:51:59.677777+0000","last_undegraded":"2024-03-08T23:51:59.677777+0000","last_fullsized":"2024-03-08T23:51:59.677777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:57:03.971483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123537+0000","last_change":"2024-03-08T23:51:52.256518+0000","last_active":"2024-03-08T23:51:58.123537+0000","last_peered":"2024-03-08T23:51:58.123537+0000","last_clean":"2024-03-08T23:51:58.123537+0000","last_became_active":"2024-03-08T23:51:52.253481+0000","last_became_peered":"2024-03-08T23:51:52.253481+0000","last_unstale":"2024-03-08T23:51:58.123537+0000","last_undegraded":"2024-03-08T23:51:58.123537+0000","last_fullsized":"2024-03-08T23:51:58.123537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:35.287408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.343701+0000","last_change":"2024-03-08T23:51:51.237921+0000","last_active":"2024-03-08T23:52:18.343701+0000","last_peered":"2024-03-08T23:52:18.343701+0000","last_clean":"2024-03-08T23:52:18.343701+0000","last_became_active":"2024-03-08T23:51:51.237568+0000","last_became_peered":"2024-03-08T23:51:51.237568+0000","last_unstale":"2024-03-08T23:52:18.343701+0000","last_undegraded":"2024-03-08T23:52:18.343701+0000","last_fullsized":"2024-03-08T23:52:18.343701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:11.256814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263913+0000","last_change":"2024-03-08T23:50:41.802121+0000","last_active":"2024-03-08T23:51:57.263913+0000","last_peered":"2024-03-08T23:51:57.263913+0000","last_clean":"2024-03-08T23:51:57.263913+0000","last_became_active":"2024-03-08T23:50:41.801678+0000","last_became_peered":"2024-03-08T23:50:41.801678+0000","last_unstale":"2024-03-08T23:51:57.263913+0000","last_undegraded":"2024-03-08T23:51:57.263913+0000","last_fullsized":"2024-03-08T23:51:57.263913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_clean_scrub_stamp":"2024-03-08T23:47:12.382547+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:15.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278544+0000","last_change":"2024-03-08T23:51:56.278465+0000","last_active":"2024-03-08T23:51:57.278544+0000","last_peered":"2024-03-08T23:51:57.278544+0000","last_clean":"2024-03-08T23:51:57.278544+0000","last_became_active":"2024-03-08T23:51:56.278079+0000","last_became_peered":"2024-03-08T23:51:56.278079+0000","last_unstale":"2024-03-08T23:51:57.278544+0000","last_undegraded":"2024-03-08T23:51:57.278544+0000","last_fullsized":"2024-03-08T23:51:57.278544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:47.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.650867+0000","last_change":"2024-03-08T23:51:52.251895+0000","last_active":"2024-03-08T23:51:57.650867+0000","last_peered":"2024-03-08T23:51:57.650867+0000","last_clean":"2024-03-08T23:51:57.650867+0000","last_became_active":"2024-03-08T23:51:52.251591+0000","last_became_peered":"2024-03-08T23:51:52.251591+0000","last_unstale":"2024-03-08T23:51:57.650867+0000","last_undegraded":"2024-03-08T23:51:57.650867+0000","last_fullsized":"2024-03-08T23:51:57.650867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:06.276636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:17.341418+0000","last_change":"2024-03-08T23:51:51.245981+0000","last_active":"2024-03-08T23:52:17.341418+0000","last_peered":"2024-03-08T23:52:17.341418+0000","last_clean":"2024-03-08T23:52:17.341418+0000","last_became_active":"2024-03-08T23:51:51.244681+0000","last_became_peered":"2024-03-08T23:51:51.244681+0000","last_unstale":"2024-03-08T23:52:17.341418+0000","last_undegraded":"2024-03-08T23:52:17.341418+0000","last_fullsized":"2024-03-08T23:52:17.341418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:06:21.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403676+0000","last_change":"2024-03-08T23:51:56.279057+0000","last_active":"2024-03-08T23:51:58.403676+0000","last_peered":"2024-03-08T23:51:58.403676+0000","last_clean":"2024-03-08T23:51:58.403676+0000","last_became_active":"2024-03-08T23:51:56.278634+0000","last_became_peered":"2024-03-08T23:51:56.278634+0000","last_unstale":"2024-03-08T23:51:58.403676+0000","last_undegraded":"2024-03-08T23:51:58.403676+0000","last_fullsized":"2024-03-08T23:51:58.403676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:30.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.240055+0000","last_change":"2024-03-08T23:51:51.240378+0000","last_active":"2024-03-08T23:51:58.240055+0000","last_peered":"2024-03-08T23:51:58.240055+0000","last_clean":"2024-03-08T23:51:58.240055+0000","last_became_active":"2024-03-08T23:51:51.239880+0000","last_became_peered":"2024-03-08T23:51:51.239880+0000","last_unstale":"2024-03-08T23:51:58.240055+0000","last_undegraded":"2024-03-08T23:51:58.240055+0000","last_fullsized":"2024-03-08T23:51:58.240055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:20.804261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651404+0000","last_change":"2024-03-08T23:51:52.240338+0000","last_active":"2024-03-08T23:51:57.651404+0000","last_peered":"2024-03-08T23:51:57.651404+0000","last_clean":"2024-03-08T23:51:57.651404+0000","last_became_active":"2024-03-08T23:51:52.239987+0000","last_became_peered":"2024-03-08T23:51:52.239987+0000","last_unstale":"2024-03-08T23:51:57.651404+0000","last_undegraded":"2024-03-08T23:51:57.651404+0000","last_fullsized":"2024-03-08T23:51:57.651404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:26.590204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123711+0000","last_change":"2024-03-08T23:51:56.281009+0000","last_active":"2024-03-08T23:51:58.123711+0000","last_peered":"2024-03-08T23:51:58.123711+0000","last_clean":"2024-03-08T23:51:58.123711+0000","last_became_active":"2024-03-08T23:51:56.280619+0000","last_became_peered":"2024-03-08T23:51:56.280619+0000","last_unstale":"2024-03-08T23:51:58.123711+0000","last_undegraded":"2024-03-08T23:51:58.123711+0000","last_fullsized":"2024-03-08T23:51:58.123711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:06.920777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264295+0000","last_change":"2024-03-08T23:51:51.241682+0000","last_active":"2024-03-08T23:51:57.264295+0000","last_peered":"2024-03-08T23:51:57.264295+0000","last_clean":"2024-03-08T23:51:57.264295+0000","last_became_active":"2024-03-08T23:51:51.241313+0000","last_became_peered":"2024-03-08T23:51:51.241313+0000","last_unstale":"2024-03-08T23:51:57.264295+0000","last_undegraded":"2024-03-08T23:51:57.264295+0000","last_fullsized":"2024-03-08T23:51:57.264295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:04:13.515856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123737+0000","last_change":"2024-03-08T23:51:52.249378+0000","last_active":"2024-03-08T23:51:58.123737+0000","last_peered":"2024-03-08T23:51:58.123737+0000","last_clean":"2024-03-08T23:51:58.123737+0000","last_became_active":"2024-03-08T23:51:52.249084+0000","last_became_peered":"2024-03-08T23:51:52.249084+0000","last_unstale":"2024-03-08T23:51:58.123737+0000","last_undegraded":"2024-03-08T23:51:58.123737+0000","last_fullsized":"2024-03-08T23:51:58.123737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:10:36.513260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265304+0000","last_change":"2024-03-08T23:51:56.280717+0000","last_active":"2024-03-08T23:51:57.265304+0000","last_peered":"2024-03-08T23:51:57.265304+0000","last_clean":"2024-03-08T23:51:57.265304+0000","last_became_active":"2024-03-08T23:51:56.280329+0000","last_became_peered":"2024-03-08T23:51:56.280329+0000","last_unstale":"2024-03-08T23:51:57.265304+0000","last_undegraded":"2024-03-08T23:51:57.265304+0000","last_fullsized":"2024-03-08T23:51:57.265304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:47:40.829142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.330305+0000","last_change":"2024-03-08T23:51:51.238058+0000","last_active":"2024-03-08T23:52:16.330305+0000","last_peered":"2024-03-08T23:52:16.330305+0000","last_clean":"2024-03-08T23:52:16.330305+0000","last_became_active":"2024-03-08T23:51:51.237727+0000","last_became_peered":"2024-03-08T23:51:51.237727+0000","last_unstale":"2024-03-08T23:52:16.330305+0000","last_undegraded":"2024-03-08T23:52:16.330305+0000","last_fullsized":"2024-03-08T23:52:16.330305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:13:54.080004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130951+0000","last_change":"2024-03-08T23:51:52.249830+0000","last_active":"2024-03-08T23:51:58.130951+0000","last_peered":"2024-03-08T23:51:58.130951+0000","last_clean":"2024-03-08T23:51:58.130951+0000","last_became_active":"2024-03-08T23:51:52.249518+0000","last_became_peered":"2024-03-08T23:51:52.249518+0000","last_unstale":"2024-03-08T23:51:58.130951+0000","last_undegraded":"2024-03-08T23:51:58.130951+0000","last_fullsized":"2024-03-08T23:51:58.130951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:24:33.721423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378369+0000","last_change":"2024-03-08T23:51:56.288595+0000","last_active":"2024-03-08T23:51:57.378369+0000","last_peered":"2024-03-08T23:51:57.378369+0000","last_clean":"2024-03-08T23:51:57.378369+0000","last_became_active":"2024-03-08T23:51:56.288146+0000","last_became_peered":"2024-03-08T23:51:56.288146+0000","last_unstale":"2024-03-08T23:51:57.378369+0000","last_undegraded":"2024-03-08T23:51:57.378369+0000","last_fullsized":"2024-03-08T23:51:57.378369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:34:58.321754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.331159+0000","last_change":"2024-03-08T23:51:51.242065+0000","last_active":"2024-03-08T23:52:16.331159+0000","last_peered":"2024-03-08T23:52:16.331159+0000","last_clean":"2024-03-08T23:52:16.331159+0000","last_became_active":"2024-03-08T23:51:51.241513+0000","last_became_peered":"2024-03-08T23:51:51.241513+0000","last_unstale":"2024-03-08T23:52:16.331159+0000","last_undegraded":"2024-03-08T23:52:16.331159+0000","last_fullsized":"2024-03-08T23:52:16.331159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:08:02.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239458+0000","last_change":"2024-03-08T23:51:52.243074+0000","last_active":"2024-03-08T23:51:58.239458+0000","last_peered":"2024-03-08T23:51:58.239458+0000","last_clean":"2024-03-08T23:51:58.239458+0000","last_became_active":"2024-03-08T23:51:52.242736+0000","last_became_peered":"2024-03-08T23:51:52.242736+0000","last_unstale":"2024-03-08T23:51:58.239458+0000","last_undegraded":"2024-03-08T23:51:58.239458+0000","last_fullsized":"2024-03-08T23:51:58.239458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:57:37.174056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264565+0000","last_change":"2024-03-08T23:51:56.276284+0000","last_active":"2024-03-08T23:51:57.264565+0000","last_peered":"2024-03-08T23:51:57.264565+0000","last_clean":"2024-03-08T23:51:57.264565+0000","last_became_active":"2024-03-08T23:51:56.275800+0000","last_became_peered":"2024-03-08T23:51:56.275800+0000","last_unstale":"2024-03-08T23:51:57.264565+0000","last_undegraded":"2024-03-08T23:51:57.264565+0000","last_fullsized":"2024-03-08T23:51:57.264565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:21:11.221266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.333264+0000","last_change":"2024-03-08T23:51:51.245548+0000","last_active":"2024-03-08T23:52:16.333264+0000","last_peered":"2024-03-08T23:52:16.333264+0000","last_clean":"2024-03-08T23:52:16.333264+0000","last_became_active":"2024-03-08T23:51:51.245238+0000","last_became_peered":"2024-03-08T23:51:51.245238+0000","last_unstale":"2024-03-08T23:52:16.333264+0000","last_undegraded":"2024-03-08T23:52:16.333264+0000","last_fullsized":"2024-03-08T23:52:16.333264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:21:19.557196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403466+0000","last_change":"2024-03-08T23:51:52.244422+0000","last_active":"2024-03-08T23:51:58.403466+0000","last_peered":"2024-03-08T23:51:58.403466+0000","last_clean":"2024-03-08T23:51:58.403466+0000","last_became_active":"2024-03-08T23:51:52.244032+0000","last_became_peered":"2024-03-08T23:51:52.244032+0000","last_unstale":"2024-03-08T23:51:58.403466+0000","last_undegraded":"2024-03-08T23:51:58.403466+0000","last_fullsized":"2024-03-08T23:51:58.403466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:04.064691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378138+0000","last_change":"2024-03-08T23:51:56.288632+0000","last_active":"2024-03-08T23:51:57.378138+0000","last_peered":"2024-03-08T23:51:57.378138+0000","last_clean":"2024-03-08T23:51:57.378138+0000","last_became_active":"2024-03-08T23:51:56.288175+0000","last_became_peered":"2024-03-08T23:51:56.288175+0000","last_unstale":"2024-03-08T23:51:57.378138+0000","last_undegraded":"2024-03-08T23:51:57.378138+0000","last_fullsized":"2024-03-08T23:51:57.378138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:54:51.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265416+0000","last_change":"2024-03-08T23:51:51.247968+0000","last_active":"2024-03-08T23:51:57.265416+0000","last_peered":"2024-03-08T23:51:57.265416+0000","last_clean":"2024-03-08T23:51:57.265416+0000","last_became_active":"2024-03-08T23:51:51.247660+0000","last_became_peered":"2024-03-08T23:51:51.247660+0000","last_unstale":"2024-03-08T23:51:57.265416+0000","last_undegraded":"2024-03-08T23:51:57.265416+0000","last_fullsized":"2024-03-08T23:51:57.265416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:40.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123736+0000","last_change":"2024-03-08T23:51:52.250778+0000","last_active":"2024-03-08T23:51:58.123736+0000","last_peered":"2024-03-08T23:51:58.123736+0000","last_clean":"2024-03-08T23:51:58.123736+0000","last_became_active":"2024-03-08T23:51:52.250475+0000","last_became_peered":"2024-03-08T23:51:52.250475+0000","last_unstale":"2024-03-08T23:51:58.123736+0000","last_undegraded":"2024-03-08T23:51:58.123736+0000","last_fullsized":"2024-03-08T23:51:58.123736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:36:01.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378204+0000","last_change":"2024-03-08T23:51:56.285631+0000","last_active":"2024-03-08T23:51:57.378204+0000","last_peered":"2024-03-08T23:51:57.378204+0000","last_clean":"2024-03-08T23:51:57.378204+0000","last_became_active":"2024-03-08T23:51:56.285183+0000","last_became_peered":"2024-03-08T23:51:56.285183+0000","last_unstale":"2024-03-08T23:51:57.378204+0000","last_undegraded":"2024-03-08T23:51:57.378204+0000","last_fullsized":"2024-03-08T23:51:57.378204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:15.002086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239656+0000","last_change":"2024-03-08T23:51:51.246848+0000","last_active":"2024-03-08T23:51:58.239656+0000","last_peered":"2024-03-08T23:51:58.239656+0000","last_clean":"2024-03-08T23:51:58.239656+0000","last_became_active":"2024-03-08T23:51:51.246533+0000","last_became_peered":"2024-03-08T23:51:51.246533+0000","last_unstale":"2024-03-08T23:51:58.239656+0000","last_undegraded":"2024-03-08T23:51:58.239656+0000","last_fullsized":"2024-03-08T23:51:58.239656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:20:16.669820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264182+0000","last_change":"2024-03-08T23:51:52.251575+0000","last_active":"2024-03-08T23:51:57.264182+0000","last_peered":"2024-03-08T23:51:57.264182+0000","last_clean":"2024-03-08T23:51:57.264182+0000","last_became_active":"2024-03-08T23:51:52.251232+0000","last_became_peered":"2024-03-08T23:51:52.251232+0000","last_unstale":"2024-03-08T23:51:57.264182+0000","last_undegraded":"2024-03-08T23:51:57.264182+0000","last_fullsized":"2024-03-08T23:51:57.264182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:33.573407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403778+0000","last_change":"2024-03-08T23:51:56.282841+0000","last_active":"2024-03-08T23:51:58.403778+0000","last_peered":"2024-03-08T23:51:58.403778+0000","last_clean":"2024-03-08T23:51:58.403778+0000","last_became_active":"2024-03-08T23:51:56.282366+0000","last_became_peered":"2024-03-08T23:51:56.282366+0000","last_unstale":"2024-03-08T23:51:58.403778+0000","last_undegraded":"2024-03-08T23:51:58.403778+0000","last_fullsized":"2024-03-08T23:51:58.403778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:23:53.164544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265884+0000","last_change":"2024-03-08T23:51:51.242081+0000","last_active":"2024-03-08T23:51:57.265884+0000","last_peered":"2024-03-08T23:51:57.265884+0000","last_clean":"2024-03-08T23:51:57.265884+0000","last_became_active":"2024-03-08T23:51:51.241636+0000","last_became_peered":"2024-03-08T23:51:51.241636+0000","last_unstale":"2024-03-08T23:51:57.265884+0000","last_undegraded":"2024-03-08T23:51:57.265884+0000","last_fullsized":"2024-03-08T23:51:57.265884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:51:57.497723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264712+0000","last_change":"2024-03-08T23:51:52.252796+0000","last_active":"2024-03-08T23:51:57.264712+0000","last_peered":"2024-03-08T23:51:57.264712+0000","last_clean":"2024-03-08T23:51:57.264712+0000","last_became_active":"2024-03-08T23:51:52.252361+0000","last_became_peered":"2024-03-08T23:51:52.252361+0000","last_unstale":"2024-03-08T23:51:57.264712+0000","last_undegraded":"2024-03-08T23:51:57.264712+0000","last_fullsized":"2024-03-08T23:51:57.264712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:07.968262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130671+0000","last_change":"2024-03-08T23:51:56.284396+0000","last_active":"2024-03-08T23:51:58.130671+0000","last_peered":"2024-03-08T23:51:58.130671+0000","last_clean":"2024-03-08T23:51:58.130671+0000","last_became_active":"2024-03-08T23:51:56.283951+0000","last_became_peered":"2024-03-08T23:51:56.283951+0000","last_unstale":"2024-03-08T23:51:58.130671+0000","last_undegraded":"2024-03-08T23:51:58.130671+0000","last_fullsized":"2024-03-08T23:51:58.130671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:09:40.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278456+0000","last_change":"2024-03-08T23:51:51.248838+0000","last_active":"2024-03-08T23:51:57.278456+0000","last_peered":"2024-03-08T23:51:57.278456+0000","last_clean":"2024-03-08T23:51:57.278456+0000","last_became_active":"2024-03-08T23:51:51.248534+0000","last_became_peered":"2024-03-08T23:51:51.248534+0000","last_unstale":"2024-03-08T23:51:57.278456+0000","last_undegraded":"2024-03-08T23:51:57.278456+0000","last_fullsized":"2024-03-08T23:51:57.278456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:37:25.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264281+0000","last_change":"2024-03-08T23:51:52.264170+0000","last_active":"2024-03-08T23:51:57.264281+0000","last_peered":"2024-03-08T23:51:57.264281+0000","last_clean":"2024-03-08T23:51:57.264281+0000","last_became_active":"2024-03-08T23:51:52.263744+0000","last_became_peered":"2024-03-08T23:51:52.263744+0000","last_unstale":"2024-03-08T23:51:57.264281+0000","last_undegraded":"2024-03-08T23:51:57.264281+0000","last_fullsized":"2024-03-08T23:51:57.264281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:42:18.287699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278007+0000","last_change":"2024-03-08T23:51:56.283316+0000","last_active":"2024-03-08T23:51:57.278007+0000","last_peered":"2024-03-08T23:51:57.278007+0000","last_clean":"2024-03-08T23:51:57.278007+0000","last_became_active":"2024-03-08T23:51:56.282515+0000","last_became_peered":"2024-03-08T23:51:56.282515+0000","last_unstale":"2024-03-08T23:51:57.278007+0000","last_undegraded":"2024-03-08T23:51:57.278007+0000","last_fullsized":"2024-03-08T23:51:57.278007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:40:15.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651353+0000","last_change":"2024-03-08T23:51:51.240970+0000","last_active":"2024-03-08T23:51:57.651353+0000","last_peered":"2024-03-08T23:51:57.651353+0000","last_clean":"2024-03-08T23:51:57.651353+0000","last_became_active":"2024-03-08T23:51:51.239982+0000","last_became_peered":"2024-03-08T23:51:51.239982+0000","last_unstale":"2024-03-08T23:51:57.651353+0000","last_undegraded":"2024-03-08T23:51:57.651353+0000","last_fullsized":"2024-03-08T23:51:57.651353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:47:40.313753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264709+0000","last_change":"2024-03-08T23:51:52.261673+0000","last_active":"2024-03-08T23:51:57.264709+0000","last_peered":"2024-03-08T23:51:57.264709+0000","last_clean":"2024-03-08T23:51:57.264709+0000","last_became_active":"2024-03-08T23:51:52.261333+0000","last_became_peered":"2024-03-08T23:51:52.261333+0000","last_unstale":"2024-03-08T23:51:57.264709+0000","last_undegraded":"2024-03-08T23:51:57.264709+0000","last_fullsized":"2024-03-08T23:51:57.264709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:08:26.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403537+0000","last_change":"2024-03-08T23:51:52.243189+0000","last_active":"2024-03-08T23:51:58.403537+0000","last_peered":"2024-03-08T23:51:58.403537+0000","last_clean":"2024-03-08T23:51:58.403537+0000","last_became_active":"2024-03-08T23:51:52.242870+0000","last_became_peered":"2024-03-08T23:51:52.242870+0000","last_unstale":"2024-03-08T23:51:58.403537+0000","last_undegraded":"2024-03-08T23:51:58.403537+0000","last_fullsized":"2024-03-08T23:51:58.403537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.438045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265347+0000","last_change":"2024-03-08T23:51:52.252854+0000","last_active":"2024-03-08T23:51:57.265347+0000","last_peered":"2024-03-08T23:51:57.265347+0000","last_clean":"2024-03-08T23:51:57.265347+0000","last_became_active":"2024-03-08T23:51:52.252534+0000","last_became_peered":"2024-03-08T23:51:52.252534+0000","last_unstale":"2024-03-08T23:51:57.265347+0000","last_undegraded":"2024-03-08T23:51:57.265347+0000","last_fullsized":"2024-03-08T23:51:57.265347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:21:13.049717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265266+0000","last_change":"2024-03-08T23:51:52.251397+0000","last_active":"2024-03-08T23:51:57.265266+0000","last_peered":"2024-03-08T23:51:57.265266+0000","last_clean":"2024-03-08T23:51:57.265266+0000","last_became_active":"2024-03-08T23:51:52.251059+0000","last_became_peered":"2024-03-08T23:51:52.251059+0000","last_unstale":"2024-03-08T23:51:57.265266+0000","last_undegraded":"2024-03-08T23:51:57.265266+0000","last_fullsized":"2024-03-08T23:51:57.265266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:19:30.363457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130727+0000","last_change":"2024-03-08T23:51:52.249039+0000","last_active":"2024-03-08T23:51:58.130727+0000","last_peered":"2024-03-08T23:51:58.130727+0000","last_clean":"2024-03-08T23:51:58.130727+0000","last_became_active":"2024-03-08T23:51:52.248560+0000","last_became_peered":"2024-03-08T23:51:52.248560+0000","last_unstale":"2024-03-08T23:51:58.130727+0000","last_undegraded":"2024-03-08T23:51:58.130727+0000","last_fullsized":"2024-03-08T23:51:58.130727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651307+0000","last_change":"2024-03-08T23:51:52.247269+0000","last_active":"2024-03-08T23:51:57.651307+0000","last_peered":"2024-03-08T23:51:57.651307+0000","last_clean":"2024-03-08T23:51:57.651307+0000","last_became_active":"2024-03-08T23:51:52.246921+0000","last_became_peered":"2024-03-08T23:51:52.246921+0000","last_unstale":"2024-03-08T23:51:57.651307+0000","last_undegraded":"2024-03-08T23:51:57.651307+0000","last_fullsized":"2024-03-08T23:51:57.651307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:00:21.828530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403856+0000","last_change":"2024-03-08T23:51:52.238999+0000","last_active":"2024-03-08T23:51:58.403856+0000","last_peered":"2024-03-08T23:51:58.403856+0000","last_clean":"2024-03-08T23:51:58.403856+0000","last_became_active":"2024-03-08T23:51:52.238662+0000","last_became_peered":"2024-03-08T23:51:52.238662+0000","last_unstale":"2024-03-08T23:51:58.403856+0000","last_undegraded":"2024-03-08T23:51:58.403856+0000","last_fullsized":"2024-03-08T23:51:58.403856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:22:08.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265818+0000","last_change":"2024-03-08T23:51:52.238793+0000","last_active":"2024-03-08T23:51:57.265818+0000","last_peered":"2024-03-08T23:51:57.265818+0000","last_clean":"2024-03-08T23:51:57.265818+0000","last_became_active":"2024-03-08T23:51:52.238464+0000","last_became_peered":"2024-03-08T23:51:52.238464+0000","last_unstale":"2024-03-08T23:51:57.265818+0000","last_undegraded":"2024-03-08T23:51:57.265818+0000","last_fullsized":"2024-03-08T23:51:57.265818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:25.597428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123869+0000","last_change":"2024-03-08T23:51:52.253778+0000","last_active":"2024-03-08T23:51:58.123869+0000","last_peered":"2024-03-08T23:51:58.123869+0000","last_clean":"2024-03-08T23:51:58.123869+0000","last_became_active":"2024-03-08T23:51:52.253387+0000","last_became_peered":"2024-03-08T23:51:52.253387+0000","last_unstale":"2024-03-08T23:51:58.123869+0000","last_undegraded":"2024-03-08T23:51:58.123869+0000","last_fullsized":"2024-03-08T23:51:58.123869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:50:03.142133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278481+0000","last_change":"2024-03-08T23:51:52.245288+0000","last_active":"2024-03-08T23:51:57.278481+0000","last_peered":"2024-03-08T23:51:57.278481+0000","last_clean":"2024-03-08T23:51:57.278481+0000","last_became_active":"2024-03-08T23:51:52.244960+0000","last_became_peered":"2024-03-08T23:51:52.244960+0000","last_unstale":"2024-03-08T23:51:57.278481+0000","last_undegraded":"2024-03-08T23:51:57.278481+0000","last_fullsized":"2024-03-08T23:51:57.278481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:20:39.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123856+0000","last_change":"2024-03-08T23:51:52.252291+0000","last_active":"2024-03-08T23:51:58.123856+0000","last_peered":"2024-03-08T23:51:58.123856+0000","last_clean":"2024-03-08T23:51:58.123856+0000","last_became_active":"2024-03-08T23:51:52.251979+0000","last_became_peered":"2024-03-08T23:51:52.251979+0000","last_unstale":"2024-03-08T23:51:58.123856+0000","last_undegraded":"2024-03-08T23:51:58.123856+0000","last_fullsized":"2024-03-08T23:51:58.123856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:19:34.314784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403696+0000","last_change":"2024-03-08T23:51:52.251048+0000","last_active":"2024-03-08T23:51:58.403696+0000","last_peered":"2024-03-08T23:51:58.403696+0000","last_clean":"2024-03-08T23:51:58.403696+0000","last_became_active":"2024-03-08T23:51:52.250508+0000","last_became_peered":"2024-03-08T23:51:52.250508+0000","last_unstale":"2024-03-08T23:51:58.403696+0000","last_undegraded":"2024-03-08T23:51:58.403696+0000","last_fullsized":"2024-03-08T23:51:58.403696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:57:37.519307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651127+0000","last_change":"2024-03-08T23:51:52.244203+0000","last_active":"2024-03-08T23:51:57.651127+0000","last_peered":"2024-03-08T23:51:57.651127+0000","last_clean":"2024-03-08T23:51:57.651127+0000","last_became_active":"2024-03-08T23:51:52.243878+0000","last_became_peered":"2024-03-08T23:51:52.243878+0000","last_unstale":"2024-03-08T23:51:57.651127+0000","last_undegraded":"2024-03-08T23:51:57.651127+0000","last_fullsized":"2024-03-08T23:51:57.651127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:53:40.938657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403921+0000","last_change":"2024-03-08T23:51:52.248334+0000","last_active":"2024-03-08T23:51:58.403921+0000","last_peered":"2024-03-08T23:51:58.403921+0000","last_clean":"2024-03-08T23:51:58.403921+0000","last_became_active":"2024-03-08T23:51:52.248025+0000","last_became_peered":"2024-03-08T23:51:52.248025+0000","last_unstale":"2024-03-08T23:51:58.403921+0000","last_undegraded":"2024-03-08T23:51:58.403921+0000","last_fullsized":"2024-03-08T23:51:58.403921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:02:06.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":26035,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":18,"num_read_kb":36,"num_write":104,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":39906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1073003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":987136,"data_stored":477797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1070157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":481057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":477409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":477859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1069059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T23:52:21.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/642389645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102ec0 msgr2=0x7f8ef4109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/642389645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102ec0 0x7f8ef4109750 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009ca0 tx=0x7f8eec02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/642389645 shutdown_connections 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/642389645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102ec0 0x7f8ef4109750 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/642389645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef4102500 0x7f8ef4102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/642389645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4101310 0x7f8ef4101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/642389645 >> 172.21.15.22:0/642389645 conn(0x7f8ef40fc8a0 msgr2=0x7f8ef40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/642389645 shutdown_connections 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.261+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/642389645 wait complete. 2024-03-08T23:52:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 Processor -- start 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 -- start start 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef4101310 0x7f8ef41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102500 0x7f8ef4199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102ec0 0x7f8ef41a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef410cfa0 con 0x7f8ef4102500 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef410ce20 con 0x7f8ef4101310 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef9ccd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef410d120 con 0x7f8ef4102ec0 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102500 0x7f8ef4199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102500 0x7f8ef4199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45102/0 (socket says 172.21.15.22:45102) 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef2ffd640 1 -- 172.21.15.22:0/1899513961 learned_addr learned my addr 172.21.15.22:0/1899513961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.262+0000 7f8ef37fe640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef4101310 0x7f8ef41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef2ffd640 1 -- 172.21.15.22:0/1899513961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102ec0 msgr2=0x7f8ef41a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef2ffd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102ec0 0x7f8ef41a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef2ffd640 1 -- 172.21.15.22:0/1899513961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef4101310 msgr2=0x7f8ef41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef2ffd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef4101310 0x7f8ef41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef2ffd640 1 -- 172.21.15.22:0/1899513961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef41a13d0 con 0x7f8ef4102500 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef2ffd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102500 0x7f8ef4199cf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c890 tx=0x7f8ee800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8004220 con 0x7f8ef4102500 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ee80043c0 con 0x7f8ef4102500 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef41a16c0 con 0x7f8ef4102500 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80049e0 con 0x7f8ef4102500 2024-03-08T23:52:21.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.263+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef41a1c10 con 0x7f8ef4102500 2024-03-08T23:52:21.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.264+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef41a1ed0 con 0x7f8ef4102500 2024-03-08T23:52:21.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.265+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8ee80079b0 con 0x7f8ef4102500 2024-03-08T23:52:21.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.265+0000 7f8ef0ff9640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8edc0a2800 0x7f8edc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:21.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.266+0000 7f8ef37fe640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8edc0a2800 0x7f8edc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:21.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.266+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8ee80cedc0 con 0x7f8ef4102500 2024-03-08T23:52:21.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.266+0000 7f8ef37fe640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8edc0a2800 0x7f8edc0a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4005fd0 tx=0x7f8ee400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:21.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.270+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8ee8014030 con 0x7f8ef4102500 2024-03-08T23:52:21.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.356+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8ee8097b30 con 0x7f8ef4102500 2024-03-08T23:52:21.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.455+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ef41a20a0 con 0x7f8edc0a2800 2024-03-08T23:52:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.465+0000 7f8ef0ff9640 1 -- 172.21.15.22:0/1899513961 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340287 (secure 0 0 0) 0x7f8ef41a20a0 con 0x7f8edc0a2800 2024-03-08T23:52:21.465 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:21.466 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-08T23:52:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8edc0a2800 msgr2=0x7f8edc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8edc0a2800 0x7f8edc0a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4005fd0 tx=0x7f8ee400b390 comp rx=0 tx=0).stop 2024-03-08T23:52:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102500 msgr2=0x7f8ef4199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102500 0x7f8ef4199cf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c890 tx=0x7f8ee800cd50 comp rx=0 tx=0).stop 2024-03-08T23:52:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 shutdown_connections 2024-03-08T23:52:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102ec0 0x7f8ef41a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8edc0a2800 0x7f8edc0a4cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef4102500 0x7f8ef4199cf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 --2- 172.21.15.22:0/1899513961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef4101310 0x7f8ef41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:21.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 >> 172.21.15.22:0/1899513961 conn(0x7f8ef40fc8a0 msgr2=0x7f8ef4073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:21.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 shutdown_connections 2024-03-08T23:52:21.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:21.468+0000 7f8ef9ccd640 1 -- 172.21.15.22:0/1899513961 wait complete. 2024-03-08T23:52:21.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:21 smithi022 ceph-mon[31894]: pgmap v307: 129 pgs: 129 active+clean; 602 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-08T23:52:21.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:21 smithi022 ceph-mon[31894]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:21.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:21 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/889740035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:52:21.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:21 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2240445123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:21.476 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-03-08T23:52:19.792459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616520,"num_objects":48,"num_object_clones":0,"num_object_copies":145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":154,"num_read_kb":155,"num_write":286,"num_write_kb":2994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362608,"kb_used_data":13420,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579216,"statfs":{"total":1151940427776,"available":1151569117184,"internally_reserved":0,"allocated":13742080,"data_stored":7517076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21653,"num_objects":24,"num_object_clones":0,"num_object_copies":73,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":18,"num_read_kb":36,"num_write":58,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003774"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404049+0000","last_change":"2024-03-08T23:51:52.246270+0000","last_active":"2024-03-08T23:51:58.404049+0000","last_peered":"2024-03-08T23:51:58.404049+0000","last_clean":"2024-03-08T23:51:58.404049+0000","last_became_active":"2024-03-08T23:51:52.245321+0000","last_became_peered":"2024-03-08T23:51:52.245321+0000","last_unstale":"2024-03-08T23:51:58.404049+0000","last_undegraded":"2024-03-08T23:51:58.404049+0000","last_fullsized":"2024-03-08T23:51:58.404049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:39.993965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130853+0000","last_change":"2024-03-08T23:51:52.245433+0000","last_active":"2024-03-08T23:51:58.130853+0000","last_peered":"2024-03-08T23:51:58.130853+0000","last_clean":"2024-03-08T23:51:58.130853+0000","last_became_active":"2024-03-08T23:51:52.245106+0000","last_became_peered":"2024-03-08T23:51:52.245106+0000","last_unstale":"2024-03-08T23:51:58.130853+0000","last_undegraded":"2024-03-08T23:51:58.130853+0000","last_fullsized":"2024-03-08T23:51:58.130853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:02:20.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130821+0000","last_change":"2024-03-08T23:51:52.244112+0000","last_active":"2024-03-08T23:51:58.130821+0000","last_peered":"2024-03-08T23:51:58.130821+0000","last_clean":"2024-03-08T23:51:58.130821+0000","last_became_active":"2024-03-08T23:51:52.243783+0000","last_became_peered":"2024-03-08T23:51:52.243783+0000","last_unstale":"2024-03-08T23:51:58.130821+0000","last_undegraded":"2024-03-08T23:51:58.130821+0000","last_fullsized":"2024-03-08T23:51:58.130821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:48:43.719188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378367+0000","last_change":"2024-03-08T23:51:52.249375+0000","last_active":"2024-03-08T23:51:57.378367+0000","last_peered":"2024-03-08T23:51:57.378367+0000","last_clean":"2024-03-08T23:51:57.378367+0000","last_became_active":"2024-03-08T23:51:52.249045+0000","last_became_peered":"2024-03-08T23:51:52.249045+0000","last_unstale":"2024-03-08T23:51:57.378367+0000","last_undegraded":"2024-03-08T23:51:57.378367+0000","last_fullsized":"2024-03-08T23:51:57.378367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:43:33.788427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239576+0000","last_change":"2024-03-08T23:51:52.247091+0000","last_active":"2024-03-08T23:51:58.239576+0000","last_peered":"2024-03-08T23:51:58.239576+0000","last_clean":"2024-03-08T23:51:58.239576+0000","last_became_active":"2024-03-08T23:51:52.246761+0000","last_became_peered":"2024-03-08T23:51:52.246761+0000","last_unstale":"2024-03-08T23:51:58.239576+0000","last_undegraded":"2024-03-08T23:51:58.239576+0000","last_fullsized":"2024-03-08T23:51:58.239576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:46:46.810857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403691+0000","last_change":"2024-03-08T23:51:52.257558+0000","last_active":"2024-03-08T23:51:58.403691+0000","last_peered":"2024-03-08T23:51:58.403691+0000","last_clean":"2024-03-08T23:51:58.403691+0000","last_became_active":"2024-03-08T23:51:52.257116+0000","last_became_peered":"2024-03-08T23:51:52.257116+0000","last_unstale":"2024-03-08T23:51:58.403691+0000","last_undegraded":"2024-03-08T23:51:58.403691+0000","last_fullsized":"2024-03-08T23:51:58.403691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:13:05.325780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263915+0000","last_change":"2024-03-08T23:51:52.253624+0000","last_active":"2024-03-08T23:51:57.263915+0000","last_peered":"2024-03-08T23:51:57.263915+0000","last_clean":"2024-03-08T23:51:57.263915+0000","last_became_active":"2024-03-08T23:51:52.253284+0000","last_became_peered":"2024-03-08T23:51:52.253284+0000","last_unstale":"2024-03-08T23:51:57.263915+0000","last_undegraded":"2024-03-08T23:51:57.263915+0000","last_fullsized":"2024-03-08T23:51:57.263915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:09:33.789502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123350+0000","last_change":"2024-03-08T23:51:52.255321+0000","last_active":"2024-03-08T23:51:58.123350+0000","last_peered":"2024-03-08T23:51:58.123350+0000","last_clean":"2024-03-08T23:51:58.123350+0000","last_became_active":"2024-03-08T23:51:52.254975+0000","last_became_peered":"2024-03-08T23:51:52.254975+0000","last_unstale":"2024-03-08T23:51:58.123350+0000","last_undegraded":"2024-03-08T23:51:58.123350+0000","last_fullsized":"2024-03-08T23:51:58.123350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:02:56.788519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403983+0000","last_change":"2024-03-08T23:51:52.244623+0000","last_active":"2024-03-08T23:51:58.403983+0000","last_peered":"2024-03-08T23:51:58.403983+0000","last_clean":"2024-03-08T23:51:58.403983+0000","last_became_active":"2024-03-08T23:51:52.244162+0000","last_became_peered":"2024-03-08T23:51:52.244162+0000","last_unstale":"2024-03-08T23:51:58.403983+0000","last_undegraded":"2024-03-08T23:51:58.403983+0000","last_fullsized":"2024-03-08T23:51:58.403983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:30:43.050616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404325+0000","last_change":"2024-03-08T23:51:52.258006+0000","last_active":"2024-03-08T23:51:58.404325+0000","last_peered":"2024-03-08T23:51:58.404325+0000","last_clean":"2024-03-08T23:51:58.404325+0000","last_became_active":"2024-03-08T23:51:52.256770+0000","last_became_peered":"2024-03-08T23:51:52.256770+0000","last_unstale":"2024-03-08T23:51:58.404325+0000","last_undegraded":"2024-03-08T23:51:58.404325+0000","last_fullsized":"2024-03-08T23:51:58.404325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:18.584212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264671+0000","last_change":"2024-03-08T23:51:52.260530+0000","last_active":"2024-03-08T23:51:57.264671+0000","last_peered":"2024-03-08T23:51:57.264671+0000","last_clean":"2024-03-08T23:51:57.264671+0000","last_became_active":"2024-03-08T23:51:52.260193+0000","last_became_peered":"2024-03-08T23:51:52.260193+0000","last_unstale":"2024-03-08T23:51:57.264671+0000","last_undegraded":"2024-03-08T23:51:57.264671+0000","last_fullsized":"2024-03-08T23:51:57.264671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:02.549683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378265+0000","last_change":"2024-03-08T23:51:52.248679+0000","last_active":"2024-03-08T23:51:57.378265+0000","last_peered":"2024-03-08T23:51:57.378265+0000","last_clean":"2024-03-08T23:51:57.378265+0000","last_became_active":"2024-03-08T23:51:52.248358+0000","last_became_peered":"2024-03-08T23:51:52.248358+0000","last_unstale":"2024-03-08T23:51:57.378265+0000","last_undegraded":"2024-03-08T23:51:57.378265+0000","last_fullsized":"2024-03-08T23:51:57.378265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:26.680887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651313+0000","last_change":"2024-03-08T23:51:52.231797+0000","last_active":"2024-03-08T23:51:57.651313+0000","last_peered":"2024-03-08T23:51:57.651313+0000","last_clean":"2024-03-08T23:51:57.651313+0000","last_became_active":"2024-03-08T23:51:52.231425+0000","last_became_peered":"2024-03-08T23:51:52.231425+0000","last_unstale":"2024-03-08T23:51:57.651313+0000","last_undegraded":"2024-03-08T23:51:57.651313+0000","last_fullsized":"2024-03-08T23:51:57.651313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:51:30.659309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130452+0000","last_change":"2024-03-08T23:51:52.245394+0000","last_active":"2024-03-08T23:51:58.130452+0000","last_peered":"2024-03-08T23:51:58.130452+0000","last_clean":"2024-03-08T23:51:58.130452+0000","last_became_active":"2024-03-08T23:51:52.245038+0000","last_became_peered":"2024-03-08T23:51:52.245038+0000","last_unstale":"2024-03-08T23:51:58.130452+0000","last_undegraded":"2024-03-08T23:51:58.130452+0000","last_fullsized":"2024-03-08T23:51:58.130452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:15.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130510+0000","last_change":"2024-03-08T23:51:52.250861+0000","last_active":"2024-03-08T23:51:58.130510+0000","last_peered":"2024-03-08T23:51:58.130510+0000","last_clean":"2024-03-08T23:51:58.130510+0000","last_became_active":"2024-03-08T23:51:52.250548+0000","last_became_peered":"2024-03-08T23:51:52.250548+0000","last_unstale":"2024-03-08T23:51:58.130510+0000","last_undegraded":"2024-03-08T23:51:58.130510+0000","last_fullsized":"2024-03-08T23:51:58.130510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:00.280043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403190+0000","last_change":"2024-03-08T23:51:52.254996+0000","last_active":"2024-03-08T23:51:58.403190+0000","last_peered":"2024-03-08T23:51:58.403190+0000","last_clean":"2024-03-08T23:51:58.403190+0000","last_became_active":"2024-03-08T23:51:52.254634+0000","last_became_peered":"2024-03-08T23:51:52.254634+0000","last_unstale":"2024-03-08T23:51:58.403190+0000","last_undegraded":"2024-03-08T23:51:58.403190+0000","last_fullsized":"2024-03-08T23:51:58.403190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:15.449988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403921+0000","last_change":"2024-03-08T23:51:52.246558+0000","last_active":"2024-03-08T23:51:58.403921+0000","last_peered":"2024-03-08T23:51:58.403921+0000","last_clean":"2024-03-08T23:51:58.403921+0000","last_became_active":"2024-03-08T23:51:52.245741+0000","last_became_peered":"2024-03-08T23:51:52.245741+0000","last_unstale":"2024-03-08T23:51:58.403921+0000","last_undegraded":"2024-03-08T23:51:58.403921+0000","last_fullsized":"2024-03-08T23:51:58.403921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:36.737017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264263+0000","last_change":"2024-03-08T23:51:52.240111+0000","last_active":"2024-03-08T23:51:57.264263+0000","last_peered":"2024-03-08T23:51:57.264263+0000","last_clean":"2024-03-08T23:51:57.264263+0000","last_became_active":"2024-03-08T23:51:52.238961+0000","last_became_peered":"2024-03-08T23:51:52.238961+0000","last_unstale":"2024-03-08T23:51:57.264263+0000","last_undegraded":"2024-03-08T23:51:57.264263+0000","last_fullsized":"2024-03-08T23:51:57.264263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:03:55.313117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265808+0000","last_change":"2024-03-08T23:51:52.245979+0000","last_active":"2024-03-08T23:51:57.265808+0000","last_peered":"2024-03-08T23:51:57.265808+0000","last_clean":"2024-03-08T23:51:57.265808+0000","last_became_active":"2024-03-08T23:51:52.245666+0000","last_became_peered":"2024-03-08T23:51:52.245666+0000","last_unstale":"2024-03-08T23:51:57.265808+0000","last_undegraded":"2024-03-08T23:51:57.265808+0000","last_fullsized":"2024-03-08T23:51:57.265808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:20.789144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265553+0000","last_change":"2024-03-08T23:51:56.288161+0000","last_active":"2024-03-08T23:51:57.265553+0000","last_peered":"2024-03-08T23:51:57.265553+0000","last_clean":"2024-03-08T23:51:57.265553+0000","last_became_active":"2024-03-08T23:51:56.287778+0000","last_became_peered":"2024-03-08T23:51:56.287778+0000","last_unstale":"2024-03-08T23:51:57.265553+0000","last_undegraded":"2024-03-08T23:51:57.265553+0000","last_fullsized":"2024-03-08T23:51:57.265553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:12:59.636228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264583+0000","last_change":"2024-03-08T23:51:51.244911+0000","last_active":"2024-03-08T23:51:57.264583+0000","last_peered":"2024-03-08T23:51:57.264583+0000","last_clean":"2024-03-08T23:51:57.264583+0000","last_became_active":"2024-03-08T23:51:51.244561+0000","last_became_peered":"2024-03-08T23:51:51.244561+0000","last_unstale":"2024-03-08T23:51:57.264583+0000","last_undegraded":"2024-03-08T23:51:57.264583+0000","last_fullsized":"2024-03-08T23:51:57.264583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:58:45.908307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123954+0000","last_change":"2024-03-08T23:51:52.253854+0000","last_active":"2024-03-08T23:51:58.123954+0000","last_peered":"2024-03-08T23:51:58.123954+0000","last_clean":"2024-03-08T23:51:58.123954+0000","last_became_active":"2024-03-08T23:51:52.253538+0000","last_became_peered":"2024-03-08T23:51:52.253538+0000","last_unstale":"2024-03-08T23:51:58.123954+0000","last_undegraded":"2024-03-08T23:51:58.123954+0000","last_fullsized":"2024-03-08T23:51:58.123954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:33.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.377727+0000","last_change":"2024-03-08T23:51:56.283307+0000","last_active":"2024-03-08T23:51:57.377727+0000","last_peered":"2024-03-08T23:51:57.377727+0000","last_clean":"2024-03-08T23:51:57.377727+0000","last_became_active":"2024-03-08T23:51:56.282868+0000","last_became_peered":"2024-03-08T23:51:56.282868+0000","last_unstale":"2024-03-08T23:51:57.377727+0000","last_undegraded":"2024-03-08T23:51:57.377727+0000","last_fullsized":"2024-03-08T23:51:57.377727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:15.027743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.124107+0000","last_change":"2024-03-08T23:51:51.247368+0000","last_active":"2024-03-08T23:51:58.124107+0000","last_peered":"2024-03-08T23:51:58.124107+0000","last_clean":"2024-03-08T23:51:58.124107+0000","last_became_active":"2024-03-08T23:51:51.247023+0000","last_became_peered":"2024-03-08T23:51:51.247023+0000","last_unstale":"2024-03-08T23:51:58.124107+0000","last_undegraded":"2024-03-08T23:51:58.124107+0000","last_fullsized":"2024-03-08T23:51:58.124107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:28.311850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130889+0000","last_change":"2024-03-08T23:51:52.245805+0000","last_active":"2024-03-08T23:51:58.130889+0000","last_peered":"2024-03-08T23:51:58.130889+0000","last_clean":"2024-03-08T23:51:58.130889+0000","last_became_active":"2024-03-08T23:51:52.244455+0000","last_became_peered":"2024-03-08T23:51:52.244455+0000","last_unstale":"2024-03-08T23:51:58.130889+0000","last_undegraded":"2024-03-08T23:51:58.130889+0000","last_fullsized":"2024-03-08T23:51:58.130889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:11:17.809148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.377931+0000","last_change":"2024-03-08T23:51:56.286666+0000","last_active":"2024-03-08T23:51:57.377931+0000","last_peered":"2024-03-08T23:51:57.377931+0000","last_clean":"2024-03-08T23:51:57.377931+0000","last_became_active":"2024-03-08T23:51:56.285968+0000","last_became_peered":"2024-03-08T23:51:56.285968+0000","last_unstale":"2024-03-08T23:51:57.377931+0000","last_undegraded":"2024-03-08T23:51:57.377931+0000","last_fullsized":"2024-03-08T23:51:57.377931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:51.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.240154+0000","last_change":"2024-03-08T23:51:51.247684+0000","last_active":"2024-03-08T23:51:58.240154+0000","last_peered":"2024-03-08T23:51:58.240154+0000","last_clean":"2024-03-08T23:51:58.240154+0000","last_became_active":"2024-03-08T23:51:51.247374+0000","last_became_peered":"2024-03-08T23:51:51.247374+0000","last_unstale":"2024-03-08T23:51:58.240154+0000","last_undegraded":"2024-03-08T23:51:58.240154+0000","last_fullsized":"2024-03-08T23:51:58.240154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:20.425302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403526+0000","last_change":"2024-03-08T23:51:52.248702+0000","last_active":"2024-03-08T23:51:58.403526+0000","last_peered":"2024-03-08T23:51:58.403526+0000","last_clean":"2024-03-08T23:51:58.403526+0000","last_became_active":"2024-03-08T23:51:52.248388+0000","last_became_peered":"2024-03-08T23:51:52.248388+0000","last_unstale":"2024-03-08T23:51:58.403526+0000","last_undegraded":"2024-03-08T23:51:58.403526+0000","last_fullsized":"2024-03-08T23:51:58.403526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:27:12.251957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.377883+0000","last_change":"2024-03-08T23:51:56.286572+0000","last_active":"2024-03-08T23:51:57.377883+0000","last_peered":"2024-03-08T23:51:57.377883+0000","last_clean":"2024-03-08T23:51:57.377883+0000","last_became_active":"2024-03-08T23:51:56.286066+0000","last_became_peered":"2024-03-08T23:51:56.286066+0000","last_unstale":"2024-03-08T23:51:57.377883+0000","last_undegraded":"2024-03-08T23:51:57.377883+0000","last_fullsized":"2024-03-08T23:51:57.377883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:13.694615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239846+0000","last_change":"2024-03-08T23:51:51.244207+0000","last_active":"2024-03-08T23:51:58.239846+0000","last_peered":"2024-03-08T23:51:58.239846+0000","last_clean":"2024-03-08T23:51:58.239846+0000","last_became_active":"2024-03-08T23:51:51.241605+0000","last_became_peered":"2024-03-08T23:51:51.241605+0000","last_unstale":"2024-03-08T23:51:58.239846+0000","last_undegraded":"2024-03-08T23:51:58.239846+0000","last_fullsized":"2024-03-08T23:51:58.239846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:03:51.766681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278595+0000","last_change":"2024-03-08T23:51:52.251015+0000","last_active":"2024-03-08T23:51:57.278595+0000","last_peered":"2024-03-08T23:51:57.278595+0000","last_clean":"2024-03-08T23:51:57.278595+0000","last_became_active":"2024-03-08T23:51:52.250690+0000","last_became_peered":"2024-03-08T23:51:52.250690+0000","last_unstale":"2024-03-08T23:51:57.278595+0000","last_undegraded":"2024-03-08T23:51:57.278595+0000","last_fullsized":"2024-03-08T23:51:57.278595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:57:23.033172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265749+0000","last_change":"2024-03-08T23:51:56.281066+0000","last_active":"2024-03-08T23:51:57.265749+0000","last_peered":"2024-03-08T23:51:57.265749+0000","last_clean":"2024-03-08T23:51:57.265749+0000","last_became_active":"2024-03-08T23:51:56.280654+0000","last_became_peered":"2024-03-08T23:51:56.280654+0000","last_unstale":"2024-03-08T23:51:57.265749+0000","last_undegraded":"2024-03-08T23:51:57.265749+0000","last_fullsized":"2024-03-08T23:51:57.265749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:53.999115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.332528+0000","last_change":"2024-03-08T23:51:51.245947+0000","last_active":"2024-03-08T23:52:16.332528+0000","last_peered":"2024-03-08T23:52:16.332528+0000","last_clean":"2024-03-08T23:52:16.332528+0000","last_became_active":"2024-03-08T23:51:51.245505+0000","last_became_peered":"2024-03-08T23:51:51.245505+0000","last_unstale":"2024-03-08T23:52:16.332528+0000","last_undegraded":"2024-03-08T23:52:16.332528+0000","last_fullsized":"2024-03-08T23:52:16.332528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:20:44.692598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239595+0000","last_change":"2024-03-08T23:51:52.241590+0000","last_active":"2024-03-08T23:51:58.239595+0000","last_peered":"2024-03-08T23:51:58.239595+0000","last_clean":"2024-03-08T23:51:58.239595+0000","last_became_active":"2024-03-08T23:51:52.241296+0000","last_became_peered":"2024-03-08T23:51:52.241296+0000","last_unstale":"2024-03-08T23:51:58.239595+0000","last_undegraded":"2024-03-08T23:51:58.239595+0000","last_fullsized":"2024-03-08T23:51:58.239595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:52:48.042360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265748+0000","last_change":"2024-03-08T23:51:56.280897+0000","last_active":"2024-03-08T23:51:57.265748+0000","last_peered":"2024-03-08T23:51:57.265748+0000","last_clean":"2024-03-08T23:51:57.265748+0000","last_became_active":"2024-03-08T23:51:56.280511+0000","last_became_peered":"2024-03-08T23:51:56.280511+0000","last_unstale":"2024-03-08T23:51:57.265748+0000","last_undegraded":"2024-03-08T23:51:57.265748+0000","last_fullsized":"2024-03-08T23:51:57.265748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:12:26.963132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:17.342088+0000","last_change":"2024-03-08T23:51:51.241384+0000","last_active":"2024-03-08T23:52:17.342088+0000","last_peered":"2024-03-08T23:52:17.342088+0000","last_clean":"2024-03-08T23:52:17.342088+0000","last_became_active":"2024-03-08T23:51:51.241018+0000","last_became_peered":"2024-03-08T23:51:51.241018+0000","last_unstale":"2024-03-08T23:52:17.342088+0000","last_undegraded":"2024-03-08T23:52:17.342088+0000","last_fullsized":"2024-03-08T23:52:17.342088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:29:50.860663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278276+0000","last_change":"2024-03-08T23:51:52.250207+0000","last_active":"2024-03-08T23:51:57.278276+0000","last_peered":"2024-03-08T23:51:57.278276+0000","last_clean":"2024-03-08T23:51:57.278276+0000","last_became_active":"2024-03-08T23:51:52.249863+0000","last_became_peered":"2024-03-08T23:51:52.249863+0000","last_unstale":"2024-03-08T23:51:57.278276+0000","last_undegraded":"2024-03-08T23:51:57.278276+0000","last_fullsized":"2024-03-08T23:51:57.278276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:18:48.732419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403965+0000","last_change":"2024-03-08T23:51:56.282092+0000","last_active":"2024-03-08T23:51:58.403965+0000","last_peered":"2024-03-08T23:51:58.403965+0000","last_clean":"2024-03-08T23:51:58.403965+0000","last_became_active":"2024-03-08T23:51:56.281657+0000","last_became_peered":"2024-03-08T23:51:56.281657+0000","last_unstale":"2024-03-08T23:51:58.403965+0000","last_undegraded":"2024-03-08T23:51:58.403965+0000","last_fullsized":"2024-03-08T23:51:58.403965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:50.440520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:17.342820+0000","last_change":"2024-03-08T23:51:51.236816+0000","last_active":"2024-03-08T23:52:17.342820+0000","last_peered":"2024-03-08T23:52:17.342820+0000","last_clean":"2024-03-08T23:52:17.342820+0000","last_became_active":"2024-03-08T23:51:51.236485+0000","last_became_peered":"2024-03-08T23:51:51.236485+0000","last_unstale":"2024-03-08T23:52:17.342820+0000","last_undegraded":"2024-03-08T23:52:17.342820+0000","last_fullsized":"2024-03-08T23:52:17.342820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:27.300950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123476+0000","last_change":"2024-03-08T23:51:52.252146+0000","last_active":"2024-03-08T23:51:58.123476+0000","last_peered":"2024-03-08T23:51:58.123476+0000","last_clean":"2024-03-08T23:51:58.123476+0000","last_became_active":"2024-03-08T23:51:52.244201+0000","last_became_peered":"2024-03-08T23:51:52.244201+0000","last_unstale":"2024-03-08T23:51:58.123476+0000","last_undegraded":"2024-03-08T23:51:58.123476+0000","last_fullsized":"2024-03-08T23:51:58.123476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:07.647078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130945+0000","last_change":"2024-03-08T23:51:56.279727+0000","last_active":"2024-03-08T23:51:58.130945+0000","last_peered":"2024-03-08T23:51:58.130945+0000","last_clean":"2024-03-08T23:51:58.130945+0000","last_became_active":"2024-03-08T23:51:56.279184+0000","last_became_peered":"2024-03-08T23:51:56.279184+0000","last_unstale":"2024-03-08T23:51:58.130945+0000","last_undegraded":"2024-03-08T23:51:58.130945+0000","last_fullsized":"2024-03-08T23:51:58.130945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:21.804410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278148+0000","last_change":"2024-03-08T23:51:51.244812+0000","last_active":"2024-03-08T23:51:57.278148+0000","last_peered":"2024-03-08T23:51:57.278148+0000","last_clean":"2024-03-08T23:51:57.278148+0000","last_became_active":"2024-03-08T23:51:51.244513+0000","last_became_peered":"2024-03-08T23:51:51.244513+0000","last_unstale":"2024-03-08T23:51:57.278148+0000","last_undegraded":"2024-03-08T23:51:57.278148+0000","last_fullsized":"2024-03-08T23:51:57.278148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:56:07.235214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263775+0000","last_change":"2024-03-08T23:51:52.244566+0000","last_active":"2024-03-08T23:51:57.263775+0000","last_peered":"2024-03-08T23:51:57.263775+0000","last_clean":"2024-03-08T23:51:57.263775+0000","last_became_active":"2024-03-08T23:51:52.244250+0000","last_became_peered":"2024-03-08T23:51:52.244250+0000","last_unstale":"2024-03-08T23:51:57.263775+0000","last_undegraded":"2024-03-08T23:51:57.263775+0000","last_fullsized":"2024-03-08T23:51:57.263775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:55:50.596364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123631+0000","last_change":"2024-03-08T23:51:56.282487+0000","last_active":"2024-03-08T23:51:58.123631+0000","last_peered":"2024-03-08T23:51:58.123631+0000","last_clean":"2024-03-08T23:51:58.123631+0000","last_became_active":"2024-03-08T23:51:56.282104+0000","last_became_peered":"2024-03-08T23:51:56.282104+0000","last_unstale":"2024-03-08T23:51:58.123631+0000","last_undegraded":"2024-03-08T23:51:58.123631+0000","last_fullsized":"2024-03-08T23:51:58.123631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:23:23.787431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.345408+0000","last_change":"2024-03-08T23:51:51.236612+0000","last_active":"2024-03-08T23:52:18.345408+0000","last_peered":"2024-03-08T23:52:18.345408+0000","last_clean":"2024-03-08T23:52:18.345408+0000","last_became_active":"2024-03-08T23:51:51.236274+0000","last_became_peered":"2024-03-08T23:51:51.236274+0000","last_unstale":"2024-03-08T23:52:18.345408+0000","last_undegraded":"2024-03-08T23:52:18.345408+0000","last_fullsized":"2024-03-08T23:52:18.345408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:37.551870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404194+0000","last_change":"2024-03-08T23:51:52.244564+0000","last_active":"2024-03-08T23:51:58.404194+0000","last_peered":"2024-03-08T23:51:58.404194+0000","last_clean":"2024-03-08T23:51:58.404194+0000","last_became_active":"2024-03-08T23:51:52.243651+0000","last_became_peered":"2024-03-08T23:51:52.243651+0000","last_unstale":"2024-03-08T23:51:58.404194+0000","last_undegraded":"2024-03-08T23:51:58.404194+0000","last_fullsized":"2024-03-08T23:51:58.404194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:35:32.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263880+0000","last_change":"2024-03-08T23:51:56.279354+0000","last_active":"2024-03-08T23:51:57.263880+0000","last_peered":"2024-03-08T23:51:57.263880+0000","last_clean":"2024-03-08T23:51:57.263880+0000","last_became_active":"2024-03-08T23:51:56.278894+0000","last_became_peered":"2024-03-08T23:51:56.278894+0000","last_unstale":"2024-03-08T23:51:57.263880+0000","last_undegraded":"2024-03-08T23:51:57.263880+0000","last_fullsized":"2024-03-08T23:51:57.263880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:20:30.164542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.331291+0000","last_change":"2024-03-08T23:51:51.246417+0000","last_active":"2024-03-08T23:52:16.331291+0000","last_peered":"2024-03-08T23:52:16.331291+0000","last_clean":"2024-03-08T23:52:16.331291+0000","last_became_active":"2024-03-08T23:51:51.246058+0000","last_became_peered":"2024-03-08T23:51:51.246058+0000","last_unstale":"2024-03-08T23:52:16.331291+0000","last_undegraded":"2024-03-08T23:52:16.331291+0000","last_fullsized":"2024-03-08T23:52:16.331291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:48:53.569295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130924+0000","last_change":"2024-03-08T23:51:52.255357+0000","last_active":"2024-03-08T23:51:58.130924+0000","last_peered":"2024-03-08T23:51:58.130924+0000","last_clean":"2024-03-08T23:51:58.130924+0000","last_became_active":"2024-03-08T23:51:52.255044+0000","last_became_peered":"2024-03-08T23:51:52.255044+0000","last_unstale":"2024-03-08T23:51:58.130924+0000","last_undegraded":"2024-03-08T23:51:58.130924+0000","last_fullsized":"2024-03-08T23:51:58.130924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:13:40.710283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378179+0000","last_change":"2024-03-08T23:51:56.284282+0000","last_active":"2024-03-08T23:51:57.378179+0000","last_peered":"2024-03-08T23:51:57.378179+0000","last_clean":"2024-03-08T23:51:57.378179+0000","last_became_active":"2024-03-08T23:51:56.283863+0000","last_became_peered":"2024-03-08T23:51:56.283863+0000","last_unstale":"2024-03-08T23:51:57.378179+0000","last_undegraded":"2024-03-08T23:51:57.378179+0000","last_fullsized":"2024-03-08T23:51:57.378179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:59:21.972283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278376+0000","last_change":"2024-03-08T23:51:51.245199+0000","last_active":"2024-03-08T23:51:57.278376+0000","last_peered":"2024-03-08T23:51:57.278376+0000","last_clean":"2024-03-08T23:51:57.278376+0000","last_became_active":"2024-03-08T23:51:51.244906+0000","last_became_peered":"2024-03-08T23:51:51.244906+0000","last_unstale":"2024-03-08T23:51:57.278376+0000","last_undegraded":"2024-03-08T23:51:57.278376+0000","last_fullsized":"2024-03-08T23:51:57.278376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:54:39.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403693+0000","last_change":"2024-03-08T23:51:52.251209+0000","last_active":"2024-03-08T23:51:58.403693+0000","last_peered":"2024-03-08T23:51:58.403693+0000","last_clean":"2024-03-08T23:51:58.403693+0000","last_became_active":"2024-03-08T23:51:52.250392+0000","last_became_peered":"2024-03-08T23:51:52.250392+0000","last_unstale":"2024-03-08T23:51:58.403693+0000","last_undegraded":"2024-03-08T23:51:58.403693+0000","last_fullsized":"2024-03-08T23:51:58.403693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:56:17.811505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263768+0000","last_change":"2024-03-08T23:51:56.281154+0000","last_active":"2024-03-08T23:51:57.263768+0000","last_peered":"2024-03-08T23:51:57.263768+0000","last_clean":"2024-03-08T23:51:57.263768+0000","last_became_active":"2024-03-08T23:51:56.280736+0000","last_became_peered":"2024-03-08T23:51:56.280736+0000","last_unstale":"2024-03-08T23:51:57.263768+0000","last_undegraded":"2024-03-08T23:51:57.263768+0000","last_fullsized":"2024-03-08T23:51:57.263768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:01:49.909731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'19","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:19.349714+0000","last_change":"2024-03-08T23:51:51.239716+0000","last_active":"2024-03-08T23:52:19.349714+0000","last_peered":"2024-03-08T23:52:19.349714+0000","last_clean":"2024-03-08T23:52:19.349714+0000","last_became_active":"2024-03-08T23:51:51.239384+0000","last_became_peered":"2024-03-08T23:51:51.239384+0000","last_unstale":"2024-03-08T23:52:19.349714+0000","last_undegraded":"2024-03-08T23:52:19.349714+0000","last_fullsized":"2024-03-08T23:52:19.349714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:40.892723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239953+0000","last_change":"2024-03-08T23:51:52.248715+0000","last_active":"2024-03-08T23:51:58.239953+0000","last_peered":"2024-03-08T23:51:58.239953+0000","last_clean":"2024-03-08T23:51:58.239953+0000","last_became_active":"2024-03-08T23:51:52.248258+0000","last_became_peered":"2024-03-08T23:51:52.248258+0000","last_unstale":"2024-03-08T23:51:58.239953+0000","last_undegraded":"2024-03-08T23:51:58.239953+0000","last_fullsized":"2024-03-08T23:51:58.239953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:38.185469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278754+0000","last_change":"2024-03-08T23:51:56.278268+0000","last_active":"2024-03-08T23:51:57.278754+0000","last_peered":"2024-03-08T23:51:57.278754+0000","last_clean":"2024-03-08T23:51:57.278754+0000","last_became_active":"2024-03-08T23:51:56.277831+0000","last_became_peered":"2024-03-08T23:51:56.277831+0000","last_unstale":"2024-03-08T23:51:57.278754+0000","last_undegraded":"2024-03-08T23:51:57.278754+0000","last_fullsized":"2024-03-08T23:51:57.278754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:32:13.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.331231+0000","last_change":"2024-03-08T23:51:51.247553+0000","last_active":"2024-03-08T23:52:16.331231+0000","last_peered":"2024-03-08T23:52:16.331231+0000","last_clean":"2024-03-08T23:52:16.331231+0000","last_became_active":"2024-03-08T23:51:51.247225+0000","last_became_peered":"2024-03-08T23:51:51.247225+0000","last_unstale":"2024-03-08T23:52:16.331231+0000","last_undegraded":"2024-03-08T23:52:16.331231+0000","last_fullsized":"2024-03-08T23:52:16.331231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:53:29.846303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651115+0000","last_change":"2024-03-08T23:51:52.247733+0000","last_active":"2024-03-08T23:51:57.651115+0000","last_peered":"2024-03-08T23:51:57.651115+0000","last_clean":"2024-03-08T23:51:57.651115+0000","last_became_active":"2024-03-08T23:51:52.247435+0000","last_became_peered":"2024-03-08T23:51:52.247435+0000","last_unstale":"2024-03-08T23:51:57.651115+0000","last_undegraded":"2024-03-08T23:51:57.651115+0000","last_fullsized":"2024-03-08T23:51:57.651115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:46:12.739601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403746+0000","last_change":"2024-03-08T23:51:56.279865+0000","last_active":"2024-03-08T23:51:58.403746+0000","last_peered":"2024-03-08T23:51:58.403746+0000","last_clean":"2024-03-08T23:51:58.403746+0000","last_became_active":"2024-03-08T23:51:56.279446+0000","last_became_peered":"2024-03-08T23:51:56.279446+0000","last_unstale":"2024-03-08T23:51:58.403746+0000","last_undegraded":"2024-03-08T23:51:58.403746+0000","last_fullsized":"2024-03-08T23:51:58.403746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:07:33.672490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.332381+0000","last_change":"2024-03-08T23:51:51.246860+0000","last_active":"2024-03-08T23:52:16.332381+0000","last_peered":"2024-03-08T23:52:16.332381+0000","last_clean":"2024-03-08T23:52:16.332381+0000","last_became_active":"2024-03-08T23:51:51.246545+0000","last_became_peered":"2024-03-08T23:51:51.246545+0000","last_unstale":"2024-03-08T23:52:16.332381+0000","last_undegraded":"2024-03-08T23:52:16.332381+0000","last_fullsized":"2024-03-08T23:52:16.332381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:11:07.246657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264375+0000","last_change":"2024-03-08T23:51:52.247353+0000","last_active":"2024-03-08T23:51:57.264375+0000","last_peered":"2024-03-08T23:51:57.264375+0000","last_clean":"2024-03-08T23:51:57.264375+0000","last_became_active":"2024-03-08T23:51:52.247008+0000","last_became_peered":"2024-03-08T23:51:52.247008+0000","last_unstale":"2024-03-08T23:51:57.264375+0000","last_undegraded":"2024-03-08T23:51:57.264375+0000","last_fullsized":"2024-03-08T23:51:57.264375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:08:50.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264384+0000","last_change":"2024-03-08T23:51:56.284103+0000","last_active":"2024-03-08T23:51:57.264384+0000","last_peered":"2024-03-08T23:51:57.264384+0000","last_clean":"2024-03-08T23:51:57.264384+0000","last_became_active":"2024-03-08T23:51:56.283561+0000","last_became_peered":"2024-03-08T23:51:56.283561+0000","last_unstale":"2024-03-08T23:51:57.264384+0000","last_undegraded":"2024-03-08T23:51:57.264384+0000","last_fullsized":"2024-03-08T23:51:57.264384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:47:20.909292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.345862+0000","last_change":"2024-03-08T23:51:51.242483+0000","last_active":"2024-03-08T23:52:18.345862+0000","last_peered":"2024-03-08T23:52:18.345862+0000","last_clean":"2024-03-08T23:52:18.345862+0000","last_became_active":"2024-03-08T23:51:51.242153+0000","last_became_peered":"2024-03-08T23:51:51.242153+0000","last_unstale":"2024-03-08T23:52:18.345862+0000","last_undegraded":"2024-03-08T23:52:18.345862+0000","last_fullsized":"2024-03-08T23:52:18.345862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:44:07.728700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264068+0000","last_change":"2024-03-08T23:51:52.255411+0000","last_active":"2024-03-08T23:51:57.264068+0000","last_peered":"2024-03-08T23:51:57.264068+0000","last_clean":"2024-03-08T23:51:57.264068+0000","last_became_active":"2024-03-08T23:51:52.255048+0000","last_became_peered":"2024-03-08T23:51:52.255048+0000","last_unstale":"2024-03-08T23:51:57.264068+0000","last_undegraded":"2024-03-08T23:51:57.264068+0000","last_fullsized":"2024-03-08T23:51:57.264068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:11.836187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651515+0000","last_change":"2024-03-08T23:51:56.286405+0000","last_active":"2024-03-08T23:51:57.651515+0000","last_peered":"2024-03-08T23:51:57.651515+0000","last_clean":"2024-03-08T23:51:57.651515+0000","last_became_active":"2024-03-08T23:51:56.285996+0000","last_became_peered":"2024-03-08T23:51:56.285996+0000","last_unstale":"2024-03-08T23:51:57.651515+0000","last_undegraded":"2024-03-08T23:51:57.651515+0000","last_fullsized":"2024-03-08T23:51:57.651515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:48:30.928436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239452+0000","last_change":"2024-03-08T23:51:51.240146+0000","last_active":"2024-03-08T23:51:58.239452+0000","last_peered":"2024-03-08T23:51:58.239452+0000","last_clean":"2024-03-08T23:51:58.239452+0000","last_became_active":"2024-03-08T23:51:51.239795+0000","last_became_peered":"2024-03-08T23:51:51.239795+0000","last_unstale":"2024-03-08T23:51:58.239452+0000","last_undegraded":"2024-03-08T23:51:58.239452+0000","last_fullsized":"2024-03-08T23:51:58.239452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:55.409922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278114+0000","last_change":"2024-03-08T23:51:52.254701+0000","last_active":"2024-03-08T23:51:57.278114+0000","last_peered":"2024-03-08T23:51:57.278114+0000","last_clean":"2024-03-08T23:51:57.278114+0000","last_became_active":"2024-03-08T23:51:52.254391+0000","last_became_peered":"2024-03-08T23:51:52.254391+0000","last_unstale":"2024-03-08T23:51:57.278114+0000","last_undegraded":"2024-03-08T23:51:57.278114+0000","last_fullsized":"2024-03-08T23:51:57.278114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:11:12.258181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278244+0000","last_change":"2024-03-08T23:51:56.277878+0000","last_active":"2024-03-08T23:51:57.278244+0000","last_peered":"2024-03-08T23:51:57.278244+0000","last_clean":"2024-03-08T23:51:57.278244+0000","last_became_active":"2024-03-08T23:51:56.277493+0000","last_became_peered":"2024-03-08T23:51:56.277493+0000","last_unstale":"2024-03-08T23:51:57.278244+0000","last_undegraded":"2024-03-08T23:51:57.278244+0000","last_fullsized":"2024-03-08T23:51:57.278244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:44:16.968632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.240086+0000","last_change":"2024-03-08T23:51:51.241457+0000","last_active":"2024-03-08T23:51:58.240086+0000","last_peered":"2024-03-08T23:51:58.240086+0000","last_clean":"2024-03-08T23:51:58.240086+0000","last_became_active":"2024-03-08T23:51:51.241148+0000","last_became_peered":"2024-03-08T23:51:51.241148+0000","last_unstale":"2024-03-08T23:51:58.240086+0000","last_undegraded":"2024-03-08T23:51:58.240086+0000","last_fullsized":"2024-03-08T23:51:58.240086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:13:27.246265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403822+0000","last_change":"2024-03-08T23:51:52.248465+0000","last_active":"2024-03-08T23:51:58.403822+0000","last_peered":"2024-03-08T23:51:58.403822+0000","last_clean":"2024-03-08T23:51:58.403822+0000","last_became_active":"2024-03-08T23:51:52.248135+0000","last_became_peered":"2024-03-08T23:51:52.248135+0000","last_unstale":"2024-03-08T23:51:58.403822+0000","last_undegraded":"2024-03-08T23:51:58.403822+0000","last_fullsized":"2024-03-08T23:51:58.403822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:55:53.712958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239727+0000","last_change":"2024-03-08T23:51:56.276019+0000","last_active":"2024-03-08T23:51:58.239727+0000","last_peered":"2024-03-08T23:51:58.239727+0000","last_clean":"2024-03-08T23:51:58.239727+0000","last_became_active":"2024-03-08T23:51:56.275564+0000","last_became_peered":"2024-03-08T23:51:56.275564+0000","last_unstale":"2024-03-08T23:51:58.239727+0000","last_undegraded":"2024-03-08T23:51:58.239727+0000","last_fullsized":"2024-03-08T23:51:58.239727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:12:36.096470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263962+0000","last_change":"2024-03-08T23:51:51.241629+0000","last_active":"2024-03-08T23:51:57.263962+0000","last_peered":"2024-03-08T23:51:57.263962+0000","last_clean":"2024-03-08T23:51:57.263962+0000","last_became_active":"2024-03-08T23:51:51.241300+0000","last_became_peered":"2024-03-08T23:51:51.241300+0000","last_unstale":"2024-03-08T23:51:57.263962+0000","last_undegraded":"2024-03-08T23:51:57.263962+0000","last_fullsized":"2024-03-08T23:51:57.263962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:01:45.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404171+0000","last_change":"2024-03-08T23:51:52.246024+0000","last_active":"2024-03-08T23:51:58.404171+0000","last_peered":"2024-03-08T23:51:58.404171+0000","last_clean":"2024-03-08T23:51:58.404171+0000","last_became_active":"2024-03-08T23:51:52.245706+0000","last_became_peered":"2024-03-08T23:51:52.245706+0000","last_unstale":"2024-03-08T23:51:58.404171+0000","last_undegraded":"2024-03-08T23:51:58.404171+0000","last_fullsized":"2024-03-08T23:51:58.404171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:00:44.557269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651351+0000","last_change":"2024-03-08T23:51:56.285649+0000","last_active":"2024-03-08T23:51:57.651351+0000","last_peered":"2024-03-08T23:51:57.651351+0000","last_clean":"2024-03-08T23:51:57.651351+0000","last_became_active":"2024-03-08T23:51:56.282731+0000","last_became_peered":"2024-03-08T23:51:56.282731+0000","last_unstale":"2024-03-08T23:51:57.651351+0000","last_undegraded":"2024-03-08T23:51:57.651351+0000","last_fullsized":"2024-03-08T23:51:57.651351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:23:54.103637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264292+0000","last_change":"2024-03-08T23:51:51.249442+0000","last_active":"2024-03-08T23:51:57.264292+0000","last_peered":"2024-03-08T23:51:57.264292+0000","last_clean":"2024-03-08T23:51:57.264292+0000","last_became_active":"2024-03-08T23:51:51.249084+0000","last_became_peered":"2024-03-08T23:51:51.249084+0000","last_unstale":"2024-03-08T23:51:57.264292+0000","last_undegraded":"2024-03-08T23:51:57.264292+0000","last_fullsized":"2024-03-08T23:51:57.264292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:48:19.370095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.404058+0000","last_change":"2024-03-08T23:51:52.257586+0000","last_active":"2024-03-08T23:51:58.404058+0000","last_peered":"2024-03-08T23:51:58.404058+0000","last_clean":"2024-03-08T23:51:58.404058+0000","last_became_active":"2024-03-08T23:51:52.253741+0000","last_became_peered":"2024-03-08T23:51:52.253741+0000","last_unstale":"2024-03-08T23:51:58.404058+0000","last_undegraded":"2024-03-08T23:51:58.404058+0000","last_fullsized":"2024-03-08T23:51:58.404058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:18.633166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651390+0000","last_change":"2024-03-08T23:51:56.285264+0000","last_active":"2024-03-08T23:51:57.651390+0000","last_peered":"2024-03-08T23:51:57.651390+0000","last_clean":"2024-03-08T23:51:57.651390+0000","last_became_active":"2024-03-08T23:51:56.284727+0000","last_became_peered":"2024-03-08T23:51:56.284727+0000","last_unstale":"2024-03-08T23:51:57.651390+0000","last_undegraded":"2024-03-08T23:51:57.651390+0000","last_fullsized":"2024-03-08T23:51:57.651390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:01:20.723010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278562+0000","last_change":"2024-03-08T23:51:52.251461+0000","last_active":"2024-03-08T23:51:57.278562+0000","last_peered":"2024-03-08T23:51:57.278562+0000","last_clean":"2024-03-08T23:51:57.278562+0000","last_became_active":"2024-03-08T23:51:52.250996+0000","last_became_peered":"2024-03-08T23:51:52.250996+0000","last_unstale":"2024-03-08T23:51:57.278562+0000","last_undegraded":"2024-03-08T23:51:57.278562+0000","last_fullsized":"2024-03-08T23:51:57.278562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:51.719142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.345280+0000","last_change":"2024-03-08T23:51:51.245698+0000","last_active":"2024-03-08T23:52:18.345280+0000","last_peered":"2024-03-08T23:52:18.345280+0000","last_clean":"2024-03-08T23:52:18.345280+0000","last_became_active":"2024-03-08T23:51:51.245389+0000","last_became_peered":"2024-03-08T23:51:51.245389+0000","last_unstale":"2024-03-08T23:52:18.345280+0000","last_undegraded":"2024-03-08T23:52:18.345280+0000","last_fullsized":"2024-03-08T23:52:18.345280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:14:08.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:59.677777+0000","last_change":"2024-03-08T23:51:56.291884+0000","last_active":"2024-03-08T23:51:59.677777+0000","last_peered":"2024-03-08T23:51:59.677777+0000","last_clean":"2024-03-08T23:51:59.677777+0000","last_became_active":"2024-03-08T23:51:56.291419+0000","last_became_peered":"2024-03-08T23:51:56.291419+0000","last_unstale":"2024-03-08T23:51:59.677777+0000","last_undegraded":"2024-03-08T23:51:59.677777+0000","last_fullsized":"2024-03-08T23:51:59.677777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:57:03.971483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123537+0000","last_change":"2024-03-08T23:51:52.256518+0000","last_active":"2024-03-08T23:51:58.123537+0000","last_peered":"2024-03-08T23:51:58.123537+0000","last_clean":"2024-03-08T23:51:58.123537+0000","last_became_active":"2024-03-08T23:51:52.253481+0000","last_became_peered":"2024-03-08T23:51:52.253481+0000","last_unstale":"2024-03-08T23:51:58.123537+0000","last_undegraded":"2024-03-08T23:51:58.123537+0000","last_fullsized":"2024-03-08T23:51:58.123537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:35.287408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:18.343701+0000","last_change":"2024-03-08T23:51:51.237921+0000","last_active":"2024-03-08T23:52:18.343701+0000","last_peered":"2024-03-08T23:52:18.343701+0000","last_clean":"2024-03-08T23:52:18.343701+0000","last_became_active":"2024-03-08T23:51:51.237568+0000","last_became_peered":"2024-03-08T23:51:51.237568+0000","last_unstale":"2024-03-08T23:52:18.343701+0000","last_undegraded":"2024-03-08T23:52:18.343701+0000","last_fullsized":"2024-03-08T23:52:18.343701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:11.256814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.263913+0000","last_change":"2024-03-08T23:50:41.802121+0000","last_active":"2024-03-08T23:51:57.263913+0000","last_peered":"2024-03-08T23:51:57.263913+0000","last_clean":"2024-03-08T23:51:57.263913+0000","last_became_active":"2024-03-08T23:50:41.801678+0000","last_became_peered":"2024-03-08T23:50:41.801678+0000","last_unstale":"2024-03-08T23:51:57.263913+0000","last_undegraded":"2024-03-08T23:51:57.263913+0000","last_fullsized":"2024-03-08T23:51:57.263913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_clean_scrub_stamp":"2024-03-08T23:47:12.382547+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:15.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278544+0000","last_change":"2024-03-08T23:51:56.278465+0000","last_active":"2024-03-08T23:51:57.278544+0000","last_peered":"2024-03-08T23:51:57.278544+0000","last_clean":"2024-03-08T23:51:57.278544+0000","last_became_active":"2024-03-08T23:51:56.278079+0000","last_became_peered":"2024-03-08T23:51:56.278079+0000","last_unstale":"2024-03-08T23:51:57.278544+0000","last_undegraded":"2024-03-08T23:51:57.278544+0000","last_fullsized":"2024-03-08T23:51:57.278544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:47.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.650867+0000","last_change":"2024-03-08T23:51:52.251895+0000","last_active":"2024-03-08T23:51:57.650867+0000","last_peered":"2024-03-08T23:51:57.650867+0000","last_clean":"2024-03-08T23:51:57.650867+0000","last_became_active":"2024-03-08T23:51:52.251591+0000","last_became_peered":"2024-03-08T23:51:52.251591+0000","last_unstale":"2024-03-08T23:51:57.650867+0000","last_undegraded":"2024-03-08T23:51:57.650867+0000","last_fullsized":"2024-03-08T23:51:57.650867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:06.276636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:17.341418+0000","last_change":"2024-03-08T23:51:51.245981+0000","last_active":"2024-03-08T23:52:17.341418+0000","last_peered":"2024-03-08T23:52:17.341418+0000","last_clean":"2024-03-08T23:52:17.341418+0000","last_became_active":"2024-03-08T23:51:51.244681+0000","last_became_peered":"2024-03-08T23:51:51.244681+0000","last_unstale":"2024-03-08T23:52:17.341418+0000","last_undegraded":"2024-03-08T23:52:17.341418+0000","last_fullsized":"2024-03-08T23:52:17.341418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:06:21.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403676+0000","last_change":"2024-03-08T23:51:56.279057+0000","last_active":"2024-03-08T23:51:58.403676+0000","last_peered":"2024-03-08T23:51:58.403676+0000","last_clean":"2024-03-08T23:51:58.403676+0000","last_became_active":"2024-03-08T23:51:56.278634+0000","last_became_peered":"2024-03-08T23:51:56.278634+0000","last_unstale":"2024-03-08T23:51:58.403676+0000","last_undegraded":"2024-03-08T23:51:58.403676+0000","last_fullsized":"2024-03-08T23:51:58.403676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:30.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.240055+0000","last_change":"2024-03-08T23:51:51.240378+0000","last_active":"2024-03-08T23:51:58.240055+0000","last_peered":"2024-03-08T23:51:58.240055+0000","last_clean":"2024-03-08T23:51:58.240055+0000","last_became_active":"2024-03-08T23:51:51.239880+0000","last_became_peered":"2024-03-08T23:51:51.239880+0000","last_unstale":"2024-03-08T23:51:58.240055+0000","last_undegraded":"2024-03-08T23:51:58.240055+0000","last_fullsized":"2024-03-08T23:51:58.240055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:20.804261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651404+0000","last_change":"2024-03-08T23:51:52.240338+0000","last_active":"2024-03-08T23:51:57.651404+0000","last_peered":"2024-03-08T23:51:57.651404+0000","last_clean":"2024-03-08T23:51:57.651404+0000","last_became_active":"2024-03-08T23:51:52.239987+0000","last_became_peered":"2024-03-08T23:51:52.239987+0000","last_unstale":"2024-03-08T23:51:57.651404+0000","last_undegraded":"2024-03-08T23:51:57.651404+0000","last_fullsized":"2024-03-08T23:51:57.651404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:26.590204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123711+0000","last_change":"2024-03-08T23:51:56.281009+0000","last_active":"2024-03-08T23:51:58.123711+0000","last_peered":"2024-03-08T23:51:58.123711+0000","last_clean":"2024-03-08T23:51:58.123711+0000","last_became_active":"2024-03-08T23:51:56.280619+0000","last_became_peered":"2024-03-08T23:51:56.280619+0000","last_unstale":"2024-03-08T23:51:58.123711+0000","last_undegraded":"2024-03-08T23:51:58.123711+0000","last_fullsized":"2024-03-08T23:51:58.123711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:06.920777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264295+0000","last_change":"2024-03-08T23:51:51.241682+0000","last_active":"2024-03-08T23:51:57.264295+0000","last_peered":"2024-03-08T23:51:57.264295+0000","last_clean":"2024-03-08T23:51:57.264295+0000","last_became_active":"2024-03-08T23:51:51.241313+0000","last_became_peered":"2024-03-08T23:51:51.241313+0000","last_unstale":"2024-03-08T23:51:57.264295+0000","last_undegraded":"2024-03-08T23:51:57.264295+0000","last_fullsized":"2024-03-08T23:51:57.264295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:04:13.515856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123737+0000","last_change":"2024-03-08T23:51:52.249378+0000","last_active":"2024-03-08T23:51:58.123737+0000","last_peered":"2024-03-08T23:51:58.123737+0000","last_clean":"2024-03-08T23:51:58.123737+0000","last_became_active":"2024-03-08T23:51:52.249084+0000","last_became_peered":"2024-03-08T23:51:52.249084+0000","last_unstale":"2024-03-08T23:51:58.123737+0000","last_undegraded":"2024-03-08T23:51:58.123737+0000","last_fullsized":"2024-03-08T23:51:58.123737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:10:36.513260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265304+0000","last_change":"2024-03-08T23:51:56.280717+0000","last_active":"2024-03-08T23:51:57.265304+0000","last_peered":"2024-03-08T23:51:57.265304+0000","last_clean":"2024-03-08T23:51:57.265304+0000","last_became_active":"2024-03-08T23:51:56.280329+0000","last_became_peered":"2024-03-08T23:51:56.280329+0000","last_unstale":"2024-03-08T23:51:57.265304+0000","last_undegraded":"2024-03-08T23:51:57.265304+0000","last_fullsized":"2024-03-08T23:51:57.265304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:47:40.829142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.330305+0000","last_change":"2024-03-08T23:51:51.238058+0000","last_active":"2024-03-08T23:52:16.330305+0000","last_peered":"2024-03-08T23:52:16.330305+0000","last_clean":"2024-03-08T23:52:16.330305+0000","last_became_active":"2024-03-08T23:51:51.237727+0000","last_became_peered":"2024-03-08T23:51:51.237727+0000","last_unstale":"2024-03-08T23:52:16.330305+0000","last_undegraded":"2024-03-08T23:52:16.330305+0000","last_fullsized":"2024-03-08T23:52:16.330305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:13:54.080004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130951+0000","last_change":"2024-03-08T23:51:52.249830+0000","last_active":"2024-03-08T23:51:58.130951+0000","last_peered":"2024-03-08T23:51:58.130951+0000","last_clean":"2024-03-08T23:51:58.130951+0000","last_became_active":"2024-03-08T23:51:52.249518+0000","last_became_peered":"2024-03-08T23:51:52.249518+0000","last_unstale":"2024-03-08T23:51:58.130951+0000","last_undegraded":"2024-03-08T23:51:58.130951+0000","last_fullsized":"2024-03-08T23:51:58.130951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:24:33.721423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378369+0000","last_change":"2024-03-08T23:51:56.288595+0000","last_active":"2024-03-08T23:51:57.378369+0000","last_peered":"2024-03-08T23:51:57.378369+0000","last_clean":"2024-03-08T23:51:57.378369+0000","last_became_active":"2024-03-08T23:51:56.288146+0000","last_became_peered":"2024-03-08T23:51:56.288146+0000","last_unstale":"2024-03-08T23:51:57.378369+0000","last_undegraded":"2024-03-08T23:51:57.378369+0000","last_fullsized":"2024-03-08T23:51:57.378369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:34:58.321754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.331159+0000","last_change":"2024-03-08T23:51:51.242065+0000","last_active":"2024-03-08T23:52:16.331159+0000","last_peered":"2024-03-08T23:52:16.331159+0000","last_clean":"2024-03-08T23:52:16.331159+0000","last_became_active":"2024-03-08T23:51:51.241513+0000","last_became_peered":"2024-03-08T23:51:51.241513+0000","last_unstale":"2024-03-08T23:52:16.331159+0000","last_undegraded":"2024-03-08T23:52:16.331159+0000","last_fullsized":"2024-03-08T23:52:16.331159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:08:02.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239458+0000","last_change":"2024-03-08T23:51:52.243074+0000","last_active":"2024-03-08T23:51:58.239458+0000","last_peered":"2024-03-08T23:51:58.239458+0000","last_clean":"2024-03-08T23:51:58.239458+0000","last_became_active":"2024-03-08T23:51:52.242736+0000","last_became_peered":"2024-03-08T23:51:52.242736+0000","last_unstale":"2024-03-08T23:51:58.239458+0000","last_undegraded":"2024-03-08T23:51:58.239458+0000","last_fullsized":"2024-03-08T23:51:58.239458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:57:37.174056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264565+0000","last_change":"2024-03-08T23:51:56.276284+0000","last_active":"2024-03-08T23:51:57.264565+0000","last_peered":"2024-03-08T23:51:57.264565+0000","last_clean":"2024-03-08T23:51:57.264565+0000","last_became_active":"2024-03-08T23:51:56.275800+0000","last_became_peered":"2024-03-08T23:51:56.275800+0000","last_unstale":"2024-03-08T23:51:57.264565+0000","last_undegraded":"2024-03-08T23:51:57.264565+0000","last_fullsized":"2024-03-08T23:51:57.264565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:21:11.221266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:52:16.333264+0000","last_change":"2024-03-08T23:51:51.245548+0000","last_active":"2024-03-08T23:52:16.333264+0000","last_peered":"2024-03-08T23:52:16.333264+0000","last_clean":"2024-03-08T23:52:16.333264+0000","last_became_active":"2024-03-08T23:51:51.245238+0000","last_became_peered":"2024-03-08T23:51:51.245238+0000","last_unstale":"2024-03-08T23:52:16.333264+0000","last_undegraded":"2024-03-08T23:52:16.333264+0000","last_fullsized":"2024-03-08T23:52:16.333264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:21:19.557196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403466+0000","last_change":"2024-03-08T23:51:52.244422+0000","last_active":"2024-03-08T23:51:58.403466+0000","last_peered":"2024-03-08T23:51:58.403466+0000","last_clean":"2024-03-08T23:51:58.403466+0000","last_became_active":"2024-03-08T23:51:52.244032+0000","last_became_peered":"2024-03-08T23:51:52.244032+0000","last_unstale":"2024-03-08T23:51:58.403466+0000","last_undegraded":"2024-03-08T23:51:58.403466+0000","last_fullsized":"2024-03-08T23:51:58.403466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:04.064691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378138+0000","last_change":"2024-03-08T23:51:56.288632+0000","last_active":"2024-03-08T23:51:57.378138+0000","last_peered":"2024-03-08T23:51:57.378138+0000","last_clean":"2024-03-08T23:51:57.378138+0000","last_became_active":"2024-03-08T23:51:56.288175+0000","last_became_peered":"2024-03-08T23:51:56.288175+0000","last_unstale":"2024-03-08T23:51:57.378138+0000","last_undegraded":"2024-03-08T23:51:57.378138+0000","last_fullsized":"2024-03-08T23:51:57.378138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:54:51.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265416+0000","last_change":"2024-03-08T23:51:51.247968+0000","last_active":"2024-03-08T23:51:57.265416+0000","last_peered":"2024-03-08T23:51:57.265416+0000","last_clean":"2024-03-08T23:51:57.265416+0000","last_became_active":"2024-03-08T23:51:51.247660+0000","last_became_peered":"2024-03-08T23:51:51.247660+0000","last_unstale":"2024-03-08T23:51:57.265416+0000","last_undegraded":"2024-03-08T23:51:57.265416+0000","last_fullsized":"2024-03-08T23:51:57.265416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:40.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123736+0000","last_change":"2024-03-08T23:51:52.250778+0000","last_active":"2024-03-08T23:51:58.123736+0000","last_peered":"2024-03-08T23:51:58.123736+0000","last_clean":"2024-03-08T23:51:58.123736+0000","last_became_active":"2024-03-08T23:51:52.250475+0000","last_became_peered":"2024-03-08T23:51:52.250475+0000","last_unstale":"2024-03-08T23:51:58.123736+0000","last_undegraded":"2024-03-08T23:51:58.123736+0000","last_fullsized":"2024-03-08T23:51:58.123736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:36:01.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.378204+0000","last_change":"2024-03-08T23:51:56.285631+0000","last_active":"2024-03-08T23:51:57.378204+0000","last_peered":"2024-03-08T23:51:57.378204+0000","last_clean":"2024-03-08T23:51:57.378204+0000","last_became_active":"2024-03-08T23:51:56.285183+0000","last_became_peered":"2024-03-08T23:51:56.285183+0000","last_unstale":"2024-03-08T23:51:57.378204+0000","last_undegraded":"2024-03-08T23:51:57.378204+0000","last_fullsized":"2024-03-08T23:51:57.378204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:15.002086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.239656+0000","last_change":"2024-03-08T23:51:51.246848+0000","last_active":"2024-03-08T23:51:58.239656+0000","last_peered":"2024-03-08T23:51:58.239656+0000","last_clean":"2024-03-08T23:51:58.239656+0000","last_became_active":"2024-03-08T23:51:51.246533+0000","last_became_peered":"2024-03-08T23:51:51.246533+0000","last_unstale":"2024-03-08T23:51:58.239656+0000","last_undegraded":"2024-03-08T23:51:58.239656+0000","last_fullsized":"2024-03-08T23:51:58.239656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:20:16.669820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264182+0000","last_change":"2024-03-08T23:51:52.251575+0000","last_active":"2024-03-08T23:51:57.264182+0000","last_peered":"2024-03-08T23:51:57.264182+0000","last_clean":"2024-03-08T23:51:57.264182+0000","last_became_active":"2024-03-08T23:51:52.251232+0000","last_became_peered":"2024-03-08T23:51:52.251232+0000","last_unstale":"2024-03-08T23:51:57.264182+0000","last_undegraded":"2024-03-08T23:51:57.264182+0000","last_fullsized":"2024-03-08T23:51:57.264182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:33.573407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403778+0000","last_change":"2024-03-08T23:51:56.282841+0000","last_active":"2024-03-08T23:51:58.403778+0000","last_peered":"2024-03-08T23:51:58.403778+0000","last_clean":"2024-03-08T23:51:58.403778+0000","last_became_active":"2024-03-08T23:51:56.282366+0000","last_became_peered":"2024-03-08T23:51:56.282366+0000","last_unstale":"2024-03-08T23:51:58.403778+0000","last_undegraded":"2024-03-08T23:51:58.403778+0000","last_fullsized":"2024-03-08T23:51:58.403778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:23:53.164544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265884+0000","last_change":"2024-03-08T23:51:51.242081+0000","last_active":"2024-03-08T23:51:57.265884+0000","last_peered":"2024-03-08T23:51:57.265884+0000","last_clean":"2024-03-08T23:51:57.265884+0000","last_became_active":"2024-03-08T23:51:51.241636+0000","last_became_peered":"2024-03-08T23:51:51.241636+0000","last_unstale":"2024-03-08T23:51:57.265884+0000","last_undegraded":"2024-03-08T23:51:57.265884+0000","last_fullsized":"2024-03-08T23:51:57.265884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:51:57.497723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264712+0000","last_change":"2024-03-08T23:51:52.252796+0000","last_active":"2024-03-08T23:51:57.264712+0000","last_peered":"2024-03-08T23:51:57.264712+0000","last_clean":"2024-03-08T23:51:57.264712+0000","last_became_active":"2024-03-08T23:51:52.252361+0000","last_became_peered":"2024-03-08T23:51:52.252361+0000","last_unstale":"2024-03-08T23:51:57.264712+0000","last_undegraded":"2024-03-08T23:51:57.264712+0000","last_fullsized":"2024-03-08T23:51:57.264712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:07.968262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130671+0000","last_change":"2024-03-08T23:51:56.284396+0000","last_active":"2024-03-08T23:51:58.130671+0000","last_peered":"2024-03-08T23:51:58.130671+0000","last_clean":"2024-03-08T23:51:58.130671+0000","last_became_active":"2024-03-08T23:51:56.283951+0000","last_became_peered":"2024-03-08T23:51:56.283951+0000","last_unstale":"2024-03-08T23:51:58.130671+0000","last_undegraded":"2024-03-08T23:51:58.130671+0000","last_fullsized":"2024-03-08T23:51:58.130671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:09:40.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278456+0000","last_change":"2024-03-08T23:51:51.248838+0000","last_active":"2024-03-08T23:51:57.278456+0000","last_peered":"2024-03-08T23:51:57.278456+0000","last_clean":"2024-03-08T23:51:57.278456+0000","last_became_active":"2024-03-08T23:51:51.248534+0000","last_became_peered":"2024-03-08T23:51:51.248534+0000","last_unstale":"2024-03-08T23:51:57.278456+0000","last_undegraded":"2024-03-08T23:51:57.278456+0000","last_fullsized":"2024-03-08T23:51:57.278456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:37:25.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264281+0000","last_change":"2024-03-08T23:51:52.264170+0000","last_active":"2024-03-08T23:51:57.264281+0000","last_peered":"2024-03-08T23:51:57.264281+0000","last_clean":"2024-03-08T23:51:57.264281+0000","last_became_active":"2024-03-08T23:51:52.263744+0000","last_became_peered":"2024-03-08T23:51:52.263744+0000","last_unstale":"2024-03-08T23:51:57.264281+0000","last_undegraded":"2024-03-08T23:51:57.264281+0000","last_fullsized":"2024-03-08T23:51:57.264281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:42:18.287699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278007+0000","last_change":"2024-03-08T23:51:56.283316+0000","last_active":"2024-03-08T23:51:57.278007+0000","last_peered":"2024-03-08T23:51:57.278007+0000","last_clean":"2024-03-08T23:51:57.278007+0000","last_became_active":"2024-03-08T23:51:56.282515+0000","last_became_peered":"2024-03-08T23:51:56.282515+0000","last_unstale":"2024-03-08T23:51:57.278007+0000","last_undegraded":"2024-03-08T23:51:57.278007+0000","last_fullsized":"2024-03-08T23:51:57.278007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:40:15.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651353+0000","last_change":"2024-03-08T23:51:51.240970+0000","last_active":"2024-03-08T23:51:57.651353+0000","last_peered":"2024-03-08T23:51:57.651353+0000","last_clean":"2024-03-08T23:51:57.651353+0000","last_became_active":"2024-03-08T23:51:51.239982+0000","last_became_peered":"2024-03-08T23:51:51.239982+0000","last_unstale":"2024-03-08T23:51:57.651353+0000","last_undegraded":"2024-03-08T23:51:57.651353+0000","last_fullsized":"2024-03-08T23:51:57.651353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:47:40.313753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.264709+0000","last_change":"2024-03-08T23:51:52.261673+0000","last_active":"2024-03-08T23:51:57.264709+0000","last_peered":"2024-03-08T23:51:57.264709+0000","last_clean":"2024-03-08T23:51:57.264709+0000","last_became_active":"2024-03-08T23:51:52.261333+0000","last_became_peered":"2024-03-08T23:51:52.261333+0000","last_unstale":"2024-03-08T23:51:57.264709+0000","last_undegraded":"2024-03-08T23:51:57.264709+0000","last_fullsized":"2024-03-08T23:51:57.264709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:08:26.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403537+0000","last_change":"2024-03-08T23:51:52.243189+0000","last_active":"2024-03-08T23:51:58.403537+0000","last_peered":"2024-03-08T23:51:58.403537+0000","last_clean":"2024-03-08T23:51:58.403537+0000","last_became_active":"2024-03-08T23:51:52.242870+0000","last_became_peered":"2024-03-08T23:51:52.242870+0000","last_unstale":"2024-03-08T23:51:58.403537+0000","last_undegraded":"2024-03-08T23:51:58.403537+0000","last_fullsized":"2024-03-08T23:51:58.403537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.438045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265347+0000","last_change":"2024-03-08T23:51:52.252854+0000","last_active":"2024-03-08T23:51:57.265347+0000","last_peered":"2024-03-08T23:51:57.265347+0000","last_clean":"2024-03-08T23:51:57.265347+0000","last_became_active":"2024-03-08T23:51:52.252534+0000","last_became_peered":"2024-03-08T23:51:52.252534+0000","last_unstale":"2024-03-08T23:51:57.265347+0000","last_undegraded":"2024-03-08T23:51:57.265347+0000","last_fullsized":"2024-03-08T23:51:57.265347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:21:13.049717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265266+0000","last_change":"2024-03-08T23:51:52.251397+0000","last_active":"2024-03-08T23:51:57.265266+0000","last_peered":"2024-03-08T23:51:57.265266+0000","last_clean":"2024-03-08T23:51:57.265266+0000","last_became_active":"2024-03-08T23:51:52.251059+0000","last_became_peered":"2024-03-08T23:51:52.251059+0000","last_unstale":"2024-03-08T23:51:57.265266+0000","last_undegraded":"2024-03-08T23:51:57.265266+0000","last_fullsized":"2024-03-08T23:51:57.265266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:19:30.363457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.130727+0000","last_change":"2024-03-08T23:51:52.249039+0000","last_active":"2024-03-08T23:51:58.130727+0000","last_peered":"2024-03-08T23:51:58.130727+0000","last_clean":"2024-03-08T23:51:58.130727+0000","last_became_active":"2024-03-08T23:51:52.248560+0000","last_became_peered":"2024-03-08T23:51:52.248560+0000","last_unstale":"2024-03-08T23:51:58.130727+0000","last_undegraded":"2024-03-08T23:51:58.130727+0000","last_fullsized":"2024-03-08T23:51:58.130727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651307+0000","last_change":"2024-03-08T23:51:52.247269+0000","last_active":"2024-03-08T23:51:57.651307+0000","last_peered":"2024-03-08T23:51:57.651307+0000","last_clean":"2024-03-08T23:51:57.651307+0000","last_became_active":"2024-03-08T23:51:52.246921+0000","last_became_peered":"2024-03-08T23:51:52.246921+0000","last_unstale":"2024-03-08T23:51:57.651307+0000","last_undegraded":"2024-03-08T23:51:57.651307+0000","last_fullsized":"2024-03-08T23:51:57.651307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:00:21.828530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403856+0000","last_change":"2024-03-08T23:51:52.238999+0000","last_active":"2024-03-08T23:51:58.403856+0000","last_peered":"2024-03-08T23:51:58.403856+0000","last_clean":"2024-03-08T23:51:58.403856+0000","last_became_active":"2024-03-08T23:51:52.238662+0000","last_became_peered":"2024-03-08T23:51:52.238662+0000","last_unstale":"2024-03-08T23:51:58.403856+0000","last_undegraded":"2024-03-08T23:51:58.403856+0000","last_fullsized":"2024-03-08T23:51:58.403856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:22:08.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.265818+0000","last_change":"2024-03-08T23:51:52.238793+0000","last_active":"2024-03-08T23:51:57.265818+0000","last_peered":"2024-03-08T23:51:57.265818+0000","last_clean":"2024-03-08T23:51:57.265818+0000","last_became_active":"2024-03-08T23:51:52.238464+0000","last_became_peered":"2024-03-08T23:51:52.238464+0000","last_unstale":"2024-03-08T23:51:57.265818+0000","last_undegraded":"2024-03-08T23:51:57.265818+0000","last_fullsized":"2024-03-08T23:51:57.265818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:25.597428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123869+0000","last_change":"2024-03-08T23:51:52.253778+0000","last_active":"2024-03-08T23:51:58.123869+0000","last_peered":"2024-03-08T23:51:58.123869+0000","last_clean":"2024-03-08T23:51:58.123869+0000","last_became_active":"2024-03-08T23:51:52.253387+0000","last_became_peered":"2024-03-08T23:51:52.253387+0000","last_unstale":"2024-03-08T23:51:58.123869+0000","last_undegraded":"2024-03-08T23:51:58.123869+0000","last_fullsized":"2024-03-08T23:51:58.123869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:50:03.142133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.278481+0000","last_change":"2024-03-08T23:51:52.245288+0000","last_active":"2024-03-08T23:51:57.278481+0000","last_peered":"2024-03-08T23:51:57.278481+0000","last_clean":"2024-03-08T23:51:57.278481+0000","last_became_active":"2024-03-08T23:51:52.244960+0000","last_became_peered":"2024-03-08T23:51:52.244960+0000","last_unstale":"2024-03-08T23:51:57.278481+0000","last_undegraded":"2024-03-08T23:51:57.278481+0000","last_fullsized":"2024-03-08T23:51:57.278481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:20:39.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.123856+0000","last_change":"2024-03-08T23:51:52.252291+0000","last_active":"2024-03-08T23:51:58.123856+0000","last_peered":"2024-03-08T23:51:58.123856+0000","last_clean":"2024-03-08T23:51:58.123856+0000","last_became_active":"2024-03-08T23:51:52.251979+0000","last_became_peered":"2024-03-08T23:51:52.251979+0000","last_unstale":"2024-03-08T23:51:58.123856+0000","last_undegraded":"2024-03-08T23:51:58.123856+0000","last_fullsized":"2024-03-08T23:51:58.123856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:19:34.314784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403696+0000","last_change":"2024-03-08T23:51:52.251048+0000","last_active":"2024-03-08T23:51:58.403696+0000","last_peered":"2024-03-08T23:51:58.403696+0000","last_clean":"2024-03-08T23:51:58.403696+0000","last_became_active":"2024-03-08T23:51:52.250508+0000","last_became_peered":"2024-03-08T23:51:52.250508+0000","last_unstale":"2024-03-08T23:51:58.403696+0000","last_undegraded":"2024-03-08T23:51:58.403696+0000","last_fullsized":"2024-03-08T23:51:58.403696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:57:37.519307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:57.651127+0000","last_change":"2024-03-08T23:51:52.244203+0000","last_active":"2024-03-08T23:51:57.651127+0000","last_peered":"2024-03-08T23:51:57.651127+0000","last_clean":"2024-03-08T23:51:57.651127+0000","last_became_active":"2024-03-08T23:51:52.243878+0000","last_became_peered":"2024-03-08T23:51:52.243878+0000","last_unstale":"2024-03-08T23:51:57.651127+0000","last_undegraded":"2024-03-08T23:51:57.651127+0000","last_fullsized":"2024-03-08T23:51:57.651127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:53:40.938657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-08T23:51:58.403921+0000","last_change":"2024-03-08T23:51:52.248334+0000","last_active":"2024-03-08T23:51:58.403921+0000","last_peered":"2024-03-08T23:51:58.403921+0000","last_clean":"2024-03-08T23:51:58.403921+0000","last_became_active":"2024-03-08T23:51:52.248025+0000","last_became_peered":"2024-03-08T23:51:52.248025+0000","last_unstale":"2024-03-08T23:51:58.403921+0000","last_undegraded":"2024-03-08T23:51:58.403921+0000","last_fullsized":"2024-03-08T23:51:58.403921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:02:06.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":26035,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":18,"num_read_kb":36,"num_write":104,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":39906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1073003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":987136,"data_stored":477797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1070157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":481057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":477409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":477859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1069059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T23:52:21.481 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T23:52:21.481 INFO:tasks.cephadm:Setting up client nodes... 2024-03-08T23:52:21.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-08T23:52:21.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:21 smithi062 ceph-mon[36129]: pgmap v307: 129 pgs: 129 active+clean; 602 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-08T23:52:21.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:21 smithi062 ceph-mon[36129]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:21.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:21 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/889740035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:52:21.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:21 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2240445123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:21.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:21 smithi192 ceph-mon[35995]: pgmap v307: 129 pgs: 129 active+clean; 602 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-08T23:52:21.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:21 smithi192 ceph-mon[35995]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:52:21.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:21 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/889740035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:52:21.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:21 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2240445123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:21.761 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:22.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:22 smithi022 ceph-mon[31894]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:52:22.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:22 smithi022 ceph-mon[31894]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-08T23:52:22.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:22 smithi022 ceph-mon[31894]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:52:22.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:22 smithi062 ceph-mon[36129]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:52:22.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:22 smithi062 ceph-mon[36129]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-08T23:52:22.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:22 smithi062 ceph-mon[36129]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:52:22.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:22 smithi192 ceph-mon[35995]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:52:22.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:22 smithi192 ceph-mon[35995]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-08T23:52:22.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:22 smithi192 ceph-mon[35995]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:52:23.477 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:23 smithi022 ceph-mon[31894]: pgmap v308: 129 pgs: 129 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-03-08T23:52:23.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:23 smithi062 ceph-mon[36129]: pgmap v308: 129 pgs: 129 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-03-08T23:52:23.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:23 smithi192 ceph-mon[35995]: pgmap v308: 129 pgs: 129 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 -- 172.21.15.22:0/3984815064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a401028e0 msgr2=0x7f1a40109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 --2- 172.21.15.22:0/3984815064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a401028e0 0x7f1a40109190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800b0d0 tx=0x7f1a3800b3e0 comp rx=0 tx=0).stop 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 -- 172.21.15.22:0/3984815064 shutdown_connections 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 --2- 172.21.15.22:0/3984815064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a401028e0 0x7f1a40109190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 --2- 172.21.15.22:0/3984815064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a40100d40 0x7f1a40101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 --2- 172.21.15.22:0/3984815064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a40101f40 0x7f1a401023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 -- 172.21.15.22:0/3984815064 >> 172.21.15.22:0/3984815064 conn(0x7f1a400fc2e0 msgr2=0x7f1a400fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 -- 172.21.15.22:0/3984815064 shutdown_connections 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.749+0000 7f1a4639f700 1 -- 172.21.15.22:0/3984815064 wait complete. 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 Processor -- start 2024-03-08T23:52:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 -- start start 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a40100d40 0x7f1a4006b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a40101f40 0x7f1a4019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401028e0 0x7f1a4019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4010c930 con 0x7f1a40101f40 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4010c7b0 con 0x7f1a401028e0 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a4639f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4010cab0 con 0x7f1a40100d40 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a3f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a40101f40 0x7f1a4019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a3f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a40101f40 0x7f1a4019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45118/0 (socket says 172.21.15.22:45118) 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a3f7fe700 1 -- 172.21.15.22:0/1390226316 learned_addr learned my addr 172.21.15.22:0/1390226316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.750+0000 7f1a3ffff700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a40100d40 0x7f1a4006b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a4493c700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401028e0 0x7f1a4019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3f7fe700 1 -- 172.21.15.22:0/1390226316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a40100d40 msgr2=0x7f1a4006b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:23.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3f7fe700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a40100d40 0x7f1a4006b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3f7fe700 1 -- 172.21.15.22:0/1390226316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401028e0 msgr2=0x7f1a4019f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3f7fe700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401028e0 0x7f1a4019f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3f7fe700 1 -- 172.21.15.22:0/1390226316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4019fba0 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a4493c700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401028e0 0x7f1a4019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3f7fe700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a40101f40 0x7f1a4019cc90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d950 tx=0x7f1a3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3d7fa700 1 -- 172.21.15.22:0/1390226316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34016070 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3d7fa700 1 -- 172.21.15.22:0/1390226316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a34009e20 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a3d7fa700 1 -- 172.21.15.22:0/1390226316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400f620 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4019fe90 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.751+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a401a0390 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.752+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a400502f0 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.753+0000 7f1a3d7fa700 1 -- 172.21.15.22:0/1390226316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1a34019070 con 0x7f1a40101f40 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.754+0000 7f1a3d7fa700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1a280a2a50 0x7f1a280a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:23.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.754+0000 7f1a3ffff700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1a280a2a50 0x7f1a280a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:23.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.754+0000 7f1a3d7fa700 1 -- 172.21.15.22:0/1390226316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1a340ce990 con 0x7f1a40101f40 2024-03-08T23:52:23.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.754+0000 7f1a3ffff700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1a280a2a50 0x7f1a280a4f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a40101da0 tx=0x7f1a30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:23.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:23.758+0000 7f1a3d7fa700 1 -- 172.21.15.22:0/1390226316 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1a3409b630 con 0x7f1a40101f40 2024-03-08T23:52:24.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.047+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1a4002d9e0 con 0x7f1a40101f40 2024-03-08T23:52:24.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.049+0000 7f1a3d7fa700 1 -- 172.21.15.22:0/1390226316 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f1a4002d9e0 con 0x7f1a40101f40 2024-03-08T23:52:24.045 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-03-08T23:52:24.046 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQC4pOtlvuLlAhAA8B3JfO/GZN/Lqay4swlHzA== 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1a280a2a50 msgr2=0x7f1a280a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1a280a2a50 0x7f1a280a4f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a40101da0 tx=0x7f1a30009260 comp rx=0 tx=0).stop 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a40101f40 msgr2=0x7f1a4019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a40101f40 0x7f1a4019cc90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d950 tx=0x7f1a3400dc60 comp rx=0 tx=0).stop 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 shutdown_connections 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401028e0 0x7f1a4019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a40100d40 0x7f1a4006b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1a280a2a50 0x7f1a280a4f10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 --2- 172.21.15.22:0/1390226316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a40101f40 0x7f1a4019cc90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:24.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 >> 172.21.15.22:0/1390226316 conn(0x7f1a400fc2e0 msgr2=0x7f1a40105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 shutdown_connections 2024-03-08T23:52:24.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:24.051+0000 7f1a4639f700 1 -- 172.21.15.22:0/1390226316 wait complete. 2024-03-08T23:52:24.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:24 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1390226316' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-08T23:52:24.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:24 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1390226316' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-08T23:52:24.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:24 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1390226316' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-08T23:52:24.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:24 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1390226316' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-08T23:52:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:24 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1390226316' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-08T23:52:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:24 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1390226316' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-08T23:52:25.019 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:52:25.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-08T23:52:25.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-08T23:52:25.061 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-08T23:52:25.062 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-08T23:52:25.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph mgr dump --format=json 2024-03-08T23:52:25.347 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:25.608 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:25 smithi022 ceph-mon[31894]: pgmap v309: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-08T23:52:25.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:25 smithi062 ceph-mon[36129]: pgmap v309: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-08T23:52:25.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:25 smithi192 ceph-mon[35995]: pgmap v309: 129 pgs: 129 active+clean; 604 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-03-08T23:52:27.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- 172.21.15.22:0/1237008060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 msgr2=0x7ff530101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 --2- 172.21.15.22:0/1237008060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530101380 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff5240099c0 tx=0x7ff524009cd0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- 172.21.15.22:0/1237008060 shutdown_connections 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 --2- 172.21.15.22:0/1237008060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff530105ff0 0x7ff53010dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 --2- 172.21.15.22:0/1237008060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530101380 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 --2- 172.21.15.22:0/1237008060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff530101950 0x7ff530105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- 172.21.15.22:0/1237008060 >> 172.21.15.22:0/1237008060 conn(0x7ff530076d30 msgr2=0x7ff530077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- 172.21.15.22:0/1237008060 shutdown_connections 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- 172.21.15.22:0/1237008060 wait complete. 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 Processor -- start 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- start start 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:27.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff530101950 0x7ff5301998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff530105ff0 0x7ff5301a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff530110fd0 con 0x7ff530100fa0 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff530110e50 con 0x7ff530105ff0 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.250+0000 7ff536fad700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff530111150 con 0x7ff530101950 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45134/0 (socket says 172.21.15.22:45134) 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 -- 172.21.15.22:0/430615709 learned_addr learned my addr 172.21.15.22:0/430615709 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff52ffff700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff530101950 0x7ff5301998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 -- 172.21.15.22:0/430615709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff530101950 msgr2=0x7ff5301998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff530101950 0x7ff5301998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff53554a700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff530105ff0 0x7ff5301a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:27.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 -- 172.21.15.22:0/430615709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff530105ff0 msgr2=0x7ff5301a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff530105ff0 0x7ff5301a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 -- 172.21.15.22:0/430615709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5301a4950 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff52ffff700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff530101950 0x7ff5301998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff534d49700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530199390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5240059d0 tx=0x7ff52400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff52dffb700 1 -- 172.21.15.22:0/430615709 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524027070 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5301a4b20 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff52dffb700 1 -- 172.21.15.22:0/430615709 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff524003680 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff52dffb700 1 -- 172.21.15.22:0/430615709 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524017650 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.251+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5301a4f70 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.252+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff530068aa0 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.253+0000 7ff52dffb700 1 -- 172.21.15.22:0/430615709 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff52402a070 con 0x7ff530100fa0 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.254+0000 7ff52dffb700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5200a2920 0x7ff5200a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.254+0000 7ff52ffff700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5200a2920 0x7ff5200a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:27.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.254+0000 7ff52dffb700 1 -- 172.21.15.22:0/430615709 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff5240cfae0 con 0x7ff530100fa0 2024-03-08T23:52:27.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.254+0000 7ff52ffff700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5200a2920 0x7ff5200a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005950 tx=0x7ff51c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:27.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.257+0000 7ff52dffb700 1 -- 172.21.15.22:0/430615709 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff524013070 con 0x7ff530100fa0 2024-03-08T23:52:27.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:27 smithi022 ceph-mon[31894]: pgmap v310: 129 pgs: 129 active+clean; 621 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-03-08T23:52:27.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.523+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff53002d9f0 con 0x7ff530100fa0 2024-03-08T23:52:27.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.527+0000 7ff52dffb700 1 -- 172.21.15.22:0/430615709 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262660 (secure 0 0 0) 0x7ff52409c810 con 0x7ff530100fa0 2024-03-08T23:52:27.528 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5200a2920 msgr2=0x7ff5200a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5200a2920 0x7ff5200a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005950 tx=0x7ff51c0058e0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 msgr2=0x7ff530199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530199390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5240059d0 tx=0x7ff52400fab0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 shutdown_connections 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff530105ff0 0x7ff5301a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5200a2920 0x7ff5200a4de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff530100fa0 0x7ff530199390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 --2- 172.21.15.22:0/430615709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff530101950 0x7ff5301998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 >> 172.21.15.22:0/430615709 conn(0x7ff530076d30 msgr2=0x7ff5300fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 shutdown_connections 2024-03-08T23:52:27.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:27.530+0000 7ff536fad700 1 -- 172.21.15.22:0/430615709 wait complete. 2024-03-08T23:52:27.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:27 smithi062 ceph-mon[36129]: pgmap v310: 129 pgs: 129 active+clean; 621 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-03-08T23:52:27.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:27 smithi192 ceph-mon[35995]: pgmap v310: 129 pgs: 129 active+clean; 621 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-03-08T23:52:28.351 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2813827512},{"type":"v1","addr":"172.21.15.22:6801","nonce":2813827512}]},"active_addr":"172.21.15.22:6801/2813827512","active_change":"2024-03-08T23:44:43.706126+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1865576829}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":3477373324}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":3461287105}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1683058859}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":548698673}]}]} 2024-03-08T23:52:28.356 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-08T23:52:28.356 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-08T23:52:28.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-08T23:52:28.606 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:28.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:28 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/430615709' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-08T23:52:28.727 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:28 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/430615709' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-08T23:52:28.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:28 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/430615709' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-08T23:52:29.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:29 smithi192 ceph-mon[35995]: pgmap v311: 129 pgs: 129 active+clean; 621 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-03-08T23:52:29.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:29 smithi022 ceph-mon[31894]: pgmap v311: 129 pgs: 129 active+clean; 621 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-03-08T23:52:29.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:29 smithi062 ceph-mon[36129]: pgmap v311: 129 pgs: 129 active+clean; 621 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-03-08T23:52:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 -- 172.21.15.22:0/1008934504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0101680 msgr2=0x7fe9e010db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/1008934504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0101680 0x7fe9e010db20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009990 tx=0x7fe9d0009ca0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 -- 172.21.15.22:0/1008934504 shutdown_connections 2024-03-08T23:52:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/1008934504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0101680 0x7fe9e010db20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/1008934504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e0106d70 0x7fe9e0107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/1008934504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e0100cc0 0x7fe9e0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 -- 172.21.15.22:0/1008934504 >> 172.21.15.22:0/1008934504 conn(0x7fe9e0077cf0 msgr2=0x7fe9e0078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 -- 172.21.15.22:0/1008934504 shutdown_connections 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 -- 172.21.15.22:0/1008934504 wait complete. 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 Processor -- start 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.434+0000 7fe9e6d40700 1 -- start start 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e6d40700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e0100cc0 0x7fe9e01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e6d40700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e0101680 0x7fe9e0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e6d40700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0106d70 0x7fe9e019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e6d40700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0111020 con 0x7fe9e0106d70 2024-03-08T23:52:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e6d40700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0110ea0 con 0x7fe9e0100cc0 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e6d40700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01111a0 con 0x7fe9e0101680 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0106d70 0x7fe9e019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0106d70 0x7fe9e019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45154/0 (socket says 172.21.15.22:45154) 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 -- 172.21.15.22:0/3081583437 learned_addr learned my addr 172.21.15.22:0/3081583437 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9dffff700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e0101680 0x7fe9e0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e4adc700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e0100cc0 0x7fe9e01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 -- 172.21.15.22:0/3081583437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e0101680 msgr2=0x7fe9e0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e0101680 0x7fe9e0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 -- 172.21.15.22:0/3081583437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e0100cc0 msgr2=0x7fe9e01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e0100cc0 0x7fe9e01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 -- 172.21.15.22:0/3081583437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e019d4d0 con 0x7fe9e0106d70 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9e52dd700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0106d70 0x7fe9e019cd90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00047b0 tx=0x7fe9d0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9ddffb700 1 -- 172.21.15.22:0/3081583437 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0026070 con 0x7fe9e0106d70 2024-03-08T23:52:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9ddffb700 1 -- 172.21.15.22:0/3081583437 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9d0004cc0 con 0x7fe9e0106d70 2024-03-08T23:52:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.435+0000 7fe9ddffb700 1 -- 172.21.15.22:0/3081583437 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0003ba0 con 0x7fe9e0106d70 2024-03-08T23:52:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.436+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e019d760 con 0x7fe9e0106d70 2024-03-08T23:52:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.436+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01a5040 con 0x7fe9e0106d70 2024-03-08T23:52:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.437+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e0068aa0 con 0x7fe9e0106d70 2024-03-08T23:52:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.438+0000 7fe9ddffb700 1 -- 172.21.15.22:0/3081583437 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe9d0029070 con 0x7fe9e0106d70 2024-03-08T23:52:30.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.439+0000 7fe9ddffb700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9c80a29a0 0x7fe9c80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:30.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.439+0000 7fe9e4adc700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9c80a29a0 0x7fe9c80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:30.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.439+0000 7fe9ddffb700 1 -- 172.21.15.22:0/3081583437 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe9d00cf640 con 0x7fe9e0106d70 2024-03-08T23:52:30.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.439+0000 7fe9e4adc700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9c80a29a0 0x7fe9c80a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8003f90 tx=0x7fe9d8009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:30.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.443+0000 7fe9ddffb700 1 -- 172.21.15.22:0/3081583437 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe9d005d260 con 0x7fe9e0106d70 2024-03-08T23:52:30.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.646+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9e01a5210 con 0x7fe9e0106d70 2024-03-08T23:52:30.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.648+0000 7fe9ddffb700 1 -- 172.21.15.22:0/3081583437 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7fe9d0013070 con 0x7fe9e0106d70 2024-03-08T23:52:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:30.645 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:52:30.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9c80a29a0 msgr2=0x7fe9c80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9c80a29a0 0x7fe9c80a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8003f90 tx=0x7fe9d8009180 comp rx=0 tx=0).stop 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0106d70 msgr2=0x7fe9e019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0106d70 0x7fe9e019cd90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00047b0 tx=0x7fe9d0005e80 comp rx=0 tx=0).stop 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 shutdown_connections 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9c80a29a0 0x7fe9c80a4e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e0106d70 0x7fe9e019cd90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e0100cc0 0x7fe9e01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 --2- 172.21.15.22:0/3081583437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e0101680 0x7fe9e0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 >> 172.21.15.22:0/3081583437 conn(0x7fe9e0077cf0 msgr2=0x7fe9e0100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 shutdown_connections 2024-03-08T23:52:30.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:30.650+0000 7fe9e6d40700 1 -- 172.21.15.22:0/3081583437 wait complete. 2024-03-08T23:52:31.402 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-08T23:52:31.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-08T23:52:31.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:31 smithi192 ceph-mon[35995]: pgmap v312: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-08T23:52:31.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:31 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3081583437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:31.648 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:31.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:31 smithi022 ceph-mon[31894]: pgmap v312: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-08T23:52:31.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:31 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3081583437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:31.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:31 smithi062 ceph-mon[36129]: pgmap v312: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-08T23:52:31.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:31 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3081583437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 -- 172.21.15.22:0/644203461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c100ce0 msgr2=0x7f306c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 --2- 172.21.15.22:0/644203461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c100ce0 0x7f306c101140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3060009990 tx=0x7f3060009ca0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 -- 172.21.15.22:0/644203461 shutdown_connections 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 --2- 172.21.15.22:0/644203461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f306c101680 0x7f306c10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 --2- 172.21.15.22:0/644203461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306c106d20 0x7f306c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 --2- 172.21.15.22:0/644203461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c100ce0 0x7f306c101140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 -- 172.21.15.22:0/644203461 >> 172.21.15.22:0/644203461 conn(0x7f306c077c10 msgr2=0x7f306c078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 -- 172.21.15.22:0/644203461 shutdown_connections 2024-03-08T23:52:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.477+0000 7f3073496700 1 -- 172.21.15.22:0/644203461 wait complete. 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 Processor -- start 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 -- start start 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f306c100ce0 0x7f306c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306c101680 0x7f306c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c106d20 0x7f306c1a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c111020 con 0x7f306c106d20 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c110ea0 con 0x7f306c101680 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3073496700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1111a0 con 0x7f306c100ce0 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c106d20 0x7f306c1a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c106d20 0x7f306c1a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39422/0 (socket says 172.21.15.22:39422) 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 -- 172.21.15.22:0/422731894 learned_addr learned my addr 172.21.15.22:0/422731894 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071232700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f306c100ce0 0x7f306c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3070a31700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306c101680 0x7f306c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 -- 172.21.15.22:0/422731894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f306c100ce0 msgr2=0x7f306c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f306c100ce0 0x7f306c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 -- 172.21.15.22:0/422731894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306c101680 msgr2=0x7f306c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306c101680 0x7f306c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.478+0000 7f3071a33700 1 -- 172.21.15.22:0/422731894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c1a4a00 con 0x7f306c106d20 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.479+0000 7f3071a33700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c106d20 0x7f306c1a44c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f306400b550 tx=0x7f306400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.479+0000 7f306a7fc700 1 -- 172.21.15.22:0/422731894 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064015020 con 0x7f306c106d20 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.479+0000 7f306a7fc700 1 -- 172.21.15.22:0/422731894 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3064011d10 con 0x7f306c106d20 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.479+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c1a4c90 con 0x7f306c106d20 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.479+0000 7f306a7fc700 1 -- 172.21.15.22:0/422731894 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306400f2e0 con 0x7f306c106d20 2024-03-08T23:52:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.479+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c1a51a0 con 0x7f306c106d20 2024-03-08T23:52:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.480+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c068aa0 con 0x7f306c106d20 2024-03-08T23:52:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.481+0000 7f306a7fc700 1 -- 172.21.15.22:0/422731894 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f3064018020 con 0x7f306c106d20 2024-03-08T23:52:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.481+0000 7f306a7fc700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f30540a29a0 0x7f30540a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.482+0000 7f306a7fc700 1 -- 172.21.15.22:0/422731894 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f30640ce4c0 con 0x7f306c106d20 2024-03-08T23:52:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.482+0000 7f3071232700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f30540a29a0 0x7f30540a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.482+0000 7f3071232700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f30540a29a0 0x7f30540a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f305c0060b0 tx=0x7f305c007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.486+0000 7f306a7fc700 1 -- 172.21.15.22:0/422731894 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f306401f020 con 0x7f306c106d20 2024-03-08T23:52:33.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:33 smithi192 ceph-mon[35995]: pgmap v313: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-03-08T23:52:33.933 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:33 smithi062 ceph-mon[36129]: pgmap v313: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-03-08T23:52:33.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.694+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f306c02d830 con 0x7f306c106d20 2024-03-08T23:52:33.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:33 smithi022 ceph-mon[31894]: pgmap v313: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-03-08T23:52:33.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.696+0000 7f306a7fc700 1 -- 172.21.15.22:0/422731894 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24032 (secure 0 0 0) 0x7f306409b2d0 con 0x7f306c106d20 2024-03-08T23:52:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:52:33.936 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:51:56.628710+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f30540a29a0 msgr2=0x7f30540a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f30540a29a0 0x7f30540a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f305c0060b0 tx=0x7f305c007b80 comp rx=0 tx=0).stop 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c106d20 msgr2=0x7f306c1a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c106d20 0x7f306c1a44c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f306400b550 tx=0x7f306400b910 comp rx=0 tx=0).stop 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 shutdown_connections 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f30540a29a0 0x7f30540a4e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f306c106d20 0x7f306c1a44c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f306c100ce0 0x7f306c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 --2- 172.21.15.22:0/422731894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306c101680 0x7f306c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 >> 172.21.15.22:0/422731894 conn(0x7f306c077c10 msgr2=0x7f306c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 shutdown_connections 2024-03-08T23:52:33.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:33.698+0000 7f3073496700 1 -- 172.21.15.22:0/422731894 wait complete. 2024-03-08T23:52:34.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-08T23:52:34.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-08T23:52:34.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-08T23:52:34.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-08T23:52:34.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-08T23:52:34.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-08T23:52:34.518 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-08T23:52:34.518 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-08T23:52:34.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-08T23:52:34.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-08T23:52:34.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-08T23:52:34.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-08T23:52:34.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:34 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/422731894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:34 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/422731894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:34.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:34 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/422731894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:52:35.197 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:35.278 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:35.279 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:35.279 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:35.281 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:35.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:35 smithi192 ceph-mon[35995]: pgmap v314: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-03-08T23:52:35.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:35 smithi022 ceph-mon[31894]: pgmap v314: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-03-08T23:52:35.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:35 smithi062 ceph-mon[36129]: pgmap v314: 129 pgs: 129 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-03-08T23:52:37.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:37 smithi022 ceph-mon[31894]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-03-08T23:52:37.557 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:37.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:37 smithi062 ceph-mon[36129]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-03-08T23:52:37.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:37 smithi192 ceph-mon[35995]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-03-08T23:52:38.490 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:38.491 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:38.492 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:38.493 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:38.494 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:38.500 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:52:39.713 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:39 smithi022 ceph-mon[31894]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-03-08T23:52:39.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:39 smithi062 ceph-mon[36129]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-03-08T23:52:39.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:39 smithi192 ceph-mon[35995]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-03-08T23:52:41.723 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:41 smithi022 ceph-mon[31894]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-08T23:52:41.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:41 smithi062 ceph-mon[36129]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-08T23:52:41.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:41 smithi192 ceph-mon[35995]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-08T23:52:42.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.431+0000 7f18aec17700 1 -- 172.21.15.22:0/3318073766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 msgr2=0x7f18a810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.431+0000 7f18aec17700 1 --2- 172.21.15.22:0/3318073766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 0x7f18a810dc30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b0d0 tx=0x7f18a000b3e0 comp rx=0 tx=0).stop 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.431+0000 7f18aec17700 1 -- 172.21.15.22:0/3318073766 shutdown_connections 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.431+0000 7f18aec17700 1 --2- 172.21.15.22:0/3318073766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 0x7f18a810dc30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.431+0000 7f18aec17700 1 --2- 172.21.15.22:0/3318073766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a8101740 0x7f18a8101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.431+0000 7f18aec17700 1 --2- 172.21.15.22:0/3318073766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a81020f0 0x7f18a8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.431+0000 7f18aec17700 1 -- 172.21.15.22:0/3318073766 >> 172.21.15.22:0/3318073766 conn(0x7f18a80772e0 msgr2=0x7f18a80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.432+0000 7f18aec17700 1 -- 172.21.15.22:0/3318073766 shutdown_connections 2024-03-08T23:52:42.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.432+0000 7f18aec17700 1 -- 172.21.15.22:0/3318073766 wait complete. 2024-03-08T23:52:42.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 Processor -- start 2024-03-08T23:52:42.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 -- start start 2024-03-08T23:52:42.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a8101740 0x7f18a81974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:42.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a81020f0 0x7f18a81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:42.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 0x7f18a819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8111050 con 0x7f18a8106060 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8110ed0 con 0x7f18a8101740 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18aec17700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a81111d0 con 0x7f18a81020f0 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18ac9b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a8101740 0x7f18a81974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18ac9b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a8101740 0x7f18a81974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34828/0 (socket says 172.21.15.22:34828) 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18ac9b3700 1 -- 172.21.15.22:0/1484421135 learned_addr learned my addr 172.21.15.22:0/1484421135 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.433+0000 7f18a7fff700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a81020f0 0x7f18a81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ad1b4700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 0x7f18a819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ac9b3700 1 -- 172.21.15.22:0/1484421135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a81020f0 msgr2=0x7f18a81979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ac9b3700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a81020f0 0x7f18a81979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ac9b3700 1 -- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 msgr2=0x7f18a819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ac9b3700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 0x7f18a819bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:42.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ac9b3700 1 -- 172.21.15.22:0/1484421135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a819c4b0 con 0x7f18a8101740 2024-03-08T23:52:42.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ad1b4700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18a8106060 0x7f18a819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:42.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18ac9b3700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a8101740 0x7f18a81974a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f189c00d980 tx=0x7f189c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:42.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c016070 con 0x7f18a8101740 2024-03-08T23:52:42.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f189c004d10 con 0x7f18a8101740 2024-03-08T23:52:42.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c0055f0 con 0x7f18a8101740 2024-03-08T23:52:42.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.434+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a819c7a0 con 0x7f18a8101740 2024-03-08T23:52:42.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.435+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a81003d0 con 0x7f18a8101740 2024-03-08T23:52:42.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.436+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18a819cbd0 con 0x7f18a8101740 2024-03-08T23:52:42.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.437+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f189c004750 con 0x7f18a8101740 2024-03-08T23:52:42.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.437+0000 7f18a5ffb700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f18980a4bb0 0x7f18980a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:42.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.438+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f189c0cec80 con 0x7f18a8101740 2024-03-08T23:52:42.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.438+0000 7f18a5ffb700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] conn(0x7f18980af870 0x7f18980b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:42.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.438+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 --> [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f189c00fec0 con 0x7f18980af870 2024-03-08T23:52:42.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.438+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f189c0d5020 con 0x7f18a8101740 2024-03-08T23:52:42.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.438+0000 7f18a7fff700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f18980a4bb0 0x7f18980a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:42.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.438+0000 7f18ad1b4700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] conn(0x7f18980af870 0x7f18980b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:42.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.439+0000 7f18a7fff700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f18980a4bb0 0x7f18980a7070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f18a8103c20 tx=0x7f189400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:42.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.439+0000 7f18ad1b4700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] conn(0x7f18980af870 0x7f18980b1cb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:42.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.440+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== osd.10 v2:172.21.15.62:6824/1674155650 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f189c00fec0 con 0x7f18980af870 2024-03-08T23:52:42.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.457+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 --> [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18a8100bf0 con 0x7f18980af870 2024-03-08T23:52:42.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.457+0000 7f18a5ffb700 1 -- 172.21.15.22:0/1484421135 <== osd.10 v2:172.21.15.62:6824/1674155650 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f18a8100bf0 con 0x7f18980af870 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] conn(0x7f18980af870 msgr2=0x7f18980b1cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] conn(0x7f18980af870 0x7f18980b1cb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f18980a4bb0 msgr2=0x7f18980a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f18980a4bb0 0x7f18980a7070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f18a8103c20 tx=0x7f189400b410 comp rx=0 tx=0).stop 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a8101740 msgr2=0x7f18a81974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 --2- 172.21.15.22:0/1484421135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a8101740 0x7f18a81974a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f189c00d980 tx=0x7f189c00dc90 comp rx=0 tx=0).stop 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18ad1b4700 1 -- 172.21.15.22:0/1484421135 reap_dead start 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 shutdown_connections 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 >> 172.21.15.22:0/1484421135 conn(0x7f18a80772e0 msgr2=0x7f18a8104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 shutdown_connections 2024-03-08T23:52:42.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:42.458+0000 7f18aec17700 1 -- 172.21.15.22:0/1484421135 wait complete. 2024-03-08T23:52:43.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 -- 172.21.15.22:0/99661498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78100f60 msgr2=0x7fea78101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:43.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 --2- 172.21.15.22:0/99661498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78100f60 0x7fea78101340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea7000b0d0 tx=0x7fea7000b3e0 comp rx=0 tx=0).stop 2024-03-08T23:52:43.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 -- 172.21.15.22:0/99661498 shutdown_connections 2024-03-08T23:52:43.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 --2- 172.21.15.22:0/99661498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea78106040 0x7fea7810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 --2- 172.21.15.22:0/99661498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78100f60 0x7fea78101340 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 --2- 172.21.15.22:0/99661498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea78101910 0x7fea78105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 -- 172.21.15.22:0/99661498 >> 172.21.15.22:0/99661498 conn(0x7fea78077d40 msgr2=0x7fea78078150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 -- 172.21.15.22:0/99661498 shutdown_connections 2024-03-08T23:52:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.020+0000 7fea7f3bc700 1 -- 172.21.15.22:0/99661498 wait complete. 2024-03-08T23:52:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 Processor -- start 2024-03-08T23:52:43.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 -- start start 2024-03-08T23:52:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea78100f60 0x7fea78199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78101910 0x7fea781999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea78106040 0x7fea781a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78111050 con 0x7fea78101910 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78110ed0 con 0x7fea78100f60 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7f3bc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea781111d0 con 0x7fea78106040 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7c957700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78101910 0x7fea781999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7c957700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78101910 0x7fea781999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60532/0 (socket says 172.21.15.22:60532) 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.021+0000 7fea7c957700 1 -- 172.21.15.22:0/3186566111 learned_addr learned my addr 172.21.15.22:0/3186566111 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7d158700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea78100f60 0x7fea78199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7d959700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea78106040 0x7fea781a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7c957700 1 -- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea78106040 msgr2=0x7fea781a44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7c957700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea78106040 0x7fea781a44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7c957700 1 -- 172.21.15.22:0/3186566111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea78100f60 msgr2=0x7fea78199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7c957700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea78100f60 0x7fea78199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7c957700 1 -- 172.21.15.22:0/3186566111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea781a4a30 con 0x7fea78101910 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7c957700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78101910 0x7fea781999b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00e970 tx=0x7fea6c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:43.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00cbd0 con 0x7fea78101910 2024-03-08T23:52:43.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea6c00cd70 con 0x7fea78101910 2024-03-08T23:52:43.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea781a4c60 con 0x7fea78101910 2024-03-08T23:52:43.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c010430 con 0x7fea78101910 2024-03-08T23:52:43.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.022+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea781a5170 con 0x7fea78101910 2024-03-08T23:52:43.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.023+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea78062470 con 0x7fea78101910 2024-03-08T23:52:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.024+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fea6c004750 con 0x7fea78101910 2024-03-08T23:52:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea767fc700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fea600a29a0 0x7fea600a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea7d158700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fea600a29a0 0x7fea600a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fea6c0cf110 con 0x7fea78101910 2024-03-08T23:52:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea767fc700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] conn(0x7fea600ad660 0x7fea600afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 --> [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea600b0170 con 0x7fea600ad660 2024-03-08T23:52:43.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea6c0d6050 con 0x7fea78101910 2024-03-08T23:52:43.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea7d959700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] conn(0x7fea600ad660 0x7fea600afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:43.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.025+0000 7fea7d158700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fea600a29a0 0x7fea600a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea70009fd0 tx=0x7fea7000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:43.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.026+0000 7fea7d959700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] conn(0x7fea600ad660 0x7fea600afaa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:43.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.027+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== osd.11 v2:172.21.15.192:6824/3259500463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fea600b0170 con 0x7fea600ad660 2024-03-08T23:52:43.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.043+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 --> [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea780628a0 con 0x7fea600ad660 2024-03-08T23:52:43.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.043+0000 7fea767fc700 1 -- 172.21.15.22:0/3186566111 <== osd.11 v2:172.21.15.192:6824/3259500463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fea780628a0 con 0x7fea600ad660 2024-03-08T23:52:43.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] conn(0x7fea600ad660 msgr2=0x7fea600afaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] conn(0x7fea600ad660 0x7fea600afaa0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fea600a29a0 msgr2=0x7fea600a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fea600a29a0 0x7fea600a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea70009fd0 tx=0x7fea7000b040 comp rx=0 tx=0).stop 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78101910 msgr2=0x7fea781999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 --2- 172.21.15.22:0/3186566111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea78101910 0x7fea781999b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00e970 tx=0x7fea6c00ed30 comp rx=0 tx=0).stop 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7d959700 1 -- 172.21.15.22:0/3186566111 reap_dead start 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 shutdown_connections 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 >> 172.21.15.22:0/3186566111 conn(0x7fea78077d40 msgr2=0x7fea780fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 shutdown_connections 2024-03-08T23:52:43.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:43.044+0000 7fea7f3bc700 1 -- 172.21.15.22:0/3186566111 wait complete. 2024-03-08T23:52:43.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:43 smithi022 ceph-mon[31894]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-08T23:52:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:43 smithi062 ceph-mon[36129]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-08T23:52:43.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:43 smithi192 ceph-mon[35995]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-08T23:52:44.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-03-08T23:52:44.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-03-08T23:52:44.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-08T23:52:44.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-08T23:52:44.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-03-08T23:52:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-03-08T23:52:45.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-08T23:52:45.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:45 smithi022 ceph-mon[31894]: osdmap e90: 12 total, 12 up, 12 in 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-08T23:52:45.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-03-08T23:52:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-03-08T23:52:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-08T23:52:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:45 smithi062 ceph-mon[36129]: osdmap e90: 12 total, 12 up, 12 in 2024-03-08T23:52:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-08T23:52:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-08T23:52:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:45 smithi192 ceph-mon[35995]: osdmap e90: 12 total, 12 up, 12 in 2024-03-08T23:52:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 -- 172.21.15.22:0/2821034361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 msgr2=0x7f02d0069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 --2- 172.21.15.22:0/2821034361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 0x7f02d0069ef0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f02c40099c0 tx=0x7f02c4009cd0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 -- 172.21.15.22:0/2821034361 shutdown_connections 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 --2- 172.21.15.22:0/2821034361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02d006a430 0x7f02d010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 --2- 172.21.15.22:0/2821034361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d0107e30 0x7f02d0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 --2- 172.21.15.22:0/2821034361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 0x7f02d0069ef0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 -- 172.21.15.22:0/2821034361 >> 172.21.15.22:0/2821034361 conn(0x7f02d0077c10 msgr2=0x7f02d0078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 -- 172.21.15.22:0/2821034361 shutdown_connections 2024-03-08T23:52:46.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.168+0000 7f02d6523700 1 -- 172.21.15.22:0/2821034361 wait complete. 2024-03-08T23:52:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 Processor -- start 2024-03-08T23:52:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 -- start start 2024-03-08T23:52:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 0x7f02d01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02d006a430 0x7f02d0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d0107e30 0x7f02d019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d01120b0 con 0x7f02d0069a90 2024-03-08T23:52:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0111f30 con 0x7f02d0107e30 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02d6523700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0112230 con 0x7f02d006a430 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02cffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 0x7f02d01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02cffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 0x7f02d01985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60556/0 (socket says 172.21.15.22:60556) 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.169+0000 7f02cffff700 1 -- 172.21.15.22:0/3501468433 learned_addr learned my addr 172.21.15.22:0/3501468433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02d4ac0700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d0107e30 0x7f02d019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cffff700 1 -- 172.21.15.22:0/3501468433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02d006a430 msgr2=0x7f02d0198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cf7fe700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02d006a430 0x7f02d0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cffff700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02d006a430 0x7f02d0198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cffff700 1 -- 172.21.15.22:0/3501468433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d0107e30 msgr2=0x7f02d019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cffff700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d0107e30 0x7f02d019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cffff700 1 -- 172.21.15.22:0/3501468433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d019d5a0 con 0x7f02d0069a90 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02d4ac0700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d0107e30 0x7f02d019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cf7fe700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02d006a430 0x7f02d0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cffff700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 0x7f02d01985e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c920 tx=0x7f02c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0007b00 con 0x7f02d0069a90 2024-03-08T23:52:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d019d890 con 0x7f02d0069a90 2024-03-08T23:52:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02c0004d10 con 0x7f02d0069a90 2024-03-08T23:52:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00056e0 con 0x7f02d0069a90 2024-03-08T23:52:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.170+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d01a5110 con 0x7f02d0069a90 2024-03-08T23:52:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.171+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02d00624f0 con 0x7f02d0069a90 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.172+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f02c0007ca0 con 0x7f02d0069a90 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02cd7fa700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f02b80a2920 0x7f02b80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02cf7fe700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f02b80a2920 0x7f02b80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f02c00cef10 con 0x7f02d0069a90 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02cd7fa700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] conn(0x7f02b80ada80 0x7f02b80aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 --> [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c0012ec0 con 0x7f02b80ada80 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02c00cf360 con 0x7f02d0069a90 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02d4ac0700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] conn(0x7f02b80ada80 0x7f02b80aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.173+0000 7f02cf7fe700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f02b80a2920 0x7f02b80a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009750 tx=0x7f02c4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:46.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.174+0000 7f02d4ac0700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] conn(0x7f02b80ada80 0x7f02b80aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:46.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.174+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== osd.3 v2:172.21.15.22:6810/2127248159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f02c0012ec0 con 0x7f02b80ada80 2024-03-08T23:52:46.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 --> [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f02d00628f0 con 0x7f02b80ada80 2024-03-08T23:52:46.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02cd7fa700 1 -- 172.21.15.22:0/3501468433 <== osd.3 v2:172.21.15.22:6810/2127248159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f02d00628f0 con 0x7f02b80ada80 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] conn(0x7f02b80ada80 msgr2=0x7f02b80aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] conn(0x7f02b80ada80 0x7f02b80aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f02b80a2920 msgr2=0x7f02b80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f02b80a2920 0x7f02b80a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009750 tx=0x7f02c4009f10 comp rx=0 tx=0).stop 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 msgr2=0x7f02d01985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 --2- 172.21.15.22:0/3501468433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02d0069a90 0x7f02d01985e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c920 tx=0x7f02c000cce0 comp rx=0 tx=0).stop 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d4ac0700 1 -- 172.21.15.22:0/3501468433 reap_dead start 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.192+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 shutdown_connections 2024-03-08T23:52:46.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.193+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 >> 172.21.15.22:0/3501468433 conn(0x7f02d0077c10 msgr2=0x7f02d00ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:46.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.193+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 shutdown_connections 2024-03-08T23:52:46.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:46.193+0000 7f02d6523700 1 -- 172.21.15.22:0/3501468433 wait complete. 2024-03-08T23:52:46.663 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:46 smithi022 ceph-mon[31894]: osdmap e91: 12 total, 12 up, 12 in 2024-03-08T23:52:46.665 INFO:teuthology.orchestra.run.smithi022.stdout:343597383703 2024-03-08T23:52:46.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-08T23:52:46.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:46 smithi062 ceph-mon[36129]: osdmap e91: 12 total, 12 up, 12 in 2024-03-08T23:52:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:46 smithi192 ceph-mon[35995]: osdmap e91: 12 total, 12 up, 12 in 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 -- 172.21.15.22:0/607160485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 msgr2=0x7f8fa81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 --2- 172.21.15.22:0/607160485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 0x7f8fa81022b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0099c0 tx=0x7f8f9c009cd0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 -- 172.21.15.22:0/607160485 shutdown_connections 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 --2- 172.21.15.22:0/607160485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa81027f0 0x7f8fa810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 --2- 172.21.15.22:0/607160485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa8107e10 0x7f8fa81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 --2- 172.21.15.22:0/607160485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 0x7f8fa81022b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 -- 172.21.15.22:0/607160485 >> 172.21.15.22:0/607160485 conn(0x7f8fa8078f10 msgr2=0x7f8fa8079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 -- 172.21.15.22:0/607160485 shutdown_connections 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 -- 172.21.15.22:0/607160485 wait complete. 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.350+0000 7f8fae6dc700 1 Processor -- start 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fae6dc700 1 -- start start 2024-03-08T23:52:47.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fae6dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 0x7f8fa80740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fae6dc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa81027f0 0x7f8fa8074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fae6dc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa8107e10 0x7f8fa8074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fae6dc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa81120b0 con 0x7f8fa8101e30 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fae6dc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa8111f30 con 0x7f8fa81027f0 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fae6dc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa8112230 con 0x7f8fa8107e10 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 0x7f8fa80740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 0x7f8fa80740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60576/0 (socket says 172.21.15.22:60576) 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 -- 172.21.15.22:0/2441500618 learned_addr learned my addr 172.21.15.22:0/2441500618 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa77fe700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa81027f0 0x7f8fa8074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8facc79700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa8107e10 0x7f8fa8074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 -- 172.21.15.22:0/2441500618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa8107e10 msgr2=0x7f8fa8074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa8107e10 0x7f8fa8074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 -- 172.21.15.22:0/2441500618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa81027f0 msgr2=0x7f8fa8074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa81027f0 0x7f8fa8074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.351+0000 7f8fa7fff700 1 -- 172.21.15.22:0/2441500618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa81a6a30 con 0x7f8fa8101e30 2024-03-08T23:52:47.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.352+0000 7f8fa7fff700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 0x7f8fa80740e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800d980 tx=0x7f8f9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:47.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.352+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98017070 con 0x7f8fa8101e30 2024-03-08T23:52:47.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.352+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa81a6d20 con 0x7f8fa8101e30 2024-03-08T23:52:47.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.352+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f98005020 con 0x7f8fa8101e30 2024-03-08T23:52:47.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.352+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800fbf0 con 0x7f8fa8101e30 2024-03-08T23:52:47.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.352+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa81a7180 con 0x7f8fa8101e30 2024-03-08T23:52:47.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.353+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8fa80624f0 con 0x7f8fa8101e30 2024-03-08T23:52:47.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.354+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8f980051d0 con 0x7f8fa8101e30 2024-03-08T23:52:47.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.354+0000 7f8fa57fa700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8f900a2920 0x7f8f900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8fa77fe700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8f900a2920 0x7f8f900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8f980cf290 con 0x7f8fa8101e30 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8fa57fa700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] conn(0x7f8f900ada80 0x7f8f900aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 --> [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f900b05f0 con 0x7f8f900ada80 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f980cf640 con 0x7f8fa8101e30 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8facc79700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] conn(0x7f8f900ada80 0x7f8f900aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8fa77fe700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8f900a2920 0x7f8f900a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8075b70 tx=0x7f8f9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.355+0000 7f8facc79700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] conn(0x7f8f900ada80 0x7f8f900aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:47.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.356+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== osd.9 v2:172.21.15.22:6826/460204078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8f900b05f0 con 0x7f8f900ada80 2024-03-08T23:52:47.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.372+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 --> [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8fa8062970 con 0x7f8f900ada80 2024-03-08T23:52:47.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fa57fa700 1 -- 172.21.15.22:0/2441500618 <== osd.9 v2:172.21.15.22:6826/460204078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8fa8062970 con 0x7f8f900ada80 2024-03-08T23:52:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] conn(0x7f8f900ada80 msgr2=0x7f8f900aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] conn(0x7f8f900ada80 0x7f8f900aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8f900a2920 msgr2=0x7f8f900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8f900a2920 0x7f8f900a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8075b70 tx=0x7f8f9c005fb0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 msgr2=0x7f8fa80740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 --2- 172.21.15.22:0/2441500618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fa8101e30 0x7f8fa80740e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800d980 tx=0x7f8f9800dc90 comp rx=0 tx=0).stop 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8facc79700 1 -- 172.21.15.22:0/2441500618 reap_dead start 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 shutdown_connections 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 >> 172.21.15.22:0/2441500618 conn(0x7f8fa8078f10 msgr2=0x7f8fa81009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 shutdown_connections 2024-03-08T23:52:47.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.373+0000 7f8fae6dc700 1 -- 172.21.15.22:0/2441500618 wait complete. 2024-03-08T23:52:47.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:47 smithi022 ceph-mon[31894]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-08T23:52:47.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 -- 172.21.15.22:0/3522072564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 msgr2=0x7f114c10ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 --2- 172.21.15.22:0/3522072564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 0x7f114c10ecd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f114400b0d0 tx=0x7f114400b3e0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 -- 172.21.15.22:0/3522072564 shutdown_connections 2024-03-08T23:52:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 --2- 172.21.15.22:0/3522072564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 0x7f114c10ecd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 --2- 172.21.15.22:0/3522072564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c102020 0x7f114c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 --2- 172.21.15.22:0/3522072564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1029d0 0x7f114c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 -- 172.21.15.22:0/3522072564 >> 172.21.15.22:0/3522072564 conn(0x7f114c0fd890 msgr2=0x7f114c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 -- 172.21.15.22:0/3522072564 shutdown_connections 2024-03-08T23:52:47.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 -- 172.21.15.22:0/3522072564 wait complete. 2024-03-08T23:52:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 Processor -- start 2024-03-08T23:52:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.650+0000 7f1152fa8700 1 -- start start 2024-03-08T23:52:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1152fa8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c102020 0x7f114c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1152fa8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1029d0 0x7f114c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1152fa8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 0x7f114c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1152fa8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c112110 con 0x7f114c107100 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1152fa8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c111f90 con 0x7f114c102020 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1152fa8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c112290 con 0x7f114c1029d0 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 0x7f114c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 0x7f114c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60604/0 (socket says 172.21.15.22:60604) 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f114bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1029d0 0x7f114c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 -- 172.21.15.22:0/485910120 learned_addr learned my addr 172.21.15.22:0/485910120 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1150d44700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c102020 0x7f114c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 -- 172.21.15.22:0/485910120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1029d0 msgr2=0x7f114c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1029d0 0x7f114c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 -- 172.21.15.22:0/485910120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c102020 msgr2=0x7f114c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c102020 0x7f114c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 -- 172.21.15.22:0/485910120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c19d500 con 0x7f114c107100 2024-03-08T23:52:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.651+0000 7f1151545700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 0x7f114c19cdc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1144006070 tx=0x7f1144004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:47.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.652+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144021070 con 0x7f114c107100 2024-03-08T23:52:47.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.652+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c19d790 con 0x7f114c107100 2024-03-08T23:52:47.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.652+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1144007b80 con 0x7f114c107100 2024-03-08T23:52:47.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.652+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440052a0 con 0x7f114c107100 2024-03-08T23:52:47.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.652+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c1a5070 con 0x7f114c107100 2024-03-08T23:52:47.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.653+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f114c0fed50 con 0x7f114c107100 2024-03-08T23:52:47.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.654+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1144024070 con 0x7f114c107100 2024-03-08T23:52:47.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f1149ffb700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f113c0a2920 0x7f113c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f114400f020 con 0x7f114c107100 2024-03-08T23:52:47.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f1150d44700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f113c0a2920 0x7f113c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f1149ffb700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] conn(0x7f113c0ada80 0x7f113c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:47.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 --> [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11440020b0 con 0x7f113c0ada80 2024-03-08T23:52:47.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11440d0b90 con 0x7f114c107100 2024-03-08T23:52:47.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f1150d44700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f113c0a2920 0x7f113c0a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1140005950 tx=0x7f114000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:47.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.655+0000 7f114bfff700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] conn(0x7f113c0ada80 0x7f113c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:47.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.656+0000 7f114bfff700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] conn(0x7f113c0ada80 0x7f113c0aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:47.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.657+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== osd.5 v2:172.21.15.192:6808/3950935286 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f11440020b0 con 0x7f113c0ada80 2024-03-08T23:52:47.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.676+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 --> [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f114c103500 con 0x7f113c0ada80 2024-03-08T23:52:47.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.676+0000 7f1149ffb700 1 -- 172.21.15.22:0/485910120 <== osd.5 v2:172.21.15.192:6808/3950935286 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f114c103500 con 0x7f113c0ada80 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 >> [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] conn(0x7f113c0ada80 msgr2=0x7f113c0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] conn(0x7f113c0ada80 0x7f113c0aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f113c0a2920 msgr2=0x7f113c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f113c0a2920 0x7f113c0a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1140005950 tx=0x7f114000b410 comp rx=0 tx=0).stop 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 msgr2=0x7f114c19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 --2- 172.21.15.22:0/485910120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f114c107100 0x7f114c19cdc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1144006070 tx=0x7f1144004d00 comp rx=0 tx=0).stop 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1151545700 1 -- 172.21.15.22:0/485910120 reap_dead start 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 shutdown_connections 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 >> 172.21.15.22:0/485910120 conn(0x7f114c0fd890 msgr2=0x7f114c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 shutdown_connections 2024-03-08T23:52:47.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:47.677+0000 7f1152fa8700 1 -- 172.21.15.22:0/485910120 wait complete. 2024-03-08T23:52:47.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:47 smithi062 ceph-mon[36129]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-08T23:52:47.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:47 smithi192 ceph-mon[35995]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-08T23:52:49.720 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:49 smithi022 ceph-mon[31894]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-08T23:52:49.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:49 smithi062 ceph-mon[36129]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-08T23:52:49.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:49 smithi192 ceph-mon[35995]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 640 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-08T23:52:51.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:51 smithi022 ceph-mon[31894]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-03-08T23:52:51.803 INFO:teuthology.orchestra.run.smithi022.stdout:107374182464 2024-03-08T23:52:51.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-08T23:52:51.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:51 smithi062 ceph-mon[36129]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-03-08T23:52:51.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:51 smithi192 ceph-mon[35995]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-03-08T23:52:52.323 INFO:teuthology.orchestra.run.smithi022.stdout:309237645339 2024-03-08T23:52:52.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-08T23:52:52.716 INFO:teuthology.orchestra.run.smithi022.stdout:287762808865 2024-03-08T23:52:52.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-08T23:52:53.698 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:53 smithi192 ceph-mon[35995]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 383 B/s wr, 3 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-03-08T23:52:53.699 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:53 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:52:53.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:53 smithi022 ceph-mon[31894]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 383 B/s wr, 3 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-03-08T23:52:53.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:53 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:52:53.829 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:53 smithi062 ceph-mon[36129]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 383 B/s wr, 3 op/s; 7.4 KiB/s, 0 objects/s recovering 2024-03-08T23:52:53.829 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:53 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:52:54.089 INFO:teuthology.orchestra.run.smithi022.stdout:163208757303 2024-03-08T23:52:54.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-08T23:52:55.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:55 smithi022 ceph-mon[31894]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 326 B/s wr, 4 op/s; 6.3 KiB/s, 0 objects/s recovering 2024-03-08T23:52:55.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:55 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:55 smithi062 ceph-mon[36129]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 326 B/s wr, 4 op/s; 6.3 KiB/s, 0 objects/s recovering 2024-03-08T23:52:55.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:55 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:55 smithi192 ceph-mon[35995]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 326 B/s wr, 4 op/s; 6.3 KiB/s, 0 objects/s recovering 2024-03-08T23:52:55.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:55.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:55 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 -- 172.21.15.22:0/3921710059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 msgr2=0x7f94e4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 --2- 172.21.15.22:0/3921710059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 0x7f94e4100d50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94d40099c0 tx=0x7f94d4009cd0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 -- 172.21.15.22:0/3921710059 shutdown_connections 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 --2- 172.21.15.22:0/3921710059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94e4107190 0x7f94e410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 --2- 172.21.15.22:0/3921710059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 0x7f94e4100d50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 --2- 172.21.15.22:0/3921710059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e4101320 0x7f94e4106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 -- 172.21.15.22:0/3921710059 >> 172.21.15.22:0/3921710059 conn(0x7f94e4076010 msgr2=0x7f94e4076420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 -- 172.21.15.22:0/3921710059 shutdown_connections 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.002+0000 7f94ea2e7700 1 -- 172.21.15.22:0/3921710059 wait complete. 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 Processor -- start 2024-03-08T23:52:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 -- start start 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 0x7f94e4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e4101320 0x7f94e4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94e4107190 0x7f94e419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4112120 con 0x7f94e4100970 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4111fa0 con 0x7f94e4107190 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94ea2e7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e41122a0 con 0x7f94e4101320 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94e3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 0x7f94e4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94e3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 0x7f94e4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50410/0 (socket says 172.21.15.22:50410) 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.003+0000 7f94e3fff700 1 -- 172.21.15.22:0/2005483895 learned_addr learned my addr 172.21.15.22:0/2005483895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e3fff700 1 -- 172.21.15.22:0/2005483895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e4101320 msgr2=0x7f94e4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e37fe700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e4101320 0x7f94e4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e8884700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94e4107190 0x7f94e419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e3fff700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e4101320 0x7f94e4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e3fff700 1 -- 172.21.15.22:0/2005483895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94e4107190 msgr2=0x7f94e419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e3fff700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94e4107190 0x7f94e419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e3fff700 1 -- 172.21.15.22:0/2005483895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e419d4d0 con 0x7f94e4100970 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e37fe700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e4101320 0x7f94e4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e3fff700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 0x7f94e4198510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f94d4006010 tx=0x7f94d4005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4027070 con 0x7f94e4100970 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e419d760 con 0x7f94e4100970 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94d40040d0 con 0x7f94e4100970 2024-03-08T23:52:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400f460 con 0x7f94e4100970 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.004+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e41a5040 con 0x7f94e4100970 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.005+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94e40624f0 con 0x7f94e4100970 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.006+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f94d402a070 con 0x7f94e4100970 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e17fa700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f94cc0a2920 0x7f94cc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e37fe700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f94cc0a2920 0x7f94cc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f94d40d0980 con 0x7f94e4100970 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e17fa700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] conn(0x7f94cc0ada00 0x7f94cc0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 --> [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94d4010ec0 con 0x7f94cc0ada00 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94d40d0d30 con 0x7f94e4100970 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e8884700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] conn(0x7f94cc0ada00 0x7f94cc0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e37fe700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f94cc0a2920 0x7f94cc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f94d8005fd0 tx=0x7f94d8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.007+0000 7f94e8884700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] conn(0x7f94cc0ada00 0x7f94cc0afea0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.008+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== osd.6 v2:172.21.15.22:6818/1438134132 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f94d4010ec0 con 0x7f94cc0ada00 2024-03-08T23:52:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 --> [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94e40628f0 con 0x7f94cc0ada00 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94e17fa700 1 -- 172.21.15.22:0/2005483895 <== osd.6 v2:172.21.15.22:6818/1438134132 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f94e40628f0 con 0x7f94cc0ada00 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] conn(0x7f94cc0ada00 msgr2=0x7f94cc0afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94ea2e7700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] conn(0x7f94cc0ada00 0x7f94cc0afea0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f94cc0a2920 msgr2=0x7f94cc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94ea2e7700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f94cc0a2920 0x7f94cc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f94d8005fd0 tx=0x7f94d8005e20 comp rx=0 tx=0).stop 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 msgr2=0x7f94e4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94ea2e7700 1 --2- 172.21.15.22:0/2005483895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94e4100970 0x7f94e4198510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f94d4006010 tx=0x7f94d4005640 comp rx=0 tx=0).stop 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.025+0000 7f94e8884700 1 -- 172.21.15.22:0/2005483895 reap_dead start 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.026+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 shutdown_connections 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.026+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 >> 172.21.15.22:0/2005483895 conn(0x7f94e4076010 msgr2=0x7f94e40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.026+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 shutdown_connections 2024-03-08T23:52:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:52.026+0000 7f94ea2e7700 1 -- 172.21.15.22:0/2005483895 wait complete. 2024-03-08T23:52:56.716 INFO:teuthology.orchestra.run.smithi022.stdout:197568495667 2024-03-08T23:52:56.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-08T23:52:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 -- 172.21.15.22:0/2081944795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4105fc0 msgr2=0x7f54b410db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 --2- 172.21.15.22:0/2081944795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4105fc0 0x7f54b410db90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00b0d0 tx=0x7f54ac00b3e0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 -- 172.21.15.22:0/2081944795 shutdown_connections 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 --2- 172.21.15.22:0/2081944795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4105fc0 0x7f54b410db90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 --2- 172.21.15.22:0/2081944795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b4100f70 0x7f54b4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 --2- 172.21.15.22:0/2081944795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54b4101920 0x7f54b41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 -- 172.21.15.22:0/2081944795 >> 172.21.15.22:0/2081944795 conn(0x7f54b4076d30 msgr2=0x7f54b4077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 -- 172.21.15.22:0/2081944795 shutdown_connections 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.923+0000 7f54b8fba700 1 -- 172.21.15.22:0/2081944795 wait complete. 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 Processor -- start 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 -- start start 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4100f70 0x7f54b4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54b4101920 0x7f54b4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b4105fc0 0x7f54b419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4110fa0 con 0x7f54b4100f70 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4110e20 con 0x7f54b4101920 2024-03-08T23:52:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b8fba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4111120 con 0x7f54b4105fc0 2024-03-08T23:52:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4100f70 0x7f54b4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4100f70 0x7f54b4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50420/0 (socket says 172.21.15.22:50420) 2024-03-08T23:52:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.924+0000 7f54b259c700 1 -- 172.21.15.22:0/2902176915 learned_addr learned my addr 172.21.15.22:0/2902176915 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b2d9d700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b4105fc0 0x7f54b419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b1d9b700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54b4101920 0x7f54b4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b259c700 1 -- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b4105fc0 msgr2=0x7f54b419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b259c700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b4105fc0 0x7f54b419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b259c700 1 -- 172.21.15.22:0/2902176915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54b4101920 msgr2=0x7f54b4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b259c700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54b4101920 0x7f54b4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b259c700 1 -- 172.21.15.22:0/2902176915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b419c340 con 0x7f54b4100f70 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b2d9d700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54b4105fc0 0x7f54b419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b1d9b700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54b4101920 0x7f54b4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b259c700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4100f70 0x7f54b4197330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54a400c9b0 tx=0x7f54a400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4007b00 con 0x7f54b4100f70 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b419c630 con 0x7f54b4100f70 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54a4007ca0 con 0x7f54b4100f70 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4018780 con 0x7f54b4100f70 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.925+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b41a3eb0 con 0x7f54b4100f70 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.926+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54b40ff600 con 0x7f54b4100f70 2024-03-08T23:52:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.927+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f54a400f450 con 0x7f54b4100f70 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.927+0000 7f54a37fe700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f549c0a2920 0x7f549c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.928+0000 7f54b1d9b700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f549c0a2920 0x7f549c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.928+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f54a40cf400 con 0x7f54b4100f70 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.928+0000 7f54a37fe700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] conn(0x7f549c0ada80 0x7f549c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.928+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 --> [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54a4015e70 con 0x7f549c0ada80 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.928+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a40cf850 con 0x7f54b4100f70 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.928+0000 7f54b1d9b700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f549c0a2920 0x7f549c0a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f54a8005e10 tx=0x7f54a8005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.928+0000 7f54b2d9d700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] conn(0x7f549c0ada80 0x7f549c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.929+0000 7f54b2d9d700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] conn(0x7f549c0ada80 0x7f549c0aff20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.930+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== osd.8 v2:172.21.15.192:6816/3107982312 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f54a4015e70 con 0x7f549c0ada80 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.946+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 --> [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54b41023c0 con 0x7f549c0ada80 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54a37fe700 1 -- 172.21.15.22:0/2902176915 <== osd.8 v2:172.21.15.192:6816/3107982312 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f54b41023c0 con 0x7f549c0ada80 2024-03-08T23:52:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] conn(0x7f549c0ada80 msgr2=0x7f549c0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] conn(0x7f549c0ada80 0x7f549c0aff20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f549c0a2920 msgr2=0x7f549c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f549c0a2920 0x7f549c0a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f54a8005e10 tx=0x7f54a8005d60 comp rx=0 tx=0).stop 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4100f70 msgr2=0x7f54b4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 --2- 172.21.15.22:0/2902176915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54b4100f70 0x7f54b4197330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54a400c9b0 tx=0x7f54a400cd70 comp rx=0 tx=0).stop 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b2d9d700 1 -- 172.21.15.22:0/2902176915 reap_dead start 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 shutdown_connections 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 >> 172.21.15.22:0/2902176915 conn(0x7f54b4076d30 msgr2=0x7f54b40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 shutdown_connections 2024-03-08T23:52:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:56.947+0000 7f54b8fba700 1 -- 172.21.15.22:0/2902176915 wait complete. 2024-03-08T23:52:57.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:57 smithi022 ceph-mon[31894]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 295 B/s wr, 5 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-03-08T23:52:57.858 INFO:teuthology.orchestra.run.smithi022.stdout:253403070505 2024-03-08T23:52:57.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-08T23:52:57.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:57 smithi062 ceph-mon[36129]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 295 B/s wr, 5 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-03-08T23:52:57.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:57 smithi192 ceph-mon[35995]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 295 B/s wr, 5 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-03-08T23:52:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 -- 172.21.15.22:0/3716160776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740101960 msgr2=0x7f17401059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 --2- 172.21.15.22:0/3716160776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740101960 0x7f17401059b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f173c0099c0 tx=0x7f173c009cd0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 -- 172.21.15.22:0/3716160776 shutdown_connections 2024-03-08T23:52:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 --2- 172.21.15.22:0/3716160776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740106090 0x7f174010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 --2- 172.21.15.22:0/3716160776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740100fb0 0x7f1740101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 --2- 172.21.15.22:0/3716160776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740101960 0x7f17401059b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 -- 172.21.15.22:0/3716160776 >> 172.21.15.22:0/3716160776 conn(0x7f1740077e10 msgr2=0x7f1740078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 -- 172.21.15.22:0/3716160776 shutdown_connections 2024-03-08T23:52:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.621+0000 7f1747488700 1 -- 172.21.15.22:0/3716160776 wait complete. 2024-03-08T23:52:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 Processor -- start 2024-03-08T23:52:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 -- start start 2024-03-08T23:52:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100fb0 0x7f1740198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740101960 0x7f1740198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740106090 0x7f174019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401110d0 con 0x7f1740106090 2024-03-08T23:52:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1740110f50 con 0x7f1740100fb0 2024-03-08T23:52:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1747488700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1740111250 con 0x7f1740101960 2024-03-08T23:52:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1745a25700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740106090 0x7f174019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1745a25700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740106090 0x7f174019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50442/0 (socket says 172.21.15.22:50442) 2024-03-08T23:52:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1745a25700 1 -- 172.21.15.22:0/930251529 learned_addr learned my addr 172.21.15.22:0/930251529 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.622+0000 7f1745224700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100fb0 0x7f1740198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1744a23700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740101960 0x7f1740198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1745a25700 1 -- 172.21.15.22:0/930251529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740101960 msgr2=0x7f1740198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1745a25700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1740101960 0x7f1740198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1745a25700 1 -- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100fb0 msgr2=0x7f1740198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1745a25700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1740100fb0 0x7f1740198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1745a25700 1 -- 172.21.15.22:0/930251529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174019d660 con 0x7f1740106090 2024-03-08T23:52:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1745a25700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740106090 0x7f174019cf20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1738007d70 tx=0x7f173800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738016020 con 0x7f1740106090 2024-03-08T23:52:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f173800ad40 con 0x7f1740106090 2024-03-08T23:52:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380116f0 con 0x7f1740106090 2024-03-08T23:52:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174019d950 con 0x7f1740106090 2024-03-08T23:52:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.623+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17401a51d0 con 0x7f1740106090 2024-03-08T23:52:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.624+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17400624f0 con 0x7f1740106090 2024-03-08T23:52:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.625+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1738019020 con 0x7f1740106090 2024-03-08T23:52:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f17367fc700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f172c0a2920 0x7f172c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f1745224700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f172c0a2920 0x7f172c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f17380cf470 con 0x7f1740106090 2024-03-08T23:52:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f17367fc700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] conn(0x7f172c0ada80 0x7f172c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 --> [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1738019ec0 con 0x7f172c0ada80 2024-03-08T23:52:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17380cf820 con 0x7f1740106090 2024-03-08T23:52:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f1745224700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f172c0a2920 0x7f172c0a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1730006fd0 tx=0x7f1730008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.626+0000 7f1744a23700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] conn(0x7f172c0ada80 0x7f172c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.627+0000 7f1744a23700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] conn(0x7f172c0ada80 0x7f172c0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.628+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== osd.1 v2:172.21.15.62:6800/782240245 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1738019ec0 con 0x7f172c0ada80 2024-03-08T23:52:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.643+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 --> [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17400628f0 con 0x7f172c0ada80 2024-03-08T23:52:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f17367fc700 1 -- 172.21.15.22:0/930251529 <== osd.1 v2:172.21.15.62:6800/782240245 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f17400628f0 con 0x7f172c0ada80 2024-03-08T23:52:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] conn(0x7f172c0ada80 msgr2=0x7f172c0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] conn(0x7f172c0ada80 0x7f172c0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f172c0a2920 msgr2=0x7f172c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f172c0a2920 0x7f172c0a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1730006fd0 tx=0x7f1730008040 comp rx=0 tx=0).stop 2024-03-08T23:52:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740106090 msgr2=0x7f174019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 --2- 172.21.15.22:0/930251529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1740106090 0x7f174019cf20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1738007d70 tx=0x7f173800a340 comp rx=0 tx=0).stop 2024-03-08T23:52:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1745a25700 1 -- 172.21.15.22:0/930251529 reap_dead start 2024-03-08T23:52:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 shutdown_connections 2024-03-08T23:52:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.644+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 >> 172.21.15.22:0/930251529 conn(0x7f1740077e10 msgr2=0x7f174006a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.645+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 shutdown_connections 2024-03-08T23:52:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:57.645+0000 7f1747488700 1 -- 172.21.15.22:0/930251529 wait complete. 2024-03-08T23:52:58.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.910+0000 7fbebd331700 1 -- 172.21.15.22:0/1818361235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 msgr2=0x7fbeb8101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.910+0000 7fbebd331700 1 --2- 172.21.15.22:0/1818361235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 0x7fbeb8101350 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009990 tx=0x7fbea8009ca0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.910+0000 7fbebd331700 1 -- 172.21.15.22:0/1818361235 shutdown_connections 2024-03-08T23:52:58.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.910+0000 7fbebd331700 1 --2- 172.21.15.22:0/1818361235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8105ff0 0x7fbeb810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.910+0000 7fbebd331700 1 --2- 172.21.15.22:0/1818361235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 0x7fbeb8101350 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.910+0000 7fbebd331700 1 --2- 172.21.15.22:0/1818361235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8101920 0x7fbeb8105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.910+0000 7fbebd331700 1 -- 172.21.15.22:0/1818361235 >> 172.21.15.22:0/1818361235 conn(0x7fbeb8076d20 msgr2=0x7fbeb8077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 -- 172.21.15.22:0/1818361235 shutdown_connections 2024-03-08T23:52:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 -- 172.21.15.22:0/1818361235 wait complete. 2024-03-08T23:52:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 Processor -- start 2024-03-08T23:52:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 -- start start 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 0x7fbeb8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8105ff0 0x7fbeb819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8111030 con 0x7fbeb8100f70 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8110eb0 con 0x7fbeb8101920 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.911+0000 7fbebd331700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81111b0 con 0x7fbeb8105ff0 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 0x7fbeb8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 0x7fbeb8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50462/0 (socket says 172.21.15.22:50462) 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 -- 172.21.15.22:0/2851692684 learned_addr learned my addr 172.21.15.22:0/2851692684 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb67fc700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb77fe700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8105ff0 0x7fbeb819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 -- 172.21.15.22:0/2851692684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8105ff0 msgr2=0x7fbeb819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8105ff0 0x7fbeb819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 -- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 msgr2=0x7fbeb8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 -- 172.21.15.22:0/2851692684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb819d720 con 0x7fbeb8100f70 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb67fc700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb8101920 0x7fbeb8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb77fe700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbeb8105ff0 0x7fbeb819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbeb6ffd700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 0x7fbeb8198710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbea800bae0 tx=0x7fbea800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8026070 con 0x7fbeb8100f70 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbea8003d70 con 0x7fbeb8100f70 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8021410 con 0x7fbeb8100f70 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb819d9b0 con 0x7fbeb8100f70 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.912+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb81a5290 con 0x7fbeb8100f70 2024-03-08T23:52:58.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.913+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbeb80624f0 con 0x7fbeb8100f70 2024-03-08T23:52:58.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.915+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fbea8029070 con 0x7fbeb8100f70 2024-03-08T23:52:58.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.915+0000 7fbe9ffff700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbea00a2920 0x7fbea00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.916+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbea80d08c0 con 0x7fbeb8100f70 2024-03-08T23:52:58.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.916+0000 7fbeb67fc700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbea00a2920 0x7fbea00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.916+0000 7fbe9ffff700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] conn(0x7fbea00ada80 0x7fbea00aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:58.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.916+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 --> [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbea802ae50 con 0x7fbea00ada80 2024-03-08T23:52:58.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.916+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea80d0d10 con 0x7fbeb8100f70 2024-03-08T23:52:58.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.916+0000 7fbeb77fe700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] conn(0x7fbea00ada80 0x7fbea00aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:58.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.916+0000 7fbeb67fc700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbea00a2920 0x7fbea00a4de0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8105c00 tx=0x7fbeac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:58.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.917+0000 7fbeb77fe700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] conn(0x7fbea00ada80 0x7fbea00aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:58.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.917+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== osd.7 v2:172.21.15.62:6816/92390165 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbea802ae50 con 0x7fbea00ada80 2024-03-08T23:52:58.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.934+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 --> [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbeb80628f0 con 0x7fbea00ada80 2024-03-08T23:52:58.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.934+0000 7fbe9ffff700 1 -- 172.21.15.22:0/2851692684 <== osd.7 v2:172.21.15.62:6816/92390165 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbeb80628f0 con 0x7fbea00ada80 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] conn(0x7fbea00ada80 msgr2=0x7fbea00aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] conn(0x7fbea00ada80 0x7fbea00aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbea00a2920 msgr2=0x7fbea00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fbea00a2920 0x7fbea00a4de0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8105c00 tx=0x7fbeac009190 comp rx=0 tx=0).stop 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 msgr2=0x7fbeb8198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 --2- 172.21.15.22:0/2851692684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbeb8100f70 0x7fbeb8198710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbea800bae0 tx=0x7fbea800bb10 comp rx=0 tx=0).stop 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbeb77fe700 1 -- 172.21.15.22:0/2851692684 reap_dead start 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 shutdown_connections 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 >> 172.21.15.22:0/2851692684 conn(0x7fbeb8076d20 msgr2=0x7fbeb806a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 shutdown_connections 2024-03-08T23:52:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:58.935+0000 7fbebd331700 1 -- 172.21.15.22:0/2851692684 wait complete. 2024-03-08T23:52:59.380 INFO:teuthology.orchestra.run.smithi022.stdout:55834574924 2024-03-08T23:52:59.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 -- 172.21.15.22:0/924250428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44106020 msgr2=0x7fae4410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 --2- 172.21.15.22:0/924250428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44106020 0x7fae4410dbf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b0d0 tx=0x7fae3c00b3e0 comp rx=0 tx=0).stop 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 -- 172.21.15.22:0/924250428 shutdown_connections 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 --2- 172.21.15.22:0/924250428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44106020 0x7fae4410dbf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 --2- 172.21.15.22:0/924250428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae44100f40 0x7fae44101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 --2- 172.21.15.22:0/924250428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae441018f0 0x7fae44105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 -- 172.21.15.22:0/924250428 >> 172.21.15.22:0/924250428 conn(0x7fae44077db0 msgr2=0x7fae440781c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 -- 172.21.15.22:0/924250428 shutdown_connections 2024-03-08T23:52:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.574+0000 7fae4b093700 1 -- 172.21.15.22:0/924250428 wait complete. 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 Processor -- start 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 -- start start 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44100f40 0x7fae44199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae441018f0 0x7fae44199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae44106020 0x7fae441a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44111000 con 0x7fae44100f40 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44110e80 con 0x7fae441018f0 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae4b093700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44111180 con 0x7fae44106020 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae48e2f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44100f40 0x7fae44199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae48e2f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44100f40 0x7fae44199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50474/0 (socket says 172.21.15.22:50474) 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae48e2f700 1 -- 172.21.15.22:0/3564416443 learned_addr learned my addr 172.21.15.22:0/3564416443 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae43fff700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae441018f0 0x7fae44199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.575+0000 7fae49630700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae44106020 0x7fae441a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae48e2f700 1 -- 172.21.15.22:0/3564416443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae44106020 msgr2=0x7fae441a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae48e2f700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae44106020 0x7fae441a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae48e2f700 1 -- 172.21.15.22:0/3564416443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae441018f0 msgr2=0x7fae44199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae48e2f700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae441018f0 0x7fae44199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae48e2f700 1 -- 172.21.15.22:0/3564416443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae441a48c0 con 0x7fae44100f40 2024-03-08T23:52:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae49630700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae44106020 0x7fae441a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae48e2f700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44100f40 0x7fae44199300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00d950 tx=0x7fae2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c016070 con 0x7fae44100f40 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae441a4af0 con 0x7fae44100f40 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae2c009e20 con 0x7fae44100f40 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00f620 con 0x7fae44100f40 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.576+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae441a5000 con 0x7fae44100f40 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.577+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae440ff4f0 con 0x7fae44100f40 2024-03-08T23:52:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.579+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fae2c019070 con 0x7fae44100f40 2024-03-08T23:52:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.579+0000 7fae41ffb700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fae300a2920 0x7fae300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.579+0000 7fae43fff700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fae300a2920 0x7fae300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.579+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fae2c0ceed0 con 0x7fae44100f40 2024-03-08T23:52:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.580+0000 7fae41ffb700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] conn(0x7fae300ada80 0x7fae300aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:52:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.580+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 --> [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae2c00ae50 con 0x7fae300ada80 2024-03-08T23:52:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.580+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae2c0d1590 con 0x7fae44100f40 2024-03-08T23:52:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.580+0000 7fae49630700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] conn(0x7fae300ada80 0x7fae300aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:52:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.580+0000 7fae43fff700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fae300a2920 0x7fae300a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae44107330 tx=0x7fae34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.580+0000 7fae49630700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] conn(0x7fae300ada80 0x7fae300aff20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:52:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.581+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== osd.0 v2:172.21.15.22:6802/103047658 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fae2c00ae50 con 0x7fae300ada80 2024-03-08T23:52:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.599+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 --> [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae44102420 con 0x7fae300ada80 2024-03-08T23:52:59.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae41ffb700 1 -- 172.21.15.22:0/3564416443 <== osd.0 v2:172.21.15.22:6802/103047658 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fae44102420 con 0x7fae300ada80 2024-03-08T23:52:59.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] conn(0x7fae300ada80 msgr2=0x7fae300aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:59.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae4b093700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] conn(0x7fae300ada80 0x7fae300aff20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:52:59.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fae300a2920 msgr2=0x7fae300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:59.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae4b093700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fae300a2920 0x7fae300a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae44107330 tx=0x7fae34009260 comp rx=0 tx=0).stop 2024-03-08T23:52:59.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44100f40 msgr2=0x7fae44199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:52:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae4b093700 1 --2- 172.21.15.22:0/3564416443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae44100f40 0x7fae44199300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00d950 tx=0x7fae2c00dc60 comp rx=0 tx=0).stop 2024-03-08T23:52:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.600+0000 7fae49630700 1 -- 172.21.15.22:0/3564416443 reap_dead start 2024-03-08T23:52:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.601+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 shutdown_connections 2024-03-08T23:52:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.601+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 >> 172.21.15.22:0/3564416443 conn(0x7fae44077db0 msgr2=0x7fae440feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:52:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.601+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 shutdown_connections 2024-03-08T23:52:59.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:52:59.601+0000 7fae4b093700 1 -- 172.21.15.22:0/3564416443 wait complete. 2024-03-08T23:52:59.608 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:52:59 smithi022 ceph-mon[31894]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 4.9 KiB/s, 0 objects/s recovering 2024-03-08T23:52:59.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:52:59 smithi062 ceph-mon[36129]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 4.9 KiB/s, 0 objects/s recovering 2024-03-08T23:52:59.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:52:59 smithi192 ceph-mon[35995]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 4.9 KiB/s, 0 objects/s recovering 2024-03-08T23:53:00.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.411+0000 7f39e275d700 1 -- 172.21.15.22:0/1047089049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc106ed0 msgr2=0x7f39dc1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:00.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.411+0000 7f39e275d700 1 --2- 172.21.15.22:0/1047089049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc106ed0 0x7f39dc1072b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009990 tx=0x7f39cc009ca0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.411+0000 7f39e275d700 1 -- 172.21.15.22:0/1047089049 shutdown_connections 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.411+0000 7f39e275d700 1 --2- 172.21.15.22:0/1047089049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39dc101980 0x7f39dc10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.411+0000 7f39e275d700 1 --2- 172.21.15.22:0/1047089049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc106ed0 0x7f39dc1072b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.411+0000 7f39e275d700 1 --2- 172.21.15.22:0/1047089049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39dc100fc0 0x7f39dc101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.411+0000 7f39e275d700 1 -- 172.21.15.22:0/1047089049 >> 172.21.15.22:0/1047089049 conn(0x7f39dc077b40 msgr2=0x7f39dc077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 -- 172.21.15.22:0/1047089049 shutdown_connections 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 -- 172.21.15.22:0/1047089049 wait complete. 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 Processor -- start 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 -- start start 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc100fc0 0x7f39dc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39dc101980 0x7f39dc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39dc106ed0 0x7f39dc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc111040 con 0x7f39dc100fc0 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc110ec0 con 0x7f39dc106ed0 2024-03-08T23:53:00.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.412+0000 7f39e275d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc1111c0 con 0x7f39dc101980 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc100fc0 0x7f39dc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc100fc0 0x7f39dc1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50494/0 (socket says 172.21.15.22:50494) 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 -- 172.21.15.22:0/4133642726 learned_addr learned my addr 172.21.15.22:0/4133642726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39db7fe700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39dc101980 0x7f39dc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39e0cfa700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39dc106ed0 0x7f39dc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 -- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39dc101980 msgr2=0x7f39dc198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39dc101980 0x7f39dc198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 -- 172.21.15.22:0/4133642726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39dc106ed0 msgr2=0x7f39dc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39dc106ed0 0x7f39dc19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 -- 172.21.15.22:0/4133642726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc19d590 con 0x7f39dc100fc0 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39dbfff700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc100fc0 0x7f39dc1985d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00b5c0 tx=0x7f39cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc027070 con 0x7f39dc100fc0 2024-03-08T23:53:00.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39cc004030 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc19d820 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00f460 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.413+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc1a5100 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.414+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39dc104e30 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.415+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f39cc02a070 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.416+0000 7f39d97fa700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f39c40a2920 0x7f39c40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.416+0000 7f39db7fe700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f39c40a2920 0x7f39c40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.416+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f39cc0d0830 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.416+0000 7f39d97fa700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] conn(0x7f39c40ada80 0x7f39c40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.416+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 --> [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39c40b05f0 con 0x7f39c40ada80 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.416+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39cc0d0be0 con 0x7f39dc100fc0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.417+0000 7f39db7fe700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f39c40a2920 0x7f39c40a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0780b0 tx=0x7f39d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.417+0000 7f39e0cfa700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] conn(0x7f39c40ada80 0x7f39c40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.417+0000 7f39e0cfa700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] conn(0x7f39c40ada80 0x7f39c40aff20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:00.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.418+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== osd.2 v2:172.21.15.192:6800/316512323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f39c40b05f0 con 0x7f39c40ada80 2024-03-08T23:53:00.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.435+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 --> [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39dc102460 con 0x7f39c40ada80 2024-03-08T23:53:00.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.435+0000 7f39d97fa700 1 -- 172.21.15.22:0/4133642726 <== osd.2 v2:172.21.15.192:6800/316512323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f39dc102460 con 0x7f39c40ada80 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] conn(0x7f39c40ada80 msgr2=0x7f39c40aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] conn(0x7f39c40ada80 0x7f39c40aff20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f39c40a2920 msgr2=0x7f39c40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f39c40a2920 0x7f39c40a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0780b0 tx=0x7f39d000b410 comp rx=0 tx=0).stop 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc100fc0 msgr2=0x7f39dc1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 --2- 172.21.15.22:0/4133642726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39dc100fc0 0x7f39dc1985d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00b5c0 tx=0x7f39cc0047b0 comp rx=0 tx=0).stop 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e0cfa700 1 -- 172.21.15.22:0/4133642726 reap_dead start 2024-03-08T23:53:00.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 shutdown_connections 2024-03-08T23:53:00.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 >> 172.21.15.22:0/4133642726 conn(0x7f39dc077b40 msgr2=0x7f39dc100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:00.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 shutdown_connections 2024-03-08T23:53:00.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:00.436+0000 7f39e275d700 1 -- 172.21.15.22:0/4133642726 wait complete. 2024-03-08T23:53:01.086 INFO:teuthology.orchestra.run.smithi022.stdout:34359738450 2024-03-08T23:53:01.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-08T23:53:01.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:01 smithi022 ceph-mon[31894]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 4.9 KiB/s, 0 objects/s recovering 2024-03-08T23:53:01.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:01 smithi062 ceph-mon[36129]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 4.9 KiB/s, 0 objects/s recovering 2024-03-08T23:53:01.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:01 smithi192 ceph-mon[35995]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 4.9 KiB/s, 0 objects/s recovering 2024-03-08T23:53:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 -- 172.21.15.22:0/3405710626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc101920 msgr2=0x7f81fc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 --2- 172.21.15.22:0/3405710626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc101920 0x7f81fc105920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009990 tx=0x7f81ec009ca0 comp rx=0 tx=0).stop 2024-03-08T23:53:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 -- 172.21.15.22:0/3405710626 shutdown_connections 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 --2- 172.21.15.22:0/3405710626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81fc106000 0x7f81fc10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 --2- 172.21.15.22:0/3405710626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81fc100f70 0x7f81fc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 --2- 172.21.15.22:0/3405710626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc101920 0x7f81fc105920 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 -- 172.21.15.22:0/3405710626 >> 172.21.15.22:0/3405710626 conn(0x7f81fc076d00 msgr2=0x7f81fc077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 -- 172.21.15.22:0/3405710626 shutdown_connections 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.948+0000 7f8201845700 1 -- 172.21.15.22:0/3405710626 wait complete. 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 Processor -- start 2024-03-08T23:53:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 -- start start 2024-03-08T23:53:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc100f70 0x7f81fc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81fc101920 0x7f81fc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81fc106000 0x7f81fc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc112120 con 0x7f81fc100f70 2024-03-08T23:53:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc111fa0 con 0x7f81fc106000 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f8201845700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc1122a0 con 0x7f81fc101920 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f81faffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc100f70 0x7f81fc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f81faffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc100f70 0x7f81fc198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59234/0 (socket says 172.21.15.22:59234) 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.949+0000 7f81faffd700 1 -- 172.21.15.22:0/3853690430 learned_addr learned my addr 172.21.15.22:0/3853690430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81fb7fe700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81fc106000 0x7f81fc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81fa7fc700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81fc101920 0x7f81fc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81faffd700 1 -- 172.21.15.22:0/3853690430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81fc101920 msgr2=0x7f81fc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81faffd700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81fc101920 0x7f81fc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81faffd700 1 -- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81fc106000 msgr2=0x7f81fc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81faffd700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81fc106000 0x7f81fc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81faffd700 1 -- 172.21.15.22:0/3853690430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc19d530 con 0x7f81fc100f70 2024-03-08T23:53:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f81faffd700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc100f70 0x7f81fc198570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f81f000d950 tx=0x7f81f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0016070 con 0x7f81fc100f70 2024-03-08T23:53:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc19d820 con 0x7f81fc100f70 2024-03-08T23:53:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81f0009e20 con 0x7f81fc100f70 2024-03-08T23:53:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f000f620 con 0x7f81fc100f70 2024-03-08T23:53:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.950+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc1a50a0 con 0x7f81fc100f70 2024-03-08T23:53:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.951+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81fc062570 con 0x7f81fc100f70 2024-03-08T23:53:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.952+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f81f0019070 con 0x7f81fc100f70 2024-03-08T23:53:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.953+0000 7f8200843700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f81e40a2920 0x7f81e40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.953+0000 7f81fa7fc700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f81e40a2920 0x7f81e40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.953+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f81f00ceec0 con 0x7f81fc100f70 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.953+0000 7f8200843700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] conn(0x7f81e40ada80 0x7f81e40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.953+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 --> [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81f0014e80 con 0x7f81e40ada80 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.953+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81f00cf310 con 0x7f81fc100f70 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.953+0000 7f81fa7fc700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f81e40a2920 0x7f81e40a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81ec006010 tx=0x7f81ec00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.954+0000 7f81fb7fe700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] conn(0x7f81e40ada80 0x7f81e40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.954+0000 7f81fb7fe700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] conn(0x7f81e40ada80 0x7f81e40aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.955+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== osd.4 v2:172.21.15.62:6808/590253654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f81f0014e80 con 0x7f81e40ada80 2024-03-08T23:53:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.971+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 --> [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f81fc0629a0 con 0x7f81e40ada80 2024-03-08T23:53:01.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8200843700 1 -- 172.21.15.22:0/3853690430 <== osd.4 v2:172.21.15.62:6808/590253654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f81fc0629a0 con 0x7f81e40ada80 2024-03-08T23:53:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] conn(0x7f81e40ada80 msgr2=0x7f81e40aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] conn(0x7f81e40ada80 0x7f81e40aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f81e40a2920 msgr2=0x7f81e40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f81e40a2920 0x7f81e40a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81ec006010 tx=0x7f81ec00b540 comp rx=0 tx=0).stop 2024-03-08T23:53:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc100f70 msgr2=0x7f81fc198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 --2- 172.21.15.22:0/3853690430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81fc100f70 0x7f81fc198570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f81f000d950 tx=0x7f81f000dc60 comp rx=0 tx=0).stop 2024-03-08T23:53:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f81fb7fe700 1 -- 172.21.15.22:0/3853690430 reap_dead start 2024-03-08T23:53:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 shutdown_connections 2024-03-08T23:53:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.972+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 >> 172.21.15.22:0/3853690430 conn(0x7f81fc076d00 msgr2=0x7f81fc105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.973+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 shutdown_connections 2024-03-08T23:53:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:01.973+0000 7f8201845700 1 -- 172.21.15.22:0/3853690430 wait complete. 2024-03-08T23:53:02.081 INFO:teuthology.orchestra.run.smithi022.stdout:77309411400 2024-03-08T23:53:02.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-08T23:53:02.787 INFO:teuthology.orchestra.run.smithi022.stdout:141733920830 2024-03-08T23:53:02.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-08T23:53:03.300 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:03.304 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:03.317 INFO:teuthology.orchestra.run.smithi022.stdout:219043332142 2024-03-08T23:53:03.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-08T23:53:03.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:03 smithi062 ceph-mon[36129]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-08T23:53:03.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:03 smithi192 ceph-mon[35995]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-08T23:53:03.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:03 smithi022 ceph-mon[31894]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-08T23:53:04.943 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:04.945 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:04.946 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:04.946 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:05.068 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:05.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:05 smithi022 ceph-mon[31894]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:05.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:05 smithi062 ceph-mon[36129]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:05.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:05 smithi192 ceph-mon[35995]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:07.709 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:07 smithi022 ceph-mon[31894]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-08T23:53:07.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:07 smithi062 ceph-mon[36129]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-08T23:53:07.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:07 smithi192 ceph-mon[35995]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-08T23:53:09.714 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:09 smithi022 ceph-mon[31894]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:09.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:09 smithi062 ceph-mon[36129]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:09.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:09 smithi192 ceph-mon[35995]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 -- 172.21.15.22:0/1599104042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 msgr2=0x7fcad41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/1599104042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad41058e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcac40099c0 tx=0x7fcac4009cd0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 -- 172.21.15.22:0/1599104042 shutdown_connections 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/1599104042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4105fc0 0x7fcad410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/1599104042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad4100f70 0x7fcad4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/1599104042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad41058e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 -- 172.21.15.22:0/1599104042 >> 172.21.15.22:0/1599104042 conn(0x7fcad4076d30 msgr2=0x7fcad4077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:11.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 -- 172.21.15.22:0/1599104042 shutdown_connections 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.288+0000 7fcadbb7a700 1 -- 172.21.15.22:0/1599104042 wait complete. 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 Processor -- start 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 -- start start 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad4100f70 0x7fcad4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4105fc0 0x7fcad419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4110f80 con 0x7fcad4101920 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4110e00 con 0x7fcad4100f70 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcadbb7a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4111100 con 0x7fcad4105fc0 2024-03-08T23:53:11.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcad9115700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stdout:287762808869 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcad9115700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad4198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55672/0 (socket says 172.21.15.22:55672) 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcad9115700 1 -- 172.21.15.22:0/216387615 learned_addr learned my addr 172.21.15.22:0/216387615 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcad9916700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad4100f70 0x7fcad4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.289+0000 7fcada117700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4105fc0 0x7fcad419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcad9115700 1 -- 172.21.15.22:0/216387615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4105fc0 msgr2=0x7fcad419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcad9115700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4105fc0 0x7fcad419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcad9115700 1 -- 172.21.15.22:0/216387615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad4100f70 msgr2=0x7fcad4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcad9115700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad4100f70 0x7fcad4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcad9115700 1 -- 172.21.15.22:0/216387615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad419d5a0 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcad9115700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad4198ad0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcac400fa00 tx=0x7fcac400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcacaffd700 1 -- 172.21.15.22:0/216387615 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4027070 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcacaffd700 1 -- 172.21.15.22:0/216387615 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcac4003680 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad419d830 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcacaffd700 1 -- 172.21.15.22:0/216387615 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac40173f0 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.290+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad41a5110 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.291+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad4068aa0 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.293+0000 7fcacaffd700 1 -- 172.21.15.22:0/216387615 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcac402a070 con 0x7fcad4101920 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.293+0000 7fcacaffd700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcac00a29a0 0x7fcac00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.294+0000 7fcacaffd700 1 -- 172.21.15.22:0/216387615 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fcac40d02b0 con 0x7fcad4101920 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.294+0000 7fcad9916700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcac00a29a0 0x7fcac00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.294+0000 7fcad9916700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcac00a29a0 0x7fcac00a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009c60 tx=0x7fcad0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.297+0000 7fcacaffd700 1 -- 172.21.15.22:0/216387615 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fcac4031070 con 0x7fcad4101920 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.504+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fcad41a52e0 con 0x7fcad4101920 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.504+0000 7fcacaffd700 1 -- 172.21.15.22:0/216387615 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcac4013070 con 0x7fcad4101920 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcac00a29a0 msgr2=0x7fcac00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcac00a29a0 0x7fcac00a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009c60 tx=0x7fcad0009260 comp rx=0 tx=0).stop 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 msgr2=0x7fcad4198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad4198ad0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcac400fa00 tx=0x7fcac400fa30 comp rx=0 tx=0).stop 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 shutdown_connections 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad4105fc0 0x7fcad419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad4100f70 0x7fcad4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcac00a29a0 0x7fcac00a4e60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 --2- 172.21.15.22:0/216387615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcad4101920 0x7fcad4198ad0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:11.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 >> 172.21.15.22:0/216387615 conn(0x7fcad4076d30 msgr2=0x7fcad4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:11.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 shutdown_connections 2024-03-08T23:53:11.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:11.507+0000 7fcadbb7a700 1 -- 172.21.15.22:0/216387615 wait complete. 2024-03-08T23:53:11.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:11 smithi022 ceph-mon[31894]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-08T23:53:11.828 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:11 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/216387615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-08T23:53:11.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:11 smithi062 ceph-mon[36129]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-08T23:53:11.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:11 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/216387615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-08T23:53:11.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:11 smithi192 ceph-mon[35995]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-08T23:53:11.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:11 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/216387615' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-08T23:53:12.363 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.446+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/2600604825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 msgr2=0x7f9edc108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.446+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/2600604825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc108210 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80099c0 tx=0x7f9ed8009cd0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.446+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/2600604825 shutdown_connections 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.446+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/2600604825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc06a430 0x7f9edc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.446+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/2600604825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc108210 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.446+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/2600604825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9edc069a90 0x7f9edc069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.446+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/2600604825 >> 172.21.15.22:0/2600604825 conn(0x7f9edc077c10 msgr2=0x7f9edc078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.447+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/2600604825 shutdown_connections 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.447+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/2600604825 wait complete. 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.447+0000 7f9ee3fb6700 1 Processor -- start 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.447+0000 7f9ee3fb6700 1 -- start start 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.447+0000 7f9ee3fb6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9edc069a90 0x7f9edc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.447+0000 7f9ee3fb6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc06a430 0x7f9edc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.447+0000 7f9ee3fb6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:13.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee3fb6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc112070 con 0x7f9edc107e30 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee3fb6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc111ef0 con 0x7f9edc069a90 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee3fb6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc1121f0 con 0x7f9edc06a430 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55694/0 (socket says 172.21.15.22:55694) 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 -- 172.21.15.22:0/1302229473 learned_addr learned my addr 172.21.15.22:0/1302229473 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee1551700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc06a430 0x7f9edc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee1d52700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9edc069a90 0x7f9edc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 -- 172.21.15.22:0/1302229473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc06a430 msgr2=0x7f9edc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc06a430 0x7f9edc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 -- 172.21.15.22:0/1302229473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9edc069a90 msgr2=0x7f9edc198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9edc069a90 0x7f9edc198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 -- 172.21.15.22:0/1302229473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc19d470 con 0x7f9edc107e30 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee1d52700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9edc069a90 0x7f9edc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:13.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9ee2553700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc19cd30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4007d70 tx=0x7f9ed400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:13.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.448+0000 7f9eceffd700 1 -- 172.21.15.22:0/1302229473 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4016020 con 0x7f9edc107e30 2024-03-08T23:53:13.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.449+0000 7f9eceffd700 1 -- 172.21.15.22:0/1302229473 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ed400ad40 con 0x7f9edc107e30 2024-03-08T23:53:13.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.449+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc19d760 con 0x7f9edc107e30 2024-03-08T23:53:13.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.449+0000 7f9eceffd700 1 -- 172.21.15.22:0/1302229473 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40116f0 con 0x7f9edc107e30 2024-03-08T23:53:13.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.449+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc100ce0 con 0x7f9edc107e30 2024-03-08T23:53:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.450+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc068aa0 con 0x7f9edc107e30 2024-03-08T23:53:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.452+0000 7f9eceffd700 1 -- 172.21.15.22:0/1302229473 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f9ed4019020 con 0x7f9edc107e30 2024-03-08T23:53:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.452+0000 7f9eceffd700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9ec80a2920 0x7f9ec80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.452+0000 7f9ee1d52700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9ec80a2920 0x7f9ec80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.452+0000 7f9eceffd700 1 -- 172.21.15.22:0/1302229473 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9ed40cf230 con 0x7f9edc107e30 2024-03-08T23:53:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.453+0000 7f9ee1d52700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9ec80a2920 0x7f9ec80a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005b40 tx=0x7f9ed8005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:13.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.455+0000 7f9eceffd700 1 -- 172.21.15.22:0/1302229473 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9ed4020020 con 0x7f9edc107e30 2024-03-08T23:53:13.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.665+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f9edc1997c0 con 0x7f9edc107e30 2024-03-08T23:53:13.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.665+0000 7f9eceffd700 1 -- 172.21.15.22:0/1302229473 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9ed4022300 con 0x7f9edc107e30 2024-03-08T23:53:13.662 INFO:teuthology.orchestra.run.smithi022.stdout:343597383709 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9ec80a2920 msgr2=0x7f9ec80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9ec80a2920 0x7f9ec80a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005b40 tx=0x7f9ed8005ab0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 msgr2=0x7f9edc19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc19cd30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4007d70 tx=0x7f9ed400a340 comp rx=0 tx=0).stop 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 shutdown_connections 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9ec80a2920 0x7f9ec80a4de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9edc107e30 0x7f9edc19cd30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9edc069a90 0x7f9edc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 --2- 172.21.15.22:0/1302229473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc06a430 0x7f9edc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:13.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 >> 172.21.15.22:0/1302229473 conn(0x7f9edc077c10 msgr2=0x7f9edc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:13.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 shutdown_connections 2024-03-08T23:53:13.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:13.667+0000 7f9ee3fb6700 1 -- 172.21.15.22:0/1302229473 wait complete. 2024-03-08T23:53:13.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:13 smithi062 ceph-mon[36129]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:13.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:13 smithi192 ceph-mon[35995]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:13.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:13 smithi022 ceph-mon[31894]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:14.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:14 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1302229473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-08T23:53:14.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:14 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1302229473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-08T23:53:14.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:14 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1302229473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 -- 172.21.15.22:0/3264718355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824101680 msgr2=0x7f382410db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 --2- 172.21.15.22:0/3264718355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824101680 0x7f382410db60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f381c009990 tx=0x7f381c009ca0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 -- 172.21.15.22:0/3264718355 shutdown_connections 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 --2- 172.21.15.22:0/3264718355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824101680 0x7f382410db60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 --2- 172.21.15.22:0/3264718355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3824106d20 0x7f3824107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 --2- 172.21.15.22:0/3264718355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3824100ce0 0x7f3824101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 -- 172.21.15.22:0/3264718355 >> 172.21.15.22:0/3264718355 conn(0x7f3824077c10 msgr2=0x7f3824078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 -- 172.21.15.22:0/3264718355 shutdown_connections 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 -- 172.21.15.22:0/3264718355 wait complete. 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.992+0000 7f382965a700 1 Processor -- start 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f382965a700 1 -- start start 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f382965a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3824100ce0 0x7f38241984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:15.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f382965a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3824101680 0x7f38241989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f382965a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824106d20 0x7f382419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f382965a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3824112120 con 0x7f3824106d20 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f382965a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3824111fa0 con 0x7f3824101680 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f382965a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38241122a0 con 0x7f3824100ce0 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824106d20 0x7f382419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824106d20 0x7f382419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55710/0 (socket says 172.21.15.22:55710) 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 -- 172.21.15.22:0/2842945899 learned_addr learned my addr 172.21.15.22:0/2842945899 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38227fc700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3824101680 0x7f38241989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 -- 172.21.15.22:0/2842945899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3824100ce0 msgr2=0x7f38241984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3824100ce0 0x7f38241984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 -- 172.21.15.22:0/2842945899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3824101680 msgr2=0x7f38241989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3824101680 0x7f38241989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 -- 172.21.15.22:0/2842945899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382419d4c0 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38227fc700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3824101680 0x7f38241989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.993+0000 7f38237fe700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824106d20 0x7f382419cd80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f381c0047b0 tx=0x7f381c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.994+0000 7f381bfff700 1 -- 172.21.15.22:0/2842945899 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c004830 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.994+0000 7f381bfff700 1 -- 172.21.15.22:0/2842945899 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f381c005020 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.994+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382419d750 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.994+0000 7f381bfff700 1 -- 172.21.15.22:0/2842945899 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c003e60 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.994+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38241a5030 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.995+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3824109100 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.996+0000 7f381bfff700 1 -- 172.21.15.22:0/2842945899 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f381c0051d0 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.996+0000 7f381bfff700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f38100a2920 0x7f38100a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.996+0000 7f3822ffd700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f38100a2920 0x7f38100a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.996+0000 7f381bfff700 1 -- 172.21.15.22:0/2842945899 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f381c0cfda0 con 0x7f3824106d20 2024-03-08T23:53:15.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:14.997+0000 7f3822ffd700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f38100a2920 0x7f38100a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3814009710 tx=0x7f3814009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:15.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.001+0000 7f381bfff700 1 -- 172.21.15.22:0/2842945899 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f381c013070 con 0x7f3824106d20 2024-03-08T23:53:15.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.211+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f3824199810 con 0x7f3824106d20 2024-03-08T23:53:15.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.211+0000 7f381bfff700 1 -- 172.21.15.22:0/2842945899 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f381c09ca10 con 0x7f3824106d20 2024-03-08T23:53:15.208 INFO:teuthology.orchestra.run.smithi022.stdout:253403070508 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f38100a2920 msgr2=0x7f38100a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f38100a2920 0x7f38100a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3814009710 tx=0x7f3814009620 comp rx=0 tx=0).stop 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824106d20 msgr2=0x7f382419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824106d20 0x7f382419cd80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f381c0047b0 tx=0x7f381c009ee0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 shutdown_connections 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f38100a2920 0x7f38100a4de0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3824106d20 0x7f382419cd80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3824100ce0 0x7f38241984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 --2- 172.21.15.22:0/2842945899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3824101680 0x7f38241989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:15.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.213+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 >> 172.21.15.22:0/2842945899 conn(0x7f3824077c10 msgr2=0x7f38240ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:15.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.214+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 shutdown_connections 2024-03-08T23:53:15.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:15.214+0000 7f382965a700 1 -- 172.21.15.22:0/2842945899 wait complete. 2024-03-08T23:53:15.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:15 smithi062 ceph-mon[36129]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:15.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2842945899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-08T23:53:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:15 smithi192 ceph-mon[35995]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2842945899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-08T23:53:15.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:15 smithi022 ceph-mon[31894]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:15.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2842945899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-08T23:53:16.012 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070508 for osd.8 2024-03-08T23:53:16.012 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 -- 172.21.15.22:0/2761936753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 msgr2=0x7fb760101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 --2- 172.21.15.22:0/2761936753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 0x7fb760101140 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb750009990 tx=0x7fb750009ca0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 -- 172.21.15.22:0/2761936753 shutdown_connections 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 --2- 172.21.15.22:0/2761936753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb760101680 0x7fb76010db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 --2- 172.21.15.22:0/2761936753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760106d70 0x7fb760107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 --2- 172.21.15.22:0/2761936753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 0x7fb760101140 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 -- 172.21.15.22:0/2761936753 >> 172.21.15.22:0/2761936753 conn(0x7fb760077cf0 msgr2=0x7fb760078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 -- 172.21.15.22:0/2761936753 shutdown_connections 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 -- 172.21.15.22:0/2761936753 wait complete. 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.227+0000 7fb765492700 1 Processor -- start 2024-03-08T23:53:16.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb765492700 1 -- start start 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb765492700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 0x7fb760199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb765492700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760101680 0x7fb760199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb765492700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb760106d70 0x7fb7601a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb765492700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb760110fa0 con 0x7fb760100cc0 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb765492700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb760110e20 con 0x7fb760101680 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb765492700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb760111120 con 0x7fb760106d70 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 0x7fb760199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760101680 0x7fb760199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760101680 0x7fb760199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:46812/0 (socket says 172.21.15.22:46812) 2024-03-08T23:53:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 -- 172.21.15.22:0/3959168828 learned_addr learned my addr 172.21.15.22:0/3959168828 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75f7fe700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb760106d70 0x7fb7601a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 -- 172.21.15.22:0/3959168828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb760106d70 msgr2=0x7fb7601a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb760106d70 0x7fb7601a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 -- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 msgr2=0x7fb760199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 0x7fb760199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75e7fc700 1 -- 172.21.15.22:0/3959168828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7601a4900 con 0x7fb760101680 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.228+0000 7fb75effd700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 0x7fb760199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.229+0000 7fb75e7fc700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760101680 0x7fb760199880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb750000c00 tx=0x7fb7500047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.229+0000 7fb757fff700 1 -- 172.21.15.22:0/3959168828 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750027070 con 0x7fb760101680 2024-03-08T23:53:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.229+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7601a4ad0 con 0x7fb760101680 2024-03-08T23:53:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.229+0000 7fb757fff700 1 -- 172.21.15.22:0/3959168828 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb750003e40 con 0x7fb760101680 2024-03-08T23:53:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.229+0000 7fb757fff700 1 -- 172.21.15.22:0/3959168828 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750022d10 con 0x7fb760101680 2024-03-08T23:53:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.230+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7601a4f80 con 0x7fb760101680 2024-03-08T23:53:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.230+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb760068aa0 con 0x7fb760101680 2024-03-08T23:53:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.232+0000 7fb757fff700 1 -- 172.21.15.22:0/3959168828 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb75002a070 con 0x7fb760101680 2024-03-08T23:53:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.232+0000 7fb757fff700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb74c0a2920 0x7fb74c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.232+0000 7fb75effd700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb74c0a2920 0x7fb74c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.232+0000 7fb757fff700 1 -- 172.21.15.22:0/3959168828 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb7500d0310 con 0x7fb760101680 2024-03-08T23:53:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.233+0000 7fb75effd700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb74c0a2920 0x7fb74c0a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb748005950 tx=0x7fb74800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:16.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.236+0000 7fb757fff700 1 -- 172.21.15.22:0/3959168828 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb750031070 con 0x7fb760101680 2024-03-08T23:53:16.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.456+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb76002da90 con 0x7fb760101680 2024-03-08T23:53:16.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.456+0000 7fb757fff700 1 -- 172.21.15.22:0/3959168828 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb750013070 con 0x7fb760101680 2024-03-08T23:53:16.453 INFO:teuthology.orchestra.run.smithi022.stdout:107374182470 2024-03-08T23:53:16.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb74c0a2920 msgr2=0x7fb74c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:16.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb74c0a2920 0x7fb74c0a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb748005950 tx=0x7fb74800b410 comp rx=0 tx=0).stop 2024-03-08T23:53:16.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760101680 msgr2=0x7fb760199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:16.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760101680 0x7fb760199880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb750000c00 tx=0x7fb7500047b0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 shutdown_connections 2024-03-08T23:53:16.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb760106d70 0x7fb7601a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb74c0a2920 0x7fb74c0a4de0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb760100cc0 0x7fb760199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 --2- 172.21.15.22:0/3959168828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb760101680 0x7fb760199880 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:16.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 >> 172.21.15.22:0/3959168828 conn(0x7fb760077cf0 msgr2=0x7fb7600ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:16.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 shutdown_connections 2024-03-08T23:53:16.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:16.459+0000 7fb765492700 1 -- 172.21.15.22:0/3959168828 wait complete. 2024-03-08T23:53:16.690 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383709 for osd.11 2024-03-08T23:53:16.690 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:16.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:16 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3959168828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-08T23:53:16.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:16 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3959168828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-08T23:53:16.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:16 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3959168828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-08T23:53:17.326 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182464 got 107374182470 for osd.3 2024-03-08T23:53:17.326 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:17.840 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:17 smithi022 ceph-mon[31894]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:17.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:17 smithi062 ceph-mon[36129]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:17.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:17 smithi192 ceph-mon[35995]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:18.479 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 -- 172.21.15.22:0/2167376749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c1070f0 msgr2=0x7f968c10eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2167376749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c1070f0 0x7f968c10eca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f968400b0a0 tx=0x7f968400b3b0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 -- 172.21.15.22:0/2167376749 shutdown_connections 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2167376749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c1070f0 0x7f968c10eca0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2167376749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c06a580 0x7f968c06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2167376749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968c06af30 0x7f968c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 -- 172.21.15.22:0/2167376749 >> 172.21.15.22:0/2167376749 conn(0x7f968c076d30 msgr2=0x7f968c077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:19.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 -- 172.21.15.22:0/2167376749 shutdown_connections 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.052+0000 7f9691e8a700 1 -- 172.21.15.22:0/2167376749 wait complete. 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 Processor -- start 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 -- start start 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c06a580 0x7f968c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968c06af30 0x7f968c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c1070f0 0x7f968c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1120b0 con 0x7f968c06a580 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c111f30 con 0x7f968c1070f0 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f9691e8a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c112230 con 0x7f968c06af30 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c06a580 0x7f968c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c06a580 0x7f968c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55760/0 (socket says 172.21.15.22:55760) 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 -- 172.21.15.22:0/2507604101 learned_addr learned my addr 172.21.15.22:0/2507604101 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:19.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968bfff700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c1070f0 0x7f968c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968affd700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968c06af30 0x7f968c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 -- 172.21.15.22:0/2507604101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968c06af30 msgr2=0x7f968c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968c06af30 0x7f968c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 -- 172.21.15.22:0/2507604101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c1070f0 msgr2=0x7f968c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c1070f0 0x7f968c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968b7fe700 1 -- 172.21.15.22:0/2507604101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c19d460 con 0x7f968c06a580 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968bfff700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c1070f0 0x7f968c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.053+0000 7f968affd700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968c06af30 0x7f968c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.054+0000 7f968b7fe700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c06a580 0x7f968c1984a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f968000e9a0 tx=0x7f968000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.054+0000 7f9688ff9700 1 -- 172.21.15.22:0/2507604101 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968000cbd0 con 0x7f968c06a580 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.054+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c19d750 con 0x7f968c06a580 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.054+0000 7f9688ff9700 1 -- 172.21.15.22:0/2507604101 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f968000cd70 con 0x7f968c06a580 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.054+0000 7f9688ff9700 1 -- 172.21.15.22:0/2507604101 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96800187c0 con 0x7f968c06a580 2024-03-08T23:53:19.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.054+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c1a4fd0 con 0x7f968c06a580 2024-03-08T23:53:19.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.055+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c068aa0 con 0x7f968c06a580 2024-03-08T23:53:19.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.056+0000 7f9688ff9700 1 -- 172.21.15.22:0/2507604101 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f9680010430 con 0x7f968c06a580 2024-03-08T23:53:19.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.056+0000 7f9688ff9700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f96740a2920 0x7f96740a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:19.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.057+0000 7f968affd700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f96740a2920 0x7f96740a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:19.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.057+0000 7f9688ff9700 1 -- 172.21.15.22:0/2507604101 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f96800cf240 con 0x7f968c06a580 2024-03-08T23:53:19.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.057+0000 7f968affd700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f96740a2920 0x7f96740a4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f967c005d90 tx=0x7f967c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:19.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.061+0000 7f9688ff9700 1 -- 172.21.15.22:0/2507604101 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f968009be20 con 0x7f968c06a580 2024-03-08T23:53:19.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.282+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f968c02d830 con 0x7f968c06a580 2024-03-08T23:53:19.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.283+0000 7f9688ff9700 1 -- 172.21.15.22:0/2507604101 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f968009b720 con 0x7f968c06a580 2024-03-08T23:53:19.283 INFO:teuthology.orchestra.run.smithi022.stdout:77309411404 2024-03-08T23:53:19.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f96740a2920 msgr2=0x7f96740a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:19.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f96740a2920 0x7f96740a4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f967c005d90 tx=0x7f967c005d00 comp rx=0 tx=0).stop 2024-03-08T23:53:19.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c06a580 msgr2=0x7f968c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:19.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c06a580 0x7f968c1984a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f968000e9a0 tx=0x7f968000ed60 comp rx=0 tx=0).stop 2024-03-08T23:53:19.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 shutdown_connections 2024-03-08T23:53:19.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f968c1070f0 0x7f968c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f96740a2920 0x7f96740a4de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968c06a580 0x7f968c1984a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 --2- 172.21.15.22:0/2507604101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968c06af30 0x7f968c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:19.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 >> 172.21.15.22:0/2507604101 conn(0x7f968c076d30 msgr2=0x7f968c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:19.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 shutdown_connections 2024-03-08T23:53:19.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.285+0000 7f9691e8a700 1 -- 172.21.15.22:0/2507604101 wait complete. 2024-03-08T23:53:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:19 smithi062 ceph-mon[36129]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3271037952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-08T23:53:19.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2507604101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-08T23:53:19.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:19 smithi192 ceph-mon[35995]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:19.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3271037952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-08T23:53:19.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2507604101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-08T23:53:19.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:19 smithi022 ceph-mon[31894]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:19.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3271037952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-08T23:53:19.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2507604101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-08T23:53:20.077 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808869 for osd.9 2024-03-08T23:53:20.078 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:20.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 -- 172.21.15.22:0/1151159167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d801020f0 msgr2=0x7f8d80105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 --2- 172.21.15.22:0/1151159167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d801020f0 0x7f8d80105980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8d740099c0 tx=0x7f8d74009cd0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 -- 172.21.15.22:0/1151159167 shutdown_connections 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 --2- 172.21.15.22:0/1151159167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d80106060 0x7f8d8010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 --2- 172.21.15.22:0/1151159167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d80101740 0x7f8d80101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 --2- 172.21.15.22:0/1151159167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d801020f0 0x7f8d80105980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 -- 172.21.15.22:0/1151159167 >> 172.21.15.22:0/1151159167 conn(0x7f8d800772e0 msgr2=0x7f8d800fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 -- 172.21.15.22:0/1151159167 shutdown_connections 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.950+0000 7f8d84a70700 1 -- 172.21.15.22:0/1151159167 wait complete. 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 Processor -- start 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 -- start start 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d80101740 0x7f8d800fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d801020f0 0x7f8d800fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:20.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d80106060 0x7f8d800fd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80111020 con 0x7f8d80101740 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80110ea0 con 0x7f8d80106060 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d84a70700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d801111a0 con 0x7f8d801020f0 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d80101740 0x7f8d800fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d80101740 0x7f8d800fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55738/0 (socket says 172.21.15.22:55738) 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 -- 172.21.15.22:0/3271037952 learned_addr learned my addr 172.21.15.22:0/3271037952 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7dd9b700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d801020f0 0x7f8d800fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7ed9d700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d80106060 0x7f8d800fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 -- 172.21.15.22:0/3271037952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d801020f0 msgr2=0x7f8d800fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:20.885 INFO:teuthology.orchestra.run.smithi022.stdout:163208757309 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d801020f0 0x7f8d800fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 -- 172.21.15.22:0/3271037952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d80106060 msgr2=0x7f8d800fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d80106060 0x7f8d800fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.951+0000 7f8d7e59c700 1 -- 172.21.15.22:0/3271037952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d801a6ac0 con 0x7f8d80101740 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.952+0000 7f8d7e59c700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d80101740 0x7f8d800fc7c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000c9b0 tx=0x7f8d7000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.952+0000 7f8d6f7fe700 1 -- 172.21.15.22:0/3271037952 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70007b00 con 0x7f8d80101740 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.952+0000 7f8d6f7fe700 1 -- 172.21.15.22:0/3271037952 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d70007ca0 con 0x7f8d80101740 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.952+0000 7f8d6f7fe700 1 -- 172.21.15.22:0/3271037952 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70018780 con 0x7f8d80101740 2024-03-08T23:53:20.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.952+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d801a6db0 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.952+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d801a7210 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.953+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d80068aa0 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.955+0000 7f8d6f7fe700 1 -- 172.21.15.22:0/3271037952 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8d7000f450 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.956+0000 7f8d6f7fe700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d680a2920 0x7f8d680a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.956+0000 7f8d6f7fe700 1 -- 172.21.15.22:0/3271037952 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8d700cf2b0 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.956+0000 7f8d7dd9b700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d680a2920 0x7f8d680a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.956+0000 7f8d7dd9b700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d680a2920 0x7f8d680a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:18.958+0000 7f8d6f7fe700 1 -- 172.21.15.22:0/3271037952 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8d70014030 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.171+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8d800fdcf0 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.171+0000 7f8d6f7fe700 1 -- 172.21.15.22:0/3271037952 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d7009bf20 con 0x7f8d80101740 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d680a2920 msgr2=0x7f8d680a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d680a2920 0x7f8d680a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d74005fd0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d80101740 msgr2=0x7f8d800fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d80101740 0x7f8d800fc7c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000c9b0 tx=0x7f8d7000cd70 comp rx=0 tx=0).stop 2024-03-08T23:53:20.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 shutdown_connections 2024-03-08T23:53:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d80106060 0x7f8d800fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d680a2920 0x7f8d680a4de0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d80101740 0x7f8d800fc7c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 --2- 172.21.15.22:0/3271037952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d801020f0 0x7f8d800fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 >> 172.21.15.22:0/3271037952 conn(0x7f8d800772e0 msgr2=0x7f8d80100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 shutdown_connections 2024-03-08T23:53:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:19.173+0000 7f8d84a70700 1 -- 172.21.15.22:0/3271037952 wait complete. 2024-03-08T23:53:21.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:21 smithi022 ceph-mon[31894]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:21.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:21.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:21 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:21.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:21 smithi062 ceph-mon[36129]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:21.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:21.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:21 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:21.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:21 smithi192 ceph-mon[35995]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:21.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:21.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:21 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 -- 172.21.15.22:0/456726083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180107100 msgr2=0x7f118010ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 --2- 172.21.15.22:0/456726083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180107100 0x7f118010ecd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f117800b0d0 tx=0x7f117800b3e0 comp rx=0 tx=0).stop 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 -- 172.21.15.22:0/456726083 shutdown_connections 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 --2- 172.21.15.22:0/456726083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180107100 0x7f118010ecd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 --2- 172.21.15.22:0/456726083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180102020 0x7f1180102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 --2- 172.21.15.22:0/456726083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11801029d0 0x7f1180106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 -- 172.21.15.22:0/456726083 >> 172.21.15.22:0/456726083 conn(0x7f11800fd890 msgr2=0x7f11800ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 -- 172.21.15.22:0/456726083 shutdown_connections 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 -- 172.21.15.22:0/456726083 wait complete. 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 Processor -- start 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.896+0000 7f1187814700 1 -- start start 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1187814700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180102020 0x7f11800740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1187814700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11801029d0 0x7f1180074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:21.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1187814700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180107100 0x7f1180074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1187814700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11801120f0 con 0x7f1180102020 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1187814700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180111f70 con 0x7f1180107100 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1187814700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180112270 con 0x7f11801029d0 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180102020 0x7f11800740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180102020 0x7f11800740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59482/0 (socket says 172.21.15.22:59482) 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 -- 172.21.15.22:0/3888777719 learned_addr learned my addr 172.21.15.22:0/3888777719 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1184daf700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11801029d0 0x7f1180074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1185db1700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180107100 0x7f1180074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 -- 172.21.15.22:0/3888777719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11801029d0 msgr2=0x7f1180074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11801029d0 0x7f1180074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 -- 172.21.15.22:0/3888777719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180107100 msgr2=0x7f1180074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180107100 0x7f1180074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 -- 172.21.15.22:0/3888777719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11801a6a30 con 0x7f1180102020 2024-03-08T23:53:21.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1185db1700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180107100 0x7f1180074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11855b0700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180102020 0x7f11800740e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f117c00d950 tx=0x7f117c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11767fc700 1 -- 172.21.15.22:0/3888777719 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c016070 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11767fc700 1 -- 172.21.15.22:0/3888777719 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f117c009e20 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11801a6d20 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.897+0000 7f11767fc700 1 -- 172.21.15.22:0/3888777719 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00f620 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.898+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11801a7200 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.898+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1180068aa0 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.900+0000 7f11767fc700 1 -- 172.21.15.22:0/3888777719 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f117c019070 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.900+0000 7f11767fc700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f116c0a2920 0x7f116c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.900+0000 7f1184daf700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f116c0a2920 0x7f116c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.901+0000 7f11767fc700 1 -- 172.21.15.22:0/3888777719 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f117c0ceb80 con 0x7f1180102020 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.901+0000 7f1184daf700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f116c0a2920 0x7f116c0a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1180100f70 tx=0x7f117000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:21.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:21.904+0000 7f11767fc700 1 -- 172.21.15.22:0/3888777719 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f117c09b760 con 0x7f1180102020 2024-03-08T23:53:22.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.118+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f118002d9f0 con 0x7f1180102020 2024-03-08T23:53:22.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.118+0000 7f11767fc700 1 -- 172.21.15.22:0/3888777719 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f117c09b060 con 0x7f1180102020 2024-03-08T23:53:22.115 INFO:teuthology.orchestra.run.smithi022.stdout:34359738454 2024-03-08T23:53:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.120+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f116c0a2920 msgr2=0x7f116c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.120+0000 7f1187814700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f116c0a2920 0x7f116c0a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1180100f70 tx=0x7f117000a300 comp rx=0 tx=0).stop 2024-03-08T23:53:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.120+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180102020 msgr2=0x7f11800740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.120+0000 7f1187814700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180102020 0x7f11800740e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f117c00d950 tx=0x7f117c00dc60 comp rx=0 tx=0).stop 2024-03-08T23:53:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 shutdown_connections 2024-03-08T23:53:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180107100 0x7f1180074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f116c0a2920 0x7f116c0a4de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1180102020 0x7f11800740e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 --2- 172.21.15.22:0/3888777719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11801029d0 0x7f1180074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 >> 172.21.15.22:0/3888777719 conn(0x7f11800fd890 msgr2=0x7f11800fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:22.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 shutdown_connections 2024-03-08T23:53:22.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.121+0000 7f1187814700 1 -- 172.21.15.22:0/3888777719 wait complete. 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 -- 172.21.15.22:0/1907601553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c8107160 msgr2=0x7f85c810ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 --2- 172.21.15.22:0/1907601553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c8107160 0x7f85c810ece0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85c000b0d0 tx=0x7f85c000b3e0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 -- 172.21.15.22:0/1907601553 shutdown_connections 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 --2- 172.21.15.22:0/1907601553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c8107160 0x7f85c810ece0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 --2- 172.21.15.22:0/1907601553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c806a580 0x7f85c806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 --2- 172.21.15.22:0/1907601553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c806af30 0x7f85c8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 -- 172.21.15.22:0/1907601553 >> 172.21.15.22:0/1907601553 conn(0x7f85c8076d30 msgr2=0x7f85c8077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 -- 172.21.15.22:0/1907601553 shutdown_connections 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.169+0000 7f85ceecf700 1 -- 172.21.15.22:0/1907601553 wait complete. 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 Processor -- start 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 -- start start 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c806a580 0x7f85c81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:22.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c806af30 0x7f85c8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c8107160 0x7f85c819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8112120 con 0x7f85c806a580 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8111fa0 con 0x7f85c8107160 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ceecf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81122a0 con 0x7f85c806af30 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c806a580 0x7f85c81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c806a580 0x7f85c81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59510/0 (socket says 172.21.15.22:59510) 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 -- 172.21.15.22:0/4030122425 learned_addr learned my addr 172.21.15.22:0/4030122425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85cd46c700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c8107160 0x7f85c819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85c7fff700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c806af30 0x7f85c8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 -- 172.21.15.22:0/4030122425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c806af30 msgr2=0x7f85c8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c806af30 0x7f85c8198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 -- 172.21.15.22:0/4030122425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c8107160 msgr2=0x7f85c819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c8107160 0x7f85c819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85ccc6b700 1 -- 172.21.15.22:0/4030122425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c819d600 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.170+0000 7f85cd46c700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c8107160 0x7f85c819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.171+0000 7f85ccc6b700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c806a580 0x7f85c81985f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00d950 tx=0x7f85bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.171+0000 7f85c5ffb700 1 -- 172.21.15.22:0/4030122425 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc016070 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.171+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c819d8f0 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.171+0000 7f85c5ffb700 1 -- 172.21.15.22:0/4030122425 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85bc009b40 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.171+0000 7f85c5ffb700 1 -- 172.21.15.22:0/4030122425 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00f620 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.171+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c81a5170 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.172+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c8068aa0 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.173+0000 7f85c5ffb700 1 -- 172.21.15.22:0/4030122425 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f85bc019070 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.173+0000 7f85c5ffb700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f85b80a2920 0x7f85b80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.174+0000 7f85c7fff700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f85b80a2920 0x7f85b80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.174+0000 7f85c5ffb700 1 -- 172.21.15.22:0/4030122425 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f85bc0ceff0 con 0x7f85c806a580 2024-03-08T23:53:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.174+0000 7f85c7fff700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f85b80a2920 0x7f85b80a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f85b4005fd0 tx=0x7f85b4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.178+0000 7f85c5ffb700 1 -- 172.21.15.22:0/4030122425 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f85bc09bb70 con 0x7f85c806a580 2024-03-08T23:53:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.384+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f85c802d830 con 0x7f85c806a580 2024-03-08T23:53:22.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.384+0000 7f85c5ffb700 1 -- 172.21.15.22:0/4030122425 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85bc09b470 con 0x7f85c806a580 2024-03-08T23:53:22.381 INFO:teuthology.orchestra.run.smithi022.stdout:197568495673 2024-03-08T23:53:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f85b80a2920 msgr2=0x7f85b80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f85b80a2920 0x7f85b80a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f85b4005fd0 tx=0x7f85b4005ee0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c806a580 msgr2=0x7f85c81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:22.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c806a580 0x7f85c81985f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00d950 tx=0x7f85bc00dd10 comp rx=0 tx=0).stop 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 shutdown_connections 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c8107160 0x7f85c819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f85b80a2920 0x7f85b80a4de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85c806a580 0x7f85c81985f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 --2- 172.21.15.22:0/4030122425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c806af30 0x7f85c8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 >> 172.21.15.22:0/4030122425 conn(0x7f85c8076d30 msgr2=0x7f85c80feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 shutdown_connections 2024-03-08T23:53:22.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:22.387+0000 7f85ceecf700 1 -- 172.21.15.22:0/4030122425 wait complete. 2024-03-08T23:53:22.561 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757303 got 163208757309 for osd.5 2024-03-08T23:53:22.561 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:22.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:22 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3888777719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-08T23:53:22.658 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:22 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4030122425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-08T23:53:22.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:22 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3888777719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-08T23:53:22.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:22 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4030122425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-08T23:53:22.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:22 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3888777719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-08T23:53:22.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:22 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4030122425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-08T23:53:23.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 -- 172.21.15.22:0/2925854371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 msgr2=0x7fe9ac101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 --2- 172.21.15.22:0/2925854371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac101140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009990 tx=0x7fe9a0009ca0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 -- 172.21.15.22:0/2925854371 shutdown_connections 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 --2- 172.21.15.22:0/2925854371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9ac101680 0x7fe9ac10db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 --2- 172.21.15.22:0/2925854371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac106d70 0x7fe9ac107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 --2- 172.21.15.22:0/2925854371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac101140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 -- 172.21.15.22:0/2925854371 >> 172.21.15.22:0/2925854371 conn(0x7fe9ac077cf0 msgr2=0x7fe9ac078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 -- 172.21.15.22:0/2925854371 shutdown_connections 2024-03-08T23:53:23.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.098+0000 7fe9b2076700 1 -- 172.21.15.22:0/2925854371 wait complete. 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 Processor -- start 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 -- start start 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac101680 0x7fe9ac1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9ac106d70 0x7fe9ac1a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac111080 con 0x7fe9ac100cc0 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac110f00 con 0x7fe9ac101680 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9b2076700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac111200 con 0x7fe9ac106d70 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59528/0 (socket says 172.21.15.22:59528) 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 -- 172.21.15.22:0/1300335988 learned_addr learned my addr 172.21.15.22:0/1300335988 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9aaffd700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac101680 0x7fe9ac1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9abfff700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9ac106d70 0x7fe9ac1a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 -- 172.21.15.22:0/1300335988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9ac106d70 msgr2=0x7fe9ac1a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9ac106d70 0x7fe9ac1a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 -- 172.21.15.22:0/1300335988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac101680 msgr2=0x7fe9ac1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac101680 0x7fe9ac1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.099+0000 7fe9ab7fe700 1 -- 172.21.15.22:0/1300335988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac1a4a60 con 0x7fe9ac100cc0 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.100+0000 7fe9abfff700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9ac106d70 0x7fe9ac1a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.100+0000 7fe9ab7fe700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac1994a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00b530 tx=0x7fe99c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.100+0000 7fe9a8ff9700 1 -- 172.21.15.22:0/1300335988 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c015020 con 0x7fe9ac100cc0 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.100+0000 7fe9a8ff9700 1 -- 172.21.15.22:0/1300335988 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe99c011d10 con 0x7fe9ac100cc0 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.100+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac1a4c90 con 0x7fe9ac100cc0 2024-03-08T23:53:23.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.100+0000 7fe9a8ff9700 1 -- 172.21.15.22:0/1300335988 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c00f2e0 con 0x7fe9ac100cc0 2024-03-08T23:53:23.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.100+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac1a51a0 con 0x7fe9ac100cc0 2024-03-08T23:53:23.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.101+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac068aa0 con 0x7fe9ac100cc0 2024-03-08T23:53:23.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.102+0000 7fe9a8ff9700 1 -- 172.21.15.22:0/1300335988 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe99c018020 con 0x7fe9ac100cc0 2024-03-08T23:53:23.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.102+0000 7fe9a8ff9700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9940a29a0 0x7fe9940a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:23.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.103+0000 7fe9aaffd700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9940a29a0 0x7fe9940a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:23.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.103+0000 7fe9a8ff9700 1 -- 172.21.15.22:0/1300335988 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe99c0ce6a0 con 0x7fe9ac100cc0 2024-03-08T23:53:23.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.103+0000 7fe9aaffd700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9940a29a0 0x7fe9940a4e60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00098e0 tx=0x7fe9a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:23.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.107+0000 7fe9a8ff9700 1 -- 172.21.15.22:0/1300335988 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe99c01f020 con 0x7fe9ac100cc0 2024-03-08T23:53:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.313+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fe9ac02d950 con 0x7fe9ac100cc0 2024-03-08T23:53:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.313+0000 7fe9a8ff9700 1 -- 172.21.15.22:0/1300335988 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe99c09b330 con 0x7fe9ac100cc0 2024-03-08T23:53:23.310 INFO:teuthology.orchestra.run.smithi022.stdout:309237645347 2024-03-08T23:53:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9940a29a0 msgr2=0x7fe9940a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9940a29a0 0x7fe9940a4e60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00098e0 tx=0x7fe9a0005960 comp rx=0 tx=0).stop 2024-03-08T23:53:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 msgr2=0x7fe9ac1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac1994a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00b530 tx=0x7fe99c00b840 comp rx=0 tx=0).stop 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 shutdown_connections 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9ac106d70 0x7fe9ac1a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9940a29a0 0x7fe9940a4e60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9ac100cc0 0x7fe9ac1994a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 --2- 172.21.15.22:0/1300335988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ac101680 0x7fe9ac1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.315+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 >> 172.21.15.22:0/1300335988 conn(0x7fe9ac077cf0 msgr2=0x7fe9ac0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.316+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 shutdown_connections 2024-03-08T23:53:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:23.316+0000 7fe9b2076700 1 -- 172.21.15.22:0/1300335988 wait complete. 2024-03-08T23:53:23.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:23 smithi062 ceph-mon[36129]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-08T23:53:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:53:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:53:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:23 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1300335988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-08T23:53:23.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:23 smithi192 ceph-mon[35995]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-08T23:53:23.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:53:23.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:53:23.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:23.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:23 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1300335988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-08T23:53:23.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:23 smithi022 ceph-mon[31894]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-08T23:53:23.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:53:23.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:53:23.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:53:23.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:23 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1300335988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-08T23:53:24.119 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645347 for osd.10 2024-03-08T23:53:24.119 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:25.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495673 for osd.6 2024-03-08T23:53:25.480 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:25.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:25 smithi022 ceph-mon[31894]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:25.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:25 smithi062 ceph-mon[36129]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:25.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:25 smithi192 ceph-mon[35995]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:26.046 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411404 for osd.2 2024-03-08T23:53:26.046 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:26.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738454 for osd.0 2024-03-08T23:53:26.601 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 -- 172.21.15.22:0/1244660583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80281029d0 msgr2=0x7f8028106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 --2- 172.21.15.22:0/1244660583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80281029d0 0x7f8028106a20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f80180099c0 tx=0x7f8018009cd0 comp rx=0 tx=0).stop 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 -- 172.21.15.22:0/1244660583 shutdown_connections 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 --2- 172.21.15.22:0/1244660583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8028107100 0x7f802810ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 --2- 172.21.15.22:0/1244660583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8028102020 0x7f8028102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 --2- 172.21.15.22:0/1244660583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80281029d0 0x7f8028106a20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 -- 172.21.15.22:0/1244660583 >> 172.21.15.22:0/1244660583 conn(0x7f80280fd890 msgr2=0x7f80280ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 -- 172.21.15.22:0/1244660583 shutdown_connections 2024-03-08T23:53:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.990+0000 7f802d08c700 1 -- 172.21.15.22:0/1244660583 wait complete. 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 Processor -- start 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 -- start start 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8028102020 0x7f80281984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80281029d0 0x7f8028198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8028107100 0x7f802819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281120c0 con 0x7f8028102020 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8028111f40 con 0x7f80281029d0 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802d08c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8028112240 con 0x7f8028107100 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f8026d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8028102020 0x7f80281984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f8026d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8028102020 0x7f80281984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59544/0 (socket says 172.21.15.22:59544) 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f8026d9d700 1 -- 172.21.15.22:0/2856072191 learned_addr learned my addr 172.21.15.22:0/2856072191 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802659c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80281029d0 0x7f8028198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.991+0000 7f802759e700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8028107100 0x7f802819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f8026d9d700 1 -- 172.21.15.22:0/2856072191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8028107100 msgr2=0x7f802819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f8026d9d700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8028107100 0x7f802819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f8026d9d700 1 -- 172.21.15.22:0/2856072191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80281029d0 msgr2=0x7f8028198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f8026d9d700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80281029d0 0x7f8028198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f8026d9d700 1 -- 172.21.15.22:0/2856072191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802819d4e0 con 0x7f8028102020 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f8026d9d700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8028102020 0x7f80281984d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c920 tx=0x7f801c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f800ffff700 1 -- 172.21.15.22:0/2856072191 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c004a30 con 0x7f8028102020 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802819d7d0 con 0x7f8028102020 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f800ffff700 1 -- 172.21.15.22:0/2856072191 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f801c005500 con 0x7f8028102020 2024-03-08T23:53:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f800ffff700 1 -- 172.21.15.22:0/2856072191 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c007d00 con 0x7f8028102020 2024-03-08T23:53:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.992+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80281a5050 con 0x7f8028102020 2024-03-08T23:53:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.993+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8028068aa0 con 0x7f8028102020 2024-03-08T23:53:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.995+0000 7f800ffff700 1 -- 172.21.15.22:0/2856072191 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f801c005000 con 0x7f8028102020 2024-03-08T23:53:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.996+0000 7f800ffff700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f80100a2920 0x7f80100a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.996+0000 7f800ffff700 1 -- 172.21.15.22:0/2856072191 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f801c0993f0 con 0x7f8028102020 2024-03-08T23:53:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.996+0000 7f802659c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f80100a2920 0x7f80100a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:26.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:26.997+0000 7f802659c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f80100a2920 0x7f80100a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8018009990 tx=0x7f8018005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.003+0000 7f800ffff700 1 -- 172.21.15.22:0/2856072191 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f801c014030 con 0x7f8028102020 2024-03-08T23:53:27.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.212+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8028199860 con 0x7f8028102020 2024-03-08T23:53:27.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.212+0000 7f800ffff700 1 -- 172.21.15.22:0/2856072191 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8028199860 con 0x7f8028102020 2024-03-08T23:53:27.209 INFO:teuthology.orchestra.run.smithi022.stdout:141733920835 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f80100a2920 msgr2=0x7f80100a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f80100a2920 0x7f80100a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8018009990 tx=0x7f8018005fb0 comp rx=0 tx=0).stop 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8028102020 msgr2=0x7f80281984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8028102020 0x7f80281984d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c920 tx=0x7f801c00cc30 comp rx=0 tx=0).stop 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 shutdown_connections 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8028107100 0x7f802819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f80100a2920 0x7f80100a4de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8028102020 0x7f80281984d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 --2- 172.21.15.22:0/2856072191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80281029d0 0x7f8028198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:27.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 >> 172.21.15.22:0/2856072191 conn(0x7f80280fd890 msgr2=0x7f80280fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:27.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 shutdown_connections 2024-03-08T23:53:27.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:27.214+0000 7f802d08c700 1 -- 172.21.15.22:0/2856072191 wait complete. 2024-03-08T23:53:27.773 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:27.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:27 smithi062 ceph-mon[36129]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:27.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:27 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2856072191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-08T23:53:27.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:27 smithi022 ceph-mon[31894]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:27.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:27 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2856072191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-08T23:53:27.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:27 smithi192 ceph-mon[35995]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:27.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:27 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2856072191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-08T23:53:28.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 -- 172.21.15.22:0/3285243295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7006aea0 msgr2=0x7f4e7010eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:28.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/3285243295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7006aea0 0x7f4e7010eda0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b0a0 tx=0x7f4e6800b3b0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 -- 172.21.15.22:0/3285243295 shutdown_connections 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/3285243295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7006aea0 0x7f4e7010eda0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/3285243295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e70103d40 0x7f4e70104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/3285243295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e7006a4e0 0x7f4e7006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 -- 172.21.15.22:0/3285243295 >> 172.21.15.22:0/3285243295 conn(0x7f4e70076d00 msgr2=0x7f4e70077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 -- 172.21.15.22:0/3285243295 shutdown_connections 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 -- 172.21.15.22:0/3285243295 wait complete. 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.258+0000 7f4e76cd7700 1 Processor -- start 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e76cd7700 1 -- start start 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e76cd7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e7006a4e0 0x7f4e70198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e76cd7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7006aea0 0x7f4e70198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:28.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e76cd7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e70103d40 0x7f4e7019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e76cd7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701121c0 con 0x7f4e70103d40 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e76cd7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70112040 con 0x7f4e7006aea0 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e76cd7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70112340 con 0x7f4e7006a4e0 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e70103d40 0x7f4e7019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e70103d40 0x7f4e7019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59558/0 (socket says 172.21.15.22:59558) 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 -- 172.21.15.22:0/2103946178 learned_addr learned my addr 172.21.15.22:0/2103946178 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e6ffff700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7006aea0 0x7f4e70198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 -- 172.21.15.22:0/2103946178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e7006a4e0 msgr2=0x7f4e70198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e7006a4e0 0x7f4e70198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 -- 172.21.15.22:0/2103946178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7006aea0 msgr2=0x7f4e70198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7006aea0 0x7f4e70198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 -- 172.21.15.22:0/2103946178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7019d5a0 con 0x7f4e70103d40 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e75274700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e70103d40 0x7f4e7019ce60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006070 tx=0x7f4e68004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e6dffb700 1 -- 172.21.15.22:0/2103946178 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68021070 con 0x7f4e70103d40 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.259+0000 7f4e6dffb700 1 -- 172.21.15.22:0/2103946178 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e68003c00 con 0x7f4e70103d40 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.260+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7019d830 con 0x7f4e70103d40 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.260+0000 7f4e6dffb700 1 -- 172.21.15.22:0/2103946178 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68004850 con 0x7f4e70103d40 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.260+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e701a5110 con 0x7f4e70103d40 2024-03-08T23:53:28.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.261+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e70068aa0 con 0x7f4e70103d40 2024-03-08T23:53:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.262+0000 7f4e6dffb700 1 -- 172.21.15.22:0/2103946178 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f4e68007980 con 0x7f4e70103d40 2024-03-08T23:53:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.262+0000 7f4e6dffb700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4e600a29a0 0x7f4e600a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.262+0000 7f4e74a73700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4e600a29a0 0x7f4e600a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.262+0000 7f4e6dffb700 1 -- 172.21.15.22:0/2103946178 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4e6800f020 con 0x7f4e70103d40 2024-03-08T23:53:28.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.263+0000 7f4e74a73700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4e600a29a0 0x7f4e600a4e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009b60 tx=0x7f4e64009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.267+0000 7f4e6dffb700 1 -- 172.21.15.22:0/2103946178 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4e6802a020 con 0x7f4e70103d40 2024-03-08T23:53:28.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.475+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4e7002d830 con 0x7f4e70103d40 2024-03-08T23:53:28.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.475+0000 7f4e6dffb700 1 -- 172.21.15.22:0/2103946178 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e680074e0 con 0x7f4e70103d40 2024-03-08T23:53:28.472 INFO:teuthology.orchestra.run.smithi022.stdout:55834574930 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4e600a29a0 msgr2=0x7f4e600a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4e600a29a0 0x7f4e600a4e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009b60 tx=0x7f4e64009620 comp rx=0 tx=0).stop 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e70103d40 msgr2=0x7f4e7019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e70103d40 0x7f4e7019ce60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006070 tx=0x7f4e68004d00 comp rx=0 tx=0).stop 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 shutdown_connections 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4e600a29a0 0x7f4e600a4e60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e70103d40 0x7f4e7019ce60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e7006a4e0 0x7f4e70198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 --2- 172.21.15.22:0/2103946178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7006aea0 0x7f4e70198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 >> 172.21.15.22:0/2103946178 conn(0x7f4e70076d00 msgr2=0x7f4e700fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 shutdown_connections 2024-03-08T23:53:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:28.477+0000 7f4e76cd7700 1 -- 172.21.15.22:0/2103946178 wait complete. 2024-03-08T23:53:28.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:28 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2103946178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-08T23:53:28.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:28 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2103946178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-08T23:53:28.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:28 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2103946178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-08T23:53:29.760 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574930 for osd.1 2024-03-08T23:53:29.760 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:29.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:29 smithi062 ceph-mon[36129]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:29.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:29 smithi192 ceph-mon[35995]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:29.912 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:29 smithi022 ceph-mon[31894]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-08T23:53:30.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920835 for osd.4 2024-03-08T23:53:30.245 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:31.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 -- 172.21.15.22:0/3146957719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778102820 msgr2=0x7f277810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 --2- 172.21.15.22:0/3146957719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778102820 0x7f277810ecb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f277000b0a0 tx=0x7f277000b3b0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 -- 172.21.15.22:0/3146957719 shutdown_connections 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 --2- 172.21.15.22:0/3146957719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778102820 0x7f277810ecb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 --2- 172.21.15.22:0/3146957719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2778107eb0 0x7f2778108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 --2- 172.21.15.22:0/3146957719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2778101e60 0x7f27781022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 -- 172.21.15.22:0/3146957719 >> 172.21.15.22:0/3146957719 conn(0x7f2778078f30 msgr2=0x7f2778079340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 -- 172.21.15.22:0/3146957719 shutdown_connections 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.273+0000 7f277dfd3700 1 -- 172.21.15.22:0/3146957719 wait complete. 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 Processor -- start 2024-03-08T23:53:31.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 -- start start 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2778101e60 0x7f27780740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2778102820 0x7f2778074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778107eb0 0x7f2778074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781121c0 con 0x7f2778107eb0 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2778112040 con 0x7f2778101e60 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f277dfd3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2778112340 con 0x7f2778102820 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f2777fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778107eb0 0x7f2778074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f2777fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778107eb0 0x7f2778074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60286/0 (socket says 172.21.15.22:60286) 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f2777fff700 1 -- 172.21.15.22:0/1857706512 learned_addr learned my addr 172.21.15.22:0/1857706512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f2776ffd700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2778102820 0x7f2778074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.274+0000 7f27777fe700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2778101e60 0x7f27780740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:31.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2777fff700 1 -- 172.21.15.22:0/1857706512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2778102820 msgr2=0x7f2778074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2777fff700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2778102820 0x7f2778074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2777fff700 1 -- 172.21.15.22:0/1857706512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2778101e60 msgr2=0x7f27780740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2777fff700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2778101e60 0x7f27780740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2777fff700 1 -- 172.21.15.22:0/1857706512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27781a6b60 con 0x7f2778107eb0 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2776ffd700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2778102820 0x7f2778074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f27777fe700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2778101e60 0x7f27780740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2777fff700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778107eb0 0x7f2778074b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f277000b070 tx=0x7f2770007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2774ff9700 1 -- 172.21.15.22:0/1857706512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770003c00 con 0x7f2778107eb0 2024-03-08T23:53:31.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27781a6df0 con 0x7f2778107eb0 2024-03-08T23:53:31.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2774ff9700 1 -- 172.21.15.22:0/1857706512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2770003da0 con 0x7f2778107eb0 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f2774ff9700 1 -- 172.21.15.22:0/1857706512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277001da50 con 0x7f2778107eb0 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.275+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27781a72a0 con 0x7f2778107eb0 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.276+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277810a290 con 0x7f2778107eb0 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.277+0000 7f2774ff9700 1 -- 172.21.15.22:0/1857706512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2770005720 con 0x7f2778107eb0 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.278+0000 7f2774ff9700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27600a29a0 0x7f27600a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.278+0000 7f27777fe700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27600a29a0 0x7f27600a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.278+0000 7f2774ff9700 1 -- 172.21.15.22:0/1857706512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f277000f020 con 0x7f2778107eb0 2024-03-08T23:53:31.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.279+0000 7f27777fe700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27600a29a0 0x7f27600a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2768005950 tx=0x7f2768005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:31.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.282+0000 7f2774ff9700 1 -- 172.21.15.22:0/1857706512 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f277009d1b0 con 0x7f2778107eb0 2024-03-08T23:53:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.488+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2778075610 con 0x7f2778107eb0 2024-03-08T23:53:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.488+0000 7f2774ff9700 1 -- 172.21.15.22:0/1857706512 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f277009cab0 con 0x7f2778107eb0 2024-03-08T23:53:31.485 INFO:teuthology.orchestra.run.smithi022.stdout:219043332149 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.490+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27600a29a0 msgr2=0x7f27600a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.490+0000 7f277dfd3700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27600a29a0 0x7f27600a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2768005950 tx=0x7f2768005f50 comp rx=0 tx=0).stop 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.490+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778107eb0 msgr2=0x7f2778074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.490+0000 7f277dfd3700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778107eb0 0x7f2778074b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f277000b070 tx=0x7f2770007a50 comp rx=0 tx=0).stop 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 shutdown_connections 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27600a29a0 0x7f27600a4e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2778107eb0 0x7f2778074b60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2778101e60 0x7f27780740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 --2- 172.21.15.22:0/1857706512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2778102820 0x7f2778074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 >> 172.21.15.22:0/1857706512 conn(0x7f2778078f30 msgr2=0x7f2778100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 shutdown_connections 2024-03-08T23:53:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:31.491+0000 7f277dfd3700 1 -- 172.21.15.22:0/1857706512 wait complete. 2024-03-08T23:53:31.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:31 smithi062 ceph-mon[36129]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:31.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:31 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1857706512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-08T23:53:31.865 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:31 smithi022 ceph-mon[31894]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:31.865 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:31 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1857706512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-08T23:53:31.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:31 smithi192 ceph-mon[35995]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:31.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:31 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1857706512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-08T23:53:32.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332149 for osd.7 2024-03-08T23:53:32.314 DEBUG:teuthology.parallel:result is None 2024-03-08T23:53:32.314 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-08T23:53:32.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-08T23:53:32.566 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:33.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:33 smithi062 ceph-mon[36129]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:33.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:33 smithi192 ceph-mon[35995]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:33.937 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:33 smithi022 ceph-mon[31894]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 -- 172.21.15.22:0/2846076748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 msgr2=0x7f74f410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/2846076748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f410ec30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00b0d0 tx=0x7f74ec00b3e0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 -- 172.21.15.22:0/2846076748 shutdown_connections 2024-03-08T23:53:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/2846076748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f410ec30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/2846076748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f4107e10 0x7f74f41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/2846076748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74f4101e30 0x7f74f41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 -- 172.21.15.22:0/2846076748 >> 172.21.15.22:0/2846076748 conn(0x7f74f4078f10 msgr2=0x7f74f4079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 -- 172.21.15.22:0/2846076748 shutdown_connections 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 -- 172.21.15.22:0/2846076748 wait complete. 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 Processor -- start 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.321+0000 7f74fa0a5700 1 -- start start 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74fa0a5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74f4101e30 0x7f74f4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74fa0a5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74fa0a5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f4107e10 0x7f74f419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74fa0a5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4112110 con 0x7f74f41027f0 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74fa0a5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4111f90 con 0x7f74f4107e10 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74fa0a5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4112290 con 0x7f74f4101e30 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:34.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f4198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60298/0 (socket says 172.21.15.22:60298) 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 -- 172.21.15.22:0/256602417 learned_addr learned my addr 172.21.15.22:0/256602417 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f3fff700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f4107e10 0x7f74f419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f37fe700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74f4101e30 0x7f74f4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 -- 172.21.15.22:0/256602417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74f4101e30 msgr2=0x7f74f4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74f4101e30 0x7f74f4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 -- 172.21.15.22:0/256602417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f4107e10 msgr2=0x7f74f419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f4107e10 0x7f74f419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 -- 172.21.15.22:0/256602417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f419d450 con 0x7f74f41027f0 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f37fe700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74f4101e30 0x7f74f4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f2ffd700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f4198980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b530 tx=0x7f74e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.322+0000 7f74f0ff9700 1 -- 172.21.15.22:0/256602417 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8015020 con 0x7f74f41027f0 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.323+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f419d740 con 0x7f74f41027f0 2024-03-08T23:53:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.323+0000 7f74f0ff9700 1 -- 172.21.15.22:0/256602417 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e8011d10 con 0x7f74f41027f0 2024-03-08T23:53:34.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.323+0000 7f74f0ff9700 1 -- 172.21.15.22:0/256602417 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800f2e0 con 0x7f74f41027f0 2024-03-08T23:53:34.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.323+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f41a4fc0 con 0x7f74f41027f0 2024-03-08T23:53:34.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.324+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4068aa0 con 0x7f74f41027f0 2024-03-08T23:53:34.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.325+0000 7f74f0ff9700 1 -- 172.21.15.22:0/256602417 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f74e8018020 con 0x7f74f41027f0 2024-03-08T23:53:34.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.325+0000 7f74f0ff9700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f74dc0a2920 0x7f74dc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:34.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.326+0000 7f74f37fe700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f74dc0a2920 0x7f74dc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:34.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.326+0000 7f74f0ff9700 1 -- 172.21.15.22:0/256602417 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f74e80ce740 con 0x7f74f41027f0 2024-03-08T23:53:34.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.326+0000 7f74f37fe700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f74dc0a2920 0x7f74dc0a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:34.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.330+0000 7f74f0ff9700 1 -- 172.21.15.22:0/256602417 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f74e801f020 con 0x7f74f41027f0 2024-03-08T23:53:34.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.530+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74f4100f20 con 0x7f74dc0a2920 2024-03-08T23:53:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.540+0000 7f74f0ff9700 1 -- 172.21.15.22:0/256602417 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340369 (secure 0 0 0) 0x7f74f4100f20 con 0x7f74dc0a2920 2024-03-08T23:53:34.540 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f74dc0a2920 msgr2=0x7f74dc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f74dc0a2920 0x7f74dc0a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e4005e30 comp rx=0 tx=0).stop 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 msgr2=0x7f74f4198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f4198980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b530 tx=0x7f74e800b840 comp rx=0 tx=0).stop 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 shutdown_connections 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74f4107e10 0x7f74f419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74f4101e30 0x7f74f4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f74dc0a2920 0x7f74dc0a4de0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 --2- 172.21.15.22:0/256602417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74f41027f0 0x7f74f4198980 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 >> 172.21.15.22:0/256602417 conn(0x7f74f4078f10 msgr2=0x7f74f4100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 shutdown_connections 2024-03-08T23:53:34.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:34.544+0000 7f74fa0a5700 1 -- 172.21.15.22:0/256602417 wait complete. 2024-03-08T23:53:34.544 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-08T23:53:35.283 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-03-08T23:53:33.813311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657327,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":587,"num_read_kb":411,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1241534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363316,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578508,"statfs":{"total":1151940427776,"available":1151568392192,"internally_reserved":0,"allocated":14434304,"data_stored":7957787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003345"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416389+0000","last_change":"2024-03-08T23:51:52.246270+0000","last_active":"2024-03-08T23:52:45.416389+0000","last_peered":"2024-03-08T23:52:45.416389+0000","last_clean":"2024-03-08T23:52:45.416389+0000","last_became_active":"2024-03-08T23:51:52.245321+0000","last_became_peered":"2024-03-08T23:51:52.245321+0000","last_unstale":"2024-03-08T23:52:45.416389+0000","last_undegraded":"2024-03-08T23:52:45.416389+0000","last_fullsized":"2024-03-08T23:52:45.416389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:39.993965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849289+0000","last_change":"2024-03-08T23:51:52.245433+0000","last_active":"2024-03-08T23:52:45.849289+0000","last_peered":"2024-03-08T23:52:45.849289+0000","last_clean":"2024-03-08T23:52:45.849289+0000","last_became_active":"2024-03-08T23:51:52.245106+0000","last_became_peered":"2024-03-08T23:51:52.245106+0000","last_unstale":"2024-03-08T23:52:45.849289+0000","last_undegraded":"2024-03-08T23:52:45.849289+0000","last_fullsized":"2024-03-08T23:52:45.849289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:02:20.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849269+0000","last_change":"2024-03-08T23:51:52.244112+0000","last_active":"2024-03-08T23:52:45.849269+0000","last_peered":"2024-03-08T23:52:45.849269+0000","last_clean":"2024-03-08T23:52:45.849269+0000","last_became_active":"2024-03-08T23:51:52.243783+0000","last_became_peered":"2024-03-08T23:51:52.243783+0000","last_unstale":"2024-03-08T23:52:45.849269+0000","last_undegraded":"2024-03-08T23:52:45.849269+0000","last_fullsized":"2024-03-08T23:52:45.849269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:48:43.719188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.586298+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:52:45.586298+0000","last_peered":"2024-03-08T23:52:45.586298+0000","last_clean":"2024-03-08T23:52:45.586298+0000","last_became_active":"2024-03-08T23:52:45.585292+0000","last_became_peered":"2024-03-08T23:52:45.585292+0000","last_unstale":"2024-03-08T23:52:45.586298+0000","last_undegraded":"2024-03-08T23:52:45.586298+0000","last_fullsized":"2024-03-08T23:52:45.586298+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:27:12.230920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584251+0000","last_change":"2024-03-08T23:51:52.247091+0000","last_active":"2024-03-08T23:52:45.584251+0000","last_peered":"2024-03-08T23:52:45.584251+0000","last_clean":"2024-03-08T23:52:45.584251+0000","last_became_active":"2024-03-08T23:51:52.246761+0000","last_became_peered":"2024-03-08T23:51:52.246761+0000","last_unstale":"2024-03-08T23:52:45.584251+0000","last_undegraded":"2024-03-08T23:52:45.584251+0000","last_fullsized":"2024-03-08T23:52:45.584251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:46:46.810857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411181+0000","last_change":"2024-03-08T23:51:52.257558+0000","last_active":"2024-03-08T23:52:45.411181+0000","last_peered":"2024-03-08T23:52:45.411181+0000","last_clean":"2024-03-08T23:52:45.411181+0000","last_became_active":"2024-03-08T23:51:52.257116+0000","last_became_peered":"2024-03-08T23:51:52.257116+0000","last_unstale":"2024-03-08T23:52:45.411181+0000","last_undegraded":"2024-03-08T23:52:45.411181+0000","last_fullsized":"2024-03-08T23:52:45.411181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:13:05.325780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411884+0000","last_change":"2024-03-08T23:51:52.253624+0000","last_active":"2024-03-08T23:52:45.411884+0000","last_peered":"2024-03-08T23:52:45.411884+0000","last_clean":"2024-03-08T23:52:45.411884+0000","last_became_active":"2024-03-08T23:51:52.253284+0000","last_became_peered":"2024-03-08T23:51:52.253284+0000","last_unstale":"2024-03-08T23:52:45.411884+0000","last_undegraded":"2024-03-08T23:52:45.411884+0000","last_fullsized":"2024-03-08T23:52:45.411884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:09:33.789502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410377+0000","last_change":"2024-03-08T23:51:52.255321+0000","last_active":"2024-03-08T23:52:45.410377+0000","last_peered":"2024-03-08T23:52:45.410377+0000","last_clean":"2024-03-08T23:52:45.410377+0000","last_became_active":"2024-03-08T23:51:52.254975+0000","last_became_peered":"2024-03-08T23:51:52.254975+0000","last_unstale":"2024-03-08T23:52:45.410377+0000","last_undegraded":"2024-03-08T23:52:45.410377+0000","last_fullsized":"2024-03-08T23:52:45.410377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:02:56.788519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414750+0000","last_change":"2024-03-08T23:51:52.244623+0000","last_active":"2024-03-08T23:52:45.414750+0000","last_peered":"2024-03-08T23:52:45.414750+0000","last_clean":"2024-03-08T23:52:45.414750+0000","last_became_active":"2024-03-08T23:51:52.244162+0000","last_became_peered":"2024-03-08T23:51:52.244162+0000","last_unstale":"2024-03-08T23:52:45.414750+0000","last_undegraded":"2024-03-08T23:52:45.414750+0000","last_fullsized":"2024-03-08T23:52:45.414750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:30:43.050616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411474+0000","last_change":"2024-03-08T23:51:52.258006+0000","last_active":"2024-03-08T23:52:45.411474+0000","last_peered":"2024-03-08T23:52:45.411474+0000","last_clean":"2024-03-08T23:52:45.411474+0000","last_became_active":"2024-03-08T23:51:52.256770+0000","last_became_peered":"2024-03-08T23:51:52.256770+0000","last_unstale":"2024-03-08T23:52:45.411474+0000","last_undegraded":"2024-03-08T23:52:45.411474+0000","last_fullsized":"2024-03-08T23:52:45.411474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:18.584212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412095+0000","last_change":"2024-03-08T23:51:52.260530+0000","last_active":"2024-03-08T23:52:45.412095+0000","last_peered":"2024-03-08T23:52:45.412095+0000","last_clean":"2024-03-08T23:52:45.412095+0000","last_became_active":"2024-03-08T23:51:52.260193+0000","last_became_peered":"2024-03-08T23:51:52.260193+0000","last_unstale":"2024-03-08T23:52:45.412095+0000","last_undegraded":"2024-03-08T23:52:45.412095+0000","last_fullsized":"2024-03-08T23:52:45.412095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:02.549683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409359+0000","last_change":"2024-03-08T23:51:52.248679+0000","last_active":"2024-03-08T23:52:45.409359+0000","last_peered":"2024-03-08T23:52:45.409359+0000","last_clean":"2024-03-08T23:52:45.409359+0000","last_became_active":"2024-03-08T23:51:52.248358+0000","last_became_peered":"2024-03-08T23:51:52.248358+0000","last_unstale":"2024-03-08T23:52:45.409359+0000","last_undegraded":"2024-03-08T23:52:45.409359+0000","last_fullsized":"2024-03-08T23:52:45.409359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:26.680887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415807+0000","last_change":"2024-03-08T23:52:45.415807+0000","last_active":"2024-03-08T23:52:45.415807+0000","last_peered":"2024-03-08T23:52:45.415807+0000","last_clean":"2024-03-08T23:52:45.415807+0000","last_became_active":"2024-03-08T23:52:45.415456+0000","last_became_peered":"2024-03-08T23:52:45.415456+0000","last_unstale":"2024-03-08T23:52:45.415807+0000","last_undegraded":"2024-03-08T23:52:45.415807+0000","last_fullsized":"2024-03-08T23:52:45.415807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:12:01.942744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.848919+0000","last_change":"2024-03-08T23:51:52.245394+0000","last_active":"2024-03-08T23:52:45.848919+0000","last_peered":"2024-03-08T23:52:45.848919+0000","last_clean":"2024-03-08T23:52:45.848919+0000","last_became_active":"2024-03-08T23:51:52.245038+0000","last_became_peered":"2024-03-08T23:51:52.245038+0000","last_unstale":"2024-03-08T23:52:45.848919+0000","last_undegraded":"2024-03-08T23:52:45.848919+0000","last_fullsized":"2024-03-08T23:52:45.848919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:15.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.848916+0000","last_change":"2024-03-08T23:51:52.250861+0000","last_active":"2024-03-08T23:52:45.848916+0000","last_peered":"2024-03-08T23:52:45.848916+0000","last_clean":"2024-03-08T23:52:45.848916+0000","last_became_active":"2024-03-08T23:51:52.250548+0000","last_became_peered":"2024-03-08T23:51:52.250548+0000","last_unstale":"2024-03-08T23:52:45.848916+0000","last_undegraded":"2024-03-08T23:52:45.848916+0000","last_fullsized":"2024-03-08T23:52:45.848916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:00.280043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414420+0000","last_change":"2024-03-08T23:51:52.254996+0000","last_active":"2024-03-08T23:52:45.414420+0000","last_peered":"2024-03-08T23:52:45.414420+0000","last_clean":"2024-03-08T23:52:45.414420+0000","last_became_active":"2024-03-08T23:51:52.254634+0000","last_became_peered":"2024-03-08T23:51:52.254634+0000","last_unstale":"2024-03-08T23:52:45.414420+0000","last_undegraded":"2024-03-08T23:52:45.414420+0000","last_fullsized":"2024-03-08T23:52:45.414420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:15.449988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416241+0000","last_change":"2024-03-08T23:51:52.246558+0000","last_active":"2024-03-08T23:52:45.416241+0000","last_peered":"2024-03-08T23:52:45.416241+0000","last_clean":"2024-03-08T23:52:45.416241+0000","last_became_active":"2024-03-08T23:51:52.245741+0000","last_became_peered":"2024-03-08T23:51:52.245741+0000","last_unstale":"2024-03-08T23:52:45.416241+0000","last_undegraded":"2024-03-08T23:52:45.416241+0000","last_fullsized":"2024-03-08T23:52:45.416241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:36.737017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408291+0000","last_change":"2024-03-08T23:51:52.240111+0000","last_active":"2024-03-08T23:52:45.408291+0000","last_peered":"2024-03-08T23:52:45.408291+0000","last_clean":"2024-03-08T23:52:45.408291+0000","last_became_active":"2024-03-08T23:51:52.238961+0000","last_became_peered":"2024-03-08T23:51:52.238961+0000","last_unstale":"2024-03-08T23:52:45.408291+0000","last_undegraded":"2024-03-08T23:52:45.408291+0000","last_fullsized":"2024-03-08T23:52:45.408291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:03:55.313117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412252+0000","last_change":"2024-03-08T23:51:52.245979+0000","last_active":"2024-03-08T23:52:45.412252+0000","last_peered":"2024-03-08T23:52:45.412252+0000","last_clean":"2024-03-08T23:52:45.412252+0000","last_became_active":"2024-03-08T23:51:52.245666+0000","last_became_peered":"2024-03-08T23:51:52.245666+0000","last_unstale":"2024-03-08T23:52:45.412252+0000","last_undegraded":"2024-03-08T23:52:45.412252+0000","last_fullsized":"2024-03-08T23:52:45.412252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:20.789144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412405+0000","last_change":"2024-03-08T23:51:56.288161+0000","last_active":"2024-03-08T23:52:45.412405+0000","last_peered":"2024-03-08T23:52:45.412405+0000","last_clean":"2024-03-08T23:52:45.412405+0000","last_became_active":"2024-03-08T23:51:56.287778+0000","last_became_peered":"2024-03-08T23:51:56.287778+0000","last_unstale":"2024-03-08T23:52:45.412405+0000","last_undegraded":"2024-03-08T23:52:45.412405+0000","last_fullsized":"2024-03-08T23:52:45.412405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:12:59.636228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408341+0000","last_change":"2024-03-08T23:51:51.244911+0000","last_active":"2024-03-08T23:52:45.408341+0000","last_peered":"2024-03-08T23:52:45.408341+0000","last_clean":"2024-03-08T23:52:45.408341+0000","last_became_active":"2024-03-08T23:51:51.244561+0000","last_became_peered":"2024-03-08T23:51:51.244561+0000","last_unstale":"2024-03-08T23:52:45.408341+0000","last_undegraded":"2024-03-08T23:52:45.408341+0000","last_fullsized":"2024-03-08T23:52:45.408341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:58:45.908307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411427+0000","last_change":"2024-03-08T23:51:52.253854+0000","last_active":"2024-03-08T23:52:45.411427+0000","last_peered":"2024-03-08T23:52:45.411427+0000","last_clean":"2024-03-08T23:52:45.411427+0000","last_became_active":"2024-03-08T23:51:52.253538+0000","last_became_peered":"2024-03-08T23:51:52.253538+0000","last_unstale":"2024-03-08T23:52:45.411427+0000","last_undegraded":"2024-03-08T23:52:45.411427+0000","last_fullsized":"2024-03-08T23:52:45.411427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:33.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408730+0000","last_change":"2024-03-08T23:51:56.283307+0000","last_active":"2024-03-08T23:52:45.408730+0000","last_peered":"2024-03-08T23:52:45.408730+0000","last_clean":"2024-03-08T23:52:45.408730+0000","last_became_active":"2024-03-08T23:51:56.282868+0000","last_became_peered":"2024-03-08T23:51:56.282868+0000","last_unstale":"2024-03-08T23:52:45.408730+0000","last_undegraded":"2024-03-08T23:52:45.408730+0000","last_fullsized":"2024-03-08T23:52:45.408730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:15.027743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411684+0000","last_change":"2024-03-08T23:51:51.247368+0000","last_active":"2024-03-08T23:52:45.411684+0000","last_peered":"2024-03-08T23:52:45.411684+0000","last_clean":"2024-03-08T23:52:45.411684+0000","last_became_active":"2024-03-08T23:51:51.247023+0000","last_became_peered":"2024-03-08T23:51:51.247023+0000","last_unstale":"2024-03-08T23:52:45.411684+0000","last_undegraded":"2024-03-08T23:52:45.411684+0000","last_fullsized":"2024-03-08T23:52:45.411684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:28.311850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849303+0000","last_change":"2024-03-08T23:51:52.245805+0000","last_active":"2024-03-08T23:52:45.849303+0000","last_peered":"2024-03-08T23:52:45.849303+0000","last_clean":"2024-03-08T23:52:45.849303+0000","last_became_active":"2024-03-08T23:51:52.244455+0000","last_became_peered":"2024-03-08T23:51:52.244455+0000","last_unstale":"2024-03-08T23:52:45.849303+0000","last_undegraded":"2024-03-08T23:52:45.849303+0000","last_fullsized":"2024-03-08T23:52:45.849303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:11:17.809148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408960+0000","last_change":"2024-03-08T23:51:56.286666+0000","last_active":"2024-03-08T23:52:45.408960+0000","last_peered":"2024-03-08T23:52:45.408960+0000","last_clean":"2024-03-08T23:52:45.408960+0000","last_became_active":"2024-03-08T23:51:56.285968+0000","last_became_peered":"2024-03-08T23:51:56.285968+0000","last_unstale":"2024-03-08T23:52:45.408960+0000","last_undegraded":"2024-03-08T23:52:45.408960+0000","last_fullsized":"2024-03-08T23:52:45.408960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:51.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584660+0000","last_change":"2024-03-08T23:51:51.247684+0000","last_active":"2024-03-08T23:52:45.584660+0000","last_peered":"2024-03-08T23:52:45.584660+0000","last_clean":"2024-03-08T23:52:45.584660+0000","last_became_active":"2024-03-08T23:51:51.247374+0000","last_became_peered":"2024-03-08T23:51:51.247374+0000","last_unstale":"2024-03-08T23:52:45.584660+0000","last_undegraded":"2024-03-08T23:52:45.584660+0000","last_fullsized":"2024-03-08T23:52:45.584660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:20.425302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413823+0000","last_change":"2024-03-08T23:51:52.248702+0000","last_active":"2024-03-08T23:52:45.413823+0000","last_peered":"2024-03-08T23:52:45.413823+0000","last_clean":"2024-03-08T23:52:45.413823+0000","last_became_active":"2024-03-08T23:51:52.248388+0000","last_became_peered":"2024-03-08T23:51:52.248388+0000","last_unstale":"2024-03-08T23:52:45.413823+0000","last_undegraded":"2024-03-08T23:52:45.413823+0000","last_fullsized":"2024-03-08T23:52:45.413823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:27:12.251957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408935+0000","last_change":"2024-03-08T23:51:56.286572+0000","last_active":"2024-03-08T23:52:45.408935+0000","last_peered":"2024-03-08T23:52:45.408935+0000","last_clean":"2024-03-08T23:52:45.408935+0000","last_became_active":"2024-03-08T23:51:56.286066+0000","last_became_peered":"2024-03-08T23:51:56.286066+0000","last_unstale":"2024-03-08T23:52:45.408935+0000","last_undegraded":"2024-03-08T23:52:45.408935+0000","last_fullsized":"2024-03-08T23:52:45.408935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:13.694615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584533+0000","last_change":"2024-03-08T23:51:51.244207+0000","last_active":"2024-03-08T23:52:45.584533+0000","last_peered":"2024-03-08T23:52:45.584533+0000","last_clean":"2024-03-08T23:52:45.584533+0000","last_became_active":"2024-03-08T23:51:51.241605+0000","last_became_peered":"2024-03-08T23:51:51.241605+0000","last_unstale":"2024-03-08T23:52:45.584533+0000","last_undegraded":"2024-03-08T23:52:45.584533+0000","last_fullsized":"2024-03-08T23:52:45.584533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:03:51.766681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415031+0000","last_change":"2024-03-08T23:51:52.251015+0000","last_active":"2024-03-08T23:52:45.415031+0000","last_peered":"2024-03-08T23:52:45.415031+0000","last_clean":"2024-03-08T23:52:45.415031+0000","last_became_active":"2024-03-08T23:51:52.250690+0000","last_became_peered":"2024-03-08T23:51:52.250690+0000","last_unstale":"2024-03-08T23:52:45.415031+0000","last_undegraded":"2024-03-08T23:52:45.415031+0000","last_fullsized":"2024-03-08T23:52:45.415031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:57:23.033172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412052+0000","last_change":"2024-03-08T23:51:56.281066+0000","last_active":"2024-03-08T23:52:45.412052+0000","last_peered":"2024-03-08T23:52:45.412052+0000","last_clean":"2024-03-08T23:52:45.412052+0000","last_became_active":"2024-03-08T23:51:56.280654+0000","last_became_peered":"2024-03-08T23:51:56.280654+0000","last_unstale":"2024-03-08T23:52:45.412052+0000","last_undegraded":"2024-03-08T23:52:45.412052+0000","last_fullsized":"2024-03-08T23:52:45.412052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:53.999115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:31.456562+0000","last_change":"2024-03-08T23:51:51.245947+0000","last_active":"2024-03-08T23:53:31.456562+0000","last_peered":"2024-03-08T23:53:31.456562+0000","last_clean":"2024-03-08T23:53:31.456562+0000","last_became_active":"2024-03-08T23:51:51.245505+0000","last_became_peered":"2024-03-08T23:51:51.245505+0000","last_unstale":"2024-03-08T23:53:31.456562+0000","last_undegraded":"2024-03-08T23:53:31.456562+0000","last_fullsized":"2024-03-08T23:53:31.456562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:20:44.692598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584041+0000","last_change":"2024-03-08T23:51:52.241590+0000","last_active":"2024-03-08T23:52:45.584041+0000","last_peered":"2024-03-08T23:52:45.584041+0000","last_clean":"2024-03-08T23:52:45.584041+0000","last_became_active":"2024-03-08T23:51:52.241296+0000","last_became_peered":"2024-03-08T23:51:52.241296+0000","last_unstale":"2024-03-08T23:52:45.584041+0000","last_undegraded":"2024-03-08T23:52:45.584041+0000","last_fullsized":"2024-03-08T23:52:45.584041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:52:48.042360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.585946+0000","last_change":"2024-03-08T23:52:45.585946+0000","last_active":"2024-03-08T23:52:45.585946+0000","last_peered":"2024-03-08T23:52:45.585946+0000","last_clean":"2024-03-08T23:52:45.585946+0000","last_became_active":"2024-03-08T23:52:45.585470+0000","last_became_peered":"2024-03-08T23:52:45.585470+0000","last_unstale":"2024-03-08T23:52:45.585946+0000","last_undegraded":"2024-03-08T23:52:45.585946+0000","last_fullsized":"2024-03-08T23:52:45.585946+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:28:36.264887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411059+0000","last_change":"2024-03-08T23:51:51.241384+0000","last_active":"2024-03-08T23:52:45.411059+0000","last_peered":"2024-03-08T23:52:45.411059+0000","last_clean":"2024-03-08T23:52:45.411059+0000","last_became_active":"2024-03-08T23:51:51.241018+0000","last_became_peered":"2024-03-08T23:51:51.241018+0000","last_unstale":"2024-03-08T23:52:45.411059+0000","last_undegraded":"2024-03-08T23:52:45.411059+0000","last_fullsized":"2024-03-08T23:52:45.411059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:29:50.860663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415653+0000","last_change":"2024-03-08T23:51:52.250207+0000","last_active":"2024-03-08T23:52:45.415653+0000","last_peered":"2024-03-08T23:52:45.415653+0000","last_clean":"2024-03-08T23:52:45.415653+0000","last_became_active":"2024-03-08T23:51:52.249863+0000","last_became_peered":"2024-03-08T23:51:52.249863+0000","last_unstale":"2024-03-08T23:52:45.415653+0000","last_undegraded":"2024-03-08T23:52:45.415653+0000","last_fullsized":"2024-03-08T23:52:45.415653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:18:48.732419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416096+0000","last_change":"2024-03-08T23:51:56.282092+0000","last_active":"2024-03-08T23:52:45.416096+0000","last_peered":"2024-03-08T23:52:45.416096+0000","last_clean":"2024-03-08T23:52:45.416096+0000","last_became_active":"2024-03-08T23:51:56.281657+0000","last_became_peered":"2024-03-08T23:51:56.281657+0000","last_unstale":"2024-03-08T23:52:45.416096+0000","last_undegraded":"2024-03-08T23:52:45.416096+0000","last_fullsized":"2024-03-08T23:52:45.416096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:50.440520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410588+0000","last_change":"2024-03-08T23:51:51.236816+0000","last_active":"2024-03-08T23:52:45.410588+0000","last_peered":"2024-03-08T23:52:45.410588+0000","last_clean":"2024-03-08T23:52:45.410588+0000","last_became_active":"2024-03-08T23:51:51.236485+0000","last_became_peered":"2024-03-08T23:51:51.236485+0000","last_unstale":"2024-03-08T23:52:45.410588+0000","last_undegraded":"2024-03-08T23:52:45.410588+0000","last_fullsized":"2024-03-08T23:52:45.410588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:27.300950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410618+0000","last_change":"2024-03-08T23:51:52.252146+0000","last_active":"2024-03-08T23:52:45.410618+0000","last_peered":"2024-03-08T23:52:45.410618+0000","last_clean":"2024-03-08T23:52:45.410618+0000","last_became_active":"2024-03-08T23:51:52.244201+0000","last_became_peered":"2024-03-08T23:51:52.244201+0000","last_unstale":"2024-03-08T23:52:45.410618+0000","last_undegraded":"2024-03-08T23:52:45.410618+0000","last_fullsized":"2024-03-08T23:52:45.410618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:07.647078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849262+0000","last_change":"2024-03-08T23:51:56.279727+0000","last_active":"2024-03-08T23:52:45.849262+0000","last_peered":"2024-03-08T23:52:45.849262+0000","last_clean":"2024-03-08T23:52:45.849262+0000","last_became_active":"2024-03-08T23:51:56.279184+0000","last_became_peered":"2024-03-08T23:51:56.279184+0000","last_unstale":"2024-03-08T23:52:45.849262+0000","last_undegraded":"2024-03-08T23:52:45.849262+0000","last_fullsized":"2024-03-08T23:52:45.849262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:21.804410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414779+0000","last_change":"2024-03-08T23:51:51.244812+0000","last_active":"2024-03-08T23:52:45.414779+0000","last_peered":"2024-03-08T23:52:45.414779+0000","last_clean":"2024-03-08T23:52:45.414779+0000","last_became_active":"2024-03-08T23:51:51.244513+0000","last_became_peered":"2024-03-08T23:51:51.244513+0000","last_unstale":"2024-03-08T23:52:45.414779+0000","last_undegraded":"2024-03-08T23:52:45.414779+0000","last_fullsized":"2024-03-08T23:52:45.414779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:56:07.235214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.407857+0000","last_change":"2024-03-08T23:51:52.244566+0000","last_active":"2024-03-08T23:52:45.407857+0000","last_peered":"2024-03-08T23:52:45.407857+0000","last_clean":"2024-03-08T23:52:45.407857+0000","last_became_active":"2024-03-08T23:51:52.244250+0000","last_became_peered":"2024-03-08T23:51:52.244250+0000","last_unstale":"2024-03-08T23:52:45.407857+0000","last_undegraded":"2024-03-08T23:52:45.407857+0000","last_fullsized":"2024-03-08T23:52:45.407857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:55:50.596364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410766+0000","last_change":"2024-03-08T23:51:56.282487+0000","last_active":"2024-03-08T23:52:45.410766+0000","last_peered":"2024-03-08T23:52:45.410766+0000","last_clean":"2024-03-08T23:52:45.410766+0000","last_became_active":"2024-03-08T23:51:56.282104+0000","last_became_peered":"2024-03-08T23:51:56.282104+0000","last_unstale":"2024-03-08T23:52:45.410766+0000","last_undegraded":"2024-03-08T23:52:45.410766+0000","last_fullsized":"2024-03-08T23:52:45.410766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:23:23.787431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:27.449886+0000","last_change":"2024-03-08T23:51:51.236612+0000","last_active":"2024-03-08T23:53:27.449886+0000","last_peered":"2024-03-08T23:53:27.449886+0000","last_clean":"2024-03-08T23:53:27.449886+0000","last_became_active":"2024-03-08T23:51:51.236274+0000","last_became_peered":"2024-03-08T23:51:51.236274+0000","last_unstale":"2024-03-08T23:53:27.449886+0000","last_undegraded":"2024-03-08T23:53:27.449886+0000","last_fullsized":"2024-03-08T23:53:27.449886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:37.551870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414901+0000","last_change":"2024-03-08T23:51:52.244564+0000","last_active":"2024-03-08T23:52:45.414901+0000","last_peered":"2024-03-08T23:52:45.414901+0000","last_clean":"2024-03-08T23:52:45.414901+0000","last_became_active":"2024-03-08T23:51:52.243651+0000","last_became_peered":"2024-03-08T23:51:52.243651+0000","last_unstale":"2024-03-08T23:52:45.414901+0000","last_undegraded":"2024-03-08T23:52:45.414901+0000","last_fullsized":"2024-03-08T23:52:45.414901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:35:32.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.407484+0000","last_change":"2024-03-08T23:51:56.279354+0000","last_active":"2024-03-08T23:52:45.407484+0000","last_peered":"2024-03-08T23:52:45.407484+0000","last_clean":"2024-03-08T23:52:45.407484+0000","last_became_active":"2024-03-08T23:51:56.278894+0000","last_became_peered":"2024-03-08T23:51:56.278894+0000","last_unstale":"2024-03-08T23:52:45.407484+0000","last_undegraded":"2024-03-08T23:52:45.407484+0000","last_fullsized":"2024-03-08T23:52:45.407484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:20:30.164542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411582+0000","last_change":"2024-03-08T23:51:51.246417+0000","last_active":"2024-03-08T23:52:45.411582+0000","last_peered":"2024-03-08T23:52:45.411582+0000","last_clean":"2024-03-08T23:52:45.411582+0000","last_became_active":"2024-03-08T23:51:51.246058+0000","last_became_peered":"2024-03-08T23:51:51.246058+0000","last_unstale":"2024-03-08T23:52:45.411582+0000","last_undegraded":"2024-03-08T23:52:45.411582+0000","last_fullsized":"2024-03-08T23:52:45.411582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:48:53.569295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849326+0000","last_change":"2024-03-08T23:51:52.255357+0000","last_active":"2024-03-08T23:52:45.849326+0000","last_peered":"2024-03-08T23:52:45.849326+0000","last_clean":"2024-03-08T23:52:45.849326+0000","last_became_active":"2024-03-08T23:51:52.255044+0000","last_became_peered":"2024-03-08T23:51:52.255044+0000","last_unstale":"2024-03-08T23:52:45.849326+0000","last_undegraded":"2024-03-08T23:52:45.849326+0000","last_fullsized":"2024-03-08T23:52:45.849326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:13:40.710283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409241+0000","last_change":"2024-03-08T23:51:56.284282+0000","last_active":"2024-03-08T23:52:45.409241+0000","last_peered":"2024-03-08T23:52:45.409241+0000","last_clean":"2024-03-08T23:52:45.409241+0000","last_became_active":"2024-03-08T23:51:56.283863+0000","last_became_peered":"2024-03-08T23:51:56.283863+0000","last_unstale":"2024-03-08T23:52:45.409241+0000","last_undegraded":"2024-03-08T23:52:45.409241+0000","last_fullsized":"2024-03-08T23:52:45.409241+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:59:21.972283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414320+0000","last_change":"2024-03-08T23:51:51.245199+0000","last_active":"2024-03-08T23:52:45.414320+0000","last_peered":"2024-03-08T23:52:45.414320+0000","last_clean":"2024-03-08T23:52:45.414320+0000","last_became_active":"2024-03-08T23:51:51.244906+0000","last_became_peered":"2024-03-08T23:51:51.244906+0000","last_unstale":"2024-03-08T23:52:45.414320+0000","last_undegraded":"2024-03-08T23:52:45.414320+0000","last_fullsized":"2024-03-08T23:52:45.414320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:54:39.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413987+0000","last_change":"2024-03-08T23:51:52.251209+0000","last_active":"2024-03-08T23:52:45.413987+0000","last_peered":"2024-03-08T23:52:45.413987+0000","last_clean":"2024-03-08T23:52:45.413987+0000","last_became_active":"2024-03-08T23:51:52.250392+0000","last_became_peered":"2024-03-08T23:51:52.250392+0000","last_unstale":"2024-03-08T23:52:45.413987+0000","last_undegraded":"2024-03-08T23:52:45.413987+0000","last_fullsized":"2024-03-08T23:52:45.413987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:56:17.811505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.586125+0000","last_change":"2024-03-08T23:52:45.586125+0000","last_active":"2024-03-08T23:52:45.586125+0000","last_peered":"2024-03-08T23:52:45.586125+0000","last_clean":"2024-03-08T23:52:45.586125+0000","last_became_active":"2024-03-08T23:52:45.585578+0000","last_became_peered":"2024-03-08T23:52:45.585578+0000","last_unstale":"2024-03-08T23:52:45.586125+0000","last_undegraded":"2024-03-08T23:52:45.586125+0000","last_fullsized":"2024-03-08T23:52:45.586125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:51:08.159460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":229,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:28.481502+0000","last_change":"2024-03-08T23:52:45.426691+0000","last_active":"2024-03-08T23:53:28.481502+0000","last_peered":"2024-03-08T23:53:28.481502+0000","last_clean":"2024-03-08T23:53:28.481502+0000","last_became_active":"2024-03-08T23:52:45.415742+0000","last_became_peered":"2024-03-08T23:52:45.415742+0000","last_unstale":"2024-03-08T23:53:28.481502+0000","last_undegraded":"2024-03-08T23:53:28.481502+0000","last_fullsized":"2024-03-08T23:53:28.481502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:05.389164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62333,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":138,"num_read_kb":129,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60681,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584525+0000","last_change":"2024-03-08T23:51:52.248715+0000","last_active":"2024-03-08T23:52:45.584525+0000","last_peered":"2024-03-08T23:52:45.584525+0000","last_clean":"2024-03-08T23:52:45.584525+0000","last_became_active":"2024-03-08T23:51:52.248258+0000","last_became_peered":"2024-03-08T23:51:52.248258+0000","last_unstale":"2024-03-08T23:52:45.584525+0000","last_undegraded":"2024-03-08T23:52:45.584525+0000","last_fullsized":"2024-03-08T23:52:45.584525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:38.185469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414980+0000","last_change":"2024-03-08T23:51:56.278268+0000","last_active":"2024-03-08T23:52:45.414980+0000","last_peered":"2024-03-08T23:52:45.414980+0000","last_clean":"2024-03-08T23:52:45.414980+0000","last_became_active":"2024-03-08T23:51:56.277831+0000","last_became_peered":"2024-03-08T23:51:56.277831+0000","last_unstale":"2024-03-08T23:52:45.414980+0000","last_undegraded":"2024-03-08T23:52:45.414980+0000","last_fullsized":"2024-03-08T23:52:45.414980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:32:13.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412137+0000","last_change":"2024-03-08T23:51:51.247553+0000","last_active":"2024-03-08T23:52:45.412137+0000","last_peered":"2024-03-08T23:52:45.412137+0000","last_clean":"2024-03-08T23:52:45.412137+0000","last_became_active":"2024-03-08T23:51:51.247225+0000","last_became_peered":"2024-03-08T23:51:51.247225+0000","last_unstale":"2024-03-08T23:52:45.412137+0000","last_undegraded":"2024-03-08T23:52:45.412137+0000","last_fullsized":"2024-03-08T23:52:45.412137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:53:29.846303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411212+0000","last_change":"2024-03-08T23:51:52.247733+0000","last_active":"2024-03-08T23:52:45.411212+0000","last_peered":"2024-03-08T23:52:45.411212+0000","last_clean":"2024-03-08T23:52:45.411212+0000","last_became_active":"2024-03-08T23:51:52.247435+0000","last_became_peered":"2024-03-08T23:51:52.247435+0000","last_unstale":"2024-03-08T23:52:45.411212+0000","last_undegraded":"2024-03-08T23:52:45.411212+0000","last_fullsized":"2024-03-08T23:52:45.411212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:46:12.739601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415996+0000","last_change":"2024-03-08T23:51:56.279865+0000","last_active":"2024-03-08T23:52:45.415996+0000","last_peered":"2024-03-08T23:52:45.415996+0000","last_clean":"2024-03-08T23:52:45.415996+0000","last_became_active":"2024-03-08T23:51:56.279446+0000","last_became_peered":"2024-03-08T23:51:56.279446+0000","last_unstale":"2024-03-08T23:52:45.415996+0000","last_undegraded":"2024-03-08T23:52:45.415996+0000","last_fullsized":"2024-03-08T23:52:45.415996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:07:33.672490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415220+0000","last_change":"2024-03-08T23:51:51.246860+0000","last_active":"2024-03-08T23:52:45.415220+0000","last_peered":"2024-03-08T23:52:45.415220+0000","last_clean":"2024-03-08T23:52:45.415220+0000","last_became_active":"2024-03-08T23:51:51.246545+0000","last_became_peered":"2024-03-08T23:51:51.246545+0000","last_unstale":"2024-03-08T23:52:45.415220+0000","last_undegraded":"2024-03-08T23:52:45.415220+0000","last_fullsized":"2024-03-08T23:52:45.415220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:11:07.246657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411593+0000","last_change":"2024-03-08T23:51:52.247353+0000","last_active":"2024-03-08T23:52:45.411593+0000","last_peered":"2024-03-08T23:52:45.411593+0000","last_clean":"2024-03-08T23:52:45.411593+0000","last_became_active":"2024-03-08T23:51:52.247008+0000","last_became_peered":"2024-03-08T23:51:52.247008+0000","last_unstale":"2024-03-08T23:52:45.411593+0000","last_undegraded":"2024-03-08T23:52:45.411593+0000","last_fullsized":"2024-03-08T23:52:45.411593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:08:50.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408112+0000","last_change":"2024-03-08T23:51:56.284103+0000","last_active":"2024-03-08T23:52:45.408112+0000","last_peered":"2024-03-08T23:52:45.408112+0000","last_clean":"2024-03-08T23:52:45.408112+0000","last_became_active":"2024-03-08T23:51:56.283561+0000","last_became_peered":"2024-03-08T23:51:56.283561+0000","last_unstale":"2024-03-08T23:52:45.408112+0000","last_undegraded":"2024-03-08T23:52:45.408112+0000","last_fullsized":"2024-03-08T23:52:45.408112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:47:20.909292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:31.456019+0000","last_change":"2024-03-08T23:51:51.242483+0000","last_active":"2024-03-08T23:53:31.456019+0000","last_peered":"2024-03-08T23:53:31.456019+0000","last_clean":"2024-03-08T23:53:31.456019+0000","last_became_active":"2024-03-08T23:51:51.242153+0000","last_became_peered":"2024-03-08T23:51:51.242153+0000","last_unstale":"2024-03-08T23:53:31.456019+0000","last_undegraded":"2024-03-08T23:53:31.456019+0000","last_fullsized":"2024-03-08T23:53:31.456019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:44:07.728700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411150+0000","last_change":"2024-03-08T23:51:52.255411+0000","last_active":"2024-03-08T23:52:45.411150+0000","last_peered":"2024-03-08T23:52:45.411150+0000","last_clean":"2024-03-08T23:52:45.411150+0000","last_became_active":"2024-03-08T23:51:52.255048+0000","last_became_peered":"2024-03-08T23:51:52.255048+0000","last_unstale":"2024-03-08T23:52:45.411150+0000","last_undegraded":"2024-03-08T23:52:45.411150+0000","last_fullsized":"2024-03-08T23:52:45.411150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:11.836187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411760+0000","last_change":"2024-03-08T23:51:56.286405+0000","last_active":"2024-03-08T23:52:45.411760+0000","last_peered":"2024-03-08T23:52:45.411760+0000","last_clean":"2024-03-08T23:52:45.411760+0000","last_became_active":"2024-03-08T23:51:56.285996+0000","last_became_peered":"2024-03-08T23:51:56.285996+0000","last_unstale":"2024-03-08T23:52:45.411760+0000","last_undegraded":"2024-03-08T23:52:45.411760+0000","last_fullsized":"2024-03-08T23:52:45.411760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:48:30.928436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584032+0000","last_change":"2024-03-08T23:51:51.240146+0000","last_active":"2024-03-08T23:52:45.584032+0000","last_peered":"2024-03-08T23:52:45.584032+0000","last_clean":"2024-03-08T23:52:45.584032+0000","last_became_active":"2024-03-08T23:51:51.239795+0000","last_became_peered":"2024-03-08T23:51:51.239795+0000","last_unstale":"2024-03-08T23:52:45.584032+0000","last_undegraded":"2024-03-08T23:52:45.584032+0000","last_fullsized":"2024-03-08T23:52:45.584032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:55.409922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414563+0000","last_change":"2024-03-08T23:51:52.254701+0000","last_active":"2024-03-08T23:52:45.414563+0000","last_peered":"2024-03-08T23:52:45.414563+0000","last_clean":"2024-03-08T23:52:45.414563+0000","last_became_active":"2024-03-08T23:51:52.254391+0000","last_became_peered":"2024-03-08T23:51:52.254391+0000","last_unstale":"2024-03-08T23:52:45.414563+0000","last_undegraded":"2024-03-08T23:52:45.414563+0000","last_fullsized":"2024-03-08T23:52:45.414563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:11:12.258181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413984+0000","last_change":"2024-03-08T23:51:56.277878+0000","last_active":"2024-03-08T23:52:45.413984+0000","last_peered":"2024-03-08T23:52:45.413984+0000","last_clean":"2024-03-08T23:52:45.413984+0000","last_became_active":"2024-03-08T23:51:56.277493+0000","last_became_peered":"2024-03-08T23:51:56.277493+0000","last_unstale":"2024-03-08T23:52:45.413984+0000","last_undegraded":"2024-03-08T23:52:45.413984+0000","last_fullsized":"2024-03-08T23:52:45.413984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:44:16.968632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584348+0000","last_change":"2024-03-08T23:51:51.241457+0000","last_active":"2024-03-08T23:52:45.584348+0000","last_peered":"2024-03-08T23:52:45.584348+0000","last_clean":"2024-03-08T23:52:45.584348+0000","last_became_active":"2024-03-08T23:51:51.241148+0000","last_became_peered":"2024-03-08T23:51:51.241148+0000","last_unstale":"2024-03-08T23:52:45.584348+0000","last_undegraded":"2024-03-08T23:52:45.584348+0000","last_fullsized":"2024-03-08T23:52:45.584348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:13:27.246265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416189+0000","last_change":"2024-03-08T23:51:52.248465+0000","last_active":"2024-03-08T23:52:45.416189+0000","last_peered":"2024-03-08T23:52:45.416189+0000","last_clean":"2024-03-08T23:52:45.416189+0000","last_became_active":"2024-03-08T23:51:52.248135+0000","last_became_peered":"2024-03-08T23:51:52.248135+0000","last_unstale":"2024-03-08T23:52:45.416189+0000","last_undegraded":"2024-03-08T23:52:45.416189+0000","last_fullsized":"2024-03-08T23:52:45.416189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:55:53.712958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584125+0000","last_change":"2024-03-08T23:51:56.276019+0000","last_active":"2024-03-08T23:52:45.584125+0000","last_peered":"2024-03-08T23:52:45.584125+0000","last_clean":"2024-03-08T23:52:45.584125+0000","last_became_active":"2024-03-08T23:51:56.275564+0000","last_became_peered":"2024-03-08T23:51:56.275564+0000","last_unstale":"2024-03-08T23:52:45.584125+0000","last_undegraded":"2024-03-08T23:52:45.584125+0000","last_fullsized":"2024-03-08T23:52:45.584125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:12:36.096470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.407907+0000","last_change":"2024-03-08T23:51:51.241629+0000","last_active":"2024-03-08T23:52:45.407907+0000","last_peered":"2024-03-08T23:52:45.407907+0000","last_clean":"2024-03-08T23:52:45.407907+0000","last_became_active":"2024-03-08T23:51:51.241300+0000","last_became_peered":"2024-03-08T23:51:51.241300+0000","last_unstale":"2024-03-08T23:52:45.407907+0000","last_undegraded":"2024-03-08T23:52:45.407907+0000","last_fullsized":"2024-03-08T23:52:45.407907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:01:45.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411166+0000","last_change":"2024-03-08T23:51:52.246024+0000","last_active":"2024-03-08T23:52:45.411166+0000","last_peered":"2024-03-08T23:52:45.411166+0000","last_clean":"2024-03-08T23:52:45.411166+0000","last_became_active":"2024-03-08T23:51:52.245706+0000","last_became_peered":"2024-03-08T23:51:52.245706+0000","last_unstale":"2024-03-08T23:52:45.411166+0000","last_undegraded":"2024-03-08T23:52:45.411166+0000","last_fullsized":"2024-03-08T23:52:45.411166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:00:44.557269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411212+0000","last_change":"2024-03-08T23:51:56.285649+0000","last_active":"2024-03-08T23:52:45.411212+0000","last_peered":"2024-03-08T23:52:45.411212+0000","last_clean":"2024-03-08T23:52:45.411212+0000","last_became_active":"2024-03-08T23:51:56.282731+0000","last_became_peered":"2024-03-08T23:51:56.282731+0000","last_unstale":"2024-03-08T23:52:45.411212+0000","last_undegraded":"2024-03-08T23:52:45.411212+0000","last_fullsized":"2024-03-08T23:52:45.411212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:23:54.103637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411148+0000","last_change":"2024-03-08T23:51:51.249442+0000","last_active":"2024-03-08T23:52:45.411148+0000","last_peered":"2024-03-08T23:52:45.411148+0000","last_clean":"2024-03-08T23:52:45.411148+0000","last_became_active":"2024-03-08T23:51:51.249084+0000","last_became_peered":"2024-03-08T23:51:51.249084+0000","last_unstale":"2024-03-08T23:52:45.411148+0000","last_undegraded":"2024-03-08T23:52:45.411148+0000","last_fullsized":"2024-03-08T23:52:45.411148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:48:19.370095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410713+0000","last_change":"2024-03-08T23:51:52.257586+0000","last_active":"2024-03-08T23:52:45.410713+0000","last_peered":"2024-03-08T23:52:45.410713+0000","last_clean":"2024-03-08T23:52:45.410713+0000","last_became_active":"2024-03-08T23:51:52.253741+0000","last_became_peered":"2024-03-08T23:51:52.253741+0000","last_unstale":"2024-03-08T23:52:45.410713+0000","last_undegraded":"2024-03-08T23:52:45.410713+0000","last_fullsized":"2024-03-08T23:52:45.410713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:18.633166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412107+0000","last_change":"2024-03-08T23:51:56.285264+0000","last_active":"2024-03-08T23:52:45.412107+0000","last_peered":"2024-03-08T23:52:45.412107+0000","last_clean":"2024-03-08T23:52:45.412107+0000","last_became_active":"2024-03-08T23:51:56.284727+0000","last_became_peered":"2024-03-08T23:51:56.284727+0000","last_unstale":"2024-03-08T23:52:45.412107+0000","last_undegraded":"2024-03-08T23:52:45.412107+0000","last_fullsized":"2024-03-08T23:52:45.412107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:01:20.723010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414290+0000","last_change":"2024-03-08T23:51:52.251461+0000","last_active":"2024-03-08T23:52:45.414290+0000","last_peered":"2024-03-08T23:52:45.414290+0000","last_clean":"2024-03-08T23:52:45.414290+0000","last_became_active":"2024-03-08T23:51:52.250996+0000","last_became_peered":"2024-03-08T23:51:52.250996+0000","last_unstale":"2024-03-08T23:52:45.414290+0000","last_undegraded":"2024-03-08T23:52:45.414290+0000","last_fullsized":"2024-03-08T23:52:45.414290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:51.719142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:28.450719+0000","last_change":"2024-03-08T23:51:51.245698+0000","last_active":"2024-03-08T23:53:28.450719+0000","last_peered":"2024-03-08T23:53:28.450719+0000","last_clean":"2024-03-08T23:53:28.450719+0000","last_became_active":"2024-03-08T23:51:51.245389+0000","last_became_peered":"2024-03-08T23:51:51.245389+0000","last_unstale":"2024-03-08T23:53:28.450719+0000","last_undegraded":"2024-03-08T23:53:28.450719+0000","last_fullsized":"2024-03-08T23:53:28.450719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:14:08.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.423657+0000","last_change":"2024-03-08T23:52:45.423657+0000","last_active":"2024-03-08T23:52:45.423657+0000","last_peered":"2024-03-08T23:52:45.423657+0000","last_clean":"2024-03-08T23:52:45.423657+0000","last_became_active":"2024-03-08T23:52:45.417868+0000","last_became_peered":"2024-03-08T23:52:45.417868+0000","last_unstale":"2024-03-08T23:52:45.423657+0000","last_undegraded":"2024-03-08T23:52:45.423657+0000","last_fullsized":"2024-03-08T23:52:45.423657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:34.178938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411550+0000","last_change":"2024-03-08T23:51:52.256518+0000","last_active":"2024-03-08T23:52:45.411550+0000","last_peered":"2024-03-08T23:52:45.411550+0000","last_clean":"2024-03-08T23:52:45.411550+0000","last_became_active":"2024-03-08T23:51:52.253481+0000","last_became_peered":"2024-03-08T23:51:52.253481+0000","last_unstale":"2024-03-08T23:52:45.411550+0000","last_undegraded":"2024-03-08T23:52:45.411550+0000","last_fullsized":"2024-03-08T23:52:45.411550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:35.287408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409139+0000","last_change":"2024-03-08T23:51:51.237921+0000","last_active":"2024-03-08T23:52:45.409139+0000","last_peered":"2024-03-08T23:52:45.409139+0000","last_clean":"2024-03-08T23:52:45.409139+0000","last_became_active":"2024-03-08T23:51:51.237568+0000","last_became_peered":"2024-03-08T23:51:51.237568+0000","last_unstale":"2024-03-08T23:52:45.409139+0000","last_undegraded":"2024-03-08T23:52:45.409139+0000","last_fullsized":"2024-03-08T23:52:45.409139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:11.256814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411073+0000","last_change":"2024-03-08T23:50:41.802121+0000","last_active":"2024-03-08T23:52:45.411073+0000","last_peered":"2024-03-08T23:52:45.411073+0000","last_clean":"2024-03-08T23:52:45.411073+0000","last_became_active":"2024-03-08T23:50:41.801678+0000","last_became_peered":"2024-03-08T23:50:41.801678+0000","last_unstale":"2024-03-08T23:52:45.411073+0000","last_undegraded":"2024-03-08T23:52:45.411073+0000","last_fullsized":"2024-03-08T23:52:45.411073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_clean_scrub_stamp":"2024-03-08T23:47:12.382547+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:15.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414198+0000","last_change":"2024-03-08T23:51:56.278465+0000","last_active":"2024-03-08T23:52:45.414198+0000","last_peered":"2024-03-08T23:52:45.414198+0000","last_clean":"2024-03-08T23:52:45.414198+0000","last_became_active":"2024-03-08T23:51:56.278079+0000","last_became_peered":"2024-03-08T23:51:56.278079+0000","last_unstale":"2024-03-08T23:52:45.414198+0000","last_undegraded":"2024-03-08T23:52:45.414198+0000","last_fullsized":"2024-03-08T23:52:45.414198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:47.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410331+0000","last_change":"2024-03-08T23:51:52.251895+0000","last_active":"2024-03-08T23:52:45.410331+0000","last_peered":"2024-03-08T23:52:45.410331+0000","last_clean":"2024-03-08T23:52:45.410331+0000","last_became_active":"2024-03-08T23:51:52.251591+0000","last_became_peered":"2024-03-08T23:51:52.251591+0000","last_unstale":"2024-03-08T23:52:45.410331+0000","last_undegraded":"2024-03-08T23:52:45.410331+0000","last_fullsized":"2024-03-08T23:52:45.410331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:06.276636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411107+0000","last_change":"2024-03-08T23:51:51.245981+0000","last_active":"2024-03-08T23:52:45.411107+0000","last_peered":"2024-03-08T23:52:45.411107+0000","last_clean":"2024-03-08T23:52:45.411107+0000","last_became_active":"2024-03-08T23:51:51.244681+0000","last_became_peered":"2024-03-08T23:51:51.244681+0000","last_unstale":"2024-03-08T23:52:45.411107+0000","last_undegraded":"2024-03-08T23:52:45.411107+0000","last_fullsized":"2024-03-08T23:52:45.411107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:06:21.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416298+0000","last_change":"2024-03-08T23:51:56.279057+0000","last_active":"2024-03-08T23:52:45.416298+0000","last_peered":"2024-03-08T23:52:45.416298+0000","last_clean":"2024-03-08T23:52:45.416298+0000","last_became_active":"2024-03-08T23:51:56.278634+0000","last_became_peered":"2024-03-08T23:51:56.278634+0000","last_unstale":"2024-03-08T23:52:45.416298+0000","last_undegraded":"2024-03-08T23:52:45.416298+0000","last_fullsized":"2024-03-08T23:52:45.416298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:30.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584441+0000","last_change":"2024-03-08T23:51:51.240378+0000","last_active":"2024-03-08T23:52:45.584441+0000","last_peered":"2024-03-08T23:52:45.584441+0000","last_clean":"2024-03-08T23:52:45.584441+0000","last_became_active":"2024-03-08T23:51:51.239880+0000","last_became_peered":"2024-03-08T23:51:51.239880+0000","last_unstale":"2024-03-08T23:52:45.584441+0000","last_undegraded":"2024-03-08T23:52:45.584441+0000","last_fullsized":"2024-03-08T23:52:45.584441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:20.804261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411591+0000","last_change":"2024-03-08T23:51:52.240338+0000","last_active":"2024-03-08T23:52:45.411591+0000","last_peered":"2024-03-08T23:52:45.411591+0000","last_clean":"2024-03-08T23:52:45.411591+0000","last_became_active":"2024-03-08T23:51:52.239987+0000","last_became_peered":"2024-03-08T23:51:52.239987+0000","last_unstale":"2024-03-08T23:52:45.411591+0000","last_undegraded":"2024-03-08T23:52:45.411591+0000","last_fullsized":"2024-03-08T23:52:45.411591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:26.590204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410553+0000","last_change":"2024-03-08T23:51:56.281009+0000","last_active":"2024-03-08T23:52:45.410553+0000","last_peered":"2024-03-08T23:52:45.410553+0000","last_clean":"2024-03-08T23:52:45.410553+0000","last_became_active":"2024-03-08T23:51:56.280619+0000","last_became_peered":"2024-03-08T23:51:56.280619+0000","last_unstale":"2024-03-08T23:52:45.410553+0000","last_undegraded":"2024-03-08T23:52:45.410553+0000","last_fullsized":"2024-03-08T23:52:45.410553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:06.920777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408419+0000","last_change":"2024-03-08T23:51:51.241682+0000","last_active":"2024-03-08T23:52:45.408419+0000","last_peered":"2024-03-08T23:52:45.408419+0000","last_clean":"2024-03-08T23:52:45.408419+0000","last_became_active":"2024-03-08T23:51:51.241313+0000","last_became_peered":"2024-03-08T23:51:51.241313+0000","last_unstale":"2024-03-08T23:52:45.408419+0000","last_undegraded":"2024-03-08T23:52:45.408419+0000","last_fullsized":"2024-03-08T23:52:45.408419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:04:13.515856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411826+0000","last_change":"2024-03-08T23:51:52.249378+0000","last_active":"2024-03-08T23:52:45.411826+0000","last_peered":"2024-03-08T23:52:45.411826+0000","last_clean":"2024-03-08T23:52:45.411826+0000","last_became_active":"2024-03-08T23:51:52.249084+0000","last_became_peered":"2024-03-08T23:51:52.249084+0000","last_unstale":"2024-03-08T23:52:45.411826+0000","last_undegraded":"2024-03-08T23:52:45.411826+0000","last_fullsized":"2024-03-08T23:52:45.411826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:10:36.513260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411744+0000","last_change":"2024-03-08T23:51:56.280717+0000","last_active":"2024-03-08T23:52:45.411744+0000","last_peered":"2024-03-08T23:52:45.411744+0000","last_clean":"2024-03-08T23:52:45.411744+0000","last_became_active":"2024-03-08T23:51:56.280329+0000","last_became_peered":"2024-03-08T23:51:56.280329+0000","last_unstale":"2024-03-08T23:52:45.411744+0000","last_undegraded":"2024-03-08T23:52:45.411744+0000","last_fullsized":"2024-03-08T23:52:45.411744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:47:40.829142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409898+0000","last_change":"2024-03-08T23:51:51.238058+0000","last_active":"2024-03-08T23:52:45.409898+0000","last_peered":"2024-03-08T23:52:45.409898+0000","last_clean":"2024-03-08T23:52:45.409898+0000","last_became_active":"2024-03-08T23:51:51.237727+0000","last_became_peered":"2024-03-08T23:51:51.237727+0000","last_unstale":"2024-03-08T23:52:45.409898+0000","last_undegraded":"2024-03-08T23:52:45.409898+0000","last_fullsized":"2024-03-08T23:52:45.409898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:13:54.080004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849383+0000","last_change":"2024-03-08T23:51:52.249830+0000","last_active":"2024-03-08T23:52:45.849383+0000","last_peered":"2024-03-08T23:52:45.849383+0000","last_clean":"2024-03-08T23:52:45.849383+0000","last_became_active":"2024-03-08T23:51:52.249518+0000","last_became_peered":"2024-03-08T23:51:52.249518+0000","last_unstale":"2024-03-08T23:52:45.849383+0000","last_undegraded":"2024-03-08T23:52:45.849383+0000","last_fullsized":"2024-03-08T23:52:45.849383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:24:33.721423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409483+0000","last_change":"2024-03-08T23:51:56.288595+0000","last_active":"2024-03-08T23:52:45.409483+0000","last_peered":"2024-03-08T23:52:45.409483+0000","last_clean":"2024-03-08T23:52:45.409483+0000","last_became_active":"2024-03-08T23:51:56.288146+0000","last_became_peered":"2024-03-08T23:51:56.288146+0000","last_unstale":"2024-03-08T23:52:45.409483+0000","last_undegraded":"2024-03-08T23:52:45.409483+0000","last_fullsized":"2024-03-08T23:52:45.409483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:34:58.321754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414356+0000","last_change":"2024-03-08T23:51:51.242065+0000","last_active":"2024-03-08T23:52:45.414356+0000","last_peered":"2024-03-08T23:52:45.414356+0000","last_clean":"2024-03-08T23:52:45.414356+0000","last_became_active":"2024-03-08T23:51:51.241513+0000","last_became_peered":"2024-03-08T23:51:51.241513+0000","last_unstale":"2024-03-08T23:52:45.414356+0000","last_undegraded":"2024-03-08T23:52:45.414356+0000","last_fullsized":"2024-03-08T23:52:45.414356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:08:02.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.583992+0000","last_change":"2024-03-08T23:51:52.243074+0000","last_active":"2024-03-08T23:52:45.583992+0000","last_peered":"2024-03-08T23:52:45.583992+0000","last_clean":"2024-03-08T23:52:45.583992+0000","last_became_active":"2024-03-08T23:51:52.242736+0000","last_became_peered":"2024-03-08T23:51:52.242736+0000","last_unstale":"2024-03-08T23:52:45.583992+0000","last_undegraded":"2024-03-08T23:52:45.583992+0000","last_fullsized":"2024-03-08T23:52:45.583992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:57:37.174056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408298+0000","last_change":"2024-03-08T23:51:56.276284+0000","last_active":"2024-03-08T23:52:45.408298+0000","last_peered":"2024-03-08T23:52:45.408298+0000","last_clean":"2024-03-08T23:52:45.408298+0000","last_became_active":"2024-03-08T23:51:56.275800+0000","last_became_peered":"2024-03-08T23:51:56.275800+0000","last_unstale":"2024-03-08T23:52:45.408298+0000","last_undegraded":"2024-03-08T23:52:45.408298+0000","last_fullsized":"2024-03-08T23:52:45.408298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:21:11.221266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410772+0000","last_change":"2024-03-08T23:51:51.245548+0000","last_active":"2024-03-08T23:52:45.410772+0000","last_peered":"2024-03-08T23:52:45.410772+0000","last_clean":"2024-03-08T23:52:45.410772+0000","last_became_active":"2024-03-08T23:51:51.245238+0000","last_became_peered":"2024-03-08T23:51:51.245238+0000","last_unstale":"2024-03-08T23:52:45.410772+0000","last_undegraded":"2024-03-08T23:52:45.410772+0000","last_fullsized":"2024-03-08T23:52:45.410772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:21:19.557196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416691+0000","last_change":"2024-03-08T23:51:52.244422+0000","last_active":"2024-03-08T23:52:45.416691+0000","last_peered":"2024-03-08T23:52:45.416691+0000","last_clean":"2024-03-08T23:52:45.416691+0000","last_became_active":"2024-03-08T23:51:52.244032+0000","last_became_peered":"2024-03-08T23:51:52.244032+0000","last_unstale":"2024-03-08T23:52:45.416691+0000","last_undegraded":"2024-03-08T23:52:45.416691+0000","last_fullsized":"2024-03-08T23:52:45.416691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:04.064691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409850+0000","last_change":"2024-03-08T23:51:56.288632+0000","last_active":"2024-03-08T23:52:45.409850+0000","last_peered":"2024-03-08T23:52:45.409850+0000","last_clean":"2024-03-08T23:52:45.409850+0000","last_became_active":"2024-03-08T23:51:56.288175+0000","last_became_peered":"2024-03-08T23:51:56.288175+0000","last_unstale":"2024-03-08T23:52:45.409850+0000","last_undegraded":"2024-03-08T23:52:45.409850+0000","last_fullsized":"2024-03-08T23:52:45.409850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:54:51.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412088+0000","last_change":"2024-03-08T23:51:51.247968+0000","last_active":"2024-03-08T23:52:45.412088+0000","last_peered":"2024-03-08T23:52:45.412088+0000","last_clean":"2024-03-08T23:52:45.412088+0000","last_became_active":"2024-03-08T23:51:51.247660+0000","last_became_peered":"2024-03-08T23:51:51.247660+0000","last_unstale":"2024-03-08T23:52:45.412088+0000","last_undegraded":"2024-03-08T23:52:45.412088+0000","last_fullsized":"2024-03-08T23:52:45.412088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:40.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411265+0000","last_change":"2024-03-08T23:51:52.250778+0000","last_active":"2024-03-08T23:52:45.411265+0000","last_peered":"2024-03-08T23:52:45.411265+0000","last_clean":"2024-03-08T23:52:45.411265+0000","last_became_active":"2024-03-08T23:51:52.250475+0000","last_became_peered":"2024-03-08T23:51:52.250475+0000","last_unstale":"2024-03-08T23:52:45.411265+0000","last_undegraded":"2024-03-08T23:52:45.411265+0000","last_fullsized":"2024-03-08T23:52:45.411265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:36:01.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409421+0000","last_change":"2024-03-08T23:51:56.285631+0000","last_active":"2024-03-08T23:52:45.409421+0000","last_peered":"2024-03-08T23:52:45.409421+0000","last_clean":"2024-03-08T23:52:45.409421+0000","last_became_active":"2024-03-08T23:51:56.285183+0000","last_became_peered":"2024-03-08T23:51:56.285183+0000","last_unstale":"2024-03-08T23:52:45.409421+0000","last_undegraded":"2024-03-08T23:52:45.409421+0000","last_fullsized":"2024-03-08T23:52:45.409421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:15.002086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584495+0000","last_change":"2024-03-08T23:51:51.246848+0000","last_active":"2024-03-08T23:52:45.584495+0000","last_peered":"2024-03-08T23:52:45.584495+0000","last_clean":"2024-03-08T23:52:45.584495+0000","last_became_active":"2024-03-08T23:51:51.246533+0000","last_became_peered":"2024-03-08T23:51:51.246533+0000","last_unstale":"2024-03-08T23:52:45.584495+0000","last_undegraded":"2024-03-08T23:52:45.584495+0000","last_fullsized":"2024-03-08T23:52:45.584495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:20:16.669820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411374+0000","last_change":"2024-03-08T23:51:52.251575+0000","last_active":"2024-03-08T23:52:45.411374+0000","last_peered":"2024-03-08T23:52:45.411374+0000","last_clean":"2024-03-08T23:52:45.411374+0000","last_became_active":"2024-03-08T23:51:52.251232+0000","last_became_peered":"2024-03-08T23:51:52.251232+0000","last_unstale":"2024-03-08T23:52:45.411374+0000","last_undegraded":"2024-03-08T23:52:45.411374+0000","last_fullsized":"2024-03-08T23:52:45.411374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:33.573407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.586092+0000","last_change":"2024-03-08T23:52:45.586092+0000","last_active":"2024-03-08T23:52:45.586092+0000","last_peered":"2024-03-08T23:52:45.586092+0000","last_clean":"2024-03-08T23:52:45.586092+0000","last_became_active":"2024-03-08T23:52:45.585513+0000","last_became_peered":"2024-03-08T23:52:45.585513+0000","last_unstale":"2024-03-08T23:52:45.586092+0000","last_undegraded":"2024-03-08T23:52:45.586092+0000","last_fullsized":"2024-03-08T23:52:45.586092+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:39:05.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413981+0000","last_change":"2024-03-08T23:51:51.242081+0000","last_active":"2024-03-08T23:52:45.413981+0000","last_peered":"2024-03-08T23:52:45.413981+0000","last_clean":"2024-03-08T23:52:45.413981+0000","last_became_active":"2024-03-08T23:51:51.241636+0000","last_became_peered":"2024-03-08T23:51:51.241636+0000","last_unstale":"2024-03-08T23:52:45.413981+0000","last_undegraded":"2024-03-08T23:52:45.413981+0000","last_fullsized":"2024-03-08T23:52:45.413981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:51:57.497723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412235+0000","last_change":"2024-03-08T23:51:52.252796+0000","last_active":"2024-03-08T23:52:45.412235+0000","last_peered":"2024-03-08T23:52:45.412235+0000","last_clean":"2024-03-08T23:52:45.412235+0000","last_became_active":"2024-03-08T23:51:52.252361+0000","last_became_peered":"2024-03-08T23:51:52.252361+0000","last_unstale":"2024-03-08T23:52:45.412235+0000","last_undegraded":"2024-03-08T23:52:45.412235+0000","last_fullsized":"2024-03-08T23:52:45.412235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:07.968262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.848938+0000","last_change":"2024-03-08T23:51:56.284396+0000","last_active":"2024-03-08T23:52:45.848938+0000","last_peered":"2024-03-08T23:52:45.848938+0000","last_clean":"2024-03-08T23:52:45.848938+0000","last_became_active":"2024-03-08T23:51:56.283951+0000","last_became_peered":"2024-03-08T23:51:56.283951+0000","last_unstale":"2024-03-08T23:52:45.848938+0000","last_undegraded":"2024-03-08T23:52:45.848938+0000","last_fullsized":"2024-03-08T23:52:45.848938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:09:40.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414485+0000","last_change":"2024-03-08T23:51:51.248838+0000","last_active":"2024-03-08T23:52:45.414485+0000","last_peered":"2024-03-08T23:52:45.414485+0000","last_clean":"2024-03-08T23:52:45.414485+0000","last_became_active":"2024-03-08T23:51:51.248534+0000","last_became_peered":"2024-03-08T23:51:51.248534+0000","last_unstale":"2024-03-08T23:52:45.414485+0000","last_undegraded":"2024-03-08T23:52:45.414485+0000","last_fullsized":"2024-03-08T23:52:45.414485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:37:25.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411577+0000","last_change":"2024-03-08T23:51:52.264170+0000","last_active":"2024-03-08T23:52:45.411577+0000","last_peered":"2024-03-08T23:52:45.411577+0000","last_clean":"2024-03-08T23:52:45.411577+0000","last_became_active":"2024-03-08T23:51:52.263744+0000","last_became_peered":"2024-03-08T23:51:52.263744+0000","last_unstale":"2024-03-08T23:52:45.411577+0000","last_undegraded":"2024-03-08T23:52:45.411577+0000","last_fullsized":"2024-03-08T23:52:45.411577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:42:18.287699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413828+0000","last_change":"2024-03-08T23:51:56.283316+0000","last_active":"2024-03-08T23:52:45.413828+0000","last_peered":"2024-03-08T23:52:45.413828+0000","last_clean":"2024-03-08T23:52:45.413828+0000","last_became_active":"2024-03-08T23:51:56.282515+0000","last_became_peered":"2024-03-08T23:51:56.282515+0000","last_unstale":"2024-03-08T23:52:45.413828+0000","last_undegraded":"2024-03-08T23:52:45.413828+0000","last_fullsized":"2024-03-08T23:52:45.413828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:40:15.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411644+0000","last_change":"2024-03-08T23:51:51.240970+0000","last_active":"2024-03-08T23:52:45.411644+0000","last_peered":"2024-03-08T23:52:45.411644+0000","last_clean":"2024-03-08T23:52:45.411644+0000","last_became_active":"2024-03-08T23:51:51.239982+0000","last_became_peered":"2024-03-08T23:51:51.239982+0000","last_unstale":"2024-03-08T23:52:45.411644+0000","last_undegraded":"2024-03-08T23:52:45.411644+0000","last_fullsized":"2024-03-08T23:52:45.411644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:47:40.313753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411949+0000","last_change":"2024-03-08T23:51:52.261673+0000","last_active":"2024-03-08T23:52:45.411949+0000","last_peered":"2024-03-08T23:52:45.411949+0000","last_clean":"2024-03-08T23:52:45.411949+0000","last_became_active":"2024-03-08T23:51:52.261333+0000","last_became_peered":"2024-03-08T23:51:52.261333+0000","last_unstale":"2024-03-08T23:52:45.411949+0000","last_undegraded":"2024-03-08T23:52:45.411949+0000","last_fullsized":"2024-03-08T23:52:45.411949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:08:26.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416718+0000","last_change":"2024-03-08T23:51:52.243189+0000","last_active":"2024-03-08T23:52:45.416718+0000","last_peered":"2024-03-08T23:52:45.416718+0000","last_clean":"2024-03-08T23:52:45.416718+0000","last_became_active":"2024-03-08T23:51:52.242870+0000","last_became_peered":"2024-03-08T23:51:52.242870+0000","last_unstale":"2024-03-08T23:52:45.416718+0000","last_undegraded":"2024-03-08T23:52:45.416718+0000","last_fullsized":"2024-03-08T23:52:45.416718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.438045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.417220+0000","last_change":"2024-03-08T23:52:45.417220+0000","last_active":"2024-03-08T23:52:45.417220+0000","last_peered":"2024-03-08T23:52:45.417220+0000","last_clean":"2024-03-08T23:52:45.417220+0000","last_became_active":"2024-03-08T23:52:45.416820+0000","last_became_peered":"2024-03-08T23:52:45.416820+0000","last_unstale":"2024-03-08T23:52:45.417220+0000","last_undegraded":"2024-03-08T23:52:45.417220+0000","last_fullsized":"2024-03-08T23:52:45.417220+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:31:13.249449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411814+0000","last_change":"2024-03-08T23:51:52.251397+0000","last_active":"2024-03-08T23:52:45.411814+0000","last_peered":"2024-03-08T23:52:45.411814+0000","last_clean":"2024-03-08T23:52:45.411814+0000","last_became_active":"2024-03-08T23:51:52.251059+0000","last_became_peered":"2024-03-08T23:51:52.251059+0000","last_unstale":"2024-03-08T23:52:45.411814+0000","last_undegraded":"2024-03-08T23:52:45.411814+0000","last_fullsized":"2024-03-08T23:52:45.411814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:19:30.363457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849115+0000","last_change":"2024-03-08T23:51:52.249039+0000","last_active":"2024-03-08T23:52:45.849115+0000","last_peered":"2024-03-08T23:52:45.849115+0000","last_clean":"2024-03-08T23:52:45.849115+0000","last_became_active":"2024-03-08T23:51:52.248560+0000","last_became_peered":"2024-03-08T23:51:52.248560+0000","last_unstale":"2024-03-08T23:52:45.849115+0000","last_undegraded":"2024-03-08T23:52:45.849115+0000","last_fullsized":"2024-03-08T23:52:45.849115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411479+0000","last_change":"2024-03-08T23:51:52.247269+0000","last_active":"2024-03-08T23:52:45.411479+0000","last_peered":"2024-03-08T23:52:45.411479+0000","last_clean":"2024-03-08T23:52:45.411479+0000","last_became_active":"2024-03-08T23:51:52.246921+0000","last_became_peered":"2024-03-08T23:51:52.246921+0000","last_unstale":"2024-03-08T23:52:45.411479+0000","last_undegraded":"2024-03-08T23:52:45.411479+0000","last_fullsized":"2024-03-08T23:52:45.411479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:00:21.828530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416365+0000","last_change":"2024-03-08T23:51:52.238999+0000","last_active":"2024-03-08T23:52:45.416365+0000","last_peered":"2024-03-08T23:52:45.416365+0000","last_clean":"2024-03-08T23:52:45.416365+0000","last_became_active":"2024-03-08T23:51:52.238662+0000","last_became_peered":"2024-03-08T23:51:52.238662+0000","last_unstale":"2024-03-08T23:52:45.416365+0000","last_undegraded":"2024-03-08T23:52:45.416365+0000","last_fullsized":"2024-03-08T23:52:45.416365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:22:08.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412568+0000","last_change":"2024-03-08T23:51:52.238793+0000","last_active":"2024-03-08T23:52:45.412568+0000","last_peered":"2024-03-08T23:52:45.412568+0000","last_clean":"2024-03-08T23:52:45.412568+0000","last_became_active":"2024-03-08T23:51:52.238464+0000","last_became_peered":"2024-03-08T23:51:52.238464+0000","last_unstale":"2024-03-08T23:52:45.412568+0000","last_undegraded":"2024-03-08T23:52:45.412568+0000","last_fullsized":"2024-03-08T23:52:45.412568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:25.597428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.585775+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:52:45.585775+0000","last_peered":"2024-03-08T23:52:45.585775+0000","last_clean":"2024-03-08T23:52:45.585775+0000","last_became_active":"2024-03-08T23:52:45.585280+0000","last_became_peered":"2024-03-08T23:52:45.585280+0000","last_unstale":"2024-03-08T23:52:45.585775+0000","last_undegraded":"2024-03-08T23:52:45.585775+0000","last_fullsized":"2024-03-08T23:52:45.585775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:20.029967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414171+0000","last_change":"2024-03-08T23:51:52.245288+0000","last_active":"2024-03-08T23:52:45.414171+0000","last_peered":"2024-03-08T23:52:45.414171+0000","last_clean":"2024-03-08T23:52:45.414171+0000","last_became_active":"2024-03-08T23:51:52.244960+0000","last_became_peered":"2024-03-08T23:51:52.244960+0000","last_unstale":"2024-03-08T23:52:45.414171+0000","last_undegraded":"2024-03-08T23:52:45.414171+0000","last_fullsized":"2024-03-08T23:52:45.414171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:20:39.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.418510+0000","last_change":"2024-03-08T23:52:45.418172+0000","last_active":"2024-03-08T23:52:45.418510+0000","last_peered":"2024-03-08T23:52:45.418510+0000","last_clean":"2024-03-08T23:52:45.418510+0000","last_became_active":"2024-03-08T23:52:45.417852+0000","last_became_peered":"2024-03-08T23:52:45.417852+0000","last_unstale":"2024-03-08T23:52:45.418510+0000","last_undegraded":"2024-03-08T23:52:45.418510+0000","last_fullsized":"2024-03-08T23:52:45.418510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:22:23.698238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414454+0000","last_change":"2024-03-08T23:51:52.251048+0000","last_active":"2024-03-08T23:52:45.414454+0000","last_peered":"2024-03-08T23:52:45.414454+0000","last_clean":"2024-03-08T23:52:45.414454+0000","last_became_active":"2024-03-08T23:51:52.250508+0000","last_became_peered":"2024-03-08T23:51:52.250508+0000","last_unstale":"2024-03-08T23:52:45.414454+0000","last_undegraded":"2024-03-08T23:52:45.414454+0000","last_fullsized":"2024-03-08T23:52:45.414454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:57:37.519307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410759+0000","last_change":"2024-03-08T23:51:52.244203+0000","last_active":"2024-03-08T23:52:45.410759+0000","last_peered":"2024-03-08T23:52:45.410759+0000","last_clean":"2024-03-08T23:52:45.410759+0000","last_became_active":"2024-03-08T23:51:52.243878+0000","last_became_peered":"2024-03-08T23:51:52.243878+0000","last_unstale":"2024-03-08T23:52:45.410759+0000","last_undegraded":"2024-03-08T23:52:45.410759+0000","last_fullsized":"2024-03-08T23:52:45.410759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:53:40.938657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416373+0000","last_change":"2024-03-08T23:51:52.248334+0000","last_active":"2024-03-08T23:52:45.416373+0000","last_peered":"2024-03-08T23:52:45.416373+0000","last_clean":"2024-03-08T23:52:45.416373+0000","last_became_active":"2024-03-08T23:51:52.248025+0000","last_became_peered":"2024-03-08T23:51:52.248025+0000","last_unstale":"2024-03-08T23:52:45.416373+0000","last_undegraded":"2024-03-08T23:52:45.416373+0000","last_fullsized":"2024-03-08T23:52:45.416373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:02:06.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66725,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":451,"num_read_kb":292,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60681,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1720320,"data_stored":1154911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":501059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":500979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1097279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":496235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":558478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":497623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":497886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":496429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":563609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T23:53:35.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-08T23:53:35.537 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:35.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:35 smithi062 ceph-mon[36129]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:35.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:35 smithi062 ceph-mon[36129]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:53:35.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:35 smithi192 ceph-mon[35995]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:35.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:35 smithi192 ceph-mon[35995]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:53:35.896 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:35 smithi022 ceph-mon[31894]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:35.896 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:35 smithi022 ceph-mon[31894]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:53:37.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 -- 172.21.15.22:0/2743616223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760100f70 msgr2=0x7fa760101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:37.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 --2- 172.21.15.22:0/2743616223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760100f70 0x7fa760101350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa750009990 tx=0x7fa750009ca0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 -- 172.21.15.22:0/2743616223 shutdown_connections 2024-03-08T23:53:37.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 --2- 172.21.15.22:0/2743616223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa760106000 0x7fa76010dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 --2- 172.21.15.22:0/2743616223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760100f70 0x7fa760101350 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 --2- 172.21.15.22:0/2743616223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa760101920 0x7fa760105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 -- 172.21.15.22:0/2743616223 >> 172.21.15.22:0/2743616223 conn(0x7fa760076d00 msgr2=0x7fa760077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:37.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 -- 172.21.15.22:0/2743616223 shutdown_connections 2024-03-08T23:53:37.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.297+0000 7fa7653c3700 1 -- 172.21.15.22:0/2743616223 wait complete. 2024-03-08T23:53:37.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 Processor -- start 2024-03-08T23:53:37.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 -- start start 2024-03-08T23:53:37.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa760100f70 0x7fa7601029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa760101920 0x7fa760102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760106000 0x7fa760103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa760111020 con 0x7fa760106000 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa760110ea0 con 0x7fa760101920 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa7653c3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601111a0 con 0x7fa760100f70 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.298+0000 7fa75f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760106000 0x7fa760103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760106000 0x7fa760103440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60318/0 (socket says 172.21.15.22:60318) 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 -- 172.21.15.22:0/1343902142 learned_addr learned my addr 172.21.15.22:0/1343902142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75e7fc700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa760101920 0x7fa760102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75effd700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa760100f70 0x7fa7601029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 -- 172.21.15.22:0/1343902142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa760100f70 msgr2=0x7fa7601029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:37.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa760100f70 0x7fa7601029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 -- 172.21.15.22:0/1343902142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa760101920 msgr2=0x7fa760102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa760101920 0x7fa760102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 -- 172.21.15.22:0/1343902142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7601a6b00 con 0x7fa760106000 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa75f7fe700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760106000 0x7fa760103440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa75800ada0 tx=0x7fa75800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa747fff700 1 -- 172.21.15.22:0/1343902142 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758017020 con 0x7fa760106000 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7601a6df0 con 0x7fa760106000 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa747fff700 1 -- 172.21.15.22:0/1343902142 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa75800a0d0 con 0x7fa760106000 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa747fff700 1 -- 172.21.15.22:0/1343902142 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758005480 con 0x7fa760106000 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.299+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7601a7350 con 0x7fa760106000 2024-03-08T23:53:37.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.300+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa760102400 con 0x7fa760106000 2024-03-08T23:53:37.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.301+0000 7fa747fff700 1 -- 172.21.15.22:0/1343902142 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fa758005620 con 0x7fa760106000 2024-03-08T23:53:37.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.302+0000 7fa747fff700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa7480a2920 0x7fa7480a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:37.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.302+0000 7fa747fff700 1 -- 172.21.15.22:0/1343902142 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa7580cebc0 con 0x7fa760106000 2024-03-08T23:53:37.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.302+0000 7fa75effd700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa7480a2920 0x7fa7480a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:37.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.303+0000 7fa75effd700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa7480a2920 0x7fa7480a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa750006010 tx=0x7fa75000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:37.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.306+0000 7fa747fff700 1 -- 172.21.15.22:0/1343902142 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa758021020 con 0x7fa760106000 2024-03-08T23:53:37.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.506+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa760104ae0 con 0x7fa7480a2920 2024-03-08T23:53:37.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.515+0000 7fa747fff700 1 -- 172.21.15.22:0/1343902142 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340369 (secure 0 0 0) 0x7fa760104ae0 con 0x7fa7480a2920 2024-03-08T23:53:37.515 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa7480a2920 msgr2=0x7fa7480a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa7480a2920 0x7fa7480a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa750006010 tx=0x7fa75000b540 comp rx=0 tx=0).stop 2024-03-08T23:53:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760106000 msgr2=0x7fa760103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760106000 0x7fa760103440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa75800ada0 tx=0x7fa75800c330 comp rx=0 tx=0).stop 2024-03-08T23:53:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 shutdown_connections 2024-03-08T23:53:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa7480a2920 0x7fa7480a4de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa760106000 0x7fa760103440 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa760100f70 0x7fa7601029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 --2- 172.21.15.22:0/1343902142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa760101920 0x7fa760102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:37.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 >> 172.21.15.22:0/1343902142 conn(0x7fa760076d00 msgr2=0x7fa760107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:37.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 shutdown_connections 2024-03-08T23:53:37.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:37.518+0000 7fa7653c3700 1 -- 172.21.15.22:0/1343902142 wait complete. 2024-03-08T23:53:37.519 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-08T23:53:37.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:37 smithi062 ceph-mon[36129]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:37.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:37 smithi192 ceph-mon[35995]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:37.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:37 smithi022 ceph-mon[31894]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:38.438 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-03-08T23:53:35.813888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657327,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":607,"num_read_kb":426,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1241534,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363316,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578508,"statfs":{"total":1151940427776,"available":1151568392192,"internally_reserved":0,"allocated":14434304,"data_stored":7957787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003419"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416389+0000","last_change":"2024-03-08T23:51:52.246270+0000","last_active":"2024-03-08T23:52:45.416389+0000","last_peered":"2024-03-08T23:52:45.416389+0000","last_clean":"2024-03-08T23:52:45.416389+0000","last_became_active":"2024-03-08T23:51:52.245321+0000","last_became_peered":"2024-03-08T23:51:52.245321+0000","last_unstale":"2024-03-08T23:52:45.416389+0000","last_undegraded":"2024-03-08T23:52:45.416389+0000","last_fullsized":"2024-03-08T23:52:45.416389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:39.993965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849289+0000","last_change":"2024-03-08T23:51:52.245433+0000","last_active":"2024-03-08T23:52:45.849289+0000","last_peered":"2024-03-08T23:52:45.849289+0000","last_clean":"2024-03-08T23:52:45.849289+0000","last_became_active":"2024-03-08T23:51:52.245106+0000","last_became_peered":"2024-03-08T23:51:52.245106+0000","last_unstale":"2024-03-08T23:52:45.849289+0000","last_undegraded":"2024-03-08T23:52:45.849289+0000","last_fullsized":"2024-03-08T23:52:45.849289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:02:20.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849269+0000","last_change":"2024-03-08T23:51:52.244112+0000","last_active":"2024-03-08T23:52:45.849269+0000","last_peered":"2024-03-08T23:52:45.849269+0000","last_clean":"2024-03-08T23:52:45.849269+0000","last_became_active":"2024-03-08T23:51:52.243783+0000","last_became_peered":"2024-03-08T23:51:52.243783+0000","last_unstale":"2024-03-08T23:52:45.849269+0000","last_undegraded":"2024-03-08T23:52:45.849269+0000","last_fullsized":"2024-03-08T23:52:45.849269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:48:43.719188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.586298+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:52:45.586298+0000","last_peered":"2024-03-08T23:52:45.586298+0000","last_clean":"2024-03-08T23:52:45.586298+0000","last_became_active":"2024-03-08T23:52:45.585292+0000","last_became_peered":"2024-03-08T23:52:45.585292+0000","last_unstale":"2024-03-08T23:52:45.586298+0000","last_undegraded":"2024-03-08T23:52:45.586298+0000","last_fullsized":"2024-03-08T23:52:45.586298+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:27:12.230920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584251+0000","last_change":"2024-03-08T23:51:52.247091+0000","last_active":"2024-03-08T23:52:45.584251+0000","last_peered":"2024-03-08T23:52:45.584251+0000","last_clean":"2024-03-08T23:52:45.584251+0000","last_became_active":"2024-03-08T23:51:52.246761+0000","last_became_peered":"2024-03-08T23:51:52.246761+0000","last_unstale":"2024-03-08T23:52:45.584251+0000","last_undegraded":"2024-03-08T23:52:45.584251+0000","last_fullsized":"2024-03-08T23:52:45.584251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:46:46.810857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411181+0000","last_change":"2024-03-08T23:51:52.257558+0000","last_active":"2024-03-08T23:52:45.411181+0000","last_peered":"2024-03-08T23:52:45.411181+0000","last_clean":"2024-03-08T23:52:45.411181+0000","last_became_active":"2024-03-08T23:51:52.257116+0000","last_became_peered":"2024-03-08T23:51:52.257116+0000","last_unstale":"2024-03-08T23:52:45.411181+0000","last_undegraded":"2024-03-08T23:52:45.411181+0000","last_fullsized":"2024-03-08T23:52:45.411181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:13:05.325780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411884+0000","last_change":"2024-03-08T23:51:52.253624+0000","last_active":"2024-03-08T23:52:45.411884+0000","last_peered":"2024-03-08T23:52:45.411884+0000","last_clean":"2024-03-08T23:52:45.411884+0000","last_became_active":"2024-03-08T23:51:52.253284+0000","last_became_peered":"2024-03-08T23:51:52.253284+0000","last_unstale":"2024-03-08T23:52:45.411884+0000","last_undegraded":"2024-03-08T23:52:45.411884+0000","last_fullsized":"2024-03-08T23:52:45.411884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:09:33.789502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410377+0000","last_change":"2024-03-08T23:51:52.255321+0000","last_active":"2024-03-08T23:52:45.410377+0000","last_peered":"2024-03-08T23:52:45.410377+0000","last_clean":"2024-03-08T23:52:45.410377+0000","last_became_active":"2024-03-08T23:51:52.254975+0000","last_became_peered":"2024-03-08T23:51:52.254975+0000","last_unstale":"2024-03-08T23:52:45.410377+0000","last_undegraded":"2024-03-08T23:52:45.410377+0000","last_fullsized":"2024-03-08T23:52:45.410377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:02:56.788519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414750+0000","last_change":"2024-03-08T23:51:52.244623+0000","last_active":"2024-03-08T23:52:45.414750+0000","last_peered":"2024-03-08T23:52:45.414750+0000","last_clean":"2024-03-08T23:52:45.414750+0000","last_became_active":"2024-03-08T23:51:52.244162+0000","last_became_peered":"2024-03-08T23:51:52.244162+0000","last_unstale":"2024-03-08T23:52:45.414750+0000","last_undegraded":"2024-03-08T23:52:45.414750+0000","last_fullsized":"2024-03-08T23:52:45.414750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:30:43.050616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411474+0000","last_change":"2024-03-08T23:51:52.258006+0000","last_active":"2024-03-08T23:52:45.411474+0000","last_peered":"2024-03-08T23:52:45.411474+0000","last_clean":"2024-03-08T23:52:45.411474+0000","last_became_active":"2024-03-08T23:51:52.256770+0000","last_became_peered":"2024-03-08T23:51:52.256770+0000","last_unstale":"2024-03-08T23:52:45.411474+0000","last_undegraded":"2024-03-08T23:52:45.411474+0000","last_fullsized":"2024-03-08T23:52:45.411474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:18.584212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412095+0000","last_change":"2024-03-08T23:51:52.260530+0000","last_active":"2024-03-08T23:52:45.412095+0000","last_peered":"2024-03-08T23:52:45.412095+0000","last_clean":"2024-03-08T23:52:45.412095+0000","last_became_active":"2024-03-08T23:51:52.260193+0000","last_became_peered":"2024-03-08T23:51:52.260193+0000","last_unstale":"2024-03-08T23:52:45.412095+0000","last_undegraded":"2024-03-08T23:52:45.412095+0000","last_fullsized":"2024-03-08T23:52:45.412095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:02.549683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409359+0000","last_change":"2024-03-08T23:51:52.248679+0000","last_active":"2024-03-08T23:52:45.409359+0000","last_peered":"2024-03-08T23:52:45.409359+0000","last_clean":"2024-03-08T23:52:45.409359+0000","last_became_active":"2024-03-08T23:51:52.248358+0000","last_became_peered":"2024-03-08T23:51:52.248358+0000","last_unstale":"2024-03-08T23:52:45.409359+0000","last_undegraded":"2024-03-08T23:52:45.409359+0000","last_fullsized":"2024-03-08T23:52:45.409359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:26.680887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415807+0000","last_change":"2024-03-08T23:52:45.415807+0000","last_active":"2024-03-08T23:52:45.415807+0000","last_peered":"2024-03-08T23:52:45.415807+0000","last_clean":"2024-03-08T23:52:45.415807+0000","last_became_active":"2024-03-08T23:52:45.415456+0000","last_became_peered":"2024-03-08T23:52:45.415456+0000","last_unstale":"2024-03-08T23:52:45.415807+0000","last_undegraded":"2024-03-08T23:52:45.415807+0000","last_fullsized":"2024-03-08T23:52:45.415807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:12:01.942744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.848919+0000","last_change":"2024-03-08T23:51:52.245394+0000","last_active":"2024-03-08T23:52:45.848919+0000","last_peered":"2024-03-08T23:52:45.848919+0000","last_clean":"2024-03-08T23:52:45.848919+0000","last_became_active":"2024-03-08T23:51:52.245038+0000","last_became_peered":"2024-03-08T23:51:52.245038+0000","last_unstale":"2024-03-08T23:52:45.848919+0000","last_undegraded":"2024-03-08T23:52:45.848919+0000","last_fullsized":"2024-03-08T23:52:45.848919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:15.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.848916+0000","last_change":"2024-03-08T23:51:52.250861+0000","last_active":"2024-03-08T23:52:45.848916+0000","last_peered":"2024-03-08T23:52:45.848916+0000","last_clean":"2024-03-08T23:52:45.848916+0000","last_became_active":"2024-03-08T23:51:52.250548+0000","last_became_peered":"2024-03-08T23:51:52.250548+0000","last_unstale":"2024-03-08T23:52:45.848916+0000","last_undegraded":"2024-03-08T23:52:45.848916+0000","last_fullsized":"2024-03-08T23:52:45.848916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:00.280043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414420+0000","last_change":"2024-03-08T23:51:52.254996+0000","last_active":"2024-03-08T23:52:45.414420+0000","last_peered":"2024-03-08T23:52:45.414420+0000","last_clean":"2024-03-08T23:52:45.414420+0000","last_became_active":"2024-03-08T23:51:52.254634+0000","last_became_peered":"2024-03-08T23:51:52.254634+0000","last_unstale":"2024-03-08T23:52:45.414420+0000","last_undegraded":"2024-03-08T23:52:45.414420+0000","last_fullsized":"2024-03-08T23:52:45.414420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:15.449988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416241+0000","last_change":"2024-03-08T23:51:52.246558+0000","last_active":"2024-03-08T23:52:45.416241+0000","last_peered":"2024-03-08T23:52:45.416241+0000","last_clean":"2024-03-08T23:52:45.416241+0000","last_became_active":"2024-03-08T23:51:52.245741+0000","last_became_peered":"2024-03-08T23:51:52.245741+0000","last_unstale":"2024-03-08T23:52:45.416241+0000","last_undegraded":"2024-03-08T23:52:45.416241+0000","last_fullsized":"2024-03-08T23:52:45.416241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:36.737017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408291+0000","last_change":"2024-03-08T23:51:52.240111+0000","last_active":"2024-03-08T23:52:45.408291+0000","last_peered":"2024-03-08T23:52:45.408291+0000","last_clean":"2024-03-08T23:52:45.408291+0000","last_became_active":"2024-03-08T23:51:52.238961+0000","last_became_peered":"2024-03-08T23:51:52.238961+0000","last_unstale":"2024-03-08T23:52:45.408291+0000","last_undegraded":"2024-03-08T23:52:45.408291+0000","last_fullsized":"2024-03-08T23:52:45.408291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:03:55.313117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412252+0000","last_change":"2024-03-08T23:51:52.245979+0000","last_active":"2024-03-08T23:52:45.412252+0000","last_peered":"2024-03-08T23:52:45.412252+0000","last_clean":"2024-03-08T23:52:45.412252+0000","last_became_active":"2024-03-08T23:51:52.245666+0000","last_became_peered":"2024-03-08T23:51:52.245666+0000","last_unstale":"2024-03-08T23:52:45.412252+0000","last_undegraded":"2024-03-08T23:52:45.412252+0000","last_fullsized":"2024-03-08T23:52:45.412252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:20.789144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412405+0000","last_change":"2024-03-08T23:51:56.288161+0000","last_active":"2024-03-08T23:52:45.412405+0000","last_peered":"2024-03-08T23:52:45.412405+0000","last_clean":"2024-03-08T23:52:45.412405+0000","last_became_active":"2024-03-08T23:51:56.287778+0000","last_became_peered":"2024-03-08T23:51:56.287778+0000","last_unstale":"2024-03-08T23:52:45.412405+0000","last_undegraded":"2024-03-08T23:52:45.412405+0000","last_fullsized":"2024-03-08T23:52:45.412405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:12:59.636228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408341+0000","last_change":"2024-03-08T23:51:51.244911+0000","last_active":"2024-03-08T23:52:45.408341+0000","last_peered":"2024-03-08T23:52:45.408341+0000","last_clean":"2024-03-08T23:52:45.408341+0000","last_became_active":"2024-03-08T23:51:51.244561+0000","last_became_peered":"2024-03-08T23:51:51.244561+0000","last_unstale":"2024-03-08T23:52:45.408341+0000","last_undegraded":"2024-03-08T23:52:45.408341+0000","last_fullsized":"2024-03-08T23:52:45.408341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:58:45.908307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411427+0000","last_change":"2024-03-08T23:51:52.253854+0000","last_active":"2024-03-08T23:52:45.411427+0000","last_peered":"2024-03-08T23:52:45.411427+0000","last_clean":"2024-03-08T23:52:45.411427+0000","last_became_active":"2024-03-08T23:51:52.253538+0000","last_became_peered":"2024-03-08T23:51:52.253538+0000","last_unstale":"2024-03-08T23:52:45.411427+0000","last_undegraded":"2024-03-08T23:52:45.411427+0000","last_fullsized":"2024-03-08T23:52:45.411427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:33.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408730+0000","last_change":"2024-03-08T23:51:56.283307+0000","last_active":"2024-03-08T23:52:45.408730+0000","last_peered":"2024-03-08T23:52:45.408730+0000","last_clean":"2024-03-08T23:52:45.408730+0000","last_became_active":"2024-03-08T23:51:56.282868+0000","last_became_peered":"2024-03-08T23:51:56.282868+0000","last_unstale":"2024-03-08T23:52:45.408730+0000","last_undegraded":"2024-03-08T23:52:45.408730+0000","last_fullsized":"2024-03-08T23:52:45.408730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:15.027743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411684+0000","last_change":"2024-03-08T23:51:51.247368+0000","last_active":"2024-03-08T23:52:45.411684+0000","last_peered":"2024-03-08T23:52:45.411684+0000","last_clean":"2024-03-08T23:52:45.411684+0000","last_became_active":"2024-03-08T23:51:51.247023+0000","last_became_peered":"2024-03-08T23:51:51.247023+0000","last_unstale":"2024-03-08T23:52:45.411684+0000","last_undegraded":"2024-03-08T23:52:45.411684+0000","last_fullsized":"2024-03-08T23:52:45.411684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:28.311850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849303+0000","last_change":"2024-03-08T23:51:52.245805+0000","last_active":"2024-03-08T23:52:45.849303+0000","last_peered":"2024-03-08T23:52:45.849303+0000","last_clean":"2024-03-08T23:52:45.849303+0000","last_became_active":"2024-03-08T23:51:52.244455+0000","last_became_peered":"2024-03-08T23:51:52.244455+0000","last_unstale":"2024-03-08T23:52:45.849303+0000","last_undegraded":"2024-03-08T23:52:45.849303+0000","last_fullsized":"2024-03-08T23:52:45.849303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:11:17.809148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408960+0000","last_change":"2024-03-08T23:51:56.286666+0000","last_active":"2024-03-08T23:52:45.408960+0000","last_peered":"2024-03-08T23:52:45.408960+0000","last_clean":"2024-03-08T23:52:45.408960+0000","last_became_active":"2024-03-08T23:51:56.285968+0000","last_became_peered":"2024-03-08T23:51:56.285968+0000","last_unstale":"2024-03-08T23:52:45.408960+0000","last_undegraded":"2024-03-08T23:52:45.408960+0000","last_fullsized":"2024-03-08T23:52:45.408960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:51.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584660+0000","last_change":"2024-03-08T23:51:51.247684+0000","last_active":"2024-03-08T23:52:45.584660+0000","last_peered":"2024-03-08T23:52:45.584660+0000","last_clean":"2024-03-08T23:52:45.584660+0000","last_became_active":"2024-03-08T23:51:51.247374+0000","last_became_peered":"2024-03-08T23:51:51.247374+0000","last_unstale":"2024-03-08T23:52:45.584660+0000","last_undegraded":"2024-03-08T23:52:45.584660+0000","last_fullsized":"2024-03-08T23:52:45.584660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:20.425302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413823+0000","last_change":"2024-03-08T23:51:52.248702+0000","last_active":"2024-03-08T23:52:45.413823+0000","last_peered":"2024-03-08T23:52:45.413823+0000","last_clean":"2024-03-08T23:52:45.413823+0000","last_became_active":"2024-03-08T23:51:52.248388+0000","last_became_peered":"2024-03-08T23:51:52.248388+0000","last_unstale":"2024-03-08T23:52:45.413823+0000","last_undegraded":"2024-03-08T23:52:45.413823+0000","last_fullsized":"2024-03-08T23:52:45.413823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:27:12.251957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408935+0000","last_change":"2024-03-08T23:51:56.286572+0000","last_active":"2024-03-08T23:52:45.408935+0000","last_peered":"2024-03-08T23:52:45.408935+0000","last_clean":"2024-03-08T23:52:45.408935+0000","last_became_active":"2024-03-08T23:51:56.286066+0000","last_became_peered":"2024-03-08T23:51:56.286066+0000","last_unstale":"2024-03-08T23:52:45.408935+0000","last_undegraded":"2024-03-08T23:52:45.408935+0000","last_fullsized":"2024-03-08T23:52:45.408935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:13.694615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584533+0000","last_change":"2024-03-08T23:51:51.244207+0000","last_active":"2024-03-08T23:52:45.584533+0000","last_peered":"2024-03-08T23:52:45.584533+0000","last_clean":"2024-03-08T23:52:45.584533+0000","last_became_active":"2024-03-08T23:51:51.241605+0000","last_became_peered":"2024-03-08T23:51:51.241605+0000","last_unstale":"2024-03-08T23:52:45.584533+0000","last_undegraded":"2024-03-08T23:52:45.584533+0000","last_fullsized":"2024-03-08T23:52:45.584533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:03:51.766681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415031+0000","last_change":"2024-03-08T23:51:52.251015+0000","last_active":"2024-03-08T23:52:45.415031+0000","last_peered":"2024-03-08T23:52:45.415031+0000","last_clean":"2024-03-08T23:52:45.415031+0000","last_became_active":"2024-03-08T23:51:52.250690+0000","last_became_peered":"2024-03-08T23:51:52.250690+0000","last_unstale":"2024-03-08T23:52:45.415031+0000","last_undegraded":"2024-03-08T23:52:45.415031+0000","last_fullsized":"2024-03-08T23:52:45.415031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:57:23.033172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412052+0000","last_change":"2024-03-08T23:51:56.281066+0000","last_active":"2024-03-08T23:52:45.412052+0000","last_peered":"2024-03-08T23:52:45.412052+0000","last_clean":"2024-03-08T23:52:45.412052+0000","last_became_active":"2024-03-08T23:51:56.280654+0000","last_became_peered":"2024-03-08T23:51:56.280654+0000","last_unstale":"2024-03-08T23:52:45.412052+0000","last_undegraded":"2024-03-08T23:52:45.412052+0000","last_fullsized":"2024-03-08T23:52:45.412052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:53.999115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:31.456562+0000","last_change":"2024-03-08T23:51:51.245947+0000","last_active":"2024-03-08T23:53:31.456562+0000","last_peered":"2024-03-08T23:53:31.456562+0000","last_clean":"2024-03-08T23:53:31.456562+0000","last_became_active":"2024-03-08T23:51:51.245505+0000","last_became_peered":"2024-03-08T23:51:51.245505+0000","last_unstale":"2024-03-08T23:53:31.456562+0000","last_undegraded":"2024-03-08T23:53:31.456562+0000","last_fullsized":"2024-03-08T23:53:31.456562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:20:44.692598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584041+0000","last_change":"2024-03-08T23:51:52.241590+0000","last_active":"2024-03-08T23:52:45.584041+0000","last_peered":"2024-03-08T23:52:45.584041+0000","last_clean":"2024-03-08T23:52:45.584041+0000","last_became_active":"2024-03-08T23:51:52.241296+0000","last_became_peered":"2024-03-08T23:51:52.241296+0000","last_unstale":"2024-03-08T23:52:45.584041+0000","last_undegraded":"2024-03-08T23:52:45.584041+0000","last_fullsized":"2024-03-08T23:52:45.584041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:52:48.042360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.585946+0000","last_change":"2024-03-08T23:52:45.585946+0000","last_active":"2024-03-08T23:52:45.585946+0000","last_peered":"2024-03-08T23:52:45.585946+0000","last_clean":"2024-03-08T23:52:45.585946+0000","last_became_active":"2024-03-08T23:52:45.585470+0000","last_became_peered":"2024-03-08T23:52:45.585470+0000","last_unstale":"2024-03-08T23:52:45.585946+0000","last_undegraded":"2024-03-08T23:52:45.585946+0000","last_fullsized":"2024-03-08T23:52:45.585946+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:28:36.264887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411059+0000","last_change":"2024-03-08T23:51:51.241384+0000","last_active":"2024-03-08T23:52:45.411059+0000","last_peered":"2024-03-08T23:52:45.411059+0000","last_clean":"2024-03-08T23:52:45.411059+0000","last_became_active":"2024-03-08T23:51:51.241018+0000","last_became_peered":"2024-03-08T23:51:51.241018+0000","last_unstale":"2024-03-08T23:52:45.411059+0000","last_undegraded":"2024-03-08T23:52:45.411059+0000","last_fullsized":"2024-03-08T23:52:45.411059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:29:50.860663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415653+0000","last_change":"2024-03-08T23:51:52.250207+0000","last_active":"2024-03-08T23:52:45.415653+0000","last_peered":"2024-03-08T23:52:45.415653+0000","last_clean":"2024-03-08T23:52:45.415653+0000","last_became_active":"2024-03-08T23:51:52.249863+0000","last_became_peered":"2024-03-08T23:51:52.249863+0000","last_unstale":"2024-03-08T23:52:45.415653+0000","last_undegraded":"2024-03-08T23:52:45.415653+0000","last_fullsized":"2024-03-08T23:52:45.415653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:18:48.732419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416096+0000","last_change":"2024-03-08T23:51:56.282092+0000","last_active":"2024-03-08T23:52:45.416096+0000","last_peered":"2024-03-08T23:52:45.416096+0000","last_clean":"2024-03-08T23:52:45.416096+0000","last_became_active":"2024-03-08T23:51:56.281657+0000","last_became_peered":"2024-03-08T23:51:56.281657+0000","last_unstale":"2024-03-08T23:52:45.416096+0000","last_undegraded":"2024-03-08T23:52:45.416096+0000","last_fullsized":"2024-03-08T23:52:45.416096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:50.440520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410588+0000","last_change":"2024-03-08T23:51:51.236816+0000","last_active":"2024-03-08T23:52:45.410588+0000","last_peered":"2024-03-08T23:52:45.410588+0000","last_clean":"2024-03-08T23:52:45.410588+0000","last_became_active":"2024-03-08T23:51:51.236485+0000","last_became_peered":"2024-03-08T23:51:51.236485+0000","last_unstale":"2024-03-08T23:52:45.410588+0000","last_undegraded":"2024-03-08T23:52:45.410588+0000","last_fullsized":"2024-03-08T23:52:45.410588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:27.300950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410618+0000","last_change":"2024-03-08T23:51:52.252146+0000","last_active":"2024-03-08T23:52:45.410618+0000","last_peered":"2024-03-08T23:52:45.410618+0000","last_clean":"2024-03-08T23:52:45.410618+0000","last_became_active":"2024-03-08T23:51:52.244201+0000","last_became_peered":"2024-03-08T23:51:52.244201+0000","last_unstale":"2024-03-08T23:52:45.410618+0000","last_undegraded":"2024-03-08T23:52:45.410618+0000","last_fullsized":"2024-03-08T23:52:45.410618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:07.647078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849262+0000","last_change":"2024-03-08T23:51:56.279727+0000","last_active":"2024-03-08T23:52:45.849262+0000","last_peered":"2024-03-08T23:52:45.849262+0000","last_clean":"2024-03-08T23:52:45.849262+0000","last_became_active":"2024-03-08T23:51:56.279184+0000","last_became_peered":"2024-03-08T23:51:56.279184+0000","last_unstale":"2024-03-08T23:52:45.849262+0000","last_undegraded":"2024-03-08T23:52:45.849262+0000","last_fullsized":"2024-03-08T23:52:45.849262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:21.804410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414779+0000","last_change":"2024-03-08T23:51:51.244812+0000","last_active":"2024-03-08T23:52:45.414779+0000","last_peered":"2024-03-08T23:52:45.414779+0000","last_clean":"2024-03-08T23:52:45.414779+0000","last_became_active":"2024-03-08T23:51:51.244513+0000","last_became_peered":"2024-03-08T23:51:51.244513+0000","last_unstale":"2024-03-08T23:52:45.414779+0000","last_undegraded":"2024-03-08T23:52:45.414779+0000","last_fullsized":"2024-03-08T23:52:45.414779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:56:07.235214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.407857+0000","last_change":"2024-03-08T23:51:52.244566+0000","last_active":"2024-03-08T23:52:45.407857+0000","last_peered":"2024-03-08T23:52:45.407857+0000","last_clean":"2024-03-08T23:52:45.407857+0000","last_became_active":"2024-03-08T23:51:52.244250+0000","last_became_peered":"2024-03-08T23:51:52.244250+0000","last_unstale":"2024-03-08T23:52:45.407857+0000","last_undegraded":"2024-03-08T23:52:45.407857+0000","last_fullsized":"2024-03-08T23:52:45.407857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:55:50.596364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410766+0000","last_change":"2024-03-08T23:51:56.282487+0000","last_active":"2024-03-08T23:52:45.410766+0000","last_peered":"2024-03-08T23:52:45.410766+0000","last_clean":"2024-03-08T23:52:45.410766+0000","last_became_active":"2024-03-08T23:51:56.282104+0000","last_became_peered":"2024-03-08T23:51:56.282104+0000","last_unstale":"2024-03-08T23:52:45.410766+0000","last_undegraded":"2024-03-08T23:52:45.410766+0000","last_fullsized":"2024-03-08T23:52:45.410766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:23:23.787431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:32.457398+0000","last_change":"2024-03-08T23:51:51.236612+0000","last_active":"2024-03-08T23:53:32.457398+0000","last_peered":"2024-03-08T23:53:32.457398+0000","last_clean":"2024-03-08T23:53:32.457398+0000","last_became_active":"2024-03-08T23:51:51.236274+0000","last_became_peered":"2024-03-08T23:51:51.236274+0000","last_unstale":"2024-03-08T23:53:32.457398+0000","last_undegraded":"2024-03-08T23:53:32.457398+0000","last_fullsized":"2024-03-08T23:53:32.457398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:37.551870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414901+0000","last_change":"2024-03-08T23:51:52.244564+0000","last_active":"2024-03-08T23:52:45.414901+0000","last_peered":"2024-03-08T23:52:45.414901+0000","last_clean":"2024-03-08T23:52:45.414901+0000","last_became_active":"2024-03-08T23:51:52.243651+0000","last_became_peered":"2024-03-08T23:51:52.243651+0000","last_unstale":"2024-03-08T23:52:45.414901+0000","last_undegraded":"2024-03-08T23:52:45.414901+0000","last_fullsized":"2024-03-08T23:52:45.414901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:35:32.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.407484+0000","last_change":"2024-03-08T23:51:56.279354+0000","last_active":"2024-03-08T23:52:45.407484+0000","last_peered":"2024-03-08T23:52:45.407484+0000","last_clean":"2024-03-08T23:52:45.407484+0000","last_became_active":"2024-03-08T23:51:56.278894+0000","last_became_peered":"2024-03-08T23:51:56.278894+0000","last_unstale":"2024-03-08T23:52:45.407484+0000","last_undegraded":"2024-03-08T23:52:45.407484+0000","last_fullsized":"2024-03-08T23:52:45.407484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:20:30.164542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411582+0000","last_change":"2024-03-08T23:51:51.246417+0000","last_active":"2024-03-08T23:52:45.411582+0000","last_peered":"2024-03-08T23:52:45.411582+0000","last_clean":"2024-03-08T23:52:45.411582+0000","last_became_active":"2024-03-08T23:51:51.246058+0000","last_became_peered":"2024-03-08T23:51:51.246058+0000","last_unstale":"2024-03-08T23:52:45.411582+0000","last_undegraded":"2024-03-08T23:52:45.411582+0000","last_fullsized":"2024-03-08T23:52:45.411582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:48:53.569295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849326+0000","last_change":"2024-03-08T23:51:52.255357+0000","last_active":"2024-03-08T23:52:45.849326+0000","last_peered":"2024-03-08T23:52:45.849326+0000","last_clean":"2024-03-08T23:52:45.849326+0000","last_became_active":"2024-03-08T23:51:52.255044+0000","last_became_peered":"2024-03-08T23:51:52.255044+0000","last_unstale":"2024-03-08T23:52:45.849326+0000","last_undegraded":"2024-03-08T23:52:45.849326+0000","last_fullsized":"2024-03-08T23:52:45.849326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:13:40.710283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409241+0000","last_change":"2024-03-08T23:51:56.284282+0000","last_active":"2024-03-08T23:52:45.409241+0000","last_peered":"2024-03-08T23:52:45.409241+0000","last_clean":"2024-03-08T23:52:45.409241+0000","last_became_active":"2024-03-08T23:51:56.283863+0000","last_became_peered":"2024-03-08T23:51:56.283863+0000","last_unstale":"2024-03-08T23:52:45.409241+0000","last_undegraded":"2024-03-08T23:52:45.409241+0000","last_fullsized":"2024-03-08T23:52:45.409241+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:59:21.972283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414320+0000","last_change":"2024-03-08T23:51:51.245199+0000","last_active":"2024-03-08T23:52:45.414320+0000","last_peered":"2024-03-08T23:52:45.414320+0000","last_clean":"2024-03-08T23:52:45.414320+0000","last_became_active":"2024-03-08T23:51:51.244906+0000","last_became_peered":"2024-03-08T23:51:51.244906+0000","last_unstale":"2024-03-08T23:52:45.414320+0000","last_undegraded":"2024-03-08T23:52:45.414320+0000","last_fullsized":"2024-03-08T23:52:45.414320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:54:39.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413987+0000","last_change":"2024-03-08T23:51:52.251209+0000","last_active":"2024-03-08T23:52:45.413987+0000","last_peered":"2024-03-08T23:52:45.413987+0000","last_clean":"2024-03-08T23:52:45.413987+0000","last_became_active":"2024-03-08T23:51:52.250392+0000","last_became_peered":"2024-03-08T23:51:52.250392+0000","last_unstale":"2024-03-08T23:52:45.413987+0000","last_undegraded":"2024-03-08T23:52:45.413987+0000","last_fullsized":"2024-03-08T23:52:45.413987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:56:17.811505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.586125+0000","last_change":"2024-03-08T23:52:45.586125+0000","last_active":"2024-03-08T23:52:45.586125+0000","last_peered":"2024-03-08T23:52:45.586125+0000","last_clean":"2024-03-08T23:52:45.586125+0000","last_became_active":"2024-03-08T23:52:45.585578+0000","last_became_peered":"2024-03-08T23:52:45.585578+0000","last_unstale":"2024-03-08T23:52:45.586125+0000","last_undegraded":"2024-03-08T23:52:45.586125+0000","last_fullsized":"2024-03-08T23:52:45.586125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:51:08.159460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":239,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:33.488801+0000","last_change":"2024-03-08T23:52:45.426691+0000","last_active":"2024-03-08T23:53:33.488801+0000","last_peered":"2024-03-08T23:53:33.488801+0000","last_clean":"2024-03-08T23:53:33.488801+0000","last_became_active":"2024-03-08T23:52:45.415742+0000","last_became_peered":"2024-03-08T23:52:45.415742+0000","last_unstale":"2024-03-08T23:53:33.488801+0000","last_undegraded":"2024-03-08T23:53:33.488801+0000","last_fullsized":"2024-03-08T23:53:33.488801+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:05.389164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62333,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":148,"num_read_kb":134,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60681,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584525+0000","last_change":"2024-03-08T23:51:52.248715+0000","last_active":"2024-03-08T23:52:45.584525+0000","last_peered":"2024-03-08T23:52:45.584525+0000","last_clean":"2024-03-08T23:52:45.584525+0000","last_became_active":"2024-03-08T23:51:52.248258+0000","last_became_peered":"2024-03-08T23:51:52.248258+0000","last_unstale":"2024-03-08T23:52:45.584525+0000","last_undegraded":"2024-03-08T23:52:45.584525+0000","last_fullsized":"2024-03-08T23:52:45.584525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:38.185469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414980+0000","last_change":"2024-03-08T23:51:56.278268+0000","last_active":"2024-03-08T23:52:45.414980+0000","last_peered":"2024-03-08T23:52:45.414980+0000","last_clean":"2024-03-08T23:52:45.414980+0000","last_became_active":"2024-03-08T23:51:56.277831+0000","last_became_peered":"2024-03-08T23:51:56.277831+0000","last_unstale":"2024-03-08T23:52:45.414980+0000","last_undegraded":"2024-03-08T23:52:45.414980+0000","last_fullsized":"2024-03-08T23:52:45.414980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:32:13.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412137+0000","last_change":"2024-03-08T23:51:51.247553+0000","last_active":"2024-03-08T23:52:45.412137+0000","last_peered":"2024-03-08T23:52:45.412137+0000","last_clean":"2024-03-08T23:52:45.412137+0000","last_became_active":"2024-03-08T23:51:51.247225+0000","last_became_peered":"2024-03-08T23:51:51.247225+0000","last_unstale":"2024-03-08T23:52:45.412137+0000","last_undegraded":"2024-03-08T23:52:45.412137+0000","last_fullsized":"2024-03-08T23:52:45.412137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:53:29.846303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411212+0000","last_change":"2024-03-08T23:51:52.247733+0000","last_active":"2024-03-08T23:52:45.411212+0000","last_peered":"2024-03-08T23:52:45.411212+0000","last_clean":"2024-03-08T23:52:45.411212+0000","last_became_active":"2024-03-08T23:51:52.247435+0000","last_became_peered":"2024-03-08T23:51:52.247435+0000","last_unstale":"2024-03-08T23:52:45.411212+0000","last_undegraded":"2024-03-08T23:52:45.411212+0000","last_fullsized":"2024-03-08T23:52:45.411212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:46:12.739601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415996+0000","last_change":"2024-03-08T23:51:56.279865+0000","last_active":"2024-03-08T23:52:45.415996+0000","last_peered":"2024-03-08T23:52:45.415996+0000","last_clean":"2024-03-08T23:52:45.415996+0000","last_became_active":"2024-03-08T23:51:56.279446+0000","last_became_peered":"2024-03-08T23:51:56.279446+0000","last_unstale":"2024-03-08T23:52:45.415996+0000","last_undegraded":"2024-03-08T23:52:45.415996+0000","last_fullsized":"2024-03-08T23:52:45.415996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:07:33.672490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.415220+0000","last_change":"2024-03-08T23:51:51.246860+0000","last_active":"2024-03-08T23:52:45.415220+0000","last_peered":"2024-03-08T23:52:45.415220+0000","last_clean":"2024-03-08T23:52:45.415220+0000","last_became_active":"2024-03-08T23:51:51.246545+0000","last_became_peered":"2024-03-08T23:51:51.246545+0000","last_unstale":"2024-03-08T23:52:45.415220+0000","last_undegraded":"2024-03-08T23:52:45.415220+0000","last_fullsized":"2024-03-08T23:52:45.415220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:11:07.246657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411593+0000","last_change":"2024-03-08T23:51:52.247353+0000","last_active":"2024-03-08T23:52:45.411593+0000","last_peered":"2024-03-08T23:52:45.411593+0000","last_clean":"2024-03-08T23:52:45.411593+0000","last_became_active":"2024-03-08T23:51:52.247008+0000","last_became_peered":"2024-03-08T23:51:52.247008+0000","last_unstale":"2024-03-08T23:52:45.411593+0000","last_undegraded":"2024-03-08T23:52:45.411593+0000","last_fullsized":"2024-03-08T23:52:45.411593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:08:50.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408112+0000","last_change":"2024-03-08T23:51:56.284103+0000","last_active":"2024-03-08T23:52:45.408112+0000","last_peered":"2024-03-08T23:52:45.408112+0000","last_clean":"2024-03-08T23:52:45.408112+0000","last_became_active":"2024-03-08T23:51:56.283561+0000","last_became_peered":"2024-03-08T23:51:56.283561+0000","last_unstale":"2024-03-08T23:52:45.408112+0000","last_undegraded":"2024-03-08T23:52:45.408112+0000","last_fullsized":"2024-03-08T23:52:45.408112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:47:20.909292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:31.456019+0000","last_change":"2024-03-08T23:51:51.242483+0000","last_active":"2024-03-08T23:53:31.456019+0000","last_peered":"2024-03-08T23:53:31.456019+0000","last_clean":"2024-03-08T23:53:31.456019+0000","last_became_active":"2024-03-08T23:51:51.242153+0000","last_became_peered":"2024-03-08T23:51:51.242153+0000","last_unstale":"2024-03-08T23:53:31.456019+0000","last_undegraded":"2024-03-08T23:53:31.456019+0000","last_fullsized":"2024-03-08T23:53:31.456019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:44:07.728700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411150+0000","last_change":"2024-03-08T23:51:52.255411+0000","last_active":"2024-03-08T23:52:45.411150+0000","last_peered":"2024-03-08T23:52:45.411150+0000","last_clean":"2024-03-08T23:52:45.411150+0000","last_became_active":"2024-03-08T23:51:52.255048+0000","last_became_peered":"2024-03-08T23:51:52.255048+0000","last_unstale":"2024-03-08T23:52:45.411150+0000","last_undegraded":"2024-03-08T23:52:45.411150+0000","last_fullsized":"2024-03-08T23:52:45.411150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:11.836187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411760+0000","last_change":"2024-03-08T23:51:56.286405+0000","last_active":"2024-03-08T23:52:45.411760+0000","last_peered":"2024-03-08T23:52:45.411760+0000","last_clean":"2024-03-08T23:52:45.411760+0000","last_became_active":"2024-03-08T23:51:56.285996+0000","last_became_peered":"2024-03-08T23:51:56.285996+0000","last_unstale":"2024-03-08T23:52:45.411760+0000","last_undegraded":"2024-03-08T23:52:45.411760+0000","last_fullsized":"2024-03-08T23:52:45.411760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:48:30.928436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584032+0000","last_change":"2024-03-08T23:51:51.240146+0000","last_active":"2024-03-08T23:52:45.584032+0000","last_peered":"2024-03-08T23:52:45.584032+0000","last_clean":"2024-03-08T23:52:45.584032+0000","last_became_active":"2024-03-08T23:51:51.239795+0000","last_became_peered":"2024-03-08T23:51:51.239795+0000","last_unstale":"2024-03-08T23:52:45.584032+0000","last_undegraded":"2024-03-08T23:52:45.584032+0000","last_fullsized":"2024-03-08T23:52:45.584032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:55.409922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414563+0000","last_change":"2024-03-08T23:51:52.254701+0000","last_active":"2024-03-08T23:52:45.414563+0000","last_peered":"2024-03-08T23:52:45.414563+0000","last_clean":"2024-03-08T23:52:45.414563+0000","last_became_active":"2024-03-08T23:51:52.254391+0000","last_became_peered":"2024-03-08T23:51:52.254391+0000","last_unstale":"2024-03-08T23:52:45.414563+0000","last_undegraded":"2024-03-08T23:52:45.414563+0000","last_fullsized":"2024-03-08T23:52:45.414563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:11:12.258181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413984+0000","last_change":"2024-03-08T23:51:56.277878+0000","last_active":"2024-03-08T23:52:45.413984+0000","last_peered":"2024-03-08T23:52:45.413984+0000","last_clean":"2024-03-08T23:52:45.413984+0000","last_became_active":"2024-03-08T23:51:56.277493+0000","last_became_peered":"2024-03-08T23:51:56.277493+0000","last_unstale":"2024-03-08T23:52:45.413984+0000","last_undegraded":"2024-03-08T23:52:45.413984+0000","last_fullsized":"2024-03-08T23:52:45.413984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:44:16.968632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584348+0000","last_change":"2024-03-08T23:51:51.241457+0000","last_active":"2024-03-08T23:52:45.584348+0000","last_peered":"2024-03-08T23:52:45.584348+0000","last_clean":"2024-03-08T23:52:45.584348+0000","last_became_active":"2024-03-08T23:51:51.241148+0000","last_became_peered":"2024-03-08T23:51:51.241148+0000","last_unstale":"2024-03-08T23:52:45.584348+0000","last_undegraded":"2024-03-08T23:52:45.584348+0000","last_fullsized":"2024-03-08T23:52:45.584348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:13:27.246265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416189+0000","last_change":"2024-03-08T23:51:52.248465+0000","last_active":"2024-03-08T23:52:45.416189+0000","last_peered":"2024-03-08T23:52:45.416189+0000","last_clean":"2024-03-08T23:52:45.416189+0000","last_became_active":"2024-03-08T23:51:52.248135+0000","last_became_peered":"2024-03-08T23:51:52.248135+0000","last_unstale":"2024-03-08T23:52:45.416189+0000","last_undegraded":"2024-03-08T23:52:45.416189+0000","last_fullsized":"2024-03-08T23:52:45.416189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:55:53.712958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584125+0000","last_change":"2024-03-08T23:51:56.276019+0000","last_active":"2024-03-08T23:52:45.584125+0000","last_peered":"2024-03-08T23:52:45.584125+0000","last_clean":"2024-03-08T23:52:45.584125+0000","last_became_active":"2024-03-08T23:51:56.275564+0000","last_became_peered":"2024-03-08T23:51:56.275564+0000","last_unstale":"2024-03-08T23:52:45.584125+0000","last_undegraded":"2024-03-08T23:52:45.584125+0000","last_fullsized":"2024-03-08T23:52:45.584125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:12:36.096470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.407907+0000","last_change":"2024-03-08T23:51:51.241629+0000","last_active":"2024-03-08T23:52:45.407907+0000","last_peered":"2024-03-08T23:52:45.407907+0000","last_clean":"2024-03-08T23:52:45.407907+0000","last_became_active":"2024-03-08T23:51:51.241300+0000","last_became_peered":"2024-03-08T23:51:51.241300+0000","last_unstale":"2024-03-08T23:52:45.407907+0000","last_undegraded":"2024-03-08T23:52:45.407907+0000","last_fullsized":"2024-03-08T23:52:45.407907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:01:45.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411166+0000","last_change":"2024-03-08T23:51:52.246024+0000","last_active":"2024-03-08T23:52:45.411166+0000","last_peered":"2024-03-08T23:52:45.411166+0000","last_clean":"2024-03-08T23:52:45.411166+0000","last_became_active":"2024-03-08T23:51:52.245706+0000","last_became_peered":"2024-03-08T23:51:52.245706+0000","last_unstale":"2024-03-08T23:52:45.411166+0000","last_undegraded":"2024-03-08T23:52:45.411166+0000","last_fullsized":"2024-03-08T23:52:45.411166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:00:44.557269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411212+0000","last_change":"2024-03-08T23:51:56.285649+0000","last_active":"2024-03-08T23:52:45.411212+0000","last_peered":"2024-03-08T23:52:45.411212+0000","last_clean":"2024-03-08T23:52:45.411212+0000","last_became_active":"2024-03-08T23:51:56.282731+0000","last_became_peered":"2024-03-08T23:51:56.282731+0000","last_unstale":"2024-03-08T23:52:45.411212+0000","last_undegraded":"2024-03-08T23:52:45.411212+0000","last_fullsized":"2024-03-08T23:52:45.411212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:23:54.103637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411148+0000","last_change":"2024-03-08T23:51:51.249442+0000","last_active":"2024-03-08T23:52:45.411148+0000","last_peered":"2024-03-08T23:52:45.411148+0000","last_clean":"2024-03-08T23:52:45.411148+0000","last_became_active":"2024-03-08T23:51:51.249084+0000","last_became_peered":"2024-03-08T23:51:51.249084+0000","last_unstale":"2024-03-08T23:52:45.411148+0000","last_undegraded":"2024-03-08T23:52:45.411148+0000","last_fullsized":"2024-03-08T23:52:45.411148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:48:19.370095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410713+0000","last_change":"2024-03-08T23:51:52.257586+0000","last_active":"2024-03-08T23:52:45.410713+0000","last_peered":"2024-03-08T23:52:45.410713+0000","last_clean":"2024-03-08T23:52:45.410713+0000","last_became_active":"2024-03-08T23:51:52.253741+0000","last_became_peered":"2024-03-08T23:51:52.253741+0000","last_unstale":"2024-03-08T23:52:45.410713+0000","last_undegraded":"2024-03-08T23:52:45.410713+0000","last_fullsized":"2024-03-08T23:52:45.410713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:18.633166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412107+0000","last_change":"2024-03-08T23:51:56.285264+0000","last_active":"2024-03-08T23:52:45.412107+0000","last_peered":"2024-03-08T23:52:45.412107+0000","last_clean":"2024-03-08T23:52:45.412107+0000","last_became_active":"2024-03-08T23:51:56.284727+0000","last_became_peered":"2024-03-08T23:51:56.284727+0000","last_unstale":"2024-03-08T23:52:45.412107+0000","last_undegraded":"2024-03-08T23:52:45.412107+0000","last_fullsized":"2024-03-08T23:52:45.412107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:01:20.723010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414290+0000","last_change":"2024-03-08T23:51:52.251461+0000","last_active":"2024-03-08T23:52:45.414290+0000","last_peered":"2024-03-08T23:52:45.414290+0000","last_clean":"2024-03-08T23:52:45.414290+0000","last_became_active":"2024-03-08T23:51:52.250996+0000","last_became_peered":"2024-03-08T23:51:52.250996+0000","last_unstale":"2024-03-08T23:52:45.414290+0000","last_undegraded":"2024-03-08T23:52:45.414290+0000","last_fullsized":"2024-03-08T23:52:45.414290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:51.719142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:53:33.458381+0000","last_change":"2024-03-08T23:51:51.245698+0000","last_active":"2024-03-08T23:53:33.458381+0000","last_peered":"2024-03-08T23:53:33.458381+0000","last_clean":"2024-03-08T23:53:33.458381+0000","last_became_active":"2024-03-08T23:51:51.245389+0000","last_became_peered":"2024-03-08T23:51:51.245389+0000","last_unstale":"2024-03-08T23:53:33.458381+0000","last_undegraded":"2024-03-08T23:53:33.458381+0000","last_fullsized":"2024-03-08T23:53:33.458381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:14:08.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.423657+0000","last_change":"2024-03-08T23:52:45.423657+0000","last_active":"2024-03-08T23:52:45.423657+0000","last_peered":"2024-03-08T23:52:45.423657+0000","last_clean":"2024-03-08T23:52:45.423657+0000","last_became_active":"2024-03-08T23:52:45.417868+0000","last_became_peered":"2024-03-08T23:52:45.417868+0000","last_unstale":"2024-03-08T23:52:45.423657+0000","last_undegraded":"2024-03-08T23:52:45.423657+0000","last_fullsized":"2024-03-08T23:52:45.423657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:34.178938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411550+0000","last_change":"2024-03-08T23:51:52.256518+0000","last_active":"2024-03-08T23:52:45.411550+0000","last_peered":"2024-03-08T23:52:45.411550+0000","last_clean":"2024-03-08T23:52:45.411550+0000","last_became_active":"2024-03-08T23:51:52.253481+0000","last_became_peered":"2024-03-08T23:51:52.253481+0000","last_unstale":"2024-03-08T23:52:45.411550+0000","last_undegraded":"2024-03-08T23:52:45.411550+0000","last_fullsized":"2024-03-08T23:52:45.411550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:35.287408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409139+0000","last_change":"2024-03-08T23:51:51.237921+0000","last_active":"2024-03-08T23:52:45.409139+0000","last_peered":"2024-03-08T23:52:45.409139+0000","last_clean":"2024-03-08T23:52:45.409139+0000","last_became_active":"2024-03-08T23:51:51.237568+0000","last_became_peered":"2024-03-08T23:51:51.237568+0000","last_unstale":"2024-03-08T23:52:45.409139+0000","last_undegraded":"2024-03-08T23:52:45.409139+0000","last_fullsized":"2024-03-08T23:52:45.409139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:11.256814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411073+0000","last_change":"2024-03-08T23:50:41.802121+0000","last_active":"2024-03-08T23:52:45.411073+0000","last_peered":"2024-03-08T23:52:45.411073+0000","last_clean":"2024-03-08T23:52:45.411073+0000","last_became_active":"2024-03-08T23:50:41.801678+0000","last_became_peered":"2024-03-08T23:50:41.801678+0000","last_unstale":"2024-03-08T23:52:45.411073+0000","last_undegraded":"2024-03-08T23:52:45.411073+0000","last_fullsized":"2024-03-08T23:52:45.411073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_clean_scrub_stamp":"2024-03-08T23:47:12.382547+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:15.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414198+0000","last_change":"2024-03-08T23:51:56.278465+0000","last_active":"2024-03-08T23:52:45.414198+0000","last_peered":"2024-03-08T23:52:45.414198+0000","last_clean":"2024-03-08T23:52:45.414198+0000","last_became_active":"2024-03-08T23:51:56.278079+0000","last_became_peered":"2024-03-08T23:51:56.278079+0000","last_unstale":"2024-03-08T23:52:45.414198+0000","last_undegraded":"2024-03-08T23:52:45.414198+0000","last_fullsized":"2024-03-08T23:52:45.414198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:47.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410331+0000","last_change":"2024-03-08T23:51:52.251895+0000","last_active":"2024-03-08T23:52:45.410331+0000","last_peered":"2024-03-08T23:52:45.410331+0000","last_clean":"2024-03-08T23:52:45.410331+0000","last_became_active":"2024-03-08T23:51:52.251591+0000","last_became_peered":"2024-03-08T23:51:52.251591+0000","last_unstale":"2024-03-08T23:52:45.410331+0000","last_undegraded":"2024-03-08T23:52:45.410331+0000","last_fullsized":"2024-03-08T23:52:45.410331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:06.276636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411107+0000","last_change":"2024-03-08T23:51:51.245981+0000","last_active":"2024-03-08T23:52:45.411107+0000","last_peered":"2024-03-08T23:52:45.411107+0000","last_clean":"2024-03-08T23:52:45.411107+0000","last_became_active":"2024-03-08T23:51:51.244681+0000","last_became_peered":"2024-03-08T23:51:51.244681+0000","last_unstale":"2024-03-08T23:52:45.411107+0000","last_undegraded":"2024-03-08T23:52:45.411107+0000","last_fullsized":"2024-03-08T23:52:45.411107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:06:21.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416298+0000","last_change":"2024-03-08T23:51:56.279057+0000","last_active":"2024-03-08T23:52:45.416298+0000","last_peered":"2024-03-08T23:52:45.416298+0000","last_clean":"2024-03-08T23:52:45.416298+0000","last_became_active":"2024-03-08T23:51:56.278634+0000","last_became_peered":"2024-03-08T23:51:56.278634+0000","last_unstale":"2024-03-08T23:52:45.416298+0000","last_undegraded":"2024-03-08T23:52:45.416298+0000","last_fullsized":"2024-03-08T23:52:45.416298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:30.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584441+0000","last_change":"2024-03-08T23:51:51.240378+0000","last_active":"2024-03-08T23:52:45.584441+0000","last_peered":"2024-03-08T23:52:45.584441+0000","last_clean":"2024-03-08T23:52:45.584441+0000","last_became_active":"2024-03-08T23:51:51.239880+0000","last_became_peered":"2024-03-08T23:51:51.239880+0000","last_unstale":"2024-03-08T23:52:45.584441+0000","last_undegraded":"2024-03-08T23:52:45.584441+0000","last_fullsized":"2024-03-08T23:52:45.584441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:20.804261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411591+0000","last_change":"2024-03-08T23:51:52.240338+0000","last_active":"2024-03-08T23:52:45.411591+0000","last_peered":"2024-03-08T23:52:45.411591+0000","last_clean":"2024-03-08T23:52:45.411591+0000","last_became_active":"2024-03-08T23:51:52.239987+0000","last_became_peered":"2024-03-08T23:51:52.239987+0000","last_unstale":"2024-03-08T23:52:45.411591+0000","last_undegraded":"2024-03-08T23:52:45.411591+0000","last_fullsized":"2024-03-08T23:52:45.411591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:26.590204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410553+0000","last_change":"2024-03-08T23:51:56.281009+0000","last_active":"2024-03-08T23:52:45.410553+0000","last_peered":"2024-03-08T23:52:45.410553+0000","last_clean":"2024-03-08T23:52:45.410553+0000","last_became_active":"2024-03-08T23:51:56.280619+0000","last_became_peered":"2024-03-08T23:51:56.280619+0000","last_unstale":"2024-03-08T23:52:45.410553+0000","last_undegraded":"2024-03-08T23:52:45.410553+0000","last_fullsized":"2024-03-08T23:52:45.410553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:06.920777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408419+0000","last_change":"2024-03-08T23:51:51.241682+0000","last_active":"2024-03-08T23:52:45.408419+0000","last_peered":"2024-03-08T23:52:45.408419+0000","last_clean":"2024-03-08T23:52:45.408419+0000","last_became_active":"2024-03-08T23:51:51.241313+0000","last_became_peered":"2024-03-08T23:51:51.241313+0000","last_unstale":"2024-03-08T23:52:45.408419+0000","last_undegraded":"2024-03-08T23:52:45.408419+0000","last_fullsized":"2024-03-08T23:52:45.408419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:04:13.515856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411826+0000","last_change":"2024-03-08T23:51:52.249378+0000","last_active":"2024-03-08T23:52:45.411826+0000","last_peered":"2024-03-08T23:52:45.411826+0000","last_clean":"2024-03-08T23:52:45.411826+0000","last_became_active":"2024-03-08T23:51:52.249084+0000","last_became_peered":"2024-03-08T23:51:52.249084+0000","last_unstale":"2024-03-08T23:52:45.411826+0000","last_undegraded":"2024-03-08T23:52:45.411826+0000","last_fullsized":"2024-03-08T23:52:45.411826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:10:36.513260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411744+0000","last_change":"2024-03-08T23:51:56.280717+0000","last_active":"2024-03-08T23:52:45.411744+0000","last_peered":"2024-03-08T23:52:45.411744+0000","last_clean":"2024-03-08T23:52:45.411744+0000","last_became_active":"2024-03-08T23:51:56.280329+0000","last_became_peered":"2024-03-08T23:51:56.280329+0000","last_unstale":"2024-03-08T23:52:45.411744+0000","last_undegraded":"2024-03-08T23:52:45.411744+0000","last_fullsized":"2024-03-08T23:52:45.411744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:47:40.829142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409898+0000","last_change":"2024-03-08T23:51:51.238058+0000","last_active":"2024-03-08T23:52:45.409898+0000","last_peered":"2024-03-08T23:52:45.409898+0000","last_clean":"2024-03-08T23:52:45.409898+0000","last_became_active":"2024-03-08T23:51:51.237727+0000","last_became_peered":"2024-03-08T23:51:51.237727+0000","last_unstale":"2024-03-08T23:52:45.409898+0000","last_undegraded":"2024-03-08T23:52:45.409898+0000","last_fullsized":"2024-03-08T23:52:45.409898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:13:54.080004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849383+0000","last_change":"2024-03-08T23:51:52.249830+0000","last_active":"2024-03-08T23:52:45.849383+0000","last_peered":"2024-03-08T23:52:45.849383+0000","last_clean":"2024-03-08T23:52:45.849383+0000","last_became_active":"2024-03-08T23:51:52.249518+0000","last_became_peered":"2024-03-08T23:51:52.249518+0000","last_unstale":"2024-03-08T23:52:45.849383+0000","last_undegraded":"2024-03-08T23:52:45.849383+0000","last_fullsized":"2024-03-08T23:52:45.849383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:24:33.721423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409483+0000","last_change":"2024-03-08T23:51:56.288595+0000","last_active":"2024-03-08T23:52:45.409483+0000","last_peered":"2024-03-08T23:52:45.409483+0000","last_clean":"2024-03-08T23:52:45.409483+0000","last_became_active":"2024-03-08T23:51:56.288146+0000","last_became_peered":"2024-03-08T23:51:56.288146+0000","last_unstale":"2024-03-08T23:52:45.409483+0000","last_undegraded":"2024-03-08T23:52:45.409483+0000","last_fullsized":"2024-03-08T23:52:45.409483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:34:58.321754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414356+0000","last_change":"2024-03-08T23:51:51.242065+0000","last_active":"2024-03-08T23:52:45.414356+0000","last_peered":"2024-03-08T23:52:45.414356+0000","last_clean":"2024-03-08T23:52:45.414356+0000","last_became_active":"2024-03-08T23:51:51.241513+0000","last_became_peered":"2024-03-08T23:51:51.241513+0000","last_unstale":"2024-03-08T23:52:45.414356+0000","last_undegraded":"2024-03-08T23:52:45.414356+0000","last_fullsized":"2024-03-08T23:52:45.414356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:08:02.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.583992+0000","last_change":"2024-03-08T23:51:52.243074+0000","last_active":"2024-03-08T23:52:45.583992+0000","last_peered":"2024-03-08T23:52:45.583992+0000","last_clean":"2024-03-08T23:52:45.583992+0000","last_became_active":"2024-03-08T23:51:52.242736+0000","last_became_peered":"2024-03-08T23:51:52.242736+0000","last_unstale":"2024-03-08T23:52:45.583992+0000","last_undegraded":"2024-03-08T23:52:45.583992+0000","last_fullsized":"2024-03-08T23:52:45.583992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:57:37.174056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.408298+0000","last_change":"2024-03-08T23:51:56.276284+0000","last_active":"2024-03-08T23:52:45.408298+0000","last_peered":"2024-03-08T23:52:45.408298+0000","last_clean":"2024-03-08T23:52:45.408298+0000","last_became_active":"2024-03-08T23:51:56.275800+0000","last_became_peered":"2024-03-08T23:51:56.275800+0000","last_unstale":"2024-03-08T23:52:45.408298+0000","last_undegraded":"2024-03-08T23:52:45.408298+0000","last_fullsized":"2024-03-08T23:52:45.408298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:21:11.221266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410772+0000","last_change":"2024-03-08T23:51:51.245548+0000","last_active":"2024-03-08T23:52:45.410772+0000","last_peered":"2024-03-08T23:52:45.410772+0000","last_clean":"2024-03-08T23:52:45.410772+0000","last_became_active":"2024-03-08T23:51:51.245238+0000","last_became_peered":"2024-03-08T23:51:51.245238+0000","last_unstale":"2024-03-08T23:52:45.410772+0000","last_undegraded":"2024-03-08T23:52:45.410772+0000","last_fullsized":"2024-03-08T23:52:45.410772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:21:19.557196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416691+0000","last_change":"2024-03-08T23:51:52.244422+0000","last_active":"2024-03-08T23:52:45.416691+0000","last_peered":"2024-03-08T23:52:45.416691+0000","last_clean":"2024-03-08T23:52:45.416691+0000","last_became_active":"2024-03-08T23:51:52.244032+0000","last_became_peered":"2024-03-08T23:51:52.244032+0000","last_unstale":"2024-03-08T23:52:45.416691+0000","last_undegraded":"2024-03-08T23:52:45.416691+0000","last_fullsized":"2024-03-08T23:52:45.416691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:04.064691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409850+0000","last_change":"2024-03-08T23:51:56.288632+0000","last_active":"2024-03-08T23:52:45.409850+0000","last_peered":"2024-03-08T23:52:45.409850+0000","last_clean":"2024-03-08T23:52:45.409850+0000","last_became_active":"2024-03-08T23:51:56.288175+0000","last_became_peered":"2024-03-08T23:51:56.288175+0000","last_unstale":"2024-03-08T23:52:45.409850+0000","last_undegraded":"2024-03-08T23:52:45.409850+0000","last_fullsized":"2024-03-08T23:52:45.409850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:54:51.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412088+0000","last_change":"2024-03-08T23:51:51.247968+0000","last_active":"2024-03-08T23:52:45.412088+0000","last_peered":"2024-03-08T23:52:45.412088+0000","last_clean":"2024-03-08T23:52:45.412088+0000","last_became_active":"2024-03-08T23:51:51.247660+0000","last_became_peered":"2024-03-08T23:51:51.247660+0000","last_unstale":"2024-03-08T23:52:45.412088+0000","last_undegraded":"2024-03-08T23:52:45.412088+0000","last_fullsized":"2024-03-08T23:52:45.412088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:40.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411265+0000","last_change":"2024-03-08T23:51:52.250778+0000","last_active":"2024-03-08T23:52:45.411265+0000","last_peered":"2024-03-08T23:52:45.411265+0000","last_clean":"2024-03-08T23:52:45.411265+0000","last_became_active":"2024-03-08T23:51:52.250475+0000","last_became_peered":"2024-03-08T23:51:52.250475+0000","last_unstale":"2024-03-08T23:52:45.411265+0000","last_undegraded":"2024-03-08T23:52:45.411265+0000","last_fullsized":"2024-03-08T23:52:45.411265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:36:01.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.409421+0000","last_change":"2024-03-08T23:51:56.285631+0000","last_active":"2024-03-08T23:52:45.409421+0000","last_peered":"2024-03-08T23:52:45.409421+0000","last_clean":"2024-03-08T23:52:45.409421+0000","last_became_active":"2024-03-08T23:51:56.285183+0000","last_became_peered":"2024-03-08T23:51:56.285183+0000","last_unstale":"2024-03-08T23:52:45.409421+0000","last_undegraded":"2024-03-08T23:52:45.409421+0000","last_fullsized":"2024-03-08T23:52:45.409421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:15.002086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.584495+0000","last_change":"2024-03-08T23:51:51.246848+0000","last_active":"2024-03-08T23:52:45.584495+0000","last_peered":"2024-03-08T23:52:45.584495+0000","last_clean":"2024-03-08T23:52:45.584495+0000","last_became_active":"2024-03-08T23:51:51.246533+0000","last_became_peered":"2024-03-08T23:51:51.246533+0000","last_unstale":"2024-03-08T23:52:45.584495+0000","last_undegraded":"2024-03-08T23:52:45.584495+0000","last_fullsized":"2024-03-08T23:52:45.584495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:20:16.669820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411374+0000","last_change":"2024-03-08T23:51:52.251575+0000","last_active":"2024-03-08T23:52:45.411374+0000","last_peered":"2024-03-08T23:52:45.411374+0000","last_clean":"2024-03-08T23:52:45.411374+0000","last_became_active":"2024-03-08T23:51:52.251232+0000","last_became_peered":"2024-03-08T23:51:52.251232+0000","last_unstale":"2024-03-08T23:52:45.411374+0000","last_undegraded":"2024-03-08T23:52:45.411374+0000","last_fullsized":"2024-03-08T23:52:45.411374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:33.573407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.586092+0000","last_change":"2024-03-08T23:52:45.586092+0000","last_active":"2024-03-08T23:52:45.586092+0000","last_peered":"2024-03-08T23:52:45.586092+0000","last_clean":"2024-03-08T23:52:45.586092+0000","last_became_active":"2024-03-08T23:52:45.585513+0000","last_became_peered":"2024-03-08T23:52:45.585513+0000","last_unstale":"2024-03-08T23:52:45.586092+0000","last_undegraded":"2024-03-08T23:52:45.586092+0000","last_fullsized":"2024-03-08T23:52:45.586092+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:39:05.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413981+0000","last_change":"2024-03-08T23:51:51.242081+0000","last_active":"2024-03-08T23:52:45.413981+0000","last_peered":"2024-03-08T23:52:45.413981+0000","last_clean":"2024-03-08T23:52:45.413981+0000","last_became_active":"2024-03-08T23:51:51.241636+0000","last_became_peered":"2024-03-08T23:51:51.241636+0000","last_unstale":"2024-03-08T23:52:45.413981+0000","last_undegraded":"2024-03-08T23:52:45.413981+0000","last_fullsized":"2024-03-08T23:52:45.413981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:51:57.497723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412235+0000","last_change":"2024-03-08T23:51:52.252796+0000","last_active":"2024-03-08T23:52:45.412235+0000","last_peered":"2024-03-08T23:52:45.412235+0000","last_clean":"2024-03-08T23:52:45.412235+0000","last_became_active":"2024-03-08T23:51:52.252361+0000","last_became_peered":"2024-03-08T23:51:52.252361+0000","last_unstale":"2024-03-08T23:52:45.412235+0000","last_undegraded":"2024-03-08T23:52:45.412235+0000","last_fullsized":"2024-03-08T23:52:45.412235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:07.968262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.848938+0000","last_change":"2024-03-08T23:51:56.284396+0000","last_active":"2024-03-08T23:52:45.848938+0000","last_peered":"2024-03-08T23:52:45.848938+0000","last_clean":"2024-03-08T23:52:45.848938+0000","last_became_active":"2024-03-08T23:51:56.283951+0000","last_became_peered":"2024-03-08T23:51:56.283951+0000","last_unstale":"2024-03-08T23:52:45.848938+0000","last_undegraded":"2024-03-08T23:52:45.848938+0000","last_fullsized":"2024-03-08T23:52:45.848938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:09:40.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414485+0000","last_change":"2024-03-08T23:51:51.248838+0000","last_active":"2024-03-08T23:52:45.414485+0000","last_peered":"2024-03-08T23:52:45.414485+0000","last_clean":"2024-03-08T23:52:45.414485+0000","last_became_active":"2024-03-08T23:51:51.248534+0000","last_became_peered":"2024-03-08T23:51:51.248534+0000","last_unstale":"2024-03-08T23:52:45.414485+0000","last_undegraded":"2024-03-08T23:52:45.414485+0000","last_fullsized":"2024-03-08T23:52:45.414485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:37:25.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411577+0000","last_change":"2024-03-08T23:51:52.264170+0000","last_active":"2024-03-08T23:52:45.411577+0000","last_peered":"2024-03-08T23:52:45.411577+0000","last_clean":"2024-03-08T23:52:45.411577+0000","last_became_active":"2024-03-08T23:51:52.263744+0000","last_became_peered":"2024-03-08T23:51:52.263744+0000","last_unstale":"2024-03-08T23:52:45.411577+0000","last_undegraded":"2024-03-08T23:52:45.411577+0000","last_fullsized":"2024-03-08T23:52:45.411577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:42:18.287699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.413828+0000","last_change":"2024-03-08T23:51:56.283316+0000","last_active":"2024-03-08T23:52:45.413828+0000","last_peered":"2024-03-08T23:52:45.413828+0000","last_clean":"2024-03-08T23:52:45.413828+0000","last_became_active":"2024-03-08T23:51:56.282515+0000","last_became_peered":"2024-03-08T23:51:56.282515+0000","last_unstale":"2024-03-08T23:52:45.413828+0000","last_undegraded":"2024-03-08T23:52:45.413828+0000","last_fullsized":"2024-03-08T23:52:45.413828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:40:15.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411644+0000","last_change":"2024-03-08T23:51:51.240970+0000","last_active":"2024-03-08T23:52:45.411644+0000","last_peered":"2024-03-08T23:52:45.411644+0000","last_clean":"2024-03-08T23:52:45.411644+0000","last_became_active":"2024-03-08T23:51:51.239982+0000","last_became_peered":"2024-03-08T23:51:51.239982+0000","last_unstale":"2024-03-08T23:52:45.411644+0000","last_undegraded":"2024-03-08T23:52:45.411644+0000","last_fullsized":"2024-03-08T23:52:45.411644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:47:40.313753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411949+0000","last_change":"2024-03-08T23:51:52.261673+0000","last_active":"2024-03-08T23:52:45.411949+0000","last_peered":"2024-03-08T23:52:45.411949+0000","last_clean":"2024-03-08T23:52:45.411949+0000","last_became_active":"2024-03-08T23:51:52.261333+0000","last_became_peered":"2024-03-08T23:51:52.261333+0000","last_unstale":"2024-03-08T23:52:45.411949+0000","last_undegraded":"2024-03-08T23:52:45.411949+0000","last_fullsized":"2024-03-08T23:52:45.411949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:08:26.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416718+0000","last_change":"2024-03-08T23:51:52.243189+0000","last_active":"2024-03-08T23:52:45.416718+0000","last_peered":"2024-03-08T23:52:45.416718+0000","last_clean":"2024-03-08T23:52:45.416718+0000","last_became_active":"2024-03-08T23:51:52.242870+0000","last_became_peered":"2024-03-08T23:51:52.242870+0000","last_unstale":"2024-03-08T23:52:45.416718+0000","last_undegraded":"2024-03-08T23:52:45.416718+0000","last_fullsized":"2024-03-08T23:52:45.416718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.438045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.417220+0000","last_change":"2024-03-08T23:52:45.417220+0000","last_active":"2024-03-08T23:52:45.417220+0000","last_peered":"2024-03-08T23:52:45.417220+0000","last_clean":"2024-03-08T23:52:45.417220+0000","last_became_active":"2024-03-08T23:52:45.416820+0000","last_became_peered":"2024-03-08T23:52:45.416820+0000","last_unstale":"2024-03-08T23:52:45.417220+0000","last_undegraded":"2024-03-08T23:52:45.417220+0000","last_fullsized":"2024-03-08T23:52:45.417220+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:31:13.249449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411814+0000","last_change":"2024-03-08T23:51:52.251397+0000","last_active":"2024-03-08T23:52:45.411814+0000","last_peered":"2024-03-08T23:52:45.411814+0000","last_clean":"2024-03-08T23:52:45.411814+0000","last_became_active":"2024-03-08T23:51:52.251059+0000","last_became_peered":"2024-03-08T23:51:52.251059+0000","last_unstale":"2024-03-08T23:52:45.411814+0000","last_undegraded":"2024-03-08T23:52:45.411814+0000","last_fullsized":"2024-03-08T23:52:45.411814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:19:30.363457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.849115+0000","last_change":"2024-03-08T23:51:52.249039+0000","last_active":"2024-03-08T23:52:45.849115+0000","last_peered":"2024-03-08T23:52:45.849115+0000","last_clean":"2024-03-08T23:52:45.849115+0000","last_became_active":"2024-03-08T23:51:52.248560+0000","last_became_peered":"2024-03-08T23:51:52.248560+0000","last_unstale":"2024-03-08T23:52:45.849115+0000","last_undegraded":"2024-03-08T23:52:45.849115+0000","last_fullsized":"2024-03-08T23:52:45.849115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.411479+0000","last_change":"2024-03-08T23:51:52.247269+0000","last_active":"2024-03-08T23:52:45.411479+0000","last_peered":"2024-03-08T23:52:45.411479+0000","last_clean":"2024-03-08T23:52:45.411479+0000","last_became_active":"2024-03-08T23:51:52.246921+0000","last_became_peered":"2024-03-08T23:51:52.246921+0000","last_unstale":"2024-03-08T23:52:45.411479+0000","last_undegraded":"2024-03-08T23:52:45.411479+0000","last_fullsized":"2024-03-08T23:52:45.411479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:00:21.828530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416365+0000","last_change":"2024-03-08T23:51:52.238999+0000","last_active":"2024-03-08T23:52:45.416365+0000","last_peered":"2024-03-08T23:52:45.416365+0000","last_clean":"2024-03-08T23:52:45.416365+0000","last_became_active":"2024-03-08T23:51:52.238662+0000","last_became_peered":"2024-03-08T23:51:52.238662+0000","last_unstale":"2024-03-08T23:52:45.416365+0000","last_undegraded":"2024-03-08T23:52:45.416365+0000","last_fullsized":"2024-03-08T23:52:45.416365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:22:08.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.412568+0000","last_change":"2024-03-08T23:51:52.238793+0000","last_active":"2024-03-08T23:52:45.412568+0000","last_peered":"2024-03-08T23:52:45.412568+0000","last_clean":"2024-03-08T23:52:45.412568+0000","last_became_active":"2024-03-08T23:51:52.238464+0000","last_became_peered":"2024-03-08T23:51:52.238464+0000","last_unstale":"2024-03-08T23:52:45.412568+0000","last_undegraded":"2024-03-08T23:52:45.412568+0000","last_fullsized":"2024-03-08T23:52:45.412568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:25.597428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.585775+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:52:45.585775+0000","last_peered":"2024-03-08T23:52:45.585775+0000","last_clean":"2024-03-08T23:52:45.585775+0000","last_became_active":"2024-03-08T23:52:45.585280+0000","last_became_peered":"2024-03-08T23:52:45.585280+0000","last_unstale":"2024-03-08T23:52:45.585775+0000","last_undegraded":"2024-03-08T23:52:45.585775+0000","last_fullsized":"2024-03-08T23:52:45.585775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:20.029967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414171+0000","last_change":"2024-03-08T23:51:52.245288+0000","last_active":"2024-03-08T23:52:45.414171+0000","last_peered":"2024-03-08T23:52:45.414171+0000","last_clean":"2024-03-08T23:52:45.414171+0000","last_became_active":"2024-03-08T23:51:52.244960+0000","last_became_peered":"2024-03-08T23:51:52.244960+0000","last_unstale":"2024-03-08T23:52:45.414171+0000","last_undegraded":"2024-03-08T23:52:45.414171+0000","last_fullsized":"2024-03-08T23:52:45.414171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:20:39.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.418510+0000","last_change":"2024-03-08T23:52:45.418172+0000","last_active":"2024-03-08T23:52:45.418510+0000","last_peered":"2024-03-08T23:52:45.418510+0000","last_clean":"2024-03-08T23:52:45.418510+0000","last_became_active":"2024-03-08T23:52:45.417852+0000","last_became_peered":"2024-03-08T23:52:45.417852+0000","last_unstale":"2024-03-08T23:52:45.418510+0000","last_undegraded":"2024-03-08T23:52:45.418510+0000","last_fullsized":"2024-03-08T23:52:45.418510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:22:23.698238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.414454+0000","last_change":"2024-03-08T23:51:52.251048+0000","last_active":"2024-03-08T23:52:45.414454+0000","last_peered":"2024-03-08T23:52:45.414454+0000","last_clean":"2024-03-08T23:52:45.414454+0000","last_became_active":"2024-03-08T23:51:52.250508+0000","last_became_peered":"2024-03-08T23:51:52.250508+0000","last_unstale":"2024-03-08T23:52:45.414454+0000","last_undegraded":"2024-03-08T23:52:45.414454+0000","last_fullsized":"2024-03-08T23:52:45.414454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:57:37.519307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.410759+0000","last_change":"2024-03-08T23:51:52.244203+0000","last_active":"2024-03-08T23:52:45.410759+0000","last_peered":"2024-03-08T23:52:45.410759+0000","last_clean":"2024-03-08T23:52:45.410759+0000","last_became_active":"2024-03-08T23:51:52.243878+0000","last_became_peered":"2024-03-08T23:51:52.243878+0000","last_unstale":"2024-03-08T23:52:45.410759+0000","last_undegraded":"2024-03-08T23:52:45.410759+0000","last_fullsized":"2024-03-08T23:52:45.410759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:53:40.938657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-08T23:52:45.416373+0000","last_change":"2024-03-08T23:51:52.248334+0000","last_active":"2024-03-08T23:52:45.416373+0000","last_peered":"2024-03-08T23:52:45.416373+0000","last_clean":"2024-03-08T23:52:45.416373+0000","last_became_active":"2024-03-08T23:51:52.248025+0000","last_became_peered":"2024-03-08T23:51:52.248025+0000","last_unstale":"2024-03-08T23:52:45.416373+0000","last_undegraded":"2024-03-08T23:52:45.416373+0000","last_fullsized":"2024-03-08T23:52:45.416373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:02:06.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66725,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":471,"num_read_kb":307,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60681,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1720320,"data_stored":1154911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":501059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":500979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1097279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":496235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":558478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":497623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":497886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":496429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":563609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T23:53:38.443 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-08T23:53:38.443 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-08T23:53:38.443 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-08T23:53:38.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph health --format=json 2024-03-08T23:53:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:38.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:38 smithi062 ceph-mon[36129]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:53:38.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:38 smithi192 ceph-mon[35995]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:53:38.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:38 smithi022 ceph-mon[31894]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:53:39.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:39 smithi022 ceph-mon[31894]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:39.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:39 smithi062 ceph-mon[36129]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:39.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:39 smithi192 ceph-mon[35995]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:40.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 -- 172.21.15.22:0/3686677320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 msgr2=0x7f1c2c10ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:40.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 --2- 172.21.15.22:0/3686677320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c10ecd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c240099c0 tx=0x7f1c24009cd0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 -- 172.21.15.22:0/3686677320 shutdown_connections 2024-03-08T23:53:40.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 --2- 172.21.15.22:0/3686677320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c10ecd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 --2- 172.21.15.22:0/3686677320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c2c102020 0x7f1c2c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 --2- 172.21.15.22:0/3686677320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c2c1029d0 0x7f1c2c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 -- 172.21.15.22:0/3686677320 >> 172.21.15.22:0/3686677320 conn(0x7f1c2c0fd890 msgr2=0x7f1c2c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 -- 172.21.15.22:0/3686677320 shutdown_connections 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.402+0000 7f1c31448700 1 -- 172.21.15.22:0/3686677320 wait complete. 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 Processor -- start 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 -- start start 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c2c102020 0x7f1c2c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c2c1029d0 0x7f1c2c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c076080 con 0x7f1c2c107100 2024-03-08T23:53:40.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c075f00 con 0x7f1c2c1029d0 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.403+0000 7f1c31448700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c1100f0 con 0x7f1c2c102020 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60332/0 (socket says 172.21.15.22:60332) 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 -- 172.21.15.22:0/1924280795 learned_addr learned my addr 172.21.15.22:0/1924280795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2affd700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c2c102020 0x7f1c2c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2a7fc700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c2c1029d0 0x7f1c2c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 -- 172.21.15.22:0/1924280795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c2c102020 msgr2=0x7f1c2c196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c2c102020 0x7f1c2c196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 -- 172.21.15.22:0/1924280795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c2c1029d0 msgr2=0x7f1c2c196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c2c1029d0 0x7f1c2c196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 -- 172.21.15.22:0/1924280795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c19b3e0 con 0x7f1c2c107100 2024-03-08T23:53:40.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2a7fc700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c2c1029d0 0x7f1c2c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:40.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2affd700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c2c102020 0x7f1c2c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:40.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c2b7fe700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c19aca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c240059d0 tx=0x7f1c24009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:40.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c23fff700 1 -- 172.21.15.22:0/1924280795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24027070 con 0x7f1c2c107100 2024-03-08T23:53:40.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c23fff700 1 -- 172.21.15.22:0/1924280795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c24004050 con 0x7f1c2c107100 2024-03-08T23:53:40.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c19b670 con 0x7f1c2c107100 2024-03-08T23:53:40.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c23fff700 1 -- 172.21.15.22:0/1924280795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24022db0 con 0x7f1c2c107100 2024-03-08T23:53:40.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.404+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c1a2f50 con 0x7f1c2c107100 2024-03-08T23:53:40.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.405+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c103500 con 0x7f1c2c107100 2024-03-08T23:53:40.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.406+0000 7f1c23fff700 1 -- 172.21.15.22:0/1924280795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1c2402a070 con 0x7f1c2c107100 2024-03-08T23:53:40.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.407+0000 7f1c23fff700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1c180a6d30 0x7f1c180a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:40.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.407+0000 7f1c2affd700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1c180a6d30 0x7f1c180a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:40.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.407+0000 7f1c23fff700 1 -- 172.21.15.22:0/1924280795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f1c240cfe30 con 0x7f1c2c107100 2024-03-08T23:53:40.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.408+0000 7f1c2affd700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1c180a6d30 0x7f1c180a91f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009dd0 tx=0x7f1c1c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:40.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.411+0000 7f1c23fff700 1 -- 172.21.15.22:0/1924280795 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1c2405e260 con 0x7f1c2c107100 2024-03-08T23:53:40.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.664+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1c2c02d830 con 0x7f1c2c107100 2024-03-08T23:53:40.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.665+0000 7f1c23fff700 1 -- 172.21.15.22:0/1924280795 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1c24013070 con 0x7f1c2c107100 2024-03-08T23:53:40.662 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:40.662 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-08T23:53:40.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1c180a6d30 msgr2=0x7f1c180a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:40.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1c180a6d30 0x7f1c180a91f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009dd0 tx=0x7f1c1c009040 comp rx=0 tx=0).stop 2024-03-08T23:53:40.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 msgr2=0x7f1c2c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:40.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c19aca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c240059d0 tx=0x7f1c24009f80 comp rx=0 tx=0).stop 2024-03-08T23:53:40.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 shutdown_connections 2024-03-08T23:53:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1c180a6d30 0x7f1c180a91f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c2c107100 0x7f1c2c19aca0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c2c102020 0x7f1c2c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 --2- 172.21.15.22:0/1924280795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c2c1029d0 0x7f1c2c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 >> 172.21.15.22:0/1924280795 conn(0x7f1c2c0fd890 msgr2=0x7f1c2c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 shutdown_connections 2024-03-08T23:53:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:40.667+0000 7f1c31448700 1 -- 172.21.15.22:0/1924280795 wait complete. 2024-03-08T23:53:41.435 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-08T23:53:41.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:53:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 -- 172.21.15.22:0/2863766722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a81024f0 msgr2=0x7ff1a8104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/2863766722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a81024f0 0x7ff1a8104960 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff198009900 tx=0x7ff19802bc30 comp rx=0 tx=0).stop 2024-03-08T23:53:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 -- 172.21.15.22:0/2863766722 shutdown_connections 2024-03-08T23:53:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/2863766722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1a8104ea0 0x7ff1a8105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/2863766722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a81024f0 0x7ff1a8104960 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/2863766722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a80ffb80 0x7ff1a8101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 -- 172.21.15.22:0/2863766722 >> 172.21.15.22:0/2863766722 conn(0x7ff1a80fb780 msgr2=0x7ff1a80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 -- 172.21.15.22:0/2863766722 shutdown_connections 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 -- 172.21.15.22:0/2863766722 wait complete. 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 Processor -- start 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 -- start start 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1a80ffb80 0x7ff1a819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.553+0000 7ff1afc9a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a81024f0 0x7ff1a819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1afc9a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8104ea0 0x7ff1a81a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1afc9a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8113580 con 0x7ff1a8104ea0 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1afc9a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8113400 con 0x7ff1a81024f0 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1afc9a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8113700 con 0x7ff1a80ffb80 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8104ea0 0x7ff1a81a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ada36640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1a80ffb80 0x7ff1a819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8104ea0 0x7ff1a81a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44582/0 (socket says 172.21.15.22:44582) 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 -- 172.21.15.22:0/192735887 learned_addr learned my addr 172.21.15.22:0/192735887 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ad235640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a81024f0 0x7ff1a819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 -- 172.21.15.22:0/192735887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1a80ffb80 msgr2=0x7ff1a819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1a80ffb80 0x7ff1a819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 -- 172.21.15.22:0/192735887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a81024f0 msgr2=0x7ff1a819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a81024f0 0x7ff1a819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 -- 172.21.15.22:0/192735887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a81a5810 con 0x7ff1a8104ea0 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ada36640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1a80ffb80 0x7ff1a819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1ae237640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8104ea0 0x7ff1a81a51a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000eb10 tx=0x7ff1a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff19effd640 1 -- 172.21.15.22:0/192735887 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0004b20 con 0x7ff1a8104ea0 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff19effd640 1 -- 172.21.15.22:0/192735887 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1a0004cc0 con 0x7ff1a8104ea0 2024-03-08T23:53:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff19effd640 1 -- 172.21.15.22:0/192735887 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0010760 con 0x7ff1a8104ea0 2024-03-08T23:53:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a81a5b00 con 0x7ff1a8104ea0 2024-03-08T23:53:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.554+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a81a6020 con 0x7ff1a8104ea0 2024-03-08T23:53:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.556+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a80fd270 con 0x7ff1a8104ea0 2024-03-08T23:53:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.557+0000 7ff19effd640 1 -- 172.21.15.22:0/192735887 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff1a0005020 con 0x7ff1a8104ea0 2024-03-08T23:53:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.557+0000 7ff19effd640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1940a2890 0x7ff1940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.557+0000 7ff1ada36640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1940a2890 0x7ff1940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.557+0000 7ff19effd640 1 -- 172.21.15.22:0/192735887 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff1a00cf610 con 0x7ff1a8104ea0 2024-03-08T23:53:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.558+0000 7ff1ada36640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1940a2890 0x7ff1940a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400bfd0 tx=0x7ff1a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:41.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.562+0000 7ff19effd640 1 -- 172.21.15.22:0/192735887 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff1a009c1f0 con 0x7ff1a8104ea0 2024-03-08T23:53:41.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:41 smithi022 ceph-mon[31894]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:41.722 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:41 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1924280795' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-08T23:53:41.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.749+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1a802d090 con 0x7ff1a8104ea0 2024-03-08T23:53:41.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.751+0000 7ff19effd640 1 -- 172.21.15.22:0/192735887 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24504 (secure 0 0 0) 0x7ff1a009baf0 con 0x7ff1a8104ea0 2024-03-08T23:53:41.747 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:41.748 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":91,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:52:45.399861+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:53:41.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1940a2890 msgr2=0x7ff1940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1940a2890 0x7ff1940a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400bfd0 tx=0x7ff1a400b390 comp rx=0 tx=0).stop 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8104ea0 msgr2=0x7ff1a81a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8104ea0 0x7ff1a81a51a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000eb10 tx=0x7ff1a000c2d0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 shutdown_connections 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff1940a2890 0x7ff1940a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1a8104ea0 0x7ff1a81a51a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a81024f0 0x7ff1a819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 --2- 172.21.15.22:0/192735887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1a80ffb80 0x7ff1a819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 >> 172.21.15.22:0/192735887 conn(0x7ff1a80fb780 msgr2=0x7ff1a8100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 shutdown_connections 2024-03-08T23:53:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.753+0000 7ff1afc9a640 1 -- 172.21.15.22:0/192735887 wait complete. 2024-03-08T23:53:41.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:53:41.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:53:41.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:41 smithi062 ceph-mon[36129]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:41.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:41 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1924280795' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 -- 172.21.15.22:0/2474847058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 msgr2=0x7f4f541040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 --2- 172.21.15.22:0/2474847058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541040a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009900 tx=0x7f4f4402bc30 comp rx=0 tx=0).stop 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 -- 172.21.15.22:0/2474847058 shutdown_connections 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 --2- 172.21.15.22:0/2474847058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f54106f10 0x7f4f54109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 --2- 172.21.15.22:0/2474847058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f541045e0 0x7f4f541069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 --2- 172.21.15.22:0/2474847058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541040a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 -- 172.21.15.22:0/2474847058 >> 172.21.15.22:0/2474847058 conn(0x7f4f540fb6a0 msgr2=0x7f4f540fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 -- 172.21.15.22:0/2474847058 shutdown_connections 2024-03-08T23:53:41.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.874+0000 7f4f593ff640 1 -- 172.21.15.22:0/2474847058 wait complete. 2024-03-08T23:53:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 Processor -- start 2024-03-08T23:53:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 -- start start 2024-03-08T23:53:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f541045e0 0x7f4f541979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f54106f10 0x7f4f5419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5410cec0 con 0x7f4f54101cb0 2024-03-08T23:53:41.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5410cd40 con 0x7f4f541045e0 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f593ff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5410d040 con 0x7f4f54106f10 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f52ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f52ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44602/0 (socket says 172.21.15.22:44602) 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f52ffd640 1 -- 172.21.15.22:0/139684312 learned_addr learned my addr 172.21.15.22:0/139684312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.875+0000 7f4f527fc640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f541045e0 0x7f4f541979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f537fe640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f54106f10 0x7f4f5419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f52ffd640 1 -- 172.21.15.22:0/139684312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f54106f10 msgr2=0x7f4f5419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f52ffd640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f54106f10 0x7f4f5419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f52ffd640 1 -- 172.21.15.22:0/139684312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f541045e0 msgr2=0x7f4f541979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f52ffd640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f541045e0 0x7f4f541979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f52ffd640 1 -- 172.21.15.22:0/139684312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5419f0d0 con 0x7f4f54101cb0 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f537fe640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f54106f10 0x7f4f5419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f52ffd640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541974b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009230 tx=0x7f4f44005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:41.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f3bfff640 1 -- 172.21.15.22:0/139684312 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44041070 con 0x7f4f54101cb0 2024-03-08T23:53:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5419f360 con 0x7f4f54101cb0 2024-03-08T23:53:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f3bfff640 1 -- 172.21.15.22:0/139684312 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f440044d0 con 0x7f4f54101cb0 2024-03-08T23:53:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f3bfff640 1 -- 172.21.15.22:0/139684312 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44034440 con 0x7f4f54101cb0 2024-03-08T23:53:41.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.876+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5419f8a0 con 0x7f4f54101cb0 2024-03-08T23:53:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.877+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f540fd190 con 0x7f4f54101cb0 2024-03-08T23:53:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.878+0000 7f4f3bfff640 1 -- 172.21.15.22:0/139684312 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f4f44044070 con 0x7f4f54101cb0 2024-03-08T23:53:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.879+0000 7f4f3bfff640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4f3c0a2890 0x7f4f3c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:41.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.879+0000 7f4f3bfff640 1 -- 172.21.15.22:0/139684312 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4f440f0eb0 con 0x7f4f54101cb0 2024-03-08T23:53:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.879+0000 7f4f527fc640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4f3c0a2890 0x7f4f3c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:41.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.880+0000 7f4f527fc640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4f3c0a2890 0x7f4f3c0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f541989c0 tx=0x7f4f4800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:41.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:41.883+0000 7f4f3bfff640 1 -- 172.21.15.22:0/139684312 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4f440bdb40 con 0x7f4f54101cb0 2024-03-08T23:53:41.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:41 smithi192 ceph-mon[35995]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:53:41.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:41 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1924280795' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-08T23:53:42.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.069+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f54061ef0 con 0x7f4f54101cb0 2024-03-08T23:53:42.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.069+0000 7f4f3bfff640 1 -- 172.21.15.22:0/139684312 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f440bd440 con 0x7f4f54101cb0 2024-03-08T23:53:42.066 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4f3c0a2890 msgr2=0x7f4f3c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4f3c0a2890 0x7f4f3c0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f541989c0 tx=0x7f4f4800a2d0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 msgr2=0x7f4f541974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541974b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009230 tx=0x7f4f44005d50 comp rx=0 tx=0).stop 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 shutdown_connections 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f54106f10 0x7f4f5419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f541045e0 0x7f4f541979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4f3c0a2890 0x7f4f3c0a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 --2- 172.21.15.22:0/139684312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f54101cb0 0x7f4f541974b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 >> 172.21.15.22:0/139684312 conn(0x7f4f540fb6a0 msgr2=0x7f4f54102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.071+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 shutdown_connections 2024-03-08T23:53:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.072+0000 7f4f593ff640 1 -- 172.21.15.22:0/139684312 wait complete. 2024-03-08T23:53:42.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:53:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- 172.21.15.22:0/2919630392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 msgr2=0x7ff544102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 --2- 172.21.15.22:0/2919630392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544102970 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff530009900 tx=0x7ff53002bc30 comp rx=0 tx=0).stop 2024-03-08T23:53:42.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- 172.21.15.22:0/2919630392 shutdown_connections 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 --2- 172.21.15.22:0/2919630392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff544102eb0 0x7ff544109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 --2- 172.21.15.22:0/2919630392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544102970 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 --2- 172.21.15.22:0/2919630392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff544101300 0x7ff544101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- 172.21.15.22:0/2919630392 >> 172.21.15.22:0/2919630392 conn(0x7ff5440fc870 msgr2=0x7ff5440fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- 172.21.15.22:0/2919630392 shutdown_connections 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- 172.21.15.22:0/2919630392 wait complete. 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 Processor -- start 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- start start 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff544101300 0x7ff5441975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff544102eb0 0x7ff54419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54410cf90 con 0x7ff5441024f0 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54410ce10 con 0x7ff544102eb0 2024-03-08T23:53:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.190+0000 7ff54aca4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54410d110 con 0x7ff544101300 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44620/0 (socket says 172.21.15.22:44620) 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 -- 172.21.15.22:0/1607034828 learned_addr learned my addr 172.21.15.22:0/1607034828 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff548a40640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff544101300 0x7ff5441975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff549241640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff544102eb0 0x7ff54419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 -- 172.21.15.22:0/1607034828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff544101300 msgr2=0x7ff5441975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff544101300 0x7ff5441975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 -- 172.21.15.22:0/1607034828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff544102eb0 msgr2=0x7ff54419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff544102eb0 0x7ff54419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 -- 172.21.15.22:0/1607034828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54419f210 con 0x7ff5441024f0 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff543fff640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544197b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff530009230 tx=0x7ff530005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff541ffb640 1 -- 172.21.15.22:0/1607034828 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530041070 con 0x7ff5441024f0 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54419f4a0 con 0x7ff5441024f0 2024-03-08T23:53:42.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff541ffb640 1 -- 172.21.15.22:0/1607034828 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5300054e0 con 0x7ff5441024f0 2024-03-08T23:53:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.191+0000 7ff541ffb640 1 -- 172.21.15.22:0/1607034828 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530034440 con 0x7ff5441024f0 2024-03-08T23:53:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.192+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54419f9e0 con 0x7ff5441024f0 2024-03-08T23:53:42.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.193+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5441086f0 con 0x7ff5441024f0 2024-03-08T23:53:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.194+0000 7ff541ffb640 1 -- 172.21.15.22:0/1607034828 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff530044070 con 0x7ff5441024f0 2024-03-08T23:53:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.195+0000 7ff541ffb640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5340a2890 0x7ff5340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.195+0000 7ff548a40640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5340a2890 0x7ff5340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.195+0000 7ff541ffb640 1 -- 172.21.15.22:0/1607034828 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff5300f0af0 con 0x7ff5441024f0 2024-03-08T23:53:42.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.195+0000 7ff548a40640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5340a2890 0x7ff5340a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff544102350 tx=0x7ff53800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:42.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.199+0000 7ff541ffb640 1 -- 172.21.15.22:0/1607034828 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff5300bd130 con 0x7ff5441024f0 2024-03-08T23:53:42.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.382+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff54402ce70 con 0x7ff5441024f0 2024-03-08T23:53:42.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.383+0000 7ff541ffb640 1 -- 172.21.15.22:0/1607034828 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5300c11d0 con 0x7ff5441024f0 2024-03-08T23:53:42.379 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:53:42.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5340a2890 msgr2=0x7ff5340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5340a2890 0x7ff5340a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff544102350 tx=0x7ff53800b390 comp rx=0 tx=0).stop 2024-03-08T23:53:42.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 msgr2=0x7ff544197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544197b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff530009230 tx=0x7ff530005dd0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 shutdown_connections 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff544102eb0 0x7ff54419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff5340a2890 0x7ff5340a4d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5441024f0 0x7ff544197b30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 --2- 172.21.15.22:0/1607034828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff544101300 0x7ff5441975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 >> 172.21.15.22:0/1607034828 conn(0x7ff5440fc870 msgr2=0x7ff5441058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 shutdown_connections 2024-03-08T23:53:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.385+0000 7ff54aca4640 1 -- 172.21.15.22:0/1607034828 wait complete. 2024-03-08T23:53:42.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:53:42.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 -- 172.21.15.22:0/3326555258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708101dc0 msgr2=0x7fb708108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 --2- 172.21.15.22:0/3326555258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708101dc0 0x7fb708108650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb700009ca0 tx=0x7fb70002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 -- 172.21.15.22:0/3326555258 shutdown_connections 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 --2- 172.21.15.22:0/3326555258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708101dc0 0x7fb708108650 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 --2- 172.21.15.22:0/3326555258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb708101400 0x7fb708101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 --2- 172.21.15.22:0/3326555258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb708100210 0x7fb708100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 -- 172.21.15.22:0/3326555258 >> 172.21.15.22:0/3326555258 conn(0x7fb7080fb7a0 msgr2=0x7fb7080fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 -- 172.21.15.22:0/3326555258 shutdown_connections 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.502+0000 7fb70d569640 1 -- 172.21.15.22:0/3326555258 wait complete. 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 Processor -- start 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 -- start start 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708100210 0x7fb7081041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb708101400 0x7fb7081027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb708101dc0 0x7fb708102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70810bea0 con 0x7fb708100210 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70810bd20 con 0x7fb708101dc0 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb70d569640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70810c020 con 0x7fb708101400 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708100210 0x7fb7081041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708100210 0x7fb7081041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44636/0 (socket says 172.21.15.22:44636) 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 -- 172.21.15.22:0/925536956 learned_addr learned my addr 172.21.15.22:0/925536956 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb7077fe640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb708101dc0 0x7fb708102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 -- 172.21.15.22:0/925536956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb708101400 msgr2=0x7fb7081027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb708101400 0x7fb7081027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 -- 172.21.15.22:0/925536956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb708101dc0 msgr2=0x7fb708102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb708101dc0 0x7fb708102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.503+0000 7fb706ffd640 1 -- 172.21.15.22:0/925536956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7081035b0 con 0x7fb708100210 2024-03-08T23:53:42.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.504+0000 7fb706ffd640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708100210 0x7fb7081041a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800d950 tx=0x7fb6f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:42.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.504+0000 7fb6effff640 1 -- 172.21.15.22:0/925536956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8017070 con 0x7fb708100210 2024-03-08T23:53:42.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.504+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7081a1730 con 0x7fb708100210 2024-03-08T23:53:42.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.504+0000 7fb6effff640 1 -- 172.21.15.22:0/925536956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6f8009c30 con 0x7fb708100210 2024-03-08T23:53:42.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.504+0000 7fb6effff640 1 -- 172.21.15.22:0/925536956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f800f600 con 0x7fb708100210 2024-03-08T23:53:42.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.504+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7081a1c80 con 0x7fb708100210 2024-03-08T23:53:42.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.505+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7081a1f40 con 0x7fb708100210 2024-03-08T23:53:42.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.506+0000 7fb6effff640 1 -- 172.21.15.22:0/925536956 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb6f801a070 con 0x7fb708100210 2024-03-08T23:53:42.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.507+0000 7fb6effff640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6f00a2890 0x7fb6f00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.507+0000 7fb7067fc640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6f00a2890 0x7fb6f00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.507+0000 7fb6effff640 1 -- 172.21.15.22:0/925536956 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb6f80ceaa0 con 0x7fb708100210 2024-03-08T23:53:42.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.507+0000 7fb7067fc640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6f00a2890 0x7fb6f00a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0058c0 tx=0x7fb6fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:42.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.511+0000 7fb6effff640 1 -- 172.21.15.22:0/925536956 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb6f8009de0 con 0x7fb708100210 2024-03-08T23:53:42.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:42 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/192735887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:42.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:42 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/139684312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:53:42.691 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:42 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1607034828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:53:42.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.696+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7081a2110 con 0x7fb708100210 2024-03-08T23:53:42.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.696+0000 7fb6effff640 1 -- 172.21.15.22:0/925536956 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6f809b620 con 0x7fb708100210 2024-03-08T23:53:42.692 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:53:42.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6f00a2890 msgr2=0x7fb6f00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6f00a2890 0x7fb6f00a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0058c0 tx=0x7fb6fc005850 comp rx=0 tx=0).stop 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708100210 msgr2=0x7fb7081041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708100210 0x7fb7081041a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800d950 tx=0x7fb6f800de10 comp rx=0 tx=0).stop 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 shutdown_connections 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb708101dc0 0x7fb708102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb708101400 0x7fb7081027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb6f00a2890 0x7fb6f00a4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 --2- 172.21.15.22:0/925536956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb708100210 0x7fb7081041a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 >> 172.21.15.22:0/925536956 conn(0x7fb7080fb7a0 msgr2=0x7fb7080fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 shutdown_connections 2024-03-08T23:53:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.698+0000 7fb70d569640 1 -- 172.21.15.22:0/925536956 wait complete. 2024-03-08T23:53:42.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:53:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.814+0000 7fd90316f640 1 -- 172.21.15.22:0/1906799114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1001f0 msgr2=0x7fd8fc100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.814+0000 7fd90316f640 1 --2- 172.21.15.22:0/1906799114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1001f0 0x7fd8fc100610 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009900 tx=0x7fd8f402bc30 comp rx=0 tx=0).stop 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.814+0000 7fd90316f640 1 -- 172.21.15.22:0/1906799114 shutdown_connections 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.814+0000 7fd90316f640 1 --2- 172.21.15.22:0/1906799114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8fc101da0 0x7fd8fc108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.814+0000 7fd90316f640 1 --2- 172.21.15.22:0/1906799114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc1013e0 0x7fd8fc101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.814+0000 7fd90316f640 1 --2- 172.21.15.22:0/1906799114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1001f0 0x7fd8fc100610 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.814+0000 7fd90316f640 1 -- 172.21.15.22:0/1906799114 >> 172.21.15.22:0/1906799114 conn(0x7fd8fc0fb780 msgr2=0x7fd8fc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 -- 172.21.15.22:0/1906799114 shutdown_connections 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 -- 172.21.15.22:0/1906799114 wait complete. 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 Processor -- start 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 -- start start 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc1001f0 0x7fd8fc068fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1013e0 0x7fd8fc19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8fc101da0 0x7fd8fc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc10bf40 con 0x7fd8fc1013e0 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc10bdc0 con 0x7fd8fc1001f0 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90316f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc10c0c0 con 0x7fd8fc101da0 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd8fbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1013e0 0x7fd8fc19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd8fbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1013e0 0x7fd8fc19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44658/0 (socket says 172.21.15.22:44658) 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd8fbfff640 1 -- 172.21.15.22:0/3400573319 learned_addr learned my addr 172.21.15.22:0/3400573319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd900f0b640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc1001f0 0x7fd8fc068fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.815+0000 7fd90170c640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8fc101da0 0x7fd8fc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8fbfff640 1 -- 172.21.15.22:0/3400573319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8fc101da0 msgr2=0x7fd8fc19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8fbfff640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8fc101da0 0x7fd8fc19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8fbfff640 1 -- 172.21.15.22:0/3400573319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc1001f0 msgr2=0x7fd8fc068fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:42.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8fbfff640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc1001f0 0x7fd8fc068fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8fbfff640 1 -- 172.21.15.22:0/3400573319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc19f3a0 con 0x7fd8fc1013e0 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd90170c640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8fc101da0 0x7fd8fc19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8fbfff640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1013e0 0x7fd8fc19c470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00ba50 tx=0x7fd8ec00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8f9ffb640 1 -- 172.21.15.22:0/3400573319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec019070 con 0x7fd8fc1013e0 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc19f630 con 0x7fd8fc1013e0 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8f9ffb640 1 -- 172.21.15.22:0/3400573319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8ec00cbe0 con 0x7fd8fc1013e0 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd8f9ffb640 1 -- 172.21.15.22:0/3400573319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0076e0 con 0x7fd8fc1013e0 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.816+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc19fbd0 con 0x7fd8fc1013e0 2024-03-08T23:53:42.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.817+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc107590 con 0x7fd8fc1013e0 2024-03-08T23:53:42.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.818+0000 7fd8f9ffb640 1 -- 172.21.15.22:0/3400573319 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd8ec01c070 con 0x7fd8fc1013e0 2024-03-08T23:53:42.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.819+0000 7fd8f9ffb640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8e80a2890 0x7fd8e80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:42.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.819+0000 7fd900f0b640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8e80a2890 0x7fd8e80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:42.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.819+0000 7fd8f9ffb640 1 -- 172.21.15.22:0/3400573319 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd8ec0ce350 con 0x7fd8fc1013e0 2024-03-08T23:53:42.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.819+0000 7fd900f0b640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8e80a2890 0x7fd8e80a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4000c00 tx=0x7fd8f4004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:42.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:42.823+0000 7fd8f9ffb640 1 -- 172.21.15.22:0/3400573319 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd8ec09aa90 con 0x7fd8fc1013e0 2024-03-08T23:53:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:42 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/192735887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:42 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/139684312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:53:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:42 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1607034828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:53:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:42 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/192735887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:42 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/139684312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:53:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:42 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1607034828' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:53:43.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.009+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd8fc02d070 con 0x7fd8fc1013e0 2024-03-08T23:53:43.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.009+0000 7fd8f9ffb640 1 -- 172.21.15.22:0/3400573319 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd8ec09eb30 con 0x7fd8fc1013e0 2024-03-08T23:53:43.005 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8e80a2890 msgr2=0x7fd8e80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8e80a2890 0x7fd8e80a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4000c00 tx=0x7fd8f4004f10 comp rx=0 tx=0).stop 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1013e0 msgr2=0x7fd8fc19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1013e0 0x7fd8fc19c470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00ba50 tx=0x7fd8ec00bf10 comp rx=0 tx=0).stop 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 shutdown_connections 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8fc101da0 0x7fd8fc19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8e80a2890 0x7fd8e80a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8fc1013e0 0x7fd8fc19c470 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 --2- 172.21.15.22:0/3400573319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc1001f0 0x7fd8fc068fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 >> 172.21.15.22:0/3400573319 conn(0x7fd8fc0fb780 msgr2=0x7fd8fc104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 shutdown_connections 2024-03-08T23:53:43.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.011+0000 7fd90316f640 1 -- 172.21.15.22:0/3400573319 wait complete. 2024-03-08T23:53:43.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:53:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.124+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2919770254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 msgr2=0x7f5fd0104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.124+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2919770254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0104180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009900 tx=0x7f5fc802bc30 comp rx=0 tx=0).stop 2024-03-08T23:53:43.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.124+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2919770254 shutdown_connections 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.124+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2919770254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd0106ff0 0x7f5fd0109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.124+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2919770254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd01046c0 0x7f5fd0106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.124+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2919770254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0104180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.124+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2919770254 >> 172.21.15.22:0/2919770254 conn(0x7f5fd00fb780 msgr2=0x7f5fd00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2919770254 shutdown_connections 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2919770254 wait complete. 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 Processor -- start 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 -- start start 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd01046c0 0x7f5fd0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0106ff0 0x7f5fd019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd010cff0 con 0x7f5fd0101d90 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd010ce70 con 0x7f5fd01046c0 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd78ca640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd010d170 con 0x7f5fd0106ff0 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd5666640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd5666640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44676/0 (socket says 172.21.15.22:44676) 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.125+0000 7f5fd5666640 1 -- 172.21.15.22:0/2143045579 learned_addr learned my addr 172.21.15.22:0/2143045579 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5e67640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0106ff0 0x7f5fd019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd4e65640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd01046c0 0x7f5fd0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5666640 1 -- 172.21.15.22:0/2143045579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0106ff0 msgr2=0x7f5fd019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5666640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0106ff0 0x7f5fd019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5666640 1 -- 172.21.15.22:0/2143045579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd01046c0 msgr2=0x7f5fd0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5666640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd01046c0 0x7f5fd0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5666640 1 -- 172.21.15.22:0/2143045579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd019f2b0 con 0x7f5fd0101d90 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5e67640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0106ff0 0x7f5fd019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd5666640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0197690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005ab0 tx=0x7f5fc802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fce7fc640 1 -- 172.21.15.22:0/2143045579 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8043020 con 0x7f5fd0101d90 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd019f540 con 0x7f5fd0101d90 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fce7fc640 1 -- 172.21.15.22:0/2143045579 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc80054e0 con 0x7f5fd0101d90 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fce7fc640 1 -- 172.21.15.22:0/2143045579 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8003ce0 con 0x7f5fd0101d90 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.126+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd019fa90 con 0x7f5fd0101d90 2024-03-08T23:53:43.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.127+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd00fd270 con 0x7f5fd0101d90 2024-03-08T23:53:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.128+0000 7f5fce7fc640 1 -- 172.21.15.22:0/2143045579 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f5fc8046020 con 0x7f5fd0101d90 2024-03-08T23:53:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.129+0000 7f5fce7fc640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5fb80a2890 0x7f5fb80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.129+0000 7f5fd4e65640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5fb80a2890 0x7f5fb80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.129+0000 7f5fce7fc640 1 -- 172.21.15.22:0/2143045579 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f5fc80f1490 con 0x7f5fd0101d90 2024-03-08T23:53:43.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.129+0000 7f5fd4e65640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5fb80a2890 0x7f5fb80a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0198ba0 tx=0x7f5fc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:43.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.132+0000 7f5fce7fc640 1 -- 172.21.15.22:0/2143045579 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5fc804d020 con 0x7f5fd0101d90 2024-03-08T23:53:43.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.354+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd002ce70 con 0x7f5fd0101d90 2024-03-08T23:53:43.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.355+0000 7f5fce7fc640 1 -- 172.21.15.22:0/2143045579 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f5fc803e450 con 0x7f5fd0101d90 2024-03-08T23:53:43.351 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:43.352 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:52:18.337548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:53:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-03-08T23:53:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5fb80a2890 msgr2=0x7f5fb80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5fb80a2890 0x7f5fb80a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0198ba0 tx=0x7f5fc4005f90 comp rx=0 tx=0).stop 2024-03-08T23:53:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 msgr2=0x7f5fd0197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0197690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005ab0 tx=0x7f5fc802c6c0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 shutdown_connections 2024-03-08T23:53:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0106ff0 0x7f5fd019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd01046c0 0x7f5fd0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5fb80a2890 0x7f5fb80a4d50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 --2- 172.21.15.22:0/2143045579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fd0101d90 0x7f5fd0197690 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 >> 172.21.15.22:0/2143045579 conn(0x7f5fd00fb780 msgr2=0x7f5fd01026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 shutdown_connections 2024-03-08T23:53:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.357+0000 7f5fd78ca640 1 -- 172.21.15.22:0/2143045579 wait complete. 2024-03-08T23:53:43.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:53:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.470+0000 7f621bb7e640 1 -- 172.21.15.22:0/126990131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6214102f70 msgr2=0x7f6214109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.470+0000 7f621bb7e640 1 --2- 172.21.15.22:0/126990131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6214102f70 0x7f6214109800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f620c009cd0 tx=0x7f620c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- 172.21.15.22:0/126990131 shutdown_connections 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 --2- 172.21.15.22:0/126990131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6214102f70 0x7f6214109800 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 --2- 172.21.15.22:0/126990131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62141025b0 0x7f6214102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 --2- 172.21.15.22:0/126990131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62140ff5e0 0x7f62140ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- 172.21.15.22:0/126990131 >> 172.21.15.22:0/126990131 conn(0x7f62140faf10 msgr2=0x7f62140fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- 172.21.15.22:0/126990131 shutdown_connections 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- 172.21.15.22:0/126990131 wait complete. 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 Processor -- start 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- start start 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62140ff5e0 0x7f6214101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62141025b0 0x7f6214100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6214102f70 0x7f6214100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621410cfd0 con 0x7f62140ff5e0 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621410ce50 con 0x7f62141025b0 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.471+0000 7f621bb7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621410d150 con 0x7f6214102f70 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62140ff5e0 0x7f6214101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62140ff5e0 0x7f6214101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44686/0 (socket says 172.21.15.22:44686) 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 -- 172.21.15.22:0/2349860702 learned_addr learned my addr 172.21.15.22:0/2349860702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f6219119640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62141025b0 0x7f6214100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621a11b640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6214102f70 0x7f6214100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 -- 172.21.15.22:0/2349860702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6214102f70 msgr2=0x7f6214100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6214102f70 0x7f6214100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 -- 172.21.15.22:0/2349860702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62141025b0 msgr2=0x7f6214100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62141025b0 0x7f6214100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 -- 172.21.15.22:0/2349860702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214101330 con 0x7f62140ff5e0 2024-03-08T23:53:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621991a640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62140ff5e0 0x7f6214101f20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f621000d770 tx=0x7f621000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f620affd640 1 -- 172.21.15.22:0/2349860702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210017070 con 0x7f62140ff5e0 2024-03-08T23:53:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f620affd640 1 -- 172.21.15.22:0/2349860702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6210009c30 con 0x7f62140ff5e0 2024-03-08T23:53:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62141a1700 con 0x7f62140ff5e0 2024-03-08T23:53:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f620affd640 1 -- 172.21.15.22:0/2349860702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210003dd0 con 0x7f62140ff5e0 2024-03-08T23:53:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.472+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62141a1c50 con 0x7f62140ff5e0 2024-03-08T23:53:43.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.473+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62141086e0 con 0x7f62140ff5e0 2024-03-08T23:53:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.475+0000 7f620affd640 1 -- 172.21.15.22:0/2349860702 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f6210003fb0 con 0x7f62140ff5e0 2024-03-08T23:53:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.475+0000 7f620affd640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62000a2890 0x7f62000a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.475+0000 7f620affd640 1 -- 172.21.15.22:0/2349860702 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f62100cecf0 con 0x7f62140ff5e0 2024-03-08T23:53:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.475+0000 7f6219119640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62000a2890 0x7f62000a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.476+0000 7f6219119640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62000a2890 0x7f62000a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6214101c90 tx=0x7f620400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:43.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.479+0000 7f620affd640 1 -- 172.21.15.22:0/2349860702 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6210009de0 con 0x7f62140ff5e0 2024-03-08T23:53:43.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.664+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f621402d120 con 0x7f62140ff5e0 2024-03-08T23:53:43.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.666+0000 7f620affd640 1 -- 172.21.15.22:0/2349860702 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24504 (secure 0 0 0) 0x7f621009b8d0 con 0x7f62140ff5e0 2024-03-08T23:53:43.663 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:43.663 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":91,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:52:45.399861+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:53:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62000a2890 msgr2=0x7f62000a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62000a2890 0x7f62000a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6214101c90 tx=0x7f620400a2d0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62140ff5e0 msgr2=0x7f6214101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62140ff5e0 0x7f6214101f20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f621000d770 tx=0x7f621000dc30 comp rx=0 tx=0).stop 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 shutdown_connections 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6214102f70 0x7f6214100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62141025b0 0x7f6214100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62000a2890 0x7f62000a4d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 --2- 172.21.15.22:0/2349860702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62140ff5e0 0x7f6214101f20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 >> 172.21.15.22:0/2349860702 conn(0x7f62140faf10 msgr2=0x7f62141058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 shutdown_connections 2024-03-08T23:53:43.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.668+0000 7f621bb7e640 1 -- 172.21.15.22:0/2349860702 wait complete. 2024-03-08T23:53:43.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:43 smithi022 ceph-mon[31894]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:43.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:43 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/925536956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:53:43.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:43 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3400573319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:53:43.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:43 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2143045579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:43.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:53:43.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:53:43.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.786+0000 7f1723e44640 1 -- 172.21.15.22:0/1150842399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 msgr2=0x7f171c109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.786+0000 7f1723e44640 1 --2- 172.21.15.22:0/1150842399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 0x7f171c109740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1714009ca0 tx=0x7f171402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.786+0000 7f1723e44640 1 -- 172.21.15.22:0/1150842399 shutdown_connections 2024-03-08T23:53:43.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.787+0000 7f1723e44640 1 --2- 172.21.15.22:0/1150842399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 0x7f171c109740 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.787+0000 7f1723e44640 1 --2- 172.21.15.22:0/1150842399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171c1024f0 0x7f171c102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.787+0000 7f1723e44640 1 --2- 172.21.15.22:0/1150842399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f171c101300 0x7f171c101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.787+0000 7f1723e44640 1 -- 172.21.15.22:0/1150842399 >> 172.21.15.22:0/1150842399 conn(0x7f171c0fc870 msgr2=0x7f171c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:43.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.787+0000 7f1723e44640 1 -- 172.21.15.22:0/1150842399 shutdown_connections 2024-03-08T23:53:43.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.787+0000 7f1723e44640 1 -- 172.21.15.22:0/1150842399 wait complete. 2024-03-08T23:53:43.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.787+0000 7f1723e44640 1 Processor -- start 2024-03-08T23:53:43.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1723e44640 1 -- start start 2024-03-08T23:53:43.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1723e44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f171c101300 0x7f171c0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1723e44640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171c1024f0 0x7f171c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1723e44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 0x7f171c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1723e44640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c10cff0 con 0x7f171c102eb0 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1723e44640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c10ce70 con 0x7f171c1024f0 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1723e44640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c10d170 con 0x7f171c101300 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1721be0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f171c101300 0x7f171c0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1721be0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f171c101300 0x7f171c0750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50506/0 (socket says 172.21.15.22:50506) 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f1721be0640 1 -- 172.21.15.22:0/4062291926 learned_addr learned my addr 172.21.15.22:0/4062291926 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f17213df640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171c1024f0 0x7f171c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.788+0000 7f17223e1640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 0x7f171c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.789+0000 7f17223e1640 1 -- 172.21.15.22:0/4062291926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f171c101300 msgr2=0x7f171c0750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.789+0000 7f17223e1640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f171c101300 0x7f171c0750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.789+0000 7f17223e1640 1 -- 172.21.15.22:0/4062291926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171c1024f0 msgr2=0x7f171c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:43.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.789+0000 7f17223e1640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171c1024f0 0x7f171c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:43.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.789+0000 7f17223e1640 1 -- 172.21.15.22:0/4062291926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0744b0 con 0x7f171c102eb0 2024-03-08T23:53:43.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.789+0000 7f17223e1640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 0x7f171c073c20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f171402c630 tx=0x7f1714032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:43.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.789+0000 7f1712ffd640 1 -- 172.21.15.22:0/4062291926 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714039040 con 0x7f171c102eb0 2024-03-08T23:53:43.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.790+0000 7f1712ffd640 1 -- 172.21.15.22:0/4062291926 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1714044070 con 0x7f171c102eb0 2024-03-08T23:53:43.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.790+0000 7f1712ffd640 1 -- 172.21.15.22:0/4062291926 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140042a0 con 0x7f171c102eb0 2024-03-08T23:53:43.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.791+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c1a16f0 con 0x7f171c102eb0 2024-03-08T23:53:43.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.791+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c1a1bc0 con 0x7f171c102eb0 2024-03-08T23:53:43.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.792+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c067850 con 0x7f171c102eb0 2024-03-08T23:53:43.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.795+0000 7f1712ffd640 1 -- 172.21.15.22:0/4062291926 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1714032690 con 0x7f171c102eb0 2024-03-08T23:53:43.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.796+0000 7f1712ffd640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17080a2990 0x7f17080a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:43.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.796+0000 7f1712ffd640 1 -- 172.21.15.22:0/4062291926 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f17140f15a0 con 0x7f171c102eb0 2024-03-08T23:53:43.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.797+0000 7f1721be0640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17080a2990 0x7f17080a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:43.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.797+0000 7f1721be0640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17080a2990 0x7f17080a4e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f17180058c0 tx=0x7f1718005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:43.808+0000 7f1712ffd640 1 -- 172.21.15.22:0/4062291926 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f171404d020 con 0x7f171c102eb0 2024-03-08T23:53:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:43 smithi062 ceph-mon[36129]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:43 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/925536956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:53:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:43 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3400573319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:53:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:43 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2143045579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:43.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:43 smithi192 ceph-mon[35995]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:53:43.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:43 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/925536956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:53:43.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:43 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3400573319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:53:43.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:43 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2143045579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.008+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f171c02ce70 con 0x7f171c102eb0 2024-03-08T23:53:44.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.009+0000 7f1712ffd640 1 -- 172.21.15.22:0/4062291926 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17140be3e0 con 0x7f171c102eb0 2024-03-08T23:53:44.005 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:53:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17080a2990 msgr2=0x7f17080a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17080a2990 0x7f17080a4e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f17180058c0 tx=0x7f1718005f90 comp rx=0 tx=0).stop 2024-03-08T23:53:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 msgr2=0x7f171c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 0x7f171c073c20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f171402c630 tx=0x7f1714032f20 comp rx=0 tx=0).stop 2024-03-08T23:53:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 shutdown_connections 2024-03-08T23:53:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f17080a2990 0x7f17080a4e50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f171c102eb0 0x7f171c073c20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171c1024f0 0x7f171c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 --2- 172.21.15.22:0/4062291926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f171c101300 0x7f171c0750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 >> 172.21.15.22:0/4062291926 conn(0x7f171c0fc870 msgr2=0x7f171c105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 shutdown_connections 2024-03-08T23:53:44.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.011+0000 7f1723e44640 1 -- 172.21.15.22:0/4062291926 wait complete. 2024-03-08T23:53:44.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:53:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 -- 172.21.15.22:0/2501988966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c106fd0 msgr2=0x7fe91c109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2501988966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c106fd0 0x7fe91c109420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009ca0 tx=0x7fe90c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 -- 172.21.15.22:0/2501988966 shutdown_connections 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2501988966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c106fd0 0x7fe91c109420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2501988966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe91c1046a0 0x7fe91c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2501988966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c101d70 0x7fe91c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 -- 172.21.15.22:0/2501988966 >> 172.21.15.22:0/2501988966 conn(0x7fe91c0fb780 msgr2=0x7fe91c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 -- 172.21.15.22:0/2501988966 shutdown_connections 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.127+0000 7fe9214c6640 1 -- 172.21.15.22:0/2501988966 wait complete. 2024-03-08T23:53:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 Processor -- start 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 -- start start 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe91c101d70 0x7fe91c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c1046a0 0x7fe91c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c106fd0 0x7fe91c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c10cf80 con 0x7fe91c1046a0 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c10ce00 con 0x7fe91c106fd0 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe9214c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c10d100 con 0x7fe91c101d70 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe91a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c1046a0 0x7fe91c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe91a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c1046a0 0x7fe91c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44730/0 (socket says 172.21.15.22:44730) 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe91a7fc640 1 -- 172.21.15.22:0/2730820515 learned_addr learned my addr 172.21.15.22:0/2730820515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.128+0000 7fe91affd640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe91c101d70 0x7fe91c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91b7fe640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c106fd0 0x7fe91c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91a7fc640 1 -- 172.21.15.22:0/2730820515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe91c101d70 msgr2=0x7fe91c197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91a7fc640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe91c101d70 0x7fe91c197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91a7fc640 1 -- 172.21.15.22:0/2730820515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c106fd0 msgr2=0x7fe91c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91a7fc640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c106fd0 0x7fe91c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91a7fc640 1 -- 172.21.15.22:0/2730820515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c19f220 con 0x7fe91c1046a0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91b7fe640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c106fd0 0x7fe91c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe91a7fc640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c1046a0 0x7fe91c197b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe90400d740 tx=0x7fe90400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe913fff640 1 -- 172.21.15.22:0/2730820515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904017070 con 0x7fe91c1046a0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe913fff640 1 -- 172.21.15.22:0/2730820515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe90400de50 con 0x7fe91c1046a0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91c19f510 con 0x7fe91c1046a0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe913fff640 1 -- 172.21.15.22:0/2730820515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904004e00 con 0x7fe91c1046a0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.129+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c19fa10 con 0x7fe91c1046a0 2024-03-08T23:53:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.130+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91c0fd270 con 0x7fe91c1046a0 2024-03-08T23:53:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.131+0000 7fe913fff640 1 -- 172.21.15.22:0/2730820515 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe904004fa0 con 0x7fe91c1046a0 2024-03-08T23:53:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.132+0000 7fe913fff640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9080a2890 0x7fe9080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.132+0000 7fe91affd640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9080a2890 0x7fe9080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.132+0000 7fe913fff640 1 -- 172.21.15.22:0/2730820515 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe9040ce950 con 0x7fe91c1046a0 2024-03-08T23:53:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.132+0000 7fe91affd640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9080a2890 0x7fe9080a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe91400bbe0 tx=0x7fe91400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.136+0000 7fe913fff640 1 -- 172.21.15.22:0/2730820515 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe90409b4d0 con 0x7fe91c1046a0 2024-03-08T23:53:44.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.321+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe91c02d0c0 con 0x7fe91c1046a0 2024-03-08T23:53:44.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.322+0000 7fe913fff640 1 -- 172.21.15.22:0/2730820515 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe90409add0 con 0x7fe91c1046a0 2024-03-08T23:53:44.318 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:53:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.323+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9080a2890 msgr2=0x7fe9080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.323+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9080a2890 0x7fe9080a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe91400bbe0 tx=0x7fe91400b390 comp rx=0 tx=0).stop 2024-03-08T23:53:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.323+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c1046a0 msgr2=0x7fe91c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.323+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c1046a0 0x7fe91c197b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe90400d740 tx=0x7fe90400dc00 comp rx=0 tx=0).stop 2024-03-08T23:53:44.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 shutdown_connections 2024-03-08T23:53:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c106fd0 0x7fe91c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe9080a2890 0x7fe9080a4d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe91c1046a0 0x7fe91c197b40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 --2- 172.21.15.22:0/2730820515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe91c101d70 0x7fe91c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 >> 172.21.15.22:0/2730820515 conn(0x7fe91c0fb780 msgr2=0x7fe91c1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 shutdown_connections 2024-03-08T23:53:44.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.324+0000 7fe9214c6640 1 -- 172.21.15.22:0/2730820515 wait complete. 2024-03-08T23:53:44.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:53:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.440+0000 7faf9ce85640 1 -- 172.21.15.22:0/3513577049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98101dd0 msgr2=0x7faf981041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.440+0000 7faf9ce85640 1 --2- 172.21.15.22:0/3513577049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98101dd0 0x7faf981041c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faf90009900 tx=0x7faf9002bc30 comp rx=0 tx=0).stop 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.440+0000 7faf9ce85640 1 -- 172.21.15.22:0/3513577049 shutdown_connections 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.440+0000 7faf9ce85640 1 --2- 172.21.15.22:0/3513577049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf98107030 0x7faf98109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.440+0000 7faf9ce85640 1 --2- 172.21.15.22:0/3513577049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf98104700 0x7faf98106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.440+0000 7faf9ce85640 1 --2- 172.21.15.22:0/3513577049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98101dd0 0x7faf981041c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.440+0000 7faf9ce85640 1 -- 172.21.15.22:0/3513577049 >> 172.21.15.22:0/3513577049 conn(0x7faf980fb780 msgr2=0x7faf980fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 -- 172.21.15.22:0/3513577049 shutdown_connections 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 -- 172.21.15.22:0/3513577049 wait complete. 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 Processor -- start 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 -- start start 2024-03-08T23:53:44.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf98101dd0 0x7faf981976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf98104700 0x7faf98197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98107030 0x7faf9819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9810cf80 con 0x7faf98107030 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9810ce00 con 0x7faf98101dd0 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.441+0000 7faf9ce85640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9810d100 con 0x7faf98104700 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98107030 0x7faf9819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98107030 0x7faf9819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44754/0 (socket says 172.21.15.22:44754) 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 -- 172.21.15.22:0/4031861456 learned_addr learned my addr 172.21.15.22:0/4031861456 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:44.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf95d9b640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf98104700 0x7faf98197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf9659c640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf98101dd0 0x7faf981976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 -- 172.21.15.22:0/4031861456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf98104700 msgr2=0x7faf98197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf98104700 0x7faf98197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 -- 172.21.15.22:0/4031861456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf98101dd0 msgr2=0x7faf981976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf98101dd0 0x7faf981976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 -- 172.21.15.22:0/4031861456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9819f2d0 con 0x7faf98107030 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf9659c640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf98101dd0 0x7faf981976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf96d9d640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98107030 0x7faf9819ec60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faf8800cad0 tx=0x7faf8800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf8f7fe640 1 -- 172.21.15.22:0/4031861456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8800ae20 con 0x7faf98107030 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9819f5c0 con 0x7faf98107030 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf8f7fe640 1 -- 172.21.15.22:0/4031861456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf88005020 con 0x7faf98107030 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf8f7fe640 1 -- 172.21.15.22:0/4031861456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8800f6f0 con 0x7faf98107030 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.442+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9819fac0 con 0x7faf98107030 2024-03-08T23:53:44.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.443+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf980fd270 con 0x7faf98107030 2024-03-08T23:53:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.444+0000 7faf8f7fe640 1 -- 172.21.15.22:0/4031861456 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7faf880051d0 con 0x7faf98107030 2024-03-08T23:53:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.445+0000 7faf8f7fe640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faf840a2890 0x7faf840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.445+0000 7faf9659c640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faf840a2890 0x7faf840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.445+0000 7faf8f7fe640 1 -- 172.21.15.22:0/4031861456 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7faf880cf020 con 0x7faf98107030 2024-03-08T23:53:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.445+0000 7faf9659c640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faf840a2890 0x7faf840a4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faf90006010 tx=0x7faf9002c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:44.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.449+0000 7faf8f7fe640 1 -- 172.21.15.22:0/4031861456 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7faf88014030 con 0x7faf98107030 2024-03-08T23:53:44.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:44 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2349860702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:44.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:44 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4062291926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:53:44.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:44 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2730820515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:53:44.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-08T23:53:44.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:44 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-08T23:53:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.645+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf98061ef0 con 0x7faf98107030 2024-03-08T23:53:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.645+0000 7faf8f7fe640 1 -- 172.21.15.22:0/4031861456 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf8809bba0 con 0x7faf98107030 2024-03-08T23:53:44.642 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:53:44.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faf840a2890 msgr2=0x7faf840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faf840a2890 0x7faf840a4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faf90006010 tx=0x7faf9002c8a0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98107030 msgr2=0x7faf9819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98107030 0x7faf9819ec60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faf8800cad0 tx=0x7faf8800cf90 comp rx=0 tx=0).stop 2024-03-08T23:53:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 shutdown_connections 2024-03-08T23:53:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7faf840a2890 0x7faf840a4d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf98107030 0x7faf9819ec60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf98104700 0x7faf98197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 --2- 172.21.15.22:0/4031861456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf98101dd0 0x7faf981976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 >> 172.21.15.22:0/4031861456 conn(0x7faf980fb780 msgr2=0x7faf98102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 shutdown_connections 2024-03-08T23:53:44.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.647+0000 7faf9ce85640 1 -- 172.21.15.22:0/4031861456 wait complete. 2024-03-08T23:53:44.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:53:44.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 -- 172.21.15.22:0/466376853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 msgr2=0x7f0e84108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/466376853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e84108640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009ca0 tx=0x7f0e7c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 -- 172.21.15.22:0/466376853 shutdown_connections 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/466376853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e84108640 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/466376853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e841013f0 0x7f0e84101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/466376853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e84100200 0x7f0e84100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 -- 172.21.15.22:0/466376853 >> 172.21.15.22:0/466376853 conn(0x7f0e840fb790 msgr2=0x7f0e840fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 -- 172.21.15.22:0/466376853 shutdown_connections 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.764+0000 7f0e8a82b640 1 -- 172.21.15.22:0/466376853 wait complete. 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 Processor -- start 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 -- start start 2024-03-08T23:53:44.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e84100200 0x7f0e84197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e841013f0 0x7f0e84197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e8419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8410bed0 con 0x7f0e84101db0 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8410bd50 con 0x7f0e84100200 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e8a82b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8410c050 con 0x7f0e841013f0 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e88dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e8419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e88dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e8419ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44762/0 (socket says 172.21.15.22:44762) 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e88dc8640 1 -- 172.21.15.22:0/4168890293 learned_addr learned my addr 172.21.15.22:0/4168890293 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.765+0000 7f0e83fff640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e84100200 0x7f0e84197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e88dc8640 1 -- 172.21.15.22:0/4168890293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e841013f0 msgr2=0x7f0e84197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e88dc8640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e841013f0 0x7f0e84197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e88dc8640 1 -- 172.21.15.22:0/4168890293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e84100200 msgr2=0x7f0e84197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e88dc8640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e84100200 0x7f0e84197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e88dc8640 1 -- 172.21.15.22:0/4168890293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8419f260 con 0x7f0e84101db0 2024-03-08T23:53:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e88dc8640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e8419ebf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005e30 tx=0x7f0e7c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e817fa640 1 -- 172.21.15.22:0/4168890293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c041020 con 0x7f0e84101db0 2024-03-08T23:53:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8419f4f0 con 0x7f0e84101db0 2024-03-08T23:53:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e817fa640 1 -- 172.21.15.22:0/4168890293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c034cd0 con 0x7f0e84101db0 2024-03-08T23:53:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e817fa640 1 -- 172.21.15.22:0/4168890293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c039040 con 0x7f0e84101db0 2024-03-08T23:53:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.766+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8419fa30 con 0x7f0e84101db0 2024-03-08T23:53:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.767+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8419fd50 con 0x7f0e84101db0 2024-03-08T23:53:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.768+0000 7f0e817fa640 1 -- 172.21.15.22:0/4168890293 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f0e7c044020 con 0x7f0e84101db0 2024-03-08T23:53:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.769+0000 7f0e817fa640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0e700a2890 0x7f0e700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.769+0000 7f0e83fff640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0e700a2890 0x7f0e700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.769+0000 7f0e817fa640 1 -- 172.21.15.22:0/4168890293 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0e7c0f0ec0 con 0x7f0e84101db0 2024-03-08T23:53:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.769+0000 7f0e83fff640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0e700a2890 0x7f0e700a4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005fd0 tx=0x7f0e7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.773+0000 7f0e817fa640 1 -- 172.21.15.22:0/4168890293 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0e7c04b020 con 0x7f0e84101db0 2024-03-08T23:53:44.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:44 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2349860702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:44.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:44 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4062291926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:53:44.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:44 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2730820515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:53:44.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-08T23:53:44.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:44 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-08T23:53:44.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.955+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e8419ff20 con 0x7f0e84101db0 2024-03-08T23:53:44.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.956+0000 7f0e817fa640 1 -- 172.21.15.22:0/4168890293 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e7c0bdb80 con 0x7f0e84101db0 2024-03-08T23:53:44.952 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:53:44.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.957+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0e700a2890 msgr2=0x7f0e700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.957+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0e700a2890 0x7f0e700a4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005fd0 tx=0x7f0e7400b410 comp rx=0 tx=0).stop 2024-03-08T23:53:44.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 msgr2=0x7f0e8419ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e8419ebf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005e30 tx=0x7f0e7c032ec0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 shutdown_connections 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0e700a2890 0x7f0e700a4d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e84101db0 0x7f0e8419ebf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e841013f0 0x7f0e84197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 --2- 172.21.15.22:0/4168890293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e84100200 0x7f0e84197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 >> 172.21.15.22:0/4168890293 conn(0x7f0e840fb790 msgr2=0x7f0e840698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 shutdown_connections 2024-03-08T23:53:44.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:44.958+0000 7f0e8a82b640 1 -- 172.21.15.22:0/4168890293 wait complete. 2024-03-08T23:53:44.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.071+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2359566541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 msgr2=0x7f0c48101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.071+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2359566541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c48101380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009900 tx=0x7f0c3802bcd0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.071+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2359566541 shutdown_connections 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.071+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2359566541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c48101380 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.071+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2359566541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c48100560 0x7f0c481009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.071+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2359566541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c480ffc00 0x7f0c48100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.071+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2359566541 >> 172.21.15.22:0/2359566541 conn(0x7f0c480fb780 msgr2=0x7f0c480fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2359566541 shutdown_connections 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2359566541 wait complete. 2024-03-08T23:53:45.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 Processor -- start 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 -- start start 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c480ffc00 0x7f0c481997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c48100560 0x7f0c48199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c481a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4810f1a0 con 0x7f0c48100f00 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4810f020 con 0x7f0c48100560 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4ed90640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4810f320 con 0x7f0c480ffc00 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4d32d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c481a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4d32d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c481a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44786/0 (socket says 172.21.15.22:44786) 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4d32d640 1 -- 172.21.15.22:0/2402701882 learned_addr learned my addr 172.21.15.22:0/2402701882 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c4cb2c640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c480ffc00 0x7f0c481997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.072+0000 7f0c47fff640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c48100560 0x7f0c48199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4d32d640 1 -- 172.21.15.22:0/2402701882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c480ffc00 msgr2=0x7f0c481997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4d32d640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c480ffc00 0x7f0c481997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4d32d640 1 -- 172.21.15.22:0/2402701882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c48100560 msgr2=0x7f0c48199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4d32d640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c48100560 0x7f0c48199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4d32d640 1 -- 172.21.15.22:0/2402701882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c481a1410 con 0x7f0c48100f00 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4d32d640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c481a0da0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c3802cf10 tx=0x7f0c38005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c45ffb640 1 -- 172.21.15.22:0/2402701882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38041070 con 0x7f0c48100f00 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c481a16a0 con 0x7f0c48100f00 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c45ffb640 1 -- 172.21.15.22:0/2402701882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c380054e0 con 0x7f0c48100f00 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c45ffb640 1 -- 172.21.15.22:0/2402701882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c380346a0 con 0x7f0c48100f00 2024-03-08T23:53:45.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.073+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c481a1b90 con 0x7f0c48100f00 2024-03-08T23:53:45.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.074+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4810aa80 con 0x7f0c48100f00 2024-03-08T23:53:45.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.075+0000 7f0c45ffb640 1 -- 172.21.15.22:0/2402701882 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f0c38044070 con 0x7f0c48100f00 2024-03-08T23:53:45.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.076+0000 7f0c45ffb640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0c300a2890 0x7f0c300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.076+0000 7f0c45ffb640 1 -- 172.21.15.22:0/2402701882 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c380f0b10 con 0x7f0c48100f00 2024-03-08T23:53:45.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.076+0000 7f0c4cb2c640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0c300a2890 0x7f0c300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.076+0000 7f0c4cb2c640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0c300a2890 0x7f0c300a4d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005fd0 tx=0x7f0c40009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:45.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.080+0000 7f0c45ffb640 1 -- 172.21.15.22:0/2402701882 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0c38003e20 con 0x7f0c48100f00 2024-03-08T23:53:45.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:44 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2349860702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:45.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:44 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4062291926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:53:45.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:44 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2730820515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:53:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-08T23:53:45.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:44 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-08T23:53:45.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.303+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c48061ef0 con 0x7f0c48100f00 2024-03-08T23:53:45.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.304+0000 7f0c45ffb640 1 -- 172.21.15.22:0/2402701882 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f0c380bd180 con 0x7f0c48100f00 2024-03-08T23:53:45.300 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:45.301 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:52:18.337548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:53:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0c300a2890 msgr2=0x7f0c300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0c300a2890 0x7f0c300a4d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005fd0 tx=0x7f0c40009680 comp rx=0 tx=0).stop 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 msgr2=0x7f0c481a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c481a0da0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c3802cf10 tx=0x7f0c38005e40 comp rx=0 tx=0).stop 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 shutdown_connections 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0c300a2890 0x7f0c300a4d50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c48100f00 0x7f0c481a0da0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c48100560 0x7f0c48199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 --2- 172.21.15.22:0/2402701882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c480ffc00 0x7f0c481997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.306+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 >> 172.21.15.22:0/2402701882 conn(0x7f0c480fb780 msgr2=0x7f0c480fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.307+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 shutdown_connections 2024-03-08T23:53:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.307+0000 7f0c4ed90640 1 -- 172.21.15.22:0/2402701882 wait complete. 2024-03-08T23:53:45.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:53:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 -- 172.21.15.22:0/3973827626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500107070 msgr2=0x7f55001094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 --2- 172.21.15.22:0/3973827626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500107070 0x7f55001094c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009ca0 tx=0x7f54f802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 -- 172.21.15.22:0/3973827626 shutdown_connections 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 --2- 172.21.15.22:0/3973827626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500107070 0x7f55001094c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 --2- 172.21.15.22:0/3973827626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5500104740 0x7f5500106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 --2- 172.21.15.22:0/3973827626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5500101e10 0x7f5500104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 -- 172.21.15.22:0/3973827626 >> 172.21.15.22:0/3973827626 conn(0x7f55000fb820 msgr2=0x7f55000fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 -- 172.21.15.22:0/3973827626 shutdown_connections 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.421+0000 7f55068c0640 1 -- 172.21.15.22:0/3973827626 wait complete. 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 Processor -- start 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 -- start start 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500101e10 0x7f5500197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5500104740 0x7f5500197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5500107070 0x7f550019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550010cfb0 con 0x7f5500101e10 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550010ce30 con 0x7f5500104740 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f55068c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550010d130 con 0x7f5500107070 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500101e10 0x7f5500197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500101e10 0x7f5500197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44806/0 (socket says 172.21.15.22:44806) 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 -- 172.21.15.22:0/815175036 learned_addr learned my addr 172.21.15.22:0/815175036 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54ff7fe640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5500104740 0x7f5500197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 -- 172.21.15.22:0/815175036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5500107070 msgr2=0x7f550019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5500107070 0x7f550019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 -- 172.21.15.22:0/815175036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5500104740 msgr2=0x7f5500197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5500104740 0x7f5500197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.422+0000 7f54fffff640 1 -- 172.21.15.22:0/815175036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550019f200 con 0x7f5500101e10 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.423+0000 7f54fffff640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500101e10 0x7f5500197650 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f54e800d950 tx=0x7f54e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.423+0000 7f54fd7fa640 1 -- 172.21.15.22:0/815175036 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8017070 con 0x7f5500101e10 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.423+0000 7f54fd7fa640 1 -- 172.21.15.22:0/815175036 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54e8009c30 con 0x7f5500101e10 2024-03-08T23:53:45.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.423+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550019f4f0 con 0x7f5500101e10 2024-03-08T23:53:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.423+0000 7f54fd7fa640 1 -- 172.21.15.22:0/815175036 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800f6b0 con 0x7f5500101e10 2024-03-08T23:53:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.423+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550019fa40 con 0x7f5500101e10 2024-03-08T23:53:45.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.424+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55000fd310 con 0x7f5500101e10 2024-03-08T23:53:45.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.425+0000 7f54fd7fa640 1 -- 172.21.15.22:0/815175036 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f54e80109d0 con 0x7f5500101e10 2024-03-08T23:53:45.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.425+0000 7f54fd7fa640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54ec0a2890 0x7f54ec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.426+0000 7f54ff7fe640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54ec0a2890 0x7f54ec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.426+0000 7f54fd7fa640 1 -- 172.21.15.22:0/815175036 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54e80ced40 con 0x7f5500101e10 2024-03-08T23:53:45.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.426+0000 7f54ff7fe640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54ec0a2890 0x7f54ec0a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005fd0 tx=0x7f54f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:45.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.430+0000 7f54fd7fa640 1 -- 172.21.15.22:0/815175036 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f54e8009de0 con 0x7f5500101e10 2024-03-08T23:53:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.615+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5500061f90 con 0x7f5500101e10 2024-03-08T23:53:45.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.617+0000 7f54fd7fa640 1 -- 172.21.15.22:0/815175036 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24654 (secure 0 0 0) 0x7f54e809b890 con 0x7f5500101e10 2024-03-08T23:53:45.613 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:45.613 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:44.610541+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54ec0a2890 msgr2=0x7f54ec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54ec0a2890 0x7f54ec0a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005fd0 tx=0x7f54f0005850 comp rx=0 tx=0).stop 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500101e10 msgr2=0x7f5500197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500101e10 0x7f5500197650 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f54e800d950 tx=0x7f54e800de10 comp rx=0 tx=0).stop 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 shutdown_connections 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5500107070 0x7f550019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5500104740 0x7f5500197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54ec0a2890 0x7f54ec0a4d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 --2- 172.21.15.22:0/815175036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5500101e10 0x7f5500197650 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 >> 172.21.15.22:0/815175036 conn(0x7f55000fb820 msgr2=0x7f55001027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 shutdown_connections 2024-03-08T23:53:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.619+0000 7f55068c0640 1 -- 172.21.15.22:0/815175036 wait complete. 2024-03-08T23:53:45.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:53:45.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 -- 172.21.15.22:0/3310097225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890102500 msgr2=0x7fd890102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 --2- 172.21.15.22:0/3310097225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890102500 0x7fd890102980 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd884009900 tx=0x7fd88402bc30 comp rx=0 tx=0).stop 2024-03-08T23:53:45.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 -- 172.21.15.22:0/3310097225 shutdown_connections 2024-03-08T23:53:45.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 --2- 172.21.15.22:0/3310097225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd890102ec0 0x7fd890109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 --2- 172.21.15.22:0/3310097225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890102500 0x7fd890102980 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 --2- 172.21.15.22:0/3310097225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd890101310 0x7fd890101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 -- 172.21.15.22:0/3310097225 >> 172.21.15.22:0/3310097225 conn(0x7fd8900fc8a0 msgr2=0x7fd8900fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:45.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 -- 172.21.15.22:0/3310097225 shutdown_connections 2024-03-08T23:53:45.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 -- 172.21.15.22:0/3310097225 wait complete. 2024-03-08T23:53:45.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 Processor -- start 2024-03-08T23:53:45.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.738+0000 7fd897c61640 1 -- start start 2024-03-08T23:53:45.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890101310 0x7fd8901975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd890102500 0x7fd890197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd890102ec0 0x7fd89019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89010cfa0 con 0x7fd890101310 2024-03-08T23:53:45.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89010ce20 con 0x7fd890102ec0 2024-03-08T23:53:45.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89010d120 con 0x7fd890102500 2024-03-08T23:53:45.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890101310 0x7fd8901975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890101310 0x7fd8901975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44826/0 (socket says 172.21.15.22:44826) 2024-03-08T23:53:45.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 -- 172.21.15.22:0/1720703799 learned_addr learned my addr 172.21.15.22:0/1720703799 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:45.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8951fc640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd890102500 0x7fd890197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8961fe640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd890102ec0 0x7fd89019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 -- 172.21.15.22:0/1720703799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd890102500 msgr2=0x7fd890197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd890102500 0x7fd890197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 -- 172.21.15.22:0/1720703799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd890102ec0 msgr2=0x7fd89019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd890102ec0 0x7fd89019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 -- 172.21.15.22:0/1720703799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89019f210 con 0x7fd890101310 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd8959fd640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890101310 0x7fd8901975f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c970 tx=0x7fd88800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd88effd640 1 -- 172.21.15.22:0/1720703799 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888004220 con 0x7fd890101310 2024-03-08T23:53:45.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd88effd640 1 -- 172.21.15.22:0/1720703799 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8880043c0 con 0x7fd890101310 2024-03-08T23:53:45.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89019f500 con 0x7fd890101310 2024-03-08T23:53:45.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd88effd640 1 -- 172.21.15.22:0/1720703799 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88800f450 con 0x7fd890101310 2024-03-08T23:53:45.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.739+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89019fa20 con 0x7fd890101310 2024-03-08T23:53:45.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.740+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd890068530 con 0x7fd890101310 2024-03-08T23:53:45.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.741+0000 7fd88effd640 1 -- 172.21.15.22:0/1720703799 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd8880079b0 con 0x7fd890101310 2024-03-08T23:53:45.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.742+0000 7fd88effd640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8780a2800 0x7fd8780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:45.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.742+0000 7fd88effd640 1 -- 172.21.15.22:0/1720703799 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8880cea80 con 0x7fd890101310 2024-03-08T23:53:45.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.742+0000 7fd8951fc640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8780a2800 0x7fd8780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:45.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.742+0000 7fd8951fc640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8780a2800 0x7fd8780a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd88402c7b0 tx=0x7fd884005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:45.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.746+0000 7fd88effd640 1 -- 172.21.15.22:0/1720703799 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd888014030 con 0x7fd890101310 2024-03-08T23:53:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:45 smithi192 ceph-mon[35995]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-08T23:53:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:45 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-08T23:53:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:45 smithi192 ceph-mon[35995]: osdmap e92: 12 total, 12 up, 12 in 2024-03-08T23:53:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:45 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4031861456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:53:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:45 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4168890293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:53:45.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:45 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2402701882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:45.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.967+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd890108630 con 0x7fd890101310 2024-03-08T23:53:45.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.967+0000 7fd88effd640 1 -- 172.21.15.22:0/1720703799 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fd88809b140 con 0x7fd890101310 2024-03-08T23:53:45.964 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:53:45.964 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:52:18.337548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:53:45.964 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8780a2800 msgr2=0x7fd8780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8780a2800 0x7fd8780a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd88402c7b0 tx=0x7fd884005fd0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890101310 msgr2=0x7fd8901975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890101310 0x7fd8901975f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c970 tx=0x7fd88800ce30 comp rx=0 tx=0).stop 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 shutdown_connections 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd890102ec0 0x7fd89019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd890102500 0x7fd890197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd8780a2800 0x7fd8780a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 --2- 172.21.15.22:0/1720703799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd890101310 0x7fd8901975f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:45.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 >> 172.21.15.22:0/1720703799 conn(0x7fd8900fc8a0 msgr2=0x7fd890105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:45.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 shutdown_connections 2024-03-08T23:53:45.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:45.969+0000 7fd897c61640 1 -- 172.21.15.22:0/1720703799 wait complete. 2024-03-08T23:53:45.975 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:45 smithi022 ceph-mon[31894]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:45.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-08T23:53:45.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:45 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-08T23:53:45.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:45 smithi022 ceph-mon[31894]: osdmap e92: 12 total, 12 up, 12 in 2024-03-08T23:53:45.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:45 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4031861456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:53:45.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:45 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4168890293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:53:45.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:45 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2402701882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:45.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-08T23:51:51.610292+0000', 'modified': '2024-03-08T23:52:18.337548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24496, 'mds_1': 24494, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/1656345994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 1656345994}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 1656345994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/1768340207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 1768340207}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 1768340207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/1631408985', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 1631408985}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 1631408985}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.192:6837/1326709093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 1326709093}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 1326709093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.62:6839/2804197530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 2804197530}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 2804197530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.192:6839/1126404206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 1126404206}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 1126404206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T23:53:45.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-08T23:53:45.978 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-08T23:53:46.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 -- 172.21.15.22:0/627962423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581023c0 msgr2=0x7f4658102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 --2- 172.21.15.22:0/627962423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581023c0 0x7f4658102840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4648009ca0 tx=0x7f464802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 -- 172.21.15.22:0/627962423 shutdown_connections 2024-03-08T23:53:46.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 --2- 172.21.15.22:0/627962423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4658102d80 0x7f4658109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 --2- 172.21.15.22:0/627962423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581023c0 0x7f4658102840 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 --2- 172.21.15.22:0/627962423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46581011d0 0x7f46581015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 -- 172.21.15.22:0/627962423 >> 172.21.15.22:0/627962423 conn(0x7f46580fc760 msgr2=0x7f46580feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 -- 172.21.15.22:0/627962423 shutdown_connections 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.085+0000 7f465dfa2640 1 -- 172.21.15.22:0/627962423 wait complete. 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 Processor -- start 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 -- start start 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581011d0 0x7f4658199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46581023c0 0x7f4658199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4658102d80 0x7f46581a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465810cec0 con 0x7f46581011d0 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465810cd40 con 0x7f46581023c0 2024-03-08T23:53:46.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465dfa2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465810d040 con 0x7f4658102d80 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581011d0 0x7f4658199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581011d0 0x7f4658199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44846/0 (socket says 172.21.15.22:44846) 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 -- 172.21.15.22:0/909796082 learned_addr learned my addr 172.21.15.22:0/909796082 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 -- 172.21.15.22:0/909796082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4658102d80 msgr2=0x7f46581a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4658102d80 0x7f46581a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 -- 172.21.15.22:0/909796082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46581023c0 msgr2=0x7f4658199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46581023c0 0x7f4658199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.086+0000 7f465cfa0640 1 -- 172.21.15.22:0/909796082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46581a1340 con 0x7f46581011d0 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.087+0000 7f465cfa0640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581011d0 0x7f4658199720 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f465000e9c0 tx=0x7f465000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.087+0000 7f4655ffb640 1 -- 172.21.15.22:0/909796082 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465000cd80 con 0x7f46581011d0 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.087+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46581a1630 con 0x7f46581011d0 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.087+0000 7f4655ffb640 1 -- 172.21.15.22:0/909796082 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46500035f0 con 0x7f46581011d0 2024-03-08T23:53:46.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.087+0000 7f4655ffb640 1 -- 172.21.15.22:0/909796082 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650010690 con 0x7f46581011d0 2024-03-08T23:53:46.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.087+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46581a1b30 con 0x7f46581011d0 2024-03-08T23:53:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.089+0000 7f4655ffb640 1 -- 172.21.15.22:0/909796082 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f46500037a0 con 0x7f46581011d0 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.089+0000 7f4661de6740 1 Processor -- start 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.089+0000 7f4655ffb640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f46440a2800 0x7f46440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.089+0000 7f4661de6740 1 -- start start 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4661de6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969250 0x55fcf4969670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4655ffb640 1 -- 172.21.15.22:0/909796082 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f46500cf340 con 0x7f46581011d0 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4661de6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 0x55fcf496e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f46440a2800 0x7f46440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4661de6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf496edb0 0x55fcf4971240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4661de6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf48ce7c0 con 0x55fcf496edb0 2024-03-08T23:53:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4661de6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf4877b50 con 0x55fcf4969c40 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4661de6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf48ca860 con 0x55fcf4969250 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf496edb0 0x55fcf4971240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf496edb0 0x55fcf4971240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44860/0 (socket says 172.21.15.22:44860) 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 -- 172.21.15.22:0/3885058928 learned_addr learned my addr 172.21.15.22:0/3885058928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969250 0x55fcf4969670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f4657fff640 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 0x55fcf496e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f46440a2800 0x7f46440a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4640009c80 tx=0x7f46400125e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 -- 172.21.15.22:0/3885058928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969250 msgr2=0x55fcf4969670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969250 0x55fcf4969670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 -- 172.21.15.22:0/3885058928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 msgr2=0x55fcf496e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 0x55fcf496e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 -- 172.21.15.22:0/3885058928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcf488b9a0 con 0x55fcf496edb0 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf496edb0 0x55fcf4971240 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4640009f10 tx=0x7f464001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998a3fc6a09bad90 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.090+0000 7f464effd640 1 -- 172.21.15.22:0/3885058928 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640024070 con 0x55fcf496edb0 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f464effd640 1 -- 172.21.15.22:0/3885058928 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4640010040 con 0x55fcf496edb0 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f464effd640 1 -- 172.21.15.22:0/3885058928 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464000f050 con 0x55fcf496edb0 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 -- 172.21.15.22:0/3885058928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf496edb0 msgr2=0x55fcf4971240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf496edb0 0x55fcf4971240 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4640009f10 tx=0x7f464001a000 comp rx=0 tx=0).stop 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 -- 172.21.15.22:0/3885058928 shutdown_connections 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf496edb0 0x55fcf4971240 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 0x55fcf496e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 --2- 172.21.15.22:0/3885058928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969250 0x55fcf4969670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 -- 172.21.15.22:0/3885058928 >> 172.21.15.22:0/3885058928 conn(0x55fcf4931400 msgr2=0x55fcf4964bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 -- 172.21.15.22:0/3885058928 shutdown_connections 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.091+0000 7f4661de6740 1 -- 172.21.15.22:0/3885058928 wait complete. 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 -- start start 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 Processor -- start 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf4973700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4973c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf497ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf4971d00 con 0x55fcf4969250 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf488af90 con 0x55fcf496edb0 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf4971e80 con 0x55fcf4969c40 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4661de6740 10 client.0 _ll_register_callbacks cb 0x55fcf48c6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf4973700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf4973700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44866/0 (socket says 172.21.15.22:44866) 2024-03-08T23:53:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 -- 172.21.15.22:0/3134587156 learned_addr learned my addr 172.21.15.22:0/3134587156 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf497ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4657fff640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4973c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 -- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 msgr2=0x55fcf4973c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4973c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 -- 172.21.15.22:0/3134587156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 msgr2=0x55fcf497ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf497ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465cfa0640 1 -- 172.21.15.22:0/3134587156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcf45e32d0 con 0x55fcf4969250 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f4657fff640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4973c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.092+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf497ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf4973700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4650009810 tx=0x7f465009c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc83228e989aff7d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4637fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4637fff640 1 -- 172.21.15.22:0/3134587156 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500b3070 con 0x55fcf4969250 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4637fff640 1 -- 172.21.15.22:0/3134587156 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4650099bf0 con 0x55fcf4969250 2024-03-08T23:53:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4640010040 con 0x55fcf4969250 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4637fff640 1 -- 172.21.15.22:0/3134587156 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4650003b10 con 0x55fcf4969250 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 10 client.15231 fetch_fsmap learned FSMap version 24 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fcf46ad6b0 con 0x55fcf4969250 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4637fff640 1 -- 172.21.15.22:0/3134587156 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4640010040 con 0x55fcf4969250 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4637fff640 1 -- 172.21.15.22:0/3134587156 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f46500a3c90 con 0x55fcf4969250 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 10 client.15231 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 10 client.15231 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 20 client.15231 populate_metadata read hostname 'smithi022' 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49802c0 0x55fcf497f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 4 client.15231 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.093+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcf48dbd00 con 0x55fcf49802c0 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.094+0000 7f4657fff640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49802c0 0x55fcf497f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.094+0000 7f4657fff640 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49802c0 0x55fcf497f3b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b58bcafb036b3b5 server_cookie=cf65657bd6412f12 in_seq=0 out_seq=0 2024-03-08T23:53:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.094+0000 7f4637fff640 10 client.15231 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4637fff640 1 -- 172.21.15.22:0/3134587156 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fcf48dbd00 con 0x55fcf49802c0 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4637fff640 10 client.15231 handle_command_reply: tid=0 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 client.15231 shutdown 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 msgr2=0x55fcf4973700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf4973700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4650009810 tx=0x7f465009c130 comp rx=0 tx=0).stop 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 shutdown_connections 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49802c0 0x55fcf497f3b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf497ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4973c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 --2- 172.21.15.22:0/3134587156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf4973700 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4637fff640 0 client.15231 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 >> 172.21.15.22:0/3134587156 conn(0x55fcf4931400 msgr2=0x55fcf4963dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 shutdown_connections 2024-03-08T23:53:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 1 -- 172.21.15.22:0/3134587156 wait complete. 2024-03-08T23:53:46.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.095+0000 7f4661de6740 20 client.15231 trim_cache size 0 max 16384 2024-03-08T23:53:46.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:45 smithi062 ceph-mon[36129]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:53:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-08T23:53:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:45 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-08T23:53:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:45 smithi062 ceph-mon[36129]: osdmap e92: 12 total, 12 up, 12 in 2024-03-08T23:53:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:45 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4031861456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:53:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:45 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4168890293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:53:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:45 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2402701882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 Processor -- start 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 -- start start 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf49646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 0x55fcf497ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf496edb0 0x55fcf4964c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf497fae0 con 0x55fcf4969250 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf4980140 con 0x55fcf4969c40 2024-03-08T23:53:46.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f4661de6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf49807a0 con 0x55fcf496edb0 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf49646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf49646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44878/0 (socket says 172.21.15.22:44878) 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 -- 172.21.15.22:0/3535046576 learned_addr learned my addr 172.21.15.22:0/3535046576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf496edb0 0x55fcf4964c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 -- 172.21.15.22:0/3535046576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf496edb0 msgr2=0x55fcf4964c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf496edb0 0x55fcf4964c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 -- 172.21.15.22:0/3535046576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 msgr2=0x55fcf497ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 0x55fcf497ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 -- 172.21.15.22:0/3535046576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcf49644f0 con 0x55fcf4969250 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf496edb0 0x55fcf4964c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf49646e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f465009c1a0 tx=0x7f46500ccad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa33351b67028ae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f464e7fc640 1 -- 172.21.15.22:0/3535046576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500b3020 con 0x55fcf4969250 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.111+0000 7f464e7fc640 1 -- 172.21.15.22:0/3535046576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4650099bf0 con 0x55fcf4969250 2024-03-08T23:53:46.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f464e7fc640 1 -- 172.21.15.22:0/3535046576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500860b0 con 0x55fcf4969250 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 -- 172.21.15.22:0/3535046576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 msgr2=0x55fcf49646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf49646e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f465009c1a0 tx=0x7f46500ccad0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 -- 172.21.15.22:0/3535046576 shutdown_connections 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf496edb0 0x55fcf4964c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf4969c40 0x55fcf497ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 --2- 172.21.15.22:0/3535046576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf49646e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 -- 172.21.15.22:0/3535046576 >> 172.21.15.22:0/3535046576 conn(0x55fcf4931400 msgr2=0x55fcf496b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 -- 172.21.15.22:0/3535046576 shutdown_connections 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 -- 172.21.15.22:0/3535046576 wait complete. 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 -- start start 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.112+0000 7f4661de6740 1 Processor -- start 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4661de6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf497aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4661de6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4977580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4661de6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf4977ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4661de6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf4981460 con 0x55fcf4969250 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4661de6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf4980e00 con 0x55fcf496edb0 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4661de6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcf4981ac0 con 0x55fcf4969c40 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4661de6740 10 client.0 _ll_register_callbacks cb 0x7f4640006f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf497aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf497aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44894/0 (socket says 172.21.15.22:44894) 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 -- 172.21.15.22:0/3744549539 learned_addr learned my addr 172.21.15.22:0/3744549539 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465d7a1640 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf4977ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4657fff640 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4977580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 -- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 msgr2=0x55fcf4977580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4977580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 -- 172.21.15.22:0/3744549539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 msgr2=0x55fcf4977ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf4977ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 -- 172.21.15.22:0/3744549539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcf49d5ec0 con 0x55fcf4969250 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f465cfa0640 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf497aa70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f465009b170 tx=0x7f4650003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9ca2ddeca220f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.113+0000 7f4637fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4637fff640 1 -- 172.21.15.22:0/3744549539 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500b3070 con 0x55fcf4969250 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4637fff640 1 -- 172.21.15.22:0/3744549539 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4650004000 con 0x55fcf4969250 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4650099bf0 con 0x55fcf4969250 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4637fff640 1 -- 172.21.15.22:0/3744549539 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f465009b250 con 0x55fcf4969250 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 10 client.15237 fetch_fsmap learned FSMap version 24 2024-03-08T23:53:46.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fcf49d7010 con 0x55fcf4969250 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4637fff640 1 -- 172.21.15.22:0/3744549539 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4650099bf0 con 0x55fcf4969250 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4637fff640 1 -- 172.21.15.22:0/3744549539 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f465009a8d0 con 0x55fcf4969250 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 10 client.15237 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 10 client.15237 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 20 client.15237 populate_metadata read hostname 'smithi022' 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49db210 0x55fcf49d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 4 client.15237 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fcf48d6050 con 0x55fcf49db210 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.114+0000 7f4657fff640 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49db210 0x55fcf49d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4637fff640 10 client.15237 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4657fff640 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49db210 0x55fcf49d7240 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3fba4a52c06606 server_cookie=ad3c2e905f3c2628 in_seq=0 out_seq=0 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4637fff640 1 -- 172.21.15.22:0/3744549539 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fcf48d6050 con 0x55fcf49db210 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4637fff640 10 client.15237 handle_command_reply: tid=0 2024-03-08T23:53:46.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 client.15237 shutdown 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 msgr2=0x55fcf497aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf497aa70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f465009b170 tx=0x7f4650003d20 comp rx=0 tx=0).stop 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 shutdown_connections 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55fcf49db210 0x55fcf49d7240 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcf496edb0 0x55fcf4977ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcf4969c40 0x55fcf4977580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4637fff640 0 client.15237 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 --2- 172.21.15.22:0/3744549539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcf4969250 0x55fcf497aa70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 >> 172.21.15.22:0/3744549539 conn(0x55fcf4931400 msgr2=0x55fcf49646e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 shutdown_connections 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 1 -- 172.21.15.22:0/3744549539 wait complete. 2024-03-08T23:53:46.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.115+0000 7f4661de6740 20 client.15237 trim_cache size 0 max 16384 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24469, 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 88.783082472000004, 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 89, 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 132.405598043 2024-03-08T23:53:46.121 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f46440a2800 msgr2=0x7f46440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f46440a2800 0x7f46440a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4640009c80 tx=0x7f46400125e0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581011d0 msgr2=0x7f4658199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581011d0 0x7f4658199720 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f465000e9c0 tx=0x7f465000ee80 comp rx=0 tx=0).stop 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 shutdown_connections 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4658102d80 0x7f46581a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46581023c0 0x7f4658199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f46440a2800 0x7f46440a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 --2- 172.21.15.22:0/909796082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46581011d0 0x7f4658199720 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 >> 172.21.15.22:0/909796082 conn(0x7f46580fc760 msgr2=0x7f4658073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 shutdown_connections 2024-03-08T23:53:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.116+0000 7f465dfa2640 1 -- 172.21.15.22:0/909796082 wait complete. 2024-03-08T23:53:46.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 -- 172.21.15.22:0/3685339048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364102ec0 msgr2=0x7f6364109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 --2- 172.21.15.22:0/3685339048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364102ec0 0x7f6364109750 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f635c009ca0 tx=0x7f635c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 -- 172.21.15.22:0/3685339048 shutdown_connections 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 --2- 172.21.15.22:0/3685339048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364102ec0 0x7f6364109750 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 --2- 172.21.15.22:0/3685339048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6364102500 0x7f6364102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 --2- 172.21.15.22:0/3685339048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6364101310 0x7f6364101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 -- 172.21.15.22:0/3685339048 >> 172.21.15.22:0/3685339048 conn(0x7f63640fc880 msgr2=0x7f63640fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 -- 172.21.15.22:0/3685339048 shutdown_connections 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.231+0000 7f6368f96640 1 -- 172.21.15.22:0/3685339048 wait complete. 2024-03-08T23:53:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 Processor -- start 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 -- start start 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364101310 0x7f6364197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6364102500 0x7f6364197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6364102ec0 0x7f636419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636410cfa0 con 0x7f6364101310 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636410ce20 con 0x7f6364102500 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6368f96640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636410d120 con 0x7f6364102ec0 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364101310 0x7f6364197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364101310 0x7f6364197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44916/0 (socket says 172.21.15.22:44916) 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 -- 172.21.15.22:0/3687518158 learned_addr learned my addr 172.21.15.22:0/3687518158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6362d9d640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6364102ec0 0x7f636419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f635bfff640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6364102500 0x7f6364197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 -- 172.21.15.22:0/3687518158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6364102ec0 msgr2=0x7f636419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6364102ec0 0x7f636419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 -- 172.21.15.22:0/3687518158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6364102500 msgr2=0x7f6364197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6364102500 0x7f6364197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f636259c640 1 -- 172.21.15.22:0/3687518158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636419f1a0 con 0x7f6364101310 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f6362d9d640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6364102ec0 0x7f636419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.232+0000 7f635bfff640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6364102500 0x7f6364197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.233+0000 7f636259c640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364101310 0x7f6364197580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9c0 tx=0x7f635400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.233+0000 7f635b7fe640 1 -- 172.21.15.22:0/3687518158 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635400cd80 con 0x7f6364101310 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.233+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636419f490 con 0x7f6364101310 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.233+0000 7f635b7fe640 1 -- 172.21.15.22:0/3687518158 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63540035f0 con 0x7f6364101310 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.233+0000 7f635b7fe640 1 -- 172.21.15.22:0/3687518158 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354010690 con 0x7f6364101310 2024-03-08T23:53:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.233+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636419fa30 con 0x7f6364101310 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.235+0000 7f635b7fe640 1 -- 172.21.15.22:0/3687518158 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f63540037a0 con 0x7f6364101310 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.235+0000 7f636bb70740 1 Processor -- start 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636bb70740 1 -- start start 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636bb70740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda282500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636bb70740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda2875d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636bb70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda287c40 0x55afda28a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636bb70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda1e77c0 con 0x55afda287c40 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636bb70740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda190b50 con 0x55afda282ad0 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636bb70740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda1e3860 con 0x55afda2820e0 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635b7fe640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f63500a2800 0x7f63500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda287c40 0x55afda28a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda287c40 0x55afda28a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44922/0 (socket says 172.21.15.22:44922) 2024-03-08T23:53:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 -- 172.21.15.22:0/1801711083 learned_addr learned my addr 172.21.15.22:0/1801711083 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f6362d9d640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f63500a2800 0x7f63500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635b7fe640 1 -- 172.21.15.22:0/3687518158 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f63540cf310 con 0x7f6364101310 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f6362d9d640 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda282500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f636259c640 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda2875d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 -- 172.21.15.22:0/1801711083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 msgr2=0x55afda282500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda282500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 -- 172.21.15.22:0/1801711083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 msgr2=0x55afda2875d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda2875d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 -- 172.21.15.22:0/1801711083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afda1a49a0 con 0x55afda287c40 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.236+0000 7f635bfff640 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda287c40 0x55afda28a0d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f634c009c20 tx=0x7f634c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83592e9a1aa298ca server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f6362d9d640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f63500a2800 0x7f63500a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f635c006010 tx=0x7f635c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f6358ff9640 1 -- 172.21.15.22:0/1801711083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c016070 con 0x55afda287c40 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f6358ff9640 1 -- 172.21.15.22:0/1801711083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f634c011950 con 0x55afda287c40 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f6358ff9640 1 -- 172.21.15.22:0/1801711083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c010930 con 0x55afda287c40 2024-03-08T23:53:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f636bb70740 1 -- 172.21.15.22:0/1801711083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda287c40 msgr2=0x55afda28a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda287c40 0x55afda28a0d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f634c009c20 tx=0x7f634c00e6c0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f636bb70740 1 -- 172.21.15.22:0/1801711083 shutdown_connections 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda287c40 0x55afda28a0d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda2875d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801711083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda282500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.237+0000 7f636bb70740 1 -- 172.21.15.22:0/1801711083 >> 172.21.15.22:0/1801711083 conn(0x55afda24a400 msgr2=0x55afda27da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 -- 172.21.15.22:0/1801711083 shutdown_connections 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 -- 172.21.15.22:0/1801711083 wait complete. 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 -- start start 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 Processor -- start 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda2820e0 0x55afda28c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda28cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda293a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda28abf0 con 0x55afda2820e0 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda1a3f90 con 0x55afda282ad0 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda28ad70 con 0x55afda287c40 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636bb70740 10 client.0 _ll_register_callbacks cb 0x55afda1df400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda2820e0 0x55afda28c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda2820e0 0x55afda28c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44924/0 (socket says 172.21.15.22:44924) 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 -- 172.21.15.22:0/1801621362 learned_addr learned my addr 172.21.15.22:0/1801621362 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f635bfff640 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda28cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 -- 172.21.15.22:0/1801621362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 msgr2=0x55afda293a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda293a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 -- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 msgr2=0x55afda28cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda28cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f636259c640 1 -- 172.21.15.22:0/1801621362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afd9efc2d0 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.238+0000 7f635bfff640 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda28cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636259c640 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda2820e0 0x55afda28c580 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f63540ccad0 tx=0x7f635409c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d873d571ddfd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f6349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f6349ffb640 1 -- 172.21.15.22:0/1801621362 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635409fa70 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f6349ffb640 1 -- 172.21.15.22:0/1801621362 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6354003b30 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f634c011950 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f6349ffb640 1 -- 172.21.15.22:0/1801621362 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63540a40c0 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 10 client.15249 fetch_fsmap learned FSMap version 24 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55afd9fc66b0 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f6349ffb640 1 -- 172.21.15.22:0/1801621362 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f634c011950 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f6349ffb640 1 -- 172.21.15.22:0/1801621362 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f635409ad70 con 0x55afda2820e0 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 10 client.15249 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 10 client.15249 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 20 client.15249 populate_metadata read hostname 'smithi022' 2024-03-08T23:53:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda299110 0x55afda298110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 4 client.15249 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.239+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afda1f4d00 con 0x55afda299110 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.240+0000 7f635bfff640 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda299110 0x55afda298110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.240+0000 7f6349ffb640 10 client.15249 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.240+0000 7f635bfff640 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda299110 0x55afda298110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbbc27ac8cb38b1f server_cookie=b41ee405df9fc1af in_seq=0 out_seq=0 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f6349ffb640 1 -- 172.21.15.22:0/1801621362 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55afda1f4d00 con 0x55afda299110 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f6349ffb640 10 client.15249 handle_command_reply: tid=0 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 client.15249 shutdown 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda2820e0 msgr2=0x55afda28c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda2820e0 0x55afda28c580 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f63540ccad0 tx=0x7f635409c110 comp rx=0 tx=0).stop 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 shutdown_connections 2024-03-08T23:53:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda299110 0x55afda298110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda293a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda282ad0 0x55afda28cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f6349ffb640 0 client.15249 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 --2- 172.21.15.22:0/1801621362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda2820e0 0x55afda28c580 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 >> 172.21.15.22:0/1801621362 conn(0x55afda24a400 msgr2=0x55afda27cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 shutdown_connections 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 1 -- 172.21.15.22:0/1801621362 wait complete. 2024-03-08T23:53:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.241+0000 7f636bb70740 20 client.15249 trim_cache size 0 max 16384 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 Processor -- start 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 -- start start 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda283fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda284510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda287c40 0x55afda28bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda298930 con 0x55afda282ad0 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda298f90 con 0x55afda287c40 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636bb70740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda2995f0 con 0x55afda2820e0 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f635bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda284510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f635bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda284510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44938/0 (socket says 172.21.15.22:44938) 2024-03-08T23:53:46.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f635bfff640 1 -- 172.21.15.22:0/898136378 learned_addr learned my addr 172.21.15.22:0/898136378 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.257+0000 7f636259c640 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda283fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f6362d9d640 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda287c40 0x55afda28bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f635bfff640 1 -- 172.21.15.22:0/898136378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 msgr2=0x55afda283fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f635bfff640 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda283fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f635bfff640 1 -- 172.21.15.22:0/898136378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda287c40 msgr2=0x55afda28bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f635bfff640 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda287c40 0x55afda28bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f635bfff640 1 -- 172.21.15.22:0/898136378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afda27dff0 con 0x55afda282ad0 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636259c640 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda283fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f6362d9d640 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda287c40 0x55afda28bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f635bfff640 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda284510 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55afda27e9a0 tx=0x7f634c009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90422e5b10dc56d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f634bfff640 1 -- 172.21.15.22:0/898136378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c02c030 con 0x55afda282ad0 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f634bfff640 1 -- 172.21.15.22:0/898136378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f634c028430 con 0x55afda282ad0 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f634bfff640 1 -- 172.21.15.22:0/898136378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c016040 con 0x55afda282ad0 2024-03-08T23:53:46.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636bb70740 1 -- 172.21.15.22:0/898136378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 msgr2=0x55afda284510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636bb70740 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda284510 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55afda27e9a0 tx=0x7f634c009fb0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636bb70740 1 -- 172.21.15.22:0/898136378 shutdown_connections 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636bb70740 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda287c40 0x55afda28bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636bb70740 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda284510 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636bb70740 1 --2- 172.21.15.22:0/898136378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda2820e0 0x55afda283fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.258+0000 7f636bb70740 1 -- 172.21.15.22:0/898136378 >> 172.21.15.22:0/898136378 conn(0x55afda24a400 msgr2=0x55afda298110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 -- 172.21.15.22:0/898136378 shutdown_connections 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 -- 172.21.15.22:0/898136378 wait complete. 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 -- start start 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 Processor -- start 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda2820e0 0x55afda2eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda2ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda2f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda29a2b0 con 0x55afda282ad0 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda299c50 con 0x55afda2820e0 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afda29a910 con 0x55afda287c40 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636bb70740 10 client.0 _ll_register_callbacks cb 0x7f634c0093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda2ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda2ebc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44940/0 (socket says 172.21.15.22:44940) 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 -- 172.21.15.22:0/4213602607 learned_addr learned my addr 172.21.15.22:0/4213602607 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f636259c640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda2820e0 0x55afda2eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f6362d9d640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda2f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 -- 172.21.15.22:0/4213602607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 msgr2=0x55afda2f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda2f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 -- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda2820e0 msgr2=0x55afda2eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda2820e0 0x55afda2eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f635bfff640 1 -- 172.21.15.22:0/4213602607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afda2f3230 con 0x55afda282ad0 2024-03-08T23:53:46.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.259+0000 7f6362d9d640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda2f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f635bfff640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda2ebc00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f634c0095a0 tx=0x7f634c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afde6a23a6b3668 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f6349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f6349ffb640 1 -- 172.21.15.22:0/4213602607 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c035070 con 0x55afda282ad0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f6349ffb640 1 -- 172.21.15.22:0/4213602607 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f634c0051e0 con 0x55afda282ad0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f634c028430 con 0x55afda282ad0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f6349ffb640 1 -- 172.21.15.22:0/4213602607 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f634c040050 con 0x55afda282ad0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 10 client.15255 fetch_fsmap learned FSMap version 24 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55afda2f44a0 con 0x55afda282ad0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f6349ffb640 1 -- 172.21.15.22:0/4213602607 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f634c028430 con 0x55afda282ad0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f6349ffb640 1 -- 172.21.15.22:0/4213602607 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f634c028ce0 con 0x55afda282ad0 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 10 client.15255 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 20 client.15255 populate_metadata read hostname 'smithi022' 2024-03-08T23:53:46.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda2f84d0 0x55afda2f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 4 client.15255 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.260+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55afda1ef050 con 0x55afda2f84d0 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636259c640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda2f84d0 0x55afda2f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636259c640 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda2f84d0 0x55afda2f4670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d22823cb49eee362 server_cookie=faa5a6dfb84aaffd in_seq=0 out_seq=0 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f6349ffb640 10 client.15255 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f6349ffb640 1 -- 172.21.15.22:0/4213602607 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55afda1ef050 con 0x55afda2f84d0 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f6349ffb640 10 client.15255 handle_command_reply: tid=0 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 client.15255 shutdown 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 msgr2=0x55afda2ebc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda2ebc00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f634c0095a0 tx=0x7f634c0092a0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 shutdown_connections 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55afda2f84d0 0x55afda2f4670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afda287c40 0x55afda2f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f6349ffb640 0 client.15255 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55afda282ad0 0x55afda2ebc00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 --2- 172.21.15.22:0/4213602607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55afda2820e0 0x55afda2eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 >> 172.21.15.22:0/4213602607 conn(0x55afda24a400 msgr2=0x55afda283fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.261+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 shutdown_connections 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f636bb70740 1 -- 172.21.15.22:0/4213602607 wait complete. 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f636bb70740 20 client.15255 trim_cache size 0 max 16384 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24494, 2024-03-08T23:53:46.268 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 89.940012483999993, 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 89, 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 118.29465573100001 2024-03-08T23:53:46.269 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f63500a2800 msgr2=0x7f63500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f63500a2800 0x7f63500a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f635c006010 tx=0x7f635c02e040 comp rx=0 tx=0).stop 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364101310 msgr2=0x7f6364197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364101310 0x7f6364197580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9c0 tx=0x7f635400ee80 comp rx=0 tx=0).stop 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 shutdown_connections 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6364102ec0 0x7f636419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6364102500 0x7f6364197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f63500a2800 0x7f63500a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 --2- 172.21.15.22:0/3687518158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6364101310 0x7f6364197580 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 >> 172.21.15.22:0/3687518158 conn(0x7f63640fc880 msgr2=0x7f6364105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 shutdown_connections 2024-03-08T23:53:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.262+0000 7f6368f96640 1 -- 172.21.15.22:0/3687518158 wait complete. 2024-03-08T23:53:46.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-08T23:53:46.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- 172.21.15.22:0/3396975435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c101dc0 msgr2=0x7f0f5c108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/3396975435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c101dc0 0x7f0f5c108650 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009ca0 tx=0x7f0f5402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- 172.21.15.22:0/3396975435 shutdown_connections 2024-03-08T23:53:46.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/3396975435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c101dc0 0x7f0f5c108650 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/3396975435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f5c101400 0x7f0f5c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/3396975435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f5c100210 0x7f0f5c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- 172.21.15.22:0/3396975435 >> 172.21.15.22:0/3396975435 conn(0x7f0f5c0fb7a0 msgr2=0x7f0f5c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- 172.21.15.22:0/3396975435 shutdown_connections 2024-03-08T23:53:46.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- 172.21.15.22:0/3396975435 wait complete. 2024-03-08T23:53:46.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 Processor -- start 2024-03-08T23:53:46.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- start start 2024-03-08T23:53:46.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c100210 0x7f0f5c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f5c101400 0x7f0f5c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f5c101dc0 0x7f0f5c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c10bea0 con 0x7f0f5c100210 2024-03-08T23:53:46.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c10bd20 con 0x7f0f5c101400 2024-03-08T23:53:46.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.379+0000 7f0f63ef8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c10c020 con 0x7f0f5c101dc0 2024-03-08T23:53:46.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c100210 0x7f0f5c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c100210 0x7f0f5c06a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44952/0 (socket says 172.21.15.22:44952) 2024-03-08T23:53:46.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 -- 172.21.15.22:0/4213584114 learned_addr learned my addr 172.21.15.22:0/4213584114 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61493640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f5c101400 0x7f0f5c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f62495640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f5c101dc0 0x7f0f5c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 -- 172.21.15.22:0/4213584114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f5c101dc0 msgr2=0x7f0f5c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f5c101dc0 0x7f0f5c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 -- 172.21.15.22:0/4213584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f5c101400 msgr2=0x7f0f5c19c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f5c101400 0x7f0f5c19c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 -- 172.21.15.22:0/4213584114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c19f230 con 0x7f0f5c100210 2024-03-08T23:53:46.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61493640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f5c101400 0x7f0f5c19c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:46.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f61c94640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c100210 0x7f0f5c06a3d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800d950 tx=0x7f0f5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f52ffd640 1 -- 172.21.15.22:0/4213584114 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58017070 con 0x7f0f5c100210 2024-03-08T23:53:46.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f52ffd640 1 -- 172.21.15.22:0/4213584114 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f58009c30 con 0x7f0f5c100210 2024-03-08T23:53:46.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c19f520 con 0x7f0f5c100210 2024-03-08T23:53:46.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f52ffd640 1 -- 172.21.15.22:0/4213584114 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800f600 con 0x7f0f5c100210 2024-03-08T23:53:46.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.380+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c19fa20 con 0x7f0f5c100210 2024-03-08T23:53:46.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.383+0000 7f0f52ffd640 1 -- 172.21.15.22:0/4213584114 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0f5801a070 con 0x7f0f5c100210 2024-03-08T23:53:46.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.383+0000 7f0f66bb8740 1 Processor -- start 2024-03-08T23:53:46.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.383+0000 7f0f66bb8740 1 -- start start 2024-03-08T23:53:46.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.383+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c980e0 0x55a001c98500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c98ad0 0x55a001c9d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001bfd7c0 con 0x55a001c9dc40 2024-03-08T23:53:46.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001ba6b50 con 0x55a001c98ad0 2024-03-08T23:53:46.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001bf9860 con 0x55a001c980e0 2024-03-08T23:53:46.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44956/0 (socket says 172.21.15.22:44956) 2024-03-08T23:53:46.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 -- 172.21.15.22:0/241431883 learned_addr learned my addr 172.21.15.22:0/241431883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f52ffd640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0f480a2800 0x7f0f480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61c94640 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c98ad0 0x55a001c9d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f62495640 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c980e0 0x55a001c98500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 -- 172.21.15.22:0/241431883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c980e0 msgr2=0x55a001c98500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c980e0 0x55a001c98500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f62495640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0f480a2800 0x7f0f480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 -- 172.21.15.22:0/241431883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c98ad0 msgr2=0x55a001c9d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c98ad0 0x55a001c9d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 -- 172.21.15.22:0/241431883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a001bba9a0 con 0x55a001c9dc40 2024-03-08T23:53:46.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f52ffd640 1 -- 172.21.15.22:0/4213584114 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f580ce960 con 0x7f0f5c100210 2024-03-08T23:53:46.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f61493640 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca00d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00cc50 tx=0x7f0f4c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdc237b73cc3b7e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f47fff640 1 -- 172.21.15.22:0/241431883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c014070 con 0x55a001c9dc40 2024-03-08T23:53:46.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f62495640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0f480a2800 0x7f0f480a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f54005e30 tx=0x7f0f540095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f47fff640 1 -- 172.21.15.22:0/241431883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c019b40 con 0x55a001c9dc40 2024-03-08T23:53:46.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.384+0000 7f0f47fff640 1 -- 172.21.15.22:0/241431883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c010c10 con 0x55a001c9dc40 2024-03-08T23:53:46.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 -- 172.21.15.22:0/241431883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 msgr2=0x55a001ca00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca00d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00cc50 tx=0x7f0f4c00e720 comp rx=0 tx=0).stop 2024-03-08T23:53:46.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 -- 172.21.15.22:0/241431883 shutdown_connections 2024-03-08T23:53:46.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca00d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c98ad0 0x55a001c9d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/241431883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c980e0 0x55a001c98500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 -- 172.21.15.22:0/241431883 >> 172.21.15.22:0/241431883 conn(0x55a001c60400 msgr2=0x55a001c93a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 -- 172.21.15.22:0/241431883 shutdown_connections 2024-03-08T23:53:46.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.385+0000 7f0f66bb8740 1 -- 172.21.15.22:0/241431883 wait complete. 2024-03-08T23:53:46.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 -- start start 2024-03-08T23:53:46.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 Processor -- start 2024-03-08T23:53:46.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c98ad0 0x55a001ca2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001ca0bf0 con 0x55a001c9dc40 2024-03-08T23:53:46.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001bb9f90 con 0x55a001c980e0 2024-03-08T23:53:46.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001ca0d70 con 0x55a001c98ad0 2024-03-08T23:53:46.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f66bb8740 10 client.0 _ll_register_callbacks cb 0x55a001bf5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:53:46.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44958/0 (socket says 172.21.15.22:44958) 2024-03-08T23:53:46.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 -- 172.21.15.22:0/1974609890 learned_addr learned my addr 172.21.15.22:0/1974609890 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f61c94640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f61493640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c98ad0 0x55a001ca2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 -- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c98ad0 msgr2=0x55a001ca2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c98ad0 0x55a001ca2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 -- 172.21.15.22:0/1974609890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 msgr2=0x55a001ca25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.386+0000 7f0f62495640 1 -- 172.21.15.22:0/1974609890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0019122d0 con 0x55a001c9dc40 2024-03-08T23:53:46.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f61493640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c98ad0 0x55a001ca2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:46.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f62495640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca9b00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0f54036000 tx=0x7f0f5402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4a0d8ffc5b5a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f457fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:53:46.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f457fa640 1 -- 172.21.15.22:0/1974609890 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54051060 con 0x55a001c9dc40 2024-03-08T23:53:46.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f457fa640 1 -- 172.21.15.22:0/1974609890 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f54004040 con 0x55a001c9dc40 2024-03-08T23:53:46.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f4c019b40 con 0x55a001c9dc40 2024-03-08T23:53:46.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f457fa640 1 -- 172.21.15.22:0/1974609890 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f54058050 con 0x55a001c9dc40 2024-03-08T23:53:46.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 10 client.15267 fetch_fsmap learned FSMap version 24 2024-03-08T23:53:46.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a0019dc6b0 con 0x55a001c9dc40 2024-03-08T23:53:46.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f457fa640 1 -- 172.21.15.22:0/1974609890 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c019b40 con 0x55a001c9dc40 2024-03-08T23:53:46.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f457fa640 1 -- 172.21.15.22:0/1974609890 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f54041a40 con 0x55a001c9dc40 2024-03-08T23:53:46.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 10 client.15267 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:53:46.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 10 client.15267 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T23:53:46.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 20 client.15267 populate_metadata read hostname 'smithi022' 2024-03-08T23:53:46.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001caf180 0x55a001cae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 4 client.15267 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:53:46.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.387+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a001c0ad00 con 0x55a001caf180 2024-03-08T23:53:46.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.388+0000 7f0f61c94640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001caf180 0x55a001cae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.388+0000 7f0f61c94640 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001caf180 0x55a001cae220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d7858c03a6fd64 server_cookie=387210bfaf8bd4b3 in_seq=0 out_seq=0 2024-03-08T23:53:46.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.388+0000 7f0f457fa640 10 client.15267 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:53:46.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f457fa640 1 -- 172.21.15.22:0/1974609890 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a001c0ad00 con 0x55a001caf180 2024-03-08T23:53:46.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f457fa640 10 client.15267 handle_command_reply: tid=0 2024-03-08T23:53:46.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 client.15267 shutdown 2024-03-08T23:53:46.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 msgr2=0x55a001ca9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca9b00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0f54036000 tx=0x7f0f5402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 shutdown_connections 2024-03-08T23:53:46.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001caf180 0x55a001cae220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c9dc40 0x55a001ca9b00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c98ad0 0x55a001ca2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/1974609890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f457fa640 0 client.15267 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:53:46.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 >> 172.21.15.22:0/1974609890 conn(0x55a001c60400 msgr2=0x55a001c92c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 shutdown_connections 2024-03-08T23:53:46.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 1 -- 172.21.15.22:0/1974609890 wait complete. 2024-03-08T23:53:46.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.389+0000 7f0f66bb8740 20 client.15267 trim_cache size 0 max 16384 2024-03-08T23:53:46.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 Processor -- start 2024-03-08T23:53:46.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 -- start start 2024-03-08T23:53:46.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001c99fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001c9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001cae9a0 con 0x55a001c98ad0 2024-03-08T23:53:46.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001caf000 con 0x55a001c980e0 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001caf660 con 0x55a001c9dc40 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44966/0 (socket says 172.21.15.22:44966) 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 -- 172.21.15.22:0/3884517216 learned_addr learned my addr 172.21.15.22:0/3884517216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61c94640 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001c99fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f62495640 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001c9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 -- 172.21.15.22:0/3884517216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 msgr2=0x55a001c9a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001c9a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 -- 172.21.15.22:0/3884517216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 msgr2=0x55a001c99fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001c99fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 -- 172.21.15.22:0/3884517216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a001ca23e0 con 0x55a001c98ad0 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.405+0000 7f0f61493640 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca9ac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00cc50 tx=0x7f0f4c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3247d038484fd0b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f477fe640 1 -- 172.21.15.22:0/3884517216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c014050 con 0x55a001c98ad0 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f477fe640 1 -- 172.21.15.22:0/3884517216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c004020 con 0x55a001c98ad0 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f477fe640 1 -- 172.21.15.22:0/3884517216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c019a70 con 0x55a001c98ad0 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3884517216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 msgr2=0x55a001ca9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca9ac0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00cc50 tx=0x7f0f4c00c9a0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3884517216 shutdown_connections 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001c9a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca9ac0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3884517216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001c99fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3884517216 >> 172.21.15.22:0/3884517216 conn(0x55a001c60400 msgr2=0x55a001ca1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3884517216 shutdown_connections 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3884517216 wait complete. 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.406+0000 7f0f66bb8740 1 -- start start 2024-03-08T23:53:46.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 1 Processor -- start 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001d04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001cb0320 con 0x55a001c98ad0 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001cafcc0 con 0x55a001c980e0 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a001cb0980 con 0x55a001c9dc40 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f66bb8740 10 client.0 _ll_register_callbacks cb 0x7f0f4c00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44982/0 (socket says 172.21.15.22:44982) 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 -- 172.21.15.22:0/3318747412 learned_addr learned my addr 172.21.15.22:0/3318747412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61c94640 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f62495640 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001d04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 -- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 msgr2=0x55a001d04d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001d04d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 -- 172.21.15.22:0/3318747412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 msgr2=0x55a001ca6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 -- 172.21.15.22:0/3318747412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a001d052c0 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f61493640 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca6ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0058c0 tx=0x7f0f4c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c70ac2460d1950 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.407+0000 7f0f457fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f457fa640 1 -- 172.21.15.22:0/3318747412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0199c0 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f4c004020 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f457fa640 1 -- 172.21.15.22:0/3318747412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f4c005200 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f457fa640 1 -- 172.21.15.22:0/3318747412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f4c03b050 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 10 client.15273 fetch_fsmap learned FSMap version 24 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a001d06560 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f457fa640 1 -- 172.21.15.22:0/3318747412 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c004020 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f457fa640 1 -- 172.21.15.22:0/3318747412 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f4c0035f0 con 0x55a001c98ad0 2024-03-08T23:53:46.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 10 client.15273 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 20 client.15273 populate_metadata read hostname 'smithi022' 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001d0a6b0 0x55a001d06730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 4 client.15273 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a001c05050 con 0x55a001d0a6b0 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.408+0000 7f0f61c94640 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001d0a6b0 0x55a001d06730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f457fa640 10 client.15273 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f61c94640 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001d0a6b0 0x55a001d06730 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73cdf3bfdd457488 server_cookie=408f6b748d72f5ff in_seq=0 out_seq=0 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f457fa640 1 -- 172.21.15.22:0/3318747412 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a001c05050 con 0x55a001d0a6b0 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f457fa640 10 client.15273 handle_command_reply: tid=0 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 client.15273 shutdown 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 msgr2=0x55a001ca6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca6ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0058c0 tx=0x7f0f4c00c8b0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 shutdown_connections 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a001d0a6b0 0x55a001d06730 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a001c9dc40 0x55a001d04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a001c98ad0 0x55a001ca6ed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 --2- 172.21.15.22:0/3318747412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a001c980e0 0x55a001ca6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f457fa640 0 client.15273 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 >> 172.21.15.22:0/3318747412 conn(0x55a001c60400 msgr2=0x55a001ca1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 shutdown_connections 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 1 -- 172.21.15.22:0/3318747412 wait complete. 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.409+0000 7f0f66bb8740 20 client.15273 trim_cache size 0 max 16384 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:53:46.431 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24496, 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 114.17184197100001, 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 89, 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 126.857753163 2024-03-08T23:53:46.432 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:53:46.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0f480a2800 msgr2=0x7f0f480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0f480a2800 0x7f0f480a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f54005e30 tx=0x7f0f540095c0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c100210 msgr2=0x7f0f5c06a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c100210 0x7f0f5c06a3d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800d950 tx=0x7f0f5800de10 comp rx=0 tx=0).stop 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 shutdown_connections 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f5c101dc0 0x7f0f5c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f5c101400 0x7f0f5c19c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0f480a2800 0x7f0f480a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 --2- 172.21.15.22:0/4213584114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f5c100210 0x7f0f5c06a3d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 >> 172.21.15.22:0/4213584114 conn(0x7f0f5c0fb7a0 msgr2=0x7f0f5c1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 shutdown_connections 2024-03-08T23:53:46.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:46.410+0000 7f0f63ef8640 1 -- 172.21.15.22:0/4213584114 wait complete. 2024-03-08T23:53:46.435 INFO:tasks.cephadm:Setup complete, yielding 2024-03-08T23:53:46.435 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-08T23:53:46.445 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-08T23:53:46.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- bash -c 'ceph orch status' 2024-03-08T23:53:46.688 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:46 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/815175036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:46 smithi192 ceph-mon[35995]: osdmap e93: 12 total, 12 up, 12 in 2024-03-08T23:53:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:46 smithi192 ceph-mon[35995]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-08T23:53:46.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:46 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1720703799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:46.952 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:46 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/815175036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:46.952 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:46 smithi022 ceph-mon[31894]: osdmap e93: 12 total, 12 up, 12 in 2024-03-08T23:53:46.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:46 smithi022 ceph-mon[31894]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-08T23:53:46.953 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:46 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1720703799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:47.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:46 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/815175036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:53:47.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:46 smithi062 ceph-mon[36129]: osdmap e93: 12 total, 12 up, 12 in 2024-03-08T23:53:47.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:46 smithi062 ceph-mon[36129]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-08T23:53:47.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:46 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1720703799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:53:47.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:47 smithi192 ceph-mon[35995]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s 2024-03-08T23:53:47.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:47 smithi192 ceph-mon[35995]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-08T23:53:47.971 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:47 smithi022 ceph-mon[31894]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s 2024-03-08T23:53:47.971 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:47 smithi022 ceph-mon[31894]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-08T23:53:48.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:47 smithi062 ceph-mon[36129]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s 2024-03-08T23:53:48.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:47 smithi062 ceph-mon[36129]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-08T23:53:48.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 -- 172.21.15.22:0/3126577502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25681028a0 msgr2=0x7f2568109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:48.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 --2- 172.21.15.22:0/3126577502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25681028a0 0x7f2568109150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f256000b0d0 tx=0x7f256000b3e0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 -- 172.21.15.22:0/3126577502 shutdown_connections 2024-03-08T23:53:48.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 --2- 172.21.15.22:0/3126577502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25681028a0 0x7f2568109150 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 --2- 172.21.15.22:0/3126577502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2568100d00 0x7f2568101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 --2- 172.21.15.22:0/3126577502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2568101f00 0x7f2568102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 -- 172.21.15.22:0/3126577502 >> 172.21.15.22:0/3126577502 conn(0x7f25680fc2a0 msgr2=0x7f25680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 -- 172.21.15.22:0/3126577502 shutdown_connections 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 -- 172.21.15.22:0/3126577502 wait complete. 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 Processor -- start 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.547+0000 7f256f4d3700 1 -- start start 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256f4d3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2568100d00 0x7f256806b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256f4d3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2568101f00 0x7f256819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256f4d3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25681028a0 0x7f256819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256f4d3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810c960 con 0x7f2568101f00 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256f4d3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810c7e0 con 0x7f2568100d00 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256f4d3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256810cae0 con 0x7f25681028a0 2024-03-08T23:53:48.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2568101f00 0x7f256819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2568101f00 0x7f256819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44988/0 (socket says 172.21.15.22:44988) 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 -- 172.21.15.22:0/403678908 learned_addr learned my addr 172.21.15.22:0/403678908 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256d26f700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2568100d00 0x7f256806b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256da70700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25681028a0 0x7f256819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 -- 172.21.15.22:0/403678908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25681028a0 msgr2=0x7f256819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25681028a0 0x7f256819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 -- 172.21.15.22:0/403678908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2568100d00 msgr2=0x7f256806b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2568100d00 0x7f256806b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 -- 172.21.15.22:0/403678908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256819fc20 con 0x7f2568101f00 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f256ca6e700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2568101f00 0x7f256819ccc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f255800c9b0 tx=0x7f255800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.548+0000 7f255e7fc700 1 -- 172.21.15.22:0/403678908 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558007b00 con 0x7f2568101f00 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.549+0000 7f255e7fc700 1 -- 172.21.15.22:0/403678908 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f255800ce50 con 0x7f2568101f00 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.549+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256819ff10 con 0x7f2568101f00 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.549+0000 7f255e7fc700 1 -- 172.21.15.22:0/403678908 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558018660 con 0x7f2568101f00 2024-03-08T23:53:48.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.549+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a0490 con 0x7f2568101f00 2024-03-08T23:53:48.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.550+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2568068aa0 con 0x7f2568101f00 2024-03-08T23:53:48.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.551+0000 7f255e7fc700 1 -- 172.21.15.22:0/403678908 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f255800f450 con 0x7f2568101f00 2024-03-08T23:53:48.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.551+0000 7f255e7fc700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f25540a2910 0x7f25540a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:48.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.552+0000 7f256d26f700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f25540a2910 0x7f25540a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:48.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.552+0000 7f255e7fc700 1 -- 172.21.15.22:0/403678908 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f25580cec50 con 0x7f2568101f00 2024-03-08T23:53:48.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.552+0000 7f256d26f700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f25540a2910 0x7f25540a4dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2568101d60 tx=0x7f2564009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.556+0000 7f255e7fc700 1 -- 172.21.15.22:0/403678908 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f25580d4020 con 0x7f2568101f00 2024-03-08T23:53:48.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.780+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2568062220 con 0x7f25540a2910 2024-03-08T23:53:48.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.781+0000 7f255e7fc700 1 -- 172.21.15.22:0/403678908 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2568062220 con 0x7f25540a2910 2024-03-08T23:53:48.777 INFO:teuthology.orchestra.run.smithi022.stdout:Backend: cephadm 2024-03-08T23:53:48.777 INFO:teuthology.orchestra.run.smithi022.stdout:Available: Yes 2024-03-08T23:53:48.777 INFO:teuthology.orchestra.run.smithi022.stdout:Paused: No 2024-03-08T23:53:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f25540a2910 msgr2=0x7f25540a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f25540a2910 0x7f25540a4dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2568101d60 tx=0x7f2564009190 comp rx=0 tx=0).stop 2024-03-08T23:53:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2568101f00 msgr2=0x7f256819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:48.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2568101f00 0x7f256819ccc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f255800c9b0 tx=0x7f255800ccc0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 shutdown_connections 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25681028a0 0x7f256819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2568100d00 0x7f256806b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f25540a2910 0x7f25540a4dd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 --2- 172.21.15.22:0/403678908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2568101f00 0x7f256819ccc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 >> 172.21.15.22:0/403678908 conn(0x7f25680fc2a0 msgr2=0x7f2568105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 shutdown_connections 2024-03-08T23:53:48.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:48.783+0000 7f256f4d3700 1 -- 172.21.15.22:0/403678908 wait complete. 2024-03-08T23:53:49.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-08T23:53:49.786 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:49.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:49 smithi192 ceph-mon[35995]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-08T23:53:49.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:49 smithi192 ceph-mon[35995]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:49.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:49 smithi022 ceph-mon[31894]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-08T23:53:49.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:49 smithi022 ceph-mon[31894]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:50.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:49 smithi062 ceph-mon[36129]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-08T23:53:50.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:49 smithi062 ceph-mon[36129]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 -- 172.21.15.22:0/3217992534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c101f80 msgr2=0x7fa44c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 --2- 172.21.15.22:0/3217992534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c101f80 0x7fa44c1023e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0099c0 tx=0x7fa43c009cd0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 -- 172.21.15.22:0/3217992534 shutdown_connections 2024-03-08T23:53:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 --2- 172.21.15.22:0/3217992534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa44c102920 0x7fa44c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 --2- 172.21.15.22:0/3217992534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44c100d80 0x7fa44c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 --2- 172.21.15.22:0/3217992534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c101f80 0x7fa44c1023e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 -- 172.21.15.22:0/3217992534 >> 172.21.15.22:0/3217992534 conn(0x7fa44c0fc340 msgr2=0x7fa44c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 -- 172.21.15.22:0/3217992534 shutdown_connections 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 -- 172.21.15.22:0/3217992534 wait complete. 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 Processor -- start 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.655+0000 7fa4547eb700 1 -- start start 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4547eb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c100d80 0x7fa44c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4547eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44c101f80 0x7fa44c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4547eb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa44c102920 0x7fa44c1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4547eb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c10c990 con 0x7fa44c100d80 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4547eb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c10c810 con 0x7fa44c101f80 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4547eb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c10cb10 con 0x7fa44c102920 2024-03-08T23:53:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c100d80 0x7fa44c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c100d80 0x7fa44c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42670/0 (socket says 172.21.15.22:42670) 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 -- 172.21.15.22:0/2328918248 learned_addr learned my addr 172.21.15.22:0/2328918248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452d88700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa44c102920 0x7fa44c1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa451d86700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44c101f80 0x7fa44c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 -- 172.21.15.22:0/2328918248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa44c102920 msgr2=0x7fa44c1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa44c102920 0x7fa44c1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 -- 172.21.15.22:0/2328918248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44c101f80 msgr2=0x7fa44c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44c101f80 0x7fa44c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 -- 172.21.15.22:0/2328918248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c1a1f30 con 0x7fa44c100d80 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa451d86700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44c101f80 0x7fa44c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa452587700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c100d80 0x7fa44c19a2d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa44800b530 tx=0x7fa44800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4437fe700 1 -- 172.21.15.22:0/2328918248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448015020 con 0x7fa44c100d80 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.656+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c1a2220 con 0x7fa44c100d80 2024-03-08T23:53:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.657+0000 7fa4437fe700 1 -- 172.21.15.22:0/2328918248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa448011d10 con 0x7fa44c100d80 2024-03-08T23:53:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.657+0000 7fa4437fe700 1 -- 172.21.15.22:0/2328918248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44800f2e0 con 0x7fa44c100d80 2024-03-08T23:53:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.657+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c1a2830 con 0x7fa44c100d80 2024-03-08T23:53:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.658+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c068aa0 con 0x7fa44c100d80 2024-03-08T23:53:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.659+0000 7fa4437fe700 1 -- 172.21.15.22:0/2328918248 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa448018020 con 0x7fa44c100d80 2024-03-08T23:53:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.659+0000 7fa4437fe700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa4380a2890 0x7fa4380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.659+0000 7fa4437fe700 1 -- 172.21.15.22:0/2328918248 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa4480ce9c0 con 0x7fa44c100d80 2024-03-08T23:53:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.659+0000 7fa451d86700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa4380a2890 0x7fa4380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.660+0000 7fa451d86700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa4380a2890 0x7fa4380a4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa43c000c00 tx=0x7fa43c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.663+0000 7fa4437fe700 1 -- 172.21.15.22:0/2328918248 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa44801f020 con 0x7fa44c100d80 2024-03-08T23:53:51.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:51 smithi022 ceph-mon[31894]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 8 op/s 2024-03-08T23:53:51.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.878+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa44c0fe4d0 con 0x7fa4380a2890 2024-03-08T23:53:51.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:51 smithi192 ceph-mon[35995]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 8 op/s 2024-03-08T23:53:51.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.892+0000 7fa4437fe700 1 -- 172.21.15.22:0/2328918248 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4092 (secure 0 0 0) 0x7fa44c0fe4d0 con 0x7fa4380a2890 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.a smithi022 running (2m) 31s ago 2m 12.9M - 19.0.0-1965-g7d63482a a272146cb810 8957338e3b2f 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.b smithi062 running (2m) 57s ago 2m 16.3M - 19.0.0-1965-g7d63482a a272146cb810 e0f4eae37122 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.c smithi192 running (2m) 57s ago 2m 16.2M - 19.0.0-1965-g7d63482a a272146cb810 3fca2d1aea84 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.d smithi022 running (2m) 31s ago 2m 14.0M - 19.0.0-1965-g7d63482a a272146cb810 859152d61f58 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.e smithi062 running (2m) 57s ago 2m 16.0M - 19.0.0-1965-g7d63482a a272146cb810 898377f6b379 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.f smithi192 running (2m) 57s ago 2m 15.1M - 19.0.0-1965-g7d63482a a272146cb810 f9a3eec91580 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.g smithi022 running (2m) 31s ago 2m 15.3M - 19.0.0-1965-g7d63482a a272146cb810 56d084ddba18 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.h smithi062 running (2m) 57s ago 2m 17.2M - 19.0.0-1965-g7d63482a a272146cb810 12de8012086c 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.i smithi192 running (2m) 57s ago 2m 25.3M - 19.0.0-1965-g7d63482a a272146cb810 60a42d821339 2024-03-08T23:53:51.889 INFO:teuthology.orchestra.run.smithi022.stdout:mds.j smithi022 running (2m) 31s ago 2m 16.4M - 19.0.0-1965-g7d63482a a272146cb810 962b59335e90 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mds.k smithi062 running (2m) 57s ago 2m 14.0M - 19.0.0-1965-g7d63482a a272146cb810 d0630ecc7c86 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mds.l smithi192 running (2m) 57s ago 2m 16.3M - 19.0.0-1965-g7d63482a a272146cb810 7f6a1d705b91 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.x smithi022 *:9283,8765 running (9m) 31s ago 9m 514M - 19.0.0-1965-g7d63482a a272146cb810 a551c7f49a0d 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.y smithi062 *:8443,8765 running (7m) 57s ago 7m 451M - 19.0.0-1965-g7d63482a a272146cb810 6914d8786658 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.z smithi192 *:8443,8765 running (7m) 57s ago 7m 451M - 19.0.0-1965-g7d63482a a272146cb810 44bdf83cbba1 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a smithi022 running (9m) 31s ago 10m 50.6M 2048M 19.0.0-1965-g7d63482a a272146cb810 6c0c3ed4de50 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b smithi062 running (8m) 57s ago 8m 42.9M 2048M 19.0.0-1965-g7d63482a a272146cb810 18fb8795a387 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c smithi192 running (8m) 57s ago 8m 40.1M 2048M 19.0.0-1965-g7d63482a a272146cb810 00f2a2c5a4e4 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (7m) 31s ago 7m 53.5M 4273M 19.0.0-1965-g7d63482a a272146cb810 5cf56178b529 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi062 running (7m) 57s ago 7m 47.6M 4270M 19.0.0-1965-g7d63482a a272146cb810 d4195be68f00 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi192 running (6m) 57s ago 6m 45.9M 4270M 19.0.0-1965-g7d63482a a272146cb810 8ca3932f4926 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi022 running (6m) 31s ago 6m 52.9M 4273M 19.0.0-1965-g7d63482a a272146cb810 33a88f6a2a0b 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi062 running (5m) 57s ago 5m 42.9M 4270M 19.0.0-1965-g7d63482a a272146cb810 95d2bdb2b314 2024-03-08T23:53:51.890 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi192 running (5m) 57s ago 5m 47.7M 4270M 19.0.0-1965-g7d63482a a272146cb810 e21f7cb1c6f2 2024-03-08T23:53:51.891 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 smithi022 running (5m) 31s ago 5m 49.1M 4273M 19.0.0-1965-g7d63482a a272146cb810 a07cabf19e78 2024-03-08T23:53:51.891 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 smithi062 running (4m) 57s ago 4m 44.5M 4270M 19.0.0-1965-g7d63482a a272146cb810 fa4d34717fc8 2024-03-08T23:53:51.891 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 smithi192 running (4m) 57s ago 4m 45.3M 4270M 19.0.0-1965-g7d63482a a272146cb810 4ae33a334892 2024-03-08T23:53:51.891 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 smithi022 running (3m) 31s ago 3m 43.8M 4273M 19.0.0-1965-g7d63482a a272146cb810 50eee3980a2c 2024-03-08T23:53:51.891 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 smithi062 running (3m) 57s ago 3m 44.4M 4270M 19.0.0-1965-g7d63482a a272146cb810 2973e1683e53 2024-03-08T23:53:51.891 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 smithi192 running (2m) 57s ago 2m 41.7M 4270M 19.0.0-1965-g7d63482a a272146cb810 25e450962a9c 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa4380a2890 msgr2=0x7fa4380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa4380a2890 0x7fa4380a4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa43c000c00 tx=0x7fa43c005fd0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c100d80 msgr2=0x7fa44c19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c100d80 0x7fa44c19a2d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa44800b530 tx=0x7fa44800b840 comp rx=0 tx=0).stop 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 shutdown_connections 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa44c102920 0x7fa44c1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa4380a2890 0x7fa4380a4d50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa44c100d80 0x7fa44c19a2d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 --2- 172.21.15.22:0/2328918248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa44c101f80 0x7fa44c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:51.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 >> 172.21.15.22:0/2328918248 conn(0x7fa44c0fc340 msgr2=0x7fa44c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:51.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 shutdown_connections 2024-03-08T23:53:51.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:51.894+0000 7fa4547eb700 1 -- 172.21.15.22:0/2328918248 wait complete. 2024-03-08T23:53:52.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:51 smithi062 ceph-mon[36129]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 8 op/s 2024-03-08T23:53:52.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-08T23:53:52.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:52 smithi022 ceph-mon[31894]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-08T23:53:52.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:52 smithi022 ceph-mon[31894]: Cluster is now healthy 2024-03-08T23:53:53.103 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:53.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:52 smithi062 ceph-mon[36129]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-08T23:53:53.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:52 smithi062 ceph-mon[36129]: Cluster is now healthy 2024-03-08T23:53:53.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:52 smithi192 ceph-mon[35995]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-08T23:53:53.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:52 smithi192 ceph-mon[35995]: Cluster is now healthy 2024-03-08T23:53:53.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:53 smithi022 ceph-mon[31894]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:53.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:53 smithi022 ceph-mon[31894]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:54.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:53 smithi062 ceph-mon[36129]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:54.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:53 smithi062 ceph-mon[36129]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:54.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:53 smithi192 ceph-mon[35995]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:54.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:53 smithi192 ceph-mon[35995]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:54.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 -- 172.21.15.22:0/2031603026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080101f80 msgr2=0x7fb0801023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:54.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 --2- 172.21.15.22:0/2031603026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080101f80 0x7fb0801023e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb07800b0d0 tx=0x7fb07800b3e0 comp rx=0 tx=0).stop 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 -- 172.21.15.22:0/2031603026 shutdown_connections 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 --2- 172.21.15.22:0/2031603026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb080102920 0x7fb0801091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 --2- 172.21.15.22:0/2031603026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080100d80 0x7fb0801011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 --2- 172.21.15.22:0/2031603026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080101f80 0x7fb0801023e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 -- 172.21.15.22:0/2031603026 >> 172.21.15.22:0/2031603026 conn(0x7fb0800fc340 msgr2=0x7fb0800fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 -- 172.21.15.22:0/2031603026 shutdown_connections 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 -- 172.21.15.22:0/2031603026 wait complete. 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.957+0000 7fb087d9d700 1 Processor -- start 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb087d9d700 1 -- start start 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb087d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080100d80 0x7fb08006b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb087d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb080101f80 0x7fb08019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb087d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080102920 0x7fb0801a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb087d9d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08010c8d0 con 0x7fb080100d80 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb087d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08010c750 con 0x7fb080101f80 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb087d9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08010ca50 con 0x7fb080102920 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080100d80 0x7fb08006b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080100d80 0x7fb08006b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42694/0 (socket says 172.21.15.22:42694) 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 -- 172.21.15.22:0/3793140097 learned_addr learned my addr 172.21.15.22:0/3793140097 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085338700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb080101f80 0x7fb08019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb08633a700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080102920 0x7fb0801a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 -- 172.21.15.22:0/3793140097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080102920 msgr2=0x7fb0801a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080102920 0x7fb0801a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 -- 172.21.15.22:0/3793140097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb080101f80 msgr2=0x7fb08019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb080101f80 0x7fb08019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 -- 172.21.15.22:0/3793140097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0801a1e70 con 0x7fb080100d80 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085338700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb080101f80 0x7fb08019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.958+0000 7fb085b39700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080100d80 0x7fb08006b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00d980 tx=0x7fb07c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.959+0000 7fb076ffd700 1 -- 172.21.15.22:0/3793140097 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c017030 con 0x7fb080100d80 2024-03-08T23:53:54.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.959+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0801a2160 con 0x7fb080100d80 2024-03-08T23:53:54.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.959+0000 7fb076ffd700 1 -- 172.21.15.22:0/3793140097 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb07c004500 con 0x7fb080100d80 2024-03-08T23:53:54.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.959+0000 7fb076ffd700 1 -- 172.21.15.22:0/3793140097 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c003aa0 con 0x7fb080100d80 2024-03-08T23:53:54.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.959+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0801a26f0 con 0x7fb080100d80 2024-03-08T23:53:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.960+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb080068aa0 con 0x7fb080100d80 2024-03-08T23:53:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.961+0000 7fb076ffd700 1 -- 172.21.15.22:0/3793140097 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb07c01a070 con 0x7fb080100d80 2024-03-08T23:53:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.961+0000 7fb076ffd700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb06c0a2890 0x7fb06c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.962+0000 7fb085338700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb06c0a2890 0x7fb06c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.962+0000 7fb076ffd700 1 -- 172.21.15.22:0/3793140097 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb07c0ce780 con 0x7fb080100d80 2024-03-08T23:53:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.962+0000 7fb085338700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb06c0a2890 0x7fb06c0a4d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb078000f80 tx=0x7fb07800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:54.966+0000 7fb076ffd700 1 -- 172.21.15.22:0/3793140097 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb07c0232f0 con 0x7fb080100d80 2024-03-08T23:53:55.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.183+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb080062220 con 0x7fb06c0a2890 2024-03-08T23:53:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.189+0000 7fb076ffd700 1 -- 172.21.15.22:0/3793140097 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fb080062220 con 0x7fb06c0a2890 2024-03-08T23:53:55.186 INFO:teuthology.orchestra.run.smithi022.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-08T23:53:55.186 INFO:teuthology.orchestra.run.smithi022.stdout:mds.all 12/12 60s ago 2m smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-08T23:53:55.186 INFO:teuthology.orchestra.run.smithi022.stdout:mgr 3/3 60s ago 8m smithi022=x;smithi062=y;smithi192=z;count:3 2024-03-08T23:53:55.186 INFO:teuthology.orchestra.run.smithi022.stdout:mon 3/3 60s ago 8m smithi022:172.21.15.22=a;smithi062:172.21.15.62=b;smithi192:172.21.15.192=c;count:3 2024-03-08T23:53:55.186 INFO:teuthology.orchestra.run.smithi022.stdout:osd 12 60s ago - 2024-03-08T23:53:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.191+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb06c0a2890 msgr2=0x7fb06c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.191+0000 7fb087d9d700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb06c0a2890 0x7fb06c0a4d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb078000f80 tx=0x7fb07800b040 comp rx=0 tx=0).stop 2024-03-08T23:53:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.191+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080100d80 msgr2=0x7fb08006b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.191+0000 7fb087d9d700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080100d80 0x7fb08006b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00d980 tx=0x7fb07c00dc90 comp rx=0 tx=0).stop 2024-03-08T23:53:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 shutdown_connections 2024-03-08T23:53:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080102920 0x7fb0801a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:55.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fb06c0a2890 0x7fb06c0a4d50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb080100d80 0x7fb08006b0a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 --2- 172.21.15.22:0/3793140097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb080101f80 0x7fb08019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 >> 172.21.15.22:0/3793140097 conn(0x7fb0800fc340 msgr2=0x7fb080105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 shutdown_connections 2024-03-08T23:53:55.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:55.192+0000 7fb087d9d700 1 -- 172.21.15.22:0/3793140097 wait complete. 2024-03-08T23:53:55.816 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:55 smithi022 ceph-mon[31894]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 333 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:55.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-08T23:53:55.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:55 smithi192 ceph-mon[35995]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 333 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:56.086 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:56.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:55 smithi062 ceph-mon[36129]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 333 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:56.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:56 smithi022 ceph-mon[31894]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:57.012 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:56 smithi062 ceph-mon[36129]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:57.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:56 smithi192 ceph-mon[35995]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:53:57.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.782+0000 7f9c443ab700 1 -- 172.21.15.22:0/2870705491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c1028a0 msgr2=0x7f9c3c109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:57.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.782+0000 7f9c443ab700 1 --2- 172.21.15.22:0/2870705491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c1028a0 0x7f9c3c109150 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400b0d0 tx=0x7f9c3400b3e0 comp rx=0 tx=0).stop 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- 172.21.15.22:0/2870705491 shutdown_connections 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 --2- 172.21.15.22:0/2870705491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c1028a0 0x7f9c3c109150 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 --2- 172.21.15.22:0/2870705491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c100d00 0x7f9c3c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 --2- 172.21.15.22:0/2870705491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c3c101f00 0x7f9c3c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- 172.21.15.22:0/2870705491 >> 172.21.15.22:0/2870705491 conn(0x7f9c3c0fc2a0 msgr2=0x7f9c3c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- 172.21.15.22:0/2870705491 shutdown_connections 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- 172.21.15.22:0/2870705491 wait complete. 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 Processor -- start 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- start start 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c100d00 0x7f9c3c196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c101f00 0x7f9c3c1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c3c1028a0 0x7f9c3c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c10c960 con 0x7f9c3c101f00 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c10c7e0 con 0x7f9c3c1028a0 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c443ab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c10cae0 con 0x7f9c3c100d00 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.783+0000 7f9c41946700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c101f00 0x7f9c3c1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c101f00 0x7f9c3c1973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42720/0 (socket says 172.21.15.22:42720) 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 -- 172.21.15.22:0/354053071 learned_addr learned my addr 172.21.15.22:0/354053071 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c42948700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c3c1028a0 0x7f9c3c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c42147700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c100d00 0x7f9c3c196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 -- 172.21.15.22:0/354053071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c100d00 msgr2=0x7f9c3c196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c100d00 0x7f9c3c196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 -- 172.21.15.22:0/354053071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c3c1028a0 msgr2=0x7f9c3c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c3c1028a0 0x7f9c3c19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 -- 172.21.15.22:0/354053071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c19eb10 con 0x7f9c3c101f00 2024-03-08T23:53:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c41946700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c101f00 0x7f9c3c1973f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00c9e0 tx=0x7f9c2c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c337fe700 1 -- 172.21.15.22:0/354053071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c007ac0 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c19ee00 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c337fe700 1 -- 172.21.15.22:0/354053071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c2c007c60 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c337fe700 1 -- 172.21.15.22:0/354053071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c018730 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.784+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c19f380 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.785+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c068aa0 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.786+0000 7f9c337fe700 1 -- 172.21.15.22:0/354053071 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9c2c0188d0 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.787+0000 7f9c337fe700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c280a2890 0x7f9c280a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.787+0000 7f9c42147700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c280a2890 0x7f9c280a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.787+0000 7f9c337fe700 1 -- 172.21.15.22:0/354053071 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9c2c0cf010 con 0x7f9c3c101f00 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.787+0000 7f9c42147700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c280a2890 0x7f9c280a4d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9c38006fd0 tx=0x7f9c38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:53:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:57.790+0000 7f9c337fe700 1 -- 172.21.15.22:0/354053071 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9c2c014030 con 0x7f9c3c101f00 2024-03-08T23:53:57.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:57 smithi022 ceph-mon[31894]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 200 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:57.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.000+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c3c062220 con 0x7f9c280a2890 2024-03-08T23:53:57.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.002+0000 7f9c337fe700 1 -- 172.21.15.22:0/354053071 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f9c3c062220 con 0x7f9c280a2890 2024-03-08T23:53:57.998 INFO:teuthology.orchestra.run.smithi022.stdout:HOST ADDR LABELS STATUS 2024-03-08T23:53:57.998 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 172.21.15.22 2024-03-08T23:53:57.998 INFO:teuthology.orchestra.run.smithi022.stdout:smithi062 172.21.15.62 2024-03-08T23:53:57.998 INFO:teuthology.orchestra.run.smithi022.stdout:smithi192 172.21.15.192 2024-03-08T23:53:57.998 INFO:teuthology.orchestra.run.smithi022.stdout:3 hosts in cluster 2024-03-08T23:53:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c280a2890 msgr2=0x7f9c280a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c280a2890 0x7f9c280a4d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9c38006fd0 tx=0x7f9c38008040 comp rx=0 tx=0).stop 2024-03-08T23:53:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c101f00 msgr2=0x7f9c3c1973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:53:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c101f00 0x7f9c3c1973f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00c9e0 tx=0x7f9c2c00ccf0 comp rx=0 tx=0).stop 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 shutdown_connections 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c3c1028a0 0x7f9c3c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c3c100d00 0x7f9c3c196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f9c280a2890 0x7f9c280a4d50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 --2- 172.21.15.22:0/354053071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c3c101f00 0x7f9c3c1973f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 >> 172.21.15.22:0/354053071 conn(0x7f9c3c0fc2a0 msgr2=0x7f9c3c105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 shutdown_connections 2024-03-08T23:53:58.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:53:58.004+0000 7f9c443ab700 1 -- 172.21.15.22:0/354053071 wait complete. 2024-03-08T23:53:58.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:57 smithi062 ceph-mon[36129]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 200 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:58.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:57 smithi192 ceph-mon[35995]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 200 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-03-08T23:53:58.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-08T23:53:58.967 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:53:59.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:59 smithi022 ceph-mon[31894]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:53:59.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:53:59 smithi022 ceph-mon[31894]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:00.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:59 smithi062 ceph-mon[36129]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:00.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:53:59 smithi062 ceph-mon[36129]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:00.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:59 smithi192 ceph-mon[35995]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:00.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:53:59 smithi192 ceph-mon[35995]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.803+0000 7fa630c5c700 1 -- 172.21.15.22:0/2028043451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c102920 msgr2=0x7fa62c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.803+0000 7fa630c5c700 1 --2- 172.21.15.22:0/2028043451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c102920 0x7fa62c1091d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b0d0 tx=0x7fa62400b3e0 comp rx=0 tx=0).stop 2024-03-08T23:54:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- 172.21.15.22:0/2028043451 shutdown_connections 2024-03-08T23:54:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 --2- 172.21.15.22:0/2028043451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c102920 0x7fa62c1091d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 --2- 172.21.15.22:0/2028043451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa62c100d80 0x7fa62c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 --2- 172.21.15.22:0/2028043451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa62c101f80 0x7fa62c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:00.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- 172.21.15.22:0/2028043451 >> 172.21.15.22:0/2028043451 conn(0x7fa62c0fc340 msgr2=0x7fa62c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:00.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- 172.21.15.22:0/2028043451 shutdown_connections 2024-03-08T23:54:00.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- 172.21.15.22:0/2028043451 wait complete. 2024-03-08T23:54:00.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 Processor -- start 2024-03-08T23:54:00.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- start start 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c100d80 0x7fa62c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa62c101f80 0x7fa62c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa62c102920 0x7fa62c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c10c950 con 0x7fa62c100d80 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c10c7d0 con 0x7fa62c102920 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.804+0000 7fa630c5c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c10cad0 con 0x7fa62c101f80 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c100d80 0x7fa62c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c100d80 0x7fa62c197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59128/0 (socket says 172.21.15.22:59128) 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 -- 172.21.15.22:0/1422340407 learned_addr learned my addr 172.21.15.22:0/1422340407 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 -- 172.21.15.22:0/1422340407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa62c101f80 msgr2=0x7fa62c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62ad9d700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa62c102920 0x7fa62c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa62c101f80 0x7fa62c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:00.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 -- 172.21.15.22:0/1422340407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa62c102920 msgr2=0x7fa62c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa62c102920 0x7fa62c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 -- 172.21.15.22:0/1422340407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c19fc20 con 0x7fa62c100d80 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62ad9d700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa62c102920 0x7fa62c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa62a59c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c100d80 0x7fa62c197fc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa62000d9b0 tx=0x7fa62000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa61b7fe700 1 -- 172.21.15.22:0/1422340407 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620017070 con 0x7fa62c100d80 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c19ff10 con 0x7fa62c100d80 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa61b7fe700 1 -- 172.21.15.22:0/1422340407 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa620009b40 con 0x7fa62c100d80 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa61b7fe700 1 -- 172.21.15.22:0/1422340407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000f620 con 0x7fa62c100d80 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.805+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c1a0360 con 0x7fa62c100d80 2024-03-08T23:54:00.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.806+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c068aa0 con 0x7fa62c100d80 2024-03-08T23:54:00.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.807+0000 7fa61b7fe700 1 -- 172.21.15.22:0/1422340407 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa620009cf0 con 0x7fa62c100d80 2024-03-08T23:54:00.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.808+0000 7fa61b7fe700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6140a2890 0x7fa6140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:00.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.808+0000 7fa61b7fe700 1 -- 172.21.15.22:0/1422340407 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6200cea50 con 0x7fa62c100d80 2024-03-08T23:54:00.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.808+0000 7fa629d9b700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6140a2890 0x7fa6140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:00.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.808+0000 7fa629d9b700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6140a2890 0x7fa6140a4d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005fd0 tx=0x7fa61c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:00.812+0000 7fa61b7fe700 1 -- 172.21.15.22:0/1422340407 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa62009b760 con 0x7fa62c100d80 2024-03-08T23:54:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.024+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa62c0fe3c0 con 0x7fa6140a2890 2024-03-08T23:54:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.025+0000 7fa61b7fe700 1 -- 172.21.15.22:0/1422340407 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fa62c0fe3c0 con 0x7fa6140a2890 2024-03-08T23:54:01.022 INFO:teuthology.orchestra.run.smithi022.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-08T23:54:01.022 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-08T23:54:01.022 INFO:teuthology.orchestra.run.smithi022.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-08T23:54:01.022 INFO:teuthology.orchestra.run.smithi022.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6140a2890 msgr2=0x7fa6140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6140a2890 0x7fa6140a4d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005fd0 tx=0x7fa61c005d00 comp rx=0 tx=0).stop 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c100d80 msgr2=0x7fa62c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c100d80 0x7fa62c197fc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa62000d9b0 tx=0x7fa62000dd70 comp rx=0 tx=0).stop 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 shutdown_connections 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa62c102920 0x7fa62c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6140a2890 0x7fa6140a4d50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c100d80 0x7fa62c197fc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 --2- 172.21.15.22:0/1422340407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa62c101f80 0x7fa62c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.027+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 >> 172.21.15.22:0/1422340407 conn(0x7fa62c0fc340 msgr2=0x7fa62c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.028+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 shutdown_connections 2024-03-08T23:54:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:01.028+0000 7fa630c5c700 1 -- 172.21.15.22:0/1422340407 wait complete. 2024-03-08T23:54:01.767 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-08T23:54:01.777 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-08T23:54:01.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-08T23:54:01.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:01 smithi022 ceph-mon[31894]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:02.027 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:54:02.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:01 smithi062 ceph-mon[36129]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:02.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:01 smithi192 ceph-mon[35995]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:02.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:02 smithi022 ceph-mon[31894]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:03.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:02 smithi062 ceph-mon[36129]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:03.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:02 smithi192 ceph-mon[35995]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:03.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 -- 172.21.15.22:0/1397448698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638101f00 msgr2=0x7fa638102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:03.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 --2- 172.21.15.22:0/1397448698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638101f00 0x7fa638102360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0099c0 tx=0x7fa62c009cd0 comp rx=0 tx=0).stop 2024-03-08T23:54:03.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 -- 172.21.15.22:0/1397448698 shutdown_connections 2024-03-08T23:54:03.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 --2- 172.21.15.22:0/1397448698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6381028a0 0x7fa638109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:03.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 --2- 172.21.15.22:0/1397448698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638100d00 0x7fa638101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 --2- 172.21.15.22:0/1397448698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa638101f00 0x7fa638102360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 -- 172.21.15.22:0/1397448698 >> 172.21.15.22:0/1397448698 conn(0x7fa6380fc2a0 msgr2=0x7fa6380fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 -- 172.21.15.22:0/1397448698 shutdown_connections 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.782+0000 7fa63fd49700 1 -- 172.21.15.22:0/1397448698 wait complete. 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 Processor -- start 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 -- start start 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638100d00 0x7fa638196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638101f00 0x7fa638197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:03.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6381028a0 0x7fa63819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63810c8f0 con 0x7fa6381028a0 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63810c770 con 0x7fa638100d00 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63fd49700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63810ca70 con 0x7fa638101f00 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6381028a0 0x7fa63819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6381028a0 0x7fa63819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59150/0 (socket says 172.21.15.22:59150) 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 -- 172.21.15.22:0/2280819505 learned_addr learned my addr 172.21.15.22:0/2280819505 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63dae5700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638100d00 0x7fa638196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63d2e4700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638101f00 0x7fa638197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 -- 172.21.15.22:0/2280819505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638101f00 msgr2=0x7fa638197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638101f00 0x7fa638197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 -- 172.21.15.22:0/2280819505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638100d00 msgr2=0x7fa638196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:03.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638100d00 0x7fa638196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.783+0000 7fa63e2e6700 1 -- 172.21.15.22:0/2280819505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63819eb30 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.784+0000 7fa63e2e6700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6381028a0 0x7fa63819e490 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa630009900 tx=0x7fa630009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.784+0000 7fa636ffd700 1 -- 172.21.15.22:0/2280819505 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63000d020 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.784+0000 7fa636ffd700 1 -- 172.21.15.22:0/2280819505 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa63000d1c0 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.784+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63806a380 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.784+0000 7fa636ffd700 1 -- 172.21.15.22:0/2280819505 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63000c460 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.784+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63806a8c0 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.785+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa638068aa0 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.787+0000 7fa636ffd700 1 -- 172.21.15.22:0/2280819505 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa63001f020 con 0x7fa6381028a0 2024-03-08T23:54:03.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.787+0000 7fa636ffd700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6200a2890 0x7fa6200a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:03.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.787+0000 7fa63dae5700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6200a2890 0x7fa6200a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:03.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.787+0000 7fa636ffd700 1 -- 172.21.15.22:0/2280819505 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6300d07f0 con 0x7fa6381028a0 2024-03-08T23:54:03.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.788+0000 7fa63dae5700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6200a2890 0x7fa6200a4d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa628009550 tx=0x7fa6280091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:03.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:03.791+0000 7fa636ffd700 1 -- 172.21.15.22:0/2280819505 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa630026030 con 0x7fa6381028a0 2024-03-08T23:54:03.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:03 smithi022 ceph-mon[31894]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:04.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.040+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa638107b20 con 0x7fa6381028a0 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.040+0000 7fa636ffd700 1 -- 172.21.15.22:0/2280819505 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2942 (secure 0 0 0) 0x7fa63009d500 con 0x7fa6381028a0 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:e24 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2024-03-08T23:54:04.037 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 24 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-03-08T23:51:51.610292+0000 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-03-08T23:52:18.337548+0000 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 300 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 0 2024-03-08T23:54:04.038 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 3 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:in 0,1,2 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=24496,1=24494,2=24469} 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3,4] 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data disabled 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 0 2024-03-08T23:54:04.039 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.i{0:24496} state up:active seq 5 addr [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.h{0:24464} state up:standby-replay seq 1 addr [v2:172.21.15.62:6836/1631408985,v1:172.21.15.62:6837/1631408985] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b{1:24494} state up:active seq 9 addr [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.f{1:24460} state up:standby-replay seq 1 addr [v2:172.21.15.192:6834/1768340207,v1:172.21.15.192:6835/1768340207] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c{2:24469} state up:active seq 12 addr [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.e{2:24437} state up:standby-replay seq 1 addr [v2:172.21.15.62:6834/1656345994,v1:172.21.15.62:6835/1656345994] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.j{-1:14562} state up:standby seq 1 addr [v2:172.21.15.22:6834/1092846184,v1:172.21.15.22:6835/1092846184] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d{-1:14580} state up:standby seq 1 addr [v2:172.21.15.22:6836/1255227444,v1:172.21.15.22:6837/1255227444] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.g{-1:14640} state up:standby seq 1 addr [v2:172.21.15.22:6838/1763775736,v1:172.21.15.22:6839/1763775736] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a{-1:14652} state up:standby seq 1 addr [v2:172.21.15.22:6840/2885682684,v1:172.21.15.22:6841/2885682684] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.62:6832/292631857,v1:172.21.15.62:6833/292631857] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.192:6832/3599891094,v1:172.21.15.192:6833/3599891094] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T23:54:04.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6200a2890 msgr2=0x7fa6200a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:04.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6200a2890 0x7fa6200a4d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa628009550 tx=0x7fa6280091b0 comp rx=0 tx=0).stop 2024-03-08T23:54:04.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6381028a0 msgr2=0x7fa63819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:04.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6381028a0 0x7fa63819e490 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa630009900 tx=0x7fa630009c10 comp rx=0 tx=0).stop 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 shutdown_connections 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6200a2890 0x7fa6200a4d50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6381028a0 0x7fa63819e490 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638100d00 0x7fa638196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 --2- 172.21.15.22:0/2280819505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638101f00 0x7fa638197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.042+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 >> 172.21.15.22:0/2280819505 conn(0x7fa6380fc2a0 msgr2=0x7fa638105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.043+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 shutdown_connections 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:04.043+0000 7fa63fd49700 1 -- 172.21.15.22:0/2280819505 wait complete. 2024-03-08T23:54:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-03-08T23:54:04.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:03 smithi062 ceph-mon[36129]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:04.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:03 smithi192 ceph-mon[35995]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1 B/s, 0 objects/s recovering 2024-03-08T23:54:04.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-08T23:54:04.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:04 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2280819505' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-08T23:54:05.031 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:54:05.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:04 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2280819505' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-08T23:54:05.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:04 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2280819505' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-08T23:54:05.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:05 smithi022 ceph-mon[31894]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:06.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:05 smithi062 ceph-mon[36129]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:06.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:05 smithi192 ceph-mon[35995]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 -- 172.21.15.22:0/1850865674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20074840 msgr2=0x7fca20074cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 --2- 172.21.15.22:0/1850865674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20074840 0x7fca20074cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fca14009990 tx=0x7fca14009ca0 comp rx=0 tx=0).stop 2024-03-08T23:54:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 -- 172.21.15.22:0/1850865674 shutdown_connections 2024-03-08T23:54:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 --2- 172.21.15.22:0/1850865674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca20109d00 0x7fca2010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 --2- 172.21.15.22:0/1850865674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca20075e10 0x7fca20074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 --2- 172.21.15.22:0/1850865674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20074840 0x7fca20074cc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:06.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 -- 172.21.15.22:0/1850865674 >> 172.21.15.22:0/1850865674 conn(0x7fca200fd3f0 msgr2=0x7fca200ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 -- 172.21.15.22:0/1850865674 shutdown_connections 2024-03-08T23:54:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.848+0000 7fca25392700 1 -- 172.21.15.22:0/1850865674 wait complete. 2024-03-08T23:54:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 Processor -- start 2024-03-08T23:54:06.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 -- start start 2024-03-08T23:54:06.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca20074840 0x7fca2019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:06.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20075e10 0x7fca2019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:06.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca20109d00 0x7fca201a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:06.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2010fc10 con 0x7fca20075e10 2024-03-08T23:54:06.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2010fa90 con 0x7fca20074840 2024-03-08T23:54:06.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca25392700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2010fd90 con 0x7fca20109d00 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca1e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20075e10 0x7fca2019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca1e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20075e10 0x7fca2019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59166/0 (socket says 172.21.15.22:59166) 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca1e7fc700 1 -- 172.21.15.22:0/4206448661 learned_addr learned my addr 172.21.15.22:0/4206448661 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca1f7fe700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca20109d00 0x7fca201a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.849+0000 7fca1effd700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca20074840 0x7fca2019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca1e7fc700 1 -- 172.21.15.22:0/4206448661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca20109d00 msgr2=0x7fca201a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca1e7fc700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca20109d00 0x7fca201a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca1e7fc700 1 -- 172.21.15.22:0/4206448661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca20074840 msgr2=0x7fca2019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca1e7fc700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca20074840 0x7fca2019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca1e7fc700 1 -- 172.21.15.22:0/4206448661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca201a1e50 con 0x7fca20075e10 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca1e7fc700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20075e10 0x7fca2019a730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fca14000c00 tx=0x7fca140057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca07fff700 1 -- 172.21.15.22:0/4206448661 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14027070 con 0x7fca20075e10 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca07fff700 1 -- 172.21.15.22:0/4206448661 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca14004150 con 0x7fca20075e10 2024-03-08T23:54:06.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca201a20e0 con 0x7fca20075e10 2024-03-08T23:54:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca07fff700 1 -- 172.21.15.22:0/4206448661 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1400f460 con 0x7fca20075e10 2024-03-08T23:54:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.850+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca201a2530 con 0x7fca20075e10 2024-03-08T23:54:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.851+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca20068aa0 con 0x7fca20075e10 2024-03-08T23:54:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.852+0000 7fca07fff700 1 -- 172.21.15.22:0/4206448661 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fca1402a070 con 0x7fca20075e10 2024-03-08T23:54:06.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.853+0000 7fca07fff700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca080a2890 0x7fca080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.853+0000 7fca1effd700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca080a2890 0x7fca080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.853+0000 7fca07fff700 1 -- 172.21.15.22:0/4206448661 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fca140cfbd0 con 0x7fca20075e10 2024-03-08T23:54:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.853+0000 7fca1effd700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca080a2890 0x7fca080a4d50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca10005950 tx=0x7fca100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:06.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:06.857+0000 7fca07fff700 1 -- 172.21.15.22:0/4206448661 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fca1405e260 con 0x7fca20075e10 2024-03-08T23:54:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.064+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fca200629f0 con 0x7fca20075e10 2024-03-08T23:54:07.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.065+0000 7fca07fff700 1 -- 172.21.15.22:0/4206448661 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5623 (secure 0 0 0) 0x7fca14013070 con 0x7fca20075e10 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 93 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-03-08T23:43:41.144090+0000 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-03-08T23:53:45.618289+0000 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:crush_version 27 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:full_ratio 0.95 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:backfillfull_ratio 0.9 2024-03-08T23:54:07.062 INFO:teuthology.orchestra.run.smithi022.stdout:nearfull_ratio 0.85 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:require_min_compat_client luminous 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:min_compat_client luminous 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:require_osd_release squid 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:stretch_mode_enabled false 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:max_osd 12 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6802/103047658,v1:172.21.15.22:6803/103047658] [v2:172.21.15.22:6804/103047658,v1:172.21.15.22:6805/103047658] exists,up 3bc68a9c-ec9a-4342-aa98-70c2835d2f06 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6800/782240245,v1:172.21.15.62:6801/782240245] [v2:172.21.15.62:6802/782240245,v1:172.21.15.62:6803/782240245] exists,up 06bad818-44be-4c08-a916-5e7cf0548753 2024-03-08T23:54:07.063 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/316512323,v1:172.21.15.192:6801/316512323] [v2:172.21.15.192:6802/316512323,v1:172.21.15.192:6803/316512323] exists,up d5674ba8-0f77-4cca-baa2-8d27058e2259 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 up in weight 1 up_from 25 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6810/2127248159,v1:172.21.15.22:6811/2127248159] [v2:172.21.15.22:6812/2127248159,v1:172.21.15.22:6813/2127248159] exists,up da0b079a-88ee-483c-b094-40c3cc5fdcd2 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6808/590253654,v1:172.21.15.62:6809/590253654] [v2:172.21.15.62:6810/590253654,v1:172.21.15.62:6811/590253654] exists,up 9b0bac61-2019-4ebf-8688-8ae47e9457ac 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/3950935286,v1:172.21.15.192:6809/3950935286] [v2:172.21.15.192:6810/3950935286,v1:172.21.15.192:6811/3950935286] exists,up 45fe7b13-24f3-4e73-984b-04ff364a0d6a 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6818/1438134132,v1:172.21.15.22:6819/1438134132] [v2:172.21.15.22:6820/1438134132,v1:172.21.15.22:6821/1438134132] exists,up be6cb9f2-a8f1-410a-a997-76adcb1fa023 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6816/92390165,v1:172.21.15.62:6817/92390165] [v2:172.21.15.62:6818/92390165,v1:172.21.15.62:6819/92390165] exists,up db83755e-a0a0-435c-963a-41f2eeb41266 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 up in weight 1 up_from 59 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/3107982312,v1:172.21.15.192:6817/3107982312] [v2:172.21.15.192:6818/3107982312,v1:172.21.15.192:6819/3107982312] exists,up 766c75f7-7820-4cb6-ac6b-ab7596735fc0 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6826/460204078,v1:172.21.15.22:6827/460204078] [v2:172.21.15.22:6828/460204078,v1:172.21.15.22:6829/460204078] exists,up 029fa555-d0bd-4863-9199-b610bec7e240 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6824/1674155650,v1:172.21.15.62:6825/1674155650] [v2:172.21.15.62:6826/1674155650,v1:172.21.15.62:6827/1674155650] exists,up 0094504e-2371-485f-b601-d2e36f3062ae 2024-03-08T23:54:07.064 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/3259500463,v1:172.21.15.192:6825/3259500463] [v2:172.21.15.192:6826/3259500463,v1:172.21.15.192:6827/3259500463] exists,up 182eae10-50a1-43f2-b776-13486e2939b3 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.a [2,11] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.f [8,11] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.14 [2,4] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.21 [4,6] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.27 [4,8] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.29 [3,9] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.33 [5,9] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.3c [4,9] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.5 [9,10] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.a [5,8] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.12 [9,10] 2024-03-08T23:54:07.065 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.1c [9,8] 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/3201080518 expires 2024-03-09T23:44:43.705970+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/3384641534 expires 2024-03-09T23:44:16.515766+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/99443044 expires 2024-03-09T23:44:43.705970+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1968010400 expires 2024-03-09T23:44:16.515766+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/3762004902 expires 2024-03-09T23:44:16.515766+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/3384641534 expires 2024-03-09T23:44:16.515766+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/4028157579 expires 2024-03-09T23:44:43.705970+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/4028157579 expires 2024-03-09T23:44:43.705970+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1133673382 expires 2024-03-09T23:44:16.515766+0000 2024-03-08T23:54:07.066 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2337825839 expires 2024-03-09T23:44:43.705970+0000 2024-03-08T23:54:07.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.068+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca080a2890 msgr2=0x7fca080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:07.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.068+0000 7fca25392700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca080a2890 0x7fca080a4d50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca10005950 tx=0x7fca100058e0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.068+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20075e10 msgr2=0x7fca2019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.068+0000 7fca25392700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20075e10 0x7fca2019a730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fca14000c00 tx=0x7fca140057c0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.068+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 shutdown_connections 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.068+0000 7fca25392700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca20109d00 0x7fca201a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.068+0000 7fca25392700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca20074840 0x7fca2019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.069+0000 7fca25392700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fca080a2890 0x7fca080a4d50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.069+0000 7fca25392700 1 --2- 172.21.15.22:0/4206448661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca20075e10 0x7fca2019a730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.069+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 >> 172.21.15.22:0/4206448661 conn(0x7fca200fd3f0 msgr2=0x7fca20105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.069+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 shutdown_connections 2024-03-08T23:54:07.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.069+0000 7fca25392700 1 -- 172.21.15.22:0/4206448661 wait complete. 2024-03-08T23:54:07.815 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:07 smithi022 ceph-mon[31894]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:54:07.815 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:07 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4206448661' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-08T23:54:07.842 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-08T23:54:07.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:54:07.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 -- 172.21.15.22:0/1250669635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68107050 msgr2=0x7f8d68109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:07.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 --2- 172.21.15.22:0/1250669635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68107050 0x7f8d68109440 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009ca0 tx=0x7f8d6002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 -- 172.21.15.22:0/1250669635 shutdown_connections 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 --2- 172.21.15.22:0/1250669635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68107050 0x7f8d68109440 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 --2- 172.21.15.22:0/1250669635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d68104720 0x7f8d68106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 --2- 172.21.15.22:0/1250669635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d68101df0 0x7f8d681041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 -- 172.21.15.22:0/1250669635 >> 172.21.15.22:0/1250669635 conn(0x7f8d680fb780 msgr2=0x7f8d680fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 -- 172.21.15.22:0/1250669635 shutdown_connections 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.963+0000 7f8d6c960640 1 -- 172.21.15.22:0/1250669635 wait complete. 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 Processor -- start 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 -- start start 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68101df0 0x7f8d681975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d68104720 0x7f8d68197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d68107050 0x7f8d6819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6810cfa0 con 0x7f8d68101df0 2024-03-08T23:54:07.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6810ce20 con 0x7f8d68107050 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6c960640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6810d120 con 0x7f8d68104720 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68101df0 0x7f8d681975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68101df0 0x7f8d681975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59178/0 (socket says 172.21.15.22:59178) 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d6659c640 1 -- 172.21.15.22:0/5387961 learned_addr learned my addr 172.21.15.22:0/5387961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.964+0000 7f8d65d9b640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d68104720 0x7f8d68197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d66d9d640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d68107050 0x7f8d6819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6659c640 1 -- 172.21.15.22:0/5387961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d68104720 msgr2=0x7f8d68197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6659c640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d68104720 0x7f8d68197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6659c640 1 -- 172.21.15.22:0/5387961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d68107050 msgr2=0x7f8d6819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6659c640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d68107050 0x7f8d6819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6659c640 1 -- 172.21.15.22:0/5387961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6819f200 con 0x7f8d68101df0 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6659c640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68101df0 0x7f8d681975e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c890 tx=0x7f8d5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d577fe640 1 -- 172.21.15.22:0/5387961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58004220 con 0x7f8d68101df0 2024-03-08T23:54:07.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d577fe640 1 -- 172.21.15.22:0/5387961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d580043c0 con 0x7f8d68101df0 2024-03-08T23:54:07.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6819f4f0 con 0x7f8d68101df0 2024-03-08T23:54:07.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d577fe640 1 -- 172.21.15.22:0/5387961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d580049e0 con 0x7f8d68101df0 2024-03-08T23:54:07.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.965+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6819fa40 con 0x7f8d68101df0 2024-03-08T23:54:07.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.966+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d680fd270 con 0x7f8d68101df0 2024-03-08T23:54:07.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.967+0000 7f8d577fe640 1 -- 172.21.15.22:0/5387961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8d580079b0 con 0x7f8d68101df0 2024-03-08T23:54:07.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.968+0000 7f8d577fe640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d500a2800 0x7f8d500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:07.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.968+0000 7f8d65d9b640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d500a2800 0x7f8d500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:07.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.968+0000 7f8d577fe640 1 -- 172.21.15.22:0/5387961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d580cef10 con 0x7f8d68101df0 2024-03-08T23:54:07.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.968+0000 7f8d65d9b640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d500a2800 0x7f8d500a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d68198af0 tx=0x7f8d5c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:07.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:07.972+0000 7f8d577fe640 1 -- 172.21.15.22:0/5387961 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8d58014030 con 0x7f8d68101df0 2024-03-08T23:54:08.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:07 smithi062 ceph-mon[36129]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:54:08.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:07 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4206448661' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-08T23:54:08.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:07 smithi192 ceph-mon[35995]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:54:08.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:07 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4206448661' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-08T23:54:08.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.156+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d68061ef0 con 0x7f8d68101df0 2024-03-08T23:54:08.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.157+0000 7f8d577fe640 1 -- 172.21.15.22:0/5387961 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24654 (secure 0 0 0) 0x7f8d5809bbc0 con 0x7f8d68101df0 2024-03-08T23:54:08.154 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:08.154 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:45.618289+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d500a2800 msgr2=0x7f8d500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d500a2800 0x7f8d500a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d68198af0 tx=0x7f8d5c00b390 comp rx=0 tx=0).stop 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68101df0 msgr2=0x7f8d681975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68101df0 0x7f8d681975e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c890 tx=0x7f8d5800cd50 comp rx=0 tx=0).stop 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 shutdown_connections 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d68107050 0x7f8d6819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d68104720 0x7f8d68197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f8d500a2800 0x7f8d500a4cc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 --2- 172.21.15.22:0/5387961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d68101df0 0x7f8d681975e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 >> 172.21.15.22:0/5387961 conn(0x7f8d680fb780 msgr2=0x7f8d68102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 shutdown_connections 2024-03-08T23:54:08.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.159+0000 7f8d6c960640 1 -- 172.21.15.22:0/5387961 wait complete. 2024-03-08T23:54:08.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:54:08.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:54:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 -- 172.21.15.22:0/406731289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 msgr2=0x7f714c101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 --2- 172.21.15.22:0/406731289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c101860 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f713c009900 tx=0x7f713c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 -- 172.21.15.22:0/406731289 shutdown_connections 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 --2- 172.21.15.22:0/406731289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c101da0 0x7f714c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 --2- 172.21.15.22:0/406731289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c101860 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 --2- 172.21.15.22:0/406731289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f714c1001f0 0x7f714c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 -- 172.21.15.22:0/406731289 >> 172.21.15.22:0/406731289 conn(0x7f714c0fb780 msgr2=0x7f714c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 -- 172.21.15.22:0/406731289 shutdown_connections 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.283+0000 7f715359d640 1 -- 172.21.15.22:0/406731289 wait complete. 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 Processor -- start 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 -- start start 2024-03-08T23:54:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f714c1001f0 0x7f714c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c101da0 0x7f714c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c10bfb0 con 0x7f714c1013e0 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c10be30 con 0x7f714c101da0 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f715359d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c10c130 con 0x7f714c1001f0 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f7150b38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f7150b38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59200/0 (socket says 172.21.15.22:59200) 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f7150b38640 1 -- 172.21.15.22:0/2347568280 learned_addr learned my addr 172.21.15.22:0/2347568280 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f7151b3a640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c101da0 0x7f714c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.284+0000 7f7151339640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f714c1001f0 0x7f714c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7150b38640 1 -- 172.21.15.22:0/2347568280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f714c1001f0 msgr2=0x7f714c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7150b38640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f714c1001f0 0x7f714c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7150b38640 1 -- 172.21.15.22:0/2347568280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c101da0 msgr2=0x7f714c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7150b38640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c101da0 0x7f714c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7150b38640 1 -- 172.21.15.22:0/2347568280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c19f330 con 0x7f714c1013e0 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7151339640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f714c1001f0 0x7f714c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7151b3a640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c101da0 0x7f714c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f7150b38640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c197c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f71427fc640 1 -- 172.21.15.22:0/2347568280 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c043020 con 0x7f714c1013e0 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f71427fc640 1 -- 172.21.15.22:0/2347568280 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f713c004440 con 0x7f714c1013e0 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714c19f5c0 con 0x7f714c1013e0 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f71427fc640 1 -- 172.21.15.22:0/2347568280 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c03e450 con 0x7f714c1013e0 2024-03-08T23:54:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.285+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714c19fb10 con 0x7f714c1013e0 2024-03-08T23:54:08.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.286+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714c19fe30 con 0x7f714c1013e0 2024-03-08T23:54:08.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.287+0000 7f71427fc640 1 -- 172.21.15.22:0/2347568280 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f713c046020 con 0x7f714c1013e0 2024-03-08T23:54:08.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.288+0000 7f71427fc640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f71380a2800 0x7f71380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.288+0000 7f7151339640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f71380a2800 0x7f71380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.288+0000 7f71427fc640 1 -- 172.21.15.22:0/2347568280 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f713c0f16c0 con 0x7f714c1013e0 2024-03-08T23:54:08.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.288+0000 7f7151339640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f71380a2800 0x7f71380a4cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7148006fd0 tx=0x7f7148008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:08.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.292+0000 7f71427fc640 1 -- 172.21.15.22:0/2347568280 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f713c04d020 con 0x7f714c1013e0 2024-03-08T23:54:08.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.471+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f714c1a0000 con 0x7f714c1013e0 2024-03-08T23:54:08.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.472+0000 7f71427fc640 1 -- 172.21.15.22:0/2347568280 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f713c0be3d0 con 0x7f714c1013e0 2024-03-08T23:54:08.468 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f71380a2800 msgr2=0x7f71380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f71380a2800 0x7f71380a4cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7148006fd0 tx=0x7f7148008040 comp rx=0 tx=0).stop 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 msgr2=0x7f714c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c197c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c004040 comp rx=0 tx=0).stop 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 shutdown_connections 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f714c101da0 0x7f714c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f71380a2800 0x7f71380a4cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f714c1013e0 0x7f714c197c50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 --2- 172.21.15.22:0/2347568280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f714c1001f0 0x7f714c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 >> 172.21.15.22:0/2347568280 conn(0x7f714c0fb780 msgr2=0x7f714c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:08.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 shutdown_connections 2024-03-08T23:54:08.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.474+0000 7f715359d640 1 -- 172.21.15.22:0/2347568280 wait complete. 2024-03-08T23:54:08.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:54:08.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.590+0000 7f1b748a4640 1 -- 172.21.15.22:0/1401851562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701013e0 msgr2=0x7f1b70101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.590+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1401851562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701013e0 0x7f1b70101860 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009900 tx=0x7f1b6402bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- 172.21.15.22:0/1401851562 shutdown_connections 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1401851562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b70101da0 0x7f1b70108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1401851562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701013e0 0x7f1b70101860 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1401851562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b701001f0 0x7f1b70100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- 172.21.15.22:0/1401851562 >> 172.21.15.22:0/1401851562 conn(0x7f1b700fb780 msgr2=0x7f1b700fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- 172.21.15.22:0/1401851562 shutdown_connections 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- 172.21.15.22:0/1401851562 wait complete. 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 Processor -- start 2024-03-08T23:54:08.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- start start 2024-03-08T23:54:08.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701001f0 0x7f1b70199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b701013e0 0x7f1b70199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b70101da0 0x7f1b701a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7010bed0 con 0x7f1b701001f0 2024-03-08T23:54:08.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7010bd50 con 0x7f1b701013e0 2024-03-08T23:54:08.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.591+0000 7f1b748a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7010c050 con 0x7f1b70101da0 2024-03-08T23:54:08.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701001f0 0x7f1b70199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701001f0 0x7f1b70199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59222/0 (socket says 172.21.15.22:59222) 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 -- 172.21.15.22:0/1376512677 learned_addr learned my addr 172.21.15.22:0/1376512677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6dd9b640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b701013e0 0x7f1b70199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6ed9d640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b70101da0 0x7f1b701a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 -- 172.21.15.22:0/1376512677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b70101da0 msgr2=0x7f1b701a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b70101da0 0x7f1b701a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 -- 172.21.15.22:0/1376512677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b701013e0 msgr2=0x7f1b70199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b701013e0 0x7f1b70199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 -- 172.21.15.22:0/1376512677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b701a13b0 con 0x7f1b701001f0 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6dd9b640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b701013e0 0x7f1b70199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6ed9d640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b70101da0 0x7f1b701a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b6e59c640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701001f0 0x7f1b70199790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c970 tx=0x7f1b6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b5f7fe640 1 -- 172.21.15.22:0/1376512677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60004220 con 0x7f1b701001f0 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b5f7fe640 1 -- 172.21.15.22:0/1376512677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b600043c0 con 0x7f1b701001f0 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b701a16a0 con 0x7f1b701001f0 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b5f7fe640 1 -- 172.21.15.22:0/1376512677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000f450 con 0x7f1b701001f0 2024-03-08T23:54:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.592+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b701a1c40 con 0x7f1b701001f0 2024-03-08T23:54:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.593+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b701a1f00 con 0x7f1b701001f0 2024-03-08T23:54:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.594+0000 7f1b5f7fe640 1 -- 172.21.15.22:0/1376512677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1b600079b0 con 0x7f1b701001f0 2024-03-08T23:54:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.595+0000 7f1b5f7fe640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1b580a2880 0x7f1b580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.595+0000 7f1b6dd9b640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1b580a2880 0x7f1b580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.595+0000 7f1b5f7fe640 1 -- 172.21.15.22:0/1376512677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1b600cefd0 con 0x7f1b701001f0 2024-03-08T23:54:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.596+0000 7f1b6dd9b640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1b580a2880 0x7f1b580a4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b640096c0 tx=0x7f1b6402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.599+0000 7f1b5f7fe640 1 -- 172.21.15.22:0/1376512677 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1b60014030 con 0x7f1b701001f0 2024-03-08T23:54:08.731 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/5387961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:08.731 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2347568280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:08.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.782+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b701a20d0 con 0x7f1b701001f0 2024-03-08T23:54:08.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.783+0000 7f1b5f7fe640 1 -- 172.21.15.22:0/1376512677 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b6009bd00 con 0x7f1b701001f0 2024-03-08T23:54:08.779 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:08.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1b580a2880 msgr2=0x7f1b580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1b580a2880 0x7f1b580a4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b640096c0 tx=0x7f1b6402df90 comp rx=0 tx=0).stop 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701001f0 msgr2=0x7f1b70199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701001f0 0x7f1b70199790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c970 tx=0x7f1b6000ce30 comp rx=0 tx=0).stop 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 shutdown_connections 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b70101da0 0x7f1b701a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b701013e0 0x7f1b70199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1b580a2880 0x7f1b580a4d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 --2- 172.21.15.22:0/1376512677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b701001f0 0x7f1b70199790 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 >> 172.21.15.22:0/1376512677 conn(0x7f1b700fb780 msgr2=0x7f1b700fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 shutdown_connections 2024-03-08T23:54:08.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.785+0000 7f1b748a4640 1 -- 172.21.15.22:0/1376512677 wait complete. 2024-03-08T23:54:08.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 -- 172.21.15.22:0/3199293066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 msgr2=0x7eff9c101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 --2- 172.21.15.22:0/3199293066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c101010 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7eff90009900 tx=0x7eff9002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 -- 172.21.15.22:0/3199293066 shutdown_connections 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 --2- 172.21.15.22:0/3199293066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c101550 0x7eff9c107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 --2- 172.21.15.22:0/3199293066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c101010 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 --2- 172.21.15.22:0/3199293066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c0ff9a0 0x7eff9c0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 -- 172.21.15.22:0/3199293066 >> 172.21.15.22:0/3199293066 conn(0x7eff9c0faef0 msgr2=0x7eff9c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 -- 172.21.15.22:0/3199293066 shutdown_connections 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.902+0000 7effa1bb2640 1 -- 172.21.15.22:0/3199293066 wait complete. 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 Processor -- start 2024-03-08T23:54:08.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 -- start start 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0ff9a0 0x7eff9c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c101550 0x7eff9c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c10beb0 con 0x7eff9c100b90 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c10bd30 con 0x7eff9c101550 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7effa1bb2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c10c030 con 0x7eff9c0ff9a0 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7eff9affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7eff9affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59238/0 (socket says 172.21.15.22:59238) 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.903+0000 7eff9affd640 1 -- 172.21.15.22:0/4002913474 learned_addr learned my addr 172.21.15.22:0/4002913474 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9bfff640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c101550 0x7eff9c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9b7fe640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0ff9a0 0x7eff9c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9affd640 1 -- 172.21.15.22:0/4002913474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0ff9a0 msgr2=0x7eff9c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9affd640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0ff9a0 0x7eff9c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9affd640 1 -- 172.21.15.22:0/4002913474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c101550 msgr2=0x7eff9c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9affd640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c101550 0x7eff9c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9affd640 1 -- 172.21.15.22:0/4002913474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c19f310 con 0x7eff9c100b90 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9b7fe640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0ff9a0 0x7eff9c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff9affd640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c197c30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7eff90006010 tx=0x7eff90005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff98ff9640 1 -- 172.21.15.22:0/4002913474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90041070 con 0x7eff9c100b90 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff98ff9640 1 -- 172.21.15.22:0/4002913474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff900054e0 con 0x7eff9c100b90 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c19f5a0 con 0x7eff9c100b90 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7eff98ff9640 1 -- 172.21.15.22:0/4002913474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90034440 con 0x7eff9c100b90 2024-03-08T23:54:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.904+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c19fa90 con 0x7eff9c100b90 2024-03-08T23:54:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.905+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c19fcf0 con 0x7eff9c100b90 2024-03-08T23:54:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.906+0000 7eff98ff9640 1 -- 172.21.15.22:0/4002913474 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7eff90044070 con 0x7eff9c100b90 2024-03-08T23:54:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.907+0000 7eff98ff9640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff840a2800 0x7eff840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.907+0000 7eff9b7fe640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff840a2800 0x7eff840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.907+0000 7eff98ff9640 1 -- 172.21.15.22:0/4002913474 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff900f0ae0 con 0x7eff9c100b90 2024-03-08T23:54:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.907+0000 7eff9b7fe640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff840a2800 0x7eff840a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff9c1009f0 tx=0x7eff8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:08.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:08.911+0000 7eff98ff9640 1 -- 172.21.15.22:0/4002913474 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7eff900bd7f0 con 0x7eff9c100b90 2024-03-08T23:54:09.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.093+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff9c19fec0 con 0x7eff9c100b90 2024-03-08T23:54:09.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.093+0000 7eff98ff9640 1 -- 172.21.15.22:0/4002913474 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff900bd0f0 con 0x7eff9c100b90 2024-03-08T23:54:09.090 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:54:09.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff840a2800 msgr2=0x7eff840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff840a2800 0x7eff840a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff9c1009f0 tx=0x7eff8c005f90 comp rx=0 tx=0).stop 2024-03-08T23:54:09.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 msgr2=0x7eff9c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c197c30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7eff90006010 tx=0x7eff90005dd0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 shutdown_connections 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c101550 0x7eff9c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff840a2800 0x7eff840a4cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff9c100b90 0x7eff9c197c30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 --2- 172.21.15.22:0/4002913474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0ff9a0 0x7eff9c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.095+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 >> 172.21.15.22:0/4002913474 conn(0x7eff9c0faef0 msgr2=0x7eff9c0fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.096+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 shutdown_connections 2024-03-08T23:54:09.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.096+0000 7effa1bb2640 1 -- 172.21.15.22:0/4002913474 wait complete. 2024-03-08T23:54:09.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:54:09.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:08 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/5387961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:09.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:08 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2347568280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:09.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:08 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/5387961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:09.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:08 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2347568280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 -- 172.21.15.22:0/1264175248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc0ff990 msgr2=0x7f64fc0ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 --2- 172.21.15.22:0/1264175248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc0ff990 0x7f64fc0ffdb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009900 tx=0x7f64ec02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 -- 172.21.15.22:0/1264175248 shutdown_connections 2024-03-08T23:54:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 --2- 172.21.15.22:0/1264175248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc101540 0x7f64fc107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 --2- 172.21.15.22:0/1264175248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc100b80 0x7f64fc101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 --2- 172.21.15.22:0/1264175248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc0ff990 0x7f64fc0ffdb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 -- 172.21.15.22:0/1264175248 >> 172.21.15.22:0/1264175248 conn(0x7f64fc0faee0 msgr2=0x7f64fc0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 -- 172.21.15.22:0/1264175248 shutdown_connections 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.212+0000 7f6500ade640 1 -- 172.21.15.22:0/1264175248 wait complete. 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 Processor -- start 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 -- start start 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ff990 0x7f64fc1038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc100b80 0x7f64fc101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc101540 0x7f64fc1024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc10bea0 con 0x7f64fc100b80 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc10bd20 con 0x7f64fc0ff990 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f6500ade640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc10c020 con 0x7f64fc101540 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f64f9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc100b80 0x7f64fc101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f64f9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc100b80 0x7f64fc101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59254/0 (socket says 172.21.15.22:59254) 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.213+0000 7f64f9d9b640 1 -- 172.21.15.22:0/1931804689 learned_addr learned my addr 172.21.15.22:0/1931804689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64fad9d640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc101540 0x7f64fc1024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64f9d9b640 1 -- 172.21.15.22:0/1931804689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc101540 msgr2=0x7f64fc1024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64f9d9b640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc101540 0x7f64fc1024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64f9d9b640 1 -- 172.21.15.22:0/1931804689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ff990 msgr2=0x7f64fc1038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64f9d9b640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ff990 0x7f64fc1038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64f9d9b640 1 -- 172.21.15.22:0/1931804689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc102d60 con 0x7f64fc100b80 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64f9d9b640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc100b80 0x7f64fc101f60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c970 tx=0x7f64f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64eb7fe640 1 -- 172.21.15.22:0/1931804689 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007bb0 con 0x7f64fc100b80 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64eb7fe640 1 -- 172.21.15.22:0/1931804689 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64f0007d50 con 0x7f64fc100b80 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc1a1790 con 0x7f64fc100b80 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f64eb7fe640 1 -- 172.21.15.22:0/1931804689 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000f450 con 0x7f64fc100b80 2024-03-08T23:54:09.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.214+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc1a1cb0 con 0x7f64fc100b80 2024-03-08T23:54:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.215+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc1a1f70 con 0x7f64fc100b80 2024-03-08T23:54:09.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.216+0000 7f64eb7fe640 1 -- 172.21.15.22:0/1931804689 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f64f0005020 con 0x7f64fc100b80 2024-03-08T23:54:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.216+0000 7f64eb7fe640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f64e40a2880 0x7f64e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.217+0000 7f64fa59c640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f64e40a2880 0x7f64e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.217+0000 7f64eb7fe640 1 -- 172.21.15.22:0/1931804689 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64f00cec40 con 0x7f64fc100b80 2024-03-08T23:54:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.217+0000 7f64fa59c640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f64e40a2880 0x7f64e40a4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64ec036040 tx=0x7f64ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:09.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.221+0000 7f64eb7fe640 1 -- 172.21.15.22:0/1931804689 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f64f0014030 con 0x7f64fc100b80 2024-03-08T23:54:09.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.405+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f64fc1a2140 con 0x7f64fc100b80 2024-03-08T23:54:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.405+0000 7f64eb7fe640 1 -- 172.21.15.22:0/1931804689 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f64f009b8f0 con 0x7f64fc100b80 2024-03-08T23:54:09.402 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f64e40a2880 msgr2=0x7f64e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f64e40a2880 0x7f64e40a4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64ec036040 tx=0x7f64ec000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc100b80 msgr2=0x7f64fc101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc100b80 0x7f64fc101f60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c970 tx=0x7f64f000ce30 comp rx=0 tx=0).stop 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 shutdown_connections 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc101540 0x7f64fc1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f64e40a2880 0x7f64e40a4d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64fc100b80 0x7f64fc101f60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 --2- 172.21.15.22:0/1931804689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64fc0ff990 0x7f64fc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 >> 172.21.15.22:0/1931804689 conn(0x7f64fc0faee0 msgr2=0x7f64fc0fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 shutdown_connections 2024-03-08T23:54:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.407+0000 7f6500ade640 1 -- 172.21.15.22:0/1931804689 wait complete. 2024-03-08T23:54:09.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:54:09.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/1857253014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8102510 msgr2=0x7ff8f8102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/1857253014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8102510 0x7ff8f8102990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009900 tx=0x7ff8e802bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/1857253014 shutdown_connections 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/1857253014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f8102ed0 0x7ff8f8109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/1857253014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8102510 0x7ff8f8102990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/1857253014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f8101320 0x7ff8f8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/1857253014 >> 172.21.15.22:0/1857253014 conn(0x7ff8f80fc890 msgr2=0x7ff8f80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/1857253014 shutdown_connections 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.523+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/1857253014 wait complete. 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 Processor -- start 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 -- start start 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8101320 0x7ff8f80750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f8102510 0x7ff8f80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f8102ed0 0x7ff8f8073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f810cfb0 con 0x7ff8f8101320 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f810ce30 con 0x7ff8f8102ed0 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8fdb7d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f810d130 con 0x7ff8f8102510 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8101320 0x7ff8f80750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8101320 0x7ff8f80750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59260/0 (socket says 172.21.15.22:59260) 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 -- 172.21.15.22:0/813188442 learned_addr learned my addr 172.21.15.22:0/813188442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f7fff640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f8102ed0 0x7ff8f8073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f6ffd640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f8102510 0x7ff8f80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 -- 172.21.15.22:0/813188442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f8102510 msgr2=0x7ff8f80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f8102510 0x7ff8f80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 -- 172.21.15.22:0/813188442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f8102ed0 msgr2=0x7ff8f8073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f8102ed0 0x7ff8f8073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.524+0000 7ff8f77fe640 1 -- 172.21.15.22:0/813188442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80744b0 con 0x7ff8f8101320 2024-03-08T23:54:09.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.525+0000 7ff8f77fe640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8101320 0x7ff8f80750a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00e9c0 tx=0x7ff8ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:09.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.525+0000 7ff8f4ff9640 1 -- 172.21.15.22:0/813188442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec00cd80 con 0x7ff8f8101320 2024-03-08T23:54:09.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.525+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f81a1750 con 0x7ff8f8101320 2024-03-08T23:54:09.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.525+0000 7ff8f4ff9640 1 -- 172.21.15.22:0/813188442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8ec0035f0 con 0x7ff8f8101320 2024-03-08T23:54:09.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.525+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f81a1c70 con 0x7ff8f8101320 2024-03-08T23:54:09.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.525+0000 7ff8f4ff9640 1 -- 172.21.15.22:0/813188442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec010690 con 0x7ff8f8101320 2024-03-08T23:54:09.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.526+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f8067850 con 0x7ff8f8101320 2024-03-08T23:54:09.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.527+0000 7ff8f4ff9640 1 -- 172.21.15.22:0/813188442 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff8ec0037a0 con 0x7ff8f8101320 2024-03-08T23:54:09.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.528+0000 7ff8f4ff9640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8e00a2800 0x7ff8e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.528+0000 7ff8f6ffd640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8e00a2800 0x7ff8e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.528+0000 7ff8f4ff9640 1 -- 172.21.15.22:0/813188442 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8ec0cf270 con 0x7ff8f8101320 2024-03-08T23:54:09.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.528+0000 7ff8f6ffd640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8e00a2800 0x7ff8e00a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006010 tx=0x7ff8e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:09.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.536+0000 7ff8f4ff9640 1 -- 172.21.15.22:0/813188442 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff8ec09bf80 con 0x7ff8f8101320 2024-03-08T23:54:09.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.758+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8f8061ef0 con 0x7ff8f8101320 2024-03-08T23:54:09.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.760+0000 7ff8f4ff9640 1 -- 172.21.15.22:0/813188442 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff8ec09b880 con 0x7ff8f8101320 2024-03-08T23:54:09.756 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:09.756 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:52:18.337548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:54:09.757 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8e00a2800 msgr2=0x7ff8e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8e00a2800 0x7ff8e00a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006010 tx=0x7ff8e8000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8101320 msgr2=0x7ff8f80750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8101320 0x7ff8f80750a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00e9c0 tx=0x7ff8ec00ee80 comp rx=0 tx=0).stop 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 shutdown_connections 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f8102ed0 0x7ff8f8073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f8102510 0x7ff8f80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8e00a2800 0x7ff8e00a4cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 --2- 172.21.15.22:0/813188442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f8101320 0x7ff8f80750a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 >> 172.21.15.22:0/813188442 conn(0x7ff8f80fc890 msgr2=0x7ff8f8105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 shutdown_connections 2024-03-08T23:54:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.762+0000 7ff8fdb7d640 1 -- 172.21.15.22:0/813188442 wait complete. 2024-03-08T23:54:09.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:54:09.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 -- 172.21.15.22:0/1072900469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102500 msgr2=0x7fa674102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 --2- 172.21.15.22:0/1072900469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102500 0x7fa674102980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009ca0 tx=0x7fa65c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 -- 172.21.15.22:0/1072900469 shutdown_connections 2024-03-08T23:54:09.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 --2- 172.21.15.22:0/1072900469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa674102ec0 0x7fa674109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 --2- 172.21.15.22:0/1072900469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102500 0x7fa674102980 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 --2- 172.21.15.22:0/1072900469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674101310 0x7fa674101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 -- 172.21.15.22:0/1072900469 >> 172.21.15.22:0/1072900469 conn(0x7fa6740fc8a0 msgr2=0x7fa6740fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.878+0000 7fa6799ab640 1 -- 172.21.15.22:0/1072900469 shutdown_connections 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 -- 172.21.15.22:0/1072900469 wait complete. 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 Processor -- start 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 -- start start 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa674101310 0x7fa6741975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674102500 0x7fa674197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102ec0 0x7fa67419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67410cfa0 con 0x7fa674102ec0 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67410ce20 con 0x7fa674102500 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.879+0000 7fa6799ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67410d120 con 0x7fa674101310 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102ec0 0x7fa67419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102ec0 0x7fa67419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59276/0 (socket says 172.21.15.22:59276) 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 -- 172.21.15.22:0/2256439700 learned_addr learned my addr 172.21.15.22:0/2256439700 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6727fc640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674102500 0x7fa674197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa672ffd640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa674101310 0x7fa6741975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 -- 172.21.15.22:0/2256439700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa674101310 msgr2=0x7fa6741975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa674101310 0x7fa6741975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 -- 172.21.15.22:0/2256439700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674102500 msgr2=0x7fa674197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674102500 0x7fa674197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:09.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 -- 172.21.15.22:0/2256439700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67419f210 con 0x7fa674102ec0 2024-03-08T23:54:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6737fe640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102ec0 0x7fa67419eba0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa66400d750 tx=0x7fa66400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6789a9640 1 -- 172.21.15.22:0/2256439700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664017070 con 0x7fa674102ec0 2024-03-08T23:54:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6789a9640 1 -- 172.21.15.22:0/2256439700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa664005500 con 0x7fa674102ec0 2024-03-08T23:54:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67419f500 con 0x7fa674102ec0 2024-03-08T23:54:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6789a9640 1 -- 172.21.15.22:0/2256439700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664004d90 con 0x7fa674102ec0 2024-03-08T23:54:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.880+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67419fa20 con 0x7fa674102ec0 2024-03-08T23:54:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.881+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa674108630 con 0x7fa674102ec0 2024-03-08T23:54:09.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.883+0000 7fa6789a9640 1 -- 172.21.15.22:0/2256439700 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa664005020 con 0x7fa674102ec0 2024-03-08T23:54:09.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.883+0000 7fa6789a9640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6600a2880 0x7fa6600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:09.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.883+0000 7fa672ffd640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6600a2880 0x7fa6600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:09.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.883+0000 7fa6789a9640 1 -- 172.21.15.22:0/2256439700 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6640ce720 con 0x7fa674102ec0 2024-03-08T23:54:09.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.884+0000 7fa672ffd640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6600a2880 0x7fa6600a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa674102360 tx=0x7fa66c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:09.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:09.892+0000 7fa6789a9640 1 -- 172.21.15.22:0/2256439700 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa66409ad80 con 0x7fa674102ec0 2024-03-08T23:54:09.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:09 smithi022 ceph-mon[31894]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:09.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:09 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1376512677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:09.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:09 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4002913474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:09.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:09 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1931804689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:10.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.104+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa674061ef0 con 0x7fa674102ec0 2024-03-08T23:54:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.106+0000 7fa6789a9640 1 -- 172.21.15.22:0/2256439700 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24654 (secure 0 0 0) 0x7fa66409ee20 con 0x7fa674102ec0 2024-03-08T23:54:10.103 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:10.103 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:45.618289+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:54:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6600a2880 msgr2=0x7fa6600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6600a2880 0x7fa6600a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa674102360 tx=0x7fa66c00b390 comp rx=0 tx=0).stop 2024-03-08T23:54:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102ec0 msgr2=0x7fa67419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102ec0 0x7fa67419eba0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa66400d750 tx=0x7fa66400dc10 comp rx=0 tx=0).stop 2024-03-08T23:54:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 shutdown_connections 2024-03-08T23:54:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa6600a2880 0x7fa6600a4d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa674102ec0 0x7fa67419eba0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674102500 0x7fa674197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 --2- 172.21.15.22:0/2256439700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa674101310 0x7fa6741975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 >> 172.21.15.22:0/2256439700 conn(0x7fa6740fc8a0 msgr2=0x7fa674105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 shutdown_connections 2024-03-08T23:54:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.109+0000 7fa6799ab640 1 -- 172.21.15.22:0/2256439700 wait complete. 2024-03-08T23:54:10.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:09 smithi062 ceph-mon[36129]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:10.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:09 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1376512677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:10.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:09 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4002913474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:10.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:09 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1931804689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:10.120 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:54:10.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:54:10.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:09 smithi192 ceph-mon[35995]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:10.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:09 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1376512677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:10.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:09 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4002913474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:10.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:09 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1931804689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:10.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 -- 172.21.15.22:0/3401667329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c0ffbe0 msgr2=0x7fda9c100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/3401667329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c0ffbe0 0x7fda9c100000 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009900 tx=0x7fda8c02bcd0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 -- 172.21.15.22:0/3401667329 shutdown_connections 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/3401667329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9c100ee0 0x7fda9c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/3401667329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c100540 0x7fda9c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/3401667329 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c0ffbe0 0x7fda9c100000 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 -- 172.21.15.22:0/3401667329 >> 172.21.15.22:0/3401667329 conn(0x7fda9c0fb780 msgr2=0x7fda9c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 -- 172.21.15.22:0/3401667329 shutdown_connections 2024-03-08T23:54:10.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.231+0000 7fdaa28e9640 1 -- 172.21.15.22:0/3401667329 wait complete. 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 Processor -- start 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 -- start start 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9c0ffbe0 0x7fda9c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c100540 0x7fda9c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c100ee0 0x7fda9c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c10f180 con 0x7fda9c100540 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c10f000 con 0x7fda9c100ee0 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa28e9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c10f300 con 0x7fda9c0ffbe0 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c100540 0x7fda9c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c100540 0x7fda9c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59288/0 (socket says 172.21.15.22:59288) 2024-03-08T23:54:10.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 -- 172.21.15.22:0/157530167 learned_addr learned my addr 172.21.15.22:0/157530167 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa0e86640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c100ee0 0x7fda9c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9bfff640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9c0ffbe0 0x7fda9c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 -- 172.21.15.22:0/157530167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9c0ffbe0 msgr2=0x7fda9c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9c0ffbe0 0x7fda9c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 -- 172.21.15.22:0/157530167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c100ee0 msgr2=0x7fda9c1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c100ee0 0x7fda9c1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fda9b7fe640 1 -- 172.21.15.22:0/157530167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c1a13e0 con 0x7fda9c100540 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.232+0000 7fdaa0e86640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c100ee0 0x7fda9c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.233+0000 7fda9b7fe640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c100540 0x7fda9c199d00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fda9000e9f0 tx=0x7fda9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.233+0000 7fda997fa640 1 -- 172.21.15.22:0/157530167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000cd80 con 0x7fda9c100540 2024-03-08T23:54:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.233+0000 7fda997fa640 1 -- 172.21.15.22:0/157530167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda90010910 con 0x7fda9c100540 2024-03-08T23:54:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.233+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c1a16d0 con 0x7fda9c100540 2024-03-08T23:54:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.233+0000 7fda997fa640 1 -- 172.21.15.22:0/157530167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90018990 con 0x7fda9c100540 2024-03-08T23:54:10.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.233+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c1a1c70 con 0x7fda9c100540 2024-03-08T23:54:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.234+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c10aae0 con 0x7fda9c100540 2024-03-08T23:54:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.235+0000 7fda997fa640 1 -- 172.21.15.22:0/157530167 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fda90010ac0 con 0x7fda9c100540 2024-03-08T23:54:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.236+0000 7fda997fa640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fda840a2880 0x7fda840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.236+0000 7fda9bfff640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fda840a2880 0x7fda840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.236+0000 7fda997fa640 1 -- 172.21.15.22:0/157530167 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda900cf200 con 0x7fda9c100540 2024-03-08T23:54:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.236+0000 7fda9bfff640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fda840a2880 0x7fda840a4d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda8c006010 tx=0x7fda8c02c940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.240+0000 7fda997fa640 1 -- 172.21.15.22:0/157530167 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fda9009bf10 con 0x7fda9c100540 2024-03-08T23:54:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.421+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda9c061ef0 con 0x7fda9c100540 2024-03-08T23:54:10.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.422+0000 7fda997fa640 1 -- 172.21.15.22:0/157530167 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda9009b810 con 0x7fda9c100540 2024-03-08T23:54:10.418 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:54:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.424+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fda840a2880 msgr2=0x7fda840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.424+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fda840a2880 0x7fda840a4d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda8c006010 tx=0x7fda8c02c940 comp rx=0 tx=0).stop 2024-03-08T23:54:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.424+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c100540 msgr2=0x7fda9c199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.424+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c100540 0x7fda9c199d00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fda9000e9f0 tx=0x7fda9000eeb0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 shutdown_connections 2024-03-08T23:54:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c100ee0 0x7fda9c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fda840a2880 0x7fda840a4d40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9c100540 0x7fda9c199d00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 --2- 172.21.15.22:0/157530167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9c0ffbe0 0x7fda9c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 >> 172.21.15.22:0/157530167 conn(0x7fda9c0fb780 msgr2=0x7fda9c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 shutdown_connections 2024-03-08T23:54:10.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.425+0000 7fdaa28e9640 1 -- 172.21.15.22:0/157530167 wait complete. 2024-03-08T23:54:10.435 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:54:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 -- 172.21.15.22:0/1479341922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc1001f0 msgr2=0x7fd2bc100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/1479341922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc1001f0 0x7fd2bc100610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009900 tx=0x7fd2ac02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 -- 172.21.15.22:0/1479341922 shutdown_connections 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/1479341922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2bc101da0 0x7fd2bc108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/1479341922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2bc1013e0 0x7fd2bc101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/1479341922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc1001f0 0x7fd2bc100610 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 -- 172.21.15.22:0/1479341922 >> 172.21.15.22:0/1479341922 conn(0x7fd2bc0fb780 msgr2=0x7fd2bc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 -- 172.21.15.22:0/1479341922 shutdown_connections 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.545+0000 7fd2c3ade640 1 -- 172.21.15.22:0/1479341922 wait complete. 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 Processor -- start 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 -- start start 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2bc1001f0 0x7fd2bc1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2bc1013e0 0x7fd2bc197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc101da0 0x7fd2bc19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10be80 con 0x7fd2bc101da0 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10bd00 con 0x7fd2bc1013e0 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c3ade640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc10c000 con 0x7fd2bc1001f0 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc101da0 0x7fd2bc19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc101da0 0x7fd2bc19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52396/0 (socket says 172.21.15.22:52396) 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 -- 172.21.15.22:0/591309372 learned_addr learned my addr 172.21.15.22:0/591309372 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c1079640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2bc1013e0 0x7fd2bc197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c187a640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2bc1001f0 0x7fd2bc1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 -- 172.21.15.22:0/591309372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2bc1001f0 msgr2=0x7fd2bc1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2bc1001f0 0x7fd2bc1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 -- 172.21.15.22:0/591309372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2bc1013e0 msgr2=0x7fd2bc197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2bc1013e0 0x7fd2bc197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c207b640 1 -- 172.21.15.22:0/591309372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc19f2e0 con 0x7fd2bc101da0 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.546+0000 7fd2c187a640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2bc1001f0 0x7fd2bc1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.547+0000 7fd2c207b640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc101da0 0x7fd2bc19ec70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009fd0 tx=0x7fd2b400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.547+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/591309372 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017070 con 0x7fd2bc101da0 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.547+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/591309372 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2b4003e50 con 0x7fd2bc101da0 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.547+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2bc19f5d0 con 0x7fd2bc101da0 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.547+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/591309372 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40106c0 con 0x7fd2bc101da0 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.547+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc19fad0 con 0x7fd2bc101da0 2024-03-08T23:54:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.548+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2bc19fd30 con 0x7fd2bc101da0 2024-03-08T23:54:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.550+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/591309372 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd2b4003970 con 0x7fd2bc101da0 2024-03-08T23:54:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.551+0000 7fd2b2ffd640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2a80a2800 0x7fd2a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.551+0000 7fd2c187a640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2a80a2800 0x7fd2a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.551+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/591309372 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd2b40ceb80 con 0x7fd2bc101da0 2024-03-08T23:54:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.551+0000 7fd2c187a640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2a80a2800 0x7fd2a80a4cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac02c7b0 tx=0x7fd2ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.554+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/591309372 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd2b409b890 con 0x7fd2bc101da0 2024-03-08T23:54:10.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.738+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2bc1a0020 con 0x7fd2bc101da0 2024-03-08T23:54:10.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.738+0000 7fd2b2ffd640 1 -- 172.21.15.22:0/591309372 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2b409b190 con 0x7fd2bc101da0 2024-03-08T23:54:10.735 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:10.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2a80a2800 msgr2=0x7fd2a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2a80a2800 0x7fd2a80a4cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac02c7b0 tx=0x7fd2ac005fd0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc101da0 msgr2=0x7fd2bc19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc101da0 0x7fd2bc19ec70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009fd0 tx=0x7fd2b400b260 comp rx=0 tx=0).stop 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 shutdown_connections 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fd2a80a2800 0x7fd2a80a4cc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2bc101da0 0x7fd2bc19ec70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2bc1013e0 0x7fd2bc197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 --2- 172.21.15.22:0/591309372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2bc1001f0 0x7fd2bc1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 >> 172.21.15.22:0/591309372 conn(0x7fd2bc0fb780 msgr2=0x7fd2bc068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 shutdown_connections 2024-03-08T23:54:10.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.740+0000 7fd2c3ade640 1 -- 172.21.15.22:0/591309372 wait complete. 2024-03-08T23:54:10.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:54:10.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 -- 172.21.15.22:0/1446034133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8102eb0 msgr2=0x7fc6c8109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1446034133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8102eb0 0x7fc6c8109740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009ca0 tx=0x7fc6c402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 -- 172.21.15.22:0/1446034133 shutdown_connections 2024-03-08T23:54:10.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1446034133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8102eb0 0x7fc6c8109740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1446034133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c81024f0 0x7fc6c8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1446034133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8101300 0x7fc6c8101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 -- 172.21.15.22:0/1446034133 >> 172.21.15.22:0/1446034133 conn(0x7fc6c80fc890 msgr2=0x7fc6c80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 -- 172.21.15.22:0/1446034133 shutdown_connections 2024-03-08T23:54:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.858+0000 7fc6d0054640 1 -- 172.21.15.22:0/1446034133 wait complete. 2024-03-08T23:54:10.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 Processor -- start 2024-03-08T23:54:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 -- start start 2024-03-08T23:54:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8101300 0x7fc6c80750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c81024f0 0x7fc6c80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8102eb0 0x7fc6c8073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c810cf90 con 0x7fc6c8101300 2024-03-08T23:54:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c810ce10 con 0x7fc6c8102eb0 2024-03-08T23:54:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6d0054640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c810d110 con 0x7fc6c81024f0 2024-03-08T23:54:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8101300 0x7fc6c80750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8101300 0x7fc6c80750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52420/0 (socket says 172.21.15.22:52420) 2024-03-08T23:54:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 -- 172.21.15.22:0/1511108153 learned_addr learned my addr 172.21.15.22:0/1511108153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6ce5f1640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8102eb0 0x7fc6c8073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 -- 172.21.15.22:0/1511108153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c81024f0 msgr2=0x7fc6c80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c81024f0 0x7fc6c80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 -- 172.21.15.22:0/1511108153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8102eb0 msgr2=0x7fc6c8073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8102eb0 0x7fc6c8073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:10.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 -- 172.21.15.22:0/1511108153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c80744b0 con 0x7fc6c8101300 2024-03-08T23:54:10.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6cddf0640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8101300 0x7fc6c80750a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000d770 tx=0x7fc6c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:10.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6beffd640 1 -- 172.21.15.22:0/1511108153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0017070 con 0x7fc6c8101300 2024-03-08T23:54:10.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.859+0000 7fc6beffd640 1 -- 172.21.15.22:0/1511108153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6c0009c30 con 0x7fc6c8101300 2024-03-08T23:54:10.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.860+0000 7fc6beffd640 1 -- 172.21.15.22:0/1511108153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0003dd0 con 0x7fc6c8101300 2024-03-08T23:54:10.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.860+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c81a16e0 con 0x7fc6c8101300 2024-03-08T23:54:10.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.860+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c81a1be0 con 0x7fc6c8101300 2024-03-08T23:54:10.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.861+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c8108620 con 0x7fc6c8101300 2024-03-08T23:54:10.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.862+0000 7fc6beffd640 1 -- 172.21.15.22:0/1511108153 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc6c0003fb0 con 0x7fc6c8101300 2024-03-08T23:54:10.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.862+0000 7fc6beffd640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc6b40a2800 0x7fc6b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:10.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.862+0000 7fc6cd5ef640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc6b40a2800 0x7fc6b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:10.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.862+0000 7fc6beffd640 1 -- 172.21.15.22:0/1511108153 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc6c00ce820 con 0x7fc6c8101300 2024-03-08T23:54:10.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.863+0000 7fc6cd5ef640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc6b40a2800 0x7fc6b40a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80058c0 tx=0x7fc6b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:10.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:10.867+0000 7fc6beffd640 1 -- 172.21.15.22:0/1511108153 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc6c009b530 con 0x7fc6c8101300 2024-03-08T23:54:10.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:10 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/813188442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:10.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:10 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2256439700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:10.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:10 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/157530167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:11.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.051+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6c8061ef0 con 0x7fc6c8101300 2024-03-08T23:54:11.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.051+0000 7fc6beffd640 1 -- 172.21.15.22:0/1511108153 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6c0009de0 con 0x7fc6c8101300 2024-03-08T23:54:11.048 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:54:11.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.053+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc6b40a2800 msgr2=0x7fc6b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.053+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc6b40a2800 0x7fc6b40a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80058c0 tx=0x7fc6b8005850 comp rx=0 tx=0).stop 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.053+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8101300 msgr2=0x7fc6c80750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.053+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8101300 0x7fc6c80750a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000d770 tx=0x7fc6c000dc30 comp rx=0 tx=0).stop 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 shutdown_connections 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8102eb0 0x7fc6c8073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c81024f0 0x7fc6c80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc6b40a2800 0x7fc6b40a4cc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 --2- 172.21.15.22:0/1511108153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6c8101300 0x7fc6c80750a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 >> 172.21.15.22:0/1511108153 conn(0x7fc6c80fc890 msgr2=0x7fc6c81057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 shutdown_connections 2024-03-08T23:54:11.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.054+0000 7fc6d0054640 1 -- 172.21.15.22:0/1511108153 wait complete. 2024-03-08T23:54:11.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:54:11.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:10 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/813188442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:11.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:10 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2256439700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:11.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:10 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/157530167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:11.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:10 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/813188442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:11.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:10 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2256439700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:11.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:10 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/157530167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 -- 172.21.15.22:0/3675874298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c14101db0 msgr2=0x7f2c14108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/3675874298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c14101db0 0x7f2c14108640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009ca0 tx=0x7f2c0c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 -- 172.21.15.22:0/3675874298 shutdown_connections 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/3675874298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c14101db0 0x7f2c14108640 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/3675874298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c141013f0 0x7f2c14101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/3675874298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c14100200 0x7f2c14100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 -- 172.21.15.22:0/3675874298 >> 172.21.15.22:0/3675874298 conn(0x7f2c140fb790 msgr2=0x7f2c140fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 -- 172.21.15.22:0/3675874298 shutdown_connections 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 -- 172.21.15.22:0/3675874298 wait complete. 2024-03-08T23:54:11.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 Processor -- start 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.170+0000 7f2c18dd4640 1 -- start start 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c14100200 0x7f2c1406a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c141013f0 0x7f2c1419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c14101db0 0x7f2c1419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1410be90 con 0x7f2c141013f0 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1410bd10 con 0x7f2c14101db0 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1410c010 con 0x7f2c14100200 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c141013f0 0x7f2c1419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c141013f0 0x7f2c1419c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52436/0 (socket says 172.21.15.22:52436) 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 -- 172.21.15.22:0/2146397738 learned_addr learned my addr 172.21.15.22:0/2146397738 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c1259c640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c14100200 0x7f2c1406a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c12d9d640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c14101db0 0x7f2c1419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 -- 172.21.15.22:0/2146397738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c14100200 msgr2=0x7f2c1406a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c14100200 0x7f2c1406a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 -- 172.21.15.22:0/2146397738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c14101db0 msgr2=0x7f2c1419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c14101db0 0x7f2c1419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 -- 172.21.15.22:0/2146397738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1419f290 con 0x7f2c141013f0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c12d9d640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c14101db0 0x7f2c1419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0bfff640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c141013f0 0x7f2c1419c360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00d770 tx=0x7f2bfc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2146397738 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc017070 con 0x7f2c141013f0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2146397738 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bfc005500 con 0x7f2c141013f0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1419f520 con 0x7f2c141013f0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2146397738 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc004e00 con 0x7f2c141013f0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.171+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1419fa20 con 0x7f2c141013f0 2024-03-08T23:54:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.172+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c14107520 con 0x7f2c141013f0 2024-03-08T23:54:11.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.173+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2146397738 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2bfc004fa0 con 0x7f2c141013f0 2024-03-08T23:54:11.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.174+0000 7f2c0b7fe640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2c000a2800 0x7f2c000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.174+0000 7f2c1259c640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2c000a2800 0x7f2c000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.174+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2146397738 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2bfc0ce820 con 0x7f2c141013f0 2024-03-08T23:54:11.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.174+0000 7f2c1259c640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2c000a2800 0x7f2c000a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c04005fd0 tx=0x7f2c0400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:11.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.178+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2146397738 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2bfc09b4d0 con 0x7f2c141013f0 2024-03-08T23:54:11.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.362+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c14061f20 con 0x7f2c141013f0 2024-03-08T23:54:11.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.362+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2146397738 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2bfc09add0 con 0x7f2c141013f0 2024-03-08T23:54:11.359 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2c000a2800 msgr2=0x7f2c000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2c000a2800 0x7f2c000a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c04005fd0 tx=0x7f2c0400b410 comp rx=0 tx=0).stop 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c141013f0 msgr2=0x7f2c1419c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c141013f0 0x7f2c1419c360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00d770 tx=0x7f2bfc00dc30 comp rx=0 tx=0).stop 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 shutdown_connections 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c14101db0 0x7f2c1419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2c000a2800 0x7f2c000a4cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c141013f0 0x7f2c1419c360 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 --2- 172.21.15.22:0/2146397738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c14100200 0x7f2c1406a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 >> 172.21.15.22:0/2146397738 conn(0x7f2c140fb790 msgr2=0x7f2c141046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:11.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 shutdown_connections 2024-03-08T23:54:11.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.364+0000 7f2c18dd4640 1 -- 172.21.15.22:0/2146397738 wait complete. 2024-03-08T23:54:11.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:54:11.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/553158697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc100bb0 msgr2=0x7f4fcc101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/553158697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc100bb0 0x7f4fcc101030 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009900 tx=0x7f4fb402bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/553158697 shutdown_connections 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/553158697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fcc101570 0x7f4fcc107e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/553158697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc100bb0 0x7f4fcc101030 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/553158697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fcc0ff9c0 0x7f4fcc0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/553158697 >> 172.21.15.22:0/553158697 conn(0x7f4fcc0faf30 msgr2=0x7f4fcc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/553158697 shutdown_connections 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.478+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/553158697 wait complete. 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 Processor -- start 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 -- start start 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc0ff9c0 0x7f4fcc1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fcc100bb0 0x7f4fcc199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fcc101570 0x7f4fcc1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc10b620 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc10b4a0 con 0x7f4fcc100bb0 2024-03-08T23:54:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fd0a0d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc10b7a0 con 0x7f4fcc101570 2024-03-08T23:54:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc0ff9c0 0x7f4fcc1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc0ff9c0 0x7f4fcc1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52456/0 (socket says 172.21.15.22:52456) 2024-03-08T23:54:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 -- 172.21.15.22:0/2034623923 learned_addr learned my addr 172.21.15.22:0/2034623923 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fcad9d640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fcc101570 0x7f4fcc1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fc9d9b640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fcc100bb0 0x7f4fcc199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 -- 172.21.15.22:0/2034623923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fcc101570 msgr2=0x7f4fcc1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fcc101570 0x7f4fcc1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 -- 172.21.15.22:0/2034623923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fcc100bb0 msgr2=0x7f4fcc199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fcc100bb0 0x7f4fcc199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 -- 172.21.15.22:0/2034623923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc1a13d0 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fca59c640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc0ff9c0 0x7f4fcc1997b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c970 tx=0x7f4fc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.479+0000 7f4fc37fe640 1 -- 172.21.15.22:0/2034623923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4007c10 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.480+0000 7f4fc37fe640 1 -- 172.21.15.22:0/2034623923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fc4007db0 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.480+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc1a16c0 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.480+0000 7f4fc37fe640 1 -- 172.21.15.22:0/2034623923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400f8a0 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.480+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc1a1be0 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.481+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc106500 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.482+0000 7f4fc37fe640 1 -- 172.21.15.22:0/2034623923 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4fc400fa60 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.482+0000 7f4fc37fe640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fb80a2800 0x7f4fb80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.483+0000 7f4fc9d9b640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fb80a2800 0x7f4fb80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.483+0000 7f4fc37fe640 1 -- 172.21.15.22:0/2034623923 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4fc40cf110 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.483+0000 7f4fc9d9b640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fb80a2800 0x7f4fb80a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006010 tx=0x7f4fb4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:11.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.487+0000 7f4fc37fe640 1 -- 172.21.15.22:0/2034623923 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4fc409beb0 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.707+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fcc02ce70 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.708+0000 7f4fc37fe640 1 -- 172.21.15.22:0/2034623923 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f4fc409b7b0 con 0x7f4fcc0ff9c0 2024-03-08T23:54:11.705 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:11.705 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:52:18.337548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:54:11.705 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2024-03-08T23:54:11.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fb80a2800 msgr2=0x7f4fb80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fb80a2800 0x7f4fb80a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006010 tx=0x7f4fb4000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc0ff9c0 msgr2=0x7f4fcc1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc0ff9c0 0x7f4fcc1997b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c970 tx=0x7f4fc400ce30 comp rx=0 tx=0).stop 2024-03-08T23:54:11.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 shutdown_connections 2024-03-08T23:54:11.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fcc101570 0x7f4fcc1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fcc100bb0 0x7f4fcc199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4fb80a2800 0x7f4fb80a4cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 --2- 172.21.15.22:0/2034623923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fcc0ff9c0 0x7f4fcc1997b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 >> 172.21.15.22:0/2034623923 conn(0x7f4fcc0faf30 msgr2=0x7f4fcc0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:11.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 shutdown_connections 2024-03-08T23:54:11.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.710+0000 7f4fd0a0d640 1 -- 172.21.15.22:0/2034623923 wait complete. 2024-03-08T23:54:11.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:54:11.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 -- 172.21.15.22:0/3132071732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980ff600 msgr2=0x7f0a980ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/3132071732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980ff600 0x7f0a980ffa20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009930 tx=0x7f0a9002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:11.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 -- 172.21.15.22:0/3132071732 shutdown_connections 2024-03-08T23:54:11.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/3132071732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a98102f90 0x7f0a98109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/3132071732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a981025d0 0x7f0a98102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/3132071732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980ff600 0x7f0a980ffa20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 -- 172.21.15.22:0/3132071732 >> 172.21.15.22:0/3132071732 conn(0x7f0a980faf30 msgr2=0x7f0a980fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 -- 172.21.15.22:0/3132071732 shutdown_connections 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.824+0000 7f0a9fff1640 1 -- 172.21.15.22:0/3132071732 wait complete. 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 Processor -- start 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 -- start start 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a980ff600 0x7f0a98199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a981025d0 0x7f0a98199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a98102f90 0x7f0a981a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9810cff0 con 0x7f0a98102f90 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9810ce70 con 0x7f0a981025d0 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9fff1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9810d170 con 0x7f0a980ff600 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a98102f90 0x7f0a981a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a98102f90 0x7f0a981a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52478/0 (socket says 172.21.15.22:52478) 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 -- 172.21.15.22:0/4094346172 learned_addr learned my addr 172.21.15.22:0/4094346172 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:11.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9d58c640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a981025d0 0x7f0a98199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9dd8d640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a980ff600 0x7f0a98199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 -- 172.21.15.22:0/4094346172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a980ff600 msgr2=0x7f0a98199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a980ff600 0x7f0a98199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 -- 172.21.15.22:0/4094346172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a981025d0 msgr2=0x7f0a98199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a981025d0 0x7f0a98199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.825+0000 7f0a9e58e640 1 -- 172.21.15.22:0/4094346172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a981a14b0 con 0x7f0a98102f90 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.826+0000 7f0a9e58e640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a98102f90 0x7f0a981a0e40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800eaa0 tx=0x7f0a8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.826+0000 7f0a96ffd640 1 -- 172.21.15.22:0/4094346172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800cdb0 con 0x7f0a98102f90 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.826+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a981a17a0 con 0x7f0a98102f90 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.826+0000 7f0a96ffd640 1 -- 172.21.15.22:0/4094346172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a88005020 con 0x7f0a98102f90 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.826+0000 7f0a96ffd640 1 -- 172.21.15.22:0/4094346172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880116b0 con 0x7f0a98102f90 2024-03-08T23:54:11.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.826+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a981a1cf0 con 0x7f0a98102f90 2024-03-08T23:54:11.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.827+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a981a1fb0 con 0x7f0a98102f90 2024-03-08T23:54:11.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.828+0000 7f0a96ffd640 1 -- 172.21.15.22:0/4094346172 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0a88005560 con 0x7f0a98102f90 2024-03-08T23:54:11.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.828+0000 7f0a96ffd640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a800a2800 0x7f0a800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:11.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.829+0000 7f0a9dd8d640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a800a2800 0x7f0a800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:11.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.829+0000 7f0a96ffd640 1 -- 172.21.15.22:0/4094346172 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0a880cf0b0 con 0x7f0a98102f90 2024-03-08T23:54:11.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.829+0000 7f0a9dd8d640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a800a2800 0x7f0a800a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0a9002c7b0 tx=0x7f0a90000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:11.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:11.833+0000 7f0a96ffd640 1 -- 172.21.15.22:0/4094346172 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0a88016030 con 0x7f0a98102f90 2024-03-08T23:54:11.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:11 smithi022 ceph-mon[31894]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-08T23:54:11.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:11 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/591309372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:11.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:11 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1511108153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:11.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:11 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2146397738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:11.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:11 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2034623923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:12.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.018+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a981a2180 con 0x7f0a98102f90 2024-03-08T23:54:12.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.020+0000 7f0a96ffd640 1 -- 172.21.15.22:0/4094346172 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24654 (secure 0 0 0) 0x7f0a8809bd60 con 0x7f0a98102f90 2024-03-08T23:54:12.016 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:12.017 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:45.618289+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:54:12.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a800a2800 msgr2=0x7f0a800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a800a2800 0x7f0a800a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0a9002c7b0 tx=0x7f0a90000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a98102f90 msgr2=0x7f0a981a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a98102f90 0x7f0a981a0e40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800eaa0 tx=0x7f0a8800ef60 comp rx=0 tx=0).stop 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 shutdown_connections 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0a800a2800 0x7f0a800a4cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a98102f90 0x7f0a981a0e40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a981025d0 0x7f0a98199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 --2- 172.21.15.22:0/4094346172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a980ff600 0x7f0a98199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 >> 172.21.15.22:0/4094346172 conn(0x7f0a980faf30 msgr2=0x7f0a98107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 shutdown_connections 2024-03-08T23:54:12.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.022+0000 7f0a9fff1640 1 -- 172.21.15.22:0/4094346172 wait complete. 2024-03-08T23:54:12.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-08T23:51:51.610292+0000', 'modified': '2024-03-08T23:52:18.337548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24496, 'mds_1': 24494, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/1656345994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 1656345994}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 1656345994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/1768340207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 1768340207}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 1768340207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/1631408985', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 1631408985}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 1631408985}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.192:6837/1326709093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 1326709093}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 1326709093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.62:6839/2804197530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 2804197530}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 2804197530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.192:6839/1126404206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 1126404206}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 1126404206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T23:54:12.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-08T23:54:12.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-08T23:54:12.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:11 smithi062 ceph-mon[36129]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-08T23:54:12.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:11 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/591309372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:12.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:11 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1511108153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:12.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:11 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2146397738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:12.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:11 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2034623923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:12.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:11 smithi192 ceph-mon[35995]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-08T23:54:12.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:11 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/591309372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:12.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:11 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1511108153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:12.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:11 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2146397738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:12.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:11 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2034623923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.136+0000 7f59b0ac6640 1 -- 172.21.15.22:0/3592289515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 msgr2=0x7f59ac100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.136+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/3592289515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac100620 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f599c009900 tx=0x7f599c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.136+0000 7f59b0ac6640 1 -- 172.21.15.22:0/3592289515 shutdown_connections 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.136+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/3592289515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ac101db0 0x7f59ac108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.136+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/3592289515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac1013f0 0x7f59ac101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.136+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/3592289515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac100620 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 -- 172.21.15.22:0/3592289515 >> 172.21.15.22:0/3592289515 conn(0x7f59ac0fb790 msgr2=0x7f59ac0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 -- 172.21.15.22:0/3592289515 shutdown_connections 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 -- 172.21.15.22:0/3592289515 wait complete. 2024-03-08T23:54:12.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 Processor -- start 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 -- start start 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac1013f0 0x7f59ac197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ac101db0 0x7f59ac19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac10be90 con 0x7f59ac100200 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac10bd10 con 0x7f59ac1013f0 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.137+0000 7f59b0ac6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac10c010 con 0x7f59ac101db0 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52504/0 (socket says 172.21.15.22:52504) 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 -- 172.21.15.22:0/1907403577 learned_addr learned my addr 172.21.15.22:0/1907403577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59a3fff640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac1013f0 0x7f59ac197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 -- 172.21.15.22:0/1907403577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ac101db0 msgr2=0x7f59ac19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ac101db0 0x7f59ac19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 -- 172.21.15.22:0/1907403577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac1013f0 msgr2=0x7f59ac197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac1013f0 0x7f59ac197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 -- 172.21.15.22:0/1907403577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac19f170 con 0x7f59ac100200 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59a3fff640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac1013f0 0x7f59ac197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59aa59c640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac197550 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f599c000c00 tx=0x7f599c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59a37fe640 1 -- 172.21.15.22:0/1907403577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c041070 con 0x7f59ac100200 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59a37fe640 1 -- 172.21.15.22:0/1907403577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f599c0054e0 con 0x7f59ac100200 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac19f400 con 0x7f59ac100200 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59a37fe640 1 -- 172.21.15.22:0/1907403577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c0346a0 con 0x7f59ac100200 2024-03-08T23:54:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.138+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac19f8f0 con 0x7f59ac100200 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59a37fe640 1 -- 172.21.15.22:0/1907403577 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f599c044070 con 0x7f59ac100200 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 Processor -- start 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 -- start start 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc60500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc60ad0 0x55db7fc655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc65c40 0x55db7fc680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fbc57c0 con 0x55db7fc60ad0 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fb6eb50 con 0x55db7fc65c40 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.141+0000 7f59b3786740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fbc1860 con 0x55db7fc600e0 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc60ad0 0x55db7fc655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc60ad0 0x55db7fc655d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52518/0 (socket says 172.21.15.22:52518) 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 -- 172.21.15.22:0/367873908 learned_addr learned my addr 172.21.15.22:0/367873908 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59a37fe640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f59980a2800 0x7f59980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aad9d640 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc60500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59a3fff640 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc65c40 0x55db7fc680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59a37fe640 1 -- 172.21.15.22:0/1907403577 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f599c0f0c60 con 0x7f59ac100200 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aad9d640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f59980a2800 0x7f59980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 -- 172.21.15.22:0/367873908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 msgr2=0x55db7fc60500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc60500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 -- 172.21.15.22:0/367873908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc65c40 msgr2=0x55db7fc680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc65c40 0x55db7fc680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.142+0000 7f59aa59c640 1 -- 172.21.15.22:0/367873908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7fb829a0 con 0x55db7fc60ad0 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.143+0000 7f59aa59c640 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc60ad0 0x55db7fc655d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f599c0c5480 tx=0x7f599c003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93625f44f1323bab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.143+0000 7f59aad9d640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f59980a2800 0x7f59980a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005fd0 tx=0x7f59a4005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.143+0000 7f59a0ff9640 1 -- 172.21.15.22:0/367873908 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c04b070 con 0x55db7fc60ad0 2024-03-08T23:54:12.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.143+0000 7f59a0ff9640 1 -- 172.21.15.22:0/367873908 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f599c005690 con 0x55db7fc60ad0 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.143+0000 7f59a0ff9640 1 -- 172.21.15.22:0/367873908 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c0c59b0 con 0x55db7fc60ad0 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- 172.21.15.22:0/367873908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc60ad0 msgr2=0x55db7fc655d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc60ad0 0x55db7fc655d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f599c0c5480 tx=0x7f599c003bb0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- 172.21.15.22:0/367873908 shutdown_connections 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc65c40 0x55db7fc680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc60ad0 0x55db7fc655d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 --2- 172.21.15.22:0/367873908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc60500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- 172.21.15.22:0/367873908 >> 172.21.15.22:0/367873908 conn(0x55db7fc28400 msgr2=0x55db7fc5ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- 172.21.15.22:0/367873908 shutdown_connections 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- 172.21.15.22:0/367873908 wait complete. 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- start start 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 Processor -- start 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc6a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc689b0 con 0x55db7fc65c40 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fb81f90 con 0x55db7fc60ad0 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc68b30 con 0x55db7fc600e0 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59b3786740 10 client.0 _ll_register_callbacks cb 0x55db7fbbd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc71860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52522/0 (socket says 172.21.15.22:52522) 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.144+0000 7f59aad9d640 1 -- 172.21.15.22:0/744612080 learned_addr learned my addr 172.21.15.22:0/744612080 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59a3fff640 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc6a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59aa59c640 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59aad9d640 1 -- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 msgr2=0x55db7fc6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59aad9d640 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc6a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59aad9d640 1 -- 172.21.15.22:0/744612080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 msgr2=0x55db7fc6a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59aad9d640 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc6a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59aad9d640 1 -- 172.21.15.22:0/744612080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db7f8da2d0 con 0x55db7fc65c40 2024-03-08T23:54:12.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59aad9d640 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc71860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f59a4011a70 tx=0x7f59a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6619775e0b734c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f5991ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f5991ffb640 1 -- 172.21.15.22:0/744612080 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4024070 con 0x55db7fc65c40 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f5991ffb640 1 -- 172.21.15.22:0/744612080 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f59a400ccf0 con 0x55db7fc65c40 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db7fbd2d00 con 0x55db7fc65c40 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f5991ffb640 1 -- 172.21.15.22:0/744612080 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a4030050 con 0x55db7fc65c40 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59b3786740 10 client.15405 fetch_fsmap learned FSMap version 24 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55db7f9a46b0 con 0x55db7fc65c40 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.145+0000 7f5991ffb640 1 -- 172.21.15.22:0/744612080 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55db7fbd2d00 con 0x55db7fc65c40 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f5991ffb640 1 -- 172.21.15.22:0/744612080 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f59a400f040 con 0x55db7fc65c40 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59b3786740 10 client.15405 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59b3786740 10 client.15405 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59b3786740 20 client.15405 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59b3786740 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fc76ee0 0x55db7fc75ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59b3786740 4 client.15405 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db7fbcd050 con 0x55db7fc76ee0 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59aa59c640 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fc76ee0 0x55db7fc75ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f5991ffb640 10 client.15405 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.146+0000 7f59aa59c640 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fc76ee0 0x55db7fc75ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2983f95a88c2607 server_cookie=65c7ac9a990e2ec in_seq=0 out_seq=0 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f5991ffb640 1 -- 172.21.15.22:0/744612080 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db7fbcd050 con 0x55db7fc76ee0 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f5991ffb640 10 client.15405 handle_command_reply: tid=0 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 client.15405 shutdown 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 msgr2=0x55db7fc71860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc71860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f59a4011a70 tx=0x7f59a4004a40 comp rx=0 tx=0).stop 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 shutdown_connections 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fc76ee0 0x55db7fc75ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc71860 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc6a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f5991ffb640 0 client.15405 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 --2- 172.21.15.22:0/744612080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 >> 172.21.15.22:0/744612080 conn(0x55db7fc28400 msgr2=0x55db7fc5ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 shutdown_connections 2024-03-08T23:54:12.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 1 -- 172.21.15.22:0/744612080 wait complete. 2024-03-08T23:54:12.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.147+0000 7f59b3786740 20 client.15405 trim_cache size 0 max 16384 2024-03-08T23:54:12.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 Processor -- start 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 -- start start 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc68e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc7f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc76700 con 0x55db7fc65c40 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc76d60 con 0x55db7fc60ad0 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59b3786740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc773c0 con 0x55db7fc600e0 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc7f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc7f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52532/0 (socket says 172.21.15.22:52532) 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 -- 172.21.15.22:0/2282046350 learned_addr learned my addr 172.21.15.22:0/2282046350 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 -- 172.21.15.22:0/2282046350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 msgr2=0x55db7fc68e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc68e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59a3fff640 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 -- 172.21.15.22:0/2282046350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 msgr2=0x55db7fc71820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 -- 172.21.15.22:0/2282046350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7fc65200 con 0x55db7fc65c40 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59a3fff640 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.163+0000 7f59aad9d640 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc7f380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f59a40117c0 tx=0x7f59a4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1923f71cae881e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f5993fff640 1 -- 172.21.15.22:0/2282046350 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a402d070 con 0x55db7fc65c40 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f5993fff640 1 -- 172.21.15.22:0/2282046350 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a4013230 con 0x55db7fc65c40 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f5993fff640 1 -- 172.21.15.22:0/2282046350 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4024040 con 0x55db7fc65c40 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 -- 172.21.15.22:0/2282046350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 msgr2=0x55db7fc7f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc7f380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f59a40117c0 tx=0x7f59a4009ec0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 -- 172.21.15.22:0/2282046350 shutdown_connections 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc7f380 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 --2- 172.21.15.22:0/2282046350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc68e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 -- 172.21.15.22:0/2282046350 >> 172.21.15.22:0/2282046350 conn(0x55db7fc28400 msgr2=0x55db7fc75ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 -- 172.21.15.22:0/2282046350 shutdown_connections 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 -- 172.21.15.22:0/2282046350 wait complete. 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.164+0000 7f59b3786740 1 -- start start 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 1 Processor -- start 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc71310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc6df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc78080 con 0x55db7fc65c40 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc77a20 con 0x55db7fc60ad0 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7fc786e0 con 0x55db7fc600e0 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59b3786740 10 client.0 _ll_register_callbacks cb 0x7f599c0b9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc6df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc6df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52540/0 (socket says 172.21.15.22:52540) 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 -- 172.21.15.22:0/4246165859 learned_addr learned my addr 172.21.15.22:0/4246165859 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59a3fff640 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aa59c640 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc71310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 -- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 msgr2=0x55db7fc71310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc71310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 -- 172.21.15.22:0/4246165859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 msgr2=0x55db7fc71850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 -- 172.21.15.22:0/4246165859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db7fc6e740 con 0x55db7fc65c40 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f59aad9d640 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc6df10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f59a4012010 tx=0x7f59a4005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8914bd2f433d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f5991ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.165+0000 7f5991ffb640 1 -- 172.21.15.22:0/4246165859 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4024070 con 0x55db7fc65c40 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f5991ffb640 1 -- 172.21.15.22:0/4246165859 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f59a401d7d0 con 0x55db7fc65c40 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59a4013230 con 0x55db7fc65c40 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f5991ffb640 1 -- 172.21.15.22:0/4246165859 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a4048050 con 0x55db7fc65c40 2024-03-08T23:54:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 10 client.15411 fetch_fsmap learned FSMap version 24 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55db7fccd8b0 con 0x55db7fc65c40 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f5991ffb640 1 -- 172.21.15.22:0/4246165859 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a4013230 con 0x55db7fc65c40 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f5991ffb640 1 -- 172.21.15.22:0/4246165859 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f59a40044d0 con 0x55db7fc65c40 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 10 client.15411 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 10 client.15411 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 20 client.15411 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fcd1bb0 0x55db7fccdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 4 client.15411 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55db7fb4b9a0 con 0x55db7fcd1bb0 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.166+0000 7f59aa59c640 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fcd1bb0 0x55db7fccdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f5991ffb640 10 client.15411 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59aa59c640 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fcd1bb0 0x55db7fccdae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bdc67f2f6e81588 server_cookie=cc6242e0dca3da2d in_seq=0 out_seq=0 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f5991ffb640 1 -- 172.21.15.22:0/4246165859 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55db7fb4b9a0 con 0x55db7fcd1bb0 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f5991ffb640 10 client.15411 handle_command_reply: tid=0 2024-03-08T23:54:12.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 client.15411 shutdown 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 msgr2=0x55db7fc6df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc6df10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f59a4012010 tx=0x7f59a4005e90 comp rx=0 tx=0).stop 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 shutdown_connections 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55db7fcd1bb0 0x55db7fccdae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db7fc65c40 0x55db7fc6df10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f5991ffb640 0 client.15411 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db7fc60ad0 0x55db7fc71850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 --2- 172.21.15.22:0/4246165859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db7fc600e0 0x55db7fc71310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 >> 172.21.15.22:0/4246165859 conn(0x55db7fc28400 msgr2=0x55db7fc68e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 shutdown_connections 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 1 -- 172.21.15.22:0/4246165859 wait complete. 2024-03-08T23:54:12.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.167+0000 7f59b3786740 20 client.15411 trim_cache size 0 max 16384 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24469, 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 114.834949266, 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 89, 2024-03-08T23:54:12.171 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 158.457464864 2024-03-08T23:54:12.172 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:12.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f59980a2800 msgr2=0x7f59980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f59980a2800 0x7f59980a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005fd0 tx=0x7f59a4005f00 comp rx=0 tx=0).stop 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 msgr2=0x7f59ac197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac197550 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f599c000c00 tx=0x7f599c004040 comp rx=0 tx=0).stop 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 shutdown_connections 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ac101db0 0x7f59ac19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac1013f0 0x7f59ac197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f59980a2800 0x7f59980a4cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 --2- 172.21.15.22:0/1907403577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f59ac100200 0x7f59ac197550 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 >> 172.21.15.22:0/1907403577 conn(0x7f59ac0fb790 msgr2=0x7f59ac0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 shutdown_connections 2024-03-08T23:54:12.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.168+0000 7f59b0ac6640 1 -- 172.21.15.22:0/1907403577 wait complete. 2024-03-08T23:54:12.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.282+0000 7f7d1c145640 1 -- 172.21.15.22:0/18940285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d14101d80 msgr2=0x7f7d14104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.282+0000 7f7d1c145640 1 --2- 172.21.15.22:0/18940285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d14101d80 0x7f7d14104170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009900 tx=0x7f7d1002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- 172.21.15.22:0/18940285 shutdown_connections 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 --2- 172.21.15.22:0/18940285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d14106fe0 0x7f7d14109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 --2- 172.21.15.22:0/18940285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d141046b0 0x7f7d14106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 --2- 172.21.15.22:0/18940285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d14101d80 0x7f7d14104170 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- 172.21.15.22:0/18940285 >> 172.21.15.22:0/18940285 conn(0x7f7d140fb790 msgr2=0x7f7d140fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- 172.21.15.22:0/18940285 shutdown_connections 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- 172.21.15.22:0/18940285 wait complete. 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 Processor -- start 2024-03-08T23:54:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- start start 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d14101d80 0x7f7d141016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d141046b0 0x7f7d140ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d14106fe0 0x7f7d14100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1410cf90 con 0x7f7d141046b0 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1410ce10 con 0x7f7d14101d80 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.283+0000 7f7d1c145640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1410d110 con 0x7f7d14106fe0 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d19ee1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d14101d80 0x7f7d141016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d141046b0 0x7f7d140ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d19ee1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d14101d80 0x7f7d141016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:42940/0 (socket says 172.21.15.22:42940) 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d19ee1640 1 -- 172.21.15.22:0/798572170 learned_addr learned my addr 172.21.15.22:0/798572170 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d1a6e2640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d14106fe0 0x7f7d14100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d196e0640 1 -- 172.21.15.22:0/798572170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d14106fe0 msgr2=0x7f7d14100270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d196e0640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d14106fe0 0x7f7d14100270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d196e0640 1 -- 172.21.15.22:0/798572170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d14101d80 msgr2=0x7f7d141016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d196e0640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d14101d80 0x7f7d141016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d196e0640 1 -- 172.21.15.22:0/798572170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14100b00 con 0x7f7d141046b0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d14101d80 0x7f7d141016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d196e0640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d141046b0 0x7f7d140ffd30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7d04005fd0 tx=0x7f7d0400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d0affd640 1 -- 172.21.15.22:0/798572170 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04009c00 con 0x7f7d141046b0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d141a1780 con 0x7f7d141046b0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d141a1d20 con 0x7f7d141046b0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.284+0000 7f7d0affd640 1 -- 172.21.15.22:0/798572170 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d04009da0 con 0x7f7d141046b0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.285+0000 7f7d0affd640 1 -- 172.21.15.22:0/798572170 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04010460 con 0x7f7d141046b0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.287+0000 7f7d0affd640 1 -- 172.21.15.22:0/798572170 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7d04003f50 con 0x7f7d141046b0 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.287+0000 7f7d1edfe740 1 Processor -- start 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.287+0000 7f7d1edfe740 1 -- start start 2024-03-08T23:54:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b134500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b134ad0 0x560e9b1395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b139c40 0x560e9b13c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b0997c0 con 0x560e9b139c40 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b042b50 con 0x560e9b1340e0 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b095860 con 0x560e9b134ad0 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b139c40 0x560e9b13c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b139c40 0x560e9b13c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52576/0 (socket says 172.21.15.22:52576) 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d196e0640 1 -- 172.21.15.22:0/4044070515 learned_addr learned my addr 172.21.15.22:0/4044070515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b134ad0 0x560e9b1395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d0affd640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f7d000a2800 0x7f7d000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1a6e2640 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d1a6e2640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f7d000a2800 0x7f7d000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d0affd640 1 -- 172.21.15.22:0/798572170 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d040cecd0 con 0x7f7d141046b0 2024-03-08T23:54:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d19ee1640 1 -- 172.21.15.22:0/4044070515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 msgr2=0x560e9b134500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b134500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d19ee1640 1 -- 172.21.15.22:0/4044070515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b139c40 msgr2=0x560e9b13c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b139c40 0x560e9b13c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.288+0000 7f7d19ee1640 1 -- 172.21.15.22:0/4044070515 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9b0569a0 con 0x560e9b134ad0 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.289+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b134ad0 0x560e9b1395d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d1002c7b0 tx=0x7f7d100095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98c1b883496d08bf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.289+0000 7f7d1a6e2640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f7d000a2800 0x7f7d000a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x560e9b135330 tx=0x7f7d0c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.289+0000 7f7cfffff640 1 -- 172.21.15.22:0/4044070515 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10042070 con 0x560e9b134ad0 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.289+0000 7f7cfffff640 1 -- 172.21.15.22:0/4044070515 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d10004ba0 con 0x560e9b134ad0 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.289+0000 7f7cfffff640 1 -- 172.21.15.22:0/4044070515 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10005370 con 0x560e9b134ad0 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 -- 172.21.15.22:0/4044070515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b134ad0 msgr2=0x560e9b1395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b134ad0 0x560e9b1395d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d1002c7b0 tx=0x7f7d100095e0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 -- 172.21.15.22:0/4044070515 shutdown_connections 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b139c40 0x560e9b13c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b134ad0 0x560e9b1395d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/4044070515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b134500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 -- 172.21.15.22:0/4044070515 >> 172.21.15.22:0/4044070515 conn(0x560e9b0fc400 msgr2=0x560e9b12fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 -- 172.21.15.22:0/4044070515 shutdown_connections 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 -- 172.21.15.22:0/4044070515 wait complete. 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 -- start start 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 Processor -- start 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.290+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b13e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b13eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b145b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b13cb90 con 0x560e9b134ad0 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b055f90 con 0x560e9b1340e0 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b13cd10 con 0x560e9b139c40 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1edfe740 10 client.0 _ll_register_callbacks cb 0x560e9b091400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b13eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b13eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52582/0 (socket says 172.21.15.22:52582) 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d196e0640 1 -- 172.21.15.22:0/2430799290 learned_addr learned my addr 172.21.15.22:0/2430799290 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b13e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1a6e2640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d19ee1640 1 -- 172.21.15.22:0/2430799290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 msgr2=0x560e9b145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d19ee1640 1 -- 172.21.15.22:0/2430799290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 msgr2=0x560e9b13eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b13eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d19ee1640 1 -- 172.21.15.22:0/2430799290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e9adae2d0 con 0x560e9b1340e0 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d196e0640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b13eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b13e5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x560e9b1309a0 tx=0x7f7d10004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4e0f9f34919088 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7cfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/2430799290 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10042040 con 0x560e9b1340e0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d10004ba0 con 0x560e9b1340e0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.291+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/2430799290 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d100052b0 con 0x560e9b1340e0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/2430799290 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d100054b0 con 0x560e9b1340e0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 10 client.25064 fetch_fsmap learned FSMap version 24 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e9ae786b0 con 0x560e9b1340e0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/2430799290 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d10004ba0 con 0x560e9b1340e0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/2430799290 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7d100346e0 con 0x560e9b1340e0 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 10 client.25064 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:54:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 10 client.25064 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 20 client.25064 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b14aba0 0x560e9b14f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 4 client.25064 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e9b0a6d00 con 0x560e9b14aba0 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.292+0000 7f7d196e0640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b14aba0 0x560e9b14f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.293+0000 7f7cfd7fa640 10 client.25064 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.293+0000 7f7d196e0640 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b14aba0 0x560e9b14f020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0534aa0774623e4 server_cookie=5fe3aa74380a4c22 in_seq=0 out_seq=0 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.293+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/2430799290 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e9b0a6d00 con 0x560e9b14aba0 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7cfd7fa640 10 client.25064 handle_command_reply: tid=0 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 client.25064 shutdown 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 msgr2=0x560e9b13e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b13e5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x560e9b1309a0 tx=0x7f7d10004270 comp rx=0 tx=0).stop 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 shutdown_connections 2024-03-08T23:54:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b14aba0 0x560e9b14f020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b13eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/2430799290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b13e5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7cfd7fa640 0 client.25064 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 >> 172.21.15.22:0/2430799290 conn(0x560e9b0fc400 msgr2=0x560e9b12ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 shutdown_connections 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 1 -- 172.21.15.22:0/2430799290 wait complete. 2024-03-08T23:54:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.294+0000 7f7d1edfe740 20 client.25064 trim_cache size 0 max 16384 2024-03-08T23:54:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 Processor -- start 2024-03-08T23:54:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 -- start start 2024-03-08T23:54:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b132c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b145a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b1331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b14a3c0 con 0x560e9b134ad0 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b14aa20 con 0x560e9b1340e0 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b14b080 con 0x560e9b139c40 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52590/0 (socket says 172.21.15.22:52590) 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 -- 172.21.15.22:0/286546264 learned_addr learned my addr 172.21.15.22:0/286546264 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b132c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d1a6e2640 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b1331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 -- 172.21.15.22:0/286546264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 msgr2=0x560e9b1331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b1331d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 -- 172.21.15.22:0/286546264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 msgr2=0x560e9b132c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b132c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 -- 172.21.15.22:0/286546264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9b1307a0 con 0x560e9b134ad0 2024-03-08T23:54:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b132c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.310+0000 7f7d196e0640 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b145a40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7d04095e30 tx=0x7f7d040cc310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8194c65f96868691 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7cff7fe640 1 -- 172.21.15.22:0/286546264 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04020040 con 0x560e9b134ad0 2024-03-08T23:54:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7cff7fe640 1 -- 172.21.15.22:0/286546264 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d040d0040 con 0x560e9b134ad0 2024-03-08T23:54:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7cff7fe640 1 -- 172.21.15.22:0/286546264 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0406e5c0 con 0x560e9b134ad0 2024-03-08T23:54:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 -- 172.21.15.22:0/286546264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 msgr2=0x560e9b145a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b145a40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7d04095e30 tx=0x7f7d040cc310 comp rx=0 tx=0).stop 2024-03-08T23:54:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 -- 172.21.15.22:0/286546264 shutdown_connections 2024-03-08T23:54:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b139c40 0x560e9b1331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b145a40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/286546264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b1340e0 0x560e9b132c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 -- 172.21.15.22:0/286546264 >> 172.21.15.22:0/286546264 conn(0x560e9b0fc400 msgr2=0x560e9b136030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 -- 172.21.15.22:0/286546264 shutdown_connections 2024-03-08T23:54:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 -- 172.21.15.22:0/286546264 wait complete. 2024-03-08T23:54:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.311+0000 7f7d1edfe740 1 -- start start 2024-03-08T23:54:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 1 Processor -- start 2024-03-08T23:54:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b1340e0 0x560e9b1426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b142c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b139c40 0x560e9b1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b14bd40 con 0x560e9b134ad0 2024-03-08T23:54:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b14b6e0 con 0x560e9b139c40 2024-03-08T23:54:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9b14c3a0 con 0x560e9b1340e0 2024-03-08T23:54:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d1edfe740 10 client.0 _ll_register_callbacks cb 0x7f7d10005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b142c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52602/0 (socket says 172.21.15.22:52602) 2024-03-08T23:54:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 -- 172.21.15.22:0/570059628 learned_addr learned my addr 172.21.15.22:0/570059628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b1340e0 0x560e9b1426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 -- 172.21.15.22:0/570059628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b1340e0 msgr2=0x560e9b1426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b1340e0 0x560e9b1426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 -- 172.21.15.22:0/570059628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b139c40 msgr2=0x560e9b1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b139c40 0x560e9b1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 -- 172.21.15.22:0/570059628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e9b145d00 con 0x560e9b134ad0 2024-03-08T23:54:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7d196e0640 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b142c30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7d040977f0 tx=0x7f7d0400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dcc72c97c0b1c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.312+0000 7f7cfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/570059628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04064020 con 0x560e9b134ad0 2024-03-08T23:54:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/570059628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d04099690 con 0x560e9b134ad0 2024-03-08T23:54:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d040d0040 con 0x560e9b134ad0 2024-03-08T23:54:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/570059628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d04020030 con 0x560e9b134ad0 2024-03-08T23:54:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 10 client.15429 fetch_fsmap learned FSMap version 24 2024-03-08T23:54:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e9b1a2110 con 0x560e9b134ad0 2024-03-08T23:54:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/570059628 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d040d0040 con 0x560e9b134ad0 2024-03-08T23:54:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/570059628 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7d040bb0b0 con 0x560e9b134ad0 2024-03-08T23:54:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 10 client.15429 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:54:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 10 client.15429 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T23:54:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 20 client.15429 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b1a63d0 0x560e9b1a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 4 client.15429 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T23:54:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560e9b0a1050 con 0x560e9b1a63d0 2024-03-08T23:54:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.313+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b1a63d0 0x560e9b1a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d19ee1640 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b1a63d0 0x560e9b1a22e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67cf7a42ad8b1173 server_cookie=51acf5439d81ca63 in_seq=0 out_seq=0 2024-03-08T23:54:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7cfd7fa640 10 client.15429 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7cfd7fa640 1 -- 172.21.15.22:0/570059628 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560e9b0a1050 con 0x560e9b1a63d0 2024-03-08T23:54:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7cfd7fa640 10 client.15429 handle_command_reply: tid=0 2024-03-08T23:54:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 client.15429 shutdown 2024-03-08T23:54:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 msgr2=0x560e9b142c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b142c30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7d040977f0 tx=0x7f7d0400df90 comp rx=0 tx=0).stop 2024-03-08T23:54:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 shutdown_connections 2024-03-08T23:54:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560e9b1a63d0 0x560e9b1a22e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9b139c40 0x560e9b1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560e9b134ad0 0x560e9b142c30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 --2- 172.21.15.22:0/570059628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e9b1340e0 0x560e9b1426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7cfd7fa640 0 client.15429 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:12.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 >> 172.21.15.22:0/570059628 conn(0x560e9b0fc400 msgr2=0x560e9b132c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 shutdown_connections 2024-03-08T23:54:12.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 1 -- 172.21.15.22:0/570059628 wait complete. 2024-03-08T23:54:12.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.314+0000 7f7d1edfe740 20 client.15429 trim_cache size 0 max 16384 2024-03-08T23:54:12.323 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:12.323 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:54:12.323 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2024-03-08T23:54:12.324 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24494, 2024-03-08T23:54:12.324 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-08T23:54:12.324 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-08T23:54:12.324 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-08T23:54:12.324 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 115.992715596, 2024-03-08T23:54:12.324 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-03-08T23:54:12.324 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2024-03-08T23:54:12.325 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 89, 2024-03-08T23:54:12.325 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 144.34735925300001 2024-03-08T23:54:12.325 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:12.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f7d000a2800 msgr2=0x7f7d000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f7d000a2800 0x7f7d000a4cc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x560e9b135330 tx=0x7f7d0c004ab0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d141046b0 msgr2=0x7f7d140ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d141046b0 0x7f7d140ffd30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7d04005fd0 tx=0x7f7d0400dde0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 shutdown_connections 2024-03-08T23:54:12.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d14106fe0 0x7f7d14100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f7d000a2800 0x7f7d000a4cc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d141046b0 0x7f7d140ffd30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 --2- 172.21.15.22:0/798572170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d14101d80 0x7f7d141016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 >> 172.21.15.22:0/798572170 conn(0x7f7d140fb790 msgr2=0x7f7d14102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 shutdown_connections 2024-03-08T23:54:12.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.315+0000 7f7d1c145640 1 -- 172.21.15.22:0/798572170 wait complete. 2024-03-08T23:54:12.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-08T23:54:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 -- 172.21.15.22:0/2606684852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098101300 msgr2=0x7f1098101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 --2- 172.21.15.22:0/2606684852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098101300 0x7f1098101720 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1090009860 tx=0x7f109002bb70 comp rx=0 tx=0).stop 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 -- 172.21.15.22:0/2606684852 shutdown_connections 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 --2- 172.21.15.22:0/2606684852 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098102eb0 0x7f1098109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 --2- 172.21.15.22:0/2606684852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10981024f0 0x7f1098102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 --2- 172.21.15.22:0/2606684852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098101300 0x7f1098101720 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 -- 172.21.15.22:0/2606684852 >> 172.21.15.22:0/2606684852 conn(0x7f10980fc890 msgr2=0x7f10980fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 -- 172.21.15.22:0/2606684852 shutdown_connections 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.437+0000 7f109cd37640 1 -- 172.21.15.22:0/2606684852 wait complete. 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 Processor -- start 2024-03-08T23:54:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 -- start start 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098101300 0x7f10981976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10981024f0 0x7f1098197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098102eb0 0x7f109819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109810cf90 con 0x7f1098101300 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109810ce10 con 0x7f1098102eb0 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109cd37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109810d110 con 0x7f10981024f0 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098101300 0x7f10981976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098101300 0x7f10981976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52612/0 (socket says 172.21.15.22:52612) 2024-03-08T23:54:12.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 -- 172.21.15.22:0/4032755244 learned_addr learned my addr 172.21.15.22:0/4032755244 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f1096d9d640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098102eb0 0x7f109819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f1095d9b640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10981024f0 0x7f1098197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 -- 172.21.15.22:0/4032755244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10981024f0 msgr2=0x7f1098197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10981024f0 0x7f1098197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 -- 172.21.15.22:0/4032755244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098102eb0 msgr2=0x7f109819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098102eb0 0x7f109819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f109659c640 1 -- 172.21.15.22:0/4032755244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109819f2c0 con 0x7f1098101300 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.438+0000 7f1095d9b640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10981024f0 0x7f1098197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:12.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.439+0000 7f109659c640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098101300 0x7f10981976a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f109002c0b0 tx=0x7f1090032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.439+0000 7f108f7fe640 1 -- 172.21.15.22:0/4032755244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090041020 con 0x7f1098101300 2024-03-08T23:54:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.439+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109819f550 con 0x7f1098101300 2024-03-08T23:54:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.439+0000 7f108f7fe640 1 -- 172.21.15.22:0/4032755244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f109002c9f0 con 0x7f1098101300 2024-03-08T23:54:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.439+0000 7f108f7fe640 1 -- 172.21.15.22:0/4032755244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090039040 con 0x7f1098101300 2024-03-08T23:54:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.439+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109819fa40 con 0x7f1098101300 2024-03-08T23:54:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.441+0000 7f108f7fe640 1 -- 172.21.15.22:0/4032755244 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1090044020 con 0x7f1098101300 2024-03-08T23:54:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 Processor -- start 2024-03-08T23:54:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 -- start start 2024-03-08T23:54:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b1fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b1597c0 con 0x55945b1f4ad0 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b102b50 con 0x55945b1f9c40 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109f9ee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b155860 con 0x55945b1f40e0 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f108f7fe640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10840a2800 0x7f10840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52626/0 (socket says 172.21.15.22:52626) 2024-03-08T23:54:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f109659c640 1 -- 172.21.15.22:0/4026433001 learned_addr learned my addr 172.21.15.22:0/4026433001 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f1096d9d640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10840a2800 0x7f10840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f1096d9d640 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.442+0000 7f1095d9b640 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b1fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f108f7fe640 1 -- 172.21.15.22:0/4032755244 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f10900f0770 con 0x7f1098101300 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f109659c640 1 -- 172.21.15.22:0/4026433001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 msgr2=0x55945b1f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f109659c640 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f109659c640 1 -- 172.21.15.22:0/4026433001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 msgr2=0x55945b1fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f109659c640 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b1fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f109659c640 1 -- 172.21.15.22:0/4026433001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945b1169a0 con 0x55945b1f4ad0 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f1095d9b640 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b1fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f1096d9d640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10840a2800 0x7f10840a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1088005fd0 tx=0x7f1088009e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f109659c640 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f95d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f10900edee0 tx=0x7f1090032970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509e6a965e572c4c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f108cff9640 1 -- 172.21.15.22:0/4026433001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10900bc8c0 con 0x55945b1f4ad0 2024-03-08T23:54:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f108cff9640 1 -- 172.21.15.22:0/4026433001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10900bd480 con 0x55945b1f4ad0 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.443+0000 7f108cff9640 1 -- 172.21.15.22:0/4026433001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10900bc6e0 con 0x55945b1f4ad0 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- 172.21.15.22:0/4026433001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 msgr2=0x55945b1f95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f95d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f10900edee0 tx=0x7f1090032970 comp rx=0 tx=0).stop 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- 172.21.15.22:0/4026433001 shutdown_connections 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b1fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f95d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 --2- 172.21.15.22:0/4026433001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- 172.21.15.22:0/4026433001 >> 172.21.15.22:0/4026433001 conn(0x55945b1bc400 msgr2=0x55945b1efa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- 172.21.15.22:0/4026433001 shutdown_connections 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- 172.21.15.22:0/4026433001 wait complete. 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- start start 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 Processor -- start 2024-03-08T23:54:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b1fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b205870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b1fc9b0 con 0x55945b1f9c40 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b115f90 con 0x55945b1f4ad0 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b1fcb30 con 0x55945b1f40e0 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.444+0000 7f109f9ee740 10 client.0 _ll_register_callbacks cb 0x55945b151400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b205870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b205870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52634/0 (socket says 172.21.15.22:52634) 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 -- 172.21.15.22:0/656747946 learned_addr learned my addr 172.21.15.22:0/656747946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f109659c640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1095d9b640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b1fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 -- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 msgr2=0x55945b1fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 -- 172.21.15.22:0/656747946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 msgr2=0x55945b1fe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b1fe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 -- 172.21.15.22:0/656747946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55945ae6e2d0 con 0x55945b1f9c40 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f109659c640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1095d9b640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b1fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f1096d9d640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b205870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f108801d9b0 tx=0x7f108801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5554b507035b425 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f107dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f107dffb640 1 -- 172.21.15.22:0/656747946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108801ddb0 con 0x55945b1f9c40 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f107dffb640 1 -- 172.21.15.22:0/656747946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f108801c930 con 0x55945b1f9c40 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10900bd480 con 0x55945b1f9c40 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f107dffb640 1 -- 172.21.15.22:0/656747946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1088034050 con 0x55945b1f9c40 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f109f9ee740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.445+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55945af386b0 con 0x55945b1f9c40 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f107dffb640 1 -- 172.21.15.22:0/656747946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10900bd480 con 0x55945b1f9c40 2024-03-08T23:54:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f107dffb640 1 -- 172.21.15.22:0/656747946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f108800b040 con 0x55945b1f9c40 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109f9ee740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109f9ee740 10 client.15441 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109f9ee740 20 client.15441 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109f9ee740 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b20af00 0x55945b209ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109f9ee740 4 client.15441 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55945b166d00 con 0x55945b20af00 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109659c640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b20af00 0x55945b209ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f107dffb640 10 client.15441 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.446+0000 7f109659c640 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b20af00 0x55945b209ee0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908ff4ce73e4cf05 server_cookie=8436a44e2de5113 in_seq=0 out_seq=0 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f107dffb640 1 -- 172.21.15.22:0/656747946 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55945b166d00 con 0x55945b20af00 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f107dffb640 10 client.15441 handle_command_reply: tid=0 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 client.15441 shutdown 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 msgr2=0x55945b205870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b205870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f108801d9b0 tx=0x7f108801d000 comp rx=0 tx=0).stop 2024-03-08T23:54:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 shutdown_connections 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b20af00 0x55945b209ee0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b205870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b1fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 --2- 172.21.15.22:0/656747946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f107dffb640 0 client.15441 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 >> 172.21.15.22:0/656747946 conn(0x55945b1bc400 msgr2=0x55945b1eec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 shutdown_connections 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 1 -- 172.21.15.22:0/656747946 wait complete. 2024-03-08T23:54:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.447+0000 7f109f9ee740 20 client.15441 trim_cache size 0 max 16384 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 Processor -- start 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 -- start start 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b205830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b20a720 con 0x55945b1f4ad0 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b20ad80 con 0x55945b1f9c40 2024-03-08T23:54:12.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f109f9ee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b20b3e0 con 0x55945b1f40e0 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f1095d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.463+0000 7f1095d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52648/0 (socket says 172.21.15.22:52648) 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1095d9b640 1 -- 172.21.15.22:0/1679768284 learned_addr learned my addr 172.21.15.22:0/1679768284 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f109659c640 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1095d9b640 1 -- 172.21.15.22:0/1679768284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 msgr2=0x55945b1f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1095d9b640 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1096d9d640 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b205830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1095d9b640 1 -- 172.21.15.22:0/1679768284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 msgr2=0x55945b205830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1095d9b640 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b205830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1095d9b640 1 -- 172.21.15.22:0/1679768284 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945b20a550 con 0x55945b1f4ad0 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1096d9d640 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b205830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f1095d9b640 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f6570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1080009bc0 tx=0x7f108000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0367b8cd116ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f107ffff640 1 -- 172.21.15.22:0/1679768284 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080009d70 con 0x55945b1f4ad0 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f107ffff640 1 -- 172.21.15.22:0/1679768284 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1080005500 con 0x55945b1f4ad0 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.464+0000 7f107ffff640 1 -- 172.21.15.22:0/1679768284 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108001c6d0 con 0x55945b1f4ad0 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- 172.21.15.22:0/1679768284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 msgr2=0x55945b1f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f6570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1080009bc0 tx=0x7f108000f7f0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- 172.21.15.22:0/1679768284 shutdown_connections 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f9c40 0x55945b205830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f4ad0 0x55945b1f6570 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1679768284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b1f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- 172.21.15.22:0/1679768284 >> 172.21.15.22:0/1679768284 conn(0x55945b1bc400 msgr2=0x55945b2133a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- 172.21.15.22:0/1679768284 shutdown_connections 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- 172.21.15.22:0/1679768284 wait complete. 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- start start 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 Processor -- start 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b204a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b204f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b2054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b20c0a0 con 0x55945b1f9c40 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b20ba40 con 0x55945b1f4ad0 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945b20c700 con 0x55945b1f40e0 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109f9ee740 10 client.0 _ll_register_callbacks cb 0x7f10900b8900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f1096d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b2054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f1096d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b2054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52660/0 (socket says 172.21.15.22:52660) 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f1096d9d640 1 -- 172.21.15.22:0/1480073131 learned_addr learned my addr 172.21.15.22:0/1480073131 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.465+0000 7f109659c640 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b204a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f1096d9d640 1 -- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 msgr2=0x55945b204a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f1096d9d640 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b204a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f1096d9d640 1 -- 172.21.15.22:0/1480073131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 msgr2=0x55945b204f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f1096d9d640 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b204f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f1096d9d640 1 -- 172.21.15.22:0/1480073131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55945b260b00 con 0x55945b1f9c40 2024-03-08T23:54:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f1096d9d640 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b2054e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1088004a80 tx=0x7f1088004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb05079f025e542b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f107dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f107dffb640 1 -- 172.21.15.22:0/1480073131 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108801cb90 con 0x55945b1f9c40 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f107dffb640 1 -- 172.21.15.22:0/1480073131 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f108801cd30 con 0x55945b1f9c40 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1080005500 con 0x55945b1f9c40 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f107dffb640 1 -- 172.21.15.22:0/1480073131 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1088046050 con 0x55945b1f9c40 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f109f9ee740 10 client.15447 fetch_fsmap learned FSMap version 24 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55945b261c20 con 0x55945b1f9c40 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f107dffb640 1 -- 172.21.15.22:0/1480073131 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1080005500 con 0x55945b1f9c40 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f107dffb640 1 -- 172.21.15.22:0/1480073131 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f10880413f0 con 0x55945b1f9c40 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.466+0000 7f109f9ee740 10 client.15447 fetch_fsmap finished waiting for FSMap version 24 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 10 client.15447 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 20 client.15447 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b266360 0x55945b261eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 4 client.15447 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55945b161050 con 0x55945b266360 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109659c640 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b266360 0x55945b261eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109659c640 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b266360 0x55945b261eb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b855373ae3ed96a server_cookie=f4e82be019941c19 in_seq=0 out_seq=0 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f107dffb640 10 client.15447 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f107dffb640 1 -- 172.21.15.22:0/1480073131 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55945b161050 con 0x55945b266360 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f107dffb640 10 client.15447 handle_command_reply: tid=0 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 client.15447 shutdown 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 msgr2=0x55945b2054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b2054e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1088004a80 tx=0x7f1088004ab0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 shutdown_connections 2024-03-08T23:54:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55945b266360 0x55945b261eb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55945b1f9c40 0x55945b2054e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55945b1f4ad0 0x55945b204f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 --2- 172.21.15.22:0/1480073131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55945b1f40e0 0x55945b204a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 >> 172.21.15.22:0/1480073131 conn(0x55945b1bc400 msgr2=0x55945b1f6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.467+0000 7f107dffb640 0 client.15447 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 shutdown_connections 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109f9ee740 1 -- 172.21.15.22:0/1480073131 wait complete. 2024-03-08T23:54:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109f9ee740 20 client.15447 trim_cache size 0 max 16384 2024-03-08T23:54:12.473 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:12.473 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "8c5a3016-dda5-11ee-95c6-87774f69a715", 2024-03-08T23:54:12.473 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24496, 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 140.23019440600001, 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 24, 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 89, 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 152.91610613 2024-03-08T23:54:12.474 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10840a2800 msgr2=0x7f10840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10840a2800 0x7f10840a4cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1088005fd0 tx=0x7f1088009e10 comp rx=0 tx=0).stop 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098101300 msgr2=0x7f10981976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098101300 0x7f10981976a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f109002c0b0 tx=0x7f1090032430 comp rx=0 tx=0).stop 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 shutdown_connections 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1098102eb0 0x7f109819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10981024f0 0x7f1098197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f10840a2800 0x7f10840a4cc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 --2- 172.21.15.22:0/4032755244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1098101300 0x7f10981976a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 >> 172.21.15.22:0/4032755244 conn(0x7f10980fc890 msgr2=0x7f1098105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 shutdown_connections 2024-03-08T23:54:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:12.468+0000 7f109cd37640 1 -- 172.21.15.22:0/4032755244 wait complete. 2024-03-08T23:54:12.477 INFO:teuthology.run_tasks:Running task kclient... 2024-03-08T23:54:12.490 INFO:tasks.kclient:Mounting kernel clients... 2024-03-08T23:54:12.490 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-08T23:54:12.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:12.490 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-08T23:54:12.505 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-08T23:54:12.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:12.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-08T23:54:12.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:12.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-03-08T23:54:12.678 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-08T23:54:12.678 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T23:54:12.678 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:12.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-08T23:54:12.926 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T23:54:12.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:12 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4094346172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:13.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:12 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4094346172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:13.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:12 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4094346172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:13.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:13 smithi022 ceph-mon[31894]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:54:14.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:13 smithi062 ceph-mon[36129]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:54:14.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:13 smithi192 ceph-mon[35995]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 -- 172.21.15.22:0/3782838464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4100720 msgr2=0x7f1ab4100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3782838464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4100720 0x7f1ab4100ba0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009990 tx=0x7f1aa4009ca0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 -- 172.21.15.22:0/3782838464 shutdown_connections 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3782838464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab41010e0 0x7f1ab410d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3782838464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ab4106590 0x7f1ab4106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3782838464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4100720 0x7f1ab4100ba0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 -- 172.21.15.22:0/3782838464 >> 172.21.15.22:0/3782838464 conn(0x7f1ab4076040 msgr2=0x7f1ab4076450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 -- 172.21.15.22:0/3782838464 shutdown_connections 2024-03-08T23:54:14.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.697+0000 7f1ab9435700 1 -- 172.21.15.22:0/3782838464 wait complete. 2024-03-08T23:54:14.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.698+0000 7f1ab9435700 1 Processor -- start 2024-03-08T23:54:14.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.698+0000 7f1ab9435700 1 -- start start 2024-03-08T23:54:14.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.698+0000 7f1ab9435700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab4100720 0x7f1ab41020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:14.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.698+0000 7f1ab9435700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ab41010e0 0x7f1ab41025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:14.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.698+0000 7f1ab9435700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4106590 0x7f1ab4102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab9435700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4110fd0 con 0x7f1ab4106590 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab9435700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4110e50 con 0x7f1ab4100720 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab9435700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4111150 con 0x7f1ab41010e0 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4106590 0x7f1ab4102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4106590 0x7f1ab4102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52676/0 (socket says 172.21.15.22:52676) 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 -- 172.21.15.22:0/3965181203 learned_addr learned my addr 172.21.15.22:0/3965181203 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab2ffd700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab4100720 0x7f1ab41020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab27fc700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ab41010e0 0x7f1ab41025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 -- 172.21.15.22:0/3965181203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ab41010e0 msgr2=0x7f1ab41025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ab41010e0 0x7f1ab41025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 -- 172.21.15.22:0/3965181203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab4100720 msgr2=0x7f1ab41020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab4100720 0x7f1ab41020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 -- 172.21.15.22:0/3965181203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab41a6af0 con 0x7f1ab4106590 2024-03-08T23:54:14.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab37fe700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4106590 0x7f1ab4102b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007ee0 tx=0x7f1aac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1aabfff700 1 -- 172.21.15.22:0/3965181203 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac004bb0 con 0x7f1ab4106590 2024-03-08T23:54:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab41a6de0 con 0x7f1ab4106590 2024-03-08T23:54:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1aabfff700 1 -- 172.21.15.22:0/3965181203 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1aac004d50 con 0x7f1ab4106590 2024-03-08T23:54:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.699+0000 7f1aabfff700 1 -- 172.21.15.22:0/3965181203 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac005960 con 0x7f1ab4106590 2024-03-08T23:54:14.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.700+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab41a72e0 con 0x7f1ab4106590 2024-03-08T23:54:14.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.701+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab4068aa0 con 0x7f1ab4106590 2024-03-08T23:54:14.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.702+0000 7f1aabfff700 1 -- 172.21.15.22:0/3965181203 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1aac01a020 con 0x7f1ab4106590 2024-03-08T23:54:14.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.702+0000 7f1aabfff700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1aa00a2890 0x7f1aa00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:14.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.702+0000 7f1ab2ffd700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1aa00a2890 0x7f1aa00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:14.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.702+0000 7f1aabfff700 1 -- 172.21.15.22:0/3965181203 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1aac0ce470 con 0x7f1ab4106590 2024-03-08T23:54:14.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.703+0000 7f1ab2ffd700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1aa00a2890 0x7f1aa00a4d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0045b0 tx=0x7f1a9c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:14.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.707+0000 7f1aabfff700 1 -- 172.21.15.22:0/3965181203 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1aac010070 con 0x7f1ab4106590 2024-03-08T23:54:14.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.957+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ab402d830 con 0x7f1ab4106590 2024-03-08T23:54:14.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.957+0000 7f1aabfff700 1 -- 172.21.15.22:0/3965181203 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1aac09b180 con 0x7f1ab4106590 2024-03-08T23:54:14.954 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1aa00a2890 msgr2=0x7f1aa00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1aa00a2890 0x7f1aa00a4d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0045b0 tx=0x7f1a9c00b3c0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4106590 msgr2=0x7f1ab4102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4106590 0x7f1ab4102b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007ee0 tx=0x7f1aac00d3b0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 shutdown_connections 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1aa00a2890 0x7f1aa00a4d50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4106590 0x7f1ab4102b30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab4100720 0x7f1ab41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 --2- 172.21.15.22:0/3965181203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ab41010e0 0x7f1ab41025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:14.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.959+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 >> 172.21.15.22:0/3965181203 conn(0x7f1ab4076040 msgr2=0x7f1ab410dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:14.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.960+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 shutdown_connections 2024-03-08T23:54:14.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:14.960+0000 7f1ab9435700 1 -- 172.21.15.22:0/3965181203 wait complete. 2024-03-08T23:54:15.724 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T23:54:15.725 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T23:54:15.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:15.725 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-03-08T23:54:15.742 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T23:54:15.742 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T23:54:15.742 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T23:54:15.742 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 41971sec preferred_lft 41971sec 2024-03-08T23:54:15.743 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-03-08T23:54:15.744 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-03-08T23:54:15.744 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T23:54:15.744 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-03-08T23:54:15.744 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-03-08T23:54:15.744 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T23:54:15.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:15.744 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:54:15.744 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:54:15.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-03-08T23:54:15.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-03-08T23:54:15.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-03-08T23:54:15.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T23:54:15.745 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:54:15.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:15.855 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T23:54:15.882 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-03-08T23:54:15.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:15.884 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-03-08T23:54:15.940 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-03-08T23:54:15.940 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T23:54:15.940 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-08T23:54:15.940 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-08T23:54:15.940 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T23:54:15.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:15.940 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:54:15.941 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:54:15.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-08T23:54:15.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-08T23:54:15.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-08T23:54:15.941 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:54:15.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:15 smithi022 ceph-mon[31894]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:15.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3965181203' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T23:54:16.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.035 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-08T23:54:16.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.089 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-03-08T23:54:16.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:15 smithi062 ceph-mon[36129]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:16.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3965181203' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T23:54:16.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:15 smithi192 ceph-mon[35995]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:16.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3965181203' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T23:54:16.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.143 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:54:16.143 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:54:16.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T23:54:16.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-08T23:54:16.144 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:54:16.225 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-08T23:54:16.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.225 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:54:16.226 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:54:16.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-08T23:54:16.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T23:54:16.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-08T23:54:16.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-08T23:54:16.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-08T23:54:16.226 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:54:16.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.350 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2024-03-08T23:54:16.353 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-03-08T23:54:16.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-03-08T23:54:16.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T23:54:16.354 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-03-08T23:54:16.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T23:54:16.395 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:16.451 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T23:54:16.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T23:54:16.451 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:16.505 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-08T23:54:16.505 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-08T23:54:16.506 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/d3c2f801-c4f0-4116-844e-51d32882c535 2024-03-08T23:54:16.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/d3c2f801-c4f0-4116-844e-51d32882c535 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-08T23:54:16.590 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-08T23:54:16.590 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-08T23:54:16.590 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-03-08T23:54:16.719 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-08T23:54:16.719 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-08T23:54:16.719 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-03-08T23:54:16.719 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.62:6789,172.21.15.192:6789" 2024-03-08T23:54:16.719 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.62:6789,172.21.15.192:6789:/volumes/qa/sv_1/d3c2f801-c4f0-4116-844e-51d32882c535 2024-03-08T23:54:16.720 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=8c5a3016-dda5-11ee-95c6-87774f69a715" will pass to kernel 2024-03-08T23:54:16.720 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-08T23:54:16.720 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-03-08T23:54:16.720 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T23:54:16.720 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T23:54:16.721 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T23:54:16.722 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T23:54:16.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:16.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.794 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T23:54:16.851 INFO:teuthology.orchestra.run.smithi022.stdout:client15456 2024-03-08T23:54:16.851 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:54:16.852 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T23:54:16.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.907 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T23:54:16.964 INFO:teuthology.orchestra.run.smithi022.stdout:client15456 2024-03-08T23:54:16.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:16.964 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T23:54:17.020 INFO:teuthology.orchestra.run.smithi022.stdout:client15456 2024-03-08T23:54:17.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:17.021 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715.client15456/status') 2024-03-08T23:54:17.088 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.15456 (3)192.168.0.1:0/447112747 2024-03-08T23:54:17.088 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2024-03-08T23:54:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2024-03-08T23:54:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2024-03-08T23:54:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:69 bytes copied, 2.1271e-05 s, 3.2 MB/s 2024-03-08T23:54:17.090 INFO:teuthology.run_tasks:Running task exec... 2024-03-08T23:54:17.106 INFO:teuthology.task.exec:Executing custom commands... 2024-03-08T23:54:17.106 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-08T23:54:17.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-08T23:54:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 -- 172.21.15.22:0/3940917902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 msgr2=0x7f6120100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 --2- 172.21.15.22:0/3940917902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120100650 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6110009900 tx=0x7f611002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 -- 172.21.15.22:0/3940917902 shutdown_connections 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 --2- 172.21.15.22:0/3940917902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6120101de0 0x7f6120108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 --2- 172.21.15.22:0/3940917902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6120101420 0x7f61201018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 --2- 172.21.15.22:0/3940917902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120100650 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 -- 172.21.15.22:0/3940917902 >> 172.21.15.22:0/3940917902 conn(0x7f61200fb7a0 msgr2=0x7f61200fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 -- 172.21.15.22:0/3940917902 shutdown_connections 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.238+0000 7f6126a32640 1 -- 172.21.15.22:0/3940917902 wait complete. 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 Processor -- start 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 -- start start 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6120101420 0x7f6120197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6120101de0 0x7f612019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612010bec0 con 0x7f6120100230 2024-03-08T23:54:17.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612010bd40 con 0x7f6120101de0 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6126a32640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612010c040 con 0x7f6120101420 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52690/0 (socket says 172.21.15.22:52690) 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 -- 172.21.15.22:0/1708639192 learned_addr learned my addr 172.21.15.22:0/1708639192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 -- 172.21.15.22:0/1708639192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6120101420 msgr2=0x7f6120197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6120101420 0x7f6120197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 -- 172.21.15.22:0/1708639192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6120101de0 msgr2=0x7f612019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f6124fcf640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6120101de0 0x7f612019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6120101de0 0x7f612019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 -- 172.21.15.22:0/1708639192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612019f170 con 0x7f6120100230 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.239+0000 7f611ffff640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120197550 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6110005ab0 tx=0x7f6110005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.240+0000 7f611d7fa640 1 -- 172.21.15.22:0/1708639192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110041070 con 0x7f6120100230 2024-03-08T23:54:17.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.240+0000 7f611d7fa640 1 -- 172.21.15.22:0/1708639192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61100054e0 con 0x7f6120100230 2024-03-08T23:54:17.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.240+0000 7f611d7fa640 1 -- 172.21.15.22:0/1708639192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110034440 con 0x7f6120100230 2024-03-08T23:54:17.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.240+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612019f400 con 0x7f6120100230 2024-03-08T23:54:17.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.240+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612019f8f0 con 0x7f6120100230 2024-03-08T23:54:17.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.241+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612019fb50 con 0x7f6120100230 2024-03-08T23:54:17.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.242+0000 7f611d7fa640 1 -- 172.21.15.22:0/1708639192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6110044070 con 0x7f6120100230 2024-03-08T23:54:17.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.242+0000 7f611d7fa640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f61080a28a0 0x7f61080a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.242+0000 7f611f7fe640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f61080a28a0 0x7f61080a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:17.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.242+0000 7f611d7fa640 1 -- 172.21.15.22:0/1708639192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f61100f0b10 con 0x7f6120100230 2024-03-08T23:54:17.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.243+0000 7f611f7fe640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f61080a28a0 0x7f61080a4d60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6120198a60 tx=0x7f6114005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:17.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.247+0000 7f611d7fa640 1 -- 172.21.15.22:0/1708639192 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f61100bd820 con 0x7f6120100230 2024-03-08T23:54:17.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.469+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f612019fee0 con 0x7f6120100230 2024-03-08T23:54:17.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.737+0000 7f611d7fa640 1 -- 172.21.15.22:0/1708639192 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f61100bd120 con 0x7f6120100230 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f61080a28a0 msgr2=0x7f61080a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f61080a28a0 0x7f61080a4d60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6120198a60 tx=0x7f6114005ed0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 msgr2=0x7f6120197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120197550 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6110005ab0 tx=0x7f6110005dc0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 shutdown_connections 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6120101de0 0x7f612019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6120101420 0x7f6120197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f61080a28a0 0x7f61080a4d60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 --2- 172.21.15.22:0/1708639192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6120100230 0x7f6120197550 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 >> 172.21.15.22:0/1708639192 conn(0x7f61200fb7a0 msgr2=0x7f61200fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.739+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 shutdown_connections 2024-03-08T23:54:17.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.740+0000 7f6126a32640 1 -- 172.21.15.22:0/1708639192 wait complete. 2024-03-08T23:54:17.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-08T23:54:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- 172.21.15.22:0/2839785725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101410 msgr2=0x7fa16c101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 --2- 172.21.15.22:0/2839785725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101410 0x7fa16c101890 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa160009900 tx=0x7fa16002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- 172.21.15.22:0/2839785725 shutdown_connections 2024-03-08T23:54:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 --2- 172.21.15.22:0/2839785725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c101dd0 0x7fa16c108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 --2- 172.21.15.22:0/2839785725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101410 0x7fa16c101890 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 --2- 172.21.15.22:0/2839785725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c100220 0x7fa16c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- 172.21.15.22:0/2839785725 >> 172.21.15.22:0/2839785725 conn(0x7fa16c0fb790 msgr2=0x7fa16c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- 172.21.15.22:0/2839785725 shutdown_connections 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- 172.21.15.22:0/2839785725 wait complete. 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 Processor -- start 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- start start 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c100220 0x7fa16c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c101410 0x7fa16c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101dd0 0x7fa16c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c10cfc0 con 0x7fa16c101dd0 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c10ce40 con 0x7fa16c100220 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.858+0000 7fa173569640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c10d140 con 0x7fa16c101410 2024-03-08T23:54:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101dd0 0x7fa16c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101dd0 0x7fa16c19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52712/0 (socket says 172.21.15.22:52712) 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 -- 172.21.15.22:0/2415933106 learned_addr learned my addr 172.21.15.22:0/2415933106 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa170b04640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c101410 0x7fa16c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171305640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c100220 0x7fa16c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 -- 172.21.15.22:0/2415933106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c101410 msgr2=0x7fa16c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c101410 0x7fa16c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 -- 172.21.15.22:0/2415933106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c100220 msgr2=0x7fa16c197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c100220 0x7fa16c197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 -- 172.21.15.22:0/2415933106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c19f1a0 con 0x7fa16c101dd0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa171b06640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101dd0 0x7fa16c19eb30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00eb10 tx=0x7fa15c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa16a7fc640 1 -- 172.21.15.22:0/2415933106 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004b20 con 0x7fa16c101dd0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa16a7fc640 1 -- 172.21.15.22:0/2415933106 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c004cc0 con 0x7fa16c101dd0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c19f490 con 0x7fa16c101dd0 2024-03-08T23:54:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa16a7fc640 1 -- 172.21.15.22:0/2415933106 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c010760 con 0x7fa16c101dd0 2024-03-08T23:54:17.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.859+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c19f9b0 con 0x7fa16c101dd0 2024-03-08T23:54:17.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.860+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c107540 con 0x7fa16c101dd0 2024-03-08T23:54:17.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.861+0000 7fa16a7fc640 1 -- 172.21.15.22:0/2415933106 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa15c005020 con 0x7fa16c101dd0 2024-03-08T23:54:17.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.862+0000 7fa16a7fc640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa1540a2920 0x7fa1540a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:17.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.862+0000 7fa171305640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa1540a2920 0x7fa1540a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:17.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.862+0000 7fa16a7fc640 1 -- 172.21.15.22:0/2415933106 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa15c0cf450 con 0x7fa16c101dd0 2024-03-08T23:54:17.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.862+0000 7fa171305640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa1540a2920 0x7fa1540a4de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa16c101270 tx=0x7fa164005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:17.866+0000 7fa16a7fc640 1 -- 172.21.15.22:0/2415933106 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa15c09c160 con 0x7fa16c101dd0 2024-03-08T23:54:18.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.076+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fa16c061730 con 0x7fa1540a2920 2024-03-08T23:54:18.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:17 smithi022 ceph-mon[31894]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:54:18.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1708639192' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-08T23:54:18.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.096+0000 7fa16a7fc640 1 -- 172.21.15.22:0/2415933106 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fa16c061730 con 0x7fa1540a2920 2024-03-08T23:54:18.092 INFO:teuthology.orchestra.run.smithi022.stdout:{} 2024-03-08T23:54:18.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa1540a2920 msgr2=0x7fa1540a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa1540a2920 0x7fa1540a4de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa16c101270 tx=0x7fa164005f90 comp rx=0 tx=0).stop 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101dd0 msgr2=0x7fa16c19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101dd0 0x7fa16c19eb30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00eb10 tx=0x7fa15c00c2d0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 shutdown_connections 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa1540a2920 0x7fa1540a4de0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa16c101dd0 0x7fa16c19eb30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa16c101410 0x7fa16c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 --2- 172.21.15.22:0/2415933106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c100220 0x7fa16c197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 >> 172.21.15.22:0/2415933106 conn(0x7fa16c0fb790 msgr2=0x7fa16c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 shutdown_connections 2024-03-08T23:54:18.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.098+0000 7fa173569640 1 -- 172.21.15.22:0/2415933106 wait complete. 2024-03-08T23:54:18.105 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-08T23:54:18.116 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-08T23:54:18.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:54:18.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:17 smithi062 ceph-mon[36129]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:54:18.127 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1708639192' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-08T23:54:18.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:17 smithi192 ceph-mon[35995]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-08T23:54:18.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1708639192' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 -- 172.21.15.22:0/3497595738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 msgr2=0x7fc288101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 --2- 172.21.15.22:0/3497595738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288101730 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc280009900 tx=0x7fc28002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 -- 172.21.15.22:0/3497595738 shutdown_connections 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 --2- 172.21.15.22:0/3497595738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288102ec0 0x7fc288109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 --2- 172.21.15.22:0/3497595738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc288102500 0x7fc288102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 --2- 172.21.15.22:0/3497595738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288101730 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 -- 172.21.15.22:0/3497595738 >> 172.21.15.22:0/3497595738 conn(0x7fc2880fc8a0 msgr2=0x7fc2880fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 -- 172.21.15.22:0/3497595738 shutdown_connections 2024-03-08T23:54:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.233+0000 7fc28d41f640 1 -- 172.21.15.22:0/3497595738 wait complete. 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 Processor -- start 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 -- start start 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc288102500 0x7fc288197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288102ec0 0x7fc28819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28810cfa0 con 0x7fc288101310 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28810ce20 con 0x7fc288102ec0 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc28d41f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28810d120 con 0x7fc288102500 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc286ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc286ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52750/0 (socket says 172.21.15.22:52750) 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc286ffd640 1 -- 172.21.15.22:0/2593744670 learned_addr learned my addr 172.21.15.22:0/2593744670 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc2867fc640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc288102500 0x7fc288197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.234+0000 7fc2877fe640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288102ec0 0x7fc28819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc286ffd640 1 -- 172.21.15.22:0/2593744670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc288102500 msgr2=0x7fc288197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc286ffd640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc288102500 0x7fc288197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc286ffd640 1 -- 172.21.15.22:0/2593744670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288102ec0 msgr2=0x7fc28819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc286ffd640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288102ec0 0x7fc28819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc286ffd640 1 -- 172.21.15.22:0/2593744670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28819f240 con 0x7fc288101310 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc286ffd640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288197620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc280000c00 tx=0x7fc280005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc26ffff640 1 -- 172.21.15.22:0/2593744670 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280041070 con 0x7fc288101310 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28819f4d0 con 0x7fc288101310 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc26ffff640 1 -- 172.21.15.22:0/2593744670 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2800044d0 con 0x7fc288101310 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc26ffff640 1 -- 172.21.15.22:0/2593744670 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280034440 con 0x7fc288101310 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.235+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28819fa10 con 0x7fc288101310 2024-03-08T23:54:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.236+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2881086b0 con 0x7fc288101310 2024-03-08T23:54:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.237+0000 7fc26ffff640 1 -- 172.21.15.22:0/2593744670 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc280044070 con 0x7fc288101310 2024-03-08T23:54:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.237+0000 7fc26ffff640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2700a2800 0x7fc2700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.238+0000 7fc2867fc640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2700a2800 0x7fc2700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.238+0000 7fc26ffff640 1 -- 172.21.15.22:0/2593744670 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc2800f0940 con 0x7fc288101310 2024-03-08T23:54:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.238+0000 7fc2867fc640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2700a2800 0x7fc2700a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc288198b30 tx=0x7fc27c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.242+0000 7fc26ffff640 1 -- 172.21.15.22:0/2593744670 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc2800bcfa0 con 0x7fc288101310 2024-03-08T23:54:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.425+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc288061ef0 con 0x7fc288101310 2024-03-08T23:54:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.427+0000 7fc26ffff640 1 -- 172.21.15.22:0/2593744670 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24654 (secure 0 0 0) 0x7fc2800c1040 con 0x7fc288101310 2024-03-08T23:54:18.423 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:18.424 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:45.618289+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:54:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2700a2800 msgr2=0x7fc2700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2700a2800 0x7fc2700a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc288198b30 tx=0x7fc27c00b390 comp rx=0 tx=0).stop 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 msgr2=0x7fc288197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288197620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc280000c00 tx=0x7fc280005d50 comp rx=0 tx=0).stop 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 shutdown_connections 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288102ec0 0x7fc28819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc288102500 0x7fc288197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc2700a2800 0x7fc2700a4cc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 --2- 172.21.15.22:0/2593744670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc288101310 0x7fc288197620 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 >> 172.21.15.22:0/2593744670 conn(0x7fc2880fc8a0 msgr2=0x7fc288105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 shutdown_connections 2024-03-08T23:54:18.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.429+0000 7fc28d41f640 1 -- 172.21.15.22:0/2593744670 wait complete. 2024-03-08T23:54:18.435 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:18.080+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:18.435 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:18.080+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:18.435 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:18.080+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:18.435 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:18.080+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:18.435 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:18.080+0000 7f7695906700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:18.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:54:18.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:54:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.546+0000 7f5476297640 1 -- 172.21.15.22:0/672646050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 msgr2=0x7f5470101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.546+0000 7f5476297640 1 --2- 172.21.15.22:0/672646050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f5470101860 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5464009900 tx=0x7f546402bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.546+0000 7f5476297640 1 -- 172.21.15.22:0/672646050 shutdown_connections 2024-03-08T23:54:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.546+0000 7f5476297640 1 --2- 172.21.15.22:0/672646050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5470101da0 0x7f5470108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.546+0000 7f5476297640 1 --2- 172.21.15.22:0/672646050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f5470101860 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.546+0000 7f5476297640 1 --2- 172.21.15.22:0/672646050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54701001f0 0x7f5470100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.546+0000 7f5476297640 1 -- 172.21.15.22:0/672646050 >> 172.21.15.22:0/672646050 conn(0x7f54700fb780 msgr2=0x7f54700fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 -- 172.21.15.22:0/672646050 shutdown_connections 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 -- 172.21.15.22:0/672646050 wait complete. 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 Processor -- start 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 -- start start 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54701001f0 0x7f5470068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f547019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5470101da0 0x7f547019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547010bed0 con 0x7f54701013e0 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547010bd50 con 0x7f5470101da0 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f5476297640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547010c050 con 0x7f54701001f0 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f547019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f546f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f547019c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52772/0 (socket says 172.21.15.22:52772) 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.547+0000 7f546f7fe640 1 -- 172.21.15.22:0/1717631513 learned_addr learned my addr 172.21.15.22:0/1717631513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546ffff640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54701001f0 0x7f5470068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f5474834640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5470101da0 0x7f547019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546f7fe640 1 -- 172.21.15.22:0/1717631513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54701001f0 msgr2=0x7f5470068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546f7fe640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54701001f0 0x7f5470068fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546f7fe640 1 -- 172.21.15.22:0/1717631513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5470101da0 msgr2=0x7f547019e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546f7fe640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5470101da0 0x7f547019e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546f7fe640 1 -- 172.21.15.22:0/1717631513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547019f280 con 0x7f54701013e0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546ffff640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54701001f0 0x7f5470068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546f7fe640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f547019c380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5464005ab0 tx=0x7f546402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546d7fa640 1 -- 172.21.15.22:0/1717631513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464043020 con 0x7f54701013e0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546d7fa640 1 -- 172.21.15.22:0/1717631513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54640054e0 con 0x7f54701013e0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f546d7fa640 1 -- 172.21.15.22:0/1717631513 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464003ce0 con 0x7f54701013e0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547019f510 con 0x7f54701013e0 2024-03-08T23:54:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.548+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547019fa60 con 0x7f54701013e0 2024-03-08T23:54:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.549+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5470107510 con 0x7f54701013e0 2024-03-08T23:54:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.550+0000 7f546d7fa640 1 -- 172.21.15.22:0/1717631513 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5464046020 con 0x7f54701013e0 2024-03-08T23:54:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.551+0000 7f546d7fa640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54580a2800 0x7f54580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.551+0000 7f546ffff640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54580a2800 0x7f54580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.551+0000 7f546d7fa640 1 -- 172.21.15.22:0/1717631513 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54640f1070 con 0x7f54701013e0 2024-03-08T23:54:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.551+0000 7f546ffff640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54580a2800 0x7f54580a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54600058c0 tx=0x7f5460005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.555+0000 7f546d7fa640 1 -- 172.21.15.22:0/1717631513 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f546404d020 con 0x7f54701013e0 2024-03-08T23:54:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.738+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f547002ce70 con 0x7f54701013e0 2024-03-08T23:54:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.739+0000 7f546d7fa640 1 -- 172.21.15.22:0/1717631513 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54640bdd20 con 0x7f54701013e0 2024-03-08T23:54:18.735 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:54:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54580a2800 msgr2=0x7f54580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54580a2800 0x7f54580a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54600058c0 tx=0x7f5460005850 comp rx=0 tx=0).stop 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 msgr2=0x7f547019c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f547019c380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5464005ab0 tx=0x7f546402c6c0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 shutdown_connections 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5470101da0 0x7f547019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f54580a2800 0x7f54580a4cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54701013e0 0x7f547019c380 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 --2- 172.21.15.22:0/1717631513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54701001f0 0x7f5470068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 >> 172.21.15.22:0/1717631513 conn(0x7f54700fb780 msgr2=0x7f54701046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 shutdown_connections 2024-03-08T23:54:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.741+0000 7f5476297640 1 -- 172.21.15.22:0/1717631513 wait complete. 2024-03-08T23:54:18.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:54:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 -- 172.21.15.22:0/3674033655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 msgr2=0x7ff8f0100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/3674033655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f0100500 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009900 tx=0x7ff8ec02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 -- 172.21.15.22:0/3674033655 shutdown_connections 2024-03-08T23:54:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/3674033655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f0101c90 0x7ff8f0108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/3674033655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f01012d0 0x7ff8f0101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/3674033655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f0100500 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 -- 172.21.15.22:0/3674033655 >> 172.21.15.22:0/3674033655 conn(0x7ff8f00fb670 msgr2=0x7ff8f00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 -- 172.21.15.22:0/3674033655 shutdown_connections 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 -- 172.21.15.22:0/3674033655 wait complete. 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 Processor -- start 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.860+0000 7ff8f69a2640 1 -- start start 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f01974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f01012d0 0x7ff8f01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f0101c90 0x7ff8f019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f010bd70 con 0x7ff8f01000e0 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f010bbf0 con 0x7ff8f0101c90 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f010bef0 con 0x7ff8f01012d0 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f01974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f01974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52790/0 (socket says 172.21.15.22:52790) 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 -- 172.21.15.22:0/804994098 learned_addr learned my addr 172.21.15.22:0/804994098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f61a1640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f0101c90 0x7ff8f019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 -- 172.21.15.22:0/804994098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f01012d0 msgr2=0x7ff8f01979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f01012d0 0x7ff8f01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 -- 172.21.15.22:0/804994098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f0101c90 msgr2=0x7ff8f019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f0101c90 0x7ff8f019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 -- 172.21.15.22:0/804994098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f019f0d0 con 0x7ff8f01000e0 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f59a0640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f01974b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec005ab0 tx=0x7ff8ec005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8e6ffd640 1 -- 172.21.15.22:0/804994098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec041070 con 0x7ff8f01000e0 2024-03-08T23:54:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8e6ffd640 1 -- 172.21.15.22:0/804994098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8ec0054e0 con 0x7ff8f01000e0 2024-03-08T23:54:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f019f360 con 0x7ff8f01000e0 2024-03-08T23:54:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8e6ffd640 1 -- 172.21.15.22:0/804994098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec034440 con 0x7ff8f01000e0 2024-03-08T23:54:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.861+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f019f850 con 0x7ff8f01000e0 2024-03-08T23:54:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.862+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f019fb70 con 0x7ff8f01000e0 2024-03-08T23:54:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.863+0000 7ff8e6ffd640 1 -- 172.21.15.22:0/804994098 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff8ec044070 con 0x7ff8f01000e0 2024-03-08T23:54:18.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.864+0000 7ff8e6ffd640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8dc0a2800 0x7ff8dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:18.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.864+0000 7ff8f519f640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8dc0a2800 0x7ff8dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:18.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.864+0000 7ff8e6ffd640 1 -- 172.21.15.22:0/804994098 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8ec0f0ae0 con 0x7ff8f01000e0 2024-03-08T23:54:18.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.864+0000 7ff8f519f640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8dc0a2800 0x7ff8dc0a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8f01989c0 tx=0x7ff8e0005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:18.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:18.868+0000 7ff8e6ffd640 1 -- 172.21.15.22:0/804994098 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff8ec0bd7f0 con 0x7ff8f01000e0 2024-03-08T23:54:19.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.051+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8f01a01c0 con 0x7ff8f01000e0 2024-03-08T23:54:19.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.051+0000 7ff8e6ffd640 1 -- 172.21.15.22:0/804994098 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8ec0bd0f0 con 0x7ff8f01000e0 2024-03-08T23:54:19.048 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:19.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8dc0a2800 msgr2=0x7ff8dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8dc0a2800 0x7ff8dc0a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8f01989c0 tx=0x7ff8e0005c70 comp rx=0 tx=0).stop 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 msgr2=0x7ff8f01974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f01974b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec005ab0 tx=0x7ff8ec005dc0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 shutdown_connections 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8f0101c90 0x7ff8f019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8f01012d0 0x7ff8f01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff8dc0a2800 0x7ff8dc0a4cc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 --2- 172.21.15.22:0/804994098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8f01000e0 0x7ff8f01974b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 >> 172.21.15.22:0/804994098 conn(0x7ff8f00fb670 msgr2=0x7ff8f00fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:19.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 shutdown_connections 2024-03-08T23:54:19.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.053+0000 7ff8f69a2640 1 -- 172.21.15.22:0/804994098 wait complete. 2024-03-08T23:54:19.059 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1708639192' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-08T23:54:19.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:54:19.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2593744670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:19.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1717631513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:19.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:54:19.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1708639192' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-08T23:54:19.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:18 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:54:19.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2593744670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:19.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1717631513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:19.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1708639192' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-08T23:54:19.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:18 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:54:19.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2593744670' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:19.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1717631513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 -- 172.21.15.22:0/4149751756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90100210 msgr2=0x7f4b90100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 --2- 172.21.15.22:0/4149751756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90100210 0x7f4b90100630 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009900 tx=0x7f4b8802bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 -- 172.21.15.22:0/4149751756 shutdown_connections 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 --2- 172.21.15.22:0/4149751756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b90101dc0 0x7f4b90108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 --2- 172.21.15.22:0/4149751756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b90101400 0x7f4b90101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 --2- 172.21.15.22:0/4149751756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90100210 0x7f4b90100630 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 -- 172.21.15.22:0/4149751756 >> 172.21.15.22:0/4149751756 conn(0x7f4b900fb7a0 msgr2=0x7f4b900fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 -- 172.21.15.22:0/4149751756 shutdown_connections 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.171+0000 7f4b971c6640 1 -- 172.21.15.22:0/4149751756 wait complete. 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 Processor -- start 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 -- start start 2024-03-08T23:54:19.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b90100210 0x7f4b90196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90101400 0x7f4b901969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b90101dc0 0x7f4b9019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9010be70 con 0x7f4b90101400 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9010bcf0 con 0x7f4b90101dc0 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b971c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9010bff0 con 0x7f4b90100210 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b8ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90101400 0x7f4b901969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b8ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90101400 0x7f4b901969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52802/0 (socket says 172.21.15.22:52802) 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b8ffff640 1 -- 172.21.15.22:0/1845438299 learned_addr learned my addr 172.21.15.22:0/1845438299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b95763640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b90101dc0 0x7f4b9019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.172+0000 7f4b94f62640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b90100210 0x7f4b90196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8ffff640 1 -- 172.21.15.22:0/1845438299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b90100210 msgr2=0x7f4b90196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8ffff640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b90100210 0x7f4b90196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8ffff640 1 -- 172.21.15.22:0/1845438299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b90101dc0 msgr2=0x7f4b9019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8ffff640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b90101dc0 0x7f4b9019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8ffff640 1 -- 172.21.15.22:0/1845438299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9019e0a0 con 0x7f4b90101400 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b94f62640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b90100210 0x7f4b90196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8ffff640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90101400 0x7f4b901969c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000ebc0 tx=0x7f4b8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8dffb640 1 -- 172.21.15.22:0/1845438299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80010630 con 0x7f4b90101400 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8dffb640 1 -- 172.21.15.22:0/1845438299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b80010cb0 con 0x7f4b90101400 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9019e390 con 0x7f4b90101400 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b8dffb640 1 -- 172.21.15.22:0/1845438299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80018bd0 con 0x7f4b90101400 2024-03-08T23:54:19.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.173+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9019e930 con 0x7f4b90101400 2024-03-08T23:54:19.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.174+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b90107530 con 0x7f4b90101400 2024-03-08T23:54:19.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.175+0000 7f4b8dffb640 1 -- 172.21.15.22:0/1845438299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4b800107d0 con 0x7f4b90101400 2024-03-08T23:54:19.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.176+0000 7f4b8dffb640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4b7c0a2880 0x7f4b7c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.176+0000 7f4b94f62640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4b7c0a2880 0x7f4b7c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.176+0000 7f4b8dffb640 1 -- 172.21.15.22:0/1845438299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b800cf460 con 0x7f4b90101400 2024-03-08T23:54:19.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.176+0000 7f4b94f62640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4b7c0a2880 0x7f4b7c0a4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b88036040 tx=0x7f4b88000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:19.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.180+0000 7f4b8dffb640 1 -- 172.21.15.22:0/1845438299 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4b8009c110 con 0x7f4b90101400 2024-03-08T23:54:19.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.365+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b90061ef0 con 0x7f4b90101400 2024-03-08T23:54:19.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.365+0000 7f4b8dffb640 1 -- 172.21.15.22:0/1845438299 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b8009ba10 con 0x7f4b90101400 2024-03-08T23:54:19.362 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4b7c0a2880 msgr2=0x7f4b7c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4b7c0a2880 0x7f4b7c0a4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b88036040 tx=0x7f4b88000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90101400 msgr2=0x7f4b901969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90101400 0x7f4b901969c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000ebc0 tx=0x7f4b8000c6a0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 shutdown_connections 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b90101dc0 0x7f4b9019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f4b7c0a2880 0x7f4b7c0a4d40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4b90101400 0x7f4b901969c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 --2- 172.21.15.22:0/1845438299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b90100210 0x7f4b90196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 >> 172.21.15.22:0/1845438299 conn(0x7f4b900fb7a0 msgr2=0x7f4b90104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 shutdown_connections 2024-03-08T23:54:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.367+0000 7f4b971c6640 1 -- 172.21.15.22:0/1845438299 wait complete. 2024-03-08T23:54:19.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 -- 172.21.15.22:0/1509045183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 msgr2=0x7f9020109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 --2- 172.21.15.22:0/1509045183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f9020109740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9018009900 tx=0x7f901802bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 -- 172.21.15.22:0/1509045183 shutdown_connections 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 --2- 172.21.15.22:0/1509045183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f9020109740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 --2- 172.21.15.22:0/1509045183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90201024f0 0x7f9020102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 --2- 172.21.15.22:0/1509045183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101300 0x7f9020101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 -- 172.21.15.22:0/1509045183 >> 172.21.15.22:0/1509045183 conn(0x7f90200fc870 msgr2=0x7f90200fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 -- 172.21.15.22:0/1509045183 shutdown_connections 2024-03-08T23:54:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.483+0000 7f9025c8e640 1 -- 172.21.15.22:0/1509045183 wait complete. 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 Processor -- start 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 -- start start 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101300 0x7f90201997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90201024f0 0x7f9020199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f90201a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902010cf60 con 0x7f9020102eb0 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902010cde0 con 0x7f90201024f0 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f9025c8e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902010d0e0 con 0x7f9020101300 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f901ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f90201a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f901ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f90201a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52826/0 (socket says 172.21.15.22:52826) 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f901ffff640 1 -- 172.21.15.22:0/2472880375 learned_addr learned my addr 172.21.15.22:0/2472880375 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.484+0000 7f901f7fe640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101300 0x7f90201997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901effd640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90201024f0 0x7f9020199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901ffff640 1 -- 172.21.15.22:0/2472880375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101300 msgr2=0x7f90201997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901ffff640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101300 0x7f90201997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901ffff640 1 -- 172.21.15.22:0/2472880375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90201024f0 msgr2=0x7f9020199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901ffff640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90201024f0 0x7f9020199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901ffff640 1 -- 172.21.15.22:0/2472880375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90201a1410 con 0x7f9020102eb0 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901ffff640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f90201a0da0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f901802c7b0 tx=0x7f9018005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901cff9640 1 -- 172.21.15.22:0/2472880375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018041070 con 0x7f9020102eb0 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901cff9640 1 -- 172.21.15.22:0/2472880375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90180054e0 con 0x7f9020102eb0 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90201a16a0 con 0x7f9020102eb0 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f901cff9640 1 -- 172.21.15.22:0/2472880375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018034440 con 0x7f9020102eb0 2024-03-08T23:54:19.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.485+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90201a1b90 con 0x7f9020102eb0 2024-03-08T23:54:19.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.486+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90201a1df0 con 0x7f9020102eb0 2024-03-08T23:54:19.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.487+0000 7f901cff9640 1 -- 172.21.15.22:0/2472880375 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9018044070 con 0x7f9020102eb0 2024-03-08T23:54:19.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.488+0000 7f901cff9640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f900c0a2800 0x7f900c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.488+0000 7f901f7fe640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f900c0a2800 0x7f900c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.488+0000 7f901cff9640 1 -- 172.21.15.22:0/2472880375 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f90180f0f50 con 0x7f9020102eb0 2024-03-08T23:54:19.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.488+0000 7f901f7fe640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f900c0a2800 0x7f900c0a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9020102350 tx=0x7f90100098f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:19.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.491+0000 7f901cff9640 1 -- 172.21.15.22:0/2472880375 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f90180bdc60 con 0x7f9020102eb0 2024-03-08T23:54:19.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.671+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f90201a2180 con 0x7f9020102eb0 2024-03-08T23:54:19.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.672+0000 7f901cff9640 1 -- 172.21.15.22:0/2472880375 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f90180bd560 con 0x7f9020102eb0 2024-03-08T23:54:19.668 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:19.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f900c0a2800 msgr2=0x7f900c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f900c0a2800 0x7f900c0a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9020102350 tx=0x7f90100098f0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 msgr2=0x7f90201a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f90201a0da0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f901802c7b0 tx=0x7f9018005dd0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 shutdown_connections 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f900c0a2800 0x7f900c0a4cc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9020102eb0 0x7f90201a0da0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90201024f0 0x7f9020199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 --2- 172.21.15.22:0/2472880375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020101300 0x7f90201997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 >> 172.21.15.22:0/2472880375 conn(0x7f90200fc870 msgr2=0x7f9020073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 shutdown_connections 2024-03-08T23:54:19.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.674+0000 7f9025c8e640 1 -- 172.21.15.22:0/2472880375 wait complete. 2024-03-08T23:54:19.681 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:54:19.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 -- 172.21.15.22:0/3804736971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c0ff9b0 msgr2=0x7fc50c0ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 --2- 172.21.15.22:0/3804736971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c0ff9b0 0x7fc50c0ffdd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc508009900 tx=0x7fc50802bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:19.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 -- 172.21.15.22:0/3804736971 shutdown_connections 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 --2- 172.21.15.22:0/3804736971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c101560 0x7fc50c107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 --2- 172.21.15.22:0/3804736971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc50c100ba0 0x7fc50c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 --2- 172.21.15.22:0/3804736971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c0ff9b0 0x7fc50c0ffdd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 -- 172.21.15.22:0/3804736971 >> 172.21.15.22:0/3804736971 conn(0x7fc50c0faf20 msgr2=0x7fc50c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 -- 172.21.15.22:0/3804736971 shutdown_connections 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 -- 172.21.15.22:0/3804736971 wait complete. 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 Processor -- start 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.792+0000 7fc512c81640 1 -- start start 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc512c81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc50c0ff9b0 0x7fc50c0fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc512c81640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c100ba0 0x7fc50c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc512c81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c101560 0x7fc50c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc512c81640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c10b700 con 0x7fc50c101560 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc512c81640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c10b580 con 0x7fc50c0ff9b0 2024-03-08T23:54:19.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc512c81640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c10b880 con 0x7fc50c100ba0 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c101560 0x7fc50c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c101560 0x7fc50c19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52854/0 (socket says 172.21.15.22:52854) 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 -- 172.21.15.22:0/1659029752 learned_addr learned my addr 172.21.15.22:0/1659029752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc503fff640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c100ba0 0x7fc50c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc510a1d640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc50c0ff9b0 0x7fc50c0fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 -- 172.21.15.22:0/1659029752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c100ba0 msgr2=0x7fc50c19c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c100ba0 0x7fc50c19c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 -- 172.21.15.22:0/1659029752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc50c0ff9b0 msgr2=0x7fc50c0fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc50c0ff9b0 0x7fc50c0fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 -- 172.21.15.22:0/1659029752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c19f3c0 con 0x7fc50c101560 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc51121e640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c101560 0x7fc50c19eab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b970 tx=0x7fc50400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc501ffb640 1 -- 172.21.15.22:0/1659029752 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504019070 con 0x7fc50c101560 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc501ffb640 1 -- 172.21.15.22:0/1659029752 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc504014440 con 0x7fc50c101560 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c19f650 con 0x7fc50c101560 2024-03-08T23:54:19.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.793+0000 7fc501ffb640 1 -- 172.21.15.22:0/1659029752 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5040075d0 con 0x7fc50c101560 2024-03-08T23:54:19.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.794+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c19fb50 con 0x7fc50c101560 2024-03-08T23:54:19.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.795+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c107100 con 0x7fc50c101560 2024-03-08T23:54:19.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.796+0000 7fc501ffb640 1 -- 172.21.15.22:0/1659029752 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fc50401c070 con 0x7fc50c101560 2024-03-08T23:54:19.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.796+0000 7fc501ffb640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc4f40a2890 0x7fc4f40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:19.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.796+0000 7fc510a1d640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc4f40a2890 0x7fc4f40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:19.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.796+0000 7fc501ffb640 1 -- 172.21.15.22:0/1659029752 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc5040ce330 con 0x7fc50c101560 2024-03-08T23:54:19.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.797+0000 7fc510a1d640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc4f40a2890 0x7fc4f40a4d50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc508000c00 tx=0x7fc508004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:19.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.801+0000 7fc501ffb640 1 -- 172.21.15.22:0/1659029752 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc50409aa40 con 0x7fc50c101560 2024-03-08T23:54:19.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.984+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc50c061ef0 con 0x7fc50c101560 2024-03-08T23:54:19.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.986+0000 7fc501ffb640 1 -- 172.21.15.22:0/1659029752 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24654 (secure 0 0 0) 0x7fc50409eae0 con 0x7fc50c101560 2024-03-08T23:54:19.982 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:19.982 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:45.618289+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:54:19.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc4f40a2890 msgr2=0x7fc4f40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc4f40a2890 0x7fc4f40a4d50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc508000c00 tx=0x7fc508004f10 comp rx=0 tx=0).stop 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c101560 msgr2=0x7fc50c19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c101560 0x7fc50c19eab0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b970 tx=0x7fc50400bd70 comp rx=0 tx=0).stop 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 shutdown_connections 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc4f40a2890 0x7fc4f40a4d50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50c101560 0x7fc50c19eab0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc50c100ba0 0x7fc50c19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 --2- 172.21.15.22:0/1659029752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc50c0ff9b0 0x7fc50c0fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 >> 172.21.15.22:0/1659029752 conn(0x7fc50c0faf20 msgr2=0x7fc50c1042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 shutdown_connections 2024-03-08T23:54:19.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:19.988+0000 7fc512c81640 1 -- 172.21.15.22:0/1659029752 wait complete. 2024-03-08T23:54:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:19 smithi022 ceph-mon[31894]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:19 smithi022 ceph-mon[31894]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/804994098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1845438299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:19.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:19 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2472880375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:19.997 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:54:19.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 -- 172.21.15.22:0/1824577042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 msgr2=0x7ff4d4100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 --2- 172.21.15.22:0/1824577042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d4100610 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009900 tx=0x7ff4bc02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 -- 172.21.15.22:0/1824577042 shutdown_connections 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 --2- 172.21.15.22:0/1824577042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4d4101da0 0x7ff4d4108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 --2- 172.21.15.22:0/1824577042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d41013e0 0x7ff4d4101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 --2- 172.21.15.22:0/1824577042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d4100610 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 -- 172.21.15.22:0/1824577042 >> 172.21.15.22:0/1824577042 conn(0x7ff4d40fb780 msgr2=0x7ff4d40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 -- 172.21.15.22:0/1824577042 shutdown_connections 2024-03-08T23:54:20.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.107+0000 7ff4da569640 1 -- 172.21.15.22:0/1824577042 wait complete. 2024-03-08T23:54:20.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 Processor -- start 2024-03-08T23:54:20.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 -- start start 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d41976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4d41013e0 0x7ff4d4197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101da0 0x7ff4d419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410be80 con 0x7ff4d41001f0 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410bd00 con 0x7ff4d41013e0 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4da569640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410c000 con 0x7ff4d4101da0 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d41976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d41976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52866/0 (socket says 172.21.15.22:52866) 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.108+0000 7ff4d3fff640 1 -- 172.21.15.22:0/384098995 learned_addr learned my addr 172.21.15.22:0/384098995 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d8b06640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101da0 0x7ff4d419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d37fe640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4d41013e0 0x7ff4d4197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d3fff640 1 -- 172.21.15.22:0/384098995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101da0 msgr2=0x7ff4d419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d3fff640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101da0 0x7ff4d419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d3fff640 1 -- 172.21.15.22:0/384098995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4d41013e0 msgr2=0x7ff4d4197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d3fff640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4d41013e0 0x7ff4d4197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d3fff640 1 -- 172.21.15.22:0/384098995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d419f290 con 0x7ff4d41001f0 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d3fff640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d41976c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc005ab0 tx=0x7ff4bc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d17fa640 1 -- 172.21.15.22:0/384098995 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc041070 con 0x7ff4d41001f0 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d17fa640 1 -- 172.21.15.22:0/384098995 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4bc0054e0 con 0x7ff4d41001f0 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d419f520 con 0x7ff4d41001f0 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4d17fa640 1 -- 172.21.15.22:0/384098995 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc034440 con 0x7ff4d41001f0 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.109+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d419fa10 con 0x7ff4d41001f0 2024-03-08T23:54:20.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.110+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d419fc70 con 0x7ff4d41001f0 2024-03-08T23:54:20.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.111+0000 7ff4d17fa640 1 -- 172.21.15.22:0/384098995 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff4bc044070 con 0x7ff4d41001f0 2024-03-08T23:54:20.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.112+0000 7ff4d17fa640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff4c00a2890 0x7ff4c00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.112+0000 7ff4d17fa640 1 -- 172.21.15.22:0/384098995 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff4bc0f0cc0 con 0x7ff4d41001f0 2024-03-08T23:54:20.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.112+0000 7ff4d37fe640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff4c00a2890 0x7ff4c00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.112+0000 7ff4d37fe640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff4c00a2890 0x7ff4c00a4d50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005fd0 tx=0x7ff4c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:20.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:19 smithi062 ceph-mon[36129]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:20.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:19 smithi062 ceph-mon[36129]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:20.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/804994098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1845438299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:19 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2472880375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.115+0000 7ff4d17fa640 1 -- 172.21.15.22:0/384098995 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff4bc0bd920 con 0x7ff4d41001f0 2024-03-08T23:54:20.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:19 smithi192 ceph-mon[35995]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-08T23:54:20.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:19 smithi192 ceph-mon[35995]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:20.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/804994098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1845438299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:19 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2472880375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.302+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4d419fe40 con 0x7ff4d41001f0 2024-03-08T23:54:20.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.302+0000 7ff4d17fa640 1 -- 172.21.15.22:0/384098995 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4bc0bd220 con 0x7ff4d41001f0 2024-03-08T23:54:20.299 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:54:20.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff4c00a2890 msgr2=0x7ff4c00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff4c00a2890 0x7ff4c00a4d50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005fd0 tx=0x7ff4c4005850 comp rx=0 tx=0).stop 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 msgr2=0x7ff4d41976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d41976c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc005ab0 tx=0x7ff4bc005dc0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 shutdown_connections 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101da0 0x7ff4d419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4d41013e0 0x7ff4d4197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff4c00a2890 0x7ff4c00a4d50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 --2- 172.21.15.22:0/384098995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41001f0 0x7ff4d41976c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 >> 172.21.15.22:0/384098995 conn(0x7ff4d40fb780 msgr2=0x7ff4d40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 shutdown_connections 2024-03-08T23:54:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.304+0000 7ff4da569640 1 -- 172.21.15.22:0/384098995 wait complete. 2024-03-08T23:54:20.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:54:20.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.421+0000 7f138b316640 1 -- 172.21.15.22:0/2194301111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13841001f0 msgr2=0x7f1384100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.421+0000 7f138b316640 1 --2- 172.21.15.22:0/2194301111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13841001f0 0x7f1384100610 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1380009900 tx=0x7f138002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:20.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- 172.21.15.22:0/2194301111 shutdown_connections 2024-03-08T23:54:20.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 --2- 172.21.15.22:0/2194301111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1384101da0 0x7f1384108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 --2- 172.21.15.22:0/2194301111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841013e0 0x7f1384101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 --2- 172.21.15.22:0/2194301111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13841001f0 0x7f1384100610 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- 172.21.15.22:0/2194301111 >> 172.21.15.22:0/2194301111 conn(0x7f13840fb780 msgr2=0x7f13840fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- 172.21.15.22:0/2194301111 shutdown_connections 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- 172.21.15.22:0/2194301111 wait complete. 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 Processor -- start 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- start start 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841001f0 0x7f1384104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13841013e0 0x7f13841027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384101da0 0x7f1384102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138410be80 con 0x7f1384101da0 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138410bd00 con 0x7f13841001f0 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.422+0000 7f138b316640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138410c000 con 0x7f13841013e0 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384101da0 0x7f1384102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384101da0 0x7f1384102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52896/0 (socket says 172.21.15.22:52896) 2024-03-08T23:54:20.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 -- 172.21.15.22:0/3561420867 learned_addr learned my addr 172.21.15.22:0/3561420867 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13890b2640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841001f0 0x7f1384104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13888b1640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13841013e0 0x7f13841027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 -- 172.21.15.22:0/3561420867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13841013e0 msgr2=0x7f13841027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13841013e0 0x7f13841027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 -- 172.21.15.22:0/3561420867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841001f0 msgr2=0x7f1384104180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841001f0 0x7f1384104180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 -- 172.21.15.22:0/3561420867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384103590 con 0x7f1384101da0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13890b2640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841001f0 0x7f1384104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f13898b3640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384101da0 0x7f1384102d00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f137c00eaa0 tx=0x7f137c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f137a7fc640 1 -- 172.21.15.22:0/3561420867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00cdb0 con 0x7f1384101da0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13841a17b0 con 0x7f1384101da0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f137a7fc640 1 -- 172.21.15.22:0/3561420867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f137c005020 con 0x7f1384101da0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f137a7fc640 1 -- 172.21.15.22:0/3561420867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0116d0 con 0x7f1384101da0 2024-03-08T23:54:20.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.423+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13841a1d00 con 0x7f1384101da0 2024-03-08T23:54:20.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.424+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13841a1fc0 con 0x7f1384101da0 2024-03-08T23:54:20.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.425+0000 7f137a7fc640 1 -- 172.21.15.22:0/3561420867 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f137c0054d0 con 0x7f1384101da0 2024-03-08T23:54:20.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.426+0000 7f137a7fc640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13700a2890 0x7f13700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.426+0000 7f13890b2640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13700a2890 0x7f13700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.426+0000 7f137a7fc640 1 -- 172.21.15.22:0/3561420867 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f137c0cf2e0 con 0x7f1384101da0 2024-03-08T23:54:20.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.426+0000 7f13890b2640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13700a2890 0x7f13700a4d50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1380036040 tx=0x7f1380000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:20.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.430+0000 7f137a7fc640 1 -- 172.21.15.22:0/3561420867 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f137c016030 con 0x7f1384101da0 2024-03-08T23:54:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.611+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13841a2190 con 0x7f1384101da0 2024-03-08T23:54:20.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.612+0000 7f137a7fc640 1 -- 172.21.15.22:0/3561420867 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f137c09be90 con 0x7f1384101da0 2024-03-08T23:54:20.608 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:20.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13700a2890 msgr2=0x7f13700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13700a2890 0x7f13700a4d50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1380036040 tx=0x7f1380000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384101da0 msgr2=0x7f1384102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384101da0 0x7f1384102d00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f137c00eaa0 tx=0x7f137c00ef60 comp rx=0 tx=0).stop 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 shutdown_connections 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f13700a2890 0x7f13700a4d50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1384101da0 0x7f1384102d00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13841013e0 0x7f13841027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 --2- 172.21.15.22:0/3561420867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841001f0 0x7f1384104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 >> 172.21.15.22:0/3561420867 conn(0x7f13840fb780 msgr2=0x7f13840684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 shutdown_connections 2024-03-08T23:54:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.614+0000 7f138b316640 1 -- 172.21.15.22:0/3561420867 wait complete. 2024-03-08T23:54:20.621 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:54:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 -- 172.21.15.22:0/2412608578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 msgr2=0x7fdde4101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/2412608578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4101c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009900 tx=0x7fddd802bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 -- 172.21.15.22:0/2412608578 shutdown_connections 2024-03-08T23:54:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/2412608578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde4100e30 0x7fdde41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/2412608578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde4100470 0x7fdde41008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/2412608578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4101c40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 -- 172.21.15.22:0/2412608578 >> 172.21.15.22:0/2412608578 conn(0x7fdde40faf20 msgr2=0x7fdde40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 -- 172.21.15.22:0/2412608578 shutdown_connections 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.731+0000 7fdde9a8c640 1 -- 172.21.15.22:0/2412608578 wait complete. 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 Processor -- start 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 -- start start 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde4100470 0x7fdde4104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde4100e30 0x7fdde4102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde410bf50 con 0x7fdde4101820 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde410bdd0 con 0x7fdde4100470 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde9a8c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde410c0d0 con 0x7fdde4100e30 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51838/0 (socket says 172.21.15.22:51838) 2024-03-08T23:54:20.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 -- 172.21.15.22:0/3529120916 learned_addr learned my addr 172.21.15.22:0/3529120916 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde37fe640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde4100470 0x7fdde4104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde2ffd640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde4100e30 0x7fdde4102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 -- 172.21.15.22:0/3529120916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde4100e30 msgr2=0x7fdde4102860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde4100e30 0x7fdde4102860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 -- 172.21.15.22:0/3529120916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde4100470 msgr2=0x7fdde4104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde4100470 0x7fdde4104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.732+0000 7fdde3fff640 1 -- 172.21.15.22:0/3529120916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4103630 con 0x7fdde4101820 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.733+0000 7fdde2ffd640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde4100e30 0x7fdde4102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.733+0000 7fdde3fff640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4102da0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00eb10 tx=0x7fdddc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.733+0000 7fdde0ff9640 1 -- 172.21.15.22:0/3529120916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc004b20 con 0x7fdde4101820 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.733+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde41a17b0 con 0x7fdde4101820 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.733+0000 7fdde0ff9640 1 -- 172.21.15.22:0/3529120916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdddc004cc0 con 0x7fdde4101820 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.733+0000 7fdde0ff9640 1 -- 172.21.15.22:0/3529120916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc010760 con 0x7fdde4101820 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.733+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde41a1d50 con 0x7fdde4101820 2024-03-08T23:54:20.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.734+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde41a2010 con 0x7fdde4101820 2024-03-08T23:54:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.735+0000 7fdde0ff9640 1 -- 172.21.15.22:0/3529120916 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fdddc005020 con 0x7fdde4101820 2024-03-08T23:54:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.736+0000 7fdde0ff9640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fddcc0a2890 0x7fddcc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.736+0000 7fdde37fe640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fddcc0a2890 0x7fddcc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.736+0000 7fdde0ff9640 1 -- 172.21.15.22:0/3529120916 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdddc0cf6e0 con 0x7fdde4101820 2024-03-08T23:54:20.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.736+0000 7fdde37fe640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fddcc0a2890 0x7fddcc0a4d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddd802c170 tx=0x7fddd802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:20.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.740+0000 7fdde0ff9640 1 -- 172.21.15.22:0/3529120916 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdddc09c290 con 0x7fdde4101820 2024-03-08T23:54:20.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.923+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdde41a23a0 con 0x7fdde4101820 2024-03-08T23:54:20.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.923+0000 7fdde0ff9640 1 -- 172.21.15.22:0/3529120916 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdddc09bb90 con 0x7fdde4101820 2024-03-08T23:54:20.920 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fddcc0a2890 msgr2=0x7fddcc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fddcc0a2890 0x7fddcc0a4d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddd802c170 tx=0x7fddd802df90 comp rx=0 tx=0).stop 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 msgr2=0x7fdde4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4102da0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00eb10 tx=0x7fdddc00c2d0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 shutdown_connections 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fddcc0a2890 0x7fddcc0a4d50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde4101820 0x7fdde4102da0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde4100e30 0x7fdde4102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 --2- 172.21.15.22:0/3529120916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde4100470 0x7fdde4104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:20.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.925+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 >> 172.21.15.22:0/3529120916 conn(0x7fdde40faf20 msgr2=0x7fdde40fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:20.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.926+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 shutdown_connections 2024-03-08T23:54:20.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:20.926+0000 7fdde9a8c640 1 -- 172.21.15.22:0/3529120916 wait complete. 2024-03-08T23:54:20.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:20 smithi022 ceph-mon[31894]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-08T23:54:20.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:20 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1659029752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:20.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:20 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/384098995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:20 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3561420867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:20.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:54:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 -- 172.21.15.22:0/3994519592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102ec0 msgr2=0x7f27c0109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 --2- 172.21.15.22:0/3994519592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102ec0 0x7f27c0109750 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009ca0 tx=0x7f27b802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 -- 172.21.15.22:0/3994519592 shutdown_connections 2024-03-08T23:54:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 --2- 172.21.15.22:0/3994519592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102ec0 0x7f27c0109750 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 --2- 172.21.15.22:0/3994519592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c0102500 0x7f27c0102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 --2- 172.21.15.22:0/3994519592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c0101310 0x7f27c0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 -- 172.21.15.22:0/3994519592 >> 172.21.15.22:0/3994519592 conn(0x7f27c00fc860 msgr2=0x7f27c00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 -- 172.21.15.22:0/3994519592 shutdown_connections 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 -- 172.21.15.22:0/3994519592 wait complete. 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 Processor -- start 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.043+0000 7f27c6522640 1 -- start start 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c0101310 0x7f27c0197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102500 0x7f27c0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c0102ec0 0x7f27c019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c010d000 con 0x7f27c0102500 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c010ce80 con 0x7f27c0102ec0 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c010d180 con 0x7f27c0101310 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102500 0x7f27c0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102500 0x7f27c0197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51856/0 (socket says 172.21.15.22:51856) 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 -- 172.21.15.22:0/2181573082 learned_addr learned my addr 172.21.15.22:0/2181573082 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bffff640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c0101310 0x7f27c0197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c4abf640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c0102ec0 0x7f27c019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 -- 172.21.15.22:0/2181573082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c0101310 msgr2=0x7f27c0197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c0101310 0x7f27c0197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 -- 172.21.15.22:0/2181573082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c0102ec0 msgr2=0x7f27c019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c0102ec0 0x7f27c019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 -- 172.21.15.22:0/2181573082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c019f230 con 0x7f27c0102500 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bf7fe640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102500 0x7f27c0197b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b4a0 tx=0x7f27b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bd7fa640 1 -- 172.21.15.22:0/2181573082 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4014070 con 0x7f27c0102500 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c019f520 con 0x7f27c0102500 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bd7fa640 1 -- 172.21.15.22:0/2181573082 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27b4010af0 con 0x7f27c0102500 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c019fa70 con 0x7f27c0102500 2024-03-08T23:54:21.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.044+0000 7f27bd7fa640 1 -- 172.21.15.22:0/2181573082 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b401e060 con 0x7f27c0102500 2024-03-08T23:54:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.045+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c0108630 con 0x7f27c0102500 2024-03-08T23:54:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.046+0000 7f27bd7fa640 1 -- 172.21.15.22:0/2181573082 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f27b4017070 con 0x7f27c0102500 2024-03-08T23:54:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.047+0000 7f27bd7fa640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27a80a2890 0x7f27a80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.047+0000 7f27bd7fa640 1 -- 172.21.15.22:0/2181573082 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f27b40ce6e0 con 0x7f27c0102500 2024-03-08T23:54:21.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.047+0000 7f27bffff640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27a80a2890 0x7f27a80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.047+0000 7f27bffff640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27a80a2890 0x7f27a80a4d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005fd0 tx=0x7f27b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.050+0000 7f27bd7fa640 1 -- 172.21.15.22:0/2181573082 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f27b409b3a0 con 0x7f27c0102500 2024-03-08T23:54:21.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:20 smithi062 ceph-mon[36129]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-08T23:54:21.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:20 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1659029752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:21.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:20 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/384098995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:21.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:20 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3561420867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:21.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:20 smithi192 ceph-mon[35995]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-08T23:54:21.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:20 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1659029752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:21.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:20 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/384098995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:21.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:20 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3561420867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:21.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.232+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f27c002d0e0 con 0x7f27c0102500 2024-03-08T23:54:21.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.232+0000 7f27bd7fa640 1 -- 172.21.15.22:0/2181573082 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f27b409aca0 con 0x7f27c0102500 2024-03-08T23:54:21.229 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27a80a2890 msgr2=0x7f27a80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27a80a2890 0x7f27a80a4d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005fd0 tx=0x7f27b000b390 comp rx=0 tx=0).stop 2024-03-08T23:54:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102500 msgr2=0x7f27c0197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102500 0x7f27c0197b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b4a0 tx=0x7f27b400b960 comp rx=0 tx=0).stop 2024-03-08T23:54:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 shutdown_connections 2024-03-08T23:54:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c0102ec0 0x7f27c019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f27a80a2890 0x7f27a80a4d50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27c0102500 0x7f27c0197b50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 --2- 172.21.15.22:0/2181573082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c0101310 0x7f27c0197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 >> 172.21.15.22:0/2181573082 conn(0x7f27c00fc860 msgr2=0x7f27c0105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 shutdown_connections 2024-03-08T23:54:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.235+0000 7f27c6522640 1 -- 172.21.15.22:0/2181573082 wait complete. 2024-03-08T23:54:21.242 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-08T23:54:21.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:54:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 -- 172.21.15.22:0/183524373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258100210 msgr2=0x7f5258100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 --2- 172.21.15.22:0/183524373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258100210 0x7f5258100630 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f524c009900 tx=0x7f524c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 -- 172.21.15.22:0/183524373 shutdown_connections 2024-03-08T23:54:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 --2- 172.21.15.22:0/183524373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5258101dc0 0x7f5258108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 --2- 172.21.15.22:0/183524373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5258101400 0x7f5258101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 --2- 172.21.15.22:0/183524373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258100210 0x7f5258100630 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 -- 172.21.15.22:0/183524373 >> 172.21.15.22:0/183524373 conn(0x7f52580fb780 msgr2=0x7f52580fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 -- 172.21.15.22:0/183524373 shutdown_connections 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.348+0000 7f525edb2640 1 -- 172.21.15.22:0/183524373 wait complete. 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 Processor -- start 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 -- start start 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5258100210 0x7f5258068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258101400 0x7f525819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5258101dc0 0x7f525819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525810bea0 con 0x7f5258101400 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525810bd20 con 0x7f5258100210 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525edb2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525810c020 con 0x7f5258101dc0 2024-03-08T23:54:21.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258101400 0x7f525819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258101400 0x7f525819c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51874/0 (socket says 172.21.15.22:51874) 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 -- 172.21.15.22:0/934699217 learned_addr learned my addr 172.21.15.22:0/934699217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525cb4e640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5258100210 0x7f5258068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f525d34f640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5258101dc0 0x7f525819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 -- 172.21.15.22:0/934699217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5258101dc0 msgr2=0x7f525819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5258101dc0 0x7f525819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 -- 172.21.15.22:0/934699217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5258100210 msgr2=0x7f5258068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5258100210 0x7f5258068fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 -- 172.21.15.22:0/934699217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525819f260 con 0x7f5258101400 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5257fff640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258101400 0x7f525819c360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f524400c890 tx=0x7f524400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5255ffb640 1 -- 172.21.15.22:0/934699217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244004220 con 0x7f5258101400 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5255ffb640 1 -- 172.21.15.22:0/934699217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52440043c0 con 0x7f5258101400 2024-03-08T23:54:21.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.349+0000 7f5255ffb640 1 -- 172.21.15.22:0/934699217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440049e0 con 0x7f5258101400 2024-03-08T23:54:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.350+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525819f550 con 0x7f5258101400 2024-03-08T23:54:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.350+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525819faf0 con 0x7f5258101400 2024-03-08T23:54:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.351+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5258107530 con 0x7f5258101400 2024-03-08T23:54:21.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.352+0000 7f5255ffb640 1 -- 172.21.15.22:0/934699217 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f52440079b0 con 0x7f5258101400 2024-03-08T23:54:21.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.352+0000 7f5255ffb640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52480a2910 0x7f52480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.352+0000 7f5255ffb640 1 -- 172.21.15.22:0/934699217 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f52440cefa0 con 0x7f5258101400 2024-03-08T23:54:21.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.352+0000 7f525cb4e640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52480a2910 0x7f52480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.353+0000 7f525cb4e640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52480a2910 0x7f52480a4dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f524c000c00 tx=0x7f524c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:21.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.356+0000 7f5255ffb640 1 -- 172.21.15.22:0/934699217 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5244014030 con 0x7f5258101400 2024-03-08T23:54:21.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.580+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f525802d020 con 0x7f5258101400 2024-03-08T23:54:21.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.581+0000 7f5255ffb640 1 -- 172.21.15.22:0/934699217 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f524409bb50 con 0x7f5258101400 2024-03-08T23:54:21.577 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:21.578 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:54:17.470857+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:54:21.578 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-03-08T23:54:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.583+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52480a2910 msgr2=0x7f52480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52480a2910 0x7f52480a4dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f524c000c00 tx=0x7f524c005fd0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258101400 msgr2=0x7f525819c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258101400 0x7f525819c360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f524400c890 tx=0x7f524400cd50 comp rx=0 tx=0).stop 2024-03-08T23:54:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 shutdown_connections 2024-03-08T23:54:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5258101dc0 0x7f525819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f52480a2910 0x7f52480a4dd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5258101400 0x7f525819c360 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 --2- 172.21.15.22:0/934699217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5258100210 0x7f5258068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 >> 172.21.15.22:0/934699217 conn(0x7f52580fb780 msgr2=0x7f5258104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 shutdown_connections 2024-03-08T23:54:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.584+0000 7f525edb2640 1 -- 172.21.15.22:0/934699217 wait complete. 2024-03-08T23:54:21.591 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-08T23:54:21.591 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-08T23:54:21.591 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T23:54:21.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 -- 172.21.15.22:0/249017531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04101310 msgr2=0x7f1e04101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 --2- 172.21.15.22:0/249017531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04101310 0x7f1e04101730 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009900 tx=0x7f1e0002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 -- 172.21.15.22:0/249017531 shutdown_connections 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 --2- 172.21.15.22:0/249017531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e04102ec0 0x7f1e04109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 --2- 172.21.15.22:0/249017531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04102500 0x7f1e04102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 --2- 172.21.15.22:0/249017531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04101310 0x7f1e04101730 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 -- 172.21.15.22:0/249017531 >> 172.21.15.22:0/249017531 conn(0x7f1e040fc8a0 msgr2=0x7f1e040fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 -- 172.21.15.22:0/249017531 shutdown_connections 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.697+0000 7f1e0b590640 1 -- 172.21.15.22:0/249017531 wait complete. 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 Processor -- start 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 -- start start 2024-03-08T23:54:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04101310 0x7f1e041975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e04102500 0x7f1e04197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04102ec0 0x7f1e0419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0410cf70 con 0x7f1e04102ec0 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0410cdf0 con 0x7f1e04101310 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0b590640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0410d0f0 con 0x7f1e04102500 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04102ec0 0x7f1e0419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04102ec0 0x7f1e0419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51884/0 (socket says 172.21.15.22:51884) 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 -- 172.21.15.22:0/2073266763 learned_addr learned my addr 172.21.15.22:0/2073266763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e08b2b640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e04102500 0x7f1e04197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e0932c640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04101310 0x7f1e041975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 -- 172.21.15.22:0/2073266763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e04102500 msgr2=0x7f1e04197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e04102500 0x7f1e04197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 -- 172.21.15.22:0/2073266763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04101310 msgr2=0x7f1e041975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04101310 0x7f1e041975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e09b2d640 1 -- 172.21.15.22:0/2073266763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0419f200 con 0x7f1e04102ec0 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.698+0000 7f1e08b2b640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e04102500 0x7f1e04197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.699+0000 7f1e09b2d640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04102ec0 0x7f1e0419eb90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00cce0 tx=0x7f1dfc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.699+0000 7f1dfa7fc640 1 -- 172.21.15.22:0/2073266763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00f450 con 0x7f1e04102ec0 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.699+0000 7f1dfa7fc640 1 -- 172.21.15.22:0/2073266763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dfc00fad0 con 0x7f1e04102ec0 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.699+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0419f4f0 con 0x7f1e04102ec0 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.699+0000 7f1dfa7fc640 1 -- 172.21.15.22:0/2073266763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc018c10 con 0x7f1e04102ec0 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.699+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0419f9f0 con 0x7f1e04102ec0 2024-03-08T23:54:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.700+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04108630 con 0x7f1e04102ec0 2024-03-08T23:54:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.701+0000 7f1dfa7fc640 1 -- 172.21.15.22:0/2073266763 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1dfc00f5f0 con 0x7f1e04102ec0 2024-03-08T23:54:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.701+0000 7f1dfa7fc640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df00a2890 0x7f1df00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.702+0000 7f1e0932c640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df00a2890 0x7f1df00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.702+0000 7f1dfa7fc640 1 -- 172.21.15.22:0/2073266763 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1dfc0cf7a0 con 0x7f1e04102ec0 2024-03-08T23:54:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.702+0000 7f1e0932c640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df00a2890 0x7f1df00a4d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1e0002c7b0 tx=0x7f1e00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.706+0000 7f1dfa7fc640 1 -- 172.21.15.22:0/2073266763 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1dfc014030 con 0x7f1e04102ec0 2024-03-08T23:54:21.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.884+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e04061730 con 0x7f1df00a2890 2024-03-08T23:54:21.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.894+0000 7f1dfa7fc640 1 -- 172.21.15.22:0/2073266763 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340501 (secure 0 0 0) 0x7f1e04061730 con 0x7f1df00a2890 2024-03-08T23:54:21.894 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:21.896 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-08T23:54:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df00a2890 msgr2=0x7f1df00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df00a2890 0x7f1df00a4d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1e0002c7b0 tx=0x7f1e00005fd0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04102ec0 msgr2=0x7f1e0419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04102ec0 0x7f1e0419eb90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00cce0 tx=0x7f1dfc00a620 comp rx=0 tx=0).stop 2024-03-08T23:54:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 shutdown_connections 2024-03-08T23:54:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f1df00a2890 0x7f1df00a4d50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e04102ec0 0x7f1e0419eb90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e04102500 0x7f1e04197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 --2- 172.21.15.22:0/2073266763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04101310 0x7f1e041975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 >> 172.21.15.22:0/2073266763 conn(0x7f1e040fc8a0 msgr2=0x7f1e04105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 shutdown_connections 2024-03-08T23:54:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:21.897+0000 7f1e0b590640 1 -- 172.21.15.22:0/2073266763 wait complete. 2024-03-08T23:54:21.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:21 smithi022 ceph-mon[31894]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-08T23:54:21.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:21 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3529120916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:21.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:21 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2181573082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:21.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:21 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/934699217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:21.904 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-03-08T23:54:21.826311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664629,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":882,"num_read_kb":571,"num_write":365,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1241556,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363676,"kb_used_data":14488,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578148,"statfs":{"total":1151940427776,"available":1151568023552,"internally_reserved":0,"allocated":14835712,"data_stored":8199512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6771,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":38,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003064"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633752+0000","last_change":"2024-03-08T23:51:52.246270+0000","last_active":"2024-03-08T23:53:45.633752+0000","last_peered":"2024-03-08T23:53:45.633752+0000","last_clean":"2024-03-08T23:53:45.633752+0000","last_became_active":"2024-03-08T23:51:52.245321+0000","last_became_peered":"2024-03-08T23:51:52.245321+0000","last_unstale":"2024-03-08T23:53:45.633752+0000","last_undegraded":"2024-03-08T23:53:45.633752+0000","last_fullsized":"2024-03-08T23:53:45.633752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:39.993965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000208+0000","last_change":"2024-03-08T23:51:52.245433+0000","last_active":"2024-03-08T23:53:46.000208+0000","last_peered":"2024-03-08T23:53:46.000208+0000","last_clean":"2024-03-08T23:53:46.000208+0000","last_became_active":"2024-03-08T23:51:52.245106+0000","last_became_peered":"2024-03-08T23:51:52.245106+0000","last_unstale":"2024-03-08T23:53:46.000208+0000","last_undegraded":"2024-03-08T23:53:46.000208+0000","last_fullsized":"2024-03-08T23:53:46.000208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:02:20.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000027+0000","last_change":"2024-03-08T23:51:52.244112+0000","last_active":"2024-03-08T23:53:46.000027+0000","last_peered":"2024-03-08T23:53:46.000027+0000","last_clean":"2024-03-08T23:53:46.000027+0000","last_became_active":"2024-03-08T23:51:52.243783+0000","last_became_peered":"2024-03-08T23:51:52.243783+0000","last_unstale":"2024-03-08T23:53:46.000027+0000","last_undegraded":"2024-03-08T23:53:46.000027+0000","last_fullsized":"2024-03-08T23:53:46.000027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:48:43.719188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875262+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:53:45.875262+0000","last_peered":"2024-03-08T23:53:45.875262+0000","last_clean":"2024-03-08T23:53:45.875262+0000","last_became_active":"2024-03-08T23:52:45.585292+0000","last_became_peered":"2024-03-08T23:52:45.585292+0000","last_unstale":"2024-03-08T23:53:45.875262+0000","last_undegraded":"2024-03-08T23:53:45.875262+0000","last_fullsized":"2024-03-08T23:53:45.875262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:27:12.230920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637664+0000","last_change":"2024-03-08T23:51:52.247091+0000","last_active":"2024-03-08T23:53:45.637664+0000","last_peered":"2024-03-08T23:53:45.637664+0000","last_clean":"2024-03-08T23:53:45.637664+0000","last_became_active":"2024-03-08T23:51:52.246761+0000","last_became_peered":"2024-03-08T23:51:52.246761+0000","last_unstale":"2024-03-08T23:53:45.637664+0000","last_undegraded":"2024-03-08T23:53:45.637664+0000","last_fullsized":"2024-03-08T23:53:45.637664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:46:46.810857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001715+0000","last_change":"2024-03-08T23:51:52.257558+0000","last_active":"2024-03-08T23:53:46.001715+0000","last_peered":"2024-03-08T23:53:46.001715+0000","last_clean":"2024-03-08T23:53:46.001715+0000","last_became_active":"2024-03-08T23:51:52.257116+0000","last_became_peered":"2024-03-08T23:51:52.257116+0000","last_unstale":"2024-03-08T23:53:46.001715+0000","last_undegraded":"2024-03-08T23:53:46.001715+0000","last_fullsized":"2024-03-08T23:53:46.001715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:13:05.325780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000412+0000","last_change":"2024-03-08T23:51:52.253624+0000","last_active":"2024-03-08T23:53:46.000412+0000","last_peered":"2024-03-08T23:53:46.000412+0000","last_clean":"2024-03-08T23:53:46.000412+0000","last_became_active":"2024-03-08T23:51:52.253284+0000","last_became_peered":"2024-03-08T23:51:52.253284+0000","last_unstale":"2024-03-08T23:53:46.000412+0000","last_undegraded":"2024-03-08T23:53:46.000412+0000","last_fullsized":"2024-03-08T23:53:46.000412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:09:33.789502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636159+0000","last_change":"2024-03-08T23:51:52.255321+0000","last_active":"2024-03-08T23:53:45.636159+0000","last_peered":"2024-03-08T23:53:45.636159+0000","last_clean":"2024-03-08T23:53:45.636159+0000","last_became_active":"2024-03-08T23:51:52.254975+0000","last_became_peered":"2024-03-08T23:51:52.254975+0000","last_unstale":"2024-03-08T23:53:45.636159+0000","last_undegraded":"2024-03-08T23:53:45.636159+0000","last_fullsized":"2024-03-08T23:53:45.636159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:02:56.788519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001266+0000","last_change":"2024-03-08T23:51:52.244623+0000","last_active":"2024-03-08T23:53:46.001266+0000","last_peered":"2024-03-08T23:53:46.001266+0000","last_clean":"2024-03-08T23:53:46.001266+0000","last_became_active":"2024-03-08T23:51:52.244162+0000","last_became_peered":"2024-03-08T23:51:52.244162+0000","last_unstale":"2024-03-08T23:53:46.001266+0000","last_undegraded":"2024-03-08T23:53:46.001266+0000","last_fullsized":"2024-03-08T23:53:46.001266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:30:43.050616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001879+0000","last_change":"2024-03-08T23:51:52.258006+0000","last_active":"2024-03-08T23:53:46.001879+0000","last_peered":"2024-03-08T23:53:46.001879+0000","last_clean":"2024-03-08T23:53:46.001879+0000","last_became_active":"2024-03-08T23:51:52.256770+0000","last_became_peered":"2024-03-08T23:51:52.256770+0000","last_unstale":"2024-03-08T23:53:46.001879+0000","last_undegraded":"2024-03-08T23:53:46.001879+0000","last_fullsized":"2024-03-08T23:53:46.001879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:18.584212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002234+0000","last_change":"2024-03-08T23:51:52.260530+0000","last_active":"2024-03-08T23:53:46.002234+0000","last_peered":"2024-03-08T23:53:46.002234+0000","last_clean":"2024-03-08T23:53:46.002234+0000","last_became_active":"2024-03-08T23:51:52.260193+0000","last_became_peered":"2024-03-08T23:51:52.260193+0000","last_unstale":"2024-03-08T23:53:46.002234+0000","last_undegraded":"2024-03-08T23:53:46.002234+0000","last_fullsized":"2024-03-08T23:53:46.002234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:02.549683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629380+0000","last_change":"2024-03-08T23:51:52.248679+0000","last_active":"2024-03-08T23:53:45.629380+0000","last_peered":"2024-03-08T23:53:45.629380+0000","last_clean":"2024-03-08T23:53:45.629380+0000","last_became_active":"2024-03-08T23:51:52.248358+0000","last_became_peered":"2024-03-08T23:51:52.248358+0000","last_unstale":"2024-03-08T23:53:45.629380+0000","last_undegraded":"2024-03-08T23:53:45.629380+0000","last_fullsized":"2024-03-08T23:53:45.629380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:26.680887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626733+0000","last_change":"2024-03-08T23:52:45.415807+0000","last_active":"2024-03-08T23:53:45.626733+0000","last_peered":"2024-03-08T23:53:45.626733+0000","last_clean":"2024-03-08T23:53:45.626733+0000","last_became_active":"2024-03-08T23:52:45.415456+0000","last_became_peered":"2024-03-08T23:52:45.415456+0000","last_unstale":"2024-03-08T23:53:45.626733+0000","last_undegraded":"2024-03-08T23:53:45.626733+0000","last_fullsized":"2024-03-08T23:53:45.626733+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:12:01.942744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999714+0000","last_change":"2024-03-08T23:51:52.245394+0000","last_active":"2024-03-08T23:53:45.999714+0000","last_peered":"2024-03-08T23:53:45.999714+0000","last_clean":"2024-03-08T23:53:45.999714+0000","last_became_active":"2024-03-08T23:51:52.245038+0000","last_became_peered":"2024-03-08T23:51:52.245038+0000","last_unstale":"2024-03-08T23:53:45.999714+0000","last_undegraded":"2024-03-08T23:53:45.999714+0000","last_fullsized":"2024-03-08T23:53:45.999714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:15.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999804+0000","last_change":"2024-03-08T23:51:52.250861+0000","last_active":"2024-03-08T23:53:45.999804+0000","last_peered":"2024-03-08T23:53:45.999804+0000","last_clean":"2024-03-08T23:53:45.999804+0000","last_became_active":"2024-03-08T23:51:52.250548+0000","last_became_peered":"2024-03-08T23:51:52.250548+0000","last_unstale":"2024-03-08T23:53:45.999804+0000","last_undegraded":"2024-03-08T23:53:45.999804+0000","last_fullsized":"2024-03-08T23:53:45.999804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:00.280043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000326+0000","last_change":"2024-03-08T23:51:52.254996+0000","last_active":"2024-03-08T23:53:46.000326+0000","last_peered":"2024-03-08T23:53:46.000326+0000","last_clean":"2024-03-08T23:53:46.000326+0000","last_became_active":"2024-03-08T23:51:52.254634+0000","last_became_peered":"2024-03-08T23:51:52.254634+0000","last_unstale":"2024-03-08T23:53:46.000326+0000","last_undegraded":"2024-03-08T23:53:46.000326+0000","last_fullsized":"2024-03-08T23:53:46.000326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:15.449988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633637+0000","last_change":"2024-03-08T23:51:52.246558+0000","last_active":"2024-03-08T23:53:45.633637+0000","last_peered":"2024-03-08T23:53:45.633637+0000","last_clean":"2024-03-08T23:53:45.633637+0000","last_became_active":"2024-03-08T23:51:52.245741+0000","last_became_peered":"2024-03-08T23:51:52.245741+0000","last_unstale":"2024-03-08T23:53:45.633637+0000","last_undegraded":"2024-03-08T23:53:45.633637+0000","last_fullsized":"2024-03-08T23:53:45.633637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:36.737017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875498+0000","last_change":"2024-03-08T23:51:52.240111+0000","last_active":"2024-03-08T23:53:45.875498+0000","last_peered":"2024-03-08T23:53:45.875498+0000","last_clean":"2024-03-08T23:53:45.875498+0000","last_became_active":"2024-03-08T23:51:52.238961+0000","last_became_peered":"2024-03-08T23:51:52.238961+0000","last_unstale":"2024-03-08T23:53:45.875498+0000","last_undegraded":"2024-03-08T23:53:45.875498+0000","last_fullsized":"2024-03-08T23:53:45.875498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:03:55.313117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638666+0000","last_change":"2024-03-08T23:51:52.245979+0000","last_active":"2024-03-08T23:53:45.638666+0000","last_peered":"2024-03-08T23:53:45.638666+0000","last_clean":"2024-03-08T23:53:45.638666+0000","last_became_active":"2024-03-08T23:51:52.245666+0000","last_became_peered":"2024-03-08T23:51:52.245666+0000","last_unstale":"2024-03-08T23:53:45.638666+0000","last_undegraded":"2024-03-08T23:53:45.638666+0000","last_fullsized":"2024-03-08T23:53:45.638666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:20.789144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638219+0000","last_change":"2024-03-08T23:51:56.288161+0000","last_active":"2024-03-08T23:53:45.638219+0000","last_peered":"2024-03-08T23:53:45.638219+0000","last_clean":"2024-03-08T23:53:45.638219+0000","last_became_active":"2024-03-08T23:51:56.287778+0000","last_became_peered":"2024-03-08T23:51:56.287778+0000","last_unstale":"2024-03-08T23:53:45.638219+0000","last_undegraded":"2024-03-08T23:53:45.638219+0000","last_fullsized":"2024-03-08T23:53:45.638219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:12:59.636228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875582+0000","last_change":"2024-03-08T23:51:51.244911+0000","last_active":"2024-03-08T23:53:45.875582+0000","last_peered":"2024-03-08T23:53:45.875582+0000","last_clean":"2024-03-08T23:53:45.875582+0000","last_became_active":"2024-03-08T23:51:51.244561+0000","last_became_peered":"2024-03-08T23:51:51.244561+0000","last_unstale":"2024-03-08T23:53:45.875582+0000","last_undegraded":"2024-03-08T23:53:45.875582+0000","last_fullsized":"2024-03-08T23:53:45.875582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:58:45.908307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636719+0000","last_change":"2024-03-08T23:51:52.253854+0000","last_active":"2024-03-08T23:53:45.636719+0000","last_peered":"2024-03-08T23:53:45.636719+0000","last_clean":"2024-03-08T23:53:45.636719+0000","last_became_active":"2024-03-08T23:51:52.253538+0000","last_became_peered":"2024-03-08T23:51:52.253538+0000","last_unstale":"2024-03-08T23:53:45.636719+0000","last_undegraded":"2024-03-08T23:53:45.636719+0000","last_fullsized":"2024-03-08T23:53:45.636719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:33.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628202+0000","last_change":"2024-03-08T23:51:56.283307+0000","last_active":"2024-03-08T23:53:45.628202+0000","last_peered":"2024-03-08T23:53:45.628202+0000","last_clean":"2024-03-08T23:53:45.628202+0000","last_became_active":"2024-03-08T23:51:56.282868+0000","last_became_peered":"2024-03-08T23:51:56.282868+0000","last_unstale":"2024-03-08T23:53:45.628202+0000","last_undegraded":"2024-03-08T23:53:45.628202+0000","last_fullsized":"2024-03-08T23:53:45.628202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:15.027743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636962+0000","last_change":"2024-03-08T23:51:51.247368+0000","last_active":"2024-03-08T23:53:45.636962+0000","last_peered":"2024-03-08T23:53:45.636962+0000","last_clean":"2024-03-08T23:53:45.636962+0000","last_became_active":"2024-03-08T23:51:51.247023+0000","last_became_peered":"2024-03-08T23:51:51.247023+0000","last_unstale":"2024-03-08T23:53:45.636962+0000","last_undegraded":"2024-03-08T23:53:45.636962+0000","last_fullsized":"2024-03-08T23:53:45.636962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:28.311850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000106+0000","last_change":"2024-03-08T23:51:52.245805+0000","last_active":"2024-03-08T23:53:46.000106+0000","last_peered":"2024-03-08T23:53:46.000106+0000","last_clean":"2024-03-08T23:53:46.000106+0000","last_became_active":"2024-03-08T23:51:52.244455+0000","last_became_peered":"2024-03-08T23:51:52.244455+0000","last_unstale":"2024-03-08T23:53:46.000106+0000","last_undegraded":"2024-03-08T23:53:46.000106+0000","last_fullsized":"2024-03-08T23:53:46.000106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:11:17.809148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628232+0000","last_change":"2024-03-08T23:51:56.286666+0000","last_active":"2024-03-08T23:53:45.628232+0000","last_peered":"2024-03-08T23:53:45.628232+0000","last_clean":"2024-03-08T23:53:45.628232+0000","last_became_active":"2024-03-08T23:51:56.285968+0000","last_became_peered":"2024-03-08T23:51:56.285968+0000","last_unstale":"2024-03-08T23:53:45.628232+0000","last_undegraded":"2024-03-08T23:53:45.628232+0000","last_fullsized":"2024-03-08T23:53:45.628232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:51.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638792+0000","last_change":"2024-03-08T23:51:51.247684+0000","last_active":"2024-03-08T23:53:45.638792+0000","last_peered":"2024-03-08T23:53:45.638792+0000","last_clean":"2024-03-08T23:53:45.638792+0000","last_became_active":"2024-03-08T23:51:51.247374+0000","last_became_peered":"2024-03-08T23:51:51.247374+0000","last_unstale":"2024-03-08T23:53:45.638792+0000","last_undegraded":"2024-03-08T23:53:45.638792+0000","last_fullsized":"2024-03-08T23:53:45.638792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:20.425302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000566+0000","last_change":"2024-03-08T23:51:52.248702+0000","last_active":"2024-03-08T23:53:46.000566+0000","last_peered":"2024-03-08T23:53:46.000566+0000","last_clean":"2024-03-08T23:53:46.000566+0000","last_became_active":"2024-03-08T23:51:52.248388+0000","last_became_peered":"2024-03-08T23:51:52.248388+0000","last_unstale":"2024-03-08T23:53:46.000566+0000","last_undegraded":"2024-03-08T23:53:46.000566+0000","last_fullsized":"2024-03-08T23:53:46.000566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:27:12.251957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628238+0000","last_change":"2024-03-08T23:51:56.286572+0000","last_active":"2024-03-08T23:53:45.628238+0000","last_peered":"2024-03-08T23:53:45.628238+0000","last_clean":"2024-03-08T23:53:45.628238+0000","last_became_active":"2024-03-08T23:51:56.286066+0000","last_became_peered":"2024-03-08T23:51:56.286066+0000","last_unstale":"2024-03-08T23:53:45.628238+0000","last_undegraded":"2024-03-08T23:53:45.628238+0000","last_fullsized":"2024-03-08T23:53:45.628238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:13.694615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638236+0000","last_change":"2024-03-08T23:51:51.244207+0000","last_active":"2024-03-08T23:53:45.638236+0000","last_peered":"2024-03-08T23:53:45.638236+0000","last_clean":"2024-03-08T23:53:45.638236+0000","last_became_active":"2024-03-08T23:51:51.241605+0000","last_became_peered":"2024-03-08T23:51:51.241605+0000","last_unstale":"2024-03-08T23:53:45.638236+0000","last_undegraded":"2024-03-08T23:53:45.638236+0000","last_fullsized":"2024-03-08T23:53:45.638236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:03:51.766681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629253+0000","last_change":"2024-03-08T23:51:52.251015+0000","last_active":"2024-03-08T23:53:45.629253+0000","last_peered":"2024-03-08T23:53:45.629253+0000","last_clean":"2024-03-08T23:53:45.629253+0000","last_became_active":"2024-03-08T23:51:52.250690+0000","last_became_peered":"2024-03-08T23:51:52.250690+0000","last_unstale":"2024-03-08T23:53:45.629253+0000","last_undegraded":"2024-03-08T23:53:45.629253+0000","last_fullsized":"2024-03-08T23:53:45.629253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:57:23.033172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638368+0000","last_change":"2024-03-08T23:51:56.281066+0000","last_active":"2024-03-08T23:53:45.638368+0000","last_peered":"2024-03-08T23:53:45.638368+0000","last_clean":"2024-03-08T23:53:45.638368+0000","last_became_active":"2024-03-08T23:51:56.280654+0000","last_became_peered":"2024-03-08T23:51:56.280654+0000","last_unstale":"2024-03-08T23:53:45.638368+0000","last_undegraded":"2024-03-08T23:53:45.638368+0000","last_fullsized":"2024-03-08T23:53:45.638368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:53.999115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:16.524728+0000","last_change":"2024-03-08T23:51:51.245947+0000","last_active":"2024-03-08T23:54:16.524728+0000","last_peered":"2024-03-08T23:54:16.524728+0000","last_clean":"2024-03-08T23:54:16.524728+0000","last_became_active":"2024-03-08T23:51:51.245505+0000","last_became_peered":"2024-03-08T23:51:51.245505+0000","last_unstale":"2024-03-08T23:54:16.524728+0000","last_undegraded":"2024-03-08T23:54:16.524728+0000","last_fullsized":"2024-03-08T23:54:16.524728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:20:44.692598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637141+0000","last_change":"2024-03-08T23:51:52.241590+0000","last_active":"2024-03-08T23:53:45.637141+0000","last_peered":"2024-03-08T23:53:45.637141+0000","last_clean":"2024-03-08T23:53:45.637141+0000","last_became_active":"2024-03-08T23:51:52.241296+0000","last_became_peered":"2024-03-08T23:51:52.241296+0000","last_unstale":"2024-03-08T23:53:45.637141+0000","last_undegraded":"2024-03-08T23:53:45.637141+0000","last_fullsized":"2024-03-08T23:53:45.637141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:52:48.042360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638174+0000","last_change":"2024-03-08T23:52:45.585946+0000","last_active":"2024-03-08T23:53:45.638174+0000","last_peered":"2024-03-08T23:53:45.638174+0000","last_clean":"2024-03-08T23:53:45.638174+0000","last_became_active":"2024-03-08T23:52:45.585470+0000","last_became_peered":"2024-03-08T23:52:45.585470+0000","last_unstale":"2024-03-08T23:53:45.638174+0000","last_undegraded":"2024-03-08T23:53:45.638174+0000","last_fullsized":"2024-03-08T23:53:45.638174+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:28:36.264887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001568+0000","last_change":"2024-03-08T23:51:51.241384+0000","last_active":"2024-03-08T23:53:46.001568+0000","last_peered":"2024-03-08T23:53:46.001568+0000","last_clean":"2024-03-08T23:53:46.001568+0000","last_became_active":"2024-03-08T23:51:51.241018+0000","last_became_peered":"2024-03-08T23:51:51.241018+0000","last_unstale":"2024-03-08T23:53:46.001568+0000","last_undegraded":"2024-03-08T23:53:46.001568+0000","last_fullsized":"2024-03-08T23:53:46.001568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:29:50.860663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629076+0000","last_change":"2024-03-08T23:51:52.250207+0000","last_active":"2024-03-08T23:53:45.629076+0000","last_peered":"2024-03-08T23:53:45.629076+0000","last_clean":"2024-03-08T23:53:45.629076+0000","last_became_active":"2024-03-08T23:51:52.249863+0000","last_became_peered":"2024-03-08T23:51:52.249863+0000","last_unstale":"2024-03-08T23:53:45.629076+0000","last_undegraded":"2024-03-08T23:53:45.629076+0000","last_fullsized":"2024-03-08T23:53:45.629076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:18:48.732419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633519+0000","last_change":"2024-03-08T23:51:56.282092+0000","last_active":"2024-03-08T23:53:45.633519+0000","last_peered":"2024-03-08T23:53:45.633519+0000","last_clean":"2024-03-08T23:53:45.633519+0000","last_became_active":"2024-03-08T23:51:56.281657+0000","last_became_peered":"2024-03-08T23:51:56.281657+0000","last_unstale":"2024-03-08T23:53:45.633519+0000","last_undegraded":"2024-03-08T23:53:45.633519+0000","last_fullsized":"2024-03-08T23:53:45.633519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:50.440520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626316+0000","last_change":"2024-03-08T23:51:51.236816+0000","last_active":"2024-03-08T23:53:45.626316+0000","last_peered":"2024-03-08T23:53:45.626316+0000","last_clean":"2024-03-08T23:53:45.626316+0000","last_became_active":"2024-03-08T23:51:51.236485+0000","last_became_peered":"2024-03-08T23:51:51.236485+0000","last_unstale":"2024-03-08T23:53:45.626316+0000","last_undegraded":"2024-03-08T23:53:45.626316+0000","last_fullsized":"2024-03-08T23:53:45.626316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:27.300950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636406+0000","last_change":"2024-03-08T23:51:52.252146+0000","last_active":"2024-03-08T23:53:45.636406+0000","last_peered":"2024-03-08T23:53:45.636406+0000","last_clean":"2024-03-08T23:53:45.636406+0000","last_became_active":"2024-03-08T23:51:52.244201+0000","last_became_peered":"2024-03-08T23:51:52.244201+0000","last_unstale":"2024-03-08T23:53:45.636406+0000","last_undegraded":"2024-03-08T23:53:45.636406+0000","last_fullsized":"2024-03-08T23:53:45.636406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:07.647078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000164+0000","last_change":"2024-03-08T23:51:56.279727+0000","last_active":"2024-03-08T23:53:46.000164+0000","last_peered":"2024-03-08T23:53:46.000164+0000","last_clean":"2024-03-08T23:53:46.000164+0000","last_became_active":"2024-03-08T23:51:56.279184+0000","last_became_peered":"2024-03-08T23:51:56.279184+0000","last_unstale":"2024-03-08T23:53:46.000164+0000","last_undegraded":"2024-03-08T23:53:46.000164+0000","last_fullsized":"2024-03-08T23:53:46.000164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:21.804410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629021+0000","last_change":"2024-03-08T23:51:51.244812+0000","last_active":"2024-03-08T23:53:45.629021+0000","last_peered":"2024-03-08T23:53:45.629021+0000","last_clean":"2024-03-08T23:53:45.629021+0000","last_became_active":"2024-03-08T23:51:51.244513+0000","last_became_peered":"2024-03-08T23:51:51.244513+0000","last_unstale":"2024-03-08T23:53:45.629021+0000","last_undegraded":"2024-03-08T23:53:45.629021+0000","last_fullsized":"2024-03-08T23:53:45.629021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:56:07.235214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875255+0000","last_change":"2024-03-08T23:51:52.244566+0000","last_active":"2024-03-08T23:53:45.875255+0000","last_peered":"2024-03-08T23:53:45.875255+0000","last_clean":"2024-03-08T23:53:45.875255+0000","last_became_active":"2024-03-08T23:51:52.244250+0000","last_became_peered":"2024-03-08T23:51:52.244250+0000","last_unstale":"2024-03-08T23:53:45.875255+0000","last_undegraded":"2024-03-08T23:53:45.875255+0000","last_fullsized":"2024-03-08T23:53:45.875255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:55:50.596364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636425+0000","last_change":"2024-03-08T23:51:56.282487+0000","last_active":"2024-03-08T23:53:45.636425+0000","last_peered":"2024-03-08T23:53:45.636425+0000","last_clean":"2024-03-08T23:53:45.636425+0000","last_became_active":"2024-03-08T23:51:56.282104+0000","last_became_peered":"2024-03-08T23:51:56.282104+0000","last_unstale":"2024-03-08T23:53:45.636425+0000","last_undegraded":"2024-03-08T23:53:45.636425+0000","last_fullsized":"2024-03-08T23:53:45.636425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:23:23.787431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.527612+0000","last_change":"2024-03-08T23:51:51.236612+0000","last_active":"2024-03-08T23:54:18.527612+0000","last_peered":"2024-03-08T23:54:18.527612+0000","last_clean":"2024-03-08T23:54:18.527612+0000","last_became_active":"2024-03-08T23:51:51.236274+0000","last_became_peered":"2024-03-08T23:51:51.236274+0000","last_unstale":"2024-03-08T23:54:18.527612+0000","last_undegraded":"2024-03-08T23:54:18.527612+0000","last_fullsized":"2024-03-08T23:54:18.527612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:37.551870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000849+0000","last_change":"2024-03-08T23:51:52.244564+0000","last_active":"2024-03-08T23:53:46.000849+0000","last_peered":"2024-03-08T23:53:46.000849+0000","last_clean":"2024-03-08T23:53:46.000849+0000","last_became_active":"2024-03-08T23:51:52.243651+0000","last_became_peered":"2024-03-08T23:51:52.243651+0000","last_unstale":"2024-03-08T23:53:46.000849+0000","last_undegraded":"2024-03-08T23:53:46.000849+0000","last_fullsized":"2024-03-08T23:53:46.000849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:35:32.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875029+0000","last_change":"2024-03-08T23:51:56.279354+0000","last_active":"2024-03-08T23:53:45.875029+0000","last_peered":"2024-03-08T23:53:45.875029+0000","last_clean":"2024-03-08T23:53:45.875029+0000","last_became_active":"2024-03-08T23:51:56.278894+0000","last_became_peered":"2024-03-08T23:51:56.278894+0000","last_unstale":"2024-03-08T23:53:45.875029+0000","last_undegraded":"2024-03-08T23:53:45.875029+0000","last_fullsized":"2024-03-08T23:53:45.875029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:20:30.164542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002108+0000","last_change":"2024-03-08T23:51:51.246417+0000","last_active":"2024-03-08T23:53:46.002108+0000","last_peered":"2024-03-08T23:53:46.002108+0000","last_clean":"2024-03-08T23:53:46.002108+0000","last_became_active":"2024-03-08T23:51:51.246058+0000","last_became_peered":"2024-03-08T23:51:51.246058+0000","last_unstale":"2024-03-08T23:53:46.002108+0000","last_undegraded":"2024-03-08T23:53:46.002108+0000","last_fullsized":"2024-03-08T23:53:46.002108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:48:53.569295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000099+0000","last_change":"2024-03-08T23:51:52.255357+0000","last_active":"2024-03-08T23:53:46.000099+0000","last_peered":"2024-03-08T23:53:46.000099+0000","last_clean":"2024-03-08T23:53:46.000099+0000","last_became_active":"2024-03-08T23:51:52.255044+0000","last_became_peered":"2024-03-08T23:51:52.255044+0000","last_unstale":"2024-03-08T23:53:46.000099+0000","last_undegraded":"2024-03-08T23:53:46.000099+0000","last_fullsized":"2024-03-08T23:53:46.000099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:13:40.710283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628526+0000","last_change":"2024-03-08T23:51:56.284282+0000","last_active":"2024-03-08T23:53:45.628526+0000","last_peered":"2024-03-08T23:53:45.628526+0000","last_clean":"2024-03-08T23:53:45.628526+0000","last_became_active":"2024-03-08T23:51:56.283863+0000","last_became_peered":"2024-03-08T23:51:56.283863+0000","last_unstale":"2024-03-08T23:53:45.628526+0000","last_undegraded":"2024-03-08T23:53:45.628526+0000","last_fullsized":"2024-03-08T23:53:45.628526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:59:21.972283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629263+0000","last_change":"2024-03-08T23:51:51.245199+0000","last_active":"2024-03-08T23:53:45.629263+0000","last_peered":"2024-03-08T23:53:45.629263+0000","last_clean":"2024-03-08T23:53:45.629263+0000","last_became_active":"2024-03-08T23:51:51.244906+0000","last_became_peered":"2024-03-08T23:51:51.244906+0000","last_unstale":"2024-03-08T23:53:45.629263+0000","last_undegraded":"2024-03-08T23:53:45.629263+0000","last_fullsized":"2024-03-08T23:53:45.629263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:54:39.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000875+0000","last_change":"2024-03-08T23:51:52.251209+0000","last_active":"2024-03-08T23:53:46.000875+0000","last_peered":"2024-03-08T23:53:46.000875+0000","last_clean":"2024-03-08T23:53:46.000875+0000","last_became_active":"2024-03-08T23:51:52.250392+0000","last_became_peered":"2024-03-08T23:51:52.250392+0000","last_unstale":"2024-03-08T23:53:46.000875+0000","last_undegraded":"2024-03-08T23:53:46.000875+0000","last_fullsized":"2024-03-08T23:53:46.000875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:56:17.811505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001099+0000","last_change":"2024-03-08T23:52:45.586125+0000","last_active":"2024-03-08T23:53:46.001099+0000","last_peered":"2024-03-08T23:53:46.001099+0000","last_clean":"2024-03-08T23:53:46.001099+0000","last_became_active":"2024-03-08T23:52:45.585578+0000","last_became_peered":"2024-03-08T23:52:45.585578+0000","last_unstale":"2024-03-08T23:53:46.001099+0000","last_undegraded":"2024-03-08T23:53:46.001099+0000","last_fullsized":"2024-03-08T23:53:46.001099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:51:08.159460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"93'40","reported_seq":343,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.553773+0000","last_change":"2024-03-08T23:52:45.426691+0000","last_active":"2024-03-08T23:54:18.553773+0000","last_peered":"2024-03-08T23:54:18.553773+0000","last_clean":"2024-03-08T23:54:18.553773+0000","last_became_active":"2024-03-08T23:52:45.415742+0000","last_became_peered":"2024-03-08T23:52:45.415742+0000","last_unstale":"2024-03-08T23:54:18.553773+0000","last_undegraded":"2024-03-08T23:54:18.553773+0000","last_fullsized":"2024-03-08T23:54:18.553773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:05.389164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68654,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":187,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60681,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638662+0000","last_change":"2024-03-08T23:51:52.248715+0000","last_active":"2024-03-08T23:53:45.638662+0000","last_peered":"2024-03-08T23:53:45.638662+0000","last_clean":"2024-03-08T23:53:45.638662+0000","last_became_active":"2024-03-08T23:51:52.248258+0000","last_became_peered":"2024-03-08T23:51:52.248258+0000","last_unstale":"2024-03-08T23:53:45.638662+0000","last_undegraded":"2024-03-08T23:53:45.638662+0000","last_fullsized":"2024-03-08T23:53:45.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:38.185469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629520+0000","last_change":"2024-03-08T23:51:56.278268+0000","last_active":"2024-03-08T23:53:45.629520+0000","last_peered":"2024-03-08T23:53:45.629520+0000","last_clean":"2024-03-08T23:53:45.629520+0000","last_became_active":"2024-03-08T23:51:56.277831+0000","last_became_peered":"2024-03-08T23:51:56.277831+0000","last_unstale":"2024-03-08T23:53:45.629520+0000","last_undegraded":"2024-03-08T23:53:45.629520+0000","last_fullsized":"2024-03-08T23:53:45.629520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:32:13.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637455+0000","last_change":"2024-03-08T23:51:51.247553+0000","last_active":"2024-03-08T23:53:45.637455+0000","last_peered":"2024-03-08T23:53:45.637455+0000","last_clean":"2024-03-08T23:53:45.637455+0000","last_became_active":"2024-03-08T23:51:51.247225+0000","last_became_peered":"2024-03-08T23:51:51.247225+0000","last_unstale":"2024-03-08T23:53:45.637455+0000","last_undegraded":"2024-03-08T23:53:45.637455+0000","last_fullsized":"2024-03-08T23:53:45.637455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:53:29.846303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626580+0000","last_change":"2024-03-08T23:51:52.247733+0000","last_active":"2024-03-08T23:53:45.626580+0000","last_peered":"2024-03-08T23:53:45.626580+0000","last_clean":"2024-03-08T23:53:45.626580+0000","last_became_active":"2024-03-08T23:51:52.247435+0000","last_became_peered":"2024-03-08T23:51:52.247435+0000","last_unstale":"2024-03-08T23:53:45.626580+0000","last_undegraded":"2024-03-08T23:53:45.626580+0000","last_fullsized":"2024-03-08T23:53:45.626580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:46:12.739601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.632494+0000","last_change":"2024-03-08T23:51:56.279865+0000","last_active":"2024-03-08T23:53:45.632494+0000","last_peered":"2024-03-08T23:53:45.632494+0000","last_clean":"2024-03-08T23:53:45.632494+0000","last_became_active":"2024-03-08T23:51:56.279446+0000","last_became_peered":"2024-03-08T23:51:56.279446+0000","last_unstale":"2024-03-08T23:53:45.632494+0000","last_undegraded":"2024-03-08T23:53:45.632494+0000","last_fullsized":"2024-03-08T23:53:45.632494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:07:33.672490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629392+0000","last_change":"2024-03-08T23:51:51.246860+0000","last_active":"2024-03-08T23:53:45.629392+0000","last_peered":"2024-03-08T23:53:45.629392+0000","last_clean":"2024-03-08T23:53:45.629392+0000","last_became_active":"2024-03-08T23:51:51.246545+0000","last_became_peered":"2024-03-08T23:51:51.246545+0000","last_unstale":"2024-03-08T23:53:45.629392+0000","last_undegraded":"2024-03-08T23:53:45.629392+0000","last_fullsized":"2024-03-08T23:53:45.629392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:11:07.246657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001899+0000","last_change":"2024-03-08T23:51:52.247353+0000","last_active":"2024-03-08T23:53:46.001899+0000","last_peered":"2024-03-08T23:53:46.001899+0000","last_clean":"2024-03-08T23:53:46.001899+0000","last_became_active":"2024-03-08T23:51:52.247008+0000","last_became_peered":"2024-03-08T23:51:52.247008+0000","last_unstale":"2024-03-08T23:53:46.001899+0000","last_undegraded":"2024-03-08T23:53:46.001899+0000","last_fullsized":"2024-03-08T23:53:46.001899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:08:50.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875481+0000","last_change":"2024-03-08T23:51:56.284103+0000","last_active":"2024-03-08T23:53:45.875481+0000","last_peered":"2024-03-08T23:53:45.875481+0000","last_clean":"2024-03-08T23:53:45.875481+0000","last_became_active":"2024-03-08T23:51:56.283561+0000","last_became_peered":"2024-03-08T23:51:56.283561+0000","last_unstale":"2024-03-08T23:53:45.875481+0000","last_undegraded":"2024-03-08T23:53:45.875481+0000","last_fullsized":"2024-03-08T23:53:45.875481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:47:20.909292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:17.524146+0000","last_change":"2024-03-08T23:51:51.242483+0000","last_active":"2024-03-08T23:54:17.524146+0000","last_peered":"2024-03-08T23:54:17.524146+0000","last_clean":"2024-03-08T23:54:17.524146+0000","last_became_active":"2024-03-08T23:51:51.242153+0000","last_became_peered":"2024-03-08T23:51:51.242153+0000","last_unstale":"2024-03-08T23:54:17.524146+0000","last_undegraded":"2024-03-08T23:54:17.524146+0000","last_fullsized":"2024-03-08T23:54:17.524146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:44:07.728700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001140+0000","last_change":"2024-03-08T23:51:52.255411+0000","last_active":"2024-03-08T23:53:46.001140+0000","last_peered":"2024-03-08T23:53:46.001140+0000","last_clean":"2024-03-08T23:53:46.001140+0000","last_became_active":"2024-03-08T23:51:52.255048+0000","last_became_peered":"2024-03-08T23:51:52.255048+0000","last_unstale":"2024-03-08T23:53:46.001140+0000","last_undegraded":"2024-03-08T23:53:46.001140+0000","last_fullsized":"2024-03-08T23:53:46.001140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:11.836187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626682+0000","last_change":"2024-03-08T23:51:56.286405+0000","last_active":"2024-03-08T23:53:45.626682+0000","last_peered":"2024-03-08T23:53:45.626682+0000","last_clean":"2024-03-08T23:53:45.626682+0000","last_became_active":"2024-03-08T23:51:56.285996+0000","last_became_peered":"2024-03-08T23:51:56.285996+0000","last_unstale":"2024-03-08T23:53:45.626682+0000","last_undegraded":"2024-03-08T23:53:45.626682+0000","last_fullsized":"2024-03-08T23:53:45.626682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:48:30.928436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637190+0000","last_change":"2024-03-08T23:51:51.240146+0000","last_active":"2024-03-08T23:53:45.637190+0000","last_peered":"2024-03-08T23:53:45.637190+0000","last_clean":"2024-03-08T23:53:45.637190+0000","last_became_active":"2024-03-08T23:51:51.239795+0000","last_became_peered":"2024-03-08T23:51:51.239795+0000","last_unstale":"2024-03-08T23:53:45.637190+0000","last_undegraded":"2024-03-08T23:53:45.637190+0000","last_fullsized":"2024-03-08T23:53:45.637190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:55.409922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629399+0000","last_change":"2024-03-08T23:51:52.254701+0000","last_active":"2024-03-08T23:53:45.629399+0000","last_peered":"2024-03-08T23:53:45.629399+0000","last_clean":"2024-03-08T23:53:45.629399+0000","last_became_active":"2024-03-08T23:51:52.254391+0000","last_became_peered":"2024-03-08T23:51:52.254391+0000","last_unstale":"2024-03-08T23:53:45.629399+0000","last_undegraded":"2024-03-08T23:53:45.629399+0000","last_fullsized":"2024-03-08T23:53:45.629399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:11:12.258181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628931+0000","last_change":"2024-03-08T23:51:56.277878+0000","last_active":"2024-03-08T23:53:45.628931+0000","last_peered":"2024-03-08T23:53:45.628931+0000","last_clean":"2024-03-08T23:53:45.628931+0000","last_became_active":"2024-03-08T23:51:56.277493+0000","last_became_peered":"2024-03-08T23:51:56.277493+0000","last_unstale":"2024-03-08T23:53:45.628931+0000","last_undegraded":"2024-03-08T23:53:45.628931+0000","last_fullsized":"2024-03-08T23:53:45.628931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:44:16.968632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.643148+0000","last_change":"2024-03-08T23:53:45.643148+0000","last_active":"2024-03-08T23:53:45.643148+0000","last_peered":"2024-03-08T23:53:45.643148+0000","last_clean":"2024-03-08T23:53:45.643148+0000","last_became_active":"2024-03-08T23:53:45.638975+0000","last_became_peered":"2024-03-08T23:53:45.638975+0000","last_unstale":"2024-03-08T23:53:45.643148+0000","last_undegraded":"2024-03-08T23:53:45.643148+0000","last_fullsized":"2024-03-08T23:53:45.643148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:42:39.188486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.632141+0000","last_change":"2024-03-08T23:51:52.248465+0000","last_active":"2024-03-08T23:53:45.632141+0000","last_peered":"2024-03-08T23:53:45.632141+0000","last_clean":"2024-03-08T23:53:45.632141+0000","last_became_active":"2024-03-08T23:51:52.248135+0000","last_became_peered":"2024-03-08T23:51:52.248135+0000","last_unstale":"2024-03-08T23:53:45.632141+0000","last_undegraded":"2024-03-08T23:53:45.632141+0000","last_fullsized":"2024-03-08T23:53:45.632141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:55:53.712958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638275+0000","last_change":"2024-03-08T23:51:56.276019+0000","last_active":"2024-03-08T23:53:45.638275+0000","last_peered":"2024-03-08T23:53:45.638275+0000","last_clean":"2024-03-08T23:53:45.638275+0000","last_became_active":"2024-03-08T23:51:56.275564+0000","last_became_peered":"2024-03-08T23:51:56.275564+0000","last_unstale":"2024-03-08T23:53:45.638275+0000","last_undegraded":"2024-03-08T23:53:45.638275+0000","last_fullsized":"2024-03-08T23:53:45.638275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:12:36.096470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875254+0000","last_change":"2024-03-08T23:51:51.241629+0000","last_active":"2024-03-08T23:53:45.875254+0000","last_peered":"2024-03-08T23:53:45.875254+0000","last_clean":"2024-03-08T23:53:45.875254+0000","last_became_active":"2024-03-08T23:51:51.241300+0000","last_became_peered":"2024-03-08T23:51:51.241300+0000","last_unstale":"2024-03-08T23:53:45.875254+0000","last_undegraded":"2024-03-08T23:53:45.875254+0000","last_fullsized":"2024-03-08T23:53:45.875254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:01:45.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001329+0000","last_change":"2024-03-08T23:51:52.246024+0000","last_active":"2024-03-08T23:53:46.001329+0000","last_peered":"2024-03-08T23:53:46.001329+0000","last_clean":"2024-03-08T23:53:46.001329+0000","last_became_active":"2024-03-08T23:51:52.245706+0000","last_became_peered":"2024-03-08T23:51:52.245706+0000","last_unstale":"2024-03-08T23:53:46.001329+0000","last_undegraded":"2024-03-08T23:53:46.001329+0000","last_fullsized":"2024-03-08T23:53:46.001329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:00:44.557269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626476+0000","last_change":"2024-03-08T23:51:56.285649+0000","last_active":"2024-03-08T23:53:45.626476+0000","last_peered":"2024-03-08T23:53:45.626476+0000","last_clean":"2024-03-08T23:53:45.626476+0000","last_became_active":"2024-03-08T23:51:56.282731+0000","last_became_peered":"2024-03-08T23:51:56.282731+0000","last_unstale":"2024-03-08T23:53:45.626476+0000","last_undegraded":"2024-03-08T23:53:45.626476+0000","last_fullsized":"2024-03-08T23:53:45.626476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:23:54.103637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001789+0000","last_change":"2024-03-08T23:51:51.249442+0000","last_active":"2024-03-08T23:53:46.001789+0000","last_peered":"2024-03-08T23:53:46.001789+0000","last_clean":"2024-03-08T23:53:46.001789+0000","last_became_active":"2024-03-08T23:51:51.249084+0000","last_became_peered":"2024-03-08T23:51:51.249084+0000","last_unstale":"2024-03-08T23:53:46.001789+0000","last_undegraded":"2024-03-08T23:53:46.001789+0000","last_fullsized":"2024-03-08T23:53:46.001789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:48:19.370095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001661+0000","last_change":"2024-03-08T23:51:52.257586+0000","last_active":"2024-03-08T23:53:46.001661+0000","last_peered":"2024-03-08T23:53:46.001661+0000","last_clean":"2024-03-08T23:53:46.001661+0000","last_became_active":"2024-03-08T23:51:52.253741+0000","last_became_peered":"2024-03-08T23:51:52.253741+0000","last_unstale":"2024-03-08T23:53:46.001661+0000","last_undegraded":"2024-03-08T23:53:46.001661+0000","last_fullsized":"2024-03-08T23:53:46.001661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:18.633166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626404+0000","last_change":"2024-03-08T23:51:56.285264+0000","last_active":"2024-03-08T23:53:45.626404+0000","last_peered":"2024-03-08T23:53:45.626404+0000","last_clean":"2024-03-08T23:53:45.626404+0000","last_became_active":"2024-03-08T23:51:56.284727+0000","last_became_peered":"2024-03-08T23:51:56.284727+0000","last_unstale":"2024-03-08T23:53:45.626404+0000","last_undegraded":"2024-03-08T23:53:45.626404+0000","last_fullsized":"2024-03-08T23:53:45.626404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:01:20.723010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629149+0000","last_change":"2024-03-08T23:51:52.251461+0000","last_active":"2024-03-08T23:53:45.629149+0000","last_peered":"2024-03-08T23:53:45.629149+0000","last_clean":"2024-03-08T23:53:45.629149+0000","last_became_active":"2024-03-08T23:51:52.250996+0000","last_became_peered":"2024-03-08T23:51:52.250996+0000","last_unstale":"2024-03-08T23:53:45.629149+0000","last_undegraded":"2024-03-08T23:53:45.629149+0000","last_fullsized":"2024-03-08T23:53:45.629149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:51.719142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.524820+0000","last_change":"2024-03-08T23:51:51.245698+0000","last_active":"2024-03-08T23:54:18.524820+0000","last_peered":"2024-03-08T23:54:18.524820+0000","last_clean":"2024-03-08T23:54:18.524820+0000","last_became_active":"2024-03-08T23:51:51.245389+0000","last_became_peered":"2024-03-08T23:51:51.245389+0000","last_unstale":"2024-03-08T23:54:18.524820+0000","last_undegraded":"2024-03-08T23:54:18.524820+0000","last_fullsized":"2024-03-08T23:54:18.524820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:14:08.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002007+0000","last_change":"2024-03-08T23:52:45.423657+0000","last_active":"2024-03-08T23:53:46.002007+0000","last_peered":"2024-03-08T23:53:46.002007+0000","last_clean":"2024-03-08T23:53:46.002007+0000","last_became_active":"2024-03-08T23:52:45.417868+0000","last_became_peered":"2024-03-08T23:52:45.417868+0000","last_unstale":"2024-03-08T23:53:46.002007+0000","last_undegraded":"2024-03-08T23:53:46.002007+0000","last_fullsized":"2024-03-08T23:53:46.002007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:34.178938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636452+0000","last_change":"2024-03-08T23:51:52.256518+0000","last_active":"2024-03-08T23:53:45.636452+0000","last_peered":"2024-03-08T23:53:45.636452+0000","last_clean":"2024-03-08T23:53:45.636452+0000","last_became_active":"2024-03-08T23:51:52.253481+0000","last_became_peered":"2024-03-08T23:51:52.253481+0000","last_unstale":"2024-03-08T23:53:45.636452+0000","last_undegraded":"2024-03-08T23:53:45.636452+0000","last_fullsized":"2024-03-08T23:53:45.636452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:35.287408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628496+0000","last_change":"2024-03-08T23:51:51.237921+0000","last_active":"2024-03-08T23:53:45.628496+0000","last_peered":"2024-03-08T23:53:45.628496+0000","last_clean":"2024-03-08T23:53:45.628496+0000","last_became_active":"2024-03-08T23:51:51.237568+0000","last_became_peered":"2024-03-08T23:51:51.237568+0000","last_unstale":"2024-03-08T23:53:45.628496+0000","last_undegraded":"2024-03-08T23:53:45.628496+0000","last_fullsized":"2024-03-08T23:53:45.628496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:11.256814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000644+0000","last_change":"2024-03-08T23:50:41.802121+0000","last_active":"2024-03-08T23:53:46.000644+0000","last_peered":"2024-03-08T23:53:46.000644+0000","last_clean":"2024-03-08T23:53:46.000644+0000","last_became_active":"2024-03-08T23:50:41.801678+0000","last_became_peered":"2024-03-08T23:50:41.801678+0000","last_unstale":"2024-03-08T23:53:46.000644+0000","last_undegraded":"2024-03-08T23:53:46.000644+0000","last_fullsized":"2024-03-08T23:53:46.000644+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_clean_scrub_stamp":"2024-03-08T23:47:12.382547+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:15.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629562+0000","last_change":"2024-03-08T23:51:56.278465+0000","last_active":"2024-03-08T23:53:45.629562+0000","last_peered":"2024-03-08T23:53:45.629562+0000","last_clean":"2024-03-08T23:53:45.629562+0000","last_became_active":"2024-03-08T23:51:56.278079+0000","last_became_peered":"2024-03-08T23:51:56.278079+0000","last_unstale":"2024-03-08T23:53:45.629562+0000","last_undegraded":"2024-03-08T23:53:45.629562+0000","last_fullsized":"2024-03-08T23:53:45.629562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:47.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626180+0000","last_change":"2024-03-08T23:51:52.251895+0000","last_active":"2024-03-08T23:53:45.626180+0000","last_peered":"2024-03-08T23:53:45.626180+0000","last_clean":"2024-03-08T23:53:45.626180+0000","last_became_active":"2024-03-08T23:51:52.251591+0000","last_became_peered":"2024-03-08T23:51:52.251591+0000","last_unstale":"2024-03-08T23:53:45.626180+0000","last_undegraded":"2024-03-08T23:53:45.626180+0000","last_fullsized":"2024-03-08T23:53:45.626180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:06.276636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001022+0000","last_change":"2024-03-08T23:51:51.245981+0000","last_active":"2024-03-08T23:53:46.001022+0000","last_peered":"2024-03-08T23:53:46.001022+0000","last_clean":"2024-03-08T23:53:46.001022+0000","last_became_active":"2024-03-08T23:51:51.244681+0000","last_became_peered":"2024-03-08T23:51:51.244681+0000","last_unstale":"2024-03-08T23:53:46.001022+0000","last_undegraded":"2024-03-08T23:53:46.001022+0000","last_fullsized":"2024-03-08T23:53:46.001022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:06:21.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.631890+0000","last_change":"2024-03-08T23:51:56.279057+0000","last_active":"2024-03-08T23:53:45.631890+0000","last_peered":"2024-03-08T23:53:45.631890+0000","last_clean":"2024-03-08T23:53:45.631890+0000","last_became_active":"2024-03-08T23:51:56.278634+0000","last_became_peered":"2024-03-08T23:51:56.278634+0000","last_unstale":"2024-03-08T23:53:45.631890+0000","last_undegraded":"2024-03-08T23:53:45.631890+0000","last_fullsized":"2024-03-08T23:53:45.631890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:30.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638685+0000","last_change":"2024-03-08T23:51:51.240378+0000","last_active":"2024-03-08T23:53:45.638685+0000","last_peered":"2024-03-08T23:53:45.638685+0000","last_clean":"2024-03-08T23:53:45.638685+0000","last_became_active":"2024-03-08T23:51:51.239880+0000","last_became_peered":"2024-03-08T23:51:51.239880+0000","last_unstale":"2024-03-08T23:53:45.638685+0000","last_undegraded":"2024-03-08T23:53:45.638685+0000","last_fullsized":"2024-03-08T23:53:45.638685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:20.804261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626738+0000","last_change":"2024-03-08T23:51:52.240338+0000","last_active":"2024-03-08T23:53:45.626738+0000","last_peered":"2024-03-08T23:53:45.626738+0000","last_clean":"2024-03-08T23:53:45.626738+0000","last_became_active":"2024-03-08T23:51:52.239987+0000","last_became_peered":"2024-03-08T23:51:52.239987+0000","last_unstale":"2024-03-08T23:53:45.626738+0000","last_undegraded":"2024-03-08T23:53:45.626738+0000","last_fullsized":"2024-03-08T23:53:45.626738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:26.590204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636290+0000","last_change":"2024-03-08T23:51:56.281009+0000","last_active":"2024-03-08T23:53:45.636290+0000","last_peered":"2024-03-08T23:53:45.636290+0000","last_clean":"2024-03-08T23:53:45.636290+0000","last_became_active":"2024-03-08T23:51:56.280619+0000","last_became_peered":"2024-03-08T23:51:56.280619+0000","last_unstale":"2024-03-08T23:53:45.636290+0000","last_undegraded":"2024-03-08T23:53:45.636290+0000","last_fullsized":"2024-03-08T23:53:45.636290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:06.920777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:17.267361+0000","last_change":"2024-03-08T23:51:51.241682+0000","last_active":"2024-03-08T23:54:17.267361+0000","last_peered":"2024-03-08T23:54:17.267361+0000","last_clean":"2024-03-08T23:54:17.267361+0000","last_became_active":"2024-03-08T23:51:51.241313+0000","last_became_peered":"2024-03-08T23:51:51.241313+0000","last_unstale":"2024-03-08T23:54:17.267361+0000","last_undegraded":"2024-03-08T23:54:17.267361+0000","last_fullsized":"2024-03-08T23:54:17.267361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:04:13.515856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636824+0000","last_change":"2024-03-08T23:51:52.249378+0000","last_active":"2024-03-08T23:53:45.636824+0000","last_peered":"2024-03-08T23:53:45.636824+0000","last_clean":"2024-03-08T23:53:45.636824+0000","last_became_active":"2024-03-08T23:51:52.249084+0000","last_became_peered":"2024-03-08T23:51:52.249084+0000","last_unstale":"2024-03-08T23:53:45.636824+0000","last_undegraded":"2024-03-08T23:53:45.636824+0000","last_fullsized":"2024-03-08T23:53:45.636824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:10:36.513260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638049+0000","last_change":"2024-03-08T23:51:56.280717+0000","last_active":"2024-03-08T23:53:45.638049+0000","last_peered":"2024-03-08T23:53:45.638049+0000","last_clean":"2024-03-08T23:53:45.638049+0000","last_became_active":"2024-03-08T23:51:56.280329+0000","last_became_peered":"2024-03-08T23:51:56.280329+0000","last_unstale":"2024-03-08T23:53:45.638049+0000","last_undegraded":"2024-03-08T23:53:45.638049+0000","last_fullsized":"2024-03-08T23:53:45.638049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:47:40.829142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629763+0000","last_change":"2024-03-08T23:51:51.238058+0000","last_active":"2024-03-08T23:53:45.629763+0000","last_peered":"2024-03-08T23:53:45.629763+0000","last_clean":"2024-03-08T23:53:45.629763+0000","last_became_active":"2024-03-08T23:51:51.237727+0000","last_became_peered":"2024-03-08T23:51:51.237727+0000","last_unstale":"2024-03-08T23:53:45.629763+0000","last_undegraded":"2024-03-08T23:53:45.629763+0000","last_fullsized":"2024-03-08T23:53:45.629763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:13:54.080004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000263+0000","last_change":"2024-03-08T23:51:52.249830+0000","last_active":"2024-03-08T23:53:46.000263+0000","last_peered":"2024-03-08T23:53:46.000263+0000","last_clean":"2024-03-08T23:53:46.000263+0000","last_became_active":"2024-03-08T23:51:52.249518+0000","last_became_peered":"2024-03-08T23:51:52.249518+0000","last_unstale":"2024-03-08T23:53:46.000263+0000","last_undegraded":"2024-03-08T23:53:46.000263+0000","last_fullsized":"2024-03-08T23:53:46.000263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:24:33.721423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629649+0000","last_change":"2024-03-08T23:51:56.288595+0000","last_active":"2024-03-08T23:53:45.629649+0000","last_peered":"2024-03-08T23:53:45.629649+0000","last_clean":"2024-03-08T23:53:45.629649+0000","last_became_active":"2024-03-08T23:51:56.288146+0000","last_became_peered":"2024-03-08T23:51:56.288146+0000","last_unstale":"2024-03-08T23:53:45.629649+0000","last_undegraded":"2024-03-08T23:53:45.629649+0000","last_fullsized":"2024-03-08T23:53:45.629649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:34:58.321754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000190+0000","last_change":"2024-03-08T23:51:51.242065+0000","last_active":"2024-03-08T23:53:46.000190+0000","last_peered":"2024-03-08T23:53:46.000190+0000","last_clean":"2024-03-08T23:53:46.000190+0000","last_became_active":"2024-03-08T23:51:51.241513+0000","last_became_peered":"2024-03-08T23:51:51.241513+0000","last_unstale":"2024-03-08T23:53:46.000190+0000","last_undegraded":"2024-03-08T23:53:46.000190+0000","last_fullsized":"2024-03-08T23:53:46.000190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:08:02.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637325+0000","last_change":"2024-03-08T23:51:52.243074+0000","last_active":"2024-03-08T23:53:45.637325+0000","last_peered":"2024-03-08T23:53:45.637325+0000","last_clean":"2024-03-08T23:53:45.637325+0000","last_became_active":"2024-03-08T23:51:52.242736+0000","last_became_peered":"2024-03-08T23:51:52.242736+0000","last_unstale":"2024-03-08T23:53:45.637325+0000","last_undegraded":"2024-03-08T23:53:45.637325+0000","last_fullsized":"2024-03-08T23:53:45.637325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:57:37.174056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875547+0000","last_change":"2024-03-08T23:51:56.276284+0000","last_active":"2024-03-08T23:53:45.875547+0000","last_peered":"2024-03-08T23:53:45.875547+0000","last_clean":"2024-03-08T23:53:45.875547+0000","last_became_active":"2024-03-08T23:51:56.275800+0000","last_became_peered":"2024-03-08T23:51:56.275800+0000","last_unstale":"2024-03-08T23:53:45.875547+0000","last_undegraded":"2024-03-08T23:53:45.875547+0000","last_fullsized":"2024-03-08T23:53:45.875547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:21:11.221266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636532+0000","last_change":"2024-03-08T23:51:51.245548+0000","last_active":"2024-03-08T23:53:45.636532+0000","last_peered":"2024-03-08T23:53:45.636532+0000","last_clean":"2024-03-08T23:53:45.636532+0000","last_became_active":"2024-03-08T23:51:51.245238+0000","last_became_peered":"2024-03-08T23:51:51.245238+0000","last_unstale":"2024-03-08T23:53:45.636532+0000","last_undegraded":"2024-03-08T23:53:45.636532+0000","last_fullsized":"2024-03-08T23:53:45.636532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:21:19.557196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.635139+0000","last_change":"2024-03-08T23:51:52.244422+0000","last_active":"2024-03-08T23:53:45.635139+0000","last_peered":"2024-03-08T23:53:45.635139+0000","last_clean":"2024-03-08T23:53:45.635139+0000","last_became_active":"2024-03-08T23:51:52.244032+0000","last_became_peered":"2024-03-08T23:51:52.244032+0000","last_unstale":"2024-03-08T23:53:45.635139+0000","last_undegraded":"2024-03-08T23:53:45.635139+0000","last_fullsized":"2024-03-08T23:53:45.635139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:04.064691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.630134+0000","last_change":"2024-03-08T23:51:56.288632+0000","last_active":"2024-03-08T23:53:45.630134+0000","last_peered":"2024-03-08T23:53:45.630134+0000","last_clean":"2024-03-08T23:53:45.630134+0000","last_became_active":"2024-03-08T23:51:56.288175+0000","last_became_peered":"2024-03-08T23:51:56.288175+0000","last_unstale":"2024-03-08T23:53:45.630134+0000","last_undegraded":"2024-03-08T23:53:45.630134+0000","last_fullsized":"2024-03-08T23:53:45.630134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:54:51.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.645297+0000","last_change":"2024-03-08T23:53:45.645297+0000","last_active":"2024-03-08T23:53:45.645297+0000","last_peered":"2024-03-08T23:53:45.645297+0000","last_clean":"2024-03-08T23:53:45.645297+0000","last_became_active":"2024-03-08T23:53:45.639667+0000","last_became_peered":"2024-03-08T23:53:45.639667+0000","last_unstale":"2024-03-08T23:53:45.645297+0000","last_undegraded":"2024-03-08T23:53:45.645297+0000","last_fullsized":"2024-03-08T23:53:45.645297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:30:30.658165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636550+0000","last_change":"2024-03-08T23:51:52.250778+0000","last_active":"2024-03-08T23:53:45.636550+0000","last_peered":"2024-03-08T23:53:45.636550+0000","last_clean":"2024-03-08T23:53:45.636550+0000","last_became_active":"2024-03-08T23:51:52.250475+0000","last_became_peered":"2024-03-08T23:51:52.250475+0000","last_unstale":"2024-03-08T23:53:45.636550+0000","last_undegraded":"2024-03-08T23:53:45.636550+0000","last_fullsized":"2024-03-08T23:53:45.636550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:36:01.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629535+0000","last_change":"2024-03-08T23:51:56.285631+0000","last_active":"2024-03-08T23:53:45.629535+0000","last_peered":"2024-03-08T23:53:45.629535+0000","last_clean":"2024-03-08T23:53:45.629535+0000","last_became_active":"2024-03-08T23:51:56.285183+0000","last_became_peered":"2024-03-08T23:51:56.285183+0000","last_unstale":"2024-03-08T23:53:45.629535+0000","last_undegraded":"2024-03-08T23:53:45.629535+0000","last_fullsized":"2024-03-08T23:53:45.629535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:15.002086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637835+0000","last_change":"2024-03-08T23:51:51.246848+0000","last_active":"2024-03-08T23:53:45.637835+0000","last_peered":"2024-03-08T23:53:45.637835+0000","last_clean":"2024-03-08T23:53:45.637835+0000","last_became_active":"2024-03-08T23:51:51.246533+0000","last_became_peered":"2024-03-08T23:51:51.246533+0000","last_unstale":"2024-03-08T23:53:45.637835+0000","last_undegraded":"2024-03-08T23:53:45.637835+0000","last_fullsized":"2024-03-08T23:53:45.637835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:20:16.669820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002055+0000","last_change":"2024-03-08T23:51:52.251575+0000","last_active":"2024-03-08T23:53:46.002055+0000","last_peered":"2024-03-08T23:53:46.002055+0000","last_clean":"2024-03-08T23:53:46.002055+0000","last_became_active":"2024-03-08T23:51:52.251232+0000","last_became_peered":"2024-03-08T23:51:52.251232+0000","last_unstale":"2024-03-08T23:53:46.002055+0000","last_undegraded":"2024-03-08T23:53:46.002055+0000","last_fullsized":"2024-03-08T23:53:46.002055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:33.573407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000476+0000","last_change":"2024-03-08T23:52:45.586092+0000","last_active":"2024-03-08T23:53:46.000476+0000","last_peered":"2024-03-08T23:53:46.000476+0000","last_clean":"2024-03-08T23:53:46.000476+0000","last_became_active":"2024-03-08T23:52:45.585513+0000","last_became_peered":"2024-03-08T23:52:45.585513+0000","last_unstale":"2024-03-08T23:53:46.000476+0000","last_undegraded":"2024-03-08T23:53:46.000476+0000","last_fullsized":"2024-03-08T23:53:46.000476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:39:05.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638551+0000","last_change":"2024-03-08T23:51:51.242081+0000","last_active":"2024-03-08T23:53:45.638551+0000","last_peered":"2024-03-08T23:53:45.638551+0000","last_clean":"2024-03-08T23:53:45.638551+0000","last_became_active":"2024-03-08T23:51:51.241636+0000","last_became_peered":"2024-03-08T23:51:51.241636+0000","last_unstale":"2024-03-08T23:53:45.638551+0000","last_undegraded":"2024-03-08T23:53:45.638551+0000","last_fullsized":"2024-03-08T23:53:45.638551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:51:57.497723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002288+0000","last_change":"2024-03-08T23:51:52.252796+0000","last_active":"2024-03-08T23:53:46.002288+0000","last_peered":"2024-03-08T23:53:46.002288+0000","last_clean":"2024-03-08T23:53:46.002288+0000","last_became_active":"2024-03-08T23:51:52.252361+0000","last_became_peered":"2024-03-08T23:51:52.252361+0000","last_unstale":"2024-03-08T23:53:46.002288+0000","last_undegraded":"2024-03-08T23:53:46.002288+0000","last_fullsized":"2024-03-08T23:53:46.002288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:07.968262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999883+0000","last_change":"2024-03-08T23:51:56.284396+0000","last_active":"2024-03-08T23:53:45.999883+0000","last_peered":"2024-03-08T23:53:45.999883+0000","last_clean":"2024-03-08T23:53:45.999883+0000","last_became_active":"2024-03-08T23:51:56.283951+0000","last_became_peered":"2024-03-08T23:51:56.283951+0000","last_unstale":"2024-03-08T23:53:45.999883+0000","last_undegraded":"2024-03-08T23:53:45.999883+0000","last_fullsized":"2024-03-08T23:53:45.999883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:09:40.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629375+0000","last_change":"2024-03-08T23:51:51.248838+0000","last_active":"2024-03-08T23:53:45.629375+0000","last_peered":"2024-03-08T23:53:45.629375+0000","last_clean":"2024-03-08T23:53:45.629375+0000","last_became_active":"2024-03-08T23:51:51.248534+0000","last_became_peered":"2024-03-08T23:51:51.248534+0000","last_unstale":"2024-03-08T23:53:45.629375+0000","last_undegraded":"2024-03-08T23:53:45.629375+0000","last_fullsized":"2024-03-08T23:53:45.629375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:37:25.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001834+0000","last_change":"2024-03-08T23:51:52.264170+0000","last_active":"2024-03-08T23:53:46.001834+0000","last_peered":"2024-03-08T23:53:46.001834+0000","last_clean":"2024-03-08T23:53:46.001834+0000","last_became_active":"2024-03-08T23:51:52.263744+0000","last_became_peered":"2024-03-08T23:51:52.263744+0000","last_unstale":"2024-03-08T23:53:46.001834+0000","last_undegraded":"2024-03-08T23:53:46.001834+0000","last_fullsized":"2024-03-08T23:53:46.001834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:42:18.287699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628650+0000","last_change":"2024-03-08T23:51:56.283316+0000","last_active":"2024-03-08T23:53:45.628650+0000","last_peered":"2024-03-08T23:53:45.628650+0000","last_clean":"2024-03-08T23:53:45.628650+0000","last_became_active":"2024-03-08T23:51:56.282515+0000","last_became_peered":"2024-03-08T23:51:56.282515+0000","last_unstale":"2024-03-08T23:53:45.628650+0000","last_undegraded":"2024-03-08T23:53:45.628650+0000","last_fullsized":"2024-03-08T23:53:45.628650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:40:15.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626731+0000","last_change":"2024-03-08T23:51:51.240970+0000","last_active":"2024-03-08T23:53:45.626731+0000","last_peered":"2024-03-08T23:53:45.626731+0000","last_clean":"2024-03-08T23:53:45.626731+0000","last_became_active":"2024-03-08T23:51:51.239982+0000","last_became_peered":"2024-03-08T23:51:51.239982+0000","last_unstale":"2024-03-08T23:53:45.626731+0000","last_undegraded":"2024-03-08T23:53:45.626731+0000","last_fullsized":"2024-03-08T23:53:45.626731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:47:40.313753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002207+0000","last_change":"2024-03-08T23:51:52.261673+0000","last_active":"2024-03-08T23:53:46.002207+0000","last_peered":"2024-03-08T23:53:46.002207+0000","last_clean":"2024-03-08T23:53:46.002207+0000","last_became_active":"2024-03-08T23:51:52.261333+0000","last_became_peered":"2024-03-08T23:51:52.261333+0000","last_unstale":"2024-03-08T23:53:46.002207+0000","last_undegraded":"2024-03-08T23:53:46.002207+0000","last_fullsized":"2024-03-08T23:53:46.002207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:08:26.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.635064+0000","last_change":"2024-03-08T23:51:52.243189+0000","last_active":"2024-03-08T23:53:45.635064+0000","last_peered":"2024-03-08T23:53:45.635064+0000","last_clean":"2024-03-08T23:53:45.635064+0000","last_became_active":"2024-03-08T23:51:52.242870+0000","last_became_peered":"2024-03-08T23:51:52.242870+0000","last_unstale":"2024-03-08T23:53:45.635064+0000","last_undegraded":"2024-03-08T23:53:45.635064+0000","last_fullsized":"2024-03-08T23:53:45.635064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.438045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638156+0000","last_change":"2024-03-08T23:52:45.417220+0000","last_active":"2024-03-08T23:53:45.638156+0000","last_peered":"2024-03-08T23:53:45.638156+0000","last_clean":"2024-03-08T23:53:45.638156+0000","last_became_active":"2024-03-08T23:52:45.416820+0000","last_became_peered":"2024-03-08T23:52:45.416820+0000","last_unstale":"2024-03-08T23:53:45.638156+0000","last_undegraded":"2024-03-08T23:53:45.638156+0000","last_fullsized":"2024-03-08T23:53:45.638156+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:31:13.249449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637319+0000","last_change":"2024-03-08T23:51:52.251397+0000","last_active":"2024-03-08T23:53:45.637319+0000","last_peered":"2024-03-08T23:53:45.637319+0000","last_clean":"2024-03-08T23:53:45.637319+0000","last_became_active":"2024-03-08T23:51:52.251059+0000","last_became_peered":"2024-03-08T23:51:52.251059+0000","last_unstale":"2024-03-08T23:53:45.637319+0000","last_undegraded":"2024-03-08T23:53:45.637319+0000","last_fullsized":"2024-03-08T23:53:45.637319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:19:30.363457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000034+0000","last_change":"2024-03-08T23:51:52.249039+0000","last_active":"2024-03-08T23:53:46.000034+0000","last_peered":"2024-03-08T23:53:46.000034+0000","last_clean":"2024-03-08T23:53:46.000034+0000","last_became_active":"2024-03-08T23:51:52.248560+0000","last_became_peered":"2024-03-08T23:51:52.248560+0000","last_unstale":"2024-03-08T23:53:46.000034+0000","last_undegraded":"2024-03-08T23:53:46.000034+0000","last_fullsized":"2024-03-08T23:53:46.000034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626651+0000","last_change":"2024-03-08T23:51:52.247269+0000","last_active":"2024-03-08T23:53:45.626651+0000","last_peered":"2024-03-08T23:53:45.626651+0000","last_clean":"2024-03-08T23:53:45.626651+0000","last_became_active":"2024-03-08T23:51:52.246921+0000","last_became_peered":"2024-03-08T23:51:52.246921+0000","last_unstale":"2024-03-08T23:53:45.626651+0000","last_undegraded":"2024-03-08T23:53:45.626651+0000","last_fullsized":"2024-03-08T23:53:45.626651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:00:21.828530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633238+0000","last_change":"2024-03-08T23:51:52.238999+0000","last_active":"2024-03-08T23:53:45.633238+0000","last_peered":"2024-03-08T23:53:45.633238+0000","last_clean":"2024-03-08T23:53:45.633238+0000","last_became_active":"2024-03-08T23:51:52.238662+0000","last_became_peered":"2024-03-08T23:51:52.238662+0000","last_unstale":"2024-03-08T23:53:45.633238+0000","last_undegraded":"2024-03-08T23:53:45.633238+0000","last_fullsized":"2024-03-08T23:53:45.633238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:22:08.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637860+0000","last_change":"2024-03-08T23:51:52.238793+0000","last_active":"2024-03-08T23:53:45.637860+0000","last_peered":"2024-03-08T23:53:45.637860+0000","last_clean":"2024-03-08T23:53:45.637860+0000","last_became_active":"2024-03-08T23:51:52.238464+0000","last_became_peered":"2024-03-08T23:51:52.238464+0000","last_unstale":"2024-03-08T23:53:45.637860+0000","last_undegraded":"2024-03-08T23:53:45.637860+0000","last_fullsized":"2024-03-08T23:53:45.637860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:25.597428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636926+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:53:45.636926+0000","last_peered":"2024-03-08T23:53:45.636926+0000","last_clean":"2024-03-08T23:53:45.636926+0000","last_became_active":"2024-03-08T23:52:45.585280+0000","last_became_peered":"2024-03-08T23:52:45.585280+0000","last_unstale":"2024-03-08T23:53:45.636926+0000","last_undegraded":"2024-03-08T23:53:45.636926+0000","last_fullsized":"2024-03-08T23:53:45.636926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:20.029967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629208+0000","last_change":"2024-03-08T23:51:52.245288+0000","last_active":"2024-03-08T23:53:45.629208+0000","last_peered":"2024-03-08T23:53:45.629208+0000","last_clean":"2024-03-08T23:53:45.629208+0000","last_became_active":"2024-03-08T23:51:52.244960+0000","last_became_peered":"2024-03-08T23:51:52.244960+0000","last_unstale":"2024-03-08T23:53:45.629208+0000","last_undegraded":"2024-03-08T23:53:45.629208+0000","last_fullsized":"2024-03-08T23:53:45.629208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:20:39.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.874977+0000","last_change":"2024-03-08T23:52:45.418172+0000","last_active":"2024-03-08T23:53:45.874977+0000","last_peered":"2024-03-08T23:53:45.874977+0000","last_clean":"2024-03-08T23:53:45.874977+0000","last_became_active":"2024-03-08T23:52:45.417852+0000","last_became_peered":"2024-03-08T23:52:45.417852+0000","last_unstale":"2024-03-08T23:53:45.874977+0000","last_undegraded":"2024-03-08T23:53:45.874977+0000","last_fullsized":"2024-03-08T23:53:45.874977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:22:23.698238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000860+0000","last_change":"2024-03-08T23:51:52.251048+0000","last_active":"2024-03-08T23:53:46.000860+0000","last_peered":"2024-03-08T23:53:46.000860+0000","last_clean":"2024-03-08T23:53:46.000860+0000","last_became_active":"2024-03-08T23:51:52.250508+0000","last_became_peered":"2024-03-08T23:51:52.250508+0000","last_unstale":"2024-03-08T23:53:46.000860+0000","last_undegraded":"2024-03-08T23:53:46.000860+0000","last_fullsized":"2024-03-08T23:53:46.000860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:57:37.519307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626525+0000","last_change":"2024-03-08T23:51:52.244203+0000","last_active":"2024-03-08T23:53:45.626525+0000","last_peered":"2024-03-08T23:53:45.626525+0000","last_clean":"2024-03-08T23:53:45.626525+0000","last_became_active":"2024-03-08T23:51:52.243878+0000","last_became_peered":"2024-03-08T23:51:52.243878+0000","last_unstale":"2024-03-08T23:53:45.626525+0000","last_undegraded":"2024-03-08T23:53:45.626525+0000","last_fullsized":"2024-03-08T23:53:45.626525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:53:40.938657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633137+0000","last_change":"2024-03-08T23:51:52.248334+0000","last_active":"2024-03-08T23:53:45.633137+0000","last_peered":"2024-03-08T23:53:45.633137+0000","last_clean":"2024-03-08T23:53:45.633137+0000","last_became_active":"2024-03-08T23:51:52.248025+0000","last_became_peered":"2024-03-08T23:51:52.248025+0000","last_unstale":"2024-03-08T23:53:45.633137+0000","last_undegraded":"2024-03-08T23:53:45.633137+0000","last_fullsized":"2024-03-08T23:53:45.633137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:02:06.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74027,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":746,"num_read_kb":452,"num_write":182,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":60703,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":222863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":519611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1180838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":519825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":519723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1117417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":583565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":517184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":583381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1111383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T23:54:21.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T23:54:22.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/1219607007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48100210 msgr2=0x7f0b48100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/1219607007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48100210 0x7f0b48100630 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009900 tx=0x7f0b4002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/1219607007 shutdown_connections 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/1219607007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b48101dc0 0x7f0b48108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/1219607007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b48101400 0x7f0b48101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/1219607007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48100210 0x7f0b48100630 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/1219607007 >> 172.21.15.22:0/1219607007 conn(0x7f0b480fb780 msgr2=0x7f0b480fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/1219607007 shutdown_connections 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.051+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/1219607007 wait complete. 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 Processor -- start 2024-03-08T23:54:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 -- start start 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b48100210 0x7f0b48197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b48101400 0x7f0b48197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48101dc0 0x7f0b4819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4810bf00 con 0x7f0b48101dc0 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4810bd80 con 0x7f0b48100210 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b4c9d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4810c080 con 0x7f0b48101400 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.052+0000 7f0b46d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48101dc0 0x7f0b4819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48101dc0 0x7f0b4819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51904/0 (socket says 172.21.15.22:51904) 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 -- 172.21.15.22:0/4170248348 learned_addr learned my addr 172.21.15.22:0/4170248348 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b4659c640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b48100210 0x7f0b48197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b45d9b640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b48101400 0x7f0b48197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 -- 172.21.15.22:0/4170248348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b48101400 msgr2=0x7f0b48197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b48101400 0x7f0b48197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 -- 172.21.15.22:0/4170248348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b48100210 msgr2=0x7f0b48197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b48100210 0x7f0b48197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 -- 172.21.15.22:0/4170248348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4819f300 con 0x7f0b48101dc0 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b46d9d640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48101dc0 0x7f0b4819ec90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800b970 tx=0x7f0b3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b3f7fe640 1 -- 172.21.15.22:0/4170248348 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38019070 con 0x7f0b48101dc0 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4819f5f0 con 0x7f0b48101dc0 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b3f7fe640 1 -- 172.21.15.22:0/4170248348 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b3800cb40 con 0x7f0b48101dc0 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b3f7fe640 1 -- 172.21.15.22:0/4170248348 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38007650 con 0x7f0b48101dc0 2024-03-08T23:54:22.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.053+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4819faf0 con 0x7f0b48101dc0 2024-03-08T23:54:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.054+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4819fdb0 con 0x7f0b48101dc0 2024-03-08T23:54:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.055+0000 7f0b3f7fe640 1 -- 172.21.15.22:0/4170248348 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f0b3801c070 con 0x7f0b48101dc0 2024-03-08T23:54:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.056+0000 7f0b3f7fe640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0b340a2890 0x7f0b340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.056+0000 7f0b4659c640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0b340a2890 0x7f0b340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.056+0000 7f0b3f7fe640 1 -- 172.21.15.22:0/4170248348 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0b380ce550 con 0x7f0b48101dc0 2024-03-08T23:54:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.056+0000 7f0b4659c640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0b340a2890 0x7f0b340a4d50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b4002c7b0 tx=0x7f0b40000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:22.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.060+0000 7f0b3f7fe640 1 -- 172.21.15.22:0/4170248348 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0b3809b1b0 con 0x7f0b48101dc0 2024-03-08T23:54:22.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:21 smithi062 ceph-mon[36129]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-08T23:54:22.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:21 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3529120916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:22.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:21 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2181573082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:22.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:21 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/934699217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:22.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:21 smithi192 ceph-mon[35995]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-08T23:54:22.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:21 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3529120916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:22.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:21 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2181573082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:22.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:21 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/934699217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:22.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.239+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b4819ff80 con 0x7f0b340a2890 2024-03-08T23:54:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.248+0000 7f0b3f7fe640 1 -- 172.21.15.22:0/4170248348 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340501 (secure 0 0 0) 0x7f0b4819ff80 con 0x7f0b340a2890 2024-03-08T23:54:22.248 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:22.250 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0b340a2890 msgr2=0x7f0b340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0b340a2890 0x7f0b340a4d50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b4002c7b0 tx=0x7f0b40000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48101dc0 msgr2=0x7f0b4819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48101dc0 0x7f0b4819ec90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800b970 tx=0x7f0b3800be30 comp rx=0 tx=0).stop 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 shutdown_connections 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f0b340a2890 0x7f0b340a4d50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b48101dc0 0x7f0b4819ec90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b48101400 0x7f0b48197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 --2- 172.21.15.22:0/4170248348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b48100210 0x7f0b48197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 >> 172.21.15.22:0/4170248348 conn(0x7f0b480fb780 msgr2=0x7f0b480fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 shutdown_connections 2024-03-08T23:54:22.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.252+0000 7f0b4c9d4640 1 -- 172.21.15.22:0/4170248348 wait complete. 2024-03-08T23:54:22.259 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-03-08T23:54:21.826311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664629,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":882,"num_read_kb":571,"num_write":365,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1241556,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363676,"kb_used_data":14488,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578148,"statfs":{"total":1151940427776,"available":1151568023552,"internally_reserved":0,"allocated":14835712,"data_stored":8199512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6771,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":38,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003064"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633752+0000","last_change":"2024-03-08T23:51:52.246270+0000","last_active":"2024-03-08T23:53:45.633752+0000","last_peered":"2024-03-08T23:53:45.633752+0000","last_clean":"2024-03-08T23:53:45.633752+0000","last_became_active":"2024-03-08T23:51:52.245321+0000","last_became_peered":"2024-03-08T23:51:52.245321+0000","last_unstale":"2024-03-08T23:53:45.633752+0000","last_undegraded":"2024-03-08T23:53:45.633752+0000","last_fullsized":"2024-03-08T23:53:45.633752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:39.993965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000208+0000","last_change":"2024-03-08T23:51:52.245433+0000","last_active":"2024-03-08T23:53:46.000208+0000","last_peered":"2024-03-08T23:53:46.000208+0000","last_clean":"2024-03-08T23:53:46.000208+0000","last_became_active":"2024-03-08T23:51:52.245106+0000","last_became_peered":"2024-03-08T23:51:52.245106+0000","last_unstale":"2024-03-08T23:53:46.000208+0000","last_undegraded":"2024-03-08T23:53:46.000208+0000","last_fullsized":"2024-03-08T23:53:46.000208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:02:20.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000027+0000","last_change":"2024-03-08T23:51:52.244112+0000","last_active":"2024-03-08T23:53:46.000027+0000","last_peered":"2024-03-08T23:53:46.000027+0000","last_clean":"2024-03-08T23:53:46.000027+0000","last_became_active":"2024-03-08T23:51:52.243783+0000","last_became_peered":"2024-03-08T23:51:52.243783+0000","last_unstale":"2024-03-08T23:53:46.000027+0000","last_undegraded":"2024-03-08T23:53:46.000027+0000","last_fullsized":"2024-03-08T23:53:46.000027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:48:43.719188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875262+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:53:45.875262+0000","last_peered":"2024-03-08T23:53:45.875262+0000","last_clean":"2024-03-08T23:53:45.875262+0000","last_became_active":"2024-03-08T23:52:45.585292+0000","last_became_peered":"2024-03-08T23:52:45.585292+0000","last_unstale":"2024-03-08T23:53:45.875262+0000","last_undegraded":"2024-03-08T23:53:45.875262+0000","last_fullsized":"2024-03-08T23:53:45.875262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:27:12.230920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637664+0000","last_change":"2024-03-08T23:51:52.247091+0000","last_active":"2024-03-08T23:53:45.637664+0000","last_peered":"2024-03-08T23:53:45.637664+0000","last_clean":"2024-03-08T23:53:45.637664+0000","last_became_active":"2024-03-08T23:51:52.246761+0000","last_became_peered":"2024-03-08T23:51:52.246761+0000","last_unstale":"2024-03-08T23:53:45.637664+0000","last_undegraded":"2024-03-08T23:53:45.637664+0000","last_fullsized":"2024-03-08T23:53:45.637664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:46:46.810857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001715+0000","last_change":"2024-03-08T23:51:52.257558+0000","last_active":"2024-03-08T23:53:46.001715+0000","last_peered":"2024-03-08T23:53:46.001715+0000","last_clean":"2024-03-08T23:53:46.001715+0000","last_became_active":"2024-03-08T23:51:52.257116+0000","last_became_peered":"2024-03-08T23:51:52.257116+0000","last_unstale":"2024-03-08T23:53:46.001715+0000","last_undegraded":"2024-03-08T23:53:46.001715+0000","last_fullsized":"2024-03-08T23:53:46.001715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:13:05.325780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000412+0000","last_change":"2024-03-08T23:51:52.253624+0000","last_active":"2024-03-08T23:53:46.000412+0000","last_peered":"2024-03-08T23:53:46.000412+0000","last_clean":"2024-03-08T23:53:46.000412+0000","last_became_active":"2024-03-08T23:51:52.253284+0000","last_became_peered":"2024-03-08T23:51:52.253284+0000","last_unstale":"2024-03-08T23:53:46.000412+0000","last_undegraded":"2024-03-08T23:53:46.000412+0000","last_fullsized":"2024-03-08T23:53:46.000412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:09:33.789502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636159+0000","last_change":"2024-03-08T23:51:52.255321+0000","last_active":"2024-03-08T23:53:45.636159+0000","last_peered":"2024-03-08T23:53:45.636159+0000","last_clean":"2024-03-08T23:53:45.636159+0000","last_became_active":"2024-03-08T23:51:52.254975+0000","last_became_peered":"2024-03-08T23:51:52.254975+0000","last_unstale":"2024-03-08T23:53:45.636159+0000","last_undegraded":"2024-03-08T23:53:45.636159+0000","last_fullsized":"2024-03-08T23:53:45.636159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:02:56.788519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001266+0000","last_change":"2024-03-08T23:51:52.244623+0000","last_active":"2024-03-08T23:53:46.001266+0000","last_peered":"2024-03-08T23:53:46.001266+0000","last_clean":"2024-03-08T23:53:46.001266+0000","last_became_active":"2024-03-08T23:51:52.244162+0000","last_became_peered":"2024-03-08T23:51:52.244162+0000","last_unstale":"2024-03-08T23:53:46.001266+0000","last_undegraded":"2024-03-08T23:53:46.001266+0000","last_fullsized":"2024-03-08T23:53:46.001266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:30:43.050616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001879+0000","last_change":"2024-03-08T23:51:52.258006+0000","last_active":"2024-03-08T23:53:46.001879+0000","last_peered":"2024-03-08T23:53:46.001879+0000","last_clean":"2024-03-08T23:53:46.001879+0000","last_became_active":"2024-03-08T23:51:52.256770+0000","last_became_peered":"2024-03-08T23:51:52.256770+0000","last_unstale":"2024-03-08T23:53:46.001879+0000","last_undegraded":"2024-03-08T23:53:46.001879+0000","last_fullsized":"2024-03-08T23:53:46.001879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:18.584212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002234+0000","last_change":"2024-03-08T23:51:52.260530+0000","last_active":"2024-03-08T23:53:46.002234+0000","last_peered":"2024-03-08T23:53:46.002234+0000","last_clean":"2024-03-08T23:53:46.002234+0000","last_became_active":"2024-03-08T23:51:52.260193+0000","last_became_peered":"2024-03-08T23:51:52.260193+0000","last_unstale":"2024-03-08T23:53:46.002234+0000","last_undegraded":"2024-03-08T23:53:46.002234+0000","last_fullsized":"2024-03-08T23:53:46.002234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:02.549683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629380+0000","last_change":"2024-03-08T23:51:52.248679+0000","last_active":"2024-03-08T23:53:45.629380+0000","last_peered":"2024-03-08T23:53:45.629380+0000","last_clean":"2024-03-08T23:53:45.629380+0000","last_became_active":"2024-03-08T23:51:52.248358+0000","last_became_peered":"2024-03-08T23:51:52.248358+0000","last_unstale":"2024-03-08T23:53:45.629380+0000","last_undegraded":"2024-03-08T23:53:45.629380+0000","last_fullsized":"2024-03-08T23:53:45.629380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:26.680887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626733+0000","last_change":"2024-03-08T23:52:45.415807+0000","last_active":"2024-03-08T23:53:45.626733+0000","last_peered":"2024-03-08T23:53:45.626733+0000","last_clean":"2024-03-08T23:53:45.626733+0000","last_became_active":"2024-03-08T23:52:45.415456+0000","last_became_peered":"2024-03-08T23:52:45.415456+0000","last_unstale":"2024-03-08T23:53:45.626733+0000","last_undegraded":"2024-03-08T23:53:45.626733+0000","last_fullsized":"2024-03-08T23:53:45.626733+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:12:01.942744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999714+0000","last_change":"2024-03-08T23:51:52.245394+0000","last_active":"2024-03-08T23:53:45.999714+0000","last_peered":"2024-03-08T23:53:45.999714+0000","last_clean":"2024-03-08T23:53:45.999714+0000","last_became_active":"2024-03-08T23:51:52.245038+0000","last_became_peered":"2024-03-08T23:51:52.245038+0000","last_unstale":"2024-03-08T23:53:45.999714+0000","last_undegraded":"2024-03-08T23:53:45.999714+0000","last_fullsized":"2024-03-08T23:53:45.999714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:15.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999804+0000","last_change":"2024-03-08T23:51:52.250861+0000","last_active":"2024-03-08T23:53:45.999804+0000","last_peered":"2024-03-08T23:53:45.999804+0000","last_clean":"2024-03-08T23:53:45.999804+0000","last_became_active":"2024-03-08T23:51:52.250548+0000","last_became_peered":"2024-03-08T23:51:52.250548+0000","last_unstale":"2024-03-08T23:53:45.999804+0000","last_undegraded":"2024-03-08T23:53:45.999804+0000","last_fullsized":"2024-03-08T23:53:45.999804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:00.280043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000326+0000","last_change":"2024-03-08T23:51:52.254996+0000","last_active":"2024-03-08T23:53:46.000326+0000","last_peered":"2024-03-08T23:53:46.000326+0000","last_clean":"2024-03-08T23:53:46.000326+0000","last_became_active":"2024-03-08T23:51:52.254634+0000","last_became_peered":"2024-03-08T23:51:52.254634+0000","last_unstale":"2024-03-08T23:53:46.000326+0000","last_undegraded":"2024-03-08T23:53:46.000326+0000","last_fullsized":"2024-03-08T23:53:46.000326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:15.449988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633637+0000","last_change":"2024-03-08T23:51:52.246558+0000","last_active":"2024-03-08T23:53:45.633637+0000","last_peered":"2024-03-08T23:53:45.633637+0000","last_clean":"2024-03-08T23:53:45.633637+0000","last_became_active":"2024-03-08T23:51:52.245741+0000","last_became_peered":"2024-03-08T23:51:52.245741+0000","last_unstale":"2024-03-08T23:53:45.633637+0000","last_undegraded":"2024-03-08T23:53:45.633637+0000","last_fullsized":"2024-03-08T23:53:45.633637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:36.737017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875498+0000","last_change":"2024-03-08T23:51:52.240111+0000","last_active":"2024-03-08T23:53:45.875498+0000","last_peered":"2024-03-08T23:53:45.875498+0000","last_clean":"2024-03-08T23:53:45.875498+0000","last_became_active":"2024-03-08T23:51:52.238961+0000","last_became_peered":"2024-03-08T23:51:52.238961+0000","last_unstale":"2024-03-08T23:53:45.875498+0000","last_undegraded":"2024-03-08T23:53:45.875498+0000","last_fullsized":"2024-03-08T23:53:45.875498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:03:55.313117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638666+0000","last_change":"2024-03-08T23:51:52.245979+0000","last_active":"2024-03-08T23:53:45.638666+0000","last_peered":"2024-03-08T23:53:45.638666+0000","last_clean":"2024-03-08T23:53:45.638666+0000","last_became_active":"2024-03-08T23:51:52.245666+0000","last_became_peered":"2024-03-08T23:51:52.245666+0000","last_unstale":"2024-03-08T23:53:45.638666+0000","last_undegraded":"2024-03-08T23:53:45.638666+0000","last_fullsized":"2024-03-08T23:53:45.638666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:20.789144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638219+0000","last_change":"2024-03-08T23:51:56.288161+0000","last_active":"2024-03-08T23:53:45.638219+0000","last_peered":"2024-03-08T23:53:45.638219+0000","last_clean":"2024-03-08T23:53:45.638219+0000","last_became_active":"2024-03-08T23:51:56.287778+0000","last_became_peered":"2024-03-08T23:51:56.287778+0000","last_unstale":"2024-03-08T23:53:45.638219+0000","last_undegraded":"2024-03-08T23:53:45.638219+0000","last_fullsized":"2024-03-08T23:53:45.638219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:12:59.636228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875582+0000","last_change":"2024-03-08T23:51:51.244911+0000","last_active":"2024-03-08T23:53:45.875582+0000","last_peered":"2024-03-08T23:53:45.875582+0000","last_clean":"2024-03-08T23:53:45.875582+0000","last_became_active":"2024-03-08T23:51:51.244561+0000","last_became_peered":"2024-03-08T23:51:51.244561+0000","last_unstale":"2024-03-08T23:53:45.875582+0000","last_undegraded":"2024-03-08T23:53:45.875582+0000","last_fullsized":"2024-03-08T23:53:45.875582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:58:45.908307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636719+0000","last_change":"2024-03-08T23:51:52.253854+0000","last_active":"2024-03-08T23:53:45.636719+0000","last_peered":"2024-03-08T23:53:45.636719+0000","last_clean":"2024-03-08T23:53:45.636719+0000","last_became_active":"2024-03-08T23:51:52.253538+0000","last_became_peered":"2024-03-08T23:51:52.253538+0000","last_unstale":"2024-03-08T23:53:45.636719+0000","last_undegraded":"2024-03-08T23:53:45.636719+0000","last_fullsized":"2024-03-08T23:53:45.636719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:33.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628202+0000","last_change":"2024-03-08T23:51:56.283307+0000","last_active":"2024-03-08T23:53:45.628202+0000","last_peered":"2024-03-08T23:53:45.628202+0000","last_clean":"2024-03-08T23:53:45.628202+0000","last_became_active":"2024-03-08T23:51:56.282868+0000","last_became_peered":"2024-03-08T23:51:56.282868+0000","last_unstale":"2024-03-08T23:53:45.628202+0000","last_undegraded":"2024-03-08T23:53:45.628202+0000","last_fullsized":"2024-03-08T23:53:45.628202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:15.027743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636962+0000","last_change":"2024-03-08T23:51:51.247368+0000","last_active":"2024-03-08T23:53:45.636962+0000","last_peered":"2024-03-08T23:53:45.636962+0000","last_clean":"2024-03-08T23:53:45.636962+0000","last_became_active":"2024-03-08T23:51:51.247023+0000","last_became_peered":"2024-03-08T23:51:51.247023+0000","last_unstale":"2024-03-08T23:53:45.636962+0000","last_undegraded":"2024-03-08T23:53:45.636962+0000","last_fullsized":"2024-03-08T23:53:45.636962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:28.311850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000106+0000","last_change":"2024-03-08T23:51:52.245805+0000","last_active":"2024-03-08T23:53:46.000106+0000","last_peered":"2024-03-08T23:53:46.000106+0000","last_clean":"2024-03-08T23:53:46.000106+0000","last_became_active":"2024-03-08T23:51:52.244455+0000","last_became_peered":"2024-03-08T23:51:52.244455+0000","last_unstale":"2024-03-08T23:53:46.000106+0000","last_undegraded":"2024-03-08T23:53:46.000106+0000","last_fullsized":"2024-03-08T23:53:46.000106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:11:17.809148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628232+0000","last_change":"2024-03-08T23:51:56.286666+0000","last_active":"2024-03-08T23:53:45.628232+0000","last_peered":"2024-03-08T23:53:45.628232+0000","last_clean":"2024-03-08T23:53:45.628232+0000","last_became_active":"2024-03-08T23:51:56.285968+0000","last_became_peered":"2024-03-08T23:51:56.285968+0000","last_unstale":"2024-03-08T23:53:45.628232+0000","last_undegraded":"2024-03-08T23:53:45.628232+0000","last_fullsized":"2024-03-08T23:53:45.628232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:51.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638792+0000","last_change":"2024-03-08T23:51:51.247684+0000","last_active":"2024-03-08T23:53:45.638792+0000","last_peered":"2024-03-08T23:53:45.638792+0000","last_clean":"2024-03-08T23:53:45.638792+0000","last_became_active":"2024-03-08T23:51:51.247374+0000","last_became_peered":"2024-03-08T23:51:51.247374+0000","last_unstale":"2024-03-08T23:53:45.638792+0000","last_undegraded":"2024-03-08T23:53:45.638792+0000","last_fullsized":"2024-03-08T23:53:45.638792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:20.425302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000566+0000","last_change":"2024-03-08T23:51:52.248702+0000","last_active":"2024-03-08T23:53:46.000566+0000","last_peered":"2024-03-08T23:53:46.000566+0000","last_clean":"2024-03-08T23:53:46.000566+0000","last_became_active":"2024-03-08T23:51:52.248388+0000","last_became_peered":"2024-03-08T23:51:52.248388+0000","last_unstale":"2024-03-08T23:53:46.000566+0000","last_undegraded":"2024-03-08T23:53:46.000566+0000","last_fullsized":"2024-03-08T23:53:46.000566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:27:12.251957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628238+0000","last_change":"2024-03-08T23:51:56.286572+0000","last_active":"2024-03-08T23:53:45.628238+0000","last_peered":"2024-03-08T23:53:45.628238+0000","last_clean":"2024-03-08T23:53:45.628238+0000","last_became_active":"2024-03-08T23:51:56.286066+0000","last_became_peered":"2024-03-08T23:51:56.286066+0000","last_unstale":"2024-03-08T23:53:45.628238+0000","last_undegraded":"2024-03-08T23:53:45.628238+0000","last_fullsized":"2024-03-08T23:53:45.628238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:13.694615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638236+0000","last_change":"2024-03-08T23:51:51.244207+0000","last_active":"2024-03-08T23:53:45.638236+0000","last_peered":"2024-03-08T23:53:45.638236+0000","last_clean":"2024-03-08T23:53:45.638236+0000","last_became_active":"2024-03-08T23:51:51.241605+0000","last_became_peered":"2024-03-08T23:51:51.241605+0000","last_unstale":"2024-03-08T23:53:45.638236+0000","last_undegraded":"2024-03-08T23:53:45.638236+0000","last_fullsized":"2024-03-08T23:53:45.638236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:03:51.766681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629253+0000","last_change":"2024-03-08T23:51:52.251015+0000","last_active":"2024-03-08T23:53:45.629253+0000","last_peered":"2024-03-08T23:53:45.629253+0000","last_clean":"2024-03-08T23:53:45.629253+0000","last_became_active":"2024-03-08T23:51:52.250690+0000","last_became_peered":"2024-03-08T23:51:52.250690+0000","last_unstale":"2024-03-08T23:53:45.629253+0000","last_undegraded":"2024-03-08T23:53:45.629253+0000","last_fullsized":"2024-03-08T23:53:45.629253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:57:23.033172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638368+0000","last_change":"2024-03-08T23:51:56.281066+0000","last_active":"2024-03-08T23:53:45.638368+0000","last_peered":"2024-03-08T23:53:45.638368+0000","last_clean":"2024-03-08T23:53:45.638368+0000","last_became_active":"2024-03-08T23:51:56.280654+0000","last_became_peered":"2024-03-08T23:51:56.280654+0000","last_unstale":"2024-03-08T23:53:45.638368+0000","last_undegraded":"2024-03-08T23:53:45.638368+0000","last_fullsized":"2024-03-08T23:53:45.638368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:53.999115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:16.524728+0000","last_change":"2024-03-08T23:51:51.245947+0000","last_active":"2024-03-08T23:54:16.524728+0000","last_peered":"2024-03-08T23:54:16.524728+0000","last_clean":"2024-03-08T23:54:16.524728+0000","last_became_active":"2024-03-08T23:51:51.245505+0000","last_became_peered":"2024-03-08T23:51:51.245505+0000","last_unstale":"2024-03-08T23:54:16.524728+0000","last_undegraded":"2024-03-08T23:54:16.524728+0000","last_fullsized":"2024-03-08T23:54:16.524728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:20:44.692598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637141+0000","last_change":"2024-03-08T23:51:52.241590+0000","last_active":"2024-03-08T23:53:45.637141+0000","last_peered":"2024-03-08T23:53:45.637141+0000","last_clean":"2024-03-08T23:53:45.637141+0000","last_became_active":"2024-03-08T23:51:52.241296+0000","last_became_peered":"2024-03-08T23:51:52.241296+0000","last_unstale":"2024-03-08T23:53:45.637141+0000","last_undegraded":"2024-03-08T23:53:45.637141+0000","last_fullsized":"2024-03-08T23:53:45.637141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:52:48.042360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638174+0000","last_change":"2024-03-08T23:52:45.585946+0000","last_active":"2024-03-08T23:53:45.638174+0000","last_peered":"2024-03-08T23:53:45.638174+0000","last_clean":"2024-03-08T23:53:45.638174+0000","last_became_active":"2024-03-08T23:52:45.585470+0000","last_became_peered":"2024-03-08T23:52:45.585470+0000","last_unstale":"2024-03-08T23:53:45.638174+0000","last_undegraded":"2024-03-08T23:53:45.638174+0000","last_fullsized":"2024-03-08T23:53:45.638174+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:28:36.264887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001568+0000","last_change":"2024-03-08T23:51:51.241384+0000","last_active":"2024-03-08T23:53:46.001568+0000","last_peered":"2024-03-08T23:53:46.001568+0000","last_clean":"2024-03-08T23:53:46.001568+0000","last_became_active":"2024-03-08T23:51:51.241018+0000","last_became_peered":"2024-03-08T23:51:51.241018+0000","last_unstale":"2024-03-08T23:53:46.001568+0000","last_undegraded":"2024-03-08T23:53:46.001568+0000","last_fullsized":"2024-03-08T23:53:46.001568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:29:50.860663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629076+0000","last_change":"2024-03-08T23:51:52.250207+0000","last_active":"2024-03-08T23:53:45.629076+0000","last_peered":"2024-03-08T23:53:45.629076+0000","last_clean":"2024-03-08T23:53:45.629076+0000","last_became_active":"2024-03-08T23:51:52.249863+0000","last_became_peered":"2024-03-08T23:51:52.249863+0000","last_unstale":"2024-03-08T23:53:45.629076+0000","last_undegraded":"2024-03-08T23:53:45.629076+0000","last_fullsized":"2024-03-08T23:53:45.629076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:18:48.732419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633519+0000","last_change":"2024-03-08T23:51:56.282092+0000","last_active":"2024-03-08T23:53:45.633519+0000","last_peered":"2024-03-08T23:53:45.633519+0000","last_clean":"2024-03-08T23:53:45.633519+0000","last_became_active":"2024-03-08T23:51:56.281657+0000","last_became_peered":"2024-03-08T23:51:56.281657+0000","last_unstale":"2024-03-08T23:53:45.633519+0000","last_undegraded":"2024-03-08T23:53:45.633519+0000","last_fullsized":"2024-03-08T23:53:45.633519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:50.440520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626316+0000","last_change":"2024-03-08T23:51:51.236816+0000","last_active":"2024-03-08T23:53:45.626316+0000","last_peered":"2024-03-08T23:53:45.626316+0000","last_clean":"2024-03-08T23:53:45.626316+0000","last_became_active":"2024-03-08T23:51:51.236485+0000","last_became_peered":"2024-03-08T23:51:51.236485+0000","last_unstale":"2024-03-08T23:53:45.626316+0000","last_undegraded":"2024-03-08T23:53:45.626316+0000","last_fullsized":"2024-03-08T23:53:45.626316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:27.300950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636406+0000","last_change":"2024-03-08T23:51:52.252146+0000","last_active":"2024-03-08T23:53:45.636406+0000","last_peered":"2024-03-08T23:53:45.636406+0000","last_clean":"2024-03-08T23:53:45.636406+0000","last_became_active":"2024-03-08T23:51:52.244201+0000","last_became_peered":"2024-03-08T23:51:52.244201+0000","last_unstale":"2024-03-08T23:53:45.636406+0000","last_undegraded":"2024-03-08T23:53:45.636406+0000","last_fullsized":"2024-03-08T23:53:45.636406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:07.647078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000164+0000","last_change":"2024-03-08T23:51:56.279727+0000","last_active":"2024-03-08T23:53:46.000164+0000","last_peered":"2024-03-08T23:53:46.000164+0000","last_clean":"2024-03-08T23:53:46.000164+0000","last_became_active":"2024-03-08T23:51:56.279184+0000","last_became_peered":"2024-03-08T23:51:56.279184+0000","last_unstale":"2024-03-08T23:53:46.000164+0000","last_undegraded":"2024-03-08T23:53:46.000164+0000","last_fullsized":"2024-03-08T23:53:46.000164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:21.804410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629021+0000","last_change":"2024-03-08T23:51:51.244812+0000","last_active":"2024-03-08T23:53:45.629021+0000","last_peered":"2024-03-08T23:53:45.629021+0000","last_clean":"2024-03-08T23:53:45.629021+0000","last_became_active":"2024-03-08T23:51:51.244513+0000","last_became_peered":"2024-03-08T23:51:51.244513+0000","last_unstale":"2024-03-08T23:53:45.629021+0000","last_undegraded":"2024-03-08T23:53:45.629021+0000","last_fullsized":"2024-03-08T23:53:45.629021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:56:07.235214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875255+0000","last_change":"2024-03-08T23:51:52.244566+0000","last_active":"2024-03-08T23:53:45.875255+0000","last_peered":"2024-03-08T23:53:45.875255+0000","last_clean":"2024-03-08T23:53:45.875255+0000","last_became_active":"2024-03-08T23:51:52.244250+0000","last_became_peered":"2024-03-08T23:51:52.244250+0000","last_unstale":"2024-03-08T23:53:45.875255+0000","last_undegraded":"2024-03-08T23:53:45.875255+0000","last_fullsized":"2024-03-08T23:53:45.875255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:55:50.596364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636425+0000","last_change":"2024-03-08T23:51:56.282487+0000","last_active":"2024-03-08T23:53:45.636425+0000","last_peered":"2024-03-08T23:53:45.636425+0000","last_clean":"2024-03-08T23:53:45.636425+0000","last_became_active":"2024-03-08T23:51:56.282104+0000","last_became_peered":"2024-03-08T23:51:56.282104+0000","last_unstale":"2024-03-08T23:53:45.636425+0000","last_undegraded":"2024-03-08T23:53:45.636425+0000","last_fullsized":"2024-03-08T23:53:45.636425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:23:23.787431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.527612+0000","last_change":"2024-03-08T23:51:51.236612+0000","last_active":"2024-03-08T23:54:18.527612+0000","last_peered":"2024-03-08T23:54:18.527612+0000","last_clean":"2024-03-08T23:54:18.527612+0000","last_became_active":"2024-03-08T23:51:51.236274+0000","last_became_peered":"2024-03-08T23:51:51.236274+0000","last_unstale":"2024-03-08T23:54:18.527612+0000","last_undegraded":"2024-03-08T23:54:18.527612+0000","last_fullsized":"2024-03-08T23:54:18.527612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:37.551870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000849+0000","last_change":"2024-03-08T23:51:52.244564+0000","last_active":"2024-03-08T23:53:46.000849+0000","last_peered":"2024-03-08T23:53:46.000849+0000","last_clean":"2024-03-08T23:53:46.000849+0000","last_became_active":"2024-03-08T23:51:52.243651+0000","last_became_peered":"2024-03-08T23:51:52.243651+0000","last_unstale":"2024-03-08T23:53:46.000849+0000","last_undegraded":"2024-03-08T23:53:46.000849+0000","last_fullsized":"2024-03-08T23:53:46.000849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:35:32.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875029+0000","last_change":"2024-03-08T23:51:56.279354+0000","last_active":"2024-03-08T23:53:45.875029+0000","last_peered":"2024-03-08T23:53:45.875029+0000","last_clean":"2024-03-08T23:53:45.875029+0000","last_became_active":"2024-03-08T23:51:56.278894+0000","last_became_peered":"2024-03-08T23:51:56.278894+0000","last_unstale":"2024-03-08T23:53:45.875029+0000","last_undegraded":"2024-03-08T23:53:45.875029+0000","last_fullsized":"2024-03-08T23:53:45.875029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:20:30.164542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002108+0000","last_change":"2024-03-08T23:51:51.246417+0000","last_active":"2024-03-08T23:53:46.002108+0000","last_peered":"2024-03-08T23:53:46.002108+0000","last_clean":"2024-03-08T23:53:46.002108+0000","last_became_active":"2024-03-08T23:51:51.246058+0000","last_became_peered":"2024-03-08T23:51:51.246058+0000","last_unstale":"2024-03-08T23:53:46.002108+0000","last_undegraded":"2024-03-08T23:53:46.002108+0000","last_fullsized":"2024-03-08T23:53:46.002108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:48:53.569295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000099+0000","last_change":"2024-03-08T23:51:52.255357+0000","last_active":"2024-03-08T23:53:46.000099+0000","last_peered":"2024-03-08T23:53:46.000099+0000","last_clean":"2024-03-08T23:53:46.000099+0000","last_became_active":"2024-03-08T23:51:52.255044+0000","last_became_peered":"2024-03-08T23:51:52.255044+0000","last_unstale":"2024-03-08T23:53:46.000099+0000","last_undegraded":"2024-03-08T23:53:46.000099+0000","last_fullsized":"2024-03-08T23:53:46.000099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:13:40.710283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628526+0000","last_change":"2024-03-08T23:51:56.284282+0000","last_active":"2024-03-08T23:53:45.628526+0000","last_peered":"2024-03-08T23:53:45.628526+0000","last_clean":"2024-03-08T23:53:45.628526+0000","last_became_active":"2024-03-08T23:51:56.283863+0000","last_became_peered":"2024-03-08T23:51:56.283863+0000","last_unstale":"2024-03-08T23:53:45.628526+0000","last_undegraded":"2024-03-08T23:53:45.628526+0000","last_fullsized":"2024-03-08T23:53:45.628526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:59:21.972283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629263+0000","last_change":"2024-03-08T23:51:51.245199+0000","last_active":"2024-03-08T23:53:45.629263+0000","last_peered":"2024-03-08T23:53:45.629263+0000","last_clean":"2024-03-08T23:53:45.629263+0000","last_became_active":"2024-03-08T23:51:51.244906+0000","last_became_peered":"2024-03-08T23:51:51.244906+0000","last_unstale":"2024-03-08T23:53:45.629263+0000","last_undegraded":"2024-03-08T23:53:45.629263+0000","last_fullsized":"2024-03-08T23:53:45.629263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:54:39.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000875+0000","last_change":"2024-03-08T23:51:52.251209+0000","last_active":"2024-03-08T23:53:46.000875+0000","last_peered":"2024-03-08T23:53:46.000875+0000","last_clean":"2024-03-08T23:53:46.000875+0000","last_became_active":"2024-03-08T23:51:52.250392+0000","last_became_peered":"2024-03-08T23:51:52.250392+0000","last_unstale":"2024-03-08T23:53:46.000875+0000","last_undegraded":"2024-03-08T23:53:46.000875+0000","last_fullsized":"2024-03-08T23:53:46.000875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:56:17.811505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001099+0000","last_change":"2024-03-08T23:52:45.586125+0000","last_active":"2024-03-08T23:53:46.001099+0000","last_peered":"2024-03-08T23:53:46.001099+0000","last_clean":"2024-03-08T23:53:46.001099+0000","last_became_active":"2024-03-08T23:52:45.585578+0000","last_became_peered":"2024-03-08T23:52:45.585578+0000","last_unstale":"2024-03-08T23:53:46.001099+0000","last_undegraded":"2024-03-08T23:53:46.001099+0000","last_fullsized":"2024-03-08T23:53:46.001099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:51:08.159460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"93'40","reported_seq":343,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.553773+0000","last_change":"2024-03-08T23:52:45.426691+0000","last_active":"2024-03-08T23:54:18.553773+0000","last_peered":"2024-03-08T23:54:18.553773+0000","last_clean":"2024-03-08T23:54:18.553773+0000","last_became_active":"2024-03-08T23:52:45.415742+0000","last_became_peered":"2024-03-08T23:52:45.415742+0000","last_unstale":"2024-03-08T23:54:18.553773+0000","last_undegraded":"2024-03-08T23:54:18.553773+0000","last_fullsized":"2024-03-08T23:54:18.553773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:05.389164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68654,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":187,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60681,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638662+0000","last_change":"2024-03-08T23:51:52.248715+0000","last_active":"2024-03-08T23:53:45.638662+0000","last_peered":"2024-03-08T23:53:45.638662+0000","last_clean":"2024-03-08T23:53:45.638662+0000","last_became_active":"2024-03-08T23:51:52.248258+0000","last_became_peered":"2024-03-08T23:51:52.248258+0000","last_unstale":"2024-03-08T23:53:45.638662+0000","last_undegraded":"2024-03-08T23:53:45.638662+0000","last_fullsized":"2024-03-08T23:53:45.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:38.185469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629520+0000","last_change":"2024-03-08T23:51:56.278268+0000","last_active":"2024-03-08T23:53:45.629520+0000","last_peered":"2024-03-08T23:53:45.629520+0000","last_clean":"2024-03-08T23:53:45.629520+0000","last_became_active":"2024-03-08T23:51:56.277831+0000","last_became_peered":"2024-03-08T23:51:56.277831+0000","last_unstale":"2024-03-08T23:53:45.629520+0000","last_undegraded":"2024-03-08T23:53:45.629520+0000","last_fullsized":"2024-03-08T23:53:45.629520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:32:13.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637455+0000","last_change":"2024-03-08T23:51:51.247553+0000","last_active":"2024-03-08T23:53:45.637455+0000","last_peered":"2024-03-08T23:53:45.637455+0000","last_clean":"2024-03-08T23:53:45.637455+0000","last_became_active":"2024-03-08T23:51:51.247225+0000","last_became_peered":"2024-03-08T23:51:51.247225+0000","last_unstale":"2024-03-08T23:53:45.637455+0000","last_undegraded":"2024-03-08T23:53:45.637455+0000","last_fullsized":"2024-03-08T23:53:45.637455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:53:29.846303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626580+0000","last_change":"2024-03-08T23:51:52.247733+0000","last_active":"2024-03-08T23:53:45.626580+0000","last_peered":"2024-03-08T23:53:45.626580+0000","last_clean":"2024-03-08T23:53:45.626580+0000","last_became_active":"2024-03-08T23:51:52.247435+0000","last_became_peered":"2024-03-08T23:51:52.247435+0000","last_unstale":"2024-03-08T23:53:45.626580+0000","last_undegraded":"2024-03-08T23:53:45.626580+0000","last_fullsized":"2024-03-08T23:53:45.626580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:46:12.739601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.632494+0000","last_change":"2024-03-08T23:51:56.279865+0000","last_active":"2024-03-08T23:53:45.632494+0000","last_peered":"2024-03-08T23:53:45.632494+0000","last_clean":"2024-03-08T23:53:45.632494+0000","last_became_active":"2024-03-08T23:51:56.279446+0000","last_became_peered":"2024-03-08T23:51:56.279446+0000","last_unstale":"2024-03-08T23:53:45.632494+0000","last_undegraded":"2024-03-08T23:53:45.632494+0000","last_fullsized":"2024-03-08T23:53:45.632494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:07:33.672490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629392+0000","last_change":"2024-03-08T23:51:51.246860+0000","last_active":"2024-03-08T23:53:45.629392+0000","last_peered":"2024-03-08T23:53:45.629392+0000","last_clean":"2024-03-08T23:53:45.629392+0000","last_became_active":"2024-03-08T23:51:51.246545+0000","last_became_peered":"2024-03-08T23:51:51.246545+0000","last_unstale":"2024-03-08T23:53:45.629392+0000","last_undegraded":"2024-03-08T23:53:45.629392+0000","last_fullsized":"2024-03-08T23:53:45.629392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:11:07.246657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001899+0000","last_change":"2024-03-08T23:51:52.247353+0000","last_active":"2024-03-08T23:53:46.001899+0000","last_peered":"2024-03-08T23:53:46.001899+0000","last_clean":"2024-03-08T23:53:46.001899+0000","last_became_active":"2024-03-08T23:51:52.247008+0000","last_became_peered":"2024-03-08T23:51:52.247008+0000","last_unstale":"2024-03-08T23:53:46.001899+0000","last_undegraded":"2024-03-08T23:53:46.001899+0000","last_fullsized":"2024-03-08T23:53:46.001899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:08:50.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875481+0000","last_change":"2024-03-08T23:51:56.284103+0000","last_active":"2024-03-08T23:53:45.875481+0000","last_peered":"2024-03-08T23:53:45.875481+0000","last_clean":"2024-03-08T23:53:45.875481+0000","last_became_active":"2024-03-08T23:51:56.283561+0000","last_became_peered":"2024-03-08T23:51:56.283561+0000","last_unstale":"2024-03-08T23:53:45.875481+0000","last_undegraded":"2024-03-08T23:53:45.875481+0000","last_fullsized":"2024-03-08T23:53:45.875481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:47:20.909292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:17.524146+0000","last_change":"2024-03-08T23:51:51.242483+0000","last_active":"2024-03-08T23:54:17.524146+0000","last_peered":"2024-03-08T23:54:17.524146+0000","last_clean":"2024-03-08T23:54:17.524146+0000","last_became_active":"2024-03-08T23:51:51.242153+0000","last_became_peered":"2024-03-08T23:51:51.242153+0000","last_unstale":"2024-03-08T23:54:17.524146+0000","last_undegraded":"2024-03-08T23:54:17.524146+0000","last_fullsized":"2024-03-08T23:54:17.524146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:44:07.728700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001140+0000","last_change":"2024-03-08T23:51:52.255411+0000","last_active":"2024-03-08T23:53:46.001140+0000","last_peered":"2024-03-08T23:53:46.001140+0000","last_clean":"2024-03-08T23:53:46.001140+0000","last_became_active":"2024-03-08T23:51:52.255048+0000","last_became_peered":"2024-03-08T23:51:52.255048+0000","last_unstale":"2024-03-08T23:53:46.001140+0000","last_undegraded":"2024-03-08T23:53:46.001140+0000","last_fullsized":"2024-03-08T23:53:46.001140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:11.836187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626682+0000","last_change":"2024-03-08T23:51:56.286405+0000","last_active":"2024-03-08T23:53:45.626682+0000","last_peered":"2024-03-08T23:53:45.626682+0000","last_clean":"2024-03-08T23:53:45.626682+0000","last_became_active":"2024-03-08T23:51:56.285996+0000","last_became_peered":"2024-03-08T23:51:56.285996+0000","last_unstale":"2024-03-08T23:53:45.626682+0000","last_undegraded":"2024-03-08T23:53:45.626682+0000","last_fullsized":"2024-03-08T23:53:45.626682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:48:30.928436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637190+0000","last_change":"2024-03-08T23:51:51.240146+0000","last_active":"2024-03-08T23:53:45.637190+0000","last_peered":"2024-03-08T23:53:45.637190+0000","last_clean":"2024-03-08T23:53:45.637190+0000","last_became_active":"2024-03-08T23:51:51.239795+0000","last_became_peered":"2024-03-08T23:51:51.239795+0000","last_unstale":"2024-03-08T23:53:45.637190+0000","last_undegraded":"2024-03-08T23:53:45.637190+0000","last_fullsized":"2024-03-08T23:53:45.637190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:55.409922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629399+0000","last_change":"2024-03-08T23:51:52.254701+0000","last_active":"2024-03-08T23:53:45.629399+0000","last_peered":"2024-03-08T23:53:45.629399+0000","last_clean":"2024-03-08T23:53:45.629399+0000","last_became_active":"2024-03-08T23:51:52.254391+0000","last_became_peered":"2024-03-08T23:51:52.254391+0000","last_unstale":"2024-03-08T23:53:45.629399+0000","last_undegraded":"2024-03-08T23:53:45.629399+0000","last_fullsized":"2024-03-08T23:53:45.629399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:11:12.258181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628931+0000","last_change":"2024-03-08T23:51:56.277878+0000","last_active":"2024-03-08T23:53:45.628931+0000","last_peered":"2024-03-08T23:53:45.628931+0000","last_clean":"2024-03-08T23:53:45.628931+0000","last_became_active":"2024-03-08T23:51:56.277493+0000","last_became_peered":"2024-03-08T23:51:56.277493+0000","last_unstale":"2024-03-08T23:53:45.628931+0000","last_undegraded":"2024-03-08T23:53:45.628931+0000","last_fullsized":"2024-03-08T23:53:45.628931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:44:16.968632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.643148+0000","last_change":"2024-03-08T23:53:45.643148+0000","last_active":"2024-03-08T23:53:45.643148+0000","last_peered":"2024-03-08T23:53:45.643148+0000","last_clean":"2024-03-08T23:53:45.643148+0000","last_became_active":"2024-03-08T23:53:45.638975+0000","last_became_peered":"2024-03-08T23:53:45.638975+0000","last_unstale":"2024-03-08T23:53:45.643148+0000","last_undegraded":"2024-03-08T23:53:45.643148+0000","last_fullsized":"2024-03-08T23:53:45.643148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:42:39.188486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.632141+0000","last_change":"2024-03-08T23:51:52.248465+0000","last_active":"2024-03-08T23:53:45.632141+0000","last_peered":"2024-03-08T23:53:45.632141+0000","last_clean":"2024-03-08T23:53:45.632141+0000","last_became_active":"2024-03-08T23:51:52.248135+0000","last_became_peered":"2024-03-08T23:51:52.248135+0000","last_unstale":"2024-03-08T23:53:45.632141+0000","last_undegraded":"2024-03-08T23:53:45.632141+0000","last_fullsized":"2024-03-08T23:53:45.632141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:55:53.712958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638275+0000","last_change":"2024-03-08T23:51:56.276019+0000","last_active":"2024-03-08T23:53:45.638275+0000","last_peered":"2024-03-08T23:53:45.638275+0000","last_clean":"2024-03-08T23:53:45.638275+0000","last_became_active":"2024-03-08T23:51:56.275564+0000","last_became_peered":"2024-03-08T23:51:56.275564+0000","last_unstale":"2024-03-08T23:53:45.638275+0000","last_undegraded":"2024-03-08T23:53:45.638275+0000","last_fullsized":"2024-03-08T23:53:45.638275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:12:36.096470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875254+0000","last_change":"2024-03-08T23:51:51.241629+0000","last_active":"2024-03-08T23:53:45.875254+0000","last_peered":"2024-03-08T23:53:45.875254+0000","last_clean":"2024-03-08T23:53:45.875254+0000","last_became_active":"2024-03-08T23:51:51.241300+0000","last_became_peered":"2024-03-08T23:51:51.241300+0000","last_unstale":"2024-03-08T23:53:45.875254+0000","last_undegraded":"2024-03-08T23:53:45.875254+0000","last_fullsized":"2024-03-08T23:53:45.875254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:01:45.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001329+0000","last_change":"2024-03-08T23:51:52.246024+0000","last_active":"2024-03-08T23:53:46.001329+0000","last_peered":"2024-03-08T23:53:46.001329+0000","last_clean":"2024-03-08T23:53:46.001329+0000","last_became_active":"2024-03-08T23:51:52.245706+0000","last_became_peered":"2024-03-08T23:51:52.245706+0000","last_unstale":"2024-03-08T23:53:46.001329+0000","last_undegraded":"2024-03-08T23:53:46.001329+0000","last_fullsized":"2024-03-08T23:53:46.001329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:00:44.557269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626476+0000","last_change":"2024-03-08T23:51:56.285649+0000","last_active":"2024-03-08T23:53:45.626476+0000","last_peered":"2024-03-08T23:53:45.626476+0000","last_clean":"2024-03-08T23:53:45.626476+0000","last_became_active":"2024-03-08T23:51:56.282731+0000","last_became_peered":"2024-03-08T23:51:56.282731+0000","last_unstale":"2024-03-08T23:53:45.626476+0000","last_undegraded":"2024-03-08T23:53:45.626476+0000","last_fullsized":"2024-03-08T23:53:45.626476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:23:54.103637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001789+0000","last_change":"2024-03-08T23:51:51.249442+0000","last_active":"2024-03-08T23:53:46.001789+0000","last_peered":"2024-03-08T23:53:46.001789+0000","last_clean":"2024-03-08T23:53:46.001789+0000","last_became_active":"2024-03-08T23:51:51.249084+0000","last_became_peered":"2024-03-08T23:51:51.249084+0000","last_unstale":"2024-03-08T23:53:46.001789+0000","last_undegraded":"2024-03-08T23:53:46.001789+0000","last_fullsized":"2024-03-08T23:53:46.001789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:48:19.370095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001661+0000","last_change":"2024-03-08T23:51:52.257586+0000","last_active":"2024-03-08T23:53:46.001661+0000","last_peered":"2024-03-08T23:53:46.001661+0000","last_clean":"2024-03-08T23:53:46.001661+0000","last_became_active":"2024-03-08T23:51:52.253741+0000","last_became_peered":"2024-03-08T23:51:52.253741+0000","last_unstale":"2024-03-08T23:53:46.001661+0000","last_undegraded":"2024-03-08T23:53:46.001661+0000","last_fullsized":"2024-03-08T23:53:46.001661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:18.633166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626404+0000","last_change":"2024-03-08T23:51:56.285264+0000","last_active":"2024-03-08T23:53:45.626404+0000","last_peered":"2024-03-08T23:53:45.626404+0000","last_clean":"2024-03-08T23:53:45.626404+0000","last_became_active":"2024-03-08T23:51:56.284727+0000","last_became_peered":"2024-03-08T23:51:56.284727+0000","last_unstale":"2024-03-08T23:53:45.626404+0000","last_undegraded":"2024-03-08T23:53:45.626404+0000","last_fullsized":"2024-03-08T23:53:45.626404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:01:20.723010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629149+0000","last_change":"2024-03-08T23:51:52.251461+0000","last_active":"2024-03-08T23:53:45.629149+0000","last_peered":"2024-03-08T23:53:45.629149+0000","last_clean":"2024-03-08T23:53:45.629149+0000","last_became_active":"2024-03-08T23:51:52.250996+0000","last_became_peered":"2024-03-08T23:51:52.250996+0000","last_unstale":"2024-03-08T23:53:45.629149+0000","last_undegraded":"2024-03-08T23:53:45.629149+0000","last_fullsized":"2024-03-08T23:53:45.629149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:51.719142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.524820+0000","last_change":"2024-03-08T23:51:51.245698+0000","last_active":"2024-03-08T23:54:18.524820+0000","last_peered":"2024-03-08T23:54:18.524820+0000","last_clean":"2024-03-08T23:54:18.524820+0000","last_became_active":"2024-03-08T23:51:51.245389+0000","last_became_peered":"2024-03-08T23:51:51.245389+0000","last_unstale":"2024-03-08T23:54:18.524820+0000","last_undegraded":"2024-03-08T23:54:18.524820+0000","last_fullsized":"2024-03-08T23:54:18.524820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:14:08.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002007+0000","last_change":"2024-03-08T23:52:45.423657+0000","last_active":"2024-03-08T23:53:46.002007+0000","last_peered":"2024-03-08T23:53:46.002007+0000","last_clean":"2024-03-08T23:53:46.002007+0000","last_became_active":"2024-03-08T23:52:45.417868+0000","last_became_peered":"2024-03-08T23:52:45.417868+0000","last_unstale":"2024-03-08T23:53:46.002007+0000","last_undegraded":"2024-03-08T23:53:46.002007+0000","last_fullsized":"2024-03-08T23:53:46.002007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:34.178938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636452+0000","last_change":"2024-03-08T23:51:52.256518+0000","last_active":"2024-03-08T23:53:45.636452+0000","last_peered":"2024-03-08T23:53:45.636452+0000","last_clean":"2024-03-08T23:53:45.636452+0000","last_became_active":"2024-03-08T23:51:52.253481+0000","last_became_peered":"2024-03-08T23:51:52.253481+0000","last_unstale":"2024-03-08T23:53:45.636452+0000","last_undegraded":"2024-03-08T23:53:45.636452+0000","last_fullsized":"2024-03-08T23:53:45.636452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:35.287408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628496+0000","last_change":"2024-03-08T23:51:51.237921+0000","last_active":"2024-03-08T23:53:45.628496+0000","last_peered":"2024-03-08T23:53:45.628496+0000","last_clean":"2024-03-08T23:53:45.628496+0000","last_became_active":"2024-03-08T23:51:51.237568+0000","last_became_peered":"2024-03-08T23:51:51.237568+0000","last_unstale":"2024-03-08T23:53:45.628496+0000","last_undegraded":"2024-03-08T23:53:45.628496+0000","last_fullsized":"2024-03-08T23:53:45.628496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:11.256814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000644+0000","last_change":"2024-03-08T23:50:41.802121+0000","last_active":"2024-03-08T23:53:46.000644+0000","last_peered":"2024-03-08T23:53:46.000644+0000","last_clean":"2024-03-08T23:53:46.000644+0000","last_became_active":"2024-03-08T23:50:41.801678+0000","last_became_peered":"2024-03-08T23:50:41.801678+0000","last_unstale":"2024-03-08T23:53:46.000644+0000","last_undegraded":"2024-03-08T23:53:46.000644+0000","last_fullsized":"2024-03-08T23:53:46.000644+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_clean_scrub_stamp":"2024-03-08T23:47:12.382547+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:15.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629562+0000","last_change":"2024-03-08T23:51:56.278465+0000","last_active":"2024-03-08T23:53:45.629562+0000","last_peered":"2024-03-08T23:53:45.629562+0000","last_clean":"2024-03-08T23:53:45.629562+0000","last_became_active":"2024-03-08T23:51:56.278079+0000","last_became_peered":"2024-03-08T23:51:56.278079+0000","last_unstale":"2024-03-08T23:53:45.629562+0000","last_undegraded":"2024-03-08T23:53:45.629562+0000","last_fullsized":"2024-03-08T23:53:45.629562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:47.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626180+0000","last_change":"2024-03-08T23:51:52.251895+0000","last_active":"2024-03-08T23:53:45.626180+0000","last_peered":"2024-03-08T23:53:45.626180+0000","last_clean":"2024-03-08T23:53:45.626180+0000","last_became_active":"2024-03-08T23:51:52.251591+0000","last_became_peered":"2024-03-08T23:51:52.251591+0000","last_unstale":"2024-03-08T23:53:45.626180+0000","last_undegraded":"2024-03-08T23:53:45.626180+0000","last_fullsized":"2024-03-08T23:53:45.626180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:06.276636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001022+0000","last_change":"2024-03-08T23:51:51.245981+0000","last_active":"2024-03-08T23:53:46.001022+0000","last_peered":"2024-03-08T23:53:46.001022+0000","last_clean":"2024-03-08T23:53:46.001022+0000","last_became_active":"2024-03-08T23:51:51.244681+0000","last_became_peered":"2024-03-08T23:51:51.244681+0000","last_unstale":"2024-03-08T23:53:46.001022+0000","last_undegraded":"2024-03-08T23:53:46.001022+0000","last_fullsized":"2024-03-08T23:53:46.001022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:06:21.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.631890+0000","last_change":"2024-03-08T23:51:56.279057+0000","last_active":"2024-03-08T23:53:45.631890+0000","last_peered":"2024-03-08T23:53:45.631890+0000","last_clean":"2024-03-08T23:53:45.631890+0000","last_became_active":"2024-03-08T23:51:56.278634+0000","last_became_peered":"2024-03-08T23:51:56.278634+0000","last_unstale":"2024-03-08T23:53:45.631890+0000","last_undegraded":"2024-03-08T23:53:45.631890+0000","last_fullsized":"2024-03-08T23:53:45.631890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:30.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638685+0000","last_change":"2024-03-08T23:51:51.240378+0000","last_active":"2024-03-08T23:53:45.638685+0000","last_peered":"2024-03-08T23:53:45.638685+0000","last_clean":"2024-03-08T23:53:45.638685+0000","last_became_active":"2024-03-08T23:51:51.239880+0000","last_became_peered":"2024-03-08T23:51:51.239880+0000","last_unstale":"2024-03-08T23:53:45.638685+0000","last_undegraded":"2024-03-08T23:53:45.638685+0000","last_fullsized":"2024-03-08T23:53:45.638685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:20.804261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626738+0000","last_change":"2024-03-08T23:51:52.240338+0000","last_active":"2024-03-08T23:53:45.626738+0000","last_peered":"2024-03-08T23:53:45.626738+0000","last_clean":"2024-03-08T23:53:45.626738+0000","last_became_active":"2024-03-08T23:51:52.239987+0000","last_became_peered":"2024-03-08T23:51:52.239987+0000","last_unstale":"2024-03-08T23:53:45.626738+0000","last_undegraded":"2024-03-08T23:53:45.626738+0000","last_fullsized":"2024-03-08T23:53:45.626738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:26.590204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636290+0000","last_change":"2024-03-08T23:51:56.281009+0000","last_active":"2024-03-08T23:53:45.636290+0000","last_peered":"2024-03-08T23:53:45.636290+0000","last_clean":"2024-03-08T23:53:45.636290+0000","last_became_active":"2024-03-08T23:51:56.280619+0000","last_became_peered":"2024-03-08T23:51:56.280619+0000","last_unstale":"2024-03-08T23:53:45.636290+0000","last_undegraded":"2024-03-08T23:53:45.636290+0000","last_fullsized":"2024-03-08T23:53:45.636290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:06.920777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:17.267361+0000","last_change":"2024-03-08T23:51:51.241682+0000","last_active":"2024-03-08T23:54:17.267361+0000","last_peered":"2024-03-08T23:54:17.267361+0000","last_clean":"2024-03-08T23:54:17.267361+0000","last_became_active":"2024-03-08T23:51:51.241313+0000","last_became_peered":"2024-03-08T23:51:51.241313+0000","last_unstale":"2024-03-08T23:54:17.267361+0000","last_undegraded":"2024-03-08T23:54:17.267361+0000","last_fullsized":"2024-03-08T23:54:17.267361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:04:13.515856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636824+0000","last_change":"2024-03-08T23:51:52.249378+0000","last_active":"2024-03-08T23:53:45.636824+0000","last_peered":"2024-03-08T23:53:45.636824+0000","last_clean":"2024-03-08T23:53:45.636824+0000","last_became_active":"2024-03-08T23:51:52.249084+0000","last_became_peered":"2024-03-08T23:51:52.249084+0000","last_unstale":"2024-03-08T23:53:45.636824+0000","last_undegraded":"2024-03-08T23:53:45.636824+0000","last_fullsized":"2024-03-08T23:53:45.636824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:10:36.513260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638049+0000","last_change":"2024-03-08T23:51:56.280717+0000","last_active":"2024-03-08T23:53:45.638049+0000","last_peered":"2024-03-08T23:53:45.638049+0000","last_clean":"2024-03-08T23:53:45.638049+0000","last_became_active":"2024-03-08T23:51:56.280329+0000","last_became_peered":"2024-03-08T23:51:56.280329+0000","last_unstale":"2024-03-08T23:53:45.638049+0000","last_undegraded":"2024-03-08T23:53:45.638049+0000","last_fullsized":"2024-03-08T23:53:45.638049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:47:40.829142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629763+0000","last_change":"2024-03-08T23:51:51.238058+0000","last_active":"2024-03-08T23:53:45.629763+0000","last_peered":"2024-03-08T23:53:45.629763+0000","last_clean":"2024-03-08T23:53:45.629763+0000","last_became_active":"2024-03-08T23:51:51.237727+0000","last_became_peered":"2024-03-08T23:51:51.237727+0000","last_unstale":"2024-03-08T23:53:45.629763+0000","last_undegraded":"2024-03-08T23:53:45.629763+0000","last_fullsized":"2024-03-08T23:53:45.629763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:13:54.080004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000263+0000","last_change":"2024-03-08T23:51:52.249830+0000","last_active":"2024-03-08T23:53:46.000263+0000","last_peered":"2024-03-08T23:53:46.000263+0000","last_clean":"2024-03-08T23:53:46.000263+0000","last_became_active":"2024-03-08T23:51:52.249518+0000","last_became_peered":"2024-03-08T23:51:52.249518+0000","last_unstale":"2024-03-08T23:53:46.000263+0000","last_undegraded":"2024-03-08T23:53:46.000263+0000","last_fullsized":"2024-03-08T23:53:46.000263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:24:33.721423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629649+0000","last_change":"2024-03-08T23:51:56.288595+0000","last_active":"2024-03-08T23:53:45.629649+0000","last_peered":"2024-03-08T23:53:45.629649+0000","last_clean":"2024-03-08T23:53:45.629649+0000","last_became_active":"2024-03-08T23:51:56.288146+0000","last_became_peered":"2024-03-08T23:51:56.288146+0000","last_unstale":"2024-03-08T23:53:45.629649+0000","last_undegraded":"2024-03-08T23:53:45.629649+0000","last_fullsized":"2024-03-08T23:53:45.629649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:34:58.321754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000190+0000","last_change":"2024-03-08T23:51:51.242065+0000","last_active":"2024-03-08T23:53:46.000190+0000","last_peered":"2024-03-08T23:53:46.000190+0000","last_clean":"2024-03-08T23:53:46.000190+0000","last_became_active":"2024-03-08T23:51:51.241513+0000","last_became_peered":"2024-03-08T23:51:51.241513+0000","last_unstale":"2024-03-08T23:53:46.000190+0000","last_undegraded":"2024-03-08T23:53:46.000190+0000","last_fullsized":"2024-03-08T23:53:46.000190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:08:02.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637325+0000","last_change":"2024-03-08T23:51:52.243074+0000","last_active":"2024-03-08T23:53:45.637325+0000","last_peered":"2024-03-08T23:53:45.637325+0000","last_clean":"2024-03-08T23:53:45.637325+0000","last_became_active":"2024-03-08T23:51:52.242736+0000","last_became_peered":"2024-03-08T23:51:52.242736+0000","last_unstale":"2024-03-08T23:53:45.637325+0000","last_undegraded":"2024-03-08T23:53:45.637325+0000","last_fullsized":"2024-03-08T23:53:45.637325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:57:37.174056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875547+0000","last_change":"2024-03-08T23:51:56.276284+0000","last_active":"2024-03-08T23:53:45.875547+0000","last_peered":"2024-03-08T23:53:45.875547+0000","last_clean":"2024-03-08T23:53:45.875547+0000","last_became_active":"2024-03-08T23:51:56.275800+0000","last_became_peered":"2024-03-08T23:51:56.275800+0000","last_unstale":"2024-03-08T23:53:45.875547+0000","last_undegraded":"2024-03-08T23:53:45.875547+0000","last_fullsized":"2024-03-08T23:53:45.875547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:21:11.221266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636532+0000","last_change":"2024-03-08T23:51:51.245548+0000","last_active":"2024-03-08T23:53:45.636532+0000","last_peered":"2024-03-08T23:53:45.636532+0000","last_clean":"2024-03-08T23:53:45.636532+0000","last_became_active":"2024-03-08T23:51:51.245238+0000","last_became_peered":"2024-03-08T23:51:51.245238+0000","last_unstale":"2024-03-08T23:53:45.636532+0000","last_undegraded":"2024-03-08T23:53:45.636532+0000","last_fullsized":"2024-03-08T23:53:45.636532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:21:19.557196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.635139+0000","last_change":"2024-03-08T23:51:52.244422+0000","last_active":"2024-03-08T23:53:45.635139+0000","last_peered":"2024-03-08T23:53:45.635139+0000","last_clean":"2024-03-08T23:53:45.635139+0000","last_became_active":"2024-03-08T23:51:52.244032+0000","last_became_peered":"2024-03-08T23:51:52.244032+0000","last_unstale":"2024-03-08T23:53:45.635139+0000","last_undegraded":"2024-03-08T23:53:45.635139+0000","last_fullsized":"2024-03-08T23:53:45.635139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:04.064691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.630134+0000","last_change":"2024-03-08T23:51:56.288632+0000","last_active":"2024-03-08T23:53:45.630134+0000","last_peered":"2024-03-08T23:53:45.630134+0000","last_clean":"2024-03-08T23:53:45.630134+0000","last_became_active":"2024-03-08T23:51:56.288175+0000","last_became_peered":"2024-03-08T23:51:56.288175+0000","last_unstale":"2024-03-08T23:53:45.630134+0000","last_undegraded":"2024-03-08T23:53:45.630134+0000","last_fullsized":"2024-03-08T23:53:45.630134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:54:51.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.645297+0000","last_change":"2024-03-08T23:53:45.645297+0000","last_active":"2024-03-08T23:53:45.645297+0000","last_peered":"2024-03-08T23:53:45.645297+0000","last_clean":"2024-03-08T23:53:45.645297+0000","last_became_active":"2024-03-08T23:53:45.639667+0000","last_became_peered":"2024-03-08T23:53:45.639667+0000","last_unstale":"2024-03-08T23:53:45.645297+0000","last_undegraded":"2024-03-08T23:53:45.645297+0000","last_fullsized":"2024-03-08T23:53:45.645297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:30:30.658165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636550+0000","last_change":"2024-03-08T23:51:52.250778+0000","last_active":"2024-03-08T23:53:45.636550+0000","last_peered":"2024-03-08T23:53:45.636550+0000","last_clean":"2024-03-08T23:53:45.636550+0000","last_became_active":"2024-03-08T23:51:52.250475+0000","last_became_peered":"2024-03-08T23:51:52.250475+0000","last_unstale":"2024-03-08T23:53:45.636550+0000","last_undegraded":"2024-03-08T23:53:45.636550+0000","last_fullsized":"2024-03-08T23:53:45.636550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:36:01.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629535+0000","last_change":"2024-03-08T23:51:56.285631+0000","last_active":"2024-03-08T23:53:45.629535+0000","last_peered":"2024-03-08T23:53:45.629535+0000","last_clean":"2024-03-08T23:53:45.629535+0000","last_became_active":"2024-03-08T23:51:56.285183+0000","last_became_peered":"2024-03-08T23:51:56.285183+0000","last_unstale":"2024-03-08T23:53:45.629535+0000","last_undegraded":"2024-03-08T23:53:45.629535+0000","last_fullsized":"2024-03-08T23:53:45.629535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:15.002086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637835+0000","last_change":"2024-03-08T23:51:51.246848+0000","last_active":"2024-03-08T23:53:45.637835+0000","last_peered":"2024-03-08T23:53:45.637835+0000","last_clean":"2024-03-08T23:53:45.637835+0000","last_became_active":"2024-03-08T23:51:51.246533+0000","last_became_peered":"2024-03-08T23:51:51.246533+0000","last_unstale":"2024-03-08T23:53:45.637835+0000","last_undegraded":"2024-03-08T23:53:45.637835+0000","last_fullsized":"2024-03-08T23:53:45.637835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:20:16.669820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002055+0000","last_change":"2024-03-08T23:51:52.251575+0000","last_active":"2024-03-08T23:53:46.002055+0000","last_peered":"2024-03-08T23:53:46.002055+0000","last_clean":"2024-03-08T23:53:46.002055+0000","last_became_active":"2024-03-08T23:51:52.251232+0000","last_became_peered":"2024-03-08T23:51:52.251232+0000","last_unstale":"2024-03-08T23:53:46.002055+0000","last_undegraded":"2024-03-08T23:53:46.002055+0000","last_fullsized":"2024-03-08T23:53:46.002055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:33.573407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000476+0000","last_change":"2024-03-08T23:52:45.586092+0000","last_active":"2024-03-08T23:53:46.000476+0000","last_peered":"2024-03-08T23:53:46.000476+0000","last_clean":"2024-03-08T23:53:46.000476+0000","last_became_active":"2024-03-08T23:52:45.585513+0000","last_became_peered":"2024-03-08T23:52:45.585513+0000","last_unstale":"2024-03-08T23:53:46.000476+0000","last_undegraded":"2024-03-08T23:53:46.000476+0000","last_fullsized":"2024-03-08T23:53:46.000476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:39:05.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638551+0000","last_change":"2024-03-08T23:51:51.242081+0000","last_active":"2024-03-08T23:53:45.638551+0000","last_peered":"2024-03-08T23:53:45.638551+0000","last_clean":"2024-03-08T23:53:45.638551+0000","last_became_active":"2024-03-08T23:51:51.241636+0000","last_became_peered":"2024-03-08T23:51:51.241636+0000","last_unstale":"2024-03-08T23:53:45.638551+0000","last_undegraded":"2024-03-08T23:53:45.638551+0000","last_fullsized":"2024-03-08T23:53:45.638551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:51:57.497723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002288+0000","last_change":"2024-03-08T23:51:52.252796+0000","last_active":"2024-03-08T23:53:46.002288+0000","last_peered":"2024-03-08T23:53:46.002288+0000","last_clean":"2024-03-08T23:53:46.002288+0000","last_became_active":"2024-03-08T23:51:52.252361+0000","last_became_peered":"2024-03-08T23:51:52.252361+0000","last_unstale":"2024-03-08T23:53:46.002288+0000","last_undegraded":"2024-03-08T23:53:46.002288+0000","last_fullsized":"2024-03-08T23:53:46.002288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:07.968262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999883+0000","last_change":"2024-03-08T23:51:56.284396+0000","last_active":"2024-03-08T23:53:45.999883+0000","last_peered":"2024-03-08T23:53:45.999883+0000","last_clean":"2024-03-08T23:53:45.999883+0000","last_became_active":"2024-03-08T23:51:56.283951+0000","last_became_peered":"2024-03-08T23:51:56.283951+0000","last_unstale":"2024-03-08T23:53:45.999883+0000","last_undegraded":"2024-03-08T23:53:45.999883+0000","last_fullsized":"2024-03-08T23:53:45.999883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:09:40.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629375+0000","last_change":"2024-03-08T23:51:51.248838+0000","last_active":"2024-03-08T23:53:45.629375+0000","last_peered":"2024-03-08T23:53:45.629375+0000","last_clean":"2024-03-08T23:53:45.629375+0000","last_became_active":"2024-03-08T23:51:51.248534+0000","last_became_peered":"2024-03-08T23:51:51.248534+0000","last_unstale":"2024-03-08T23:53:45.629375+0000","last_undegraded":"2024-03-08T23:53:45.629375+0000","last_fullsized":"2024-03-08T23:53:45.629375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:37:25.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001834+0000","last_change":"2024-03-08T23:51:52.264170+0000","last_active":"2024-03-08T23:53:46.001834+0000","last_peered":"2024-03-08T23:53:46.001834+0000","last_clean":"2024-03-08T23:53:46.001834+0000","last_became_active":"2024-03-08T23:51:52.263744+0000","last_became_peered":"2024-03-08T23:51:52.263744+0000","last_unstale":"2024-03-08T23:53:46.001834+0000","last_undegraded":"2024-03-08T23:53:46.001834+0000","last_fullsized":"2024-03-08T23:53:46.001834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:42:18.287699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628650+0000","last_change":"2024-03-08T23:51:56.283316+0000","last_active":"2024-03-08T23:53:45.628650+0000","last_peered":"2024-03-08T23:53:45.628650+0000","last_clean":"2024-03-08T23:53:45.628650+0000","last_became_active":"2024-03-08T23:51:56.282515+0000","last_became_peered":"2024-03-08T23:51:56.282515+0000","last_unstale":"2024-03-08T23:53:45.628650+0000","last_undegraded":"2024-03-08T23:53:45.628650+0000","last_fullsized":"2024-03-08T23:53:45.628650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:40:15.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626731+0000","last_change":"2024-03-08T23:51:51.240970+0000","last_active":"2024-03-08T23:53:45.626731+0000","last_peered":"2024-03-08T23:53:45.626731+0000","last_clean":"2024-03-08T23:53:45.626731+0000","last_became_active":"2024-03-08T23:51:51.239982+0000","last_became_peered":"2024-03-08T23:51:51.239982+0000","last_unstale":"2024-03-08T23:53:45.626731+0000","last_undegraded":"2024-03-08T23:53:45.626731+0000","last_fullsized":"2024-03-08T23:53:45.626731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:47:40.313753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002207+0000","last_change":"2024-03-08T23:51:52.261673+0000","last_active":"2024-03-08T23:53:46.002207+0000","last_peered":"2024-03-08T23:53:46.002207+0000","last_clean":"2024-03-08T23:53:46.002207+0000","last_became_active":"2024-03-08T23:51:52.261333+0000","last_became_peered":"2024-03-08T23:51:52.261333+0000","last_unstale":"2024-03-08T23:53:46.002207+0000","last_undegraded":"2024-03-08T23:53:46.002207+0000","last_fullsized":"2024-03-08T23:53:46.002207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:08:26.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.635064+0000","last_change":"2024-03-08T23:51:52.243189+0000","last_active":"2024-03-08T23:53:45.635064+0000","last_peered":"2024-03-08T23:53:45.635064+0000","last_clean":"2024-03-08T23:53:45.635064+0000","last_became_active":"2024-03-08T23:51:52.242870+0000","last_became_peered":"2024-03-08T23:51:52.242870+0000","last_unstale":"2024-03-08T23:53:45.635064+0000","last_undegraded":"2024-03-08T23:53:45.635064+0000","last_fullsized":"2024-03-08T23:53:45.635064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.438045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638156+0000","last_change":"2024-03-08T23:52:45.417220+0000","last_active":"2024-03-08T23:53:45.638156+0000","last_peered":"2024-03-08T23:53:45.638156+0000","last_clean":"2024-03-08T23:53:45.638156+0000","last_became_active":"2024-03-08T23:52:45.416820+0000","last_became_peered":"2024-03-08T23:52:45.416820+0000","last_unstale":"2024-03-08T23:53:45.638156+0000","last_undegraded":"2024-03-08T23:53:45.638156+0000","last_fullsized":"2024-03-08T23:53:45.638156+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:31:13.249449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637319+0000","last_change":"2024-03-08T23:51:52.251397+0000","last_active":"2024-03-08T23:53:45.637319+0000","last_peered":"2024-03-08T23:53:45.637319+0000","last_clean":"2024-03-08T23:53:45.637319+0000","last_became_active":"2024-03-08T23:51:52.251059+0000","last_became_peered":"2024-03-08T23:51:52.251059+0000","last_unstale":"2024-03-08T23:53:45.637319+0000","last_undegraded":"2024-03-08T23:53:45.637319+0000","last_fullsized":"2024-03-08T23:53:45.637319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:19:30.363457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000034+0000","last_change":"2024-03-08T23:51:52.249039+0000","last_active":"2024-03-08T23:53:46.000034+0000","last_peered":"2024-03-08T23:53:46.000034+0000","last_clean":"2024-03-08T23:53:46.000034+0000","last_became_active":"2024-03-08T23:51:52.248560+0000","last_became_peered":"2024-03-08T23:51:52.248560+0000","last_unstale":"2024-03-08T23:53:46.000034+0000","last_undegraded":"2024-03-08T23:53:46.000034+0000","last_fullsized":"2024-03-08T23:53:46.000034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626651+0000","last_change":"2024-03-08T23:51:52.247269+0000","last_active":"2024-03-08T23:53:45.626651+0000","last_peered":"2024-03-08T23:53:45.626651+0000","last_clean":"2024-03-08T23:53:45.626651+0000","last_became_active":"2024-03-08T23:51:52.246921+0000","last_became_peered":"2024-03-08T23:51:52.246921+0000","last_unstale":"2024-03-08T23:53:45.626651+0000","last_undegraded":"2024-03-08T23:53:45.626651+0000","last_fullsized":"2024-03-08T23:53:45.626651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:00:21.828530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633238+0000","last_change":"2024-03-08T23:51:52.238999+0000","last_active":"2024-03-08T23:53:45.633238+0000","last_peered":"2024-03-08T23:53:45.633238+0000","last_clean":"2024-03-08T23:53:45.633238+0000","last_became_active":"2024-03-08T23:51:52.238662+0000","last_became_peered":"2024-03-08T23:51:52.238662+0000","last_unstale":"2024-03-08T23:53:45.633238+0000","last_undegraded":"2024-03-08T23:53:45.633238+0000","last_fullsized":"2024-03-08T23:53:45.633238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:22:08.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637860+0000","last_change":"2024-03-08T23:51:52.238793+0000","last_active":"2024-03-08T23:53:45.637860+0000","last_peered":"2024-03-08T23:53:45.637860+0000","last_clean":"2024-03-08T23:53:45.637860+0000","last_became_active":"2024-03-08T23:51:52.238464+0000","last_became_peered":"2024-03-08T23:51:52.238464+0000","last_unstale":"2024-03-08T23:53:45.637860+0000","last_undegraded":"2024-03-08T23:53:45.637860+0000","last_fullsized":"2024-03-08T23:53:45.637860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:25.597428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636926+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:53:45.636926+0000","last_peered":"2024-03-08T23:53:45.636926+0000","last_clean":"2024-03-08T23:53:45.636926+0000","last_became_active":"2024-03-08T23:52:45.585280+0000","last_became_peered":"2024-03-08T23:52:45.585280+0000","last_unstale":"2024-03-08T23:53:45.636926+0000","last_undegraded":"2024-03-08T23:53:45.636926+0000","last_fullsized":"2024-03-08T23:53:45.636926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:20.029967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629208+0000","last_change":"2024-03-08T23:51:52.245288+0000","last_active":"2024-03-08T23:53:45.629208+0000","last_peered":"2024-03-08T23:53:45.629208+0000","last_clean":"2024-03-08T23:53:45.629208+0000","last_became_active":"2024-03-08T23:51:52.244960+0000","last_became_peered":"2024-03-08T23:51:52.244960+0000","last_unstale":"2024-03-08T23:53:45.629208+0000","last_undegraded":"2024-03-08T23:53:45.629208+0000","last_fullsized":"2024-03-08T23:53:45.629208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:20:39.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.874977+0000","last_change":"2024-03-08T23:52:45.418172+0000","last_active":"2024-03-08T23:53:45.874977+0000","last_peered":"2024-03-08T23:53:45.874977+0000","last_clean":"2024-03-08T23:53:45.874977+0000","last_became_active":"2024-03-08T23:52:45.417852+0000","last_became_peered":"2024-03-08T23:52:45.417852+0000","last_unstale":"2024-03-08T23:53:45.874977+0000","last_undegraded":"2024-03-08T23:53:45.874977+0000","last_fullsized":"2024-03-08T23:53:45.874977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:22:23.698238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000860+0000","last_change":"2024-03-08T23:51:52.251048+0000","last_active":"2024-03-08T23:53:46.000860+0000","last_peered":"2024-03-08T23:53:46.000860+0000","last_clean":"2024-03-08T23:53:46.000860+0000","last_became_active":"2024-03-08T23:51:52.250508+0000","last_became_peered":"2024-03-08T23:51:52.250508+0000","last_unstale":"2024-03-08T23:53:46.000860+0000","last_undegraded":"2024-03-08T23:53:46.000860+0000","last_fullsized":"2024-03-08T23:53:46.000860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:57:37.519307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626525+0000","last_change":"2024-03-08T23:51:52.244203+0000","last_active":"2024-03-08T23:53:45.626525+0000","last_peered":"2024-03-08T23:53:45.626525+0000","last_clean":"2024-03-08T23:53:45.626525+0000","last_became_active":"2024-03-08T23:51:52.243878+0000","last_became_peered":"2024-03-08T23:51:52.243878+0000","last_unstale":"2024-03-08T23:53:45.626525+0000","last_undegraded":"2024-03-08T23:53:45.626525+0000","last_fullsized":"2024-03-08T23:53:45.626525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:53:40.938657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633137+0000","last_change":"2024-03-08T23:51:52.248334+0000","last_active":"2024-03-08T23:53:45.633137+0000","last_peered":"2024-03-08T23:53:45.633137+0000","last_clean":"2024-03-08T23:53:45.633137+0000","last_became_active":"2024-03-08T23:51:52.248025+0000","last_became_peered":"2024-03-08T23:51:52.248025+0000","last_unstale":"2024-03-08T23:53:45.633137+0000","last_undegraded":"2024-03-08T23:53:45.633137+0000","last_fullsized":"2024-03-08T23:53:45.633137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:02:06.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74027,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":746,"num_read_kb":452,"num_write":182,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":60703,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":222863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":519611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1180838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":519825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":519723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1117417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":583565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":517184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":583381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1111383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T23:54:22.265 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-08T23:54:22.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T23:54:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 -- 172.21.15.22:0/653090510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 msgr2=0x7f5a54101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 --2- 172.21.15.22:0/653090510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54101750 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009900 tx=0x7f5a4c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 -- 172.21.15.22:0/653090510 shutdown_connections 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 --2- 172.21.15.22:0/653090510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a54102ee0 0x7f5a54109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 --2- 172.21.15.22:0/653090510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a54102520 0x7f5a541029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 --2- 172.21.15.22:0/653090510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54101750 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 -- 172.21.15.22:0/653090510 >> 172.21.15.22:0/653090510 conn(0x7f5a540fc8a0 msgr2=0x7f5a540fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 -- 172.21.15.22:0/653090510 shutdown_connections 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.406+0000 7f5a5944b640 1 -- 172.21.15.22:0/653090510 wait complete. 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 Processor -- start 2024-03-08T23:54:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 -- start start 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a54102520 0x7f5a54197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a54102ee0 0x7f5a5419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5410cfc0 con 0x7f5a54101330 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5410ce40 con 0x7f5a54102ee0 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a5944b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5410d140 con 0x7f5a54102520 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a52ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a52ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51928/0 (socket says 172.21.15.22:51928) 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a52ffd640 1 -- 172.21.15.22:0/3597068083 learned_addr learned my addr 172.21.15.22:0/3597068083 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a537fe640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a54102ee0 0x7f5a5419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.407+0000 7f5a527fc640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a54102520 0x7f5a54197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a52ffd640 1 -- 172.21.15.22:0/3597068083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a54102520 msgr2=0x7f5a54197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a52ffd640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a54102520 0x7f5a54197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a52ffd640 1 -- 172.21.15.22:0/3597068083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a54102ee0 msgr2=0x7f5a5419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a52ffd640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a54102ee0 0x7f5a5419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a52ffd640 1 -- 172.21.15.22:0/3597068083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5419f290 con 0x7f5a54101330 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a527fc640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a54102520 0x7f5a54197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a52ffd640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54197670 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005ab0 tx=0x7f5a4c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a4bfff640 1 -- 172.21.15.22:0/3597068083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c041070 con 0x7f5a54101330 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a4bfff640 1 -- 172.21.15.22:0/3597068083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a4c0054e0 con 0x7f5a54101330 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5419f520 con 0x7f5a54101330 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a4bfff640 1 -- 172.21.15.22:0/3597068083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c034440 con 0x7f5a54101330 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.408+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5419fa10 con 0x7f5a54101330 2024-03-08T23:54:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.409+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a54108720 con 0x7f5a54101330 2024-03-08T23:54:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.410+0000 7f5a4bfff640 1 -- 172.21.15.22:0/3597068083 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f5a4c044070 con 0x7f5a54101330 2024-03-08T23:54:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.410+0000 7f5a4bfff640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5a400a2890 0x7f5a400a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.411+0000 7f5a527fc640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5a400a2890 0x7f5a400a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.411+0000 7f5a4bfff640 1 -- 172.21.15.22:0/3597068083 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5a4c0f0ae0 con 0x7f5a54101330 2024-03-08T23:54:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.411+0000 7f5a527fc640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5a400a2890 0x7f5a400a4d50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005fd0 tx=0x7f5a3c005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.415+0000 7f5a4bfff640 1 -- 172.21.15.22:0/3597068083 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5a4c0bd7a0 con 0x7f5a54101330 2024-03-08T23:54:22.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.596+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a54061730 con 0x7f5a400a2890 2024-03-08T23:54:22.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.606+0000 7f5a4bfff640 1 -- 172.21.15.22:0/3597068083 <== mgr.14152 v2:172.21.15.22:6800/2813827512 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340501 (secure 0 0 0) 0x7f5a54061730 con 0x7f5a400a2890 2024-03-08T23:54:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:22.608 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-03-08T23:54:22.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5a400a2890 msgr2=0x7f5a400a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5a400a2890 0x7f5a400a4d50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005fd0 tx=0x7f5a3c005ea0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 msgr2=0x7f5a54197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54197670 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005ab0 tx=0x7f5a4c005dc0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 shutdown_connections 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a54102ee0 0x7f5a5419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a54102520 0x7f5a54197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f5a400a2890 0x7f5a400a4d50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.609+0000 7f5a5944b640 1 --2- 172.21.15.22:0/3597068083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a54101330 0x7f5a54197670 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.610+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 >> 172.21.15.22:0/3597068083 conn(0x7f5a540fc8a0 msgr2=0x7f5a541058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.610+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 shutdown_connections 2024-03-08T23:54:22.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.610+0000 7f5a5944b640 1 -- 172.21.15.22:0/3597068083 wait complete. 2024-03-08T23:54:22.617 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-03-08T23:54:21.826311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664629,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":882,"num_read_kb":571,"num_write":365,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1241556,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363676,"kb_used_data":14488,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578148,"statfs":{"total":1151940427776,"available":1151568023552,"internally_reserved":0,"allocated":14835712,"data_stored":8199512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6771,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":38,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003064"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633752+0000","last_change":"2024-03-08T23:51:52.246270+0000","last_active":"2024-03-08T23:53:45.633752+0000","last_peered":"2024-03-08T23:53:45.633752+0000","last_clean":"2024-03-08T23:53:45.633752+0000","last_became_active":"2024-03-08T23:51:52.245321+0000","last_became_peered":"2024-03-08T23:51:52.245321+0000","last_unstale":"2024-03-08T23:53:45.633752+0000","last_undegraded":"2024-03-08T23:53:45.633752+0000","last_fullsized":"2024-03-08T23:53:45.633752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:39.993965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000208+0000","last_change":"2024-03-08T23:51:52.245433+0000","last_active":"2024-03-08T23:53:46.000208+0000","last_peered":"2024-03-08T23:53:46.000208+0000","last_clean":"2024-03-08T23:53:46.000208+0000","last_became_active":"2024-03-08T23:51:52.245106+0000","last_became_peered":"2024-03-08T23:51:52.245106+0000","last_unstale":"2024-03-08T23:53:46.000208+0000","last_undegraded":"2024-03-08T23:53:46.000208+0000","last_fullsized":"2024-03-08T23:53:46.000208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:02:20.755123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000027+0000","last_change":"2024-03-08T23:51:52.244112+0000","last_active":"2024-03-08T23:53:46.000027+0000","last_peered":"2024-03-08T23:53:46.000027+0000","last_clean":"2024-03-08T23:53:46.000027+0000","last_became_active":"2024-03-08T23:51:52.243783+0000","last_became_peered":"2024-03-08T23:51:52.243783+0000","last_unstale":"2024-03-08T23:53:46.000027+0000","last_undegraded":"2024-03-08T23:53:46.000027+0000","last_fullsized":"2024-03-08T23:53:46.000027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:48:43.719188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875262+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:53:45.875262+0000","last_peered":"2024-03-08T23:53:45.875262+0000","last_clean":"2024-03-08T23:53:45.875262+0000","last_became_active":"2024-03-08T23:52:45.585292+0000","last_became_peered":"2024-03-08T23:52:45.585292+0000","last_unstale":"2024-03-08T23:53:45.875262+0000","last_undegraded":"2024-03-08T23:53:45.875262+0000","last_fullsized":"2024-03-08T23:53:45.875262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:27:12.230920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637664+0000","last_change":"2024-03-08T23:51:52.247091+0000","last_active":"2024-03-08T23:53:45.637664+0000","last_peered":"2024-03-08T23:53:45.637664+0000","last_clean":"2024-03-08T23:53:45.637664+0000","last_became_active":"2024-03-08T23:51:52.246761+0000","last_became_peered":"2024-03-08T23:51:52.246761+0000","last_unstale":"2024-03-08T23:53:45.637664+0000","last_undegraded":"2024-03-08T23:53:45.637664+0000","last_fullsized":"2024-03-08T23:53:45.637664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:46:46.810857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001715+0000","last_change":"2024-03-08T23:51:52.257558+0000","last_active":"2024-03-08T23:53:46.001715+0000","last_peered":"2024-03-08T23:53:46.001715+0000","last_clean":"2024-03-08T23:53:46.001715+0000","last_became_active":"2024-03-08T23:51:52.257116+0000","last_became_peered":"2024-03-08T23:51:52.257116+0000","last_unstale":"2024-03-08T23:53:46.001715+0000","last_undegraded":"2024-03-08T23:53:46.001715+0000","last_fullsized":"2024-03-08T23:53:46.001715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:13:05.325780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000412+0000","last_change":"2024-03-08T23:51:52.253624+0000","last_active":"2024-03-08T23:53:46.000412+0000","last_peered":"2024-03-08T23:53:46.000412+0000","last_clean":"2024-03-08T23:53:46.000412+0000","last_became_active":"2024-03-08T23:51:52.253284+0000","last_became_peered":"2024-03-08T23:51:52.253284+0000","last_unstale":"2024-03-08T23:53:46.000412+0000","last_undegraded":"2024-03-08T23:53:46.000412+0000","last_fullsized":"2024-03-08T23:53:46.000412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:09:33.789502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636159+0000","last_change":"2024-03-08T23:51:52.255321+0000","last_active":"2024-03-08T23:53:45.636159+0000","last_peered":"2024-03-08T23:53:45.636159+0000","last_clean":"2024-03-08T23:53:45.636159+0000","last_became_active":"2024-03-08T23:51:52.254975+0000","last_became_peered":"2024-03-08T23:51:52.254975+0000","last_unstale":"2024-03-08T23:53:45.636159+0000","last_undegraded":"2024-03-08T23:53:45.636159+0000","last_fullsized":"2024-03-08T23:53:45.636159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:02:56.788519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001266+0000","last_change":"2024-03-08T23:51:52.244623+0000","last_active":"2024-03-08T23:53:46.001266+0000","last_peered":"2024-03-08T23:53:46.001266+0000","last_clean":"2024-03-08T23:53:46.001266+0000","last_became_active":"2024-03-08T23:51:52.244162+0000","last_became_peered":"2024-03-08T23:51:52.244162+0000","last_unstale":"2024-03-08T23:53:46.001266+0000","last_undegraded":"2024-03-08T23:53:46.001266+0000","last_fullsized":"2024-03-08T23:53:46.001266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:30:43.050616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001879+0000","last_change":"2024-03-08T23:51:52.258006+0000","last_active":"2024-03-08T23:53:46.001879+0000","last_peered":"2024-03-08T23:53:46.001879+0000","last_clean":"2024-03-08T23:53:46.001879+0000","last_became_active":"2024-03-08T23:51:52.256770+0000","last_became_peered":"2024-03-08T23:51:52.256770+0000","last_unstale":"2024-03-08T23:53:46.001879+0000","last_undegraded":"2024-03-08T23:53:46.001879+0000","last_fullsized":"2024-03-08T23:53:46.001879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:18.584212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002234+0000","last_change":"2024-03-08T23:51:52.260530+0000","last_active":"2024-03-08T23:53:46.002234+0000","last_peered":"2024-03-08T23:53:46.002234+0000","last_clean":"2024-03-08T23:53:46.002234+0000","last_became_active":"2024-03-08T23:51:52.260193+0000","last_became_peered":"2024-03-08T23:51:52.260193+0000","last_unstale":"2024-03-08T23:53:46.002234+0000","last_undegraded":"2024-03-08T23:53:46.002234+0000","last_fullsized":"2024-03-08T23:53:46.002234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:02.549683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629380+0000","last_change":"2024-03-08T23:51:52.248679+0000","last_active":"2024-03-08T23:53:45.629380+0000","last_peered":"2024-03-08T23:53:45.629380+0000","last_clean":"2024-03-08T23:53:45.629380+0000","last_became_active":"2024-03-08T23:51:52.248358+0000","last_became_peered":"2024-03-08T23:51:52.248358+0000","last_unstale":"2024-03-08T23:53:45.629380+0000","last_undegraded":"2024-03-08T23:53:45.629380+0000","last_fullsized":"2024-03-08T23:53:45.629380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:26.680887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626733+0000","last_change":"2024-03-08T23:52:45.415807+0000","last_active":"2024-03-08T23:53:45.626733+0000","last_peered":"2024-03-08T23:53:45.626733+0000","last_clean":"2024-03-08T23:53:45.626733+0000","last_became_active":"2024-03-08T23:52:45.415456+0000","last_became_peered":"2024-03-08T23:52:45.415456+0000","last_unstale":"2024-03-08T23:53:45.626733+0000","last_undegraded":"2024-03-08T23:53:45.626733+0000","last_fullsized":"2024-03-08T23:53:45.626733+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:12:01.942744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999714+0000","last_change":"2024-03-08T23:51:52.245394+0000","last_active":"2024-03-08T23:53:45.999714+0000","last_peered":"2024-03-08T23:53:45.999714+0000","last_clean":"2024-03-08T23:53:45.999714+0000","last_became_active":"2024-03-08T23:51:52.245038+0000","last_became_peered":"2024-03-08T23:51:52.245038+0000","last_unstale":"2024-03-08T23:53:45.999714+0000","last_undegraded":"2024-03-08T23:53:45.999714+0000","last_fullsized":"2024-03-08T23:53:45.999714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:15.339295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999804+0000","last_change":"2024-03-08T23:51:52.250861+0000","last_active":"2024-03-08T23:53:45.999804+0000","last_peered":"2024-03-08T23:53:45.999804+0000","last_clean":"2024-03-08T23:53:45.999804+0000","last_became_active":"2024-03-08T23:51:52.250548+0000","last_became_peered":"2024-03-08T23:51:52.250548+0000","last_unstale":"2024-03-08T23:53:45.999804+0000","last_undegraded":"2024-03-08T23:53:45.999804+0000","last_fullsized":"2024-03-08T23:53:45.999804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:34:00.280043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000326+0000","last_change":"2024-03-08T23:51:52.254996+0000","last_active":"2024-03-08T23:53:46.000326+0000","last_peered":"2024-03-08T23:53:46.000326+0000","last_clean":"2024-03-08T23:53:46.000326+0000","last_became_active":"2024-03-08T23:51:52.254634+0000","last_became_peered":"2024-03-08T23:51:52.254634+0000","last_unstale":"2024-03-08T23:53:46.000326+0000","last_undegraded":"2024-03-08T23:53:46.000326+0000","last_fullsized":"2024-03-08T23:53:46.000326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:15.449988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633637+0000","last_change":"2024-03-08T23:51:52.246558+0000","last_active":"2024-03-08T23:53:45.633637+0000","last_peered":"2024-03-08T23:53:45.633637+0000","last_clean":"2024-03-08T23:53:45.633637+0000","last_became_active":"2024-03-08T23:51:52.245741+0000","last_became_peered":"2024-03-08T23:51:52.245741+0000","last_unstale":"2024-03-08T23:53:45.633637+0000","last_undegraded":"2024-03-08T23:53:45.633637+0000","last_fullsized":"2024-03-08T23:53:45.633637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:36.737017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875498+0000","last_change":"2024-03-08T23:51:52.240111+0000","last_active":"2024-03-08T23:53:45.875498+0000","last_peered":"2024-03-08T23:53:45.875498+0000","last_clean":"2024-03-08T23:53:45.875498+0000","last_became_active":"2024-03-08T23:51:52.238961+0000","last_became_peered":"2024-03-08T23:51:52.238961+0000","last_unstale":"2024-03-08T23:53:45.875498+0000","last_undegraded":"2024-03-08T23:53:45.875498+0000","last_fullsized":"2024-03-08T23:53:45.875498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:03:55.313117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638666+0000","last_change":"2024-03-08T23:51:52.245979+0000","last_active":"2024-03-08T23:53:45.638666+0000","last_peered":"2024-03-08T23:53:45.638666+0000","last_clean":"2024-03-08T23:53:45.638666+0000","last_became_active":"2024-03-08T23:51:52.245666+0000","last_became_peered":"2024-03-08T23:51:52.245666+0000","last_unstale":"2024-03-08T23:53:45.638666+0000","last_undegraded":"2024-03-08T23:53:45.638666+0000","last_fullsized":"2024-03-08T23:53:45.638666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:20.789144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638219+0000","last_change":"2024-03-08T23:51:56.288161+0000","last_active":"2024-03-08T23:53:45.638219+0000","last_peered":"2024-03-08T23:53:45.638219+0000","last_clean":"2024-03-08T23:53:45.638219+0000","last_became_active":"2024-03-08T23:51:56.287778+0000","last_became_peered":"2024-03-08T23:51:56.287778+0000","last_unstale":"2024-03-08T23:53:45.638219+0000","last_undegraded":"2024-03-08T23:53:45.638219+0000","last_fullsized":"2024-03-08T23:53:45.638219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:12:59.636228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875582+0000","last_change":"2024-03-08T23:51:51.244911+0000","last_active":"2024-03-08T23:53:45.875582+0000","last_peered":"2024-03-08T23:53:45.875582+0000","last_clean":"2024-03-08T23:53:45.875582+0000","last_became_active":"2024-03-08T23:51:51.244561+0000","last_became_peered":"2024-03-08T23:51:51.244561+0000","last_unstale":"2024-03-08T23:53:45.875582+0000","last_undegraded":"2024-03-08T23:53:45.875582+0000","last_fullsized":"2024-03-08T23:53:45.875582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:58:45.908307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636719+0000","last_change":"2024-03-08T23:51:52.253854+0000","last_active":"2024-03-08T23:53:45.636719+0000","last_peered":"2024-03-08T23:53:45.636719+0000","last_clean":"2024-03-08T23:53:45.636719+0000","last_became_active":"2024-03-08T23:51:52.253538+0000","last_became_peered":"2024-03-08T23:51:52.253538+0000","last_unstale":"2024-03-08T23:53:45.636719+0000","last_undegraded":"2024-03-08T23:53:45.636719+0000","last_fullsized":"2024-03-08T23:53:45.636719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:33.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628202+0000","last_change":"2024-03-08T23:51:56.283307+0000","last_active":"2024-03-08T23:53:45.628202+0000","last_peered":"2024-03-08T23:53:45.628202+0000","last_clean":"2024-03-08T23:53:45.628202+0000","last_became_active":"2024-03-08T23:51:56.282868+0000","last_became_peered":"2024-03-08T23:51:56.282868+0000","last_unstale":"2024-03-08T23:53:45.628202+0000","last_undegraded":"2024-03-08T23:53:45.628202+0000","last_fullsized":"2024-03-08T23:53:45.628202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:15.027743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636962+0000","last_change":"2024-03-08T23:51:51.247368+0000","last_active":"2024-03-08T23:53:45.636962+0000","last_peered":"2024-03-08T23:53:45.636962+0000","last_clean":"2024-03-08T23:53:45.636962+0000","last_became_active":"2024-03-08T23:51:51.247023+0000","last_became_peered":"2024-03-08T23:51:51.247023+0000","last_unstale":"2024-03-08T23:53:45.636962+0000","last_undegraded":"2024-03-08T23:53:45.636962+0000","last_fullsized":"2024-03-08T23:53:45.636962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:28.311850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000106+0000","last_change":"2024-03-08T23:51:52.245805+0000","last_active":"2024-03-08T23:53:46.000106+0000","last_peered":"2024-03-08T23:53:46.000106+0000","last_clean":"2024-03-08T23:53:46.000106+0000","last_became_active":"2024-03-08T23:51:52.244455+0000","last_became_peered":"2024-03-08T23:51:52.244455+0000","last_unstale":"2024-03-08T23:53:46.000106+0000","last_undegraded":"2024-03-08T23:53:46.000106+0000","last_fullsized":"2024-03-08T23:53:46.000106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:11:17.809148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628232+0000","last_change":"2024-03-08T23:51:56.286666+0000","last_active":"2024-03-08T23:53:45.628232+0000","last_peered":"2024-03-08T23:53:45.628232+0000","last_clean":"2024-03-08T23:53:45.628232+0000","last_became_active":"2024-03-08T23:51:56.285968+0000","last_became_peered":"2024-03-08T23:51:56.285968+0000","last_unstale":"2024-03-08T23:53:45.628232+0000","last_undegraded":"2024-03-08T23:53:45.628232+0000","last_fullsized":"2024-03-08T23:53:45.628232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:51.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638792+0000","last_change":"2024-03-08T23:51:51.247684+0000","last_active":"2024-03-08T23:53:45.638792+0000","last_peered":"2024-03-08T23:53:45.638792+0000","last_clean":"2024-03-08T23:53:45.638792+0000","last_became_active":"2024-03-08T23:51:51.247374+0000","last_became_peered":"2024-03-08T23:51:51.247374+0000","last_unstale":"2024-03-08T23:53:45.638792+0000","last_undegraded":"2024-03-08T23:53:45.638792+0000","last_fullsized":"2024-03-08T23:53:45.638792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:20.425302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000566+0000","last_change":"2024-03-08T23:51:52.248702+0000","last_active":"2024-03-08T23:53:46.000566+0000","last_peered":"2024-03-08T23:53:46.000566+0000","last_clean":"2024-03-08T23:53:46.000566+0000","last_became_active":"2024-03-08T23:51:52.248388+0000","last_became_peered":"2024-03-08T23:51:52.248388+0000","last_unstale":"2024-03-08T23:53:46.000566+0000","last_undegraded":"2024-03-08T23:53:46.000566+0000","last_fullsized":"2024-03-08T23:53:46.000566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:27:12.251957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628238+0000","last_change":"2024-03-08T23:51:56.286572+0000","last_active":"2024-03-08T23:53:45.628238+0000","last_peered":"2024-03-08T23:53:45.628238+0000","last_clean":"2024-03-08T23:53:45.628238+0000","last_became_active":"2024-03-08T23:51:56.286066+0000","last_became_peered":"2024-03-08T23:51:56.286066+0000","last_unstale":"2024-03-08T23:53:45.628238+0000","last_undegraded":"2024-03-08T23:53:45.628238+0000","last_fullsized":"2024-03-08T23:53:45.628238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:13.694615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638236+0000","last_change":"2024-03-08T23:51:51.244207+0000","last_active":"2024-03-08T23:53:45.638236+0000","last_peered":"2024-03-08T23:53:45.638236+0000","last_clean":"2024-03-08T23:53:45.638236+0000","last_became_active":"2024-03-08T23:51:51.241605+0000","last_became_peered":"2024-03-08T23:51:51.241605+0000","last_unstale":"2024-03-08T23:53:45.638236+0000","last_undegraded":"2024-03-08T23:53:45.638236+0000","last_fullsized":"2024-03-08T23:53:45.638236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:03:51.766681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629253+0000","last_change":"2024-03-08T23:51:52.251015+0000","last_active":"2024-03-08T23:53:45.629253+0000","last_peered":"2024-03-08T23:53:45.629253+0000","last_clean":"2024-03-08T23:53:45.629253+0000","last_became_active":"2024-03-08T23:51:52.250690+0000","last_became_peered":"2024-03-08T23:51:52.250690+0000","last_unstale":"2024-03-08T23:53:45.629253+0000","last_undegraded":"2024-03-08T23:53:45.629253+0000","last_fullsized":"2024-03-08T23:53:45.629253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:57:23.033172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638368+0000","last_change":"2024-03-08T23:51:56.281066+0000","last_active":"2024-03-08T23:53:45.638368+0000","last_peered":"2024-03-08T23:53:45.638368+0000","last_clean":"2024-03-08T23:53:45.638368+0000","last_became_active":"2024-03-08T23:51:56.280654+0000","last_became_peered":"2024-03-08T23:51:56.280654+0000","last_unstale":"2024-03-08T23:53:45.638368+0000","last_undegraded":"2024-03-08T23:53:45.638368+0000","last_fullsized":"2024-03-08T23:53:45.638368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:53.999115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:16.524728+0000","last_change":"2024-03-08T23:51:51.245947+0000","last_active":"2024-03-08T23:54:16.524728+0000","last_peered":"2024-03-08T23:54:16.524728+0000","last_clean":"2024-03-08T23:54:16.524728+0000","last_became_active":"2024-03-08T23:51:51.245505+0000","last_became_peered":"2024-03-08T23:51:51.245505+0000","last_unstale":"2024-03-08T23:54:16.524728+0000","last_undegraded":"2024-03-08T23:54:16.524728+0000","last_fullsized":"2024-03-08T23:54:16.524728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:20:44.692598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637141+0000","last_change":"2024-03-08T23:51:52.241590+0000","last_active":"2024-03-08T23:53:45.637141+0000","last_peered":"2024-03-08T23:53:45.637141+0000","last_clean":"2024-03-08T23:53:45.637141+0000","last_became_active":"2024-03-08T23:51:52.241296+0000","last_became_peered":"2024-03-08T23:51:52.241296+0000","last_unstale":"2024-03-08T23:53:45.637141+0000","last_undegraded":"2024-03-08T23:53:45.637141+0000","last_fullsized":"2024-03-08T23:53:45.637141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:52:48.042360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638174+0000","last_change":"2024-03-08T23:52:45.585946+0000","last_active":"2024-03-08T23:53:45.638174+0000","last_peered":"2024-03-08T23:53:45.638174+0000","last_clean":"2024-03-08T23:53:45.638174+0000","last_became_active":"2024-03-08T23:52:45.585470+0000","last_became_peered":"2024-03-08T23:52:45.585470+0000","last_unstale":"2024-03-08T23:53:45.638174+0000","last_undegraded":"2024-03-08T23:53:45.638174+0000","last_fullsized":"2024-03-08T23:53:45.638174+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:28:36.264887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001568+0000","last_change":"2024-03-08T23:51:51.241384+0000","last_active":"2024-03-08T23:53:46.001568+0000","last_peered":"2024-03-08T23:53:46.001568+0000","last_clean":"2024-03-08T23:53:46.001568+0000","last_became_active":"2024-03-08T23:51:51.241018+0000","last_became_peered":"2024-03-08T23:51:51.241018+0000","last_unstale":"2024-03-08T23:53:46.001568+0000","last_undegraded":"2024-03-08T23:53:46.001568+0000","last_fullsized":"2024-03-08T23:53:46.001568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:29:50.860663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629076+0000","last_change":"2024-03-08T23:51:52.250207+0000","last_active":"2024-03-08T23:53:45.629076+0000","last_peered":"2024-03-08T23:53:45.629076+0000","last_clean":"2024-03-08T23:53:45.629076+0000","last_became_active":"2024-03-08T23:51:52.249863+0000","last_became_peered":"2024-03-08T23:51:52.249863+0000","last_unstale":"2024-03-08T23:53:45.629076+0000","last_undegraded":"2024-03-08T23:53:45.629076+0000","last_fullsized":"2024-03-08T23:53:45.629076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:18:48.732419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633519+0000","last_change":"2024-03-08T23:51:56.282092+0000","last_active":"2024-03-08T23:53:45.633519+0000","last_peered":"2024-03-08T23:53:45.633519+0000","last_clean":"2024-03-08T23:53:45.633519+0000","last_became_active":"2024-03-08T23:51:56.281657+0000","last_became_peered":"2024-03-08T23:51:56.281657+0000","last_unstale":"2024-03-08T23:53:45.633519+0000","last_undegraded":"2024-03-08T23:53:45.633519+0000","last_fullsized":"2024-03-08T23:53:45.633519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:50.440520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626316+0000","last_change":"2024-03-08T23:51:51.236816+0000","last_active":"2024-03-08T23:53:45.626316+0000","last_peered":"2024-03-08T23:53:45.626316+0000","last_clean":"2024-03-08T23:53:45.626316+0000","last_became_active":"2024-03-08T23:51:51.236485+0000","last_became_peered":"2024-03-08T23:51:51.236485+0000","last_unstale":"2024-03-08T23:53:45.626316+0000","last_undegraded":"2024-03-08T23:53:45.626316+0000","last_fullsized":"2024-03-08T23:53:45.626316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:27.300950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636406+0000","last_change":"2024-03-08T23:51:52.252146+0000","last_active":"2024-03-08T23:53:45.636406+0000","last_peered":"2024-03-08T23:53:45.636406+0000","last_clean":"2024-03-08T23:53:45.636406+0000","last_became_active":"2024-03-08T23:51:52.244201+0000","last_became_peered":"2024-03-08T23:51:52.244201+0000","last_unstale":"2024-03-08T23:53:45.636406+0000","last_undegraded":"2024-03-08T23:53:45.636406+0000","last_fullsized":"2024-03-08T23:53:45.636406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:07.647078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000164+0000","last_change":"2024-03-08T23:51:56.279727+0000","last_active":"2024-03-08T23:53:46.000164+0000","last_peered":"2024-03-08T23:53:46.000164+0000","last_clean":"2024-03-08T23:53:46.000164+0000","last_became_active":"2024-03-08T23:51:56.279184+0000","last_became_peered":"2024-03-08T23:51:56.279184+0000","last_unstale":"2024-03-08T23:53:46.000164+0000","last_undegraded":"2024-03-08T23:53:46.000164+0000","last_fullsized":"2024-03-08T23:53:46.000164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:28:21.804410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629021+0000","last_change":"2024-03-08T23:51:51.244812+0000","last_active":"2024-03-08T23:53:45.629021+0000","last_peered":"2024-03-08T23:53:45.629021+0000","last_clean":"2024-03-08T23:53:45.629021+0000","last_became_active":"2024-03-08T23:51:51.244513+0000","last_became_peered":"2024-03-08T23:51:51.244513+0000","last_unstale":"2024-03-08T23:53:45.629021+0000","last_undegraded":"2024-03-08T23:53:45.629021+0000","last_fullsized":"2024-03-08T23:53:45.629021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:56:07.235214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875255+0000","last_change":"2024-03-08T23:51:52.244566+0000","last_active":"2024-03-08T23:53:45.875255+0000","last_peered":"2024-03-08T23:53:45.875255+0000","last_clean":"2024-03-08T23:53:45.875255+0000","last_became_active":"2024-03-08T23:51:52.244250+0000","last_became_peered":"2024-03-08T23:51:52.244250+0000","last_unstale":"2024-03-08T23:53:45.875255+0000","last_undegraded":"2024-03-08T23:53:45.875255+0000","last_fullsized":"2024-03-08T23:53:45.875255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:55:50.596364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636425+0000","last_change":"2024-03-08T23:51:56.282487+0000","last_active":"2024-03-08T23:53:45.636425+0000","last_peered":"2024-03-08T23:53:45.636425+0000","last_clean":"2024-03-08T23:53:45.636425+0000","last_became_active":"2024-03-08T23:51:56.282104+0000","last_became_peered":"2024-03-08T23:51:56.282104+0000","last_unstale":"2024-03-08T23:53:45.636425+0000","last_undegraded":"2024-03-08T23:53:45.636425+0000","last_fullsized":"2024-03-08T23:53:45.636425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:23:23.787431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.527612+0000","last_change":"2024-03-08T23:51:51.236612+0000","last_active":"2024-03-08T23:54:18.527612+0000","last_peered":"2024-03-08T23:54:18.527612+0000","last_clean":"2024-03-08T23:54:18.527612+0000","last_became_active":"2024-03-08T23:51:51.236274+0000","last_became_peered":"2024-03-08T23:51:51.236274+0000","last_unstale":"2024-03-08T23:54:18.527612+0000","last_undegraded":"2024-03-08T23:54:18.527612+0000","last_fullsized":"2024-03-08T23:54:18.527612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:37.551870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000849+0000","last_change":"2024-03-08T23:51:52.244564+0000","last_active":"2024-03-08T23:53:46.000849+0000","last_peered":"2024-03-08T23:53:46.000849+0000","last_clean":"2024-03-08T23:53:46.000849+0000","last_became_active":"2024-03-08T23:51:52.243651+0000","last_became_peered":"2024-03-08T23:51:52.243651+0000","last_unstale":"2024-03-08T23:53:46.000849+0000","last_undegraded":"2024-03-08T23:53:46.000849+0000","last_fullsized":"2024-03-08T23:53:46.000849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:35:32.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875029+0000","last_change":"2024-03-08T23:51:56.279354+0000","last_active":"2024-03-08T23:53:45.875029+0000","last_peered":"2024-03-08T23:53:45.875029+0000","last_clean":"2024-03-08T23:53:45.875029+0000","last_became_active":"2024-03-08T23:51:56.278894+0000","last_became_peered":"2024-03-08T23:51:56.278894+0000","last_unstale":"2024-03-08T23:53:45.875029+0000","last_undegraded":"2024-03-08T23:53:45.875029+0000","last_fullsized":"2024-03-08T23:53:45.875029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:20:30.164542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002108+0000","last_change":"2024-03-08T23:51:51.246417+0000","last_active":"2024-03-08T23:53:46.002108+0000","last_peered":"2024-03-08T23:53:46.002108+0000","last_clean":"2024-03-08T23:53:46.002108+0000","last_became_active":"2024-03-08T23:51:51.246058+0000","last_became_peered":"2024-03-08T23:51:51.246058+0000","last_unstale":"2024-03-08T23:53:46.002108+0000","last_undegraded":"2024-03-08T23:53:46.002108+0000","last_fullsized":"2024-03-08T23:53:46.002108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:48:53.569295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000099+0000","last_change":"2024-03-08T23:51:52.255357+0000","last_active":"2024-03-08T23:53:46.000099+0000","last_peered":"2024-03-08T23:53:46.000099+0000","last_clean":"2024-03-08T23:53:46.000099+0000","last_became_active":"2024-03-08T23:51:52.255044+0000","last_became_peered":"2024-03-08T23:51:52.255044+0000","last_unstale":"2024-03-08T23:53:46.000099+0000","last_undegraded":"2024-03-08T23:53:46.000099+0000","last_fullsized":"2024-03-08T23:53:46.000099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:13:40.710283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628526+0000","last_change":"2024-03-08T23:51:56.284282+0000","last_active":"2024-03-08T23:53:45.628526+0000","last_peered":"2024-03-08T23:53:45.628526+0000","last_clean":"2024-03-08T23:53:45.628526+0000","last_became_active":"2024-03-08T23:51:56.283863+0000","last_became_peered":"2024-03-08T23:51:56.283863+0000","last_unstale":"2024-03-08T23:53:45.628526+0000","last_undegraded":"2024-03-08T23:53:45.628526+0000","last_fullsized":"2024-03-08T23:53:45.628526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:59:21.972283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629263+0000","last_change":"2024-03-08T23:51:51.245199+0000","last_active":"2024-03-08T23:53:45.629263+0000","last_peered":"2024-03-08T23:53:45.629263+0000","last_clean":"2024-03-08T23:53:45.629263+0000","last_became_active":"2024-03-08T23:51:51.244906+0000","last_became_peered":"2024-03-08T23:51:51.244906+0000","last_unstale":"2024-03-08T23:53:45.629263+0000","last_undegraded":"2024-03-08T23:53:45.629263+0000","last_fullsized":"2024-03-08T23:53:45.629263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:54:39.581870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000875+0000","last_change":"2024-03-08T23:51:52.251209+0000","last_active":"2024-03-08T23:53:46.000875+0000","last_peered":"2024-03-08T23:53:46.000875+0000","last_clean":"2024-03-08T23:53:46.000875+0000","last_became_active":"2024-03-08T23:51:52.250392+0000","last_became_peered":"2024-03-08T23:51:52.250392+0000","last_unstale":"2024-03-08T23:53:46.000875+0000","last_undegraded":"2024-03-08T23:53:46.000875+0000","last_fullsized":"2024-03-08T23:53:46.000875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:56:17.811505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001099+0000","last_change":"2024-03-08T23:52:45.586125+0000","last_active":"2024-03-08T23:53:46.001099+0000","last_peered":"2024-03-08T23:53:46.001099+0000","last_clean":"2024-03-08T23:53:46.001099+0000","last_became_active":"2024-03-08T23:52:45.585578+0000","last_became_peered":"2024-03-08T23:52:45.585578+0000","last_unstale":"2024-03-08T23:53:46.001099+0000","last_undegraded":"2024-03-08T23:53:46.001099+0000","last_fullsized":"2024-03-08T23:53:46.001099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:51:08.159460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"93'40","reported_seq":343,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.553773+0000","last_change":"2024-03-08T23:52:45.426691+0000","last_active":"2024-03-08T23:54:18.553773+0000","last_peered":"2024-03-08T23:54:18.553773+0000","last_clean":"2024-03-08T23:54:18.553773+0000","last_became_active":"2024-03-08T23:52:45.415742+0000","last_became_peered":"2024-03-08T23:52:45.415742+0000","last_unstale":"2024-03-08T23:54:18.553773+0000","last_undegraded":"2024-03-08T23:54:18.553773+0000","last_fullsized":"2024-03-08T23:54:18.553773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:05.389164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68654,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":187,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60681,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638662+0000","last_change":"2024-03-08T23:51:52.248715+0000","last_active":"2024-03-08T23:53:45.638662+0000","last_peered":"2024-03-08T23:53:45.638662+0000","last_clean":"2024-03-08T23:53:45.638662+0000","last_became_active":"2024-03-08T23:51:52.248258+0000","last_became_peered":"2024-03-08T23:51:52.248258+0000","last_unstale":"2024-03-08T23:53:45.638662+0000","last_undegraded":"2024-03-08T23:53:45.638662+0000","last_fullsized":"2024-03-08T23:53:45.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:54:38.185469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629520+0000","last_change":"2024-03-08T23:51:56.278268+0000","last_active":"2024-03-08T23:53:45.629520+0000","last_peered":"2024-03-08T23:53:45.629520+0000","last_clean":"2024-03-08T23:53:45.629520+0000","last_became_active":"2024-03-08T23:51:56.277831+0000","last_became_peered":"2024-03-08T23:51:56.277831+0000","last_unstale":"2024-03-08T23:53:45.629520+0000","last_undegraded":"2024-03-08T23:53:45.629520+0000","last_fullsized":"2024-03-08T23:53:45.629520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:32:13.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637455+0000","last_change":"2024-03-08T23:51:51.247553+0000","last_active":"2024-03-08T23:53:45.637455+0000","last_peered":"2024-03-08T23:53:45.637455+0000","last_clean":"2024-03-08T23:53:45.637455+0000","last_became_active":"2024-03-08T23:51:51.247225+0000","last_became_peered":"2024-03-08T23:51:51.247225+0000","last_unstale":"2024-03-08T23:53:45.637455+0000","last_undegraded":"2024-03-08T23:53:45.637455+0000","last_fullsized":"2024-03-08T23:53:45.637455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:53:29.846303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626580+0000","last_change":"2024-03-08T23:51:52.247733+0000","last_active":"2024-03-08T23:53:45.626580+0000","last_peered":"2024-03-08T23:53:45.626580+0000","last_clean":"2024-03-08T23:53:45.626580+0000","last_became_active":"2024-03-08T23:51:52.247435+0000","last_became_peered":"2024-03-08T23:51:52.247435+0000","last_unstale":"2024-03-08T23:53:45.626580+0000","last_undegraded":"2024-03-08T23:53:45.626580+0000","last_fullsized":"2024-03-08T23:53:45.626580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:46:12.739601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.632494+0000","last_change":"2024-03-08T23:51:56.279865+0000","last_active":"2024-03-08T23:53:45.632494+0000","last_peered":"2024-03-08T23:53:45.632494+0000","last_clean":"2024-03-08T23:53:45.632494+0000","last_became_active":"2024-03-08T23:51:56.279446+0000","last_became_peered":"2024-03-08T23:51:56.279446+0000","last_unstale":"2024-03-08T23:53:45.632494+0000","last_undegraded":"2024-03-08T23:53:45.632494+0000","last_fullsized":"2024-03-08T23:53:45.632494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:07:33.672490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629392+0000","last_change":"2024-03-08T23:51:51.246860+0000","last_active":"2024-03-08T23:53:45.629392+0000","last_peered":"2024-03-08T23:53:45.629392+0000","last_clean":"2024-03-08T23:53:45.629392+0000","last_became_active":"2024-03-08T23:51:51.246545+0000","last_became_peered":"2024-03-08T23:51:51.246545+0000","last_unstale":"2024-03-08T23:53:45.629392+0000","last_undegraded":"2024-03-08T23:53:45.629392+0000","last_fullsized":"2024-03-08T23:53:45.629392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:11:07.246657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001899+0000","last_change":"2024-03-08T23:51:52.247353+0000","last_active":"2024-03-08T23:53:46.001899+0000","last_peered":"2024-03-08T23:53:46.001899+0000","last_clean":"2024-03-08T23:53:46.001899+0000","last_became_active":"2024-03-08T23:51:52.247008+0000","last_became_peered":"2024-03-08T23:51:52.247008+0000","last_unstale":"2024-03-08T23:53:46.001899+0000","last_undegraded":"2024-03-08T23:53:46.001899+0000","last_fullsized":"2024-03-08T23:53:46.001899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:08:50.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875481+0000","last_change":"2024-03-08T23:51:56.284103+0000","last_active":"2024-03-08T23:53:45.875481+0000","last_peered":"2024-03-08T23:53:45.875481+0000","last_clean":"2024-03-08T23:53:45.875481+0000","last_became_active":"2024-03-08T23:51:56.283561+0000","last_became_peered":"2024-03-08T23:51:56.283561+0000","last_unstale":"2024-03-08T23:53:45.875481+0000","last_undegraded":"2024-03-08T23:53:45.875481+0000","last_fullsized":"2024-03-08T23:53:45.875481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:47:20.909292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":164,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:17.524146+0000","last_change":"2024-03-08T23:51:51.242483+0000","last_active":"2024-03-08T23:54:17.524146+0000","last_peered":"2024-03-08T23:54:17.524146+0000","last_clean":"2024-03-08T23:54:17.524146+0000","last_became_active":"2024-03-08T23:51:51.242153+0000","last_became_peered":"2024-03-08T23:51:51.242153+0000","last_unstale":"2024-03-08T23:54:17.524146+0000","last_undegraded":"2024-03-08T23:54:17.524146+0000","last_fullsized":"2024-03-08T23:54:17.524146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:44:07.728700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001140+0000","last_change":"2024-03-08T23:51:52.255411+0000","last_active":"2024-03-08T23:53:46.001140+0000","last_peered":"2024-03-08T23:53:46.001140+0000","last_clean":"2024-03-08T23:53:46.001140+0000","last_became_active":"2024-03-08T23:51:52.255048+0000","last_became_peered":"2024-03-08T23:51:52.255048+0000","last_unstale":"2024-03-08T23:53:46.001140+0000","last_undegraded":"2024-03-08T23:53:46.001140+0000","last_fullsized":"2024-03-08T23:53:46.001140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:11.836187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626682+0000","last_change":"2024-03-08T23:51:56.286405+0000","last_active":"2024-03-08T23:53:45.626682+0000","last_peered":"2024-03-08T23:53:45.626682+0000","last_clean":"2024-03-08T23:53:45.626682+0000","last_became_active":"2024-03-08T23:51:56.285996+0000","last_became_peered":"2024-03-08T23:51:56.285996+0000","last_unstale":"2024-03-08T23:53:45.626682+0000","last_undegraded":"2024-03-08T23:53:45.626682+0000","last_fullsized":"2024-03-08T23:53:45.626682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:48:30.928436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637190+0000","last_change":"2024-03-08T23:51:51.240146+0000","last_active":"2024-03-08T23:53:45.637190+0000","last_peered":"2024-03-08T23:53:45.637190+0000","last_clean":"2024-03-08T23:53:45.637190+0000","last_became_active":"2024-03-08T23:51:51.239795+0000","last_became_peered":"2024-03-08T23:51:51.239795+0000","last_unstale":"2024-03-08T23:53:45.637190+0000","last_undegraded":"2024-03-08T23:53:45.637190+0000","last_fullsized":"2024-03-08T23:53:45.637190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:55.409922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629399+0000","last_change":"2024-03-08T23:51:52.254701+0000","last_active":"2024-03-08T23:53:45.629399+0000","last_peered":"2024-03-08T23:53:45.629399+0000","last_clean":"2024-03-08T23:53:45.629399+0000","last_became_active":"2024-03-08T23:51:52.254391+0000","last_became_peered":"2024-03-08T23:51:52.254391+0000","last_unstale":"2024-03-08T23:53:45.629399+0000","last_undegraded":"2024-03-08T23:53:45.629399+0000","last_fullsized":"2024-03-08T23:53:45.629399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:11:12.258181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628931+0000","last_change":"2024-03-08T23:51:56.277878+0000","last_active":"2024-03-08T23:53:45.628931+0000","last_peered":"2024-03-08T23:53:45.628931+0000","last_clean":"2024-03-08T23:53:45.628931+0000","last_became_active":"2024-03-08T23:51:56.277493+0000","last_became_peered":"2024-03-08T23:51:56.277493+0000","last_unstale":"2024-03-08T23:53:45.628931+0000","last_undegraded":"2024-03-08T23:53:45.628931+0000","last_fullsized":"2024-03-08T23:53:45.628931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:44:16.968632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.643148+0000","last_change":"2024-03-08T23:53:45.643148+0000","last_active":"2024-03-08T23:53:45.643148+0000","last_peered":"2024-03-08T23:53:45.643148+0000","last_clean":"2024-03-08T23:53:45.643148+0000","last_became_active":"2024-03-08T23:53:45.638975+0000","last_became_peered":"2024-03-08T23:53:45.638975+0000","last_unstale":"2024-03-08T23:53:45.643148+0000","last_undegraded":"2024-03-08T23:53:45.643148+0000","last_fullsized":"2024-03-08T23:53:45.643148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:42:39.188486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.632141+0000","last_change":"2024-03-08T23:51:52.248465+0000","last_active":"2024-03-08T23:53:45.632141+0000","last_peered":"2024-03-08T23:53:45.632141+0000","last_clean":"2024-03-08T23:53:45.632141+0000","last_became_active":"2024-03-08T23:51:52.248135+0000","last_became_peered":"2024-03-08T23:51:52.248135+0000","last_unstale":"2024-03-08T23:53:45.632141+0000","last_undegraded":"2024-03-08T23:53:45.632141+0000","last_fullsized":"2024-03-08T23:53:45.632141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:55:53.712958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638275+0000","last_change":"2024-03-08T23:51:56.276019+0000","last_active":"2024-03-08T23:53:45.638275+0000","last_peered":"2024-03-08T23:53:45.638275+0000","last_clean":"2024-03-08T23:53:45.638275+0000","last_became_active":"2024-03-08T23:51:56.275564+0000","last_became_peered":"2024-03-08T23:51:56.275564+0000","last_unstale":"2024-03-08T23:53:45.638275+0000","last_undegraded":"2024-03-08T23:53:45.638275+0000","last_fullsized":"2024-03-08T23:53:45.638275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:12:36.096470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875254+0000","last_change":"2024-03-08T23:51:51.241629+0000","last_active":"2024-03-08T23:53:45.875254+0000","last_peered":"2024-03-08T23:53:45.875254+0000","last_clean":"2024-03-08T23:53:45.875254+0000","last_became_active":"2024-03-08T23:51:51.241300+0000","last_became_peered":"2024-03-08T23:51:51.241300+0000","last_unstale":"2024-03-08T23:53:45.875254+0000","last_undegraded":"2024-03-08T23:53:45.875254+0000","last_fullsized":"2024-03-08T23:53:45.875254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:01:45.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001329+0000","last_change":"2024-03-08T23:51:52.246024+0000","last_active":"2024-03-08T23:53:46.001329+0000","last_peered":"2024-03-08T23:53:46.001329+0000","last_clean":"2024-03-08T23:53:46.001329+0000","last_became_active":"2024-03-08T23:51:52.245706+0000","last_became_peered":"2024-03-08T23:51:52.245706+0000","last_unstale":"2024-03-08T23:53:46.001329+0000","last_undegraded":"2024-03-08T23:53:46.001329+0000","last_fullsized":"2024-03-08T23:53:46.001329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:00:44.557269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626476+0000","last_change":"2024-03-08T23:51:56.285649+0000","last_active":"2024-03-08T23:53:45.626476+0000","last_peered":"2024-03-08T23:53:45.626476+0000","last_clean":"2024-03-08T23:53:45.626476+0000","last_became_active":"2024-03-08T23:51:56.282731+0000","last_became_peered":"2024-03-08T23:51:56.282731+0000","last_unstale":"2024-03-08T23:53:45.626476+0000","last_undegraded":"2024-03-08T23:53:45.626476+0000","last_fullsized":"2024-03-08T23:53:45.626476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:23:54.103637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001789+0000","last_change":"2024-03-08T23:51:51.249442+0000","last_active":"2024-03-08T23:53:46.001789+0000","last_peered":"2024-03-08T23:53:46.001789+0000","last_clean":"2024-03-08T23:53:46.001789+0000","last_became_active":"2024-03-08T23:51:51.249084+0000","last_became_peered":"2024-03-08T23:51:51.249084+0000","last_unstale":"2024-03-08T23:53:46.001789+0000","last_undegraded":"2024-03-08T23:53:46.001789+0000","last_fullsized":"2024-03-08T23:53:46.001789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:48:19.370095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001661+0000","last_change":"2024-03-08T23:51:52.257586+0000","last_active":"2024-03-08T23:53:46.001661+0000","last_peered":"2024-03-08T23:53:46.001661+0000","last_clean":"2024-03-08T23:53:46.001661+0000","last_became_active":"2024-03-08T23:51:52.253741+0000","last_became_peered":"2024-03-08T23:51:52.253741+0000","last_unstale":"2024-03-08T23:53:46.001661+0000","last_undegraded":"2024-03-08T23:53:46.001661+0000","last_fullsized":"2024-03-08T23:53:46.001661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:18.633166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626404+0000","last_change":"2024-03-08T23:51:56.285264+0000","last_active":"2024-03-08T23:53:45.626404+0000","last_peered":"2024-03-08T23:53:45.626404+0000","last_clean":"2024-03-08T23:53:45.626404+0000","last_became_active":"2024-03-08T23:51:56.284727+0000","last_became_peered":"2024-03-08T23:51:56.284727+0000","last_unstale":"2024-03-08T23:53:45.626404+0000","last_undegraded":"2024-03-08T23:53:45.626404+0000","last_fullsized":"2024-03-08T23:53:45.626404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:01:20.723010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629149+0000","last_change":"2024-03-08T23:51:52.251461+0000","last_active":"2024-03-08T23:53:45.629149+0000","last_peered":"2024-03-08T23:53:45.629149+0000","last_clean":"2024-03-08T23:53:45.629149+0000","last_became_active":"2024-03-08T23:51:52.250996+0000","last_became_peered":"2024-03-08T23:51:52.250996+0000","last_unstale":"2024-03-08T23:53:45.629149+0000","last_undegraded":"2024-03-08T23:53:45.629149+0000","last_fullsized":"2024-03-08T23:53:45.629149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:51.719142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:18.524820+0000","last_change":"2024-03-08T23:51:51.245698+0000","last_active":"2024-03-08T23:54:18.524820+0000","last_peered":"2024-03-08T23:54:18.524820+0000","last_clean":"2024-03-08T23:54:18.524820+0000","last_became_active":"2024-03-08T23:51:51.245389+0000","last_became_peered":"2024-03-08T23:51:51.245389+0000","last_unstale":"2024-03-08T23:54:18.524820+0000","last_undegraded":"2024-03-08T23:54:18.524820+0000","last_fullsized":"2024-03-08T23:54:18.524820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:14:08.765142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002007+0000","last_change":"2024-03-08T23:52:45.423657+0000","last_active":"2024-03-08T23:53:46.002007+0000","last_peered":"2024-03-08T23:53:46.002007+0000","last_clean":"2024-03-08T23:53:46.002007+0000","last_became_active":"2024-03-08T23:52:45.417868+0000","last_became_peered":"2024-03-08T23:52:45.417868+0000","last_unstale":"2024-03-08T23:53:46.002007+0000","last_undegraded":"2024-03-08T23:53:46.002007+0000","last_fullsized":"2024-03-08T23:53:46.002007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:34.178938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636452+0000","last_change":"2024-03-08T23:51:52.256518+0000","last_active":"2024-03-08T23:53:45.636452+0000","last_peered":"2024-03-08T23:53:45.636452+0000","last_clean":"2024-03-08T23:53:45.636452+0000","last_became_active":"2024-03-08T23:51:52.253481+0000","last_became_peered":"2024-03-08T23:51:52.253481+0000","last_unstale":"2024-03-08T23:53:45.636452+0000","last_undegraded":"2024-03-08T23:53:45.636452+0000","last_fullsized":"2024-03-08T23:53:45.636452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:22:35.287408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628496+0000","last_change":"2024-03-08T23:51:51.237921+0000","last_active":"2024-03-08T23:53:45.628496+0000","last_peered":"2024-03-08T23:53:45.628496+0000","last_clean":"2024-03-08T23:53:45.628496+0000","last_became_active":"2024-03-08T23:51:51.237568+0000","last_became_peered":"2024-03-08T23:51:51.237568+0000","last_unstale":"2024-03-08T23:53:45.628496+0000","last_undegraded":"2024-03-08T23:53:45.628496+0000","last_fullsized":"2024-03-08T23:53:45.628496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:11.256814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000644+0000","last_change":"2024-03-08T23:50:41.802121+0000","last_active":"2024-03-08T23:53:46.000644+0000","last_peered":"2024-03-08T23:53:46.000644+0000","last_clean":"2024-03-08T23:53:46.000644+0000","last_became_active":"2024-03-08T23:50:41.801678+0000","last_became_peered":"2024-03-08T23:50:41.801678+0000","last_unstale":"2024-03-08T23:53:46.000644+0000","last_undegraded":"2024-03-08T23:53:46.000644+0000","last_fullsized":"2024-03-08T23:53:46.000644+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:47:12.382547+0000","last_clean_scrub_stamp":"2024-03-08T23:47:12.382547+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:15.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629562+0000","last_change":"2024-03-08T23:51:56.278465+0000","last_active":"2024-03-08T23:53:45.629562+0000","last_peered":"2024-03-08T23:53:45.629562+0000","last_clean":"2024-03-08T23:53:45.629562+0000","last_became_active":"2024-03-08T23:51:56.278079+0000","last_became_peered":"2024-03-08T23:51:56.278079+0000","last_unstale":"2024-03-08T23:53:45.629562+0000","last_undegraded":"2024-03-08T23:53:45.629562+0000","last_fullsized":"2024-03-08T23:53:45.629562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:06:47.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626180+0000","last_change":"2024-03-08T23:51:52.251895+0000","last_active":"2024-03-08T23:53:45.626180+0000","last_peered":"2024-03-08T23:53:45.626180+0000","last_clean":"2024-03-08T23:53:45.626180+0000","last_became_active":"2024-03-08T23:51:52.251591+0000","last_became_peered":"2024-03-08T23:51:52.251591+0000","last_unstale":"2024-03-08T23:53:45.626180+0000","last_undegraded":"2024-03-08T23:53:45.626180+0000","last_fullsized":"2024-03-08T23:53:45.626180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:06.276636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001022+0000","last_change":"2024-03-08T23:51:51.245981+0000","last_active":"2024-03-08T23:53:46.001022+0000","last_peered":"2024-03-08T23:53:46.001022+0000","last_clean":"2024-03-08T23:53:46.001022+0000","last_became_active":"2024-03-08T23:51:51.244681+0000","last_became_peered":"2024-03-08T23:51:51.244681+0000","last_unstale":"2024-03-08T23:53:46.001022+0000","last_undegraded":"2024-03-08T23:53:46.001022+0000","last_fullsized":"2024-03-08T23:53:46.001022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:06:21.370027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.631890+0000","last_change":"2024-03-08T23:51:56.279057+0000","last_active":"2024-03-08T23:53:45.631890+0000","last_peered":"2024-03-08T23:53:45.631890+0000","last_clean":"2024-03-08T23:53:45.631890+0000","last_became_active":"2024-03-08T23:51:56.278634+0000","last_became_peered":"2024-03-08T23:51:56.278634+0000","last_unstale":"2024-03-08T23:53:45.631890+0000","last_undegraded":"2024-03-08T23:53:45.631890+0000","last_fullsized":"2024-03-08T23:53:45.631890+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:30.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638685+0000","last_change":"2024-03-08T23:51:51.240378+0000","last_active":"2024-03-08T23:53:45.638685+0000","last_peered":"2024-03-08T23:53:45.638685+0000","last_clean":"2024-03-08T23:53:45.638685+0000","last_became_active":"2024-03-08T23:51:51.239880+0000","last_became_peered":"2024-03-08T23:51:51.239880+0000","last_unstale":"2024-03-08T23:53:45.638685+0000","last_undegraded":"2024-03-08T23:53:45.638685+0000","last_fullsized":"2024-03-08T23:53:45.638685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:53:20.804261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626738+0000","last_change":"2024-03-08T23:51:52.240338+0000","last_active":"2024-03-08T23:53:45.626738+0000","last_peered":"2024-03-08T23:53:45.626738+0000","last_clean":"2024-03-08T23:53:45.626738+0000","last_became_active":"2024-03-08T23:51:52.239987+0000","last_became_peered":"2024-03-08T23:51:52.239987+0000","last_unstale":"2024-03-08T23:53:45.626738+0000","last_undegraded":"2024-03-08T23:53:45.626738+0000","last_fullsized":"2024-03-08T23:53:45.626738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:05:26.590204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636290+0000","last_change":"2024-03-08T23:51:56.281009+0000","last_active":"2024-03-08T23:53:45.636290+0000","last_peered":"2024-03-08T23:53:45.636290+0000","last_clean":"2024-03-08T23:53:45.636290+0000","last_became_active":"2024-03-08T23:51:56.280619+0000","last_became_peered":"2024-03-08T23:51:56.280619+0000","last_unstale":"2024-03-08T23:53:45.636290+0000","last_undegraded":"2024-03-08T23:53:45.636290+0000","last_fullsized":"2024-03-08T23:53:45.636290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:50:06.920777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:54:17.267361+0000","last_change":"2024-03-08T23:51:51.241682+0000","last_active":"2024-03-08T23:54:17.267361+0000","last_peered":"2024-03-08T23:54:17.267361+0000","last_clean":"2024-03-08T23:54:17.267361+0000","last_became_active":"2024-03-08T23:51:51.241313+0000","last_became_peered":"2024-03-08T23:51:51.241313+0000","last_unstale":"2024-03-08T23:54:17.267361+0000","last_undegraded":"2024-03-08T23:54:17.267361+0000","last_fullsized":"2024-03-08T23:54:17.267361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:04:13.515856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636824+0000","last_change":"2024-03-08T23:51:52.249378+0000","last_active":"2024-03-08T23:53:45.636824+0000","last_peered":"2024-03-08T23:53:45.636824+0000","last_clean":"2024-03-08T23:53:45.636824+0000","last_became_active":"2024-03-08T23:51:52.249084+0000","last_became_peered":"2024-03-08T23:51:52.249084+0000","last_unstale":"2024-03-08T23:53:45.636824+0000","last_undegraded":"2024-03-08T23:53:45.636824+0000","last_fullsized":"2024-03-08T23:53:45.636824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:10:36.513260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638049+0000","last_change":"2024-03-08T23:51:56.280717+0000","last_active":"2024-03-08T23:53:45.638049+0000","last_peered":"2024-03-08T23:53:45.638049+0000","last_clean":"2024-03-08T23:53:45.638049+0000","last_became_active":"2024-03-08T23:51:56.280329+0000","last_became_peered":"2024-03-08T23:51:56.280329+0000","last_unstale":"2024-03-08T23:53:45.638049+0000","last_undegraded":"2024-03-08T23:53:45.638049+0000","last_fullsized":"2024-03-08T23:53:45.638049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:47:40.829142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629763+0000","last_change":"2024-03-08T23:51:51.238058+0000","last_active":"2024-03-08T23:53:45.629763+0000","last_peered":"2024-03-08T23:53:45.629763+0000","last_clean":"2024-03-08T23:53:45.629763+0000","last_became_active":"2024-03-08T23:51:51.237727+0000","last_became_peered":"2024-03-08T23:51:51.237727+0000","last_unstale":"2024-03-08T23:53:45.629763+0000","last_undegraded":"2024-03-08T23:53:45.629763+0000","last_fullsized":"2024-03-08T23:53:45.629763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:13:54.080004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000263+0000","last_change":"2024-03-08T23:51:52.249830+0000","last_active":"2024-03-08T23:53:46.000263+0000","last_peered":"2024-03-08T23:53:46.000263+0000","last_clean":"2024-03-08T23:53:46.000263+0000","last_became_active":"2024-03-08T23:51:52.249518+0000","last_became_peered":"2024-03-08T23:51:52.249518+0000","last_unstale":"2024-03-08T23:53:46.000263+0000","last_undegraded":"2024-03-08T23:53:46.000263+0000","last_fullsized":"2024-03-08T23:53:46.000263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:24:33.721423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629649+0000","last_change":"2024-03-08T23:51:56.288595+0000","last_active":"2024-03-08T23:53:45.629649+0000","last_peered":"2024-03-08T23:53:45.629649+0000","last_clean":"2024-03-08T23:53:45.629649+0000","last_became_active":"2024-03-08T23:51:56.288146+0000","last_became_peered":"2024-03-08T23:51:56.288146+0000","last_unstale":"2024-03-08T23:53:45.629649+0000","last_undegraded":"2024-03-08T23:53:45.629649+0000","last_fullsized":"2024-03-08T23:53:45.629649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:34:58.321754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000190+0000","last_change":"2024-03-08T23:51:51.242065+0000","last_active":"2024-03-08T23:53:46.000190+0000","last_peered":"2024-03-08T23:53:46.000190+0000","last_clean":"2024-03-08T23:53:46.000190+0000","last_became_active":"2024-03-08T23:51:51.241513+0000","last_became_peered":"2024-03-08T23:51:51.241513+0000","last_unstale":"2024-03-08T23:53:46.000190+0000","last_undegraded":"2024-03-08T23:53:46.000190+0000","last_fullsized":"2024-03-08T23:53:46.000190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:08:02.505240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637325+0000","last_change":"2024-03-08T23:51:52.243074+0000","last_active":"2024-03-08T23:53:45.637325+0000","last_peered":"2024-03-08T23:53:45.637325+0000","last_clean":"2024-03-08T23:53:45.637325+0000","last_became_active":"2024-03-08T23:51:52.242736+0000","last_became_peered":"2024-03-08T23:51:52.242736+0000","last_unstale":"2024-03-08T23:53:45.637325+0000","last_undegraded":"2024-03-08T23:53:45.637325+0000","last_fullsized":"2024-03-08T23:53:45.637325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:57:37.174056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.875547+0000","last_change":"2024-03-08T23:51:56.276284+0000","last_active":"2024-03-08T23:53:45.875547+0000","last_peered":"2024-03-08T23:53:45.875547+0000","last_clean":"2024-03-08T23:53:45.875547+0000","last_became_active":"2024-03-08T23:51:56.275800+0000","last_became_peered":"2024-03-08T23:51:56.275800+0000","last_unstale":"2024-03-08T23:53:45.875547+0000","last_undegraded":"2024-03-08T23:53:45.875547+0000","last_fullsized":"2024-03-08T23:53:45.875547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:21:11.221266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636532+0000","last_change":"2024-03-08T23:51:51.245548+0000","last_active":"2024-03-08T23:53:45.636532+0000","last_peered":"2024-03-08T23:53:45.636532+0000","last_clean":"2024-03-08T23:53:45.636532+0000","last_became_active":"2024-03-08T23:51:51.245238+0000","last_became_peered":"2024-03-08T23:51:51.245238+0000","last_unstale":"2024-03-08T23:53:45.636532+0000","last_undegraded":"2024-03-08T23:53:45.636532+0000","last_fullsized":"2024-03-08T23:53:45.636532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:21:19.557196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.635139+0000","last_change":"2024-03-08T23:51:52.244422+0000","last_active":"2024-03-08T23:53:45.635139+0000","last_peered":"2024-03-08T23:53:45.635139+0000","last_clean":"2024-03-08T23:53:45.635139+0000","last_became_active":"2024-03-08T23:51:52.244032+0000","last_became_peered":"2024-03-08T23:51:52.244032+0000","last_unstale":"2024-03-08T23:53:45.635139+0000","last_undegraded":"2024-03-08T23:53:45.635139+0000","last_fullsized":"2024-03-08T23:53:45.635139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:04.064691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.630134+0000","last_change":"2024-03-08T23:51:56.288632+0000","last_active":"2024-03-08T23:53:45.630134+0000","last_peered":"2024-03-08T23:53:45.630134+0000","last_clean":"2024-03-08T23:53:45.630134+0000","last_became_active":"2024-03-08T23:51:56.288175+0000","last_became_peered":"2024-03-08T23:51:56.288175+0000","last_unstale":"2024-03-08T23:53:45.630134+0000","last_undegraded":"2024-03-08T23:53:45.630134+0000","last_fullsized":"2024-03-08T23:53:45.630134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:54:51.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.645297+0000","last_change":"2024-03-08T23:53:45.645297+0000","last_active":"2024-03-08T23:53:45.645297+0000","last_peered":"2024-03-08T23:53:45.645297+0000","last_clean":"2024-03-08T23:53:45.645297+0000","last_became_active":"2024-03-08T23:53:45.639667+0000","last_became_peered":"2024-03-08T23:53:45.639667+0000","last_unstale":"2024-03-08T23:53:45.645297+0000","last_undegraded":"2024-03-08T23:53:45.645297+0000","last_fullsized":"2024-03-08T23:53:45.645297+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:30:30.658165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636550+0000","last_change":"2024-03-08T23:51:52.250778+0000","last_active":"2024-03-08T23:53:45.636550+0000","last_peered":"2024-03-08T23:53:45.636550+0000","last_clean":"2024-03-08T23:53:45.636550+0000","last_became_active":"2024-03-08T23:51:52.250475+0000","last_became_peered":"2024-03-08T23:51:52.250475+0000","last_unstale":"2024-03-08T23:53:45.636550+0000","last_undegraded":"2024-03-08T23:53:45.636550+0000","last_fullsized":"2024-03-08T23:53:45.636550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:36:01.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629535+0000","last_change":"2024-03-08T23:51:56.285631+0000","last_active":"2024-03-08T23:53:45.629535+0000","last_peered":"2024-03-08T23:53:45.629535+0000","last_clean":"2024-03-08T23:53:45.629535+0000","last_became_active":"2024-03-08T23:51:56.285183+0000","last_became_peered":"2024-03-08T23:51:56.285183+0000","last_unstale":"2024-03-08T23:53:45.629535+0000","last_undegraded":"2024-03-08T23:53:45.629535+0000","last_fullsized":"2024-03-08T23:53:45.629535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:15.002086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637835+0000","last_change":"2024-03-08T23:51:51.246848+0000","last_active":"2024-03-08T23:53:45.637835+0000","last_peered":"2024-03-08T23:53:45.637835+0000","last_clean":"2024-03-08T23:53:45.637835+0000","last_became_active":"2024-03-08T23:51:51.246533+0000","last_became_peered":"2024-03-08T23:51:51.246533+0000","last_unstale":"2024-03-08T23:53:45.637835+0000","last_undegraded":"2024-03-08T23:53:45.637835+0000","last_fullsized":"2024-03-08T23:53:45.637835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:20:16.669820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002055+0000","last_change":"2024-03-08T23:51:52.251575+0000","last_active":"2024-03-08T23:53:46.002055+0000","last_peered":"2024-03-08T23:53:46.002055+0000","last_clean":"2024-03-08T23:53:46.002055+0000","last_became_active":"2024-03-08T23:51:52.251232+0000","last_became_peered":"2024-03-08T23:51:52.251232+0000","last_unstale":"2024-03-08T23:53:46.002055+0000","last_undegraded":"2024-03-08T23:53:46.002055+0000","last_fullsized":"2024-03-08T23:53:46.002055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:51:33.573407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000476+0000","last_change":"2024-03-08T23:52:45.586092+0000","last_active":"2024-03-08T23:53:46.000476+0000","last_peered":"2024-03-08T23:53:46.000476+0000","last_clean":"2024-03-08T23:53:46.000476+0000","last_became_active":"2024-03-08T23:52:45.585513+0000","last_became_peered":"2024-03-08T23:52:45.585513+0000","last_unstale":"2024-03-08T23:53:46.000476+0000","last_undegraded":"2024-03-08T23:53:46.000476+0000","last_fullsized":"2024-03-08T23:53:46.000476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:39:05.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638551+0000","last_change":"2024-03-08T23:51:51.242081+0000","last_active":"2024-03-08T23:53:45.638551+0000","last_peered":"2024-03-08T23:53:45.638551+0000","last_clean":"2024-03-08T23:53:45.638551+0000","last_became_active":"2024-03-08T23:51:51.241636+0000","last_became_peered":"2024-03-08T23:51:51.241636+0000","last_unstale":"2024-03-08T23:53:45.638551+0000","last_undegraded":"2024-03-08T23:53:45.638551+0000","last_fullsized":"2024-03-08T23:53:45.638551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:51:57.497723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002288+0000","last_change":"2024-03-08T23:51:52.252796+0000","last_active":"2024-03-08T23:53:46.002288+0000","last_peered":"2024-03-08T23:53:46.002288+0000","last_clean":"2024-03-08T23:53:46.002288+0000","last_became_active":"2024-03-08T23:51:52.252361+0000","last_became_peered":"2024-03-08T23:51:52.252361+0000","last_unstale":"2024-03-08T23:53:46.002288+0000","last_undegraded":"2024-03-08T23:53:46.002288+0000","last_fullsized":"2024-03-08T23:53:46.002288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:07.968262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.999883+0000","last_change":"2024-03-08T23:51:56.284396+0000","last_active":"2024-03-08T23:53:45.999883+0000","last_peered":"2024-03-08T23:53:45.999883+0000","last_clean":"2024-03-08T23:53:45.999883+0000","last_became_active":"2024-03-08T23:51:56.283951+0000","last_became_peered":"2024-03-08T23:51:56.283951+0000","last_unstale":"2024-03-08T23:53:45.999883+0000","last_undegraded":"2024-03-08T23:53:45.999883+0000","last_fullsized":"2024-03-08T23:53:45.999883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:09:40.061228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629375+0000","last_change":"2024-03-08T23:51:51.248838+0000","last_active":"2024-03-08T23:53:45.629375+0000","last_peered":"2024-03-08T23:53:45.629375+0000","last_clean":"2024-03-08T23:53:45.629375+0000","last_became_active":"2024-03-08T23:51:51.248534+0000","last_became_peered":"2024-03-08T23:51:51.248534+0000","last_unstale":"2024-03-08T23:53:45.629375+0000","last_undegraded":"2024-03-08T23:53:45.629375+0000","last_fullsized":"2024-03-08T23:53:45.629375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:37:25.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.001834+0000","last_change":"2024-03-08T23:51:52.264170+0000","last_active":"2024-03-08T23:53:46.001834+0000","last_peered":"2024-03-08T23:53:46.001834+0000","last_clean":"2024-03-08T23:53:46.001834+0000","last_became_active":"2024-03-08T23:51:52.263744+0000","last_became_peered":"2024-03-08T23:51:52.263744+0000","last_unstale":"2024-03-08T23:53:46.001834+0000","last_undegraded":"2024-03-08T23:53:46.001834+0000","last_fullsized":"2024-03-08T23:53:46.001834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:42:18.287699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.628650+0000","last_change":"2024-03-08T23:51:56.283316+0000","last_active":"2024-03-08T23:53:45.628650+0000","last_peered":"2024-03-08T23:53:45.628650+0000","last_clean":"2024-03-08T23:53:45.628650+0000","last_became_active":"2024-03-08T23:51:56.282515+0000","last_became_peered":"2024-03-08T23:51:56.282515+0000","last_unstale":"2024-03-08T23:53:45.628650+0000","last_undegraded":"2024-03-08T23:53:45.628650+0000","last_fullsized":"2024-03-08T23:53:45.628650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:55.247550+0000","last_clean_scrub_stamp":"2024-03-08T23:51:55.247550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:40:15.909626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626731+0000","last_change":"2024-03-08T23:51:51.240970+0000","last_active":"2024-03-08T23:53:45.626731+0000","last_peered":"2024-03-08T23:53:45.626731+0000","last_clean":"2024-03-08T23:53:45.626731+0000","last_became_active":"2024-03-08T23:51:51.239982+0000","last_became_peered":"2024-03-08T23:51:51.239982+0000","last_unstale":"2024-03-08T23:53:45.626731+0000","last_undegraded":"2024-03-08T23:53:45.626731+0000","last_fullsized":"2024-03-08T23:53:45.626731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:50.221451+0000","last_clean_scrub_stamp":"2024-03-08T23:51:50.221451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:47:40.313753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.002207+0000","last_change":"2024-03-08T23:51:52.261673+0000","last_active":"2024-03-08T23:53:46.002207+0000","last_peered":"2024-03-08T23:53:46.002207+0000","last_clean":"2024-03-08T23:53:46.002207+0000","last_became_active":"2024-03-08T23:51:52.261333+0000","last_became_peered":"2024-03-08T23:51:52.261333+0000","last_unstale":"2024-03-08T23:53:46.002207+0000","last_undegraded":"2024-03-08T23:53:46.002207+0000","last_fullsized":"2024-03-08T23:53:46.002207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:08:26.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.635064+0000","last_change":"2024-03-08T23:51:52.243189+0000","last_active":"2024-03-08T23:53:45.635064+0000","last_peered":"2024-03-08T23:53:45.635064+0000","last_clean":"2024-03-08T23:53:45.635064+0000","last_became_active":"2024-03-08T23:51:52.242870+0000","last_became_peered":"2024-03-08T23:51:52.242870+0000","last_unstale":"2024-03-08T23:53:45.635064+0000","last_undegraded":"2024-03-08T23:53:45.635064+0000","last_fullsized":"2024-03-08T23:53:45.635064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.438045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.638156+0000","last_change":"2024-03-08T23:52:45.417220+0000","last_active":"2024-03-08T23:53:45.638156+0000","last_peered":"2024-03-08T23:53:45.638156+0000","last_clean":"2024-03-08T23:53:45.638156+0000","last_became_active":"2024-03-08T23:52:45.416820+0000","last_became_peered":"2024-03-08T23:52:45.416820+0000","last_unstale":"2024-03-08T23:53:45.638156+0000","last_undegraded":"2024-03-08T23:53:45.638156+0000","last_fullsized":"2024-03-08T23:53:45.638156+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:31:13.249449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637319+0000","last_change":"2024-03-08T23:51:52.251397+0000","last_active":"2024-03-08T23:53:45.637319+0000","last_peered":"2024-03-08T23:53:45.637319+0000","last_clean":"2024-03-08T23:53:45.637319+0000","last_became_active":"2024-03-08T23:51:52.251059+0000","last_became_peered":"2024-03-08T23:51:52.251059+0000","last_unstale":"2024-03-08T23:53:45.637319+0000","last_undegraded":"2024-03-08T23:53:45.637319+0000","last_fullsized":"2024-03-08T23:53:45.637319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:19:30.363457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000034+0000","last_change":"2024-03-08T23:51:52.249039+0000","last_active":"2024-03-08T23:53:46.000034+0000","last_peered":"2024-03-08T23:53:46.000034+0000","last_clean":"2024-03-08T23:53:46.000034+0000","last_became_active":"2024-03-08T23:51:52.248560+0000","last_became_peered":"2024-03-08T23:51:52.248560+0000","last_unstale":"2024-03-08T23:53:46.000034+0000","last_undegraded":"2024-03-08T23:53:46.000034+0000","last_fullsized":"2024-03-08T23:53:46.000034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626651+0000","last_change":"2024-03-08T23:51:52.247269+0000","last_active":"2024-03-08T23:53:45.626651+0000","last_peered":"2024-03-08T23:53:45.626651+0000","last_clean":"2024-03-08T23:53:45.626651+0000","last_became_active":"2024-03-08T23:51:52.246921+0000","last_became_peered":"2024-03-08T23:51:52.246921+0000","last_unstale":"2024-03-08T23:53:45.626651+0000","last_undegraded":"2024-03-08T23:53:45.626651+0000","last_fullsized":"2024-03-08T23:53:45.626651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:00:21.828530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633238+0000","last_change":"2024-03-08T23:51:52.238999+0000","last_active":"2024-03-08T23:53:45.633238+0000","last_peered":"2024-03-08T23:53:45.633238+0000","last_clean":"2024-03-08T23:53:45.633238+0000","last_became_active":"2024-03-08T23:51:52.238662+0000","last_became_peered":"2024-03-08T23:51:52.238662+0000","last_unstale":"2024-03-08T23:53:45.633238+0000","last_undegraded":"2024-03-08T23:53:45.633238+0000","last_fullsized":"2024-03-08T23:53:45.633238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:22:08.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.637860+0000","last_change":"2024-03-08T23:51:52.238793+0000","last_active":"2024-03-08T23:53:45.637860+0000","last_peered":"2024-03-08T23:53:45.637860+0000","last_clean":"2024-03-08T23:53:45.637860+0000","last_became_active":"2024-03-08T23:51:52.238464+0000","last_became_peered":"2024-03-08T23:51:52.238464+0000","last_unstale":"2024-03-08T23:53:45.637860+0000","last_undegraded":"2024-03-08T23:53:45.637860+0000","last_fullsized":"2024-03-08T23:53:45.637860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:25.597428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.636926+0000","last_change":"2024-03-08T23:52:45.585775+0000","last_active":"2024-03-08T23:53:45.636926+0000","last_peered":"2024-03-08T23:53:45.636926+0000","last_clean":"2024-03-08T23:53:45.636926+0000","last_became_active":"2024-03-08T23:52:45.585280+0000","last_became_peered":"2024-03-08T23:52:45.585280+0000","last_unstale":"2024-03-08T23:53:45.636926+0000","last_undegraded":"2024-03-08T23:53:45.636926+0000","last_fullsized":"2024-03-08T23:53:45.636926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:11:20.029967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.629208+0000","last_change":"2024-03-08T23:51:52.245288+0000","last_active":"2024-03-08T23:53:45.629208+0000","last_peered":"2024-03-08T23:53:45.629208+0000","last_clean":"2024-03-08T23:53:45.629208+0000","last_became_active":"2024-03-08T23:51:52.244960+0000","last_became_peered":"2024-03-08T23:51:52.244960+0000","last_unstale":"2024-03-08T23:53:45.629208+0000","last_undegraded":"2024-03-08T23:53:45.629208+0000","last_fullsized":"2024-03-08T23:53:45.629208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:20:39.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.874977+0000","last_change":"2024-03-08T23:52:45.418172+0000","last_active":"2024-03-08T23:53:45.874977+0000","last_peered":"2024-03-08T23:53:45.874977+0000","last_clean":"2024-03-08T23:53:45.874977+0000","last_became_active":"2024-03-08T23:52:45.417852+0000","last_became_peered":"2024-03-08T23:52:45.417852+0000","last_unstale":"2024-03-08T23:53:45.874977+0000","last_undegraded":"2024-03-08T23:53:45.874977+0000","last_fullsized":"2024-03-08T23:53:45.874977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:22:23.698238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:46.000860+0000","last_change":"2024-03-08T23:51:52.251048+0000","last_active":"2024-03-08T23:53:46.000860+0000","last_peered":"2024-03-08T23:53:46.000860+0000","last_clean":"2024-03-08T23:53:46.000860+0000","last_became_active":"2024-03-08T23:51:52.250508+0000","last_became_peered":"2024-03-08T23:51:52.250508+0000","last_unstale":"2024-03-08T23:53:46.000860+0000","last_undegraded":"2024-03-08T23:53:46.000860+0000","last_fullsized":"2024-03-08T23:53:46.000860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:57:37.519307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.626525+0000","last_change":"2024-03-08T23:51:52.244203+0000","last_active":"2024-03-08T23:53:45.626525+0000","last_peered":"2024-03-08T23:53:45.626525+0000","last_clean":"2024-03-08T23:53:45.626525+0000","last_became_active":"2024-03-08T23:51:52.243878+0000","last_became_peered":"2024-03-08T23:51:52.243878+0000","last_unstale":"2024-03-08T23:53:45.626525+0000","last_undegraded":"2024-03-08T23:53:45.626525+0000","last_fullsized":"2024-03-08T23:53:45.626525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:53:40.938657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-08T23:53:45.633137+0000","last_change":"2024-03-08T23:51:52.248334+0000","last_active":"2024-03-08T23:53:45.633137+0000","last_peered":"2024-03-08T23:53:45.633137+0000","last_clean":"2024-03-08T23:53:45.633137+0000","last_became_active":"2024-03-08T23:51:52.248025+0000","last_became_peered":"2024-03-08T23:51:52.248025+0000","last_unstale":"2024-03-08T23:53:45.633137+0000","last_undegraded":"2024-03-08T23:53:45.633137+0000","last_fullsized":"2024-03-08T23:53:45.633137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T23:51:51.223069+0000","last_clean_scrub_stamp":"2024-03-08T23:51:51.223069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:02:06.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74027,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":746,"num_read_kb":452,"num_write":182,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":60703,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":222863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":519611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1757184,"data_stored":1180838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":519825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":519723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1117417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":583565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":517184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":583381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1111383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T23:54:22.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 -- 172.21.15.22:0/942554479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101de0 msgr2=0x7f60c4108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 --2- 172.21.15.22:0/942554479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101de0 0x7f60c4108670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009ca0 tx=0x7f60bc02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 -- 172.21.15.22:0/942554479 shutdown_connections 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 --2- 172.21.15.22:0/942554479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101de0 0x7f60c4108670 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 --2- 172.21.15.22:0/942554479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4101420 0x7f60c41018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 --2- 172.21.15.22:0/942554479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c4100230 0x7f60c4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 -- 172.21.15.22:0/942554479 >> 172.21.15.22:0/942554479 conn(0x7f60c40fb7a0 msgr2=0x7f60c40fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:22.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 -- 172.21.15.22:0/942554479 shutdown_connections 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.763+0000 7f60cbb29640 1 -- 172.21.15.22:0/942554479 wait complete. 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 Processor -- start 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 -- start start 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4100230 0x7f60c406a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101420 0x7f60c419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c4101de0 0x7f60c419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410bec0 con 0x7f60c4101420 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410bd40 con 0x7f60c4100230 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60cbb29640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410c040 con 0x7f60c4101de0 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60c90c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101420 0x7f60c419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60c90c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101420 0x7f60c419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51956/0 (socket says 172.21.15.22:51956) 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60c90c4640 1 -- 172.21.15.22:0/1126034593 learned_addr learned my addr 172.21.15.22:0/1126034593 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60ca0c6640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c4101de0 0x7f60c419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.764+0000 7f60c98c5640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4100230 0x7f60c406a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60c90c4640 1 -- 172.21.15.22:0/1126034593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c4101de0 msgr2=0x7f60c419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60c90c4640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c4101de0 0x7f60c419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60c90c4640 1 -- 172.21.15.22:0/1126034593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4100230 msgr2=0x7f60c406a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60c90c4640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4100230 0x7f60c406a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60c90c4640 1 -- 172.21.15.22:0/1126034593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c419f210 con 0x7f60c4101420 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60c90c4640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101420 0x7f60c419c2e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f60b400c890 tx=0x7f60b400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60baffd640 1 -- 172.21.15.22:0/1126034593 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4004220 con 0x7f60c4101420 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60baffd640 1 -- 172.21.15.22:0/1126034593 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b40043c0 con 0x7f60c4101420 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c419f500 con 0x7f60c4101420 2024-03-08T23:54:22.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60baffd640 1 -- 172.21.15.22:0/1126034593 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b40049e0 con 0x7f60c4101420 2024-03-08T23:54:22.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.765+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c419fa50 con 0x7f60c4101420 2024-03-08T23:54:22.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.766+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c41074f0 con 0x7f60c4101420 2024-03-08T23:54:22.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.767+0000 7f60baffd640 1 -- 172.21.15.22:0/1126034593 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f60b40079b0 con 0x7f60c4101420 2024-03-08T23:54:22.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.767+0000 7f60baffd640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f60b00a2890 0x7f60b00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:22.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.767+0000 7f60c98c5640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f60b00a2890 0x7f60b00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:22.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.768+0000 7f60baffd640 1 -- 172.21.15.22:0/1126034593 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f60b40cf0a0 con 0x7f60c4101420 2024-03-08T23:54:22.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.768+0000 7f60c98c5640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f60b00a2890 0x7f60b00a4d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005fd0 tx=0x7f60c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:22.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.772+0000 7f60baffd640 1 -- 172.21.15.22:0/1126034593 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f60b4014030 con 0x7f60c4101420 2024-03-08T23:54:22.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.958+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60c402d020 con 0x7f60c4101420 2024-03-08T23:54:22.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.960+0000 7f60baffd640 1 -- 172.21.15.22:0/1126034593 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24654 (secure 0 0 0) 0x7f60b409bbf0 con 0x7f60c4101420 2024-03-08T23:54:22.957 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:22.957 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:45.618289+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:54:22.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f60b00a2890 msgr2=0x7f60b00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f60b00a2890 0x7f60b00a4d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005fd0 tx=0x7f60c0005850 comp rx=0 tx=0).stop 2024-03-08T23:54:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101420 msgr2=0x7f60c419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101420 0x7f60c419c2e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f60b400c890 tx=0x7f60b400cd50 comp rx=0 tx=0).stop 2024-03-08T23:54:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 shutdown_connections 2024-03-08T23:54:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c4101de0 0x7f60c419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f60b00a2890 0x7f60b00a4d50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60c4101420 0x7f60c419c2e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 --2- 172.21.15.22:0/1126034593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60c4100230 0x7f60c406a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:22.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 >> 172.21.15.22:0/1126034593 conn(0x7f60c40fb7a0 msgr2=0x7f60c41046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:22.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 shutdown_connections 2024-03-08T23:54:22.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:22.962+0000 7f60cbb29640 1 -- 172.21.15.22:0/1126034593 wait complete. 2024-03-08T23:54:22.972 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-08T23:54:22.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T23:54:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.083+0000 7ff346789640 1 -- 172.21.15.22:0/35781580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401013e0 msgr2=0x7ff340101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.083+0000 7ff346789640 1 --2- 172.21.15.22:0/35781580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401013e0 0x7ff340101860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff330009900 tx=0x7ff33002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.083+0000 7ff346789640 1 -- 172.21.15.22:0/35781580 shutdown_connections 2024-03-08T23:54:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.083+0000 7ff346789640 1 --2- 172.21.15.22:0/35781580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff340101da0 0x7ff340108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.083+0000 7ff346789640 1 --2- 172.21.15.22:0/35781580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401013e0 0x7ff340101860 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.083+0000 7ff346789640 1 --2- 172.21.15.22:0/35781580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3401001f0 0x7ff340100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.083+0000 7ff346789640 1 -- 172.21.15.22:0/35781580 >> 172.21.15.22:0/35781580 conn(0x7ff3400fb780 msgr2=0x7ff3400fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 -- 172.21.15.22:0/35781580 shutdown_connections 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 -- 172.21.15.22:0/35781580 wait complete. 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 Processor -- start 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 -- start start 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401001f0 0x7ff340197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3401013e0 0x7ff340197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff340101da0 0x7ff34019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34010be80 con 0x7ff3401001f0 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34010bd00 con 0x7ff340101da0 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.084+0000 7ff346789640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34010c000 con 0x7ff3401013e0 2024-03-08T23:54:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401001f0 0x7ff340197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401001f0 0x7ff340197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51980/0 (socket says 172.21.15.22:51980) 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 -- 172.21.15.22:0/3235228681 learned_addr learned my addr 172.21.15.22:0/3235228681 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33f7fe640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3401013e0 0x7ff340197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff344d26640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff340101da0 0x7ff34019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 -- 172.21.15.22:0/3235228681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3401013e0 msgr2=0x7ff340197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3401013e0 0x7ff340197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 -- 172.21.15.22:0/3235228681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff340101da0 msgr2=0x7ff34019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff340101da0 0x7ff34019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 -- 172.21.15.22:0/3235228681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34019f230 con 0x7ff3401001f0 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33ffff640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401001f0 0x7ff340197660 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9c0 tx=0x7ff32800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33d7fa640 1 -- 172.21.15.22:0/3235228681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800cd80 con 0x7ff3401001f0 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34019f520 con 0x7ff3401001f0 2024-03-08T23:54:23.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33d7fa640 1 -- 172.21.15.22:0/3235228681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3280035f0 con 0x7ff3401001f0 2024-03-08T23:54:23.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff33d7fa640 1 -- 172.21.15.22:0/3235228681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328010690 con 0x7ff3401001f0 2024-03-08T23:54:23.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.085+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34019fa90 con 0x7ff3401001f0 2024-03-08T23:54:23.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.087+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34019fd50 con 0x7ff3401001f0 2024-03-08T23:54:23.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.088+0000 7ff33d7fa640 1 -- 172.21.15.22:0/3235228681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff3280037a0 con 0x7ff3401001f0 2024-03-08T23:54:23.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.088+0000 7ff33d7fa640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff32c0a2890 0x7ff32c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.088+0000 7ff33f7fe640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff32c0a2890 0x7ff32c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.088+0000 7ff33d7fa640 1 -- 172.21.15.22:0/3235228681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3280cf330 con 0x7ff3401001f0 2024-03-08T23:54:23.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.089+0000 7ff33f7fe640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff32c0a2890 0x7ff32c0a4d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff33002c7b0 tx=0x7ff330005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:23.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.097+0000 7ff33d7fa640 1 -- 172.21.15.22:0/3235228681 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff32809bee0 con 0x7ff3401001f0 2024-03-08T23:54:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.281+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff34019ff20 con 0x7ff3401001f0 2024-03-08T23:54:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.282+0000 7ff33d7fa640 1 -- 172.21.15.22:0/3235228681 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff32809b7e0 con 0x7ff3401001f0 2024-03-08T23:54:23.278 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-08T23:54:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff32c0a2890 msgr2=0x7ff32c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff32c0a2890 0x7ff32c0a4d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff33002c7b0 tx=0x7ff330005fd0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401001f0 msgr2=0x7ff340197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401001f0 0x7ff340197660 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9c0 tx=0x7ff32800ee80 comp rx=0 tx=0).stop 2024-03-08T23:54:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 shutdown_connections 2024-03-08T23:54:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff340101da0 0x7ff34019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3401013e0 0x7ff340197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7ff32c0a2890 0x7ff32c0a4d50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 --2- 172.21.15.22:0/3235228681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3401001f0 0x7ff340197660 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 >> 172.21.15.22:0/3235228681 conn(0x7ff3400fb780 msgr2=0x7ff3400fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 shutdown_connections 2024-03-08T23:54:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.284+0000 7ff346789640 1 -- 172.21.15.22:0/3235228681 wait complete. 2024-03-08T23:54:23.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T23:54:23.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 -- 172.21.15.22:0/1964513916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 msgr2=0x7fdcf8108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 --2- 172.21.15.22:0/1964513916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 0x7fdcf8108640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009ca0 tx=0x7fdcf002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 -- 172.21.15.22:0/1964513916 shutdown_connections 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 --2- 172.21.15.22:0/1964513916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 0x7fdcf8108640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 --2- 172.21.15.22:0/1964513916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 0x7fdcf8101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 --2- 172.21.15.22:0/1964513916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8100200 0x7fdcf8100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 -- 172.21.15.22:0/1964513916 >> 172.21.15.22:0/1964513916 conn(0x7fdcf80fb790 msgr2=0x7fdcf80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 -- 172.21.15.22:0/1964513916 shutdown_connections 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.416+0000 7fdcfe806640 1 -- 172.21.15.22:0/1964513916 wait complete. 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 Processor -- start 2024-03-08T23:54:23.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 -- start start 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8100200 0x7fdcf8197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 0x7fdcf8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 0x7fdcf819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf810be90 con 0x7fdcf81013f0 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf810bd10 con 0x7fdcf8100200 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfe806640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf810c010 con 0x7fdcf8101db0 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcf77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 0x7fdcf8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcf77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 0x7fdcf8197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51986/0 (socket says 172.21.15.22:51986) 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcf77fe640 1 -- 172.21.15.22:0/2408889555 learned_addr learned my addr 172.21.15.22:0/2408889555 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcfcda3640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 0x7fdcf819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.417+0000 7fdcf7fff640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8100200 0x7fdcf8197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf77fe640 1 -- 172.21.15.22:0/2408889555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 msgr2=0x7fdcf819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf77fe640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 0x7fdcf819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf77fe640 1 -- 172.21.15.22:0/2408889555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8100200 msgr2=0x7fdcf8197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf77fe640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8100200 0x7fdcf8197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf77fe640 1 -- 172.21.15.22:0/2408889555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf819f1b0 con 0x7fdcf81013f0 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcfcda3640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 0x7fdcf819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf77fe640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 0x7fdcf8197ad0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdce800ea60 tx=0x7fdce800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf57fa640 1 -- 172.21.15.22:0/2408889555 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800cde0 con 0x7fdcf81013f0 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf57fa640 1 -- 172.21.15.22:0/2408889555 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdce80035f0 con 0x7fdcf81013f0 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf819f4a0 con 0x7fdcf81013f0 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcf57fa640 1 -- 172.21.15.22:0/2408889555 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8010670 con 0x7fdcf81013f0 2024-03-08T23:54:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.418+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf819f9f0 con 0x7fdcf81013f0 2024-03-08T23:54:23.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.419+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf819fcb0 con 0x7fdcf81013f0 2024-03-08T23:54:23.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.420+0000 7fdcf57fa640 1 -- 172.21.15.22:0/2408889555 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fdce80037a0 con 0x7fdcf81013f0 2024-03-08T23:54:23.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.421+0000 7fdcf57fa640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdce00a2890 0x7fdce00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.421+0000 7fdcf7fff640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdce00a2890 0x7fdce00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.421+0000 7fdcf57fa640 1 -- 172.21.15.22:0/2408889555 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdce80cf330 con 0x7fdcf81013f0 2024-03-08T23:54:23.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.421+0000 7fdcf7fff640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdce00a2890 0x7fdce00a4d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8101250 tx=0x7fdcec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:23.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.428+0000 7fdcf57fa640 1 -- 172.21.15.22:0/2408889555 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdce809bf70 con 0x7fdcf81013f0 2024-03-08T23:54:23.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.629+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcf819fe80 con 0x7fdcf81013f0 2024-03-08T23:54:23.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.629+0000 7fdcf57fa640 1 -- 172.21.15.22:0/2408889555 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdce809b870 con 0x7fdcf81013f0 2024-03-08T23:54:23.625 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:23.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdce00a2890 msgr2=0x7fdce00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdce00a2890 0x7fdce00a4d50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8101250 tx=0x7fdcec005f90 comp rx=0 tx=0).stop 2024-03-08T23:54:23.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 msgr2=0x7fdcf8197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 0x7fdcf8197ad0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdce800ea60 tx=0x7fdce800ef20 comp rx=0 tx=0).stop 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 shutdown_connections 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcf8101db0 0x7fdcf819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fdce00a2890 0x7fdce00a4d50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcf81013f0 0x7fdcf8197ad0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 --2- 172.21.15.22:0/2408889555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8100200 0x7fdcf8197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 >> 172.21.15.22:0/2408889555 conn(0x7fdcf80fb790 msgr2=0x7fdcf80fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 shutdown_connections 2024-03-08T23:54:23.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.631+0000 7fdcfe806640 1 -- 172.21.15.22:0/2408889555 wait complete. 2024-03-08T23:54:23.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T23:54:23.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 -- 172.21.15.22:0/420555765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 msgr2=0x7f62d81008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 --2- 172.21.15.22:0/420555765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81008e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009900 tx=0x7f62cc02bc80 comp rx=0 tx=0).stop 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 -- 172.21.15.22:0/420555765 shutdown_connections 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 --2- 172.21.15.22:0/420555765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d8100e20 0x7f62d81086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 --2- 172.21.15.22:0/420555765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81008e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 --2- 172.21.15.22:0/420555765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d81018a0 0x7f62d8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 -- 172.21.15.22:0/420555765 >> 172.21.15.22:0/420555765 conn(0x7f62d80faef0 msgr2=0x7f62d80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 -- 172.21.15.22:0/420555765 shutdown_connections 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.749+0000 7f62dcde1640 1 -- 172.21.15.22:0/420555765 wait complete. 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 Processor -- start 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 -- start start 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d8100e20 0x7f62d8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d81018a0 0x7f62d819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d810bef0 con 0x7f62d8100460 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d810bd70 con 0x7f62d8100e20 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d810c070 con 0x7f62d81018a0 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52014/0 (socket says 172.21.15.22:52014) 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 -- 172.21.15.22:0/67166792 learned_addr learned my addr 172.21.15.22:0/67166792 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d5d9b640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d8100e20 0x7f62d8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d6d9d640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d81018a0 0x7f62d819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 -- 172.21.15.22:0/67166792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d81018a0 msgr2=0x7f62d819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d81018a0 0x7f62d819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 -- 172.21.15.22:0/67166792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d8100e20 msgr2=0x7f62d8197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d8100e20 0x7f62d8197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 -- 172.21.15.22:0/67166792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d819f280 con 0x7f62d8100460 2024-03-08T23:54:23.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d6d9d640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d81018a0 0x7f62d819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:23.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62d659c640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81976b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f62c800e9f0 tx=0x7f62c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:23.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62c77fe640 1 -- 172.21.15.22:0/67166792 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800cd80 con 0x7f62d8100460 2024-03-08T23:54:23.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62c77fe640 1 -- 172.21.15.22:0/67166792 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62c8010910 con 0x7f62d8100460 2024-03-08T23:54:23.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d819f570 con 0x7f62d8100460 2024-03-08T23:54:23.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62c77fe640 1 -- 172.21.15.22:0/67166792 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8018990 con 0x7f62d8100460 2024-03-08T23:54:23.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.750+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d819fa90 con 0x7f62d8100460 2024-03-08T23:54:23.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.751+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8068530 con 0x7f62d8100460 2024-03-08T23:54:23.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.752+0000 7f62c77fe640 1 -- 172.21.15.22:0/67166792 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f62c8010ac0 con 0x7f62d8100460 2024-03-08T23:54:23.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.753+0000 7f62c77fe640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62c00a2890 0x7f62c00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:23.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.753+0000 7f62d5d9b640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62c00a2890 0x7f62c00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:23.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.753+0000 7f62c77fe640 1 -- 172.21.15.22:0/67166792 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f62c80cf740 con 0x7f62d8100460 2024-03-08T23:54:23.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.754+0000 7f62d5d9b640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62c00a2890 0x7f62c00a4d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f62cc02c7a0 tx=0x7f62cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:23.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.756+0000 7f62c77fe640 1 -- 172.21.15.22:0/67166792 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f62c809c2f0 con 0x7f62d8100460 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1126034593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3235228681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:23.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:23 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2408889555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.944+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62d819fdb0 con 0x7f62d8100460 2024-03-08T23:54:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.945+0000 7f62c77fe640 1 -- 172.21.15.22:0/67166792 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62c809bbf0 con 0x7f62d8100460 2024-03-08T23:54:23.941 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-08T23:54:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.946+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62c00a2890 msgr2=0x7f62c00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.946+0000 7f62dcde1640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62c00a2890 0x7f62c00a4d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f62cc02c7a0 tx=0x7f62cc000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.946+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 msgr2=0x7f62d81976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.946+0000 7f62dcde1640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81976b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f62c800e9f0 tx=0x7f62c800eeb0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.946+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 shutdown_connections 2024-03-08T23:54:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.946+0000 7f62dcde1640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d81018a0 0x7f62d819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.946+0000 7f62dcde1640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d8100e20 0x7f62d8197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.947+0000 7f62dcde1640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f62c00a2890 0x7f62c00a4d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.947+0000 7f62dcde1640 1 --2- 172.21.15.22:0/67166792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62d8100460 0x7f62d81976b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.947+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 >> 172.21.15.22:0/67166792 conn(0x7f62d80faef0 msgr2=0x7f62d80fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.947+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 shutdown_connections 2024-03-08T23:54:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:23.947+0000 7f62dcde1640 1 -- 172.21.15.22:0/67166792 wait complete. 2024-03-08T23:54:23.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-08T23:54:24.015 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-08T23:54:24.015 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:24.015 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:24.015 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:24.015 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:24.015 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1126034593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:24.016 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3235228681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:24.016 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:23 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2408889555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 -- 172.21.15.22:0/3355779554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78104710 msgr2=0x7eff78106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/3355779554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78104710 0x7eff78106b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff68009900 tx=0x7eff6802bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 -- 172.21.15.22:0/3355779554 shutdown_connections 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/3355779554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff78107040 0x7eff78109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/3355779554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78104710 0x7eff78106b00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/3355779554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff78101de0 0x7eff781041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 -- 172.21.15.22:0/3355779554 >> 172.21.15.22:0/3355779554 conn(0x7eff780fb790 msgr2=0x7eff780fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 -- 172.21.15.22:0/3355779554 shutdown_connections 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.064+0000 7eff7f6c0640 1 -- 172.21.15.22:0/3355779554 wait complete. 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 Processor -- start 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 -- start start 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff78101de0 0x7eff78197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff78104710 0x7eff78197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78107040 0x7eff7819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7810cf90 con 0x7eff78107040 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7810ce10 con 0x7eff78101de0 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7f6c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7810d110 con 0x7eff78104710 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78107040 0x7eff7819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78107040 0x7eff7819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52028/0 (socket says 172.21.15.22:52028) 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 -- 172.21.15.22:0/2877475498 learned_addr learned my addr 172.21.15.22:0/2877475498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7cc5b640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff78104710 0x7eff78197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7d45c640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff78101de0 0x7eff78197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 -- 172.21.15.22:0/2877475498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff78104710 msgr2=0x7eff78197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff78104710 0x7eff78197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 -- 172.21.15.22:0/2877475498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff78101de0 msgr2=0x7eff78197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff78101de0 0x7eff78197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.065+0000 7eff7dc5d640 1 -- 172.21.15.22:0/2877475498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7819f2a0 con 0x7eff78107040 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.066+0000 7eff7dc5d640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78107040 0x7eff7819ec30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff7000eb10 tx=0x7eff7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.066+0000 7eff6e7fc640 1 -- 172.21.15.22:0/2877475498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70004b20 con 0x7eff78107040 2024-03-08T23:54:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.066+0000 7eff6e7fc640 1 -- 172.21.15.22:0/2877475498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff70004cc0 con 0x7eff78107040 2024-03-08T23:54:24.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.066+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7819f590 con 0x7eff78107040 2024-03-08T23:54:24.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.066+0000 7eff6e7fc640 1 -- 172.21.15.22:0/2877475498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70010760 con 0x7eff78107040 2024-03-08T23:54:24.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.066+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7819fa60 con 0x7eff78107040 2024-03-08T23:54:24.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.067+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff780fd280 con 0x7eff78107040 2024-03-08T23:54:24.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.068+0000 7eff6e7fc640 1 -- 172.21.15.22:0/2877475498 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7eff70005020 con 0x7eff78107040 2024-03-08T23:54:24.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.068+0000 7eff6e7fc640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff640a2890 0x7eff640a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.068+0000 7eff6e7fc640 1 -- 172.21.15.22:0/2877475498 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff700cf600 con 0x7eff78107040 2024-03-08T23:54:24.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.068+0000 7eff7d45c640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff640a2890 0x7eff640a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.069+0000 7eff7d45c640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff640a2890 0x7eff640a4d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7eff74009aa0 tx=0x7eff74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:24.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.073+0000 7eff6e7fc640 1 -- 172.21.15.22:0/2877475498 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7eff7009c150 con 0x7eff78107040 2024-03-08T23:54:24.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-08T23:54:24.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T23:54:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1126034593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3235228681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T23:54:24.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:23 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2408889555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T23:54:24.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.258+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7eff78061ef0 con 0x7eff78107040 2024-03-08T23:54:24.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.258+0000 7eff6e7fc640 1 -- 172.21.15.22:0/2877475498 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7eff7009ba50 con 0x7eff78107040 2024-03-08T23:54:24.255 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-08T23:54:24.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.260+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff640a2890 msgr2=0x7eff640a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.260+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff640a2890 0x7eff640a4d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7eff74009aa0 tx=0x7eff74005f90 comp rx=0 tx=0).stop 2024-03-08T23:54:24.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.260+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78107040 msgr2=0x7eff7819ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.260+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78107040 0x7eff7819ec30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff7000eb10 tx=0x7eff7000c2d0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 shutdown_connections 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7eff640a2890 0x7eff640a4d50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff78107040 0x7eff7819ec30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff78104710 0x7eff78197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 --2- 172.21.15.22:0/2877475498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff78101de0 0x7eff78197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 >> 172.21.15.22:0/2877475498 conn(0x7eff780fb790 msgr2=0x7eff781028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 shutdown_connections 2024-03-08T23:54:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.261+0000 7eff7f6c0640 1 -- 172.21.15.22:0/2877475498 wait complete. 2024-03-08T23:54:24.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 -- 172.21.15.22:0/1531563661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 msgr2=0x7fcfa8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/1531563661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8101880 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009900 tx=0x7fcf9802bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 -- 172.21.15.22:0/1531563661 shutdown_connections 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/1531563661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa8101dc0 0x7fcfa8108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/1531563661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8101880 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/1531563661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfa8100210 0x7fcfa8100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 -- 172.21.15.22:0/1531563661 >> 172.21.15.22:0/1531563661 conn(0x7fcfa80fb780 msgr2=0x7fcfa80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:24.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 -- 172.21.15.22:0/1531563661 shutdown_connections 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.384+0000 7fcfb07c4640 1 -- 172.21.15.22:0/1531563661 wait complete. 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 Processor -- start 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 -- start start 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa8100210 0x7fcfa81976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfa8101dc0 0x7fcfa819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa810bea0 con 0x7fcfa8101400 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa810bd20 con 0x7fcfa8100210 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.385+0000 7fcfb07c4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa810c020 con 0x7fcfa8101dc0 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52054/0 (socket says 172.21.15.22:52054) 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 -- 172.21.15.22:0/2192363917 learned_addr learned my addr 172.21.15.22:0/2192363917 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfae560640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa8100210 0x7fcfa81976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfaed61640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfa8101dc0 0x7fcfa819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 -- 172.21.15.22:0/2192363917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfa8101dc0 msgr2=0x7fcfa819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfa8101dc0 0x7fcfa819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 -- 172.21.15.22:0/2192363917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa8100210 msgr2=0x7fcfa81976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa8100210 0x7fcfa81976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 -- 172.21.15.22:0/2192363917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa819f280 con 0x7fcfa8101400 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfadd5f640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8197c10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009230 tx=0x7fcf9802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcf9f7fe640 1 -- 172.21.15.22:0/2192363917 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98041070 con 0x7fcfa8101400 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa819f510 con 0x7fcfa8101400 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcf9f7fe640 1 -- 172.21.15.22:0/2192363917 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf98004450 con 0x7fcfa8101400 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcf9f7fe640 1 -- 172.21.15.22:0/2192363917 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98003e20 con 0x7fcfa8101400 2024-03-08T23:54:24.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.386+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa819fa00 con 0x7fcfa8101400 2024-03-08T23:54:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.387+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa819fd20 con 0x7fcfa8101400 2024-03-08T23:54:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.388+0000 7fcf9f7fe640 1 -- 172.21.15.22:0/2192363917 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcf98044070 con 0x7fcfa8101400 2024-03-08T23:54:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.389+0000 7fcf9f7fe640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcf940a2890 0x7fcf940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.389+0000 7fcf9f7fe640 1 -- 172.21.15.22:0/2192363917 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcf980f0a20 con 0x7fcfa8101400 2024-03-08T23:54:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.389+0000 7fcfae560640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcf940a2890 0x7fcf940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.389+0000 7fcfae560640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcf940a2890 0x7fcf940a4d50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8101260 tx=0x7fcfa4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:24.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.394+0000 7fcf9f7fe640 1 -- 172.21.15.22:0/2192363917 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fcf980bd090 con 0x7fcfa8101400 2024-03-08T23:54:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.641+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfa819fef0 con 0x7fcfa8101400 2024-03-08T23:54:24.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.642+0000 7fcf9f7fe640 1 -- 172.21.15.22:0/2192363917 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fcf980c1130 con 0x7fcfa8101400 2024-03-08T23:54:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:24.639 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:54:17.470857+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:54:24.639 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-03-08T23:54:24.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcf940a2890 msgr2=0x7fcf940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcf940a2890 0x7fcf940a4d50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8101260 tx=0x7fcfa4005f90 comp rx=0 tx=0).stop 2024-03-08T23:54:24.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 msgr2=0x7fcfa8197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8197c10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009230 tx=0x7fcf9802c650 comp rx=0 tx=0).stop 2024-03-08T23:54:24.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 shutdown_connections 2024-03-08T23:54:24.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfa8101dc0 0x7fcfa819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fcf940a2890 0x7fcf940a4d50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfa8101400 0x7fcfa8197c10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 --2- 172.21.15.22:0/2192363917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa8100210 0x7fcfa81976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 >> 172.21.15.22:0/2192363917 conn(0x7fcfa80fb780 msgr2=0x7fcfa80fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:24.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 shutdown_connections 2024-03-08T23:54:24.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.644+0000 7fcfb07c4640 1 -- 172.21.15.22:0/2192363917 wait complete. 2024-03-08T23:54:24.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T23:54:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1590770623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 msgr2=0x7f2f9c104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1590770623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c104240 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009860 tx=0x7f2f8c02bb70 comp rx=0 tx=0).stop 2024-03-08T23:54:24.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1590770623 shutdown_connections 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1590770623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f9c1070b0 0x7f2f9c1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1590770623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9c104780 0x7f2f9c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1590770623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c104240 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1590770623 >> 172.21.15.22:0/1590770623 conn(0x7f2f9c0fb7a0 msgr2=0x7f2f9c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1590770623 shutdown_connections 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.767+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1590770623 wait complete. 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 Processor -- start 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 -- start start 2024-03-08T23:54:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c1017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f9c104780 0x7f2f9c0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9c1070b0 0x7f2f9c100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c10cfb0 con 0x7f2f9c101e50 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c10ce30 con 0x7f2f9c1070b0 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa32a1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c10d130 con 0x7f2f9c104780 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c1017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c1017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52072/0 (socket says 172.21.15.22:52072) 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 -- 172.21.15.22:0/1233204333 learned_addr learned my addr 172.21.15.22:0/1233204333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa183e640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9c1070b0 0x7f2f9c100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa083c640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f9c104780 0x7f2f9c0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 -- 172.21.15.22:0/1233204333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f9c104780 msgr2=0x7f2f9c0ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f9c104780 0x7f2f9c0ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 -- 172.21.15.22:0/1233204333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9c1070b0 msgr2=0x7f2f9c100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9c1070b0 0x7f2f9c100340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa103d640 1 -- 172.21.15.22:0/1233204333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c100bd0 con 0x7f2f9c101e50 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.768+0000 7f2fa183e640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9c1070b0 0x7f2f9c100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.769+0000 7f2fa103d640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c1017c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005940 tx=0x7f2f8c032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.769+0000 7f2f927fc640 1 -- 172.21.15.22:0/1233204333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c041020 con 0x7f2f9c101e50 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.769+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9c1a1720 con 0x7f2f9c101e50 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.769+0000 7f2f927fc640 1 -- 172.21.15.22:0/1233204333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f8c02c9f0 con 0x7f2f9c101e50 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.769+0000 7f2f927fc640 1 -- 172.21.15.22:0/1233204333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c039040 con 0x7f2f9c101e50 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.769+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c1a1cc0 con 0x7f2f9c101e50 2024-03-08T23:54:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.770+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9c0fd290 con 0x7f2f9c101e50 2024-03-08T23:54:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.771+0000 7f2f927fc640 1 -- 172.21.15.22:0/1233204333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2f8c044020 con 0x7f2f9c101e50 2024-03-08T23:54:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.772+0000 7f2f927fc640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2f880a2890 0x7f2f880a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.772+0000 7f2f927fc640 1 -- 172.21.15.22:0/1233204333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2f8c0f0bd0 con 0x7f2f9c101e50 2024-03-08T23:54:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.772+0000 7f2fa083c640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2f880a2890 0x7f2f880a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.772+0000 7f2fa083c640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2f880a2890 0x7f2f880a4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c101530 tx=0x7f2f98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.775+0000 7f2f927fc640 1 -- 172.21.15.22:0/1233204333 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2f8c04b020 con 0x7f2f9c101e50 2024-03-08T23:54:24.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.964+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f9c061ef0 con 0x7f2f9c101e50 2024-03-08T23:54:24.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.967+0000 7f2f927fc640 1 -- 172.21.15.22:0/1233204333 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24654 (secure 0 0 0) 0x7f2f8c0bd890 con 0x7f2f9c101e50 2024-03-08T23:54:24.963 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:24.963 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-08T23:53:45.618289+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T23:54:24.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2f880a2890 msgr2=0x7f2f880a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2f880a2890 0x7f2f880a4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c101530 tx=0x7f2f98009260 comp rx=0 tx=0).stop 2024-03-08T23:54:24.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 msgr2=0x7f2f9c1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:24.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c1017c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005940 tx=0x7f2f8c032470 comp rx=0 tx=0).stop 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 shutdown_connections 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9c1070b0 0x7f2f9c100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f9c104780 0x7f2f9c0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f2f880a2890 0x7f2f880a4d50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 --2- 172.21.15.22:0/1233204333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f9c101e50 0x7f2f9c1017c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 >> 172.21.15.22:0/1233204333 conn(0x7f2f9c0fb7a0 msgr2=0x7f2f9c102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 shutdown_connections 2024-03-08T23:54:24.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:24.969+0000 7f2fa32a1640 1 -- 172.21.15.22:0/1233204333 wait complete. 2024-03-08T23:54:24.975 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/67166792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:24.975 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2877475498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:24.975 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2192363917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:24.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:24.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:24.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:24.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:24.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:24.976 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:24 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:24.977 DEBUG:tasks.fwd_scrub:Yielding 2024-03-08T23:54:24.978 INFO:teuthology.run_tasks:Running task exec... 2024-03-08T23:54:24.987 INFO:teuthology.task.exec:Executing custom commands... 2024-03-08T23:54:24.988 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-08T23:54:24.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-08T23:54:24.988 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-08T23:54:24.988 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:54:24.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:24.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:54:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 -- 172.21.15.22:0/74475065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102540 msgr2=0x7fe5ec1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 --2- 172.21.15.22:0/74475065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102540 0x7fe5ec1029c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009900 tx=0x7fe5dc02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 -- 172.21.15.22:0/74475065 shutdown_connections 2024-03-08T23:54:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 --2- 172.21.15.22:0/74475065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec102f00 0x7fe5ec109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 --2- 172.21.15.22:0/74475065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102540 0x7fe5ec1029c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 --2- 172.21.15.22:0/74475065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ec101350 0x7fe5ec101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 -- 172.21.15.22:0/74475065 >> 172.21.15.22:0/74475065 conn(0x7fe5ec0fc8c0 msgr2=0x7fe5ec0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 -- 172.21.15.22:0/74475065 shutdown_connections 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 -- 172.21.15.22:0/74475065 wait complete. 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.098+0000 7fe5f3380640 1 Processor -- start 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f3380640 1 -- start start 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f3380640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec101350 0x7fe5ec197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f3380640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ec102540 0x7fe5ec197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f3380640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102f00 0x7fe5ec19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f3380640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec10cfe0 con 0x7fe5ec102f00 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f3380640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec10ce60 con 0x7fe5ec101350 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f3380640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec10d160 con 0x7fe5ec102540 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec101350 0x7fe5ec197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f191d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102f00 0x7fe5ec19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec101350 0x7fe5ec197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41658/0 (socket says 172.21.15.22:41658) 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 -- 172.21.15.22:0/3954025492 learned_addr learned my addr 172.21.15.22:0/3954025492 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f091b640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ec102540 0x7fe5ec197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 -- 172.21.15.22:0/3954025492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ec102540 msgr2=0x7fe5ec197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ec102540 0x7fe5ec197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 -- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102f00 msgr2=0x7fe5ec19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102f00 0x7fe5ec19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f111c640 1 -- 172.21.15.22:0/3954025492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec19f1a0 con 0x7fe5ec101350 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.099+0000 7fe5f191d640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102f00 0x7fe5ec19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.100+0000 7fe5f111c640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec101350 0x7fe5ec197580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800e9c0 tx=0x7fe5e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.100+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800cd80 con 0x7fe5ec101350 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.100+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec19f490 con 0x7fe5ec101350 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.100+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5e80035f0 con 0x7fe5ec101350 2024-03-08T23:54:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.100+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8010690 con 0x7fe5ec101350 2024-03-08T23:54:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.100+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec19fa00 con 0x7fe5ec101350 2024-03-08T23:54:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 -- 172.21.15.22:0/3755668295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce60106fd0 msgr2=0x7fce60109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 --2- 172.21.15.22:0/3755668295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce60106fd0 0x7fce60109420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fce58009ca0 tx=0x7fce5802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec108670 con 0x7fe5ec101350 2024-03-08T23:54:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 -- 172.21.15.22:0/3755668295 shutdown_connections 2024-03-08T23:54:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 --2- 172.21.15.22:0/3755668295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce60106fd0 0x7fce60109420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 --2- 172.21.15.22:0/3755668295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce601046a0 0x7fce60106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 --2- 172.21.15.22:0/3755668295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce60101d70 0x7fce60104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 -- 172.21.15.22:0/3755668295 >> 172.21.15.22:0/3755668295 conn(0x7fce600fb780 msgr2=0x7fce600fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 -- 172.21.15.22:0/3755668295 shutdown_connections 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 -- 172.21.15.22:0/3755668295 wait complete. 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.101+0000 7fce671f1640 1 Processor -- start 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce671f1640 1 -- start start 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce671f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce60101d70 0x7fce601016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce671f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce601046a0 0x7fce600ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce671f1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce60106fd0 0x7fce60100260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce671f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6010cf80 con 0x7fce601046a0 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce671f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6010ce00 con 0x7fce60101d70 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce671f1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6010d100 con 0x7fce60106fd0 2024-03-08T23:54:25.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce601046a0 0x7fce600ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce64f8d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce60101d70 0x7fce601016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce601046a0 0x7fce600ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52110/0 (socket says 172.21.15.22:52110) 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 -- 172.21.15.22:0/3361499283 learned_addr learned my addr 172.21.15.22:0/3361499283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce6578e640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce60106fd0 0x7fce60100260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 -- 172.21.15.22:0/3361499283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce60106fd0 msgr2=0x7fce60100260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce60106fd0 0x7fce60100260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 -- 172.21.15.22:0/3361499283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce60101d70 msgr2=0x7fce601016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce60101d70 0x7fce601016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 -- 172.21.15.22:0/3361499283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60100af0 con 0x7fce601046a0 2024-03-08T23:54:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe5e80037a0 con 0x7fe5ec101350 2024-03-08T23:54:25.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.102+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce601046a0 0x7fce600ffd20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00e9f0 tx=0x7fce4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.103+0000 7fce5dffb640 1 -- 172.21.15.22:0/3361499283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00ccc0 con 0x7fce601046a0 2024-03-08T23:54:25.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.103+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce601a1720 con 0x7fce601046a0 2024-03-08T23:54:25.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.103+0000 7fce5dffb640 1 -- 172.21.15.22:0/3361499283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c005500 con 0x7fce601046a0 2024-03-08T23:54:25.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.103+0000 7fce5dffb640 1 -- 172.21.15.22:0/3361499283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c010690 con 0x7fce601046a0 2024-03-08T23:54:25.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.103+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce601a1c70 con 0x7fce601046a0 2024-03-08T23:54:25.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.103+0000 7fe5e27fc640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe5d80a2940 0x7fe5d80a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.104+0000 7fe5f091b640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe5d80a2940 0x7fe5d80a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.104+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe5e80cf410 con 0x7fe5ec101350 2024-03-08T23:54:25.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.104+0000 7fe5f091b640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe5d80a2940 0x7fe5d80a4e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc02c7b0 tx=0x7fe5dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce5dffb640 1 -- 172.21.15.22:0/3361499283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fce4c0046c0 con 0x7fce601046a0 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 Processor -- start 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 -- start start 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fb11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fb6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d633fb8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d633f15b40 con 0x55d633fb6900 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d633ebed80 con 0x55d633fb1790 2024-03-08T23:54:25.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce69dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d633f11be0 con 0x55d633fb0da0 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce5dffb640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fce500a2890 0x7fce500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d633fb8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce64f8d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fb6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce6578e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fb11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d633fb8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52120/0 (socket says 172.21.15.22:52120) 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce5ffff640 1 -- 172.21.15.22:0/550334123 learned_addr learned my addr 172.21.15.22:0/550334123 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.106+0000 7fce6578e640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fce500a2890 0x7fce500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce5dffb640 1 -- 172.21.15.22:0/3361499283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fce4c0cf340 con 0x7fce601046a0 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce5ffff640 1 -- 172.21.15.22:0/550334123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 msgr2=0x55d633fb11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce5ffff640 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fb11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce5ffff640 1 -- 172.21.15.22:0/550334123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 msgr2=0x55d633fb6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce5ffff640 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fb6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce5ffff640 1 -- 172.21.15.22:0/550334123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d633cf27e0 con 0x55d633fb6900 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce6578e640 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fb11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce64f8d640 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fb6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce5ffff640 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d633fb8d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c6a0 tx=0x7fce4c004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f300baa9a058b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce6578e640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fce500a2890 0x7fce500a4d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce58005e30 tx=0x7fce58032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce4affd640 1 -- 172.21.15.22:0/550334123 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c09b150 con 0x55d633fb6900 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce4affd640 1 -- 172.21.15.22:0/550334123 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0cb040 con 0x55d633fb6900 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.107+0000 7fce4affd640 1 -- 172.21.15.22:0/550334123 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0860d0 con 0x55d633fb6900 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 -- 172.21.15.22:0/550334123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 msgr2=0x55d633fb8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d633fb8d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c6a0 tx=0x7fce4c004ba0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 -- 172.21.15.22:0/550334123 shutdown_connections 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d633fb8d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fb6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe5e809c050 con 0x7fe5ec101350 2024-03-08T23:54:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 --2- 172.21.15.22:0/550334123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fb11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 -- 172.21.15.22:0/550334123 >> 172.21.15.22:0/550334123 conn(0x55d633fac310 msgr2=0x55d633fac710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 -- 172.21.15.22:0/550334123 shutdown_connections 2024-03-08T23:54:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 -- 172.21.15.22:0/550334123 wait complete. 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.108+0000 7fce69dd5740 1 -- start start 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 1 Processor -- start 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633fbb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fbb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fc2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d633fb98b0 con 0x55d633fb0da0 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d633ed21c0 con 0x55d633fb1790 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d633fb9a30 con 0x55d633fb6900 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fbb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce64f8d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633fbb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce69dd5740 10 client.0 _ll_register_callbacks cb 0x55d633ec17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fbb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41696/0 (socket says 172.21.15.22:41696) 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 -- 172.21.15.22:0/3272524889 learned_addr learned my addr 172.21.15.22:0/3272524889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce6578e640 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fc2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 -- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 msgr2=0x55d633fc2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fc2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 -- 172.21.15.22:0/3272524889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 msgr2=0x55d633fbb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633fbb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.109+0000 7fce5ffff640 1 -- 172.21.15.22:0/3272524889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d633d52f70 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce64f8d640 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633fbb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fbb5f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce4c09b780 tx=0x7fce4c0cc8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3267a22f99f3e73b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce3ffff640 1 -- 172.21.15.22:0/3272524889 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c09b150 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce3ffff640 1 -- 172.21.15.22:0/3272524889 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fce4c099db0 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce4c0cb040 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce3ffff640 1 -- 172.21.15.22:0/3272524889 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4c09b7f0 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce69dd5740 10 client.25205 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d633bea170 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.110+0000 7fce3ffff640 1 -- 172.21.15.22:0/3272524889 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0cb040 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce3ffff640 1 -- 172.21.15.22:0/3272524889 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fce4c0bb040 con 0x55d633fb1790 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce69dd5740 10 client.25205 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce69dd5740 10 client.25205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce69dd5740 20 client.25205 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d633fc8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce69dd5740 4 client.25205 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d633eeb960 con 0x55d633fc6b20 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce64f8d640 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d633fc8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce64f8d640 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d633fc8f10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349fcd807c5ae1ec server_cookie=3fb18f6845b6b257 in_seq=0 out_seq=0 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.111+0000 7fce3ffff640 10 client.25205 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce3ffff640 1 -- 172.21.15.22:0/3272524889 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d633eeb960 con 0x55d633fc6b20 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce3ffff640 10 client.25205 handle_command_reply: tid=0 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 client.25205 shutdown 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 msgr2=0x55d633fbb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fbb5f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce4c09b780 tx=0x7fce4c0cc8a0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 shutdown_connections 2024-03-08T23:54:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d633fc8f10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fc2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce3ffff640 0 client.25205 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fbb5f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3272524889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633fbb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 >> 172.21.15.22:0/3272524889 conn(0x55d633fac310 msgr2=0x55d633fab480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 shutdown_connections 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 1 -- 172.21.15.22:0/3272524889 wait complete. 2024-03-08T23:54:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.112+0000 7fce69dd5740 20 client.25205 trim_cache size 0 max 16384 2024-03-08T23:54:25.118 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/67166792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2877475498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2192363917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.119 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:24 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 Processor -- start 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 -- start start 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633faabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601aa0c0 con 0x55d633fb0da0 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a7c60 con 0x55d633fb1790 2024-03-08T23:54:25.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce69dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a5d20 con 0x55d633fb6900 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce64f8d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633faabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce64f8d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633faabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52138/0 (socket says 172.21.15.22:52138) 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce64f8d640 1 -- 172.21.15.22:0/1739527288 learned_addr learned my addr 172.21.15.22:0/1739527288 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce6578e640 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce5ffff640 1 -- 172.21.15.22:0/1739527288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 msgr2=0x55d633fab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce5ffff640 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce5ffff640 1 -- 172.21.15.22:0/1739527288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 msgr2=0x55d633faabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce5ffff640 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633faabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce5ffff640 1 -- 172.21.15.22:0/1739527288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d633fbc230 con 0x55d633fb1790 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.128+0000 7fce64f8d640 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633faabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.129+0000 7fce5ffff640 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc25f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0097f0 tx=0x7fce4c004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a20f2ec09d6b153 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.129+0000 7fce4a7fc640 1 -- 172.21.15.22:0/1739527288 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c09b150 con 0x55d633fb1790 2024-03-08T23:54:25.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.129+0000 7fce4a7fc640 1 -- 172.21.15.22:0/1739527288 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0cf170 con 0x55d633fb1790 2024-03-08T23:54:25.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.129+0000 7fce4a7fc640 1 -- 172.21.15.22:0/1739527288 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c09f310 con 0x55d633fb1790 2024-03-08T23:54:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 -- 172.21.15.22:0/1739527288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 msgr2=0x55d633fc25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc25f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0097f0 tx=0x7fce4c004ba0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 -- 172.21.15.22:0/1739527288 shutdown_connections 2024-03-08T23:54:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb6900 0x55d633fab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc25f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 --2- 172.21.15.22:0/1739527288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb0da0 0x55d633faabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 -- 172.21.15.22:0/1739527288 >> 172.21.15.22:0/1739527288 conn(0x55d633fac310 msgr2=0x55d633fb2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 -- 172.21.15.22:0/1739527288 shutdown_connections 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 -- 172.21.15.22:0/1739527288 wait complete. 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 -- start start 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.130+0000 7fce69dd5740 1 Processor -- start 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fc1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce69dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d63401ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce69dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a6320 con 0x55d633fb6900 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce69dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a7600 con 0x55d633fb1790 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce69dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a6940 con 0x55d633fb0da0 2024-03-08T23:54:25.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce69dd5740 10 client.0 _ll_register_callbacks cb 0x7fce4c097370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce6578e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d63401ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce64f8d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fc1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41724/0 (socket says 172.21.15.22:41724) 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 -- 172.21.15.22:0/3542223578 learned_addr learned my addr 172.21.15.22:0/3542223578 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 -- 172.21.15.22:0/3542223578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 msgr2=0x55d633fc1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fc1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 -- 172.21.15.22:0/3542223578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 msgr2=0x55d63401ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d63401ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce5ffff640 1 -- 172.21.15.22:0/3542223578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d63401d430 con 0x55d633fb1790 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.131+0000 7fce6578e640 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d63401ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce5ffff640 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc1b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0974d0 tx=0x7fce4c09bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1770c768fca4b90c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce3ffff640 1 -- 172.21.15.22:0/3542223578 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c07f060 con 0x55d633fb1790 2024-03-08T23:54:25.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce3ffff640 1 -- 172.21.15.22:0/3542223578 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fce4c0a3b00 con 0x55d633fb1790 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce4c0cf170 con 0x55d633fb1790 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce3ffff640 1 -- 172.21.15.22:0/3542223578 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4c09f310 con 0x55d633fb1790 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 10 client.25211 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d63401e770 con 0x55d633fb1790 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce3ffff640 1 -- 172.21.15.22:0/3542223578 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0cf170 con 0x55d633fb1790 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce3ffff640 1 -- 172.21.15.22:0/3542223578 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fce4c0bb040 con 0x55d633fb1790 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 10 client.25211 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 10 client.25211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 20 client.25211 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d634020c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 4 client.25211 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d633ed7080 con 0x55d633fc6b20 2024-03-08T23:54:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.132+0000 7fce64f8d640 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d634020c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.133+0000 7fce64f8d640 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d634020c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533968436ef409fa server_cookie=e0cfcf85d34705d3 in_seq=0 out_seq=0 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.133+0000 7fce3ffff640 10 client.25211 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce3ffff640 1 -- 172.21.15.22:0/3542223578 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d633ed7080 con 0x55d633fc6b20 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce3ffff640 10 client.25211 handle_command_reply: tid=0 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 client.25211 shutdown 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 msgr2=0x55d633fc1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc1b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0974d0 tx=0x7fce4c09bc60 comp rx=0 tx=0).stop 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 shutdown_connections 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d633fc6b20 0x55d634020c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d633fb6900 0x55d63401ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d633fb1790 0x55d633fc1b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 --2- 172.21.15.22:0/3542223578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d633fb0da0 0x55d633fc1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 >> 172.21.15.22:0/3542223578 conn(0x55d633fac310 msgr2=0x55d633faf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.134+0000 7fce3ffff640 0 client.25211 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 shutdown_connections 2024-03-08T23:54:25.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce69dd5740 1 -- 172.21.15.22:0/3542223578 wait complete. 2024-03-08T23:54:25.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce69dd5740 20 client.25211 trim_cache size 0 max 16384 2024-03-08T23:54:25.136 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:54:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "fc2a0b18-8048-4ae5-8624-e1da499868c9", 2024-03-08T23:54:25.136 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:54:25.136 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fce500a2890 msgr2=0x7fce500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fce500a2890 0x7fce500a4d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce58005e30 tx=0x7fce58032590 comp rx=0 tx=0).stop 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce601046a0 msgr2=0x7fce600ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce601046a0 0x7fce600ffd20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00e9f0 tx=0x7fce4c00eeb0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 shutdown_connections 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce60106fd0 0x7fce60100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fce500a2890 0x7fce500a4d50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fce601046a0 0x7fce600ffd20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 --2- 172.21.15.22:0/3361499283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce60101d70 0x7fce601016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.135+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 >> 172.21.15.22:0/3361499283 conn(0x7fce600fb780 msgr2=0x7fce601026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.136+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 shutdown_connections 2024-03-08T23:54:25.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.136+0000 7fce671f1640 1 -- 172.21.15.22:0/3361499283 wait complete. 2024-03-08T23:54:25.138 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/67166792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T23:54:25.138 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2877475498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-08T23:54:25.138 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2192363917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:25.138 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.138 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.139 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.139 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.139 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.139 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:24 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:25.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:54:25.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 -- 172.21.15.22:0/4042495762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa528101da0 msgr2=0x7fa528108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 --2- 172.21.15.22:0/4042495762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa528101da0 0x7fa528108630 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa520009ca0 tx=0x7fa52002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 -- 172.21.15.22:0/4042495762 shutdown_connections 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 --2- 172.21.15.22:0/4042495762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa528101da0 0x7fa528108630 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 --2- 172.21.15.22:0/4042495762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5281013e0 0x7fa528101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 --2- 172.21.15.22:0/4042495762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281001f0 0x7fa528100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 -- 172.21.15.22:0/4042495762 >> 172.21.15.22:0/4042495762 conn(0x7fa5280fb780 msgr2=0x7fa5280fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 -- 172.21.15.22:0/4042495762 shutdown_connections 2024-03-08T23:54:25.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.261+0000 7fa5303c4640 1 -- 172.21.15.22:0/4042495762 wait complete. 2024-03-08T23:54:25.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 Processor -- start 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 -- start start 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5281001f0 0x7fa5281975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281013e0 0x7fa528197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa528101da0 0x7fa52819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52810be80 con 0x7fa5281001f0 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52810bd00 con 0x7fa528101da0 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.262+0000 7fa5303c4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52810c000 con 0x7fa5281013e0 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5281001f0 0x7fa5281975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5281001f0 0x7fa5281975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52152/0 (socket says 172.21.15.22:52152) 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 -- 172.21.15.22:0/3496922012 learned_addr learned my addr 172.21.15.22:0/3496922012 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52d95f640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281013e0 0x7fa528197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e961640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa528101da0 0x7fa52819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 -- 172.21.15.22:0/3496922012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281013e0 msgr2=0x7fa528197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281013e0 0x7fa528197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 -- 172.21.15.22:0/3496922012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa528101da0 msgr2=0x7fa52819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa528101da0 0x7fa52819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 -- 172.21.15.22:0/3496922012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52819f210 con 0x7fa5281001f0 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52d95f640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281013e0 0x7fa528197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e961640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa528101da0 0x7fa52819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa52e160640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5281001f0 0x7fa5281975f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9c0 tx=0x7fa52400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa51f7fe640 1 -- 172.21.15.22:0/3496922012 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52400cd80 con 0x7fa5281001f0 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52819f500 con 0x7fa5281001f0 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.263+0000 7fa51f7fe640 1 -- 172.21.15.22:0/3496922012 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5240035f0 con 0x7fa5281001f0 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.264+0000 7fa51f7fe640 1 -- 172.21.15.22:0/3496922012 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524010690 con 0x7fa5281001f0 2024-03-08T23:54:25.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.264+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52819fa50 con 0x7fa5281001f0 2024-03-08T23:54:25.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.267+0000 7fa51f7fe640 1 -- 172.21.15.22:0/3496922012 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 19) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fa5240037a0 con 0x7fa5281001f0 2024-03-08T23:54:25.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.267+0000 7fa532fa6740 1 Processor -- start 2024-03-08T23:54:25.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa532fa6740 1 -- start start 2024-03-08T23:54:25.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bef6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa51f7fe640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa5140a2890 0x7fa5140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef7580 0x55793befc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793befeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa532fa6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793be56e00 con 0x55793bef7580 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa532fa6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793bdffb10 con 0x55793bef6b90 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa532fa6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793be52ea0 con 0x55793befc6f0 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52e961640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa5140a2890 0x7fa5140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52d95f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef7580 0x55793befc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52e160640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bef6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52d95f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef7580 0x55793befc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52158/0 (socket says 172.21.15.22:52158) 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52d95f640 1 -- 172.21.15.22:0/2649135105 learned_addr learned my addr 172.21.15.22:0/2649135105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52e961640 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793befeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52e160640 1 -- 172.21.15.22:0/2649135105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 msgr2=0x55793befeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52e160640 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793befeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52e160640 1 -- 172.21.15.22:0/2649135105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef7580 msgr2=0x55793befc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.268+0000 7fa52e160640 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef7580 0x55793befc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa52e160640 1 -- 172.21.15.22:0/2649135105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793bc357d0 con 0x55793bef6b90 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa52d95f640 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef7580 0x55793befc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa51f7fe640 1 -- 172.21.15.22:0/3496922012 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa5240cf2e0 con 0x7fa5281001f0 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa52e961640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa5140a2890 0x7fa5140a4d50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa520005e30 tx=0x7fa520009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa52e160640 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bef6fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa5240cc900 tx=0x7fa524021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700c1efe9762836 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa51cbfe640 1 -- 172.21.15.22:0/2649135105 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52409be90 con 0x55793bef6b90 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa51cbfe640 1 -- 172.21.15.22:0/2649135105 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5240a5040 con 0x55793bef6b90 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.269+0000 7fa51cbfe640 1 -- 172.21.15.22:0/2649135105 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5240a3f30 con 0x55793bef6b90 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 -- 172.21.15.22:0/2649135105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 msgr2=0x55793bef6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bef6fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa5240cc900 tx=0x7fa524021000 comp rx=0 tx=0).stop 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 -- 172.21.15.22:0/2649135105 shutdown_connections 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793befeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef7580 0x55793befc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2649135105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bef6fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 -- 172.21.15.22:0/2649135105 >> 172.21.15.22:0/2649135105 conn(0x55793beedcf0 msgr2=0x55793beee0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 -- 172.21.15.22:0/2649135105 shutdown_connections 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 -- 172.21.15.22:0/2649135105 wait complete. 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 -- start start 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 Processor -- start 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.270+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf01070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa532fa6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793beff640 con 0x55793befc6f0 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa532fa6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793be130a0 con 0x55793bef7580 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa532fa6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793beff7c0 con 0x55793bef6b90 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa532fa6740 10 client.0 _ll_register_callbacks cb 0x55793be12a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:25.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf01070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52e961640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52e160640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf01070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41768/0 (socket says 172.21.15.22:41768) 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 -- 172.21.15.22:0/2662964848 learned_addr learned my addr 172.21.15.22:0/2662964848 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52e160640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60222/0 (socket says 172.21.15.22:60222) 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 -- 172.21.15.22:0/2662964848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef6b90 msgr2=0x55793bf00b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 -- 172.21.15.22:0/2662964848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 msgr2=0x55793bf080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 -- 172.21.15.22:0/2662964848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793bb6b3f0 con 0x55793bef7580 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52e961640 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa52d95f640 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf01070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa518011830 tx=0x7fa518011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db661b612b8a4039 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:54:25.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa511ffb640 1 -- 172.21.15.22:0/2662964848 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51801f060 con 0x55793bef7580 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5240a5040 con 0x55793bef7580 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.271+0000 7fa511ffb640 1 -- 172.21.15.22:0/2662964848 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa518010ba0 con 0x55793bef7580 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa511ffb640 1 -- 172.21.15.22:0/2662964848 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa518026050 con 0x55793bef7580 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 10 client.25223 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55793bb2b0f0 con 0x55793bef7580 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa511ffb640 1 -- 172.21.15.22:0/2662964848 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5240a5040 con 0x55793bef7580 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa511ffb640 1 -- 172.21.15.22:0/2662964848 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa518019370 con 0x55793bef7580 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 10 client.25223 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 10 client.25223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 20 client.25223 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf0c5f0 0x55793bf0ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 4 client.25223 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55793be64340 con 0x55793bf0c5f0 2024-03-08T23:54:25.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.272+0000 7fa52e160640 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf0c5f0 0x55793bf0ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.273+0000 7fa52e160640 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf0c5f0 0x55793bf0ea30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e9db4c1ed4547b server_cookie=15d3c52aad1c5779 in_seq=0 out_seq=0 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.273+0000 7fa511ffb640 10 client.25223 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa511ffb640 1 -- 172.21.15.22:0/2662964848 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55793be64340 con 0x55793bf0c5f0 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa511ffb640 10 client.25223 handle_command_reply: tid=0 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 client.25223 shutdown 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 msgr2=0x55793bf01070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf01070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa518011830 tx=0x7fa518011cf0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 shutdown_connections 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf0c5f0 0x55793bf0ea30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf01070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa511ffb640 0 client.25223 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2662964848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 >> 172.21.15.22:0/2662964848 conn(0x55793beedcf0 msgr2=0x55793beecef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 shutdown_connections 2024-03-08T23:54:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 1 -- 172.21.15.22:0/2662964848 wait complete. 2024-03-08T23:54:25.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.274+0000 7fa532fa6740 20 client.25223 trim_cache size 0 max 16384 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.293+0000 7fa532fa6740 1 Processor -- start 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.293+0000 7fa532fa6740 1 -- start start 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.293+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.293+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef7580 0x55793bf01070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.293+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf08070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.293+0000 7fa532fa6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281aa0c0 con 0x55793befc6f0 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa532fa6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281a7c60 con 0x55793bef6b90 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa532fa6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281a5d00 con 0x55793bef7580 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52e961640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf08070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52e961640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf08070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52184/0 (socket says 172.21.15.22:52184) 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52e961640 1 -- 172.21.15.22:0/1407523370 learned_addr learned my addr 172.21.15.22:0/1407523370 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52d95f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef7580 0x55793bf01070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52e160640 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52d95f640 1 -- 172.21.15.22:0/1407523370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 msgr2=0x55793bf00b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52d95f640 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52d95f640 1 -- 172.21.15.22:0/1407523370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 msgr2=0x55793bf08070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52d95f640 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf08070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52d95f640 1 -- 172.21.15.22:0/1407523370 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793bf01b20 con 0x55793bef7580 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52e961640 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf08070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa52d95f640 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef7580 0x55793bf01070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa518019710 tx=0x7fa51800be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=658bc6d9e12a77c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa513fff640 1 -- 172.21.15.22:0/1407523370 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51801f040 con 0x55793bef7580 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa513fff640 1 -- 172.21.15.22:0/1407523370 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa518036070 con 0x55793bef7580 2024-03-08T23:54:25.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.294+0000 7fa513fff640 1 -- 172.21.15.22:0/1407523370 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518010e00 con 0x55793bef7580 2024-03-08T23:54:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 -- 172.21.15.22:0/1407523370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef7580 msgr2=0x55793bf01070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef7580 0x55793bf01070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa518019710 tx=0x7fa51800be60 comp rx=0 tx=0).stop 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 -- 172.21.15.22:0/1407523370 shutdown_connections 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793befc6f0 0x55793bf08070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793bef7580 0x55793bf01070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 --2- 172.21.15.22:0/1407523370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef6b90 0x55793bf00b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 -- 172.21.15.22:0/1407523370 >> 172.21.15.22:0/1407523370 conn(0x55793beedcf0 msgr2=0x55793bf13390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 -- 172.21.15.22:0/1407523370 shutdown_connections 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.295+0000 7fa532fa6740 1 -- 172.21.15.22:0/1407523370 wait complete. 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 -- start start 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 Processor -- start 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef6b90 0x55793bf042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793bf06d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281a6300 con 0x55793bef6b90 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281a7600 con 0x55793bef7580 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281a6940 con 0x55793befc6f0 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa532fa6740 10 client.0 _ll_register_callbacks cb 0x7fa518011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52e160640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef6b90 0x55793bf042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41784/0 (socket says 172.21.15.22:41784) 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 -- 172.21.15.22:0/2362863654 learned_addr learned my addr 172.21.15.22:0/2362863654 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52e961640 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793bf06d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 -- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 msgr2=0x55793bf06d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793bf06d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 -- 172.21.15.22:0/2362863654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef6b90 msgr2=0x55793bf042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef6b90 0x55793bf042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 -- 172.21.15.22:0/2362863654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793bf075a0 con 0x55793bef7580 2024-03-08T23:54:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa52d95f640 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf067f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa518035000 tx=0x7fa518019600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebeeb776dae6b18b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.296+0000 7fa511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa511ffb640 1 -- 172.21.15.22:0/2362863654 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51801f070 con 0x55793bef7580 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa511ffb640 1 -- 172.21.15.22:0/2362863654 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa518010db0 con 0x55793bef7580 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa518036070 con 0x55793bef7580 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa511ffb640 1 -- 172.21.15.22:0/2362863654 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa518049050 con 0x55793bef7580 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 10 client.25229 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55793bf5ecf0 con 0x55793bef7580 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa511ffb640 1 -- 172.21.15.22:0/2362863654 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa518036070 con 0x55793bef7580 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa511ffb640 1 -- 172.21.15.22:0/2362863654 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa518004660 con 0x55793bef7580 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 10 client.25229 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 10 client.25229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 20 client.25229 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf5f560 0x55793bf61980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 4 client.25229 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:54:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.297+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55793bf00920 con 0x55793bf5f560 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa52e160640 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf5f560 0x55793bf61980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa511ffb640 10 client.25229 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa52e160640 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf5f560 0x55793bf61980 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca7e1d231160b550 server_cookie=5b9201faaf396815 in_seq=0 out_seq=0 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa511ffb640 1 -- 172.21.15.22:0/2362863654 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55793bf00920 con 0x55793bf5f560 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa511ffb640 10 client.25229 handle_command_reply: tid=0 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 client.25229 shutdown 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 msgr2=0x55793bf067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf067f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa518035000 tx=0x7fa518019600 comp rx=0 tx=0).stop 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 shutdown_connections 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55793bf5f560 0x55793bf61980 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55793befc6f0 0x55793bf06d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa511ffb640 0 client.25229 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55793bef7580 0x55793bf067f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 --2- 172.21.15.22:0/2362863654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55793bef6b90 0x55793bf042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 >> 172.21.15.22:0/2362863654 conn(0x55793beedcf0 msgr2=0x55793befb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 shutdown_connections 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.298+0000 7fa532fa6740 1 -- 172.21.15.22:0/2362863654 wait complete. 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa532fa6740 20 client.25229 trim_cache size 0 max 16384 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "fc2a0b18-8048-4ae5-8624-e1da499868c9": { 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:54:25.301 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "fc2a0b18-8048-4ae5-8624-e1da499868c9", 2024-03-08T23:54:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:54:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:54:25.302 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:54:25.302 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa5140a2890 msgr2=0x7fa5140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa5140a2890 0x7fa5140a4d50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa520005e30 tx=0x7fa520009880 comp rx=0 tx=0).stop 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5281001f0 msgr2=0x7fa5281975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5281001f0 0x7fa5281975f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9c0 tx=0x7fa52400ee80 comp rx=0 tx=0).stop 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 shutdown_connections 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa528101da0 0x7fa52819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281013e0 0x7fa528197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fa5140a2890 0x7fa5140a4d50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 --2- 172.21.15.22:0/3496922012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5281001f0 0x7fa5281975f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 >> 172.21.15.22:0/3496922012 conn(0x7fa5280fb780 msgr2=0x7fa5280fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 shutdown_connections 2024-03-08T23:54:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.299+0000 7fa5303c4640 1 -- 172.21.15.22:0/3496922012 wait complete. 2024-03-08T23:54:25.306 INFO:tasks.cephfs.filesystem:scrub status for tag:fc2a0b18-8048-4ae5-8624-e1da499868c9 - {'path': '/', 'tag': 'fc2a0b18-8048-4ae5-8624-e1da499868c9', 'options': 'recursive,force'} 2024-03-08T23:54:25.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.375+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe5ec061ef0 con 0x7fe5ec101350 2024-03-08T23:54:25.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.792+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 20) v1 ==== 137358+0+0 (secure 0 0 0) 0x7fe5e8097fc0 con 0x7fe5ec101350 2024-03-08T23:54:25.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.793+0000 7fe5e27fc640 1 -- 172.21.15.22:0/3954025492 <== mon.2 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe5e80c7950 con 0x7fe5ec101350 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.795+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe5d80a2940 msgr2=0x7fe5d80a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.795+0000 7fe5f3380640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe5d80a2940 0x7fe5d80a4e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc02c7b0 tx=0x7fe5dc005fd0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.795+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec101350 msgr2=0x7fe5ec197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.795+0000 7fe5f3380640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec101350 0x7fe5ec197580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800e9c0 tx=0x7fe5e800ee80 comp rx=0 tx=0).stop 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 shutdown_connections 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fe5d80a2940 0x7fe5d80a4e00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5ec102f00 0x7fe5ec19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ec102540 0x7fe5ec197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 --2- 172.21.15.22:0/3954025492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ec101350 0x7fe5ec197580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 >> 172.21.15.22:0/3954025492 conn(0x7fe5ec0fc8c0 msgr2=0x7fe5ec105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 shutdown_connections 2024-03-08T23:54:25.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.796+0000 7fe5f3380640 1 -- 172.21.15.22:0/3954025492 wait complete. 2024-03-08T23:54:25.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 -- 172.21.15.22:0/523344687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8101400 msgr2=0x7f58c8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 --2- 172.21.15.22:0/523344687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8101400 0x7f58c8101880 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009900 tx=0x7f58bc02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 -- 172.21.15.22:0/523344687 shutdown_connections 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 --2- 172.21.15.22:0/523344687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c8101dc0 0x7f58c8108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 --2- 172.21.15.22:0/523344687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8101400 0x7f58c8101880 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 --2- 172.21.15.22:0/523344687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58c8100210 0x7f58c8100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 -- 172.21.15.22:0/523344687 >> 172.21.15.22:0/523344687 conn(0x7f58c80fb780 msgr2=0x7f58c80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 -- 172.21.15.22:0/523344687 shutdown_connections 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.918+0000 7f58cea44640 1 -- 172.21.15.22:0/523344687 wait complete. 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 Processor -- start 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 -- start start 2024-03-08T23:54:25.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8100210 0x7f58c81041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c8101400 0x7f58c81027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58c8101dc0 0x7f58c8102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c810bea0 con 0x7f58c8100210 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c810bd20 con 0x7f58c8101dc0 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58cea44640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c810c020 con 0x7f58c8101400 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58c7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8100210 0x7f58c81041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58c7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8100210 0x7f58c81041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52234/0 (socket says 172.21.15.22:52234) 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58c7fff640 1 -- 172.21.15.22:0/84279980 learned_addr learned my addr 172.21.15.22:0/84279980 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58ccfe1640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58c8101dc0 0x7f58c8102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.919+0000 7f58c77fe640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c8101400 0x7f58c81027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c7fff640 1 -- 172.21.15.22:0/84279980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c8101400 msgr2=0x7f58c81027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c7fff640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c8101400 0x7f58c81027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c7fff640 1 -- 172.21.15.22:0/84279980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58c8101dc0 msgr2=0x7f58c8102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:25.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c7fff640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58c8101dc0 0x7f58c8102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c7fff640 1 -- 172.21.15.22:0/84279980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c81035b0 con 0x7f58c8100210 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58ccfe1640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58c8101dc0 0x7f58c8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c7fff640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8100210 0x7f58c81041a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f58b800d950 tx=0x7f58b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c57fa640 1 -- 172.21.15.22:0/84279980 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8017020 con 0x7f58c8100210 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c81a16d0 con 0x7f58c8100210 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c57fa640 1 -- 172.21.15.22:0/84279980 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58b8009c30 con 0x7f58c8100210 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58c57fa640 1 -- 172.21.15.22:0/84279980 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8005220 con 0x7f58c8100210 2024-03-08T23:54:25.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.920+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c81a1c40 con 0x7f58c8100210 2024-03-08T23:54:25.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.921+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c81a1f60 con 0x7f58c8100210 2024-03-08T23:54:25.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.923+0000 7f58c57fa640 1 -- 172.21.15.22:0/84279980 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 20) v1 ==== 137358+0+0 (secure 0 0 0) 0x7f58b801a020 con 0x7f58c8100210 2024-03-08T23:54:25.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.923+0000 7f58c57fa640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f58b00a2990 0x7f58b00a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:25.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.924+0000 7f58c77fe640 1 -- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f58b00a2990 msgr2=0x7f58b00a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:25.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.924+0000 7f58c57fa640 1 -- 172.21.15.22:0/84279980 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f58b80cf300 con 0x7f58c8100210 2024-03-08T23:54:25.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.924+0000 7f58c77fe640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f58b00a2990 0x7f58b00a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:54:25.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:25.926+0000 7f58c57fa640 1 -- 172.21.15.22:0/84279980 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f58b8014020 con 0x7f58c8100210 2024-03-08T23:54:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.113+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f58c81a2600 con 0x7f58c8100210 2024-03-08T23:54:26.111 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: ignoring --setuser ceph since I am not root 2024-03-08T23:54:26.111 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: ignoring --setgroup ceph since I am not root 2024-03-08T23:54:26.111 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mgr[37529]: -- 172.21.15.62:0/1454207617 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557753e7e5a0 con 0x557753e52c00 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1233204333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3954025492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:26.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:26.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:25 smithi062 ceph-mon[36129]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:26.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.116+0000 7f58c57fa640 1 -- 172.21.15.22:0/84279980 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) v1 ==== 138+0+0 (secure 0 0 0) 0x7f58b8009de0 con 0x7f58c8100210 2024-03-08T23:54:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f58b00a2990 msgr2=0x7f58b00a4e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:54:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f58b00a2990 0x7f58b00a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8100210 msgr2=0x7f58c81041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8100210 0x7f58c81041a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f58b800d950 tx=0x7f58b800de10 comp rx=0 tx=0).stop 2024-03-08T23:54:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 shutdown_connections 2024-03-08T23:54:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58c8101dc0 0x7f58c8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c8101400 0x7f58c81027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f58b00a2990 0x7f58b00a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 --2- 172.21.15.22:0/84279980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58c8100210 0x7f58c81041a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 >> 172.21.15.22:0/84279980 conn(0x7f58c80fb780 msgr2=0x7f58c80fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 shutdown_connections 2024-03-08T23:54:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.119+0000 7f58cea44640 1 -- 172.21.15.22:0/84279980 wait complete. 2024-03-08T23:54:26.125 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: ignoring --setuser ceph since I am not root 2024-03-08T23:54:26.125 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: ignoring --setgroup ceph since I am not root 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1233204333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3954025492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:26.126 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:25 smithi022 ceph-mon[31894]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:26.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-08T23:54:26.131 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: ignoring --setuser ceph since I am not root 2024-03-08T23:54:26.131 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: ignoring --setgroup ceph since I am not root 2024-03-08T23:54:26.131 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mgr[38021]: -- 172.21.15.192:0/302628273 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559db8c865a0 con 0x559db8c5a800 2024-03-08T23:54:26.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-08T23:54:26.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1233204333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T23:54:26.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3954025492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:26.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:25 smithi192 ceph-mon[35995]: from='mgr.14152 172.21.15.22:0/2912677454' entity='mgr.x' 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.236+0000 7f400fac0640 1 -- 172.21.15.22:0/1001504544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101dd0 msgr2=0x7f4008108660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.236+0000 7f400fac0640 1 --2- 172.21.15.22:0/1001504544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101dd0 0x7f4008108660 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4000009ca0 tx=0x7f400002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.236+0000 7f400fac0640 1 -- 172.21.15.22:0/1001504544 shutdown_connections 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.236+0000 7f400fac0640 1 --2- 172.21.15.22:0/1001504544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101dd0 0x7f4008108660 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.236+0000 7f400fac0640 1 --2- 172.21.15.22:0/1001504544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008101410 0x7f4008101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.236+0000 7f400fac0640 1 --2- 172.21.15.22:0/1001504544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4008100220 0x7f4008100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.236+0000 7f400fac0640 1 -- 172.21.15.22:0/1001504544 >> 172.21.15.22:0/1001504544 conn(0x7f40080fb7b0 msgr2=0x7f40080fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 -- 172.21.15.22:0/1001504544 shutdown_connections 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 -- 172.21.15.22:0/1001504544 wait complete. 2024-03-08T23:54:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 Processor -- start 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 -- start start 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4008100220 0x7f40081974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101410 0x7f4008197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008101dd0 0x7f400819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400810beb0 con 0x7f4008101410 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400810bd30 con 0x7f4008101dd0 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400fac0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400810c030 con 0x7f4008100220 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400d05b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101410 0x7f4008197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400d05b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101410 0x7f4008197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52262/0 (socket says 172.21.15.22:52262) 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400d05b640 1 -- 172.21.15.22:0/772390990 learned_addr learned my addr 172.21.15.22:0/772390990 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.237+0000 7f400d85c640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4008100220 0x7f40081974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400e05d640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008101dd0 0x7f400819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400d05b640 1 -- 172.21.15.22:0/772390990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4008100220 msgr2=0x7f40081974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400d05b640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4008100220 0x7f40081974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400d05b640 1 -- 172.21.15.22:0/772390990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008101dd0 msgr2=0x7f400819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400d05b640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008101dd0 0x7f400819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400d05b640 1 -- 172.21.15.22:0/772390990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400819f0b0 con 0x7f4008101410 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400d05b640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101410 0x7f4008197a20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800ea60 tx=0x7f3ff800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f3ffeffd640 1 -- 172.21.15.22:0/772390990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800cde0 con 0x7f4008101410 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f3ffeffd640 1 -- 172.21.15.22:0/772390990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ff80035f0 con 0x7f4008101410 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400819f3a0 con 0x7f4008101410 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f3ffeffd640 1 -- 172.21.15.22:0/772390990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8010670 con 0x7f4008101410 2024-03-08T23:54:26.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.238+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400819f8f0 con 0x7f4008101410 2024-03-08T23:54:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.239+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400819fbb0 con 0x7f4008101410 2024-03-08T23:54:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.240+0000 7f3ffeffd640 1 -- 172.21.15.22:0/772390990 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 20) v1 ==== 137358+0+0 (secure 0 0 0) 0x7f3ff80037a0 con 0x7f4008101410 2024-03-08T23:54:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.241+0000 7f3ffeffd640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3ff40a2990 0x7f3ff40a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.241+0000 7f400d85c640 1 -- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3ff40a2990 msgr2=0x7f3ff40a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.241+0000 7f3ffeffd640 1 -- 172.21.15.22:0/772390990 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3ff80cf330 con 0x7f4008101410 2024-03-08T23:54:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.241+0000 7f400d85c640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3ff40a2990 0x7f3ff40a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:54:26.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.246+0000 7f3ffeffd640 1 -- 172.21.15.22:0/772390990 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3ff809be80 con 0x7f4008101410 2024-03-08T23:54:26.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.429+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f400819fd80 con 0x7f4008101410 2024-03-08T23:54:26.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.433+0000 7f3ffeffd640 1 -- 172.21.15.22:0/772390990 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3ff809b780 con 0x7f4008101410 2024-03-08T23:54:26.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3ff40a2990 msgr2=0x7f3ff40a4e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:54:26.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3ff40a2990 0x7f3ff40a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101410 msgr2=0x7f4008197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101410 0x7f4008197a20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800ea60 tx=0x7f3ff800ef20 comp rx=0 tx=0).stop 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 shutdown_connections 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008101dd0 0x7f400819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7f3ff40a2990 0x7f3ff40a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4008101410 0x7f4008197a20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 --2- 172.21.15.22:0/772390990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4008100220 0x7f40081974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 >> 172.21.15.22:0/772390990 conn(0x7f40080fb7b0 msgr2=0x7f40080fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 shutdown_connections 2024-03-08T23:54:26.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.435+0000 7f400fac0640 1 -- 172.21.15.22:0/772390990 wait complete. 2024-03-08T23:54:26.441 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:26 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:26.377+0000 7f12220b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:54:26.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- 172.21.15.22:0/385774579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc368101d90 msgr2=0x7fc368104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 --2- 172.21.15.22:0/385774579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc368101d90 0x7fc368104180 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009900 tx=0x7fc35c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- 172.21.15.22:0/385774579 shutdown_connections 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 --2- 172.21.15.22:0/385774579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368106ff0 0x7fc368109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 --2- 172.21.15.22:0/385774579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3681046c0 0x7fc368106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 --2- 172.21.15.22:0/385774579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc368101d90 0x7fc368104180 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- 172.21.15.22:0/385774579 >> 172.21.15.22:0/385774579 conn(0x7fc3680fb780 msgr2=0x7fc3680fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- 172.21.15.22:0/385774579 shutdown_connections 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- 172.21.15.22:0/385774579 wait complete. 2024-03-08T23:54:26.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 Processor -- start 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- start start 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368101d90 0x7fc3681975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3681046c0 0x7fc368197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368106ff0 0x7fc36819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36810cfa0 con 0x7fc3681046c0 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36810ce20 con 0x7fc368101d90 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc36f030640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36810d120 con 0x7fc368106ff0 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.553+0000 7fc367fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3681046c0 0x7fc368197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3681046c0 0x7fc368197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52288/0 (socket says 172.21.15.22:52288) 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 -- 172.21.15.22:0/1082106440 learned_addr learned my addr 172.21.15.22:0/1082106440 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:26.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc36d5cd640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368106ff0 0x7fc36819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368101d90 0x7fc3681975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368106ff0 msgr2=0x7fc36819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368106ff0 0x7fc36819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368101d90 msgr2=0x7fc3681975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368101d90 0x7fc3681975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 -- 172.21.15.22:0/1082106440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36819f1d0 con 0x7fc3681046c0 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc367fff640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3681046c0 0x7fc368197af0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc35400e9c0 tx=0x7fc35400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400cd80 con 0x7fc3681046c0 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36819f4c0 con 0x7fc3681046c0 2024-03-08T23:54:26.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc354005500 con 0x7fc3681046c0 2024-03-08T23:54:26.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354010690 con 0x7fc3681046c0 2024-03-08T23:54:26.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.554+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36819fa60 con 0x7fc3681046c0 2024-03-08T23:54:26.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.555+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3680fd270 con 0x7fc3681046c0 2024-03-08T23:54:26.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.556+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 20) v1 ==== 137358+0+0 (secure 0 0 0) 0x7fc3540046c0 con 0x7fc3681046c0 2024-03-08T23:54:26.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.557+0000 7fc365ffb640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:26.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.557+0000 7fc36cdcc640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 msgr2=0x7fc3580a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:26.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.557+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc3540cf340 con 0x7fc3681046c0 2024-03-08T23:54:26.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.557+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T23:54:26.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.561+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc35409bef0 con 0x7fc3681046c0 2024-03-08T23:54:26.631 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:26.352+0000 7f389562f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:54:26.631 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:26.447+0000 7f389562f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:54:26.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:26 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:26.477+0000 7f12220b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:54:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.758+0000 7fc36cdcc640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 msgr2=0x7fc3580a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:26.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.758+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T23:54:26.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:26.767+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 --> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc368061730 con 0x7fc3580a2990 2024-03-08T23:54:26.796 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:26.369+0000 7f67e9527200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T23:54:26.796 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:26.467+0000 7f67e9527200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T23:54:27.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:26 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-08T23:54:27.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:26 smithi192 ceph-mon[35995]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-08T23:54:27.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:26 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/84279980' entity='client.admin' 2024-03-08T23:54:27.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:26 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/772390990' entity='client.admin' 2024-03-08T23:54:27.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:26 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-08T23:54:27.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:26 smithi062 ceph-mon[36129]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-08T23:54:27.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:26 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/84279980' entity='client.admin' 2024-03-08T23:54:27.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:26 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/772390990' entity='client.admin' 2024-03-08T23:54:27.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:26 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-08T23:54:27.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:26 smithi022 ceph-mon[31894]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-08T23:54:27.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:26 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/84279980' entity='client.admin' 2024-03-08T23:54:27.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:26 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/772390990' entity='client.admin' 2024-03-08T23:54:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:27.159+0000 7fc36cdcc640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 msgr2=0x7fc3580a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:27.159+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-08T23:54:27.375 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:27.131+0000 7f67e9527200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:54:27.375 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:27.230+0000 7f67e9527200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:54:27.375 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:27.379+0000 7f67e9527200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:54:27.381 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.086+0000 7f389562f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:54:27.381 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.176+0000 7f389562f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:54:27.381 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.318+0000 7f389562f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:54:27.473 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:27.140+0000 7f12220b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T23:54:27.474 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:27.236+0000 7f12220b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T23:54:27.474 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:27.379+0000 7f12220b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T23:54:27.474 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:27.477+0000 7f12220b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:54:27.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:27.563+0000 7f12220b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:54:27.756 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.414+0000 7f389562f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:54:27.756 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.495+0000 7f389562f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:54:27.847 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:27.482+0000 7f67e9527200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T23:54:27.848 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:27.574+0000 7f67e9527200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T23:54:27.848 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:27.851+0000 7f67e9527200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:54:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:27.960+0000 7fc36cdcc640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 msgr2=0x7fc3580a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:27.960+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-08T23:54:28.041 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.759+0000 7f389562f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:54:28.041 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.867+0000 7f389562f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:54:28.041 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:27.951+0000 7f389562f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:54:28.041 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:28.044+0000 7f389562f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:54:28.111 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:27.966+0000 7f67e9527200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:54:28.111 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:28.056+0000 7f67e9527200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:54:28.127 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:27.833+0000 7f12220b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T23:54:28.127 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:27.945+0000 7f12220b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T23:54:28.127 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:28 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:28.034+0000 7f12220b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T23:54:28.127 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:28 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:28.131+0000 7f12220b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:54:28.381 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:28.219+0000 7f389562f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:54:28.485 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:28 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:28.312+0000 7f12220b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:54:28.611 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:28.155+0000 7f67e9527200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T23:54:28.611 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:28.339+0000 7f67e9527200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T23:54:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:29.561+0000 7fc36cdcc640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 msgr2=0x7fc3580a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:29.561+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-08T23:54:29.611 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:29.351+0000 7f67e9527200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:54:29.611 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:29.486+0000 7f67e9527200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:54:29.631 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:29.169+0000 7f389562f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:54:29.631 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:29.298+0000 7f389562f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:54:29.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:29.303+0000 7f12220b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T23:54:29.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:29.438+0000 7f12220b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T23:54:29.912 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:29.635+0000 7f389562f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:54:29.913 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:29.724+0000 7f389562f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:54:29.913 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:29.916+0000 7f389562f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:54:30.071 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:29.778+0000 7f12220b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:54:30.071 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:29.872+0000 7f12220b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:54:30.111 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:29.831+0000 7f67e9527200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T23:54:30.111 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:29.924+0000 7f67e9527200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T23:54:30.375 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:30.099+0000 7f389562f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:54:30.485 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:30.074+0000 7f12220b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:54:30.485 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:30.267+0000 7f12220b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:54:30.604 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:30.124+0000 7f67e9527200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:54:30.604 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:30.314+0000 7f67e9527200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T23:54:30.631 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:30.379+0000 7f389562f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:54:30.825 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:30.564+0000 7f12220b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:54:30.861 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:30.608+0000 7f67e9527200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T23:54:31.131 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:30.642+0000 7f389562f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:54:31.131 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:30.874+0000 7f389562f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:54:31.235 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:30.827+0000 7f12220b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:54:31.235 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:31 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:31.074+0000 7f12220b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:54:31.361 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:30.877+0000 7f67e9527200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T23:54:31.361 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:31.120+0000 7f67e9527200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T23:54:32.131 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:31.673+0000 7f389562f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:54:32.235 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:31 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:31.914+0000 7f12220b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:54:32.361 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:31.953+0000 7f67e9527200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T23:54:32.552 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:32.242+0000 7f389562f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:54:32.552 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:32.343+0000 7f389562f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:54:32.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:32.631+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 21) v1 ==== 137358+0+0 (secure 0 0 0) 0x7fc354097e60 con 0x7fc3681046c0 2024-03-08T23:54:32.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:32.764+0000 7fc36cdcc640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 msgr2=0x7fc3580a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/2813827512 2024-03-08T23:54:32.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:32.764+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-08T23:54:32.840 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:32.506+0000 7f12220b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:54:32.840 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:32.612+0000 7f12220b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:54:32.840 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-mon[31894]: Standby manager daemon z restarted 2024-03-08T23:54:32.840 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-mon[31894]: Standby manager daemon z started 2024-03-08T23:54:32.840 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T23:54:32.841 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:54:32.841 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T23:54:32.841 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:54:32.861 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:32.549+0000 7f67e9527200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T23:54:32.861 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:32.653+0000 7f67e9527200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T23:54:32.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-mon[36129]: Standby manager daemon z restarted 2024-03-08T23:54:32.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-mon[36129]: Standby manager daemon z started 2024-03-08T23:54:32.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T23:54:32.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:54:32.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T23:54:32.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:54:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:32.867+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 22) v1 ==== 136975+0+0 (secure 0 0 0) 0x7fc3540994b0 con 0x7fc3681046c0 2024-03-08T23:54:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:32.867+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 msgr2=0x7fc3580a4e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T23:54:32.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:32.867+0000 7fc365ffb640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2813827512,v1:172.21.15.22:6801/2813827512] conn(0x7fc3580a2990 0x7fc3580a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:32.881 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z[38017]: 2024-03-08T23:54:32.556+0000 7f389562f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:54:32.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-mon[35995]: Standby manager daemon z restarted 2024-03-08T23:54:32.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-mon[35995]: Standby manager daemon z started 2024-03-08T23:54:32.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T23:54:32.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:54:32.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T23:54:32.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:32 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.192:0/4182148955' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:54:33.113 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:32.843+0000 7f12220b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:54:33.178 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 08 23:54:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y[37525]: 2024-03-08T23:54:32.882+0000 7f67e9527200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T23:54:33.485 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1180a52700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.485 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1180a52700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1180a52700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1180a52700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1180a52700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1184b5a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1184b5a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1184b5a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1184b5a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:54:33.116+0000 7f1184b5a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:54:33.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.873+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mon.0 v2:172.21.15.22:3300/0 9 ==== mgrmap(e 23) v1 ==== 137102+0+0 (secure 0 0 0) 0x7fc35406c5c0 con 0x7fc3681046c0 2024-03-08T23:54:33.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.874+0000 7fc365ffb640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc358035bf0 0x7fc358035fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:33.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.874+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 --> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc358041260 con 0x7fc358035bf0 2024-03-08T23:54:33.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.877+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc358035bf0 0x7fc358035fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:33.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.879+0000 7fc36cdcc640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc358035bf0 0x7fc358035fd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc35c02c7b0 tx=0x7fc35c005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:33.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.946+0000 7fc365ffb640 1 -- 172.21.15.22:0/1082106440 <== mgr.15660 v2:172.21.15.22:6800/2739225064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc358041260 con 0x7fc358035bf0 2024-03-08T23:54:33.943 INFO:teuthology.orchestra.run.smithi022.stdout:Schedule set for path / 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc358035bf0 msgr2=0x7fc358035fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc358035bf0 0x7fc358035fd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc35c02c7b0 tx=0x7fc35c005fd0 comp rx=0 tx=0).stop 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3681046c0 msgr2=0x7fc368197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 --2- 172.21.15.22:0/1082106440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3681046c0 0x7fc368197af0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc35400e9c0 tx=0x7fc35400ee80 comp rx=0 tx=0).stop 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36d5cd640 1 -- 172.21.15.22:0/1082106440 reap_dead start 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 shutdown_connections 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 >> 172.21.15.22:0/1082106440 conn(0x7fc3680fb780 msgr2=0x7fc3681026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 shutdown_connections 2024-03-08T23:54:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:33.949+0000 7fc36f030640 1 -- 172.21.15.22:0/1082106440 wait complete. 2024-03-08T23:54:33.959 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: mgrmap e21: x(active, since 9m), standbys: y, z 2024-03-08T23:54:33.959 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: Active manager daemon x restarted 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: Activating manager daemon x 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: osdmap e94: 12 total, 12 up, 12 in 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: mgrmap e22: x(active, starting, since 0.0116554s), standbys: y, z 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T23:54:33.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:54:33.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:54:33.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:54:33.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:54:33.962 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:54:33.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-08T23:54:33.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:54:33.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:54:33.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:54:33.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:54:33.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:54:33.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:54:33.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: Standby manager daemon y restarted 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: Standby manager daemon y started 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: Manager daemon x is now available 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:33.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T23:54:33.966 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:33 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 -- 172.21.15.22:0/1172090301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 msgr2=0x7f60d4104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 --2- 172.21.15.22:0/1172090301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d4104200 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009900 tx=0x7f60c402bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 -- 172.21.15.22:0/1172090301 shutdown_connections 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 --2- 172.21.15.22:0/1172090301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d4107070 0x7f60d4109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 --2- 172.21.15.22:0/1172090301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d4104740 0x7f60d4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 --2- 172.21.15.22:0/1172090301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d4104200 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 -- 172.21.15.22:0/1172090301 >> 172.21.15.22:0/1172090301 conn(0x7f60d40fb7c0 msgr2=0x7f60d40fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 -- 172.21.15.22:0/1172090301 shutdown_connections 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.078+0000 7f60d98a4640 1 -- 172.21.15.22:0/1172090301 wait complete. 2024-03-08T23:54:34.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 Processor -- start 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 -- start start 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d4104740 0x7f60d4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d4107070 0x7f60d419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d410cfc0 con 0x7f60d4101e10 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d410ce40 con 0x7f60d4107070 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d98a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d410d140 con 0x7f60d4104740 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d41975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41780/0 (socket says 172.21.15.22:41780) 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d2ffd640 1 -- 172.21.15.22:0/2040762239 learned_addr learned my addr 172.21.15.22:0/2040762239 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d27fc640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d4104740 0x7f60d4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d37fe640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d4107070 0x7f60d419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.079+0000 7f60d2ffd640 1 -- 172.21.15.22:0/2040762239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d4104740 msgr2=0x7f60d4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d2ffd640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d4104740 0x7f60d4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d2ffd640 1 -- 172.21.15.22:0/2040762239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d4107070 msgr2=0x7f60d419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d2ffd640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d4107070 0x7f60d419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d2ffd640 1 -- 172.21.15.22:0/2040762239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d419f1c0 con 0x7f60d4101e10 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d27fc640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d4104740 0x7f60d4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d2ffd640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d41975a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009230 tx=0x7f60c4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d88a2640 1 -- 172.21.15.22:0/2040762239 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4041070 con 0x7f60d4101e10 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d88a2640 1 -- 172.21.15.22:0/2040762239 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60c40054e0 con 0x7f60d4101e10 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d419f450 con 0x7f60d4101e10 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d88a2640 1 -- 172.21.15.22:0/2040762239 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4034440 con 0x7f60d4101e10 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.080+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d419f940 con 0x7f60d4101e10 2024-03-08T23:54:34.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.081+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d40fd2b0 con 0x7f60d4101e10 2024-03-08T23:54:34.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.082+0000 7f60d88a2640 1 -- 172.21.15.22:0/2040762239 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 23) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f60c4044070 con 0x7f60d4101e10 2024-03-08T23:54:34.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.082+0000 7f60d88a2640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60bc0a26b0 0x7f60bc0a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.083+0000 7f60d27fc640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60bc0a26b0 0x7f60bc0a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.083+0000 7f60d88a2640 1 -- 172.21.15.22:0/2040762239 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f60c40f08e0 con 0x7f60d4101e10 2024-03-08T23:54:34.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.083+0000 7f60d27fc640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60bc0a26b0 0x7f60bc0a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005d00 tx=0x7f60c8005c70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:34.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.087+0000 7f60d88a2640 1 -- 172.21.15.22:0/2040762239 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f60c40bd7b0 con 0x7f60d4101e10 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: mgrmap e21: x(active, since 9m), standbys: y, z 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: Active manager daemon x restarted 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: Activating manager daemon x 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: osdmap e94: 12 total, 12 up, 12 in 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: mgrmap e22: x(active, starting, since 0.0116554s), standbys: y, z 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:54:34.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T23:54:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:54:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: Standby manager daemon y restarted 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: Standby manager daemon y started 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: Manager daemon x is now available 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:34.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T23:54:34.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:33 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T23:54:34.128 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: mgrmap e21: x(active, since 9m), standbys: y, z 2024-03-08T23:54:34.128 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: Active manager daemon x restarted 2024-03-08T23:54:34.128 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: Activating manager daemon x 2024-03-08T23:54:34.128 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: osdmap e94: 12 total, 12 up, 12 in 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: mgrmap e22: x(active, starting, since 0.0116554s), standbys: y, z 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:54:34.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T23:54:34.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: Standby manager daemon y restarted 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: Standby manager daemon y started 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.? 172.21.15.62:0/1439024365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: Manager daemon x is now available 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:34.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T23:54:34.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:33 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T23:54:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.293+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 --> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f60d4061890 con 0x7f60bc0a26b0 2024-03-08T23:54:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.354+0000 7f60d88a2640 1 -- 172.21.15.22:0/2040762239 <== mgr.15660 v2:172.21.15.22:6800/2739225064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f60d4061890 con 0x7f60bc0a26b0 2024-03-08T23:54:34.350 INFO:teuthology.orchestra.run.smithi022.stdout:Retention added to path / 2024-03-08T23:54:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60bc0a26b0 msgr2=0x7f60bc0a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60bc0a26b0 0x7f60bc0a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005d00 tx=0x7f60c8005c70 comp rx=0 tx=0).stop 2024-03-08T23:54:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 msgr2=0x7f60d41975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d41975a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009230 tx=0x7f60c4005dd0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 shutdown_connections 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60bc0a26b0 0x7f60bc0a4b70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d4107070 0x7f60d419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d4104740 0x7f60d4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 --2- 172.21.15.22:0/2040762239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60d4101e10 0x7f60d41975a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 >> 172.21.15.22:0/2040762239 conn(0x7f60d40fb7c0 msgr2=0x7f60d4102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 shutdown_connections 2024-03-08T23:54:34.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.356+0000 7f60d98a4640 1 -- 172.21.15.22:0/2040762239 wait complete. 2024-03-08T23:54:34.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 -- 172.21.15.22:0/1609694240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 msgr2=0x7ff5e41041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/1609694240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e41041d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009900 tx=0x7ff5e002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 -- 172.21.15.22:0/1609694240 shutdown_connections 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/1609694240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4107040 0x7ff5e4109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/1609694240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e4104710 0x7ff5e4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/1609694240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e41041d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 -- 172.21.15.22:0/1609694240 >> 172.21.15.22:0/1609694240 conn(0x7ff5e40fb7d0 msgr2=0x7ff5e40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 -- 172.21.15.22:0/1609694240 shutdown_connections 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.478+0000 7ff5eb61f640 1 -- 172.21.15.22:0/1609694240 wait complete. 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 Processor -- start 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 -- start start 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e4197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4104710 0x7ff5e4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e4107040 0x7ff5e419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e410cff0 con 0x7ff5e4101de0 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e410ce70 con 0x7ff5e4107040 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5eb61f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e410d170 con 0x7ff5e4104710 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e4197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e4197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41806/0 (socket says 172.21.15.22:41806) 2024-03-08T23:54:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 -- 172.21.15.22:0/3353192408 learned_addr learned my addr 172.21.15.22:0/3353192408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e9bbc640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e4107040 0x7ff5e419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e8bba640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4104710 0x7ff5e4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 -- 172.21.15.22:0/3353192408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4104710 msgr2=0x7ff5e4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4104710 0x7ff5e4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 -- 172.21.15.22:0/3353192408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e4107040 msgr2=0x7ff5e419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e4107040 0x7ff5e419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.479+0000 7ff5e93bb640 1 -- 172.21.15.22:0/3353192408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e419f180 con 0x7ff5e4101de0 2024-03-08T23:54:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.480+0000 7ff5e93bb640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e4197560 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009230 tx=0x7ff5e0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.480+0000 7ff5da7fc640 1 -- 172.21.15.22:0/3353192408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0041070 con 0x7ff5e4101de0 2024-03-08T23:54:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.480+0000 7ff5da7fc640 1 -- 172.21.15.22:0/3353192408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5e00054e0 con 0x7ff5e4101de0 2024-03-08T23:54:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.480+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e419f410 con 0x7ff5e4101de0 2024-03-08T23:54:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.480+0000 7ff5da7fc640 1 -- 172.21.15.22:0/3353192408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0034440 con 0x7ff5e4101de0 2024-03-08T23:54:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.480+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e419f950 con 0x7ff5e4101de0 2024-03-08T23:54:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.481+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e40fd2c0 con 0x7ff5e4101de0 2024-03-08T23:54:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.482+0000 7ff5da7fc640 1 -- 172.21.15.22:0/3353192408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 23) v1 ==== 137102+0+0 (secure 0 0 0) 0x7ff5e0044070 con 0x7ff5e4101de0 2024-03-08T23:54:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.483+0000 7ff5da7fc640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5d00a2730 0x7ff5d00a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.483+0000 7ff5da7fc640 1 -- 172.21.15.22:0/3353192408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff5e00f07b0 con 0x7ff5e4101de0 2024-03-08T23:54:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.483+0000 7ff5e8bba640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5d00a2730 0x7ff5d00a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.483+0000 7ff5e8bba640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5d00a2730 0x7ff5d00a4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005d00 tx=0x7ff5d4005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:34.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.487+0000 7ff5da7fc640 1 -- 172.21.15.22:0/3353192408 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff5e00bd680 con 0x7ff5e4101de0 2024-03-08T23:54:34.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.692+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 --> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5e4061730 con 0x7ff5d00a2730 2024-03-08T23:54:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.700+0000 7ff5da7fc640 1 -- 172.21.15.22:0/3353192408 <== mgr.15660 v2:172.21.15.22:6800/2739225064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7ff5e4061730 con 0x7ff5d00a2730 2024-03-08T23:54:34.696 INFO:teuthology.orchestra.run.smithi022.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-08T00:00:00", "created": "2024-03-08T23:54:33", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-08T23:54:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5d00a2730 msgr2=0x7ff5d00a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5d00a2730 0x7ff5d00a4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005d00 tx=0x7ff5d4005c50 comp rx=0 tx=0).stop 2024-03-08T23:54:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 msgr2=0x7ff5e4197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e4197560 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009230 tx=0x7ff5e0005dd0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 shutdown_connections 2024-03-08T23:54:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5d00a2730 0x7ff5d00a4bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e4107040 0x7ff5e419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4104710 0x7ff5e4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 --2- 172.21.15.22:0/3353192408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e4101de0 0x7ff5e4197560 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 >> 172.21.15.22:0/3353192408 conn(0x7ff5e40fb7d0 msgr2=0x7ff5e4102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 shutdown_connections 2024-03-08T23:54:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.702+0000 7ff5eb61f640 1 -- 172.21.15.22:0/3353192408 wait complete. 2024-03-08T23:54:34.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-08T23:54:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.820+0000 7f84ea3e1640 1 -- 172.21.15.22:0/2171815600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4107090 msgr2=0x7f84e4109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.820+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/2171815600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4107090 0x7f84e4109480 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009ca0 tx=0x7f84dc02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 -- 172.21.15.22:0/2171815600 shutdown_connections 2024-03-08T23:54:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/2171815600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4107090 0x7f84e4109480 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/2171815600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4104760 0x7f84e4106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/2171815600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e4101e30 0x7f84e4104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 -- 172.21.15.22:0/2171815600 >> 172.21.15.22:0/2171815600 conn(0x7f84e40fb780 msgr2=0x7f84e40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 -- 172.21.15.22:0/2171815600 shutdown_connections 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 -- 172.21.15.22:0/2171815600 wait complete. 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 Processor -- start 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 -- start start 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.821+0000 7f84ea3e1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4101e30 0x7f84e4197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84ea3e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4104760 0x7f84e4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84ea3e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e4107090 0x7f84e419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84ea3e1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e410cfc0 con 0x7f84e4104760 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84ea3e1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e410ce40 con 0x7f84e4107090 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84ea3e1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e410d140 con 0x7f84e4101e30 2024-03-08T23:54:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4104760 0x7f84e4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4104760 0x7f84e4197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41832/0 (socket says 172.21.15.22:41832) 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e37fe640 1 -- 172.21.15.22:0/4073695203 learned_addr learned my addr 172.21.15.22:0/4073695203 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e3fff640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4101e30 0x7f84e4197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e897e640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e4107090 0x7f84e419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e3fff640 1 -- 172.21.15.22:0/4073695203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e4107090 msgr2=0x7f84e419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e3fff640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e4107090 0x7f84e419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e3fff640 1 -- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4104760 msgr2=0x7f84e4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e3fff640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4104760 0x7f84e4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e3fff640 1 -- 172.21.15.22:0/4073695203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e419f1a0 con 0x7f84e4101e30 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e37fe640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4104760 0x7f84e4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.822+0000 7f84e3fff640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4101e30 0x7f84e4197580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84d400e9f0 tx=0x7f84d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.823+0000 7f84e17fa640 1 -- 172.21.15.22:0/4073695203 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d400ccc0 con 0x7f84e4101e30 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.823+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e419f490 con 0x7f84e4101e30 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.823+0000 7f84e17fa640 1 -- 172.21.15.22:0/4073695203 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84d4010910 con 0x7f84e4101e30 2024-03-08T23:54:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.823+0000 7f84e17fa640 1 -- 172.21.15.22:0/4073695203 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4018960 con 0x7f84e4101e30 2024-03-08T23:54:34.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.823+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e419fa30 con 0x7f84e4101e30 2024-03-08T23:54:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.824+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e40fd270 con 0x7f84e4101e30 2024-03-08T23:54:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.825+0000 7f84e17fa640 1 -- 172.21.15.22:0/4073695203 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f84d4010ac0 con 0x7f84e4101e30 2024-03-08T23:54:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.825+0000 7f84e17fa640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84d00a26b0 0x7f84d00a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.826+0000 7f84e17fa640 1 -- 172.21.15.22:0/4073695203 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f84d40ceec0 con 0x7f84e4101e30 2024-03-08T23:54:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.826+0000 7f84e37fe640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84d00a26b0 0x7f84d00a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.826+0000 7f84e37fe640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84d00a26b0 0x7f84d00a4b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84e4198a90 tx=0x7f84cc005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:34.829+0000 7f84e17fa640 1 -- 172.21.15.22:0/4073695203 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f84d409b740 con 0x7f84e4101e30 2024-03-08T23:54:34.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:34 smithi192 ceph-mon[35995]: [08/Mar/2024:23:54:33] ENGINE Bus STARTING 2024-03-08T23:54:34.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:34 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:34.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:34 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:34.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:34 smithi192 ceph-mon[35995]: mgrmap e23: x(active, since 1.01718s), standbys: z, y 2024-03-08T23:54:34.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:34 smithi022 ceph-mon[31894]: [08/Mar/2024:23:54:33] ENGINE Bus STARTING 2024-03-08T23:54:34.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:34 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:34.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:34 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:34.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:34 smithi022 ceph-mon[31894]: mgrmap e23: x(active, since 1.01718s), standbys: z, y 2024-03-08T23:54:35.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.044+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 --> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f84e4061730 con 0x7f84d00a26b0 2024-03-08T23:54:35.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.050+0000 7f84e17fa640 1 -- 172.21.15.22:0/4073695203 <== mgr.15660 v2:172.21.15.22:6800/2739225064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f84e4061730 con 0x7f84d00a26b0 2024-03-08T23:54:35.047 INFO:teuthology.orchestra.run.smithi022.stdout:/ 1m 6m3h 2024-03-08T23:54:35.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84d00a26b0 msgr2=0x7f84d00a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:35.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84d00a26b0 0x7f84d00a4b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84e4198a90 tx=0x7f84cc005f90 comp rx=0 tx=0).stop 2024-03-08T23:54:35.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4101e30 msgr2=0x7f84e4197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4101e30 0x7f84e4197580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84d400e9f0 tx=0x7f84d400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 shutdown_connections 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84d00a26b0 0x7f84d00a4b70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e4107090 0x7f84e419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e4104760 0x7f84e4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 --2- 172.21.15.22:0/4073695203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4101e30 0x7f84e4197580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 >> 172.21.15.22:0/4073695203 conn(0x7f84e40fb780 msgr2=0x7f84e4102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 shutdown_connections 2024-03-08T23:54:35.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:35.053+0000 7f84ea3e1640 1 -- 172.21.15.22:0/4073695203 wait complete. 2024-03-08T23:54:35.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-08T23:54:35.088 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-08T23:54:35.099 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-08T23:54:35.110 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-08T23:54:35.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-08T23:54:35.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:34 smithi062 ceph-mon[36129]: [08/Mar/2024:23:54:33] ENGINE Bus STARTING 2024-03-08T23:54:35.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:34 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:34 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:34 smithi062 ceph-mon[36129]: mgrmap e23: x(active, since 1.01718s), standbys: z, y 2024-03-08T23:54:35.130 INFO:teuthology.run_tasks:Running task workunit... 2024-03-08T23:54:35.145 INFO:tasks.workunit:Pulling workunits from ref 7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:54:35.146 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-08T23:54:35.146 INFO:tasks.workunit:timeout=3h 2024-03-08T23:54:35.146 INFO:tasks.workunit:cleanup=True 2024-03-08T23:54:35.146 DEBUG:teuthology.orchestra.run.smithi022:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:35.162 INFO:teuthology.orchestra.run.smithi022.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:35.162 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-08T23:54:35.162 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-08T23:54:35.162 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T23:54:35.162 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-08T23:54:35.162 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-03-08 23:52:19.766579171 +0000 2024-03-08T23:54:35.162 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-03-08 23:52:19.766579171 +0000 2024-03-08T23:54:35.163 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-03-08 23:54:16.794443665 +0000 2024-03-08T23:54:35.163 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-03-08 23:52:19.766579171 +0000 2024-03-08T23:54:35.163 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-08T23:54:35.163 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-08T23:54:35.236 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7d63482aa89628d733e298984a762389b625c761 2024-03-08T23:54:35.292 INFO:tasks.workunit.client.0.smithi022.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-08T23:54:35.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:35.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:35.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: [08/Mar/2024:23:54:33] ENGINE Serving on https://172.21.15.22:7150 2024-03-08T23:54:35.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: [08/Mar/2024:23:54:34] ENGINE Serving on http://172.21.15.22:8765 2024-03-08T23:54:35.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: [08/Mar/2024:23:54:34] ENGINE Bus STARTED 2024-03-08T23:54:35.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:35.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:35.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:35 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:35.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:35.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: [08/Mar/2024:23:54:33] ENGINE Serving on https://172.21.15.22:7150 2024-03-08T23:54:35.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: [08/Mar/2024:23:54:34] ENGINE Serving on http://172.21.15.22:8765 2024-03-08T23:54:35.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: [08/Mar/2024:23:54:34] ENGINE Bus STARTED 2024-03-08T23:54:35.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:35.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:35.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:35.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:35 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:36.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:36.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: [08/Mar/2024:23:54:33] ENGINE Serving on https://172.21.15.22:7150 2024-03-08T23:54:36.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: [08/Mar/2024:23:54:34] ENGINE Serving on http://172.21.15.22:8765 2024-03-08T23:54:36.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: [08/Mar/2024:23:54:34] ENGINE Bus STARTED 2024-03-08T23:54:36.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:36.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:36.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:35 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:36.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='client.25297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:36.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: mgrmap e24: x(active, since 3s), standbys: z, y 2024-03-08T23:54:36.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:36.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='client.25297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:36.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: mgrmap e24: x(active, since 3s), standbys: z, y 2024-03-08T23:54:36.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:36.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:37.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='client.25297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: mgrmap e24: x(active, since 3s), standbys: z, y 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:37.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Unable to set osd_memory_target on smithi192 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi062 to 174.1M 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Unable to set osd_memory_target on smithi062 to 182642790: error parsing value: Value '182642790' is below minimum 939524096 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi022 to 177.2M 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Unable to set osd_memory_target on smithi022 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-08T23:54:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-08T23:54:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-08T23:54:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:37 smithi062 ceph-mon[36129]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Unable to set osd_memory_target on smithi192 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi062 to 174.1M 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Unable to set osd_memory_target on smithi062 to 182642790: error parsing value: Value '182642790' is below minimum 939524096 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi022 to 177.2M 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Unable to set osd_memory_target on smithi022 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-08T23:54:38.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:37 smithi192 ceph-mon[35995]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-08T23:54:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-08T23:54:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Unable to set osd_memory_target on smithi192 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-08T23:54:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi062 to 174.1M 2024-03-08T23:54:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Unable to set osd_memory_target on smithi062 to 182642790: error parsing value: Value '182642790' is below minimum 939524096 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi022 to 177.2M 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Unable to set osd_memory_target on smithi022 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Updating smithi022:/etc/ceph/ceph.conf 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-08T23:54:38.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:37 smithi022 ceph-mon[31894]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:39.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:39.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:39.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:39.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: Updating smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:54:39.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:54:40.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:39 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:54:40.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:39 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:54:40.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:39 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:54:41.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:40 smithi062 ceph-mon[36129]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-03-08T23:54:41.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:40 smithi192 ceph-mon[35995]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-03-08T23:54:41.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:40 smithi022 ceph-mon[31894]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-03-08T23:54:43.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:42 smithi062 ceph-mon[36129]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-08T23:54:43.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:42 smithi192 ceph-mon[35995]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-08T23:54:43.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:42 smithi022 ceph-mon[31894]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-08T23:54:45.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:44 smithi062 ceph-mon[36129]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-08T23:54:45.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:44 smithi192 ceph-mon[35995]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-08T23:54:45.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:44 smithi022 ceph-mon[31894]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-08T23:54:47.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:46 smithi062 ceph-mon[36129]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-08T23:54:47.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:46 smithi192 ceph-mon[35995]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-08T23:54:47.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:46 smithi022 ceph-mon[31894]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-08T23:54:48.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:47 smithi022 ceph-mon[31894]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-08T23:54:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 -- 172.21.15.22:0/1906581189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54100430 msgr2=0x7f8e54100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 --2- 172.21.15.22:0/1906581189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54100430 0x7f8e54100810 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009900 tx=0x7f8e4402bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 -- 172.21.15.22:0/1906581189 shutdown_connections 2024-03-08T23:54:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 --2- 172.21.15.22:0/1906581189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e54105500 0x7f8e5410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 --2- 172.21.15.22:0/1906581189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e54100de0 0x7f8e54104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 --2- 172.21.15.22:0/1906581189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54100430 0x7f8e54100810 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 -- 172.21.15.22:0/1906581189 >> 172.21.15.22:0/1906581189 conn(0x7f8e54077320 msgr2=0x7f8e54077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 -- 172.21.15.22:0/1906581189 shutdown_connections 2024-03-08T23:54:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.279+0000 7f8e5a685640 1 -- 172.21.15.22:0/1906581189 wait complete. 2024-03-08T23:54:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 Processor -- start 2024-03-08T23:54:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 -- start start 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e54100430 0x7f8e5419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54105500 0x7f8e5419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e54197910 0x7f8e54197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54110590 con 0x7f8e54105500 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54110410 con 0x7f8e54100430 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e5a685640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54110710 con 0x7f8e54197910 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.280+0000 7f8e537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54105500 0x7f8e5419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54105500 0x7f8e5419dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49786/0 (socket says 172.21.15.22:49786) 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 -- 172.21.15.22:0/153980181 learned_addr learned my addr 172.21.15.22:0/153980181 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e53fff640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e54100430 0x7f8e5419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e58c22640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e54197910 0x7f8e54197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 -- 172.21.15.22:0/153980181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e54197910 msgr2=0x7f8e54197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e54197910 0x7f8e54197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 -- 172.21.15.22:0/153980181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e54100430 msgr2=0x7f8e5419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e54100430 0x7f8e5419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 -- 172.21.15.22:0/153980181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e54198430 con 0x7f8e54105500 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e537fe640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54105500 0x7f8e5419dc50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d770 tx=0x7f8e4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e517fa640 1 -- 172.21.15.22:0/153980181 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48017070 con 0x7f8e54105500 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e517fa640 1 -- 172.21.15.22:0/153980181 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e48009c30 con 0x7f8e54105500 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e54198720 con 0x7f8e54105500 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e517fa640 1 -- 172.21.15.22:0/153980181 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800f620 con 0x7f8e54105500 2024-03-08T23:54:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.281+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e541a4920 con 0x7f8e54105500 2024-03-08T23:54:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e517fa640 1 -- 172.21.15.22:0/153980181 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f8e4801a070 con 0x7f8e54105500 2024-03-08T23:54:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 Processor -- start 2024-03-08T23:54:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 -- start start 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e517fa640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e3c0a2ab0 0x7f8e3c0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7073a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7074450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa707d550 0x55aaa707f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e517fa640 1 -- 172.21.15.22:0/153980181 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e480cee80 con 0x7f8e54105500 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa6fd85f0 con 0x55aaa7074d40 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa6f81930 con 0x55aaa7073fd0 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa6fd4690 con 0x55aaa707d550 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e58c22640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e3c0a2ab0 0x7f8e3c0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49798/0 (socket says 172.21.15.22:49798) 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 -- 172.21.15.22:0/133662103 learned_addr learned my addr 172.21.15.22:0/133662103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e58c22640 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa707d550 0x55aaa707f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e537fe640 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7074450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 -- 172.21.15.22:0/133662103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa707d550 msgr2=0x55aaa707f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa707d550 0x55aaa707f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 -- 172.21.15.22:0/133662103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 msgr2=0x55aaa7074450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7074450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 -- 172.21.15.22:0/133662103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa6ced210 con 0x55aaa7074d40 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e58c22640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e3c0a2ab0 0x7f8e3c0a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c013a30 tx=0x7f8e4c00ac70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e53fff640 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7073a90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e4402c170 tx=0x7f8e44005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d125804aa591474 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e3a7fc640 1 -- 172.21.15.22:0/133662103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44042070 con 0x55aaa7074d40 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e3a7fc640 1 -- 172.21.15.22:0/133662103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e4402ca80 con 0x55aaa7074d40 2024-03-08T23:54:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.285+0000 7f8e3a7fc640 1 -- 172.21.15.22:0/133662103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44005180 con 0x55aaa7074d40 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 -- 172.21.15.22:0/133662103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 msgr2=0x55aaa7073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7073a90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e4402c170 tx=0x7f8e44005c00 comp rx=0 tx=0).stop 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 -- 172.21.15.22:0/133662103 shutdown_connections 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa707d550 0x55aaa707f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/133662103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7073a90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 -- 172.21.15.22:0/133662103 >> 172.21.15.22:0/133662103 conn(0x55aaa703b5b0 msgr2=0x55aaa706ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 -- 172.21.15.22:0/133662103 shutdown_connections 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 -- 172.21.15.22:0/133662103 wait complete. 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.286+0000 7f8e5d26d740 1 -- start start 2024-03-08T23:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 1 Processor -- start 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa707a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa707ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa707b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa7080220 con 0x55aaa707d550 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa6f94d90 con 0x55aaa7073fd0 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa70803a0 con 0x55aaa7074d40 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e5d26d740 10 client.0 _ll_register_callbacks cb 0x55aaa6ceca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa707b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa707b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49808/0 (socket says 172.21.15.22:49808) 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 -- 172.21.15.22:0/2784161756 learned_addr learned my addr 172.21.15.22:0/2784161756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e53fff640 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa707a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e537fe640 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa707ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 -- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 msgr2=0x55aaa707ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa707ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 -- 172.21.15.22:0/2784161756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 msgr2=0x55aaa707a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa707a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.287+0000 7f8e58c22640 1 -- 172.21.15.22:0/2784161756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaa6db9620 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e58c22640 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa707b070 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c010f90 tx=0x7f8e4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cb4c4cfcd1de8e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e337fe640 1 -- 172.21.15.22:0/2784161756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c024070 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e337fe640 1 -- 172.21.15.22:0/2784161756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e4c004d20 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e4402ca80 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e337fe640 1 -- 172.21.15.22:0/2784161756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e4c02f050 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e5d26d740 10 client.15750 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aaa708c660 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e337fe640 1 -- 172.21.15.22:0/2784161756 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e4402ca80 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e337fe640 1 -- 172.21.15.22:0/2784161756 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e4c005020 con 0x55aaa707d550 2024-03-08T23:54:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e5d26d740 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e5d26d740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.288+0000 7f8e5d26d740 20 client.15750 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.289+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa708dad0 0x55aaa708fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.289+0000 7f8e5d26d740 4 client.15750 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.289+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaa6fe5b30 con 0x55aaa708dad0 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.289+0000 7f8e53fff640 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa708dad0 0x55aaa708fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.289+0000 7f8e337fe640 10 client.15750 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.289+0000 7f8e53fff640 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa708dad0 0x55aaa708fec0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6e4ce9fd3fc392 server_cookie=436f7175854779f in_seq=0 out_seq=0 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e337fe640 1 -- 172.21.15.22:0/2784161756 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aaa6fe5b30 con 0x55aaa708dad0 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e337fe640 10 client.15750 handle_command_reply: tid=0 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 client.15750 shutdown 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 msgr2=0x55aaa707b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa707b070 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c010f90 tx=0x7f8e4c004ab0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 shutdown_connections 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa708dad0 0x55aaa708fec0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa707b070 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa707ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2784161756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa707a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e337fe640 0 client.15750 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 >> 172.21.15.22:0/2784161756 conn(0x55aaa703b5b0 msgr2=0x55aaa7070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 shutdown_connections 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2784161756 wait complete. 2024-03-08T23:54:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.290+0000 7f8e5d26d740 20 client.15750 trim_cache size 0 max 16384 2024-03-08T23:54:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 Processor -- start 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 -- start start 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7084610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa7084b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa7078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541b0930 con 0x55aaa707d550 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541ae4d0 con 0x55aaa7073fd0 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541ac570 con 0x55aaa7074d40 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa7078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa7078ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49822/0 (socket says 172.21.15.22:49822) 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 -- 172.21.15.22:0/2902087192 learned_addr learned my addr 172.21.15.22:0/2902087192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e53fff640 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7084610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e537fe640 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa7084b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 -- 172.21.15.22:0/2902087192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 msgr2=0x55aaa7084b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa7084b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 -- 172.21.15.22:0/2902087192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 msgr2=0x55aaa7084610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7084610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.304+0000 7f8e58c22640 1 -- 172.21.15.22:0/2902087192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa70732a0 con 0x55aaa707d550 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e58c22640 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa7078ec0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c014010 tx=0x7f8e4c010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d434f1c26f07d3e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e39ffb640 1 -- 172.21.15.22:0/2902087192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c024040 con 0x55aaa707d550 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e39ffb640 1 -- 172.21.15.22:0/2902087192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e4c013350 con 0x55aaa707d550 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e39ffb640 1 -- 172.21.15.22:0/2902087192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c020690 con 0x55aaa707d550 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2902087192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 msgr2=0x55aaa7078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa7078ec0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c014010 tx=0x7f8e4c010c40 comp rx=0 tx=0).stop 2024-03-08T23:54:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2902087192 shutdown_connections 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa707d550 0x55aaa7078ec0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7074d40 0x55aaa7084b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/2902087192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa7073fd0 0x55aaa7084610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2902087192 >> 172.21.15.22:0/2902087192 conn(0x55aaa703b5b0 msgr2=0x55aaa706e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2902087192 shutdown_connections 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.305+0000 7f8e5d26d740 1 -- 172.21.15.22:0/2902087192 wait complete. 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 -- start start 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 Processor -- start 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7073fd0 0x55aaa7084a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7080e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa707d550 0x55aaa7081440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541acb70 con 0x55aaa7074d40 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541ade70 con 0x55aaa707d550 2024-03-08T23:54:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541ad1b0 con 0x55aaa7073fd0 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e5d26d740 10 client.0 _ll_register_callbacks cb 0x7f8e44005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7080e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7080e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49828/0 (socket says 172.21.15.22:49828) 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e537fe640 1 -- 172.21.15.22:0/891548076 learned_addr learned my addr 172.21.15.22:0/891548076 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e53fff640 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7073fd0 0x55aaa7084a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.306+0000 7f8e58c22640 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa707d550 0x55aaa7081440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e537fe640 1 -- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7073fd0 msgr2=0x55aaa7084a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e537fe640 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7073fd0 0x55aaa7084a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e537fe640 1 -- 172.21.15.22:0/891548076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa707d550 msgr2=0x55aaa7081440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e537fe640 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa707d550 0x55aaa7081440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e537fe640 1 -- 172.21.15.22:0/891548076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaa7081c10 con 0x55aaa7074d40 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e537fe640 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7080e30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8e480cc200 tx=0x7f8e48009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12afcfcd5e29956 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e337fe640 1 -- 172.21.15.22:0/891548076 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48021040 con 0x55aaa7074d40 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e337fe640 1 -- 172.21.15.22:0/891548076 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e48095b40 con 0x55aaa7074d40 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e4c013350 con 0x55aaa7074d40 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e337fe640 1 -- 172.21.15.22:0/891548076 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e480cc620 con 0x55aaa7074d40 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e5d26d740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aaa70e5120 con 0x55aaa7074d40 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e337fe640 1 -- 172.21.15.22:0/891548076 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e4c013350 con 0x55aaa7074d40 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e337fe640 1 -- 172.21.15.22:0/891548076 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e4809a5b0 con 0x55aaa7074d40 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e5d26d740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e5d26d740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e5d26d740 20 client.15756 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.307+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa70e5800 0x55aaa70e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.308+0000 7f8e5d26d740 4 client.15756 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.308+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aaa6fdfe80 con 0x55aaa70e5800 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.308+0000 7f8e53fff640 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa70e5800 0x55aaa70e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.308+0000 7f8e337fe640 10 client.15756 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.308+0000 7f8e53fff640 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa70e5800 0x55aaa70e7c20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45d5d22c9a94fda server_cookie=f597027bc3eff9e5 in_seq=0 out_seq=0 2024-03-08T23:54:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e337fe640 1 -- 172.21.15.22:0/891548076 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aaa6fdfe80 con 0x55aaa70e5800 2024-03-08T23:54:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e337fe640 10 client.15756 handle_command_reply: tid=0 2024-03-08T23:54:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 client.15756 shutdown 2024-03-08T23:54:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 msgr2=0x55aaa7080e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7080e30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8e480cc200 tx=0x7f8e48009ea0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 shutdown_connections 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aaa70e5800 0x55aaa70e7c20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aaa707d550 0x55aaa7081440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aaa7074d40 0x55aaa7080e30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 --2- 172.21.15.22:0/891548076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aaa7073fd0 0x55aaa7084a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e337fe640 0 client.15756 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 >> 172.21.15.22:0/891548076 conn(0x55aaa703b5b0 msgr2=0x55aaa7078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 shutdown_connections 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 1 -- 172.21.15.22:0/891548076 wait complete. 2024-03-08T23:54:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.322+0000 7f8e5d26d740 20 client.15756 trim_cache size 0 max 16384 2024-03-08T23:54:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:54:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:54:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:54:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e3c0a2ab0 msgr2=0x7f8e3c0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e3c0a2ab0 0x7f8e3c0a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c013a30 tx=0x7f8e4c00ac70 comp rx=0 tx=0).stop 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54105500 msgr2=0x7f8e5419dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54105500 0x7f8e5419dc50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d770 tx=0x7f8e4800dc30 comp rx=0 tx=0).stop 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 shutdown_connections 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e3c0a2ab0 0x7f8e3c0a4f70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e54197910 0x7f8e54197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e54105500 0x7f8e5419dc50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 --2- 172.21.15.22:0/153980181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e54100430 0x7f8e5419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 >> 172.21.15.22:0/153980181 conn(0x7f8e54077320 msgr2=0x7f8e54069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 shutdown_connections 2024-03-08T23:54:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:54:48.323+0000 7f8e5a685640 1 -- 172.21.15.22:0/153980181 wait complete. 2024-03-08T23:54:48.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:47 smithi062 ceph-mon[36129]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-08T23:54:48.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:47 smithi192 ceph-mon[35995]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-08T23:54:50.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:49 smithi022 ceph-mon[31894]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-08T23:54:50.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:49 smithi062 ceph-mon[36129]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-08T23:54:50.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:49 smithi192 ceph-mon[35995]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-08T23:54:52.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:51 smithi022 ceph-mon[31894]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-03-08T23:54:52.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:51 smithi062 ceph-mon[36129]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-03-08T23:54:52.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:51 smithi192 ceph-mon[35995]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-03-08T23:54:54.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:53 smithi022 ceph-mon[31894]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-03-08T23:54:54.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:53 smithi062 ceph-mon[36129]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-03-08T23:54:54.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:53 smithi192 ceph-mon[35995]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-03-08T23:54:55.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:54:55.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:54:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 -- 172.21.15.22:0/1248733879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 msgr2=0x7f00001007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 --2- 172.21.15.22:0/1248733879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f00001007b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efff0009900 tx=0x7efff002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 -- 172.21.15.22:0/1248733879 shutdown_connections 2024-03-08T23:54:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 --2- 172.21.15.22:0/1248733879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00001054a0 0x7f000010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 --2- 172.21.15.22:0/1248733879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0000100d80 0x7f0000104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 --2- 172.21.15.22:0/1248733879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f00001007b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 -- 172.21.15.22:0/1248733879 >> 172.21.15.22:0/1248733879 conn(0x7f00000772f0 msgr2=0x7f0000077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 -- 172.21.15.22:0/1248733879 shutdown_connections 2024-03-08T23:54:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.431+0000 7f0005137640 1 -- 172.21.15.22:0/1248733879 wait complete. 2024-03-08T23:54:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 Processor -- start 2024-03-08T23:54:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 -- start start 2024-03-08T23:54:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f000019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00001054a0 0x7f000019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00001979b0 0x7f0000197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0000110560 con 0x7f00001003d0 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001103e0 con 0x7f00001054a0 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7f0005137640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001106e0 con 0x7f00001979b0 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f000019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f000019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38664/0 (socket says 172.21.15.22:38664) 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 -- 172.21.15.22:0/3004703570 learned_addr learned my addr 172.21.15.22:0/3004703570 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efff7fff640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00001054a0 0x7f000019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7effff59e640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00001979b0 0x7f0000197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 -- 172.21.15.22:0/3004703570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00001979b0 msgr2=0x7f0000197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00001979b0 0x7f0000197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 -- 172.21.15.22:0/3004703570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00001054a0 msgr2=0x7f000019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00001054a0 0x7f000019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 -- 172.21.15.22:0/3004703570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00001984d0 con 0x7f00001003d0 2024-03-08T23:54:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.432+0000 7efffed9d640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f000019d800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.433+0000 7efffcd99640 1 -- 172.21.15.22:0/3004703570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0041070 con 0x7f00001003d0 2024-03-08T23:54:55.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.433+0000 7efffcd99640 1 -- 172.21.15.22:0/3004703570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efff0004450 con 0x7f00001003d0 2024-03-08T23:54:55.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.433+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0000198760 con 0x7f00001003d0 2024-03-08T23:54:55.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.433+0000 7efffcd99640 1 -- 172.21.15.22:0/3004703570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0003e00 con 0x7f00001003d0 2024-03-08T23:54:55.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.433+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00001a4950 con 0x7f00001003d0 2024-03-08T23:54:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.435+0000 7efffcd99640 1 -- 172.21.15.22:0/3004703570 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7efff0044070 con 0x7f00001003d0 2024-03-08T23:54:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.435+0000 7f0007dee740 1 Processor -- start 2024-03-08T23:54:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.435+0000 7f0007dee740 1 -- start start 2024-03-08T23:54:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efffcd99640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7effec0a2a00 0x7effec0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7effff59e640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7effec0a2a00 0x7effec0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efffcd99640 1 -- 172.21.15.22:0/3004703570 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7efff00f0e70 con 0x7f00001003d0 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7f0007dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7f0007dee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7f0007dee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7f0007dee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d5e87e00 con 0x5618d5f25080 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7f0007dee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d5e30b10 con 0x5618d5f2dc60 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7f0007dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d5e83ea0 con 0x5618d5f24760 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38680/0 (socket says 172.21.15.22:38680) 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 -- 172.21.15.22:0/3902194779 learned_addr learned my addr 172.21.15.22:0/3902194779 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7effff59e640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7effec0a2a00 0x7effec0a4ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efff8005fd0 tx=0x7efff8009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efffed9d640 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7effff59e640 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 -- 172.21.15.22:0/3902194779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 msgr2=0x5618d5f24b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f24b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 -- 172.21.15.22:0/3902194779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 msgr2=0x5618d5f30050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f30050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efff7fff640 1 -- 172.21.15.22:0/3902194779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618d5c667d0 con 0x5618d5f25080 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7effff59e640 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.436+0000 7efffed9d640 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f24b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.437+0000 7efff7fff640 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2d720 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7effe800bc30 tx=0x7effe80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1395a5a340a638 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.437+0000 7efff57fa640 1 -- 172.21.15.22:0/3902194779 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe801c480 con 0x5618d5f25080 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.437+0000 7efff57fa640 1 -- 172.21.15.22:0/3902194779 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe801cb00 con 0x5618d5f25080 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.437+0000 7efff57fa640 1 -- 172.21.15.22:0/3902194779 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8011be0 con 0x5618d5f25080 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.437+0000 7f0007dee740 1 -- 172.21.15.22:0/3902194779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 msgr2=0x5618d5f2d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.437+0000 7f0007dee740 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2d720 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7effe800bc30 tx=0x7effe80107a0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- 172.21.15.22:0/3902194779 shutdown_connections 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2d720 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 --2- 172.21.15.22:0/3902194779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f24b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- 172.21.15.22:0/3902194779 >> 172.21.15.22:0/3902194779 conn(0x5618d5f1ee90 msgr2=0x5618d5f1f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- 172.21.15.22:0/3902194779 shutdown_connections 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- 172.21.15.22:0/3902194779 wait complete. 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- start start 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 Processor -- start 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f2d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f2acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d5f30a50 con 0x5618d5f25080 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d5e440a0 con 0x5618d5f24760 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d5f30bd0 con 0x5618d5f2dc60 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7f0007dee740 10 client.0 _ll_register_callbacks cb 0x5618d5e43a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:55.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38688/0 (socket says 172.21.15.22:38688) 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7efff7fff640 1 -- 172.21.15.22:0/2596799293 learned_addr learned my addr 172.21.15.22:0/2596799293 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.438+0000 7efffed9d640 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f2d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7effff59e640 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f2acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7efff7fff640 1 -- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 msgr2=0x5618d5f2acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7efff7fff640 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f2acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7efff7fff640 1 -- 172.21.15.22:0/2596799293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 msgr2=0x5618d5f2d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7efff7fff640 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f2d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7efff7fff640 1 -- 172.21.15.22:0/2596799293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618d5b9c3f0 con 0x5618d5f25080 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7efff7fff640 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2a770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7effe8000c00 tx=0x7effe801cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f29d6980aec1ac server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7effe67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7effe67fc640 1 -- 172.21.15.22:0/2596799293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe802b060 con 0x5618d5f25080 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7effe67fc640 1 -- 172.21.15.22:0/2596799293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7effe801c690 con 0x5618d5f25080 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effe801cb00 con 0x5618d5f25080 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7effe67fc640 1 -- 172.21.15.22:0/2596799293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe8032050 con 0x5618d5f25080 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7f0007dee740 10 client.15768 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.439+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5618d5b5c0f0 con 0x5618d5f25080 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7effe67fc640 1 -- 172.21.15.22:0/2596799293 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe801cb00 con 0x5618d5f25080 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7effe67fc640 1 -- 172.21.15.22:0/2596799293 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7effe8005330 con 0x5618d5f25080 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7f0007dee740 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7f0007dee740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7f0007dee740 20 client.15768 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7f0007dee740 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f3d600 0x5618d5f3f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7f0007dee740 4 client.15768 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618d5e95340 con 0x5618d5f3d600 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7efffed9d640 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f3d600 0x5618d5f3f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7efffed9d640 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f3d600 0x5618d5f3f9f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75aa36b01d34720 server_cookie=cd1af578956b0135 in_seq=0 out_seq=0 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.440+0000 7effe67fc640 10 client.15768 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7effe67fc640 1 -- 172.21.15.22:0/2596799293 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618d5e95340 con 0x5618d5f3d600 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7effe67fc640 10 client.15768 handle_command_reply: tid=0 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 client.15768 shutdown 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 msgr2=0x5618d5f2a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2a770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7effe8000c00 tx=0x7effe801cee0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 shutdown_connections 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f3d600 0x5618d5f3f9f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f2acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2a770 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 --2- 172.21.15.22:0/2596799293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f2d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7effe67fc640 0 client.15768 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 >> 172.21.15.22:0/2596799293 conn(0x5618d5f1ee90 msgr2=0x5618d5f20be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 shutdown_connections 2024-03-08T23:54:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 1 -- 172.21.15.22:0/2596799293 wait complete. 2024-03-08T23:54:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.441+0000 7f0007dee740 20 client.15768 trim_cache size 0 max 16384 2024-03-08T23:54:55.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.456+0000 7f0007dee740 1 Processor -- start 2024-03-08T23:54:55.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.456+0000 7f0007dee740 1 -- start start 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7f0007dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f25b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7f0007dee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7f0007dee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f2d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7f0007dee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001b0a40 con 0x5618d5f25080 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7f0007dee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001ae5e0 con 0x5618d5f2dc60 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7f0007dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001ac680 con 0x5618d5f24760 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38702/0 (socket says 172.21.15.22:38702) 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 -- 172.21.15.22:0/4191649394 learned_addr learned my addr 172.21.15.22:0/4191649394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efffed9d640 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f25b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7effff59e640 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f2d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 -- 172.21.15.22:0/4191649394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 msgr2=0x5618d5f25b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f25b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 -- 172.21.15.22:0/4191649394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 msgr2=0x5618d5f2d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f2d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 -- 172.21.15.22:0/4191649394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618d5f24eb0 con 0x5618d5f25080 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7effff59e640 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f2d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.457+0000 7efff7fff640 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2cea0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7effe80054c0 tx=0x7effe80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2133132abbbc389 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7efff4ff9640 1 -- 172.21.15.22:0/4191649394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe802b040 con 0x5618d5f25080 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7efff4ff9640 1 -- 172.21.15.22:0/4191649394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe800be50 con 0x5618d5f25080 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7efff4ff9640 1 -- 172.21.15.22:0/4191649394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe80043b0 con 0x5618d5f25080 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 -- 172.21.15.22:0/4191649394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 msgr2=0x5618d5f2cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2cea0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7effe80054c0 tx=0x7effe80107a0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 -- 172.21.15.22:0/4191649394 shutdown_connections 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f2dc60 0x5618d5f2d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f2cea0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 --2- 172.21.15.22:0/4191649394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f24760 0x5618d5f25b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 -- 172.21.15.22:0/4191649394 >> 172.21.15.22:0/4191649394 conn(0x5618d5f1ee90 msgr2=0x5618d5f25480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 -- 172.21.15.22:0/4191649394 shutdown_connections 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.458+0000 7f0007dee740 1 -- 172.21.15.22:0/4191649394 wait complete. 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 -- start start 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 Processor -- start 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f41d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f3e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001acc80 con 0x5618d5f25080 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001adf80 con 0x5618d5f24760 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001ad2c0 con 0x5618d5f2dc60 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7f0007dee740 10 client.0 _ll_register_callbacks cb 0x7effe800ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f3e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38714/0 (socket says 172.21.15.22:38714) 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 -- 172.21.15.22:0/1749601981 learned_addr learned my addr 172.21.15.22:0/1749601981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efffed9d640 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f41d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7effff59e640 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f3e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 -- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 msgr2=0x5618d5f3e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f3e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 -- 172.21.15.22:0/1749601981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 msgr2=0x5618d5f41d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f41d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 -- 172.21.15.22:0/1749601981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618d5f3ede0 con 0x5618d5f25080 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7efff7fff640 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f3e0a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7effe8020010 tx=0x7effe8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aabc42f5b170c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.459+0000 7effe67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7effe67fc640 1 -- 172.21.15.22:0/1749601981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe802b040 con 0x5618d5f25080 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7effe67fc640 1 -- 172.21.15.22:0/1749601981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7effe801c690 con 0x5618d5f25080 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effe800be50 con 0x5618d5f25080 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7effe67fc640 1 -- 172.21.15.22:0/1749601981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe800b820 con 0x5618d5f25080 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5618d5f945e0 con 0x5618d5f25080 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7effe67fc640 1 -- 172.21.15.22:0/1749601981 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe800be50 con 0x5618d5f25080 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7effe67fc640 1 -- 172.21.15.22:0/1749601981 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7effe80035f0 con 0x5618d5f25080 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 20 client.15774 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f94bf0 0x5618d5f96fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 4 client.15774 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5618d5e8f690 con 0x5618d5f94bf0 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.460+0000 7efffed9d640 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f94bf0 0x5618d5f96fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7effe67fc640 10 client.15774 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7efffed9d640 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f94bf0 0x5618d5f96fe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861a6f88cab96c0e server_cookie=a07b72fe8f8a2adf in_seq=0 out_seq=0 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7effe67fc640 1 -- 172.21.15.22:0/1749601981 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5618d5e8f690 con 0x5618d5f94bf0 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7effe67fc640 10 client.15774 handle_command_reply: tid=0 2024-03-08T23:54:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 client.15774 shutdown 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 msgr2=0x5618d5f3e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f3e0a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7effe8020010 tx=0x7effe8005c50 comp rx=0 tx=0).stop 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 shutdown_connections 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5618d5f94bf0 0x5618d5f96fe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5f2dc60 0x5618d5f3e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7effe67fc640 0 client.15774 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5618d5f25080 0x5618d5f3e0a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 --2- 172.21.15.22:0/1749601981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5618d5f24760 0x5618d5f41d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 >> 172.21.15.22:0/1749601981 conn(0x5618d5f1ee90 msgr2=0x5618d5f20a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 shutdown_connections 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 1 -- 172.21.15.22:0/1749601981 wait complete. 2024-03-08T23:54:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.461+0000 7f0007dee740 20 client.15774 trim_cache size 0 max 16384 2024-03-08T23:54:55.465 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:54:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:54:55.465 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7effec0a2a00 msgr2=0x7effec0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7effec0a2a00 0x7effec0a4ec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efff8005fd0 tx=0x7efff8009500 comp rx=0 tx=0).stop 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 msgr2=0x7f000019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f000019d800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff002cb30 comp rx=0 tx=0).stop 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 shutdown_connections 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7effec0a2a00 0x7effec0a4ec0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00001979b0 0x7f0000197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00001054a0 0x7f000019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 --2- 172.21.15.22:0/3004703570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00001003d0 0x7f000019d800 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 >> 172.21.15.22:0/3004703570 conn(0x7f00000772f0 msgr2=0x7f00000687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 shutdown_connections 2024-03-08T23:54:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.462+0000 7f0005137640 1 -- 172.21.15.22:0/3004703570 wait complete. 2024-03-08T23:54:55.470 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:54:55.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:54:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.576+0000 7ff09ca29640 1 -- 172.21.15.22:0/2978250600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff098100e10 msgr2=0x7ff098104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.576+0000 7ff09ca29640 1 --2- 172.21.15.22:0/2978250600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff098100e10 0x7ff098104e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.576+0000 7ff09ca29640 1 -- 172.21.15.22:0/2978250600 shutdown_connections 2024-03-08T23:54:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.576+0000 7ff09ca29640 1 --2- 172.21.15.22:0/2978250600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff098105530 0x7ff09810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.577+0000 7ff09ca29640 1 --2- 172.21.15.22:0/2978250600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff098100e10 0x7ff098104e50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.577+0000 7ff09ca29640 1 --2- 172.21.15.22:0/2978250600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff098100460 0x7ff098100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.577+0000 7ff09ca29640 1 -- 172.21.15.22:0/2978250600 >> 172.21.15.22:0/2978250600 conn(0x7ff0980772f0 msgr2=0x7ff098077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.577+0000 7ff09ca29640 1 -- 172.21.15.22:0/2978250600 shutdown_connections 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.577+0000 7ff09ca29640 1 -- 172.21.15.22:0/2978250600 wait complete. 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.577+0000 7ff09ca29640 1 Processor -- start 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.577+0000 7ff09ca29640 1 -- start start 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff098100460 0x7ff09819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff098105530 0x7ff09819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0981969a0 0x7ff098196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0981105c0 con 0x7ff0981969a0 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff098110440 con 0x7ff098105530 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff098110740 con 0x7ff098100460 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0981969a0 0x7ff098196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0981969a0 0x7ff098196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38734/0 (socket says 172.21.15.22:38734) 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 -- 172.21.15.22:0/3461907961 learned_addr learned my addr 172.21.15.22:0/3461907961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09659c640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff098100460 0x7ff09819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff08ffff640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff098105530 0x7ff09819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 -- 172.21.15.22:0/3461907961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff098100460 msgr2=0x7ff09819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff098100460 0x7ff09819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 -- 172.21.15.22:0/3461907961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff098105530 msgr2=0x7ff09819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff098105530 0x7ff09819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 -- 172.21.15.22:0/3461907961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0981974c0 con 0x7ff0981969a0 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff096d9d640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0981969a0 0x7ff098196e50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff09000eaa0 tx=0x7ff09000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff08f7fe640 1 -- 172.21.15.22:0/3461907961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09000cdb0 con 0x7ff0981969a0 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff08f7fe640 1 -- 172.21.15.22:0/3461907961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff090005020 con 0x7ff0981969a0 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0981977b0 con 0x7ff0981969a0 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff08f7fe640 1 -- 172.21.15.22:0/3461907961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0900116b0 con 0x7ff0981969a0 2024-03-08T23:54:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.578+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0981a3960 con 0x7ff0981969a0 2024-03-08T23:54:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.579+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff098101930 con 0x7ff0981969a0 2024-03-08T23:54:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.581+0000 7ff08f7fe640 1 -- 172.21.15.22:0/3461907961 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7ff090005560 con 0x7ff0981969a0 2024-03-08T23:54:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.581+0000 7ff08f7fe640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0840a2a00 0x7ff0840a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.581+0000 7ff09659c640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0840a2a00 0x7ff0840a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.581+0000 7ff08f7fe640 1 -- 172.21.15.22:0/3461907961 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff0900cf8d0 con 0x7ff0981969a0 2024-03-08T23:54:55.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.582+0000 7ff09659c640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0840a2a00 0x7ff0840a4ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0981051f0 tx=0x7ff08800b390 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.585+0000 7ff08f7fe640 1 -- 172.21.15.22:0/3461907961 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff090016030 con 0x7ff0981969a0 2024-03-08T23:54:55.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.806+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0980680e0 con 0x7ff0981969a0 2024-03-08T23:54:55.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.807+0000 7ff08f7fe640 1 -- 172.21.15.22:0/3461907961 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff0980680e0 con 0x7ff0981969a0 2024-03-08T23:54:55.804 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:54:55.804 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:54:17.470857+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1631408985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1631408985},{"type":"v1","addr":"172.21.15.62:6837","nonce":1631408985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:54:55.804 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 25 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0840a2a00 msgr2=0x7ff0840a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0840a2a00 0x7ff0840a4ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0981051f0 tx=0x7ff08800b390 comp rx=0 tx=0).stop 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0981969a0 msgr2=0x7ff098196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0981969a0 0x7ff098196e50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff09000eaa0 tx=0x7ff09000ef60 comp rx=0 tx=0).stop 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 shutdown_connections 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0840a2a00 0x7ff0840a4ec0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0981969a0 0x7ff098196e50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff098105530 0x7ff09819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 --2- 172.21.15.22:0/3461907961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff098100460 0x7ff09819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.809+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 >> 172.21.15.22:0/3461907961 conn(0x7ff0980772f0 msgr2=0x7ff0980fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.810+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 shutdown_connections 2024-03-08T23:54:55.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.810+0000 7ff09ca29640 1 -- 172.21.15.22:0/3461907961 wait complete. 2024-03-08T23:54:55.817 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:54:55.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:54:55.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 -- 172.21.15.22:0/3080447433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c1073b0 msgr2=0x7f3d7c107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3080447433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c1073b0 0x7f3d7c107790 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009930 tx=0x7f3d6c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:55.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 -- 172.21.15.22:0/3080447433 shutdown_connections 2024-03-08T23:54:55.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3080447433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c06a320 0x7f3d7c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3080447433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d7c069960 0x7f3d7c069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3080447433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c1073b0 0x7f3d7c107790 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 -- 172.21.15.22:0/3080447433 >> 172.21.15.22:0/3080447433 conn(0x7f3d7c076f90 msgr2=0x7f3d7c0773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 -- 172.21.15.22:0/3080447433 shutdown_connections 2024-03-08T23:54:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 -- 172.21.15.22:0/3080447433 wait complete. 2024-03-08T23:54:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 Processor -- start 2024-03-08T23:54:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.928+0000 7f3d8220b640 1 -- start start 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d8220b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c069960 0x7f3d7c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d8220b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c06a320 0x7f3d7c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d8220b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d7c1073b0 0x7f3d7c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d8220b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c111640 con 0x7f3d7c069960 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d8220b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1114c0 con 0x7f3d7c1073b0 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d8220b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1117c0 con 0x7f3d7c06a320 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c069960 0x7f3d7c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c069960 0x7f3d7c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38764/0 (socket says 172.21.15.22:38764) 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3828422612 learned_addr learned my addr 172.21.15.22:0/3828422612 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d7c1073b0 0x7f3d7c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3828422612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c06a320 msgr2=0x7f3d7c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c06a320 0x7f3d7c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3828422612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d7c1073b0 msgr2=0x7f3d7c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d7c1073b0 0x7f3d7c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3828422612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c198010 con 0x7f3d7c069960 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c069960 0x7f3d7c19d6d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009230 tx=0x7f3d6c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.929+0000 7f3d78ff9640 1 -- 172.21.15.22:0/3828422612 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c041070 con 0x7f3d7c069960 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.930+0000 7f3d78ff9640 1 -- 172.21.15.22:0/3828422612 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d6c004450 con 0x7f3d7c069960 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.930+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c1982a0 con 0x7f3d7c069960 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.930+0000 7f3d78ff9640 1 -- 172.21.15.22:0/3828422612 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c034440 con 0x7f3d7c069960 2024-03-08T23:54:55.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.930+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c1a4520 con 0x7f3d7c069960 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d78ff9640 1 -- 172.21.15.22:0/3828422612 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f3d6c044070 con 0x7f3d7c069960 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 Processor -- start 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 -- start start 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dba6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbaf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbb1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0db6d360 con 0x55ba0dba61a0 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0db09ec0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d84deb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0dab2d30 con 0x55ba0dbaf6a0 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d78ff9640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3d640a2a00 0x7f3d640a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dba6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dba6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38768/0 (socket says 172.21.15.22:38768) 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d7bfff640 1 -- 172.21.15.22:0/827514287 learned_addr learned my addr 172.21.15.22:0/827514287 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3d640a2a00 0x7f3d640a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d78ff9640 1 -- 172.21.15.22:0/3828422612 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3d6c0f0dc0 con 0x7f3d7c069960 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.932+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbaf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7affd640 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbb1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7bfff640 1 -- 172.21.15.22:0/827514287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 msgr2=0x55ba0dbb1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbb1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7bfff640 1 -- 172.21.15.22:0/827514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 msgr2=0x55ba0dbaf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbaf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7bfff640 1 -- 172.21.15.22:0/827514287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba0dab1bc0 con 0x55ba0dba61a0 2024-03-08T23:54:55.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dba6580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3d74012d80 tx=0x7f3d74010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2083f8a4ee62b1b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3d640a2a00 0x7f3d640a4ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d74010c40 tx=0x7f3d7400c000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d61ffb640 1 -- 172.21.15.22:0/827514287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74024070 con 0x55ba0dba61a0 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d61ffb640 1 -- 172.21.15.22:0/827514287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d74009d80 con 0x55ba0dba61a0 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.933+0000 7f3d61ffb640 1 -- 172.21.15.22:0/827514287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74005420 con 0x55ba0dba61a0 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- 172.21.15.22:0/827514287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 msgr2=0x55ba0dba6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dba6580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3d74012d80 tx=0x7f3d74010740 comp rx=0 tx=0).stop 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- 172.21.15.22:0/827514287 shutdown_connections 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbaf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 --2- 172.21.15.22:0/827514287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dba6580 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- 172.21.15.22:0/827514287 >> 172.21.15.22:0/827514287 conn(0x55ba0dba08d0 msgr2=0x55ba0dba0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- 172.21.15.22:0/827514287 shutdown_connections 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- 172.21.15.22:0/827514287 wait complete. 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- start start 2024-03-08T23:54:55.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 Processor -- start 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba61a0 0x55ba0dbac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbacca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0daa84a0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0db05f60 con 0x55ba0dba61a0 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba0dbb2430 con 0x55ba0dbaf6a0 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.934+0000 7f3d84deb740 10 client.0 _ll_register_callbacks cb 0x55ba0db03d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbacca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba61a0 0x55ba0dbac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbacca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38784/0 (socket says 172.21.15.22:38784) 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 -- 172.21.15.22:0/4162720008 learned_addr learned my addr 172.21.15.22:0/4162720008 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 -- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 msgr2=0x55ba0dbad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 -- 172.21.15.22:0/4162720008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba61a0 msgr2=0x55ba0dbac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba61a0 0x55ba0dbac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 -- 172.21.15.22:0/4162720008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba0d81e3f0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba61a0 0x55ba0dbac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d7affd640 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbacca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000d5f0 tx=0x7f3d7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e55a8c7da35e5b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d5affd640 1 -- 172.21.15.22:0/4162720008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70004220 con 0x55ba0dba6ac0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d5affd640 1 -- 172.21.15.22:0/4162720008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3d700043c0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d74009d80 con 0x55ba0dba6ac0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d5affd640 1 -- 172.21.15.22:0/4162720008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d70005020 con 0x55ba0dba6ac0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d84deb740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.935+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba0d8e87d0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d5affd640 1 -- 172.21.15.22:0/4162720008 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d74009d80 con 0x55ba0dba6ac0 2024-03-08T23:54:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d5affd640 1 -- 172.21.15.22:0/4162720008 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3d7000f6f0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d84deb740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d84deb740 10 client.15792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d84deb740 20 client.15792 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d84deb740 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dbbf930 0x55ba0dbc1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d84deb740 4 client.15792 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba0db17400 con 0x55ba0dbbf930 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.936+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dbbf930 0x55ba0dbc1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dbbf930 0x55ba0dbc1d20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c682fd2de96f1569 server_cookie=7a8813a88d66ae23 in_seq=0 out_seq=0 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d5affd640 10 client.15792 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d5affd640 1 -- 172.21.15.22:0/4162720008 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba0db17400 con 0x55ba0dbbf930 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d5affd640 10 client.15792 handle_command_reply: tid=0 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d84deb740 1 client.15792 shutdown 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 msgr2=0x55ba0dbacca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d84deb740 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbacca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000d5f0 tx=0x7f3d7000dab0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 shutdown_connections 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.937+0000 7f3d84deb740 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dbbf930 0x55ba0dbc1d20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d84deb740 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d84deb740 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbacca0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d84deb740 1 --2- 172.21.15.22:0/4162720008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba61a0 0x55ba0dbac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d5affd640 0 client.15792 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 >> 172.21.15.22:0/4162720008 conn(0x55ba0dba08d0 msgr2=0x55ba0dba2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 shutdown_connections 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d84deb740 1 -- 172.21.15.22:0/4162720008 wait complete. 2024-03-08T23:54:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.938+0000 7f3d84deb740 20 client.15792 trim_cache size 0 max 16384 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 Processor -- start 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 -- start start 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dba61a0 0x55ba0dbaee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dba7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dba7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1b09a0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1ae540 con 0x55ba0dbaf6a0 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.949+0000 7f3d84deb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1ac5e0 con 0x55ba0dba61a0 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dba7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dba7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38788/0 (socket says 172.21.15.22:38788) 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 -- 172.21.15.22:0/3260892645 learned_addr learned my addr 172.21.15.22:0/3260892645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dba61a0 0x55ba0dbaee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dba7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 -- 172.21.15.22:0/3260892645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dba61a0 msgr2=0x55ba0dbaee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dba61a0 0x55ba0dbaee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 -- 172.21.15.22:0/3260892645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dbaf6a0 msgr2=0x55ba0dba7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dba7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 -- 172.21.15.22:0/3260892645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba0dba0110 con 0x55ba0dba6ac0 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7bfff640 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dba7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dba61a0 0x55ba0dbaee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d7affd640 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dba7a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000ba40 tx=0x7f3d7000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e80e6ef476fb0ac server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d617fa640 1 -- 172.21.15.22:0/3260892645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70029070 con 0x55ba0dba6ac0 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d617fa640 1 -- 172.21.15.22:0/3260892645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d7000be50 con 0x55ba0dba6ac0 2024-03-08T23:54:55.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.950+0000 7f3d617fa640 1 -- 172.21.15.22:0/3260892645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70017430 con 0x55ba0dba6ac0 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- 172.21.15.22:0/3260892645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 msgr2=0x55ba0dba7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dba7a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000ba40 tx=0x7f3d7000b400 comp rx=0 tx=0).stop 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- 172.21.15.22:0/3260892645 shutdown_connections 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dba7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba6ac0 0x55ba0dba7a20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3260892645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dba61a0 0x55ba0dbaee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- 172.21.15.22:0/3260892645 >> 172.21.15.22:0/3260892645 conn(0x55ba0dba08d0 msgr2=0x55ba0dba6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- 172.21.15.22:0/3260892645 shutdown_connections 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- 172.21.15.22:0/3260892645 wait complete. 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- start start 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 Processor -- start 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dbb6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbb6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbb4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1acbe0 con 0x55ba0dba61a0 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1adee0 con 0x55ba0dba6ac0 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.951+0000 7f3d84deb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1ad220 con 0x55ba0dbaf6a0 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d84deb740 10 client.0 _ll_register_callbacks cb 0x7f3d740123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dbb6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dbb6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38790/0 (socket says 172.21.15.22:38790) 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3390777800 learned_addr learned my addr 172.21.15.22:0/3390777800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3390777800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 msgr2=0x55ba0dbb4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbb4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3390777800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 msgr2=0x55ba0dbb6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbb6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 -- 172.21.15.22:0/3390777800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba0dbb53c0 con 0x55ba0dba61a0 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d7b7fe640 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dbb6650 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b99a0 tx=0x7f3d6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909c86a7e6294471 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:55.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d5affd640 1 -- 172.21.15.22:0/3390777800 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0d5070 con 0x55ba0dba61a0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d5affd640 1 -- 172.21.15.22:0/3390777800 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3d6c0ee360 con 0x55ba0dba61a0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d7000be50 con 0x55ba0dba61a0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d5affd640 1 -- 172.21.15.22:0/3390777800 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d6c04b040 con 0x55ba0dba61a0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d84deb740 10 client.15798 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba0dc16af0 con 0x55ba0dba61a0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.952+0000 7f3d5affd640 1 -- 172.21.15.22:0/3390777800 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d7000be50 con 0x55ba0dba61a0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d5affd640 1 -- 172.21.15.22:0/3390777800 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3d6c0c5330 con 0x55ba0dba61a0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d84deb740 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d84deb740 10 client.15798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d84deb740 20 client.15798 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dc171d0 0x55ba0dc19670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d84deb740 4 client.15798 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ba0db11750 con 0x55ba0dc171d0 2024-03-08T23:54:55.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d7affd640 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dc171d0 0x55ba0dc19670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d5affd640 10 client.15798 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.953+0000 7f3d7affd640 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dc171d0 0x55ba0dc19670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26bb29a1f86899db server_cookie=966272b1d928a341 in_seq=0 out_seq=0 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d5affd640 1 -- 172.21.15.22:0/3390777800 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ba0db11750 con 0x55ba0dc171d0 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d5affd640 10 client.15798 handle_command_reply: tid=0 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 client.15798 shutdown 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 msgr2=0x55ba0dbb6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dbb6650 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b99a0 tx=0x7f3d6c000bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 shutdown_connections 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ba0dc171d0 0x55ba0dc19670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba0dbaf6a0 0x55ba0dbb4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba0dba6ac0 0x55ba0dbb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d5affd640 0 client.15798 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:54:55.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 --2- 172.21.15.22:0/3390777800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba0dba61a0 0x55ba0dbb6650 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 >> 172.21.15.22:0/3390777800 conn(0x55ba0dba08d0 msgr2=0x55ba0dba7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 shutdown_connections 2024-03-08T23:54:55.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 1 -- 172.21.15.22:0/3390777800 wait complete. 2024-03-08T23:54:55.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d84deb740 20 client.15798 trim_cache size 0 max 16384 2024-03-08T23:54:55.957 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3d640a2a00 msgr2=0x7f3d640a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3d640a2a00 0x7f3d640a4ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d74010c40 tx=0x7f3d7400c000 comp rx=0 tx=0).stop 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c069960 msgr2=0x7f3d7c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c069960 0x7f3d7c19d6d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009230 tx=0x7f3d6c02cc70 comp rx=0 tx=0).stop 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 shutdown_connections 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3d640a2a00 0x7f3d640a4ec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d7c1073b0 0x7f3d7c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c06a320 0x7f3d7c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 --2- 172.21.15.22:0/3828422612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d7c069960 0x7f3d7c19d6d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.954+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 >> 172.21.15.22:0/3828422612 conn(0x7f3d7c076f90 msgr2=0x7f3d7c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.955+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 shutdown_connections 2024-03-08T23:54:55.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:55.955+0000 7f3d8220b640 1 -- 172.21.15.22:0/3828422612 wait complete. 2024-03-08T23:54:55.961 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:54:55.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 -- 172.21.15.22:0/2787101526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c100d90 msgr2=0x7fee6c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 --2- 172.21.15.22:0/2787101526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c100d90 0x7fee6c104dd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009900 tx=0x7fee5c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 -- 172.21.15.22:0/2787101526 shutdown_connections 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 --2- 172.21.15.22:0/2787101526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee6c1054b0 0x7fee6c10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 --2- 172.21.15.22:0/2787101526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c100d90 0x7fee6c104dd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 --2- 172.21.15.22:0/2787101526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee6c1003e0 0x7fee6c1007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 -- 172.21.15.22:0/2787101526 >> 172.21.15.22:0/2787101526 conn(0x7fee6c0773a0 msgr2=0x7fee6c0777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.069+0000 7fee73309640 1 -- 172.21.15.22:0/2787101526 shutdown_connections 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 -- 172.21.15.22:0/2787101526 wait complete. 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 Processor -- start 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 -- start start 2024-03-08T23:54:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c1003e0 0x7fee6c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee6c1054b0 0x7fee6c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee6c1994d0 0x7fee6c1a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c110540 con 0x7fee6c1003e0 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1103c0 con 0x7fee6c1054b0 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee73309640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1106c0 con 0x7fee6c1994d0 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee710a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c1003e0 0x7fee6c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee710a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c1003e0 0x7fee6c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38798/0 (socket says 172.21.15.22:38798) 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee710a5640 1 -- 172.21.15.22:0/2712684552 learned_addr learned my addr 172.21.15.22:0/2712684552 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.070+0000 7fee718a6640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee6c1994d0 0x7fee6c1a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee708a4640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee6c1054b0 0x7fee6c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee710a5640 1 -- 172.21.15.22:0/2712684552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee6c1994d0 msgr2=0x7fee6c1a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee710a5640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee6c1994d0 0x7fee6c1a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee710a5640 1 -- 172.21.15.22:0/2712684552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee6c1054b0 msgr2=0x7fee6c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee710a5640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee6c1054b0 0x7fee6c198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee710a5640 1 -- 172.21.15.22:0/2712684552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c1a4250 con 0x7fee6c1003e0 2024-03-08T23:54:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee710a5640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c1003e0 0x7fee6c198a00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c890 tx=0x7fee6800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee627fc640 1 -- 172.21.15.22:0/2712684552 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68004220 con 0x7fee6c1003e0 2024-03-08T23:54:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee627fc640 1 -- 172.21.15.22:0/2712684552 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee680043c0 con 0x7fee6c1003e0 2024-03-08T23:54:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c1a4540 con 0x7fee6c1003e0 2024-03-08T23:54:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee627fc640 1 -- 172.21.15.22:0/2712684552 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680049e0 con 0x7fee6c1003e0 2024-03-08T23:54:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.071+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c1a4a90 con 0x7fee6c1003e0 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.073+0000 7fee627fc640 1 -- 172.21.15.22:0/2712684552 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7fee680079b0 con 0x7fee6c1003e0 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 Processor -- start 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 -- start start 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee627fc640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fee580a2a00 0x7fee580a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57be160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b577c360 con 0x5639b57b51a0 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b5718ec0 con 0x5639b57be6a0 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee75fc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b56c1d30 con 0x5639b57b5ac0 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee718a6640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fee580a2a00 0x7fee580a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38808/0 (socket says 172.21.15.22:38808) 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 -- 172.21.15.22:0/1641271144 learned_addr learned my addr 172.21.15.22:0/1641271144 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee718a6640 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee627fc640 1 -- 172.21.15.22:0/2712684552 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fee680cf500 con 0x7fee6c1003e0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee708a4640 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57be160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 -- 172.21.15.22:0/1641271144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 msgr2=0x5639b57be160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57be160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 -- 172.21.15.22:0/1641271144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 msgr2=0x5639b57c0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.074+0000 7fee710a5640 1 -- 172.21.15.22:0/1641271144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b56c0bc0 con 0x5639b57b51a0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.075+0000 7fee718a6640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fee580a2a00 0x7fee580a4ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee640139f0 tx=0x7fee6400ac20 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.075+0000 7fee710a5640 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57b5580 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fee6809b600 tx=0x7fee68020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65819424a671cf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.075+0000 7fee577fe640 1 -- 172.21.15.22:0/1641271144 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6801d070 con 0x5639b57b51a0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.075+0000 7fee577fe640 1 -- 172.21.15.22:0/1641271144 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee6809bd70 con 0x5639b57b51a0 2024-03-08T23:54:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.075+0000 7fee577fe640 1 -- 172.21.15.22:0/1641271144 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680bb040 con 0x5639b57b51a0 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.075+0000 7fee75fc6740 1 -- 172.21.15.22:0/1641271144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 msgr2=0x5639b57b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.075+0000 7fee75fc6740 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57b5580 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fee6809b600 tx=0x7fee68020000 comp rx=0 tx=0).stop 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- 172.21.15.22:0/1641271144 shutdown_connections 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57be160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 --2- 172.21.15.22:0/1641271144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57b5580 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- 172.21.15.22:0/1641271144 >> 172.21.15.22:0/1641271144 conn(0x5639b57af8d0 msgr2=0x5639b57afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- 172.21.15.22:0/1641271144 shutdown_connections 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- 172.21.15.22:0/1641271144 wait complete. 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- start start 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 Processor -- start 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57bb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57bbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57bc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b56b74a0 con 0x5639b57b51a0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b5714f60 con 0x5639b57be6a0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b57c13d0 con 0x5639b57b5ac0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee75fc6740 10 client.0 _ll_register_callbacks cb 0x5639b5712d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee710a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.076+0000 7fee710a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57bb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38810/0 (socket says 172.21.15.22:38810) 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee710a5640 1 -- 172.21.15.22:0/2657405671 learned_addr learned my addr 172.21.15.22:0/2657405671 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee718a6640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57bc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee708a4640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57bbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee710a5640 1 -- 172.21.15.22:0/2657405671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 msgr2=0x5639b57bbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee710a5640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57bbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee710a5640 1 -- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 msgr2=0x5639b57bc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee710a5640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57bc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee710a5640 1 -- 172.21.15.22:0/2657405671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639b542d3f0 con 0x5639b57b51a0 2024-03-08T23:54:56.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee718a6640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee710a5640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57bb700 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fee68098040 tx=0x7fee68099cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fe42b31fa2854c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee54ff9640 1 -- 172.21.15.22:0/2657405671 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680b3070 con 0x5639b57b51a0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee54ff9640 1 -- 172.21.15.22:0/2657405671 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fee68099010 con 0x5639b57b51a0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee6809bd70 con 0x5639b57b51a0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee54ff9640 1 -- 172.21.15.22:0/2657405671 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee680bb050 con 0x5639b57b51a0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee75fc6740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639b54f77d0 con 0x5639b57b51a0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.077+0000 7fee54ff9640 1 -- 172.21.15.22:0/2657405671 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee6809bd70 con 0x5639b57b51a0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee54ff9640 1 -- 172.21.15.22:0/2657405671 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fee68097ce0 con 0x5639b57b51a0 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee75fc6740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee75fc6740 10 client.15810 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee75fc6740 20 client.15810 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b57ce6a0 0x5639b57d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee75fc6740 4 client.15810 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639b5726400 con 0x5639b57ce6a0 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee708a4640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b57ce6a0 0x5639b57d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee54ff9640 10 client.15810 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.078+0000 7fee708a4640 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b57ce6a0 0x5639b57d0a90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aec53942f15296d4 server_cookie=bc2b552e91207acb in_seq=0 out_seq=0 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee54ff9640 1 -- 172.21.15.22:0/2657405671 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639b5726400 con 0x5639b57ce6a0 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee54ff9640 10 client.15810 handle_command_reply: tid=0 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 client.15810 shutdown 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 msgr2=0x5639b57bb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57bb700 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fee68098040 tx=0x7fee68099cc0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 shutdown_connections 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b57ce6a0 0x5639b57d0a90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b5ac0 0x5639b57bbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2657405671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b51a0 0x5639b57bb700 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee54ff9640 0 client.15810 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:56.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 >> 172.21.15.22:0/2657405671 conn(0x5639b57af8d0 msgr2=0x5639b57b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 shutdown_connections 2024-03-08T23:54:56.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 1 -- 172.21.15.22:0/2657405671 wait complete. 2024-03-08T23:54:56.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.079+0000 7fee75fc6740 20 client.15810 trim_cache size 0 max 16384 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 Processor -- start 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 -- start start 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b57bde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b5ac0 0x5639b57d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1b0b90 con 0x5639b57b5ac0 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1ae730 con 0x5639b57be6a0 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee75fc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1ac7d0 con 0x5639b57b51a0 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee708a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b5ac0 0x5639b57d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee708a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b5ac0 0x5639b57d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38824/0 (socket says 172.21.15.22:38824) 2024-03-08T23:54:56.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.091+0000 7fee708a4640 1 -- 172.21.15.22:0/2039306765 learned_addr learned my addr 172.21.15.22:0/2039306765 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee708a4640 1 -- 172.21.15.22:0/2039306765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 msgr2=0x5639b57bde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee718a6640 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee708a4640 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b57bde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee708a4640 1 -- 172.21.15.22:0/2039306765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 msgr2=0x5639b57c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee708a4640 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee708a4640 1 -- 172.21.15.22:0/2039306765 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b57b4290 con 0x5639b57b5ac0 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee718a6640 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee708a4640 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b5ac0 0x5639b57d2e90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c02c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec9b5efde6c3edc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee56ffd640 1 -- 172.21.15.22:0/2039306765 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c02cd40 con 0x5639b57b5ac0 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee56ffd640 1 -- 172.21.15.22:0/2039306765 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee5c005040 con 0x5639b57b5ac0 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee56ffd640 1 -- 172.21.15.22:0/2039306765 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004b80 con 0x5639b57b5ac0 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee75fc6740 1 -- 172.21.15.22:0/2039306765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b5ac0 msgr2=0x5639b57d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.092+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b5ac0 0x5639b57d2e90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c02c3d0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- 172.21.15.22:0/2039306765 shutdown_connections 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57be6a0 0x5639b57c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57b5ac0 0x5639b57d2e90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 --2- 172.21.15.22:0/2039306765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b57bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- 172.21.15.22:0/2039306765 >> 172.21.15.22:0/2039306765 conn(0x5639b57af8d0 msgr2=0x5639b57c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- 172.21.15.22:0/2039306765 shutdown_connections 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- 172.21.15.22:0/2039306765 wait complete. 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- start start 2024-03-08T23:54:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 Processor -- start 2024-03-08T23:54:56.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b5822810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57b5ac0 0x5639b5822d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57be6a0 0x5639b581cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1acdd0 con 0x5639b57be6a0 2024-03-08T23:54:56.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1ae0d0 con 0x5639b57b5ac0 2024-03-08T23:54:56.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1ad410 con 0x5639b57b51a0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee75fc6740 10 client.0 _ll_register_callbacks cb 0x7fee680cc6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee718a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57be6a0 0x5639b581cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee718a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57be6a0 0x5639b581cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38830/0 (socket says 172.21.15.22:38830) 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.093+0000 7fee718a6640 1 -- 172.21.15.22:0/4151297756 learned_addr learned my addr 172.21.15.22:0/4151297756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee710a5640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b5822810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee708a4640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57b5ac0 0x5639b5822d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee718a6640 1 -- 172.21.15.22:0/4151297756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 msgr2=0x5639b5822810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee718a6640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b5822810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee718a6640 1 -- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57b5ac0 msgr2=0x5639b5822d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee718a6640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57b5ac0 0x5639b5822d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee718a6640 1 -- 172.21.15.22:0/4151297756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639b581d310 con 0x5639b57be6a0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee710a5640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b5822810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee718a6640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57be6a0 0x5639b581cc20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fee640135f0 tx=0x7fee64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f143a18859395492 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:56.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee54ff9640 1 -- 172.21.15.22:0/4151297756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64012c00 con 0x5639b57be6a0 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee54ff9640 1 -- 172.21.15.22:0/4151297756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fee64012da0 con 0x5639b57be6a0 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee5c005040 con 0x5639b57be6a0 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee54ff9640 1 -- 172.21.15.22:0/4151297756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee64003970 con 0x5639b57be6a0 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee75fc6740 10 client.15816 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639b5829fb0 con 0x5639b57be6a0 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee54ff9640 1 -- 172.21.15.22:0/4151297756 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee5c005040 con 0x5639b57be6a0 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.094+0000 7fee54ff9640 1 -- 172.21.15.22:0/4151297756 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fee6401c040 con 0x5639b57be6a0 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 10 client.15816 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 10 client.15816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 20 client.15816 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b582a790 0x5639b582cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 4 client.15816 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639b5720750 con 0x5639b582a790 2024-03-08T23:54:56.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee710a5640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b582a790 0x5639b582cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee710a5640 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b582a790 0x5639b582cbb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f7d267124dba073 server_cookie=6ebd4852debc632 in_seq=0 out_seq=0 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee54ff9640 10 client.15816 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee54ff9640 1 -- 172.21.15.22:0/4151297756 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639b5720750 con 0x5639b582a790 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee54ff9640 10 client.15816 handle_command_reply: tid=0 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 client.15816 shutdown 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57be6a0 msgr2=0x5639b581cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57be6a0 0x5639b581cc20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fee640135f0 tx=0x7fee64004ab0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 shutdown_connections 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5639b582a790 0x5639b582cbb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5639b57be6a0 0x5639b581cc20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639b57b5ac0 0x5639b5822d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee75fc6740 1 --2- 172.21.15.22:0/4151297756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639b57b51a0 0x5639b5822810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.095+0000 7fee54ff9640 0 client.15816 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 >> 172.21.15.22:0/4151297756 conn(0x5639b57af8d0 msgr2=0x5639b57bbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 shutdown_connections 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee75fc6740 1 -- 172.21.15.22:0/4151297756 wait complete. 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee75fc6740 20 client.15816 trim_cache size 0 max 16384 2024-03-08T23:54:56.099 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:54:56.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fee580a2a00 msgr2=0x7fee580a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fee580a2a00 0x7fee580a4ec0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee640139f0 tx=0x7fee6400ac20 comp rx=0 tx=0).stop 2024-03-08T23:54:56.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c1003e0 msgr2=0x7fee6c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c1003e0 0x7fee6c198a00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c890 tx=0x7fee6800cd50 comp rx=0 tx=0).stop 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 shutdown_connections 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fee580a2a00 0x7fee580a4ec0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee6c1994d0 0x7fee6c1a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee6c1054b0 0x7fee6c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 --2- 172.21.15.22:0/2712684552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee6c1003e0 0x7fee6c198a00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 >> 172.21.15.22:0/2712684552 conn(0x7fee6c0773a0 msgr2=0x7fee6c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 shutdown_connections 2024-03-08T23:54:56.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.096+0000 7fee73309640 1 -- 172.21.15.22:0/2712684552 wait complete. 2024-03-08T23:54:56.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:55 smithi022 ceph-mon[31894]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-08T23:54:56.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:55 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3461907961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:56.103 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:54:56.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:54:56.183 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:55 smithi192 ceph-mon[35995]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-08T23:54:56.183 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:55 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3461907961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:56.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 -- 172.21.15.22:0/1616965808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 msgr2=0x7f14a410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 --2- 172.21.15.22:0/1616965808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a410e130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f149c009cd0 tx=0x7f149c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 -- 172.21.15.22:0/1616965808 shutdown_connections 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 --2- 172.21.15.22:0/1616965808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a410e130 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 --2- 172.21.15.22:0/1616965808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14a4069960 0x7f14a4069de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 --2- 172.21.15.22:0/1616965808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41073b0 0x7f14a4107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 -- 172.21.15.22:0/1616965808 >> 172.21.15.22:0/1616965808 conn(0x7f14a4076f90 msgr2=0x7f14a40773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 -- 172.21.15.22:0/1616965808 shutdown_connections 2024-03-08T23:54:56.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.211+0000 7f14abba9640 1 -- 172.21.15.22:0/1616965808 wait complete. 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 Processor -- start 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 -- start start 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a4069960 0x7f14a419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14a41073b0 0x7f14a4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4111640 con 0x7f14a406a320 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41114c0 con 0x7f14a4069960 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14abba9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41117c0 con 0x7f14a41073b0 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a419dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38854/0 (socket says 172.21.15.22:38854) 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 -- 172.21.15.22:0/140394471 learned_addr learned my addr 172.21.15.22:0/140394471 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9945640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a4069960 0x7f14a419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14aa146640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14a41073b0 0x7f14a4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 -- 172.21.15.22:0/140394471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14a41073b0 msgr2=0x7f14a4197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14a41073b0 0x7f14a4197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 -- 172.21.15.22:0/140394471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a4069960 msgr2=0x7f14a419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a4069960 0x7f14a419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14a9144640 1 -- 172.21.15.22:0/140394471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4197fe0 con 0x7f14a406a320 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.212+0000 7f14aa146640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14a41073b0 0x7f14a4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.213+0000 7f14a9144640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a419dbf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9f0 tx=0x7f14a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.213+0000 7f149affd640 1 -- 172.21.15.22:0/140394471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000cd80 con 0x7f14a406a320 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.213+0000 7f149affd640 1 -- 172.21.15.22:0/140394471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a0005500 con 0x7f14a406a320 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.213+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a41982d0 con 0x7f14a406a320 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.213+0000 7f149affd640 1 -- 172.21.15.22:0/140394471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0010690 con 0x7f14a406a320 2024-03-08T23:54:56.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.213+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a41a4570 con 0x7f14a406a320 2024-03-08T23:54:56.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.215+0000 7f149affd640 1 -- 172.21.15.22:0/140394471 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f14a00046c0 con 0x7f14a406a320 2024-03-08T23:54:56.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.215+0000 7f14ae789740 1 Processor -- start 2024-03-08T23:54:56.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.215+0000 7f14ae789740 1 -- start start 2024-03-08T23:54:56.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14ae789740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef873580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14ae789740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef873ac0 0x55eaef87c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14ae789740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14ae789740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaef83a360 con 0x55eaef87c6a0 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14ae789740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaef7d6ec0 con 0x55eaef8731a0 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14ae789740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaef77fd30 con 0x55eaef873ac0 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f149affd640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14900a2a00 0x7f14900a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14aa146640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef873580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38858/0 (socket says 172.21.15.22:38858) 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 -- 172.21.15.22:0/3823870135 learned_addr learned my addr 172.21.15.22:0/3823870135 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14aa146640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef873580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:48882/0 (socket says 172.21.15.22:48882) 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14aa146640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14900a2a00 0x7f14900a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9945640 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef873ac0 0x55eaef87c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f149affd640 1 -- 172.21.15.22:0/140394471 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f14a00cf8a0 con 0x7f14a406a320 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 -- 172.21.15.22:0/3823870135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef873ac0 msgr2=0x55eaef87c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef873ac0 0x55eaef87c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 -- 172.21.15.22:0/3823870135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 msgr2=0x55eaef873580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef873580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 -- 172.21.15.22:0/3823870135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaef77ebc0 con 0x55eaef87c6a0 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9945640 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef873ac0 0x55eaef87c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14a9144640 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87ea90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f14a00ccaa0 tx=0x7f14a0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e7b4e3488e7f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.216+0000 7f14aa146640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14900a2a00 0x7f14900a4ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f149c02c630 tx=0x7f149c0095c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f148ffff640 1 -- 172.21.15.22:0/3823870135 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a009c090 con 0x55eaef87c6a0 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f148ffff640 1 -- 172.21.15.22:0/3823870135 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a009b990 con 0x55eaef87c6a0 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f148ffff640 1 -- 172.21.15.22:0/3823870135 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00a3ca0 con 0x55eaef87c6a0 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 -- 172.21.15.22:0/3823870135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 msgr2=0x55eaef87ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87ea90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f14a00ccaa0 tx=0x7f14a0005740 comp rx=0 tx=0).stop 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 -- 172.21.15.22:0/3823870135 shutdown_connections 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87ea90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef873ac0 0x55eaef87c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 --2- 172.21.15.22:0/3823870135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef873580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 -- 172.21.15.22:0/3823870135 >> 172.21.15.22:0/3823870135 conn(0x55eaef86d8d0 msgr2=0x55eaef86dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.217+0000 7f14ae789740 1 -- 172.21.15.22:0/3823870135 shutdown_connections 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 -- 172.21.15.22:0/3823870135 wait complete. 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 -- start start 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 Processor -- start 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef879630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef879b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaef7754a0 con 0x55eaef87c6a0 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaef7d2f60 con 0x55eaef873ac0 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaef87f3d0 con 0x55eaef8731a0 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14ae789740 10 client.0 _ll_register_callbacks cb 0x55eaef7d0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14aa146640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14aa146640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38866/0 (socket says 172.21.15.22:38866) 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.218+0000 7f14aa146640 1 -- 172.21.15.22:0/642001078 learned_addr learned my addr 172.21.15.22:0/642001078 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14a9144640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef879b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14a9945640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef879630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14aa146640 1 -- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 msgr2=0x55eaef879630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14aa146640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef879630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14aa146640 1 -- 172.21.15.22:0/642001078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 msgr2=0x55eaef879b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14aa146640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef879b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14aa146640 1 -- 172.21.15.22:0/642001078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaef4eb3f0 con 0x55eaef87c6a0 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14a9144640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef879b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14a9945640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef879630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14aa146640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87a0b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f149c003c50 tx=0x7f149c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bab2d35b7dbadb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f148d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f148d7fa640 1 -- 172.21.15.22:0/642001078 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c053060 con 0x55eaef87c6a0 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f148d7fa640 1 -- 172.21.15.22:0/642001078 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f149c0047c0 con 0x55eaef87c6a0 2024-03-08T23:54:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14a009b990 con 0x55eaef87c6a0 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f148d7fa640 1 -- 172.21.15.22:0/642001078 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f149c039070 con 0x55eaef87c6a0 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14ae789740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55eaef5b57d0 con 0x55eaef87c6a0 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.219+0000 7f148d7fa640 1 -- 172.21.15.22:0/642001078 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a009b990 con 0x55eaef87c6a0 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f148d7fa640 1 -- 172.21.15.22:0/642001078 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f149c04d070 con 0x55eaef87c6a0 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14ae789740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14ae789740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14ae789740 20 client.15828 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14ae789740 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef88c770 0x55eaef88ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14ae789740 4 client.15828 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaef7e4400 con 0x55eaef88c770 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14a9945640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef88c770 0x55eaef88ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f148d7fa640 10 client.15828 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.220+0000 7f14a9945640 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef88c770 0x55eaef88ebf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c21cc8c8d71b6ae server_cookie=d9f68a8eaa2eaa67 in_seq=0 out_seq=0 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f148d7fa640 1 -- 172.21.15.22:0/642001078 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eaef7e4400 con 0x55eaef88c770 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f148d7fa640 10 client.15828 handle_command_reply: tid=0 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 client.15828 shutdown 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 msgr2=0x55eaef87a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87a0b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f149c003c50 tx=0x7f149c009360 comp rx=0 tx=0).stop 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 shutdown_connections 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef88c770 0x55eaef88ebf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef87a0b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f148d7fa640 0 client.15828 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef879b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 --2- 172.21.15.22:0/642001078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef879630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 >> 172.21.15.22:0/642001078 conn(0x55eaef86d8d0 msgr2=0x55eaef86f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 shutdown_connections 2024-03-08T23:54:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 1 -- 172.21.15.22:0/642001078 wait complete. 2024-03-08T23:54:56.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.221+0000 7f14ae789740 20 client.15828 trim_cache size 0 max 16384 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 Processor -- start 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 -- start start 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef87bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef876890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef876dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41b0a90 con 0x55eaef87c6a0 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41ae630 con 0x55eaef873ac0 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14ae789740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41ac6d0 con 0x55eaef8731a0 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef876dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef876dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38882/0 (socket says 172.21.15.22:38882) 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 -- 172.21.15.22:0/1890940995 learned_addr learned my addr 172.21.15.22:0/1890940995 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14a9945640 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef87bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 -- 172.21.15.22:0/1890940995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 msgr2=0x55eaef87bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef87bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 -- 172.21.15.22:0/1890940995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 msgr2=0x55eaef876890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14a9144640 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef876890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef876890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14aa146640 1 -- 172.21.15.22:0/1890940995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaef877b80 con 0x55eaef87c6a0 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.233+0000 7f14a9144640 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef876890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14aa146640 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef876dd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f149c005ff0 tx=0x7f149c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a144911ab41533c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f148f7fe640 1 -- 172.21.15.22:0/1890940995 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c063070 con 0x55eaef87c6a0 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f148f7fe640 1 -- 172.21.15.22:0/1890940995 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f149c03a040 con 0x55eaef87c6a0 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f148f7fe640 1 -- 172.21.15.22:0/1890940995 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c053070 con 0x55eaef87c6a0 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 -- 172.21.15.22:0/1890940995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 msgr2=0x55eaef876dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef876dd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f149c005ff0 tx=0x7f149c004a80 comp rx=0 tx=0).stop 2024-03-08T23:54:56.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 -- 172.21.15.22:0/1890940995 shutdown_connections 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef87c6a0 0x55eaef876dd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef873ac0 0x55eaef876890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 --2- 172.21.15.22:0/1890940995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef8731a0 0x55eaef87bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 -- 172.21.15.22:0/1890940995 >> 172.21.15.22:0/1890940995 conn(0x55eaef86d8d0 msgr2=0x55eaef86dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 -- 172.21.15.22:0/1890940995 shutdown_connections 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.234+0000 7f14ae789740 1 -- 172.21.15.22:0/1890940995 wait complete. 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 -- start start 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 Processor -- start 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef882050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef873ac0 0x55eaef882590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef87c6a0 0x55eaef882ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41accd0 con 0x55eaef873ac0 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41adfd0 con 0x55eaef8731a0 2024-03-08T23:54:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41ad310 con 0x55eaef87c6a0 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14ae789740 10 client.0 _ll_register_callbacks cb 0x7f14a00cc9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9144640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef873ac0 0x55eaef882590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9144640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef873ac0 0x55eaef882590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38898/0 (socket says 172.21.15.22:38898) 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9144640 1 -- 172.21.15.22:0/1477245677 learned_addr learned my addr 172.21.15.22:0/1477245677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9945640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef882050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14aa146640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef87c6a0 0x55eaef882ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9144640 1 -- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef87c6a0 msgr2=0x55eaef882ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9144640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef87c6a0 0x55eaef882ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9144640 1 -- 172.21.15.22:0/1477245677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 msgr2=0x55eaef882050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.235+0000 7f14a9144640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef882050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14a9144640 1 -- 172.21.15.22:0/1477245677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaef8e2d30 con 0x55eaef873ac0 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14aa146640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef87c6a0 0x55eaef882ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14a9144640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef873ac0 0x55eaef882590 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f14a00cfb90 tx=0x7f14a00ccb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b971a578c2cfd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:56.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f148d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f148d7fa640 1 -- 172.21.15.22:0/1477245677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00cf630 con 0x55eaef873ac0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f148d7fa640 1 -- 172.21.15.22:0/1477245677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f14a00a3ca0 con 0x55eaef873ac0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f149c03a040 con 0x55eaef873ac0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f148d7fa640 1 -- 172.21.15.22:0/1477245677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a00a4130 con 0x55eaef873ac0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 10 client.15834 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55eaef8e3fa0 con 0x55eaef873ac0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f148d7fa640 1 -- 172.21.15.22:0/1477245677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f149c03a040 con 0x55eaef873ac0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f148d7fa640 1 -- 172.21.15.22:0/1477245677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f14a00860d0 con 0x55eaef873ac0 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 20 client.15834 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef8e4330 0x55eaef8e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 4 client.15834 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.236+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eaef7de750 con 0x55eaef8e4330 2024-03-08T23:54:56.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14a9945640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef8e4330 0x55eaef8e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f148d7fa640 10 client.15834 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14a9945640 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef8e4330 0x55eaef8e6720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1930f485557ebeb server_cookie=1b94943280c90c0a in_seq=0 out_seq=0 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f148d7fa640 1 -- 172.21.15.22:0/1477245677 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eaef7de750 con 0x55eaef8e4330 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f148d7fa640 10 client.15834 handle_command_reply: tid=0 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 client.15834 shutdown 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef873ac0 msgr2=0x55eaef882590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef873ac0 0x55eaef882590 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f14a00cfb90 tx=0x7f14a00ccb60 comp rx=0 tx=0).stop 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 shutdown_connections 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eaef8e4330 0x55eaef8e6720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eaef87c6a0 0x55eaef882ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eaef873ac0 0x55eaef882590 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 --2- 172.21.15.22:0/1477245677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaef8731a0 0x55eaef882050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f148d7fa640 0 client.15834 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:56.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 >> 172.21.15.22:0/1477245677 conn(0x55eaef86d8d0 msgr2=0x55eaef877f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 shutdown_connections 2024-03-08T23:54:56.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.237+0000 7f14ae789740 1 -- 172.21.15.22:0/1477245677 wait complete. 2024-03-08T23:54:56.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14ae789740 20 client.15834 trim_cache size 0 max 16384 2024-03-08T23:54:56.241 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14900a2a00 msgr2=0x7f14900a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14900a2a00 0x7f14900a4ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f149c02c630 tx=0x7f149c0095c0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 msgr2=0x7f14a419dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a419dbf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9f0 tx=0x7f14a000edf0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 shutdown_connections 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14900a2a00 0x7f14900a4ec0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14a41073b0 0x7f14a4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14a406a320 0x7f14a419dbf0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 --2- 172.21.15.22:0/140394471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a4069960 0x7f14a419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:56.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 >> 172.21.15.22:0/140394471 conn(0x7f14a4076f90 msgr2=0x7f14a40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:56.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 shutdown_connections 2024-03-08T23:54:56.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:56.238+0000 7f14abba9640 1 -- 172.21.15.22:0/140394471 wait complete. 2024-03-08T23:54:56.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:55 smithi062 ceph-mon[36129]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-08T23:54:56.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:55 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3461907961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:54:57.246 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:54:57.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:54:57.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:54:57.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 -- 172.21.15.22:0/2167865489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102e50 msgr2=0x7f828010b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 --2- 172.21.15.22:0/2167865489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102e50 0x7f828010b330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8274009900 tx=0x7f827402bcd0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 -- 172.21.15.22:0/2167865489 shutdown_connections 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 --2- 172.21.15.22:0/2167865489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828010b900 0x7f828010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 --2- 172.21.15.22:0/2167865489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102e50 0x7f828010b330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 --2- 172.21.15.22:0/2167865489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8280102530 0x7f8280102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 -- 172.21.15.22:0/2167865489 >> 172.21.15.22:0/2167865489 conn(0x7f8280076570 msgr2=0x7f8280076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 -- 172.21.15.22:0/2167865489 shutdown_connections 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.370+0000 7f828731b640 1 -- 172.21.15.22:0/2167865489 wait complete. 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 Processor -- start 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 -- start start 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102530 0x7f828019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8280102e50 0x7f828019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828010b900 0x7f8280197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801116e0 con 0x7f8280102530 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280111560 con 0x7f828010b900 2024-03-08T23:54:57.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f828731b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280111860 con 0x7f8280102e50 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102530 0x7f828019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102530 0x7f828019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38918/0 (socket says 172.21.15.22:38918) 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 -- 172.21.15.22:0/1627161957 learned_addr learned my addr 172.21.15.22:0/1627161957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82848b6640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8280102e50 0x7f828019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82858b8640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828010b900 0x7f8280197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 -- 172.21.15.22:0/1627161957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8280102e50 msgr2=0x7f828019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8280102e50 0x7f828019dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 -- 172.21.15.22:0/1627161957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828010b900 msgr2=0x7f8280197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828010b900 0x7f8280197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 -- 172.21.15.22:0/1627161957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280198180 con 0x7f8280102530 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82848b6640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8280102e50 0x7f828019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f82850b7640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102530 0x7f828019d850 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f827000ea80 tx=0x7f827000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.371+0000 7f827e7fc640 1 -- 172.21.15.22:0/1627161957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827000ce40 con 0x7f8280102530 2024-03-08T23:54:57.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.372+0000 7f827e7fc640 1 -- 172.21.15.22:0/1627161957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8270005020 con 0x7f8280102530 2024-03-08T23:54:57.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.372+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8280198470 con 0x7f8280102530 2024-03-08T23:54:57.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.372+0000 7f827e7fc640 1 -- 172.21.15.22:0/1627161957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270011760 con 0x7f8280102530 2024-03-08T23:54:57.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.372+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82801a4710 con 0x7f8280102530 2024-03-08T23:54:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.374+0000 7f827e7fc640 1 -- 172.21.15.22:0/1627161957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f82700051d0 con 0x7f8280102530 2024-03-08T23:54:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.374+0000 7f8289fd2740 1 Processor -- start 2024-03-08T23:54:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.374+0000 7f8289fd2740 1 -- start start 2024-03-08T23:54:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x5619306a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f8289fd2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619305fdb40 con 0x56193069a6a0 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f8289fd2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619305a6d80 con 0x5619306a3280 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f8289fd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619305f9be0 con 0x561930699d80 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f827e7fc640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f82680a2a80 0x7f82680a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82850b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x5619306a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58480/0 (socket says 172.21.15.22:58480) 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 -- 172.21.15.22:0/3352774520 learned_addr learned my addr 172.21.15.22:0/3352774520 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f82680a2a80 0x7f82680a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82848b6640 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f827e7fc640 1 -- 172.21.15.22:0/1627161957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f82700cfa00 con 0x7f8280102530 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 -- 172.21.15.22:0/3352774520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 msgr2=0x5619306a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 -- 172.21.15.22:0/3352774520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 msgr2=0x5619306a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x5619306a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.375+0000 7f82858b8640 1 -- 172.21.15.22:0/3352774520 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619303da7e0 con 0x561930699d80 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f82858b8640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f82680a2a80 0x7f82680a4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8278005e00 tx=0x7f827801c000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f82858b8640 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069a160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8278013b70 tx=0x7f82780117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b925cd4e5cff4f4c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f82677fe640 1 -- 172.21.15.22:0/3352774520 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278022070 con 0x561930699d80 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f82677fe640 1 -- 172.21.15.22:0/3352774520 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8278004020 con 0x561930699d80 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f82677fe640 1 -- 172.21.15.22:0/3352774520 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278005220 con 0x561930699d80 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f8289fd2740 1 -- 172.21.15.22:0/3352774520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 msgr2=0x56193069a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f8289fd2740 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069a160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8278013b70 tx=0x7f82780117b0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f8289fd2740 1 -- 172.21.15.22:0/3352774520 shutdown_connections 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f8289fd2740 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f8289fd2740 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x5619306a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.376+0000 7f8289fd2740 1 --2- 172.21.15.22:0/3352774520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069a160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 -- 172.21.15.22:0/3352774520 >> 172.21.15.22:0/3352774520 conn(0x5619306944b0 msgr2=0x561930694890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 -- 172.21.15.22:0/3352774520 shutdown_connections 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 -- 172.21.15.22:0/3352774520 wait complete. 2024-03-08T23:54:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 -- start start 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 Processor -- start 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561930699d80 0x5619306a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56193069a6a0 0x56193069ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619306a6010 con 0x561930699d80 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619305ba1c0 con 0x5619306a3280 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619306a6190 con 0x56193069a6a0 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.377+0000 7f8289fd2740 10 client.0 _ll_register_callbacks cb 0x5619305a97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82850b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561930699d80 0x5619306a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82848b6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56193069a6a0 0x56193069ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82850b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561930699d80 0x5619306a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38948/0 (socket says 172.21.15.22:38948) 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82850b7640 1 -- 172.21.15.22:0/2882379881 learned_addr learned my addr 172.21.15.22:0/2882379881 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82858b8640 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82848b6640 1 -- 172.21.15.22:0/2882379881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 msgr2=0x5619306a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82848b6640 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82848b6640 1 -- 172.21.15.22:0/2882379881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561930699d80 msgr2=0x5619306a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82848b6640 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561930699d80 0x5619306a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82848b6640 1 -- 172.21.15.22:0/2882379881 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56193043af70 con 0x56193069a6a0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82850b7640 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561930699d80 0x5619306a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f82848b6640 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56193069a6a0 0x56193069ff70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f827402c210 tx=0x7f827402d930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49895da28cc950a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f8264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f8264ff9640 1 -- 172.21.15.22:0/2882379881 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274043070 con 0x56193069a6a0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8278004020 con 0x56193069a6a0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f8264ff9640 1 -- 172.21.15.22:0/2882379881 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8274005030 con 0x56193069a6a0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f8264ff9640 1 -- 172.21.15.22:0/2882379881 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f827404f050 con 0x56193069a6a0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f8289fd2740 10 client.25378 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.378+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619302d2170 con 0x56193069a6a0 2024-03-08T23:54:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8264ff9640 1 -- 172.21.15.22:0/2882379881 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8278004020 con 0x56193069a6a0 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8264ff9640 1 -- 172.21.15.22:0/2882379881 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f82740035f0 con 0x56193069a6a0 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8289fd2740 10 client.25378 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8289fd2740 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8289fd2740 20 client.25378 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8289fd2740 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5619306b3240 0x5619306b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8289fd2740 4 client.25378 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619305d3960 con 0x5619306b3240 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.379+0000 7f82850b7640 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5619306b3240 0x5619306b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.380+0000 7f82850b7640 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5619306b3240 0x5619306b5670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a93dd0218d6e99c server_cookie=afd85c070ba4f096 in_seq=0 out_seq=0 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.380+0000 7f8264ff9640 10 client.25378 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.380+0000 7f8264ff9640 1 -- 172.21.15.22:0/2882379881 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619305d3960 con 0x5619306b3240 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.380+0000 7f8264ff9640 10 client.25378 handle_command_reply: tid=0 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.380+0000 7f8289fd2740 1 client.25378 shutdown 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56193069a6a0 msgr2=0x56193069ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56193069a6a0 0x56193069ff70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f827402c210 tx=0x7f827402d930 comp rx=0 tx=0).stop 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 shutdown_connections 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5619306b3240 0x5619306b5670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56193069a6a0 0x56193069ff70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8264ff9640 0 client.25378 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 --2- 172.21.15.22:0/2882379881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561930699d80 0x5619306a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 >> 172.21.15.22:0/2882379881 conn(0x5619306944b0 msgr2=0x561930696200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 shutdown_connections 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 1 -- 172.21.15.22:0/2882379881 wait complete. 2024-03-08T23:54:57.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.381+0000 7f8289fd2740 20 client.25378 trim_cache size 0 max 16384 2024-03-08T23:54:57.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 Processor -- start 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 -- start start 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x5619306aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x56193069ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801b09d0 con 0x56193069a6a0 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801ae570 con 0x5619306a3280 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.396+0000 7f8289fd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801ac610 con 0x561930699d80 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x56193069ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x56193069ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38962/0 (socket says 172.21.15.22:38962) 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 -- 172.21.15.22:0/724575371 learned_addr learned my addr 172.21.15.22:0/724575371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82850b7640 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x5619306aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82858b8640 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 -- 172.21.15.22:0/724575371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 msgr2=0x5619306aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x5619306aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 -- 172.21.15.22:0/724575371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 msgr2=0x5619306a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 -- 172.21.15.22:0/724575371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56193069a4d0 con 0x56193069a6a0 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f82848b6640 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x56193069ef90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8274009850 tx=0x7f8274005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61252ad819e7d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f8266ffd640 1 -- 172.21.15.22:0/724575371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827404c070 con 0x56193069a6a0 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f8266ffd640 1 -- 172.21.15.22:0/724575371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8274003b00 con 0x56193069a6a0 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.397+0000 7f8266ffd640 1 -- 172.21.15.22:0/724575371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274043040 con 0x56193069a6a0 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 -- 172.21.15.22:0/724575371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 msgr2=0x56193069ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x56193069ef90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8274009850 tx=0x7f8274005bc0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 -- 172.21.15.22:0/724575371 shutdown_connections 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5619306a3280 0x5619306a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56193069a6a0 0x56193069ef90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 --2- 172.21.15.22:0/724575371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x5619306aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 -- 172.21.15.22:0/724575371 >> 172.21.15.22:0/724575371 conn(0x5619306944b0 msgr2=0x56193069e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 -- 172.21.15.22:0/724575371 shutdown_connections 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 -- 172.21.15.22:0/724575371 wait complete. 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 -- start start 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.398+0000 7f8289fd2740 1 Processor -- start 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56193069a6a0 0x56193069d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619306a3280 0x56193069d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801acc10 con 0x5619306a3280 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801adf10 con 0x56193069a6a0 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801ad250 con 0x561930699d80 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 10 client.0 _ll_register_callbacks cb 0x7f827800c5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619306a3280 0x56193069d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619306a3280 0x56193069d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38974/0 (socket says 172.21.15.22:38974) 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 -- 172.21.15.22:0/317462708 learned_addr learned my addr 172.21.15.22:0/317462708 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82850b7640 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82848b6640 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56193069a6a0 0x56193069d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 -- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 msgr2=0x56193069ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 -- 172.21.15.22:0/317462708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56193069a6a0 msgr2=0x56193069d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56193069a6a0 0x56193069d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 -- 172.21.15.22:0/317462708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619307099a0 con 0x5619306a3280 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f82858b8640 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619306a3280 0x56193069d900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f827800c730 tx=0x7f82780056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffab8a24af4ab616 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8264ff9640 1 -- 172.21.15.22:0/317462708 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278022040 con 0x5619306a3280 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8264ff9640 1 -- 172.21.15.22:0/317462708 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8278009d10 con 0x5619306a3280 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.399+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8274003b00 con 0x5619306a3280 2024-03-08T23:54:57.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8264ff9640 1 -- 172.21.15.22:0/317462708 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f827803f050 con 0x5619306a3280 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 10 client.15852 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56193070ace0 con 0x5619306a3280 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8264ff9640 1 -- 172.21.15.22:0/317462708 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8274003b00 con 0x5619306a3280 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8264ff9640 1 -- 172.21.15.22:0/317462708 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f827800fc00 con 0x5619306a3280 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 10 client.15852 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 20 client.15852 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56193070b320 0x56193070d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 4 client.15852 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5619305bf080 con 0x56193070b320 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.400+0000 7f82850b7640 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56193070b320 0x56193070d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.401+0000 7f8264ff9640 10 client.15852 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.401+0000 7f82850b7640 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56193070b320 0x56193070d740 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99639f3ad149a0f0 server_cookie=1cf90323cc96b3c7 in_seq=0 out_seq=0 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.401+0000 7f8264ff9640 1 -- 172.21.15.22:0/317462708 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5619305bf080 con 0x56193070b320 2024-03-08T23:54:57.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.401+0000 7f8264ff9640 10 client.15852 handle_command_reply: tid=0 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.401+0000 7f8289fd2740 1 client.15852 shutdown 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619306a3280 msgr2=0x56193069d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619306a3280 0x56193069d900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f827800c730 tx=0x7f82780056c0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 shutdown_connections 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56193070b320 0x56193070d740 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5619306a3280 0x56193069d900 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56193069a6a0 0x56193069d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 --2- 172.21.15.22:0/317462708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561930699d80 0x56193069ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8264ff9640 0 client.15852 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 >> 172.21.15.22:0/317462708 conn(0x5619306944b0 msgr2=0x561930695790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 shutdown_connections 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 1 -- 172.21.15.22:0/317462708 wait complete. 2024-03-08T23:54:57.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f8289fd2740 20 client.15852 trim_cache size 0 max 16384 2024-03-08T23:54:57.405 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:54:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "32cf56a9-453b-4303-972a-f2ad6c3330fd", 2024-03-08T23:54:57.405 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:54:57.405 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f82680a2a80 msgr2=0x7f82680a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f828731b640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f82680a2a80 0x7f82680a4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8278005e00 tx=0x7f827801c000 comp rx=0 tx=0).stop 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102530 msgr2=0x7f828019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.402+0000 7f828731b640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102530 0x7f828019d850 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f827000ea80 tx=0x7f827000ef40 comp rx=0 tx=0).stop 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 shutdown_connections 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f82680a2a80 0x7f82680a4f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828010b900 0x7f8280197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8280102e50 0x7f828019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 --2- 172.21.15.22:0/1627161957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8280102530 0x7f828019d850 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 >> 172.21.15.22:0/1627161957 conn(0x7f8280076570 msgr2=0x7f82800fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 shutdown_connections 2024-03-08T23:54:57.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.403+0000 7f828731b640 1 -- 172.21.15.22:0/1627161957 wait complete. 2024-03-08T23:54:57.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:54:57.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:54:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 -- 172.21.15.22:0/3788710385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670105550 msgr2=0x7fe67010d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 --2- 172.21.15.22:0/3788710385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670105550 0x7fe67010d120 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe668009ca0 tx=0x7fe66802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 -- 172.21.15.22:0/3788710385 shutdown_connections 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 --2- 172.21.15.22:0/3788710385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670105550 0x7fe67010d120 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 --2- 172.21.15.22:0/3788710385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe670100e30 0x7fe670104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 --2- 172.21.15.22:0/3788710385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe670100480 0x7fe670100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 -- 172.21.15.22:0/3788710385 >> 172.21.15.22:0/3788710385 conn(0x7fe670077350 msgr2=0x7fe670077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 -- 172.21.15.22:0/3788710385 shutdown_connections 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.518+0000 7fe6773d4640 1 -- 172.21.15.22:0/3788710385 wait complete. 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 Processor -- start 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 -- start start 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670100480 0x7fe6700fe020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe670105550 0x7fe6700fe560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6700feaa0 0x7fe6701a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701105f0 con 0x7fe670100480 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670110470 con 0x7fe670105550 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe6773d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670110770 con 0x7fe6700feaa0 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670100480 0x7fe6700fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670100480 0x7fe6700fe020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38998/0 (socket says 172.21.15.22:38998) 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 -- 172.21.15.22:0/1252238253 learned_addr learned my addr 172.21.15.22:0/1252238253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675971640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6700feaa0 0x7fe6701a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe67496f640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe670105550 0x7fe6700fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 -- 172.21.15.22:0/1252238253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6700feaa0 msgr2=0x7fe6701a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6700feaa0 0x7fe6701a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 -- 172.21.15.22:0/1252238253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe670105550 msgr2=0x7fe6700fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe670105550 0x7fe6700fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 -- 172.21.15.22:0/1252238253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6701a6470 con 0x7fe670100480 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675971640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6700feaa0 0x7fe6701a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe67496f640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe670105550 0x7fe6700fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.519+0000 7fe675170640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670100480 0x7fe6700fe020 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00d950 tx=0x7fe66c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.520+0000 7fe6667fc640 1 -- 172.21.15.22:0/1252238253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c017070 con 0x7fe670100480 2024-03-08T23:54:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.520+0000 7fe6667fc640 1 -- 172.21.15.22:0/1252238253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66c009c30 con 0x7fe670100480 2024-03-08T23:54:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.520+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6701a6760 con 0x7fe670100480 2024-03-08T23:54:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.520+0000 7fe6667fc640 1 -- 172.21.15.22:0/1252238253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00f6b0 con 0x7fe670100480 2024-03-08T23:54:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.520+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6701a6d00 con 0x7fe670100480 2024-03-08T23:54:57.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe6667fc640 1 -- 172.21.15.22:0/1252238253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7fe66c0109d0 con 0x7fe670100480 2024-03-08T23:54:57.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 Processor -- start 2024-03-08T23:54:57.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 -- start start 2024-03-08T23:54:57.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe6667fc640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe65c0a2ab0 0x7fe65c0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44255b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44256080 0x55bb4425e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb44261050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb441b8e00 con 0x55bb4425ec60 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb44161b10 con 0x55bb44256080 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe679fbb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb441b4ea0 con 0x55bb44255760 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.522+0000 7fe6667fc640 1 -- 172.21.15.22:0/1252238253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fe66c0cf030 con 0x7fe670100480 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe65c0a2ab0 0x7fe65c0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb44261050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675170640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44255b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb44261050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39002/0 (socket says 172.21.15.22:39002) 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 -- 172.21.15.22:0/737972435 learned_addr learned my addr 172.21.15.22:0/737972435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe67496f640 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44256080 0x55bb4425e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 -- 172.21.15.22:0/737972435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 msgr2=0x55bb44255b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44255b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 -- 172.21.15.22:0/737972435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44256080 msgr2=0x55bb4425e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44256080 0x55bb4425e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 -- 172.21.15.22:0/737972435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb43f977d0 con 0x55bb4425ec60 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe65c0a2ab0 0x7fe65c0a4f70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe668032530 tx=0x7fe668009880 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe675971640 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb44261050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe668036000 tx=0x7fe668046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2861370d6f13af server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe65b7fe640 1 -- 172.21.15.22:0/737972435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668032a20 con 0x55bb4425ec60 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe65b7fe640 1 -- 172.21.15.22:0/737972435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe668051070 con 0x55bb4425ec60 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.523+0000 7fe65b7fe640 1 -- 172.21.15.22:0/737972435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668004770 con 0x55bb4425ec60 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 -- 172.21.15.22:0/737972435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 msgr2=0x55bb44261050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb44261050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe668036000 tx=0x7fe668046000 comp rx=0 tx=0).stop 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 -- 172.21.15.22:0/737972435 shutdown_connections 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb44261050 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44256080 0x55bb4425e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 --2- 172.21.15.22:0/737972435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44255b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 -- 172.21.15.22:0/737972435 >> 172.21.15.22:0/737972435 conn(0x55bb4424fe90 msgr2=0x55bb44250270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 -- 172.21.15.22:0/737972435 shutdown_connections 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 -- 172.21.15.22:0/737972435 wait complete. 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 -- start start 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.524+0000 7fe679fbb740 1 Processor -- start 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44255760 0x55bb4425e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44256080 0x55bb4425b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb4425bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe679fbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb442619f0 con 0x55bb4425ec60 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe679fbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb441750a0 con 0x55bb44255760 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe679fbb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb44261b70 con 0x55bb44256080 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe679fbb740 10 client.0 _ll_register_callbacks cb 0x55bb44174a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb4425bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb4425bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39016/0 (socket says 172.21.15.22:39016) 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 -- 172.21.15.22:0/3691052635 learned_addr learned my addr 172.21.15.22:0/3691052635 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe67496f640 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44256080 0x55bb4425b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675170640 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44255760 0x55bb4425e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 -- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44256080 msgr2=0x55bb4425b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44256080 0x55bb4425b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 -- 172.21.15.22:0/3691052635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44255760 msgr2=0x55bb4425e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44255760 0x55bb4425e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.525+0000 7fe675971640 1 -- 172.21.15.22:0/3691052635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb43ecd3f0 con 0x55bb4425ec60 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe675971640 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb4425bcc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe66802c130 tx=0x7fe668032ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c86fc99324c3b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe658ff9640 1 -- 172.21.15.22:0/3691052635 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668004040 con 0x55bb4425ec60 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe658ff9640 1 -- 172.21.15.22:0/3691052635 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fe668003970 con 0x55bb4425ec60 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe668051070 con 0x55bb4425ec60 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe658ff9640 1 -- 172.21.15.22:0/3691052635 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe668003b70 con 0x55bb4425ec60 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 10 client.15864 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb43e8d0f0 con 0x55bb4425ec60 2024-03-08T23:54:57.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe658ff9640 1 -- 172.21.15.22:0/3691052635 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe668051070 con 0x55bb4425ec60 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe658ff9640 1 -- 172.21.15.22:0/3691052635 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe66804b070 con 0x55bb4425ec60 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 20 client.15864 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb4426e5f0 0x55bb442709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 4 client.15864 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.526+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb441c6340 con 0x55bb4426e5f0 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.527+0000 7fe675170640 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb4426e5f0 0x55bb442709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.527+0000 7fe675170640 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb4426e5f0 0x55bb442709e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101f5d94b5f45c9d server_cookie=a22c05696780fee3 in_seq=0 out_seq=0 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.527+0000 7fe658ff9640 10 client.15864 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.527+0000 7fe658ff9640 1 -- 172.21.15.22:0/3691052635 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb441c6340 con 0x55bb4426e5f0 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.527+0000 7fe658ff9640 10 client.15864 handle_command_reply: tid=0 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.527+0000 7fe679fbb740 1 client.15864 shutdown 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 msgr2=0x55bb4425bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb4425bcc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe66802c130 tx=0x7fe668032ad0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 shutdown_connections 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb4426e5f0 0x55bb442709e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb4425ec60 0x55bb4425bcc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe658ff9640 0 client.15864 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44256080 0x55bb4425b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3691052635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb44255760 0x55bb4425e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 >> 172.21.15.22:0/3691052635 conn(0x55bb4424fe90 msgr2=0x55bb44251be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 shutdown_connections 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 1 -- 172.21.15.22:0/3691052635 wait complete. 2024-03-08T23:54:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.528+0000 7fe679fbb740 20 client.15864 trim_cache size 0 max 16384 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 Processor -- start 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 -- start start 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb4425ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44258f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701b2c50 con 0x55bb44256080 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701b07f0 con 0x55bb4425ec60 2024-03-08T23:54:57.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe679fbb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701ae890 con 0x55bb44255760 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39018/0 (socket says 172.21.15.22:39018) 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 -- 172.21.15.22:0/4143627399 learned_addr learned my addr 172.21.15.22:0/4143627399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe675170640 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb4425ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 -- 172.21.15.22:0/4143627399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 msgr2=0x55bb4425ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb4425ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 -- 172.21.15.22:0/4143627399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 msgr2=0x55bb44258f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe675971640 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44258f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44258f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.543+0000 7fe67496f640 1 -- 172.21.15.22:0/4143627399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb4424f670 con 0x55bb44256080 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe67496f640 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442589e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe66000bc20 tx=0x7fe660012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704dc421c1d9d345 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe65affd640 1 -- 172.21.15.22:0/4143627399 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000bd10 con 0x55bb44256080 2024-03-08T23:54:57.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe65affd640 1 -- 172.21.15.22:0/4143627399 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe660011920 con 0x55bb44256080 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe65affd640 1 -- 172.21.15.22:0/4143627399 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000f850 con 0x55bb44256080 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 -- 172.21.15.22:0/4143627399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 msgr2=0x55bb442589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442589e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe66000bc20 tx=0x7fe660012820 comp rx=0 tx=0).stop 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 -- 172.21.15.22:0/4143627399 shutdown_connections 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44258f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442589e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 --2- 172.21.15.22:0/4143627399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb4425ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 -- 172.21.15.22:0/4143627399 >> 172.21.15.22:0/4143627399 conn(0x55bb4424fe90 msgr2=0x55bb44275000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 -- 172.21.15.22:0/4143627399 shutdown_connections 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.544+0000 7fe679fbb740 1 -- 172.21.15.22:0/4143627399 wait complete. 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 -- start start 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 Processor -- start 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44265da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44264320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701aee90 con 0x55bb44256080 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701b0190 con 0x55bb4425ec60 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701af4d0 con 0x55bb44255760 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe679fbb740 10 client.0 _ll_register_callbacks cb 0x7fe66c0cc1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe67496f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe67496f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442662e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39032/0 (socket says 172.21.15.22:39032) 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe67496f640 1 -- 172.21.15.22:0/3223870276 learned_addr learned my addr 172.21.15.22:0/3223870276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe675170640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44265da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.545+0000 7fe675971640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44264320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe67496f640 1 -- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 msgr2=0x55bb44265da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe67496f640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44265da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe67496f640 1 -- 172.21.15.22:0/3223870276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 msgr2=0x55bb44264320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:54:57.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe67496f640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44264320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe67496f640 1 -- 172.21.15.22:0/3223870276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb44264b50 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe675971640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44264320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe67496f640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442662e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe660005c60 tx=0x7fe6600116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653a58021de826a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe658ff9640 1 -- 172.21.15.22:0/3223870276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660021040 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe658ff9640 1 -- 172.21.15.22:0/3223870276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fe660011ad0 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe660011920 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe658ff9640 1 -- 172.21.15.22:0/3223870276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe660033050 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 10 client.15870 fetch_fsmap learned FSMap version 25 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb442c5400 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe658ff9640 1 -- 172.21.15.22:0/3223870276 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe660011920 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe658ff9640 1 -- 172.21.15.22:0/3223870276 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe660024070 con 0x55bb44256080 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 10 client.15870 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:54:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 20 client.15870 populate_metadata read hostname 'smithi022' 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb442c5a60 0x55bb442c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 4 client.15870 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.546+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb441c0690 con 0x55bb442c5a60 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe675170640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb442c5a60 0x55bb442c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe675170640 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb442c5a60 0x55bb442c7e80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f37b7e2e936af11 server_cookie=466a8f63622e6a71 in_seq=0 out_seq=0 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe658ff9640 10 client.15870 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe658ff9640 1 -- 172.21.15.22:0/3223870276 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bb441c0690 con 0x55bb442c5a60 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe658ff9640 10 client.15870 handle_command_reply: tid=0 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 client.15870 shutdown 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 msgr2=0x55bb442662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442662e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe660005c60 tx=0x7fe6600116f0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 shutdown_connections 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb442c5a60 0x55bb442c7e80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb4425ec60 0x55bb44264320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb44256080 0x55bb442662e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 --2- 172.21.15.22:0/3223870276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb44255760 0x55bb44265da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe658ff9640 0 client.15870 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 >> 172.21.15.22:0/3223870276 conn(0x55bb4424fe90 msgr2=0x55bb4424e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 shutdown_connections 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.547+0000 7fe679fbb740 1 -- 172.21.15.22:0/3223870276 wait complete. 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe679fbb740 20 client.15870 trim_cache size 0 max 16384 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:54:57.551 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:54:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "32cf56a9-453b-4303-972a-f2ad6c3330fd": { 2024-03-08T23:54:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:54:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "32cf56a9-453b-4303-972a-f2ad6c3330fd", 2024-03-08T23:54:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:54:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:54:57.552 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:54:57.552 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe65c0a2ab0 msgr2=0x7fe65c0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe65c0a2ab0 0x7fe65c0a4f70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe668032530 tx=0x7fe668009880 comp rx=0 tx=0).stop 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670100480 msgr2=0x7fe6700fe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670100480 0x7fe6700fe020 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00d950 tx=0x7fe66c00de10 comp rx=0 tx=0).stop 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 shutdown_connections 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe65c0a2ab0 0x7fe65c0a4f70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6700feaa0 0x7fe6701a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe670105550 0x7fe6700fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 --2- 172.21.15.22:0/1252238253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe670100480 0x7fe6700fe020 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 >> 172.21.15.22:0/1252238253 conn(0x7fe670077350 msgr2=0x7fe670069b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:54:57.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 shutdown_connections 2024-03-08T23:54:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:54:57.548+0000 7fe6773d4640 1 -- 172.21.15.22:0/1252238253 wait complete. 2024-03-08T23:54:57.555 INFO:tasks.cephfs.filesystem:scrub status for tag:32cf56a9-453b-4303-972a-f2ad6c3330fd - {'path': '/', 'tag': '32cf56a9-453b-4303-972a-f2ad6c3330fd', 'options': 'recursive,force'} 2024-03-08T23:54:58.163 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:57 smithi062 ceph-mon[36129]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-03-08T23:54:58.164 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:57 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:54:58.164 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:57 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:58.164 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:57 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:54:58.164 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:57 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:58.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:57 smithi022 ceph-mon[31894]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-03-08T23:54:58.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:57 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:54:58.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:57 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:58.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:57 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:54:58.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:57 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:58.378 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:57 smithi192 ceph-mon[35995]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-03-08T23:54:58.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:57 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:54:58.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:57 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:54:58.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:57 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:54:58.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:57 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:55:00.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:59 smithi022 ceph-mon[31894]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-08T23:55:00.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:54:59 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:55:00.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:59 smithi062 ceph-mon[36129]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-08T23:55:00.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:54:59 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:55:00.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:59 smithi192 ceph-mon[35995]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-08T23:55:00.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:54:59 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:55:00.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:55:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:55:00.366+0000 7f1162696700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:55:00.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:55:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:55:00.366+0000 7f1162696700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:55:00.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:55:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:55:00.366+0000 7f1162696700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:55:00.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:55:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:55:00.366+0000 7f1162696700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:55:00.735 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 08 23:55:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-08T23:55:00.366+0000 7f1162696700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T23:55:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 -- 172.21.15.22:0/2743654265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 msgr2=0x7f041c105c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 --2- 172.21.15.22:0/2743654265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c105c10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0414009cd0 tx=0x7f041402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 -- 172.21.15.22:0/2743654265 shutdown_connections 2024-03-08T23:55:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 --2- 172.21.15.22:0/2743654265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c105c10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 --2- 172.21.15.22:0/2743654265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f041c100be0 0x7f041c1050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 --2- 172.21.15.22:0/2743654265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c1002c0 0x7f041c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 -- 172.21.15.22:0/2743654265 >> 172.21.15.22:0/2743654265 conn(0x7f041c077320 msgr2=0x7f041c077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 -- 172.21.15.22:0/2743654265 shutdown_connections 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 -- 172.21.15.22:0/2743654265 wait complete. 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.421+0000 7f0423f74640 1 Processor -- start 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0423f74640 1 -- start start 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0423f74640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c1002c0 0x7f041c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0423f74640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f041c100be0 0x7f041c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0423f74640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c1a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0423f74640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c117d30 con 0x7f041c105760 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0423f74640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c117bb0 con 0x7f041c1002c0 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0423f74640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c117eb0 con 0x7f041c100be0 2024-03-08T23:55:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c1a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c1a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57542/0 (socket says 172.21.15.22:57542) 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 -- 172.21.15.22:0/1257230412 learned_addr learned my addr 172.21.15.22:0/1257230412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f042150f640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f041c100be0 0x7f041c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0421d10640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c1002c0 0x7f041c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 -- 172.21.15.22:0/1257230412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f041c100be0 msgr2=0x7f041c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f041c100be0 0x7f041c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 -- 172.21.15.22:0/1257230412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c1002c0 msgr2=0x7f041c19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c1002c0 0x7f041c19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 -- 172.21.15.22:0/1257230412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c1a3080 con 0x7f041c105760 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0421d10640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c1002c0 0x7f041c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.422+0000 7f0422511640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c1a2950 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f04140348f0 tx=0x7f0414032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.423+0000 7f0412ffd640 1 -- 172.21.15.22:0/1257230412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414041020 con 0x7f041c105760 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.423+0000 7f0412ffd640 1 -- 172.21.15.22:0/1257230412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0414034cd0 con 0x7f041c105760 2024-03-08T23:55:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.423+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c1a3310 con 0x7f041c105760 2024-03-08T23:55:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.423+0000 7f0412ffd640 1 -- 172.21.15.22:0/1257230412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414039040 con 0x7f041c105760 2024-03-08T23:55:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.423+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c1aac00 con 0x7f041c105760 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0412ffd640 1 -- 172.21.15.22:0/1257230412 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 25) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f0414044020 con 0x7f041c105760 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 Processor -- start 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 -- start start 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f47a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f48450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 0x55c629f53940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c629eac5f0 con 0x55c629f48d40 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c629e55930 con 0x55c629f47fd0 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0426c2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c629ea8690 con 0x55c629f51550 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f042150f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f47a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.426+0000 7f0412ffd640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04080a2ab0 0x7f04080a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f47a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57550/0 (socket says 172.21.15.22:57550) 2024-03-08T23:55:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 -- 172.21.15.22:0/2129895451 learned_addr learned my addr 172.21.15.22:0/2129895451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f0421d10640 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f48450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f0422511640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04080a2ab0 0x7f04080a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 0x55c629f53940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f0412ffd640 1 -- 172.21.15.22:0/1257230412 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f04140f1140 con 0x7f041c105760 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 -- 172.21.15.22:0/2129895451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 msgr2=0x55c629f53940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 0x55c629f53940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 -- 172.21.15.22:0/2129895451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 msgr2=0x55c629f48450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f48450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 -- 172.21.15.22:0/2129895451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c629bc1210 con 0x55c629f48d40 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f042150f640 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f47a90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f040c012530 tx=0x7f040c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e30407dcb1aaae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f0422511640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04080a2ab0 0x7f04080a4f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f04140ee5a0 tx=0x7f041402e040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f0407fff640 1 -- 172.21.15.22:0/2129895451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c005020 con 0x55c629f48d40 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f0407fff640 1 -- 172.21.15.22:0/2129895451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f040c0056a0 con 0x55c629f48d40 2024-03-08T23:55:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.427+0000 7f0407fff640 1 -- 172.21.15.22:0/2129895451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c010660 con 0x55c629f48d40 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 -- 172.21.15.22:0/2129895451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 msgr2=0x55c629f47a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f47a90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f040c012530 tx=0x7f040c00b650 comp rx=0 tx=0).stop 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 -- 172.21.15.22:0/2129895451 shutdown_connections 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 0x55c629f53940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f48450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2129895451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f47a90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 -- 172.21.15.22:0/2129895451 >> 172.21.15.22:0/2129895451 conn(0x55c629f0f5b0 msgr2=0x55c629f42cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 -- 172.21.15.22:0/2129895451 shutdown_connections 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.428+0000 7f0426c2c740 1 -- 172.21.15.22:0/2129895451 wait complete. 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 -- start start 2024-03-08T23:55:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 Processor -- start 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f4e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c629f54280 con 0x55c629f51550 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c629e68d90 con 0x55c629f48d40 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c629f54400 con 0x55c629f47fd0 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0426c2c740 10 client.0 _ll_register_callbacks cb 0x55c629bc0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f042150f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f042150f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57564/0 (socket says 172.21.15.22:57564) 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f042150f640 1 -- 172.21.15.22:0/489142321 learned_addr learned my addr 172.21.15.22:0/489142321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.429+0000 7f0421d10640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f042150f640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f4e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f042150f640 1 -- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 msgr2=0x55c629f4e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f042150f640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f4e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f042150f640 1 -- 172.21.15.22:0/489142321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 msgr2=0x55c629f4eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f042150f640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f042150f640 1 -- 172.21.15.22:0/489142321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c629c8d620 con 0x55c629f51550 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f0421d10640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f042150f640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4efe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f040c0122a0 tx=0x7f040c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8bccd09e39f177 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f04057fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f04057fa640 1 -- 172.21.15.22:0/489142321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c004f40 con 0x55c629f51550 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f04057fa640 1 -- 172.21.15.22:0/489142321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f040c0050e0 con 0x55c629f51550 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f040c0056a0 con 0x55c629f51550 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f04057fa640 1 -- 172.21.15.22:0/489142321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f040c0052e0 con 0x55c629f51550 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f0426c2c740 10 client.15888 fetch_fsmap learned FSMap version 25 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c629f60570 con 0x55c629f51550 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.430+0000 7f04057fa640 1 -- 172.21.15.22:0/489142321 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f040c0056a0 con 0x55c629f51550 2024-03-08T23:55:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f04057fa640 1 -- 172.21.15.22:0/489142321 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f040c00e490 con 0x55c629f51550 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0426c2c740 10 client.15888 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0426c2c740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0426c2c740 20 client.15888 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0426c2c740 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629f618c0 0x55c629f63cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0426c2c740 4 client.15888 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c629eb9b30 con 0x55c629f618c0 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0421d10640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629f618c0 0x55c629f63cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f0421d10640 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629f618c0 0x55c629f63cb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecb7fcbc93c2c76 server_cookie=c6effd03cfdb4a0f in_seq=0 out_seq=0 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.431+0000 7f04057fa640 10 client.15888 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f04057fa640 1 -- 172.21.15.22:0/489142321 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c629eb9b30 con 0x55c629f618c0 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f04057fa640 10 client.15888 handle_command_reply: tid=0 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 client.15888 shutdown 2024-03-08T23:55:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 msgr2=0x55c629f4efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4efe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f040c0122a0 tx=0x7f040c00b410 comp rx=0 tx=0).stop 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 shutdown_connections 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629f618c0 0x55c629f63cb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4efe0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f04057fa640 0 client.15888 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 --2- 172.21.15.22:0/489142321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f4e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 >> 172.21.15.22:0/489142321 conn(0x55c629f0f5b0 msgr2=0x55c629f44a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 shutdown_connections 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 1 -- 172.21.15.22:0/489142321 wait complete. 2024-03-08T23:55:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.432+0000 7f0426c2c740 20 client.15888 trim_cache size 0 max 16384 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 Processor -- start 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 -- start start 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f50c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1b6f90 con 0x55c629f51550 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1b4b30 con 0x55c629f48d40 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0426c2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1b2bd0 con 0x55c629f47fd0 2024-03-08T23:55:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57578/0 (socket says 172.21.15.22:57578) 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 -- 172.21.15.22:0/2717944116 learned_addr learned my addr 172.21.15.22:0/2717944116 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0421d10640 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f50c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 -- 172.21.15.22:0/2717944116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 msgr2=0x55c629f50c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f50c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 -- 172.21.15.22:0/2717944116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 msgr2=0x55c629f4ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 -- 172.21.15.22:0/2717944116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c629f472a0 con 0x55c629f51550 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f0421d10640 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.446+0000 7f042150f640 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4d370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e930 tx=0x7f040c00b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744e7be048156ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f04077fe640 1 -- 172.21.15.22:0/2717944116 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00b8c0 con 0x55c629f51550 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f04077fe640 1 -- 172.21.15.22:0/2717944116 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f040c004cb0 con 0x55c629f51550 2024-03-08T23:55:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f04077fe640 1 -- 172.21.15.22:0/2717944116 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c010460 con 0x55c629f51550 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 -- 172.21.15.22:0/2717944116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 msgr2=0x55c629f4d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4d370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e930 tx=0x7f040c00b520 comp rx=0 tx=0).stop 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 -- 172.21.15.22:0/2717944116 shutdown_connections 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f51550 0x55c629f4d370 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f48d40 0x55c629f4ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2717944116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f47fd0 0x55c629f50c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 -- 172.21.15.22:0/2717944116 >> 172.21.15.22:0/2717944116 conn(0x55c629f0f5b0 msgr2=0x55c629f49870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 -- 172.21.15.22:0/2717944116 shutdown_connections 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.447+0000 7f0426c2c740 1 -- 172.21.15.22:0/2717944116 wait complete. 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 -- start start 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 Processor -- start 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f56fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 0x55c629f57a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1b31d0 con 0x55c629f48d40 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1b44d0 con 0x55c629f47fd0 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1b3810 con 0x55c629f51550 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0426c2c740 10 client.0 _ll_register_callbacks cb 0x7f040c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f57500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57594/0 (socket says 172.21.15.22:57594) 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 -- 172.21.15.22:0/2427669798 learned_addr learned my addr 172.21.15.22:0/2427669798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f042150f640 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f56fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 -- 172.21.15.22:0/2427669798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 msgr2=0x55c629f57a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 0x55c629f57a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 -- 172.21.15.22:0/2427669798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 msgr2=0x55c629f56fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f56fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.448+0000 7f0421d10640 1 -- 172.21.15.22:0/2427669798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c629fb7cd0 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0421d10640 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f57500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55c629f494a0 tx=0x7f0418011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a876bab373ea169 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f04057fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f04057fa640 1 -- 172.21.15.22:0/2427669798 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418004440 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f04057fa640 1 -- 172.21.15.22:0/2427669798 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f041801c310 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f040c004cb0 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f04057fa640 1 -- 172.21.15.22:0/2427669798 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f041802d050 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 10 client.15894 fetch_fsmap learned FSMap version 25 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c629fb8f50 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f04057fa640 1 -- 172.21.15.22:0/2427669798 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f040c004cb0 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f04057fa640 1 -- 172.21.15.22:0/2427669798 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0418025420 con 0x55c629f48d40 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 10 client.15894 fetch_fsmap finished waiting for FSMap version 25 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 20 client.15894 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629fb9700 0x55c629fbbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 4 client.15894 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:55:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.449+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c629f47cc0 con 0x55c629fb9700 2024-03-08T23:55:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.450+0000 7f042150f640 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629fb9700 0x55c629fbbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.450+0000 7f04057fa640 10 client.15894 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.450+0000 7f042150f640 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629fb9700 0x55c629fbbb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ea49971762ec73 server_cookie=46d427b909a875ba in_seq=0 out_seq=0 2024-03-08T23:55:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:01.930+0000 7f0412ffd640 1 -- 172.21.15.22:0/1257230412 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f0414003970 con 0x7f041c105760 2024-03-08T23:55:02.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:01 smithi022 ceph-mon[31894]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-08T23:55:02.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:01 smithi062 ceph-mon[36129]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-08T23:55:02.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:01 smithi192 ceph-mon[35995]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-08T23:55:03.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:02 smithi022 ceph-mon[31894]: mgrmap e26: x(active, since 29s), standbys: z, y 2024-03-08T23:55:03.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:02 smithi062 ceph-mon[36129]: mgrmap e26: x(active, since 29s), standbys: z, y 2024-03-08T23:55:03.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:02 smithi192 ceph-mon[35995]: mgrmap e26: x(active, since 29s), standbys: z, y 2024-03-08T23:55:04.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:03 smithi022 ceph-mon[31894]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-08T23:55:04.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:03 smithi062 ceph-mon[36129]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-08T23:55:04.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:03 smithi192 ceph-mon[35995]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f04057fa640 1 -- 172.21.15.22:0/2427669798 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c629f47cc0 con 0x55c629fb9700 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f04057fa640 10 client.15894 handle_command_reply: tid=0 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 client.15894 shutdown 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 msgr2=0x55c629f57500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f57500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55c629f494a0 tx=0x7f0418011810 comp rx=0 tx=0).stop 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 shutdown_connections 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c629fb9700 0x55c629fbbb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c629f51550 0x55c629f57a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c629f48d40 0x55c629f57500 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f04057fa640 0 client.15894 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 --2- 172.21.15.22:0/2427669798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c629f47fd0 0x55c629f56fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 >> 172.21.15.22:0/2427669798 conn(0x55c629f0f5b0 msgr2=0x55c629f4ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 shutdown_connections 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 1 -- 172.21.15.22:0/2427669798 wait complete. 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.566+0000 7f0426c2c740 20 client.15894 trim_cache size 0 max 16384 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:55:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:55:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04080a2ab0 msgr2=0x7f04080a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04080a2ab0 0x7f04080a4f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f04140ee5a0 tx=0x7f041402e040 comp rx=0 tx=0).stop 2024-03-08T23:55:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 msgr2=0x7f041c1a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c1a2950 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f04140348f0 tx=0x7f0414032ec0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 shutdown_connections 2024-03-08T23:55:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04080a2ab0 0x7f04080a4f70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f041c105760 0x7f041c1a2950 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f041c100be0 0x7f041c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 --2- 172.21.15.22:0/1257230412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c1002c0 0x7f041c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 >> 172.21.15.22:0/1257230412 conn(0x7f041c077320 msgr2=0x7f041c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 shutdown_connections 2024-03-08T23:55:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:04.567+0000 7f0423f74640 1 -- 172.21.15.22:0/1257230412 wait complete. 2024-03-08T23:55:04.976 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: *** Caught signal (Segmentation fault) ** 2024-03-08T23:55:04.976 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: in thread 7fc22c015700 thread_name:md_log_replay 2024-03-08T23:55:04.976 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T23:55:04.976 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc23d3d6d20] 2024-03-08T23:55:04.976 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55fca53e5ee2] 2024-03-08T23:55:04.976 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fca53e9773] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 4: (MDLog::_replay_thread()+0x617) [0x55fca5445587] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fca544e091] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc23d3cc1ca] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 7: clone() 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2024-03-08T23:55:04.617+0000 7fc22c015700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: in thread 7fc22c015700 thread_name:md_log_replay 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc23d3d6d20] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55fca53e5ee2] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fca53e9773] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 4: (MDLog::_replay_thread()+0x617) [0x55fca5445587] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fca544e091] 2024-03-08T23:55:04.977 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc23d3cc1ca] 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 7: clone() 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 0> 2024-03-08T23:55:04.617+0000 7fc22c015700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: in thread 7fc22c015700 thread_name:md_log_replay 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc23d3d6d20] 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55fca53e5ee2] 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fca53e9773] 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 4: (MDLog::_replay_thread()+0x617) [0x55fca5445587] 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fca544e091] 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc23d3cc1ca] 2024-03-08T23:55:04.978 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 7: clone() 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 0> 2024-03-08T23:55:04.617+0000 7fc22c015700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: in thread 7fc22c015700 thread_name:md_log_replay 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc23d3d6d20] 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55fca53e5ee2] 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fca53e9773] 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 4: (MDLog::_replay_thread()+0x617) [0x55fca5445587] 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fca544e091] 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc23d3cc1ca] 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 7: clone() 2024-03-08T23:55:04.979 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T23:55:04.980 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[66305]: 2024-03-08T23:55:04.980 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:04 smithi062 podman[69984]: 2024-03-08 23:55:04.679178846 +0000 UTC m=+0.022824013 container died 12de8012086c79af9fc2f973552e1d1e0e718e1061a2ce1c633031345414ba07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-08T23:55:05.606 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:05 smithi062 podman[69984]: 2024-03-08 23:55:05.355142091 +0000 UTC m=+0.698787247 container remove 12de8012086c79af9fc2f973552e1d1e0e718e1061a2ce1c633031345414ba07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-08T23:55:05.606 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:05 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-03-08T23:55:05.861 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:05 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.h.service: Failed with result 'exit-code'. 2024-03-08T23:55:06.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:05 smithi022 ceph-mon[31894]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-08T23:55:06.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:05 smithi062 ceph-mon[36129]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-08T23:55:06.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:05 smithi192 ceph-mon[35995]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-08T23:55:08.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:07 smithi022 ceph-mon[31894]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-08T23:55:08.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:07 smithi062 ceph-mon[36129]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-08T23:55:08.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:07 smithi192 ceph-mon[35995]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-08T23:55:10.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:09 smithi022 ceph-mon[31894]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-03-08T23:55:10.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:09 smithi062 ceph-mon[36129]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-03-08T23:55:10.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:09 smithi192 ceph-mon[35995]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-03-08T23:55:12.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:11 smithi022 ceph-mon[31894]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-03-08T23:55:12.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:11 smithi062 ceph-mon[36129]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-03-08T23:55:12.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:11 smithi192 ceph-mon[35995]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-03-08T23:55:14.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:13 smithi022 ceph-mon[31894]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-03-08T23:55:14.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:13 smithi062 ceph-mon[36129]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-03-08T23:55:14.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:13 smithi192 ceph-mon[35995]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-03-08T23:55:15.958 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:15 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-03-08T23:55:15.958 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:15 smithi062 systemd[1]: Stopped Ceph mds.h for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:55:15.958 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:15 smithi062 systemd[1]: Starting Ceph mds.h for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-08T23:55:15.958 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:15 smithi062 podman[70076]: 2024-03-08 23:55:15.757596978 +0000 UTC m=+0.018712516 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T23:55:15.959 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:15 smithi062 podman[70076]: 2024-03-08 23:55:15.864398354 +0000 UTC m=+0.125513876 container create cb149f6106b746f619e02354969149b60a35442c50d6161c03010bdff4768b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux ) 2024-03-08T23:55:16.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:15 smithi022 ceph-mon[31894]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-08T23:55:16.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:15 smithi062 ceph-mon[36129]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-08T23:55:16.361 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:16 smithi062 podman[70076]: 2024-03-08 23:55:16.108897239 +0000 UTC m=+0.370012775 container init cb149f6106b746f619e02354969149b60a35442c50d6161c03010bdff4768b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-08T23:55:16.362 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:16 smithi062 podman[70076]: 2024-03-08 23:55:16.111679059 +0000 UTC m=+0.372794590 container start cb149f6106b746f619e02354969149b60a35442c50d6161c03010bdff4768b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-08T23:55:16.362 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:16 smithi062 ceph-mds[70095]: -- 172.21.15.62:0/2008355923 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563de6b3ba40 con 0x563de6b0e800 2024-03-08T23:55:16.362 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:16 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: starting mds.h at 2024-03-08T23:55:16.362 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:16 smithi062 bash[70076]: cb149f6106b746f619e02354969149b60a35442c50d6161c03010bdff4768b28 2024-03-08T23:55:16.362 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:55:16 smithi062 systemd[1]: Started Ceph mds.h for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-08T23:55:16.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:15 smithi192 ceph-mon[35995]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-08T23:55:17.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:16 smithi022 ceph-mon[31894]: daemon mds.h restarted 2024-03-08T23:55:17.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:16 smithi062 ceph-mon[36129]: daemon mds.h restarted 2024-03-08T23:55:17.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:16 smithi192 ceph-mon[35995]: daemon mds.h restarted 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 -- 172.21.15.22:0/2600479030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 msgr2=0x7f43c41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 --2- 172.21.15.22:0/2600479030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c41077e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b402bc30 comp rx=0 tx=0).stop 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 -- 172.21.15.22:0/2600479030 shutdown_connections 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 --2- 172.21.15.22:0/2600479030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c4069310 0x7f43c410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 --2- 172.21.15.22:0/2600479030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c4068970 0x7f43c4068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 --2- 172.21.15.22:0/2600479030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c41077e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 -- 172.21.15.22:0/2600479030 >> 172.21.15.22:0/2600479030 conn(0x7f43c4077040 msgr2=0x7f43c4077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 -- 172.21.15.22:0/2600479030 shutdown_connections 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.662+0000 7f43c9673640 1 -- 172.21.15.22:0/2600479030 wait complete. 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 Processor -- start 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 -- start start 2024-03-08T23:55:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c4068970 0x7f43c4100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c4069310 0x7f43c4100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c4100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4111640 con 0x7f43c4107400 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41114c0 con 0x7f43c4069310 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c9673640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41117c0 con 0x7f43c4068970 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c4100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c4100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36666/0 (socket says 172.21.15.22:36666) 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.663+0000 7f43c37fe640 1 -- 172.21.15.22:0/2216217872 learned_addr learned my addr 172.21.15.22:0/2216217872 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c4069310 0x7f43c4100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c4068970 0x7f43c4100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c37fe640 1 -- 172.21.15.22:0/2216217872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c4068970 msgr2=0x7f43c4100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c37fe640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c4068970 0x7f43c4100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c37fe640 1 -- 172.21.15.22:0/2216217872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c4069310 msgr2=0x7f43c4100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c37fe640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c4069310 0x7f43c4100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c37fe640 1 -- 172.21.15.22:0/2216217872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c41a5f60 con 0x7f43c4107400 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c4068970 0x7f43c4100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c37fe640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c4100f90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00ea60 tx=0x7f43bc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43abfff640 1 -- 172.21.15.22:0/2216217872 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00ce40 con 0x7f43c4107400 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43abfff640 1 -- 172.21.15.22:0/2216217872 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43bc005020 con 0x7f43c4107400 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c41a6250 con 0x7f43c4107400 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43abfff640 1 -- 172.21.15.22:0/2216217872 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc010690 con 0x7f43c4107400 2024-03-08T23:55:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.664+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c41a6750 con 0x7f43c4107400 2024-03-08T23:55:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43abfff640 1 -- 172.21.15.22:0/2216217872 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f43bc005520 con 0x7f43c4107400 2024-03-08T23:55:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 Processor -- start 2024-03-08T23:55:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 -- start start 2024-03-08T23:55:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43abfff640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43ac0a2b30 0x7f43ac0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb07450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb12940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddca6b5f0 con 0x55dddcb06fd0 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddca14930 con 0x55dddcb07d40 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43cc252740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddca67690 con 0x55dddcb10550 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43abfff640 1 -- 172.21.15.22:0/2216217872 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f43bc0cf9c0 con 0x7f43c4107400 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43ac0a2b30 0x7f43ac0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb07450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb07450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36676/0 (socket says 172.21.15.22:36676) 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.667+0000 7f43c27fc640 1 -- 172.21.15.22:0/1808237927 learned_addr learned my addr 172.21.15.22:0/1808237927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c37fe640 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb12940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c27fc640 1 -- 172.21.15.22:0/1808237927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 msgr2=0x55dddcb12940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c27fc640 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb12940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c27fc640 1 -- 172.21.15.22:0/1808237927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 msgr2=0x55dddcb06a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c27fc640 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb06a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c27fc640 1 -- 172.21.15.22:0/1808237927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dddc780210 con 0x55dddcb06fd0 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43ac0a2b30 0x7f43ac0a4ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43b8012980 tx=0x7f43b800bb80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43c27fc640 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb07450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f43b8010740 tx=0x7f43b801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffba1a35a7390e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43a97fa640 1 -- 172.21.15.22:0/1808237927 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800b710 con 0x55dddcb06fd0 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43a97fa640 1 -- 172.21.15.22:0/1808237927 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b8025070 con 0x55dddcb06fd0 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.668+0000 7f43a97fa640 1 -- 172.21.15.22:0/1808237927 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8020690 con 0x55dddcb06fd0 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 -- 172.21.15.22:0/1808237927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 msgr2=0x55dddcb07450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb07450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f43b8010740 tx=0x7f43b801c000 comp rx=0 tx=0).stop 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 -- 172.21.15.22:0/1808237927 shutdown_connections 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb12940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb07450 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 --2- 172.21.15.22:0/1808237927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb06a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 -- 172.21.15.22:0/1808237927 >> 172.21.15.22:0/1808237927 conn(0x55dddcace5b0 msgr2=0x55dddcb01cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 -- 172.21.15.22:0/1808237927 shutdown_connections 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 -- 172.21.15.22:0/1808237927 wait complete. 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 -- start start 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.669+0000 7f43cc252740 1 Processor -- start 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43cc252740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb0d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43cc252740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb0db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43cc252740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43cc252740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddcb13220 con 0x55dddcb06fd0 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43cc252740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddca27d90 con 0x55dddcb07d40 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43cc252740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddcb133a0 con 0x55dddcb10550 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43cc252740 10 client.0 _ll_register_callbacks cb 0x55dddc77fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb0d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb0d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36684/0 (socket says 172.21.15.22:36684) 2024-03-08T23:55:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 -- 172.21.15.22:0/3989250377 learned_addr learned my addr 172.21.15.22:0/3989250377 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c37fe640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c27fc640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb0db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 -- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 msgr2=0x55dddcb0e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 -- 172.21.15.22:0/3989250377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 msgr2=0x55dddcb0db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb0db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 -- 172.21.15.22:0/3989250377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dddc84c620 con 0x55dddcb06fd0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c27fc640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb0d5d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cb81f3ee552045 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.670+0000 7f43a6ff5640 1 -- 172.21.15.22:0/3989250377 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4005030 con 0x55dddcb06fd0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43a6ff5640 1 -- 172.21.15.22:0/3989250377 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f43b40051d0 con 0x55dddcb06fd0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43b8025070 con 0x55dddcb06fd0 2024-03-08T23:55:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43a6ff5640 1 -- 172.21.15.22:0/3989250377 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43b404e050 con 0x55dddcb06fd0 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 10 client.15912 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dddcb1f5a0 con 0x55dddcb06fd0 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43a6ff5640 1 -- 172.21.15.22:0/3989250377 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b8025070 con 0x55dddcb06fd0 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43a6ff5640 1 -- 172.21.15.22:0/3989250377 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f43b40035f0 con 0x55dddcb06fd0 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 20 client.15912 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb20a60 0x55dddcb22e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 4 client.15912 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dddca78b30 con 0x55dddcb20a60 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.671+0000 7f43c27fc640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb20a60 0x55dddcb22e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.672+0000 7f43a6ff5640 10 client.15912 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.672+0000 7f43c27fc640 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb20a60 0x55dddcb22e50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0233f7bd3b0443e server_cookie=d0b9123f09d8e9cc in_seq=0 out_seq=0 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43a6ff5640 1 -- 172.21.15.22:0/3989250377 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dddca78b30 con 0x55dddcb20a60 2024-03-08T23:55:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43a6ff5640 10 client.15912 handle_command_reply: tid=0 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 client.15912 shutdown 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 msgr2=0x55dddcb0d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb0d5d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b402c7e0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 shutdown_connections 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb20a60 0x55dddcb22e50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43a6ff5640 0 client.15912 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb07d40 0x55dddcb0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 --2- 172.21.15.22:0/3989250377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb06fd0 0x55dddcb0d5d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 >> 172.21.15.22:0/3989250377 conn(0x55dddcace5b0 msgr2=0x55dddcb03a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 shutdown_connections 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 1 -- 172.21.15.22:0/3989250377 wait complete. 2024-03-08T23:55:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.673+0000 7f43cc252740 20 client.15912 trim_cache size 0 max 16384 2024-03-08T23:55:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 Processor -- start 2024-03-08T23:55:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 -- start start 2024-03-08T23:55:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb06fd0 0x55dddcb0fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb0bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41b2ba0 con 0x55dddcb07d40 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41b0740 con 0x55dddcb06fd0 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43cc252740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41ae7e0 con 0x55dddcb10550 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb0bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb0bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36698/0 (socket says 172.21.15.22:36698) 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c27fc640 1 -- 172.21.15.22:0/2401067595 learned_addr learned my addr 172.21.15.22:0/2401067595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb06fd0 0x55dddcb0fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c37fe640 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c27fc640 1 -- 172.21.15.22:0/2401067595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 msgr2=0x55dddcb0c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.686+0000 7f43c27fc640 1 -- 172.21.15.22:0/2401067595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb06fd0 msgr2=0x55dddcb0fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb06fd0 0x55dddcb0fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43c27fc640 1 -- 172.21.15.22:0/2401067595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dddcb062a0 con 0x55dddcb07d40 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43c37fe640 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb06fd0 0x55dddcb0fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb0bea0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f43b8010740 tx=0x7f43b8012580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b72d66cb3254ff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43a8ff9640 1 -- 172.21.15.22:0/2401067595 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800d920 con 0x55dddcb07d40 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43a8ff9640 1 -- 172.21.15.22:0/2401067595 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b8025040 con 0x55dddcb07d40 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.687+0000 7f43a8ff9640 1 -- 172.21.15.22:0/2401067595 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8004660 con 0x55dddcb07d40 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- 172.21.15.22:0/2401067595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 msgr2=0x55dddcb0bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb0bea0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f43b8010740 tx=0x7f43b8012580 comp rx=0 tx=0).stop 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- 172.21.15.22:0/2401067595 shutdown_connections 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb10550 0x55dddcb0c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb0bea0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 --2- 172.21.15.22:0/2401067595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb06fd0 0x55dddcb0fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- 172.21.15.22:0/2401067595 >> 172.21.15.22:0/2401067595 conn(0x55dddcace5b0 msgr2=0x55dddcb017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- 172.21.15.22:0/2401067595 shutdown_connections 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- 172.21.15.22:0/2401067595 wait complete. 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- start start 2024-03-08T23:55:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 Processor -- start 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb06fd0 0x55dddcb17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb10550 0x55dddcb166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41aede0 con 0x55dddcb07d40 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41b00e0 con 0x55dddcb10550 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41af420 con 0x55dddcb06fd0 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43cc252740 10 client.0 _ll_register_callbacks cb 0x7f43b80124d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb15f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36700/0 (socket says 172.21.15.22:36700) 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43c27fc640 1 -- 172.21.15.22:0/2573537152 learned_addr learned my addr 172.21.15.22:0/2573537152 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.688+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb06fd0 0x55dddcb17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43c27fc640 1 -- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb06fd0 msgr2=0x55dddcb17bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb06fd0 0x55dddcb17bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43c27fc640 1 -- 172.21.15.22:0/2573537152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb10550 msgr2=0x55dddcb166a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb10550 0x55dddcb166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43c27fc640 1 -- 172.21.15.22:0/2573537152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dddcb76f60 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb06fd0 0x55dddcb17bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43c27fc640 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb15f60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f43b8034000 tx=0x7f43b8010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9d23a2aea9151f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43a6ff5640 1 -- 172.21.15.22:0/2573537152 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b803a070 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43a6ff5640 1 -- 172.21.15.22:0/2573537152 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f43b800db30 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43b8025040 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43a6ff5640 1 -- 172.21.15.22:0/2573537152 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43b8005a70 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43cc252740 10 client.15918 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dddcb78090 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.689+0000 7f43a6ff5640 1 -- 172.21.15.22:0/2573537152 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b8025040 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43a6ff5640 1 -- 172.21.15.22:0/2573537152 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f43b8032070 con 0x55dddcb07d40 2024-03-08T23:55:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43cc252740 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43cc252740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43cc252740 20 client.15918 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43cc252740 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb78610 0x55dddcb7aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43cc252740 4 client.15918 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dddca72e80 con 0x55dddcb78610 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb78610 0x55dddcb7aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43a6ff5640 10 client.15918 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.690+0000 7f43c2ffd640 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb78610 0x55dddcb7aa30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63749dd76ca7aa0 server_cookie=31787bc9742c9aca in_seq=0 out_seq=0 2024-03-08T23:55:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.698+0000 7f43a6ff5640 1 -- 172.21.15.22:0/2573537152 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dddca72e80 con 0x55dddcb78610 2024-03-08T23:55:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.698+0000 7f43a6ff5640 10 client.15918 handle_command_reply: tid=0 2024-03-08T23:55:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 client.15918 shutdown 2024-03-08T23:55:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 msgr2=0x55dddcb15f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb15f60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f43b8034000 tx=0x7f43b8010e80 comp rx=0 tx=0).stop 2024-03-08T23:55:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 shutdown_connections 2024-03-08T23:55:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dddcb78610 0x55dddcb7aa30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dddcb10550 0x55dddcb166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dddcb07d40 0x55dddcb15f60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 --2- 172.21.15.22:0/2573537152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dddcb06fd0 0x55dddcb17bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43a6ff5640 0 client.15918 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 >> 172.21.15.22:0/2573537152 conn(0x55dddcace5b0 msgr2=0x55dddcb0ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 shutdown_connections 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 1 -- 172.21.15.22:0/2573537152 wait complete. 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43cc252740 20 client.15918 trim_cache size 0 max 16384 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:55:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:55:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:55:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43ac0a2b30 msgr2=0x7f43ac0a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43ac0a2b30 0x7f43ac0a4ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43b8012980 tx=0x7f43b800bb80 comp rx=0 tx=0).stop 2024-03-08T23:55:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 msgr2=0x7f43c4100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c4100f90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00ea60 tx=0x7f43bc00ef20 comp rx=0 tx=0).stop 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 shutdown_connections 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43ac0a2b30 0x7f43ac0a4ff0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c4107400 0x7f43c4100f90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c4069310 0x7f43c4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.699+0000 7f43c9673640 1 --2- 172.21.15.22:0/2216217872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c4068970 0x7f43c4100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.700+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 >> 172.21.15.22:0/2216217872 conn(0x7f43c4077040 msgr2=0x7f43c40fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.700+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 shutdown_connections 2024-03-08T23:55:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:17.700+0000 7f43c9673640 1 -- 172.21.15.22:0/2216217872 wait complete. 2024-03-08T23:55:18.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:17 smithi022 ceph-mon[31894]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2024-03-08T23:55:18.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:17 smithi022 ceph-mon[31894]: osdmap e95: 12 total, 12 up, 12 in 2024-03-08T23:55:18.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:17 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6836/1375407074,v1:172.21.15.62:6837/1375407074] up:boot 2024-03-08T23:55:18.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:17 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:55:18.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:17 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:55:18.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:17 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:55:18.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:17 smithi062 ceph-mon[36129]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2024-03-08T23:55:18.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:17 smithi062 ceph-mon[36129]: osdmap e95: 12 total, 12 up, 12 in 2024-03-08T23:55:18.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:17 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6836/1375407074,v1:172.21.15.62:6837/1375407074] up:boot 2024-03-08T23:55:18.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:17 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:55:18.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:17 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:55:18.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:17 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:55:18.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:17 smithi192 ceph-mon[35995]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2024-03-08T23:55:18.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:17 smithi192 ceph-mon[35995]: osdmap e95: 12 total, 12 up, 12 in 2024-03-08T23:55:18.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:17 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6836/1375407074,v1:172.21.15.62:6837/1375407074] up:boot 2024-03-08T23:55:18.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:17 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:55:18.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:17 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:55:18.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:17 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:55:20.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:19 smithi022 ceph-mon[31894]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:20.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:19 smithi062 ceph-mon[36129]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:20.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:19 smithi192 ceph-mon[35995]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:22.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:21 smithi022 ceph-mon[31894]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:22.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:21 smithi062 ceph-mon[36129]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:22.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:21 smithi192 ceph-mon[35995]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:24.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:23 smithi022 ceph-mon[31894]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:24.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:23 smithi062 ceph-mon[36129]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:24.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:23 smithi192 ceph-mon[35995]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-08T23:55:26.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:25 smithi022 ceph-mon[31894]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-08T23:55:26.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:25 smithi062 ceph-mon[36129]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-08T23:55:26.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:25 smithi192 ceph-mon[35995]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-08T23:55:27.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:55:27.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:55:27.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa40affd640 1 -- 172.21.15.22:0/1353943407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40002cc20 con 0x7fa40c101eb0 2024-03-08T23:55:27.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 -- 172.21.15.22:0/1353943407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c101eb0 msgr2=0x7fa40c105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1353943407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c101eb0 0x7fa40c105ef0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa400009900 tx=0x7fa40002bc50 comp rx=0 tx=0).stop 2024-03-08T23:55:27.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 -- 172.21.15.22:0/1353943407 shutdown_connections 2024-03-08T23:55:27.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1353943407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa40c1065d0 0x7fa40c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1353943407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c101eb0 0x7fa40c105ef0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1353943407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa40c101500 0x7fa40c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 -- 172.21.15.22:0/1353943407 >> 172.21.15.22:0/1353943407 conn(0x7fa40c0fcd20 msgr2=0x7fa40c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:27.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 -- 172.21.15.22:0/1353943407 shutdown_connections 2024-03-08T23:55:27.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.727+0000 7fa4129d4640 1 -- 172.21.15.22:0/1353943407 wait complete. 2024-03-08T23:55:27.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.728+0000 7fa4129d4640 1 Processor -- start 2024-03-08T23:55:27.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.728+0000 7fa4129d4640 1 -- start start 2024-03-08T23:55:27.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.728+0000 7fa4129d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa40c101500 0x7fa40c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.728+0000 7fa4129d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa40c1065d0 0x7fa40c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.728+0000 7fa4129d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c197a50 0x7fa40c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa4129d4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1116b0 con 0x7fa40c197a50 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c197a50 0x7fa40c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c197a50 0x7fa40c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39138/0 (socket says 172.21.15.22:39138) 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 -- 172.21.15.22:0/1651125797 learned_addr learned my addr 172.21.15.22:0/1651125797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa4129d4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c111530 con 0x7fa40c101500 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c111830 con 0x7fa40c1065d0 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa40bfff640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa40c101500 0x7fa40c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa40c1065d0 0x7fa40c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 -- 172.21.15.22:0/1651125797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa40c1065d0 msgr2=0x7fa40c19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa40c1065d0 0x7fa40c19dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 -- 172.21.15.22:0/1651125797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa40c101500 msgr2=0x7fa40c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa40c101500 0x7fa40c19d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 -- 172.21.15.22:0/1651125797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c198570 con 0x7fa40c197a50 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa40c1065d0 0x7fa40c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.729+0000 7fa410f71640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c197a50 0x7fa40c197f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa40400eaa0 tx=0x7fa40400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.730+0000 7fa4097fa640 1 -- 172.21.15.22:0/1651125797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40400cdb0 con 0x7fa40c197a50 2024-03-08T23:55:27.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.730+0000 7fa4097fa640 1 -- 172.21.15.22:0/1651125797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa404005020 con 0x7fa40c197a50 2024-03-08T23:55:27.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.730+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c198860 con 0x7fa40c197a50 2024-03-08T23:55:27.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.730+0000 7fa4097fa640 1 -- 172.21.15.22:0/1651125797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4040116b0 con 0x7fa40c197a50 2024-03-08T23:55:27.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.730+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c1a4a50 con 0x7fa40c197a50 2024-03-08T23:55:27.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.733+0000 7fa4097fa640 1 -- 172.21.15.22:0/1651125797 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fa404005560 con 0x7fa40c197a50 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.733+0000 7fa4155b5740 1 Processor -- start 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.733+0000 7fa4155b5740 1 -- start start 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.733+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.733+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa4155b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8600de00 con 0x564e860ab080 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa4155b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e85fb6b10 con 0x564e860aa760 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa4155b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e86009ea0 con 0x564e860b3c60 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa4097fa640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa3f40a2b10 0x7fa3f40a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45824/0 (socket says 172.21.15.22:45824) 2024-03-08T23:55:27.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40b7fe640 1 -- 172.21.15.22:0/1229920824 learned_addr learned my addr 172.21.15.22:0/1229920824 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:27.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa410f71640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa3f40a2b10 0x7fa3f40a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa4097fa640 1 -- 172.21.15.22:0/1651125797 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa4040cf590 con 0x7fa40c197a50 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40bfff640 1 -- 172.21.15.22:0/1229920824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 msgr2=0x564e860b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40bfff640 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.734+0000 7fa40bfff640 1 -- 172.21.15.22:0/1229920824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 msgr2=0x564e860aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.735+0000 7fa40bfff640 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.735+0000 7fa40bfff640 1 -- 172.21.15.22:0/1229920824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e85dec7d0 con 0x564e860ab080 2024-03-08T23:55:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.735+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.735+0000 7fa410f71640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa3f40a2b10 0x7fa3f40a4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cc820 tx=0x7fa40401e6e0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.735+0000 7fa40bfff640 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860b3720 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009bd0 tx=0x7fa3fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41da8c02ded30a5e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.735+0000 7fa3f27fc640 1 -- 172.21.15.22:0/1229920824 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc004230 con 0x564e860ab080 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.735+0000 7fa3f27fc640 1 -- 172.21.15.22:0/1229920824 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3fc009e40 con 0x564e860ab080 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 -- 172.21.15.22:0/1229920824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 msgr2=0x564e860b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860b3720 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009bd0 tx=0x7fa3fc011810 comp rx=0 tx=0).stop 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 -- 172.21.15.22:0/1229920824 shutdown_connections 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860b3720 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1229920824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 -- 172.21.15.22:0/1229920824 >> 172.21.15.22:0/1229920824 conn(0x564e860a4e90 msgr2=0x564e860a5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 -- 172.21.15.22:0/1229920824 shutdown_connections 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.736+0000 7fa4155b5740 1 -- 172.21.15.22:0/1229920824 wait complete. 2024-03-08T23:55:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.737+0000 7fa4155b5740 1 -- start start 2024-03-08T23:55:27.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.737+0000 7fa4155b5740 1 Processor -- start 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.737+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860b3c60 0x564e860b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa4155b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e860baf00 con 0x564e860b3c60 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa4155b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e85fca0a0 con 0x564e860aa760 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860b3c60 0x564e860b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa4155b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e860bb080 con 0x564e860ab080 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860b3c60 0x564e860b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39160/0 (socket says 172.21.15.22:39160) 2024-03-08T23:55:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 -- 172.21.15.22:0/2060014111 learned_addr learned my addr 172.21.15.22:0/2060014111 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40bfff640 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 -- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 msgr2=0x564e860b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 -- 172.21.15.22:0/2060014111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 msgr2=0x564e860b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 -- 172.21.15.22:0/2060014111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e85d223f0 con 0x564e860b3c60 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860b3c60 0x564e860b0b20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa400009900 tx=0x7fa4000095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2455077803bbda server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.738+0000 7fa3eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa4155b5740 10 client.0 _ll_register_callbacks cb 0x564e85fc9a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e8601b340 con 0x564e860b3c60 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/2060014111 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400050050 con 0x564e860b3c60 2024-03-08T23:55:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/2060014111 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa400004250 con 0x564e860b3c60 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/2060014111 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa400046070 con 0x564e860b3c60 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa4155b5740 10 client.15930 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x564e85ce20f0 con 0x564e860b3c60 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/2060014111 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa400053020 con 0x564e860b3c60 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa4155b5740 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa4155b5740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.739+0000 7fa4155b5740 20 client.15930 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.740+0000 7fa4155b5740 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e860c7780 0x564e860c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.740+0000 7fa4155b5740 4 client.15930 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.740+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e86015690 con 0x564e860c7780 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.740+0000 7fa40bfff640 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e860c7780 0x564e860c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.740+0000 7fa3eb7fe640 10 client.15930 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.740+0000 7fa40bfff640 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e860c7780 0x564e860c9c10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627f8defbe824302 server_cookie=356fd68bad1e6295 in_seq=0 out_seq=0 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.741+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/2060014111 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e86015690 con 0x564e860c7780 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.741+0000 7fa3eb7fe640 10 client.15930 handle_command_reply: tid=0 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.741+0000 7fa4155b5740 1 client.15930 shutdown 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.741+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860b3c60 msgr2=0x564e860b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.741+0000 7fa4155b5740 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860b3c60 0x564e860b0b20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa400009900 tx=0x7fa4000095f0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 shutdown_connections 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e860c7780 0x564e860c9c10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860b3c60 0x564e860b0b20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa3eb7fe640 0 client.15930 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 --2- 172.21.15.22:0/2060014111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 >> 172.21.15.22:0/2060014111 conn(0x564e860a4e90 msgr2=0x564e860a6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 shutdown_connections 2024-03-08T23:55:27.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 1 -- 172.21.15.22:0/2060014111 wait complete. 2024-03-08T23:55:27.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.742+0000 7fa4155b5740 20 client.15930 trim_cache size 0 max 16384 2024-03-08T23:55:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 Processor -- start 2024-03-08T23:55:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 -- start start 2024-03-08T23:55:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1ae940 con 0x564e860ab080 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1ac4e0 con 0x564e860aa760 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa4155b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1aa580 con 0x564e860b3c60 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39162/0 (socket says 172.21.15.22:39162) 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.772+0000 7fa40bfff640 1 -- 172.21.15.22:0/1196705231 learned_addr learned my addr 172.21.15.22:0/1196705231 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40bfff640 1 -- 172.21.15.22:0/1196705231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 msgr2=0x564e860b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40bfff640 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40bfff640 1 -- 172.21.15.22:0/1196705231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 msgr2=0x564e860abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40bfff640 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40bfff640 1 -- 172.21.15.22:0/1196705231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e860aaeb0 con 0x564e860ab080 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa40bfff640 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860ce5b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009bd0 tx=0x7fa3fc004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee44b5c008fa6ff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa3f1ffb640 1 -- 172.21.15.22:0/1196705231 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc01e040 con 0x564e860ab080 2024-03-08T23:55:27.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa3f1ffb640 1 -- 172.21.15.22:0/1196705231 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3fc004050 con 0x564e860ab080 2024-03-08T23:55:27.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.773+0000 7fa3f1ffb640 1 -- 172.21.15.22:0/1196705231 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc003970 con 0x564e860ab080 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 -- 172.21.15.22:0/1196705231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 msgr2=0x564e860ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860ce5b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009bd0 tx=0x7fa3fc004950 comp rx=0 tx=0).stop 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 -- 172.21.15.22:0/1196705231 shutdown_connections 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860b3c60 0x564e860b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860ab080 0x564e860ce5b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 --2- 172.21.15.22:0/1196705231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860aa760 0x564e860abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 -- 172.21.15.22:0/1196705231 >> 172.21.15.22:0/1196705231 conn(0x564e860a4e90 msgr2=0x564e860a3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 -- 172.21.15.22:0/1196705231 shutdown_connections 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 -- 172.21.15.22:0/1196705231 wait complete. 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.774+0000 7fa4155b5740 1 -- start start 2024-03-08T23:55:27.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 1 Processor -- start 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860aa760 0x564e860bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860b3c60 0x564e8611dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1aab80 con 0x564e860aa760 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1abe80 con 0x564e860b3c60 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1ab1c0 con 0x564e860ab080 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa4155b5740 10 client.0 _ll_register_callbacks cb 0x7fa400009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860aa760 0x564e860bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860aa760 0x564e860bc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39166/0 (socket says 172.21.15.22:39166) 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 -- 172.21.15.22:0/281376246 learned_addr learned my addr 172.21.15.22:0/281376246 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40bfff640 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860b3c60 0x564e8611dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 -- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 msgr2=0x564e860bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 -- 172.21.15.22:0/281376246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860b3c60 msgr2=0x564e8611dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860b3c60 0x564e8611dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.775+0000 7fa40b7fe640 1 -- 172.21.15.22:0/281376246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e8611e270 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa40b7fe640 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860aa760 0x564e860bc360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa400009850 tx=0x7fa40002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ffe6c7711227e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa3eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/281376246 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400050070 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/281376246 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa400034a10 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3fc004050 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/281376246 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa400065050 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa4155b5740 10 client.15936 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564e8611f480 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.776+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/281376246 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564e8601b340 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/281376246 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa400053030 con 0x564e860aa760 2024-03-08T23:55:27.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa4155b5740 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa4155b5740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa4155b5740 20 client.15936 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa4155b5740 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e8611faf0 0x564e86121ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa4155b5740 4 client.15936 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fa3fc004050 con 0x564e8611faf0 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa40bfff640 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e8611faf0 0x564e86121ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa3eb7fe640 10 client.15936 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.777+0000 7fa40bfff640 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e8611faf0 0x564e86121ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08916e14e591605 server_cookie=571f287a20b9813a in_seq=0 out_seq=0 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa3eb7fe640 1 -- 172.21.15.22:0/281376246 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fa3fc004050 con 0x564e8611faf0 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa3eb7fe640 10 client.15936 handle_command_reply: tid=0 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 client.15936 shutdown 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860aa760 msgr2=0x564e860bc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860aa760 0x564e860bc360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa400009850 tx=0x7fa40002e070 comp rx=0 tx=0).stop 2024-03-08T23:55:27.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 shutdown_connections 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e8611faf0 0x564e86121ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e860b3c60 0x564e8611dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e860ab080 0x564e860bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 --2- 172.21.15.22:0/281376246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e860aa760 0x564e860bc360 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 >> 172.21.15.22:0/281376246 conn(0x564e860a4e90 msgr2=0x564e860a3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa3eb7fe640 0 client.15936 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 shutdown_connections 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 1 -- 172.21.15.22:0/281376246 wait complete. 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.778+0000 7fa4155b5740 20 client.15936 trim_cache size 0 max 16384 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:55:27.779 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:55:27.780 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:55:27.780 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:55:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa3f40a2b10 msgr2=0x7fa3f40a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa3f40a2b10 0x7fa3f40a4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cc820 tx=0x7fa40401e6e0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c197a50 msgr2=0x7fa40c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c197a50 0x7fa40c197f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa40400eaa0 tx=0x7fa40400ef60 comp rx=0 tx=0).stop 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 shutdown_connections 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa3f40a2b10 0x7fa3f40a4fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c197a50 0x7fa40c197f00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa40c1065d0 0x7fa40c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 --2- 172.21.15.22:0/1651125797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa40c101500 0x7fa40c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 >> 172.21.15.22:0/1651125797 conn(0x7fa40c0fcd20 msgr2=0x7fa40c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 shutdown_connections 2024-03-08T23:55:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.779+0000 7fa4129d4640 1 -- 172.21.15.22:0/1651125797 wait complete. 2024-03-08T23:55:27.789 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:55:27.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.899+0000 7fda9f6ea640 1 -- 172.21.15.22:0/4003074982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9810b930 msgr2=0x7fda9810ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.899+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/4003074982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9810b930 0x7fda9810ddd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fda90009ca0 tx=0x7fda9002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.899+0000 7fda9f6ea640 1 -- 172.21.15.22:0/4003074982 shutdown_connections 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.899+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/4003074982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda9810b930 0x7fda9810ddd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.899+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/4003074982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda98102e80 0x7fda9810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.899+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/4003074982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98102560 0x7fda98102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.899+0000 7fda9f6ea640 1 -- 172.21.15.22:0/4003074982 >> 172.21.15.22:0/4003074982 conn(0x7fda980772f0 msgr2=0x7fda98077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:27.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.900+0000 7fda9f6ea640 1 -- 172.21.15.22:0/4003074982 shutdown_connections 2024-03-08T23:55:27.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.900+0000 7fda9f6ea640 1 -- 172.21.15.22:0/4003074982 wait complete. 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.900+0000 7fda9f6ea640 1 Processor -- start 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.900+0000 7fda9f6ea640 1 -- start start 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9f6ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98102560 0x7fda9819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9f6ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda98102e80 0x7fda9819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9f6ea640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9810b930 0x7fda981979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9f6ea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda981116f0 con 0x7fda98102e80 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9f6ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda98111570 con 0x7fda98102560 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9f6ea640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda98111870 con 0x7fda9810b930 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda98102e80 0x7fda9819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda98102e80 0x7fda9819dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39180/0 (socket says 172.21.15.22:39180) 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 -- 172.21.15.22:0/3173385916 learned_addr learned my addr 172.21.15.22:0/3173385916 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9d486640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98102560 0x7fda9819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9dc87640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9810b930 0x7fda981979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 -- 172.21.15.22:0/3173385916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9810b930 msgr2=0x7fda981979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9810b930 0x7fda981979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 -- 172.21.15.22:0/3173385916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98102560 msgr2=0x7fda9819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98102560 0x7fda9819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9cc85640 1 -- 172.21.15.22:0/3173385916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda98198130 con 0x7fda98102e80 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.901+0000 7fda9dc87640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9810b930 0x7fda981979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.902+0000 7fda9cc85640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda98102e80 0x7fda9819dcf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fda8800d950 tx=0x7fda8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.902+0000 7fda8e7fc640 1 -- 172.21.15.22:0/3173385916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88017070 con 0x7fda98102e80 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.902+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda98198420 con 0x7fda98102e80 2024-03-08T23:55:27.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.902+0000 7fda8e7fc640 1 -- 172.21.15.22:0/3173385916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda88009c30 con 0x7fda98102e80 2024-03-08T23:55:27.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.902+0000 7fda8e7fc640 1 -- 172.21.15.22:0/3173385916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8800f6b0 con 0x7fda98102e80 2024-03-08T23:55:27.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.902+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda981000e0 con 0x7fda98102e80 2024-03-08T23:55:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.903+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda980680e0 con 0x7fda98102e80 2024-03-08T23:55:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.904+0000 7fda8e7fc640 1 -- 172.21.15.22:0/3173385916 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fda880109d0 con 0x7fda98102e80 2024-03-08T23:55:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.905+0000 7fda8e7fc640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda840a2a90 0x7fda840a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.905+0000 7fda9d486640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda840a2a90 0x7fda840a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.905+0000 7fda8e7fc640 1 -- 172.21.15.22:0/3173385916 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fda880cf140 con 0x7fda98102e80 2024-03-08T23:55:27.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.906+0000 7fda9d486640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda840a2a90 0x7fda840a4f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda980ff5a0 tx=0x7fda9400b390 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:27.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:27.914+0000 7fda8e7fc640 1 -- 172.21.15.22:0/3173385916 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fda88009de0 con 0x7fda98102e80 2024-03-08T23:55:27.972 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:27 smithi022 ceph-mon[31894]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-08T23:55:28.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.138+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda98100340 con 0x7fda98102e80 2024-03-08T23:55:28.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.139+0000 7fda8e7fc640 1 -- 172.21.15.22:0/3173385916 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fda8809b8b0 con 0x7fda98102e80 2024-03-08T23:55:28.136 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:55:28.136 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:55:16.974110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25439":{"gid":25439,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1375407074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1375407074},{"type":"v1","addr":"172.21.15.62:6837","nonce":1375407074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:55:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-08T23:55:28.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda840a2a90 msgr2=0x7fda840a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda840a2a90 0x7fda840a4f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda980ff5a0 tx=0x7fda9400b390 comp rx=0 tx=0).stop 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda98102e80 msgr2=0x7fda9819dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda98102e80 0x7fda9819dcf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fda8800d950 tx=0x7fda8800de10 comp rx=0 tx=0).stop 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 shutdown_connections 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda840a2a90 0x7fda840a4f50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda9810b930 0x7fda981979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda98102e80 0x7fda9819dcf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 --2- 172.21.15.22:0/3173385916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98102560 0x7fda9819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 >> 172.21.15.22:0/3173385916 conn(0x7fda980772f0 msgr2=0x7fda980fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 shutdown_connections 2024-03-08T23:55:28.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.142+0000 7fda9f6ea640 1 -- 172.21.15.22:0/3173385916 wait complete. 2024-03-08T23:55:28.153 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:55:28.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:55:28.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/3102677743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28102510 msgr2=0x7f5d281028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/3102677743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28102510 0x7f5d281028f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009900 tx=0x7f5d1c02bcd0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/3102677743 shutdown_connections 2024-03-08T23:55:28.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/3102677743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d2810b850 0x7f5d2810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/3102677743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d28102e30 0x7f5d2810b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/3102677743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d28102510 0x7f5d281028f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/3102677743 >> 172.21.15.22:0/3102677743 conn(0x7f5d280773a0 msgr2=0x7f5d280777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/3102677743 shutdown_connections 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.275+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/3102677743 wait complete. 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 Processor -- start 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 -- start start 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d28102510 0x7f5d2819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d28102e30 0x7f5d2819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2810b850 0x7f5d281979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28111680 con 0x7f5d2810b850 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28111500 con 0x7f5d28102510 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2ed3b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28111800 con 0x7f5d28102e30 2024-03-08T23:55:28.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2d2d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2810b850 0x7f5d281979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2d2d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2810b850 0x7f5d281979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39196/0 (socket says 172.21.15.22:39196) 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2d2d8640 1 -- 172.21.15.22:0/2385468383 learned_addr learned my addr 172.21.15.22:0/2385468383 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.276+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d28102510 0x7f5d2819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d28102e30 0x7f5d2819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d2d2d8640 1 -- 172.21.15.22:0/2385468383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d28102e30 msgr2=0x7f5d2819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d2d2d8640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d28102e30 0x7f5d2819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d2d2d8640 1 -- 172.21.15.22:0/2385468383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d28102510 msgr2=0x7f5d2819d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d2d2d8640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d28102510 0x7f5d2819d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d2d2d8640 1 -- 172.21.15.22:0/2385468383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d28198120 con 0x7f5d2810b850 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d2d2d8640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2810b850 0x7f5d281979a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009fd0 tx=0x7f5d2000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d25ffb640 1 -- 172.21.15.22:0/2385468383 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20017070 con 0x7f5d2810b850 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d25ffb640 1 -- 172.21.15.22:0/2385468383 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d20003e50 con 0x7f5d2810b850 2024-03-08T23:55:28.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d28198410 con 0x7f5d2810b850 2024-03-08T23:55:28.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.277+0000 7f5d25ffb640 1 -- 172.21.15.22:0/2385468383 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d200106c0 con 0x7f5d2810b850 2024-03-08T23:55:28.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.278+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d281a4660 con 0x7f5d2810b850 2024-03-08T23:55:28.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.280+0000 7f5d25ffb640 1 -- 172.21.15.22:0/2385468383 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f5d20003970 con 0x7f5d2810b850 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d25ffb640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5d180a2b10 0x7f5d180a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d319f2740 1 Processor -- start 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d319f2740 1 -- start start 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d25ffb640 1 -- 172.21.15.22:0/2385468383 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5d200cf100 con 0x7f5d2810b850 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5d180a2b10 0x7f5d180a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d35075cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d35074a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.281+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350764700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d319f2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d350717360 con 0x55d35075c830 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d319f2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3506b3ec0 con 0x55d35074a380 2024-03-08T23:55:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d319f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35065cd30 con 0x55d35074ad40 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5d180a2b10 0x7f5d180a4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ab0 tx=0x7f5d1c005a20 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d35075cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d35075cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39210/0 (socket says 172.21.15.22:39210) 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 -- 172.21.15.22:0/2141547391 learned_addr learned my addr 172.21.15.22:0/2141547391 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2d2d8640 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350764700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d35074a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 -- 172.21.15.22:0/2141547391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 msgr2=0x55d350764700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350764700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 -- 172.21.15.22:0/2141547391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 msgr2=0x55d35074a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d35074a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 -- 172.21.15.22:0/2141547391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35065bbc0 con 0x55d35075c830 2024-03-08T23:55:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d35075cc10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006010 tx=0x7f5d1c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ef58b2af949e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d12ffd640 1 -- 172.21.15.22:0/2141547391 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c045070 con 0x55d35075c830 2024-03-08T23:55:28.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d12ffd640 1 -- 172.21.15.22:0/2141547391 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c051070 con 0x55d35075c830 2024-03-08T23:55:28.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.282+0000 7f5d12ffd640 1 -- 172.21.15.22:0/2141547391 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0050e0 con 0x55d35075c830 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 -- 172.21.15.22:0/2141547391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 msgr2=0x55d35075cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d35075cc10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006010 tx=0x7f5d1c03f000 comp rx=0 tx=0).stop 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 -- 172.21.15.22:0/2141547391 shutdown_connections 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350764700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d35074a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2141547391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d35075cc10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 -- 172.21.15.22:0/2141547391 >> 172.21.15.22:0/2141547391 conn(0x55d35074ce40 msgr2=0x55d35074d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 -- 172.21.15.22:0/2141547391 shutdown_connections 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.283+0000 7f5d319f2740 1 -- 172.21.15.22:0/2141547391 wait complete. 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.284+0000 7f5d319f2740 1 -- start start 2024-03-08T23:55:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.284+0000 7f5d319f2740 1 Processor -- start 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.284+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d350761bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.284+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350762110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.284+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d350762650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d2d2d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d350762650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d2d2d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d350762650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39214/0 (socket says 172.21.15.22:39214) 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d2d2d8640 1 -- 172.21.15.22:0/2278113487 learned_addr learned my addr 172.21.15.22:0/2278113487 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3506524a0 con 0x55d35075c830 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3506aff60 con 0x55d35074a380 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d350764d00 con 0x55d35074ad40 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350762110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d350761bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d319f2740 10 client.0 _ll_register_callbacks cb 0x55d3506add60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:28.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d27fff640 1 -- 172.21.15.22:0/2278113487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 msgr2=0x55d350761bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d350761bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d27fff640 1 -- 172.21.15.22:0/2278113487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 msgr2=0x55d350762650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d350762650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d27fff640 1 -- 172.21.15.22:0/2278113487 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3503c83f0 con 0x55d35074ad40 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d2d2d8640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d350762650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350762110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005fd0 tx=0x7f5d1400bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da6d329b0484aae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.285+0000 7f5d07fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.286+0000 7f5d07fff640 1 -- 172.21.15.22:0/2278113487 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1401b440 con 0x55d35074ad40 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.286+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d1c051070 con 0x55d35074ad40 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.286+0000 7f5d07fff640 1 -- 172.21.15.22:0/2278113487 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5d14011440 con 0x55d35074ad40 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.286+0000 7f5d07fff640 1 -- 172.21.15.22:0/2278113487 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d14011640 con 0x55d35074ad40 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.286+0000 7f5d319f2740 10 client.25471 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.286+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d3504927d0 con 0x55d35074ad40 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.286+0000 7f5d07fff640 1 -- 172.21.15.22:0/2278113487 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c051070 con 0x55d35074ad40 2024-03-08T23:55:28.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d07fff640 1 -- 172.21.15.22:0/2278113487 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d14019020 con 0x55d35074ad40 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d319f2740 10 client.25471 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d319f2740 10 client.25471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d319f2740 20 client.25471 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d350774550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d319f2740 4 client.25471 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3506c1400 con 0x55d350772140 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d350774550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d350774550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be15355bcc7250d server_cookie=dfe54fae90130bbd in_seq=0 out_seq=0 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.287+0000 7f5d07fff640 10 client.25471 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d07fff640 1 -- 172.21.15.22:0/2278113487 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3506c1400 con 0x55d350772140 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d07fff640 10 client.25471 handle_command_reply: tid=0 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 client.25471 shutdown 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 msgr2=0x55d350762110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350762110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005fd0 tx=0x7f5d1400bcc0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 shutdown_connections 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d350774550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d07fff640 0 client.25471 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35075c830 0x55d350762650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350762110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2278113487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35074a380 0x55d350761bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.288+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 >> 172.21.15.22:0/2278113487 conn(0x55d35074ce40 msgr2=0x55d35074c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.289+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 shutdown_connections 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.289+0000 7f5d319f2740 1 -- 172.21.15.22:0/2278113487 wait complete. 2024-03-08T23:55:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.289+0000 7f5d319f2740 20 client.25471 trim_cache size 0 max 16384 2024-03-08T23:55:28.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.311+0000 7f5d319f2740 1 Processor -- start 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.311+0000 7f5d319f2740 1 -- start start 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074a380 0x55d350764300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350745870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d350778eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d319f2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281ae7b0 con 0x55d35074a380 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d319f2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281ac350 con 0x55d35075c830 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d319f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281aa3f0 con 0x55d35074ad40 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074a380 0x55d350764300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074a380 0x55d350764300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39226/0 (socket says 172.21.15.22:39226) 2024-03-08T23:55:28.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 -- 172.21.15.22:0/3882336286 learned_addr learned my addr 172.21.15.22:0/3882336286 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2d2d8640 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d350778eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d27fff640 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350745870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 -- 172.21.15.22:0/3882336286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 msgr2=0x55d350745870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350745870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 -- 172.21.15.22:0/3882336286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 msgr2=0x55d350778eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d350778eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 -- 172.21.15.22:0/3882336286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35075c470 con 0x55d35074a380 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.312+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074a380 0x55d350764300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009900 tx=0x7f5d1c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79e4442b0b11ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.313+0000 7f5d127fc640 1 -- 172.21.15.22:0/3882336286 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c045050 con 0x55d35074a380 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.313+0000 7f5d127fc640 1 -- 172.21.15.22:0/3882336286 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c051040 con 0x55d35074a380 2024-03-08T23:55:28.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.313+0000 7f5d127fc640 1 -- 172.21.15.22:0/3882336286 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c05e5c0 con 0x55d35074a380 2024-03-08T23:55:28.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 -- 172.21.15.22:0/3882336286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074a380 msgr2=0x55d350764300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074a380 0x55d350764300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009900 tx=0x7f5d1c02bcd0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 -- 172.21.15.22:0/3882336286 shutdown_connections 2024-03-08T23:55:28.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d350778eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074ad40 0x55d350745870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 --2- 172.21.15.22:0/3882336286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074a380 0x55d350764300 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 -- 172.21.15.22:0/3882336286 >> 172.21.15.22:0/3882336286 conn(0x55d35074ce40 msgr2=0x55d35075b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 -- 172.21.15.22:0/3882336286 shutdown_connections 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 -- 172.21.15.22:0/3882336286 wait complete. 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 -- start start 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.314+0000 7f5d319f2740 1 Processor -- start 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074a380 0x55d350765950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074ad40 0x55d350765e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d319f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d3507663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d319f2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281aa9f0 con 0x55d35074ad40 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d319f2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281abcf0 con 0x55d35075c830 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074ad40 0x55d350765e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d2cad7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074a380 0x55d350765950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074ad40 0x55d350765e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39234/0 (socket says 172.21.15.22:39234) 2024-03-08T23:55:28.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 -- 172.21.15.22:0/2497123003 learned_addr learned my addr 172.21.15.22:0/2497123003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281ab030 con 0x55d35074a380 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d2d2d8640 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d3507663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d319f2740 10 client.0 _ll_register_callbacks cb 0x7f5d1c005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 -- 172.21.15.22:0/2497123003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074a380 msgr2=0x55d350765950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074a380 0x55d350765950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 -- 172.21.15.22:0/2497123003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 msgr2=0x55d3507663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d3507663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 -- 172.21.15.22:0/2497123003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3507c8630 con 0x55d35074ad40 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d27fff640 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074ad40 0x55d350765e90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55d350761430 tx=0x7f5d14005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea962faaa9d7f12 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.315+0000 7f5d07fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d07fff640 1 -- 172.21.15.22:0/2497123003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1402c070 con 0x55d35074ad40 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d07fff640 1 -- 172.21.15.22:0/2497123003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5d1401b440 con 0x55d35074ad40 2024-03-08T23:55:28.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d1c051040 con 0x55d35074ad40 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d07fff640 1 -- 172.21.15.22:0/2497123003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d1403a050 con 0x55d35074ad40 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d319f2740 10 client.15960 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d3507c98a0 con 0x55d35074ad40 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d07fff640 1 -- 172.21.15.22:0/2497123003 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c051040 con 0x55d35074ad40 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.316+0000 7f5d07fff640 1 -- 172.21.15.22:0/2497123003 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d14019050 con 0x55d35074ad40 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.317+0000 7f5d319f2740 10 client.15960 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.317+0000 7f5d319f2740 10 client.15960 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.317+0000 7f5d319f2740 20 client.15960 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.317+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d3507cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.317+0000 7f5d319f2740 4 client.15960 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.317+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d35075d810 con 0x55d350772140 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.317+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d3507cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d2cad7640 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d3507cbf80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fe7490bd33d0651 server_cookie=2728ee8543e3ace9 in_seq=0 out_seq=0 2024-03-08T23:55:28.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d07fff640 10 client.15960 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d07fff640 1 -- 172.21.15.22:0/2497123003 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d35075d810 con 0x55d350772140 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d07fff640 10 client.15960 handle_command_reply: tid=0 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 client.15960 shutdown 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074ad40 msgr2=0x55d350765e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074ad40 0x55d350765e90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55d350761430 tx=0x7f5d14005c40 comp rx=0 tx=0).stop 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 shutdown_connections 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d350772140 0x55d3507cbf80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d35075c830 0x55d3507663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d07fff640 0 client.15960 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d35074ad40 0x55d350765e90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 --2- 172.21.15.22:0/2497123003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35074a380 0x55d350765950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.318+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 >> 172.21.15.22:0/2497123003 conn(0x55d35074ce40 msgr2=0x55d35075b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.319+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 shutdown_connections 2024-03-08T23:55:28.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.319+0000 7f5d319f2740 1 -- 172.21.15.22:0/2497123003 wait complete. 2024-03-08T23:55:28.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.319+0000 7f5d319f2740 20 client.15960 trim_cache size 0 max 16384 2024-03-08T23:55:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.319+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5d180a2b10 msgr2=0x7f5d180a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.319+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5d180a2b10 0x7f5d180a4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ab0 tx=0x7f5d1c005a20 comp rx=0 tx=0).stop 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.319+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2810b850 msgr2=0x7f5d281979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.319+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2810b850 0x7f5d281979a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009fd0 tx=0x7f5d2000b260 comp rx=0 tx=0).stop 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 shutdown_connections 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5d180a2b10 0x7f5d180a4fd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d2810b850 0x7f5d281979a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d28102e30 0x7f5d2819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 --2- 172.21.15.22:0/2385468383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d28102510 0x7f5d2819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 >> 172.21.15.22:0/2385468383 conn(0x7f5d280773a0 msgr2=0x7f5d280fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 shutdown_connections 2024-03-08T23:55:28.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.320+0000 7f5d2ed3b640 1 -- 172.21.15.22:0/2385468383 wait complete. 2024-03-08T23:55:28.330 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:55:28.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:55:28.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:27 smithi062 ceph-mon[36129]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-08T23:55:28.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:27 smithi192 ceph-mon[35995]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-08T23:55:28.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 -- 172.21.15.22:0/3845108282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118101e80 msgr2=0x7f5118105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 --2- 172.21.15.22:0/3845108282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118101e80 0x7f5118105ec0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f510c009900 tx=0x7f510c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:55:28.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 -- 172.21.15.22:0/3845108282 shutdown_connections 2024-03-08T23:55:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 --2- 172.21.15.22:0/3845108282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51181065a0 0x7f511810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 --2- 172.21.15.22:0/3845108282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118101e80 0x7f5118105ec0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 --2- 172.21.15.22:0/3845108282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51181014d0 0x7f51181018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 -- 172.21.15.22:0/3845108282 >> 172.21.15.22:0/3845108282 conn(0x7f51180fcd90 msgr2=0x7f51180ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 -- 172.21.15.22:0/3845108282 shutdown_connections 2024-03-08T23:55:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.445+0000 7f511e880640 1 -- 172.21.15.22:0/3845108282 wait complete. 2024-03-08T23:55:28.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 Processor -- start 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 -- start start 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51181014d0 0x7f51180735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51181065a0 0x7f5118073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118074070 0x7f51181a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118111610 con 0x7f5118074070 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118111490 con 0x7f51181014d0 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511e880640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118111790 con 0x7f51181065a0 2024-03-08T23:55:28.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511ce1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118074070 0x7f51181a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511ce1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118074070 0x7f51181a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39250/0 (socket says 172.21.15.22:39250) 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f511ce1d640 1 -- 172.21.15.22:0/2064420804 learned_addr learned my addr 172.21.15.22:0/2064420804 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f5117fff640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51181014d0 0x7f51180735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.446+0000 7f51177fe640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51181065a0 0x7f5118073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511ce1d640 1 -- 172.21.15.22:0/2064420804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51181065a0 msgr2=0x7f5118073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511ce1d640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51181065a0 0x7f5118073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511ce1d640 1 -- 172.21.15.22:0/2064420804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51181014d0 msgr2=0x7f51180735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511ce1d640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51181014d0 0x7f51180735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511ce1d640 1 -- 172.21.15.22:0/2064420804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51181a6420 con 0x7f5118074070 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f5117fff640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51181014d0 0x7f51180735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511ce1d640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118074070 0x7f51181a5d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f511000cce0 tx=0x7f511000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f51157fa640 1 -- 172.21.15.22:0/2064420804 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511000f450 con 0x7f5118074070 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f51157fa640 1 -- 172.21.15.22:0/2064420804 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f511000fad0 con 0x7f5118074070 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f51157fa640 1 -- 172.21.15.22:0/2064420804 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110018c10 con 0x7f5118074070 2024-03-08T23:55:28.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51181a6710 con 0x7f5118074070 2024-03-08T23:55:28.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.447+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51181a6be0 con 0x7f5118074070 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f51157fa640 1 -- 172.21.15.22:0/2064420804 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f511000f5f0 con 0x7f5118074070 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 Processor -- start 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 -- start start 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a611a0 0x561f12a61580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a6a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f51157fa640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51000a2b40 0x7f51000a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f12a28360 con 0x561f12a6a6a0 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f129c4ec0 con 0x561f12a611a0 2024-03-08T23:55:28.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f512145c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1296dd30 con 0x561f12a61ac0 2024-03-08T23:55:28.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f51157fa640 1 -- 172.21.15.22:0/2064420804 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f51100cf9a0 con 0x7f5118074070 2024-03-08T23:55:28.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f511ce1d640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51000a2b40 0x7f51000a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f51177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f51177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a6ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39252/0 (socket says 172.21.15.22:39252) 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.451+0000 7f51177fe640 1 -- 172.21.15.22:0/916803330 learned_addr learned my addr 172.21.15.22:0/916803330 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f5117fff640 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a6a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f5117fff640 1 -- 172.21.15.22:0/916803330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a611a0 msgr2=0x561f12a61580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f5117fff640 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a611a0 0x561f12a61580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f5117fff640 1 -- 172.21.15.22:0/916803330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 msgr2=0x561f12a6ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f5117fff640 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a6ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f5117fff640 1 -- 172.21.15.22:0/916803330 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1296cbc0 con 0x561f12a61ac0 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f511ce1d640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51000a2b40 0x7f51000a5000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51100ccc30 tx=0x7f5110008ee0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f51177fe640 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f5117fff640 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a6a160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5108009fd0 tx=0x7f51080118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b489885090855f25 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f50fe7fc640 1 -- 172.21.15.22:0/916803330 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510801e070 con 0x561f12a61ac0 2024-03-08T23:55:28.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f50fe7fc640 1 -- 172.21.15.22:0/916803330 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51080035f0 con 0x561f12a61ac0 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.452+0000 7f50fe7fc640 1 -- 172.21.15.22:0/916803330 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510800f800 con 0x561f12a61ac0 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 -- 172.21.15.22:0/916803330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 msgr2=0x561f12a6a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a6a160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5108009fd0 tx=0x7f51080118d0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 -- 172.21.15.22:0/916803330 shutdown_connections 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a6a160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 --2- 172.21.15.22:0/916803330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a611a0 0x561f12a61580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 -- 172.21.15.22:0/916803330 >> 172.21.15.22:0/916803330 conn(0x561f12a5b8d0 msgr2=0x561f12a5bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 -- 172.21.15.22:0/916803330 shutdown_connections 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.453+0000 7f512145c740 1 -- 172.21.15.22:0/916803330 wait complete. 2024-03-08T23:55:28.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 -- start start 2024-03-08T23:55:28.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 Processor -- start 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a67c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 0x561f12a68140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f129634a0 con 0x561f12a611a0 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f129c0f60 con 0x561f12a6a6a0 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.454+0000 7f512145c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f12a6d490 con 0x561f12a61ac0 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a676c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39260/0 (socket says 172.21.15.22:39260) 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f512145c740 10 client.0 _ll_register_callbacks cb 0x561f129bed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 -- 172.21.15.22:0/3766002292 learned_addr learned my addr 172.21.15.22:0/3766002292 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f5117fff640 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a67c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 -- 172.21.15.22:0/3766002292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 msgr2=0x561f12a67c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a67c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 -- 172.21.15.22:0/3766002292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 msgr2=0x561f12a68140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 0x561f12a68140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 -- 172.21.15.22:0/3766002292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f126d93f0 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f5117fff640 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f51177fe640 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a676c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f510c009230 tx=0x7f510c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bc1207fb990e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f50f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f50f77fe640 1 -- 172.21.15.22:0/3766002292 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0046c0 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f50f77fe640 1 -- 172.21.15.22:0/3766002292 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f510c004860 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51080035f0 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f50f77fe640 1 -- 172.21.15.22:0/3766002292 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f510c004a60 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f512145c740 10 client.15972 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.455+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f127a37d0 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f50f77fe640 1 -- 172.21.15.22:0/3766002292 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f510c004c00 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f50f77fe640 1 -- 172.21.15.22:0/3766002292 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f510c003950 con 0x561f12a611a0 2024-03-08T23:55:28.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f512145c740 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f512145c740 10 client.15972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f512145c740 20 client.15972 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f512145c740 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12a7a880 0x561f12a7cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f512145c740 4 client.15972 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51080035f0 con 0x561f12a7a880 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.456+0000 7f5117fff640 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12a7a880 0x561f12a7cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.457+0000 7f5117fff640 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12a7a880 0x561f12a7cc70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8a752495f7d2687 server_cookie=311e863fd186442f in_seq=0 out_seq=0 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.457+0000 7f50f77fe640 10 client.15972 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.457+0000 7f50f77fe640 1 -- 172.21.15.22:0/3766002292 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f51080035f0 con 0x561f12a7a880 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.457+0000 7f50f77fe640 10 client.15972 handle_command_reply: tid=0 2024-03-08T23:55:28.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.457+0000 7f512145c740 1 client.15972 shutdown 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 msgr2=0x561f12a676c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a676c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f510c009230 tx=0x7f510c009680 comp rx=0 tx=0).stop 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 shutdown_connections 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12a7a880 0x561f12a7cc70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 0x561f12a68140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f50f77fe640 0 client.15972 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 --2- 172.21.15.22:0/3766002292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a676c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 >> 172.21.15.22:0/3766002292 conn(0x561f12a5b8d0 msgr2=0x561f12a5d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 shutdown_connections 2024-03-08T23:55:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 1 -- 172.21.15.22:0/3766002292 wait complete. 2024-03-08T23:55:28.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.458+0000 7f512145c740 20 client.15972 trim_cache size 0 max 16384 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 Processor -- start 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 -- start start 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 0x561f12a64940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181b2c80 con 0x561f12a611a0 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181b0820 con 0x561f12a6a6a0 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f512145c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181ae8c0 con 0x561f12a61ac0 2024-03-08T23:55:28.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a69df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39270/0 (socket says 172.21.15.22:39270) 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 -- 172.21.15.22:0/405984004 learned_addr learned my addr 172.21.15.22:0/405984004 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f5117fff640 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 -- 172.21.15.22:0/405984004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 msgr2=0x561f12a666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 -- 172.21.15.22:0/405984004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 msgr2=0x561f12a64940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 0x561f12a64940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 -- 172.21.15.22:0/405984004 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f12a7cfe0 con 0x561f12a611a0 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.470+0000 7f51177fe640 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a69df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f510c009890 tx=0x7f510c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e0029872188a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f50fdffb640 1 -- 172.21.15.22:0/405984004 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c05b070 con 0x561f12a611a0 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f50fdffb640 1 -- 172.21.15.22:0/405984004 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f510c034440 con 0x561f12a611a0 2024-03-08T23:55:28.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f50fdffb640 1 -- 172.21.15.22:0/405984004 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c034c00 con 0x561f12a611a0 2024-03-08T23:55:28.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 -- 172.21.15.22:0/405984004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 msgr2=0x561f12a69df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a69df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f510c009890 tx=0x7f510c005fd0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 -- 172.21.15.22:0/405984004 shutdown_connections 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a6a6a0 0x561f12a64940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 --2- 172.21.15.22:0/405984004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a611a0 0x561f12a69df0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 -- 172.21.15.22:0/405984004 >> 172.21.15.22:0/405984004 conn(0x561f12a5b8d0 msgr2=0x561f12a65f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.471+0000 7f512145c740 1 -- 172.21.15.22:0/405984004 shutdown_connections 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 -- 172.21.15.22:0/405984004 wait complete. 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 -- start start 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 Processor -- start 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a7a880 0x561f12ad0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181aeec0 con 0x561f12a6a6a0 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181b01c0 con 0x561f12a7a880 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181af500 con 0x561f12a61ac0 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f512145c740 10 client.0 _ll_register_callbacks cb 0x7f5108009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f5117fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f5117fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39282/0 (socket says 172.21.15.22:39282) 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.472+0000 7f5117fff640 1 -- 172.21.15.22:0/1748845894 learned_addr learned my addr 172.21.15.22:0/1748845894 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f51177fe640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f51177fe640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a7a880 0x561f12ad0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f5117fff640 1 -- 172.21.15.22:0/1748845894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 msgr2=0x561f12a703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f5117fff640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f5117fff640 1 -- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a7a880 msgr2=0x561f12ad0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f5117fff640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a7a880 0x561f12ad0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f5117fff640 1 -- 172.21.15.22:0/1748845894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f12a711d0 con 0x561f12a6a6a0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f51177fe640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a7a880 0x561f12ad0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f5117fff640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a708e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5108009730 tx=0x7f5108005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80842a2ec7cc4c4f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f50f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f50f77fe640 1 -- 172.21.15.22:0/1748845894 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510802b020 con 0x561f12a6a6a0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f50f77fe640 1 -- 172.21.15.22:0/1748845894 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f51080050b0 con 0x561f12a6a6a0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f510c034440 con 0x561f12a6a6a0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f50f77fe640 1 -- 172.21.15.22:0/1748845894 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f510801e040 con 0x561f12a6a6a0 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f512145c740 10 client.15978 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:28.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.473+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f12ad22e0 con 0x561f12a6a6a0 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f50f77fe640 1 -- 172.21.15.22:0/1748845894 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f510801f040 con 0x561f12a6a6a0 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f50f77fe640 1 -- 172.21.15.22:0/1748845894 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51080276a0 con 0x561f12a6a6a0 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f512145c740 10 client.15978 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f512145c740 10 client.15978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f512145c740 20 client.15978 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f512145c740 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12ad2870 0x561f12ad4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f512145c740 4 client.15978 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f510c034440 con 0x561f12ad2870 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.474+0000 7f51177fe640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12ad2870 0x561f12ad4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f51177fe640 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12ad2870 0x561f12ad4c60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f68c8c99b097418d server_cookie=c7cde51e9dc633c1 in_seq=0 out_seq=0 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f50f77fe640 10 client.15978 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f50f77fe640 1 -- 172.21.15.22:0/1748845894 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f510c034440 con 0x561f12ad2870 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f50f77fe640 10 client.15978 handle_command_reply: tid=0 2024-03-08T23:55:28.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 client.15978 shutdown 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 msgr2=0x561f12a708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a708e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5108009730 tx=0x7f5108005d30 comp rx=0 tx=0).stop 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 shutdown_connections 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561f12ad2870 0x561f12ad4c60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f12a7a880 0x561f12ad0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f50f77fe640 0 client.15978 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f12a6a6a0 0x561f12a708e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 --2- 172.21.15.22:0/1748845894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f12a61ac0 0x561f12a703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.475+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 >> 172.21.15.22:0/1748845894 conn(0x561f12a5b8d0 msgr2=0x561f12a66820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 shutdown_connections 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f512145c740 1 -- 172.21.15.22:0/1748845894 wait complete. 2024-03-08T23:55:28.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f512145c740 20 client.15978 trim_cache size 0 max 16384 2024-03-08T23:55:28.478 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:55:28.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51000a2b40 msgr2=0x7f51000a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51000a2b40 0x7f51000a5000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51100ccc30 tx=0x7f5110008ee0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118074070 msgr2=0x7f51181a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118074070 0x7f51181a5d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f511000cce0 tx=0x7f511000a620 comp rx=0 tx=0).stop 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 shutdown_connections 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51000a2b40 0x7f51000a5000 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5118074070 0x7f51181a5d20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51181065a0 0x7f5118073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 --2- 172.21.15.22:0/2064420804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51181014d0 0x7f51180735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 >> 172.21.15.22:0/2064420804 conn(0x7f51180fcd90 msgr2=0x7f51180ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 shutdown_connections 2024-03-08T23:55:28.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.476+0000 7f511e880640 1 -- 172.21.15.22:0/2064420804 wait complete. 2024-03-08T23:55:28.487 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:55:28.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 -- 172.21.15.22:0/1433050263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd238102510 msgr2=0x7fd2381028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 --2- 172.21.15.22:0/1433050263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd238102510 0x7fd2381028f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd228009900 tx=0x7fd22802bcd0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 -- 172.21.15.22:0/1433050263 shutdown_connections 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 --2- 172.21.15.22:0/1433050263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23810b850 0x7fd23810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 --2- 172.21.15.22:0/1433050263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238102e30 0x7fd23810b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 --2- 172.21.15.22:0/1433050263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd238102510 0x7fd2381028f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 -- 172.21.15.22:0/1433050263 >> 172.21.15.22:0/1433050263 conn(0x7fd2380773a0 msgr2=0x7fd2380777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.596+0000 7fd23ca27640 1 -- 172.21.15.22:0/1433050263 shutdown_connections 2024-03-08T23:55:28.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.597+0000 7fd23ca27640 1 -- 172.21.15.22:0/1433050263 wait complete. 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.597+0000 7fd23ca27640 1 Processor -- start 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.597+0000 7fd23ca27640 1 -- start start 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.597+0000 7fd23ca27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd238102510 0x7fd238199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.597+0000 7fd23ca27640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238102e30 0x7fd23819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd23ca27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd23810b850 0x7fd23819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd23ca27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238111610 con 0x7fd23810b850 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd23ca27640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238111490 con 0x7fd238102e30 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd23ca27640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238111790 con 0x7fd238102510 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd23810b850 0x7fd23819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd23810b850 0x7fd23819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39292/0 (socket says 172.21.15.22:39292) 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 -- 172.21.15.22:0/824920542 learned_addr learned my addr 172.21.15.22:0/824920542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd235d9b640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238102e30 0x7fd23819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd23659c640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd238102510 0x7fd238199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 -- 172.21.15.22:0/824920542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd238102510 msgr2=0x7fd238199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd238102510 0x7fd238199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 -- 172.21.15.22:0/824920542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238102e30 msgr2=0x7fd23819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238102e30 0x7fd23819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 -- 172.21.15.22:0/824920542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23819ed20 con 0x7fd23810b850 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd236d9d640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd23810b850 0x7fd23819e5f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd23000cad0 tx=0x7fd23000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.598+0000 7fd2277fe640 1 -- 172.21.15.22:0/824920542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23000ae20 con 0x7fd23810b850 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.599+0000 7fd2277fe640 1 -- 172.21.15.22:0/824920542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd230005020 con 0x7fd23810b850 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.599+0000 7fd2277fe640 1 -- 172.21.15.22:0/824920542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23000f6f0 con 0x7fd23810b850 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.599+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23819f010 con 0x7fd23810b850 2024-03-08T23:55:28.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.599+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2381a68a0 con 0x7fd23810b850 2024-03-08T23:55:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.601+0000 7fd2277fe640 1 -- 172.21.15.22:0/824920542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fd2300051d0 con 0x7fd23810b850 2024-03-08T23:55:28.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.602+0000 7fd23f609740 1 Processor -- start 2024-03-08T23:55:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.602+0000 7fd2277fe640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2200a2b10 0x7fd2200a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.602+0000 7fd2277fe640 1 -- 172.21.15.22:0/824920542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd2300cf370 con 0x7fd23810b850 2024-03-08T23:55:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.602+0000 7fd235d9b640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2200a2b10 0x7fd2200a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.602+0000 7fd23f609740 1 -- start start 2024-03-08T23:55:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23f609740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf710b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23f609740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf710bac0 0x557cf7114160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23f609740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf71146a0 0x557cf7116a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23f609740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf70d2360 con 0x557cf710bac0 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23f609740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf706eec0 con 0x557cf710b1a0 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23f609740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf7017d30 con 0x557cf71146a0 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf710b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf710b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57744/0 (socket says 172.21.15.22:57744) 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23659c640 1 -- 172.21.15.22:0/4183666970 learned_addr learned my addr 172.21.15.22:0/4183666970 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd235d9b640 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf710bac0 0x557cf7114160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd236d9d640 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf71146a0 0x557cf7116a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd235d9b640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2200a2b10 0x7fd2200a4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd23810b240 tx=0x7fd22c00b390 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23659c640 1 -- 172.21.15.22:0/4183666970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf71146a0 msgr2=0x557cf7116a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23659c640 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf71146a0 0x557cf7116a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23659c640 1 -- 172.21.15.22:0/4183666970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf710bac0 msgr2=0x557cf7114160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.603+0000 7fd23659c640 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf710bac0 0x557cf7114160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.604+0000 7fd23659c640 1 -- 172.21.15.22:0/4183666970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cf7016bc0 con 0x557cf710b1a0 2024-03-08T23:55:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.604+0000 7fd236d9d640 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf71146a0 0x557cf7116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.604+0000 7fd23659c640 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf710b580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd228009230 tx=0x7fd228005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7257b5b9f302b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.604+0000 7fd224ff9640 1 -- 172.21.15.22:0/4183666970 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228042070 con 0x557cf710b1a0 2024-03-08T23:55:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.604+0000 7fd224ff9640 1 -- 172.21.15.22:0/4183666970 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd22802cb20 con 0x557cf710b1a0 2024-03-08T23:55:28.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.604+0000 7fd224ff9640 1 -- 172.21.15.22:0/4183666970 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228004f10 con 0x557cf710b1a0 2024-03-08T23:55:28.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 -- 172.21.15.22:0/4183666970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 msgr2=0x557cf710b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf710b580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd228009230 tx=0x7fd228005e00 comp rx=0 tx=0).stop 2024-03-08T23:55:28.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 -- 172.21.15.22:0/4183666970 shutdown_connections 2024-03-08T23:55:28.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf71146a0 0x557cf7116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf710bac0 0x557cf7114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 --2- 172.21.15.22:0/4183666970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf710b580 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 -- 172.21.15.22:0/4183666970 >> 172.21.15.22:0/4183666970 conn(0x557cf71058d0 msgr2=0x557cf7105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 -- 172.21.15.22:0/4183666970 shutdown_connections 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 -- 172.21.15.22:0/4183666970 wait complete. 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 -- start start 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.605+0000 7fd23f609740 1 Processor -- start 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23f609740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7111720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23f609740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf7111c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23f609740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf71121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23f609740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf700d4a0 con 0x557cf71146a0 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23f609740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf706af60 con 0x557cf710bac0 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23f609740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf7117430 con 0x557cf710b1a0 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf71121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf71121a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39314/0 (socket says 172.21.15.22:39314) 2024-03-08T23:55:28.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 -- 172.21.15.22:0/1720450353 learned_addr learned my addr 172.21.15.22:0/1720450353 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23659c640 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7111720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd23f609740 10 client.0 _ll_register_callbacks cb 0x557cf7068d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd235d9b640 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf7111c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 -- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 msgr2=0x557cf7111720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7111720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 -- 172.21.15.22:0/1720450353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 msgr2=0x557cf7111c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf7111c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.606+0000 7fd236d9d640 1 -- 172.21.15.22:0/1720450353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cf6d833f0 con 0x557cf71146a0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd236d9d640 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf71121a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd2300978f0 tx=0x7fd23009aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e03f4797340a120 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd21dffb640 1 -- 172.21.15.22:0/1720450353 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300b4060 con 0x557cf71146a0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd21dffb640 1 -- 172.21.15.22:0/1720450353 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd230005560 con 0x557cf71146a0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd22802cb20 con 0x557cf71146a0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd21dffb640 1 -- 172.21.15.22:0/1720450353 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd23001d030 con 0x557cf71146a0 2024-03-08T23:55:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd23f609740 10 client.15990 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cf6e4d7d0 con 0x557cf71146a0 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd21dffb640 1 -- 172.21.15.22:0/1720450353 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd22802cb20 con 0x557cf71146a0 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.607+0000 7fd21dffb640 1 -- 172.21.15.22:0/1720450353 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2300cc880 con 0x557cf71146a0 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23f609740 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23f609740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23f609740 20 client.15990 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23f609740 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf7124840 0x557cf7126c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23f609740 4 client.15990 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cf707c400 con 0x557cf7124840 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23659c640 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf7124840 0x557cf7126c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd21dffb640 10 client.15990 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.608+0000 7fd23659c640 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf7124840 0x557cf7126c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d90e818c2a7fc740 server_cookie=a1e99749d5f30b04 in_seq=0 out_seq=0 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd21dffb640 1 -- 172.21.15.22:0/1720450353 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fd228052a90 con 0x557cf7124840 2024-03-08T23:55:28.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd21dffb640 10 client.15990 handle_command_reply: tid=0 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 client.15990 shutdown 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 msgr2=0x557cf71121a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf71121a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd2300978f0 tx=0x7fd23009aee0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 shutdown_connections 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf7124840 0x557cf7126c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf71121a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf7111c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd21dffb640 0 client.15990 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 --2- 172.21.15.22:0/1720450353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7111720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 >> 172.21.15.22:0/1720450353 conn(0x557cf71058d0 msgr2=0x557cf7107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 shutdown_connections 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 1 -- 172.21.15.22:0/1720450353 wait complete. 2024-03-08T23:55:28.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.609+0000 7fd23f609740 20 client.15990 trim_cache size 0 max 16384 2024-03-08T23:55:28.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 Processor -- start 2024-03-08T23:55:28.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 -- start start 2024-03-08T23:55:28.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7113e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf710c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf710cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381b2ca0 con 0x557cf71146a0 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381b0840 con 0x557cf710bac0 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23f609740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381ae8e0 con 0x557cf710b1a0 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf710cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf710cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39326/0 (socket says 172.21.15.22:39326) 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd236d9d640 1 -- 172.21.15.22:0/1595731597 learned_addr learned my addr 172.21.15.22:0/1595731597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.621+0000 7fd23659c640 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd235d9b640 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf710c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd236d9d640 1 -- 172.21.15.22:0/1595731597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 msgr2=0x557cf7113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd236d9d640 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd236d9d640 1 -- 172.21.15.22:0/1595731597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 msgr2=0x557cf710c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd236d9d640 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf710c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd236d9d640 1 -- 172.21.15.22:0/1595731597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cf710fc70 con 0x557cf71146a0 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd235d9b640 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf710c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd236d9d640 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf710cf20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd23009b420 tx=0x7fd230005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303b9fe3f40e98ae server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd21ffff640 1 -- 172.21.15.22:0/1595731597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300cc880 con 0x557cf71146a0 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd21ffff640 1 -- 172.21.15.22:0/1595731597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2300cca20 con 0x557cf71146a0 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.622+0000 7fd21ffff640 1 -- 172.21.15.22:0/1595731597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300a36f0 con 0x557cf71146a0 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- 172.21.15.22:0/1595731597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 msgr2=0x557cf710cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf710cf20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd23009b420 tx=0x7fd230005960 comp rx=0 tx=0).stop 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- 172.21.15.22:0/1595731597 shutdown_connections 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf710cf20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710bac0 0x557cf710c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 --2- 172.21.15.22:0/1595731597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710b1a0 0x557cf7113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- 172.21.15.22:0/1595731597 >> 172.21.15.22:0/1595731597 conn(0x557cf71058d0 msgr2=0x557cf7105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- 172.21.15.22:0/1595731597 shutdown_connections 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- 172.21.15.22:0/1595731597 wait complete. 2024-03-08T23:55:28.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- start start 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 Processor -- start 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf7174090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710bac0 0x557cf71745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf711a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381aeee0 con 0x557cf71146a0 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381b01e0 con 0x557cf710b1a0 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.623+0000 7fd23f609740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381af520 con 0x557cf710bac0 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd23f609740 10 client.0 _ll_register_callbacks cb 0x7fd228009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd236d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf711a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd235d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710bac0 0x557cf71745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd235d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710bac0 0x557cf71745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:46032/0 (socket says 172.21.15.22:46032) 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd235d9b640 1 -- 172.21.15.22:0/474945973 learned_addr learned my addr 172.21.15.22:0/474945973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd23659c640 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf7174090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd236d9d640 1 -- 172.21.15.22:0/474945973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710bac0 msgr2=0x557cf71745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd236d9d640 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710bac0 0x557cf71745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd236d9d640 1 -- 172.21.15.22:0/474945973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 msgr2=0x557cf7174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd236d9d640 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf7174090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd236d9d640 1 -- 172.21.15.22:0/474945973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cf711b060 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd236d9d640 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf711a880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x557cf71270d0 tx=0x7fd230005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aab8d19decd714d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd21dffb640 1 -- 172.21.15.22:0/474945973 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300cc880 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd21dffb640 1 -- 172.21.15.22:0/474945973 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd2300a36f0 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.624+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2300cca20 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd21dffb640 1 -- 172.21.15.22:0/474945973 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2300b4260 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 10 client.15996 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cf717c790 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd21dffb640 1 -- 172.21.15.22:0/474945973 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2300cca20 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd21dffb640 1 -- 172.21.15.22:0/474945973 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd23009a350 con 0x557cf71146a0 2024-03-08T23:55:28.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:28.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:28.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 20 client.15996 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:28.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf717cb20 0x557cf717ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:28.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 4 client.15996 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:55:28.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557cf7076750 con 0x557cf717cb20 2024-03-08T23:55:28.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.625+0000 7fd23659c640 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf717cb20 0x557cf717ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:28.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd21dffb640 10 client.15996 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:28.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23659c640 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf717cb20 0x557cf717ef10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa744fc9ea77416e server_cookie=23675ac61f5436d9 in_seq=0 out_seq=0 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd21dffb640 1 -- 172.21.15.22:0/474945973 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557cf7076750 con 0x557cf717cb20 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd21dffb640 10 client.15996 handle_command_reply: tid=0 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 client.15996 shutdown 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 msgr2=0x557cf711a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf711a880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x557cf71270d0 tx=0x7fd230005b40 comp rx=0 tx=0).stop 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 shutdown_connections 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557cf717cb20 0x557cf717ef10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cf71146a0 0x557cf711a880 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557cf710bac0 0x557cf71745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 --2- 172.21.15.22:0/474945973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557cf710b1a0 0x557cf7174090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 >> 172.21.15.22:0/474945973 conn(0x557cf71058d0 msgr2=0x557cf710c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd21dffb640 0 client.15996 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 shutdown_connections 2024-03-08T23:55:28.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.626+0000 7fd23f609740 1 -- 172.21.15.22:0/474945973 wait complete. 2024-03-08T23:55:28.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23f609740 20 client.15996 trim_cache size 0 max 16384 2024-03-08T23:55:28.629 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2200a2b10 msgr2=0x7fd2200a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2200a2b10 0x7fd2200a4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd23810b240 tx=0x7fd22c00b390 comp rx=0 tx=0).stop 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd23810b850 msgr2=0x7fd23819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd23810b850 0x7fd23819e5f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd23000cad0 tx=0x7fd23000cf90 comp rx=0 tx=0).stop 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 shutdown_connections 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2200a2b10 0x7fd2200a4fd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd23810b850 0x7fd23819e5f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238102e30 0x7fd23819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 --2- 172.21.15.22:0/824920542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd238102510 0x7fd238199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.627+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 >> 172.21.15.22:0/824920542 conn(0x7fd2380773a0 msgr2=0x7fd2381077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.628+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 shutdown_connections 2024-03-08T23:55:28.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:28.628+0000 7fd23ca27640 1 -- 172.21.15.22:0/824920542 wait complete. 2024-03-08T23:55:28.977 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:28 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3173385916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:55:29.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:28 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3173385916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:55:29.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:28 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3173385916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:55:29.639 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:55:29.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:55:29.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:55:29.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3663719186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0068760 msgr2=0x7fbaa0068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3663719186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0068760 0x7fbaa0068b40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fba94009900 tx=0x7fba9402bc30 comp rx=0 tx=0).stop 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3663719186 shutdown_connections 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3663719186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaa0106630 0x7fbaa010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3663719186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0069110 0x7fbaa0105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3663719186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0068760 0x7fbaa0068b40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3663719186 >> 172.21.15.22:0/3663719186 conn(0x7fbaa00772f0 msgr2=0x7fbaa0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3663719186 shutdown_connections 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3663719186 wait complete. 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 Processor -- start 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.800+0000 7fbaa6f09640 1 -- start start 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa6f09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0068760 0x7fbaa01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa6f09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0106630 0x7fbaa01006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa6f09640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaa0100c30 0x7fbaa01a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa6f09640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa0111690 con 0x7fbaa0106630 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa6f09640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa0111510 con 0x7fbaa0068760 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa6f09640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa0111810 con 0x7fbaa0100c30 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0068760 0x7fbaa01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fba9ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0106630 0x7fbaa01006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0068760 0x7fbaa01001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57780/0 (socket says 172.21.15.22:57780) 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 -- 172.21.15.22:0/3608669359 learned_addr learned my addr 172.21.15.22:0/3608669359 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaa0100c30 0x7fbaa01a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 -- 172.21.15.22:0/3608669359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaa0100c30 msgr2=0x7fbaa01a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaa0100c30 0x7fbaa01a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 -- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0106630 msgr2=0x7fbaa01006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0106630 0x7fbaa01006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fbaa4ca5640 1 -- 172.21.15.22:0/3608669359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa01a6460 con 0x7fbaa0068760 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.801+0000 7fba9ffff640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0106630 0x7fbaa01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.802+0000 7fbaa4ca5640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0068760 0x7fbaa01001b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba94006010 tx=0x7fba9402cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.802+0000 7fba9dffb640 1 -- 172.21.15.22:0/3608669359 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9404d070 con 0x7fbaa0068760 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.802+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa01978e0 con 0x7fbaa0068760 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.802+0000 7fba9dffb640 1 -- 172.21.15.22:0/3608669359 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba94040de0 con 0x7fbaa0068760 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.802+0000 7fba9dffb640 1 -- 172.21.15.22:0/3608669359 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94003850 con 0x7fbaa0068760 2024-03-08T23:55:29.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.802+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa0197e20 con 0x7fbaa0068760 2024-03-08T23:55:29.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 Processor -- start 2024-03-08T23:55:29.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 -- start start 2024-03-08T23:55:29.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed26a0 0x55a109edad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109edb280 0x55a109edd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109e35b40 con 0x55a109ed1d80 2024-03-08T23:55:29.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109dded80 con 0x55a109edb280 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109e31be0 con 0x55a109ed26a0 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39370/0 (socket says 172.21.15.22:39370) 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1171126930 learned_addr learned my addr 172.21.15.22:0/1171126930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fba9ffff640 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109edb280 0x55a109edd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1171126930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed26a0 msgr2=0x55a109edad40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed26a0 0x55a109edad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1171126930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109edb280 msgr2=0x55a109edd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109edb280 0x55a109edd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1171126930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a109c127e0 con 0x55a109ed1d80 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.805+0000 7fba9dffb640 1 -- 172.21.15.22:0/3608669359 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fba94050070 con 0x7fbaa0068760 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.806+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed2160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fba98009f50 tx=0x7fba980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df53032f88a3f171 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.806+0000 7fba8affd640 1 -- 172.21.15.22:0/1171126930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98004d20 con 0x55a109ed1d80 2024-03-08T23:55:29.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.806+0000 7fba8affd640 1 -- 172.21.15.22:0/1171126930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba98004050 con 0x55a109ed1d80 2024-03-08T23:55:29.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.806+0000 7fba8affd640 1 -- 172.21.15.22:0/1171126930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98005280 con 0x55a109ed1d80 2024-03-08T23:55:29.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.806+0000 7fba9dffb640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fba900a2bc0 0x7fba900a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.806+0000 7fba9dffb640 1 -- 172.21.15.22:0/3608669359 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fba940f89c0 con 0x7fbaa0068760 2024-03-08T23:55:29.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.806+0000 7fbaa4ca5640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fba900a2bc0 0x7fba900a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1171126930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 msgr2=0x55a109ed2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa4ca5640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fba900a2bc0 0x7fba900a5080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba940f5d00 tx=0x7fba94040f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed2160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fba98009f50 tx=0x7fba980118d0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1171126930 shutdown_connections 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109edb280 0x55a109edd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed26a0 0x55a109edad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1171126930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed2160 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1171126930 >> 172.21.15.22:0/1171126930 conn(0x55a109ecc4b0 msgr2=0x55a109ecc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1171126930 shutdown_connections 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1171126930 wait complete. 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.807+0000 7fbaa9bc8740 1 -- start start 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 1 Processor -- start 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 0x55a109edace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109ed8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109ed8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109ede070 con 0x55a109edb280 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109df21c0 con 0x55a109ed26a0 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109ede1f0 con 0x55a109ed1d80 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa9bc8740 10 client.0 _ll_register_callbacks cb 0x55a109de17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa54a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109ed8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa54a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109ed8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39378/0 (socket says 172.21.15.22:39378) 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1637927617 learned_addr learned my addr 172.21.15.22:0/1637927617 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 0x55a109edace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.808+0000 7fba9ffff640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109ed8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 msgr2=0x55a109edace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 0x55a109edace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1637927617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 msgr2=0x55a109ed8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109ed8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa54a6640 1 -- 172.21.15.22:0/1637927617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a109c72f70 con 0x55a109edb280 2024-03-08T23:55:29.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fba9ffff640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109ed8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109ed8790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fba98009250 tx=0x7fba980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2153a2146e1dbb43 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fba7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fba7ffff640 1 -- 172.21.15.22:0/1637927617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98028650 con 0x55a109edb280 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fba7ffff640 1 -- 172.21.15.22:0/1637927617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fba980287f0 con 0x55a109edb280 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba98004050 con 0x55a109edb280 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fba7ffff640 1 -- 172.21.15.22:0/1637927617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba98036050 con 0x55a109edb280 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa9bc8740 10 client.16008 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.809+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a109b0a170 con 0x55a109edb280 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fba7ffff640 1 -- 172.21.15.22:0/1637927617 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba98004050 con 0x55a109edb280 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fba7ffff640 1 -- 172.21.15.22:0/1637927617 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fba9800fa20 con 0x55a109edb280 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fbaa9bc8740 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fbaa9bc8740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fbaa9bc8740 20 client.16008 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109eeb2c0 0x55a109eed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fbaa9bc8740 4 client.16008 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a109e0b960 con 0x55a109eeb2c0 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.810+0000 7fba9ffff640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109eeb2c0 0x55a109eed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.811+0000 7fba7ffff640 10 client.16008 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.811+0000 7fba9ffff640 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109eeb2c0 0x55a109eed6b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe38f810a38dd6d server_cookie=529af854ade06141 in_seq=0 out_seq=0 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.811+0000 7fba7ffff640 1 -- 172.21.15.22:0/1637927617 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a109e0b960 con 0x55a109eeb2c0 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.811+0000 7fba7ffff640 10 client.16008 handle_command_reply: tid=0 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.811+0000 7fbaa9bc8740 1 client.16008 shutdown 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 msgr2=0x55a109ed8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109ed8790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fba98009250 tx=0x7fba980047b0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 shutdown_connections 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109eeb2c0 0x55a109eed6b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109ed8790 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109ed8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/1637927617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 0x55a109edace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 >> 172.21.15.22:0/1637927617 conn(0x55a109ecc4b0 msgr2=0x55a109ece200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fba7ffff640 0 client.16008 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 shutdown_connections 2024-03-08T23:55:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/1637927617 wait complete. 2024-03-08T23:55:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.812+0000 7fbaa9bc8740 20 client.16008 trim_cache size 0 max 16384 2024-03-08T23:55:29.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.827+0000 7fbaa9bc8740 1 Processor -- start 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.827+0000 7fbaa9bc8740 1 -- start start 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.827+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.827+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109eda7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109edb280 0x55a109edace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01ae8c0 con 0x55a109ed1d80 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01ac460 con 0x55a109ed26a0 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01aa500 con 0x55a109edb280 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39390/0 (socket says 172.21.15.22:39390) 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 -- 172.21.15.22:0/519253069 learned_addr learned my addr 172.21.15.22:0/519253069 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fba9ffff640 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109eda7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 -- 172.21.15.22:0/519253069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109edb280 msgr2=0x55a109edace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109edb280 0x55a109edace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 -- 172.21.15.22:0/519253069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 msgr2=0x55a109eda7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109eda7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 -- 172.21.15.22:0/519253069 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a109ed24d0 con 0x55a109ed1d80 2024-03-08T23:55:29.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed7050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba98005d70 tx=0x7fba98027470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12696b9d44b309c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.828+0000 7fba8a7fc640 1 -- 172.21.15.22:0/519253069 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9803f070 con 0x55a109ed1d80 2024-03-08T23:55:29.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.829+0000 7fba8a7fc640 1 -- 172.21.15.22:0/519253069 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba9802d070 con 0x55a109ed1d80 2024-03-08T23:55:29.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.829+0000 7fba8a7fc640 1 -- 172.21.15.22:0/519253069 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98004950 con 0x55a109ed1d80 2024-03-08T23:55:29.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.829+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/519253069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 msgr2=0x55a109ed7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.829+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed7050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba98005d70 tx=0x7fba98027470 comp rx=0 tx=0).stop 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/519253069 shutdown_connections 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109edb280 0x55a109edace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109eda7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/519253069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109ed1d80 0x55a109ed7050 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/519253069 >> 172.21.15.22:0/519253069 conn(0x55a109ecc4b0 msgr2=0x55a109ed6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/519253069 shutdown_connections 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/519253069 wait complete. 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 -- start start 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.830+0000 7fbaa9bc8740 1 Processor -- start 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 0x55a109edee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109edf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa9bc8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109edf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01aab00 con 0x55a109edb280 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01abe00 con 0x55a109ed26a0 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa9bc8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01ab140 con 0x55a109ed1d80 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa9bc8740 10 client.0 _ll_register_callbacks cb 0x7fba98009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109edf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109edf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57818/0 (socket says 172.21.15.22:57818) 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 -- 172.21.15.22:0/143948814 learned_addr learned my addr 172.21.15.22:0/143948814 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109edf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 -- 172.21.15.22:0/143948814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 msgr2=0x55a109edee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 0x55a109edee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 -- 172.21.15.22:0/143948814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 msgr2=0x55a109edf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109edf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fba9ffff640 1 -- 172.21.15.22:0/143948814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a109f415e0 con 0x55a109ed26a0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.831+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109edf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fba9ffff640 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109edf340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbaa006aa50 tx=0x7fba8c010ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a08eabc76b3a287 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fba7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fba7ffff640 1 -- 172.21.15.22:0/143948814 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c004440 con 0x55a109ed26a0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba9802d070 con 0x55a109ed26a0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fba7ffff640 1 -- 172.21.15.22:0/143948814 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fba8c01e410 con 0x55a109ed26a0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fba7ffff640 1 -- 172.21.15.22:0/143948814 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba8c02c050 con 0x55a109ed26a0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fbaa9bc8740 10 client.25523 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a109f42960 con 0x55a109ed26a0 2024-03-08T23:55:29.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.832+0000 7fba7ffff640 1 -- 172.21.15.22:0/143948814 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba9802d070 con 0x55a109ed26a0 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fba7ffff640 1 -- 172.21.15.22:0/143948814 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fba8c003710 con 0x55a109ed26a0 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa9bc8740 10 client.25523 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa9bc8740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa9bc8740 20 client.25523 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109f430f0 0x55a109f45510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa9bc8740 4 client.25523 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a109df7080 con 0x55a109f430f0 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109f430f0 0x55a109f45510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fba7ffff640 10 client.25523 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.833+0000 7fbaa54a6640 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109f430f0 0x55a109f45510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83d353ef3f46191 server_cookie=d876e2a8685d2cbc in_seq=0 out_seq=0 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fba7ffff640 1 -- 172.21.15.22:0/143948814 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a109df7080 con 0x55a109f430f0 2024-03-08T23:55:29.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fba7ffff640 10 client.25523 handle_command_reply: tid=0 2024-03-08T23:55:29.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 client.25523 shutdown 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 msgr2=0x55a109edf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109edf340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbaa006aa50 tx=0x7fba8c010ba0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 shutdown_connections 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a109f430f0 0x55a109f45510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a109edb280 0x55a109edf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a109ed26a0 0x55a109edf340 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbaa006aa50 tx=0x7fba8c010ba0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 --2- 172.21.15.22:0/143948814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a109ed1d80 0x55a109edee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 >> 172.21.15.22:0/143948814 conn(0x55a109ecc4b0 msgr2=0x55a109ecd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fba7ffff640 0 client.25523 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 shutdown_connections 2024-03-08T23:55:29.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 1 -- 172.21.15.22:0/143948814 wait complete. 2024-03-08T23:55:29.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.835+0000 7fbaa9bc8740 20 client.25523 trim_cache size 0 max 16384 2024-03-08T23:55:29.837 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:55:29.837 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:55:29.837 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "944e0059-efbd-432f-b666-44256dda7698", 2024-03-08T23:55:29.837 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:55:29.837 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:55:29.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fba900a2bc0 msgr2=0x7fba900a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fba900a2bc0 0x7fba900a5080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba940f5d00 tx=0x7fba94040f90 comp rx=0 tx=0).stop 2024-03-08T23:55:29.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0068760 msgr2=0x7fbaa01001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0068760 0x7fbaa01001b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba94006010 tx=0x7fba9402cb30 comp rx=0 tx=0).stop 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 shutdown_connections 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fba900a2bc0 0x7fba900a5080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaa0100c30 0x7fbaa01a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaa0106630 0x7fbaa01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 --2- 172.21.15.22:0/3608669359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa0068760 0x7fbaa01001b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 >> 172.21.15.22:0/3608669359 conn(0x7fbaa00772f0 msgr2=0x7fbaa00fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 shutdown_connections 2024-03-08T23:55:29.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.836+0000 7fbaa6f09640 1 -- 172.21.15.22:0/3608669359 wait complete. 2024-03-08T23:55:29.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:55:29.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:55:29.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 -- 172.21.15.22:0/416948529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60069110 msgr2=0x7f9a60105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 --2- 172.21.15.22:0/416948529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60069110 0x7f9a60105f50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009900 tx=0x7f9a5002bc30 comp rx=0 tx=0).stop 2024-03-08T23:55:29.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 -- 172.21.15.22:0/416948529 shutdown_connections 2024-03-08T23:55:29.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 --2- 172.21.15.22:0/416948529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60106630 0x7f9a6010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 --2- 172.21.15.22:0/416948529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60069110 0x7f9a60105f50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 --2- 172.21.15.22:0/416948529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60068760 0x7f9a60068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 -- 172.21.15.22:0/416948529 >> 172.21.15.22:0/416948529 conn(0x7f9a600772f0 msgr2=0x7f9a60077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 -- 172.21.15.22:0/416948529 shutdown_connections 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.956+0000 7f9a67369640 1 -- 172.21.15.22:0/416948529 wait complete. 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 Processor -- start 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 -- start start 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60068760 0x7f9a6019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60106630 0x7f9a6019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60197990 0x7f9a60197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60111690 con 0x7f9a60068760 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60111510 con 0x7f9a60197990 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a67369640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60111810 con 0x7f9a60106630 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65105640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60068760 0x7f9a6019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65105640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60068760 0x7f9a6019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39434/0 (socket says 172.21.15.22:39434) 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65105640 1 -- 172.21.15.22:0/2228523108 learned_addr learned my addr 172.21.15.22:0/2228523108 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65906640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60197990 0x7f9a60197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a64904640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60106630 0x7f9a6019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65105640 1 -- 172.21.15.22:0/2228523108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60106630 msgr2=0x7f9a6019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65105640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60106630 0x7f9a6019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65105640 1 -- 172.21.15.22:0/2228523108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60197990 msgr2=0x7f9a60197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.957+0000 7f9a65105640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60197990 0x7f9a60197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.958+0000 7f9a65105640 1 -- 172.21.15.22:0/2228523108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a601984b0 con 0x7f9a60068760 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.958+0000 7f9a65906640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60197990 0x7f9a60197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.958+0000 7f9a65105640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60068760 0x7f9a6019d7e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00d770 tx=0x7f9a5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.958+0000 7f9a567fc640 1 -- 172.21.15.22:0/2228523108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c017070 con 0x7f9a60068760 2024-03-08T23:55:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.958+0000 7f9a567fc640 1 -- 172.21.15.22:0/2228523108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a5c005500 con 0x7f9a60068760 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.959+0000 7f9a567fc640 1 -- 172.21.15.22:0/2228523108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c004e00 con 0x7f9a60068760 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.959+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a601987a0 con 0x7f9a60068760 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.959+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a601a49f0 con 0x7f9a60068760 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.961+0000 7f9a6a026740 1 Processor -- start 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.961+0000 7f9a6a026740 1 -- start start 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.961+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a68eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.961+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a697720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.961+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a697c60 0x56147a69a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.961+0000 7f9a6a026740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a5f1e00 con 0x56147a697c60 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a6a026740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a59ab10 con 0x56147a68f080 2024-03-08T23:55:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a6a026740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a5edea0 con 0x56147a68e760 2024-03-08T23:55:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a697c60 0x56147a69a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a65906640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a68eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a697c60 0x56147a69a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39450/0 (socket says 172.21.15.22:39450) 2024-03-08T23:55:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 -- 172.21.15.22:0/654424171 learned_addr learned my addr 172.21.15.22:0/654424171 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a567fc640 1 -- 172.21.15.22:0/2228523108 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f9a5c004fa0 con 0x7f9a60068760 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 -- 172.21.15.22:0/654424171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 msgr2=0x56147a68eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 --2- 172.21.15.22:0/654424171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a68eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 -- 172.21.15.22:0/654424171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 msgr2=0x56147a697720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 --2- 172.21.15.22:0/654424171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a697720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.962+0000 7f9a64904640 1 -- 172.21.15.22:0/654424171 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56147a3d07d0 con 0x56147a697c60 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.963+0000 7f9a567fc640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9a4c0a2b10 0x7f9a4c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.963+0000 7f9a64904640 1 --2- 172.21.15.22:0/654424171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a697c60 0x56147a69a050 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009900 tx=0x7f9a50005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839b0063dc0359cc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.963+0000 7f9a567fc640 1 -- 172.21.15.22:0/2228523108 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9a5c0ceb90 con 0x7f9a60068760 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.963+0000 7f9a65906640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9a4c0a2b10 0x7f9a4c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.963+0000 7f9a4b7fe640 1 -- 172.21.15.22:0/654424171 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50044020 con 0x56147a697c60 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.963+0000 7f9a4b7fe640 1 -- 172.21.15.22:0/654424171 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a5002c880 con 0x56147a697c60 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.963+0000 7f9a4b7fe640 1 -- 172.21.15.22:0/654424171 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500046c0 con 0x56147a697c60 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 -- 172.21.15.22:0/654424171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a697c60 msgr2=0x56147a69a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a65906640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9a4c0a2b10 0x7f9a4c0a4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009fd0 tx=0x7f9a58009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 --2- 172.21.15.22:0/654424171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a697c60 0x56147a69a050 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009900 tx=0x7f9a50005d50 comp rx=0 tx=0).stop 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 -- 172.21.15.22:0/654424171 shutdown_connections 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 --2- 172.21.15.22:0/654424171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a697c60 0x56147a69a050 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 --2- 172.21.15.22:0/654424171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a697720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 --2- 172.21.15.22:0/654424171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a68eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 -- 172.21.15.22:0/654424171 >> 172.21.15.22:0/654424171 conn(0x56147a688e90 msgr2=0x56147a689270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 -- 172.21.15.22:0/654424171 shutdown_connections 2024-03-08T23:55:29.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.964+0000 7f9a6a026740 1 -- 172.21.15.22:0/654424171 wait complete. 2024-03-08T23:55:29.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 -- start start 2024-03-08T23:55:29.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 Processor -- start 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a694900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a69a9f0 con 0x56147a694e40 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a5ae0a0 con 0x56147a68f080 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a69ab70 con 0x56147a68e760 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.965+0000 7f9a6a026740 10 client.0 _ll_register_callbacks cb 0x56147a5ada70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39454/0 (socket says 172.21.15.22:39454) 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 -- 172.21.15.22:0/101018900 learned_addr learned my addr 172.21.15.22:0/101018900 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65105640 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 -- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 msgr2=0x56147a697610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 -- 172.21.15.22:0/101018900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 msgr2=0x56147a694900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a694900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 -- 172.21.15.22:0/101018900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56147a3063f0 con 0x56147a694e40 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a65906640 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6952a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9a58003e20 tx=0x7f9a58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e373e6a30e89cf90 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a48ff9640 1 -- 172.21.15.22:0/101018900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a58004550 con 0x56147a694e40 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a48ff9640 1 -- 172.21.15.22:0/101018900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9a58018030 con 0x56147a694e40 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.966+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a5002c880 con 0x56147a694e40 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a48ff9640 1 -- 172.21.15.22:0/101018900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a580046f0 con 0x56147a694e40 2024-03-08T23:55:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 10 client.16026 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56147a2c60f0 con 0x56147a694e40 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a48ff9640 1 -- 172.21.15.22:0/101018900 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a5002c880 con 0x56147a694e40 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a48ff9640 1 -- 172.21.15.22:0/101018900 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a58028420 con 0x56147a694e40 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 20 client.16026 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a6aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 4 client.16026 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.967+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56147a5ff340 con 0x56147a6a8170 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.968+0000 7f9a65105640 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a6aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.968+0000 7f9a65105640 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a6aa5f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fe76694a294cd4 server_cookie=daa8ff54af85be5a in_seq=0 out_seq=0 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.968+0000 7f9a48ff9640 10 client.16026 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a48ff9640 1 -- 172.21.15.22:0/101018900 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56147a5ff340 con 0x56147a6a8170 2024-03-08T23:55:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a48ff9640 10 client.16026 handle_command_reply: tid=0 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 client.16026 shutdown 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 msgr2=0x56147a6952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6952a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9a58003e20 tx=0x7f9a58004ab0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 shutdown_connections 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a6aa5f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6952a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a48ff9640 0 client.16026 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a694900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 --2- 172.21.15.22:0/101018900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 >> 172.21.15.22:0/101018900 conn(0x56147a688e90 msgr2=0x56147a68abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 shutdown_connections 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 1 -- 172.21.15.22:0/101018900 wait complete. 2024-03-08T23:55:29.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:29.969+0000 7f9a6a026740 20 client.16026 trim_cache size 0 max 16384 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.000+0000 7f9a6a026740 1 Processor -- start 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.000+0000 7f9a6a026740 1 -- start start 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.000+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.000+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a691b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.000+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a691b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a691b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57872/0 (socket says 172.21.15.22:57872) 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a65906640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a65906640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6920a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39458/0 (socket says 172.21.15.22:39458) 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a65105640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 -- 172.21.15.22:0/3187963198 learned_addr learned my addr 172.21.15.22:0/3187963198 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601b0a80 con 0x56147a694e40 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601ae620 con 0x56147a68f080 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601ac6c0 con 0x56147a68e760 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 -- 172.21.15.22:0/3187963198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 msgr2=0x56147a697140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 -- 172.21.15.22:0/3187963198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 msgr2=0x56147a6920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a64904640 1 -- 172.21.15.22:0/3187963198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56147a6af820 con 0x56147a68f080 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a65906640 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.001+0000 7f9a65105640 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.002+0000 7f9a64904640 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a691b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x56147a68a020 tx=0x7f9a5002c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72aafac123befca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.002+0000 7f9a4affd640 1 -- 172.21.15.22:0/3187963198 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5004d020 con 0x56147a68f080 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.002+0000 7f9a4affd640 1 -- 172.21.15.22:0/3187963198 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a50005510 con 0x56147a68f080 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.002+0000 7f9a4affd640 1 -- 172.21.15.22:0/3187963198 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50044030 con 0x56147a68f080 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 msgr2=0x56147a691b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a691b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x56147a68a020 tx=0x7f9a5002c7e0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 shutdown_connections 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a694e40 0x56147a6920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a691b60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3187963198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a68e760 0x56147a697140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 >> 172.21.15.22:0/3187963198 conn(0x56147a688e90 msgr2=0x56147a6931d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 shutdown_connections 2024-03-08T23:55:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.003+0000 7f9a6a026740 1 -- 172.21.15.22:0/3187963198 wait complete. 2024-03-08T23:55:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.004+0000 7f9a6a026740 1 -- start start 2024-03-08T23:55:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.004+0000 7f9a6a026740 1 Processor -- start 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.004+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a68e760 0x56147a69ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.004+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a69d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.004+0000 7f9a6a026740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a694e40 0x56147a69d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a65105640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a68e760 0x56147a69ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a65105640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a68e760 0x56147a69ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39466/0 (socket says 172.21.15.22:39466) 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a65105640 1 -- 172.21.15.22:0/3945070428 learned_addr learned my addr 172.21.15.22:0/3945070428 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601accc0 con 0x56147a68e760 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601adfc0 con 0x56147a68f080 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a64904640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a69d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a65906640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a694e40 0x56147a69d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601ad300 con 0x56147a694e40 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a64904640 1 -- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a694e40 msgr2=0x56147a69d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a64904640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a694e40 0x56147a69d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a64904640 1 -- 172.21.15.22:0/3945070428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a68e760 msgr2=0x56147a69ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a64904640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a68e760 0x56147a69ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a64904640 1 -- 172.21.15.22:0/3945070428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56147a69dcc0 con 0x56147a68f080 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.005+0000 7f9a65105640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a68e760 0x56147a69ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a6a026740 10 client.0 _ll_register_callbacks cb 0x7f9a58012070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a50005510 con 0x56147a68f080 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a64904640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a69d240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x56147a6a4ad0 tx=0x7f9a50005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aceee9483e94e5a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a48ff9640 10 client.25535 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a48ff9640 1 -- 172.21.15.22:0/3945070428 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50056060 con 0x56147a68f080 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a48ff9640 1 -- 172.21.15.22:0/3945070428 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9a50004e10 con 0x56147a68f080 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a48ff9640 1 -- 172.21.15.22:0/3945070428 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a5004d070 con 0x56147a68f080 2024-03-08T23:55:30.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a6a026740 10 client.25535 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.006+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x56147a6ff290 con 0x56147a68f080 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a48ff9640 1 -- 172.21.15.22:0/3945070428 <== mon.2 v2:172.21.15.62:3300/0 4 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a500050d0 con 0x56147a68f080 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a6a026740 10 client.25535 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a6a026740 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a6a026740 20 client.25535 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a701520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a6a026740 4 client.25535 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56147a5f9690 con 0x56147a6a8170 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.007+0000 7f9a65105640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a701520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a48ff9640 10 client.25535 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a65105640 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a701520 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4149a729900c78 server_cookie=535f6faeefbe31b4 in_seq=0 out_seq=0 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a48ff9640 1 -- 172.21.15.22:0/3945070428 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56147a5f9690 con 0x56147a6a8170 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a48ff9640 10 client.25535 handle_command_reply: tid=0 2024-03-08T23:55:30.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 client.25535 shutdown 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 msgr2=0x56147a69d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a69d240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x56147a6a4ad0 tx=0x7f9a50005d50 comp rx=0 tx=0).stop 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 shutdown_connections 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56147a6a8170 0x56147a701520 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56147a694e40 0x56147a69d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a48ff9640 0 client.25535 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56147a68f080 0x56147a69d240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 --2- 172.21.15.22:0/3945070428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56147a68e760 0x56147a69ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 >> 172.21.15.22:0/3945070428 conn(0x56147a688e90 msgr2=0x56147a68a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.008+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 shutdown_connections 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.009+0000 7f9a6a026740 1 -- 172.21.15.22:0/3945070428 wait complete. 2024-03-08T23:55:30.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.009+0000 7f9a6a026740 20 client.25535 trim_cache size 0 max 16384 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: "944e0059-efbd-432f-b666-44256dda7698": { 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "944e0059-efbd-432f-b666-44256dda7698", 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:55:30.012 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:55:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.009+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9a4c0a2b10 msgr2=0x7f9a4c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.009+0000 7f9a67369640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9a4c0a2b10 0x7f9a4c0a4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009fd0 tx=0x7f9a58009500 comp rx=0 tx=0).stop 2024-03-08T23:55:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.009+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60068760 msgr2=0x7f9a6019d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.009+0000 7f9a67369640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60068760 0x7f9a6019d7e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00d770 tx=0x7f9a5c00dc30 comp rx=0 tx=0).stop 2024-03-08T23:55:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 shutdown_connections 2024-03-08T23:55:30.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9a4c0a2b10 0x7f9a4c0a4fd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60197990 0x7f9a60197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60106630 0x7f9a6019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 --2- 172.21.15.22:0/2228523108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9a60068760 0x7f9a6019d7e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 >> 172.21.15.22:0/2228523108 conn(0x7f9a600772f0 msgr2=0x7f9a600fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 shutdown_connections 2024-03-08T23:55:30.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:55:30.010+0000 7f9a67369640 1 -- 172.21.15.22:0/2228523108 wait complete. 2024-03-08T23:55:30.021 INFO:tasks.cephfs.filesystem:scrub status for tag:944e0059-efbd-432f-b666-44256dda7698 - {'path': '/', 'tag': '944e0059-efbd-432f-b666-44256dda7698', 'options': 'recursive,force'} 2024-03-08T23:55:30.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:29 smithi022 ceph-mon[31894]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-08T23:55:30.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:29 smithi062 ceph-mon[36129]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-08T23:55:30.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:29 smithi192 ceph-mon[35995]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 -- 172.21.15.22:0/2145916529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c105190 msgr2=0x7fb42c105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 --2- 172.21.15.22:0/2145916529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c105190 0x7fb42c105570 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb428009900 tx=0x7fb42802bc30 comp rx=0 tx=0).stop 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 -- 172.21.15.22:0/2145916529 shutdown_connections 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 --2- 172.21.15.22:0/2145916529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb42c100e00 0x7fb42c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 --2- 172.21.15.22:0/2145916529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42c100440 0x7fb42c1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 --2- 172.21.15.22:0/2145916529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c105190 0x7fb42c105570 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 -- 172.21.15.22:0/2145916529 >> 172.21.15.22:0/2145916529 conn(0x7fb42c077040 msgr2=0x7fb42c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 -- 172.21.15.22:0/2145916529 shutdown_connections 2024-03-08T23:55:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.822+0000 7fb433dcd640 1 -- 172.21.15.22:0/2145916529 wait complete. 2024-03-08T23:55:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 Processor -- start 2024-03-08T23:55:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 -- start start 2024-03-08T23:55:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42c100440 0x7fb42c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c100e00 0x7fb42c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb42c105190 0x7fb42c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c110630 con 0x7fb42c100e00 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1104b0 con 0x7fb42c105190 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb433dcd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1107b0 con 0x7fb42c100440 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb431368640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c100e00 0x7fb42c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb431368640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c100e00 0x7fb42c19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46660/0 (socket says 172.21.15.22:46660) 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.823+0000 7fb431368640 1 -- 172.21.15.22:0/3838587766 learned_addr learned my addr 172.21.15.22:0/3838587766 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb43236a640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb42c105190 0x7fb42c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb431b69640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42c100440 0x7fb42c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb431368640 1 -- 172.21.15.22:0/3838587766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42c100440 msgr2=0x7fb42c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb431368640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42c100440 0x7fb42c19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb431368640 1 -- 172.21.15.22:0/3838587766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb42c105190 msgr2=0x7fb42c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb431368640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb42c105190 0x7fb42c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb431368640 1 -- 172.21.15.22:0/3838587766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c198170 con 0x7fb42c100e00 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb43236a640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb42c105190 0x7fb42c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb431368640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c100e00 0x7fb42c19dd30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00d950 tx=0x7fb41c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb422ffd640 1 -- 172.21.15.22:0/3838587766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c017030 con 0x7fb42c100e00 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c198460 con 0x7fb42c100e00 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb422ffd640 1 -- 172.21.15.22:0/3838587766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb41c004020 con 0x7fb42c100e00 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.824+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c101d90 con 0x7fb42c100e00 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.825+0000 7fb422ffd640 1 -- 172.21.15.22:0/3838587766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0047a0 con 0x7fb42c100e00 2024-03-08T23:55:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.827+0000 7fb4369ab740 1 Processor -- start 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb4369ab740 1 -- start start 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x56252175ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175bfd0 0x56252175c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb422ffd640 1 -- 172.21.15.22:0/3838587766 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fb41c0041d0 con 0x7fb42c100e00 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521770150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb4369ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625216c05f0 con 0x56252175cd40 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb4369ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562521669930 con 0x56252176dd60 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb4369ab740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625216bc690 con 0x56252175bfd0 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb43236a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x56252175ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb43236a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x56252175ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46664/0 (socket says 172.21.15.22:46664) 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb43236a640 1 -- 172.21.15.22:0/676596051 learned_addr learned my addr 172.21.15.22:0/676596051 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb431368640 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521770150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.828+0000 7fb431b69640 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175bfd0 0x56252175c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb422ffd640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb4180a2a20 0x7fb4180a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb43236a640 1 -- 172.21.15.22:0/676596051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175bfd0 msgr2=0x56252175c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb43236a640 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175bfd0 0x56252175c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb43236a640 1 -- 172.21.15.22:0/676596051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 msgr2=0x562521770150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb43236a640 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521770150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb43236a640 1 -- 172.21.15.22:0/676596051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625213d5210 con 0x56252175cd40 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb422ffd640 1 -- 172.21.15.22:0/3838587766 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb41c0cefa0 con 0x7fb42c100e00 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb43236a640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb4180a2a20 0x7fb4180a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.829+0000 7fb43236a640 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x56252175ba90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb4240114b0 tx=0x7fb4240118b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2caafe2a6abd2e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.830+0000 7fb43236a640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb4180a2a20 0x7fb4180a4ee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb424022000 tx=0x7fb42401f000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.830+0000 7fb417fff640 1 -- 172.21.15.22:0/676596051 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42400f6e0 con 0x56252175cd40 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.830+0000 7fb417fff640 1 -- 172.21.15.22:0/676596051 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42400fd60 con 0x56252175cd40 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb417fff640 1 -- 172.21.15.22:0/676596051 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424012d90 con 0x56252175cd40 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 -- 172.21.15.22:0/676596051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 msgr2=0x56252175ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x56252175ba90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb4240114b0 tx=0x7fb4240118b0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 -- 172.21.15.22:0/676596051 shutdown_connections 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521770150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175bfd0 0x56252175c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 --2- 172.21.15.22:0/676596051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x56252175ba90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 -- 172.21.15.22:0/676596051 >> 172.21.15.22:0/676596051 conn(0x5625217235b0 msgr2=0x562521756cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 -- 172.21.15.22:0/676596051 shutdown_connections 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.831+0000 7fb4369ab740 1 -- 172.21.15.22:0/676596051 wait complete. 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 -- start start 2024-03-08T23:55:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 Processor -- start 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x562521767990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x56252176d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521767ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252175d990 con 0x56252175bfd0 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252167cd90 con 0x56252176dd60 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252175db10 con 0x56252175cd40 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb431b69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x562521767990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb4369ab740 10 client.0 _ll_register_callbacks cb 0x5625213d4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb431b69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x562521767990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46668/0 (socket says 172.21.15.22:46668) 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.832+0000 7fb431b69640 1 -- 172.21.15.22:0/1304769799 learned_addr learned my addr 172.21.15.22:0/1304769799 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb43236a640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521767ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431368640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x56252176d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431b69640 1 -- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 msgr2=0x56252176d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431b69640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x56252176d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431b69640 1 -- 172.21.15.22:0/1304769799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 msgr2=0x562521767ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431b69640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521767ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431b69640 1 -- 172.21.15.22:0/1304769799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625214a1620 con 0x56252175bfd0 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb43236a640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521767ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431368640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x56252176d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb431b69640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x562521767990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb428005940 tx=0x7fb428004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef28668ece5fa1e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb4157fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb4157fa640 1 -- 172.21.15.22:0/1304769799 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428042070 con 0x56252175bfd0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb4157fa640 1 -- 172.21.15.22:0/1304769799 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb42802ccc0 con 0x56252175bfd0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb42400fd60 con 0x56252175bfd0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.833+0000 7fb4157fa640 1 -- 172.21.15.22:0/1304769799 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb428005030 con 0x56252175bfd0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.834+0000 7fb4369ab740 10 client.16044 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.834+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562521785510 con 0x56252175bfd0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.834+0000 7fb4157fa640 1 -- 172.21.15.22:0/1304769799 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42400fd60 con 0x56252175bfd0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4157fa640 1 -- 172.21.15.22:0/1304769799 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb4280035f0 con 0x56252175bfd0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4369ab740 10 client.16044 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4369ab740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4369ab740 20 client.16044 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217869d0 0x562521788e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4369ab740 4 client.16044 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625216cdb30 con 0x5625217869d0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb431368640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217869d0 0x562521788e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb4157fa640 10 client.16044 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.835+0000 7fb431368640 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217869d0 0x562521788e20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c9d9e940dc878f server_cookie=4176cfd338b255bd in_seq=0 out_seq=0 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4157fa640 1 -- 172.21.15.22:0/1304769799 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625216cdb30 con 0x5625217869d0 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4157fa640 10 client.16044 handle_command_reply: tid=0 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 client.16044 shutdown 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 msgr2=0x562521767990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x562521767990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb428005940 tx=0x7fb428004540 comp rx=0 tx=0).stop 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 shutdown_connections 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217869d0 0x562521788e20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x562521767ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x56252176d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1304769799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x562521767990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4157fa640 0 client.16044 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 >> 172.21.15.22:0/1304769799 conn(0x5625217235b0 msgr2=0x562521758a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 shutdown_connections 2024-03-08T23:55:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.836+0000 7fb4369ab740 1 -- 172.21.15.22:0/1304769799 wait complete. 2024-03-08T23:55:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.837+0000 7fb4369ab740 20 client.16044 trim_cache size 0 max 16384 2024-03-08T23:55:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 Processor -- start 2024-03-08T23:55:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 -- start start 2024-03-08T23:55:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x56252176d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x562521751d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x56252178d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410012210 con 0x56252175bfd0 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41000fdb0 con 0x56252176dd60 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb4369ab740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41000de50 con 0x56252175cd40 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb431b69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x56252176d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb431b69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x56252176d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46684/0 (socket says 172.21.15.22:46684) 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.850+0000 7fb431b69640 1 -- 172.21.15.22:0/1734502192 learned_addr learned my addr 172.21.15.22:0/1734502192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb43236a640 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x56252178d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb431b69640 1 -- 172.21.15.22:0/1734502192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 msgr2=0x562521751d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb431368640 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x562521751d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb431b69640 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x562521751d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb431b69640 1 -- 172.21.15.22:0/1734502192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 msgr2=0x56252178d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb431b69640 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x56252178d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb431b69640 1 -- 172.21.15.22:0/1734502192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562521757200 con 0x56252175bfd0 2024-03-08T23:55:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb431b69640 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x56252176d510 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb428009900 tx=0x7fb428005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd6d46c82ce455b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb4177fe640 1 -- 172.21.15.22:0/1734502192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428042070 con 0x56252175bfd0 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb4177fe640 1 -- 172.21.15.22:0/1734502192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42802cac0 con 0x56252175bfd0 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.851+0000 7fb4177fe640 1 -- 172.21.15.22:0/1734502192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428004920 con 0x56252175bfd0 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 -- 172.21.15.22:0/1734502192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 msgr2=0x56252176d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x56252176d510 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb428009900 tx=0x7fb428005b50 comp rx=0 tx=0).stop 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 -- 172.21.15.22:0/1734502192 shutdown_connections 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252176dd60 0x56252178d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252175cd40 0x562521751d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 --2- 172.21.15.22:0/1734502192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175bfd0 0x56252176d510 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 -- 172.21.15.22:0/1734502192 >> 172.21.15.22:0/1734502192 conn(0x5625217235b0 msgr2=0x56252175b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 -- 172.21.15.22:0/1734502192 shutdown_connections 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 -- 172.21.15.22:0/1734502192 wait complete. 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 -- start start 2024-03-08T23:55:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.852+0000 7fb4369ab740 1 Processor -- start 2024-03-08T23:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252175bfd0 0x5625217d6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x5625217d6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb4369ab740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252176dd60 0x5625217d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb4369ab740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41000e450 con 0x56252175cd40 2024-03-08T23:55:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb4369ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41000f750 con 0x56252175bfd0 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb4369ab740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41000ea90 con 0x56252176dd60 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb4369ab740 10 client.0 _ll_register_callbacks cb 0x7fb42c06ac40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x5625217d6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x5625217d6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46694/0 (socket says 172.21.15.22:46694) 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 -- 172.21.15.22:0/2888181100 learned_addr learned my addr 172.21.15.22:0/2888181100 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431b69640 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252175bfd0 0x5625217d6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 -- 172.21.15.22:0/2888181100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252176dd60 msgr2=0x5625217d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252176dd60 0x5625217d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 -- 172.21.15.22:0/2888181100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252175bfd0 msgr2=0x5625217d6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252175bfd0 0x5625217d6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431368640 1 -- 172.21.15.22:0/2888181100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625217d16d0 con 0x56252175cd40 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.853+0000 7fb431b69640 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252175bfd0 0x5625217d6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb431368640 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x5625217d6dd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb41c09afa0 tx=0x7fb41c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10464fb7af1e43e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4157fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4157fa640 1 -- 172.21.15.22:0/2888181100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c09b710 con 0x56252175cd40 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4157fa640 1 -- 172.21.15.22:0/2888181100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb41c096d60 con 0x56252175cd40 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb42802cac0 con 0x56252175cd40 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4157fa640 1 -- 172.21.15.22:0/2888181100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb41c09b010 con 0x56252175cd40 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4369ab740 10 client.16050 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625217de0b0 con 0x56252175cd40 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4157fa640 1 -- 172.21.15.22:0/2888181100 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42802cac0 con 0x56252175cd40 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4157fa640 1 -- 172.21.15.22:0/2888181100 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb41c0cc4b0 con 0x56252175cd40 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4369ab740 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.854+0000 7fb4369ab740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.855+0000 7fb4369ab740 20 client.16050 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.855+0000 7fb4369ab740 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217de790 0x5625217e0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.855+0000 7fb4369ab740 4 client.16050 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:55:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.855+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5625216c7e80 con 0x5625217de790 2024-03-08T23:55:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.855+0000 7fb431b69640 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217de790 0x5625217e0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.855+0000 7fb4157fa640 10 client.16050 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.855+0000 7fb431b69640 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217de790 0x5625217e0bb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32ce545344cbd20d server_cookie=b41353ab7169b7fa in_seq=0 out_seq=0 2024-03-08T23:55:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4157fa640 1 -- 172.21.15.22:0/2888181100 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5625216c7e80 con 0x5625217de790 2024-03-08T23:55:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4157fa640 10 client.16050 handle_command_reply: tid=0 2024-03-08T23:55:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 client.16050 shutdown 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 msgr2=0x5625217d6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x5625217d6dd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb41c09afa0 tx=0x7fb41c01c000 comp rx=0 tx=0).stop 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 shutdown_connections 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5625217de790 0x5625217e0bb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56252176dd60 0x5625217d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4157fa640 0 client.16050 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56252175cd40 0x5625217d6dd0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 --2- 172.21.15.22:0/2888181100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56252175bfd0 0x5625217d6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 >> 172.21.15.22:0/2888181100 conn(0x5625217235b0 msgr2=0x5625217614f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 shutdown_connections 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 1 -- 172.21.15.22:0/2888181100 wait complete. 2024-03-08T23:55:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.859+0000 7fb4369ab740 20 client.16050 trim_cache size 0 max 16384 2024-03-08T23:55:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:55:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:55:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:55:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb4180a2a20 msgr2=0x7fb4180a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb4180a2a20 0x7fb4180a4ee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb424022000 tx=0x7fb42401f000 comp rx=0 tx=0).stop 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c100e00 msgr2=0x7fb42c19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c100e00 0x7fb42c19dd30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00d950 tx=0x7fb41c00de10 comp rx=0 tx=0).stop 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 shutdown_connections 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb4180a2a20 0x7fb4180a4ee0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb42c105190 0x7fb42c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb42c100e00 0x7fb42c19dd30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 --2- 172.21.15.22:0/3838587766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42c100440 0x7fb42c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 >> 172.21.15.22:0/3838587766 conn(0x7fb42c077040 msgr2=0x7fb42c068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 shutdown_connections 2024-03-08T23:55:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:30.860+0000 7fb433dcd640 1 -- 172.21.15.22:0/3838587766 wait complete. 2024-03-08T23:55:31.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:30 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:55:31.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:30 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:55:31.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:30 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:55:31.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:30 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:55:31.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:30 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:55:31.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:30 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:55:31.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:30 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:55:31.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:30 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:55:31.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:30 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:55:31.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:30 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:55:31.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:30 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:55:31.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:30 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:55:32.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:31 smithi062 ceph-mon[36129]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:32.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:31 smithi192 ceph-mon[35995]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:32.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:31 smithi022 ceph-mon[31894]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:34.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:33 smithi062 ceph-mon[36129]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:34.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:33 smithi192 ceph-mon[35995]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:34.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:33 smithi022 ceph-mon[31894]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:35.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:34 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:55:35.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:34 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:55:35.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:34 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:55:36.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:35 smithi062 ceph-mon[36129]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:36.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:35 smithi192 ceph-mon[35995]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:36.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:35 smithi022 ceph-mon[31894]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:38.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:38 smithi062 ceph-mon[36129]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:38.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:38 smithi192 ceph-mon[35995]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:38.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:38 smithi022 ceph-mon[31894]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:39 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:55:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:39 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:39 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:39 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:39 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:39 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:55:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:39 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:39 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:39 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:39 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:39 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:55:39.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:39 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:39 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:39 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:39.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:39 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:40.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:40 smithi062 ceph-mon[36129]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:40.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:40 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:55:40.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:40 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:55:40.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:40 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:40 smithi192 ceph-mon[35995]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:40 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:55:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:40 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:55:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:40 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:40.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:40 smithi022 ceph-mon[31894]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:40.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:40 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:55:40.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:40 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:55:40.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:40 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:55:42.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:42 smithi062 ceph-mon[36129]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:42.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:42 smithi192 ceph-mon[35995]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:42.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:42 smithi022 ceph-mon[31894]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.987+0000 7f80c94c0640 1 -- 172.21.15.22:0/2786814545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 msgr2=0x7f80c4068d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.987+0000 7f80c94c0640 1 --2- 172.21.15.22:0/2786814545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c4068d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009900 tx=0x7f80b402bcd0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.987+0000 7f80c94c0640 1 -- 172.21.15.22:0/2786814545 shutdown_connections 2024-03-08T23:55:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.987+0000 7f80c94c0640 1 --2- 172.21.15.22:0/2786814545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c410dff0 0x7f80c41103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.987+0000 7f80c94c0640 1 --2- 172.21.15.22:0/2786814545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80c4069290 0x7f80c410dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.987+0000 7f80c94c0640 1 --2- 172.21.15.22:0/2786814545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c4068d50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.988+0000 7f80c94c0640 1 -- 172.21.15.22:0/2786814545 >> 172.21.15.22:0/2786814545 conn(0x7f80c4077040 msgr2=0x7f80c4077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.988+0000 7f80c94c0640 1 -- 172.21.15.22:0/2786814545 shutdown_connections 2024-03-08T23:55:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.988+0000 7f80c94c0640 1 -- 172.21.15.22:0/2786814545 wait complete. 2024-03-08T23:55:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.988+0000 7f80c94c0640 1 Processor -- start 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c94c0640 1 -- start start 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c94c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c419fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c94c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80c4069290 0x7f80c419ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c94c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c410dff0 0x7f80c4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c94c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41138b0 con 0x7f80c4068970 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c94c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4113730 con 0x7f80c410dff0 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c94c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4113a30 con 0x7f80c4069290 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c419fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c419fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57162/0 (socket says 172.21.15.22:57162) 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 -- 172.21.15.22:0/1549193919 learned_addr learned my addr 172.21.15.22:0/1549193919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c37fe640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c410dff0 0x7f80c4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c27fc640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80c4069290 0x7f80c419ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 -- 172.21.15.22:0/1549193919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80c4069290 msgr2=0x7f80c419ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80c4069290 0x7f80c419ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 -- 172.21.15.22:0/1549193919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c410dff0 msgr2=0x7f80c4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c410dff0 0x7f80c4199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.989+0000 7f80c2ffd640 1 -- 172.21.15.22:0/1549193919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c419a3f0 con 0x7f80c4068970 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.990+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c419fa70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f80b4005940 tx=0x7f80b402cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.990+0000 7f80abfff640 1 -- 172.21.15.22:0/1549193919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4041070 con 0x7f80c4068970 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.990+0000 7f80abfff640 1 -- 172.21.15.22:0/1549193919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b4004450 con 0x7f80c4068970 2024-03-08T23:55:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.990+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c419a680 con 0x7f80c4068970 2024-03-08T23:55:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.990+0000 7f80abfff640 1 -- 172.21.15.22:0/1549193919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4034440 con 0x7f80c4068970 2024-03-08T23:55:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.991+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c41a6810 con 0x7f80c4068970 2024-03-08T23:55:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80abfff640 1 -- 172.21.15.22:0/1549193919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f80b4044070 con 0x7f80c4068970 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 Processor -- start 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 -- start start 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0ddd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0d365f0 con 0x5585f0dd2d40 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0cdf930 con 0x5585f0ddb550 2024-03-08T23:55:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.993+0000 7f80cc179740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0d32690 con 0x5585f0dd1fd0 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57172/0 (socket says 172.21.15.22:57172) 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 -- 172.21.15.22:0/3488586643 learned_addr learned my addr 172.21.15.22:0/3488586643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80abfff640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f80ac0a2b30 0x7f80ac0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 -- 172.21.15.22:0/3488586643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 msgr2=0x5585f0dd2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c27fc640 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0ddd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 -- 172.21.15.22:0/3488586643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 msgr2=0x5585f0ddd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0ddd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 -- 172.21.15.22:0/3488586643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585f0a4b210 con 0x5585f0dd2d40 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80abfff640 1 -- 172.21.15.22:0/1549193919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f80b40f10b0 con 0x7f80c4068970 2024-03-08T23:55:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c37fe640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f80ac0a2b30 0x7f80ac0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.994+0000 7f80c27fc640 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0ddd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.995+0000 7f80c37fe640 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd1a90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00f460 tx=0x7f80bc00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35958a63033eada server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.995+0000 7f80a97fa640 1 -- 172.21.15.22:0/3488586643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc01c400 con 0x5585f0dd2d40 2024-03-08T23:55:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.995+0000 7f80a97fa640 1 -- 172.21.15.22:0/3488586643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80bc01ca80 con 0x5585f0dd2d40 2024-03-08T23:55:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.995+0000 7f80a97fa640 1 -- 172.21.15.22:0/3488586643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc01b820 con 0x5585f0dd2d40 2024-03-08T23:55:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 -- 172.21.15.22:0/3488586643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 msgr2=0x5585f0dd1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd1a90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00f460 tx=0x7f80bc00f920 comp rx=0 tx=0).stop 2024-03-08T23:55:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80c37fe640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f80ac0a2b30 0x7f80ac0a4ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80bc020000 tx=0x7f80bc01d000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 -- 172.21.15.22:0/3488586643 shutdown_connections 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0ddd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 --2- 172.21.15.22:0/3488586643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd1a90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 -- 172.21.15.22:0/3488586643 >> 172.21.15.22:0/3488586643 conn(0x5585f0d995b0 msgr2=0x5585f0dcccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 -- 172.21.15.22:0/3488586643 shutdown_connections 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 -- 172.21.15.22:0/3488586643 wait complete. 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 -- start start 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.996+0000 7f80cc179740 1 Processor -- start 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80cc179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80cc179740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80cc179740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0dd8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57174/0 (socket says 172.21.15.22:57174) 2024-03-08T23:55:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80c27fc640 1 -- 172.21.15.22:0/1837542548 learned_addr learned my addr 172.21.15.22:0/1837542548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0dde280 con 0x5585f0dd2d40 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0cf2d90 con 0x5585f0ddb550 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0dde400 con 0x5585f0dd1fd0 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80c37fe640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0dd8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80cc179740 10 client.0 _ll_register_callbacks cb 0x5585f0a4aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.997+0000 7f80c27fc640 1 -- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 msgr2=0x5585f0dd8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80c27fc640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80c27fc640 1 -- 172.21.15.22:0/1837542548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 msgr2=0x5585f0dd8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80c27fc640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0dd8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80c27fc640 1 -- 172.21.15.22:0/1837542548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585f0b17620 con 0x5585f0dd2d40 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80c37fe640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0dd8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80c27fc640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd8a90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f80b80106a0 tx=0x7f80b8010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ea500194ac38f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80a6ff5640 1 -- 172.21.15.22:0/1837542548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8012920 con 0x5585f0dd2d40 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80a6ff5640 1 -- 172.21.15.22:0/1837542548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f80b8012ac0 con 0x5585f0dd2d40 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80bc01ca80 con 0x5585f0dd2d40 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80a6ff5640 1 -- 172.21.15.22:0/1837542548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80b8012cc0 con 0x5585f0dd2d40 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.998+0000 7f80cc179740 10 client.16062 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5585f0dea500 con 0x5585f0dd2d40 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80a6ff5640 1 -- 172.21.15.22:0/1837542548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80bc01ca80 con 0x5585f0dd2d40 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80a6ff5640 1 -- 172.21.15.22:0/1837542548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f80b800f040 con 0x5585f0dd2d40 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80cc179740 10 client.16062 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80cc179740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80cc179740 20 client.16062 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80cc179740 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0deb9c0 0x5585f0deddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80cc179740 4 client.16062 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:43.999+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585f0d43b30 con 0x5585f0deb9c0 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.000+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0deb9c0 0x5585f0deddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.000+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0deb9c0 0x5585f0deddd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ed813507c7e1c1 server_cookie=cb213f4ebb04d90a in_seq=0 out_seq=0 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.000+0000 7f80a6ff5640 10 client.16062 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80a6ff5640 1 -- 172.21.15.22:0/1837542548 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585f0d43b30 con 0x5585f0deb9c0 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80a6ff5640 10 client.16062 handle_command_reply: tid=0 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 client.16062 shutdown 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 msgr2=0x5585f0dd8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd8a90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f80b80106a0 tx=0x7f80b8010aa0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 shutdown_connections 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0deb9c0 0x5585f0deddd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80a6ff5640 0 client.16062 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0ddb550 0x5585f0dd8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd8a90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 --2- 172.21.15.22:0/1837542548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0dd1fd0 0x5585f0dd8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 >> 172.21.15.22:0/1837542548 conn(0x5585f0d995b0 msgr2=0x5585f0dcea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.001+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 shutdown_connections 2024-03-08T23:55:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.002+0000 7f80cc179740 1 -- 172.21.15.22:0/1837542548 wait complete. 2024-03-08T23:55:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.002+0000 7f80cc179740 20 client.16062 trim_cache size 0 max 16384 2024-03-08T23:55:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 Processor -- start 2024-03-08T23:55:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 -- start start 2024-03-08T23:55:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0ddac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0dd7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41b2bb0 con 0x5585f0dd2d40 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41b0750 con 0x5585f0dd1fd0 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.028+0000 7f80cc179740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41ae7f0 con 0x5585f0ddb550 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0ddac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57178/0 (socket says 172.21.15.22:57178) 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 -- 172.21.15.22:0/1014834394 learned_addr learned my addr 172.21.15.22:0/1014834394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0ddac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:46598/0 (socket says 172.21.15.22:46598) 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c37fe640 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0dd7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 -- 172.21.15.22:0/1014834394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 msgr2=0x5585f0dd7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0dd7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 -- 172.21.15.22:0/1014834394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 msgr2=0x5585f0ddac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0ddac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 -- 172.21.15.22:0/1014834394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585f0df3000 con 0x5585f0dd2d40 2024-03-08T23:55:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0ddac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c37fe640 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0dd7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:55:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80c27fc640 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd6e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b5c0 tx=0x7f80b800b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc0e3c7e4538bbc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.029+0000 7f80a8ff9640 1 -- 172.21.15.22:0/1014834394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8012e60 con 0x5585f0dd2d40 2024-03-08T23:55:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.030+0000 7f80a8ff9640 1 -- 172.21.15.22:0/1014834394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b800b2c0 con 0x5585f0dd2d40 2024-03-08T23:55:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.030+0000 7f80a8ff9640 1 -- 172.21.15.22:0/1014834394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8000e60 con 0x5585f0dd2d40 2024-03-08T23:55:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.030+0000 7f80cc179740 1 -- 172.21.15.22:0/1014834394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 msgr2=0x5585f0dd6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.030+0000 7f80cc179740 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd6e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b5c0 tx=0x7f80b800b5f0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 -- 172.21.15.22:0/1014834394 shutdown_connections 2024-03-08T23:55:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0dd7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0dd2d40 0x5585f0dd6e20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 --2- 172.21.15.22:0/1014834394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0ddac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 -- 172.21.15.22:0/1014834394 >> 172.21.15.22:0/1014834394 conn(0x5585f0d995b0 msgr2=0x5585f0dd38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 -- 172.21.15.22:0/1014834394 shutdown_connections 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 -- 172.21.15.22:0/1014834394 wait complete. 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 -- start start 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 Processor -- start 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0de11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0de1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.031+0000 7f80cc179740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0deb9c0 0x5585f0de1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0deb9c0 0x5585f0de1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0deb9c0 0x5585f0de1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57184/0 (socket says 172.21.15.22:57184) 2024-03-08T23:55:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 -- 172.21.15.22:0/3850142305 learned_addr learned my addr 172.21.15.22:0/3850142305 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41aedf0 con 0x5585f0deb9c0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41b00f0 con 0x5585f0dd1fd0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41af430 con 0x5585f0ddb550 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0de11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c27fc640 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0de1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80cc179740 10 client.0 _ll_register_callbacks cb 0x7f80c406ac40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 -- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 msgr2=0x5585f0de1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0de1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 -- 172.21.15.22:0/3850142305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 msgr2=0x5585f0de11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0de11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 -- 172.21.15.22:0/3850142305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585f0e42120 con 0x5585f0deb9c0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80c37fe640 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0deb9c0 0x5585f0de1c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00f460 tx=0x7f80bc01b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4eddb03e40eb74 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.032+0000 7f80a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80a6ff5640 1 -- 172.21.15.22:0/3850142305 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc009be0 con 0x5585f0deb9c0 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80a6ff5640 1 -- 172.21.15.22:0/3850142305 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f80bc0049e0 con 0x5585f0deb9c0 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80b800b2c0 con 0x5585f0deb9c0 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80a6ff5640 1 -- 172.21.15.22:0/3850142305 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80bc03f050 con 0x5585f0deb9c0 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80cc179740 10 client.16068 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5585f0e43340 con 0x5585f0deb9c0 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80a6ff5640 1 -- 172.21.15.22:0/3850142305 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80bc036070 con 0x5585f0deb9c0 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80a6ff5640 1 -- 172.21.15.22:0/3850142305 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f80bc0057b0 con 0x5585f0deb9c0 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80cc179740 10 client.16068 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80cc179740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80cc179740 20 client.16068 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.033+0000 7f80cc179740 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0e43920 0x5585f0e45d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.034+0000 7f80cc179740 4 client.16068 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.034+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f80b800b2c0 con 0x5585f0e43920 2024-03-08T23:55:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.034+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0e43920 0x5585f0e45d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.034+0000 7f80a6ff5640 10 client.16068 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.034+0000 7f80c2ffd640 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0e43920 0x5585f0e45d40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7013b5d1fbbebfe9 server_cookie=33e7b31b91a190fd in_seq=0 out_seq=0 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.038+0000 7f80a6ff5640 1 -- 172.21.15.22:0/3850142305 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f80b800b2c0 con 0x5585f0e43920 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.038+0000 7f80a6ff5640 10 client.16068 handle_command_reply: tid=0 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.038+0000 7f80cc179740 1 client.16068 shutdown 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0deb9c0 msgr2=0x5585f0de1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0deb9c0 0x5585f0de1c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00f460 tx=0x7f80bc01b640 comp rx=0 tx=0).stop 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 shutdown_connections 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585f0e43920 0x5585f0e45d40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80a6ff5640 0 client.16068 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585f0deb9c0 0x5585f0de1c40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585f0ddb550 0x5585f0de1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 --2- 172.21.15.22:0/3850142305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585f0dd1fd0 0x5585f0de11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 >> 172.21.15.22:0/3850142305 conn(0x5585f0d995b0 msgr2=0x5585f0dcb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 shutdown_connections 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 1 -- 172.21.15.22:0/3850142305 wait complete. 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.039+0000 7f80cc179740 20 client.16068 trim_cache size 0 max 16384 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:55:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:55:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f80ac0a2b30 msgr2=0x7f80ac0a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f80ac0a2b30 0x7f80ac0a4ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80bc020000 tx=0x7f80bc01d000 comp rx=0 tx=0).stop 2024-03-08T23:55:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 msgr2=0x7f80c419fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c419fa70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f80b4005940 tx=0x7f80b402cd10 comp rx=0 tx=0).stop 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 shutdown_connections 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f80ac0a2b30 0x7f80ac0a4ff0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80c410dff0 0x7f80c4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80c4069290 0x7f80c419ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 --2- 172.21.15.22:0/1549193919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80c4068970 0x7f80c419fa70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 >> 172.21.15.22:0/1549193919 conn(0x7f80c4077040 msgr2=0x7f80c410f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 shutdown_connections 2024-03-08T23:55:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:44.040+0000 7f80c94c0640 1 -- 172.21.15.22:0/1549193919 wait complete. 2024-03-08T23:55:45.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:44 smithi062 ceph-mon[36129]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:45.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:44 smithi192 ceph-mon[35995]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:45.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:44 smithi022 ceph-mon[31894]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-08T23:55:47.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:46 smithi062 ceph-mon[36129]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:47.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:46 smithi192 ceph-mon[35995]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:47.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:46 smithi022 ceph-mon[31894]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T23:55:49.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:48 smithi062 ceph-mon[36129]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:49.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:48 smithi192 ceph-mon[35995]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:49.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:48 smithi022 ceph-mon[31894]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-08T23:55:50.763 INFO:tasks.workunit.client.0.smithi022.stderr:Updating files: 94% (11632/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-08T23:55:51.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:50 smithi062 ceph-mon[36129]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-08T23:55:51.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:50 smithi192 ceph-mon[35995]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-08T23:55:51.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:50 smithi022 ceph-mon[31894]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-08T23:55:51.426 INFO:tasks.workunit.client.0.smithi022.stderr:Note: switching to '7d63482aa89628d733e298984a762389b625c761'. 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr:state without impacting any branches by switching back to a branch. 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: git switch -c 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr:Or undo this operation with: 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: git switch - 2024-03-08T23:55:51.427 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-08T23:55:51.428 INFO:tasks.workunit.client.0.smithi022.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-08T23:55:51.428 INFO:tasks.workunit.client.0.smithi022.stderr: 2024-03-08T23:55:51.428 INFO:tasks.workunit.client.0.smithi022.stderr:HEAD is now at 7d63482aa89 Merge remote-tracking branch 'rishabh/fsshell-deprecation' into rishabh-2024mar8 2024-03-08T23:55:51.430 DEBUG:teuthology.orchestra.run.smithi022:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-08T23:55:51.492 INFO:tasks.workunit.client.0.smithi022.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-08T23:55:51.494 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-08T23:55:51.494 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-08T23:55:52.329 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-08T23:55:52.390 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-08T23:55:52.431 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-08T23:55:52.433 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-08T23:55:52.433 INFO:tasks.workunit.client.0.smithi022.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-08T23:55:52.472 INFO:tasks.workunit.client.0.smithi022.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-08T23:55:52.475 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-08T23:55:52.475 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-08T23:55:52.531 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-08T23:55:52.532 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-08T23:55:52.533 DEBUG:teuthology.orchestra.run.smithi022:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-08T23:55:52.591 INFO:tasks.workunit.client.0.smithi022.stderr:+ dbench 1 2024-03-08T23:55:52.597 INFO:tasks.workunit.client.0.smithi022.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-08T23:55:52.597 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-03-08T23:55:52.597 INFO:tasks.workunit.client.0.smithi022.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-08T23:55:52.604 INFO:tasks.workunit.client.0.smithi022.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-08T23:55:52.704 INFO:tasks.workunit.client.0.smithi022.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-08T23:55:52.704 INFO:tasks.workunit.client.0.smithi022.stdout:releasing clients 2024-03-08T23:55:53.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:52 smithi062 ceph-mon[36129]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-08T23:55:53.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:52 smithi192 ceph-mon[35995]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-08T23:55:53.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:52 smithi022 ceph-mon[31894]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-08T23:55:53.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 2982 33.52 MB/sec warmup 1 sec latency 7.414 ms 2024-03-08T23:55:54.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 5527 22.78 MB/sec warmup 2 sec latency 6.452 ms 2024-03-08T23:55:55.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:54 smithi062 ceph-mon[36129]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-08T23:55:55.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:54 smithi192 ceph-mon[35995]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-08T23:55:55.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:54 smithi022 ceph-mon[31894]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-08T23:55:55.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 8014 19.97 MB/sec warmup 3 sec latency 6.906 ms 2024-03-08T23:55:56.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 11017 19.58 MB/sec warmup 4 sec latency 20.038 ms 2024-03-08T23:55:57.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:56 smithi062 ceph-mon[36129]: pgmap v45: 129 pgs: 129 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 173 KiB/s wr, 28 op/s 2024-03-08T23:55:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 -- 172.21.15.22:0/1228262632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558105190 msgr2=0x7f9558105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 --2- 172.21.15.22:0/1228262632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558105190 0x7f9558105570 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9548009900 tx=0x7f954802bc30 comp rx=0 tx=0).stop 2024-03-08T23:55:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 -- 172.21.15.22:0/1228262632 shutdown_connections 2024-03-08T23:55:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 --2- 172.21.15.22:0/1228262632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9558100e00 0x7f955810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 --2- 172.21.15.22:0/1228262632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9558100440 0x7f95581008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 --2- 172.21.15.22:0/1228262632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558105190 0x7f9558105570 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 -- 172.21.15.22:0/1228262632 >> 172.21.15.22:0/1228262632 conn(0x7f9558077040 msgr2=0x7f9558077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 -- 172.21.15.22:0/1228262632 shutdown_connections 2024-03-08T23:55:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 -- 172.21.15.22:0/1228262632 wait complete. 2024-03-08T23:55:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.132+0000 7f955cb7e640 1 Processor -- start 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955cb7e640 1 -- start start 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955cb7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558100440 0x7f95581967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955cb7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9558100e00 0x7f9558196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955cb7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9558105190 0x7f9558197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955cb7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9558110650 con 0x7f9558100440 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955cb7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581104d0 con 0x7f9558105190 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955cb7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581107d0 con 0x7f9558100e00 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558100440 0x7f95581967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558100440 0x7f95581967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53566/0 (socket says 172.21.15.22:53566) 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 -- 172.21.15.22:0/2751503980 learned_addr learned my addr 172.21.15.22:0/2751503980 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f9556d9d640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9558105190 0x7f9558197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f954dd9b640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9558100e00 0x7f9558196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 -- 172.21.15.22:0/2751503980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9558100e00 msgr2=0x7f9558196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9558100e00 0x7f9558196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 -- 172.21.15.22:0/2751503980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9558105190 msgr2=0x7f9558197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9558105190 0x7f9558197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 -- 172.21.15.22:0/2751503980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95581a6010 con 0x7f9558100440 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.133+0000 7f955659c640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558100440 0x7f95581967a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9548005940 tx=0x7f95480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.134+0000 7f954ffff640 1 -- 172.21.15.22:0/2751503980 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548041070 con 0x7f9558100440 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.134+0000 7f954ffff640 1 -- 172.21.15.22:0/2751503980 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f954802ccc0 con 0x7f9558100440 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.134+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95581a62a0 con 0x7f9558100440 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.134+0000 7f954ffff640 1 -- 172.21.15.22:0/2751503980 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548003d80 con 0x7f9558100440 2024-03-08T23:55:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.134+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95581a6790 con 0x7f9558100440 2024-03-08T23:55:57.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:56 smithi192 ceph-mon[35995]: pgmap v45: 129 pgs: 129 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 173 KiB/s wr, 28 op/s 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.136+0000 7f954ffff640 1 -- 172.21.15.22:0/2751503980 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f9548044070 con 0x7f9558100440 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.136+0000 7f955f837740 1 Processor -- start 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955f837740 1 -- start start 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f954ffff640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95440a2b30 0x7f95440a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955f837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b87d40 0x564a59b86a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955f837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b86fd0 0x564a59b87450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955f837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b92940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955f837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a59aeb5f0 con 0x564a59b87d40 2024-03-08T23:55:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955f837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a59a94930 con 0x564a59b90550 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955f837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a59ae7690 con 0x564a59b86fd0 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f9556d9d640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95440a2b30 0x7f95440a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f954ffff640 1 -- 172.21.15.22:0/2751503980 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f95480f0e90 con 0x7f9558100440 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b87d40 0x564a59b86a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b87d40 0x564a59b86a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53580/0 (socket says 172.21.15.22:53580) 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 -- 172.21.15.22:0/4171349823 learned_addr learned my addr 172.21.15.22:0/4171349823 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f954dd9b640 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b86fd0 0x564a59b87450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f9556d9d640 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b92940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 -- 172.21.15.22:0/4171349823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b86fd0 msgr2=0x564a59b87450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b86fd0 0x564a59b87450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 -- 172.21.15.22:0/4171349823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 msgr2=0x564a59b92940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b92940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f955659c640 1 -- 172.21.15.22:0/4171349823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a59800210 con 0x564a59b87d40 2024-03-08T23:55:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.137+0000 7f9556d9d640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95440a2b30 0x7f95440a4ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9550009fd0 tx=0x7f9550011b40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955659c640 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b87d40 0x564a59b86a90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f95480b9a90 tx=0x7f954804d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd13d56f6df72394 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f954cd99640 1 -- 172.21.15.22:0/4171349823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954804b040 con 0x564a59b87d40 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f954cd99640 1 -- 172.21.15.22:0/4171349823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9548004c80 con 0x564a59b87d40 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f954cd99640 1 -- 172.21.15.22:0/4171349823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480c57b0 con 0x564a59b87d40 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955f837740 1 -- 172.21.15.22:0/4171349823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b87d40 msgr2=0x564a59b86a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955f837740 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b87d40 0x564a59b86a90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f95480b9a90 tx=0x7f954804d000 comp rx=0 tx=0).stop 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955f837740 1 -- 172.21.15.22:0/4171349823 shutdown_connections 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955f837740 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b92940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955f837740 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b86fd0 0x564a59b87450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955f837740 1 --2- 172.21.15.22:0/4171349823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b87d40 0x564a59b86a90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.138+0000 7f955f837740 1 -- 172.21.15.22:0/4171349823 >> 172.21.15.22:0/4171349823 conn(0x564a59b4e5b0 msgr2=0x564a59b81cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 -- 172.21.15.22:0/4171349823 shutdown_connections 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 -- 172.21.15.22:0/4171349823 wait complete. 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 -- start start 2024-03-08T23:55:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 Processor -- start 2024-03-08T23:55:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b86fd0 0x564a59b8d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a59b93040 con 0x564a59b86fd0 2024-03-08T23:55:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a59aa7d90 con 0x564a59b90550 2024-03-08T23:55:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a59b931c0 con 0x564a59b87d40 2024-03-08T23:55:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955f837740 10 client.0 _ll_register_callbacks cb 0x564a597ffa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b86fd0 0x564a59b8d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b86fd0 0x564a59b8d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53590/0 (socket says 172.21.15.22:53590) 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f955659c640 1 -- 172.21.15.22:0/3665480936 learned_addr learned my addr 172.21.15.22:0/3665480936 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.139+0000 7f9556d9d640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955659c640 1 -- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 msgr2=0x564a59b8da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955659c640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955659c640 1 -- 172.21.15.22:0/3665480936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 msgr2=0x564a59b8dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955659c640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b8dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955659c640 1 -- 172.21.15.22:0/3665480936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a598cc620 con 0x564a59b86fd0 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f9556d9d640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955659c640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b86fd0 0x564a59b8d540 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f954802e040 tx=0x7f95480ee180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e64a29d684f539 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f953dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f953dffb640 1 -- 172.21.15.22:0/3665480936 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480d5070 con 0x564a59b86fd0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f953dffb640 1 -- 172.21.15.22:0/3665480936 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f954804caa0 con 0x564a59b86fd0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9548004c80 con 0x564a59b86fd0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f953dffb640 1 -- 172.21.15.22:0/3665480936 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f954804b070 con 0x564a59b86fd0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955f837740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564a59b93040 con 0x564a59b86fd0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f953dffb640 1 -- 172.21.15.22:0/3665480936 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9548004c80 con 0x564a59b86fd0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f953dffb640 1 -- 172.21.15.22:0/3665480936 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95480c5320 con 0x564a59b86fd0 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955f837740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955f837740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955f837740 20 client.16080 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.140+0000 7f955f837740 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59ba04a0 0x564a59ba2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.141+0000 7f955f837740 4 client.16080 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.141+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a59af8b30 con 0x564a59ba04a0 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.141+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59ba04a0 0x564a59ba2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.141+0000 7f953dffb640 10 client.16080 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.141+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59ba04a0 0x564a59ba2890 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22beb43a215a6db4 server_cookie=8c55bad2ef64557c in_seq=0 out_seq=0 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f953dffb640 1 -- 172.21.15.22:0/3665480936 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a59af8b30 con 0x564a59ba04a0 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f953dffb640 10 client.16080 handle_command_reply: tid=0 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 client.16080 shutdown 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b86fd0 msgr2=0x564a59b8d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b86fd0 0x564a59b8d540 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f954802e040 tx=0x7f95480ee180 comp rx=0 tx=0).stop 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 shutdown_connections 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59ba04a0 0x564a59ba2890 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b90550 0x564a59b8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f953dffb640 0 client.16080 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 --2- 172.21.15.22:0/3665480936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b86fd0 0x564a59b8d540 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 >> 172.21.15.22:0/3665480936 conn(0x564a59b4e5b0 msgr2=0x564a59b83a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 shutdown_connections 2024-03-08T23:55:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 1 -- 172.21.15.22:0/3665480936 wait complete. 2024-03-08T23:55:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.142+0000 7f955f837740 20 client.16080 trim_cache size 0 max 16384 2024-03-08T23:55:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 Processor -- start 2024-03-08T23:55:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 -- start start 2024-03-08T23:55:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581b2be0 con 0x564a59b90550 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581b0780 con 0x564a59b86fd0 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955f837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581ae820 con 0x564a59b87d40 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f9556d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f9556d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b97830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53598/0 (socket says 172.21.15.22:53598) 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f955659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f9556d9d640 1 -- 172.21.15.22:0/3438528044 learned_addr learned my addr 172.21.15.22:0/3438528044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.155+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f9556d9d640 1 -- 172.21.15.22:0/3438528044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 msgr2=0x564a59b972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f9556d9d640 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f9556d9d640 1 -- 172.21.15.22:0/3438528044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 msgr2=0x564a59b8fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f9556d9d640 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f9556d9d640 1 -- 172.21.15.22:0/3438528044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a59b862a0 con 0x564a59b90550 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f955659c640 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f9556d9d640 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b97830 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f955000fc80 tx=0x7f955001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1925484a58c5607b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f953ffff640 1 -- 172.21.15.22:0/3438528044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955001a040 con 0x564a59b90550 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f953ffff640 1 -- 172.21.15.22:0/3438528044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9550018040 con 0x564a59b90550 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f953ffff640 1 -- 172.21.15.22:0/3438528044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550021c80 con 0x564a59b90550 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f955f837740 1 -- 172.21.15.22:0/3438528044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 msgr2=0x564a59b97830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.156+0000 7f955f837740 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b97830 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f955000fc80 tx=0x7f955001e000 comp rx=0 tx=0).stop 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- 172.21.15.22:0/3438528044 shutdown_connections 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b97830 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 --2- 172.21.15.22:0/3438528044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- 172.21.15.22:0/3438528044 >> 172.21.15.22:0/3438528044 conn(0x564a59b4e5b0 msgr2=0x564a59b817f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- 172.21.15.22:0/3438528044 shutdown_connections 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- 172.21.15.22:0/3438528044 wait complete. 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- start start 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 Processor -- start 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581aee20 con 0x564a59b90550 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581b0120 con 0x564a59b86fd0 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95581af460 con 0x564a59b87d40 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955f837740 10 client.0 _ll_register_callbacks cb 0x7f954000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f9556d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f954dd9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58500/0 (socket says 172.21.15.22:58500) 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f9556d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b8ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53604/0 (socket says 172.21.15.22:53604) 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.157+0000 7f955659c640 1 -- 172.21.15.22:0/3212664591 learned_addr learned my addr 172.21.15.22:0/3212664591 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955659c640 1 -- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 msgr2=0x564a59b8a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955659c640 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955659c640 1 -- 172.21.15.22:0/3212664591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 msgr2=0x564a59b8ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955659c640 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b8ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955659c640 1 -- 172.21.15.22:0/3212664591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a59bf6920 con 0x564a59b86fd0 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f9556d9d640 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955659c640 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8a080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9548000c00 tx=0x7f95480ee1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12253cb3ccd3fc4c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f953dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f953dffb640 1 -- 172.21.15.22:0/3212664591 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480d5070 con 0x564a59b86fd0 2024-03-08T23:55:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f953dffb640 1 -- 172.21.15.22:0/3212664591 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f95480dbb10 con 0x564a59b86fd0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9550018040 con 0x564a59b86fd0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f953dffb640 1 -- 172.21.15.22:0/3212664591 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95480bca20 con 0x564a59b86fd0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955f837740 10 client.25586 fetch_fsmap learned FSMap version 27 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564a59bf6920 con 0x564a59b86fd0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.158+0000 7f953dffb640 1 -- 172.21.15.22:0/3212664591 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9550018040 con 0x564a59b86fd0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f953dffb640 1 -- 172.21.15.22:0/3212664591 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95480b8e10 con 0x564a59b86fd0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f955f837740 10 client.25586 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f955f837740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f955f837740 20 client.25586 populate_metadata read hostname 'smithi022' 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f955f837740 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59bf7d70 0x564a59bfa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f955f837740 4 client.25586 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a59af2e80 con 0x564a59bf7d70 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59bf7d70 0x564a59bfa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f954dd9b640 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59bf7d70 0x564a59bfa160 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a32e31d9d2ba73 server_cookie=f792a9ff1d6670cb in_seq=0 out_seq=0 2024-03-08T23:55:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.159+0000 7f953dffb640 10 client.25586 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:57.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:56 smithi022 ceph-mon[31894]: pgmap v45: 129 pgs: 129 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 173 KiB/s wr, 28 op/s 2024-03-08T23:55:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f953dffb640 1 -- 172.21.15.22:0/3212664591 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a59af2e80 con 0x564a59bf7d70 2024-03-08T23:55:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f953dffb640 10 client.25586 handle_command_reply: tid=0 2024-03-08T23:55:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 client.25586 shutdown 2024-03-08T23:55:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 msgr2=0x564a59b8a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8a080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9548000c00 tx=0x7f95480ee1f0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 shutdown_connections 2024-03-08T23:55:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564a59bf7d70 0x564a59bfa160 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a59b90550 0x564a59b8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a59b87d40 0x564a59b8a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 --2- 172.21.15.22:0/3212664591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a59b86fd0 0x564a59b8a080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f953dffb640 0 client.25586 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.302+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 >> 172.21.15.22:0/3212664591 conn(0x564a59b4e5b0 msgr2=0x564a59b8bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 shutdown_connections 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955f837740 1 -- 172.21.15.22:0/3212664591 wait complete. 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955f837740 20 client.25586 trim_cache size 0 max 16384 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:55:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:55:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:55:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95440a2b30 msgr2=0x7f95440a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95440a2b30 0x7f95440a4ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9550009fd0 tx=0x7f9550011b40 comp rx=0 tx=0).stop 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558100440 msgr2=0x7f95581967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558100440 0x7f95581967a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9548005940 tx=0x7f95480042f0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 shutdown_connections 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95440a2b30 0x7f95440a4ff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9558105190 0x7f9558197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9558100e00 0x7f9558196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 --2- 172.21.15.22:0/2751503980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9558100440 0x7f95581967a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 >> 172.21.15.22:0/2751503980 conn(0x7f9558077040 msgr2=0x7f95580fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:55:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 shutdown_connections 2024-03-08T23:55:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:55:57.303+0000 7f955cb7e640 1 -- 172.21.15.22:0/2751503980 wait complete. 2024-03-08T23:55:57.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 13568 18.17 MB/sec warmup 5 sec latency 89.309 ms 2024-03-08T23:55:58.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:57 smithi022 ceph-mon[31894]: pgmap v46: 129 pgs: 129 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 173 KiB/s wr, 25 op/s 2024-03-08T23:55:58.303 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:57 smithi062 ceph-mon[36129]: pgmap v46: 129 pgs: 129 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 173 KiB/s wr, 25 op/s 2024-03-08T23:55:58.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:57 smithi192 ceph-mon[35995]: pgmap v46: 129 pgs: 129 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 173 KiB/s wr, 25 op/s 2024-03-08T23:55:58.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 16258 17.18 MB/sec warmup 6 sec latency 8.249 ms 2024-03-08T23:55:59.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 18617 16.74 MB/sec warmup 7 sec latency 22.888 ms 2024-03-08T23:56:00.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:56:00.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 -- 172.21.15.22:0/1964518697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1088100d80 msgr2=0x7f1088104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 --2- 172.21.15.22:0/1964518697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1088100d80 0x7f1088104dc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1078009900 tx=0x7f107802bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 -- 172.21.15.22:0/1964518697 shutdown_connections 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 --2- 172.21.15.22:0/1964518697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10881054a0 0x7f108810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 --2- 172.21.15.22:0/1964518697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1088100d80 0x7f1088104dc0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 --2- 172.21.15.22:0/1964518697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 0x7f10881007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 -- 172.21.15.22:0/1964518697 >> 172.21.15.22:0/1964518697 conn(0x7f10880772f0 msgr2=0x7f1088077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 -- 172.21.15.22:0/1964518697 shutdown_connections 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.133+0000 7f108f516640 1 -- 172.21.15.22:0/1964518697 wait complete. 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 Processor -- start 2024-03-08T23:56:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 -- start start 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 0x7f108810d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10881054a0 0x7f108810dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f108810f4a0 0x7f108810e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1088111660 con 0x7f108810f4a0 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881114e0 con 0x7f10881003d0 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108f516640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881117e0 con 0x7f10881054a0 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108dab3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f108810f4a0 0x7f108810e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108d2b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 0x7f108810d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108d2b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 0x7f108810d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58518/0 (socket says 172.21.15.22:58518) 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108d2b2640 1 -- 172.21.15.22:0/3636104954 learned_addr learned my addr 172.21.15.22:0/3636104954 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.134+0000 7f108cab1640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10881054a0 0x7f108810dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108d2b2640 1 -- 172.21.15.22:0/3636104954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10881054a0 msgr2=0x7f108810dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108d2b2640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10881054a0 0x7f108810dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108d2b2640 1 -- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f108810f4a0 msgr2=0x7f108810e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108d2b2640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f108810f4a0 0x7f108810e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108d2b2640 1 -- 172.21.15.22:0/3636104954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10881aa870 con 0x7f10881003d0 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108dab3640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f108810f4a0 0x7f108810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108d2b2640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 0x7f108810d800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f108400e9f0 tx=0x7f108400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f107e7fc640 1 -- 172.21.15.22:0/3636104954 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400cd80 con 0x7f10881003d0 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10881aab60 con 0x7f10881003d0 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f107e7fc640 1 -- 172.21.15.22:0/3636104954 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1084010910 con 0x7f10881003d0 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f107e7fc640 1 -- 172.21.15.22:0/3636104954 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840188f0 con 0x7f10881003d0 2024-03-08T23:56:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.135+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10881ab080 con 0x7f10881003d0 2024-03-08T23:56:00.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.137+0000 7f107e7fc640 1 -- 172.21.15.22:0/3636104954 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f1084010ac0 con 0x7f10881003d0 2024-03-08T23:56:00.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 Processor -- start 2024-03-08T23:56:00.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 -- start start 2024-03-08T23:56:00.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f107e7fc640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10740a2a90 0x7f10740a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa1291b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1292080 0x561aa129a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa129ac60 0x561aa129d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f107e7fc640 1 -- 172.21.15.22:0/3636104954 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f10840cf880 con 0x7f10881003d0 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa11f4e00 con 0x561aa129ac60 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa119db10 con 0x561aa1292080 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f10921d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa11f0ea0 con 0x561aa1291760 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108dab3640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10740a2a90 0x7f10740a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa1291b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa1291b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:42644/0 (socket says 172.21.15.22:42644) 2024-03-08T23:56:00.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 -- 172.21.15.22:0/2476765679 learned_addr learned my addr 172.21.15.22:0/2476765679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108dab3640 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa129ac60 0x561aa129d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108cab1640 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1292080 0x561aa129a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 -- 172.21.15.22:0/2476765679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1292080 msgr2=0x561aa129a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1292080 0x561aa129a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 -- 172.21.15.22:0/2476765679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa129ac60 msgr2=0x561aa129d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa129ac60 0x561aa129d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.138+0000 7f108d2b2640 1 -- 172.21.15.22:0/2476765679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aa0fd37d0 con 0x561aa1291760 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.139+0000 7f108dab3640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10740a2a90 0x7f10740a4f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1080009fd0 tx=0x7f1080012570 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.139+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa1291b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f108400e9c0 tx=0x7f1084008720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29694694d78d1233 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.139+0000 7f10737fe640 1 -- 172.21.15.22:0/2476765679 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840046c0 con 0x561aa1291760 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.139+0000 7f10737fe640 1 -- 172.21.15.22:0/2476765679 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1084005220 con 0x561aa1291760 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.139+0000 7f10737fe640 1 -- 172.21.15.22:0/2476765679 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840a3c00 con 0x561aa1291760 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.139+0000 7f10921d0740 1 -- 172.21.15.22:0/2476765679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 msgr2=0x561aa1291b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.139+0000 7f10921d0740 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa1291b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f108400e9c0 tx=0x7f1084008720 comp rx=0 tx=0).stop 2024-03-08T23:56:00.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- 172.21.15.22:0/2476765679 shutdown_connections 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa129ac60 0x561aa129d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1292080 0x561aa129a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 --2- 172.21.15.22:0/2476765679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa1291b40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- 172.21.15.22:0/2476765679 >> 172.21.15.22:0/2476765679 conn(0x561aa128be90 msgr2=0x561aa128c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- 172.21.15.22:0/2476765679 shutdown_connections 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- 172.21.15.22:0/2476765679 wait complete. 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- start start 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 Processor -- start 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1291760 0x561aa129a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa1297720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa129ac60 0x561aa1297c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa129da50 con 0x561aa1292080 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa11b10a0 con 0x561aa1291760 2024-03-08T23:56:00.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aa129dbd0 con 0x561aa129ac60 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f10921d0740 10 client.0 _ll_register_callbacks cb 0x561aa11b0a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.140+0000 7f108cab1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa1297720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108d2b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1291760 0x561aa129a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa1297720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53654/0 (socket says 172.21.15.22:53654) 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 -- 172.21.15.22:0/2944596511 learned_addr learned my addr 172.21.15.22:0/2944596511 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108dab3640 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa129ac60 0x561aa1297c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 -- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa129ac60 msgr2=0x561aa1297c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa129ac60 0x561aa1297c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 -- 172.21.15.22:0/2944596511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1291760 msgr2=0x561aa129a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1291760 0x561aa129a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 -- 172.21.15.22:0/2944596511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aa0f093f0 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1291760 0x561aa129a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f108cab1640 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa1297720 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1078009230 tx=0x7f107802ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8a5a21a01fe1ff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f1070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f1070ff9640 1 -- 172.21.15.22:0/2944596511 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078004300 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f1070ff9640 1 -- 172.21.15.22:0/2944596511 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f10780046c0 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1084005220 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f1070ff9640 1 -- 172.21.15.22:0/2944596511 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10780044a0 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f10921d0740 10 client.16098 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.141+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561aa0ec90f0 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f1070ff9640 1 -- 172.21.15.22:0/2944596511 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1084005220 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f1070ff9640 1 -- 172.21.15.22:0/2944596511 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f10780035f0 con 0x561aa1292080 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f10921d0740 10 client.16098 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:00.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f10921d0740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f10921d0740 20 client.16098 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f10921d0740 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa12aa720 0x561aa12acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f10921d0740 4 client.16098 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aa1202340 con 0x561aa12aa720 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa12aa720 0x561aa12acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f1070ff9640 10 client.16098 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.142+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa12aa720 0x561aa12acb10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6bf3867ed40146 server_cookie=59d5e5526c2ee418 in_seq=0 out_seq=0 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f1070ff9640 1 -- 172.21.15.22:0/2944596511 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561aa1202340 con 0x561aa12aa720 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f1070ff9640 10 client.16098 handle_command_reply: tid=0 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 client.16098 shutdown 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 msgr2=0x561aa1297720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa1297720 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1078009230 tx=0x7f107802ce30 comp rx=0 tx=0).stop 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 shutdown_connections 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa12aa720 0x561aa12acb10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa129ac60 0x561aa1297c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f1070ff9640 0 client.16098 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:00.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa1297720 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 --2- 172.21.15.22:0/2944596511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa1291760 0x561aa129a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 >> 172.21.15.22:0/2944596511 conn(0x561aa128be90 msgr2=0x561aa128dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 shutdown_connections 2024-03-08T23:56:00.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.143+0000 7f10921d0740 1 -- 172.21.15.22:0/2944596511 wait complete. 2024-03-08T23:56:00.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.144+0000 7f10921d0740 20 client.16098 trim_cache size 0 max 16384 2024-03-08T23:56:00.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 Processor -- start 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 -- start start 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1291760 0x561aa1299e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1292080 0x561aa129a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa1295ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881b70b0 con 0x561aa1291760 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881b4c50 con 0x561aa129ac60 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f10921d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881b2cf0 con 0x561aa1292080 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1291760 0x561aa1299e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1291760 0x561aa1299e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53666/0 (socket says 172.21.15.22:53666) 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 -- 172.21.15.22:0/2923287535 learned_addr learned my addr 172.21.15.22:0/2923287535 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108dab3640 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa1295ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108cab1640 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1292080 0x561aa129a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 -- 172.21.15.22:0/2923287535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1292080 msgr2=0x561aa129a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1292080 0x561aa129a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 -- 172.21.15.22:0/2923287535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 msgr2=0x561aa1295ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa1295ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 -- 172.21.15.22:0/2923287535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aa12b1d00 con 0x561aa1291760 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108cab1640 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1292080 0x561aa129a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108dab3640 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa1295ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f108d2b2640 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1291760 0x561aa1299e50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f108409b6a0 tx=0x7f1084097d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15d677390f3cb9a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f1072ffd640 1 -- 172.21.15.22:0/2923287535 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108409fa40 con 0x561aa1291760 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f1072ffd640 1 -- 172.21.15.22:0/2923287535 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10840a4090 con 0x561aa1291760 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.160+0000 7f1072ffd640 1 -- 172.21.15.22:0/2923287535 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840ccff0 con 0x561aa1291760 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 -- 172.21.15.22:0/2923287535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1291760 msgr2=0x561aa1299e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1291760 0x561aa1299e50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f108409b6a0 tx=0x7f1084097d70 comp rx=0 tx=0).stop 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 -- 172.21.15.22:0/2923287535 shutdown_connections 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa1295ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1292080 0x561aa129a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 --2- 172.21.15.22:0/2923287535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1291760 0x561aa1299e50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 -- 172.21.15.22:0/2923287535 >> 172.21.15.22:0/2923287535 conn(0x561aa128be90 msgr2=0x561aa12976e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 -- 172.21.15.22:0/2923287535 shutdown_connections 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 -- 172.21.15.22:0/2923287535 wait complete. 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 -- start start 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.161+0000 7f10921d0740 1 Processor -- start 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f10921d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa12a2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f10921d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa129e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f10921d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa129e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f10921d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881b32f0 con 0x561aa1292080 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f10921d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881b45f0 con 0x561aa129ac60 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f10921d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881b3930 con 0x561aa1291760 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f10921d0740 10 client.0 _ll_register_callbacks cb 0x7f1078009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa129e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa129e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53670/0 (socket says 172.21.15.22:53670) 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 -- 172.21.15.22:0/3403929581 learned_addr learned my addr 172.21.15.22:0/3403929581 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108d2b2640 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa12a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108dab3640 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa129e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 -- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 msgr2=0x561aa12a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa12a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 -- 172.21.15.22:0/3403929581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 msgr2=0x561aa129e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa129e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 -- 172.21.15.22:0/3403929581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aa129ef50 con 0x561aa1292080 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f108cab1640 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa129e3a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1078000c00 tx=0x7f107802c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29ce7262ba6ab82 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.162+0000 7f1070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f1070ff9640 1 -- 172.21.15.22:0/3403929581 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078004300 con 0x561aa1292080 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f1070ff9640 1 -- 172.21.15.22:0/3403929581 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f1078003bb0 con 0x561aa1292080 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10840a4090 con 0x561aa1292080 2024-03-08T23:56:00.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f1070ff9640 1 -- 172.21.15.22:0/3403929581 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1078063050 con 0x561aa1292080 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 10 client.16104 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561aa1301a90 con 0x561aa1292080 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f1070ff9640 1 -- 172.21.15.22:0/3403929581 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10840a4090 con 0x561aa1292080 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f1070ff9640 1 -- 172.21.15.22:0/3403929581 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f107805e030 con 0x561aa1292080 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 10 client.16104 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 20 client.16104 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa13021b0 0x561aa13045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 4 client.16104 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561aa11fc690 con 0x561aa13021b0 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.163+0000 7f108d2b2640 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa13021b0 0x561aa13045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f1070ff9640 10 client.16104 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f108d2b2640 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa13021b0 0x561aa13045d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c53c1656c39f89 server_cookie=7facffba6ee0269e in_seq=0 out_seq=0 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f1070ff9640 1 -- 172.21.15.22:0/3403929581 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561aa11fc690 con 0x561aa13021b0 2024-03-08T23:56:00.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f1070ff9640 10 client.16104 handle_command_reply: tid=0 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 client.16104 shutdown 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 msgr2=0x561aa129e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa129e3a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1078000c00 tx=0x7f107802c3b0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 shutdown_connections 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561aa13021b0 0x561aa13045d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561aa129ac60 0x561aa129e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f1070ff9640 0 client.16104 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561aa1292080 0x561aa129e3a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 --2- 172.21.15.22:0/3403929581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561aa1291760 0x561aa12a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.164+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 >> 172.21.15.22:0/3403929581 conn(0x561aa128be90 msgr2=0x561aa12925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 shutdown_connections 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f10921d0740 1 -- 172.21.15.22:0/3403929581 wait complete. 2024-03-08T23:56:00.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f10921d0740 20 client.16104 trim_cache size 0 max 16384 2024-03-08T23:56:00.168 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:00.168 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:56:00.168 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:56:00.168 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10740a2a90 msgr2=0x7f10740a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10740a2a90 0x7f10740a4f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1080009fd0 tx=0x7f1080012570 comp rx=0 tx=0).stop 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 msgr2=0x7f108810d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 0x7f108810d800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f108400e9f0 tx=0x7f108400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 shutdown_connections 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10740a2a90 0x7f10740a4f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f108810f4a0 0x7f108810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10881054a0 0x7f108810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 --2- 172.21.15.22:0/3636104954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10881003d0 0x7f108810d800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 >> 172.21.15.22:0/3636104954 conn(0x7f10880772f0 msgr2=0x7f10880fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 shutdown_connections 2024-03-08T23:56:00.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.165+0000 7f108f516640 1 -- 172.21.15.22:0/3636104954 wait complete. 2024-03-08T23:56:00.171 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:55:59 smithi022 ceph-mon[31894]: pgmap v47: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 868 KiB/s wr, 189 op/s 2024-03-08T23:56:00.172 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:56:00.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.284+0000 7ff473b89640 1 -- 172.21.15.22:0/2452098347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101eb0 msgr2=0x7ff46c105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.284+0000 7ff473b89640 1 --2- 172.21.15.22:0/2452098347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101eb0 0x7ff46c105ef0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009900 tx=0x7ff45c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- 172.21.15.22:0/2452098347 shutdown_connections 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 --2- 172.21.15.22:0/2452098347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff46c1065d0 0x7ff46c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 --2- 172.21.15.22:0/2452098347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101eb0 0x7ff46c105ef0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 --2- 172.21.15.22:0/2452098347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff46c101500 0x7ff46c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- 172.21.15.22:0/2452098347 >> 172.21.15.22:0/2452098347 conn(0x7ff46c0fcd80 msgr2=0x7ff46c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- 172.21.15.22:0/2452098347 shutdown_connections 2024-03-08T23:56:00.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- 172.21.15.22:0/2452098347 wait complete. 2024-03-08T23:56:00.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 Processor -- start 2024-03-08T23:56:00.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- start start 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101500 0x7ff46c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff46c1065d0 0x7ff46c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff46c074070 0x7ff46c1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c1116b0 con 0x7ff46c101500 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c111530 con 0x7ff46c074070 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.285+0000 7ff473b89640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c111830 con 0x7ff46c1065d0 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101500 0x7ff46c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101500 0x7ff46c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53680/0 (socket says 172.21.15.22:53680) 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 -- 172.21.15.22:0/2587210044 learned_addr learned my addr 172.21.15.22:0/2587210044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff472126640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff46c074070 0x7ff46c1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471124640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff46c1065d0 0x7ff46c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 -- 172.21.15.22:0/2587210044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff46c1065d0 msgr2=0x7ff46c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff46c1065d0 0x7ff46c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 -- 172.21.15.22:0/2587210044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff46c074070 msgr2=0x7ff46c1a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff46c074070 0x7ff46c1a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 -- 172.21.15.22:0/2587210044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c1a64c0 con 0x7ff46c101500 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff471925640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101500 0x7ff46c0735f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9f0 tx=0x7ff46800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff462ffd640 1 -- 172.21.15.22:0/2587210044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800cd80 con 0x7ff46c101500 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c1a67b0 con 0x7ff46c101500 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff462ffd640 1 -- 172.21.15.22:0/2587210044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff468005500 con 0x7ff46c101500 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff462ffd640 1 -- 172.21.15.22:0/2587210044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468010690 con 0x7ff46c101500 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.286+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c1a6cd0 con 0x7ff46c101500 2024-03-08T23:56:00.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.287+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c0680e0 con 0x7ff46c101500 2024-03-08T23:56:00.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.289+0000 7ff462ffd640 1 -- 172.21.15.22:0/2587210044 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7ff4680046c0 con 0x7ff46c101500 2024-03-08T23:56:00.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.289+0000 7ff462ffd640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4580a2a90 0x7ff4580a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.289+0000 7ff471124640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4580a2a90 0x7ff4580a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.290+0000 7ff462ffd640 1 -- 172.21.15.22:0/2587210044 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff4680cf8c0 con 0x7ff46c101500 2024-03-08T23:56:00.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.290+0000 7ff471124640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4580a2a90 0x7ff4580a4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff45c02c240 tx=0x7ff45c02df90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.294+0000 7ff462ffd640 1 -- 172.21.15.22:0/2587210044 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff46809c030 con 0x7ff46c101500 2024-03-08T23:56:00.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:55:59 smithi062 ceph-mon[36129]: pgmap v47: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 868 KiB/s wr, 189 op/s 2024-03-08T23:56:00.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:55:59 smithi192 ceph-mon[35995]: pgmap v47: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 868 KiB/s wr, 189 op/s 2024-03-08T23:56:00.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.537+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff46c109800 con 0x7ff46c101500 2024-03-08T23:56:00.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.538+0000 7ff462ffd640 1 -- 172.21.15.22:0/2587210044 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff46809b930 con 0x7ff46c101500 2024-03-08T23:56:00.534 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:56:00.534 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:55:16.974110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25439":{"gid":25439,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1375407074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1375407074},{"type":"v1","addr":"172.21.15.62:6837","nonce":1375407074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:56:00.535 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4580a2a90 msgr2=0x7ff4580a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4580a2a90 0x7ff4580a4f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff45c02c240 tx=0x7ff45c02df90 comp rx=0 tx=0).stop 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101500 msgr2=0x7ff46c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101500 0x7ff46c0735f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9f0 tx=0x7ff46800edf0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 shutdown_connections 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4580a2a90 0x7ff4580a4f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff46c074070 0x7ff46c1a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff46c1065d0 0x7ff46c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 --2- 172.21.15.22:0/2587210044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff46c101500 0x7ff46c0735f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.540+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 >> 172.21.15.22:0/2587210044 conn(0x7ff46c0fcd80 msgr2=0x7ff46c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.541+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 shutdown_connections 2024-03-08T23:56:00.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.541+0000 7ff473b89640 1 -- 172.21.15.22:0/2587210044 wait complete. 2024-03-08T23:56:00.548 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:00.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:56:00.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 -- 172.21.15.22:0/1444676862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0100d90 msgr2=0x7fd8c0104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 --2- 172.21.15.22:0/1444676862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0100d90 0x7fd8c0104dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009900 tx=0x7fd8a802bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 -- 172.21.15.22:0/1444676862 shutdown_connections 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 --2- 172.21.15.22:0/1444676862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01054b0 0x7fd8c010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 --2- 172.21.15.22:0/1444676862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0100d90 0x7fd8c0104dd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 --2- 172.21.15.22:0/1444676862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c01003e0 0x7fd8c01007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 -- 172.21.15.22:0/1444676862 >> 172.21.15.22:0/1444676862 conn(0x7fd8c00773a0 msgr2=0x7fd8c00777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 -- 172.21.15.22:0/1444676862 shutdown_connections 2024-03-08T23:56:00.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.661+0000 7fd8c5197640 1 -- 172.21.15.22:0/1444676862 wait complete. 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 Processor -- start 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 -- start start 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01003e0 0x7fd8c01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c01054b0 0x7fd8c0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0199440 0x7fd8c01a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0110520 con 0x7fd8c0199440 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01103a0 con 0x7fd8c01054b0 2024-03-08T23:56:00.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.662+0000 7fd8c5197640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01106a0 con 0x7fd8c01003e0 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0199440 0x7fd8c01a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01003e0 0x7fd8c01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c01054b0 0x7fd8c0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0199440 0x7fd8c01a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47052/0 (socket says 172.21.15.22:47052) 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01003e0 0x7fd8c01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59758/0 (socket says 172.21.15.22:59758) 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8bf59e640 1 -- 172.21.15.22:0/2723028410 learned_addr learned my addr 172.21.15.22:0/2723028410 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8be59c640 1 -- 172.21.15.22:0/2723028410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01003e0 msgr2=0x7fd8c01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8be59c640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01003e0 0x7fd8c01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8be59c640 1 -- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0199440 msgr2=0x7fd8c01a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8be59c640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0199440 0x7fd8c01a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8be59c640 1 -- 172.21.15.22:0/2723028410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c01a41c0 con 0x7fd8c01054b0 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01003e0 0x7fd8c01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8be59c640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c01054b0 0x7fd8c0198f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8036040 tx=0x7fd8a802cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.663+0000 7fd8b7fff640 1 -- 172.21.15.22:0/2723028410 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8041070 con 0x7fd8c01054b0 2024-03-08T23:56:00.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.664+0000 7fd8b7fff640 1 -- 172.21.15.22:0/2723028410 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8004450 con 0x7fd8c01054b0 2024-03-08T23:56:00.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.664+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c01a4450 con 0x7fd8c01054b0 2024-03-08T23:56:00.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.664+0000 7fd8b7fff640 1 -- 172.21.15.22:0/2723028410 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8003e30 con 0x7fd8c01054b0 2024-03-08T23:56:00.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.664+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c01a4940 con 0x7fd8c01054b0 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8b7fff640 1 -- 172.21.15.22:0/2723028410 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fd8a8044070 con 0x7fd8c01054b0 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 Processor -- start 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 -- start start 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6eb42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ec1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6ec1ab0 0x55efc6ec3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6e7b360 con 0x55efc6eb3ed0 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6e17ec0 con 0x55efc6ec1ab0 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6dc0d30 con 0x55efc6eb47f0 2024-03-08T23:56:00.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8b7fff640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd8ac0a2a90 0x7fd8ac0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6eb42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6eb42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47064/0 (socket says 172.21.15.22:47064) 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.666+0000 7fd8bf59e640 1 -- 172.21.15.22:0/1082658434 learned_addr learned my addr 172.21.15.22:0/1082658434 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ec1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8be59c640 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6ec1ab0 0x55efc6ec3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bf59e640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd8ac0a2a90 0x7fd8ac0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bed9d640 1 -- 172.21.15.22:0/1082658434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6ec1ab0 msgr2=0x55efc6ec3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6ec1ab0 0x55efc6ec3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bed9d640 1 -- 172.21.15.22:0/1082658434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 msgr2=0x55efc6eb42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8b7fff640 1 -- 172.21.15.22:0/2723028410 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd8a80f11c0 con 0x7fd8c01054b0 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6eb42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bed9d640 1 -- 172.21.15.22:0/1082658434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc6dbfbc0 con 0x55efc6eb47f0 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bf59e640 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6eb42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ec1570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8005fd0 tx=0x7fd8b8011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48ad0e579093d58 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8bf59e640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd8ac0a2a90 0x7fd8ac0a4f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005d80 tx=0x7fd8b0004730 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8b57fa640 1 -- 172.21.15.22:0/1082658434 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8004d20 con 0x55efc6eb47f0 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8b57fa640 1 -- 172.21.15.22:0/1082658434 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8b8003970 con 0x55efc6eb47f0 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.667+0000 7fd8b57fa640 1 -- 172.21.15.22:0/1082658434 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80043c0 con 0x55efc6eb47f0 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 -- 172.21.15.22:0/1082658434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 msgr2=0x55efc6ec1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ec1570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8005fd0 tx=0x7fd8b8011c70 comp rx=0 tx=0).stop 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 -- 172.21.15.22:0/1082658434 shutdown_connections 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6ec1ab0 0x55efc6ec3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ec1570 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/1082658434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6eb42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 -- 172.21.15.22:0/1082658434 >> 172.21.15.22:0/1082658434 conn(0x55efc6eae8d0 msgr2=0x55efc6eaecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 -- 172.21.15.22:0/1082658434 shutdown_connections 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.668+0000 7fd8c7d79740 1 -- 172.21.15.22:0/1082658434 wait complete. 2024-03-08T23:56:00.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 -- start start 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 Processor -- start 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb3ed0 0x55efc6ebeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ebeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6db64a0 con 0x55efc6ec1ab0 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6e13f60 con 0x55efc6eb47f0 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6ec4870 con 0x55efc6eb3ed0 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8c7d79740 10 client.0 _ll_register_callbacks cb 0x55efc6e11d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb3ed0 0x55efc6ebeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ebeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47076/0 (socket says 172.21.15.22:47076) 2024-03-08T23:56:00.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb3ed0 0x55efc6ebeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59770/0 (socket says 172.21.15.22:59770) 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.669+0000 7fd8bf59e640 1 -- 172.21.15.22:0/3187776309 learned_addr learned my addr 172.21.15.22:0/3187776309 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8bf59e640 1 -- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb3ed0 msgr2=0x55efc6ebeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8bf59e640 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb3ed0 0x55efc6ebeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8bf59e640 1 -- 172.21.15.22:0/3187776309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 msgr2=0x55efc6ebeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8bf59e640 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ebeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8bf59e640 1 -- 172.21.15.22:0/3187776309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efc6b2c3f0 con 0x55efc6ec1ab0 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb3ed0 0x55efc6ebeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8be59c640 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ebeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8bf59e640 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebf530 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000ea40 tx=0x7fd8b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6d1b22b66fe2ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8a67fc640 1 -- 172.21.15.22:0/3187776309 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0024030 con 0x55efc6ec1ab0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8a67fc640 1 -- 172.21.15.22:0/3187776309 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd8b0011b40 con 0x55efc6ec1ab0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8b8003970 con 0x55efc6ec1ab0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8a67fc640 1 -- 172.21.15.22:0/3187776309 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b0030050 con 0x55efc6ec1ab0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8c7d79740 10 client.16122 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55efc6bf67d0 con 0x55efc6ec1ab0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.670+0000 7fd8a67fc640 1 -- 172.21.15.22:0/3187776309 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8b8003970 con 0x55efc6ec1ab0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8a67fc640 1 -- 172.21.15.22:0/3187776309 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd8b001e040 con 0x55efc6ec1ab0 2024-03-08T23:56:00.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8c7d79740 10 client.16122 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8c7d79740 10 client.16122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8c7d79740 20 client.16122 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6ed1c80 0x55efc6ed40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8c7d79740 4 client.16122 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efc6e25400 con 0x55efc6ed1c80 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6ed1c80 0x55efc6ed40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6ed1c80 0x55efc6ed40b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc63a7ac739e8bcb server_cookie=d1551bda8fde43df in_seq=0 out_seq=0 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.671+0000 7fd8a67fc640 10 client.16122 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8a67fc640 1 -- 172.21.15.22:0/3187776309 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55efc6e25400 con 0x55efc6ed1c80 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8a67fc640 10 client.16122 handle_command_reply: tid=0 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 client.16122 shutdown 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 msgr2=0x55efc6ebf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebf530 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000ea40 tx=0x7fd8b00046c0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 shutdown_connections 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6ed1c80 0x55efc6ed40b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebf530 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ebeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3187776309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb3ed0 0x55efc6ebeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8a67fc640 0 client.16122 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 >> 172.21.15.22:0/3187776309 conn(0x55efc6eae8d0 msgr2=0x55efc6eb0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 shutdown_connections 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3187776309 wait complete. 2024-03-08T23:56:00.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.672+0000 7fd8c7d79740 20 client.16122 trim_cache size 0 max 16384 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 Processor -- start 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 -- start start 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ebbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01ae840 con 0x55efc6ec1ab0 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01ac3e0 con 0x55efc6eb3ed0 2024-03-08T23:56:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01aa480 con 0x55efc6eb47f0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.684+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8be59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ebbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8bf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47082/0 (socket says 172.21.15.22:47082) 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8bf59e640 1 -- 172.21.15.22:0/3776138572 learned_addr learned my addr 172.21.15.22:0/3776138572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8be59c640 1 -- 172.21.15.22:0/3776138572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb3ed0 msgr2=0x55efc6ec11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8be59c640 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8be59c640 1 -- 172.21.15.22:0/3776138572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 msgr2=0x55efc6ebc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8be59c640 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8be59c640 1 -- 172.21.15.22:0/3776138572 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc6ec3ea0 con 0x55efc6eb47f0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8be59c640 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ebbd10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80ee500 tx=0x7fd8a804e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b919c15faf8cb7a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8b4ff9640 1 -- 172.21.15.22:0/3776138572 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a804b040 con 0x55efc6eb47f0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8b4ff9640 1 -- 172.21.15.22:0/3776138572 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a80d61c0 con 0x55efc6eb47f0 2024-03-08T23:56:00.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.685+0000 7fd8b4ff9640 1 -- 172.21.15.22:0/3776138572 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80dd0b0 con 0x55efc6eb47f0 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3776138572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 msgr2=0x55efc6ebbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ebbd10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80ee500 tx=0x7fd8a804e000 comp rx=0 tx=0).stop 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3776138572 shutdown_connections 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6ec1ab0 0x55efc6ebc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6eb47f0 0x55efc6ebbd10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/3776138572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3776138572 >> 172.21.15.22:0/3776138572 conn(0x55efc6eae8d0 msgr2=0x55efc6eb9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3776138572 shutdown_connections 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 -- 172.21.15.22:0/3776138572 wait complete. 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.686+0000 7fd8c7d79740 1 -- start start 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 1 Processor -- start 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ec70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6ec1ab0 0x55efc6f27190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01aaa80 con 0x55efc6eb3ed0 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01abd80 con 0x55efc6eb47f0 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01ab0c0 con 0x55efc6ec1ab0 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8c7d79740 10 client.0 _ll_register_callbacks cb 0x7fd8b8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8be59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ec70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47094/0 (socket says 172.21.15.22:47094) 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 -- 172.21.15.22:0/605351622 learned_addr learned my addr 172.21.15.22:0/605351622 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bf59e640 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6ec1ab0 0x55efc6f27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 -- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6ec1ab0 msgr2=0x55efc6f27190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6ec1ab0 0x55efc6f27190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 -- 172.21.15.22:0/605351622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 msgr2=0x55efc6ec70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ec70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 -- 172.21.15.22:0/605351622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efc6f276d0 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8be59c640 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ec70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8bed9d640 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec6b80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800a010 tx=0x7fd8b8004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f420181afe3824 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.687+0000 7fd8a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8a67fc640 1 -- 172.21.15.22:0/605351622 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b801d040 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8a67fc640 1 -- 172.21.15.22:0/605351622 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd8b800f9a0 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8a80d61c0 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8a67fc640 1 -- 172.21.15.22:0/605351622 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b803c050 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 10 client.16128 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55efc6f288d0 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8a67fc640 1 -- 172.21.15.22:0/605351622 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a80d61c0 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8a67fc640 1 -- 172.21.15.22:0/605351622 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd8b8023050 con 0x55efc6eb3ed0 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 10 client.16128 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:00.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 10 client.16128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 20 client.16128 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6f28db0 0x55efc6f2b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 4 client.16128 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.688+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55efc6e1f750 con 0x55efc6f28db0 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8be59c640 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6f28db0 0x55efc6f2b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8a67fc640 10 client.16128 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8be59c640 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6f28db0 0x55efc6f2b1a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d58d9c973227718 server_cookie=4772f17cf8ce1505 in_seq=0 out_seq=0 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8a67fc640 1 -- 172.21.15.22:0/605351622 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55efc6e1f750 con 0x55efc6f28db0 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8a67fc640 10 client.16128 handle_command_reply: tid=0 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 client.16128 shutdown 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 msgr2=0x55efc6ec6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec6b80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800a010 tx=0x7fd8b8004f60 comp rx=0 tx=0).stop 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 shutdown_connections 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efc6f28db0 0x55efc6f2b1a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efc6ec1ab0 0x55efc6f27190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc6eb47f0 0x55efc6ec70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 --2- 172.21.15.22:0/605351622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efc6eb3ed0 0x55efc6ec6b80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8a67fc640 0 client.16128 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 >> 172.21.15.22:0/605351622 conn(0x55efc6eae8d0 msgr2=0x55efc6ebd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 shutdown_connections 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.689+0000 7fd8c7d79740 1 -- 172.21.15.22:0/605351622 wait complete. 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c7d79740 20 client.16128 trim_cache size 0 max 16384 2024-03-08T23:56:00.692 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd8ac0a2a90 msgr2=0x7fd8ac0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd8ac0a2a90 0x7fd8ac0a4f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005d80 tx=0x7fd8b0004730 comp rx=0 tx=0).stop 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c01054b0 msgr2=0x7fd8c0198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c01054b0 0x7fd8c0198f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8036040 tx=0x7fd8a802cbf0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 shutdown_connections 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd8ac0a2a90 0x7fd8ac0a4f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c0199440 0x7fd8c01a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c01054b0 0x7fd8c0198f00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 --2- 172.21.15.22:0/2723028410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8c01003e0 0x7fd8c01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 >> 172.21.15.22:0/2723028410 conn(0x7fd8c00773a0 msgr2=0x7fd8c00fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 shutdown_connections 2024-03-08T23:56:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.690+0000 7fd8c5197640 1 -- 172.21.15.22:0/2723028410 wait complete. 2024-03-08T23:56:00.698 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:00.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:56:00.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 21507 16.85 MB/sec warmup 8 sec latency 48.694 ms 2024-03-08T23:56:00.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 -- 172.21.15.22:0/3717878581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd44101e80 msgr2=0x7efd44105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 --2- 172.21.15.22:0/3717878581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd44101e80 0x7efd44105ec0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009900 tx=0x7efd2c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:00.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 -- 172.21.15.22:0/3717878581 shutdown_connections 2024-03-08T23:56:00.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 --2- 172.21.15.22:0/3717878581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd441065a0 0x7efd4410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 --2- 172.21.15.22:0/3717878581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd44101e80 0x7efd44105ec0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 --2- 172.21.15.22:0/3717878581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd441014d0 0x7efd441018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 -- 172.21.15.22:0/3717878581 >> 172.21.15.22:0/3717878581 conn(0x7efd440fcd90 msgr2=0x7efd440ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 -- 172.21.15.22:0/3717878581 shutdown_connections 2024-03-08T23:56:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.809+0000 7efd488c0640 1 -- 172.21.15.22:0/3717878581 wait complete. 2024-03-08T23:56:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 Processor -- start 2024-03-08T23:56:00.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 -- start start 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd441014d0 0x7efd4419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd441065a0 0x7efd4419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd44197a60 0x7efd44197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44111680 con 0x7efd441014d0 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44111500 con 0x7efd441065a0 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd488c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44111800 con 0x7efd44197a60 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd441014d0 0x7efd4419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd441014d0 0x7efd4419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47112/0 (socket says 172.21.15.22:47112) 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 -- 172.21.15.22:0/1073835089 learned_addr learned my addr 172.21.15.22:0/1073835089 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd41d9b640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd441065a0 0x7efd4419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd42d9d640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd44197a60 0x7efd44197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 -- 172.21.15.22:0/1073835089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd44197a60 msgr2=0x7efd44197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd44197a60 0x7efd44197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 -- 172.21.15.22:0/1073835089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd441065a0 msgr2=0x7efd4419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd441065a0 0x7efd4419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 -- 172.21.15.22:0/1073835089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44198580 con 0x7efd441014d0 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.810+0000 7efd4259c640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd441014d0 0x7efd4419d860 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e9f0 tx=0x7efd3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.811+0000 7efd3b7fe640 1 -- 172.21.15.22:0/1073835089 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c00cd80 con 0x7efd441014d0 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.811+0000 7efd3b7fe640 1 -- 172.21.15.22:0/1073835089 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd3c005500 con 0x7efd441014d0 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.811+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd44198870 con 0x7efd441014d0 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.811+0000 7efd3b7fe640 1 -- 172.21.15.22:0/1073835089 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c010690 con 0x7efd441014d0 2024-03-08T23:56:00.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.811+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd441a4a70 con 0x7efd441014d0 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.813+0000 7efd3b7fe640 1 -- 172.21.15.22:0/1073835089 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7efd3c0046c0 con 0x7efd441014d0 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.813+0000 7efd4b57c740 1 Processor -- start 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4b57c740 1 -- start start 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd3b7fe640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efd300a2b10 0x7efd300a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec095b0 0x556c9ec09990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec12570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec12ab0 0x556c9ec14ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4b57c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9ebcc360 con 0x556c9ec09ed0 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4b57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9eb68ec0 con 0x556c9ec12ab0 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4b57c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9eb11d30 con 0x556c9ec095b0 2024-03-08T23:56:00.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd42d9d640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efd300a2b10 0x7efd300a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec12570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd3b7fe640 1 -- 172.21.15.22:0/1073835089 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7efd3c0cf850 con 0x7efd441014d0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec095b0 0x556c9ec09990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec12570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47124/0 (socket says 172.21.15.22:47124) 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 -- 172.21.15.22:0/82382474 learned_addr learned my addr 172.21.15.22:0/82382474 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd4259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec095b0 0x556c9ec09990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59824/0 (socket says 172.21.15.22:59824) 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd42d9d640 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec12ab0 0x556c9ec14ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 -- 172.21.15.22:0/82382474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec095b0 msgr2=0x556c9ec09990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec095b0 0x556c9ec09990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 -- 172.21.15.22:0/82382474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec12ab0 msgr2=0x556c9ec14ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec12ab0 0x556c9ec14ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 -- 172.21.15.22:0/82382474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c9eb10bc0 con 0x556c9ec09ed0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd42d9d640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efd300a2b10 0x7efd300a4fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd34005fd0 tx=0x7efd34009b00 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.814+0000 7efd41d9b640 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec12570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4c82e1ab1b3ee server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd38ff9640 1 -- 172.21.15.22:0/82382474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c045070 con 0x556c9ec09ed0 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd38ff9640 1 -- 172.21.15.22:0/82382474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2c02c650 con 0x556c9ec09ed0 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd38ff9640 1 -- 172.21.15.22:0/82382474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c004ab0 con 0x556c9ec09ed0 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd4b57c740 1 -- 172.21.15.22:0/82382474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 msgr2=0x556c9ec12570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd4b57c740 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec12570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c009560 comp rx=0 tx=0).stop 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd4b57c740 1 -- 172.21.15.22:0/82382474 shutdown_connections 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd4b57c740 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec12ab0 0x556c9ec14ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd4b57c740 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec12570 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd4b57c740 1 --2- 172.21.15.22:0/82382474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec095b0 0x556c9ec09990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.815+0000 7efd4b57c740 1 -- 172.21.15.22:0/82382474 >> 172.21.15.22:0/82382474 conn(0x556c9ec03ce0 msgr2=0x556c9ec040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 -- 172.21.15.22:0/82382474 shutdown_connections 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 -- 172.21.15.22:0/82382474 wait complete. 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 -- start start 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 Processor -- start 2024-03-08T23:56:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec0fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9eb074a0 con 0x556c9ec09ed0 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9eb64f60 con 0x556c9ec095b0 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c9ec15840 con 0x556c9ec12ab0 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4b57c740 10 client.0 _ll_register_callbacks cb 0x556c9eb62d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd41d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec0fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec0fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35836/0 (socket says 172.21.15.22:35836) 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4259c640 1 -- 172.21.15.22:0/593482917 learned_addr learned my addr 172.21.15.22:0/593482917 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd42d9d640 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4259c640 1 -- 172.21.15.22:0/593482917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 msgr2=0x556c9ec104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4259c640 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4259c640 1 -- 172.21.15.22:0/593482917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 msgr2=0x556c9ec0ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.816+0000 7efd4259c640 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4259c640 1 -- 172.21.15.22:0/593482917 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c9e87d3f0 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd41d9b640 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4259c640 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec0fa70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c6a0 tx=0x7efd3c065f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=460bf442dd73f12b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd29ffb640 1 -- 172.21.15.22:0/593482917 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0b3020 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd29ffb640 1 -- 172.21.15.22:0/593482917 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7efd3c065030 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd2c02c650 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd29ffb640 1 -- 172.21.15.22:0/593482917 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd3c065230 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4b57c740 10 client.25640 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556c9e9477d0 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd29ffb640 1 -- 172.21.15.22:0/593482917 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2c02c650 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd29ffb640 1 -- 172.21.15.22:0/593482917 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efd3c0cf460 con 0x556c9ec095b0 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4b57c740 10 client.25640 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4b57c740 10 client.25640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4b57c740 20 client.25640 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.817+0000 7efd4b57c740 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec22b60 0x556c9ec24f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.818+0000 7efd4b57c740 4 client.25640 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.818+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c9eb76400 con 0x556c9ec22b60 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.818+0000 7efd41d9b640 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec22b60 0x556c9ec24f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.818+0000 7efd41d9b640 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec22b60 0x556c9ec24f90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99231bdf3b6bd5bb server_cookie=d160d0f090eade47 in_seq=0 out_seq=0 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.818+0000 7efd29ffb640 10 client.25640 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd29ffb640 1 -- 172.21.15.22:0/593482917 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c9eb76400 con 0x556c9ec22b60 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd29ffb640 10 client.25640 handle_command_reply: tid=0 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 client.25640 shutdown 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 msgr2=0x556c9ec0fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec0fa70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c6a0 tx=0x7efd3c065f20 comp rx=0 tx=0).stop 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 shutdown_connections 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec22b60 0x556c9ec24f90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd29ffb640 0 client.25640 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 --2- 172.21.15.22:0/593482917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec0fa70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 >> 172.21.15.22:0/593482917 conn(0x556c9ec03ce0 msgr2=0x556c9ec05a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 shutdown_connections 2024-03-08T23:56:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 1 -- 172.21.15.22:0/593482917 wait complete. 2024-03-08T23:56:00.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.819+0000 7efd4b57c740 20 client.25640 trim_cache size 0 max 16384 2024-03-08T23:56:00.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 Processor -- start 2024-03-08T23:56:00.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 -- start start 2024-03-08T23:56:00.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec0d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441ae710 con 0x556c9ec09ed0 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441ac2b0 con 0x556c9ec095b0 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4b57c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441aa350 con 0x556c9ec12ab0 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47154/0 (socket says 172.21.15.22:47154) 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 -- 172.21.15.22:0/345701796 learned_addr learned my addr 172.21.15.22:0/345701796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd4259c640 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd42d9d640 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec0d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 -- 172.21.15.22:0/345701796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 msgr2=0x556c9ec0d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec0d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 -- 172.21.15.22:0/345701796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 msgr2=0x556c9ec121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.831+0000 7efd41d9b640 1 -- 172.21.15.22:0/345701796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c9ec14ea0 con 0x556c9ec09ed0 2024-03-08T23:56:00.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd41d9b640 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ccd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efd2c02c210 tx=0x7efd2c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6606f8d30f5996e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd2bfff640 1 -- 172.21.15.22:0/345701796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c055070 con 0x556c9ec09ed0 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd2bfff640 1 -- 172.21.15.22:0/345701796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2c004ee0 con 0x556c9ec09ed0 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd2bfff640 1 -- 172.21.15.22:0/345701796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c045040 con 0x556c9ec09ed0 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 -- 172.21.15.22:0/345701796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 msgr2=0x556c9ec0ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ccd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efd2c02c210 tx=0x7efd2c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 -- 172.21.15.22:0/345701796 shutdown_connections 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec12ab0 0x556c9ec0d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec09ed0 0x556c9ec0ccd0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 --2- 172.21.15.22:0/345701796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 -- 172.21.15.22:0/345701796 >> 172.21.15.22:0/345701796 conn(0x556c9ec03ce0 msgr2=0x556c9ec040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 -- 172.21.15.22:0/345701796 shutdown_connections 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.832+0000 7efd4b57c740 1 -- 172.21.15.22:0/345701796 wait complete. 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 -- start start 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 Processor -- start 2024-03-08T23:56:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec19c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec09ed0 0x556c9ec17fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec12ab0 0x556c9ec184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441aa950 con 0x556c9ec12ab0 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441abc50 con 0x556c9ec095b0 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441aaf90 con 0x556c9ec09ed0 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4b57c740 10 client.0 _ll_register_callbacks cb 0x7efd3c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec12ab0 0x556c9ec184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec12ab0 0x556c9ec184f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47160/0 (socket says 172.21.15.22:47160) 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 -- 172.21.15.22:0/2223098442 learned_addr learned my addr 172.21.15.22:0/2223098442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd41d9b640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec09ed0 0x556c9ec17fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd4259c640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec19c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 -- 172.21.15.22:0/2223098442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec09ed0 msgr2=0x556c9ec17fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec09ed0 0x556c9ec17fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 -- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 msgr2=0x556c9ec19c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec19c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 -- 172.21.15.22:0/2223098442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c9ec18b00 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd41d9b640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec09ed0 0x556c9ec17fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd42d9d640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec12ab0 0x556c9ec184f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efd3400fe60 tx=0x7efd34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6590ef51865d927c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.833+0000 7efd29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd29ffb640 1 -- 172.21.15.22:0/2223098442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340046c0 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd29ffb640 1 -- 172.21.15.22:0/2223098442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7efd34004b20 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd2c004ee0 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd29ffb640 1 -- 172.21.15.22:0/2223098442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd34035050 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 10 client.16146 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556c9ec79ed0 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd29ffb640 1 -- 172.21.15.22:0/2223098442 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2c004ee0 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd29ffb640 1 -- 172.21.15.22:0/2223098442 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efd34029070 con 0x556c9ec12ab0 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 10 client.16146 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 10 client.16146 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:56:00.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 20 client.16146 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec7a680 0x556c9ec7caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 4 client.16146 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c9eb70750 con 0x556c9ec7a680 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.834+0000 7efd4259c640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec7a680 0x556c9ec7caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4259c640 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec7a680 0x556c9ec7caa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6561a5bac60a07a5 server_cookie=d529a1f3ebec17ac in_seq=0 out_seq=0 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd29ffb640 10 client.16146 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd29ffb640 1 -- 172.21.15.22:0/2223098442 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c9eb70750 con 0x556c9ec7a680 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd29ffb640 10 client.16146 handle_command_reply: tid=0 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 client.16146 shutdown 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec12ab0 msgr2=0x556c9ec184f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec12ab0 0x556c9ec184f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efd3400fe60 tx=0x7efd34004ab0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 shutdown_connections 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556c9ec7a680 0x556c9ec7caa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c9ec12ab0 0x556c9ec184f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd29ffb640 0 client.16146 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c9ec09ed0 0x556c9ec17fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 --2- 172.21.15.22:0/2223098442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c9ec095b0 0x556c9ec19c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 >> 172.21.15.22:0/2223098442 conn(0x556c9ec03ce0 msgr2=0x556c9ebfe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 shutdown_connections 2024-03-08T23:56:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 1 -- 172.21.15.22:0/2223098442 wait complete. 2024-03-08T23:56:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.835+0000 7efd4b57c740 20 client.16146 trim_cache size 0 max 16384 2024-03-08T23:56:00.838 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efd300a2b10 msgr2=0x7efd300a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efd300a2b10 0x7efd300a4fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd34005fd0 tx=0x7efd34009b00 comp rx=0 tx=0).stop 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd441014d0 msgr2=0x7efd4419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd441014d0 0x7efd4419d860 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e9f0 tx=0x7efd3c00edf0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 shutdown_connections 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efd300a2b10 0x7efd300a4fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd44197a60 0x7efd44197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd441065a0 0x7efd4419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 --2- 172.21.15.22:0/1073835089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd441014d0 0x7efd4419d860 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 >> 172.21.15.22:0/1073835089 conn(0x7efd440fcd90 msgr2=0x7efd440ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 shutdown_connections 2024-03-08T23:56:00.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.836+0000 7efd488c0640 1 -- 172.21.15.22:0/1073835089 wait complete. 2024-03-08T23:56:00.843 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:00.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:56:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 -- 172.21.15.22:0/1356160345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 msgr2=0x7f3e0810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/1356160345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0810e1f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009ca0 tx=0x7f3e0002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 -- 172.21.15.22:0/1356160345 shutdown_connections 2024-03-08T23:56:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/1356160345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0810e1f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/1356160345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e0806a600 0x7f3e08105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/1356160345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e08069c50 0x7f3e0806a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 -- 172.21.15.22:0/1356160345 >> 172.21.15.22:0/1356160345 conn(0x7f3e08076570 msgr2=0x7f3e08076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 -- 172.21.15.22:0/1356160345 shutdown_connections 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 -- 172.21.15.22:0/1356160345 wait complete. 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.994+0000 7f3e0d64b640 1 Processor -- start 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e0d64b640 1 -- start start 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e0d64b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08069c50 0x7f3e0819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e0d64b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e0d64b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e08195740 0x7f3e08195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e0d64b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0810f480 con 0x7f3e08106640 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e0d64b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0810f300 con 0x7f3e08069c50 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e0d64b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0810f600 con 0x7f3e08195740 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47180/0 (socket says 172.21.15.22:47180) 2024-03-08T23:56:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 -- 172.21.15.22:0/258114382 learned_addr learned my addr 172.21.15.22:0/258114382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08069c50 0x7f3e0819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e077fe640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e08195740 0x7f3e08195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 -- 172.21.15.22:0/258114382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e08195740 msgr2=0x7f3e08195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e08195740 0x7f3e08195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 -- 172.21.15.22:0/258114382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08069c50 msgr2=0x7f3e0819b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08069c50 0x7f3e0819b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 -- 172.21.15.22:0/258114382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08196340 con 0x7f3e08106640 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3e067fc640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0819bb70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e9f0 tx=0x7f3df000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.995+0000 7f3dfffff640 1 -- 172.21.15.22:0/258114382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df000ccc0 con 0x7f3e08106640 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.996+0000 7f3dfffff640 1 -- 172.21.15.22:0/258114382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3df0010910 con 0x7f3e08106640 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.996+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e081a23d0 con 0x7f3e08106640 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.996+0000 7f3dfffff640 1 -- 172.21.15.22:0/258114382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0018920 con 0x7f3e08106640 2024-03-08T23:56:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.996+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e081a27d0 con 0x7f3e08106640 2024-03-08T23:56:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.998+0000 7f3dfffff640 1 -- 172.21.15.22:0/258114382 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f3df0010ac0 con 0x7f3e08106640 2024-03-08T23:56:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.998+0000 7f3e1022b740 1 Processor -- start 2024-03-08T23:56:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e1022b740 1 -- start start 2024-03-08T23:56:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc088731a0 0x55bc08873580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc0887c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0887c6a0 0x55bc0887ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e1022b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0883a360 con 0x55bc0887c6a0 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e1022b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc087d6ec0 con 0x55bc088731a0 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e1022b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0877fd30 con 0x55bc08873ac0 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0887c6a0 0x55bc0887ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc0887c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0887c6a0 0x55bc0887ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47186/0 (socket says 172.21.15.22:47186) 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc0887c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59882/0 (socket says 172.21.15.22:59882) 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e067fc640 1 -- 172.21.15.22:0/3312794039 learned_addr learned my addr 172.21.15.22:0/3312794039 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e077fe640 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc088731a0 0x55bc08873580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3dfffff640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3df40a2a90 0x7f3df40a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e077fe640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3df40a2a90 0x7f3df40a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e06ffd640 1 -- 172.21.15.22:0/3312794039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc088731a0 msgr2=0x55bc08873580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc088731a0 0x55bc08873580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e06ffd640 1 -- 172.21.15.22:0/3312794039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0887c6a0 msgr2=0x55bc0887ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3dfffff640 1 -- 172.21.15.22:0/258114382 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3df00cf770 con 0x7f3e08106640 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0887c6a0 0x55bc0887ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e06ffd640 1 -- 172.21.15.22:0/3312794039 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0877ebc0 con 0x55bc08873ac0 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:00.999+0000 7f3e067fc640 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0887c6a0 0x55bc0887ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.000+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc0887c160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3df800f930 tx=0x7f3df8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db6c0b77b73f873 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.000+0000 7f3e077fe640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3df40a2a90 0x7f3df40a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e00004af0 tx=0x7f3e0002c0f0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.000+0000 7f3dfd7fa640 1 -- 172.21.15.22:0/3312794039 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df801e030 con 0x55bc08873ac0 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.000+0000 7f3dfd7fa640 1 -- 172.21.15.22:0/3312794039 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3df8005500 con 0x55bc08873ac0 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.000+0000 7f3dfd7fa640 1 -- 172.21.15.22:0/3312794039 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8004d10 con 0x55bc08873ac0 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- 172.21.15.22:0/3312794039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 msgr2=0x55bc0887c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc0887c160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3df800f930 tx=0x7f3df8005f90 comp rx=0 tx=0).stop 2024-03-08T23:56:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- 172.21.15.22:0/3312794039 shutdown_connections 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc0887c6a0 0x55bc0887ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc0887c160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 --2- 172.21.15.22:0/3312794039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc088731a0 0x55bc08873580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- 172.21.15.22:0/3312794039 >> 172.21.15.22:0/3312794039 conn(0x55bc0886d8d0 msgr2=0x55bc0886dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- 172.21.15.22:0/3312794039 shutdown_connections 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- 172.21.15.22:0/3312794039 wait complete. 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- start start 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 Processor -- start 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08879620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08879b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc0887a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc087754a0 con 0x55bc088731a0 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc087d2f60 con 0x55bc08873ac0 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0887f490 con 0x55bc0887c6a0 2024-03-08T23:56:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e1022b740 10 client.0 _ll_register_callbacks cb 0x55bc087d0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08879620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08879620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47200/0 (socket says 172.21.15.22:47200) 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.001+0000 7f3e06ffd640 1 -- 172.21.15.22:0/1450045971 learned_addr learned my addr 172.21.15.22:0/1450045971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e077fe640 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc0887a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e067fc640 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08879b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e06ffd640 1 -- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 msgr2=0x55bc0887a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc0887a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e06ffd640 1 -- 172.21.15.22:0/1450045971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 msgr2=0x55bc08879b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08879b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e06ffd640 1 -- 172.21.15.22:0/1450045971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc084eb3f0 con 0x55bc088731a0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08879620 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3df8011010 tx=0x7f3df80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1952a0809a2f13f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3dee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3dee7fc640 1 -- 172.21.15.22:0/1450045971 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8004fa0 con 0x55bc088731a0 2024-03-08T23:56:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3dee7fc640 1 -- 172.21.15.22:0/1450045971 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3df8005140 con 0x55bc088731a0 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3df8005500 con 0x55bc088731a0 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3dee7fc640 1 -- 172.21.15.22:0/1450045971 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df8005340 con 0x55bc088731a0 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e1022b740 10 client.16158 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc085b57d0 con 0x55bc088731a0 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.002+0000 7f3dee7fc640 1 -- 172.21.15.22:0/1450045971 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3df8005500 con 0x55bc088731a0 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3dee7fc640 1 -- 172.21.15.22:0/1450045971 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3df802b070 con 0x55bc088731a0 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e1022b740 10 client.16158 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e1022b740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e1022b740 20 client.16158 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e1022b740 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc0888c670 0x55bc0888ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e1022b740 4 client.16158 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc087e4400 con 0x55bc0888c670 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e067fc640 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc0888c670 0x55bc0888ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3dee7fc640 10 client.16158 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:01.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.003+0000 7f3e067fc640 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc0888c670 0x55bc0888ea60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0ed6a37e232900 server_cookie=60bb8c1beeeb449 in_seq=0 out_seq=0 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3dee7fc640 1 -- 172.21.15.22:0/1450045971 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc087e4400 con 0x55bc0888c670 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3dee7fc640 10 client.16158 handle_command_reply: tid=0 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 client.16158 shutdown 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 msgr2=0x55bc08879620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08879620 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3df8011010 tx=0x7f3df80048f0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 shutdown_connections 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc0888c670 0x55bc0888ea60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc0887a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3dee7fc640 0 client.16158 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08879b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 --2- 172.21.15.22:0/1450045971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08879620 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 >> 172.21.15.22:0/1450045971 conn(0x55bc0886d8d0 msgr2=0x55bc0886f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:01.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 shutdown_connections 2024-03-08T23:56:01.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 1 -- 172.21.15.22:0/1450045971 wait complete. 2024-03-08T23:56:01.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.004+0000 7f3e1022b740 20 client.16158 trim_cache size 0 max 16384 2024-03-08T23:56:01.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 Processor -- start 2024-03-08T23:56:01.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 -- start start 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc0887bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc088768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc0887c6a0 0x55bc08876de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081ae8c0 con 0x55bc088731a0 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081ac460 con 0x55bc0887c6a0 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e1022b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081aa500 con 0x55bc08873ac0 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc0887bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc0887bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47208/0 (socket says 172.21.15.22:47208) 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e06ffd640 1 -- 172.21.15.22:0/627485315 learned_addr learned my addr 172.21.15.22:0/627485315 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e067fc640 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc088768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.016+0000 7f3e077fe640 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc0887c6a0 0x55bc08876de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e06ffd640 1 -- 172.21.15.22:0/627485315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 msgr2=0x55bc088768a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc088768a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e06ffd640 1 -- 172.21.15.22:0/627485315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc0887c6a0 msgr2=0x55bc08876de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc0887c6a0 0x55bc08876de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e06ffd640 1 -- 172.21.15.22:0/627485315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0887ea90 con 0x55bc088731a0 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc0887bd50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3df80039c0 tx=0x7f3df8010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320059be2206e182 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3dfcff9640 1 -- 172.21.15.22:0/627485315 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df801e040 con 0x55bc088731a0 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3dfcff9640 1 -- 172.21.15.22:0/627485315 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3df8024a70 con 0x55bc088731a0 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3dfcff9640 1 -- 172.21.15.22:0/627485315 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df801a910 con 0x55bc088731a0 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e1022b740 1 -- 172.21.15.22:0/627485315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 msgr2=0x55bc0887bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e1022b740 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc0887bd50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3df80039c0 tx=0x7f3df8010fa0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.017+0000 7f3e1022b740 1 -- 172.21.15.22:0/627485315 shutdown_connections 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc0887c6a0 0x55bc08876de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc08873ac0 0x55bc088768a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 --2- 172.21.15.22:0/627485315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc0887bd50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 -- 172.21.15.22:0/627485315 >> 172.21.15.22:0/627485315 conn(0x55bc0886d8d0 msgr2=0x55bc08893210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 -- 172.21.15.22:0/627485315 shutdown_connections 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 -- 172.21.15.22:0/627485315 wait complete. 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 -- start start 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 Processor -- start 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08882330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08882870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc08882db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081aab00 con 0x55bc088731a0 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081abe00 con 0x55bc08873ac0 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081ab140 con 0x55bc0887c6a0 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e1022b740 10 client.0 _ll_register_callbacks cb 0x7f3df80042f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08882330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08882870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08882870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35908/0 (socket says 172.21.15.22:35908) 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e067fc640 1 -- 172.21.15.22:0/219078276 learned_addr learned my addr 172.21.15.22:0/219078276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.018+0000 7f3e077fe640 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc08882db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e06ffd640 1 -- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 msgr2=0x55bc08882db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc08882db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e06ffd640 1 -- 172.21.15.22:0/219078276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 msgr2=0x55bc08882870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08882870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e06ffd640 1 -- 172.21.15.22:0/219078276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc088e2ad0 con 0x55bc088731a0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e067fc640 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08882870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e06ffd640 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08882330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3df8024880 tx=0x7f3df8004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61f844da5d3a516 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3dee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3dee7fc640 1 -- 172.21.15.22:0/219078276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df801e040 con 0x55bc088731a0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3dee7fc640 1 -- 172.21.15.22:0/219078276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3df801a7b0 con 0x55bc088731a0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3df8024a70 con 0x55bc088731a0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3dee7fc640 1 -- 172.21.15.22:0/219078276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df8024c20 con 0x55bc088731a0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 10 client.16164 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc088e3db0 con 0x55bc088731a0 2024-03-08T23:56:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3dee7fc640 1 -- 172.21.15.22:0/219078276 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3df8024a70 con 0x55bc088731a0 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3dee7fc640 1 -- 172.21.15.22:0/219078276 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3df802b050 con 0x55bc088731a0 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 20 client.16164 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc088e4340 0x55bc088e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 4 client.16164 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.019+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc087de750 con 0x55bc088e4340 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e067fc640 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc088e4340 0x55bc088e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3dee7fc640 10 client.16164 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e067fc640 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc088e4340 0x55bc088e6760 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4766919c202a3aee server_cookie=d7516be8e7c4d9b4 in_seq=0 out_seq=0 2024-03-08T23:56:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3dee7fc640 1 -- 172.21.15.22:0/219078276 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc087de750 con 0x55bc088e4340 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3dee7fc640 10 client.16164 handle_command_reply: tid=0 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 client.16164 shutdown 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 msgr2=0x55bc08882330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08882330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3df8024880 tx=0x7f3df8004fd0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 shutdown_connections 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bc088e4340 0x55bc088e6760 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3dee7fc640 0 client.16164 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc0887c6a0 0x55bc08882db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc08873ac0 0x55bc08882870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 --2- 172.21.15.22:0/219078276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc088731a0 0x55bc08882330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.020+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 >> 172.21.15.22:0/219078276 conn(0x55bc0886d8d0 msgr2=0x55bc088795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 shutdown_connections 2024-03-08T23:56:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e1022b740 1 -- 172.21.15.22:0/219078276 wait complete. 2024-03-08T23:56:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e1022b740 20 client.16164 trim_cache size 0 max 16384 2024-03-08T23:56:01.025 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3df40a2a90 msgr2=0x7f3df40a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3df40a2a90 0x7f3df40a4f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e00004af0 tx=0x7f3e0002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 msgr2=0x7f3e0819bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:01.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0819bb70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e9f0 tx=0x7f3df000eeb0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 shutdown_connections 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3df40a2a90 0x7f3df40a4f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e08195740 0x7f3e08195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e08106640 0x7f3e0819bb70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 --2- 172.21.15.22:0/258114382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e08069c50 0x7f3e0819b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 >> 172.21.15.22:0/258114382 conn(0x7f3e08076570 msgr2=0x7f3e080fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 shutdown_connections 2024-03-08T23:56:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:01.021+0000 7f3e0d64b640 1 -- 172.21.15.22:0/258114382 wait complete. 2024-03-08T23:56:01.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:00 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2587210044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:01.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:00 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2587210044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:00 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2587210044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:01.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 22749 15.64 MB/sec warmup 9 sec latency 46.454 ms 2024-03-08T23:56:02.028 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:56:02.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:56:02.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:56:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 -- 172.21.15.22:0/2339775727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4241003d0 msgr2=0x7ff4241007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/2339775727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4241003d0 0x7ff4241007b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ca0 tx=0x7ff41402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 -- 172.21.15.22:0/2339775727 shutdown_connections 2024-03-08T23:56:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/2339775727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4241054a0 0x7ff42410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/2339775727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff424100d80 0x7ff424104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/2339775727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4241003d0 0x7ff4241007b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 -- 172.21.15.22:0/2339775727 >> 172.21.15.22:0/2339775727 conn(0x7ff4240772f0 msgr2=0x7ff424077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 -- 172.21.15.22:0/2339775727 shutdown_connections 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.143+0000 7ff42b2ab640 1 -- 172.21.15.22:0/2339775727 wait complete. 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 Processor -- start 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 -- start start 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4241003d0 0x7ff42419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4241054a0 0x7ff42419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff424197980 0x7ff424197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424111680 con 0x7ff424197980 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424111500 con 0x7ff4241054a0 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff42b2ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424111800 con 0x7ff4241003d0 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff424197980 0x7ff424197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff424197980 0x7ff424197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47222/0 (socket says 172.21.15.22:47222) 2024-03-08T23:56:02.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429047640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4241003d0 0x7ff42419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 -- 172.21.15.22:0/1567390919 learned_addr learned my addr 172.21.15.22:0/1567390919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff428846640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4241054a0 0x7ff42419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 -- 172.21.15.22:0/1567390919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4241003d0 msgr2=0x7ff42419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4241003d0 0x7ff42419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 -- 172.21.15.22:0/1567390919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4241054a0 msgr2=0x7ff42419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4241054a0 0x7ff42419dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.144+0000 7ff429848640 1 -- 172.21.15.22:0/1567390919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4241984a0 con 0x7ff424197980 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.145+0000 7ff429848640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff424197980 0x7ff424197e30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e970 tx=0x7ff41c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.145+0000 7ff4227fc640 1 -- 172.21.15.22:0/1567390919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00ccc0 con 0x7ff424197980 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.145+0000 7ff4227fc640 1 -- 172.21.15.22:0/1567390919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41c00ce60 con 0x7ff424197980 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.145+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff424198790 con 0x7ff424197980 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.145+0000 7ff4227fc640 1 -- 172.21.15.22:0/1567390919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c010490 con 0x7ff424197980 2024-03-08T23:56:02.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.145+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a49e0 con 0x7ff424197980 2024-03-08T23:56:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.147+0000 7ff4227fc640 1 -- 172.21.15.22:0/1567390919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7ff41c005020 con 0x7ff424197980 2024-03-08T23:56:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.147+0000 7ff42df6c740 1 Processor -- start 2024-03-08T23:56:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.147+0000 7ff42df6c740 1 -- start start 2024-03-08T23:56:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff4227fc640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff40c0a2a90 0x7ff40c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff42df6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858ba3bb40 con 0x55858bad7d80 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff42df6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858b9e4d80 con 0x55858bad86a0 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff42df6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858ba37be0 con 0x55858bae1280 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff428846640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429047640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff40c0a2a90 0x7ff40c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff428846640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47226/0 (socket says 172.21.15.22:47226) 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff428846640 1 -- 172.21.15.22:0/2842599640 learned_addr learned my addr 172.21.15.22:0/2842599640 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429848640 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff4227fc640 1 -- 172.21.15.22:0/1567390919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff41c0cf870 con 0x7ff424197980 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff428846640 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429848640 1 -- 172.21.15.22:0/2842599640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 msgr2=0x55858bae0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429848640 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429848640 1 -- 172.21.15.22:0/2842599640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 msgr2=0x55858bad8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429848640 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429848640 1 -- 172.21.15.22:0/2842599640 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55858b8187e0 con 0x55858bae1280 2024-03-08T23:56:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429047640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff40c0a2a90 0x7ff40c0a4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff414004af0 tx=0x7ff414005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.148+0000 7ff429848640 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae3670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0ccaa0 tx=0x7ff41c09b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a656301daf8bb8d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff40b7fe640 1 -- 172.21.15.22:0/2842599640 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c065dc0 con 0x55858bae1280 2024-03-08T23:56:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff40b7fe640 1 -- 172.21.15.22:0/2842599640 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41c09bfe0 con 0x55858bae1280 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff40b7fe640 1 -- 172.21.15.22:0/2842599640 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c099be0 con 0x55858bae1280 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 -- 172.21.15.22:0/2842599640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 msgr2=0x55858bae3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae3670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0ccaa0 tx=0x7ff41c09b420 comp rx=0 tx=0).stop 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 -- 172.21.15.22:0/2842599640 shutdown_connections 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae3670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 --2- 172.21.15.22:0/2842599640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 -- 172.21.15.22:0/2842599640 >> 172.21.15.22:0/2842599640 conn(0x55858bad24b0 msgr2=0x55858bad2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 -- 172.21.15.22:0/2842599640 shutdown_connections 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.149+0000 7ff42df6c740 1 -- 172.21.15.22:0/2842599640 wait complete. 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 -- start start 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 Processor -- start 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bad86a0 0x55858bade080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bae1280 0x55858bade7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858bae4070 con 0x55858bad7d80 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858b9f81c0 con 0x55858bae1280 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858bae41f0 con 0x55858bad86a0 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff42df6c740 10 client.0 _ll_register_callbacks cb 0x55858b9e77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff429047640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff428846640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bad86a0 0x55858bade080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff429848640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bae1280 0x55858bade7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff429047640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47240/0 (socket says 172.21.15.22:47240) 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.150+0000 7ff429047640 1 -- 172.21.15.22:0/1260292653 learned_addr learned my addr 172.21.15.22:0/1260292653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff429047640 1 -- 172.21.15.22:0/1260292653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bad86a0 msgr2=0x55858bade080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff429047640 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bad86a0 0x55858bade080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff429047640 1 -- 172.21.15.22:0/1260292653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bae1280 msgr2=0x55858bade7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff429047640 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bae1280 0x55858bade7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff429047640 1 -- 172.21.15.22:0/1260292653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55858b878f70 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff429848640 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bae1280 0x55858bade7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff429047640 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae0cf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ca0 tx=0x7ff41402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732c1cd45d82d14f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff408ff9640 1 -- 172.21.15.22:0/1260292653 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414050060 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff408ff9640 1 -- 172.21.15.22:0/1260292653 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff414003b70 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff41c09bfe0 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff408ff9640 1 -- 172.21.15.22:0/1260292653 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff414057050 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff42df6c740 10 client.16176 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55858b710170 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.151+0000 7ff408ff9640 1 -- 172.21.15.22:0/1260292653 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41c09bfe0 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff408ff9640 1 -- 172.21.15.22:0/1260292653 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff4140346a0 con 0x55858bad7d80 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff42df6c740 10 client.16176 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff42df6c740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:02.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff42df6c740 20 client.16176 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858baf13f0 0x55858baf37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff42df6c740 4 client.16176 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55858ba11960 con 0x55858baf13f0 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff428846640 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858baf13f0 0x55858baf37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff408ff9640 10 client.16176 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.152+0000 7ff428846640 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858baf13f0 0x55858baf37e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1d86771c591812 server_cookie=825c6ed7fb72e3b3 in_seq=0 out_seq=0 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff408ff9640 1 -- 172.21.15.22:0/1260292653 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55858ba11960 con 0x55858baf13f0 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff408ff9640 10 client.16176 handle_command_reply: tid=0 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 client.16176 shutdown 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 msgr2=0x55858bae0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae0cf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ca0 tx=0x7ff41402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 shutdown_connections 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858baf13f0 0x55858baf37e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bae1280 0x55858bade7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bad86a0 0x55858bade080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1260292653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae0cf0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff408ff9640 0 client.16176 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 >> 172.21.15.22:0/1260292653 conn(0x55858bad24b0 msgr2=0x55858bad4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 shutdown_connections 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.153+0000 7ff42df6c740 1 -- 172.21.15.22:0/1260292653 wait complete. 2024-03-08T23:56:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.154+0000 7ff42df6c740 20 client.16176 trim_cache size 0 max 16384 2024-03-08T23:56:02.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 Processor -- start 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 -- start start 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858badd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241b0a90 con 0x55858bad7d80 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241ae630 con 0x55858bad86a0 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff42df6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241ac6d0 con 0x55858bae1280 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47248/0 (socket says 172.21.15.22:47248) 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 -- 172.21.15.22:0/1232752233 learned_addr learned my addr 172.21.15.22:0/1232752233 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff428846640 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858badd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429848640 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 -- 172.21.15.22:0/1232752233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 msgr2=0x55858bae8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 -- 172.21.15.22:0/1232752233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 msgr2=0x55858badd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858badd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.169+0000 7ff429047640 1 -- 172.21.15.22:0/1232752233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55858badeb30 con 0x55858bad7d80 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff429047640 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8a80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff414004980 tx=0x7ff414009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95476f2eb48d9541 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff40affd640 1 -- 172.21.15.22:0/1232752233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414050040 con 0x55858bad7d80 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff40affd640 1 -- 172.21.15.22:0/1232752233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41404b470 con 0x55858bad7d80 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff40affd640 1 -- 172.21.15.22:0/1232752233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414044070 con 0x55858bad7d80 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 -- 172.21.15.22:0/1232752233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 msgr2=0x55858bad8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8a80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff414004980 tx=0x7ff414009a00 comp rx=0 tx=0).stop 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 -- 172.21.15.22:0/1232752233 shutdown_connections 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bae8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858badd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1232752233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bad8a80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 -- 172.21.15.22:0/1232752233 >> 172.21.15.22:0/1232752233 conn(0x55858bad24b0 msgr2=0x55858badc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 -- 172.21.15.22:0/1232752233 shutdown_connections 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.170+0000 7ff42df6c740 1 -- 172.21.15.22:0/1232752233 wait complete. 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 -- start start 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 Processor -- start 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bb478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241accd0 con 0x55858bad7d80 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241adfd0 con 0x55858bad86a0 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241ad310 con 0x55858bae1280 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff42df6c740 10 client.0 _ll_register_callbacks cb 0x7ff41c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429047640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429047640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47256/0 (socket says 172.21.15.22:47256) 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429047640 1 -- 172.21.15.22:0/1388473705 learned_addr learned my addr 172.21.15.22:0/1388473705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429848640 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bb478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff428846640 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429047640 1 -- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 msgr2=0x55858bb478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429047640 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bb478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429047640 1 -- 172.21.15.22:0/1388473705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 msgr2=0x55858bae76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.171+0000 7ff429047640 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff429047640 1 -- 172.21.15.22:0/1388473705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55858bad84d0 con 0x55858bad7d80 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff429047640 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae7180 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff4140049d0 tx=0x7ff4140346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65e7f4212725755 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff408ff9640 1 -- 172.21.15.22:0/1388473705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414044070 con 0x55858bad7d80 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff408ff9640 1 -- 172.21.15.22:0/1388473705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff41404ba30 con 0x55858bad7d80 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff41404b470 con 0x55858bad7d80 2024-03-08T23:56:02.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff408ff9640 1 -- 172.21.15.22:0/1388473705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff41406d050 con 0x55858bad7d80 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55858bb48dc0 con 0x55858bad7d80 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff408ff9640 1 -- 172.21.15.22:0/1388473705 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41404b470 con 0x55858bad7d80 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff408ff9640 1 -- 172.21.15.22:0/1388473705 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff41405b430 con 0x55858bad7d80 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 20 client.16182 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858bb49320 0x55858bb4b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 4 client.16182 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.172+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55858b9fd080 con 0x55858bb49320 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.173+0000 7ff428846640 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858bb49320 0x55858bb4b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.173+0000 7ff408ff9640 10 client.16182 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.173+0000 7ff428846640 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858bb49320 0x55858bb4b710 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f9db25b56136f6 server_cookie=8c3e555fefb72926 in_seq=0 out_seq=0 2024-03-08T23:56:02.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff408ff9640 1 -- 172.21.15.22:0/1388473705 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55858b9fd080 con 0x55858bb49320 2024-03-08T23:56:02.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff408ff9640 10 client.16182 handle_command_reply: tid=0 2024-03-08T23:56:02.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 client.16182 shutdown 2024-03-08T23:56:02.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 msgr2=0x55858bae7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae7180 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff4140049d0 tx=0x7ff4140346f0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 shutdown_connections 2024-03-08T23:56:02.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55858bb49320 0x55858bb4b710 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55858bae1280 0x55858bb478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55858bad86a0 0x55858bae76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 --2- 172.21.15.22:0/1388473705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55858bad7d80 0x55858bae7180 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 >> 172.21.15.22:0/1388473705 conn(0x55858bad24b0 msgr2=0x55858bad38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff408ff9640 0 client.16182 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 shutdown_connections 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.175+0000 7ff42df6c740 1 -- 172.21.15.22:0/1388473705 wait complete. 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42df6c740 20 client.16182 trim_cache size 0 max 16384 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:56:02.177 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "8273ace9-8a23-425a-b763-f14039ba1503", 2024-03-08T23:56:02.178 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:56:02.178 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff40c0a2a90 msgr2=0x7ff40c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff40c0a2a90 0x7ff40c0a4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff414004af0 tx=0x7ff414005c50 comp rx=0 tx=0).stop 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff424197980 msgr2=0x7ff424197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff424197980 0x7ff424197e30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e970 tx=0x7ff41c00ed70 comp rx=0 tx=0).stop 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 shutdown_connections 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff40c0a2a90 0x7ff40c0a4f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff424197980 0x7ff424197e30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4241054a0 0x7ff42419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 --2- 172.21.15.22:0/1567390919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4241003d0 0x7ff42419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 >> 172.21.15.22:0/1567390919 conn(0x7ff4240772f0 msgr2=0x7ff4240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 shutdown_connections 2024-03-08T23:56:02.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.176+0000 7ff42b2ab640 1 -- 172.21.15.22:0/1567390919 wait complete. 2024-03-08T23:56:02.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:01 smithi022 ceph-mon[31894]: pgmap v48: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 868 KiB/s wr, 186 op/s 2024-03-08T23:56:02.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:56:02.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:56:02.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 -- 172.21.15.22:0/4090787766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c0ffbb0 msgr2=0x7fd37c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 --2- 172.21.15.22:0/4090787766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c0ffbb0 0x7fd37c100030 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd370009930 tx=0x7fd37002bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:02.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 -- 172.21.15.22:0/4090787766 shutdown_connections 2024-03-08T23:56:02.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 --2- 172.21.15.22:0/4090787766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c100570 0x7fd37c10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 --2- 172.21.15.22:0/4090787766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c0ffbb0 0x7fd37c100030 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 --2- 172.21.15.22:0/4090787766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd37c1074f0 0x7fd37c1078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 -- 172.21.15.22:0/4090787766 >> 172.21.15.22:0/4090787766 conn(0x7fd37c075530 msgr2=0x7fd37c075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 -- 172.21.15.22:0/4090787766 shutdown_connections 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.294+0000 7fd382140640 1 -- 172.21.15.22:0/4090787766 wait complete. 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 Processor -- start 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 -- start start 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0ffbb0 0x7fd37c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c100570 0x7fd37c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd37c1074f0 0x7fd37c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c111720 con 0x7fd37c100570 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1115a0 con 0x7fd37c0ffbb0 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.295+0000 7fd382140640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1118a0 con 0x7fd37c1074f0 2024-03-08T23:56:02.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c100570 0x7fd37c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c100570 0x7fd37c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47270/0 (socket says 172.21.15.22:47270) 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 -- 172.21.15.22:0/1344003414 learned_addr learned my addr 172.21.15.22:0/1344003414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0ffbb0 0x7fd37c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37bfff640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd37c1074f0 0x7fd37c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 -- 172.21.15.22:0/1344003414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd37c1074f0 msgr2=0x7fd37c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd37c1074f0 0x7fd37c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 -- 172.21.15.22:0/1344003414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0ffbb0 msgr2=0x7fd37c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0ffbb0 0x7fd37c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 -- 172.21.15.22:0/1344003414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c198140 con 0x7fd37c100570 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0ffbb0 0x7fd37c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:02.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd37affd640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c100570 0x7fd37c19dd50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd370009900 tx=0x7fd370004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd378ff9640 1 -- 172.21.15.22:0/1344003414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370043020 con 0x7fd37c100570 2024-03-08T23:56:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c1983d0 con 0x7fd37c100570 2024-03-08T23:56:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd378ff9640 1 -- 172.21.15.22:0/1344003414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3700054e0 con 0x7fd37c100570 2024-03-08T23:56:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd378ff9640 1 -- 172.21.15.22:0/1344003414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370003bb0 con 0x7fd37c100570 2024-03-08T23:56:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.296+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c1a46d0 con 0x7fd37c100570 2024-03-08T23:56:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.298+0000 7fd378ff9640 1 -- 172.21.15.22:0/1344003414 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 26) v1 ==== 137523+0+0 (secure 0 0 0) 0x7fd370046020 con 0x7fd37c100570 2024-03-08T23:56:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 Processor -- start 2024-03-08T23:56:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 -- start start 2024-03-08T23:56:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bacb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bad080 0x5598f4bb5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bb8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4b0fe00 con 0x5598f4bad080 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4ab8b10 con 0x5598f4bac760 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd384df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4b0bea0 con 0x5598f4bb5c60 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd378ff9640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd3640a2a90 0x7fd3640a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bad080 0x5598f4bb5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bad080 0x5598f4bb5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47280/0 (socket says 172.21.15.22:47280) 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bb8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1306557382 learned_addr learned my addr 172.21.15.22:0/1306557382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37bfff640 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bacb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37bfff640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd3640a2a90 0x7fd3640a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1306557382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 msgr2=0x5598f4bb8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd378ff9640 1 -- 172.21.15.22:0/1344003414 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd3700f1400 con 0x7fd37c100570 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bb8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1306557382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 msgr2=0x5598f4bacb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bacb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.299+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1306557382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598f48ee7d0 con 0x5598f4bad080 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.300+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bad080 0x5598f4bb5720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0114f0 tx=0x7fd36c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b1d4618f15a19d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.300+0000 7fd37bfff640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd3640a2a90 0x7fd3640a4f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd374005fd0 tx=0x7fd374012040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.300+0000 7fd361ffb640 1 -- 172.21.15.22:0/1306557382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c01e070 con 0x5598f4bad080 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.300+0000 7fd361ffb640 1 -- 172.21.15.22:0/1306557382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd36c0035f0 con 0x5598f4bad080 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.300+0000 7fd361ffb640 1 -- 172.21.15.22:0/1306557382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00f820 con 0x5598f4bad080 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- 172.21.15.22:0/1306557382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bad080 msgr2=0x5598f4bb5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bad080 0x5598f4bb5720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0114f0 tx=0x7fd36c0119b0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- 172.21.15.22:0/1306557382 shutdown_connections 2024-03-08T23:56:02.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bb8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bad080 0x5598f4bb5720 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 --2- 172.21.15.22:0/1306557382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bacb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- 172.21.15.22:0/1306557382 >> 172.21.15.22:0/1306557382 conn(0x5598f4ba6e90 msgr2=0x5598f4ba7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- 172.21.15.22:0/1306557382 shutdown_connections 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- 172.21.15.22:0/1306557382 wait complete. 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- start start 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 Processor -- start 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bb5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bb27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4bb2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4bb8a50 con 0x5598f4bb5c60 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4acc0a0 con 0x5598f4bac760 2024-03-08T23:56:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.301+0000 7fd384df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f4bb8bd0 con 0x5598f4bad080 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd384df0740 10 client.0 _ll_register_callbacks cb 0x5598f4acba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4bb2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bb5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bb5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35984/0 (socket says 172.21.15.22:35984) 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 -- 172.21.15.22:0/427876691 learned_addr learned my addr 172.21.15.22:0/427876691 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37affd640 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bb27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 -- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 msgr2=0x5598f4bb27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bb27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 -- 172.21.15.22:0/427876691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 msgr2=0x5598f4bb2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4bb2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 -- 172.21.15.22:0/427876691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598f48243f0 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bb5450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009700 tx=0x7fd36c0039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10d7e20f573c948 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd35affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd35affd640 1 -- 172.21.15.22:0/427876691 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c01e040 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd36c0035f0 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd35affd640 1 -- 172.21.15.22:0/427876691 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd36c016050 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd35affd640 1 -- 172.21.15.22:0/427876691 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd36c031050 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd384df0740 10 client.25694 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.302+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5598f47e40f0 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd35affd640 1 -- 172.21.15.22:0/427876691 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd36c0035f0 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd35affd640 1 -- 172.21.15.22:0/427876691 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd36c021030 con 0x5598f4bac760 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd384df0740 10 client.25694 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd384df0740 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:02.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd384df0740 20 client.25694 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd384df0740 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4bc5790 0x5598f4bc7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd384df0740 4 client.25694 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598f4b1d340 con 0x5598f4bc5790 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd37affd640 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4bc5790 0x5598f4bc7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd37affd640 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4bc5790 0x5598f4bc7b80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93510156091bc783 server_cookie=464db9e01896a976 in_seq=0 out_seq=0 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.303+0000 7fd35affd640 10 client.25694 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd35affd640 1 -- 172.21.15.22:0/427876691 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598f4b1d340 con 0x5598f4bc5790 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd35affd640 10 client.25694 handle_command_reply: tid=0 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 client.25694 shutdown 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 msgr2=0x5598f4bb5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bb5450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009700 tx=0x7fd36c0039d0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 shutdown_connections 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4bc5790 0x5598f4bc7b80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4bb2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bb27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 --2- 172.21.15.22:0/427876691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bb5450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd35affd640 0 client.25694 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.304+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 >> 172.21.15.22:0/427876691 conn(0x5598f4ba6e90 msgr2=0x5598f4ba8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.305+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 shutdown_connections 2024-03-08T23:56:02.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.305+0000 7fd384df0740 1 -- 172.21.15.22:0/427876691 wait complete. 2024-03-08T23:56:02.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.305+0000 7fd384df0740 20 client.25694 trim_cache size 0 max 16384 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.319+0000 7fd384df0740 1 Processor -- start 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.319+0000 7fd384df0740 1 -- start start 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd384df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bac760 0x5598f4bb4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd384df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bad080 0x5598f4bb5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd384df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bbce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd384df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1b09c0 con 0x5598f4bac760 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd384df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1ae560 con 0x5598f4bad080 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd384df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1ac600 con 0x5598f4bb5c60 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bac760 0x5598f4bb4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bac760 0x5598f4bb4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47302/0 (socket says 172.21.15.22:47302) 2024-03-08T23:56:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 -- 172.21.15.22:0/3481350257 learned_addr learned my addr 172.21.15.22:0/3481350257 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37affd640 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bad080 0x5598f4bb5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37bfff640 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bbce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 -- 172.21.15.22:0/3481350257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 msgr2=0x5598f4bbce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bbce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 -- 172.21.15.22:0/3481350257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bad080 msgr2=0x5598f4bb5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bad080 0x5598f4bb5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 -- 172.21.15.22:0/3481350257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598f4bb8930 con 0x5598f4bac760 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37affd640 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bad080 0x5598f4bb5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bac760 0x5598f4bb4f10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0057c0 tx=0x7fd36c004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44726cf256d4a0c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd3617fa640 1 -- 172.21.15.22:0/3481350257 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c01e070 con 0x5598f4bac760 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd3617fa640 1 -- 172.21.15.22:0/3481350257 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd36c004e20 con 0x5598f4bac760 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.320+0000 7fd3617fa640 1 -- 172.21.15.22:0/3481350257 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0052a0 con 0x5598f4bac760 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 -- 172.21.15.22:0/3481350257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bac760 msgr2=0x5598f4bb4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bac760 0x5598f4bb4f10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0057c0 tx=0x7fd36c004a70 comp rx=0 tx=0).stop 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 -- 172.21.15.22:0/3481350257 shutdown_connections 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bb5c60 0x5598f4bbce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bad080 0x5598f4bb5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 --2- 172.21.15.22:0/3481350257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bac760 0x5598f4bb4f10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 -- 172.21.15.22:0/3481350257 >> 172.21.15.22:0/3481350257 conn(0x5598f4ba6e90 msgr2=0x5598f4bb27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 -- 172.21.15.22:0/3481350257 shutdown_connections 2024-03-08T23:56:02.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 -- 172.21.15.22:0/3481350257 wait complete. 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 -- start start 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.321+0000 7fd384df0740 1 Processor -- start 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bbb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bbb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4c1aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1acc00 con 0x5598f4bb5c60 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1adf00 con 0x5598f4bac760 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1ad240 con 0x5598f4bad080 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 10 client.0 _ll_register_callbacks cb 0x7fd36c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:02.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4c1aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bbb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bbb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4c1aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47306/0 (socket says 172.21.15.22:47306) 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bbb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35998/0 (socket says 172.21.15.22:35998) 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1174636760 learned_addr learned my addr 172.21.15.22:0/1174636760 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1174636760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 msgr2=0x5598f4bbb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bbb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1174636760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 msgr2=0x5598f4c1aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4c1aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 -- 172.21.15.22:0/1174636760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598f4c1b090 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd37b7fe640 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bbb050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd36c000c00 tx=0x7fd36c009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c180958cbbc51bb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd35affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd35affd640 1 -- 172.21.15.22:0/1174636760 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c03b020 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd36c004e20 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.322+0000 7fd35affd640 1 -- 172.21.15.22:0/1174636760 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd36c005040 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd35affd640 1 -- 172.21.15.22:0/1174636760 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd36c046050 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 10 client.25700 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5598f4c1c380 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd35affd640 1 -- 172.21.15.22:0/1174636760 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd36c004e20 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd35affd640 1 -- 172.21.15.22:0/1174636760 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd36c021070 con 0x5598f4bac760 2024-03-08T23:56:02.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 10 client.25700 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 20 client.25700 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4c1c930 0x5598f4c1ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 4 client.25700 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598f4b17690 con 0x5598f4c1c930 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd37affd640 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4c1c930 0x5598f4c1ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd37affd640 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4c1c930 0x5598f4c1ed50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c9e0cdb065710f server_cookie=66739608433127a5 in_seq=0 out_seq=0 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.323+0000 7fd35affd640 10 client.25700 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd35affd640 1 -- 172.21.15.22:0/1174636760 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5598f4b17690 con 0x5598f4c1c930 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd35affd640 10 client.25700 handle_command_reply: tid=0 2024-03-08T23:56:02.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 client.25700 shutdown 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 msgr2=0x5598f4bbb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bbb050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd36c000c00 tx=0x7fd36c009d80 comp rx=0 tx=0).stop 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 shutdown_connections 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5598f4c1c930 0x5598f4c1ed50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5598f4bb5c60 0x5598f4c1aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f4bad080 0x5598f4bbb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd35affd640 0 client.25700 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 --2- 172.21.15.22:0/1174636760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f4bac760 0x5598f4bbb050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.324+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 >> 172.21.15.22:0/1174636760 conn(0x5598f4ba6e90 msgr2=0x5598f4bb0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 shutdown_connections 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd384df0740 1 -- 172.21.15.22:0/1174636760 wait complete. 2024-03-08T23:56:02.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd384df0740 20 client.25700 trim_cache size 0 max 16384 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: "8273ace9-8a23-425a-b763-f14039ba1503": { 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "8273ace9-8a23-425a-b763-f14039ba1503", 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:56:02.330 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:02.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd3640a2a90 msgr2=0x7fd3640a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd3640a2a90 0x7fd3640a4f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd374005fd0 tx=0x7fd374012040 comp rx=0 tx=0).stop 2024-03-08T23:56:02.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c100570 msgr2=0x7fd37c19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:02.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c100570 0x7fd37c19dd50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd370009900 tx=0x7fd370004360 comp rx=0 tx=0).stop 2024-03-08T23:56:02.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 shutdown_connections 2024-03-08T23:56:02.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd3640a2a90 0x7fd3640a4f50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd37c1074f0 0x7fd37c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd37c100570 0x7fd37c19dd50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 --2- 172.21.15.22:0/1344003414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0ffbb0 0x7fd37c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:02.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 >> 172.21.15.22:0/1344003414 conn(0x7fd37c075530 msgr2=0x7fd37c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:02.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 shutdown_connections 2024-03-08T23:56:02.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:02.325+0000 7fd382140640 1 -- 172.21.15.22:0/1344003414 wait complete. 2024-03-08T23:56:02.333 INFO:tasks.cephfs.filesystem:scrub status for tag:8273ace9-8a23-425a-b763-f14039ba1503 - {'path': '/', 'tag': '8273ace9-8a23-425a-b763-f14039ba1503', 'options': 'recursive,force'} 2024-03-08T23:56:02.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:01 smithi062 ceph-mon[36129]: pgmap v48: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 868 KiB/s wr, 186 op/s 2024-03-08T23:56:02.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:01 smithi192 ceph-mon[35995]: pgmap v48: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 868 KiB/s wr, 186 op/s 2024-03-08T23:56:02.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 25456 15.49 MB/sec warmup 10 sec latency 17.583 ms 2024-03-08T23:56:03.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:02 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:56:03.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:02 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:03.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:02 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:56:03.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:02 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:03.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:02 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:56:03.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:02 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:03.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:02 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:56:03.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:02 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:03.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:02 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:56:03.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:02 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:03.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:02 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:56:03.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:02 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:03.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 28678 15.79 MB/sec warmup 11 sec latency 7.756 ms 2024-03-08T23:56:04.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:03 smithi022 ceph-mon[31894]: pgmap v49: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 868 KiB/s wr, 186 op/s 2024-03-08T23:56:04.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:03 smithi062 ceph-mon[36129]: pgmap v49: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 868 KiB/s wr, 186 op/s 2024-03-08T23:56:04.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:03 smithi192 ceph-mon[35995]: pgmap v49: 129 pgs: 129 active+clean; 9.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 868 KiB/s wr, 186 op/s 2024-03-08T23:56:04.704 INFO:tasks.workunit.client.0.smithi022.stdout: 1 31427 15.55 MB/sec warmup 12 sec latency 6.465 ms 2024-03-08T23:56:05.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:04 smithi022 ceph-mon[31894]: mgrmap e27: x(active, since 91s), standbys: z, y 2024-03-08T23:56:05.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:04 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:56:05.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:04 smithi062 ceph-mon[36129]: mgrmap e27: x(active, since 91s), standbys: z, y 2024-03-08T23:56:05.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:04 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:56:05.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:04 smithi192 ceph-mon[35995]: mgrmap e27: x(active, since 91s), standbys: z, y 2024-03-08T23:56:05.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:04 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:56:05.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 33659 15.26 MB/sec warmup 13 sec latency 14.521 ms 2024-03-08T23:56:06.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:05 smithi022 ceph-mon[31894]: pgmap v50: 129 pgs: 129 active+clean; 17 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 315 op/s 2024-03-08T23:56:06.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:05 smithi062 ceph-mon[36129]: pgmap v50: 129 pgs: 129 active+clean; 17 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 315 op/s 2024-03-08T23:56:06.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:05 smithi192 ceph-mon[35995]: pgmap v50: 129 pgs: 129 active+clean; 17 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 315 op/s 2024-03-08T23:56:06.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 36292 15.19 MB/sec warmup 14 sec latency 6.814 ms 2024-03-08T23:56:07.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 39423 15.42 MB/sec warmup 15 sec latency 12.460 ms 2024-03-08T23:56:08.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:07 smithi022 ceph-mon[31894]: pgmap v51: 129 pgs: 129 active+clean; 17 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-03-08T23:56:08.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:07 smithi062 ceph-mon[36129]: pgmap v51: 129 pgs: 129 active+clean; 17 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-03-08T23:56:08.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:07 smithi192 ceph-mon[35995]: pgmap v51: 129 pgs: 129 active+clean; 17 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-03-08T23:56:08.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 42038 15.24 MB/sec warmup 16 sec latency 44.342 ms 2024-03-08T23:56:09.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 44458 15.05 MB/sec warmup 17 sec latency 6.515 ms 2024-03-08T23:56:10.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:09 smithi022 ceph-mon[31894]: pgmap v52: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 380 op/s 2024-03-08T23:56:10.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:09 smithi062 ceph-mon[36129]: pgmap v52: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 380 op/s 2024-03-08T23:56:10.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:09 smithi192 ceph-mon[35995]: pgmap v52: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 380 op/s 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 -- 172.21.15.22:0/2412593664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc105230 msgr2=0x7f65cc105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 --2- 172.21.15.22:0/2412593664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc105230 0x7f65cc105610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009900 tx=0x7f65c802bc80 comp rx=0 tx=0).stop 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 -- 172.21.15.22:0/2412593664 shutdown_connections 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 --2- 172.21.15.22:0/2412593664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65cc100e70 0x7f65cc10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 --2- 172.21.15.22:0/2412593664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65cc1004d0 0x7f65cc100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 --2- 172.21.15.22:0/2412593664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc105230 0x7f65cc105610 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 -- 172.21.15.22:0/2412593664 >> 172.21.15.22:0/2412593664 conn(0x7f65cc076fd0 msgr2=0x7f65cc0773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.395+0000 7f65d40a4640 1 -- 172.21.15.22:0/2412593664 shutdown_connections 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 -- 172.21.15.22:0/2412593664 wait complete. 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 Processor -- start 2024-03-08T23:56:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 -- start start 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc1004d0 0x7f65cc0fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65cc100e70 0x7f65cc0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65cc105230 0x7f65cc0fe820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1105e0 con 0x7f65cc1004d0 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc110460 con 0x7f65cc105230 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.396+0000 7f65d40a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc110760 con 0x7f65cc100e70 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc1004d0 0x7f65cc0fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc1004d0 0x7f65cc0fdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47330/0 (socket says 172.21.15.22:47330) 2024-03-08T23:56:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 -- 172.21.15.22:0/3319150241 learned_addr learned my addr 172.21.15.22:0/3319150241 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d163f640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65cc100e70 0x7f65cc0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d2641640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65cc105230 0x7f65cc0fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 -- 172.21.15.22:0/3319150241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65cc100e70 msgr2=0x7f65cc0fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65cc100e70 0x7f65cc0fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 -- 172.21.15.22:0/3319150241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65cc105230 msgr2=0x7f65cc0fe820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65cc105230 0x7f65cc0fe820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 -- 172.21.15.22:0/3319150241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc1a6030 con 0x7f65cc1004d0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d1e40640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc1004d0 0x7f65cc0fdda0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006010 tx=0x7f65c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65c2ffd640 1 -- 172.21.15.22:0/3319150241 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8041070 con 0x7f65cc1004d0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65c2ffd640 1 -- 172.21.15.22:0/3319150241 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65c80054e0 con 0x7f65cc1004d0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc1a62c0 con 0x7f65cc1004d0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65c2ffd640 1 -- 172.21.15.22:0/3319150241 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8003cc0 con 0x7f65cc1004d0 2024-03-08T23:56:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.397+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc1a6860 con 0x7f65cc1004d0 2024-03-08T23:56:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.400+0000 7f65c2ffd640 1 -- 172.21.15.22:0/3319150241 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f65c8044070 con 0x7f65cc1004d0 2024-03-08T23:56:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.400+0000 7f65d6d65740 1 Processor -- start 2024-03-08T23:56:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d6d65740 1 -- start start 2024-03-08T23:56:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2395a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a2396450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a239f550 0x55a1a23a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d6d65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a22fa5f0 con 0x55a1a2396d40 2024-03-08T23:56:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d6d65740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a22a3930 con 0x55a1a239f550 2024-03-08T23:56:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d6d65740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a22f6690 con 0x55a1a2395fd0 2024-03-08T23:56:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65c2ffd640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65b80a2a90 0x7f65b80a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d2641640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2395a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d2641640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2395a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47332/0 (socket says 172.21.15.22:47332) 2024-03-08T23:56:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d2641640 1 -- 172.21.15.22:0/3801150067 learned_addr learned my addr 172.21.15.22:0/3801150067 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d2641640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65b80a2a90 0x7f65b80a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d163f640 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a239f550 0x55a1a23a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65c2ffd640 1 -- 172.21.15.22:0/3319150241 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f65c80f0d10 con 0x7f65cc1004d0 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.401+0000 7f65d1e40640 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a2396450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d163f640 1 -- 172.21.15.22:0/3801150067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 msgr2=0x55a1a2396450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d163f640 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a2396450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d163f640 1 -- 172.21.15.22:0/3801150067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 msgr2=0x55a1a2395a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d163f640 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2395a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d163f640 1 -- 172.21.15.22:0/3801150067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a200f210 con 0x55a1a239f550 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d2641640 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2395a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d2641640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65b80a2a90 0x7f65b80a4f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009930 tx=0x7f65c4009c00 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65d163f640 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a239f550 0x55a1a23a1940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009bf0 tx=0x7f65bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f53612090cdf9f43 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65b7fff640 1 -- 172.21.15.22:0/3801150067 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc004050 con 0x55a1a239f550 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65b7fff640 1 -- 172.21.15.22:0/3801150067 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65bc005500 con 0x55a1a239f550 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.402+0000 7f65b7fff640 1 -- 172.21.15.22:0/3801150067 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc004db0 con 0x55a1a239f550 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 -- 172.21.15.22:0/3801150067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a239f550 msgr2=0x55a1a23a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a239f550 0x55a1a23a1940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009bf0 tx=0x7f65bc011810 comp rx=0 tx=0).stop 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 -- 172.21.15.22:0/3801150067 shutdown_connections 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a239f550 0x55a1a23a1940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a2396450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 --2- 172.21.15.22:0/3801150067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2395a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 -- 172.21.15.22:0/3801150067 >> 172.21.15.22:0/3801150067 conn(0x55a1a235d5b0 msgr2=0x55a1a2390cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 -- 172.21.15.22:0/3801150067 shutdown_connections 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 -- 172.21.15.22:0/3801150067 wait complete. 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.403+0000 7f65d6d65740 1 -- start start 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 1 Processor -- start 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a239c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2396d40 0x55a1a239ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a239cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a23a2280 con 0x55a1a239f550 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a22b6d90 con 0x55a1a2395fd0 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a23a2400 con 0x55a1a2396d40 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d6d65740 10 client.0 _ll_register_callbacks cb 0x55a1a200ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a239cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a239cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47338/0 (socket says 172.21.15.22:47338) 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 -- 172.21.15.22:0/42827861 learned_addr learned my addr 172.21.15.22:0/42827861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d163f640 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2396d40 0x55a1a239ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d1e40640 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a239c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 -- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2396d40 msgr2=0x55a1a239ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2396d40 0x55a1a239ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 -- 172.21.15.22:0/42827861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 msgr2=0x55a1a239c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a239c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.404+0000 7f65d2641640 1 -- 172.21.15.22:0/42827861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1a20db620 con 0x55a1a239f550 2024-03-08T23:56:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65d2641640 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a239cf90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f65c4003ea0 tx=0x7f65c401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d928c7f9a4222b82 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65b57fa640 1 -- 172.21.15.22:0/42827861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4004d20 con 0x55a1a239f550 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65bc005500 con 0x55a1a239f550 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65b57fa640 1 -- 172.21.15.22:0/42827861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f65c4020650 con 0x55a1a239f550 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65b57fa640 1 -- 172.21.15.22:0/42827861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c4030050 con 0x55a1a239f550 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65d6d65740 10 client.16212 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a1a23ae550 con 0x55a1a239f550 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65b57fa640 1 -- 172.21.15.22:0/42827861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65bc005500 con 0x55a1a239f550 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.405+0000 7f65b57fa640 1 -- 172.21.15.22:0/42827861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f65c4027070 con 0x55a1a239f550 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d6d65740 10 client.16212 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d6d65740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d6d65740 20 client.16212 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d6d65740 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a23afe20 0x55a1a23b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d6d65740 4 client.16212 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1a2307b30 con 0x55a1a23afe20 2024-03-08T23:56:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d1e40640 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a23afe20 0x55a1a23b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65b57fa640 10 client.16212 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.406+0000 7f65d1e40640 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a23afe20 0x55a1a23b2280 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31de73e6bf9d6c3a server_cookie=3fb47106c00005ab in_seq=0 out_seq=0 2024-03-08T23:56:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.407+0000 7f65b57fa640 1 -- 172.21.15.22:0/42827861 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a1a2307b30 con 0x55a1a23afe20 2024-03-08T23:56:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.407+0000 7f65b57fa640 10 client.16212 handle_command_reply: tid=0 2024-03-08T23:56:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.407+0000 7f65d6d65740 1 client.16212 shutdown 2024-03-08T23:56:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.407+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 msgr2=0x55a1a239cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.407+0000 7f65d6d65740 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a239cf90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f65c4003ea0 tx=0x7f65c401d000 comp rx=0 tx=0).stop 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.407+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 shutdown_connections 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.407+0000 7f65d6d65740 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a23afe20 0x55a1a23b2280 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65d6d65740 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a239cf90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65d6d65740 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2396d40 0x55a1a239ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65d6d65740 1 --2- 172.21.15.22:0/42827861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a239c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 >> 172.21.15.22:0/42827861 conn(0x55a1a235d5b0 msgr2=0x55a1a2392a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65b57fa640 0 client.16212 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 shutdown_connections 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65d6d65740 1 -- 172.21.15.22:0/42827861 wait complete. 2024-03-08T23:56:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.408+0000 7f65d6d65740 20 client.16212 trim_cache size 0 max 16384 2024-03-08T23:56:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.421+0000 7f65d6d65740 1 Processor -- start 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.421+0000 7f65d6d65740 1 -- start start 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.421+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a239ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.421+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2396d40 0x55a1a23b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.421+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a23a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.421+0000 7f65d6d65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1b2b40 con 0x55a1a239f550 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.421+0000 7f65d6d65740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1b06e0 con 0x55a1a2396d40 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d1e40640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a239ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d6d65740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1ae780 con 0x55a1a2395fd0 2024-03-08T23:56:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d1e40640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a239ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60110/0 (socket says 172.21.15.22:60110) 2024-03-08T23:56:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d1e40640 1 -- 172.21.15.22:0/330231189 learned_addr learned my addr 172.21.15.22:0/330231189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d163f640 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2396d40 0x55a1a23b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d2641640 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a23a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d1e40640 1 -- 172.21.15.22:0/330231189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2396d40 msgr2=0x55a1a23b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.422+0000 7f65d1e40640 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2396d40 0x55a1a23b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.423+0000 7f65d1e40640 1 -- 172.21.15.22:0/330231189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 msgr2=0x55a1a23a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.423+0000 7f65d1e40640 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a23a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.423+0000 7f65d1e40640 1 -- 172.21.15.22:0/330231189 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a2397440 con 0x55a1a2395fd0 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.423+0000 7f65d163f640 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2396d40 0x55a1a23b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.423+0000 7f65d1e40640 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a239ec40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65c8005940 tx=0x7f65c80dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d97244c0545dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.424+0000 7f65b77fe640 1 -- 172.21.15.22:0/330231189 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80d5020 con 0x55a1a2395fd0 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.424+0000 7f65b77fe640 1 -- 172.21.15.22:0/330231189 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65c8005690 con 0x55a1a2395fd0 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.424+0000 7f65b77fe640 1 -- 172.21.15.22:0/330231189 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c804b040 con 0x55a1a2395fd0 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.425+0000 7f65d6d65740 1 -- 172.21.15.22:0/330231189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 msgr2=0x55a1a239ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.425+0000 7f65d6d65740 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a239ec40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65c8005940 tx=0x7f65c80dd040 comp rx=0 tx=0).stop 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.425+0000 7f65d6d65740 1 -- 172.21.15.22:0/330231189 shutdown_connections 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.425+0000 7f65d6d65740 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a239f550 0x55a1a23a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.425+0000 7f65d6d65740 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2396d40 0x55a1a23b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.425+0000 7f65d6d65740 1 --2- 172.21.15.22:0/330231189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a2395fd0 0x55a1a239ec40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.425+0000 7f65d6d65740 1 -- 172.21.15.22:0/330231189 >> 172.21.15.22:0/330231189 conn(0x55a1a235d5b0 msgr2=0x55a1a2397910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.426+0000 7f65d6d65740 1 -- 172.21.15.22:0/330231189 shutdown_connections 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.426+0000 7f65d6d65740 1 -- 172.21.15.22:0/330231189 wait complete. 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.426+0000 7f65d6d65740 1 -- start start 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.426+0000 7f65d6d65740 1 Processor -- start 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.427+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a23a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.427+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2398ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.427+0000 7f65d6d65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a239f550 0x55a1a23993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.427+0000 7f65d1e40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a23a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.427+0000 7f65d1e40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a23a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36040/0 (socket says 172.21.15.22:36040) 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.427+0000 7f65d1e40640 1 -- 172.21.15.22:0/874327988 learned_addr learned my addr 172.21.15.22:0/874327988 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.427+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1aed80 con 0x55a1a2396d40 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d163f640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2398ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1b0080 con 0x55a1a2395fd0 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1af3c0 con 0x55a1a239f550 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d6d65740 10 client.0 _ll_register_callbacks cb 0x7f65bc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d2641640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a239f550 0x55a1a23993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d1e40640 1 -- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a239f550 msgr2=0x55a1a23993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d1e40640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a239f550 0x55a1a23993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d1e40640 1 -- 172.21.15.22:0/874327988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 msgr2=0x55a1a2398ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d1e40640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2398ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d1e40640 1 -- 172.21.15.22:0/874327988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1a23952a0 con 0x55a1a2395fd0 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d2641640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a239f550 0x55a1a23993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65d1e40640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a23a69b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65c80b9590 tx=0x7f65c802cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804df6c97bd48040 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.428+0000 7f65b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.429+0000 7f65b57fa640 1 -- 172.21.15.22:0/874327988 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80d5070 con 0x55a1a2395fd0 2024-03-08T23:56:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.429+0000 7f65b57fa640 1 -- 172.21.15.22:0/874327988 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f65c80bb430 con 0x55a1a2395fd0 2024-03-08T23:56:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.429+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1a23a1d50 con 0x55a1a2395fd0 2024-03-08T23:56:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.429+0000 7f65b57fa640 1 -- 172.21.15.22:0/874327988 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c804b040 con 0x55a1a2395fd0 2024-03-08T23:56:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.429+0000 7f65d6d65740 10 client.25718 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.429+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a1a2407140 con 0x55a1a2395fd0 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65b57fa640 1 -- 172.21.15.22:0/874327988 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a1a23a1d50 con 0x55a1a2395fd0 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65b57fa640 1 -- 172.21.15.22:0/874327988 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f65c80ee2e0 con 0x55a1a2395fd0 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65d6d65740 10 client.25718 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65d6d65740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65d6d65740 20 client.25718 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65d6d65740 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a2407580 0x55a1a2409970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65d6d65740 4 client.25718 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.430+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1a2301e80 con 0x55a1a2407580 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.431+0000 7f65d163f640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a2407580 0x55a1a2409970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.432+0000 7f65b57fa640 10 client.25718 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.432+0000 7f65d163f640 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a2407580 0x55a1a2409970 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649c512192b547a1 server_cookie=b8bfee889e8dd94d in_seq=0 out_seq=0 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.664+0000 7f65b57fa640 1 -- 172.21.15.22:0/874327988 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1a2301e80 con 0x55a1a2407580 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.664+0000 7f65b57fa640 10 client.25718 handle_command_reply: tid=0 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.664+0000 7f65d6d65740 1 client.25718 shutdown 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.664+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 msgr2=0x55a1a23a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.664+0000 7f65d6d65740 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a23a69b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65c80b9590 tx=0x7f65c802cdb0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 shutdown_connections 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1a2407580 0x55a1a2409970 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1a239f550 0x55a1a23993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1a2396d40 0x55a1a2398ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 --2- 172.21.15.22:0/874327988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1a2395fd0 0x55a1a23a69b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65b57fa640 0 client.25718 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 >> 172.21.15.22:0/874327988 conn(0x55a1a235d5b0 msgr2=0x55a1a23b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 shutdown_connections 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 1 -- 172.21.15.22:0/874327988 wait complete. 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d6d65740 20 client.25718 trim_cache size 0 max 16384 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:56:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:56:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65b80a2a90 msgr2=0x7f65b80a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65b80a2a90 0x7f65b80a4f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009930 tx=0x7f65c4009c00 comp rx=0 tx=0).stop 2024-03-08T23:56:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc1004d0 msgr2=0x7f65cc0fdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc1004d0 0x7f65cc0fdda0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006010 tx=0x7f65c8004270 comp rx=0 tx=0).stop 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 shutdown_connections 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65b80a2a90 0x7f65b80a4f50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65cc105230 0x7f65cc0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65cc100e70 0x7f65cc0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 --2- 172.21.15.22:0/3319150241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65cc1004d0 0x7f65cc0fdda0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.665+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 >> 172.21.15.22:0/3319150241 conn(0x7f65cc076fd0 msgr2=0x7f65cc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.666+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 shutdown_connections 2024-03-08T23:56:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:10.666+0000 7f65d40a4640 1 -- 172.21.15.22:0/3319150241 wait complete. 2024-03-08T23:56:10.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 46687 14.98 MB/sec warmup 18 sec latency 121.260 ms 2024-03-08T23:56:11.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 49916 15.18 MB/sec warmup 19 sec latency 7.553 ms 2024-03-08T23:56:12.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:11 smithi022 ceph-mon[31894]: pgmap v53: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-03-08T23:56:12.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:11 smithi062 ceph-mon[36129]: pgmap v53: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-03-08T23:56:12.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:11 smithi192 ceph-mon[35995]: pgmap v53: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-03-08T23:56:12.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 52605 15.06 MB/sec warmup 20 sec latency 6.552 ms 2024-03-08T23:56:13.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 54922 14.91 MB/sec warmup 21 sec latency 14.168 ms 2024-03-08T23:56:14.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:13 smithi022 ceph-mon[31894]: pgmap v54: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-03-08T23:56:14.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:13 smithi062 ceph-mon[36129]: pgmap v54: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-03-08T23:56:14.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:13 smithi192 ceph-mon[35995]: pgmap v54: 129 pgs: 129 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.4 MiB/s wr, 216 op/s 2024-03-08T23:56:14.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 57513 14.88 MB/sec warmup 22 sec latency 15.454 ms 2024-03-08T23:56:15.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 60595 15.03 MB/sec warmup 23 sec latency 18.862 ms 2024-03-08T23:56:16.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:15 smithi022 ceph-mon[31894]: pgmap v55: 129 pgs: 129 active+clean; 18 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-03-08T23:56:16.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:15 smithi062 ceph-mon[36129]: pgmap v55: 129 pgs: 129 active+clean; 18 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-03-08T23:56:16.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:15 smithi192 ceph-mon[35995]: pgmap v55: 129 pgs: 129 active+clean; 18 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-03-08T23:56:16.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 63502 14.97 MB/sec warmup 24 sec latency 6.668 ms 2024-03-08T23:56:17.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 66272 14.89 MB/sec warmup 25 sec latency 6.867 ms 2024-03-08T23:56:18.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:17 smithi022 ceph-mon[31894]: pgmap v56: 129 pgs: 129 active+clean; 18 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-03-08T23:56:18.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:17 smithi062 ceph-mon[36129]: pgmap v56: 129 pgs: 129 active+clean; 18 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-03-08T23:56:18.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:17 smithi192 ceph-mon[35995]: pgmap v56: 129 pgs: 129 active+clean; 18 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-03-08T23:56:18.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 68513 14.86 MB/sec warmup 26 sec latency 15.343 ms 2024-03-08T23:56:19.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 71500 14.94 MB/sec warmup 27 sec latency 18.982 ms 2024-03-08T23:56:20.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:19 smithi022 ceph-mon[31894]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-03-08T23:56:20.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:19 smithi062 ceph-mon[36129]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-03-08T23:56:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:19 smithi192 ceph-mon[35995]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-03-08T23:56:20.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 74755 15.08 MB/sec warmup 28 sec latency 6.657 ms 2024-03-08T23:56:21.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 77421 14.96 MB/sec warmup 29 sec latency 6.760 ms 2024-03-08T23:56:22.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:21 smithi022 ceph-mon[31894]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-08T23:56:22.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:21 smithi062 ceph-mon[36129]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-08T23:56:22.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:21 smithi192 ceph-mon[35995]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-08T23:56:22.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 79645 14.90 MB/sec warmup 30 sec latency 15.322 ms 2024-03-08T23:56:23.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 82189 14.87 MB/sec warmup 31 sec latency 68.358 ms 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.754+0000 7f623ec80640 1 -- 172.21.15.22:0/1270856827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238106600 msgr2=0x7f623810e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.754+0000 7f623ec80640 1 --2- 172.21.15.22:0/1270856827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238106600 0x7f623810e1d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6230009ca0 tx=0x7f623002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.754+0000 7f623ec80640 1 -- 172.21.15.22:0/1270856827 shutdown_connections 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.754+0000 7f623ec80640 1 --2- 172.21.15.22:0/1270856827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238106600 0x7f623810e1d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.754+0000 7f623ec80640 1 --2- 172.21.15.22:0/1270856827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238101ee0 0x7f6238105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.754+0000 7f623ec80640 1 --2- 172.21.15.22:0/1270856827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6238101530 0x7f6238101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.754+0000 7f623ec80640 1 -- 172.21.15.22:0/1270856827 >> 172.21.15.22:0/1270856827 conn(0x7f62380fcdd0 msgr2=0x7f62380ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 -- 172.21.15.22:0/1270856827 shutdown_connections 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 -- 172.21.15.22:0/1270856827 wait complete. 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 Processor -- start 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 -- start start 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238101530 0x7f623819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238106600 0x7f623819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62381978e0 0x7f6238197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381116c0 con 0x7f6238101530 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6238111540 con 0x7f6238106600 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ec80640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6238111840 con 0x7f62381978e0 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ca1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238101530 0x7f623819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ca1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238101530 0x7f623819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58866/0 (socket says 172.21.15.22:58866) 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623ca1c640 1 -- 172.21.15.22:0/3689123419 learned_addr learned my addr 172.21.15.22:0/3689123419 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f623d21d640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62381978e0 0x7f6238197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.755+0000 7f6237fff640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238106600 0x7f623819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ca1c640 1 -- 172.21.15.22:0/3689123419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62381978e0 msgr2=0x7f6238197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ca1c640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62381978e0 0x7f6238197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ca1c640 1 -- 172.21.15.22:0/3689123419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238106600 msgr2=0x7f623819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ca1c640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238106600 0x7f623819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ca1c640 1 -- 172.21.15.22:0/3689123419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6238198400 con 0x7f6238101530 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ca1c640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238101530 0x7f623819d730 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f622c00e9f0 tx=0x7f622c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f6235ffb640 1 -- 172.21.15.22:0/3689123419 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c00cd80 con 0x7f6238101530 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f6235ffb640 1 -- 172.21.15.22:0/3689123419 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622c005500 con 0x7f6238101530 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62381986f0 con 0x7f6238101530 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f6235ffb640 1 -- 172.21.15.22:0/3689123419 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c010690 con 0x7f6238101530 2024-03-08T23:56:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.756+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62381a48f0 con 0x7f6238101530 2024-03-08T23:56:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.758+0000 7f6235ffb640 1 -- 172.21.15.22:0/3689123419 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f622c0046c0 con 0x7f6238101530 2024-03-08T23:56:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 Processor -- start 2024-03-08T23:56:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 -- start start 2024-03-08T23:56:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f6235ffb640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f62280a2a10 0x7f62280a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe080a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe081450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe08a550 0x559dfe08c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfdfe55f0 con 0x559dfe080fd0 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfdf8e930 con 0x559dfe081d40 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f624185c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfdfe1690 con 0x559dfe08a550 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f6235ffb640 1 -- 172.21.15.22:0/3689123419 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f622c0cf660 con 0x7f6238101530 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f623d21d640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f62280a2a10 0x7f62280a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f6237fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe081450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f6237fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe081450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58870/0 (socket says 172.21.15.22:58870) 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f6237fff640 1 -- 172.21.15.22:0/4111289605 learned_addr learned my addr 172.21.15.22:0/4111289605 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f623ca1c640 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe080a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.759+0000 7f623d21d640 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe08a550 0x559dfe08c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f623ca1c640 1 -- 172.21.15.22:0/4111289605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe08a550 msgr2=0x559dfe08c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f623ca1c640 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe08a550 0x559dfe08c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f623ca1c640 1 -- 172.21.15.22:0/4111289605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 msgr2=0x559dfe081450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f623ca1c640 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe081450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f623ca1c640 1 -- 172.21.15.22:0/4111289605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dfdcfa210 con 0x559dfe081d40 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f6237fff640 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe081450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f623d21d640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f62280a2a10 0x7f62280a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6230005e30 tx=0x7f623002ced0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f623ca1c640 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe080a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f622c0cc9b0 tx=0x7f622c09b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a45afb3210eb586 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f6222ffd640 1 -- 172.21.15.22:0/4111289605 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c09bfc0 con 0x559dfe081d40 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f6222ffd640 1 -- 172.21.15.22:0/4111289605 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622c09b8c0 con 0x559dfe081d40 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.760+0000 7f6222ffd640 1 -- 172.21.15.22:0/4111289605 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0ccc30 con 0x559dfe081d40 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 -- 172.21.15.22:0/4111289605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 msgr2=0x559dfe080a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe080a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f622c0cc9b0 tx=0x7f622c09b4c0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 -- 172.21.15.22:0/4111289605 shutdown_connections 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe08a550 0x559dfe08c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe081450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 --2- 172.21.15.22:0/4111289605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe080a90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 -- 172.21.15.22:0/4111289605 >> 172.21.15.22:0/4111289605 conn(0x559dfe0485b0 msgr2=0x559dfe07bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 -- 172.21.15.22:0/4111289605 shutdown_connections 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 -- 172.21.15.22:0/4111289605 wait complete. 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 -- start start 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 Processor -- start 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe080fd0 0x559dfe087610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe087b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.761+0000 7f624185c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe088090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f624185c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfe08d220 con 0x559dfe08a550 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f624185c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfdfa1d90 con 0x559dfe080fd0 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f624185c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dfe08d3a0 con 0x559dfe081d40 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f624185c740 10 client.0 _ll_register_callbacks cb 0x559dfdcf9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623d21d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe088090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623ca1c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe080fd0 0x559dfe087610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623d21d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe088090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58878/0 (socket says 172.21.15.22:58878) 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f6237fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe087b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623d21d640 1 -- 172.21.15.22:0/985516124 learned_addr learned my addr 172.21.15.22:0/985516124 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623ca1c640 1 -- 172.21.15.22:0/985516124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 msgr2=0x559dfe087b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623ca1c640 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe087b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623ca1c640 1 -- 172.21.15.22:0/985516124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 msgr2=0x559dfe088090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623ca1c640 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe088090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623ca1c640 1 -- 172.21.15.22:0/985516124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dfddc6620 con 0x559dfe080fd0 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623d21d640 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe088090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f623ca1c640 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe080fd0 0x559dfe087610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x559dfe08ced0 tx=0x7f622c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9892a009a249c8f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f6217fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f6217fff640 1 -- 172.21.15.22:0/985516124 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0b3020 con 0x559dfe080fd0 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f6217fff640 1 -- 172.21.15.22:0/985516124 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f622c0cf240 con 0x559dfe080fd0 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f622c09b8c0 con 0x559dfe080fd0 2024-03-08T23:56:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f6217fff640 1 -- 172.21.15.22:0/985516124 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f622c0cce40 con 0x559dfe080fd0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f624185c740 10 client.25727 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.762+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559dfe099590 con 0x559dfe080fd0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f6217fff640 1 -- 172.21.15.22:0/985516124 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622c09b8c0 con 0x559dfe080fd0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f6217fff640 1 -- 172.21.15.22:0/985516124 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f622c0a3bd0 con 0x559dfe080fd0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f624185c740 10 client.25727 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f624185c740 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f624185c740 20 client.25727 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f624185c740 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe09a8e0 0x559dfe09ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f624185c740 4 client.25727 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dfdff2b30 con 0x559dfe09a8e0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f6237fff640 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe09a8e0 0x559dfe09ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f6237fff640 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe09a8e0 0x559dfe09ccd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d83bbcc3150cc5 server_cookie=d7b581cbbbe656b9 in_seq=0 out_seq=0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.763+0000 7f6217fff640 10 client.25727 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.765+0000 7f6217fff640 1 -- 172.21.15.22:0/985516124 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559dfdff2b30 con 0x559dfe09a8e0 2024-03-08T23:56:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.765+0000 7f6217fff640 10 client.25727 handle_command_reply: tid=0 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.765+0000 7f624185c740 1 client.25727 shutdown 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe080fd0 msgr2=0x559dfe087610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe080fd0 0x559dfe087610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x559dfe08ced0 tx=0x7f622c004a40 comp rx=0 tx=0).stop 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 shutdown_connections 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe09a8e0 0x559dfe09ccd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe088090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe087b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 --2- 172.21.15.22:0/985516124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe080fd0 0x559dfe087610 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f6217fff640 0 client.25727 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 >> 172.21.15.22:0/985516124 conn(0x559dfe0485b0 msgr2=0x559dfe07da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 shutdown_connections 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 1 -- 172.21.15.22:0/985516124 wait complete. 2024-03-08T23:56:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.766+0000 7f624185c740 20 client.25727 trim_cache size 0 max 16384 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 Processor -- start 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 -- start start 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe089d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe091610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe08a550 0x559dfe091b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381b08f0 con 0x559dfe080fd0 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381ae490 con 0x559dfe08a550 2024-03-08T23:56:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f624185c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381ac530 con 0x559dfe081d40 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe089d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe089d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58888/0 (socket says 172.21.15.22:58888) 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 -- 172.21.15.22:0/310939776 learned_addr learned my addr 172.21.15.22:0/310939776 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623d21d640 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe08a550 0x559dfe091b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f6237fff640 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe091610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 -- 172.21.15.22:0/310939776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 msgr2=0x559dfe091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe091610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 -- 172.21.15.22:0/310939776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe08a550 msgr2=0x559dfe091b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe08a550 0x559dfe091b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 -- 172.21.15.22:0/310939776 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dfe0802a0 con 0x559dfe080fd0 2024-03-08T23:56:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f6237fff640 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe091610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f623ca1c640 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe089d40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f622c0049d0 tx=0x7f622c099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a91c8336797eef2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.780+0000 7f62227fc640 1 -- 172.21.15.22:0/310939776 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0ab070 con 0x559dfe080fd0 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f62227fc640 1 -- 172.21.15.22:0/310939776 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622c0ba190 con 0x559dfe080fd0 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f62227fc640 1 -- 172.21.15.22:0/310939776 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0b3070 con 0x559dfe080fd0 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 -- 172.21.15.22:0/310939776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 msgr2=0x559dfe089d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe089d40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f622c0049d0 tx=0x7f622c099dd0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 -- 172.21.15.22:0/310939776 shutdown_connections 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe08a550 0x559dfe091b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe081d40 0x559dfe091610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 --2- 172.21.15.22:0/310939776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe080fd0 0x559dfe089d40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 -- 172.21.15.22:0/310939776 >> 172.21.15.22:0/310939776 conn(0x559dfe0485b0 msgr2=0x559dfe07b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 -- 172.21.15.22:0/310939776 shutdown_connections 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 -- 172.21.15.22:0/310939776 wait complete. 2024-03-08T23:56:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.781+0000 7f624185c740 1 -- start start 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 1 Processor -- start 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe080fd0 0x559dfe091620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe083aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe083fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381acb30 con 0x559dfe08a550 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381ade30 con 0x559dfe081d40 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381ad170 con 0x559dfe080fd0 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f624185c740 10 client.0 _ll_register_callbacks cb 0x7f622c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623d21d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe083fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623ca1c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe080fd0 0x559dfe091620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623ca1c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe080fd0 0x559dfe091620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:55218/0 (socket says 172.21.15.22:55218) 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623d21d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe083fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58894/0 (socket says 172.21.15.22:58894) 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623ca1c640 1 -- 172.21.15.22:0/312395720 learned_addr learned my addr 172.21.15.22:0/312395720 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f6237fff640 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe083aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623d21d640 1 -- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe080fd0 msgr2=0x559dfe091620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623d21d640 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe080fd0 0x559dfe091620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623d21d640 1 -- 172.21.15.22:0/312395720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 msgr2=0x559dfe083aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623d21d640 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe083aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623d21d640 1 -- 172.21.15.22:0/312395720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dfe091e30 con 0x559dfe08a550 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.782+0000 7f623ca1c640 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe080fd0 0x559dfe091620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f623d21d640 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe083fe0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6230004af0 tx=0x7f623002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c965b987c55b56bd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f6217fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f6217fff640 1 -- 172.21.15.22:0/312395720 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6230039040 con 0x559dfe08a550 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f6217fff640 1 -- 172.21.15.22:0/312395720 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6230004510 con 0x559dfe08a550 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f622c0ba190 con 0x559dfe08a550 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f6217fff640 1 -- 172.21.15.22:0/312395720 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f623005c050 con 0x559dfe08a550 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 10 client.16236 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559dfe0f15d0 con 0x559dfe08a550 2024-03-08T23:56:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f6217fff640 1 -- 172.21.15.22:0/312395720 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622c0ba190 con 0x559dfe08a550 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f6217fff640 1 -- 172.21.15.22:0/312395720 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f623004e420 con 0x559dfe08a550 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 10 client.16236 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 20 client.16236 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe0f1d20 0x559dfe0f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 4 client.16236 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.783+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559dfdfece80 con 0x559dfe0f1d20 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.784+0000 7f623ca1c640 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe0f1d20 0x559dfe0f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.784+0000 7f623ca1c640 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe0f1d20 0x559dfe0f4140 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952e26b93d2a82b0 server_cookie=9b78204b69426357 in_seq=0 out_seq=0 2024-03-08T23:56:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:23.784+0000 7f6217fff640 10 client.16236 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.019+0000 7f6217fff640 1 -- 172.21.15.22:0/312395720 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559dfdfece80 con 0x559dfe0f1d20 2024-03-08T23:56:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.019+0000 7f6217fff640 10 client.16236 handle_command_reply: tid=0 2024-03-08T23:56:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.019+0000 7f624185c740 1 client.16236 shutdown 2024-03-08T23:56:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.019+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 msgr2=0x559dfe083fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.019+0000 7f624185c740 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe083fe0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6230004af0 tx=0x7f623002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 shutdown_connections 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559dfe0f1d20 0x559dfe0f4140 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f6217fff640 0 client.16236 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dfe08a550 0x559dfe083fe0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559dfe081d40 0x559dfe083aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 --2- 172.21.15.22:0/312395720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559dfe080fd0 0x559dfe091620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 >> 172.21.15.22:0/312395720 conn(0x559dfe0485b0 msgr2=0x559dfe085ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 shutdown_connections 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 1 -- 172.21.15.22:0/312395720 wait complete. 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.020+0000 7f624185c740 20 client.16236 trim_cache size 0 max 16384 2024-03-08T23:56:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:56:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:56:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:56:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f62280a2a10 msgr2=0x7f62280a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f62280a2a10 0x7f62280a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6230005e30 tx=0x7f623002ced0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238101530 msgr2=0x7f623819d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238101530 0x7f623819d730 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f622c00e9f0 tx=0x7f622c00edf0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 shutdown_connections 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f62280a2a10 0x7f62280a4ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62381978e0 0x7f6238197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238106600 0x7f623819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 --2- 172.21.15.22:0/3689123419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6238101530 0x7f623819d730 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 >> 172.21.15.22:0/3689123419 conn(0x7f62380fcdd0 msgr2=0x7f62380ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 shutdown_connections 2024-03-08T23:56:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:24.021+0000 7f623ec80640 1 -- 172.21.15.22:0/3689123419 wait complete. 2024-03-08T23:56:24.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:23 smithi022 ceph-mon[31894]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-08T23:56:24.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:23 smithi062 ceph-mon[36129]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-08T23:56:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:23 smithi192 ceph-mon[35995]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.7 MiB/s wr, 314 op/s 2024-03-08T23:56:24.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 84408 14.74 MB/sec warmup 32 sec latency 124.963 ms 2024-03-08T23:56:25.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 86601 14.70 MB/sec warmup 33 sec latency 8.633 ms 2024-03-08T23:56:26.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:25 smithi022 ceph-mon[31894]: pgmap v60: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 478 op/s 2024-03-08T23:56:26.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:25 smithi062 ceph-mon[36129]: pgmap v60: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 478 op/s 2024-03-08T23:56:26.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:25 smithi192 ceph-mon[35995]: pgmap v60: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 478 op/s 2024-03-08T23:56:26.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 89283 14.68 MB/sec warmup 34 sec latency 16.435 ms 2024-03-08T23:56:27.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 92376 14.77 MB/sec warmup 35 sec latency 6.907 ms 2024-03-08T23:56:28.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:27 smithi022 ceph-mon[31894]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-03-08T23:56:28.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:27 smithi062 ceph-mon[36129]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-03-08T23:56:28.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:27 smithi192 ceph-mon[35995]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-03-08T23:56:28.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 94163 14.54 MB/sec warmup 36 sec latency 57.638 ms 2024-03-08T23:56:29.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 96321 14.52 MB/sec warmup 37 sec latency 43.426 ms 2024-03-08T23:56:30.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:29 smithi022 ceph-mon[31894]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-08T23:56:30.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:29 smithi062 ceph-mon[36129]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-08T23:56:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:29 smithi192 ceph-mon[35995]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-08T23:56:30.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 99358 14.60 MB/sec warmup 38 sec latency 9.138 ms 2024-03-08T23:56:31.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 101707 14.45 MB/sec warmup 39 sec latency 12.639 ms 2024-03-08T23:56:32.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:31 smithi022 ceph-mon[31894]: pgmap v63: 129 pgs: 129 active+clean; 58 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-03-08T23:56:32.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:56:32.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:56:32.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:31 smithi062 ceph-mon[36129]: pgmap v63: 129 pgs: 129 active+clean; 58 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-03-08T23:56:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:31 smithi192 ceph-mon[35995]: pgmap v63: 129 pgs: 129 active+clean; 58 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 -- 172.21.15.22:0/2120162443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101003e0 msgr2=0x7f29101007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 --2- 172.21.15.22:0/2120162443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101003e0 0x7f29101007c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2900009900 tx=0x7f290002bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 -- 172.21.15.22:0/2120162443 shutdown_connections 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 --2- 172.21.15.22:0/2120162443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29101054b0 0x7f291010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 --2- 172.21.15.22:0/2120162443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2910100d90 0x7f2910104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 --2- 172.21.15.22:0/2120162443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101003e0 0x7f29101007c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 -- 172.21.15.22:0/2120162443 >> 172.21.15.22:0/2120162443 conn(0x7f2910076570 msgr2=0x7f2910076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 -- 172.21.15.22:0/2120162443 shutdown_connections 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.445+0000 7f29154cd640 1 -- 172.21.15.22:0/2120162443 wait complete. 2024-03-08T23:56:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 Processor -- start 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 -- start start 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29101003e0 0x7f291019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101054b0 0x7f291019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29101968d0 0x7f2910196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2910110570 con 0x7f29101054b0 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101103f0 con 0x7f29101003e0 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f29154cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101106f0 con 0x7f29101968d0 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101054b0 0x7f291019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29101968d0 0x7f2910196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101054b0 0x7f291019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42084/0 (socket says 172.21.15.22:42084) 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 -- 172.21.15.22:0/3301037188 learned_addr learned my addr 172.21.15.22:0/3301037188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f290effd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29101003e0 0x7f291019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 -- 172.21.15.22:0/3301037188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29101968d0 msgr2=0x7f2910196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29101968d0 0x7f2910196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 -- 172.21.15.22:0/3301037188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29101003e0 msgr2=0x7f291019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29101003e0 0x7f291019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 -- 172.21.15.22:0/3301037188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29101973f0 con 0x7f29101054b0 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.446+0000 7f2907fff640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101054b0 0x7f291019cc60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9c0 tx=0x7f28f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.447+0000 7f290cff9640 1 -- 172.21.15.22:0/3301037188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800cd80 con 0x7f29101054b0 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.447+0000 7f290cff9640 1 -- 172.21.15.22:0/3301037188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f8005500 con 0x7f29101054b0 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.447+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29101976e0 con 0x7f29101054b0 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.447+0000 7f290cff9640 1 -- 172.21.15.22:0/3301037188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8010690 con 0x7f29101054b0 2024-03-08T23:56:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.447+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2910069ca0 con 0x7f29101054b0 2024-03-08T23:56:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.449+0000 7f290cff9640 1 -- 172.21.15.22:0/3301037188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f28f80046c0 con 0x7f29101054b0 2024-03-08T23:56:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.449+0000 7f290cff9640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28fc0a29f0 0x7f28fc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.449+0000 7f2918181740 1 Processor -- start 2024-03-08T23:56:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.449+0000 7f2918181740 1 -- start start 2024-03-08T23:56:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.449+0000 7f290effd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28fc0a29f0 0x7f28fc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.449+0000 7f290cff9640 1 -- 172.21.15.22:0/3301037188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f28f80cf5a0 con 0x7f29101054b0 2024-03-08T23:56:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2918181740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d3f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2918181740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d402720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2918181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d405050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2918181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d35ce00 con 0x55a78d402c60 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2918181740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d305b10 con 0x55a78d3fa080 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2918181740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d358ea0 con 0x55a78d3f9760 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d405050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2907fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d402720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d405050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42096/0 (socket says 172.21.15.22:42096) 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2907fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d402720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50518/0 (socket says 172.21.15.22:50518) 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 -- 172.21.15.22:0/2587719370 learned_addr learned my addr 172.21.15.22:0/2587719370 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290effd640 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d3f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290effd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28fc0a29f0 0x7f28fc0a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2900006010 tx=0x7f290002c670 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 -- 172.21.15.22:0/2587719370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 msgr2=0x55a78d3f9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d3f9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 -- 172.21.15.22:0/2587719370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 msgr2=0x55a78d402720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d402720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 -- 172.21.15.22:0/2587719370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78d13b7d0 con 0x55a78d402c60 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f2907fff640 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d402720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f290f7fe640 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d405050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2908009f50 tx=0x7f29080118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a7766814012294 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.450+0000 7f29057fa640 1 -- 172.21.15.22:0/2587719370 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908004d20 con 0x55a78d402c60 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f29057fa640 1 -- 172.21.15.22:0/2587719370 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2908004050 con 0x55a78d402c60 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f29057fa640 1 -- 172.21.15.22:0/2587719370 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908005280 con 0x55a78d402c60 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 -- 172.21.15.22:0/2587719370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 msgr2=0x55a78d405050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d405050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2908009f50 tx=0x7f29080118d0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 -- 172.21.15.22:0/2587719370 shutdown_connections 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d405050 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d402720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 --2- 172.21.15.22:0/2587719370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d3f9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 -- 172.21.15.22:0/2587719370 >> 172.21.15.22:0/2587719370 conn(0x55a78d3f3e90 msgr2=0x55a78d3f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 -- 172.21.15.22:0/2587719370 shutdown_connections 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.451+0000 7f2918181740 1 -- 172.21.15.22:0/2587719370 wait complete. 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 -- start start 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 Processor -- start 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d402370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d3ff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d3ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d405a50 con 0x55a78d402c60 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d3190a0 con 0x55a78d3fa080 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d405bd0 con 0x55a78d3f9760 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2918181740 10 client.0 _ll_register_callbacks cb 0x55a78d318a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d3ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d3ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42110/0 (socket says 172.21.15.22:42110) 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 -- 172.21.15.22:0/64298127 learned_addr learned my addr 172.21.15.22:0/64298127 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290effd640 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d402370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f2907fff640 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d3ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 -- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 msgr2=0x55a78d402370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d402370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 -- 172.21.15.22:0/64298127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 msgr2=0x55a78d3ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d3ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.452+0000 7f290f7fe640 1 -- 172.21.15.22:0/64298127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a78d0713f0 con 0x55a78d402c60 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f290f7fe640 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d3ffc40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2908009730 tx=0x7f2908004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df63a18c8e848ac server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f28f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f28f67fc640 1 -- 172.21.15.22:0/64298127 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908029070 con 0x55a78d402c60 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2908004050 con 0x55a78d402c60 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f28f67fc640 1 -- 172.21.15.22:0/64298127 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f290800f480 con 0x55a78d402c60 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f28f67fc640 1 -- 172.21.15.22:0/64298127 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2908034050 con 0x55a78d402c60 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 10 client.16248 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a78d0310f0 con 0x55a78d402c60 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f28f67fc640 1 -- 172.21.15.22:0/64298127 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2908004050 con 0x55a78d402c60 2024-03-08T23:56:32.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f28f67fc640 1 -- 172.21.15.22:0/64298127 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2908019430 con 0x55a78d402c60 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 10 client.16248 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 20 client.16248 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d4125f0 0x55a78d4149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 4 client.16248 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a78d36a340 con 0x55a78d4125f0 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.453+0000 7f290effd640 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d4125f0 0x55a78d4149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.454+0000 7f290effd640 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d4125f0 0x55a78d4149e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7603cec41016b27 server_cookie=ac7aa6dd1eda10fe in_seq=0 out_seq=0 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.454+0000 7f28f67fc640 10 client.16248 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f28f67fc640 1 -- 172.21.15.22:0/64298127 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a78d36a340 con 0x55a78d4125f0 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f28f67fc640 10 client.16248 handle_command_reply: tid=0 2024-03-08T23:56:32.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 client.16248 shutdown 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 msgr2=0x55a78d3ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d3ffc40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2908009730 tx=0x7f2908004e50 comp rx=0 tx=0).stop 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 shutdown_connections 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d4125f0 0x55a78d4149e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d402c60 0x55a78d3ffc40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f28f67fc640 0 client.16248 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d3ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 --2- 172.21.15.22:0/64298127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3f9760 0x55a78d402370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 >> 172.21.15.22:0/64298127 conn(0x55a78d3f3e90 msgr2=0x55a78d3f5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 shutdown_connections 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 1 -- 172.21.15.22:0/64298127 wait complete. 2024-03-08T23:56:32.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.455+0000 7f2918181740 20 client.16248 trim_cache size 0 max 16384 2024-03-08T23:56:32.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 Processor -- start 2024-03-08T23:56:32.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 -- start start 2024-03-08T23:56:32.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d40a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d401e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d402c60 0x55a78d402370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101b0b70 con 0x55a78d3f9760 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101ae710 con 0x55a78d3fa080 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.471+0000 7f2918181740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101ac7b0 con 0x55a78d402c60 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d40a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f2907fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d401e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f2907fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d401e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50532/0 (socket says 172.21.15.22:50532) 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d402c60 0x55a78d402370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d402c60 0x55a78d402370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:34676/0 (socket says 172.21.15.22:34676) 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290f7fe640 1 -- 172.21.15.22:0/611224835 learned_addr learned my addr 172.21.15.22:0/611224835 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290effd640 1 -- 172.21.15.22:0/611224835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d402c60 msgr2=0x55a78d402370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290effd640 1 --2- 172.21.15.22:0/611224835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d402c60 0x55a78d402370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290effd640 1 -- 172.21.15.22:0/611224835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 msgr2=0x55a78d401e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290effd640 1 --2- 172.21.15.22:0/611224835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d401e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.472+0000 7f290effd640 1 -- 172.21.15.22:0/611224835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78d3f4810 con 0x55a78d3f9760 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.473+0000 7f290effd640 1 --2- 172.21.15.22:0/611224835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d40a1f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2910197a10 tx=0x7f2900042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ade4827832e75b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.473+0000 7f2904ff9640 1 -- 172.21.15.22:0/611224835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900051d00 con 0x55a78d3f9760 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.473+0000 7f2904ff9640 1 -- 172.21.15.22:0/611224835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2900005040 con 0x55a78d3f9760 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.473+0000 7f2904ff9640 1 -- 172.21.15.22:0/611224835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000515c0 con 0x55a78d3f9760 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- 172.21.15.22:0/611224835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 msgr2=0x55a78d40a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 --2- 172.21.15.22:0/611224835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d40a1f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2910197a10 tx=0x7f2900042000 comp rx=0 tx=0).stop 2024-03-08T23:56:32.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- 172.21.15.22:0/611224835 shutdown_connections 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 --2- 172.21.15.22:0/611224835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d402c60 0x55a78d402370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 --2- 172.21.15.22:0/611224835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d3fa080 0x55a78d401e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 --2- 172.21.15.22:0/611224835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d40a1f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- 172.21.15.22:0/611224835 >> 172.21.15.22:0/611224835 conn(0x55a78d3f3e90 msgr2=0x55a78d3fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- 172.21.15.22:0/611224835 shutdown_connections 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- 172.21.15.22:0/611224835 wait complete. 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- start start 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 Processor -- start 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d3fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3fa080 0x55a78d3fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d402c60 0x55a78d3fc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101acdb0 con 0x55a78d3f9760 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101ae0b0 con 0x55a78d402c60 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101ad3f0 con 0x55a78d3fa080 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f2918181740 10 client.0 _ll_register_callbacks cb 0x7f28f80056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.474+0000 7f290effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d3fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d3fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42132/0 (socket says 172.21.15.22:42132) 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 -- 172.21.15.22:0/1379122075 learned_addr learned my addr 172.21.15.22:0/1379122075 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f2907fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3fa080 0x55a78d3fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290f7fe640 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d402c60 0x55a78d3fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 -- 172.21.15.22:0/1379122075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3fa080 msgr2=0x55a78d3fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3fa080 0x55a78d3fbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 -- 172.21.15.22:0/1379122075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d402c60 msgr2=0x55a78d3fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d402c60 0x55a78d3fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 -- 172.21.15.22:0/1379122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a78d46c8a0 con 0x55a78d3f9760 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290f7fe640 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d402c60 0x55a78d3fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f290effd640 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d3fb990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f290002c980 tx=0x7f29000051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fa34ee636ab73b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f28f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f28f67fc640 1 -- 172.21.15.22:0/1379122075 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900055070 con 0x55a78d3f9760 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f28f67fc640 1 -- 172.21.15.22:0/1379122075 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f29000048c0 con 0x55a78d3f9760 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2900005040 con 0x55a78d3f9760 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f28f67fc640 1 -- 172.21.15.22:0/1379122075 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2900065050 con 0x55a78d3f9760 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f2918181740 10 client.16254 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.475+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a78d46db90 con 0x55a78d3f9760 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f28f67fc640 1 -- 172.21.15.22:0/1379122075 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2900005040 con 0x55a78d3f9760 2024-03-08T23:56:32.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f28f67fc640 1 -- 172.21.15.22:0/1379122075 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f290005ccd0 con 0x55a78d3f9760 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2918181740 10 client.16254 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2918181740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2918181740 20 client.16254 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2918181740 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d46e270 0x55a78d470690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2918181740 4 client.16254 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a78d364690 con 0x55a78d46e270 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2907fff640 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d46e270 0x55a78d470690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f2907fff640 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d46e270 0x55a78d470690 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d0de19640750186 server_cookie=49411e23ff407425 in_seq=0 out_seq=0 2024-03-08T23:56:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.476+0000 7f28f67fc640 10 client.16254 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:32.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.478+0000 7f28f67fc640 1 -- 172.21.15.22:0/1379122075 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a78d364690 con 0x55a78d46e270 2024-03-08T23:56:32.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.478+0000 7f28f67fc640 10 client.16254 handle_command_reply: tid=0 2024-03-08T23:56:32.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 client.16254 shutdown 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 msgr2=0x55a78d3fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d3fb990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f290002c980 tx=0x7f29000051f0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 shutdown_connections 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a78d46e270 0x55a78d470690 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a78d402c60 0x55a78d3fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a78d3fa080 0x55a78d3fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 --2- 172.21.15.22:0/1379122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a78d3f9760 0x55a78d3fb990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 >> 172.21.15.22:0/1379122075 conn(0x55a78d3f3e90 msgr2=0x55a78d3fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f28f67fc640 0 client.16254 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 shutdown_connections 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 1 -- 172.21.15.22:0/1379122075 wait complete. 2024-03-08T23:56:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.479+0000 7f2918181740 20 client.16254 trim_cache size 0 max 16384 2024-03-08T23:56:32.482 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:32.482 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:56:32.482 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:56:32.482 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28fc0a29f0 msgr2=0x7f28fc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28fc0a29f0 0x7f28fc0a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2900006010 tx=0x7f290002c670 comp rx=0 tx=0).stop 2024-03-08T23:56:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101054b0 msgr2=0x7f291019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101054b0 0x7f291019cc60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9c0 tx=0x7f28f800ee80 comp rx=0 tx=0).stop 2024-03-08T23:56:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 shutdown_connections 2024-03-08T23:56:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28fc0a29f0 0x7f28fc0a4eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29101968d0 0x7f2910196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29101054b0 0x7f291019cc60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 --2- 172.21.15.22:0/3301037188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29101003e0 0x7f291019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.480+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 >> 172.21.15.22:0/3301037188 conn(0x7f2910076570 msgr2=0x7f29100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.481+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 shutdown_connections 2024-03-08T23:56:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.482+0000 7f29154cd640 1 -- 172.21.15.22:0/3301037188 wait complete. 2024-03-08T23:56:32.495 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:56:32.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:56:32.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.605+0000 7f14926e9640 1 -- 172.21.15.22:0/3452962863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c105530 msgr2=0x7f148c10d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.605+0000 7f14926e9640 1 --2- 172.21.15.22:0/3452962863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c105530 0x7f148c10d100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1484009ca0 tx=0x7f148402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 -- 172.21.15.22:0/3452962863 shutdown_connections 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 --2- 172.21.15.22:0/3452962863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c105530 0x7f148c10d100 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 --2- 172.21.15.22:0/3452962863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f148c100e10 0x7f148c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 --2- 172.21.15.22:0/3452962863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c100460 0x7f148c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 -- 172.21.15.22:0/3452962863 >> 172.21.15.22:0/3452962863 conn(0x7f148c0772f0 msgr2=0x7f148c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 -- 172.21.15.22:0/3452962863 shutdown_connections 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 -- 172.21.15.22:0/3452962863 wait complete. 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 Processor -- start 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 -- start start 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.606+0000 7f14926e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c100460 0x7f148c10d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14926e9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f148c105530 0x7f148c10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14926e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c10e310 0x7f148c1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14926e9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1116f0 con 0x7f148c100460 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14926e9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c111570 con 0x7f148c10e310 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14926e9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c111870 con 0x7f148c105530 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c100460 0x7f148c10d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c100460 0x7f148c10d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42156/0 (socket says 172.21.15.22:42156) 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 -- 172.21.15.22:0/672035322 learned_addr learned my addr 172.21.15.22:0/672035322 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f1490c86640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c10e310 0x7f148c1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148b7fe640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f148c105530 0x7f148c10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 -- 172.21.15.22:0/672035322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f148c105530 msgr2=0x7f148c10ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f148c105530 0x7f148c10ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 -- 172.21.15.22:0/672035322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c10e310 msgr2=0x7f148c1a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c10e310 0x7f148c1a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 -- 172.21.15.22:0/672035322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c1a6480 con 0x7f148c100460 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f148bfff640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c100460 0x7f148c10d890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f147c00d950 tx=0x7f147c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14897fa640 1 -- 172.21.15.22:0/672035322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c017070 con 0x7f148c100460 2024-03-08T23:56:32.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14897fa640 1 -- 172.21.15.22:0/672035322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f147c009c30 con 0x7f148c100460 2024-03-08T23:56:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c1a6770 con 0x7f148c100460 2024-03-08T23:56:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14897fa640 1 -- 172.21.15.22:0/672035322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c00f6b0 con 0x7f148c100460 2024-03-08T23:56:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.607+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c1a6cc0 con 0x7f148c100460 2024-03-08T23:56:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.609+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c0680e0 con 0x7f148c100460 2024-03-08T23:56:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.610+0000 7f14897fa640 1 -- 172.21.15.22:0/672035322 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f147c0109d0 con 0x7f148c100460 2024-03-08T23:56:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.610+0000 7f14897fa640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14740a29a0 0x7f14740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.610+0000 7f148b7fe640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14740a29a0 0x7f14740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.610+0000 7f14897fa640 1 -- 172.21.15.22:0/672035322 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f147c0ceda0 con 0x7f148c100460 2024-03-08T23:56:32.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.611+0000 7f148b7fe640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14740a29a0 0x7f14740a4e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f148c10f320 tx=0x7f1480005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.615+0000 7f14897fa640 1 -- 172.21.15.22:0/672035322 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f147c09b110 con 0x7f148c100460 2024-03-08T23:56:32.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 103717 14.40 MB/sec warmup 40 sec latency 13.928 ms 2024-03-08T23:56:32.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.854+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f148c1087c0 con 0x7f148c100460 2024-03-08T23:56:32.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.855+0000 7f14897fa640 1 -- 172.21.15.22:0/672035322 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f147c09f1b0 con 0x7f148c100460 2024-03-08T23:56:32.851 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:56:32.851 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:55:16.974110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25439":{"gid":25439,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1375407074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1375407074},{"type":"v1","addr":"172.21.15.62:6837","nonce":1375407074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:56:32.851 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14740a29a0 msgr2=0x7f14740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14740a29a0 0x7f14740a4e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f148c10f320 tx=0x7f1480005f90 comp rx=0 tx=0).stop 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c100460 msgr2=0x7f148c10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c100460 0x7f148c10d890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f147c00d950 tx=0x7f147c00de10 comp rx=0 tx=0).stop 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 shutdown_connections 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14740a29a0 0x7f14740a4e60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c10e310 0x7f148c1a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f148c105530 0x7f148c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 --2- 172.21.15.22:0/672035322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c100460 0x7f148c10d890 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 >> 172.21.15.22:0/672035322 conn(0x7f148c0772f0 msgr2=0x7f148c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 shutdown_connections 2024-03-08T23:56:32.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.857+0000 7f14926e9640 1 -- 172.21.15.22:0/672035322 wait complete. 2024-03-08T23:56:32.864 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:32.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:56:32.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 -- 172.21.15.22:0/2631788252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81003d0 msgr2=0x7f71e81007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 --2- 172.21.15.22:0/2631788252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81003d0 0x7f71e81007b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009ca0 tx=0x7f71e402c100 comp rx=0 tx=0).stop 2024-03-08T23:56:32.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 -- 172.21.15.22:0/2631788252 shutdown_connections 2024-03-08T23:56:32.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 --2- 172.21.15.22:0/2631788252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71e81054a0 0x7f71e810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 --2- 172.21.15.22:0/2631788252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e8100d80 0x7f71e8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 --2- 172.21.15.22:0/2631788252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81003d0 0x7f71e81007b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 -- 172.21.15.22:0/2631788252 >> 172.21.15.22:0/2631788252 conn(0x7f71e8077320 msgr2=0x7f71e8077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 -- 172.21.15.22:0/2631788252 shutdown_connections 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.980+0000 7f71f0214640 1 -- 172.21.15.22:0/2631788252 wait complete. 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 Processor -- start 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 -- start start 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e81003d0 0x7f71e819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81054a0 0x7f71e819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71e8197960 0x7f71e8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8110560 con 0x7f71e81054a0 2024-03-08T23:56:32.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81103e0 con 0x7f71e81003d0 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71f0214640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81106e0 con 0x7f71e8197960 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71ed7af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81054a0 0x7f71e819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71edfb0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e81003d0 0x7f71e819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71ee7b1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71e8197960 0x7f71e8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71edfb0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e81003d0 0x7f71e819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50580/0 (socket says 172.21.15.22:50580) 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.981+0000 7f71edfb0640 1 -- 172.21.15.22:0/1696307556 learned_addr learned my addr 172.21.15.22:0/1696307556 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71edfb0640 1 -- 172.21.15.22:0/1696307556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71e8197960 msgr2=0x7f71e8197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71edfb0640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71e8197960 0x7f71e8197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71edfb0640 1 -- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81054a0 msgr2=0x7f71e819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71edfb0640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81054a0 0x7f71e819dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71edfb0640 1 -- 172.21.15.22:0/1696307556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8198480 con 0x7f71e81003d0 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71edfb0640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e81003d0 0x7f71e819d7b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71e4005e30 tx=0x7f71e4032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71deffd640 1 -- 172.21.15.22:0/1696307556 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4039040 con 0x7f71e81003d0 2024-03-08T23:56:32.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e8198710 con 0x7f71e81003d0 2024-03-08T23:56:32.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71deffd640 1 -- 172.21.15.22:0/1696307556 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e4044070 con 0x7f71e81003d0 2024-03-08T23:56:32.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71deffd640 1 -- 172.21.15.22:0/1696307556 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40042a0 con 0x7f71e81003d0 2024-03-08T23:56:32.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.982+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e81a4960 con 0x7f71e81003d0 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.984+0000 7f71deffd640 1 -- 172.21.15.22:0/1696307556 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f71e4003970 con 0x7f71e81003d0 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 Processor -- start 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 -- start start 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71deffd640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71d40a2970 0x7f71d40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c94580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59c9d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c9d6a0 0x558d59c9fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59c5b360 con 0x558d59c941a0 2024-03-08T23:56:32.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59bf7ec0 con 0x558d59c94ac0 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71f2df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59ba0d30 con 0x558d59c9d6a0 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71d40a2970 0x7f71d40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71ed7af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59c9d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71ed7af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59c9d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50590/0 (socket says 172.21.15.22:50590) 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.985+0000 7f71ed7af640 1 -- 172.21.15.22:0/1795235841 learned_addr learned my addr 172.21.15.22:0/1795235841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c9d6a0 0x558d59c9fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71edfb0640 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71deffd640 1 -- 172.21.15.22:0/1696307556 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f71e40f15f0 con 0x7f71e81003d0 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ed7af640 1 -- 172.21.15.22:0/1795235841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c9d6a0 msgr2=0x558d59c9fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ed7af640 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c9d6a0 0x558d59c9fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ed7af640 1 -- 172.21.15.22:0/1795235841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 msgr2=0x558d59c94580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ed7af640 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c94580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ed7af640 1 -- 172.21.15.22:0/1795235841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d59b9fbc0 con 0x558d59c94ac0 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71d40a2970 0x7f71d40a4e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ca0 tx=0x7f71e0009830 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71ed7af640 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59c9d160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71d8013bd0 tx=0x7f71d80097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e543b063eb3a7c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71d3fff640 1 -- 172.21.15.22:0/1795235841 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004050 con 0x558d59c94ac0 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71d3fff640 1 -- 172.21.15.22:0/1795235841 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71d80041f0 con 0x558d59c94ac0 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.986+0000 7f71d3fff640 1 -- 172.21.15.22:0/1795235841 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d801c7a0 con 0x558d59c94ac0 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 -- 172.21.15.22:0/1795235841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 msgr2=0x558d59c9d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59c9d160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71d8013bd0 tx=0x7f71d80097a0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 -- 172.21.15.22:0/1795235841 shutdown_connections 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c9d6a0 0x558d59c9fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59c9d160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 --2- 172.21.15.22:0/1795235841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c94580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 -- 172.21.15.22:0/1795235841 >> 172.21.15.22:0/1795235841 conn(0x558d59c8e8d0 msgr2=0x558d59c8ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 -- 172.21.15.22:0/1795235841 shutdown_connections 2024-03-08T23:56:32.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.987+0000 7f71f2df0740 1 -- 172.21.15.22:0/1795235841 wait complete. 2024-03-08T23:56:32.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 -- start start 2024-03-08T23:56:32.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 Processor -- start 2024-03-08T23:56:32.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59c9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c94ac0 0x558d59c9abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c9b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59b964a0 con 0x558d59c94ac0 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59bf3f60 con 0x558d59c9d6a0 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59ca0490 con 0x558d59c941a0 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.988+0000 7f71f2df0740 10 client.0 _ll_register_callbacks cb 0x558d59bf1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c94ac0 0x558d59c9abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c94ac0 0x558d59c9abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42188/0 (socket says 172.21.15.22:42188) 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 -- 172.21.15.22:0/3218698357 learned_addr learned my addr 172.21.15.22:0/3218698357 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:32.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c9b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59c9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 -- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 msgr2=0x558d59c9a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59c9a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 -- 172.21.15.22:0/3218698357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 msgr2=0x558d59c9b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c9b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 -- 172.21.15.22:0/3218698357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d5990c3f0 con 0x558d59c94ac0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c9b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71ed7af640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c94ac0 0x558d59c9abc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005db0 tx=0x7f71d8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48bc6e0f731113 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71d17fa640 1 -- 172.21.15.22:0/3218698357 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d802a020 con 0x558d59c94ac0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71d80041f0 con 0x558d59c94ac0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71d17fa640 1 -- 172.21.15.22:0/3218698357 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f71d8019050 con 0x558d59c94ac0 2024-03-08T23:56:32.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.989+0000 7f71d17fa640 1 -- 172.21.15.22:0/3218698357 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71d8035050 con 0x558d59c94ac0 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 10 client.16272 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558d599d67d0 con 0x558d59c94ac0 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71d17fa640 1 -- 172.21.15.22:0/3218698357 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71d80041f0 con 0x558d59c94ac0 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71d17fa640 1 -- 172.21.15.22:0/3218698357 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71d801d420 con 0x558d59c94ac0 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 10 client.16272 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 10 client.16272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 20 client.16272 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59cad740 0x558d59cafb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 4 client.16272 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d59c05400 con 0x558d59cad740 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59cad740 0x558d59cafb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59cad740 0x558d59cafb30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a1b6c33d44ca1ef server_cookie=c275c490f9a45cdf in_seq=0 out_seq=0 2024-03-08T23:56:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.990+0000 7f71d17fa640 10 client.16272 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71d17fa640 1 -- 172.21.15.22:0/3218698357 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d59c05400 con 0x558d59cad740 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71d17fa640 10 client.16272 handle_command_reply: tid=0 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 client.16272 shutdown 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c94ac0 msgr2=0x558d59c9abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c94ac0 0x558d59c9abc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005db0 tx=0x7f71d8009640 comp rx=0 tx=0).stop 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 shutdown_connections 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59cad740 0x558d59cafb30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c9b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71d17fa640 0 client.16272 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c94ac0 0x558d59c9abc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3218698357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59c9a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 >> 172.21.15.22:0/3218698357 conn(0x558d59c8e8d0 msgr2=0x558d59c90620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 shutdown_connections 2024-03-08T23:56:32.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.991+0000 7f71f2df0740 1 -- 172.21.15.22:0/3218698357 wait complete. 2024-03-08T23:56:32.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:32.992+0000 7f71f2df0740 20 client.16272 trim_cache size 0 max 16384 2024-03-08T23:56:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 Processor -- start 2024-03-08T23:56:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 -- start start 2024-03-08T23:56:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c9cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c94ac0 0x558d59c95940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c95e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81ae890 con 0x558d59c941a0 2024-03-08T23:56:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81ac430 con 0x558d59c9d6a0 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71f2df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81aa4d0 con 0x558d59c94ac0 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c9cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c9cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42190/0 (socket says 172.21.15.22:42190) 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 -- 172.21.15.22:0/3914418415 learned_addr learned my addr 172.21.15.22:0/3914418415 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71ed7af640 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c94ac0 0x558d59c95940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 -- 172.21.15.22:0/3914418415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c94ac0 msgr2=0x558d59c95940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c94ac0 0x558d59c95940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 -- 172.21.15.22:0/3914418415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 msgr2=0x558d59c95e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c95e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 -- 172.21.15.22:0/3914418415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d59c948f0 con 0x558d59c941a0 2024-03-08T23:56:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c95e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.005+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c9cdb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f71e4004af0 tx=0x7f71e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ede9fddd82b3ead server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71d37fe640 1 -- 172.21.15.22:0/3914418415 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40d0190 con 0x558d59c941a0 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71d37fe640 1 -- 172.21.15.22:0/3914418415 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e40bd9c0 con 0x558d59c941a0 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71d37fe640 1 -- 172.21.15.22:0/3914418415 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40ba030 con 0x558d59c941a0 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 -- 172.21.15.22:0/3914418415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 msgr2=0x558d59c9cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c9cdb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f71e4004af0 tx=0x7f71e40095c0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 -- 172.21.15.22:0/3914418415 shutdown_connections 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c9d6a0 0x558d59c95e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c94ac0 0x558d59c95940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3914418415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c941a0 0x558d59c9cdb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 -- 172.21.15.22:0/3914418415 >> 172.21.15.22:0/3914418415 conn(0x558d59c8e8d0 msgr2=0x558d59c8ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 -- 172.21.15.22:0/3914418415 shutdown_connections 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.006+0000 7f71f2df0740 1 -- 172.21.15.22:0/3914418415 wait complete. 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 -- start start 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 Processor -- start 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59ca4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59ca2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c9d6a0 0x558d59ca3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81aaad0 con 0x558d59c9d6a0 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81abdd0 con 0x558d59c94ac0 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81ab110 con 0x558d59c941a0 2024-03-08T23:56:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71f2df0740 10 client.0 _ll_register_callbacks cb 0x7f71d80137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71ed7af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59ca2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71ee7b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c9d6a0 0x558d59ca3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71edfb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59ca4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71edfb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59ca4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:34744/0 (socket says 172.21.15.22:34744) 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71ed7af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59ca2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50616/0 (socket says 172.21.15.22:50616) 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71ee7b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c9d6a0 0x558d59ca3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42192/0 (socket says 172.21.15.22:42192) 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.007+0000 7f71edfb0640 1 -- 172.21.15.22:0/3494822866 learned_addr learned my addr 172.21.15.22:0/3494822866 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71ee7b1640 1 -- 172.21.15.22:0/3494822866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 msgr2=0x558d59ca4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59ca4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71ee7b1640 1 -- 172.21.15.22:0/3494822866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 msgr2=0x558d59ca2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59ca2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71ee7b1640 1 -- 172.21.15.22:0/3494822866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d59ca3bc0 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71ed7af640 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59ca2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71ee7b1640 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c9d6a0 0x558d59ca3390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f71e0014010 tx=0x7f71e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a34ab92c706907f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71d17fa640 1 -- 172.21.15.22:0/3494822866 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000f890 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71d17fa640 1 -- 172.21.15.22:0/3494822866 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f71e000fa30 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e40bd9c0 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71d17fa640 1 -- 172.21.15.22:0/3494822866 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e0035050 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71f2df0740 10 client.16278 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558d59d049d0 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.008+0000 7f71d17fa640 1 -- 172.21.15.22:0/3494822866 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e40bd9c0 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71d17fa640 1 -- 172.21.15.22:0/3494822866 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71e0029070 con 0x558d59c9d6a0 2024-03-08T23:56:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71f2df0740 10 client.16278 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71f2df0740 10 client.16278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71f2df0740 20 client.16278 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59d05150 0x558d59d07570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71f2df0740 4 client.16278 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d59bff750 con 0x558d59d05150 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59d05150 0x558d59d07570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71d17fa640 10 client.16278 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.009+0000 7f71edfb0640 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59d05150 0x558d59d07570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be25241af8391109 server_cookie=5011c98c3f483f32 in_seq=0 out_seq=0 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71d17fa640 1 -- 172.21.15.22:0/3494822866 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d59bff750 con 0x558d59d05150 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71d17fa640 10 client.16278 handle_command_reply: tid=0 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 client.16278 shutdown 2024-03-08T23:56:33.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c9d6a0 msgr2=0x558d59ca3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c9d6a0 0x558d59ca3390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f71e0014010 tx=0x7f71e0004730 comp rx=0 tx=0).stop 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 shutdown_connections 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x558d59d05150 0x558d59d07570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558d59c9d6a0 0x558d59ca3390 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d59c94ac0 0x558d59ca2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 --2- 172.21.15.22:0/3494822866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d59c941a0 0x558d59ca4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71d17fa640 0 client.16278 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 >> 172.21.15.22:0/3494822866 conn(0x558d59c8e8d0 msgr2=0x558d59c98b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 shutdown_connections 2024-03-08T23:56:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 1 -- 172.21.15.22:0/3494822866 wait complete. 2024-03-08T23:56:33.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f2df0740 20 client.16278 trim_cache size 0 max 16384 2024-03-08T23:56:33.015 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71d40a2970 msgr2=0x7f71d40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f0214640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71d40a2970 0x7f71d40a4e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ca0 tx=0x7f71e0009830 comp rx=0 tx=0).stop 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e81003d0 msgr2=0x7f71e819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f0214640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e81003d0 0x7f71e819d7b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71e4005e30 tx=0x7f71e4032f40 comp rx=0 tx=0).stop 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 shutdown_connections 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f0214640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71d40a2970 0x7f71d40a4e30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.010+0000 7f71f0214640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71e8197960 0x7f71e8197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.011+0000 7f71f0214640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71e81054a0 0x7f71e819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.011+0000 7f71f0214640 1 --2- 172.21.15.22:0/1696307556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71e81003d0 0x7f71e819d7b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.011+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 >> 172.21.15.22:0/1696307556 conn(0x7f71e8077320 msgr2=0x7f71e80698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.011+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 shutdown_connections 2024-03-08T23:56:33.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.011+0000 7f71f0214640 1 -- 172.21.15.22:0/1696307556 wait complete. 2024-03-08T23:56:33.018 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:33.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:56:33.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 -- 172.21.15.22:0/984107739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102e30 msgr2=0x7feae410b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 --2- 172.21.15.22:0/984107739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102e30 0x7feae410b310 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fead8009900 tx=0x7fead802bcd0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 -- 172.21.15.22:0/984107739 shutdown_connections 2024-03-08T23:56:33.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 --2- 172.21.15.22:0/984107739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae410b850 0x7feae410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 --2- 172.21.15.22:0/984107739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102e30 0x7feae410b310 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 --2- 172.21.15.22:0/984107739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae4102510 0x7feae41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 -- 172.21.15.22:0/984107739 >> 172.21.15.22:0/984107739 conn(0x7feae40773a0 msgr2=0x7feae40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 -- 172.21.15.22:0/984107739 shutdown_connections 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.136+0000 7feae90a1640 1 -- 172.21.15.22:0/984107739 wait complete. 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 Processor -- start 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 -- start start 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102510 0x7feae4100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae4102e30 0x7feae41006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae410b850 0x7feae4100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4111630 con 0x7feae4102510 2024-03-08T23:56:33.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41114b0 con 0x7feae410b850 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.137+0000 7feae90a1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41117b0 con 0x7feae4102e30 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102510 0x7feae4100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102510 0x7feae4100160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42204/0 (socket says 172.21.15.22:42204) 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 -- 172.21.15.22:0/489285973 learned_addr learned my addr 172.21.15.22:0/489285973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae259c640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae4102e30 0x7feae41006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae359e640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae410b850 0x7feae4100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 -- 172.21.15.22:0/489285973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae4102e30 msgr2=0x7feae41006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae4102e30 0x7feae41006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 -- 172.21.15.22:0/489285973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae410b850 msgr2=0x7feae4100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae410b850 0x7feae4100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 -- 172.21.15.22:0/489285973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae41a5ff0 con 0x7feae4102510 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae2d9d640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102510 0x7feae4100160 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7feadc00e9f0 tx=0x7feadc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feacbfff640 1 -- 172.21.15.22:0/489285973 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00cd80 con 0x7feae4102510 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feacbfff640 1 -- 172.21.15.22:0/489285973 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feadc010910 con 0x7feae4102510 2024-03-08T23:56:33.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae41a62e0 con 0x7feae4102510 2024-03-08T23:56:33.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feacbfff640 1 -- 172.21.15.22:0/489285973 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc018990 con 0x7feae4102510 2024-03-08T23:56:33.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.138+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae41a6850 con 0x7feae4102510 2024-03-08T23:56:33.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.140+0000 7feacbfff640 1 -- 172.21.15.22:0/489285973 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7feadc010ac0 con 0x7feae4102510 2024-03-08T23:56:33.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 Processor -- start 2024-03-08T23:56:33.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 -- start start 2024-03-08T23:56:33.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82dea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feacbfff640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feacc0a2a20 0x7feacc0a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82df3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82df36a0 0x556a82df5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a82db1360 con 0x556a82dea1a0 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a82d4dec0 con 0x556a82df36a0 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feaeb922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a82cf6d30 con 0x556a82deaac0 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feae359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82dea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.141+0000 7feae2d9d640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feacc0a2a20 0x7feacc0a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82dea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42206/0 (socket says 172.21.15.22:42206) 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 -- 172.21.15.22:0/1672145614 learned_addr learned my addr 172.21.15.22:0/1672145614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae259c640 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82df3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82df36a0 0x556a82df5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feacbfff640 1 -- 172.21.15.22:0/489285973 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7feadc0cf530 con 0x7feae4102510 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 -- 172.21.15.22:0/1672145614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 msgr2=0x556a82df3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82df3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 -- 172.21.15.22:0/1672145614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82df36a0 msgr2=0x556a82df5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82df36a0 0x556a82df5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 -- 172.21.15.22:0/1672145614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a82cf5bc0 con 0x556a82dea1a0 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae2d9d640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feacc0a2a20 0x7feacc0a4ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feadc0cc990 tx=0x7feadc066ac0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feae359e640 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82dea580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fead4012a50 tx=0x7fead4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cabdea2bcc5ae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feac97fa640 1 -- 172.21.15.22:0/1672145614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4012110 con 0x556a82dea1a0 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feac97fa640 1 -- 172.21.15.22:0/1672145614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead4003e50 con 0x556a82dea1a0 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.142+0000 7feac97fa640 1 -- 172.21.15.22:0/1672145614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4004290 con 0x556a82dea1a0 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 -- 172.21.15.22:0/1672145614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 msgr2=0x556a82dea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82dea580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fead4012a50 tx=0x7fead4009a20 comp rx=0 tx=0).stop 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 -- 172.21.15.22:0/1672145614 shutdown_connections 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82df36a0 0x556a82df5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82df3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 --2- 172.21.15.22:0/1672145614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82dea580 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 -- 172.21.15.22:0/1672145614 >> 172.21.15.22:0/1672145614 conn(0x556a82de48d0 msgr2=0x556a82de4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 -- 172.21.15.22:0/1672145614 shutdown_connections 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 -- 172.21.15.22:0/1672145614 wait complete. 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 -- start start 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.143+0000 7feaeb922740 1 Processor -- start 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82df06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82deaac0 0x556a82df0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a82cec4a0 con 0x556a82dea1a0 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a82d49f60 con 0x556a82deaac0 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a82df6490 con 0x556a82df36a0 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 10 client.0 _ll_register_callbacks cb 0x556a82d47d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82df06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82df06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42220/0 (socket says 172.21.15.22:42220) 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 -- 172.21.15.22:0/2339472631 learned_addr learned my addr 172.21.15.22:0/2339472631 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae259c640 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82deaac0 0x556a82df0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 -- 172.21.15.22:0/2339472631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 msgr2=0x556a82df1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 -- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82deaac0 msgr2=0x556a82df0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82deaac0 0x556a82df0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 -- 172.21.15.22:0/2339472631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a82a623f0 con 0x556a82dea1a0 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feae359e640 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82df06c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fead4006010 tx=0x7fead400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3261c67e98d6d30b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feac6ff5640 1 -- 172.21.15.22:0/2339472631 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4004a40 con 0x556a82dea1a0 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feac6ff5640 1 -- 172.21.15.22:0/2339472631 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fead4004be0 con 0x556a82dea1a0 2024-03-08T23:56:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fead4003e50 con 0x556a82dea1a0 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feac6ff5640 1 -- 172.21.15.22:0/2339472631 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fead4030050 con 0x556a82dea1a0 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 10 client.16290 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.144+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556a82b2c7d0 con 0x556a82dea1a0 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feac6ff5640 1 -- 172.21.15.22:0/2339472631 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead4003e50 con 0x556a82dea1a0 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feac6ff5640 1 -- 172.21.15.22:0/2339472631 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fead4005040 con 0x556a82dea1a0 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feaeb922740 10 client.16290 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feaeb922740 10 client.16290 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feaeb922740 20 client.16290 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feaeb922740 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e038d0 0x556a82e05cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feaeb922740 4 client.16290 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a82d5b400 con 0x556a82e038d0 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feae259c640 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e038d0 0x556a82e05cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feae259c640 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e038d0 0x556a82e05cc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd6c2ca4dc20ca4f server_cookie=a6d4b71b1332771f in_seq=0 out_seq=0 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.145+0000 7feac6ff5640 10 client.16290 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feac6ff5640 1 -- 172.21.15.22:0/2339472631 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a82d5b400 con 0x556a82e038d0 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feac6ff5640 10 client.16290 handle_command_reply: tid=0 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 client.16290 shutdown 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 msgr2=0x556a82df06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82df06c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fead4006010 tx=0x7fead400ec90 comp rx=0 tx=0).stop 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 shutdown_connections 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e038d0 0x556a82e05cc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feac6ff5640 0 client.16290 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82deaac0 0x556a82df0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 --2- 172.21.15.22:0/2339472631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82dea1a0 0x556a82df06c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 >> 172.21.15.22:0/2339472631 conn(0x556a82de48d0 msgr2=0x556a82de6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 shutdown_connections 2024-03-08T23:56:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 1 -- 172.21.15.22:0/2339472631 wait complete. 2024-03-08T23:56:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.146+0000 7feaeb922740 20 client.16290 trim_cache size 0 max 16384 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.159+0000 7feaeb922740 1 Processor -- start 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feaeb922740 1 -- start start 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feaeb922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82df2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feaeb922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82ded960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feaeb922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82df36a0 0x556a82dedea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feaeb922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41b2c70 con 0x556a82df36a0 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feaeb922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41b0810 con 0x556a82dea1a0 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feaeb922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41ae8b0 con 0x556a82deaac0 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82df36a0 0x556a82dedea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82ded960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82df36a0 0x556a82dedea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42228/0 (socket says 172.21.15.22:42228) 2024-03-08T23:56:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 -- 172.21.15.22:0/781726729 learned_addr learned my addr 172.21.15.22:0/781726729 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82df2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 -- 172.21.15.22:0/781726729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 msgr2=0x556a82ded960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82ded960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 -- 172.21.15.22:0/781726729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 msgr2=0x556a82df2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82df2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 -- 172.21.15.22:0/781726729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a82dea8f0 con 0x556a82df36a0 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae259c640 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82ded960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.160+0000 7feae359e640 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82df36a0 0x556a82dedea0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fead400ee00 tx=0x7fead401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0e28e8110ac14 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feac8ff9640 1 -- 172.21.15.22:0/781726729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead40092b0 con 0x556a82df36a0 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feac8ff9640 1 -- 172.21.15.22:0/781726729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead4005780 con 0x556a82df36a0 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feac8ff9640 1 -- 172.21.15.22:0/781726729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4003b80 con 0x556a82df36a0 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 -- 172.21.15.22:0/781726729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82df36a0 msgr2=0x556a82dedea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82df36a0 0x556a82dedea0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fead400ee00 tx=0x7fead401b040 comp rx=0 tx=0).stop 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 -- 172.21.15.22:0/781726729 shutdown_connections 2024-03-08T23:56:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82df36a0 0x556a82dedea0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82deaac0 0x556a82ded960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 --2- 172.21.15.22:0/781726729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82df2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 -- 172.21.15.22:0/781726729 >> 172.21.15.22:0/781726729 conn(0x556a82de48d0 msgr2=0x556a82deaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 -- 172.21.15.22:0/781726729 shutdown_connections 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.161+0000 7feaeb922740 1 -- 172.21.15.22:0/781726729 wait complete. 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 -- start start 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 Processor -- start 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82dfa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82deaac0 0x556a82df8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41aeeb0 con 0x556a82deaac0 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41b01b0 con 0x556a82dea1a0 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41af4f0 con 0x556a82df36a0 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feaeb922740 10 client.0 _ll_register_callbacks cb 0x7fead4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:33.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82deaac0 0x556a82df8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82deaac0 0x556a82df8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42242/0 (socket says 172.21.15.22:42242) 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 -- 172.21.15.22:0/3986419796 learned_addr learned my addr 172.21.15.22:0/3986419796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae359e640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82dfa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae359e640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 -- 172.21.15.22:0/3986419796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 msgr2=0x556a82df92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 -- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 msgr2=0x556a82dfa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82dfa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae259c640 1 -- 172.21.15.22:0/3986419796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a82df9aa0 con 0x556a82deaac0 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.162+0000 7feae359e640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82dfa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feae259c640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82deaac0 0x556a82df8d60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fead802c4c0 tx=0x7fead8004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74a419adb0bbef2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feac6ff5640 1 -- 172.21.15.22:0/3986419796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead802cb20 con 0x556a82deaac0 2024-03-08T23:56:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feac6ff5640 1 -- 172.21.15.22:0/3986419796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fead802ccc0 con 0x556a82deaac0 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fead4005780 con 0x556a82deaac0 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feac6ff5640 1 -- 172.21.15.22:0/3986419796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fead8059050 con 0x556a82deaac0 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 10 client.16296 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556a82e5ac00 con 0x556a82deaac0 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feac6ff5640 1 -- 172.21.15.22:0/3986419796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fead4005780 con 0x556a82deaac0 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feac6ff5640 1 -- 172.21.15.22:0/3986419796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fead8047040 con 0x556a82deaac0 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 10 client.16296 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 10 client.16296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 20 client.16296 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e5b2d0 0x556a82e5d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 4 client.16296 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.163+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a82d55750 con 0x556a82e5b2d0 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feae359e640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e5b2d0 0x556a82e5d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feae359e640 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e5b2d0 0x556a82e5d6f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f7b7008c4056cbc server_cookie=5a8ffc02db95c55c in_seq=0 out_seq=0 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feac6ff5640 10 client.16296 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feac6ff5640 1 -- 172.21.15.22:0/3986419796 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a82d55750 con 0x556a82e5b2d0 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feac6ff5640 10 client.16296 handle_command_reply: tid=0 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 client.16296 shutdown 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82deaac0 msgr2=0x556a82df8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82deaac0 0x556a82df8d60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fead802c4c0 tx=0x7fead8004240 comp rx=0 tx=0).stop 2024-03-08T23:56:33.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 shutdown_connections 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556a82e5b2d0 0x556a82e5d6f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a82df36a0 0x556a82df92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a82deaac0 0x556a82df8d60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feac6ff5640 0 client.16296 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 --2- 172.21.15.22:0/3986419796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a82dea1a0 0x556a82dfa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 >> 172.21.15.22:0/3986419796 conn(0x556a82de48d0 msgr2=0x556a82deefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 shutdown_connections 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 1 -- 172.21.15.22:0/3986419796 wait complete. 2024-03-08T23:56:33.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.164+0000 7feaeb922740 20 client.16296 trim_cache size 0 max 16384 2024-03-08T23:56:33.170 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feacc0a2a20 msgr2=0x7feacc0a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feacc0a2a20 0x7feacc0a4ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feadc0cc990 tx=0x7feadc066ac0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102510 msgr2=0x7feae4100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102510 0x7feae4100160 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7feadc00e9f0 tx=0x7feadc00eeb0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 shutdown_connections 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feacc0a2a20 0x7feacc0a4ee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae410b850 0x7feae4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae4102e30 0x7feae41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 --2- 172.21.15.22:0/489285973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feae4102510 0x7feae4100160 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 >> 172.21.15.22:0/489285973 conn(0x7feae40773a0 msgr2=0x7feae40fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 shutdown_connections 2024-03-08T23:56:33.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.165+0000 7feae90a1640 1 -- 172.21.15.22:0/489285973 wait complete. 2024-03-08T23:56:33.172 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:32 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/672035322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:33.173 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:33.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:56:33.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 -- 172.21.15.22:0/1967663847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 msgr2=0x7ff450102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 --2- 172.21.15.22:0/1967663847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff450102910 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009ca0 tx=0x7ff44c02c230 comp rx=0 tx=0).stop 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 -- 172.21.15.22:0/1967663847 shutdown_connections 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 --2- 172.21.15.22:0/1967663847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff45010b870 0x7ff45010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 --2- 172.21.15.22:0/1967663847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff450102e50 0x7ff45010b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 --2- 172.21.15.22:0/1967663847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff450102910 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 -- 172.21.15.22:0/1967663847 >> 172.21.15.22:0/1967663847 conn(0x7ff450076570 msgr2=0x7ff450076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 -- 172.21.15.22:0/1967663847 shutdown_connections 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.288+0000 7ff4580e8640 1 -- 172.21.15.22:0/1967663847 wait complete. 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 Processor -- start 2024-03-08T23:56:33.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 -- start start 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff45019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff450102e50 0x7ff45019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45010b870 0x7ff450197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff450111650 con 0x7ff450102530 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501114d0 con 0x7ff45010b870 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff4580e8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501117d0 con 0x7ff450102e50 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff45019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff45019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42254/0 (socket says 172.21.15.22:42254) 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 -- 172.21.15.22:0/3281141014 learned_addr learned my addr 172.21.15.22:0/3281141014 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455683640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff450102e50 0x7ff45019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff456685640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45010b870 0x7ff450197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 -- 172.21.15.22:0/3281141014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff450102e50 msgr2=0x7ff45019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff450102e50 0x7ff45019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 -- 172.21.15.22:0/3281141014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45010b870 msgr2=0x7ff450197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45010b870 0x7ff450197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 -- 172.21.15.22:0/3281141014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff450198110 con 0x7ff450102530 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.289+0000 7ff455e84640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff45019d7e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005e30 tx=0x7ff44c035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.290+0000 7ff446ffd640 1 -- 172.21.15.22:0/3281141014 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c039040 con 0x7ff450102530 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.290+0000 7ff446ffd640 1 -- 172.21.15.22:0/3281141014 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44c044070 con 0x7ff450102530 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.290+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4501983a0 con 0x7ff450102530 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.290+0000 7ff446ffd640 1 -- 172.21.15.22:0/3281141014 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0042a0 con 0x7ff450102530 2024-03-08T23:56:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.290+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4501a46a0 con 0x7ff450102530 2024-03-08T23:56:33.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.292+0000 7ff446ffd640 1 -- 172.21.15.22:0/3281141014 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff44c032d20 con 0x7ff450102530 2024-03-08T23:56:33.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.292+0000 7ff45ada2740 1 Processor -- start 2024-03-08T23:56:33.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff45ada2740 1 -- start start 2024-03-08T23:56:33.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f2262580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f226b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f226b6a0 0x5600f226da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff45ada2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f2229360 con 0x5600f2262ac0 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff45ada2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f21c5ec0 con 0x5600f22621a0 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff45ada2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f216ed30 con 0x5600f226b6a0 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f226b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455683640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f226b6a0 0x5600f226da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f226b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42266/0 (socket says 172.21.15.22:42266) 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff456685640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f2262580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 -- 172.21.15.22:0/3592150672 learned_addr learned my addr 172.21.15.22:0/3592150672 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff446ffd640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff43c0a2970 0x7ff43c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff456685640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff43c0a2970 0x7ff43c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff446ffd640 1 -- 172.21.15.22:0/3281141014 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff44c0f1590 con 0x7ff450102530 2024-03-08T23:56:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 -- 172.21.15.22:0/3592150672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f226b6a0 msgr2=0x5600f226da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f226b6a0 0x5600f226da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 -- 172.21.15.22:0/3592150672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 msgr2=0x5600f2262580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f2262580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455e84640 1 -- 172.21.15.22:0/3592150672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600f216dbc0 con 0x5600f2262ac0 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff456685640 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f2262580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.293+0000 7ff455683640 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f226b6a0 0x5600f226da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.294+0000 7ff455e84640 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f226b160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0ee9c0 tx=0x7ff44c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aac6f9772f5327e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.294+0000 7ff43bfff640 1 -- 172.21.15.22:0/3592150672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0bd2c0 con 0x5600f2262ac0 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.294+0000 7ff456685640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff43c0a2970 0x7ff43c0a4e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff448009fd0 tx=0x7ff448004730 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.294+0000 7ff43bfff640 1 -- 172.21.15.22:0/3592150672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44c0be060 con 0x5600f2262ac0 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.294+0000 7ff43bfff640 1 -- 172.21.15.22:0/3592150672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0eebc0 con 0x5600f2262ac0 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 -- 172.21.15.22:0/3592150672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 msgr2=0x5600f226b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f226b160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0ee9c0 tx=0x7ff44c0093f0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 -- 172.21.15.22:0/3592150672 shutdown_connections 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f226b6a0 0x5600f226da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f226b160 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3592150672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f2262580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 -- 172.21.15.22:0/3592150672 >> 172.21.15.22:0/3592150672 conn(0x5600f225c8d0 msgr2=0x5600f225ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 -- 172.21.15.22:0/3592150672 shutdown_connections 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 -- 172.21.15.22:0/3592150672 wait complete. 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 -- start start 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 Processor -- start 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2268680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2268bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.295+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f2269100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff45ada2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f21644a0 con 0x5600f2262ac0 2024-03-08T23:56:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff45ada2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f21c1f60 con 0x5600f226b6a0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff45ada2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f226e250 con 0x5600f22621a0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff45ada2740 10 client.0 _ll_register_callbacks cb 0x5600f21bfd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2268bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2268bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42268/0 (socket says 172.21.15.22:42268) 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 -- 172.21.15.22:0/3436892680 learned_addr learned my addr 172.21.15.22:0/3436892680 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455e84640 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2268680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff456685640 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f2269100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 -- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 msgr2=0x5600f2268680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2268680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 -- 172.21.15.22:0/3436892680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 msgr2=0x5600f2269100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f2269100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 -- 172.21.15.22:0/3436892680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600f1eda3f0 con 0x5600f2262ac0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff455683640 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2268bc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff440012710 tx=0x7ff440012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c20840a04f809dd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff4397fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff4397fa640 1 -- 172.21.15.22:0/3436892680 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000f920 con 0x5600f2262ac0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff4397fa640 1 -- 172.21.15.22:0/3436892680 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff44000fac0 con 0x5600f2262ac0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff44c0be060 con 0x5600f2262ac0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff4397fa640 1 -- 172.21.15.22:0/3436892680 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff44000fcc0 con 0x5600f2262ac0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff45ada2740 10 client.16308 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.296+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5600f1fa47d0 con 0x5600f2262ac0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff4397fa640 1 -- 172.21.15.22:0/3436892680 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44c0be060 con 0x5600f2262ac0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff4397fa640 1 -- 172.21.15.22:0/3436892680 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff44001c420 con 0x5600f2262ac0 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff45ada2740 10 client.16308 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff45ada2740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff45ada2740 20 client.16308 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f227b560 0x5600f227d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff45ada2740 4 client.16308 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:33.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600f21d3400 con 0x5600f227b560 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.297+0000 7ff455e84640 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f227b560 0x5600f227d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.298+0000 7ff4397fa640 10 client.16308 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.298+0000 7ff455e84640 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f227b560 0x5600f227d9b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547a7f811aa93b8a server_cookie=5cf80ad4cd3f8688 in_seq=0 out_seq=0 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.298+0000 7ff4397fa640 1 -- 172.21.15.22:0/3436892680 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600f21d3400 con 0x5600f227b560 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.298+0000 7ff4397fa640 10 client.16308 handle_command_reply: tid=0 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.298+0000 7ff45ada2740 1 client.16308 shutdown 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.298+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 msgr2=0x5600f2268bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.298+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2268bc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff440012710 tx=0x7ff440012bd0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 shutdown_connections 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f227b560 0x5600f227d9b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f2269100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2268bc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff4397fa640 0 client.16308 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3436892680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2268680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 >> 172.21.15.22:0/3436892680 conn(0x5600f225c8d0 msgr2=0x5600f225e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 shutdown_connections 2024-03-08T23:56:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 1 -- 172.21.15.22:0/3436892680 wait complete. 2024-03-08T23:56:33.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.299+0000 7ff45ada2740 20 client.16308 trim_cache size 0 max 16384 2024-03-08T23:56:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.311+0000 7ff45ada2740 1 Processor -- start 2024-03-08T23:56:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.311+0000 7ff45ada2740 1 -- start start 2024-03-08T23:56:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f226adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f2262ac0 0x5600f2265900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f226b6a0 0x5600f2265e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff45ada2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501b0b30 con 0x5600f226b6a0 2024-03-08T23:56:33.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff45ada2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501ae6d0 con 0x5600f22621a0 2024-03-08T23:56:33.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff45ada2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501ac770 con 0x5600f2262ac0 2024-03-08T23:56:33.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff456685640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f226b6a0 0x5600f2265e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455683640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f2262ac0 0x5600f2265900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff456685640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f226b6a0 0x5600f2265e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42280/0 (socket says 172.21.15.22:42280) 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455e84640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f226adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff456685640 1 -- 172.21.15.22:0/3814593800 learned_addr learned my addr 172.21.15.22:0/3814593800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455683640 1 -- 172.21.15.22:0/3814593800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 msgr2=0x5600f226adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455683640 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f226adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455683640 1 -- 172.21.15.22:0/3814593800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f226b6a0 msgr2=0x5600f2265e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455683640 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f226b6a0 0x5600f2265e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455683640 1 -- 172.21.15.22:0/3814593800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600f2266bd0 con 0x5600f2262ac0 2024-03-08T23:56:33.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff456685640 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f226b6a0 0x5600f2265e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.312+0000 7ff455e84640 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f226adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.313+0000 7ff455683640 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f2262ac0 0x5600f2265900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff44001c910 tx=0x7ff44000f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4344dbc3ae5ddc9c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.313+0000 7ff43b7fe640 1 -- 172.21.15.22:0/3814593800 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000fe60 con 0x5600f2262ac0 2024-03-08T23:56:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.313+0000 7ff43b7fe640 1 -- 172.21.15.22:0/3814593800 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44001ca80 con 0x5600f2262ac0 2024-03-08T23:56:33.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.313+0000 7ff43b7fe640 1 -- 172.21.15.22:0/3814593800 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400035f0 con 0x5600f2262ac0 2024-03-08T23:56:33.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 -- 172.21.15.22:0/3814593800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f2262ac0 msgr2=0x5600f2265900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f2262ac0 0x5600f2265900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff44001c910 tx=0x7ff44000f810 comp rx=0 tx=0).stop 2024-03-08T23:56:33.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 -- 172.21.15.22:0/3814593800 shutdown_connections 2024-03-08T23:56:33.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f226b6a0 0x5600f2265e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f2262ac0 0x5600f2265900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3814593800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f22621a0 0x5600f226adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 -- 172.21.15.22:0/3814593800 >> 172.21.15.22:0/3814593800 conn(0x5600f225c8d0 msgr2=0x5600f2282350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 -- 172.21.15.22:0/3814593800 shutdown_connections 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 -- 172.21.15.22:0/3814593800 wait complete. 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 -- start start 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.314+0000 7ff45ada2740 1 Processor -- start 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2271600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2271b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff45ada2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f22cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff45ada2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501acd70 con 0x5600f2262ac0 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff45ada2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501ae070 con 0x5600f226b6a0 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff45ada2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501ad3b0 con 0x5600f22621a0 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff45ada2740 10 client.0 _ll_register_callbacks cb 0x7ff44c0b9a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455683640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2271b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455e84640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2271600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455683640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2271b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42292/0 (socket says 172.21.15.22:42292) 2024-03-08T23:56:33.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455683640 1 -- 172.21.15.22:0/3616534687 learned_addr learned my addr 172.21.15.22:0/3616534687 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff456685640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f22cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455e84640 1 -- 172.21.15.22:0/3616534687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 msgr2=0x5600f22cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455e84640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f22cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455e84640 1 -- 172.21.15.22:0/3616534687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 msgr2=0x5600f2271b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455e84640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2271b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455e84640 1 -- 172.21.15.22:0/3616534687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600f22cc120 con 0x5600f22621a0 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff456685640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f22cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.315+0000 7ff455683640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2271b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff455e84640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2271600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0c1c70 tx=0x7ff44c003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3439adf4a02ab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:33.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff4397fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff4397fa640 1 -- 172.21.15.22:0/3616534687 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0bd2c0 con 0x5600f22621a0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff44001ca80 con 0x5600f22621a0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff4397fa640 1 -- 172.21.15.22:0/3616534687 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff44c0b9fd0 con 0x5600f22621a0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff4397fa640 1 -- 172.21.15.22:0/3616534687 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff44c0b9c00 con 0x5600f22621a0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 10 client.25795 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5600f22cd420 con 0x5600f22621a0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff4397fa640 1 -- 172.21.15.22:0/3616534687 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff44001ca80 con 0x5600f22621a0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff4397fa640 1 -- 172.21.15.22:0/3616534687 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff44c03c030 con 0x5600f22621a0 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 10 client.25795 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 20 client.25795 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f22d7ee0 0x5600f22da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 4 client.25795 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:33.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.316+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5600f21cd750 con 0x5600f22d7ee0 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.317+0000 7ff455683640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f22d7ee0 0x5600f22da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.317+0000 7ff455683640 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f22d7ee0 0x5600f22da2d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d8b93c4d71519 server_cookie=1b7a148a69cd2d5f in_seq=0 out_seq=0 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.317+0000 7ff4397fa640 10 client.25795 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff4397fa640 1 -- 172.21.15.22:0/3616534687 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5600f21cd750 con 0x5600f22d7ee0 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff4397fa640 10 client.25795 handle_command_reply: tid=0 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 client.25795 shutdown 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 msgr2=0x5600f2271600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2271600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0c1c70 tx=0x7ff44c003c00 comp rx=0 tx=0).stop 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 shutdown_connections 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5600f22d7ee0 0x5600f22da2d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600f226b6a0 0x5600f22cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5600f2262ac0 0x5600f2271b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff4397fa640 0 client.25795 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:33.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 --2- 172.21.15.22:0/3616534687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5600f22621a0 0x5600f2271600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 >> 172.21.15.22:0/3616534687 conn(0x5600f225c8d0 msgr2=0x5600f225dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 shutdown_connections 2024-03-08T23:56:33.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 1 -- 172.21.15.22:0/3616534687 wait complete. 2024-03-08T23:56:33.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff45ada2740 20 client.25795 trim_cache size 0 max 16384 2024-03-08T23:56:33.323 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:33.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff43c0a2970 msgr2=0x7ff43c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.318+0000 7ff4580e8640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff43c0a2970 0x7ff43c0a4e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff448009fd0 tx=0x7ff448004730 comp rx=0 tx=0).stop 2024-03-08T23:56:33.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 msgr2=0x7ff45019d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:33.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff45019d7e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005e30 tx=0x7ff44c035d90 comp rx=0 tx=0).stop 2024-03-08T23:56:33.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 shutdown_connections 2024-03-08T23:56:33.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff43c0a2970 0x7ff43c0a4e30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45010b870 0x7ff450197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff450102e50 0x7ff45019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 --2- 172.21.15.22:0/3281141014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff450102530 0x7ff45019d7e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:33.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 >> 172.21.15.22:0/3281141014 conn(0x7ff450076570 msgr2=0x7ff4500fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:33.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 shutdown_connections 2024-03-08T23:56:33.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:33.319+0000 7ff4580e8640 1 -- 172.21.15.22:0/3281141014 wait complete. 2024-03-08T23:56:33.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:32 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/672035322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:33.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:32 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/672035322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:33.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 106780 14.50 MB/sec warmup 41 sec latency 8.799 ms 2024-03-08T23:56:34.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:33 smithi022 ceph-mon[31894]: pgmap v64: 129 pgs: 129 active+clean; 58 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-03-08T23:56:34.326 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:56:34.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:56:34.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:56:34.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:33 smithi062 ceph-mon[36129]: pgmap v64: 129 pgs: 129 active+clean; 58 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-03-08T23:56:34.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:33 smithi192 ceph-mon[35995]: pgmap v64: 129 pgs: 129 active+clean; 58 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 390 op/s 2024-03-08T23:56:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 -- 172.21.15.22:0/3426140978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c381054a0 msgr2=0x7f2c3810d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 --2- 172.21.15.22:0/3426140978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c381054a0 0x7f2c3810d070 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009ca0 tx=0x7f2c3002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 -- 172.21.15.22:0/3426140978 shutdown_connections 2024-03-08T23:56:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 --2- 172.21.15.22:0/3426140978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c381054a0 0x7f2c3810d070 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 --2- 172.21.15.22:0/3426140978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c38100d80 0x7f2c38104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 --2- 172.21.15.22:0/3426140978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 0x7f2c381007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 -- 172.21.15.22:0/3426140978 >> 172.21.15.22:0/3426140978 conn(0x7f2c380772f0 msgr2=0x7f2c38077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 -- 172.21.15.22:0/3426140978 shutdown_connections 2024-03-08T23:56:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.445+0000 7f2c40280640 1 -- 172.21.15.22:0/3426140978 wait complete. 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.446+0000 7f2c40280640 1 Processor -- start 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.446+0000 7f2c40280640 1 -- start start 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c40280640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 0x7f2c38199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c40280640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c381054a0 0x7f2c3819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c40280640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c3819e5f0 0x7f2c3819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c40280640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c38110560 con 0x7f2c3819e5f0 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c40280640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381103e0 con 0x7f2c381003d0 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c40280640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381106e0 con 0x7f2c381054a0 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 0x7f2c38199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 0x7f2c38199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50728/0 (socket says 172.21.15.22:50728) 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 -- 172.21.15.22:0/548977801 learned_addr learned my addr 172.21.15.22:0/548977801 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e81d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c3819e5f0 0x7f2c3819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c381054a0 0x7f2c3819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 -- 172.21.15.22:0/548977801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c381054a0 msgr2=0x7f2c3819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c381054a0 0x7f2c3819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 -- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c3819e5f0 msgr2=0x7f2c3819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c3819e5f0 0x7f2c3819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.447+0000 7f2c3e01c640 1 -- 172.21.15.22:0/548977801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3819f0e0 con 0x7f2c381003d0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.448+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 0x7f2c38199d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400ea60 tx=0x7f2c3400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.448+0000 7f2c2f7fe640 1 -- 172.21.15.22:0/548977801 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3400cde0 con 0x7f2c381003d0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.448+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3819f3d0 con 0x7f2c381003d0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.448+0000 7f2c2f7fe640 1 -- 172.21.15.22:0/548977801 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c340035f0 con 0x7f2c381003d0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.448+0000 7f2c2f7fe640 1 -- 172.21.15.22:0/548977801 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34010670 con 0x7f2c381003d0 2024-03-08T23:56:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.448+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c381a6bc0 con 0x7f2c381003d0 2024-03-08T23:56:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.450+0000 7f2c2f7fe640 1 -- 172.21.15.22:0/548977801 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2c340037a0 con 0x7f2c381003d0 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.450+0000 7f2c42e5c740 1 Processor -- start 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c42e5c740 1 -- start start 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c2f7fe640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c240a2970 0x7f2c240a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b4d80 0x5567e81b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81be280 0x5567e81c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e8118b40 con 0x5567e81be280 2024-03-08T23:56:34.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e80c1d80 con 0x5567e81b56a0 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e8114be0 con 0x5567e81b4d80 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c2f7fe640 1 -- 172.21.15.22:0/548977801 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2c340cf4e0 con 0x7f2c381003d0 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e81d640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c240a2970 0x7f2c240a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b4d80 0x5567e81b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b4d80 0x5567e81b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:34858/0 (socket says 172.21.15.22:34858) 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3d81b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 -- 172.21.15.22:0/1904185653 learned_addr learned my addr 172.21.15.22:0/1904185653 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e81d640 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81be280 0x5567e81c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 -- 172.21.15.22:0/1904185653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 msgr2=0x5567e81bdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 -- 172.21.15.22:0/1904185653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81be280 msgr2=0x5567e81c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81be280 0x5567e81c0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.451+0000 7f2c3e01c640 1 -- 172.21.15.22:0/1904185653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e7ef57e0 con 0x5567e81b4d80 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.452+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b4d80 0x5567e81b5160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2c340a3a30 tx=0x7f2c3409b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deda5797a5f38f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.452+0000 7f2c3e81d640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c240a2970 0x7f2c240a4e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c30004af0 tx=0x7f2c30032340 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.452+0000 7f2c2cbfe640 1 -- 172.21.15.22:0/1904185653 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34066c20 con 0x5567e81b4d80 2024-03-08T23:56:34.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.452+0000 7f2c2cbfe640 1 -- 172.21.15.22:0/1904185653 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c340670a0 con 0x5567e81b4d80 2024-03-08T23:56:34.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.452+0000 7f2c2cbfe640 1 -- 172.21.15.22:0/1904185653 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34099ac0 con 0x5567e81b4d80 2024-03-08T23:56:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 -- 172.21.15.22:0/1904185653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b4d80 msgr2=0x5567e81b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b4d80 0x5567e81b5160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2c340a3a30 tx=0x7f2c3409b6c0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 -- 172.21.15.22:0/1904185653 shutdown_connections 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81be280 0x5567e81c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/1904185653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b4d80 0x5567e81b5160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 -- 172.21.15.22:0/1904185653 >> 172.21.15.22:0/1904185653 conn(0x5567e81af4b0 msgr2=0x5567e81af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 -- 172.21.15.22:0/1904185653 shutdown_connections 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 -- 172.21.15.22:0/1904185653 wait complete. 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 -- start start 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.453+0000 7f2c42e5c740 1 Processor -- start 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e81c1070 con 0x5567e81b4d80 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e80d51c0 con 0x5567e81b56a0 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e81c11f0 con 0x5567e81be280 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42316/0 (socket says 172.21.15.22:42316) 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3d81b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 -- 172.21.15.22:0/2739652768 learned_addr learned my addr 172.21.15.22:0/2739652768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e81d640 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c42e5c740 10 client.0 _ll_register_callbacks cb 0x5567e80c47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 -- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 msgr2=0x5567e81bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81bbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 -- 172.21.15.22:0/2739652768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 msgr2=0x5567e81bb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 -- 172.21.15.22:0/2739652768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567e7f55f70 con 0x5567e81b4d80 2024-03-08T23:56:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bb0e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2c340cc960 tx=0x7f2c34003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a9d6ef58187bce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.454+0000 7f2c21ffb640 1 -- 172.21.15.22:0/2739652768 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3409b7c0 con 0x5567e81b4d80 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c21ffb640 1 -- 172.21.15.22:0/2739652768 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2c34003b30 con 0x5567e81b4d80 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c340670a0 con 0x5567e81b4d80 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c21ffb640 1 -- 172.21.15.22:0/2739652768 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3409f910 con 0x5567e81b4d80 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 10 client.16326 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5567e7ded170 con 0x5567e81b4d80 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c21ffb640 1 -- 172.21.15.22:0/2739652768 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c340670a0 con 0x5567e81b4d80 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c21ffb640 1 -- 172.21.15.22:0/2739652768 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c3406b1f0 con 0x5567e81b4d80 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 10 client.16326 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 20 client.16326 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e81ce2a0 0x5567e81d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 4 client.16326 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567e80ee960 con 0x5567e81ce2a0 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.455+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e81ce2a0 0x5567e81d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.456+0000 7f2c21ffb640 10 client.16326 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.456+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e81ce2a0 0x5567e81d0690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50bb766f4401362 server_cookie=b468cad9c61af743 in_seq=0 out_seq=0 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c21ffb640 1 -- 172.21.15.22:0/2739652768 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567e80ee960 con 0x5567e81ce2a0 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c21ffb640 10 client.16326 handle_command_reply: tid=0 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 client.16326 shutdown 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 msgr2=0x5567e81bb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bb0e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2c340cc960 tx=0x7f2c34003e30 comp rx=0 tx=0).stop 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 shutdown_connections 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e81ce2a0 0x5567e81d0690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81bb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c21ffb640 0 client.16326 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/2739652768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bb0e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 >> 172.21.15.22:0/2739652768 conn(0x5567e81af4b0 msgr2=0x5567e81b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 shutdown_connections 2024-03-08T23:56:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.457+0000 7f2c42e5c740 1 -- 172.21.15.22:0/2739652768 wait complete. 2024-03-08T23:56:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.458+0000 7f2c42e5c740 20 client.16326 trim_cache size 0 max 16384 2024-03-08T23:56:34.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 Processor -- start 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 -- start start 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381b2d40 con 0x5567e81b4d80 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381b08e0 con 0x5567e81b56a0 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381ae980 con 0x5567e81be280 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42326/0 (socket says 172.21.15.22:42326) 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c3e01c640 1 -- 172.21.15.22:0/285775937 learned_addr learned my addr 172.21.15.22:0/285775937 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.474+0000 7f2c3e81d640 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3e01c640 1 -- 172.21.15.22:0/285775937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 msgr2=0x5567e81b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3e01c640 1 -- 172.21.15.22:0/285775937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 msgr2=0x5567e81b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3e01c640 1 -- 172.21.15.22:0/285775937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e81b16f0 con 0x5567e81b4d80 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3e81d640 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bd810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c760 tx=0x7f2c3409b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a5016285688a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c23fff640 1 -- 172.21.15.22:0/285775937 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3409b120 con 0x5567e81b4d80 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c23fff640 1 -- 172.21.15.22:0/285775937 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c3409b7c0 con 0x5567e81b4d80 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.475+0000 7f2c23fff640 1 -- 172.21.15.22:0/285775937 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340cf170 con 0x5567e81b4d80 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 -- 172.21.15.22:0/285775937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 msgr2=0x5567e81bd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bd810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c760 tx=0x7f2c3409b6c0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 -- 172.21.15.22:0/285775937 shutdown_connections 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81be280 0x5567e81b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81b56a0 0x5567e81b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/285775937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e81bd810 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 -- 172.21.15.22:0/285775937 >> 172.21.15.22:0/285775937 conn(0x5567e81af4b0 msgr2=0x5567e81c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 -- 172.21.15.22:0/285775937 shutdown_connections 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 -- 172.21.15.22:0/285775937 wait complete. 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 -- start start 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.476+0000 7f2c42e5c740 1 Processor -- start 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e821d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b56a0 0x5567e821de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81be280 0x5567e81d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381aef80 con 0x5567e81b4d80 2024-03-08T23:56:34.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381b0280 con 0x5567e81be280 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381af5c0 con 0x5567e81b56a0 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 10 client.0 _ll_register_callbacks cb 0x7f2c34009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e821d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e821d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42340/0 (socket says 172.21.15.22:42340) 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 -- 172.21.15.22:0/3475471838 learned_addr learned my addr 172.21.15.22:0/3475471838 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b56a0 0x5567e821de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e81d640 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81be280 0x5567e81d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 -- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b56a0 msgr2=0x5567e821de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b56a0 0x5567e821de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 -- 172.21.15.22:0/3475471838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81be280 msgr2=0x5567e81d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81be280 0x5567e81d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 -- 172.21.15.22:0/3475471838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567e81d24a0 con 0x5567e81b4d80 2024-03-08T23:56:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c3e01c640 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e821d940 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2c3401f000 tx=0x7f2c34098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329be3e066ac779b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c21ffb640 1 -- 172.21.15.22:0/3475471838 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340a3f60 con 0x5567e81b4d80 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c21ffb640 1 -- 172.21.15.22:0/3475471838 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2c340ccb30 con 0x5567e81b4d80 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c3409b7c0 con 0x5567e81b4d80 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.477+0000 7f2c21ffb640 1 -- 172.21.15.22:0/3475471838 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c340ccd30 con 0x5567e81b4d80 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 10 client.16332 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5567e81d24a0 con 0x5567e81b4d80 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c21ffb640 1 -- 172.21.15.22:0/3475471838 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c3409b7c0 con 0x5567e81b4d80 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c21ffb640 1 -- 172.21.15.22:0/3475471838 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c34066bb0 con 0x5567e81b4d80 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 10 client.16332 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 20 client.16332 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e8226410 0x5567e8228800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 4 client.16332 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:56:34.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5567e80da080 con 0x5567e8226410 2024-03-08T23:56:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e8226410 0x5567e8228800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c21ffb640 10 client.16332 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.478+0000 7f2c3d81b640 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e8226410 0x5567e8228800 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30661d303ea04d1 server_cookie=a22497c1183e544c in_seq=0 out_seq=0 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.482+0000 7f2c21ffb640 1 -- 172.21.15.22:0/3475471838 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5567e80da080 con 0x5567e8226410 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.482+0000 7f2c21ffb640 10 client.16332 handle_command_reply: tid=0 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.482+0000 7f2c42e5c740 1 client.16332 shutdown 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 msgr2=0x5567e821d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e821d940 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2c3401f000 tx=0x7f2c34098040 comp rx=0 tx=0).stop 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 shutdown_connections 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5567e8226410 0x5567e8228800 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567e81be280 0x5567e81d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567e81b56a0 0x5567e821de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 --2- 172.21.15.22:0/3475471838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567e81b4d80 0x5567e821d940 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 >> 172.21.15.22:0/3475471838 conn(0x5567e81af4b0 msgr2=0x5567e81bb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c21ffb640 0 client.16332 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 shutdown_connections 2024-03-08T23:56:34.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 1 -- 172.21.15.22:0/3475471838 wait complete. 2024-03-08T23:56:34.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.483+0000 7f2c42e5c740 20 client.16332 trim_cache size 0 max 16384 2024-03-08T23:56:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:34.483 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:56:34.483 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "75f04e6a-8d8e-497b-b5f0-2f6a10f3da5c", 2024-03-08T23:56:34.483 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:56:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c240a2970 msgr2=0x7f2c240a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c240a2970 0x7f2c240a4e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c30004af0 tx=0x7f2c30032340 comp rx=0 tx=0).stop 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 msgr2=0x7f2c38199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 0x7f2c38199d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400ea60 tx=0x7f2c3400ef20 comp rx=0 tx=0).stop 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 shutdown_connections 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c240a2970 0x7f2c240a4e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c3819e5f0 0x7f2c3819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c381054a0 0x7f2c3819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 --2- 172.21.15.22:0/548977801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c381003d0 0x7f2c38199d30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 >> 172.21.15.22:0/548977801 conn(0x7f2c380772f0 msgr2=0x7f2c380fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 shutdown_connections 2024-03-08T23:56:34.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.484+0000 7f2c40280640 1 -- 172.21.15.22:0/548977801 wait complete. 2024-03-08T23:56:34.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:56:34.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:56:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/3044649438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a44100270 msgr2=0x7f2a4410b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/3044649438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a44100270 0x7f2a4410b2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009900 tx=0x7f2a4002bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/3044649438 shutdown_connections 2024-03-08T23:56:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/3044649438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4410b8c0 0x7f2a4410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/3044649438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a44100270 0x7f2a4410b2f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/3044649438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a440ff950 0x7f2a440ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/3044649438 >> 172.21.15.22:0/3044649438 conn(0x7f2a44074e90 msgr2=0x7f2a440752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/3044649438 shutdown_connections 2024-03-08T23:56:34.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.642+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/3044649438 wait complete. 2024-03-08T23:56:34.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.643+0000 7f2a4ab9f640 1 Processor -- start 2024-03-08T23:56:34.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.643+0000 7f2a4ab9f640 1 -- start start 2024-03-08T23:56:34.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4ab9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a440ff950 0x7f2a44199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4ab9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44100270 0x7f2a4419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4ab9f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4410b8c0 0x7f2a4419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a440ff950 0x7f2a44199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a440ff950 0x7f2a44199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42360/0 (socket says 172.21.15.22:42360) 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 -- 172.21.15.22:0/604902316 learned_addr learned my addr 172.21.15.22:0/604902316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441115f0 con 0x7f2a440ff950 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44111470 con 0x7f2a44100270 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44111770 con 0x7f2a4410b8c0 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4410b8c0 0x7f2a4419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a4939c640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44100270 0x7f2a4419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 -- 172.21.15.22:0/604902316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4410b8c0 msgr2=0x7f2a4419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4410b8c0 0x7f2a4419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 -- 172.21.15.22:0/604902316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44100270 msgr2=0x7f2a4419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44100270 0x7f2a4419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.644+0000 7f2a49b9d640 1 -- 172.21.15.22:0/604902316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4419ed10 con 0x7f2a440ff950 2024-03-08T23:56:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.645+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a440ff950 0x7f2a44199d20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9f0 tx=0x7f2a3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.646+0000 7f2a3affd640 1 -- 172.21.15.22:0/604902316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400cd80 con 0x7f2a440ff950 2024-03-08T23:56:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.646+0000 7f2a3affd640 1 -- 172.21.15.22:0/604902316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a34010910 con 0x7f2a440ff950 2024-03-08T23:56:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.646+0000 7f2a3affd640 1 -- 172.21.15.22:0/604902316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340188f0 con 0x7f2a440ff950 2024-03-08T23:56:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.646+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4419f000 con 0x7f2a440ff950 2024-03-08T23:56:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.646+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a441a6890 con 0x7f2a440ff950 2024-03-08T23:56:34.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.650+0000 7f2a4e9d9740 1 Processor -- start 2024-03-08T23:56:34.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.650+0000 7f2a3affd640 1 -- 172.21.15.22:0/604902316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2a34010ac0 con 0x7f2a440ff950 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4e9d9740 1 -- start start 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x56486096ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096f220 0x5648609778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564860977e00 0x56486097a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648608d1e00 con 0x56486096e900 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486087ab10 con 0x56486096f220 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648608cdea0 con 0x564860977e00 2024-03-08T23:56:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4a39e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x56486096ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4a39e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x56486096ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42362/0 (socket says 172.21.15.22:42362) 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.651+0000 7f2a4a39e640 1 -- 172.21.15.22:0/1001559399 learned_addr learned my addr 172.21.15.22:0/1001559399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a3affd640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2a300a29f0 0x7f2a300a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096f220 0x5648609778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2a300a29f0 0x7f2a300a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a3affd640 1 -- 172.21.15.22:0/604902316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2a340cf680 con 0x7f2a440ff950 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a4a39e640 1 -- 172.21.15.22:0/1001559399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564860977e00 msgr2=0x56486097a1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564860977e00 0x56486097a1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a4a39e640 1 -- 172.21.15.22:0/1001559399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096f220 msgr2=0x5648609778c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096f220 0x5648609778c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.652+0000 7f2a4a39e640 1 -- 172.21.15.22:0/1001559399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648606b07d0 con 0x56486096e900 2024-03-08T23:56:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.653+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x56486096ece0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c014a20 tx=0x7f2a3c014ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343a6849fc40bc43 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.653+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2a300a29f0 0x7f2a300a4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c01c000 tx=0x7f2a3c01f000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.653+0000 7f2a2ffff640 1 -- 172.21.15.22:0/1001559399 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00ec20 con 0x56486096e900 2024-03-08T23:56:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.653+0000 7f2a2ffff640 1 -- 172.21.15.22:0/1001559399 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c003e50 con 0x56486096e900 2024-03-08T23:56:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.653+0000 7f2a2ffff640 1 -- 172.21.15.22:0/1001559399 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c004380 con 0x56486096e900 2024-03-08T23:56:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.654+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/1001559399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 msgr2=0x56486096ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.654+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x56486096ece0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c014a20 tx=0x7f2a3c014ee0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.655+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/1001559399 shutdown_connections 2024-03-08T23:56:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.655+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564860977e00 0x56486097a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.655+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096f220 0x5648609778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.655+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/1001559399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x56486096ece0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.655+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/1001559399 >> 172.21.15.22:0/1001559399 conn(0x564860969030 msgr2=0x564860969410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.655+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/1001559399 shutdown_connections 2024-03-08T23:56:34.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.655+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/1001559399 wait complete. 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 -- start start 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 Processor -- start 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096e900 0x5648609775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860974980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564860977e00 0x564860974ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486098bb40 con 0x564860977e00 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486088e0a0 con 0x56486096e900 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486098bcc0 con 0x56486096f220 2024-03-08T23:56:34.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.656+0000 7f2a4e9d9740 10 client.0 _ll_register_callbacks cb 0x56486088da70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:34.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564860977e00 0x564860974ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564860977e00 0x564860974ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42378/0 (socket says 172.21.15.22:42378) 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 -- 172.21.15.22:0/4010743962 learned_addr learned my addr 172.21.15.22:0/4010743962 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4939c640 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860974980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 -- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 msgr2=0x564860974980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860974980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 -- 172.21.15.22:0/4010743962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096e900 msgr2=0x5648609775f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096e900 0x5648609775f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.657+0000 7f2a4a39e640 1 -- 172.21.15.22:0/4010743962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648605e63f0 con 0x564860977e00 2024-03-08T23:56:34.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.658+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564860977e00 0x564860974ec0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c015010 tx=0x7f2a3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba2eb689b8f87d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.658+0000 7f2a2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:34.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.658+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/4010743962 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c02d070 con 0x564860977e00 2024-03-08T23:56:34.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.658+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a3c003e50 con 0x564860977e00 2024-03-08T23:56:34.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.658+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/4010743962 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2a3c004100 con 0x564860977e00 2024-03-08T23:56:34.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.658+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/4010743962 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a3c024040 con 0x564860977e00 2024-03-08T23:56:34.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.659+0000 7f2a4e9d9740 10 client.16344 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:34.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.659+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648605a60f0 con 0x564860977e00 2024-03-08T23:56:34.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.659+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/4010743962 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c003e50 con 0x564860977e00 2024-03-08T23:56:34.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.660+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/4010743962 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2a3c004980 con 0x564860977e00 2024-03-08T23:56:34.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.660+0000 7f2a4e9d9740 10 client.16344 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:34.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.660+0000 7f2a4e9d9740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:34.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.660+0000 7f2a4e9d9740 20 client.16344 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:34.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.660+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609a1040 0x5648609a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.660+0000 7f2a4e9d9740 4 client.16344 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:34.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.660+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648608df340 con 0x5648609a1040 2024-03-08T23:56:34.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.661+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609a1040 0x5648609a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.662+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609a1040 0x5648609a3430 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88bf0205c541bde6 server_cookie=df541f0f9a39a032 in_seq=0 out_seq=0 2024-03-08T23:56:34.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.662+0000 7f2a2d7fa640 10 client.16344 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.664+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/4010743962 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648608df340 con 0x5648609a1040 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.664+0000 7f2a2d7fa640 10 client.16344 handle_command_reply: tid=0 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.664+0000 7f2a4e9d9740 1 client.16344 shutdown 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.664+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564860977e00 msgr2=0x564860974ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.664+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564860977e00 0x564860974ec0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c015010 tx=0x7f2a3c009c00 comp rx=0 tx=0).stop 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 shutdown_connections 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609a1040 0x5648609a3430 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564860977e00 0x564860974ec0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860974980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/4010743962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56486096e900 0x5648609775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 >> 172.21.15.22:0/4010743962 conn(0x564860969030 msgr2=0x56486096ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a2d7fa640 0 client.16344 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 shutdown_connections 2024-03-08T23:56:34.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/4010743962 wait complete. 2024-03-08T23:56:34.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.665+0000 7f2a4e9d9740 20 client.16344 trim_cache size 0 max 16384 2024-03-08T23:56:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.696+0000 7f2a4e9d9740 1 Processor -- start 2024-03-08T23:56:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.696+0000 7f2a4e9d9740 1 -- start start 2024-03-08T23:56:34.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096e900 0x564860974ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096f220 0x5648609738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x5648609770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44072a00 con 0x56486096f220 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440705a0 con 0x564860977e00 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4406e680 con 0x56486096e900 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4a39e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x5648609770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4a39e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x5648609770b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50796/0 (socket says 172.21.15.22:50796) 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.697+0000 7f2a4a39e640 1 -- 172.21.15.22:0/3159397824 learned_addr learned my addr 172.21.15.22:0/3159397824 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.698+0000 7f2a4939c640 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096f220 0x5648609738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.698+0000 7f2a4a39e640 1 -- 172.21.15.22:0/3159397824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096e900 msgr2=0x564860974ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.698+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096e900 0x564860974ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.698+0000 7f2a4a39e640 1 -- 172.21.15.22:0/3159397824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096f220 msgr2=0x5648609738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.698+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096f220 0x5648609738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.699+0000 7f2a4a39e640 1 -- 172.21.15.22:0/3159397824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564860972ed0 con 0x564860977e00 2024-03-08T23:56:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.699+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x5648609770b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c014f20 tx=0x7f2a3c009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4b03dd1aab3f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.699+0000 7f2a2f7fe640 1 -- 172.21.15.22:0/3159397824 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c02d070 con 0x564860977e00 2024-03-08T23:56:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.699+0000 7f2a2f7fe640 1 -- 172.21.15.22:0/3159397824 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c004f90 con 0x564860977e00 2024-03-08T23:56:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.699+0000 7f2a2f7fe640 1 -- 172.21.15.22:0/3159397824 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c005980 con 0x564860977e00 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.700+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3159397824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 msgr2=0x5648609770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.700+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x5648609770b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c014f20 tx=0x7f2a3c009e80 comp rx=0 tx=0).stop 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.700+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3159397824 shutdown_connections 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x5648609770b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096f220 0x5648609738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3159397824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096e900 0x564860974ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3159397824 >> 172.21.15.22:0/3159397824 conn(0x564860969030 msgr2=0x56486096fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3159397824 shutdown_connections 2024-03-08T23:56:34.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3159397824 wait complete. 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 -- start start 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.701+0000 7f2a4e9d9740 1 Processor -- start 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x564860998750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860998c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4e9d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x564860996ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4406ec80 con 0x56486096e900 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4406ff40 con 0x564860977e00 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4e9d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4406f280 con 0x56486096f220 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4e9d9740 10 client.0 _ll_register_callbacks cb 0x7f2a3400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.702+0000 7f2a4939c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860998c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4a39e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x564860996ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4939c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860998c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:34936/0 (socket says 172.21.15.22:34936) 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4939c640 1 -- 172.21.15.22:0/3176108815 learned_addr learned my addr 172.21.15.22:0/3176108815 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x564860998750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4a39e640 1 -- 172.21.15.22:0/3176108815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 msgr2=0x564860998c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860998c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4a39e640 1 -- 172.21.15.22:0/3176108815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 msgr2=0x564860998750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x564860998750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4a39e640 1 -- 172.21.15.22:0/3176108815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648609974a0 con 0x564860977e00 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a4a39e640 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x564860996ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c004f60 tx=0x7f2a3c0147a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3696e5837f115dce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.703+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/3176108815 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c02d070 con 0x564860977e00 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/3176108815 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2a3c005340 con 0x564860977e00 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a3c004f90 con 0x564860977e00 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/3176108815 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a3c04b050 con 0x564860977e00 2024-03-08T23:56:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a4e9d9740 10 client.25835 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648609f7e10 con 0x564860977e00 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/3176108815 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c011440 con 0x564860977e00 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/3176108815 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2a3c032440 con 0x564860977e00 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a4e9d9740 10 client.25835 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a4e9d9740 10 client.25835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.704+0000 7f2a4e9d9740 20 client.25835 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609f8420 0x5648609fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a4e9d9740 4 client.25835 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f2a3c004f90 con 0x5648609f8420 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609f8420 0x5648609fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a49b9d640 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609f8420 0x5648609fa810 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231c5c28295bc2e0 server_cookie=7c2283ad0b5d5e95 in_seq=0 out_seq=0 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a2d7fa640 10 client.25835 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a2d7fa640 1 -- 172.21.15.22:0/3176108815 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f2a3c004f90 con 0x5648609f8420 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a2d7fa640 10 client.25835 handle_command_reply: tid=0 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a4e9d9740 1 client.25835 shutdown 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 msgr2=0x564860996ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.705+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x564860996ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c004f60 tx=0x7f2a3c0147a0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 shutdown_connections 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5648609f8420 0x5648609fa810 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564860977e00 0x564860996ca0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a2d7fa640 0 client.25835 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56486096f220 0x564860998c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 --2- 172.21.15.22:0/3176108815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56486096e900 0x564860998750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 >> 172.21.15.22:0/3176108815 conn(0x564860969030 msgr2=0x564860967880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 shutdown_connections 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 1 -- 172.21.15.22:0/3176108815 wait complete. 2024-03-08T23:56:34.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4e9d9740 20 client.25835 trim_cache size 0 max 16384 2024-03-08T23:56:34.708 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:34.708 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:56:34.708 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:56:34.708 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:34.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2a300a29f0 msgr2=0x7f2a300a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2a300a29f0 0x7f2a300a4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c01c000 tx=0x7f2a3c01f000 comp rx=0 tx=0).stop 2024-03-08T23:56:34.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a440ff950 msgr2=0x7f2a44199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a440ff950 0x7f2a44199d20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9f0 tx=0x7f2a3400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 shutdown_connections 2024-03-08T23:56:34.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2a300a29f0 0x7f2a300a4eb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4410b8c0 0x7f2a4419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44100270 0x7f2a4419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 --2- 172.21.15.22:0/604902316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a440ff950 0x7f2a44199d20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 >> 172.21.15.22:0/604902316 conn(0x7f2a44074e90 msgr2=0x7f2a440fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 shutdown_connections 2024-03-08T23:56:34.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.706+0000 7f2a4ab9f640 1 -- 172.21.15.22:0/604902316 wait complete. 2024-03-08T23:56:34.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 108840 14.33 MB/sec warmup 42 sec latency 38.081 ms 2024-03-08T23:56:34.713 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:56:34.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 -- 172.21.15.22:0/3725036102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068106650 msgr2=0x7ff06810e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 --2- 172.21.15.22:0/3725036102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068106650 0x7ff06810e200 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff060009ca0 tx=0x7ff06002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 -- 172.21.15.22:0/3725036102 shutdown_connections 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 --2- 172.21.15.22:0/3725036102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068106650 0x7ff06810e200 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 --2- 172.21.15.22:0/3725036102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06806a5e0 0x7ff068105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 --2- 172.21.15.22:0/3725036102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff068069c30 0x7ff06806a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 -- 172.21.15.22:0/3725036102 >> 172.21.15.22:0/3725036102 conn(0x7ff0680772c0 msgr2=0x7ff0680776d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 -- 172.21.15.22:0/3725036102 shutdown_connections 2024-03-08T23:56:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 -- 172.21.15.22:0/3725036102 wait complete. 2024-03-08T23:56:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 Processor -- start 2024-03-08T23:56:34.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.824+0000 7ff06ec73640 1 -- start start 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ec73640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068069c30 0x7ff06819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ec73640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff068106650 0x7ff06819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ec73640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff068197980 0x7ff068197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ec73640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0681116a0 con 0x7ff068069c30 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ec73640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068111520 con 0x7ff068197980 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ec73640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068111820 con 0x7ff068106650 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068069c30 0x7ff06819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068069c30 0x7ff06819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42420/0 (socket says 172.21.15.22:42420) 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 -- 172.21.15.22:0/1563759185 learned_addr learned my addr 172.21.15.22:0/1563759185 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff067fff640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff068106650 0x7ff06819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06d210640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff068197980 0x7ff068197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 -- 172.21.15.22:0/1563759185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff068106650 msgr2=0x7ff06819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff068106650 0x7ff06819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 -- 172.21.15.22:0/1563759185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff068197980 msgr2=0x7ff068197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff068197980 0x7ff068197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 -- 172.21.15.22:0/1563759185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0681984a0 con 0x7ff068069c30 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06d210640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff068197980 0x7ff068197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ca0f640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068069c30 0x7ff06819d780 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00e9f0 tx=0x7ff05c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff065ffb640 1 -- 172.21.15.22:0/1563759185 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00cd80 con 0x7ff068069c30 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff065ffb640 1 -- 172.21.15.22:0/1563759185 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff05c010910 con 0x7ff068069c30 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff068198790 con 0x7ff068069c30 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.825+0000 7ff065ffb640 1 -- 172.21.15.22:0/1563759185 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c018990 con 0x7ff068069c30 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.826+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0681a4990 con 0x7ff068069c30 2024-03-08T23:56:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.827+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0681029e0 con 0x7ff068069c30 2024-03-08T23:56:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.828+0000 7ff065ffb640 1 -- 172.21.15.22:0/1563759185 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff05c010ac0 con 0x7ff068069c30 2024-03-08T23:56:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.828+0000 7ff065ffb640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0580a2970 0x7ff0580a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.829+0000 7ff067fff640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0580a2970 0x7ff0580a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.829+0000 7ff065ffb640 1 -- 172.21.15.22:0/1563759185 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff05c0cf610 con 0x7ff068069c30 2024-03-08T23:56:34.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.829+0000 7ff067fff640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0580a2970 0x7ff0580a4e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0680ffc90 tx=0x7ff054005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:34.833+0000 7ff065ffb640 1 -- 172.21.15.22:0/1563759185 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff05c09bfd0 con 0x7ff068069c30 2024-03-08T23:56:34.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:34 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:56:34.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:34 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:34.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:34 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:56:34.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:34 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:34.965 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:34 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:56:35.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.073+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff068109840 con 0x7ff068069c30 2024-03-08T23:56:35.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.074+0000 7ff065ffb640 1 -- 172.21.15.22:0/1563759185 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff05c09b8d0 con 0x7ff068069c30 2024-03-08T23:56:35.074 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:56:35.074 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:55:16.974110+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25439":{"gid":25439,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/1375407074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1375407074},{"type":"v1","addr":"172.21.15.62:6837","nonce":1375407074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:56:35.075 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 27 2024-03-08T23:56:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0580a2970 msgr2=0x7ff0580a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0580a2970 0x7ff0580a4e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0680ffc90 tx=0x7ff054005f90 comp rx=0 tx=0).stop 2024-03-08T23:56:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068069c30 msgr2=0x7ff06819d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068069c30 0x7ff06819d780 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00e9f0 tx=0x7ff05c00eeb0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 shutdown_connections 2024-03-08T23:56:35.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0580a2970 0x7ff0580a4e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff068197980 0x7ff068197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff068106650 0x7ff06819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 --2- 172.21.15.22:0/1563759185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff068069c30 0x7ff06819d780 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 >> 172.21.15.22:0/1563759185 conn(0x7ff0680772c0 msgr2=0x7ff0680fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 shutdown_connections 2024-03-08T23:56:35.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.076+0000 7ff06ec73640 1 -- 172.21.15.22:0/1563759185 wait complete. 2024-03-08T23:56:35.089 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:35.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:56:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 -- 172.21.15.22:0/4120859432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd006a560 msgr2=0x7f4dd0105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/4120859432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd006a560 0x7f4dd0105f80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009900 tx=0x7f4dc402bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:35.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 -- 172.21.15.22:0/4120859432 shutdown_connections 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/4120859432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd01066b0 0x7f4dd010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/4120859432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd006a560 0x7f4dd0105f80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/4120859432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dd0069bb0 0x7f4dd0069f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 -- 172.21.15.22:0/4120859432 >> 172.21.15.22:0/4120859432 conn(0x7f4dd0076fc0 msgr2=0x7f4dd00773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 -- 172.21.15.22:0/4120859432 shutdown_connections 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.213+0000 7f4dd68a7640 1 -- 172.21.15.22:0/4120859432 wait complete. 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 Processor -- start 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 -- start start 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd0069bb0 0x7f4dd0100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dd01066b0 0x7f4dd0100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd0100f90 0x7f4dd01a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01116c0 con 0x7f4dd0100f90 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0111540 con 0x7f4dd0069bb0 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd68a7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0111840 con 0x7f4dd01066b0 2024-03-08T23:56:35.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd4e44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd0100f90 0x7f4dd01a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd4e44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd0100f90 0x7f4dd01a5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42450/0 (socket says 172.21.15.22:42450) 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dd4e44640 1 -- 172.21.15.22:0/1079586821 learned_addr learned my addr 172.21.15.22:0/1079586821 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dd01066b0 0x7f4dd0100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.214+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd0069bb0 0x7f4dd0100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd4e44640 1 -- 172.21.15.22:0/1079586821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dd01066b0 msgr2=0x7f4dd0100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd4e44640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dd01066b0 0x7f4dd0100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd4e44640 1 -- 172.21.15.22:0/1079586821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd0069bb0 msgr2=0x7f4dd0100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd4e44640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd0069bb0 0x7f4dd0100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd4e44640 1 -- 172.21.15.22:0/1079586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd01a6450 con 0x7f4dd0100f90 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd4e44640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd0100f90 0x7f4dd01a5d50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800eb10 tx=0x7f4dc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dcd7fa640 1 -- 172.21.15.22:0/1079586821 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8004b20 con 0x7f4dd0100f90 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dcd7fa640 1 -- 172.21.15.22:0/1079586821 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8004cc0 con 0x7f4dd0100f90 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd01a6740 con 0x7f4dd0100f90 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dcd7fa640 1 -- 172.21.15.22:0/1079586821 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8010760 con 0x7f4dd0100f90 2024-03-08T23:56:35.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.215+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd01a6c10 con 0x7f4dd0100f90 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.217+0000 7f4dcd7fa640 1 -- 172.21.15.22:0/1079586821 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4dc8005020 con 0x7f4dd0100f90 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.217+0000 7f4dd9485740 1 Processor -- start 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.217+0000 7f4dd9485740 1 -- start start 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.217+0000 7f4dcd7fa640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4db80a29a0 0x7f4db80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dd9485740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a364783360 con 0x55a3647c56a0 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dd9485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36471fec0 con 0x55a3647bcac0 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dd9485740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3646c8d30 con 0x55a3647bc1a0 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcd7fa640 1 -- 172.21.15.22:0/1079586821 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4dc80cf750 con 0x7f4dd0100f90 2024-03-08T23:56:35.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4db80a29a0 0x7f4db80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:34982/0 (socket says 172.21.15.22:34982) 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 -- 172.21.15.22:0/1578370698 learned_addr learned my addr 172.21.15.22:0/1578370698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dd4e44640 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 -- 172.21.15.22:0/1578370698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 msgr2=0x55a3647c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 -- 172.21.15.22:0/1578370698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 msgr2=0x55a3647c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 -- 172.21.15.22:0/1578370698 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3646c7bc0 con 0x55a3647bc1a0 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4db80a29a0 0x7f4db80a4e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4036040 tx=0x7f4dc402c7e0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647bc580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009cf0 tx=0x7f4dc0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b65c418feee280 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.218+0000 7f4db67fc640 1 -- 172.21.15.22:0/1578370698 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0004020 con 0x55a3647bc1a0 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4db67fc640 1 -- 172.21.15.22:0/1578370698 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc00041c0 con 0x55a3647bc1a0 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4db67fc640 1 -- 172.21.15.22:0/1578370698 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00187d0 con 0x55a3647bc1a0 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 -- 172.21.15.22:0/1578370698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 msgr2=0x55a3647bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647bc580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009cf0 tx=0x7f4dc0011810 comp rx=0 tx=0).stop 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 -- 172.21.15.22:0/1578370698 shutdown_connections 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1578370698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647bc580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 -- 172.21.15.22:0/1578370698 >> 172.21.15.22:0/1578370698 conn(0x55a3647b68d0 msgr2=0x55a3647b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 -- 172.21.15.22:0/1578370698 shutdown_connections 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 -- 172.21.15.22:0/1578370698 wait complete. 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.219+0000 7f4dd9485740 1 -- start start 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 1 Processor -- start 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3646be4a0 con 0x55a3647c56a0 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36471bf60 con 0x55a3647bcac0 2024-03-08T23:56:35.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3647c8490 con 0x55a3647bc1a0 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd9485740 10 client.0 _ll_register_callbacks cb 0x55a364719d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd4e44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dcffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dcf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd4e44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42472/0 (socket says 172.21.15.22:42472) 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dd4e44640 1 -- 172.21.15.22:0/1012344526 learned_addr learned my addr 172.21.15.22:0/1012344526 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dcffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:34988/0 (socket says 172.21.15.22:34988) 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dcffff640 1 -- 172.21.15.22:0/1012344526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 msgr2=0x55a3647c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dcffff640 1 -- 172.21.15.22:0/1012344526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 msgr2=0x55a3647c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.220+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dcffff640 1 -- 172.21.15.22:0/1012344526 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3644343f0 con 0x55a3647bc1a0 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647c25b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009cf0 tx=0x7f4dc0003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c814d2687db3a96e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4daf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4daf7fe640 1 -- 172.21.15.22:0/1012344526 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0026020 con 0x55a3647bc1a0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4daf7fe640 1 -- 172.21.15.22:0/1012344526 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4dc0005050 con 0x55a3647bc1a0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dc00041c0 con 0x55a3647bc1a0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4daf7fe640 1 -- 172.21.15.22:0/1012344526 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc0031050 con 0x55a3647bc1a0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dd9485740 10 client.25840 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a3644fe7d0 con 0x55a3647bc1a0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4daf7fe640 1 -- 172.21.15.22:0/1012344526 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc00041c0 con 0x55a3647bc1a0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4daf7fe640 1 -- 172.21.15.22:0/1012344526 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4dc0018aa0 con 0x55a3647bc1a0 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dd9485740 10 client.25840 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.221+0000 7f4dd9485740 10 client.25840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.222+0000 7f4dd9485740 20 client.25840 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.222+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a3647d5730 0x55a3647d7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.222+0000 7f4dd9485740 4 client.25840 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:35.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.222+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a36472d400 con 0x55a3647d5730 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.222+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a3647d5730 0x55a3647d7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.222+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a3647d5730 0x55a3647d7b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e0e72b794e3bedc server_cookie=63817e1386f41a73 in_seq=0 out_seq=0 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.222+0000 7f4daf7fe640 10 client.25840 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4daf7fe640 1 -- 172.21.15.22:0/1012344526 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a36472d400 con 0x55a3647d5730 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4daf7fe640 10 client.25840 handle_command_reply: tid=0 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 client.25840 shutdown 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 msgr2=0x55a3647c25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647c25b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009cf0 tx=0x7f4dc0003c20 comp rx=0 tx=0).stop 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 shutdown_connections 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a3647d5730 0x55a3647d7b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a3647c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bcac0 0x55a3647c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1012344526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bc1a0 0x55a3647c25b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4daf7fe640 0 client.25840 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:35.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 >> 172.21.15.22:0/1012344526 conn(0x55a3647b68d0 msgr2=0x55a3647b8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 shutdown_connections 2024-03-08T23:56:35.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 1 -- 172.21.15.22:0/1012344526 wait complete. 2024-03-08T23:56:35.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.223+0000 7f4dd9485740 20 client.25840 trim_cache size 0 max 16384 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 Processor -- start 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 -- start start 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647bc1a0 0x55a3647c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647c56a0 0x55a3647ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01b2d80 con 0x55a3647bc1a0 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01b0920 con 0x55a3647c56a0 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd9485740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01ae9c0 con 0x55a3647bcac0 2024-03-08T23:56:35.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647bc1a0 0x55a3647c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647bc1a0 0x55a3647c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42476/0 (socket says 172.21.15.22:42476) 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcffff640 1 -- 172.21.15.22:0/1017461058 learned_addr learned my addr 172.21.15.22:0/1017461058 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dd4e44640 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647c56a0 0x55a3647ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcffff640 1 -- 172.21.15.22:0/1017461058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 msgr2=0x55a3647cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcffff640 1 -- 172.21.15.22:0/1017461058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647c56a0 msgr2=0x55a3647ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.236+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647c56a0 0x55a3647ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4dcffff640 1 -- 172.21.15.22:0/1017461058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3647b6700 con 0x55a3647bc1a0 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4dd4e44640 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647c56a0 0x55a3647ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4dcffff640 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647bc1a0 0x55a3647c4ce0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0019620 tx=0x7f4dc0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd5e9892a57408e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4db5ffb640 1 -- 172.21.15.22:0/1017461058 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0026040 con 0x55a3647bc1a0 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4db5ffb640 1 -- 172.21.15.22:0/1017461058 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc0005610 con 0x55a3647bc1a0 2024-03-08T23:56:35.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4db5ffb640 1 -- 172.21.15.22:0/1017461058 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0019aa0 con 0x55a3647bc1a0 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4dd9485740 1 -- 172.21.15.22:0/1017461058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647bc1a0 msgr2=0x55a3647c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.237+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647bc1a0 0x55a3647c4ce0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0019620 tx=0x7f4dc0009bf0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- 172.21.15.22:0/1017461058 shutdown_connections 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647c56a0 0x55a3647ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 --2- 172.21.15.22:0/1017461058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647bc1a0 0x55a3647c4ce0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- 172.21.15.22:0/1017461058 >> 172.21.15.22:0/1017461058 conn(0x55a3647b68d0 msgr2=0x55a3647dc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- 172.21.15.22:0/1017461058 shutdown_connections 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- 172.21.15.22:0/1017461058 wait complete. 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- start start 2024-03-08T23:56:35.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 Processor -- start 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bc1a0 0x55a3647ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a36482aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01aefc0 con 0x55a3647c56a0 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01b02c0 con 0x55a3647bc1a0 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01af600 con 0x55a3647bcac0 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd9485740 10 client.0 _ll_register_callbacks cb 0x7f4dc00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.238+0000 7f4dd4e44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a36482aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dd4e44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a36482aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42484/0 (socket says 172.21.15.22:42484) 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcf7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dd4e44640 1 -- 172.21.15.22:0/2232907863 learned_addr learned my addr 172.21.15.22:0/2232907863 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcffff640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bc1a0 0x55a3647ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcf7fe640 1 -- 172.21.15.22:0/2232907863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bc1a0 msgr2=0x55a3647ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bc1a0 0x55a3647ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcf7fe640 1 -- 172.21.15.22:0/2232907863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 msgr2=0x55a36482aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a36482aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcf7fe640 1 -- 172.21.15.22:0/2232907863 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3647be1d0 con 0x55a3647bcac0 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcffff640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bc1a0 0x55a3647ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dd4e44640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a36482aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dcf7fe640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cade0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00696f0 tx=0x7f4dc402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9530f26e2c370405 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4daf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4daf7fe640 1 -- 172.21.15.22:0/2232907863 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc40051c0 con 0x55a3647bcac0 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dc0005610 con 0x55a3647bcac0 2024-03-08T23:56:35.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4daf7fe640 1 -- 172.21.15.22:0/2232907863 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4dc4005360 con 0x55a3647bcac0 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4daf7fe640 1 -- 172.21.15.22:0/2232907863 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc4005560 con 0x55a3647bcac0 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dd9485740 10 client.25846 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.239+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a36482c1f0 con 0x55a3647bcac0 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4daf7fe640 1 -- 172.21.15.22:0/2232907863 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc0005610 con 0x55a3647bcac0 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4daf7fe640 1 -- 172.21.15.22:0/2232907863 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4dc404c040 con 0x55a3647bcac0 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dd9485740 10 client.25846 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dd9485740 10 client.25846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dd9485740 20 client.25846 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dd9485740 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a36482c920 0x55a36482ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dd9485740 4 client.25846 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a364727750 con 0x55a36482c920 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dcffff640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a36482c920 0x55a36482ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4daf7fe640 10 client.25846 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:35.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.240+0000 7f4dcffff640 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a36482c920 0x55a36482ed40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7eefcbef66ae4c2e server_cookie=466490b5cf2003c4 in_seq=0 out_seq=0 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4daf7fe640 1 -- 172.21.15.22:0/2232907863 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a364727750 con 0x55a36482c920 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4daf7fe640 10 client.25846 handle_command_reply: tid=0 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 client.25846 shutdown 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 msgr2=0x55a3647cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cade0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00696f0 tx=0x7f4dc402bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 shutdown_connections 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a36482c920 0x55a36482ed40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3647c56a0 0x55a36482aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3647bcac0 0x55a3647cade0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4daf7fe640 0 client.25846 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 --2- 172.21.15.22:0/2232907863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3647bc1a0 0x55a3647ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 >> 172.21.15.22:0/2232907863 conn(0x55a3647b68d0 msgr2=0x55a3647b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 shutdown_connections 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 1 -- 172.21.15.22:0/2232907863 wait complete. 2024-03-08T23:56:35.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd9485740 20 client.25846 trim_cache size 0 max 16384 2024-03-08T23:56:35.244 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4db80a29a0 msgr2=0x7f4db80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4db80a29a0 0x7f4db80a4e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4036040 tx=0x7f4dc402c7e0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd0100f90 msgr2=0x7f4dd01a5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.241+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd0100f90 0x7f4dd01a5d50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800eb10 tx=0x7f4dc800c2d0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 shutdown_connections 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4db80a29a0 0x7f4db80a4e60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4dd0100f90 0x7f4dd01a5d50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dd01066b0 0x7f4dd0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 --2- 172.21.15.22:0/1079586821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4dd0069bb0 0x7f4dd0100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 >> 172.21.15.22:0/1079586821 conn(0x7f4dd0076fc0 msgr2=0x7f4dd00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 shutdown_connections 2024-03-08T23:56:35.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.242+0000 7f4dd68a7640 1 -- 172.21.15.22:0/1079586821 wait complete. 2024-03-08T23:56:35.249 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:35.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 -- 172.21.15.22:0/769309855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d0101e90 msgr2=0x7fa4d0105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 --2- 172.21.15.22:0/769309855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d0101e90 0x7fa4d0105ed0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009900 tx=0x7fa4bc02bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 -- 172.21.15.22:0/769309855 shutdown_connections 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 --2- 172.21.15.22:0/769309855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4d01065b0 0x7fa4d010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 --2- 172.21.15.22:0/769309855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d0101e90 0x7fa4d0105ed0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 --2- 172.21.15.22:0/769309855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d01014e0 0x7fa4d01018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 -- 172.21.15.22:0/769309855 >> 172.21.15.22:0/769309855 conn(0x7fa4d00fcda0 msgr2=0x7fa4d00ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 -- 172.21.15.22:0/769309855 shutdown_connections 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 -- 172.21.15.22:0/769309855 wait complete. 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 Processor -- start 2024-03-08T23:56:35.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.362+0000 7fa4d7081640 1 -- start start 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d7081640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d01014e0 0x7fa4d019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d7081640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4d01065b0 0x7fa4d019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d7081640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d0197a50 0x7fa4d0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d7081640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d0111640 con 0x7fa4d01014e0 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d7081640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01114c0 con 0x7fa4d0197a50 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d7081640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01117c0 con 0x7fa4d01065b0 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d01014e0 0x7fa4d019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d01014e0 0x7fa4d019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42506/0 (socket says 172.21.15.22:42506) 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2741509382 learned_addr learned my addr 172.21.15.22:0/2741509382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4cffff640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4d01065b0 0x7fa4d019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d561e640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d0197a50 0x7fa4d0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2741509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4d01065b0 msgr2=0x7fa4d019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4d01065b0 0x7fa4d019dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2741509382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d0197a50 msgr2=0x7fa4d0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d0197a50 0x7fa4d0197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2741509382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0198570 con 0x7fa4d01014e0 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d01014e0 0x7fa4d019d850 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e9f0 tx=0x7fa4c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4cdffb640 1 -- 172.21.15.22:0/2741509382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400cd80 con 0x7fa4d01014e0 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d0198860 con 0x7fa4d01014e0 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4cdffb640 1 -- 172.21.15.22:0/2741509382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4c4010910 con 0x7fa4d01014e0 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.363+0000 7fa4cdffb640 1 -- 172.21.15.22:0/2741509382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40188f0 con 0x7fa4d01014e0 2024-03-08T23:56:35.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.364+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d01a4ab0 con 0x7fa4d01014e0 2024-03-08T23:56:35.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:34 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:56:35.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:34 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:35.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:34 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:56:35.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:34 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:35.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:34 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:56:35.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.366+0000 7fa4cdffb640 1 -- 172.21.15.22:0/2741509382 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa4c4010ac0 con 0x7fa4d01014e0 2024-03-08T23:56:35.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.366+0000 7fa4d9c68740 1 Processor -- start 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.366+0000 7fa4d9c68740 1 -- start start 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.366+0000 7fa4cdffb640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa4c00a2970 0x7fa4c00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d561e640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa4c00a2970 0x7fa4c00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb7d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4cdffb640 1 -- 172.21.15.22:0/2741509382 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa4c40cf5c0 con 0x7fa4d01014e0 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d96a0 0x5582fb7dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582fb797360 con 0x5582fb7d01a0 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582fb733ec0 con 0x5582fb7d96a0 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582fb6dcd30 con 0x5582fb7d0ac0 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb7d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb7d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42514/0 (socket says 172.21.15.22:42514) 2024-03-08T23:56:35.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2417670696 learned_addr learned my addr 172.21.15.22:0/2417670696 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4cffff640 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d561e640 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d96a0 0x5582fb7dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2417670696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 msgr2=0x5582fb7d9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2417670696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d96a0 msgr2=0x5582fb7dba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d96a0 0x5582fb7dba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/2417670696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582fb6dbbc0 con 0x5582fb7d01a0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d561e640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa4c00a2970 0x7fa4c00a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8005fd0 tx=0x7fa4c8011b40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb7d0580 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4098300 tx=0x7fa4c409b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b162012a6d8140 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4baffd640 1 -- 172.21.15.22:0/2417670696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40ccbf0 con 0x5582fb7d01a0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4baffd640 1 -- 172.21.15.22:0/2417670696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4c4097fa0 con 0x5582fb7d01a0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.367+0000 7fa4baffd640 1 -- 172.21.15.22:0/2417670696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c409cbb0 con 0x5582fb7d01a0 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 -- 172.21.15.22:0/2417670696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 msgr2=0x5582fb7d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb7d0580 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4098300 tx=0x7fa4c409b930 comp rx=0 tx=0).stop 2024-03-08T23:56:35.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 -- 172.21.15.22:0/2417670696 shutdown_connections 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d96a0 0x5582fb7dba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/2417670696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb7d0580 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 -- 172.21.15.22:0/2417670696 >> 172.21.15.22:0/2417670696 conn(0x5582fb7ca8d0 msgr2=0x5582fb7cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 -- 172.21.15.22:0/2417670696 shutdown_connections 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.368+0000 7fa4d9c68740 1 -- 172.21.15.22:0/2417670696 wait complete. 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 -- start start 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 Processor -- start 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582fb6d24a0 con 0x5582fb7d0ac0 2024-03-08T23:56:35.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582fb72ff60 con 0x5582fb7d01a0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582fb7dc430 con 0x5582fb7d96a0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d9c68740 10 client.0 _ll_register_callbacks cb 0x5582fb72dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42524/0 (socket says 172.21.15.22:42524) 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 -- 172.21.15.22:0/360192406 learned_addr learned my addr 172.21.15.22:0/360192406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d561e640 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 -- 172.21.15.22:0/360192406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 msgr2=0x5582fb7d7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7d7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 -- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 msgr2=0x5582fb7d6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.369+0000 7fa4cffff640 1 -- 172.21.15.22:0/360192406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582fb4483f0 con 0x5582fb7d0ac0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4cffff640 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6bc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009900 tx=0x7fa4bc02ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371f6d4303ff9a35 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4affff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4affff640 1 -- 172.21.15.22:0/360192406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc004460 con 0x5582fb7d0ac0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4affff640 1 -- 172.21.15.22:0/360192406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa4bc0046c0 con 0x5582fb7d0ac0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4c4097fa0 con 0x5582fb7d0ac0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4affff640 1 -- 172.21.15.22:0/360192406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4bc051050 con 0x5582fb7d0ac0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 10 client.16386 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5582fb5127d0 con 0x5582fb7d0ac0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4affff640 1 -- 172.21.15.22:0/360192406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4c4097fa0 con 0x5582fb7d0ac0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4affff640 1 -- 172.21.15.22:0/360192406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa4bc0035f0 con 0x5582fb7d0ac0 2024-03-08T23:56:35.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 10 client.16386 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 10 client.16386 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 20 client.16386 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb7e97f0 0x5582fb7ebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 4 client.16386 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.370+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582fb741400 con 0x5582fb7e97f0 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.371+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb7e97f0 0x5582fb7ebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.371+0000 7fa4affff640 10 client.16386 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.371+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb7e97f0 0x5582fb7ebbe0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3d1f019a88fdda2 server_cookie=cd60ed5ff78020e8 in_seq=0 out_seq=0 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.371+0000 7fa4affff640 1 -- 172.21.15.22:0/360192406 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582fb741400 con 0x5582fb7e97f0 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.371+0000 7fa4affff640 10 client.16386 handle_command_reply: tid=0 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.371+0000 7fa4d9c68740 1 client.16386 shutdown 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d0ac0 msgr2=0x5582fb7d6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6bc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009900 tx=0x7fa4bc02ce30 comp rx=0 tx=0).stop 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 shutdown_connections 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb7e97f0 0x5582fb7ebbe0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7d7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6bc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4affff640 0 client.16386 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/360192406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 >> 172.21.15.22:0/360192406 conn(0x5582fb7ca8d0 msgr2=0x5582fb7cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 shutdown_connections 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 1 -- 172.21.15.22:0/360192406 wait complete. 2024-03-08T23:56:35.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.372+0000 7fa4d9c68740 20 client.16386 trim_cache size 0 max 16384 2024-03-08T23:56:35.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.383+0000 7fa4d9c68740 1 Processor -- start 2024-03-08T23:56:35.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.383+0000 7fa4d9c68740 1 -- start start 2024-03-08T23:56:35.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d96a0 0x5582fb7e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01b0b40 con 0x5582fb7d96a0 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01ae6e0 con 0x5582fb7d01a0 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01ac780 con 0x5582fb7d0ac0 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d96a0 0x5582fb7e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d96a0 0x5582fb7e0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42536/0 (socket says 172.21.15.22:42536) 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 -- 172.21.15.22:0/3551410507 learned_addr learned my addr 172.21.15.22:0/3551410507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4cffff640 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 -- 172.21.15.22:0/3551410507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 msgr2=0x5582fb7d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 -- 172.21.15.22:0/3551410507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 msgr2=0x5582fb7d8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 -- 172.21.15.22:0/3551410507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582fb7ebf50 con 0x5582fb7d96a0 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4d561e640 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d96a0 0x5582fb7e0670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800fe10 tx=0x7fa4c801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dc7655024a086c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4ba7fc640 1 -- 172.21.15.22:0/3551410507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c801a040 con 0x5582fb7d96a0 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4ba7fc640 1 -- 172.21.15.22:0/3551410507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4c8025070 con 0x5582fb7d96a0 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.384+0000 7fa4ba7fc640 1 -- 172.21.15.22:0/3551410507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8021420 con 0x5582fb7d96a0 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3551410507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d96a0 msgr2=0x5582fb7e0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d96a0 0x5582fb7e0670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800fe10 tx=0x7fa4c801e000 comp rx=0 tx=0).stop 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3551410507 shutdown_connections 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d96a0 0x5582fb7e0670 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d0ac0 0x5582fb7d6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3551410507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d01a0 0x5582fb7d8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3551410507 >> 172.21.15.22:0/3551410507 conn(0x5582fb7ca8d0 msgr2=0x5582fb7d6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3551410507 shutdown_connections 2024-03-08T23:56:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.385+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3551410507 wait complete. 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 -- start start 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 Processor -- start 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb838fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d0ac0 0x5582fb839500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7df8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01acd80 con 0x5582fb7d01a0 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01ae080 con 0x5582fb7d0ac0 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01ad3c0 con 0x5582fb7d96a0 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d9c68740 10 client.0 _ll_register_callbacks cb 0x7fa4c4097bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb838fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb838fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42548/0 (socket says 172.21.15.22:42548) 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/3983430498 learned_addr learned my addr 172.21.15.22:0/3983430498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4cffff640 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d0ac0 0x5582fb839500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d561e640 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7df8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/3983430498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 msgr2=0x5582fb7df8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7df8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d0ac0 msgr2=0x5582fb839500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d0ac0 0x5582fb839500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.386+0000 7fa4d4e1d640 1 -- 172.21.15.22:0/3983430498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582fb7dffb0 con 0x5582fb7d01a0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d4e1d640 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb838fc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa4c409b8c0 tx=0x7fa4c4009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d1e43456754a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4affff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4affff640 1 -- 172.21.15.22:0/3983430498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40cf3e0 con 0x5582fb7d01a0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4affff640 1 -- 172.21.15.22:0/3983430498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa4c40860b0 con 0x5582fb7d01a0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4c8025070 con 0x5582fb7d01a0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4affff640 1 -- 172.21.15.22:0/3983430498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4c4097fa0 con 0x5582fb7d01a0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 10 client.16392 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5582fb841800 con 0x5582fb7d01a0 2024-03-08T23:56:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4affff640 1 -- 172.21.15.22:0/3983430498 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4c8025070 con 0x5582fb7d01a0 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4affff640 1 -- 172.21.15.22:0/3983430498 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa4c409cc20 con 0x5582fb7d01a0 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 10 client.16392 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 10 client.16392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 20 client.16392 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb841af0 0x5582fb843ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 4 client.16392 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.387+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5582fb73b750 con 0x5582fb841af0 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4cffff640 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb841af0 0x5582fb843ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4cffff640 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb841af0 0x5582fb843ee0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2746ec38a561dd6b server_cookie=e50b7e8c97183731 in_seq=0 out_seq=0 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4affff640 10 client.16392 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4affff640 1 -- 172.21.15.22:0/3983430498 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5582fb73b750 con 0x5582fb841af0 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4affff640 10 client.16392 handle_command_reply: tid=0 2024-03-08T23:56:35.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 client.16392 shutdown 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 msgr2=0x5582fb838fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb838fc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa4c409b8c0 tx=0x7fa4c4009700 comp rx=0 tx=0).stop 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 shutdown_connections 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5582fb841af0 0x5582fb843ee0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582fb7d96a0 0x5582fb7df8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5582fb7d0ac0 0x5582fb839500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 --2- 172.21.15.22:0/3983430498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582fb7d01a0 0x5582fb838fc0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4affff640 0 client.16392 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 >> 172.21.15.22:0/3983430498 conn(0x5582fb7ca8d0 msgr2=0x5582fb7c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 shutdown_connections 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 1 -- 172.21.15.22:0/3983430498 wait complete. 2024-03-08T23:56:35.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.388+0000 7fa4d9c68740 20 client.16392 trim_cache size 0 max 16384 2024-03-08T23:56:35.392 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:34 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:56:35.392 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:34 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:35.392 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:34 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:56:35.392 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:34 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:35.392 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:34 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:56:35.392 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:35.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa4c00a2970 msgr2=0x7fa4c00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa4c00a2970 0x7fa4c00a4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8005fd0 tx=0x7fa4c8011b40 comp rx=0 tx=0).stop 2024-03-08T23:56:35.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d01014e0 msgr2=0x7fa4d019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d01014e0 0x7fa4d019d850 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e9f0 tx=0x7fa4c400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 shutdown_connections 2024-03-08T23:56:35.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa4c00a2970 0x7fa4c00a4e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4d0197a50 0x7fa4d0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4d01065b0 0x7fa4d019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 --2- 172.21.15.22:0/2741509382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4d01014e0 0x7fa4d019d850 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 >> 172.21.15.22:0/2741509382 conn(0x7fa4d00fcda0 msgr2=0x7fa4d00ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 shutdown_connections 2024-03-08T23:56:35.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.389+0000 7fa4d7081640 1 -- 172.21.15.22:0/2741509382 wait complete. 2024-03-08T23:56:35.397 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:56:35.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- 172.21.15.22:0/2739124405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4100d90 msgr2=0x7f00a4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 --2- 172.21.15.22:0/2739124405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4100d90 0x7f00a4104dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0090009900 tx=0x7f009002bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- 172.21.15.22:0/2739124405 shutdown_connections 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 --2- 172.21.15.22:0/2739124405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41054b0 0x7f00a410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 --2- 172.21.15.22:0/2739124405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4100d90 0x7f00a4104dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 --2- 172.21.15.22:0/2739124405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a41003e0 0x7f00a41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- 172.21.15.22:0/2739124405 >> 172.21.15.22:0/2739124405 conn(0x7f00a40773a0 msgr2=0x7f00a40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- 172.21.15.22:0/2739124405 shutdown_connections 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- 172.21.15.22:0/2739124405 wait complete. 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 Processor -- start 2024-03-08T23:56:35.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- start start 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41003e0 0x7f00a419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a41054b0 0x7f00a419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4197950 0x7f00a4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4110540 con 0x7f00a41003e0 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41103c0 con 0x7f00a4197950 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.511+0000 7f00ab109640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41106c0 con 0x7f00a41054b0 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41003e0 0x7f00a419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41003e0 0x7f00a419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42572/0 (socket says 172.21.15.22:42572) 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 -- 172.21.15.22:0/2755364174 learned_addr learned my addr 172.21.15.22:0/2755364174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a3fff640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a41054b0 0x7f00a419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a96a6640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4197950 0x7f00a4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 -- 172.21.15.22:0/2755364174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a41054b0 msgr2=0x7f00a419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a41054b0 0x7f00a419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 -- 172.21.15.22:0/2755364174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4197950 msgr2=0x7f00a4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4197950 0x7f00a4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 -- 172.21.15.22:0/2755364174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a4198470 con 0x7f00a41003e0 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41003e0 0x7f00a419d7a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f009800e9f0 tx=0x7f009800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a1ffb640 1 -- 172.21.15.22:0/2755364174 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800cd80 con 0x7f00a41003e0 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a4198760 con 0x7f00a41003e0 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a1ffb640 1 -- 172.21.15.22:0/2755364174 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0098005500 con 0x7f00a41003e0 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00a1ffb640 1 -- 172.21.15.22:0/2755364174 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098010690 con 0x7f00a41003e0 2024-03-08T23:56:35.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.512+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a41a49b0 con 0x7f00a41003e0 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.514+0000 7f00a1ffb640 1 -- 172.21.15.22:0/2755364174 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f00980046c0 con 0x7f00a41003e0 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a1ffb640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00940a2970 0x7f00940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 Processor -- start 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a3fff640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00940a2970 0x7f00940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a1ffb640 1 -- 172.21.15.22:0/2755364174 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f00980cf660 con 0x7f00a41003e0 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 -- start start 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe10580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe19160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe196a0 0x55698fe1ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698fdd7360 con 0x55698fe101a0 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698fd73ec0 con 0x55698fe196a0 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00addc9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698fd1cd30 con 0x55698fe10ac0 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe10580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe10580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42576/0 (socket says 172.21.15.22:42576) 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a8ea5640 1 -- 172.21.15.22:0/3360780152 learned_addr learned my addr 172.21.15.22:0/3360780152 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a3fff640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00940a2970 0x7f00940a4e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0090009230 tx=0x7f0090005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a3fff640 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe19160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.515+0000 7f00a96a6640 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe196a0 0x55698fe1ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a8ea5640 1 -- 172.21.15.22:0/3360780152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 msgr2=0x55698fe19160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe19160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a8ea5640 1 -- 172.21.15.22:0/3360780152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe196a0 msgr2=0x55698fe1ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe196a0 0x55698fe1ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a8ea5640 1 -- 172.21.15.22:0/3360780152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55698fd1bbc0 con 0x55698fe101a0 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a96a6640 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe196a0 0x55698fe1ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a3fff640 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe19160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe10580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0098098290 tx=0x7f009809b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acab758aa6b897d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f008effd640 1 -- 172.21.15.22:0/3360780152 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009809fa10 con 0x55698fe101a0 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f008effd640 1 -- 172.21.15.22:0/3360780152 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00980a4060 con 0x55698fe101a0 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.516+0000 7f008effd640 1 -- 172.21.15.22:0/3360780152 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980cf220 con 0x55698fe101a0 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 -- 172.21.15.22:0/3360780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 msgr2=0x55698fe10580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe10580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0098098290 tx=0x7f009809b2e0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 -- 172.21.15.22:0/3360780152 shutdown_connections 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe196a0 0x55698fe1ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe19160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 --2- 172.21.15.22:0/3360780152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe10580 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 -- 172.21.15.22:0/3360780152 >> 172.21.15.22:0/3360780152 conn(0x55698fe0a8d0 msgr2=0x55698fe0acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 -- 172.21.15.22:0/3360780152 shutdown_connections 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 -- 172.21.15.22:0/3360780152 wait complete. 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 -- start start 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 Processor -- start 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.517+0000 7f00addc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe101a0 0x55698fe16750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe16c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe196a0 0x55698fe171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698fd124a0 con 0x55698fe196a0 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698fd6ff60 con 0x55698fe101a0 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698fe1c490 con 0x55698fe10ac0 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 10 client.0 _ll_register_callbacks cb 0x55698fd6dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:35.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe196a0 0x55698fe171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe196a0 0x55698fe171d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42584/0 (socket says 172.21.15.22:42584) 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 -- 172.21.15.22:0/1194455016 learned_addr learned my addr 172.21.15.22:0/1194455016 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe101a0 0x55698fe16750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a3fff640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe16c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 -- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 msgr2=0x55698fe16c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe16c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 -- 172.21.15.22:0/1194455016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe101a0 msgr2=0x55698fe16750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe101a0 0x55698fe16750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 -- 172.21.15.22:0/1194455016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55698fa883f0 con 0x55698fe196a0 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe101a0 0x55698fe16750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00a96a6640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe196a0 0x55698fe171d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f009c011a00 tx=0x7f009c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cbee0f0fc84fb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f0083fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f0083fff640 1 -- 172.21.15.22:0/1194455016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c00fd10 con 0x55698fe196a0 2024-03-08T23:56:35.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f0083fff640 1 -- 172.21.15.22:0/1194455016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f009c003970 con 0x55698fe196a0 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00980a4060 con 0x55698fe196a0 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f0083fff640 1 -- 172.21.15.22:0/1194455016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f009c025050 con 0x55698fe196a0 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 10 client.16404 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.518+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55698fb527d0 con 0x55698fe196a0 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f0083fff640 1 -- 172.21.15.22:0/1194455016 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00980a4060 con 0x55698fe196a0 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f0083fff640 1 -- 172.21.15.22:0/1194455016 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f009c01a410 con 0x55698fe196a0 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00addc9740 10 client.16404 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00addc9740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00addc9740 20 client.16404 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00addc9740 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe29980 0x55698fe2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00addc9740 4 client.16404 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55698fd81400 con 0x55698fe29980 2024-03-08T23:56:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe29980 0x55698fe2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe29980 0x55698fe2bd70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b973ea1bec61351 server_cookie=39fc0d7f3dc4e422 in_seq=0 out_seq=0 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.519+0000 7f0083fff640 10 client.16404 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f0083fff640 1 -- 172.21.15.22:0/1194455016 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55698fd81400 con 0x55698fe29980 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f0083fff640 10 client.16404 handle_command_reply: tid=0 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 client.16404 shutdown 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe196a0 msgr2=0x55698fe171d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe196a0 0x55698fe171d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f009c011a00 tx=0x7f009c011ec0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 shutdown_connections 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe29980 0x55698fe2bd70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe196a0 0x55698fe171d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe10ac0 0x55698fe16c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 --2- 172.21.15.22:0/1194455016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe101a0 0x55698fe16750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 >> 172.21.15.22:0/1194455016 conn(0x55698fe0a8d0 msgr2=0x55698fe0c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f0083fff640 0 client.16404 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 shutdown_connections 2024-03-08T23:56:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 1 -- 172.21.15.22:0/1194455016 wait complete. 2024-03-08T23:56:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.521+0000 7f00addc9740 20 client.16404 trim_cache size 0 max 16384 2024-03-08T23:56:35.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.543+0000 7f00addc9740 1 Processor -- start 2024-03-08T23:56:35.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00addc9740 1 -- start start 2024-03-08T23:56:35.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00addc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe18e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00addc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00addc9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe20c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00addc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41b0ab0 con 0x55698fe101a0 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00addc9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41ae650 con 0x55698fe10ac0 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00addc9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41ac6f0 con 0x55698fe196a0 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe18e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00a96a6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe18e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42588/0 (socket says 172.21.15.22:42588) 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00a96a6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe20c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35096/0 (socket says 172.21.15.22:35096) 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00a8ea5640 1 -- 172.21.15.22:0/1360815428 learned_addr learned my addr 172.21.15.22:0/1360815428 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.544+0000 7f00a3fff640 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.545+0000 7f00a8ea5640 1 -- 172.21.15.22:0/1360815428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 msgr2=0x55698fe20c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.545+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe20c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.545+0000 7f00a8ea5640 1 -- 172.21.15.22:0/1360815428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 msgr2=0x55698fe206c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.545+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe206c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.545+0000 7f00a8ea5640 1 -- 172.21.15.22:0/1360815428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55698fe0f290 con 0x55698fe101a0 2024-03-08T23:56:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.545+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe18e80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f009809b5c0 tx=0x7f0098005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e24cb86cada16a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.546+0000 7f008e7fc640 1 -- 172.21.15.22:0/1360815428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980a4060 con 0x55698fe101a0 2024-03-08T23:56:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.546+0000 7f008e7fc640 1 -- 172.21.15.22:0/1360815428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0098097f30 con 0x55698fe101a0 2024-03-08T23:56:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.546+0000 7f008e7fc640 1 -- 172.21.15.22:0/1360815428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980a3bd0 con 0x55698fe101a0 2024-03-08T23:56:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 -- 172.21.15.22:0/1360815428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 msgr2=0x55698fe18e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe18e80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f009809b5c0 tx=0x7f0098005760 comp rx=0 tx=0).stop 2024-03-08T23:56:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 -- 172.21.15.22:0/1360815428 shutdown_connections 2024-03-08T23:56:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe20c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe206c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 --2- 172.21.15.22:0/1360815428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe18e80 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 -- 172.21.15.22:0/1360815428 >> 172.21.15.22:0/1360815428 conn(0x55698fe0a8d0 msgr2=0x55698fe306d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 -- 172.21.15.22:0/1360815428 shutdown_connections 2024-03-08T23:56:35.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.547+0000 7f00addc9740 1 -- 172.21.15.22:0/1360815428 wait complete. 2024-03-08T23:56:35.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.548+0000 7f00addc9740 1 -- start start 2024-03-08T23:56:35.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.548+0000 7f00addc9740 1 Processor -- start 2024-03-08T23:56:35.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00addc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe1e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00addc9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe1ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00addc9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe1f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00addc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41accf0 con 0x55698fe101a0 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe1e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a8ea5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe1e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42598/0 (socket says 172.21.15.22:42598) 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a8ea5640 1 -- 172.21.15.22:0/2885010142 learned_addr learned my addr 172.21.15.22:0/2885010142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41adff0 con 0x55698fe10ac0 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41ad330 con 0x55698fe196a0 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a3fff640 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe1ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00addc9740 10 client.0 _ll_register_callbacks cb 0x7f00980056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a96a6640 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe1f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a3fff640 1 -- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 msgr2=0x55698fe1f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a3fff640 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe1f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a3fff640 1 -- 172.21.15.22:0/2885010142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 msgr2=0x55698fe1e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a3fff640 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe1e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.549+0000 7f00a3fff640 1 -- 172.21.15.22:0/2885010142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55698fe8fe60 con 0x55698fe10ac0 2024-03-08T23:56:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.550+0000 7f0083fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.550+0000 7f00a3fff640 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe1ebf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0090036040 tx=0x7f009002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e473add13c85ea server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.550+0000 7f0083fff640 1 -- 172.21.15.22:0/2885010142 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009005b060 con 0x55698fe10ac0 2024-03-08T23:56:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.551+0000 7f0083fff640 1 -- 172.21.15.22:0/2885010142 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0090004ec0 con 0x55698fe10ac0 2024-03-08T23:56:35.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.551+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0098097f30 con 0x55698fe10ac0 2024-03-08T23:56:35.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.552+0000 7f0083fff640 1 -- 172.21.15.22:0/2885010142 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00900050c0 con 0x55698fe10ac0 2024-03-08T23:56:35.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.552+0000 7f00addc9740 10 client.25892 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.552+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55698fe91130 con 0x55698fe10ac0 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f0083fff640 1 -- 172.21.15.22:0/2885010142 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0098097f30 con 0x55698fe10ac0 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f0083fff640 1 -- 172.21.15.22:0/2885010142 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f009004d440 con 0x55698fe10ac0 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f00addc9740 10 client.25892 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f00addc9740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f00addc9740 20 client.25892 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f00addc9740 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe91840 0x55698fe93c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f00addc9740 4 client.25892 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55698fd7b750 con 0x55698fe91840 2024-03-08T23:56:35.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.553+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe91840 0x55698fe93c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:35.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.556+0000 7f0083fff640 10 client.25892 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:35.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.556+0000 7f00a8ea5640 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe91840 0x55698fe93c60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb22e3c2f2dc4d5 server_cookie=26586b52547d36e in_seq=0 out_seq=0 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f0083fff640 1 -- 172.21.15.22:0/2885010142 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55698fd7b750 con 0x55698fe91840 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f0083fff640 10 client.25892 handle_command_reply: tid=0 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 client.25892 shutdown 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 msgr2=0x55698fe1ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe1ebf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0090036040 tx=0x7f009002bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 shutdown_connections 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55698fe91840 0x55698fe93c60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f0083fff640 0 client.25892 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55698fe196a0 0x55698fe1f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55698fe10ac0 0x55698fe1ebf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 --2- 172.21.15.22:0/2885010142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55698fe101a0 0x55698fe1e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.561+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 >> 172.21.15.22:0/2885010142 conn(0x55698fe0a8d0 msgr2=0x55698fe11a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 shutdown_connections 2024-03-08T23:56:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00addc9740 1 -- 172.21.15.22:0/2885010142 wait complete. 2024-03-08T23:56:35.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00addc9740 20 client.25892 trim_cache size 0 max 16384 2024-03-08T23:56:35.559 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00940a2970 msgr2=0x7f00940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00940a2970 0x7f00940a4e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0090009230 tx=0x7f0090005fd0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41003e0 msgr2=0x7f00a419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41003e0 0x7f00a419d7a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f009800e9f0 tx=0x7f009800edf0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 shutdown_connections 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00940a2970 0x7f00940a4e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a4197950 0x7f00a4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a41054b0 0x7f00a419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 --2- 172.21.15.22:0/2755364174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00a41003e0 0x7f00a419d7a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:35.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 >> 172.21.15.22:0/2755364174 conn(0x7f00a40773a0 msgr2=0x7f00a4068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 shutdown_connections 2024-03-08T23:56:35.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:35.562+0000 7f00ab109640 1 -- 172.21.15.22:0/2755364174 wait complete. 2024-03-08T23:56:35.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 110820 14.29 MB/sec warmup 43 sec latency 11.232 ms 2024-03-08T23:56:36.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:35 smithi022 ceph-mon[31894]: pgmap v65: 129 pgs: 129 active+clean; 60 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 525 op/s 2024-03-08T23:56:36.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:35 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1563759185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:36.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:35 smithi062 ceph-mon[36129]: pgmap v65: 129 pgs: 129 active+clean; 60 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 525 op/s 2024-03-08T23:56:36.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:35 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1563759185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:36.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:35 smithi192 ceph-mon[35995]: pgmap v65: 129 pgs: 129 active+clean; 60 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 525 op/s 2024-03-08T23:56:36.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:35 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1563759185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:56:36.570 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:56:36.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:56:36.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:56:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 -- 172.21.15.22:0/1533472062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd0102460 msgr2=0x7f0cd0102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/1533472062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd0102460 0x7f0cd0102840 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009900 tx=0x7f0cc002bcd0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 -- 172.21.15.22:0/1533472062 shutdown_connections 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/1533472062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cd010b830 0x7f0cd010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/1533472062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd0102d80 0x7f0cd010b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/1533472062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd0102460 0x7f0cd0102840 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 -- 172.21.15.22:0/1533472062 >> 172.21.15.22:0/1533472062 conn(0x7f0cd00fbc00 msgr2=0x7f0cd00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 -- 172.21.15.22:0/1533472062 shutdown_connections 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.688+0000 7f0cd55a2640 1 -- 172.21.15.22:0/1533472062 wait complete. 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 Processor -- start 2024-03-08T23:56:36.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 -- start start 2024-03-08T23:56:36.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cd0102460 0x7f0cd01000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd0102d80 0x7f0cd01005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd010b830 0x7f0cd0100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01115c0 con 0x7f0cd010b830 2024-03-08T23:56:36.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0111440 con 0x7f0cd0102d80 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd55a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0111740 con 0x7f0cd0102460 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd4da1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd010b830 0x7f0cd0100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd4da1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd010b830 0x7f0cd0100b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42610/0 (socket says 172.21.15.22:42610) 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0cd4da1640 1 -- 172.21.15.22:0/4141505430 learned_addr learned my addr 172.21.15.22:0/4141505430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0ccffff640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cd0102460 0x7f0cd01000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.689+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd0102d80 0x7f0cd01005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd4da1640 1 -- 172.21.15.22:0/4141505430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cd0102460 msgr2=0x7f0cd01000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd4da1640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cd0102460 0x7f0cd01000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd4da1640 1 -- 172.21.15.22:0/4141505430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd0102d80 msgr2=0x7f0cd01005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd4da1640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd0102d80 0x7f0cd01005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd4da1640 1 -- 172.21.15.22:0/4141505430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd01a6000 con 0x7f0cd010b830 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd0102d80 0x7f0cd01005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd4da1640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd010b830 0x7f0cd0100b30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800eb10 tx=0x7f0cc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0ccd7fa640 1 -- 172.21.15.22:0/4141505430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8004b20 con 0x7f0cd010b830 2024-03-08T23:56:36.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd01a62f0 con 0x7f0cd010b830 2024-03-08T23:56:36.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0ccd7fa640 1 -- 172.21.15.22:0/4141505430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cc8004cc0 con 0x7f0cd010b830 2024-03-08T23:56:36.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0ccd7fa640 1 -- 172.21.15.22:0/4141505430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8010760 con 0x7f0cd010b830 2024-03-08T23:56:36.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.690+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd01a6890 con 0x7f0cd010b830 2024-03-08T23:56:36.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.692+0000 7f0ccd7fa640 1 -- 172.21.15.22:0/4141505430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0cc8005020 con 0x7f0cd010b830 2024-03-08T23:56:36.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 Processor -- start 2024-03-08T23:56:36.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 -- start start 2024-03-08T23:56:36.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c025390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c0308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30bf88b40 con 0x55b30c02e4b0 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30bf31d80 con 0x55b30c024fb0 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30bf84be0 con 0x55b30c0258d0 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c0308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c0308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42612/0 (socket says 172.21.15.22:42612) 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/1110598216 learned_addr learned my addr 172.21.15.22:0/1110598216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccffff640 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c025390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccd7fa640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cb80a2970 0x7f0cb80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0cd4da1640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cb80a2970 0x7f0cb80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccffff640 1 -- 172.21.15.22:0/1110598216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 msgr2=0x55b30c025390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccffff640 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c025390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccffff640 1 -- 172.21.15.22:0/1110598216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 msgr2=0x55b30c0308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccffff640 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c0308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.693+0000 7f0ccffff640 1 -- 172.21.15.22:0/1110598216 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b30bd657e0 con 0x55b30c0258d0 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.694+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c0308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.694+0000 7f0ccd7fa640 1 -- 172.21.15.22:0/4141505430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0cc80cf900 con 0x7f0cd010b830 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.694+0000 7f0ccffff640 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02df70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0000c00 tx=0x7f0cc0009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d2a3b018a7bf81 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.694+0000 7f0cd4da1640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cb80a2970 0x7f0cb80a4e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8098000 tx=0x7f0cc806a9b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.694+0000 7f0cb67fc640 1 -- 172.21.15.22:0/1110598216 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0042070 con 0x55b30c0258d0 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.694+0000 7f0cb67fc640 1 -- 172.21.15.22:0/1110598216 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0004040 con 0x55b30c0258d0 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.694+0000 7f0cb67fc640 1 -- 172.21.15.22:0/1110598216 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0004c80 con 0x55b30c0258d0 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1110598216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 msgr2=0x55b30c02df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02df70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0000c00 tx=0x7f0cc0009560 comp rx=0 tx=0).stop 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1110598216 shutdown_connections 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c0308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02df70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1110598216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c025390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1110598216 >> 172.21.15.22:0/1110598216 conn(0x55b30c01f6e0 msgr2=0x55b30c01fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1110598216 shutdown_connections 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1110598216 wait complete. 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 -- start start 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 Processor -- start 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.695+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30c031240 con 0x55b30c02e4b0 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30bf451c0 con 0x55b30c024fb0 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30c0313c0 con 0x55b30c0258d0 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0cd93e9740 10 client.0 _ll_register_callbacks cb 0x55b30bf347a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35142/0 (socket says 172.21.15.22:35142) 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 -- 172.21.15.22:0/1064785269 learned_addr learned my addr 172.21.15.22:0/1064785269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 -- 172.21.15.22:0/1064785269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 msgr2=0x55b30c02b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 -- 172.21.15.22:0/1064785269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 msgr2=0x55b30c02bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 -- 172.21.15.22:0/1064785269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b30bdc5f70 con 0x55b30c0258d0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0ccffff640 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02b850 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009900 tx=0x7f0cc0004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccec236912645a0a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0caf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1064785269 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0042070 con 0x55b30c0258d0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cc0004040 con 0x55b30c0258d0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.696+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1064785269 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0cc002cb30 con 0x55b30c0258d0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1064785269 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cc002cd30 con 0x55b30c0258d0 2024-03-08T23:56:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 10 client.25894 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b30bc5d170 con 0x55b30c0258d0 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1064785269 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0004040 con 0x55b30c0258d0 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1064785269 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cc004f030 con 0x55b30c0258d0 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 10 client.25894 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 20 client.25894 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c03e490 0x55b30c040880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 4 client.25894 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b30bf5e960 con 0x55b30c03e490 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.697+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c03e490 0x55b30c040880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0caf7fe640 10 client.25894 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c03e490 0x55b30c040880 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359ccc67bbed085b server_cookie=ade84880224341d in_seq=0 out_seq=0 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1064785269 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b30bf5e960 con 0x55b30c03e490 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0caf7fe640 10 client.25894 handle_command_reply: tid=0 2024-03-08T23:56:36.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 client.25894 shutdown 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 msgr2=0x55b30c02b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02b850 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009900 tx=0x7f0cc0004cb0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 shutdown_connections 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c03e490 0x55b30c040880 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02b850 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0caf7fe640 0 client.25894 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/1064785269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 >> 172.21.15.22:0/1064785269 conn(0x55b30c01f6e0 msgr2=0x55b30c021430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 shutdown_connections 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.698+0000 7f0cd93e9740 1 -- 172.21.15.22:0/1064785269 wait complete. 2024-03-08T23:56:36.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.699+0000 7f0cd93e9740 20 client.25894 trim_cache size 0 max 16384 2024-03-08T23:56:36.705 INFO:tasks.workunit.client.0.smithi022.stdout: 1 113886 14.38 MB/sec warmup 44 sec latency 12.079 ms 2024-03-08T23:56:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 Processor -- start 2024-03-08T23:56:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 -- start start 2024-03-08T23:56:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01b2a70 con 0x55b30c02e4b0 2024-03-08T23:56:36.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01b0610 con 0x55b30c024fb0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01ae6b0 con 0x55b30c0258d0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51016/0 (socket says 172.21.15.22:51016) 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/3067467669 learned_addr learned my addr 172.21.15.22:0/3067467669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.716+0000 7f0ccffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/3067467669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 msgr2=0x55b30c02a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/3067467669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/3067467669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 msgr2=0x55b30c02bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/3067467669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/3067467669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b30c024980 con 0x55b30c024fb0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/3067467669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02da40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4000c00 tx=0x7f0cc4009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d0d33744725ed0a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0cb5ffb640 1 -- 172.21.15.22:0/3067467669 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40258e0 con 0x55b30c024fb0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0cb5ffb640 1 -- 172.21.15.22:0/3067467669 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cc400f7f0 con 0x55b30c024fb0 2024-03-08T23:56:36.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.717+0000 7f0cb5ffb640 1 -- 172.21.15.22:0/3067467669 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40036c0 con 0x55b30c024fb0 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- 172.21.15.22:0/3067467669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 msgr2=0x55b30c02da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/3067467669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02da40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4000c00 tx=0x7f0cc4009f70 comp rx=0 tx=0).stop 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- 172.21.15.22:0/3067467669 shutdown_connections 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/3067467669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c02bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/3067467669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c0258d0 0x55b30c02a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/3067467669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c024fb0 0x55b30c02da40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- 172.21.15.22:0/3067467669 >> 172.21.15.22:0/3067467669 conn(0x55b30c01f6e0 msgr2=0x55b30c029be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- 172.21.15.22:0/3067467669 shutdown_connections 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- 172.21.15.22:0/3067467669 wait complete. 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- start start 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 Processor -- start 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c024fb0 0x55b30c035220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c0258d0 0x55b30c035760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c033760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01aecb0 con 0x55b30c02e4b0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01affb0 con 0x55b30c0258d0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01af2f0 con 0x55b30c024fb0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.718+0000 7f0cd93e9740 10 client.0 _ll_register_callbacks cb 0x7f0cc80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c033760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c024fb0 0x55b30c035220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c024fb0 0x55b30c035220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35152/0 (socket says 172.21.15.22:35152) 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/2722810374 learned_addr learned my addr 172.21.15.22:0/2722810374 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccffff640 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c0258d0 0x55b30c035760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/2722810374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c0258d0 msgr2=0x55b30c035760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c0258d0 0x55b30c035760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/2722810374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 msgr2=0x55b30c033760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c033760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 -- 172.21.15.22:0/2722810374 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b30c033ea0 con 0x55b30c024fb0 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccffff640 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c0258d0 0x55b30c035760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:36.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0ccf7fe640 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c024fb0 0x55b30c035220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009530 tx=0x7f0cc4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9068de1d6af967 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0caf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0caf7fe640 1 -- 172.21.15.22:0/2722810374 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4005240 con 0x55b30c024fb0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0caf7fe640 1 -- 172.21.15.22:0/2722810374 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0cc40053e0 con 0x55b30c024fb0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cc400f7f0 con 0x55b30c024fb0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0caf7fe640 1 -- 172.21.15.22:0/2722810374 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cc4043050 con 0x55b30c024fb0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0cd93e9740 10 client.25900 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.719+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b30c0950a0 con 0x55b30c024fb0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0caf7fe640 1 -- 172.21.15.22:0/2722810374 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cc400f7f0 con 0x55b30c024fb0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0caf7fe640 1 -- 172.21.15.22:0/2722810374 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cc403e420 con 0x55b30c024fb0 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0cd93e9740 10 client.25900 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0cd93e9740 10 client.25900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0cd93e9740 20 client.25900 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c095710 0x55b30c097b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0cd93e9740 4 client.25900 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:56:36.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b30bf4a080 con 0x55b30c095710 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0ccffff640 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c095710 0x55b30c097b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0ccffff640 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c095710 0x55b30c097b00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cec738b5b7b2ebd server_cookie=bba7cff79df0d35e in_seq=0 out_seq=0 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.720+0000 7f0caf7fe640 10 client.25900 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.721+0000 7f0caf7fe640 1 -- 172.21.15.22:0/2722810374 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b30bf4a080 con 0x55b30c095710 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.721+0000 7f0caf7fe640 10 client.25900 handle_command_reply: tid=0 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.721+0000 7f0cd93e9740 1 client.25900 shutdown 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c024fb0 msgr2=0x55b30c035220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c024fb0 0x55b30c035220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009530 tx=0x7f0cc4005f90 comp rx=0 tx=0).stop 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 shutdown_connections 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b30c095710 0x55b30c097b00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b30c02e4b0 0x55b30c033760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30c0258d0 0x55b30c035760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 --2- 172.21.15.22:0/2722810374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b30c024fb0 0x55b30c035220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0caf7fe640 0 client.25900 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 >> 172.21.15.22:0/2722810374 conn(0x55b30c01f6e0 msgr2=0x55b30c020870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 shutdown_connections 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 1 -- 172.21.15.22:0/2722810374 wait complete. 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd93e9740 20 client.25900 trim_cache size 0 max 16384 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "479b2f36-6dc9-4914-abf6-885573b07e36", 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:56:36.724 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cb80a2970 msgr2=0x7f0cb80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cb80a2970 0x7f0cb80a4e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8098000 tx=0x7f0cc806a9b0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd010b830 msgr2=0x7f0cd0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.722+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd010b830 0x7f0cd0100b30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800eb10 tx=0x7f0cc800c2d0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 shutdown_connections 2024-03-08T23:56:36.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cb80a2970 0x7f0cb80a4e30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cd010b830 0x7f0cd0100b30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd0102d80 0x7f0cd01005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 --2- 172.21.15.22:0/4141505430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cd0102460 0x7f0cd01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 >> 172.21.15.22:0/4141505430 conn(0x7f0cd00fbc00 msgr2=0x7f0cd00fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 shutdown_connections 2024-03-08T23:56:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.723+0000 7f0cd55a2640 1 -- 172.21.15.22:0/4141505430 wait complete. 2024-03-08T23:56:36.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:56:36.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:56:36.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 -- 172.21.15.22:0/1736376002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488100d80 msgr2=0x7fc488104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 --2- 172.21.15.22:0/1736376002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488100d80 0x7fc488104dc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc470009900 tx=0x7fc47002bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:36.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 -- 172.21.15.22:0/1736376002 shutdown_connections 2024-03-08T23:56:36.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 --2- 172.21.15.22:0/1736376002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4881054a0 0x7fc48810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 --2- 172.21.15.22:0/1736376002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488100d80 0x7fc488104dc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 --2- 172.21.15.22:0/1736376002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4881003d0 0x7fc4881007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 -- 172.21.15.22:0/1736376002 >> 172.21.15.22:0/1736376002 conn(0x7fc4880772f0 msgr2=0x7fc488077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 -- 172.21.15.22:0/1736376002 shutdown_connections 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 -- 172.21.15.22:0/1736376002 wait complete. 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 Processor -- start 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.842+0000 7fc48d57e640 1 -- start start 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc48d57e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4881003d0 0x7fc48819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc48d57e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4881054a0 0x7fc48819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc48d57e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc488197990 0x7fc488197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc48d57e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc488110530 con 0x7fc4881003d0 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc48d57e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881103b0 con 0x7fc4881054a0 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc48d57e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881106b0 con 0x7fc488197990 2024-03-08T23:56:36.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4881003d0 0x7fc48819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4881003d0 0x7fc48819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42670/0 (socket says 172.21.15.22:42670) 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 -- 172.21.15.22:0/2525693522 learned_addr learned my addr 172.21.15.22:0/2525693522 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc4867fc640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4881054a0 0x7fc48819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc488197990 0x7fc488197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 -- 172.21.15.22:0/2525693522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc488197990 msgr2=0x7fc488197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc488197990 0x7fc488197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 -- 172.21.15.22:0/2525693522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4881054a0 msgr2=0x7fc48819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4881054a0 0x7fc48819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 -- 172.21.15.22:0/2525693522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4881984b0 con 0x7fc4881003d0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4881003d0 0x7fc48819d7e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc48000eab0 tx=0x7fc48000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc47ffff640 1 -- 172.21.15.22:0/2525693522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48000ce40 con 0x7fc4881003d0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc47ffff640 1 -- 172.21.15.22:0/2525693522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc480011950 con 0x7fc4881003d0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4881987a0 con 0x7fc4881003d0 2024-03-08T23:56:36.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.843+0000 7fc47ffff640 1 -- 172.21.15.22:0/2525693522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480010ae0 con 0x7fc4881003d0 2024-03-08T23:56:36.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.844+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4881a49f0 con 0x7fc4881003d0 2024-03-08T23:56:36.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc47ffff640 1 -- 172.21.15.22:0/2525693522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc480011b00 con 0x7fc4881003d0 2024-03-08T23:56:36.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 Processor -- start 2024-03-08T23:56:36.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 -- start start 2024-03-08T23:56:36.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc47ffff640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc4740a2970 0x7fc4740a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca1c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca29b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca2a070 0x55d53ca2c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53c97fe00 con 0x55d53ca2a070 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53c928b10 con 0x55d53ca1c490 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc490238740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53c97bea0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.846+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc4740a2970 0x7fc4740a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca29b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca2a070 0x55d53ca2c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc47ffff640 1 -- 172.21.15.22:0/2525693522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc4800cf830 con 0x7fc4881003d0 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca29b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35180/0 (socket says 172.21.15.22:35180) 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca2a070 0x55d53ca2c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42676/0 (socket says 172.21.15.22:42676) 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 -- 172.21.15.22:0/1002190816 learned_addr learned my addr 172.21.15.22:0/1002190816 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc486ffd640 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca1c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 -- 172.21.15.22:0/1002190816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 msgr2=0x55d53ca1c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca1c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 -- 172.21.15.22:0/1002190816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca2a070 msgr2=0x55d53ca2c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca2a070 0x55d53ca2c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 -- 172.21.15.22:0/1002190816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53c75e7d0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc4740a2970 0x7fc4740a4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc478005f60 tx=0x7fc478009ae0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc4867fc640 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca29b30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc48806aa50 tx=0x7fc470009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd94220e1a6700c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc47d7fa640 1 -- 172.21.15.22:0/1002190816 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470042070 con 0x55d53ca1cdb0 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc47d7fa640 1 -- 172.21.15.22:0/1002190816 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc47002ca80 con 0x55d53ca1cdb0 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.847+0000 7fc47d7fa640 1 -- 172.21.15.22:0/1002190816 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470004c00 con 0x55d53ca1cdb0 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 -- 172.21.15.22:0/1002190816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 msgr2=0x55d53ca29b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca29b30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc48806aa50 tx=0x7fc470009610 comp rx=0 tx=0).stop 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 -- 172.21.15.22:0/1002190816 shutdown_connections 2024-03-08T23:56:36.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca2a070 0x55d53ca2c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca29b30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 --2- 172.21.15.22:0/1002190816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca1c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 -- 172.21.15.22:0/1002190816 >> 172.21.15.22:0/1002190816 conn(0x55d53ca16e90 msgr2=0x55d53ca17270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 -- 172.21.15.22:0/1002190816 shutdown_connections 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.848+0000 7fc490238740 1 -- 172.21.15.22:0/1002190816 wait complete. 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 -- start start 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 Processor -- start 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1c490 0x55d53ca297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca26b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca27080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53ca2cdd0 con 0x55d53ca1c490 2024-03-08T23:56:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53c93c0a0 con 0x55d53ca2a070 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53ca2cf50 con 0x55d53ca1cdb0 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc490238740 10 client.0 _ll_register_callbacks cb 0x55d53c93ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1c490 0x55d53ca297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1c490 0x55d53ca297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42678/0 (socket says 172.21.15.22:42678) 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 -- 172.21.15.22:0/718619164 learned_addr learned my addr 172.21.15.22:0/718619164 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca26b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc4877fe640 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 -- 172.21.15.22:0/718619164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 msgr2=0x55d53ca26b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca26b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 -- 172.21.15.22:0/718619164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 msgr2=0x55d53ca27080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca27080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc486ffd640 1 -- 172.21.15.22:0/718619164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d53c6943f0 con 0x55d53ca1c490 2024-03-08T23:56:36.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.849+0000 7fc4877fe640 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca27080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc486ffd640 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1c490 0x55d53ca297b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c700 tx=0x7fc48009b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7769bd93a2e7fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc46e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc46e7fc640 1 -- 172.21.15.22:0/718619164 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800ae2c0 con 0x55d53ca1c490 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc46e7fc640 1 -- 172.21.15.22:0/718619164 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc480098160 con 0x55d53ca1c490 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc47002ca80 con 0x55d53ca1c490 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc46e7fc640 1 -- 172.21.15.22:0/718619164 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc48009baf0 con 0x55d53ca1c490 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 10 client.16440 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d53c6540f0 con 0x55d53ca1c490 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc46e7fc640 1 -- 172.21.15.22:0/718619164 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc47002ca80 con 0x55d53ca1c490 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc46e7fc640 1 -- 172.21.15.22:0/718619164 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc48000f040 con 0x55d53ca1c490 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 10 client.16440 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:36.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 20 client.16440 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca39a90 0x55d53ca3be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 4 client.16440 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d53c98d340 con 0x55d53ca39a90 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.850+0000 7fc4867fc640 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca39a90 0x55d53ca3be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.851+0000 7fc46e7fc640 10 client.16440 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.851+0000 7fc4867fc640 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca39a90 0x55d53ca3be80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e9d75c56cffe0c server_cookie=8bebbfe8955bcdc6 in_seq=0 out_seq=0 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc46e7fc640 1 -- 172.21.15.22:0/718619164 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d53c98d340 con 0x55d53ca39a90 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc46e7fc640 10 client.16440 handle_command_reply: tid=0 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 client.16440 shutdown 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1c490 msgr2=0x55d53ca297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1c490 0x55d53ca297b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c700 tx=0x7fc48009b510 comp rx=0 tx=0).stop 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 shutdown_connections 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca39a90 0x55d53ca3be80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca27080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc46e7fc640 0 client.16440 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1cdb0 0x55d53ca26b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 --2- 172.21.15.22:0/718619164 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1c490 0x55d53ca297b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 >> 172.21.15.22:0/718619164 conn(0x55d53ca16e90 msgr2=0x55d53ca18be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 shutdown_connections 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 1 -- 172.21.15.22:0/718619164 wait complete. 2024-03-08T23:56:36.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.852+0000 7fc490238740 20 client.16440 trim_cache size 0 max 16384 2024-03-08T23:56:36.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 Processor -- start 2024-03-08T23:56:36.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 -- start start 2024-03-08T23:56:36.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1c490 0x55d53ca29270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca21ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881ae6f0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881ac290 con 0x55d53ca2a070 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc490238740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881aa330 con 0x55d53ca1c490 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.867+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1c490 0x55d53ca29270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1c490 0x55d53ca29270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35204/0 (socket says 172.21.15.22:35204) 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42680/0 (socket says 172.21.15.22:42680) 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc486ffd640 1 -- 172.21.15.22:0/2564396340 learned_addr learned my addr 172.21.15.22:0/2564396340 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca21ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4867fc640 1 -- 172.21.15.22:0/2564396340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1c490 msgr2=0x55d53ca29270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4867fc640 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1c490 0x55d53ca29270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4867fc640 1 -- 172.21.15.22:0/2564396340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 msgr2=0x55d53ca21ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4867fc640 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca21ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4867fc640 1 -- 172.21.15.22:0/2564396340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53ca1cbe0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc4867fc640 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca297b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc470036040 tx=0x7fc47002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4458b007a3b8055f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc47cff9640 1 -- 172.21.15.22:0/2564396340 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470042040 con 0x55d53ca1cdb0 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc47cff9640 1 -- 172.21.15.22:0/2564396340 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc47002ccb0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.868+0000 7fc47cff9640 1 -- 172.21.15.22:0/2564396340 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47004e3d0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 -- 172.21.15.22:0/2564396340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 msgr2=0x55d53ca297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca297b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc470036040 tx=0x7fc47002bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 -- 172.21.15.22:0/2564396340 shutdown_connections 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca2a070 0x55d53ca21ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca297b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 --2- 172.21.15.22:0/2564396340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca1c490 0x55d53ca29270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 -- 172.21.15.22:0/2564396340 >> 172.21.15.22:0/2564396340 conn(0x55d53ca16e90 msgr2=0x55d53ca25410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 -- 172.21.15.22:0/2564396340 shutdown_connections 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 -- 172.21.15.22:0/2564396340 wait complete. 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 -- start start 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.869+0000 7fc490238740 1 Processor -- start 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca2dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca2e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca2a070 0x55d53ca2e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881aa930 con 0x55d53ca1cdb0 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881abc30 con 0x55d53ca1c490 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881aaf70 con 0x55d53ca2a070 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 10 client.0 _ll_register_callbacks cb 0x7fc480097d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca2e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca2e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42684/0 (socket says 172.21.15.22:42684) 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 -- 172.21.15.22:0/451515596 learned_addr learned my addr 172.21.15.22:0/451515596 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc486ffd640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca2dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4877fe640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca2a070 0x55d53ca2e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 -- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca2a070 msgr2=0x55d53ca2e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca2a070 0x55d53ca2e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 -- 172.21.15.22:0/451515596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 msgr2=0x55d53ca2dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca2dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 -- 172.21.15.22:0/451515596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d53ca90020 con 0x55d53ca1cdb0 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc486ffd640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca2dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc4867fc640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca2e240 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc47002cfd0 tx=0x7fc47002ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61461342fa1fceb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc46e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc46e7fc640 1 -- 172.21.15.22:0/451515596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470057060 con 0x55d53ca1cdb0 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc46e7fc640 1 -- 172.21.15.22:0/451515596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc470004030 con 0x55d53ca1cdb0 2024-03-08T23:56:36.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.870+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc47002ccb0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc46e7fc640 1 -- 172.21.15.22:0/451515596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc47005e050 con 0x55d53ca1cdb0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 10 client.16446 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d53ca91330 con 0x55d53ca1cdb0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc46e7fc640 1 -- 172.21.15.22:0/451515596 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc47002ccb0 con 0x55d53ca1cdb0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc46e7fc640 1 -- 172.21.15.22:0/451515596 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc470003680 con 0x55d53ca1cdb0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 10 client.16446 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 20 client.16446 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca915b0 0x55d53ca939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 4 client.16446 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d53c987690 con 0x55d53ca915b0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.871+0000 7fc486ffd640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca915b0 0x55d53ca939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.872+0000 7fc486ffd640 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca915b0 0x55d53ca939a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9252e9383951b578 server_cookie=9b35d6f76a9334cb in_seq=0 out_seq=0 2024-03-08T23:56:36.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.872+0000 7fc46e7fc640 10 client.16446 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc46e7fc640 1 -- 172.21.15.22:0/451515596 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d53c987690 con 0x55d53ca915b0 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc46e7fc640 10 client.16446 handle_command_reply: tid=0 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 client.16446 shutdown 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 msgr2=0x55d53ca2e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca2e240 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc47002cfd0 tx=0x7fc47002ca80 comp rx=0 tx=0).stop 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 shutdown_connections 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d53ca915b0 0x55d53ca939a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d53ca2a070 0x55d53ca2e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc46e7fc640 0 client.16446 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d53ca1cdb0 0x55d53ca2e240 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 --2- 172.21.15.22:0/451515596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d53ca1c490 0x55d53ca2dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.873+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 >> 172.21.15.22:0/451515596 conn(0x55d53ca16e90 msgr2=0x55d53ca24fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 shutdown_connections 2024-03-08T23:56:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc490238740 1 -- 172.21.15.22:0/451515596 wait complete. 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc490238740 20 client.16446 trim_cache size 0 max 16384 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: "479b2f36-6dc9-4914-abf6-885573b07e36": { 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "479b2f36-6dc9-4914-abf6-885573b07e36", 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:56:36.877 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:56:36.878 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:56:36.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc4740a2970 msgr2=0x7fc4740a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc4740a2970 0x7fc4740a4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc478005f60 tx=0x7fc478009ae0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4881003d0 msgr2=0x7fc48819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4881003d0 0x7fc48819d7e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc48000eab0 tx=0x7fc48000ef70 comp rx=0 tx=0).stop 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 shutdown_connections 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc4740a2970 0x7fc4740a4e30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc488197990 0x7fc488197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4881054a0 0x7fc48819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 --2- 172.21.15.22:0/2525693522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4881003d0 0x7fc48819d7e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 >> 172.21.15.22:0/2525693522 conn(0x7fc4880772f0 msgr2=0x7fc488068860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 shutdown_connections 2024-03-08T23:56:36.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:56:36.874+0000 7fc48d57e640 1 -- 172.21.15.22:0/2525693522 wait complete. 2024-03-08T23:56:36.881 INFO:tasks.cephfs.filesystem:scrub status for tag:479b2f36-6dc9-4914-abf6-885573b07e36 - {'path': '/', 'tag': '479b2f36-6dc9-4914-abf6-885573b07e36', 'options': 'recursive,force'} 2024-03-08T23:56:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 -- 172.21.15.22:0/1860143364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310106320 msgr2=0x7f2310106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1860143364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310106320 0x7f2310106700 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2308009cd0 tx=0x7f230802c140 comp rx=0 tx=0).stop 2024-03-08T23:56:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 -- 172.21.15.22:0/1860143364 shutdown_connections 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1860143364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310100de0 0x7f231010cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1860143364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2310100440 0x7f23101008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1860143364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310106320 0x7f2310106700 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 -- 172.21.15.22:0/1860143364 >> 172.21.15.22:0/1860143364 conn(0x7f2310077030 msgr2=0x7f2310077440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 -- 172.21.15.22:0/1860143364 shutdown_connections 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.113+0000 7f2314a4a640 1 -- 172.21.15.22:0/1860143364 wait complete. 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 Processor -- start 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 -- start start 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310100440 0x7f23101966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2310100de0 0x7f2310196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310106320 0x7f2310197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2310110520 con 0x7f2310100440 2024-03-08T23:56:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101103a0 con 0x7f2310106320 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f2314a4a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101106a0 con 0x7f2310100de0 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310100440 0x7f23101966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310100440 0x7f23101966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42696/0 (socket says 172.21.15.22:42696) 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f230e59c640 1 -- 172.21.15.22:0/1229465380 learned_addr learned my addr 172.21.15.22:0/1229465380 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f230dd9b640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2310100de0 0x7f2310196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.114+0000 7f230ed9d640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310106320 0x7f2310197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f230e59c640 1 -- 172.21.15.22:0/1229465380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2310100de0 msgr2=0x7f2310196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f230e59c640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2310100de0 0x7f2310196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f230e59c640 1 -- 172.21.15.22:0/1229465380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310106320 msgr2=0x7f2310197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f230e59c640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310106320 0x7f2310197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f230e59c640 1 -- 172.21.15.22:0/1229465380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23101a5e90 con 0x7f2310100440 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f230dd9b640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2310100de0 0x7f2310196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f230e59c640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310100440 0x7f23101966b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2308009df0 tx=0x7f2308034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f23077fe640 1 -- 172.21.15.22:0/1229465380 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308039050 con 0x7f2310100440 2024-03-08T23:56:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f23077fe640 1 -- 172.21.15.22:0/1229465380 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2308045070 con 0x7f2310100440 2024-03-08T23:56:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23101a6120 con 0x7f2310100440 2024-03-08T23:56:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f23077fe640 1 -- 172.21.15.22:0/1229465380 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308004570 con 0x7f2310100440 2024-03-08T23:56:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.115+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23101a6660 con 0x7f2310100440 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.117+0000 7f23077fe640 1 -- 172.21.15.22:0/1229465380 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2308004710 con 0x7f2310100440 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.117+0000 7f2317620740 1 Processor -- start 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f2317620740 1 -- start start 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f2317620740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9da6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f2317620740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9da7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f2317620740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9db2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f2317620740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f9d0b5f0 con 0x5595f9da6fd0 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f2317620740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f9cb4930 con 0x5595f9db0550 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f2317620740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f9d07690 con 0x5595f9da7d40 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9da7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f23077fe640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f22fc0a2a90 0x7f22fc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9da7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42708/0 (socket says 172.21.15.22:42708) 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 -- 172.21.15.22:0/922555720 learned_addr learned my addr 172.21.15.22:0/922555720 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9da6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230dd9b640 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9db2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230ed9d640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f22fc0a2a90 0x7f22fc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f23077fe640 1 -- 172.21.15.22:0/1229465380 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f23080f1920 con 0x7f2310100440 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 -- 172.21.15.22:0/922555720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 msgr2=0x5595f9da6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9da6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 -- 172.21.15.22:0/922555720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 msgr2=0x5595f9db2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9db2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230e59c640 1 -- 172.21.15.22:0/922555720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f9a20210 con 0x5595f9da6fd0 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.118+0000 7f230ed9d640 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9da6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.119+0000 7f230e59c640 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9da7450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23080eecf0 tx=0x7f23080095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662359d89d337318 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.119+0000 7f230ed9d640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f22fc0a2a90 0x7f22fc0a4f50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009710 tx=0x7f22f8004730 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.119+0000 7f2304ff9640 1 -- 172.21.15.22:0/922555720 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308034d90 con 0x5595f9da6fd0 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.119+0000 7f2304ff9640 1 -- 172.21.15.22:0/922555720 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23080f3e50 con 0x5595f9da6fd0 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.119+0000 7f2304ff9640 1 -- 172.21.15.22:0/922555720 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080ba300 con 0x5595f9da6fd0 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 -- 172.21.15.22:0/922555720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 msgr2=0x5595f9da7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9da7450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23080eecf0 tx=0x7f23080095c0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 -- 172.21.15.22:0/922555720 shutdown_connections 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9db2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9da7450 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 --2- 172.21.15.22:0/922555720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9da6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 -- 172.21.15.22:0/922555720 >> 172.21.15.22:0/922555720 conn(0x5595f9d6e5b0 msgr2=0x5595f9da1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 -- 172.21.15.22:0/922555720 shutdown_connections 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 -- 172.21.15.22:0/922555720 wait complete. 2024-03-08T23:56:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 -- start start 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.120+0000 7f2317620740 1 Processor -- start 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dadb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f9db3040 con 0x5595f9da6fd0 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f9cc7d90 con 0x5595f9db0550 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f9db31c0 con 0x5595f9da7d40 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 10 client.0 _ll_register_callbacks cb 0x5595f9a1fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dadb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dadb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35242/0 (socket says 172.21.15.22:35242) 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 -- 172.21.15.22:0/2657855126 learned_addr learned my addr 172.21.15.22:0/2657855126 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230e59c640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 -- 172.21.15.22:0/2657855126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 msgr2=0x5595f9dae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 -- 172.21.15.22:0/2657855126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 msgr2=0x5595f9dad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 -- 172.21.15.22:0/2657855126 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595f9aec620 con 0x5595f9da7d40 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230ed9d640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230e59c640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f230dd9b640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dadb00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2300012d50 tx=0x7f230000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44523f8cd64c90ca server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f22f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f22f5ffb640 1 -- 172.21.15.22:0/2657855126 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300012110 con 0x5595f9da7d40 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.121+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23080f3e50 con 0x5595f9da7d40 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f22f5ffb640 1 -- 172.21.15.22:0/2657855126 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2300004050 con 0x5595f9da7d40 2024-03-08T23:56:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f22f5ffb640 1 -- 172.21.15.22:0/2657855126 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23000122b0 con 0x5595f9da7d40 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 10 client.25930 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5595f9dbf360 con 0x5595f9da7d40 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f22f5ffb640 1 -- 172.21.15.22:0/2657855126 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23080f3e50 con 0x5595f9da7d40 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f22f5ffb640 1 -- 172.21.15.22:0/2657855126 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2300010430 con 0x5595f9da7d40 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 10 client.25930 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 20 client.25930 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9dc07d0 0x5595f9dc2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 4 client.25930 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595f9d18b30 con 0x5595f9dc07d0 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.122+0000 7f230e59c640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9dc07d0 0x5595f9dc2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.123+0000 7f230e59c640 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9dc07d0 0x5595f9dc2bc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd795ddb001f958f server_cookie=3fc7f88f44000cb4 in_seq=0 out_seq=0 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.123+0000 7f22f5ffb640 10 client.25930 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f22f5ffb640 1 -- 172.21.15.22:0/2657855126 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595f9d18b30 con 0x5595f9dc07d0 2024-03-08T23:56:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f22f5ffb640 10 client.25930 handle_command_reply: tid=0 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 client.25930 shutdown 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 msgr2=0x5595f9dadb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dadb00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2300012d50 tx=0x7f230000f790 comp rx=0 tx=0).stop 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 shutdown_connections 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9dc07d0 0x5595f9dc2bc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dadb00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 --2- 172.21.15.22:0/2657855126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f22f5ffb640 0 client.25930 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 >> 172.21.15.22:0/2657855126 conn(0x5595f9d6e5b0 msgr2=0x5595f9da3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 shutdown_connections 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 1 -- 172.21.15.22:0/2657855126 wait complete. 2024-03-08T23:56:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.124+0000 7f2317620740 20 client.25930 trim_cache size 0 max 16384 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 Processor -- start 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 -- start start 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dafcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dabe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101b2b30 con 0x5595f9da6fd0 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101b06d0 con 0x5595f9db0550 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f2317620740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101ae770 con 0x5595f9da7d40 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dafcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dafcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42724/0 (socket says 172.21.15.22:42724) 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230ed9d640 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 -- 172.21.15.22:0/2781675735 learned_addr learned my addr 172.21.15.22:0/2781675735 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230dd9b640 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dabe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 -- 172.21.15.22:0/2781675735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 msgr2=0x5595f9dabe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dabe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 -- 172.21.15.22:0/2781675735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 msgr2=0x5595f9dac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 -- 172.21.15.22:0/2781675735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f9da62a0 con 0x5595f9da6fd0 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230ed9d640 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.138+0000 7f230e59c640 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dafcf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f23080ba0f0 tx=0x7f2308000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67953a7cce124ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f22f7fff640 1 -- 172.21.15.22:0/2781675735 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080bd5f0 con 0x5595f9da6fd0 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f22f7fff640 1 -- 172.21.15.22:0/2781675735 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f230803a040 con 0x5595f9da6fd0 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f22f7fff640 1 -- 172.21.15.22:0/2781675735 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080c1de0 con 0x5595f9da6fd0 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f2317620740 1 -- 172.21.15.22:0/2781675735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 msgr2=0x5595f9dafcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f2317620740 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dafcf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f23080ba0f0 tx=0x7f2308000f40 comp rx=0 tx=0).stop 2024-03-08T23:56:37.139 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f2317620740 1 -- 172.21.15.22:0/2781675735 shutdown_connections 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f2317620740 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9db0550 0x5595f9dac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f2317620740 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9da7d40 0x5595f9dabe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f2317620740 1 --2- 172.21.15.22:0/2781675735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9dafcf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.139+0000 7f2317620740 1 -- 172.21.15.22:0/2781675735 >> 172.21.15.22:0/2781675735 conn(0x5595f9d6e5b0 msgr2=0x5595f9da17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 -- 172.21.15.22:0/2781675735 shutdown_connections 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 -- 172.21.15.22:0/2781675735 wait complete. 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 -- start start 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 Processor -- start 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9db59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9da7d40 0x5595f9db5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9db0550 0x5595f9db6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101aed70 con 0x5595f9da6fd0 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101b0070 con 0x5595f9da7d40 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101af3b0 con 0x5595f9db0550 2024-03-08T23:56:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f2317620740 10 client.0 _ll_register_callbacks cb 0x7f23080bda60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9db59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f230e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9db59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42732/0 (socket says 172.21.15.22:42732) 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f230e59c640 1 -- 172.21.15.22:0/973129928 learned_addr learned my addr 172.21.15.22:0/973129928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f230ed9d640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9db0550 0x5595f9db6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.140+0000 7f230dd9b640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9da7d40 0x5595f9db5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f230e59c640 1 -- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9db0550 msgr2=0x5595f9db6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f230e59c640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9db0550 0x5595f9db6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f230e59c640 1 -- 172.21.15.22:0/973129928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9da7d40 msgr2=0x5595f9db5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f230e59c640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9da7d40 0x5595f9db5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f230e59c640 1 -- 172.21.15.22:0/973129928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595f9db7640 con 0x5595f9da6fd0 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f230dd9b640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9da7d40 0x5595f9db5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f230e59c640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9db59f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f23080bd990 tx=0x7f23080c6430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490b1d5e4d3ee8b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f22f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f22f5ffb640 1 -- 172.21.15.22:0/973129928 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080d5030 con 0x5595f9da6fd0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f22f5ffb640 1 -- 172.21.15.22:0/973129928 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f23080a8040 con 0x5595f9da6fd0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f230803a040 con 0x5595f9da6fd0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f22f5ffb640 1 -- 172.21.15.22:0/973129928 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23080bd5f0 con 0x5595f9da6fd0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f2317620740 10 client.16464 fetch_fsmap learned FSMap version 27 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5595f9e17850 con 0x5595f9da6fd0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.141+0000 7f22f5ffb640 1 -- 172.21.15.22:0/973129928 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f230803a040 con 0x5595f9da6fd0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f22f5ffb640 1 -- 172.21.15.22:0/973129928 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f23080bdf80 con 0x5595f9da6fd0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f2317620740 10 client.16464 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f2317620740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f2317620740 20 client.16464 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f2317620740 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9e17aa0 0x5595f9e19e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f2317620740 4 client.16464 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5595f9d12e80 con 0x5595f9e17aa0 2024-03-08T23:56:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f230dd9b640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9e17aa0 0x5595f9e19e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f230dd9b640 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9e17aa0 0x5595f9e19e90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cde5bdb214945cf server_cookie=faffc8e7f1c29ea9 in_seq=0 out_seq=0 2024-03-08T23:56:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.142+0000 7f22f5ffb640 10 client.16464 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f22f5ffb640 1 -- 172.21.15.22:0/973129928 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5595f9d12e80 con 0x5595f9e17aa0 2024-03-08T23:56:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f22f5ffb640 10 client.16464 handle_command_reply: tid=0 2024-03-08T23:56:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 client.16464 shutdown 2024-03-08T23:56:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 msgr2=0x5595f9db59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9db59f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f23080bd990 tx=0x7f23080c6430 comp rx=0 tx=0).stop 2024-03-08T23:56:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 shutdown_connections 2024-03-08T23:56:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f9e17aa0 0x5595f9e19e90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f9db0550 0x5595f9db6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f9da7d40 0x5595f9db5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 --2- 172.21.15.22:0/973129928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f9da6fd0 0x5595f9db59f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f22f5ffb640 0 client.16464 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.365+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 >> 172.21.15.22:0/973129928 conn(0x5595f9d6e5b0 msgr2=0x5595f9da17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 shutdown_connections 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2317620740 1 -- 172.21.15.22:0/973129928 wait complete. 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2317620740 20 client.16464 trim_cache size 0 max 16384 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:56:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f22fc0a2a90 msgr2=0x7f22fc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f22fc0a2a90 0x7f22fc0a4f50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009710 tx=0x7f22f8004730 comp rx=0 tx=0).stop 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310100440 msgr2=0x7f23101966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310100440 0x7f23101966b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2308009df0 tx=0x7f2308034670 comp rx=0 tx=0).stop 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 shutdown_connections 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f22fc0a2a90 0x7f22fc0a4f50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310106320 0x7f2310197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2310100de0 0x7f2310196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 --2- 172.21.15.22:0/1229465380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2310100440 0x7f23101966b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 >> 172.21.15.22:0/1229465380 conn(0x7f2310077030 msgr2=0x7f23100ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 shutdown_connections 2024-03-08T23:56:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:37.366+0000 7f2314a4a640 1 -- 172.21.15.22:0/1229465380 wait complete. 2024-03-08T23:56:37.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 115941 14.22 MB/sec warmup 45 sec latency 116.503 ms 2024-03-08T23:56:38.111 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setuser ceph since I am not root 2024-03-08T23:56:38.111 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setgroup ceph since I am not root 2024-03-08T23:56:38.111 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-mds[70095]: -- 172.21.15.62:0/344954255 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564cb0c5da40 con 0x564cb0c31000 2024-03-08T23:56:38.111 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: starting mds.h at 2024-03-08T23:56:38.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:56:38.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:38.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:56:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-mon[36129]: pgmap v66: 129 pgs: 129 active+clean; 60 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2024-03-08T23:56:38.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:37 smithi062 ceph-mon[36129]: daemon mds.h restarted 2024-03-08T23:56:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:37 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:56:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:37 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:37 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:56:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:37 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:37 smithi022 ceph-mon[31894]: pgmap v66: 129 pgs: 129 active+clean; 60 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2024-03-08T23:56:38.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:37 smithi022 ceph-mon[31894]: daemon mds.h restarted 2024-03-08T23:56:38.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:37 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:56:38.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:37 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:38.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:37 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:56:38.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:37 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:56:38.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:37 smithi192 ceph-mon[35995]: pgmap v66: 129 pgs: 129 active+clean; 60 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2024-03-08T23:56:38.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:37 smithi192 ceph-mon[35995]: daemon mds.h restarted 2024-03-08T23:56:38.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 118003 14.19 MB/sec warmup 46 sec latency 24.328 ms 2024-03-08T23:56:39.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:38 smithi022 ceph-mon[31894]: osdmap e96: 12 total, 12 up, 12 in 2024-03-08T23:56:39.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:38 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6836/3083291387,v1:172.21.15.62:6837/3083291387] up:boot 2024-03-08T23:56:39.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:38 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:56:39.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:38 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:56:39.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:38 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:56:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:38 smithi062 ceph-mon[36129]: osdmap e96: 12 total, 12 up, 12 in 2024-03-08T23:56:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:38 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6836/3083291387,v1:172.21.15.62:6837/3083291387] up:boot 2024-03-08T23:56:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:38 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:56:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:38 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:56:39.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:38 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:56:39.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:38 smithi192 ceph-mon[35995]: osdmap e96: 12 total, 12 up, 12 in 2024-03-08T23:56:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:38 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6836/3083291387,v1:172.21.15.62:6837/3083291387] up:boot 2024-03-08T23:56:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:38 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:56:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:38 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:56:39.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:38 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:56:39.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 120617 14.24 MB/sec warmup 47 sec latency 77.565 ms 2024-03-08T23:56:40.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:39 smithi022 ceph-mon[31894]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-03-08T23:56:40.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:39 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:56:40.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:39 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:56:40.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:39 smithi062 ceph-mon[36129]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-03-08T23:56:40.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:39 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:56:40.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:39 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:56:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:39 smithi192 ceph-mon[35995]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-03-08T23:56:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:39 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:56:40.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:39 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:56:40.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 122611 14.10 MB/sec warmup 48 sec latency 12.805 ms 2024-03-08T23:56:41.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:40 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:56:41.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:40 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:56:41.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:40 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:56:41.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:40 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:56:41.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:40 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:56:41.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:40 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:56:41.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:40 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:56:41.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:40 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:56:41.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:40 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:56:41.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 124922 14.09 MB/sec warmup 49 sec latency 17.912 ms 2024-03-08T23:56:42.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:42 smithi062 ceph-mon[36129]: pgmap v69: 129 pgs: 129 active+clean; 51 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 540 op/s 2024-03-08T23:56:42.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:42 smithi192 ceph-mon[35995]: pgmap v69: 129 pgs: 129 active+clean; 51 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 540 op/s 2024-03-08T23:56:42.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 127990 14.18 MB/sec warmup 50 sec latency 8.145 ms 2024-03-08T23:56:42.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:42 smithi022 ceph-mon[31894]: pgmap v69: 129 pgs: 129 active+clean; 51 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 540 op/s 2024-03-08T23:56:43.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 130622 14.14 MB/sec warmup 51 sec latency 10.322 ms 2024-03-08T23:56:44.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:44 smithi062 ceph-mon[36129]: pgmap v70: 129 pgs: 129 active+clean; 51 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 540 op/s 2024-03-08T23:56:44.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:44 smithi192 ceph-mon[35995]: pgmap v70: 129 pgs: 129 active+clean; 51 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 540 op/s 2024-03-08T23:56:44.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 132014 14.02 MB/sec warmup 52 sec latency 90.715 ms 2024-03-08T23:56:44.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:44 smithi022 ceph-mon[31894]: pgmap v70: 129 pgs: 129 active+clean; 51 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 540 op/s 2024-03-08T23:56:45.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 134907 14.08 MB/sec warmup 53 sec latency 68.415 ms 2024-03-08T23:56:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:46 smithi062 ceph-mon[36129]: pgmap v71: 129 pgs: 129 active+clean; 47 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2024-03-08T23:56:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:46 smithi192 ceph-mon[35995]: pgmap v71: 129 pgs: 129 active+clean; 47 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2024-03-08T23:56:46.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 137222 13.98 MB/sec warmup 54 sec latency 12.717 ms 2024-03-08T23:56:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:46 smithi022 ceph-mon[31894]: pgmap v71: 129 pgs: 129 active+clean; 47 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2024-03-08T23:56:47.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 139251 13.95 MB/sec warmup 55 sec latency 13.640 ms 2024-03-08T23:56:48.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:48 smithi062 ceph-mon[36129]: pgmap v72: 129 pgs: 129 active+clean; 47 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2024-03-08T23:56:48.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:48 smithi192 ceph-mon[35995]: pgmap v72: 129 pgs: 129 active+clean; 47 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2024-03-08T23:56:48.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 142293 14.03 MB/sec warmup 56 sec latency 7.971 ms 2024-03-08T23:56:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:48 smithi022 ceph-mon[31894]: pgmap v72: 129 pgs: 129 active+clean; 47 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2024-03-08T23:56:49.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 144300 13.91 MB/sec warmup 57 sec latency 151.933 ms 2024-03-08T23:56:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 -- 172.21.15.22:0/4087885019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8101530 msgr2=0x7f8bc8101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 --2- 172.21.15.22:0/4087885019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8101530 0x7f8bc8101910 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009900 tx=0x7f8bb802bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 -- 172.21.15.22:0/4087885019 shutdown_connections 2024-03-08T23:56:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 --2- 172.21.15.22:0/4087885019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc8106600 0x7f8bc810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 --2- 172.21.15.22:0/4087885019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc8101ee0 0x7f8bc8105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 --2- 172.21.15.22:0/4087885019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8101530 0x7f8bc8101910 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 -- 172.21.15.22:0/4087885019 >> 172.21.15.22:0/4087885019 conn(0x7f8bc80fcdd0 msgr2=0x7f8bc80ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 -- 172.21.15.22:0/4087885019 shutdown_connections 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 -- 172.21.15.22:0/4087885019 wait complete. 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 Processor -- start 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.456+0000 7f8bce53b640 1 -- start start 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bce53b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc8101530 0x7f8bc819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bce53b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8106600 0x7f8bc819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bce53b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc81979a0 0x7f8bc8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bce53b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8111700 con 0x7f8bc8106600 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bce53b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8111580 con 0x7f8bc81979a0 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bce53b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8111880 con 0x7f8bc8101530 2024-03-08T23:56:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8106600 0x7f8bc819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8106600 0x7f8bc819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56160/0 (socket says 172.21.15.22:56160) 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 -- 172.21.15.22:0/2792059325 learned_addr learned my addr 172.21.15.22:0/2792059325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc8101530 0x7f8bc819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bccad8640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc81979a0 0x7f8bc8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 -- 172.21.15.22:0/2792059325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc8101530 msgr2=0x7f8bc819d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc8101530 0x7f8bc819d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 -- 172.21.15.22:0/2792059325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc81979a0 msgr2=0x7f8bc8197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc81979a0 0x7f8bc8197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 -- 172.21.15.22:0/2792059325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc81984c0 con 0x7f8bc8106600 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bccad8640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc81979a0 0x7f8bc8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8106600 0x7f8bc819dce0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e9f0 tx=0x7f8bbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc57fa640 1 -- 172.21.15.22:0/2792059325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00ccc0 con 0x7f8bc8106600 2024-03-08T23:56:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bc57fa640 1 -- 172.21.15.22:0/2792059325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc005500 con 0x7f8bc8106600 2024-03-08T23:56:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.457+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc81987b0 con 0x7f8bc8106600 2024-03-08T23:56:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.458+0000 7f8bc57fa640 1 -- 172.21.15.22:0/2792059325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc010690 con 0x7f8bc8106600 2024-03-08T23:56:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.458+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81a4960 con 0x7f8bc8106600 2024-03-08T23:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bc57fa640 1 -- 172.21.15.22:0/2792059325 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8bbc0046c0 con 0x7f8bc8106600 2024-03-08T23:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 Processor -- start 2024-03-08T23:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 -- start start 2024-03-08T23:56:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e50d40 0x564368e4fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e50450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e59550 0x564368e5b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368db45f0 con 0x564368e59550 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368d5d930 con 0x564368e4ffd0 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368db0690 con 0x564368e50d40 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.460+0000 7f8bc57fa640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8bb00a2a10 0x7f8bb00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e50450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e50450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:46456/0 (socket says 172.21.15.22:46456) 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 -- 172.21.15.22:0/1108627401 learned_addr learned my addr 172.21.15.22:0/1108627401 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e59550 0x564368e5b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bccad8640 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e50d40 0x564368e4fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bccad8640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8bb00a2a10 0x7f8bb00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc57fa640 1 -- 172.21.15.22:0/2792059325 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8bbc0cf680 con 0x7f8bc8106600 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 -- 172.21.15.22:0/1108627401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e50d40 msgr2=0x564368e4fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e50d40 0x564368e4fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 -- 172.21.15.22:0/1108627401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e59550 msgr2=0x564368e5b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e59550 0x564368e5b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 -- 172.21.15.22:0/1108627401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564368ac9210 con 0x564368e4ffd0 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e50450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009900 tx=0x7f8bb8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6cec389c5d73ea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bccad8640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8bb00a2a10 0x7f8bb00a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005c40 tx=0x7f8bc0014040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bae7fc640 1 -- 172.21.15.22:0/1108627401 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8045070 con 0x564368e4ffd0 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bae7fc640 1 -- 172.21.15.22:0/1108627401 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bb802c650 con 0x564368e4ffd0 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.461+0000 7f8bae7fc640 1 -- 172.21.15.22:0/1108627401 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80049e0 con 0x564368e4ffd0 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1108627401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 msgr2=0x564368e50450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e50450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009900 tx=0x7f8bb8009560 comp rx=0 tx=0).stop 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1108627401 shutdown_connections 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e59550 0x564368e5b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e50450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1108627401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e50d40 0x564368e4fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1108627401 >> 172.21.15.22:0/1108627401 conn(0x564368e175b0 msgr2=0x564368e4acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1108627401 shutdown_connections 2024-03-08T23:56:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1108627401 wait complete. 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.462+0000 7f8bd11f6740 1 -- start start 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 1 Processor -- start 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e56b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368e5c220 con 0x564368e50d40 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368d70d90 con 0x564368e4ffd0 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368e5c3a0 con 0x564368e59550 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bd11f6740 10 client.0 _ll_register_callbacks cb 0x564368ac8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e56b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e56b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56180/0 (socket says 172.21.15.22:56180) 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc77fe640 1 -- 172.21.15.22:0/1559930847 learned_addr learned my addr 172.21.15.22:0/1559930847 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bccad8640 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc7fff640 1 -- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 msgr2=0x564368e57060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e57060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc7fff640 1 -- 172.21.15.22:0/1559930847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 msgr2=0x564368e56b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e56b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc7fff640 1 -- 172.21.15.22:0/1559930847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564368b95620 con 0x564368e4ffd0 2024-03-08T23:56:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e56b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e565e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8bb802c240 tx=0x7f8bb802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c5e722907ee33d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.463+0000 7f8ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8ba77fe640 1 -- 172.21.15.22:0/1559930847 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8045040 con 0x564368e4ffd0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8ba77fe640 1 -- 172.21.15.22:0/1559930847 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8bb80050c0 con 0x564368e4ffd0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bb802c650 con 0x564368e4ffd0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8ba77fe640 1 -- 172.21.15.22:0/1559930847 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bb8059050 con 0x564368e4ffd0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 10 client.25964 fetch_fsmap learned FSMap version 29 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564368e68560 con 0x564368e4ffd0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8ba77fe640 1 -- 172.21.15.22:0/1559930847 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bb802c650 con 0x564368e4ffd0 2024-03-08T23:56:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8ba77fe640 1 -- 172.21.15.22:0/1559930847 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8bb804aca0 con 0x564368e4ffd0 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 10 client.25964 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 10 client.25964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 20 client.25964 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368e698b0 0x564368e6bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 4 client.25964 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564368dc1b30 con 0x564368e698b0 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.464+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368e698b0 0x564368e6bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.465+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368e698b0 0x564368e6bca0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c55a9bb66d19ac server_cookie=cd3f66a145db708f in_seq=0 out_seq=0 2024-03-08T23:56:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.465+0000 7f8ba77fe640 10 client.25964 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.465+0000 7f8ba77fe640 1 -- 172.21.15.22:0/1559930847 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564368dc1b30 con 0x564368e698b0 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.465+0000 7f8ba77fe640 10 client.25964 handle_command_reply: tid=0 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.465+0000 7f8bd11f6740 1 client.25964 shutdown 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.465+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 msgr2=0x564368e565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.465+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e565e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8bb802c240 tx=0x7f8bb802bc30 comp rx=0 tx=0).stop 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 shutdown_connections 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368e698b0 0x564368e6bca0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e57060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e56b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/1559930847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e565e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8ba77fe640 0 client.25964 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 >> 172.21.15.22:0/1559930847 conn(0x564368e175b0 msgr2=0x564368e4ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 shutdown_connections 2024-03-08T23:56:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 1 -- 172.21.15.22:0/1559930847 wait complete. 2024-03-08T23:56:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.466+0000 7f8bd11f6740 20 client.25964 trim_cache size 0 max 16384 2024-03-08T23:56:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 Processor -- start 2024-03-08T23:56:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 -- start start 2024-03-08T23:56:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e58d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e60610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e60b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81b0950 con 0x564368e50d40 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81ae4f0 con 0x564368e4ffd0 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81ac590 con 0x564368e59550 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e60610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e60610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56182/0 (socket says 172.21.15.22:56182) 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3323890473 learned_addr learned my addr 172.21.15.22:0/3323890473 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e58d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bccad8640 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e60b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3323890473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 msgr2=0x564368e60b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e60b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3323890473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 msgr2=0x564368e58d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e58d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3323890473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564368e4f2a0 con 0x564368e50d40 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.480+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e60610 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0a3b50 tx=0x7f8bbc0d1ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4153f8f2daab5734 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8badffb640 1 -- 172.21.15.22:0/3323890473 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc09b8e0 con 0x564368e50d40 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8badffb640 1 -- 172.21.15.22:0/3323890473 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc09fa30 con 0x564368e50d40 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8badffb640 1 -- 172.21.15.22:0/3323890473 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0cf460 con 0x564368e50d40 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3323890473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 msgr2=0x564368e60610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e60610 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0a3b50 tx=0x7f8bbc0d1ad0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3323890473 shutdown_connections 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e59550 0x564368e60b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e60610 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3323890473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e4ffd0 0x564368e58d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3323890473 >> 172.21.15.22:0/3323890473 conn(0x564368e175b0 msgr2=0x564368e702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3323890473 shutdown_connections 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.481+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3323890473 wait complete. 2024-03-08T23:56:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 -- start start 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 Processor -- start 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e4ffd0 0x564368e608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e5cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e59550 0x564368e5d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81acb90 con 0x564368e50d40 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81ade90 con 0x564368e59550 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81ad1d0 con 0x564368e4ffd0 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bd11f6740 10 client.0 _ll_register_callbacks cb 0x7f8bbc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e5cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e5cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56186/0 (socket says 172.21.15.22:56186) 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3227515758 learned_addr learned my addr 172.21.15.22:0/3227515758 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e4ffd0 0x564368e608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bccad8640 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e59550 0x564368e5d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e4ffd0 msgr2=0x564368e608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e4ffd0 0x564368e608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3227515758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e59550 msgr2=0x564368e5d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e59550 0x564368e5d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.482+0000 7f8bc77fe640 1 -- 172.21.15.22:0/3227515758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564368e563b0 con 0x564368e50d40 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bc77fe640 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e5cc30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0cc970 tx=0x7f8bbc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47216bae29c479db server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8ba77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8ba77fe640 1 -- 172.21.15.22:0/3227515758 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0cf200 con 0x564368e50d40 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8ba77fe640 1 -- 172.21.15.22:0/3227515758 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8bbc0cf3a0 con 0x564368e50d40 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bbc09fa30 con 0x564368e50d40 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8ba77fe640 1 -- 172.21.15.22:0/3227515758 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bbc09bb40 con 0x564368e50d40 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 10 client.16488 fetch_fsmap learned FSMap version 29 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564368ec07e0 con 0x564368e50d40 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8ba77fe640 1 -- 172.21.15.22:0/3227515758 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc09fa30 con 0x564368e50d40 2024-03-08T23:56:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8ba77fe640 1 -- 172.21.15.22:0/3227515758 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8bbc09ad30 con 0x564368e50d40 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 10 client.16488 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 20 client.16488 populate_metadata read hostname 'smithi022' 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368ec0ec0 0x564368ec32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 4 client.16488 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564368dbbe80 con 0x564368ec0ec0 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.483+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368ec0ec0 0x564368ec32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.484+0000 7f8bc7fff640 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368ec0ec0 0x564368ec32e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada132c92d984864 server_cookie=8f759d7d2798dbd4 in_seq=0 out_seq=0 2024-03-08T23:56:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.484+0000 7f8ba77fe640 10 client.16488 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:50.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:50 smithi062 ceph-mon[36129]: pgmap v73: 129 pgs: 129 active+clean; 51 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 582 op/s 2024-03-08T23:56:50.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:50 smithi192 ceph-mon[35995]: pgmap v73: 129 pgs: 129 active+clean; 51 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 582 op/s 2024-03-08T23:56:50.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 145904 13.88 MB/sec warmup 58 sec latency 160.262 ms 2024-03-08T23:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.737+0000 7f8ba77fe640 1 -- 172.21.15.22:0/3227515758 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564368dbbe80 con 0x564368ec0ec0 2024-03-08T23:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.737+0000 7f8ba77fe640 10 client.16488 handle_command_reply: tid=0 2024-03-08T23:56:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.737+0000 7f8bd11f6740 1 client.16488 shutdown 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 msgr2=0x564368e5cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e5cc30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0cc970 tx=0x7f8bbc0057b0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 shutdown_connections 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564368ec0ec0 0x564368ec32e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564368e59550 0x564368e5d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564368e50d40 0x564368e5cc30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8ba77fe640 0 client.16488 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 --2- 172.21.15.22:0/3227515758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564368e4ffd0 0x564368e608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 >> 172.21.15.22:0/3227515758 conn(0x564368e175b0 msgr2=0x564368e4a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 shutdown_connections 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 1 -- 172.21.15.22:0/3227515758 wait complete. 2024-03-08T23:56:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.738+0000 7f8bd11f6740 20 client.16488 trim_cache size 0 max 16384 2024-03-08T23:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:56:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:56:50.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:50 smithi022 ceph-mon[31894]: pgmap v73: 129 pgs: 129 active+clean; 51 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 582 op/s 2024-03-08T23:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8bb00a2a10 msgr2=0x7f8bb00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8bb00a2a10 0x7f8bb00a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005c40 tx=0x7f8bc0014040 comp rx=0 tx=0).stop 2024-03-08T23:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8106600 msgr2=0x7f8bc819dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:56:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8106600 0x7f8bc819dce0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00e9f0 tx=0x7f8bbc00eeb0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 shutdown_connections 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8bb00a2a10 0x7f8bb00a4ed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc81979a0 0x7f8bc8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bc8106600 0x7f8bc819dce0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 --2- 172.21.15.22:0/2792059325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc8101530 0x7f8bc819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 >> 172.21.15.22:0/2792059325 conn(0x7f8bc80fcdd0 msgr2=0x7f8bc80ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 shutdown_connections 2024-03-08T23:56:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:56:50.739+0000 7f8bce53b640 1 -- 172.21.15.22:0/2792059325 wait complete. 2024-03-08T23:56:51.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 148822 13.89 MB/sec warmup 59 sec latency 169.449 ms 2024-03-08T23:56:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:52 smithi062 ceph-mon[36129]: pgmap v74: 129 pgs: 129 active+clean; 49 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 391 op/s 2024-03-08T23:56:52.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:52 smithi192 ceph-mon[35995]: pgmap v74: 129 pgs: 129 active+clean; 49 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 391 op/s 2024-03-08T23:56:52.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 150861 13.83 MB/sec warmup 60 sec latency 14.314 ms 2024-03-08T23:56:52.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:52 smithi022 ceph-mon[31894]: pgmap v74: 129 pgs: 129 active+clean; 49 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 391 op/s 2024-03-08T23:56:53.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 153341 13.83 MB/sec warmup 61 sec latency 11.896 ms 2024-03-08T23:56:54.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:54 smithi062 ceph-mon[36129]: pgmap v75: 129 pgs: 129 active+clean; 49 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-03-08T23:56:54.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:54 smithi192 ceph-mon[35995]: pgmap v75: 129 pgs: 129 active+clean; 49 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-03-08T23:56:54.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 156071 13.88 MB/sec warmup 62 sec latency 70.084 ms 2024-03-08T23:56:54.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:54 smithi022 ceph-mon[31894]: pgmap v75: 129 pgs: 129 active+clean; 49 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-03-08T23:56:55.706 INFO:tasks.workunit.client.0.smithi022.stdout: 1 158343 13.78 MB/sec warmup 63 sec latency 8.119 ms 2024-03-08T23:56:56.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:56 smithi062 ceph-mon[36129]: pgmap v76: 129 pgs: 129 active+clean; 42 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 546 op/s 2024-03-08T23:56:56.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:56 smithi192 ceph-mon[35995]: pgmap v76: 129 pgs: 129 active+clean; 42 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 546 op/s 2024-03-08T23:56:56.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 160429 13.78 MB/sec warmup 64 sec latency 25.831 ms 2024-03-08T23:56:56.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:56 smithi022 ceph-mon[31894]: pgmap v76: 129 pgs: 129 active+clean; 42 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 546 op/s 2024-03-08T23:56:57.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 163324 13.84 MB/sec warmup 65 sec latency 13.043 ms 2024-03-08T23:56:58.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:56:58 smithi062 ceph-mon[36129]: pgmap v77: 129 pgs: 129 active+clean; 42 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2024-03-08T23:56:58.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:56:58 smithi192 ceph-mon[35995]: pgmap v77: 129 pgs: 129 active+clean; 42 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2024-03-08T23:56:58.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 165764 13.77 MB/sec warmup 66 sec latency 7.255 ms 2024-03-08T23:56:58.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:56:58 smithi022 ceph-mon[31894]: pgmap v77: 129 pgs: 129 active+clean; 42 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 425 op/s 2024-03-08T23:56:59.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 167454 13.73 MB/sec warmup 67 sec latency 50.558 ms 2024-03-08T23:57:00.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:00 smithi062 ceph-mon[36129]: pgmap v78: 129 pgs: 129 active+clean; 47 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 525 op/s 2024-03-08T23:57:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:00 smithi192 ceph-mon[35995]: pgmap v78: 129 pgs: 129 active+clean; 47 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 525 op/s 2024-03-08T23:57:00.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 169972 13.73 MB/sec warmup 68 sec latency 46.968 ms 2024-03-08T23:57:00.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:00 smithi022 ceph-mon[31894]: pgmap v78: 129 pgs: 129 active+clean; 47 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 525 op/s 2024-03-08T23:57:01.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 170704 13.61 MB/sec warmup 69 sec latency 60.277 ms 2024-03-08T23:57:02.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:02 smithi062 ceph-mon[36129]: pgmap v79: 129 pgs: 129 active+clean; 47 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2024-03-08T23:57:02.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:02 smithi192 ceph-mon[35995]: pgmap v79: 129 pgs: 129 active+clean; 47 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2024-03-08T23:57:02.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 173649 13.62 MB/sec warmup 70 sec latency 8.484 ms 2024-03-08T23:57:02.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:02 smithi022 ceph-mon[31894]: pgmap v79: 129 pgs: 129 active+clean; 47 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2024-03-08T23:57:03.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 175648 13.57 MB/sec warmup 71 sec latency 16.354 ms 2024-03-08T23:57:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 -- 172.21.15.22:0/1621481197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a01073e0 msgr2=0x7f16a01077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 --2- 172.21.15.22:0/1621481197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a01073e0 0x7f16a01077c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1690009900 tx=0x7f169002bcd0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 -- 172.21.15.22:0/1621481197 shutdown_connections 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 --2- 172.21.15.22:0/1621481197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a0069310 0x7f16a010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 --2- 172.21.15.22:0/1621481197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a0068970 0x7f16a0068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 --2- 172.21.15.22:0/1621481197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a01073e0 0x7f16a01077c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 -- 172.21.15.22:0/1621481197 >> 172.21.15.22:0/1621481197 conn(0x7f16a0077040 msgr2=0x7f16a0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 -- 172.21.15.22:0/1621481197 shutdown_connections 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 -- 172.21.15.22:0/1621481197 wait complete. 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.830+0000 7f16a6244640 1 Processor -- start 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f16a6244640 1 -- start start 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f16a6244640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a0068970 0x7f16a0100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f16a6244640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a0069310 0x7f16a0100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f16a6244640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a01073e0 0x7f16a0100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f16a6244640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0111640 con 0x7f16a0069310 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f16a6244640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01114c0 con 0x7f16a0068970 2024-03-08T23:57:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f16a6244640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01117c0 con 0x7f16a01073e0 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a0069310 0x7f16a0100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a0069310 0x7f16a0100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37960/0 (socket says 172.21.15.22:37960) 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169f7fe640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a0068970 0x7f16a0100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 -- 172.21.15.22:0/4205402871 learned_addr learned my addr 172.21.15.22:0/4205402871 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169ffff640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a01073e0 0x7f16a0100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 -- 172.21.15.22:0/4205402871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a01073e0 msgr2=0x7f16a0100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a01073e0 0x7f16a0100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 -- 172.21.15.22:0/4205402871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a0068970 msgr2=0x7f16a0100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a0068970 0x7f16a0100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 -- 172.21.15.22:0/4205402871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a01a5f80 con 0x7f16a0069310 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169f7fe640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a0068970 0x7f16a0100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169ffff640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a01073e0 0x7f16a0100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.831+0000 7f169effd640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a0069310 0x7f16a0100a50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f169400e9f0 tx=0x7f169400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.832+0000 7f169cff9640 1 -- 172.21.15.22:0/4205402871 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169400cd80 con 0x7f16a0069310 2024-03-08T23:57:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.832+0000 7f169cff9640 1 -- 172.21.15.22:0/4205402871 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1694010910 con 0x7f16a0069310 2024-03-08T23:57:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.832+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a01a6270 con 0x7f16a0069310 2024-03-08T23:57:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.832+0000 7f169cff9640 1 -- 172.21.15.22:0/4205402871 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694018990 con 0x7f16a0069310 2024-03-08T23:57:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.832+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a01a67c0 con 0x7f16a0069310 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.834+0000 7f169cff9640 1 -- 172.21.15.22:0/4205402871 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1694010ac0 con 0x7f16a0069310 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.834+0000 7f16a8e25740 1 Processor -- start 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.834+0000 7f16a8e25740 1 -- start start 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169cff9640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16880a2a40 0x7f16880a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c9d40 0x5635528c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528d2550 0x5635528d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f16a8e25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355282d5f0 con 0x5635528c8fd0 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f16a8e25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635527d6930 con 0x5635528d2550 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f16a8e25740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563552829690 con 0x5635528c9d40 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c9d40 0x5635528c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37970/0 (socket says 172.21.15.22:37970) 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c9d40 0x5635528c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36390/0 (socket says 172.21.15.22:36390) 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 -- 172.21.15.22:0/1887591545 learned_addr learned my addr 172.21.15.22:0/1887591545 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169f7fe640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16880a2a40 0x7f16880a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169ffff640 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528d2550 0x5635528d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169cff9640 1 -- 172.21.15.22:0/4205402871 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f16940cf460 con 0x7f16a0069310 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 -- 172.21.15.22:0/1887591545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c9d40 msgr2=0x5635528c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c9d40 0x5635528c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 -- 172.21.15.22:0/1887591545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528d2550 msgr2=0x5635528d4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528d2550 0x5635528d4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 -- 172.21.15.22:0/1887591545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563552542210 con 0x5635528c8fd0 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169f7fe640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16880a2a40 0x7f16880a4f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f169002efd0 tx=0x7f169002cf90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.835+0000 7f169effd640 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528c9450 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f169409b800 tx=0x7f169409b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bbea6bdf480e92 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f1685ffb640 1 -- 172.21.15.22:0/1887591545 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694094e60 con 0x5635528c8fd0 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f1685ffb640 1 -- 172.21.15.22:0/1887591545 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16940a4e20 con 0x5635528c8fd0 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f1685ffb640 1 -- 172.21.15.22:0/1887591545 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169409f910 con 0x5635528c8fd0 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 -- 172.21.15.22:0/1887591545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 msgr2=0x5635528c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528c9450 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f169409b800 tx=0x7f169409b1d0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 -- 172.21.15.22:0/1887591545 shutdown_connections 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528d2550 0x5635528d4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528c9450 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 --2- 172.21.15.22:0/1887591545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c9d40 0x5635528c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 -- 172.21.15.22:0/1887591545 >> 172.21.15.22:0/1887591545 conn(0x5635528905b0 msgr2=0x5635528c3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 -- 172.21.15.22:0/1887591545 shutdown_connections 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.836+0000 7f16a8e25740 1 -- 172.21.15.22:0/1887591545 wait complete. 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 -- start start 2024-03-08T23:57:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 Processor -- start 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c8fd0 0x5635528cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528d2550 0x5635528cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635528d51c0 con 0x5635528d2550 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635527e9d90 con 0x5635528c9d40 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635528d5340 con 0x5635528c8fd0 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f16a8e25740 10 client.0 _ll_register_callbacks cb 0x563552541a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f169ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528d2550 0x5635528cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f169ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528d2550 0x5635528cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37980/0 (socket says 172.21.15.22:37980) 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.837+0000 7f169ffff640 1 -- 172.21.15.22:0/2192870537 learned_addr learned my addr 172.21.15.22:0/2192870537 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169f7fe640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c8fd0 0x5635528cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169effd640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169ffff640 1 -- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c8fd0 msgr2=0x5635528cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169ffff640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c8fd0 0x5635528cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169ffff640 1 -- 172.21.15.22:0/2192870537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 msgr2=0x5635528cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169ffff640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169ffff640 1 -- 172.21.15.22:0/2192870537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56355260e620 con 0x5635528d2550 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169effd640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f169ffff640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528d2550 0x5635528cffc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1698003ad0 tx=0x7f1698003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab7670632bc88d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f167effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f167effd640 1 -- 172.21.15.22:0/2192870537 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698004b20 con 0x5635528d2550 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f167effd640 1 -- 172.21.15.22:0/2192870537 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f1698004cc0 con 0x5635528d2550 2024-03-08T23:57:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16940a4e20 con 0x5635528d2550 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f167effd640 1 -- 172.21.15.22:0/2192870537 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1698029050 con 0x5635528d2550 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f16a8e25740 10 client.16500 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.838+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5635528e1450 con 0x5635528d2550 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f167effd640 1 -- 172.21.15.22:0/2192870537 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16940a4e20 con 0x5635528d2550 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f167effd640 1 -- 172.21.15.22:0/2192870537 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f169801d070 con 0x5635528d2550 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f16a8e25740 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f16a8e25740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f16a8e25740 20 client.16500 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f16a8e25740 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5635528e2960 0x5635528e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f16a8e25740 4 client.16500 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56355283ab30 con 0x5635528e2960 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.839+0000 7f169f7fe640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5635528e2960 0x5635528e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.840+0000 7f169f7fe640 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5635528e2960 0x5635528e4d50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95116530fdd1b79d server_cookie=1578b1faf7f98a2 in_seq=0 out_seq=0 2024-03-08T23:57:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.840+0000 7f167effd640 10 client.16500 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.840+0000 7f167effd640 1 -- 172.21.15.22:0/2192870537 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56355283ab30 con 0x5635528e2960 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.840+0000 7f167effd640 10 client.16500 handle_command_reply: tid=0 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.840+0000 7f16a8e25740 1 client.16500 shutdown 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528d2550 msgr2=0x5635528cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528d2550 0x5635528cffc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1698003ad0 tx=0x7f1698003f90 comp rx=0 tx=0).stop 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 shutdown_connections 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5635528e2960 0x5635528e4d50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528d2550 0x5635528cffc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f167effd640 0 client.16500 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 --2- 172.21.15.22:0/2192870537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528c8fd0 0x5635528cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 >> 172.21.15.22:0/2192870537 conn(0x5635528905b0 msgr2=0x5635528c5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 shutdown_connections 2024-03-08T23:57:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 1 -- 172.21.15.22:0/2192870537 wait complete. 2024-03-08T23:57:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.841+0000 7f16a8e25740 20 client.16500 trim_cache size 0 max 16384 2024-03-08T23:57:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 Processor -- start 2024-03-08T23:57:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 -- start start 2024-03-08T23:57:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01b2d10 con 0x5635528c8fd0 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01b08b0 con 0x5635528c9d40 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f16a8e25740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01ae950 con 0x5635528d2550 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f169ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37994/0 (socket says 172.21.15.22:37994) 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f169f7fe640 1 -- 172.21.15.22:0/805445033 learned_addr learned my addr 172.21.15.22:0/805445033 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.855+0000 7f169effd640 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f169ffff640 1 -- 172.21.15.22:0/805445033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 msgr2=0x5635528cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f169ffff640 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f169ffff640 1 -- 172.21.15.22:0/805445033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 msgr2=0x5635528d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f169ffff640 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f169ffff640 1 -- 172.21.15.22:0/805445033 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635528c82a0 con 0x5635528d2550 2024-03-08T23:57:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f169f7fe640 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f169ffff640 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528ce350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1698000f80 tx=0x7f1698009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a02f574a075ccae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f16857fa640 1 -- 172.21.15.22:0/805445033 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698032070 con 0x5635528d2550 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f16857fa640 1 -- 172.21.15.22:0/805445033 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f169801e3f0 con 0x5635528d2550 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.856+0000 7f16857fa640 1 -- 172.21.15.22:0/805445033 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698005030 con 0x5635528d2550 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 -- 172.21.15.22:0/805445033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 msgr2=0x5635528ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528ce350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1698000f80 tx=0x7f1698009480 comp rx=0 tx=0).stop 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 -- 172.21.15.22:0/805445033 shutdown_connections 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528ce350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 --2- 172.21.15.22:0/805445033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 -- 172.21.15.22:0/805445033 >> 172.21.15.22:0/805445033 conn(0x5635528905b0 msgr2=0x5635528ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 -- 172.21.15.22:0/805445033 shutdown_connections 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 -- 172.21.15.22:0/805445033 wait complete. 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 -- start start 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.857+0000 7f16a8e25740 1 Processor -- start 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01aef50 con 0x5635528c8fd0 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01b0250 con 0x5635528c9d40 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01af590 con 0x5635528d2550 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 10 client.0 _ll_register_callbacks cb 0x7f1698003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38006/0 (socket says 172.21.15.22:38006) 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 -- 172.21.15.22:0/3967443797 learned_addr learned my addr 172.21.15.22:0/3967443797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169effd640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169ffff640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 -- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 msgr2=0x5635528d81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528d81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 -- 172.21.15.22:0/3967443797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 msgr2=0x5635528d7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528d7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 -- 172.21.15.22:0/3967443797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635528d89e0 con 0x5635528c8fd0 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169effd640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f169f7fe640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d9900 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f169002c8c0 tx=0x7f169002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9255bb9a2fd088 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f167effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f167effd640 1 -- 172.21.15.22:0/3967443797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690004450 con 0x5635528c8fd0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f167effd640 1 -- 172.21.15.22:0/3967443797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f1690004f30 con 0x5635528c8fd0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.858+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f169801e3f0 con 0x5635528c8fd0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f167effd640 1 -- 172.21.15.22:0/3967443797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1690061050 con 0x5635528c8fd0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 10 client.16506 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563552939ea0 con 0x5635528c8fd0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f167effd640 1 -- 172.21.15.22:0/3967443797 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f169801e3f0 con 0x5635528c8fd0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f167effd640 1 -- 172.21.15.22:0/3967443797 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1690003c10 con 0x5635528c8fd0 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 10 client.16506 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 20 client.16506 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56355293a550 0x56355293c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 4 client.16506 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:57:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563552834e80 con 0x56355293a550 2024-03-08T23:57:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.859+0000 7f169effd640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56355293a550 0x56355293c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.860+0000 7f167effd640 10 client.16506 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:03.860+0000 7f169effd640 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56355293a550 0x56355293c970 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d216e7bc76c0d4 server_cookie=a992036bbbf7bab3 in_seq=0 out_seq=0 2024-03-08T23:57:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.086+0000 7f167effd640 1 -- 172.21.15.22:0/3967443797 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563552834e80 con 0x56355293a550 2024-03-08T23:57:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.086+0000 7f167effd640 10 client.16506 handle_command_reply: tid=0 2024-03-08T23:57:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.086+0000 7f16a8e25740 1 client.16506 shutdown 2024-03-08T23:57:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.086+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 msgr2=0x5635528d9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.086+0000 7f16a8e25740 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d9900 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f169002c8c0 tx=0x7f169002bcd0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 shutdown_connections 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56355293a550 0x56355293c970 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5635528d2550 0x5635528d81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f167effd640 0 client.16506 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635528c9d40 0x5635528d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 --2- 172.21.15.22:0/3967443797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5635528c8fd0 0x5635528d9900 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 >> 172.21.15.22:0/3967443797 conn(0x5635528905b0 msgr2=0x5635528cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 shutdown_connections 2024-03-08T23:57:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 1 -- 172.21.15.22:0/3967443797 wait complete. 2024-03-08T23:57:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a8e25740 20 client.16506 trim_cache size 0 max 16384 2024-03-08T23:57:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:57:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:57:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:57:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:57:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.087+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16880a2a40 msgr2=0x7f16880a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16880a2a40 0x7f16880a4f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f169002efd0 tx=0x7f169002cf90 comp rx=0 tx=0).stop 2024-03-08T23:57:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a0069310 msgr2=0x7f16a0100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a0069310 0x7f16a0100a50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f169400e9f0 tx=0x7f169400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 shutdown_connections 2024-03-08T23:57:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16880a2a40 0x7f16880a4f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16a01073e0 0x7f16a0100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16a0069310 0x7f16a0100a50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 --2- 172.21.15.22:0/4205402871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a0068970 0x7f16a0100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 >> 172.21.15.22:0/4205402871 conn(0x7f16a0077040 msgr2=0x7f16a00ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 shutdown_connections 2024-03-08T23:57:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:04.088+0000 7f16a6244640 1 -- 172.21.15.22:0/4205402871 wait complete. 2024-03-08T23:57:04.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:04 smithi062 ceph-mon[36129]: pgmap v80: 129 pgs: 129 active+clean; 47 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2024-03-08T23:57:04.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:04 smithi192 ceph-mon[35995]: pgmap v80: 129 pgs: 129 active+clean; 47 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2024-03-08T23:57:04.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 177590 13.57 MB/sec warmup 72 sec latency 115.490 ms 2024-03-08T23:57:04.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:04 smithi022 ceph-mon[31894]: pgmap v80: 129 pgs: 129 active+clean; 47 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 427 op/s 2024-03-08T23:57:05.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 179948 13.49 MB/sec warmup 73 sec latency 12.551 ms 2024-03-08T23:57:06.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:06 smithi062 ceph-mon[36129]: pgmap v81: 129 pgs: 129 active+clean; 52 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 611 op/s 2024-03-08T23:57:06.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:06 smithi192 ceph-mon[35995]: pgmap v81: 129 pgs: 129 active+clean; 52 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 611 op/s 2024-03-08T23:57:06.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 182016 13.49 MB/sec warmup 74 sec latency 20.105 ms 2024-03-08T23:57:06.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:06 smithi022 ceph-mon[31894]: pgmap v81: 129 pgs: 129 active+clean; 52 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 611 op/s 2024-03-08T23:57:06.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:57:06.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 -- 172.21.15.22:0/3658202722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801054a0 msgr2=0x7fd18010d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 --2- 172.21.15.22:0/3658202722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801054a0 0x7fd18010d070 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd178009ca0 tx=0x7fd17802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 -- 172.21.15.22:0/3658202722 shutdown_connections 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 --2- 172.21.15.22:0/3658202722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801054a0 0x7fd18010d070 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 --2- 172.21.15.22:0/3658202722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd180100d80 0x7fd180104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 --2- 172.21.15.22:0/3658202722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1801003d0 0x7fd1801007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 -- 172.21.15.22:0/3658202722 >> 172.21.15.22:0/3658202722 conn(0x7fd1800772f0 msgr2=0x7fd180077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 -- 172.21.15.22:0/3658202722 shutdown_connections 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.996+0000 7fd1866b2640 1 -- 172.21.15.22:0/3658202722 wait complete. 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 Processor -- start 2024-03-08T23:57:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 -- start start 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801003d0 0x7fd180101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1801054a0 0x7fd1801023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1801028e0 0x7fd1801a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180110560 con 0x7fd1801003d0 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801103e0 con 0x7fd1801054a0 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd1866b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801106e0 con 0x7fd1801028e0 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801003d0 0x7fd180101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801003d0 0x7fd180101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38026/0 (socket says 172.21.15.22:38026) 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd17ffff640 1 -- 172.21.15.22:0/787514452 learned_addr learned my addr 172.21.15.22:0/787514452 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1801054a0 0x7fd1801023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.997+0000 7fd184c4f640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1801028e0 0x7fd1801a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17ffff640 1 -- 172.21.15.22:0/787514452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1801028e0 msgr2=0x7fd1801a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17ffff640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1801028e0 0x7fd1801a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17ffff640 1 -- 172.21.15.22:0/787514452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1801054a0 msgr2=0x7fd1801023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17ffff640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1801054a0 0x7fd1801023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17ffff640 1 -- 172.21.15.22:0/787514452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1801a64b0 con 0x7fd1801003d0 2024-03-08T23:57:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17ffff640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801003d0 0x7fd180101e60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd17000e9f0 tx=0x7fd17000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17d7fa640 1 -- 172.21.15.22:0/787514452 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000cd80 con 0x7fd1801003d0 2024-03-08T23:57:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1801a67a0 con 0x7fd1801003d0 2024-03-08T23:57:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17d7fa640 1 -- 172.21.15.22:0/787514452 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd170005500 con 0x7fd1801003d0 2024-03-08T23:57:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd17d7fa640 1 -- 172.21.15.22:0/787514452 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170010690 con 0x7fd1801003d0 2024-03-08T23:57:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:06.998+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1801a6cf0 con 0x7fd1801003d0 2024-03-08T23:57:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.000+0000 7fd17d7fa640 1 -- 172.21.15.22:0/787514452 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd1700046c0 con 0x7fd1801003d0 2024-03-08T23:57:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.000+0000 7fd189371740 1 Processor -- start 2024-03-08T23:57:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.000+0000 7fd189371740 1 -- start start 2024-03-08T23:57:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd189371740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da770b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd189371740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da779720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd189371740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da779c60 0x5650da77c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd189371740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650da6d3e00 con 0x5650da770760 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd189371740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650da67cb10 con 0x5650da771080 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd189371740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650da6cfea0 con 0x5650da779c60 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da770b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da770b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38028/0 (socket says 172.21.15.22:38028) 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 -- 172.21.15.22:0/3787614029 learned_addr learned my addr 172.21.15.22:0/3787614029 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd17d7fa640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1680a29a0 0x7fd1680a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da779c60 0x5650da77c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd17ffff640 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da779720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1680a29a0 0x7fd1680a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 -- 172.21.15.22:0/3787614029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da779c60 msgr2=0x5650da77c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da779c60 0x5650da77c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 -- 172.21.15.22:0/3787614029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 msgr2=0x5650da779720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da779720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd17d7fa640 1 -- 172.21.15.22:0/787514452 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd1700cf610 con 0x7fd1801003d0 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 -- 172.21.15.22:0/3787614029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650da4b27d0 con 0x5650da770760 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd17ffff640 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da779720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.001+0000 7fd184c4f640 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da770b40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd178037040 tx=0x7fd17802cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b06155c6cdec6b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd1667fc640 1 -- 172.21.15.22:0/3787614029 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178050070 con 0x5650da770760 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd1667fc640 1 -- 172.21.15.22:0/3787614029 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd178003e50 con 0x5650da770760 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd1667fc640 1 -- 172.21.15.22:0/3787614029 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178039040 con 0x5650da770760 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 -- 172.21.15.22:0/3787614029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 msgr2=0x5650da770b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd184c4f640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1680a29a0 0x7fd1680a4e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd17804d000 tx=0x7fd178048000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da770b40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd178037040 tx=0x7fd17802cef0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 -- 172.21.15.22:0/3787614029 shutdown_connections 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da779c60 0x5650da77c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da779720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 --2- 172.21.15.22:0/3787614029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da770b40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 -- 172.21.15.22:0/3787614029 >> 172.21.15.22:0/3787614029 conn(0x5650da76ae90 msgr2=0x5650da76b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 -- 172.21.15.22:0/3787614029 shutdown_connections 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.002+0000 7fd189371740 1 -- 172.21.15.22:0/3787614029 wait complete. 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 -- start start 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 Processor -- start 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da779340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da771080 0x5650da7766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da776c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650da77ca50 con 0x5650da770760 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650da6900a0 con 0x5650da779c60 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650da77cbd0 con 0x5650da771080 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd189371740 10 client.0 _ll_register_callbacks cb 0x5650da68fa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da779340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da771080 0x5650da7766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd184c4f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da776c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da779340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38042/0 (socket says 172.21.15.22:38042) 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.003+0000 7fd17ffff640 1 -- 172.21.15.22:0/3577082975 learned_addr learned my addr 172.21.15.22:0/3577082975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd17ffff640 1 -- 172.21.15.22:0/3577082975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da771080 msgr2=0x5650da7766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd17ffff640 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da771080 0x5650da7766d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd17ffff640 1 -- 172.21.15.22:0/3577082975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 msgr2=0x5650da776c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd17ffff640 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da776c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd17ffff640 1 -- 172.21.15.22:0/3577082975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650da3e83f0 con 0x5650da770760 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd184c4f640 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da776c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd17ffff640 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da779340 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd17000e9c0 tx=0x7fd17009b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca22ac442c55683 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd15f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd15f7fe640 1 -- 172.21.15.22:0/3577082975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700ccde0 con 0x5650da770760 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd15f7fe640 1 -- 172.21.15.22:0/3577082975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd1700cf1d0 con 0x5650da770760 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd178003e50 con 0x5650da770760 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd15f7fe640 1 -- 172.21.15.22:0/3577082975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1700cf3d0 con 0x5650da770760 2024-03-08T23:57:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd189371740 10 client.16518 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650da3a80f0 con 0x5650da770760 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.004+0000 7fd15f7fe640 1 -- 172.21.15.22:0/3577082975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd178003e50 con 0x5650da770760 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd15f7fe640 1 -- 172.21.15.22:0/3577082975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd17009cba0 con 0x5650da770760 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd189371740 10 client.16518 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd189371740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd189371740 20 client.16518 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd189371740 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da789770 0x5650da78bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd189371740 4 client.16518 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650da6e1340 con 0x5650da789770 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da789770 0x5650da78bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da789770 0x5650da78bb60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f50caff2567459 server_cookie=9be929cfe1cdf198 in_seq=0 out_seq=0 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.005+0000 7fd15f7fe640 10 client.16518 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd15f7fe640 1 -- 172.21.15.22:0/3577082975 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650da6e1340 con 0x5650da789770 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd15f7fe640 10 client.16518 handle_command_reply: tid=0 2024-03-08T23:57:07.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 client.16518 shutdown 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 msgr2=0x5650da779340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da779340 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd17000e9c0 tx=0x7fd17009b470 comp rx=0 tx=0).stop 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 shutdown_connections 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da789770 0x5650da78bb60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da776c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da771080 0x5650da7766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd15f7fe640 0 client.16518 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 --2- 172.21.15.22:0/3577082975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da770760 0x5650da779340 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 >> 172.21.15.22:0/3577082975 conn(0x5650da76ae90 msgr2=0x5650da76cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 shutdown_connections 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 1 -- 172.21.15.22:0/3577082975 wait complete. 2024-03-08T23:57:07.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.006+0000 7fd189371740 20 client.16518 trim_cache size 0 max 16384 2024-03-08T23:57:07.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 Processor -- start 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 -- start start 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da7756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da771080 0x5650da778e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da779340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801b2c70 con 0x5650da771080 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801b0810 con 0x5650da779c60 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd189371740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801ae8b0 con 0x5650da770760 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da771080 0x5650da778e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da771080 0x5650da778e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38044/0 (socket says 172.21.15.22:38044) 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd17f7fe640 1 -- 172.21.15.22:0/2764218978 learned_addr learned my addr 172.21.15.22:0/2764218978 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.022+0000 7fd17ffff640 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da7756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd184c4f640 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da779340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd17f7fe640 1 -- 172.21.15.22:0/2764218978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 msgr2=0x5650da7756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da7756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd17f7fe640 1 -- 172.21.15.22:0/2764218978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 msgr2=0x5650da779340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da779340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd17f7fe640 1 -- 172.21.15.22:0/2764218978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650da770130 con 0x5650da771080 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd184c4f640 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da779340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da771080 0x5650da778e00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd174000c00 tx=0x7fd174005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4541cdf5b16b771a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd165ffb640 1 -- 172.21.15.22:0/2764218978 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174003de0 con 0x5650da771080 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd165ffb640 1 -- 172.21.15.22:0/2764218978 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd17401f550 con 0x5650da771080 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.023+0000 7fd165ffb640 1 -- 172.21.15.22:0/2764218978 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174010c10 con 0x5650da771080 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 -- 172.21.15.22:0/2764218978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da771080 msgr2=0x5650da778e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da771080 0x5650da778e00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd174000c00 tx=0x7fd174005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 -- 172.21.15.22:0/2764218978 shutdown_connections 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da779c60 0x5650da779340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da771080 0x5650da778e00 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 --2- 172.21.15.22:0/2764218978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da7756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 -- 172.21.15.22:0/2764218978 >> 172.21.15.22:0/2764218978 conn(0x5650da76ae90 msgr2=0x5650da774fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 -- 172.21.15.22:0/2764218978 shutdown_connections 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 -- 172.21.15.22:0/2764218978 wait complete. 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 -- start start 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.024+0000 7fd189371740 1 Processor -- start 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da773630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da773b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da779c60 0x5650da7dfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801aeeb0 con 0x5650da779c60 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801b01b0 con 0x5650da771080 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801af4f0 con 0x5650da770760 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 10 client.0 _ll_register_callbacks cb 0x7fd17000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da779c60 0x5650da7dfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da779c60 0x5650da7dfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38058/0 (socket says 172.21.15.22:38058) 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 -- 172.21.15.22:0/954498262 learned_addr learned my addr 172.21.15.22:0/954498262 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da773b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd17ffff640 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da773630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 -- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 msgr2=0x5650da773630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da773630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 -- 172.21.15.22:0/954498262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 msgr2=0x5650da773b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da773b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 -- 172.21.15.22:0/954498262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650da7e0110 con 0x5650da779c60 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd17f7fe640 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da773b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd184c4f640 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da779c60 0x5650da7dfbd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd178032530 tx=0x7fd178009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2b1e54e902dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd15f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd15f7fe640 1 -- 172.21.15.22:0/954498262 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178039040 con 0x5650da779c60 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd15f7fe640 1 -- 172.21.15.22:0/954498262 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd178055440 con 0x5650da779c60 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.025+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd17401f550 con 0x5650da779c60 2024-03-08T23:57:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd15f7fe640 1 -- 172.21.15.22:0/954498262 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd178065050 con 0x5650da779c60 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650da7e1170 con 0x5650da779c60 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd15f7fe640 1 -- 172.21.15.22:0/954498262 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd17401f550 con 0x5650da779c60 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd15f7fe640 1 -- 172.21.15.22:0/954498262 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd17803f070 con 0x5650da779c60 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 20 client.16524 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da7e1460 0x5650da7e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 4 client.16524 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5650da6db690 con 0x5650da7e1460 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.026+0000 7fd17ffff640 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da7e1460 0x5650da7e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.027+0000 7fd17ffff640 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da7e1460 0x5650da7e3850 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af1e1b7506debe6 server_cookie=bcbda7849b8e4a3d in_seq=0 out_seq=0 2024-03-08T23:57:07.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.027+0000 7fd15f7fe640 10 client.16524 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.027+0000 7fd15f7fe640 1 -- 172.21.15.22:0/954498262 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5650da6db690 con 0x5650da7e1460 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.027+0000 7fd15f7fe640 10 client.16524 handle_command_reply: tid=0 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.027+0000 7fd189371740 1 client.16524 shutdown 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da779c60 msgr2=0x5650da7dfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da779c60 0x5650da7dfbd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd178032530 tx=0x7fd178009360 comp rx=0 tx=0).stop 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 shutdown_connections 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650da7e1460 0x5650da7e3850 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650da779c60 0x5650da7dfbd0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650da771080 0x5650da773b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd15f7fe640 0 client.16524 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 --2- 172.21.15.22:0/954498262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650da770760 0x5650da773630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 >> 172.21.15.22:0/954498262 conn(0x5650da76ae90 msgr2=0x5650da775830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 shutdown_connections 2024-03-08T23:57:07.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 1 -- 172.21.15.22:0/954498262 wait complete. 2024-03-08T23:57:07.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd189371740 20 client.16524 trim_cache size 0 max 16384 2024-03-08T23:57:07.030 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:57:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:57:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:57:07.030 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1680a29a0 msgr2=0x7fd1680a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd1866b2640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1680a29a0 0x7fd1680a4e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd17804d000 tx=0x7fd178048000 comp rx=0 tx=0).stop 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801003d0 msgr2=0x7fd180101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.028+0000 7fd1866b2640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801003d0 0x7fd180101e60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd17000e9f0 tx=0x7fd17000edf0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 shutdown_connections 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1680a29a0 0x7fd1680a4e60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1801028e0 0x7fd1801a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1801054a0 0x7fd1801023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 --2- 172.21.15.22:0/787514452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1801003d0 0x7fd180101e60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 >> 172.21.15.22:0/787514452 conn(0x7fd1800772f0 msgr2=0x7fd1801040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 shutdown_connections 2024-03-08T23:57:07.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.029+0000 7fd1866b2640 1 -- 172.21.15.22:0/787514452 wait complete. 2024-03-08T23:57:07.035 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:57:07.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:57:07.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 -- 172.21.15.22:0/1260229437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301065c0 msgr2=0x7f353010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 --2- 172.21.15.22:0/1260229437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301065c0 0x7f353010e190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3528009ca0 tx=0x7f352802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 -- 172.21.15.22:0/1260229437 shutdown_connections 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 --2- 172.21.15.22:0/1260229437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301065c0 0x7f353010e190 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 --2- 172.21.15.22:0/1260229437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3530101ea0 0x7f3530105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 --2- 172.21.15.22:0/1260229437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35301014f0 0x7f35301018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 -- 172.21.15.22:0/1260229437 >> 172.21.15.22:0/1260229437 conn(0x7f35300fcd90 msgr2=0x7f35300ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 -- 172.21.15.22:0/1260229437 shutdown_connections 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.145+0000 7f3536374640 1 -- 172.21.15.22:0/1260229437 wait complete. 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 Processor -- start 2024-03-08T23:57:07.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 -- start start 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35301014f0 0x7f353019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35301065c0 0x7f353019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301979e0 0x7f3530197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3530111660 con 0x7f35301979e0 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35301114e0 con 0x7f35301014f0 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3536374640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35301117e0 con 0x7f35301065c0 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3534911640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301979e0 0x7f3530197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3534911640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301979e0 0x7f3530197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38078/0 (socket says 172.21.15.22:38078) 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f3534911640 1 -- 172.21.15.22:0/3113787931 learned_addr learned my addr 172.21.15.22:0/3113787931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f352f7fe640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35301065c0 0x7f353019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.146+0000 7f352ffff640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35301014f0 0x7f353019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3534911640 1 -- 172.21.15.22:0/3113787931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35301065c0 msgr2=0x7f353019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3534911640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35301065c0 0x7f353019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3534911640 1 -- 172.21.15.22:0/3113787931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35301014f0 msgr2=0x7f353019d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3534911640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35301014f0 0x7f353019d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3534911640 1 -- 172.21.15.22:0/3113787931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3530198500 con 0x7f35301979e0 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3534911640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301979e0 0x7f3530197e90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f352802c700 tx=0x7f3528032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f352d7fa640 1 -- 172.21.15.22:0/3113787931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528041020 con 0x7f35301979e0 2024-03-08T23:57:07.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f352d7fa640 1 -- 172.21.15.22:0/3113787931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3528034cd0 con 0x7f35301979e0 2024-03-08T23:57:07.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f352d7fa640 1 -- 172.21.15.22:0/3113787931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528039040 con 0x7f35301979e0 2024-03-08T23:57:07.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3530198790 con 0x7f35301979e0 2024-03-08T23:57:07.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.147+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35301a48e0 con 0x7f35301979e0 2024-03-08T23:57:07.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.148+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3530068530 con 0x7f35301979e0 2024-03-08T23:57:07.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.149+0000 7f352d7fa640 1 -- 172.21.15.22:0/3113787931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3528044020 con 0x7f35301979e0 2024-03-08T23:57:07.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.150+0000 7f352d7fa640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f35180a2970 0x7f35180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.150+0000 7f352ffff640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f35180a2970 0x7f35180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.150+0000 7f352d7fa640 1 -- 172.21.15.22:0/3113787931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f35280f0de0 con 0x7f35301979e0 2024-03-08T23:57:07.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.150+0000 7f352ffff640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f35180a2970 0x7f35180a4e30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f35300fd230 tx=0x7f3520005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.155+0000 7f352d7fa640 1 -- 172.21.15.22:0/3113787931 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f352804b020 con 0x7f35301979e0 2024-03-08T23:57:07.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.392+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3530101ea0 con 0x7f35301979e0 2024-03-08T23:57:07.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.392+0000 7f352d7fa640 1 -- 172.21.15.22:0/3113787931 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f35280bd8b0 con 0x7f35301979e0 2024-03-08T23:57:07.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:57:07.389 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:56:37.975909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25952":{"gid":25952,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3083291387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3083291387},{"type":"v1","addr":"172.21.15.62:6837","nonce":3083291387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:57:07.389 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-03-08T23:57:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.394+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f35180a2970 msgr2=0x7f35180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.394+0000 7f3536374640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f35180a2970 0x7f35180a4e30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f35300fd230 tx=0x7f3520005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.394+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301979e0 msgr2=0x7f3530197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.394+0000 7f3536374640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301979e0 0x7f3530197e90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f352802c700 tx=0x7f3528032f20 comp rx=0 tx=0).stop 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.394+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 shutdown_connections 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.394+0000 7f3536374640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f35180a2970 0x7f35180a4e30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.395+0000 7f3536374640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35301979e0 0x7f3530197e90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.395+0000 7f3536374640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35301065c0 0x7f353019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.395+0000 7f3536374640 1 --2- 172.21.15.22:0/3113787931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35301014f0 0x7f353019d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.395+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 >> 172.21.15.22:0/3113787931 conn(0x7f35300fcd90 msgr2=0x7f35300ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.395+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 shutdown_connections 2024-03-08T23:57:07.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.395+0000 7f3536374640 1 -- 172.21.15.22:0/3113787931 wait complete. 2024-03-08T23:57:07.401 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:57:07.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 -- 172.21.15.22:0/2218366926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 msgr2=0x7ff91010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 --2- 172.21.15.22:0/2218366926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91010e1a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff900009900 tx=0x7ff90002bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 -- 172.21.15.22:0/2218366926 shutdown_connections 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 --2- 172.21.15.22:0/2218366926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91010e1a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 --2- 172.21.15.22:0/2218366926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff910101eb0 0x7ff910105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 --2- 172.21.15.22:0/2218366926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff910101500 0x7ff9101018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 -- 172.21.15.22:0/2218366926 >> 172.21.15.22:0/2218366926 conn(0x7ff9100fcda0 msgr2=0x7ff9100ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 -- 172.21.15.22:0/2218366926 shutdown_connections 2024-03-08T23:57:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 -- 172.21.15.22:0/2218366926 wait complete. 2024-03-08T23:57:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 Processor -- start 2024-03-08T23:57:07.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.512+0000 7ff915996640 1 -- start start 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff910101500 0x7ff91019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff910197970 0x7ff910197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff910111730 con 0x7ff9101065d0 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101115b0 con 0x7ff910101500 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101118b0 con 0x7ff910197970 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff910101500 0x7ff91019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38096/0 (socket says 172.21.15.22:38096) 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 -- 172.21.15.22:0/1288961502 learned_addr learned my addr 172.21.15.22:0/1288961502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 -- 172.21.15.22:0/1288961502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff910197970 msgr2=0x7ff910197e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff910197970 0x7ff910197e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 -- 172.21.15.22:0/1288961502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff910101500 msgr2=0x7ff91019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff910101500 0x7ff91019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 -- 172.21.15.22:0/1288961502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff910198490 con 0x7ff9101065d0 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90effd640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff910101500 0x7ff91019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff90e7fc640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91019dd00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800b450 tx=0x7ff8f800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff914994640 1 -- 172.21.15.22:0/1288961502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8015020 con 0x7ff9101065d0 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff914994640 1 -- 172.21.15.22:0/1288961502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8f8011d70 con 0x7ff9101065d0 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff910198780 con 0x7ff9101065d0 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff914994640 1 -- 172.21.15.22:0/1288961502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800f340 con 0x7ff9101065d0 2024-03-08T23:57:07.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.513+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9101a49d0 con 0x7ff9101065d0 2024-03-08T23:57:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff914994640 1 -- 172.21.15.22:0/1288961502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff8f8018020 con 0x7ff9101065d0 2024-03-08T23:57:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 Processor -- start 2024-03-08T23:57:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 -- start start 2024-03-08T23:57:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d103580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d10c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769d0ca360 con 0x55769d1031a0 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769d066ec0 con 0x55769d10c6a0 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff918570740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769d00fd30 con 0x55769d103ac0 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff90f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff90effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d10c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff90f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38102/0 (socket says 172.21.15.22:38102) 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1960949944 learned_addr learned my addr 172.21.15.22:0/1960949944 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.516+0000 7ff90e7fc640 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff914994640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff8fc0a29f0 0x7ff8fc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1960949944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 msgr2=0x55769d10c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d10c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1960949944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 msgr2=0x55769d10ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1960949944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55769d00ebc0 con 0x55769d1031a0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff914994640 1 -- 172.21.15.22:0/1288961502 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff8f80ceb80 con 0x7ff9101065d0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff8fc0a29f0 0x7ff8fc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d103580 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff900036040 tx=0x7ff900009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2229a0212d23e14 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff905ffb640 1 -- 172.21.15.22:0/1960949944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900044040 con 0x55769d1031a0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff8fc0a29f0 0x7ff8fc0a4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff900049000 tx=0x7ff900046000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff905ffb640 1 -- 172.21.15.22:0/1960949944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9000035f0 con 0x55769d1031a0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.517+0000 7ff905ffb640 1 -- 172.21.15.22:0/1960949944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90004a6a0 con 0x55769d1031a0 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 -- 172.21.15.22:0/1960949944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 msgr2=0x55769d103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d103580 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff900036040 tx=0x7ff900009630 comp rx=0 tx=0).stop 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 -- 172.21.15.22:0/1960949944 shutdown_connections 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 --2- 172.21.15.22:0/1960949944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d103580 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 -- 172.21.15.22:0/1960949944 >> 172.21.15.22:0/1960949944 conn(0x55769d0fd8d0 msgr2=0x55769d0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 -- 172.21.15.22:0/1960949944 shutdown_connections 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.518+0000 7ff918570740 1 -- 172.21.15.22:0/1960949944 wait complete. 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 -- start start 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 Processor -- start 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d109670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769d0054a0 con 0x55769d1031a0 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769d062f60 con 0x55769d10c6a0 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769d10f490 con 0x55769d103ac0 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff918570740 10 client.0 _ll_register_callbacks cb 0x55769d060d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff90effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff90effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38112/0 (socket says 172.21.15.22:38112) 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff90effd640 1 -- 172.21.15.22:0/1478325914 learned_addr learned my addr 172.21.15.22:0/1478325914 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff90e7fc640 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.519+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff90effd640 1 -- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 msgr2=0x55769d109bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff90effd640 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d109bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff90effd640 1 -- 172.21.15.22:0/1478325914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 msgr2=0x55769d10a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff90effd640 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff90effd640 1 -- 172.21.15.22:0/1478325914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55769cd7b3f0 con 0x55769d1031a0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff90effd640 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d109670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff90800fe50 tx=0x7ff908012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74360f4ec1779949 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff8eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1478325914 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908004b20 con 0x55769d1031a0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9000035f0 con 0x55769d1031a0 2024-03-08T23:57:07.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1478325914 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff908004cc0 con 0x55769d1031a0 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1478325914 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff908025050 con 0x55769d1031a0 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff918570740 10 client.16542 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55769ce457d0 con 0x55769d1031a0 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1478325914 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9000035f0 con 0x55769d1031a0 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.520+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1478325914 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff908010670 con 0x55769d1031a0 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff918570740 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff918570740 10 client.16542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff918570740 20 client.16542 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff918570740 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d11cb70 0x55769d11ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff918570740 4 client.16542 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55769d074400 con 0x55769d11cb70 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff90e7fc640 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d11cb70 0x55769d11ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff90e7fc640 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d11cb70 0x55769d11ef60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2a9907ae9f67f9a server_cookie=34d6e4e93d42617f in_seq=0 out_seq=0 2024-03-08T23:57:07.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.521+0000 7ff8eeffd640 10 client.16542 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1478325914 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55769d074400 con 0x55769d11cb70 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff8eeffd640 10 client.16542 handle_command_reply: tid=0 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 client.16542 shutdown 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 msgr2=0x55769d109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d109670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff90800fe50 tx=0x7ff908012820 comp rx=0 tx=0).stop 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 shutdown_connections 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d11cb70 0x55769d11ef60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d10c6a0 0x55769d10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 --2- 172.21.15.22:0/1478325914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d109670 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff8eeffd640 0 client.16542 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 >> 172.21.15.22:0/1478325914 conn(0x55769d0fd8d0 msgr2=0x55769d0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 shutdown_connections 2024-03-08T23:57:07.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 1 -- 172.21.15.22:0/1478325914 wait complete. 2024-03-08T23:57:07.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.522+0000 7ff918570740 20 client.16542 trim_cache size 0 max 16384 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 Processor -- start 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 -- start start 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d10bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d103ac0 0x55769d1068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d10c6a0 0x55769d106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101b0a10 con 0x55769d1031a0 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101ae5b0 con 0x55769d103ac0 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff918570740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101ac650 con 0x55769d10c6a0 2024-03-08T23:57:07.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d10bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d10bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38122/0 (socket says 172.21.15.22:38122) 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 -- 172.21.15.22:0/3732023228 learned_addr learned my addr 172.21.15.22:0/3732023228 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d103ac0 0x55769d1068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d10c6a0 0x55769d106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 -- 172.21.15.22:0/3732023228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d10c6a0 msgr2=0x55769d106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d10c6a0 0x55769d106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 -- 172.21.15.22:0/3732023228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d103ac0 msgr2=0x55769d1068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d103ac0 0x55769d1068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 -- 172.21.15.22:0/3732023228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55769d107bc0 con 0x55769d1031a0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.534+0000 7ff90effd640 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d10bda0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff90800f970 tx=0x7ff908011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f8d47142a3d78d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff9057fa640 1 -- 172.21.15.22:0/3732023228 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90802e070 con 0x55769d1031a0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff9057fa640 1 -- 172.21.15.22:0/3732023228 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff908010c30 con 0x55769d1031a0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff9057fa640 1 -- 172.21.15.22:0/3732023228 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908003de0 con 0x55769d1031a0 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 -- 172.21.15.22:0/3732023228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 msgr2=0x55769d10bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d10bda0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff90800f970 tx=0x7ff908011fa0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 -- 172.21.15.22:0/3732023228 shutdown_connections 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d10c6a0 0x55769d106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d103ac0 0x55769d1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 --2- 172.21.15.22:0/3732023228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d1031a0 0x55769d10bda0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 -- 172.21.15.22:0/3732023228 >> 172.21.15.22:0/3732023228 conn(0x55769d0fd8d0 msgr2=0x55769d107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 -- 172.21.15.22:0/3732023228 shutdown_connections 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 -- 172.21.15.22:0/3732023228 wait complete. 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.535+0000 7ff918570740 1 -- start start 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 1 Processor -- start 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d1031a0 0x55769d112a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d112f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d10c6a0 0x55769d16cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101acc50 con 0x55769d10c6a0 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101adf50 con 0x55769d1031a0 2024-03-08T23:57:07.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9101ad290 con 0x55769d103ac0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff918570740 10 client.0 _ll_register_callbacks cb 0x7ff9080093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d10c6a0 0x55769d16cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d10c6a0 0x55769d16cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38136/0 (socket says 172.21.15.22:38136) 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90e7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1165230634 learned_addr learned my addr 172.21.15.22:0/1165230634 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90effd640 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d1031a0 0x55769d112a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1165230634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 msgr2=0x55769d112f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d112f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1165230634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d1031a0 msgr2=0x55769d112a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d1031a0 0x55769d112a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 -- 172.21.15.22:0/1165230634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55769d16d3c0 con 0x55769d10c6a0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff90f7fe640 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d10c6a0 0x55769d16cd80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff90002c210 tx=0x7ff900003fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547213d0c91f0aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.536+0000 7ff8eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1165230634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900044040 con 0x55769d10c6a0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1165230634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff90002ca80 con 0x55769d10c6a0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff908010c30 con 0x55769d10c6a0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1165230634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90002cc80 con 0x55769d10c6a0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 10 client.16548 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55769d174790 con 0x55769d10c6a0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1165230634 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff908010c30 con 0x55769d10c6a0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1165230634 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff900048020 con 0x55769d10c6a0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 10 client.16548 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 10 client.16548 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 20 client.16548 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d174b70 0x55769d176f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 4 client.16548 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.537+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55769d06e750 con 0x55769d174b70 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff90effd640 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d174b70 0x55769d176f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff90effd640 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d174b70 0x55769d176f60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=173ac42b495f508e server_cookie=69cd103304927c83 in_seq=0 out_seq=0 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff8eeffd640 10 client.16548 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff8eeffd640 1 -- 172.21.15.22:0/1165230634 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55769d06e750 con 0x55769d174b70 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff8eeffd640 10 client.16548 handle_command_reply: tid=0 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 client.16548 shutdown 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d10c6a0 msgr2=0x55769d16cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d10c6a0 0x55769d16cd80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff90002c210 tx=0x7ff900003fc0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 shutdown_connections 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55769d174b70 0x55769d176f60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55769d10c6a0 0x55769d16cd80 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55769d103ac0 0x55769d112f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff8eeffd640 0 client.16548 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 --2- 172.21.15.22:0/1165230634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769d1031a0 0x55769d112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 >> 172.21.15.22:0/1165230634 conn(0x55769d0fd8d0 msgr2=0x55769d0fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 shutdown_connections 2024-03-08T23:57:07.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 1 -- 172.21.15.22:0/1165230634 wait complete. 2024-03-08T23:57:07.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.538+0000 7ff918570740 20 client.16548 trim_cache size 0 max 16384 2024-03-08T23:57:07.541 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff8fc0a29f0 msgr2=0x7ff8fc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff8fc0a29f0 0x7ff8fc0a4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff900049000 tx=0x7ff900046000 comp rx=0 tx=0).stop 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 msgr2=0x7ff91019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91019dd00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800b450 tx=0x7ff8f800b910 comp rx=0 tx=0).stop 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 shutdown_connections 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff8fc0a29f0 0x7ff8fc0a4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff910197970 0x7ff910197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9101065d0 0x7ff91019dd00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 --2- 172.21.15.22:0/1288961502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff910101500 0x7ff91019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 >> 172.21.15.22:0/1288961502 conn(0x7ff9100fcda0 msgr2=0x7ff9100fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 shutdown_connections 2024-03-08T23:57:07.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.539+0000 7ff915996640 1 -- 172.21.15.22:0/1288961502 wait complete. 2024-03-08T23:57:07.546 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:57:07.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2579902996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1052c0 msgr2=0x7f9c2c10ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2579902996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1052c0 0x7f9c2c10ce90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009ca0 tx=0x7f9c2402c120 comp rx=0 tx=0).stop 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2579902996 shutdown_connections 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2579902996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1052c0 0x7f9c2c10ce90 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2579902996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c100ba0 0x7f9c2c104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2579902996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c1001f0 0x7f9c2c1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2579902996 >> 172.21.15.22:0/2579902996 conn(0x7f9c2c076da0 msgr2=0x7f9c2c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2579902996 shutdown_connections 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.672+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2579902996 wait complete. 2024-03-08T23:57:07.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 Processor -- start 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 -- start start 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1001f0 0x7f9c2c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c1052c0 0x7f9c2c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c197830 0x7f9c2c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1103f0 con 0x7f9c2c1001f0 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c110270 con 0x7f9c2c1052c0 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c110570 con 0x7f9c2c197830 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1001f0 0x7f9c2c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1001f0 0x7f9c2c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38146/0 (socket says 172.21.15.22:38146) 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2a59c640 1 -- 172.21.15.22:0/2700880647 learned_addr learned my addr 172.21.15.22:0/2700880647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.673+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c197830 0x7f9c2c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c23fff640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c1052c0 0x7f9c2c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2a59c640 1 -- 172.21.15.22:0/2700880647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c197830 msgr2=0x7f9c2c197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c197830 0x7f9c2c197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2a59c640 1 -- 172.21.15.22:0/2700880647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c1052c0 msgr2=0x7f9c2c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c1052c0 0x7f9c2c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2a59c640 1 -- 172.21.15.22:0/2700880647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c1983c0 con 0x7f9c2c1001f0 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1001f0 0x7f9c2c19d6c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b450 tx=0x7f9c1c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c237fe640 1 -- 172.21.15.22:0/2700880647 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c015020 con 0x7f9c2c1001f0 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c237fe640 1 -- 172.21.15.22:0/2700880647 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c1c011d70 con 0x7f9c2c1001f0 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c1986b0 con 0x7f9c2c1001f0 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c237fe640 1 -- 172.21.15.22:0/2700880647 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00f340 con 0x7f9c2c1001f0 2024-03-08T23:57:07.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.674+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c1a4840 con 0x7f9c2c1001f0 2024-03-08T23:57:07.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.676+0000 7f9c237fe640 1 -- 172.21.15.22:0/2700880647 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9c1c018020 con 0x7f9c2c1001f0 2024-03-08T23:57:07.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 Processor -- start 2024-03-08T23:57:07.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 -- start start 2024-03-08T23:57:07.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c237fe640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c180a2970 0x7f9c180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347afe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347b07820 0x563347b09c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563347ac5360 con 0x563347b07820 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563347a61ec0 con 0x563347afec40 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c335e4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563347a0ad30 con 0x563347afe320 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c237fe640 1 -- 172.21.15.22:0/2700880647 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9c1c0ceb90 con 0x7f9c2c1001f0 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c180a2970 0x7f9c180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347afe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347afe700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36576/0 (socket says 172.21.15.22:36576) 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b072e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58250/0 (socket says 172.21.15.22:58250) 2024-03-08T23:57:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2a59c640 1 -- 172.21.15.22:0/850572763 learned_addr learned my addr 172.21.15.22:0/850572763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347b07820 0x563347b09c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2a59c640 1 -- 172.21.15.22:0/850572763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 msgr2=0x563347b072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2a59c640 1 -- 172.21.15.22:0/850572763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347b07820 msgr2=0x563347b09c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.677+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347b07820 0x563347b09c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.678+0000 7f9c2a59c640 1 -- 172.21.15.22:0/850572763 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563347a09bc0 con 0x563347afe320 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.678+0000 7f9c23fff640 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.678+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c180a2970 0x7f9c180a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c24004af0 tx=0x7f9c2402e040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.678+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347afe700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0978d0 tx=0x7f9c1c09ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b074a752b4dabf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.678+0000 7f9c20ff9640 1 -- 172.21.15.22:0/850572763 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c09af00 con 0x563347afe320 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.678+0000 7f9c20ff9640 1 -- 172.21.15.22:0/850572763 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c1c09f050 con 0x563347afe320 2024-03-08T23:57:07.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.678+0000 7f9c20ff9640 1 -- 172.21.15.22:0/850572763 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c097700 con 0x563347afe320 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- 172.21.15.22:0/850572763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 msgr2=0x563347afe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347afe700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0978d0 tx=0x7f9c1c09ae00 comp rx=0 tx=0).stop 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- 172.21.15.22:0/850572763 shutdown_connections 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347b07820 0x563347b09c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 --2- 172.21.15.22:0/850572763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347afe700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- 172.21.15.22:0/850572763 >> 172.21.15.22:0/850572763 conn(0x563347af8a50 msgr2=0x563347af8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- 172.21.15.22:0/850572763 shutdown_connections 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- 172.21.15.22:0/850572763 wait complete. 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- start start 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 Processor -- start 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347b047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b04ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347b07820 0x563347b05220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563347a004a0 con 0x563347afec40 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563347a5df60 con 0x563347b07820 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563347b0a5b0 con 0x563347afe320 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.679+0000 7f9c335e4740 10 client.0 _ll_register_callbacks cb 0x563347a5bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b04ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b04ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38162/0 (socket says 172.21.15.22:38162) 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c2a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347b047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 -- 172.21.15.22:0/3305418510 learned_addr learned my addr 172.21.15.22:0/3305418510 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347b07820 0x563347b05220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 -- 172.21.15.22:0/3305418510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 msgr2=0x563347b047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347b047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 -- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347b07820 msgr2=0x563347b05220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347b07820 0x563347b05220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 -- 172.21.15.22:0/3305418510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633477763f0 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347b07820 0x563347b05220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c23fff640 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b04ce0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9c14012700 tx=0x7f9c14012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee0965759ddb0e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3305418510 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14004030 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c1c09f050 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3305418510 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9c140041d0 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3305418510 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c140043d0 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c335e4740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.680+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5633478407d0 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3305418510 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c1c09f050 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3305418510 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c14003b40 con 0x563347afec40 2024-03-08T23:57:07.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c335e4740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c335e4740 10 client.16560 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c335e4740 20 client.16560 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b17950 0x563347b19d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c335e4740 4 client.16560 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563347a6f400 con 0x563347b17950 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b17950 0x563347b19d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c11ffb640 10 client.16560 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.681+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b17950 0x563347b19d90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78c85de1f1f78d77 server_cookie=737ac6b1352e0fc in_seq=0 out_seq=0 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3305418510 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563347a6f400 con 0x563347b17950 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c11ffb640 10 client.16560 handle_command_reply: tid=0 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 client.16560 shutdown 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 msgr2=0x563347b04ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b04ce0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9c14012700 tx=0x7f9c14012bc0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 shutdown_connections 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b17950 0x563347b19d90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347b07820 0x563347b05220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b04ce0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c11ffb640 0 client.16560 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3305418510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347afe320 0x563347b047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 >> 172.21.15.22:0/3305418510 conn(0x563347af8a50 msgr2=0x563347afa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 shutdown_connections 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 1 -- 172.21.15.22:0/3305418510 wait complete. 2024-03-08T23:57:07.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.682+0000 7f9c335e4740 20 client.16560 trim_cache size 0 max 16384 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 Processor -- start 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 -- start start 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afe320 0x563347b06ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1b0960 con 0x563347afec40 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1ae500 con 0x563347afe320 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c335e4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1ac5a0 con 0x563347b07820 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b03090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38164/0 (socket says 172.21.15.22:38164) 2024-03-08T23:57:07.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c23fff640 1 -- 172.21.15.22:0/4233035361 learned_addr learned my addr 172.21.15.22:0/4233035361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afe320 0x563347b06ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c23fff640 1 -- 172.21.15.22:0/4233035361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 msgr2=0x563347b035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c23fff640 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c23fff640 1 -- 172.21.15.22:0/4233035361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afe320 msgr2=0x563347b06ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.694+0000 7f9c23fff640 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afe320 0x563347b06ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c23fff640 1 -- 172.21.15.22:0/4233035361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563347b05770 con 0x563347afec40 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c23fff640 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b03090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9c140059d0 tx=0x7f9c14003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34784da0ba808ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c13fff640 1 -- 172.21.15.22:0/4233035361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1402e070 con 0x563347afec40 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c13fff640 1 -- 172.21.15.22:0/4233035361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c14003fe0 con 0x563347afec40 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c13fff640 1 -- 172.21.15.22:0/4233035361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400fea0 con 0x563347afec40 2024-03-08T23:57:07.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 -- 172.21.15.22:0/4233035361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 msgr2=0x563347b03090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b03090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9c140059d0 tx=0x7f9c14003c40 comp rx=0 tx=0).stop 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 -- 172.21.15.22:0/4233035361 shutdown_connections 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afec40 0x563347b03090 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 --2- 172.21.15.22:0/4233035361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afe320 0x563347b06ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 -- 172.21.15.22:0/4233035361 >> 172.21.15.22:0/4233035361 conn(0x563347af8a50 msgr2=0x563347affac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 -- 172.21.15.22:0/4233035361 shutdown_connections 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.695+0000 7f9c335e4740 1 -- 172.21.15.22:0/4233035361 wait complete. 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 -- start start 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 Processor -- start 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afe320 0x563347b0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b0d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b0dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1acba0 con 0x563347afe320 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1adea0 con 0x563347afec40 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1ad1e0 con 0x563347b07820 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c335e4740 10 client.0 _ll_register_callbacks cb 0x7f9c1c0092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afe320 0x563347b0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afe320 0x563347b0d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38172/0 (socket says 172.21.15.22:38172) 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b0d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 -- 172.21.15.22:0/3793596893 learned_addr learned my addr 172.21.15.22:0/3793596893 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2ad9d640 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b0dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 -- 172.21.15.22:0/3793596893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 msgr2=0x563347b0dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b0dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 -- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 msgr2=0x563347b0d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b0d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c2a59c640 1 -- 172.21.15.22:0/3793596893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563347b6dec0 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.696+0000 7f9c23fff640 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b0d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c2a59c640 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afe320 0x563347b0d1b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009fd0 tx=0x7f9c1c09a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2c720a3934c479 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3793596893 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c09f050 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3793596893 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9c1c097570 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c14003fe0 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3793596893 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c1c09af00 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 10 client.16566 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563347b6f0a0 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3793596893 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c14003fe0 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3793596893 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c1c096c50 con 0x563347afe320 2024-03-08T23:57:07.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 10 client.16566 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 10 client.16566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 20 client.16566 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b6f390 0x563347b71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 4 client.16566 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563347a69750 con 0x563347b6f390 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.697+0000 7f9c23fff640 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b6f390 0x563347b71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c11ffb640 10 client.16566 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c23fff640 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b6f390 0x563347b71780 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8561bad990dfa3bb server_cookie=1c55fdc6b1b88f3e in_seq=0 out_seq=0 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c11ffb640 1 -- 172.21.15.22:0/3793596893 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563347a69750 con 0x563347b6f390 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c11ffb640 10 client.16566 handle_command_reply: tid=0 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 client.16566 shutdown 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afe320 msgr2=0x563347b0d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afe320 0x563347b0d1b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009fd0 tx=0x7f9c1c09a920 comp rx=0 tx=0).stop 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 shutdown_connections 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563347b6f390 0x563347b71780 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563347b07820 0x563347b0dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563347afec40 0x563347b0d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c11ffb640 0 client.16566 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 --2- 172.21.15.22:0/3793596893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563347afe320 0x563347b0d1b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 >> 172.21.15.22:0/3793596893 conn(0x563347af8a50 msgr2=0x563347b02c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 shutdown_connections 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 1 -- 172.21.15.22:0/3793596893 wait complete. 2024-03-08T23:57:07.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.698+0000 7f9c335e4740 20 client.16566 trim_cache size 0 max 16384 2024-03-08T23:57:07.703 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:57:07.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c180a2970 msgr2=0x7f9c180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c180a2970 0x7f9c180a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c24004af0 tx=0x7f9c2402e040 comp rx=0 tx=0).stop 2024-03-08T23:57:07.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1001f0 msgr2=0x7f9c2c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1001f0 0x7f9c2c19d6c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b450 tx=0x7f9c1c00b910 comp rx=0 tx=0).stop 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 shutdown_connections 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c180a2970 0x7f9c180a4e30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c197830 0x7f9c2c197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c1052c0 0x7f9c2c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 --2- 172.21.15.22:0/2700880647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c2c1001f0 0x7f9c2c19d6c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 >> 172.21.15.22:0/2700880647 conn(0x7f9c2c076da0 msgr2=0x7f9c2c0687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 shutdown_connections 2024-03-08T23:57:07.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.699+0000 7f9c2b59e640 1 -- 172.21.15.22:0/2700880647 wait complete. 2024-03-08T23:57:07.706 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:57:07.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:57:07.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 184890 13.54 MB/sec warmup 75 sec latency 18.527 ms 2024-03-08T23:57:07.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 -- 172.21.15.22:0/2529063559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 msgr2=0x7f1258106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 --2- 172.21.15.22:0/2529063559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f1258106660 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1248009900 tx=0x7f124802bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:07.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 -- 172.21.15.22:0/2529063559 shutdown_connections 2024-03-08T23:57:07.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 --2- 172.21.15.22:0/2529063559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1258100e00 0x7f125810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 --2- 172.21.15.22:0/2529063559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1258100440 0x7f12581008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 --2- 172.21.15.22:0/2529063559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f1258106660 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 -- 172.21.15.22:0/2529063559 >> 172.21.15.22:0/2529063559 conn(0x7f1258076f90 msgr2=0x7f12580773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 -- 172.21.15.22:0/2529063559 shutdown_connections 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 -- 172.21.15.22:0/2529063559 wait complete. 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.818+0000 7f125d0ab640 1 Processor -- start 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125d0ab640 1 -- start start 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125d0ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1258100440 0x7f1258101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125d0ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1258100e00 0x7f1258102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125d0ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f12581028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125d0ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12581105a0 con 0x7f1258106280 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125d0ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1258110420 con 0x7f1258100440 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125d0ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1258110720 con 0x7f1258100e00 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f12581028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f1256d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1258100440 0x7f1258101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f12581028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38190/0 (socket says 172.21.15.22:38190) 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1258100e00 0x7f1258102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 -- 172.21.15.22:0/178448293 learned_addr learned my addr 172.21.15.22:0/178448293 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 -- 172.21.15.22:0/178448293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1258100e00 msgr2=0x7f1258102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1258100e00 0x7f1258102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 -- 172.21.15.22:0/178448293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1258100440 msgr2=0x7f1258101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1258100440 0x7f1258101e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.819+0000 7f125759e640 1 -- 172.21.15.22:0/178448293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12581a6090 con 0x7f1258106280 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.820+0000 7f125759e640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f12581028d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f125000eaa0 tx=0x7f125000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.820+0000 7f123ffff640 1 -- 172.21.15.22:0/178448293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000cdb0 con 0x7f1258106280 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.820+0000 7f123ffff640 1 -- 172.21.15.22:0/178448293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1250005020 con 0x7f1258106280 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.820+0000 7f123ffff640 1 -- 172.21.15.22:0/178448293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500116b0 con 0x7f1258106280 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.820+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12581a6380 con 0x7f1258106280 2024-03-08T23:57:07.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.820+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12581a68d0 con 0x7f1258106280 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.822+0000 7f123ffff640 1 -- 172.21.15.22:0/178448293 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1250005560 con 0x7f1258106280 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.822+0000 7f125fd6f740 1 Processor -- start 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.822+0000 7f125fd6f740 1 -- start start 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daa2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 0x56272daada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125fd6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272da69360 con 0x56272daa21a0 2024-03-08T23:57:07.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125fd6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272da05ec0 con 0x56272daab6a0 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125fd6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272d9aed30 con 0x56272daa2ac0 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f123ffff640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12400a29f0 0x7f12400a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daa2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125759e640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12400a29f0 0x7f12400a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f1256d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daa2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38194/0 (socket says 172.21.15.22:38194) 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125659c640 1 -- 172.21.15.22:0/3458223510 learned_addr learned my addr 172.21.15.22:0/3458223510 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125659c640 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 0x56272daada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f123ffff640 1 -- 172.21.15.22:0/178448293 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f12500cf300 con 0x7f1258106280 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f1256d9d640 1 -- 172.21.15.22:0/3458223510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 msgr2=0x56272daada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f1256d9d640 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 0x56272daada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f1256d9d640 1 -- 172.21.15.22:0/3458223510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 msgr2=0x56272daa2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f1256d9d640 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daa2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f1256d9d640 1 -- 172.21.15.22:0/3458223510 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56272d9adbc0 con 0x56272daa2ac0 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125659c640 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daa2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f125759e640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12400a29f0 0x7f12400a4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f125000efd0 tx=0x7f1250086040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f1256d9d640 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daab160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1248000c00 tx=0x7f1248009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7288f746b2224b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.823+0000 7f123d7fa640 1 -- 172.21.15.22:0/3458223510 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248042070 con 0x56272daa2ac0 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f123d7fa640 1 -- 172.21.15.22:0/3458223510 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f124802ca80 con 0x56272daa2ac0 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f123d7fa640 1 -- 172.21.15.22:0/3458223510 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248004f00 con 0x56272daa2ac0 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 -- 172.21.15.22:0/3458223510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 msgr2=0x56272daab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daab160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1248000c00 tx=0x7f1248009610 comp rx=0 tx=0).stop 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 -- 172.21.15.22:0/3458223510 shutdown_connections 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 0x56272daada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daab160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 --2- 172.21.15.22:0/3458223510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daa2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 -- 172.21.15.22:0/3458223510 >> 172.21.15.22:0/3458223510 conn(0x56272da9c8d0 msgr2=0x56272da9ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 -- 172.21.15.22:0/3458223510 shutdown_connections 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.824+0000 7f125fd6f740 1 -- 172.21.15.22:0/3458223510 wait complete. 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 -- start start 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 Processor -- start 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272daa87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272daa8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272daa9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272d9a44a0 con 0x56272daa2ac0 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272da01f60 con 0x56272daa21a0 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272daae430 con 0x56272daab6a0 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125fd6f740 10 client.0 _ll_register_callbacks cb 0x56272d9ffd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272daa8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272daa8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38210/0 (socket says 172.21.15.22:38210) 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 -- 172.21.15.22:0/492144735 learned_addr learned my addr 172.21.15.22:0/492144735 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125659c640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272daa9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125659c640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272daa87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 -- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 msgr2=0x56272daa9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272daa9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 -- 172.21.15.22:0/492144735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 msgr2=0x56272daa87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272daa87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 -- 172.21.15.22:0/492144735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56272d71a3f0 con 0x56272daa2ac0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f125659c640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272daa9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f1256d9d640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272daa8cf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1248006010 tx=0x7f124802c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5f734e5f994378 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.825+0000 7f123aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f123aff5640 1 -- 172.21.15.22:0/492144735 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248042070 con 0x56272daa2ac0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f123aff5640 1 -- 172.21.15.22:0/492144735 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f1248004440 con 0x56272daa2ac0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f124802ca80 con 0x56272daa2ac0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f123aff5640 1 -- 172.21.15.22:0/492144735 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1248057050 con 0x56272daa2ac0 2024-03-08T23:57:07.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56272d7e47d0 con 0x56272daa2ac0 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f123aff5640 1 -- 172.21.15.22:0/492144735 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f124802ca80 con 0x56272daa2ac0 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f123aff5640 1 -- 172.21.15.22:0/492144735 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f124804e070 con 0x56272daa2ac0 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 20 client.16578 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272dabb7b0 0x56272dabdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 4 client.16578 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56272da13400 con 0x56272dabb7b0 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.826+0000 7f125659c640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272dabb7b0 0x56272dabdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.827+0000 7f125659c640 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272dabb7b0 0x56272dabdba0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe281e1fff08346 server_cookie=9adf22c8734f11dc in_seq=0 out_seq=0 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.827+0000 7f123aff5640 10 client.16578 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f123aff5640 1 -- 172.21.15.22:0/492144735 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56272da13400 con 0x56272dabb7b0 2024-03-08T23:57:07.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f123aff5640 10 client.16578 handle_command_reply: tid=0 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 client.16578 shutdown 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 msgr2=0x56272daa8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272daa8cf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1248006010 tx=0x7f124802c880 comp rx=0 tx=0).stop 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 shutdown_connections 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272dabb7b0 0x56272dabdba0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272daa9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272daa8cf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 --2- 172.21.15.22:0/492144735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272daa87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f123aff5640 0 client.16578 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 >> 172.21.15.22:0/492144735 conn(0x56272da9c8d0 msgr2=0x56272da9e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 shutdown_connections 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 1 -- 172.21.15.22:0/492144735 wait complete. 2024-03-08T23:57:07.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.828+0000 7f125fd6f740 20 client.16578 trim_cache size 0 max 16384 2024-03-08T23:57:07.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 Processor -- start 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 -- start start 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daaaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daa3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 0x56272daa4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12581b2be0 con 0x56272daa21a0 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12581b0780 con 0x56272daab6a0 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125fd6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12581ae820 con 0x56272daa2ac0 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.840+0000 7f125659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daaaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daaaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38222/0 (socket says 172.21.15.22:38222) 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 -- 172.21.15.22:0/501212071 learned_addr learned my addr 172.21.15.22:0/501212071 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f1256d9d640 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daa3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 -- 172.21.15.22:0/501212071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 msgr2=0x56272daa3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daa3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 -- 172.21.15.22:0/501212071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 msgr2=0x56272daa4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 0x56272daa4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 -- 172.21.15.22:0/501212071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56272daae370 con 0x56272daa21a0 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f125659c640 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daaaee0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x56272dabe0e0 tx=0x7f124c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2059c02147431105 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f123cff9640 1 -- 172.21.15.22:0/501212071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c022350 con 0x56272daa21a0 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f123cff9640 1 -- 172.21.15.22:0/501212071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f124c0135f0 con 0x56272daa21a0 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.841+0000 7f123cff9640 1 -- 172.21.15.22:0/501212071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c003850 con 0x56272daa21a0 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- 172.21.15.22:0/501212071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 msgr2=0x56272daaaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daaaee0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x56272dabe0e0 tx=0x7f124c004730 comp rx=0 tx=0).stop 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- 172.21.15.22:0/501212071 shutdown_connections 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daab6a0 0x56272daa4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daa2ac0 0x56272daa3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 --2- 172.21.15.22:0/501212071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa21a0 0x56272daaaee0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- 172.21.15.22:0/501212071 >> 172.21.15.22:0/501212071 conn(0x56272da9c8d0 msgr2=0x56272daa2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- 172.21.15.22:0/501212071 shutdown_connections 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- 172.21.15.22:0/501212071 wait complete. 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- start start 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 Processor -- start 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272dab10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272dab1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272dab1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12581aee20 con 0x56272daa2ac0 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12581b0120 con 0x56272daa21a0 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12581af460 con 0x56272daab6a0 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125fd6f740 10 client.0 _ll_register_callbacks cb 0x7f1248009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f1256d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272dab1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f1256d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272dab1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38234/0 (socket says 172.21.15.22:38234) 2024-03-08T23:57:07.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f1256d9d640 1 -- 172.21.15.22:0/4047543304 learned_addr learned my addr 172.21.15.22:0/4047543304 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.842+0000 7f125659c640 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272dab1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f125659c640 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272dab10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f1256d9d640 1 -- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 msgr2=0x56272dab1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f1256d9d640 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272dab1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f1256d9d640 1 -- 172.21.15.22:0/4047543304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 msgr2=0x56272dab10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f1256d9d640 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272dab10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f1256d9d640 1 -- 172.21.15.22:0/4047543304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56272db11940 con 0x56272daa2ac0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f1256d9d640 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272dab1620 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1248005ec0 tx=0x7f12480049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33771774d1a0a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f123aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f123aff5640 1 -- 172.21.15.22:0/4047543304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248042040 con 0x56272daa2ac0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f123aff5640 1 -- 172.21.15.22:0/4047543304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f1248004240 con 0x56272daa2ac0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f124c0135f0 con 0x56272daa2ac0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f123aff5640 1 -- 172.21.15.22:0/4047543304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1248004440 con 0x56272daa2ac0 2024-03-08T23:57:07.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f125fd6f740 10 client.16584 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56272db12c20 con 0x56272daa2ac0 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f123aff5640 1 -- 172.21.15.22:0/4047543304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f124c0135f0 con 0x56272daa2ac0 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.843+0000 7f123aff5640 1 -- 172.21.15.22:0/4047543304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f124804e070 con 0x56272daa2ac0 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125fd6f740 10 client.16584 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125fd6f740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125fd6f740 20 client.16584 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125fd6f740 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272db13050 0x56272db15440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125fd6f740 4 client.16584 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56272da0d750 con 0x56272db13050 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125659c640 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272db13050 0x56272db15440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f123aff5640 10 client.16584 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.844+0000 7f125659c640 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272db13050 0x56272db15440 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b255b6956b8a38 server_cookie=75f252bd1864a49 in_seq=0 out_seq=0 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.847+0000 7f123aff5640 1 -- 172.21.15.22:0/4047543304 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56272da0d750 con 0x56272db13050 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.847+0000 7f123aff5640 10 client.16584 handle_command_reply: tid=0 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.847+0000 7f125fd6f740 1 client.16584 shutdown 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.847+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 msgr2=0x56272dab1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.847+0000 7f125fd6f740 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272dab1620 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1248005ec0 tx=0x7f12480049a0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 shutdown_connections 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56272db13050 0x56272db15440 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56272daab6a0 0x56272dab1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56272daa2ac0 0x56272dab1620 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f123aff5640 0 client.16584 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:07.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 --2- 172.21.15.22:0/4047543304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56272daa21a0 0x56272dab10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 >> 172.21.15.22:0/4047543304 conn(0x56272da9c8d0 msgr2=0x56272daa3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 shutdown_connections 2024-03-08T23:57:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 1 -- 172.21.15.22:0/4047543304 wait complete. 2024-03-08T23:57:07.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125fd6f740 20 client.16584 trim_cache size 0 max 16384 2024-03-08T23:57:07.848 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12400a29f0 msgr2=0x7f12400a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12400a29f0 0x7f12400a4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f125000efd0 tx=0x7f1250086040 comp rx=0 tx=0).stop 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 msgr2=0x7f12581028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f12581028d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f125000eaa0 tx=0x7f125000ef60 comp rx=0 tx=0).stop 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 shutdown_connections 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12400a29f0 0x7f12400a4eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1258106280 0x7f12581028d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1258100e00 0x7f1258102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 --2- 172.21.15.22:0/178448293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1258100440 0x7f1258101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:07.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 >> 172.21.15.22:0/178448293 conn(0x7f1258076f90 msgr2=0x7f1258069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:07.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 shutdown_connections 2024-03-08T23:57:07.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:07.848+0000 7f125d0ab640 1 -- 172.21.15.22:0/178448293 wait complete. 2024-03-08T23:57:08.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:08 smithi062 ceph-mon[36129]: pgmap v82: 129 pgs: 129 active+clean; 52 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 315 op/s 2024-03-08T23:57:08.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:08 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3113787931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:57:08.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:08 smithi192 ceph-mon[35995]: pgmap v82: 129 pgs: 129 active+clean; 52 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 315 op/s 2024-03-08T23:57:08.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:08 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3113787931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:57:08.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 187728 13.55 MB/sec warmup 76 sec latency 15.759 ms 2024-03-08T23:57:08.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:08 smithi022 ceph-mon[31894]: pgmap v82: 129 pgs: 129 active+clean; 52 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 315 op/s 2024-03-08T23:57:08.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:08 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3113787931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:57:08.856 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:57:08.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:57:08.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:57:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 -- 172.21.15.22:0/1835129010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8608100d80 msgr2=0x7f8608104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 --2- 172.21.15.22:0/1835129010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8608100d80 0x7f8608104dc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009900 tx=0x7f85f002bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 -- 172.21.15.22:0/1835129010 shutdown_connections 2024-03-08T23:57:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 --2- 172.21.15.22:0/1835129010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86081054a0 0x7f860810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 --2- 172.21.15.22:0/1835129010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8608100d80 0x7f8608104dc0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 --2- 172.21.15.22:0/1835129010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86081003d0 0x7f86081007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 -- 172.21.15.22:0/1835129010 >> 172.21.15.22:0/1835129010 conn(0x7f86080772f0 msgr2=0x7f8608077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 -- 172.21.15.22:0/1835129010 shutdown_connections 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.971+0000 7f860e70a640 1 -- 172.21.15.22:0/1835129010 wait complete. 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 Processor -- start 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 -- start start 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86081003d0 0x7f8608199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86081054a0 0x7f860819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f860819e600 0x7f860819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608110530 con 0x7f86081054a0 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081103b0 con 0x7f86081003d0 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860e70a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081106b0 con 0x7f860819e600 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f86077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86081054a0 0x7f860819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f86077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86081054a0 0x7f860819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38252/0 (socket says 172.21.15.22:38252) 2024-03-08T23:57:08.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f86077fe640 1 -- 172.21.15.22:0/3347591599 learned_addr learned my addr 172.21.15.22:0/3347591599 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f860cca7640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f860819e600 0x7f860819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.972+0000 7f8607fff640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86081003d0 0x7f8608199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86077fe640 1 -- 172.21.15.22:0/3347591599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f860819e600 msgr2=0x7f860819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86077fe640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f860819e600 0x7f860819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86077fe640 1 -- 172.21.15.22:0/3347591599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86081003d0 msgr2=0x7f8608199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86077fe640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86081003d0 0x7f8608199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86077fe640 1 -- 172.21.15.22:0/3347591599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860819f0f0 con 0x7f86081054a0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86077fe640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86081054a0 0x7f860819a280 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f85f0036040 tx=0x7f85f002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86057fa640 1 -- 172.21.15.22:0/3347591599 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0041070 con 0x7f86081054a0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86057fa640 1 -- 172.21.15.22:0/3347591599 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85f00054e0 con 0x7f86081054a0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860819f380 con 0x7f86081054a0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f86057fa640 1 -- 172.21.15.22:0/3347591599 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0034440 con 0x7f86081054a0 2024-03-08T23:57:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.973+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86081a6b40 con 0x7f86081054a0 2024-03-08T23:57:08.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.975+0000 7f86057fa640 1 -- 172.21.15.22:0/3347591599 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f85f0044070 con 0x7f86081054a0 2024-03-08T23:57:08.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86057fa640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f85f40a2970 0x7f85f40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f8607fff640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f85f40a2970 0x7f85f40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 Processor -- start 2024-03-08T23:57:08.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86057fa640 1 -- 172.21.15.22:0/3347591599 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f85f00f0fc0 con 0x7f86081054a0 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 -- start start 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f821160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f829d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f784b40 con 0x55817f820d80 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f72dd80 con 0x55817f82a280 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.976+0000 7f86113c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f780be0 con 0x55817f8216a0 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f85f40a2970 0x7f85f40a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009b60 tx=0x7f85f8005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f821160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f86077fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f829d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f821160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38262/0 (socket says 172.21.15.22:38262) 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 -- 172.21.15.22:0/3413778541 learned_addr learned my addr 172.21.15.22:0/3413778541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f860cca7640 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 -- 172.21.15.22:0/3413778541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 msgr2=0x55817f829d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f829d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 -- 172.21.15.22:0/3413778541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 msgr2=0x55817f82c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 -- 172.21.15.22:0/3413778541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817f5617e0 con 0x55817f820d80 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f86077fe640 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f829d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f8607fff640 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f821160 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011fa0 tx=0x7f85f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2895c2945ddce95b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f85fe7fc640 1 -- 172.21.15.22:0/3413778541 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f801f620 con 0x55817f820d80 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f85fe7fc640 1 -- 172.21.15.22:0/3413778541 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85f801fca0 con 0x55817f820d80 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.977+0000 7f85fe7fc640 1 -- 172.21.15.22:0/3413778541 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f801ed30 con 0x55817f820d80 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 -- 172.21.15.22:0/3413778541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 msgr2=0x55817f821160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f821160 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011fa0 tx=0x7f85f8014000 comp rx=0 tx=0).stop 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 -- 172.21.15.22:0/3413778541 shutdown_connections 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f829d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 --2- 172.21.15.22:0/3413778541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f821160 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 -- 172.21.15.22:0/3413778541 >> 172.21.15.22:0/3413778541 conn(0x55817f81b4b0 msgr2=0x55817f81b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 -- 172.21.15.22:0/3413778541 shutdown_connections 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 -- 172.21.15.22:0/3413778541 wait complete. 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.978+0000 7f86113c2740 1 -- start start 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 1 Processor -- start 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f829ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f827070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f827790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f82d070 con 0x55817f820d80 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f7411c0 con 0x55817f82a280 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f82d1f0 con 0x55817f8216a0 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86113c2740 10 client.0 _ll_register_callbacks cb 0x55817f7307a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f829ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86077fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f827070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f829ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38274/0 (socket says 172.21.15.22:38274) 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 -- 172.21.15.22:0/2283069129 learned_addr learned my addr 172.21.15.22:0/2283069129 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f860cca7640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f827790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 -- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 msgr2=0x55817f827070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f827070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 -- 172.21.15.22:0/2283069129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 msgr2=0x55817f827790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f827790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 -- 172.21.15.22:0/2283069129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55817f5c1f70 con 0x55817f820d80 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f86077fe640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f827070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f860cca7640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f827790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f8607fff640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f829ce0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011870 tx=0x7f85f801e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb70fa599eef4b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.979+0000 7f85e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f85e77fe640 1 -- 172.21.15.22:0/2283069129 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8036060 con 0x55817f820d80 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f85e77fe640 1 -- 172.21.15.22:0/2283069129 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f85f801ed00 con 0x55817f820d80 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85f801fca0 con 0x55817f820d80 2024-03-08T23:57:08.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f85e77fe640 1 -- 172.21.15.22:0/2283069129 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f803d050 con 0x55817f820d80 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55817f459170 con 0x55817f820d80 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f85e77fe640 1 -- 172.21.15.22:0/2283069129 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85f801fca0 con 0x55817f820d80 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f85e77fe640 1 -- 172.21.15.22:0/2283069129 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f85f801f420 con 0x55817f820d80 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 20 client.16596 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f83a370 0x55817f83c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 4 client.16596 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55817f75a960 con 0x55817f83a370 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.980+0000 7f86077fe640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f83a370 0x55817f83c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.981+0000 7f85e77fe640 10 client.16596 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:08.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.981+0000 7f86077fe640 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f83a370 0x55817f83c760 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b1c4a51b03a1ad server_cookie=44a602264b175dff in_seq=0 out_seq=0 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f85e77fe640 1 -- 172.21.15.22:0/2283069129 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55817f75a960 con 0x55817f83a370 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f85e77fe640 10 client.16596 handle_command_reply: tid=0 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 client.16596 shutdown 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 msgr2=0x55817f829ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f829ce0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011870 tx=0x7f85f801e800 comp rx=0 tx=0).stop 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 shutdown_connections 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f83a370 0x55817f83c760 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f827790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f827070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 --2- 172.21.15.22:0/2283069129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f829ce0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f85e77fe640 0 client.16596 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 >> 172.21.15.22:0/2283069129 conn(0x55817f81b4b0 msgr2=0x55817f81d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 shutdown_connections 2024-03-08T23:57:08.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 1 -- 172.21.15.22:0/2283069129 wait complete. 2024-03-08T23:57:08.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:08.983+0000 7f86113c2740 20 client.16596 trim_cache size 0 max 16384 2024-03-08T23:57:08.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 Processor -- start 2024-03-08T23:57:08.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 -- start start 2024-03-08T23:57:08.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f820d80 0x55817f83c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f825f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f82a280 0x55817f8297a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:08.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081b2bb0 con 0x55817f82a280 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081b0750 con 0x55817f820d80 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86113c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081ae7f0 con 0x55817f8216a0 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f82a280 0x55817f8297a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f82a280 0x55817f8297a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38282/0 (socket says 172.21.15.22:38282) 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 -- 172.21.15.22:0/2648071475 learned_addr learned my addr 172.21.15.22:0/2648071475 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f8607fff640 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f820d80 0x55817f83c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f86077fe640 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f825f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 -- 172.21.15.22:0/2648071475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 msgr2=0x55817f825f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f825f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 -- 172.21.15.22:0/2648071475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f820d80 msgr2=0x55817f83c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f820d80 0x55817f83c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:08.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.000+0000 7f860cca7640 1 -- 172.21.15.22:0/2648071475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817f827b00 con 0x55817f82a280 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f860cca7640 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f82a280 0x55817f8297a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8600010750 tx=0x7f8600007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36120b17dce537b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f85fdffb640 1 -- 172.21.15.22:0/2648071475 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600020070 con 0x55817f82a280 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f85fdffb640 1 -- 172.21.15.22:0/2648071475 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8600003d80 con 0x55817f82a280 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f85fdffb640 1 -- 172.21.15.22:0/2648071475 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600004730 con 0x55817f82a280 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 -- 172.21.15.22:0/2648071475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f82a280 msgr2=0x55817f8297a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f82a280 0x55817f8297a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8600010750 tx=0x7f8600007520 comp rx=0 tx=0).stop 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 -- 172.21.15.22:0/2648071475 shutdown_connections 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f82a280 0x55817f8297a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f825f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 --2- 172.21.15.22:0/2648071475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f820d80 0x55817f83c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 -- 172.21.15.22:0/2648071475 >> 172.21.15.22:0/2648071475 conn(0x55817f81b4b0 msgr2=0x55817f825850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 -- 172.21.15.22:0/2648071475 shutdown_connections 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.001+0000 7f86113c2740 1 -- 172.21.15.22:0/2648071475 wait complete. 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 -- start start 2024-03-08T23:57:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 Processor -- start 2024-03-08T23:57:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f82de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f82e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081aedf0 con 0x55817f820d80 2024-03-08T23:57:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081b00f0 con 0x55817f82a280 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081af430 con 0x55817f8216a0 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86113c2740 10 client.0 _ll_register_callbacks cb 0x7f85f80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f8607fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f82de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f8607fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f82de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38286/0 (socket says 172.21.15.22:38286) 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f8607fff640 1 -- 172.21.15.22:0/1613175405 learned_addr learned my addr 172.21.15.22:0/1613175405 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f860cca7640 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.002+0000 7f86077fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f82e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f8607fff640 1 -- 172.21.15.22:0/1613175405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 msgr2=0x55817f82e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f8607fff640 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f82e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f8607fff640 1 -- 172.21.15.22:0/1613175405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 msgr2=0x55817f82e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f8607fff640 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f8607fff640 1 -- 172.21.15.22:0/1613175405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55817f8214d0 con 0x55817f820d80 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f860cca7640 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f8607fff640 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f82de30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011870 tx=0x7f85f80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b82001d59e036 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f85e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f85e77fe640 1 -- 172.21.15.22:0/1613175405 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8046060 con 0x55817f820d80 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f85e77fe640 1 -- 172.21.15.22:0/1613175405 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f85f801f420 con 0x55817f820d80 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8600003d80 con 0x55817f820d80 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f85e77fe640 1 -- 172.21.15.22:0/1613175405 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f804d050 con 0x55817f820d80 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 10 client.16602 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55817f891660 con 0x55817f820d80 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f85e77fe640 1 -- 172.21.15.22:0/1613175405 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8600003d80 con 0x55817f820d80 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f85e77fe640 1 -- 172.21.15.22:0/1613175405 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f85f80035f0 con 0x55817f820d80 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 20 client.16602 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f891d60 0x55817f894180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 4 client.16602 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:57:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.003+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55817f746080 con 0x55817f891d60 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.005+0000 7f86077fe640 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f891d60 0x55817f894180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.006+0000 7f86077fe640 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f891d60 0x55817f894180 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7fb486364c7845 server_cookie=78a1b78d1fae3b63 in_seq=0 out_seq=0 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.006+0000 7f85e77fe640 10 client.16602 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.007+0000 7f85e77fe640 1 -- 172.21.15.22:0/1613175405 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55817f746080 con 0x55817f891d60 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.007+0000 7f85e77fe640 10 client.16602 handle_command_reply: tid=0 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.007+0000 7f86113c2740 1 client.16602 shutdown 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 msgr2=0x55817f82de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f82de30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f85f8011870 tx=0x7f85f80099e0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 shutdown_connections 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55817f891d60 0x55817f894180 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55817f82a280 0x55817f82e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f85e77fe640 0 client.16602 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55817f8216a0 0x55817f82e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 --2- 172.21.15.22:0/1613175405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55817f820d80 0x55817f82de30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 >> 172.21.15.22:0/1613175405 conn(0x55817f81b4b0 msgr2=0x55817f8260e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 shutdown_connections 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 1 -- 172.21.15.22:0/1613175405 wait complete. 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f86113c2740 20 client.16602 trim_cache size 0 max 16384 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "5bbad7b4-1b2b-43c8-b8eb-fa4692a86492", 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:57:09.008 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:57:09.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f85f40a2970 msgr2=0x7f85f40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f860e70a640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f85f40a2970 0x7f85f40a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009b60 tx=0x7f85f8005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:09.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86081054a0 msgr2=0x7f860819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.008+0000 7f860e70a640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86081054a0 0x7f860819a280 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f85f0036040 tx=0x7f85f002cbf0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 shutdown_connections 2024-03-08T23:57:09.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f85f40a2970 0x7f85f40a4e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f860819e600 0x7f860819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86081054a0 0x7f860819a280 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 --2- 172.21.15.22:0/3347591599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86081003d0 0x7f8608199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 >> 172.21.15.22:0/3347591599 conn(0x7f86080772f0 msgr2=0x7f860810c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 shutdown_connections 2024-03-08T23:57:09.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.009+0000 7f860e70a640 1 -- 172.21.15.22:0/3347591599 wait complete. 2024-03-08T23:57:09.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:57:09.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 -- 172.21.15.22:0/662174540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1065c0 msgr2=0x7fcc0c10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 --2- 172.21.15.22:0/662174540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c10e190 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009ca0 tx=0x7fcc0402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 -- 172.21.15.22:0/662174540 shutdown_connections 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 --2- 172.21.15.22:0/662174540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c10e190 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 --2- 172.21.15.22:0/662174540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c101ea0 0x7fcc0c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 --2- 172.21.15.22:0/662174540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c1014f0 0x7fcc0c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 -- 172.21.15.22:0/662174540 >> 172.21.15.22:0/662174540 conn(0x7fcc0c0fcd90 msgr2=0x7fcc0c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 -- 172.21.15.22:0/662174540 shutdown_connections 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.126+0000 7fcc10acb640 1 -- 172.21.15.22:0/662174540 wait complete. 2024-03-08T23:57:09.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 Processor -- start 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 -- start start 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1014f0 0x7fcc0c19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c197a00 0x7fcc0c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c111680 con 0x7fcc0c1014f0 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c111500 con 0x7fcc0c197a00 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc10acb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c111800 con 0x7fcc0c1065c0 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1014f0 0x7fcc0c19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1014f0 0x7fcc0c19d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38310/0 (socket says 172.21.15.22:38310) 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36760/0 (socket says 172.21.15.22:36760) 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc0a59c640 1 -- 172.21.15.22:0/2437044488 learned_addr learned my addr 172.21.15.22:0/2437044488 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c197a00 0x7fcc0c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc09d9b640 1 -- 172.21.15.22:0/2437044488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c197a00 msgr2=0x7fcc0c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c197a00 0x7fcc0c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc09d9b640 1 -- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1014f0 msgr2=0x7fcc0c19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1014f0 0x7fcc0c19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc09d9b640 1 -- 172.21.15.22:0/2437044488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c198520 con 0x7fcc0c1065c0 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.127+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1014f0 0x7fcc0c19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.128+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c19dd40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000e9c0 tx=0x7fcc0000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.128+0000 7fcbfb7fe640 1 -- 172.21.15.22:0/2437044488 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000ccc0 con 0x7fcc0c1065c0 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.128+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c198810 con 0x7fcc0c1065c0 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.128+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c1a4a10 con 0x7fcc0c1065c0 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.128+0000 7fcbfb7fe640 1 -- 172.21.15.22:0/2437044488 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc00005020 con 0x7fcc0c1065c0 2024-03-08T23:57:09.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.128+0000 7fcbfb7fe640 1 -- 172.21.15.22:0/2437044488 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0001c070 con 0x7fcc0c1065c0 2024-03-08T23:57:09.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.130+0000 7fcbfb7fe640 1 -- 172.21.15.22:0/2437044488 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fcc00005540 con 0x7fcc0c1065c0 2024-03-08T23:57:09.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.130+0000 7fcc13783740 1 Processor -- start 2024-03-08T23:57:09.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcbfb7fe640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbf40a2970 0x7fcbf40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc13783740 1 -- start start 2024-03-08T23:57:09.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc13783740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8776fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbf40a2970 0x7fcbf40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc13783740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87778720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc13783740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8777b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcbfb7fe640 1 -- 172.21.15.22:0/2437044488 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcc000cf100 con 0x7fcc0c1065c0 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc13783740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b876d2e00 con 0x560b87770080 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc13783740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8767bb10 con 0x560b8776f760 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc13783740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b876ceea0 con 0x560b87778c60 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87778720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87778720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38324/0 (socket says 172.21.15.22:38324) 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 -- 172.21.15.22:0/2003035447 learned_addr learned my addr 172.21.15.22:0/2003035447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8776fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8777b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 -- 172.21.15.22:0/2003035447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 msgr2=0x560b8777b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8777b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 -- 172.21.15.22:0/2003035447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 msgr2=0x560b8776fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8776fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 -- 172.21.15.22:0/2003035447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b874b17d0 con 0x560b87770080 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8776fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbf40a2970 0x7fcbf40a4e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc04004af0 tx=0x7fcc0402ce80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.131+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87778720 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcc000a3630 tx=0x7fcc0009b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2baa7cfaddf2d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcbf8ff9640 1 -- 172.21.15.22:0/2003035447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000b3020 con 0x560b87770080 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcbf8ff9640 1 -- 172.21.15.22:0/2003035447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc000b4260 con 0x560b87770080 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcbf8ff9640 1 -- 172.21.15.22:0/2003035447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00097bc0 con 0x560b87770080 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 -- 172.21.15.22:0/2003035447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 msgr2=0x560b87778720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87778720 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcc000a3630 tx=0x7fcc0009b030 comp rx=0 tx=0).stop 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 -- 172.21.15.22:0/2003035447 shutdown_connections 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8777b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87778720 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 --2- 172.21.15.22:0/2003035447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8776fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 -- 172.21.15.22:0/2003035447 >> 172.21.15.22:0/2003035447 conn(0x560b87769e90 msgr2=0x560b8776a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 -- 172.21.15.22:0/2003035447 shutdown_connections 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.132+0000 7fcc13783740 1 -- 172.21.15.22:0/2003035447 wait complete. 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 -- start start 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 Processor -- start 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b87778470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87775800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b87775d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8777ba50 con 0x560b87770080 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8768f0a0 con 0x560b8776f760 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8777bbd0 con 0x560b87778c60 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87775800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc13783740 10 client.0 _ll_register_callbacks cb 0x560b8768ea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87775800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38330/0 (socket says 172.21.15.22:38330) 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 -- 172.21.15.22:0/4266725433 learned_addr learned my addr 172.21.15.22:0/4266725433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b87778470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b87775d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 -- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 msgr2=0x560b87775d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b87775d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 -- 172.21.15.22:0/4266725433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 msgr2=0x560b87778470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b87778470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.133+0000 7fcc09d9b640 1 -- 172.21.15.22:0/4266725433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b873e73f0 con 0x560b87770080 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87775800 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcc0009b300 tx=0x7fcc0009f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959b49eab5b903ef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcbf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/4266725433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000b3070 con 0x560b87770080 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b876e0340 con 0x560b87770080 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/4266725433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcc000a38e0 con 0x560b87770080 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/4266725433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc00097b80 con 0x560b87770080 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 10 client.16614 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560b873a70f0 con 0x560b87770080 2024-03-08T23:57:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/4266725433 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560b876e0340 con 0x560b87770080 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/4266725433 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc0009a280 con 0x560b87770080 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 10 client.16614 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 20 client.16614 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b87788660 0x560b8778aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 4 client.16614 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b876da690 con 0x560b87788660 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.134+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b87788660 0x560b8778aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.135+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b87788660 0x560b8778aa50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440a5a7df74deed0 server_cookie=b7c193052c3319c5 in_seq=0 out_seq=0 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.135+0000 7fcbf1ffb640 10 client.16614 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.135+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/4266725433 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b876da690 con 0x560b87788660 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.135+0000 7fcbf1ffb640 10 client.16614 handle_command_reply: tid=0 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.135+0000 7fcc13783740 1 client.16614 shutdown 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 msgr2=0x560b87775800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87775800 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcc0009b300 tx=0x7fcc0009f3f0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 shutdown_connections 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b87788660 0x560b8778aa50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b87775d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b87775800 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcbf1ffb640 0 client.16614 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 --2- 172.21.15.22:0/4266725433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b87778470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 >> 172.21.15.22:0/4266725433 conn(0x560b87769e90 msgr2=0x560b8776bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 shutdown_connections 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 1 -- 172.21.15.22:0/4266725433 wait complete. 2024-03-08T23:57:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.136+0000 7fcc13783740 20 client.16614 trim_cache size 0 max 16384 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 Processor -- start 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 -- start start 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b8776f760 0x560b87770cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87770080 0x560b87777f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b87778c60 0x560b87778470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1b0a10 con 0x560b8776f760 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1ae5b0 con 0x560b87778c60 2024-03-08T23:57:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc13783740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1ac650 con 0x560b87770080 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b8776f760 0x560b87770cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b8776f760 0x560b87770cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38342/0 (socket says 172.21.15.22:38342) 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 -- 172.21.15.22:0/2690564419 learned_addr learned my addr 172.21.15.22:0/2690564419 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87770080 0x560b87777f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b87778c60 0x560b87778470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 -- 172.21.15.22:0/2690564419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87770080 msgr2=0x560b87777f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87770080 0x560b87777f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 -- 172.21.15.22:0/2690564419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b87778c60 msgr2=0x560b87778470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b87778c60 0x560b87778470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0a59c640 1 -- 172.21.15.22:0/2690564419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8777b370 con 0x560b8776f760 2024-03-08T23:57:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87770080 0x560b87777f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.151+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b87778c60 0x560b87778470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b8776f760 0x560b87770cb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc000c00 tx=0x7fcbfc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70d9c8d410f26e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcbf3fff640 1 -- 172.21.15.22:0/2690564419 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc025070 con 0x560b8776f760 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcbf3fff640 1 -- 172.21.15.22:0/2690564419 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc005500 con 0x560b8776f760 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcbf3fff640 1 -- 172.21.15.22:0/2690564419 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc003850 con 0x560b8776f760 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 -- 172.21.15.22:0/2690564419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b8776f760 msgr2=0x560b87770cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b8776f760 0x560b87770cb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc000c00 tx=0x7fcbfc005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 -- 172.21.15.22:0/2690564419 shutdown_connections 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b87778c60 0x560b87778470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87770080 0x560b87777f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 --2- 172.21.15.22:0/2690564419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b8776f760 0x560b87770cb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 -- 172.21.15.22:0/2690564419 >> 172.21.15.22:0/2690564419 conn(0x560b87769e90 msgr2=0x560b877705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 -- 172.21.15.22:0/2690564419 shutdown_connections 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.152+0000 7fcc13783740 1 -- 172.21.15.22:0/2690564419 wait complete. 2024-03-08T23:57:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 -- start start 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 Processor -- start 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8778cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b8778b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8778ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1acc50 con 0x560b87770080 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1adf50 con 0x560b8776f760 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c1ad290 con 0x560b87778c60 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc13783740 10 client.0 _ll_register_callbacks cb 0x7fcbfc009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b8778b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b8778b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38352/0 (socket says 172.21.15.22:38352) 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 -- 172.21.15.22:0/940965950 learned_addr learned my addr 172.21.15.22:0/940965950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc0ad9d640 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8778ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8778cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 -- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 msgr2=0x560b8778ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8778ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 -- 172.21.15.22:0/940965950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 msgr2=0x560b8778cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8778cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 -- 172.21.15.22:0/940965950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b877de740 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.153+0000 7fcc09d9b640 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b8778b2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcc0009b0c0 tx=0x7fcc00097710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109fd67d98f68b7b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcbf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/940965950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000b3070 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/940965950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fcc0009a810 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbfc005500 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/940965950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc000051d0 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 10 client.16620 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560b877de740 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/940965950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc005500 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/940965950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc00097110 con 0x560b87770080 2024-03-08T23:57:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 10 client.16620 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 20 client.16620 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b877dfb90 0x560b877e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 4 client.16620 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b876584e0 con 0x560b877dfb90 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.154+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b877dfb90 0x560b877e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc0a59c640 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b877dfb90 0x560b877e1fb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b691508126c0d server_cookie=c8836233313dd9e8 in_seq=0 out_seq=0 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcbf1ffb640 10 client.16620 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcbf1ffb640 1 -- 172.21.15.22:0/940965950 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560b876584e0 con 0x560b877dfb90 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcbf1ffb640 10 client.16620 handle_command_reply: tid=0 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 client.16620 shutdown 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 msgr2=0x560b8778b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b8778b2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcc0009b0c0 tx=0x7fcc00097710 comp rx=0 tx=0).stop 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 shutdown_connections 2024-03-08T23:57:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560b877dfb90 0x560b877e1fb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560b87778c60 0x560b8778ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560b87770080 0x560b8778b2f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcbf1ffb640 0 client.16620 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 --2- 172.21.15.22:0/940965950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8776f760 0x560b8778cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 >> 172.21.15.22:0/940965950 conn(0x560b87769e90 msgr2=0x560b87770e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 shutdown_connections 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 1 -- 172.21.15.22:0/940965950 wait complete. 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.155+0000 7fcc13783740 20 client.16620 trim_cache size 0 max 16384 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:57:09.158 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "5bbad7b4-1b2b-43c8-b8eb-fa4692a86492": { 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "5bbad7b4-1b2b-43c8-b8eb-fa4692a86492", 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:57:09.159 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbf40a2970 msgr2=0x7fcbf40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbf40a2970 0x7fcbf40a4e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc04004af0 tx=0x7fcc0402ce80 comp rx=0 tx=0).stop 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c1065c0 msgr2=0x7fcc0c19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c19dd40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000e9c0 tx=0x7fcc0000ee80 comp rx=0 tx=0).stop 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 shutdown_connections 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbf40a2970 0x7fcbf40a4e30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c197a00 0x7fcc0c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc0c1065c0 0x7fcc0c19dd40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 --2- 172.21.15.22:0/2437044488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc0c1014f0 0x7fcc0c19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 >> 172.21.15.22:0/2437044488 conn(0x7fcc0c0fcd90 msgr2=0x7fcc0c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:09.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 shutdown_connections 2024-03-08T23:57:09.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:09.156+0000 7fcc10acb640 1 -- 172.21.15.22:0/2437044488 wait complete. 2024-03-08T23:57:09.162 INFO:tasks.cephfs.filesystem:scrub status for tag:5bbad7b4-1b2b-43c8-b8eb-fa4692a86492 - {'path': '/', 'tag': '5bbad7b4-1b2b-43c8-b8eb-fa4692a86492', 'options': 'recursive,force'} 2024-03-08T23:57:09.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 189129 13.46 MB/sec warmup 77 sec latency 99.239 ms 2024-03-08T23:57:10.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:10 smithi062 ceph-mon[36129]: pgmap v83: 129 pgs: 129 active+clean; 47 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-03-08T23:57:10.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:10 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:57:10.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:10 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:10.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:10 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:57:10.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:10 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:10.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:10 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:57:10.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:10 smithi192 ceph-mon[35995]: pgmap v83: 129 pgs: 129 active+clean; 47 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-03-08T23:57:10.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:10 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:57:10.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:10 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:10.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:10 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:57:10.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:10 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:10.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:10 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:57:10.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 191906 13.51 MB/sec warmup 78 sec latency 103.561 ms 2024-03-08T23:57:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:10 smithi022 ceph-mon[31894]: pgmap v83: 129 pgs: 129 active+clean; 47 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-03-08T23:57:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:10 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:57:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:10 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:10 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:57:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:10 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:10 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:57:11.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 194714 13.50 MB/sec warmup 79 sec latency 13.855 ms 2024-03-08T23:57:12.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:12 smithi062 ceph-mon[36129]: pgmap v84: 129 pgs: 129 active+clean; 45 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 468 op/s 2024-03-08T23:57:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:12 smithi192 ceph-mon[35995]: pgmap v84: 129 pgs: 129 active+clean; 45 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 468 op/s 2024-03-08T23:57:12.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 196705 13.47 MB/sec warmup 80 sec latency 13.529 ms 2024-03-08T23:57:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:12 smithi022 ceph-mon[31894]: pgmap v84: 129 pgs: 129 active+clean; 45 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 468 op/s 2024-03-08T23:57:13.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 199224 13.48 MB/sec warmup 81 sec latency 15.345 ms 2024-03-08T23:57:14.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:14 smithi062 ceph-mon[36129]: pgmap v85: 129 pgs: 129 active+clean; 45 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 436 op/s 2024-03-08T23:57:14.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:14 smithi192 ceph-mon[35995]: pgmap v85: 129 pgs: 129 active+clean; 45 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 436 op/s 2024-03-08T23:57:14.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 202163 13.53 MB/sec warmup 82 sec latency 29.415 ms 2024-03-08T23:57:14.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:14 smithi022 ceph-mon[31894]: pgmap v85: 129 pgs: 129 active+clean; 45 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 436 op/s 2024-03-08T23:57:15.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 203417 13.41 MB/sec warmup 83 sec latency 109.495 ms 2024-03-08T23:57:16.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:16 smithi192 ceph-mon[35995]: pgmap v86: 129 pgs: 129 active+clean; 51 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 572 op/s 2024-03-08T23:57:16.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 206156 13.45 MB/sec warmup 84 sec latency 17.917 ms 2024-03-08T23:57:16.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:16 smithi022 ceph-mon[31894]: pgmap v86: 129 pgs: 129 active+clean; 51 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 572 op/s 2024-03-08T23:57:16.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:16 smithi062 ceph-mon[36129]: pgmap v86: 129 pgs: 129 active+clean; 51 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 572 op/s 2024-03-08T23:57:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 -- 172.21.15.22:0/3609824551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 msgr2=0x7f9978101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3609824551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f9978101740 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997402bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 -- 172.21.15.22:0/3609824551 shutdown_connections 2024-03-08T23:57:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3609824551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9978106430 0x7f997810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3609824551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9978101d10 0x7f9978105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3609824551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f9978101740 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 -- 172.21.15.22:0/3609824551 >> 172.21.15.22:0/3609824551 conn(0x7f99780fcc40 msgr2=0x7f99780ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 -- 172.21.15.22:0/3609824551 shutdown_connections 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.176+0000 7f997e9a2640 1 -- 172.21.15.22:0/3609824551 wait complete. 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 Processor -- start 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 -- start start 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f997819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9978106430 0x7f997819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9978197810 0x7f9978197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978111510 con 0x7f9978101360 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978111390 con 0x7f9978197810 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e9a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978111690 con 0x7f9978106430 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f997819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f997819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56012/0 (socket says 172.21.15.22:56012) 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 -- 172.21.15.22:0/3092349583 learned_addr learned my addr 172.21.15.22:0/3092349583 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d19f640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9978106430 0x7f997819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997e1a1640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9978197810 0x7f9978197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 -- 172.21.15.22:0/3092349583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9978106430 msgr2=0x7f997819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9978106430 0x7f997819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 -- 172.21.15.22:0/3092349583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9978197810 msgr2=0x7f9978197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9978197810 0x7f9978197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.177+0000 7f997d9a0640 1 -- 172.21.15.22:0/3092349583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99781983a0 con 0x7f9978101360 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.178+0000 7f997d9a0640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f997819d6a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f997402c240 tx=0x7f99740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.178+0000 7f996effd640 1 -- 172.21.15.22:0/3092349583 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974043020 con 0x7f9978101360 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.178+0000 7f996effd640 1 -- 172.21.15.22:0/3092349583 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99740054e0 con 0x7f9978101360 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.178+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9978198630 con 0x7f9978101360 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.178+0000 7f996effd640 1 -- 172.21.15.22:0/3092349583 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974003b30 con 0x7f9978101360 2024-03-08T23:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.178+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781a4810 con 0x7f9978101360 2024-03-08T23:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.180+0000 7f996effd640 1 -- 172.21.15.22:0/3092349583 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9974046020 con 0x7f9978101360 2024-03-08T23:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 Processor -- start 2024-03-08T23:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 -- start start 2024-03-08T23:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f996effd640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99640a2960 0x7f99640a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326246c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326247630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x561326252b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613261ab5f0 con 0x5613262471b0 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561326154930 con 0x561326247f20 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f99827e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613261a7690 con 0x561326250730 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f997e1a1640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99640a2960 0x7f99640a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f997d19f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326247630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f996effd640 1 -- 172.21.15.22:0/3092349583 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f99740f1020 con 0x7f9978101360 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f997d19f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326247630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56018/0 (socket says 172.21.15.22:56018) 2024-03-08T23:57:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f997d19f640 1 -- 172.21.15.22:0/2434597819 learned_addr learned my addr 172.21.15.22:0/2434597819 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f997e1a1640 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x561326252b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.181+0000 7f997d9a0640 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326246c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f997d19f640 1 -- 172.21.15.22:0/2434597819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 msgr2=0x561326252b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f997d19f640 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x561326252b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f997d19f640 1 -- 172.21.15.22:0/2434597819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 msgr2=0x561326246c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f997d19f640 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326246c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f997d19f640 1 -- 172.21.15.22:0/2434597819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561325ec0210 con 0x5613262471b0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f997e1a1640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99640a2960 0x7f99640a4e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9970005c40 tx=0x7f9970009d70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f997d19f640 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326247630 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9968009c20 tx=0x7f9968011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b43a65fdfcc721 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f9963fff640 1 -- 172.21.15.22:0/2434597819 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968004230 con 0x5613262471b0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f9963fff640 1 -- 172.21.15.22:0/2434597819 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99680043d0 con 0x5613262471b0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.182+0000 7f9963fff640 1 -- 172.21.15.22:0/2434597819 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968019820 con 0x5613262471b0 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 -- 172.21.15.22:0/2434597819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 msgr2=0x561326247630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326247630 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9968009c20 tx=0x7f9968011810 comp rx=0 tx=0).stop 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 -- 172.21.15.22:0/2434597819 shutdown_connections 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x561326252b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326247630 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 --2- 172.21.15.22:0/2434597819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326246c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 -- 172.21.15.22:0/2434597819 >> 172.21.15.22:0/2434597819 conn(0x56132620e5b0 msgr2=0x561326241ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 -- 172.21.15.22:0/2434597819 shutdown_connections 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 -- 172.21.15.22:0/2434597819 wait complete. 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 -- start start 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.183+0000 7f99827e3740 1 Processor -- start 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613262471b0 0x56132624d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326247f20 0x56132624dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561326250730 0x56132624e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561326253460 con 0x561326250730 2024-03-08T23:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561326167d90 con 0x5613262471b0 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613262535e0 con 0x561326247f20 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 10 client.0 _ll_register_callbacks cb 0x561325ebfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561326250730 0x56132624e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561326250730 0x56132624e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56022/0 (socket says 172.21.15.22:56022) 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 -- 172.21.15.22:0/2677512320 learned_addr learned my addr 172.21.15.22:0/2677512320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997d19f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326247f20 0x56132624dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997d9a0640 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613262471b0 0x56132624d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 -- 172.21.15.22:0/2677512320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326247f20 msgr2=0x56132624dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326247f20 0x56132624dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 -- 172.21.15.22:0/2677512320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613262471b0 msgr2=0x56132624d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613262471b0 0x56132624d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 -- 172.21.15.22:0/2677512320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561325f8c620 con 0x561326250730 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997d9a0640 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613262471b0 0x56132624d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f997e1a1640 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561326250730 0x56132624e1e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f997000edd0 tx=0x7f9970004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8e714ad134b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99617fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99617fa640 1 -- 172.21.15.22:0/2677512320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99700045c0 con 0x561326250730 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99617fa640 1 -- 172.21.15.22:0/2677512320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9970004760 con 0x561326250730 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99680043d0 con 0x561326250730 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.184+0000 7f99617fa640 1 -- 172.21.15.22:0/2677512320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9970031050 con 0x561326250730 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 10 client.16632 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56132625f740 con 0x561326250730 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99617fa640 1 -- 172.21.15.22:0/2677512320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99680043d0 con 0x561326250730 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99617fa640 1 -- 172.21.15.22:0/2677512320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f997001f050 con 0x561326250730 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 20 client.16632 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561326260c00 0x561326262ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 4 client.16632 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613261b8b30 con 0x561326260c00 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.185+0000 7f997d9a0640 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561326260c00 0x561326262ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.186+0000 7f99617fa640 10 client.16632 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.186+0000 7f997d9a0640 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561326260c00 0x561326262ff0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf40e32e9d05dc05 server_cookie=15799660ed1972b8 in_seq=0 out_seq=0 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99617fa640 1 -- 172.21.15.22:0/2677512320 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5613261b8b30 con 0x561326260c00 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99617fa640 10 client.16632 handle_command_reply: tid=0 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 client.16632 shutdown 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561326250730 msgr2=0x56132624e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561326250730 0x56132624e1e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f997000edd0 tx=0x7f9970004ab0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 shutdown_connections 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561326260c00 0x561326262ff0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561326250730 0x56132624e1e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326247f20 0x56132624dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99617fa640 0 client.16632 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 --2- 172.21.15.22:0/2677512320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613262471b0 0x56132624d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 >> 172.21.15.22:0/2677512320 conn(0x56132620e5b0 msgr2=0x561326243bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 shutdown_connections 2024-03-08T23:57:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 1 -- 172.21.15.22:0/2677512320 wait complete. 2024-03-08T23:57:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.187+0000 7f99827e3740 20 client.16632 trim_cache size 0 max 16384 2024-03-08T23:57:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 Processor -- start 2024-03-08T23:57:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 -- start start 2024-03-08T23:57:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x56132624fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x56132624a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x56132624af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781b0880 con 0x5613262471b0 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781ae420 con 0x561326247f20 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f99827e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781ac4c0 con 0x561326250730 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f997d9a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x56132624fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f997d9a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x56132624fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56034/0 (socket says 172.21.15.22:56034) 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f997d9a0640 1 -- 172.21.15.22:0/1934219366 learned_addr learned my addr 172.21.15.22:0/1934219366 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f997d19f640 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x56132624a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.201+0000 7f997e1a1640 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x56132624af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f997d9a0640 1 -- 172.21.15.22:0/1934219366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 msgr2=0x56132624af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f997d9a0640 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x56132624af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f997d9a0640 1 -- 172.21.15.22:0/1934219366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 msgr2=0x56132624a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f997d9a0640 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x56132624a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f997d9a0640 1 -- 172.21.15.22:0/1934219366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132623c2f0 con 0x5613262471b0 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f997d9a0640 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x56132624fe90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f99740ee540 tx=0x7f997402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1222f84d232cc9d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f99637fe640 1 -- 172.21.15.22:0/1934219366 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740bdaf0 con 0x5613262471b0 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f99637fe640 1 -- 172.21.15.22:0/1934219366 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99740bd3f0 con 0x5613262471b0 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.202+0000 7f99637fe640 1 -- 172.21.15.22:0/1934219366 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974003cd0 con 0x5613262471b0 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- 172.21.15.22:0/1934219366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 msgr2=0x56132624fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x56132624fe90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f99740ee540 tx=0x7f997402bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- 172.21.15.22:0/1934219366 shutdown_connections 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x56132624af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x56132624a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 --2- 172.21.15.22:0/1934219366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x56132624fe90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- 172.21.15.22:0/1934219366 >> 172.21.15.22:0/1934219366 conn(0x56132620e5b0 msgr2=0x56132624c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- 172.21.15.22:0/1934219366 shutdown_connections 2024-03-08T23:57:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- 172.21.15.22:0/1934219366 wait complete. 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- start start 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 Processor -- start 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326256a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326256f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x5613262b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781acac0 con 0x5613262471b0 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781addc0 con 0x561326247f20 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781ad100 con 0x561326250730 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f99827e3740 10 client.0 _ll_register_callbacks cb 0x7f99680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.203+0000 7f997d9a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326256a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326256a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56040/0 (socket says 172.21.15.22:56040) 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 -- 172.21.15.22:0/3621193617 learned_addr learned my addr 172.21.15.22:0/3621193617 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997e1a1640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x5613262b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d19f640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326256f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 -- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 msgr2=0x5613262b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x5613262b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 -- 172.21.15.22:0/3621193617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 msgr2=0x561326256f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326256f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 -- 172.21.15.22:0/3621193617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561326246480 con 0x5613262471b0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d19f640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326256f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f997d9a0640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326256a20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f997402c820 tx=0x7f9974009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e889ebd04b8d5ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99617fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99617fa640 1 -- 172.21.15.22:0/3621193617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740c5b90 con 0x5613262471b0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99617fa640 1 -- 172.21.15.22:0/3621193617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9974003cd0 con 0x5613262471b0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99740bd3f0 con 0x5613262471b0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99617fa640 1 -- 172.21.15.22:0/3621193617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99740bdaf0 con 0x5613262471b0 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99827e3740 10 client.16638 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5613262b8760 con 0x5613262471b0 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.204+0000 7f99617fa640 1 -- 172.21.15.22:0/3621193617 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99740bd3f0 con 0x5613262471b0 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99617fa640 1 -- 172.21.15.22:0/3621193617 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f99740b9a60 con 0x5613262471b0 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99827e3740 10 client.16638 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99827e3740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99827e3740 20 client.16638 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99827e3740 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5613262b8d70 0x5613262bb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99827e3740 4 client.16638 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5613261b2e80 con 0x5613262b8d70 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f997d19f640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5613262b8d70 0x5613262bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f997d19f640 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5613262b8d70 0x5613262bb190 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9135fce39a05e4c1 server_cookie=45067c4145767b30 in_seq=0 out_seq=0 2024-03-08T23:57:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.205+0000 7f99617fa640 10 client.16638 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99617fa640 1 -- 172.21.15.22:0/3621193617 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5613261b2e80 con 0x5613262b8d70 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99617fa640 10 client.16638 handle_command_reply: tid=0 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 client.16638 shutdown 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 msgr2=0x561326256a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326256a20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f997402c820 tx=0x7f9974009260 comp rx=0 tx=0).stop 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 shutdown_connections 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5613262b8d70 0x5613262bb190 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561326250730 0x5613262b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561326247f20 0x561326256f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 --2- 172.21.15.22:0/3621193617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5613262471b0 0x561326256a20 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.445+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 >> 172.21.15.22:0/3621193617 conn(0x56132620e5b0 msgr2=0x561326243b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f99617fa640 0 client.16638 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 shutdown_connections 2024-03-08T23:57:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f99827e3740 1 -- 172.21.15.22:0/3621193617 wait complete. 2024-03-08T23:57:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f99827e3740 20 client.16638 trim_cache size 0 max 16384 2024-03-08T23:57:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:57:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:57:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:57:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99640a2960 msgr2=0x7f99640a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99640a2960 0x7f99640a4e20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9970005c40 tx=0x7f9970009d70 comp rx=0 tx=0).stop 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 msgr2=0x7f997819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f997819d6a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f997402c240 tx=0x7f99740042f0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.446+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 shutdown_connections 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.447+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99640a2960 0x7f99640a4e20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.447+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9978197810 0x7f9978197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.447+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9978106430 0x7f997819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.447+0000 7f997e9a2640 1 --2- 172.21.15.22:0/3092349583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9978101360 0x7f997819d6a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.447+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 >> 172.21.15.22:0/3092349583 conn(0x7f99780fcc40 msgr2=0x7f99780fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.447+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 shutdown_connections 2024-03-08T23:57:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:17.447+0000 7f997e9a2640 1 -- 172.21.15.22:0/3092349583 wait complete. 2024-03-08T23:57:17.707 INFO:tasks.workunit.client.0.smithi022.stdout: 1 208219 13.39 MB/sec warmup 85 sec latency 135.934 ms 2024-03-08T23:57:18.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:18 smithi192 ceph-mon[35995]: pgmap v87: 129 pgs: 129 active+clean; 51 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 388 op/s 2024-03-08T23:57:18.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 210258 13.37 MB/sec warmup 86 sec latency 13.478 ms 2024-03-08T23:57:18.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:18 smithi022 ceph-mon[31894]: pgmap v87: 129 pgs: 129 active+clean; 51 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 388 op/s 2024-03-08T23:57:18.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:18 smithi062 ceph-mon[36129]: pgmap v87: 129 pgs: 129 active+clean; 51 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 388 op/s 2024-03-08T23:57:19.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 213202 13.43 MB/sec warmup 87 sec latency 17.391 ms 2024-03-08T23:57:20.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:20 smithi192 ceph-mon[35995]: pgmap v88: 129 pgs: 129 active+clean; 48 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-03-08T23:57:20.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 215304 13.36 MB/sec warmup 88 sec latency 58.096 ms 2024-03-08T23:57:20.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:20 smithi022 ceph-mon[31894]: pgmap v88: 129 pgs: 129 active+clean; 48 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-03-08T23:57:20.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:20 smithi062 ceph-mon[36129]: pgmap v88: 129 pgs: 129 active+clean; 48 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 490 op/s 2024-03-08T23:57:21.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 217212 13.35 MB/sec warmup 89 sec latency 13.045 ms 2024-03-08T23:57:22.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:22 smithi192 ceph-mon[35995]: pgmap v89: 129 pgs: 129 active+clean; 46 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 585 op/s 2024-03-08T23:57:22.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 220244 13.40 MB/sec warmup 90 sec latency 13.099 ms 2024-03-08T23:57:22.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:22 smithi022 ceph-mon[31894]: pgmap v89: 129 pgs: 129 active+clean; 46 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 585 op/s 2024-03-08T23:57:22.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:22 smithi062 ceph-mon[36129]: pgmap v89: 129 pgs: 129 active+clean; 46 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 585 op/s 2024-03-08T23:57:23.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 222879 13.40 MB/sec warmup 91 sec latency 6.453 ms 2024-03-08T23:57:24.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:24 smithi192 ceph-mon[35995]: pgmap v90: 129 pgs: 129 active+clean; 46 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 431 op/s 2024-03-08T23:57:24.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 224939 13.38 MB/sec warmup 92 sec latency 20.583 ms 2024-03-08T23:57:24.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:24 smithi022 ceph-mon[31894]: pgmap v90: 129 pgs: 129 active+clean; 46 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 431 op/s 2024-03-08T23:57:24.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:24 smithi062 ceph-mon[36129]: pgmap v90: 129 pgs: 129 active+clean; 46 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 431 op/s 2024-03-08T23:57:25.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 227359 13.38 MB/sec warmup 93 sec latency 65.023 ms 2024-03-08T23:57:26.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:26 smithi192 ceph-mon[35995]: pgmap v91: 129 pgs: 129 active+clean; 51 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 560 op/s 2024-03-08T23:57:26.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 229934 13.36 MB/sec warmup 94 sec latency 24.353 ms 2024-03-08T23:57:26.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:26 smithi022 ceph-mon[31894]: pgmap v91: 129 pgs: 129 active+clean; 51 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 560 op/s 2024-03-08T23:57:26.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:26 smithi062 ceph-mon[36129]: pgmap v91: 129 pgs: 129 active+clean; 51 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 560 op/s 2024-03-08T23:57:27.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 231783 13.33 MB/sec warmup 95 sec latency 18.196 ms 2024-03-08T23:57:28.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:28 smithi192 ceph-mon[35995]: pgmap v92: 129 pgs: 129 active+clean; 51 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 425 op/s 2024-03-08T23:57:28.663 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:28 smithi062 ceph-mon[36129]: pgmap v92: 129 pgs: 129 active+clean; 51 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 425 op/s 2024-03-08T23:57:28.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 234550 13.37 MB/sec warmup 96 sec latency 49.230 ms 2024-03-08T23:57:28.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:28 smithi022 ceph-mon[31894]: pgmap v92: 129 pgs: 129 active+clean; 51 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 425 op/s 2024-03-08T23:57:29.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 237320 13.36 MB/sec warmup 97 sec latency 13.897 ms 2024-03-08T23:57:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 -- 172.21.15.22:0/2051854929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 msgr2=0x7f3038100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 --2- 172.21.15.22:0/2051854929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f3038100810 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3020009900 tx=0x7f302002bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 -- 172.21.15.22:0/2051854929 shutdown_connections 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 --2- 172.21.15.22:0/2051854929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038105500 0x7f303810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 --2- 172.21.15.22:0/2051854929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038100de0 0x7f3038104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 --2- 172.21.15.22:0/2051854929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f3038100810 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 -- 172.21.15.22:0/2051854929 >> 172.21.15.22:0/2051854929 conn(0x7f3038077320 msgr2=0x7f3038077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 -- 172.21.15.22:0/2051854929 shutdown_connections 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.542+0000 7f303e6dc640 1 -- 172.21.15.22:0/2051854929 wait complete. 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 Processor -- start 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 -- start start 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f303819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038105500 0x7f303819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038196830 0x7f3038196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3038110610 con 0x7f3038100430 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3038110490 con 0x7f3038196830 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303e6dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3038110790 con 0x7f3038105500 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f3037fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f303819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f3037fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f303819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41696/0 (socket says 172.21.15.22:41696) 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f3037fff640 1 -- 172.21.15.22:0/886596116 learned_addr learned my addr 172.21.15.22:0/886596116 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f303cc79640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038196830 0x7f3038196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.543+0000 7f30377fe640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038105500 0x7f303819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f3037fff640 1 -- 172.21.15.22:0/886596116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038105500 msgr2=0x7f303819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f3037fff640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038105500 0x7f303819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f3037fff640 1 -- 172.21.15.22:0/886596116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038196830 msgr2=0x7f3038196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f3037fff640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038196830 0x7f3038196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f3037fff640 1 -- 172.21.15.22:0/886596116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30381973c0 con 0x7f3038100430 2024-03-08T23:57:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f3037fff640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f303819c670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f302002c240 tx=0x7f30200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f30357fa640 1 -- 172.21.15.22:0/886596116 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020043020 con 0x7f3038100430 2024-03-08T23:57:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3038197650 con 0x7f3038100430 2024-03-08T23:57:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f30357fa640 1 -- 172.21.15.22:0/886596116 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30200054e0 con 0x7f3038100430 2024-03-08T23:57:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f30357fa640 1 -- 172.21.15.22:0/886596116 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020003b30 con 0x7f3038100430 2024-03-08T23:57:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.544+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30381a37e0 con 0x7f3038100430 2024-03-08T23:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.546+0000 7f30357fa640 1 -- 172.21.15.22:0/886596116 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3020046020 con 0x7f3038100430 2024-03-08T23:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 Processor -- start 2024-03-08T23:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 -- start start 2024-03-08T23:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f30357fa640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30240a2a10 0x7f30240a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd24450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd31b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306dc885f0 con 0x56306dd24d40 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306dc31930 con 0x56306dd23fd0 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3041399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306dc84690 con 0x56306dd2f760 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f303cc79640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30240a2a10 0x7f30240a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f30357fa640 1 -- 172.21.15.22:0/886596116 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f30200f1030 con 0x7f3038100430 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3037fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3037fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd23a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41712/0 (socket says 172.21.15.22:41712) 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f3037fff640 1 -- 172.21.15.22:0/1747361791 learned_addr learned my addr 172.21.15.22:0/1747361791 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f303cc79640 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd31b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.547+0000 7f30377fe640 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd24450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f3037fff640 1 -- 172.21.15.22:0/1747361791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 msgr2=0x56306dd31b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f3037fff640 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd31b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f3037fff640 1 -- 172.21.15.22:0/1747361791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 msgr2=0x56306dd24450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f3037fff640 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd24450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f3037fff640 1 -- 172.21.15.22:0/1747361791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56306d99d210 con 0x56306dd24d40 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f30377fe640 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd24450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f303cc79640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30240a2a10 0x7f30240a4ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3030009fd0 tx=0x7f3030009d80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f3037fff640 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd23a90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f30200bd390 tx=0x7f30200035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7375c11da1dda9b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f302e7fc640 1 -- 172.21.15.22:0/1747361791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020005690 con 0x56306dd24d40 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f302e7fc640 1 -- 172.21.15.22:0/1747361791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30200bdb00 con 0x56306dd24d40 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.548+0000 7f302e7fc640 1 -- 172.21.15.22:0/1747361791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020003cd0 con 0x56306dd24d40 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 -- 172.21.15.22:0/1747361791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 msgr2=0x56306dd23a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd23a90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f30200bd390 tx=0x7f30200035f0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 -- 172.21.15.22:0/1747361791 shutdown_connections 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd31b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd24450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 --2- 172.21.15.22:0/1747361791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd23a90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 -- 172.21.15.22:0/1747361791 >> 172.21.15.22:0/1747361791 conn(0x56306dceb5b0 msgr2=0x56306dd1ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 -- 172.21.15.22:0/1747361791 shutdown_connections 2024-03-08T23:57:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 -- 172.21.15.22:0/1747361791 wait complete. 2024-03-08T23:57:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 -- start start 2024-03-08T23:57:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 Processor -- start 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd23fd0 0x56306dd2c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.549+0000 7f3041399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd2cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd2f760 0x56306dd2d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306dd32250 con 0x56306dd24d40 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306dc44d90 con 0x56306dd2f760 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306dd323d0 con 0x56306dd23fd0 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 10 client.0 _ll_register_callbacks cb 0x56306d99ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd2cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3037fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd23fd0 0x56306dd2c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd2cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41726/0 (socket says 172.21.15.22:41726) 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 -- 172.21.15.22:0/1221133654 learned_addr learned my addr 172.21.15.22:0/1221133654 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f303cc79640 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd2f760 0x56306dd2d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 -- 172.21.15.22:0/1221133654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd23fd0 msgr2=0x56306dd2c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd23fd0 0x56306dd2c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 -- 172.21.15.22:0/1221133654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd2f760 msgr2=0x56306dd2d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd2f760 0x56306dd2d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 -- 172.21.15.22:0/1221133654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56306da69620 con 0x56306dd24d40 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30377fe640 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd2cd00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3038069930 tx=0x7f3028011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b2130eaf3d9c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30177fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30177fe640 1 -- 172.21.15.22:0/1221133654 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028004020 con 0x56306dd24d40 2024-03-08T23:57:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30177fe640 1 -- 172.21.15.22:0/1221133654 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f30280041c0 con 0x56306dd24d40 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30200bdb00 con 0x56306dd24d40 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f30177fe640 1 -- 172.21.15.22:0/1221133654 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3028024050 con 0x56306dd24d40 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 10 client.16650 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.550+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56306dd3e450 con 0x56306dd24d40 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f30177fe640 1 -- 172.21.15.22:0/1221133654 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30200bdb00 con 0x56306dd24d40 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f30177fe640 1 -- 172.21.15.22:0/1221133654 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f30280039f0 con 0x56306dd24d40 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3041399740 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3041399740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3041399740 20 client.16650 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3041399740 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd3f960 0x56306dd41d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3041399740 4 client.16650 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56306dc95b30 con 0x56306dd3f960 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3037fff640 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd3f960 0x56306dd41d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f30177fe640 10 client.16650 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.551+0000 7f3037fff640 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd3f960 0x56306dd41d50 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7c121c25222e8f server_cookie=60942787bb1282d in_seq=0 out_seq=0 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f30177fe640 1 -- 172.21.15.22:0/1221133654 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56306dc95b30 con 0x56306dd3f960 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f30177fe640 10 client.16650 handle_command_reply: tid=0 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 client.16650 shutdown 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 msgr2=0x56306dd2cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd2cd00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3038069930 tx=0x7f3028011810 comp rx=0 tx=0).stop 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 shutdown_connections 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd3f960 0x56306dd41d50 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd2f760 0x56306dd2d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd24d40 0x56306dd2cd00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 --2- 172.21.15.22:0/1221133654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd23fd0 0x56306dd2c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f30177fe640 0 client.16650 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 >> 172.21.15.22:0/1221133654 conn(0x56306dceb5b0 msgr2=0x56306dd20a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 shutdown_connections 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 1 -- 172.21.15.22:0/1221133654 wait complete. 2024-03-08T23:57:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.552+0000 7f3041399740 20 client.16650 trim_cache size 0 max 16384 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 Processor -- start 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 -- start start 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd2eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd24d40 0x56306dd36490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd2f760 0x56306dd369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381ae7d0 con 0x56306dd2f760 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381ac370 con 0x56306dd23fd0 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3041399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381aa410 con 0x56306dd24d40 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd2f760 0x56306dd369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd2f760 0x56306dd369d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41732/0 (socket says 172.21.15.22:41732) 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 -- 172.21.15.22:0/3307632257 learned_addr learned my addr 172.21.15.22:0/3307632257 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f3037fff640 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd2eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f30377fe640 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd24d40 0x56306dd36490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 -- 172.21.15.22:0/3307632257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd24d40 msgr2=0x56306dd36490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd24d40 0x56306dd36490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 -- 172.21.15.22:0/3307632257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 msgr2=0x56306dd2eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd2eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.566+0000 7f303cc79640 1 -- 172.21.15.22:0/3307632257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56306dd232a0 con 0x56306dd2f760 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.567+0000 7f303cc79640 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd2f760 0x56306dd369d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3030013670 tx=0x7f3030009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332fab5ab0778f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.567+0000 7f302dffb640 1 -- 172.21.15.22:0/3307632257 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300131e0 con 0x56306dd2f760 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.567+0000 7f302dffb640 1 -- 172.21.15.22:0/3307632257 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3030004510 con 0x56306dd2f760 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.567+0000 7f302dffb640 1 -- 172.21.15.22:0/3307632257 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303002a9f0 con 0x56306dd2f760 2024-03-08T23:57:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.567+0000 7f3041399740 1 -- 172.21.15.22:0/3307632257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd2f760 msgr2=0x56306dd369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.567+0000 7f3041399740 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd2f760 0x56306dd369d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3030013670 tx=0x7f3030009500 comp rx=0 tx=0).stop 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- 172.21.15.22:0/3307632257 shutdown_connections 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd2f760 0x56306dd369d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd24d40 0x56306dd36490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 --2- 172.21.15.22:0/3307632257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd23fd0 0x56306dd2eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- 172.21.15.22:0/3307632257 >> 172.21.15.22:0/3307632257 conn(0x56306dceb5b0 msgr2=0x56306dd2b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- 172.21.15.22:0/3307632257 shutdown_connections 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- 172.21.15.22:0/3307632257 wait complete. 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- start start 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 Processor -- start 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd23fd0 0x56306dd364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd24d40 0x56306dd34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381aaa10 con 0x56306dd23fd0 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381abd10 con 0x56306dd24d40 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381ab050 con 0x56306dd2f760 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3041399740 10 client.0 _ll_register_callbacks cb 0x7f30200044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3037fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd23fd0 0x56306dd364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3037fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd23fd0 0x56306dd364a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41734/0 (socket says 172.21.15.22:41734) 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.568+0000 7f3037fff640 1 -- 172.21.15.22:0/2070029695 learned_addr learned my addr 172.21.15.22:0/2070029695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f303cc79640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f30377fe640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd24d40 0x56306dd34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f303cc79640 1 -- 172.21.15.22:0/2070029695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd24d40 msgr2=0x56306dd34480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f303cc79640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd24d40 0x56306dd34480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f303cc79640 1 -- 172.21.15.22:0/2070029695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd23fd0 msgr2=0x56306dd364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f303cc79640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd23fd0 0x56306dd364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f303cc79640 1 -- 172.21.15.22:0/2070029695 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56306dd31fc0 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f3037fff640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd23fd0 0x56306dd364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f303cc79640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd349c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3030003fd0 tx=0x7f30300136a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eeb5aad5f716d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f30177fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f30177fe640 1 -- 172.21.15.22:0/2070029695 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030004230 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f30177fe640 1 -- 172.21.15.22:0/2070029695 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f303002a770 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3030004510 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.569+0000 7f30177fe640 1 -- 172.21.15.22:0/2070029695 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f303002a970 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 10 client.26119 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56306dd97550 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f30177fe640 1 -- 172.21.15.22:0/2070029695 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3030004510 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f30177fe640 1 -- 172.21.15.22:0/2070029695 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3030020050 con 0x56306dd2f760 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 10 client.26119 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 10 client.26119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 20 client.26119 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd97b70 0x56306dd99f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 4 client.26119 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:57:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56306dc8fe80 con 0x56306dd97b70 2024-03-08T23:57:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.570+0000 7f3037fff640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd97b70 0x56306dd99f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.571+0000 7f30177fe640 10 client.26119 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.571+0000 7f3037fff640 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd97b70 0x56306dd99f90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f750be0cce9b6694 server_cookie=ff913fc7b3a3a019 in_seq=0 out_seq=0 2024-03-08T23:57:30.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:30 smithi192 ceph-mon[35995]: pgmap v93: 129 pgs: 129 active+clean; 51 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 455 op/s 2024-03-08T23:57:30.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 238580 13.29 MB/sec warmup 98 sec latency 71.913 ms 2024-03-08T23:57:30.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:30 smithi022 ceph-mon[31894]: pgmap v93: 129 pgs: 129 active+clean; 51 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 455 op/s 2024-03-08T23:57:30.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:30 smithi062 ceph-mon[36129]: pgmap v93: 129 pgs: 129 active+clean; 51 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 455 op/s 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.904+0000 7f30177fe640 1 -- 172.21.15.22:0/2070029695 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56306dc8fe80 con 0x56306dd97b70 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.904+0000 7f30177fe640 10 client.26119 handle_command_reply: tid=0 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.904+0000 7f3041399740 1 client.26119 shutdown 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.904+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 msgr2=0x56306dd349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.904+0000 7f3041399740 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd349c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3030003fd0 tx=0x7f30300136a0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.904+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 shutdown_connections 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.904+0000 7f3041399740 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56306dd97b70 0x56306dd99f90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f3041399740 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56306dd2f760 0x56306dd349c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f30177fe640 0 client.26119 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f3041399740 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56306dd24d40 0x56306dd34480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f3041399740 1 --2- 172.21.15.22:0/2070029695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56306dd23fd0 0x56306dd364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 >> 172.21.15.22:0/2070029695 conn(0x56306dceb5b0 msgr2=0x56306dd2c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 shutdown_connections 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f3041399740 1 -- 172.21.15.22:0/2070029695 wait complete. 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.905+0000 7f3041399740 20 client.26119 trim_cache size 0 max 16384 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:57:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:57:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30240a2a10 msgr2=0x7f30240a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30240a2a10 0x7f30240a4ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3030009fd0 tx=0x7f3030009d80 comp rx=0 tx=0).stop 2024-03-08T23:57:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 msgr2=0x7f303819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f303819c670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f302002c240 tx=0x7f30200042f0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 shutdown_connections 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30240a2a10 0x7f30240a4ed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3038196830 0x7f3038196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038105500 0x7f303819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 --2- 172.21.15.22:0/886596116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3038100430 0x7f303819c670 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 >> 172.21.15.22:0/886596116 conn(0x7f3038077320 msgr2=0x7f30380fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 shutdown_connections 2024-03-08T23:57:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:30.906+0000 7f303e6dc640 1 -- 172.21.15.22:0/886596116 wait complete. 2024-03-08T23:57:31.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 241311 13.33 MB/sec warmup 99 sec latency 75.864 ms 2024-03-08T23:57:32.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:32 smithi192 ceph-mon[35995]: pgmap v94: 129 pgs: 129 active+clean; 57 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-03-08T23:57:32.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 243444 13.27 MB/sec warmup 100 sec latency 16.672 ms 2024-03-08T23:57:32.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:32 smithi022 ceph-mon[31894]: pgmap v94: 129 pgs: 129 active+clean; 57 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-03-08T23:57:32.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:32 smithi062 ceph-mon[36129]: pgmap v94: 129 pgs: 129 active+clean; 57 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-03-08T23:57:33.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 245576 13.28 MB/sec warmup 101 sec latency 10.129 ms 2024-03-08T23:57:34.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 248574 13.33 MB/sec warmup 102 sec latency 16.576 ms 2024-03-08T23:57:34.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:34 smithi022 ceph-mon[31894]: pgmap v95: 129 pgs: 129 active+clean; 57 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-08T23:57:34.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:34 smithi062 ceph-mon[36129]: pgmap v95: 129 pgs: 129 active+clean; 57 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-08T23:57:34.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:34 smithi192 ceph-mon[35995]: pgmap v95: 129 pgs: 129 active+clean; 57 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-08T23:57:35.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 250764 13.27 MB/sec warmup 103 sec latency 50.174 ms 2024-03-08T23:57:36.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 252689 13.26 MB/sec warmup 104 sec latency 17.242 ms 2024-03-08T23:57:36.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:36 smithi022 ceph-mon[31894]: pgmap v96: 129 pgs: 129 active+clean; 47 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.6 MiB/s wr, 541 op/s 2024-03-08T23:57:36.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:36 smithi062 ceph-mon[36129]: pgmap v96: 129 pgs: 129 active+clean; 47 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.6 MiB/s wr, 541 op/s 2024-03-08T23:57:36.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:36 smithi192 ceph-mon[35995]: pgmap v96: 129 pgs: 129 active+clean; 47 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.6 MiB/s wr, 541 op/s 2024-03-08T23:57:37.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 255700 13.31 MB/sec warmup 105 sec latency 15.122 ms 2024-03-08T23:57:38.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 258017 13.26 MB/sec warmup 106 sec latency 12.360 ms 2024-03-08T23:57:38.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:38 smithi022 ceph-mon[31894]: pgmap v97: 129 pgs: 129 active+clean; 47 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 412 op/s 2024-03-08T23:57:38.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:38 smithi062 ceph-mon[36129]: pgmap v97: 129 pgs: 129 active+clean; 47 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 412 op/s 2024-03-08T23:57:38.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:38 smithi192 ceph-mon[35995]: pgmap v97: 129 pgs: 129 active+clean; 47 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 412 op/s 2024-03-08T23:57:39.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:57:39.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:57:39.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 -- 172.21.15.22:0/879399225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e881074f0 msgr2=0x7f8e881078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/879399225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e881074f0 0x7f8e881078d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009930 tx=0x7f8e8402bc20 comp rx=0 tx=0).stop 2024-03-08T23:57:39.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 -- 172.21.15.22:0/879399225 shutdown_connections 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/879399225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e88100570 0x7f8e8810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/879399225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e880ffbb0 0x7f8e88100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/879399225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e881074f0 0x7f8e881078d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 -- 172.21.15.22:0/879399225 >> 172.21.15.22:0/879399225 conn(0x7f8e88075530 msgr2=0x7f8e88075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 -- 172.21.15.22:0/879399225 shutdown_connections 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.275+0000 7f8e8f88c640 1 -- 172.21.15.22:0/879399225 wait complete. 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 Processor -- start 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 -- start start 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e880ffbb0 0x7f8e88100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e88100570 0x7f8e881012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e881074f0 0x7f8e88101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881116f0 con 0x7f8e880ffbb0 2024-03-08T23:57:39.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88111570 con 0x7f8e88100570 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8f88c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88111870 con 0x7f8e881074f0 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e880ffbb0 0x7f8e88100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e880ffbb0 0x7f8e88100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41752/0 (socket says 172.21.15.22:41752) 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 -- 172.21.15.22:0/3672474536 learned_addr learned my addr 172.21.15.22:0/3672474536 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8de29640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e881074f0 0x7f8e88101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 -- 172.21.15.22:0/3672474536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e881074f0 msgr2=0x7f8e88101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e881074f0 0x7f8e88101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 -- 172.21.15.22:0/3672474536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e88100570 msgr2=0x7f8e881012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8ce27640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e88100570 0x7f8e881012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e88100570 0x7f8e881012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8d628640 1 -- 172.21.15.22:0/3672474536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e881a60d0 con 0x7f8e880ffbb0 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.276+0000 7f8e8de29640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e881074f0 0x7f8e88101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.277+0000 7f8e8d628640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e880ffbb0 0x7f8e88100d90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009430 tx=0x7f8e8402cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.277+0000 7f8e7e7fc640 1 -- 172.21.15.22:0/3672474536 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84041070 con 0x7f8e880ffbb0 2024-03-08T23:57:39.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.277+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e881a6360 con 0x7f8e880ffbb0 2024-03-08T23:57:39.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.277+0000 7f8e7e7fc640 1 -- 172.21.15.22:0/3672474536 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e84004450 con 0x7f8e880ffbb0 2024-03-08T23:57:39.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.277+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e881a68a0 con 0x7f8e880ffbb0 2024-03-08T23:57:39.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.277+0000 7f8e7e7fc640 1 -- 172.21.15.22:0/3672474536 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84003dc0 con 0x7f8e880ffbb0 2024-03-08T23:57:39.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.279+0000 7f8e7e7fc640 1 -- 172.21.15.22:0/3672474536 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8e84044070 con 0x7f8e880ffbb0 2024-03-08T23:57:39.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 Processor -- start 2024-03-08T23:57:39.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e7e7fc640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e740a29a0 0x7f8e740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 -- start start 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8ce27640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e740a29a0 0x7f8e740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e7e7fc640 1 -- 172.21.15.22:0/3672474536 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e840f0c40 con 0x7f8e880ffbb0 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeac760 0x56139aeacb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139ae0fe00 con 0x56139aeb5c60 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139adb8b10 con 0x56139aead080 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e92461740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139ae0bea0 con 0x56139aeac760 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeac760 0x56139aeacb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8de29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeac760 0x56139aeacb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60570/0 (socket says 172.21.15.22:60570) 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8d628640 1 -- 172.21.15.22:0/502693529 learned_addr learned my addr 172.21.15.22:0/502693529 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8ce27640 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8ce27640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e740a29a0 0x7f8e740a4e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e881027d0 tx=0x7f8e78009e80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8d628640 1 -- 172.21.15.22:0/502693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 msgr2=0x56139aeb5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.280+0000 7f8e8d628640 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e8d628640 1 -- 172.21.15.22:0/502693529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 msgr2=0x56139aeb8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e8d628640 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e8d628640 1 -- 172.21.15.22:0/502693529 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56139abee7d0 con 0x56139aeac760 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e8de29640 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e8d628640 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeac760 0x56139aeacb40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e840c5280 tx=0x7f8e8402ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99d563dca9fc592 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e737fe640 1 -- 172.21.15.22:0/502693529 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8404b070 con 0x56139aeac760 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e737fe640 1 -- 172.21.15.22:0/502693529 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e840bd710 con 0x56139aeac760 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.281+0000 7f8e737fe640 1 -- 172.21.15.22:0/502693529 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840ee270 con 0x56139aeac760 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- 172.21.15.22:0/502693529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeac760 msgr2=0x56139aeacb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeac760 0x56139aeacb40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e840c5280 tx=0x7f8e8402ce70 comp rx=0 tx=0).stop 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- 172.21.15.22:0/502693529 shutdown_connections 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 --2- 172.21.15.22:0/502693529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeac760 0x56139aeacb40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- 172.21.15.22:0/502693529 >> 172.21.15.22:0/502693529 conn(0x56139aea6e90 msgr2=0x56139aea7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- 172.21.15.22:0/502693529 shutdown_connections 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- 172.21.15.22:0/502693529 wait complete. 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- start start 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 Processor -- start 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aeb53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aeb2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139aeb8810 con 0x56139aeb5c60 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139adcc0a0 con 0x56139aeac760 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.282+0000 7f8e92461740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139aeb8990 con 0x56139aead080 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e92461740 10 client.0 _ll_register_callbacks cb 0x56139adcba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aeb53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41774/0 (socket says 172.21.15.22:41774) 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 -- 172.21.15.22:0/2983606532 learned_addr learned my addr 172.21.15.22:0/2983606532 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8ce27640 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aeb2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 -- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 msgr2=0x56139aeb2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aeb2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 -- 172.21.15.22:0/2983606532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 msgr2=0x56139aeb53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aeb53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 -- 172.21.15.22:0/2983606532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56139ab243f0 con 0x56139aeb5c60 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8d628640 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aeb53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e8de29640 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb2cc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8e80011d10 tx=0x7f8e800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529b4bee58c059c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e70ff9640 1 -- 172.21.15.22:0/2983606532 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8001f060 con 0x56139aeb5c60 2024-03-08T23:57:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e70ff9640 1 -- 172.21.15.22:0/2983606532 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e8001a410 con 0x56139aeb5c60 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e840bd710 con 0x56139aeb5c60 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e70ff9640 1 -- 172.21.15.22:0/2983606532 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e80026050 con 0x56139aeb5c60 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e92461740 10 client.16668 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.283+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56139aae40f0 con 0x56139aeb5c60 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e70ff9640 1 -- 172.21.15.22:0/2983606532 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e840bd710 con 0x56139aeb5c60 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e70ff9640 1 -- 172.21.15.22:0/2983606532 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e80019370 con 0x56139aeb5c60 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e92461740 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e92461740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e92461740 20 client.16668 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e92461740 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139aec54c0 0x56139aec78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e92461740 4 client.16668 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56139ae1d340 con 0x56139aec54c0 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e8d628640 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139aec54c0 0x56139aec78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e8d628640 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139aec54c0 0x56139aec78b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6143bc67c73a96e server_cookie=65ff51cfa2ac8559 in_seq=0 out_seq=0 2024-03-08T23:57:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.284+0000 7f8e70ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.285+0000 7f8e70ff9640 1 -- 172.21.15.22:0/2983606532 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56139ae1d340 con 0x56139aec54c0 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.285+0000 7f8e70ff9640 10 client.16668 handle_command_reply: tid=0 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.285+0000 7f8e92461740 1 client.16668 shutdown 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.285+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 msgr2=0x56139aeb2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.285+0000 7f8e92461740 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb2cc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8e80011d10 tx=0x7f8e800107a0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 shutdown_connections 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139aec54c0 0x56139aec78b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139aeb2cc0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aeb2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e70ff9640 0 client.16668 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 --2- 172.21.15.22:0/2983606532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aeb53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 >> 172.21.15.22:0/2983606532 conn(0x56139aea6e90 msgr2=0x56139aea8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 shutdown_connections 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 1 -- 172.21.15.22:0/2983606532 wait complete. 2024-03-08T23:57:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.286+0000 7f8e92461740 20 client.16668 trim_cache size 0 max 16384 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.301+0000 7f8e92461740 1 Processor -- start 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.301+0000 7f8e92461740 1 -- start start 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e92461740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeac760 0x56139aeb4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e92461740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e92461740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeb5c60 0x56139aebcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e92461740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881b2da0 con 0x56139aeac760 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e92461740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881b0940 con 0x56139aead080 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e92461740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881ae9e0 con 0x56139aeb5c60 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeac760 0x56139aeb4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeac760 0x56139aeb4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41782/0 (socket says 172.21.15.22:41782) 2024-03-08T23:57:39.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 -- 172.21.15.22:0/960942914 learned_addr learned my addr 172.21.15.22:0/960942914 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8ce27640 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8de29640 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeb5c60 0x56139aebcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 -- 172.21.15.22:0/960942914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeb5c60 msgr2=0x56139aebcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeb5c60 0x56139aebcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 -- 172.21.15.22:0/960942914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 msgr2=0x56139aeb53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 -- 172.21.15.22:0/960942914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56139aec7c20 con 0x56139aeac760 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e8d628640 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeac760 0x56139aeb4eb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009900 tx=0x7f8e84005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676409ff940a54a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e72ffd640 1 -- 172.21.15.22:0/960942914 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8404b040 con 0x56139aeac760 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e72ffd640 1 -- 172.21.15.22:0/960942914 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e840bd710 con 0x56139aeac760 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.302+0000 7f8e72ffd640 1 -- 172.21.15.22:0/960942914 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840b9680 con 0x56139aeac760 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 -- 172.21.15.22:0/960942914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeac760 msgr2=0x56139aeb4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeac760 0x56139aeb4eb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009900 tx=0x7f8e84005fd0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 -- 172.21.15.22:0/960942914 shutdown_connections 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aeb5c60 0x56139aebcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aead080 0x56139aeb53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 --2- 172.21.15.22:0/960942914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeac760 0x56139aeb4eb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 -- 172.21.15.22:0/960942914 >> 172.21.15.22:0/960942914 conn(0x56139aea6e90 msgr2=0x56139aeb2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 -- 172.21.15.22:0/960942914 shutdown_connections 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 -- 172.21.15.22:0/960942914 wait complete. 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 -- start start 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.303+0000 7f8e92461740 1 Processor -- start 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e92461740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aebad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e92461740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aebb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e92461740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139af1a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e92461740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881aefe0 con 0x56139aeb5c60 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e92461740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881b02e0 con 0x56139aeac760 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e92461740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881af620 con 0x56139aead080 2024-03-08T23:57:39.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e92461740 10 client.0 _ll_register_callbacks cb 0x7f8e80009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139af1a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8d628640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aebad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139af1a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41786/0 (socket says 172.21.15.22:41786) 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 -- 172.21.15.22:0/4031230842 learned_addr learned my addr 172.21.15.22:0/4031230842 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8ce27640 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aebb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 -- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 msgr2=0x56139aebb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aebb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 -- 172.21.15.22:0/4031230842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 msgr2=0x56139aebad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aebad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 -- 172.21.15.22:0/4031230842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56139af1af80 con 0x56139aeb5c60 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8d628640 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aebad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e8de29640 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139af1a990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8e80019000 tx=0x7f8e80010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fccfc421686845 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.304+0000 7f8e70ff9640 1 -- 172.21.15.22:0/4031230842 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80019930 con 0x56139aeb5c60 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e70ff9640 1 -- 172.21.15.22:0/4031230842 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8e80019370 con 0x56139aeb5c60 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e840bd710 con 0x56139aeb5c60 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e70ff9640 1 -- 172.21.15.22:0/4031230842 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e80038050 con 0x56139aeb5c60 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 10 client.16674 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56139af1c270 con 0x56139aeb5c60 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e70ff9640 1 -- 172.21.15.22:0/4031230842 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e840bd710 con 0x56139aeb5c60 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e70ff9640 1 -- 172.21.15.22:0/4031230842 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e80005230 con 0x56139aeb5c60 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 10 client.16674 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 20 client.16674 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139af1c990 0x56139af1edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 4 client.16674 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56139ae17690 con 0x56139af1c990 2024-03-08T23:57:39.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.305+0000 7f8e8d628640 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139af1c990 0x56139af1edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e70ff9640 10 client.16674 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e8d628640 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139af1c990 0x56139af1edb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15cf122f445b24d server_cookie=ca954d1c75a14007 in_seq=0 out_seq=0 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e70ff9640 1 -- 172.21.15.22:0/4031230842 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56139ae17690 con 0x56139af1c990 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e70ff9640 10 client.16674 handle_command_reply: tid=0 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 client.16674 shutdown 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 msgr2=0x56139af1a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139af1a990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8e80019000 tx=0x7f8e80010f80 comp rx=0 tx=0).stop 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 shutdown_connections 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56139af1c990 0x56139af1edb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56139aeb5c60 0x56139af1a990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56139aead080 0x56139aebb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 --2- 172.21.15.22:0/4031230842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56139aeac760 0x56139aebad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e70ff9640 0 client.16674 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 >> 172.21.15.22:0/4031230842 conn(0x56139aea6e90 msgr2=0x56139aead5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 shutdown_connections 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 1 -- 172.21.15.22:0/4031230842 wait complete. 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.306+0000 7f8e92461740 20 client.16674 trim_cache size 0 max 16384 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:57:39.309 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:57:39.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e740a29a0 msgr2=0x7f8e740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e740a29a0 0x7f8e740a4e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e881027d0 tx=0x7f8e78009e80 comp rx=0 tx=0).stop 2024-03-08T23:57:39.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e880ffbb0 msgr2=0x7f8e88100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e880ffbb0 0x7f8e88100d90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009430 tx=0x7f8e8402cb10 comp rx=0 tx=0).stop 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 shutdown_connections 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8e740a29a0 0x7f8e740a4e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e881074f0 0x7f8e88101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e88100570 0x7f8e881012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 --2- 172.21.15.22:0/3672474536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e880ffbb0 0x7f8e88100d90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 >> 172.21.15.22:0/3672474536 conn(0x7f8e88075530 msgr2=0x7f8e880fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 shutdown_connections 2024-03-08T23:57:39.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.307+0000 7f8e8f88c640 1 -- 172.21.15.22:0/3672474536 wait complete. 2024-03-08T23:57:39.313 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:57:39.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 -- 172.21.15.22:0/1623701225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c1051d0 msgr2=0x7f443c1055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 --2- 172.21.15.22:0/1623701225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c1051d0 0x7f443c1055b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f442c009930 tx=0x7f442c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 -- 172.21.15.22:0/1623701225 shutdown_connections 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 --2- 172.21.15.22:0/1623701225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c100f00 0x7f443c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 --2- 172.21.15.22:0/1623701225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f443c100540 0x7f443c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 --2- 172.21.15.22:0/1623701225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c1051d0 0x7f443c1055b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 -- 172.21.15.22:0/1623701225 >> 172.21.15.22:0/1623701225 conn(0x7f443c075ff0 msgr2=0x7f443c076400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 -- 172.21.15.22:0/1623701225 shutdown_connections 2024-03-08T23:57:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.421+0000 7f44429b3640 1 -- 172.21.15.22:0/1623701225 wait complete. 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 Processor -- start 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 -- start start 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c100540 0x7f443c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c100f00 0x7f443c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f443c1051d0 0x7f443c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c1116e0 con 0x7f443c100540 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c111560 con 0x7f443c100f00 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f44429b3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c111860 con 0x7f443c1051d0 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f443bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c100540 0x7f443c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f443bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c100540 0x7f443c19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41800/0 (socket says 172.21.15.22:41800) 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f443bfff640 1 -- 172.21.15.22:0/3647141534 learned_addr learned my addr 172.21.15.22:0/3647141534 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f443b7fe640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c100f00 0x7f443c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.422+0000 7f4440f50640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f443c1051d0 0x7f443c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f443bfff640 1 -- 172.21.15.22:0/3647141534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f443c1051d0 msgr2=0x7f443c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f443bfff640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f443c1051d0 0x7f443c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f443bfff640 1 -- 172.21.15.22:0/3647141534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c100f00 msgr2=0x7f443c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f443bfff640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c100f00 0x7f443c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f443bfff640 1 -- 172.21.15.22:0/3647141534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c198160 con 0x7f443c100540 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f443bfff640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c100540 0x7f443c19d830 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f442c009230 tx=0x7f442c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f44397fa640 1 -- 172.21.15.22:0/3647141534 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c041070 con 0x7f443c100540 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f44397fa640 1 -- 172.21.15.22:0/3647141534 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f442c02ccc0 con 0x7f443c100540 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c1983f0 con 0x7f443c100540 2024-03-08T23:57:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f44397fa640 1 -- 172.21.15.22:0/3647141534 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c003de0 con 0x7f443c100540 2024-03-08T23:57:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.423+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c1a46f0 con 0x7f443c100540 2024-03-08T23:57:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.424+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c101910 con 0x7f443c100540 2024-03-08T23:57:39.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.425+0000 7f44397fa640 1 -- 172.21.15.22:0/3647141534 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f442c044070 con 0x7f443c100540 2024-03-08T23:57:39.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.426+0000 7f44397fa640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44240a29f0 0x7f44240a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.426+0000 7f443b7fe640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44240a29f0 0x7f44240a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.426+0000 7f44397fa640 1 -- 172.21.15.22:0/3647141534 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f442c0f0e90 con 0x7f443c100540 2024-03-08T23:57:39.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.426+0000 7f443b7fe640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44240a29f0 0x7f44240a4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f443c0fd860 tx=0x7f4430005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.429+0000 7f44397fa640 1 -- 172.21.15.22:0/3647141534 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f442c005670 con 0x7f443c100540 2024-03-08T23:57:39.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.659+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f443c068530 con 0x7f443c100540 2024-03-08T23:57:39.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.660+0000 7f44397fa640 1 -- 172.21.15.22:0/3647141534 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f442c0b7e20 con 0x7f443c100540 2024-03-08T23:57:39.656 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:57:39.656 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:56:37.975909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25952":{"gid":25952,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/3083291387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3083291387},{"type":"v1","addr":"172.21.15.62:6837","nonce":3083291387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:57:39.656 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44240a29f0 msgr2=0x7f44240a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44240a29f0 0x7f44240a4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f443c0fd860 tx=0x7f4430005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c100540 msgr2=0x7f443c19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c100540 0x7f443c19d830 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f442c009230 tx=0x7f442c0043b0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 shutdown_connections 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44240a29f0 0x7f44240a4eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f443c1051d0 0x7f443c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443c100f00 0x7f443c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 --2- 172.21.15.22:0/3647141534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f443c100540 0x7f443c19d830 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 >> 172.21.15.22:0/3647141534 conn(0x7f443c075ff0 msgr2=0x7f443c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 shutdown_connections 2024-03-08T23:57:39.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.662+0000 7f44429b3640 1 -- 172.21.15.22:0/3647141534 wait complete. 2024-03-08T23:57:39.669 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:57:39.669 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:57:39.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 260124 13.26 MB/sec warmup 107 sec latency 13.214 ms 2024-03-08T23:57:39.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 -- 172.21.15.22:0/36842817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 msgr2=0x7f77c4111d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 --2- 172.21.15.22:0/36842817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c4111d30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009900 tx=0x7f77b002bcd0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 -- 172.21.15.22:0/36842817 shutdown_connections 2024-03-08T23:57:39.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 --2- 172.21.15.22:0/36842817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c4112270 0x7f77c4114660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 --2- 172.21.15.22:0/36842817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c4111d30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 --2- 172.21.15.22:0/36842817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4073690 0x7f77c4073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 -- 172.21.15.22:0/36842817 >> 172.21.15.22:0/36842817 conn(0x7f77c40fcd90 msgr2=0x7f77c40ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 -- 172.21.15.22:0/36842817 shutdown_connections 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.783+0000 7f77caf59640 1 -- 172.21.15.22:0/36842817 wait complete. 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 Processor -- start 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 -- start start 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4073690 0x7f77c419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c4112270 0x7f77c41a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c4117c70 con 0x7f77c4073fb0 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c4117af0 con 0x7f77c4112270 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77caf59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c4117df0 con 0x7f77c4073690 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41822/0 (socket says 172.21.15.22:41822) 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 -- 172.21.15.22:0/2789162227 learned_addr learned my addr 172.21.15.22:0/2789162227 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c94f6640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c4112270 0x7f77c41a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4073690 0x7f77c419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 -- 172.21.15.22:0/2789162227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4073690 msgr2=0x7f77c419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4073690 0x7f77c419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 -- 172.21.15.22:0/2789162227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c4112270 msgr2=0x7f77c41a2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c4112270 0x7f77c41a2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 -- 172.21.15.22:0/2789162227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c41a31b0 con 0x7f77c4073fb0 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.784+0000 7f77c3fff640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c419e750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f77b0005940 tx=0x7f77b002cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.785+0000 7f77c1ffb640 1 -- 172.21.15.22:0/2789162227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0041070 con 0x7f77c4073fb0 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.785+0000 7f77c1ffb640 1 -- 172.21.15.22:0/2789162227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77b0004450 con 0x7f77c4073fb0 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.785+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c41a3440 con 0x7f77c4073fb0 2024-03-08T23:57:39.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.785+0000 7f77c1ffb640 1 -- 172.21.15.22:0/2789162227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0003df0 con 0x7f77c4073fb0 2024-03-08T23:57:39.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.785+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c41aad30 con 0x7f77c4073fb0 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77c1ffb640 1 -- 172.21.15.22:0/2789162227 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f77b0044070 con 0x7f77c4073fb0 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 Processor -- start 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 -- start start 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006921580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 0x55b00692a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00692ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0068e8360 con 0x55b0069211a0 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b006884ec0 con 0x55b00692a6a0 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77cdc06740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b00682dd30 con 0x55b006921ac0 2024-03-08T23:57:39.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.787+0000 7f77c1ffb640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77b40a2970 0x7f77b40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006921580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006921580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41836/0 (socket says 172.21.15.22:41836) 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 -- 172.21.15.22:0/1973459456 learned_addr learned my addr 172.21.15.22:0/1973459456 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77b40a2970 0x7f77b40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c3fff640 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00692ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 0x55b00692a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c1ffb640 1 -- 172.21.15.22:0/2789162227 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f77b00f0c80 con 0x7f77c4073fb0 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 -- 172.21.15.22:0/1973459456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 msgr2=0x55b00692a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 0x55b00692a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 -- 172.21.15.22:0/1973459456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 msgr2=0x55b00692ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00692ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 -- 172.21.15.22:0/1973459456 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b00682cbc0 con 0x55b0069211a0 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c3fff640 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00692ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006921580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f77bc011d70 tx=0x7f77bc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764f35c33b6a569d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77c94f6640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77b40a2970 0x7f77b40a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00fc90 tx=0x7f77bc01e000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77aeffd640 1 -- 172.21.15.22:0/1973459456 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc01dbf0 con 0x55b0069211a0 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77aeffd640 1 -- 172.21.15.22:0/1973459456 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77bc01dd90 con 0x55b0069211a0 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.788+0000 7f77aeffd640 1 -- 172.21.15.22:0/1973459456 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc004250 con 0x55b0069211a0 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 -- 172.21.15.22:0/1973459456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 msgr2=0x55b006921580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006921580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f77bc011d70 tx=0x7f77bc00f790 comp rx=0 tx=0).stop 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 -- 172.21.15.22:0/1973459456 shutdown_connections 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00692ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 0x55b00692a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1973459456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006921580 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 -- 172.21.15.22:0/1973459456 >> 172.21.15.22:0/1973459456 conn(0x55b00691b8d0 msgr2=0x55b00691bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 -- 172.21.15.22:0/1973459456 shutdown_connections 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.789+0000 7f77cdc06740 1 -- 172.21.15.22:0/1973459456 wait complete. 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 -- start start 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 Processor -- start 2024-03-08T23:57:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b0069275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b006921ac0 0x55b006927af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b00692a6a0 0x55b006928030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0068234a0 con 0x55b0069211a0 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b006880f60 con 0x55b006921ac0 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b00692d430 con 0x55b00692a6a0 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77cdc06740 10 client.0 _ll_register_callbacks cb 0x55b00687ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77c8cf5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b0069275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77c8cf5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b0069275b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41850/0 (socket says 172.21.15.22:41850) 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1097880589 learned_addr learned my addr 172.21.15.22:0/1097880589 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77c94f6640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b00692a6a0 0x55b006928030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.790+0000 7f77c3fff640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b006921ac0 0x55b006927af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b00692a6a0 msgr2=0x55b006928030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b00692a6a0 0x55b006928030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1097880589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b006921ac0 msgr2=0x55b006927af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b006921ac0 0x55b006927af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1097880589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0065993f0 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77c3fff640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b006921ac0 0x55b006927af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b0069275b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f77b800b950 tx=0x7f77b800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d7c378a1693e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77a3fff640 1 -- 172.21.15.22:0/1097880589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b801b440 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77a3fff640 1 -- 172.21.15.22:0/1097880589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f77b801b5e0 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77bc01dd90 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77a3fff640 1 -- 172.21.15.22:0/1097880589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77b801b820 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77cdc06740 10 client.16692 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b0066637d0 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77a3fff640 1 -- 172.21.15.22:0/1097880589 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77bc01dd90 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77a3fff640 1 -- 172.21.15.22:0/1097880589 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f77b8019020 con 0x55b0069211a0 2024-03-08T23:57:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77cdc06740 10 client.16692 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.791+0000 7f77cdc06740 10 client.16692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.792+0000 7f77cdc06740 20 client.16692 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.792+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b00693a790 0x55b00693cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.792+0000 7f77cdc06740 4 client.16692 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.792+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b006892400 con 0x55b00693a790 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.792+0000 7f77c3fff640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b00693a790 0x55b00693cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.792+0000 7f77a3fff640 10 client.16692 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.792+0000 7f77c3fff640 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b00693a790 0x55b00693cb80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=125907cb5cb9b7cc server_cookie=2be6cd8d4be74216 in_seq=0 out_seq=0 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.793+0000 7f77a3fff640 1 -- 172.21.15.22:0/1097880589 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b006892400 con 0x55b00693a790 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.793+0000 7f77a3fff640 10 client.16692 handle_command_reply: tid=0 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.793+0000 7f77cdc06740 1 client.16692 shutdown 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 msgr2=0x55b0069275b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b0069275b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f77b800b950 tx=0x7f77b800bd50 comp rx=0 tx=0).stop 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 shutdown_connections 2024-03-08T23:57:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b00693a790 0x55b00693cb80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b00692a6a0 0x55b006928030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b006921ac0 0x55b006927af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1097880589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b0069275b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 >> 172.21.15.22:0/1097880589 conn(0x55b00691b8d0 msgr2=0x55b00691d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77a3fff640 0 client.16692 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 shutdown_connections 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 1 -- 172.21.15.22:0/1097880589 wait complete. 2024-03-08T23:57:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.794+0000 7f77cdc06740 20 client.16692 trim_cache size 0 max 16384 2024-03-08T23:57:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 Processor -- start 2024-03-08T23:57:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 -- start start 2024-03-08T23:57:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006929ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 0x55b0069228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b006922e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c41b7080 con 0x55b0069211a0 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c41b4c20 con 0x55b00692a6a0 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77cdc06740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c41b2cc0 con 0x55b006921ac0 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77c8cf5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006929ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77c94f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b006922e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77c8cf5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006929ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41864/0 (socket says 172.21.15.22:41864) 2024-03-08T23:57:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.820+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1442695007 learned_addr learned my addr 172.21.15.22:0/1442695007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.821+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1442695007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 msgr2=0x55b0069228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.821+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 0x55b0069228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.821+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1442695007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 msgr2=0x55b006922e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.821+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b006922e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.821+0000 7f77c8cf5640 1 -- 172.21.15.22:0/1442695007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b006941d70 con 0x55b0069211a0 2024-03-08T23:57:39.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.821+0000 7f77c94f6640 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b006922e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:39.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.821+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006929ce0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f77b800b620 tx=0x7f77b8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb607b86695095b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.822+0000 7f77ae7fc640 1 -- 172.21.15.22:0/1442695007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b802c020 con 0x55b0069211a0 2024-03-08T23:57:39.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.822+0000 7f77ae7fc640 1 -- 172.21.15.22:0/1442695007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77b8005bb0 con 0x55b0069211a0 2024-03-08T23:57:39.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.822+0000 7f77ae7fc640 1 -- 172.21.15.22:0/1442695007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8011790 con 0x55b0069211a0 2024-03-08T23:57:39.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 -- 172.21.15.22:0/1442695007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 msgr2=0x55b006929ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006929ce0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f77b800b620 tx=0x7f77b8005fd0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 -- 172.21.15.22:0/1442695007 shutdown_connections 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b006922e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b006921ac0 0x55b0069228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 --2- 172.21.15.22:0/1442695007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b0069211a0 0x55b006929ce0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 -- 172.21.15.22:0/1442695007 >> 172.21.15.22:0/1442695007 conn(0x55b00691b8d0 msgr2=0x55b00691bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 -- 172.21.15.22:0/1442695007 shutdown_connections 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.823+0000 7f77cdc06740 1 -- 172.21.15.22:0/1442695007 wait complete. 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.824+0000 7f77cdc06740 1 -- start start 2024-03-08T23:57:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.824+0000 7f77cdc06740 1 Processor -- start 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0069211a0 0x55b00692f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b006921ac0 0x55b00692fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77cdc06740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00698bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77cdc06740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c41b32c0 con 0x55b006921ac0 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77cdc06740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c41b45c0 con 0x55b00692a6a0 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77cdc06740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c41b3900 con 0x55b0069211a0 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77cdc06740 10 client.0 _ll_register_callbacks cb 0x7f77b8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0069211a0 0x55b00692f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b006921ac0 0x55b00692fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0069211a0 0x55b00692f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60666/0 (socket says 172.21.15.22:60666) 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b006921ac0 0x55b00692fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41872/0 (socket says 172.21.15.22:41872) 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 -- 172.21.15.22:0/32053798 learned_addr learned my addr 172.21.15.22:0/32053798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c94f6640 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00698bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 -- 172.21.15.22:0/32053798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 msgr2=0x55b00698bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00698bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 -- 172.21.15.22:0/32053798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b006921ac0 msgr2=0x55b00692fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b006921ac0 0x55b00692fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.825+0000 7f77c8cf5640 1 -- 172.21.15.22:0/32053798 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b00698c180 con 0x55b0069211a0 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77c8cf5640 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0069211a0 0x55b00692f840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f77b800b950 tx=0x7f77b800b4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56814e942c9813c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77a3fff640 1 -- 172.21.15.22:0/32053798 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b801bdd0 con 0x55b0069211a0 2024-03-08T23:57:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77b8005bb0 con 0x55b0069211a0 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77a3fff640 1 -- 172.21.15.22:0/32053798 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f77b8011440 con 0x55b0069211a0 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77a3fff640 1 -- 172.21.15.22:0/32053798 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77b8011640 con 0x55b0069211a0 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77cdc06740 10 client.26158 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b0069a24c0 con 0x55b0069211a0 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.826+0000 7f77a3fff640 1 -- 172.21.15.22:0/32053798 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77b8005bb0 con 0x55b0069211a0 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77a3fff640 1 -- 172.21.15.22:0/32053798 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f77b8004840 con 0x55b0069211a0 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77cdc06740 10 client.26158 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77cdc06740 10 client.26158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77cdc06740 20 client.26158 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77cdc06740 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b0069a2a20 0x55b0069a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77cdc06740 4 client.26158 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b00688c750 con 0x55b0069a2a20 2024-03-08T23:57:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77c3fff640 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b0069a2a20 0x55b0069a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77a3fff640 10 client.26158 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77c3fff640 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b0069a2a20 0x55b0069a4e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=638c8e58b37a1d70 server_cookie=5a6dbec84c5b22d0 in_seq=0 out_seq=0 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77a3fff640 1 -- 172.21.15.22:0/32053798 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b00688c750 con 0x55b0069a2a20 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77a3fff640 10 client.26158 handle_command_reply: tid=0 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.827+0000 7f77cdc06740 1 client.26158 shutdown 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0069211a0 msgr2=0x55b00692f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0069211a0 0x55b00692f840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f77b800b950 tx=0x7f77b800b4d0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 shutdown_connections 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55b0069a2a20 0x55b0069a4e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b00692a6a0 0x55b00698bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b006921ac0 0x55b00692fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 --2- 172.21.15.22:0/32053798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0069211a0 0x55b00692f840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77a3fff640 0 client.26158 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:57:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 >> 172.21.15.22:0/32053798 conn(0x55b00691b8d0 msgr2=0x55b0069228e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 shutdown_connections 2024-03-08T23:57:39.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 1 -- 172.21.15.22:0/32053798 wait complete. 2024-03-08T23:57:39.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77cdc06740 20 client.26158 trim_cache size 0 max 16384 2024-03-08T23:57:39.828 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77b40a2970 msgr2=0x7f77b40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77b40a2970 0x7f77b40a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00fc90 tx=0x7f77bc01e000 comp rx=0 tx=0).stop 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 msgr2=0x7f77c419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c419e750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f77b0005940 tx=0x7f77b002cc40 comp rx=0 tx=0).stop 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 shutdown_connections 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77b40a2970 0x7f77b40a4e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77c4112270 0x7f77c41a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77c4073fb0 0x7f77c419e750 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 --2- 172.21.15.22:0/2789162227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4073690 0x7f77c419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.828+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 >> 172.21.15.22:0/2789162227 conn(0x7f77c40fcd90 msgr2=0x7f77c40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.829+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 shutdown_connections 2024-03-08T23:57:39.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.829+0000 7f77caf59640 1 -- 172.21.15.22:0/2789162227 wait complete. 2024-03-08T23:57:39.835 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:57:39.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:57:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 -- 172.21.15.22:0/4261138840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102510 msgr2=0x7f5a301028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 --2- 172.21.15.22:0/4261138840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102510 0x7f5a301028f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009900 tx=0x7f5a2002bcd0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 -- 172.21.15.22:0/4261138840 shutdown_connections 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 --2- 172.21.15.22:0/4261138840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a3010b850 0x7f5a3010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 --2- 172.21.15.22:0/4261138840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a30102e30 0x7f5a3010b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 --2- 172.21.15.22:0/4261138840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102510 0x7f5a301028f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 -- 172.21.15.22:0/4261138840 >> 172.21.15.22:0/4261138840 conn(0x7f5a300773a0 msgr2=0x7f5a300777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 -- 172.21.15.22:0/4261138840 shutdown_connections 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.988+0000 7f5a3610a640 1 -- 172.21.15.22:0/4261138840 wait complete. 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 Processor -- start 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 -- start start 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a30102510 0x7f5a30199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102e30 0x7f5a3019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a3010b850 0x7f5a3019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301116b0 con 0x7f5a30102e30 2024-03-08T23:57:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30111530 con 0x7f5a3010b850 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a3610a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30111830 con 0x7f5a30102510 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a2effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102e30 0x7f5a3019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.989+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a30102510 0x7f5a30199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102e30 0x7f5a3019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41886/0 (socket says 172.21.15.22:41886) 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2effd640 1 -- 172.21.15.22:0/3971028351 learned_addr learned my addr 172.21.15.22:0/3971028351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2ffff640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a3010b850 0x7f5a3019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/3971028351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a3010b850 msgr2=0x7f5a3019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a3010b850 0x7f5a3019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102e30 msgr2=0x7f5a3019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102e30 0x7f5a3019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/3971028351 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3019edb0 con 0x7f5a30102510 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2effd640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102e30 0x7f5a3019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a30102510 0x7f5a30199e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a20000c00 tx=0x7f5a2002cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2cff9640 1 -- 172.21.15.22:0/3971028351 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20041070 con 0x7f5a30102510 2024-03-08T23:57:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3019f040 con 0x7f5a30102510 2024-03-08T23:57:39.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2cff9640 1 -- 172.21.15.22:0/3971028351 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a20004450 con 0x7f5a30102510 2024-03-08T23:57:39.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a2cff9640 1 -- 172.21.15.22:0/3971028351 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20003e50 con 0x7f5a30102510 2024-03-08T23:57:39.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.990+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a301a6930 con 0x7f5a30102510 2024-03-08T23:57:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.992+0000 7f5a2cff9640 1 -- 172.21.15.22:0/3971028351 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5a20044070 con 0x7f5a30102510 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 Processor -- start 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 -- start start 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2cff9640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a180a2b10 0x7f5a180a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc001a0 0x55c9fbc00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2ffff640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a180a2b10 0x7f5a180a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc09160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fbbc7360 con 0x55c9fbc00ac0 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fbb63ec0 con 0x55c9fbc001a0 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fbb0cd30 con 0x55c9fbc096a0 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2cff9640 1 -- 172.21.15.22:0/3971028351 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5a200f0af0 con 0x7f5a30102510 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc001a0 0x55c9fbc00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc09160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc001a0 0x55c9fbc00580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:54050/0 (socket says 172.21.15.22:54050) 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/949076663 learned_addr learned my addr 172.21.15.22:0/949076663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.993+0000 7f5a2ffff640 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2effd640 1 -- 172.21.15.22:0/949076663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc096a0 msgr2=0x55c9fbc0ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2effd640 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2effd640 1 -- 172.21.15.22:0/949076663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc001a0 msgr2=0x55c9fbc00580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2effd640 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc001a0 0x55c9fbc00580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2effd640 1 -- 172.21.15.22:0/949076663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9fbb0bbc0 con 0x55c9fbc00ac0 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc001a0 0x55c9fbc00580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2ffff640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a180a2b10 0x7f5a180a4fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800a530 tx=0x7f5a2800a930 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a2effd640 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc09160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5a3010a8f0 tx=0x7f5a240118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6ae58aea5cd9fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a15ffb640 1 -- 172.21.15.22:0/949076663 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2401e070 con 0x55c9fbc00ac0 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a15ffb640 1 -- 172.21.15.22:0/949076663 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a2400f960 con 0x55c9fbc00ac0 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.994+0000 7f5a15ffb640 1 -- 172.21.15.22:0/949076663 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a24019a90 con 0x55c9fbc00ac0 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 -- 172.21.15.22:0/949076663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc00ac0 msgr2=0x55c9fbc09160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc09160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5a3010a8f0 tx=0x7f5a240118e0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 -- 172.21.15.22:0/949076663 shutdown_connections 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc09160 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/949076663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc001a0 0x55c9fbc00580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 -- 172.21.15.22:0/949076663 >> 172.21.15.22:0/949076663 conn(0x55c9fbbfa8d0 msgr2=0x55c9fbbfacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 -- 172.21.15.22:0/949076663 shutdown_connections 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 -- 172.21.15.22:0/949076663 wait complete. 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 -- start start 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.995+0000 7f5a38dc3740 1 Processor -- start 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc06bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fbb024a0 con 0x55c9fbc001a0 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fbb5ff60 con 0x55c9fbc096a0 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9fbc0c490 con 0x55c9fbc00ac0 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 10 client.0 _ll_register_callbacks cb 0x55c9fbb5dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41896/0 (socket says 172.21.15.22:41896) 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/2052726718 learned_addr learned my addr 172.21.15.22:0/2052726718 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2effd640 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc06bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2ffff640 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc07130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/2052726718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 msgr2=0x55c9fbc06bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc06bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 msgr2=0x55c9fbc07130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc07130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/2052726718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9fb8783f0 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc066b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5a200bcea0 tx=0x7f5a200bcb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9b8baca970f87d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a0effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a0effd640 1 -- 172.21.15.22:0/2052726718 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2004b040 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a0effd640 1 -- 172.21.15.22:0/2052726718 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5a20047bd0 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.996+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a2400f960 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a0effd640 1 -- 172.21.15.22:0/2052726718 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a200bcf10 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 10 client.16710 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c9fb9427d0 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a0effd640 1 -- 172.21.15.22:0/2052726718 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a2400f960 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a0effd640 1 -- 172.21.15.22:0/2052726718 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a200ee0c0 con 0x55c9fbc001a0 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 10 client.16710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:57:40.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 20 client.16710 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc19870 0x55c9fbc1bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 4 client.16710 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9fbb71400 con 0x55c9fbc19870 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.997+0000 7f5a2effd640 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc19870 0x55c9fbc1bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a0effd640 10 client.16710 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a2effd640 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc19870 0x55c9fbc1bc60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a47a07c27d417ba0 server_cookie=1a6bcfa4ce6bd173 in_seq=0 out_seq=0 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a0effd640 1 -- 172.21.15.22:0/2052726718 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9fbb71400 con 0x55c9fbc19870 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a0effd640 10 client.16710 handle_command_reply: tid=0 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 client.16710 shutdown 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 msgr2=0x55c9fbc066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc066b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5a200bcea0 tx=0x7f5a200bcb10 comp rx=0 tx=0).stop 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 shutdown_connections 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc19870 0x55c9fbc1bc60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc06bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a0effd640 0 client.16710 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:40.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/2052726718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc066b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 >> 172.21.15.22:0/2052726718 conn(0x55c9fbbfa8d0 msgr2=0x55c9fbbfc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 shutdown_connections 2024-03-08T23:57:40.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.998+0000 7f5a38dc3740 1 -- 172.21.15.22:0/2052726718 wait complete. 2024-03-08T23:57:40.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:39.999+0000 7f5a38dc3740 20 client.16710 trim_cache size 0 max 16384 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 Processor -- start 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 -- start start 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc001a0 0x55c9fbc08de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc03910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc096a0 0x55c9fbc03e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301b2c90 con 0x55c9fbc096a0 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301b0830 con 0x55c9fbc00ac0 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301ae8d0 con 0x55c9fbc001a0 2024-03-08T23:57:40.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc001a0 0x55c9fbc08de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc001a0 0x55c9fbc08de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60700/0 (socket says 172.21.15.22:60700) 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/3862599989 learned_addr learned my addr 172.21.15.22:0/3862599989 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a2ffff640 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc096a0 0x55c9fbc03e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.012+0000 7f5a2effd640 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc03910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/3862599989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc00ac0 msgr2=0x55c9fbc03910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc03910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/3862599989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc096a0 msgr2=0x55c9fbc03e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc096a0 0x55c9fbc03e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/3862599989 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9fbc04c00 con 0x55c9fbc001a0 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc001a0 0x55c9fbc08de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a2002c8c0 tx=0x7f5a200bb420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae4beaef3b4617a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a157fa640 1 -- 172.21.15.22:0/3862599989 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2004b070 con 0x55c9fbc001a0 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a157fa640 1 -- 172.21.15.22:0/3862599989 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a200c53b0 con 0x55c9fbc001a0 2024-03-08T23:57:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.013+0000 7f5a157fa640 1 -- 172.21.15.22:0/3862599989 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200ee290 con 0x55c9fbc001a0 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 -- 172.21.15.22:0/3862599989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc001a0 msgr2=0x55c9fbc08de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc001a0 0x55c9fbc08de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a2002c8c0 tx=0x7f5a200bb420 comp rx=0 tx=0).stop 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 -- 172.21.15.22:0/3862599989 shutdown_connections 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc096a0 0x55c9fbc03e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc03910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/3862599989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc001a0 0x55c9fbc08de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 -- 172.21.15.22:0/3862599989 >> 172.21.15.22:0/3862599989 conn(0x55c9fbbfa8d0 msgr2=0x55c9fbbfacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 -- 172.21.15.22:0/3862599989 shutdown_connections 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 -- 172.21.15.22:0/3862599989 wait complete. 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.014+0000 7f5a38dc3740 1 -- start start 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 1 Processor -- start 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc0f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc0f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301aeed0 con 0x55c9fbc001a0 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301b01d0 con 0x55c9fbc096a0 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301af510 con 0x55c9fbc00ac0 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a38dc3740 10 client.0 _ll_register_callbacks cb 0x7f5a24009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc0f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc0f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41916/0 (socket says 172.21.15.22:41916) 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/86184636 learned_addr learned my addr 172.21.15.22:0/86184636 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:40.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2effd640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc0f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2ffff640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/86184636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 msgr2=0x55c9fbc0f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc0f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 msgr2=0x55c9fbc0fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2f7fe640 1 -- 172.21.15.22:0/86184636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9fbc6fdd0 con 0x55c9fbc001a0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.015+0000 7f5a2effd640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc0f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a2f7fe640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc0f140 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5a2003e1f0 tx=0x7f5a200bcb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a8e4ca59e8dca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a0effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a0effd640 1 -- 172.21.15.22:0/86184636 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2004b070 con 0x55c9fbc001a0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a0effd640 1 -- 172.21.15.22:0/86184636 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5a200ee030 con 0x55c9fbc001a0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a200c53b0 con 0x55c9fbc001a0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a0effd640 1 -- 172.21.15.22:0/86184636 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a200c0fb0 con 0x55c9fbc001a0 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 10 client.16716 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:40.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c9fbc6fdd0 con 0x55c9fbc001a0 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a0effd640 1 -- 172.21.15.22:0/86184636 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a200c53b0 con 0x55c9fbc001a0 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a0effd640 1 -- 172.21.15.22:0/86184636 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a200bbdd0 con 0x55c9fbc001a0 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 10 client.16716 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 10 client.16716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 20 client.16716 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc71350 0x55c9fbc73740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 4 client.16716 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9fbb6b750 con 0x55c9fbc71350 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.016+0000 7f5a2effd640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc71350 0x55c9fbc73740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a2effd640 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc71350 0x55c9fbc73740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=957955054b96c8f5 server_cookie=c89eb40bb4054c19 in_seq=0 out_seq=0 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a0effd640 10 client.16716 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a0effd640 1 -- 172.21.15.22:0/86184636 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9fbb6b750 con 0x55c9fbc71350 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a0effd640 10 client.16716 handle_command_reply: tid=0 2024-03-08T23:57:40.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 client.16716 shutdown 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 msgr2=0x55c9fbc0f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc0f140 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5a2003e1f0 tx=0x7f5a200bcb70 comp rx=0 tx=0).stop 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 shutdown_connections 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c9fbc71350 0x55c9fbc73740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9fbc096a0 0x55c9fbc0fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9fbc00ac0 0x55c9fbc0f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a0effd640 0 client.16716 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 --2- 172.21.15.22:0/86184636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9fbc001a0 0x55c9fbc0f140 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 >> 172.21.15.22:0/86184636 conn(0x55c9fbbfa8d0 msgr2=0x55c9fbc04f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 shutdown_connections 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 1 -- 172.21.15.22:0/86184636 wait complete. 2024-03-08T23:57:40.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.017+0000 7f5a38dc3740 20 client.16716 trim_cache size 0 max 16384 2024-03-08T23:57:40.020 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a180a2b10 msgr2=0x7f5a180a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a180a2b10 0x7f5a180a4fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800a530 tx=0x7f5a2800a930 comp rx=0 tx=0).stop 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a30102510 msgr2=0x7f5a30199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a30102510 0x7f5a30199e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a20000c00 tx=0x7f5a2002cc90 comp rx=0 tx=0).stop 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 shutdown_connections 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a180a2b10 0x7f5a180a4fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a3010b850 0x7f5a3019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a30102e30 0x7f5a3019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 --2- 172.21.15.22:0/3971028351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a30102510 0x7f5a30199e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 >> 172.21.15.22:0/3971028351 conn(0x7f5a300773a0 msgr2=0x7f5a30107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 shutdown_connections 2024-03-08T23:57:40.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.018+0000 7f5a3610a640 1 -- 172.21.15.22:0/3971028351 wait complete. 2024-03-08T23:57:40.024 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:57:40.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:57:40.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 -- 172.21.15.22:0/980042207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca481054b0 msgr2=0x7fca4810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 --2- 172.21.15.22:0/980042207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca481054b0 0x7fca4810d080 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ca0 tx=0x7fca4002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 -- 172.21.15.22:0/980042207 shutdown_connections 2024-03-08T23:57:40.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 --2- 172.21.15.22:0/980042207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca481054b0 0x7fca4810d080 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 --2- 172.21.15.22:0/980042207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca48100d90 0x7fca48104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 --2- 172.21.15.22:0/980042207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 0x7fca481007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 -- 172.21.15.22:0/980042207 >> 172.21.15.22:0/980042207 conn(0x7fca480773a0 msgr2=0x7fca480777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 -- 172.21.15.22:0/980042207 shutdown_connections 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 -- 172.21.15.22:0/980042207 wait complete. 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 Processor -- start 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.136+0000 7fca4e53b640 1 -- start start 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4e53b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 0x7fca4819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4e53b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca481054b0 0x7fca4819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4e53b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca48196830 0x7fca48196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4e53b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca48110570 con 0x7fca48196830 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4e53b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481103f0 con 0x7fca481054b0 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4e53b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481106f0 con 0x7fca481003e0 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4cad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca48196830 0x7fca48196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 0x7fca4819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 0x7fca4819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60722/0 (socket says 172.21.15.22:60722) 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 -- 172.21.15.22:0/2305873037 learned_addr learned my addr 172.21.15.22:0/2305873037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca477fe640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca481054b0 0x7fca4819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 -- 172.21.15.22:0/2305873037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca481054b0 msgr2=0x7fca4819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca481054b0 0x7fca4819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 -- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca48196830 msgr2=0x7fca48196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca48196830 0x7fca48196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 -- 172.21.15.22:0/2305873037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca481973c0 con 0x7fca481003e0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca4cad8640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca48196830 0x7fca48196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.137+0000 7fca47fff640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 0x7fca4819c670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9f0 tx=0x7fca3800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.138+0000 7fca457fa640 1 -- 172.21.15.22:0/2305873037 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3800cd80 con 0x7fca481003e0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.138+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca481976b0 con 0x7fca481003e0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.138+0000 7fca457fa640 1 -- 172.21.15.22:0/2305873037 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca38005500 con 0x7fca481003e0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.138+0000 7fca457fa640 1 -- 172.21.15.22:0/2305873037 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38010690 con 0x7fca481003e0 2024-03-08T23:57:40.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.138+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca481a3840 con 0x7fca481003e0 2024-03-08T23:57:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca457fa640 1 -- 172.21.15.22:0/2305873037 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fca380046c0 con 0x7fca481003e0 2024-03-08T23:57:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 Processor -- start 2024-03-08T23:57:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 -- start start 2024-03-08T23:57:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe5f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5ff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe601ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbe5bb360 con 0x563bbe5f4960 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbe557ec0 con 0x563bbe5f3fb0 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.140+0000 7fca511f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbe500d30 con 0x563bbe5ff8b0 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe5f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe5f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:54110/0 (socket says 172.21.15.22:54110) 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 -- 172.21.15.22:0/2918126638 learned_addr learned my addr 172.21.15.22:0/2918126638 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca477fe640 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe601ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca47fff640 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca457fa640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca300a2970 0x7fca300a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 -- 172.21.15.22:0/2918126638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 msgr2=0x563bbe601ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe601ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 -- 172.21.15.22:0/2918126638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 msgr2=0x563bbe5ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5ff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 -- 172.21.15.22:0/2918126638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bbe4ffbc0 con 0x563bbe5f3fb0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca457fa640 1 -- 172.21.15.22:0/2305873037 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fca380cf600 con 0x7fca481003e0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca300a2970 0x7fca300a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca4cad8640 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe5f4390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ca0 tx=0x7fca40005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cd58fcf9e72ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca2e7fc640 1 -- 172.21.15.22:0/2918126638 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40045030 con 0x563bbe5f3fb0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca2e7fc640 1 -- 172.21.15.22:0/2918126638 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca40003e50 con 0x563bbe5f3fb0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.141+0000 7fca2e7fc640 1 -- 172.21.15.22:0/2918126638 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40039040 con 0x563bbe5f3fb0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 -- 172.21.15.22:0/2918126638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 msgr2=0x563bbe5f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe5f4390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ca0 tx=0x7fca40005f80 comp rx=0 tx=0).stop 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca4cad8640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca300a2970 0x7fca300a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca40049000 tx=0x7fca4004a000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 -- 172.21.15.22:0/2918126638 shutdown_connections 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe601ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 --2- 172.21.15.22:0/2918126638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe5f4390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 -- 172.21.15.22:0/2918126638 >> 172.21.15.22:0/2918126638 conn(0x563bbe5ee8d0 msgr2=0x563bbe5eecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 -- 172.21.15.22:0/2918126638 shutdown_connections 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 -- 172.21.15.22:0/2918126638 wait complete. 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.142+0000 7fca511f2740 1 -- start start 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 1 Processor -- start 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbe4f64a0 con 0x563bbe5f4960 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbe553f60 con 0x563bbe5ff8b0 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbe602670 con 0x563bbe5f3fb0 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca511f2740 10 client.0 _ll_register_callbacks cb 0x563bbe551d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60746/0 (socket says 172.21.15.22:60746) 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5fce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41936/0 (socket says 172.21.15.22:41936) 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 -- 172.21.15.22:0/906697538 learned_addr learned my addr 172.21.15.22:0/906697538 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca4cad8640 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 -- 172.21.15.22:0/906697538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5ff8b0 msgr2=0x563bbe5fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 -- 172.21.15.22:0/906697538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 msgr2=0x563bbe5fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca47fff640 1 -- 172.21.15.22:0/906697538 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bbe26c3f0 con 0x563bbe5f3fb0 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.143+0000 7fca477fe640 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca47fff640 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5fc8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9c0 tx=0x7fca38004ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e91eb416990b5343 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca277fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:57:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca277fe640 1 -- 172.21.15.22:0/906697538 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3809b220 con 0x563bbe5f3fb0 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca40003e50 con 0x563bbe5f3fb0 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca277fe640 1 -- 172.21.15.22:0/906697538 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fca38086070 con 0x563bbe5f3fb0 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca277fe640 1 -- 172.21.15.22:0/906697538 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca380cce40 con 0x563bbe5f3fb0 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 10 client.26188 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563bbe3367d0 con 0x563bbe5f3fb0 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca277fe640 1 -- 172.21.15.22:0/906697538 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca40003e50 con 0x563bbe5f3fb0 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca277fe640 1 -- 172.21.15.22:0/906697538 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca380cf200 con 0x563bbe5f3fb0 2024-03-08T23:57:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 10 client.26188 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 20 client.26188 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe60fac0 0x563bbe611eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 4 client.26188 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bbe565400 con 0x563bbe60fac0 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.144+0000 7fca477fe640 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe60fac0 0x563bbe611eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.145+0000 7fca277fe640 10 client.26188 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.145+0000 7fca477fe640 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe60fac0 0x563bbe611eb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc12625a65ce552 server_cookie=52f0f4c746ea33d8 in_seq=0 out_seq=0 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca277fe640 1 -- 172.21.15.22:0/906697538 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563bbe565400 con 0x563bbe60fac0 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca277fe640 10 client.26188 handle_command_reply: tid=0 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 client.26188 shutdown 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 msgr2=0x563bbe5fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5fc8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9c0 tx=0x7fca38004ae0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 shutdown_connections 2024-03-08T23:57:40.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe60fac0 0x563bbe611eb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe5fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 --2- 172.21.15.22:0/906697538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5fc8e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca277fe640 0 client.26188 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:40.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 >> 172.21.15.22:0/906697538 conn(0x563bbe5ee8d0 msgr2=0x563bbe5f0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 shutdown_connections 2024-03-08T23:57:40.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 1 -- 172.21.15.22:0/906697538 wait complete. 2024-03-08T23:57:40.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.146+0000 7fca511f2740 20 client.26188 trim_cache size 0 max 16384 2024-03-08T23:57:40.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 Processor -- start 2024-03-08T23:57:40.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 -- start start 2024-03-08T23:57:40.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f4960 0x563bbe5f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481ae8e0 con 0x563bbe5ff8b0 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481ac480 con 0x563bbe5f4960 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca511f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481aa520 con 0x563bbe5f3fb0 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41946/0 (socket says 172.21.15.22:41946) 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 -- 172.21.15.22:0/1622335575 learned_addr learned my addr 172.21.15.22:0/1622335575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca477fe640 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f4960 0x563bbe5f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca47fff640 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 -- 172.21.15.22:0/1622335575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 msgr2=0x563bbe5feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 -- 172.21.15.22:0/1622335575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f4960 msgr2=0x563bbe5f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f4960 0x563bbe5f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.158+0000 7fca4cad8640 1 -- 172.21.15.22:0/1622335575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bbe602010 con 0x563bbe5ff8b0 2024-03-08T23:57:40.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca4cad8640 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fa0c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fca40009600 tx=0x7fca4004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446378a2ab7f1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca2dffb640 1 -- 172.21.15.22:0/1622335575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40058070 con 0x563bbe5ff8b0 2024-03-08T23:57:40.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca2dffb640 1 -- 172.21.15.22:0/1622335575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca40003e50 con 0x563bbe5ff8b0 2024-03-08T23:57:40.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca2dffb640 1 -- 172.21.15.22:0/1622335575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40039070 con 0x563bbe5ff8b0 2024-03-08T23:57:40.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 -- 172.21.15.22:0/1622335575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5ff8b0 msgr2=0x563bbe5fa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fa0c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fca40009600 tx=0x7fca4004e000 comp rx=0 tx=0).stop 2024-03-08T23:57:40.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 -- 172.21.15.22:0/1622335575 shutdown_connections 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5ff8b0 0x563bbe5fa0c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f4960 0x563bbe5f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 --2- 172.21.15.22:0/1622335575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5f3fb0 0x563bbe5feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 -- 172.21.15.22:0/1622335575 >> 172.21.15.22:0/1622335575 conn(0x563bbe5ee8d0 msgr2=0x563bbe5f7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 -- 172.21.15.22:0/1622335575 shutdown_connections 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.159+0000 7fca511f2740 1 -- 172.21.15.22:0/1622335575 wait complete. 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 -- start start 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 Processor -- start 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe6602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe660810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe605b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481aab20 con 0x563bbe5f4960 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481abe20 con 0x563bbe5f3fb0 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca481ab160 con 0x563bbe5ff8b0 2024-03-08T23:57:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca511f2740 10 client.0 _ll_register_callbacks cb 0x7fca3c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe660810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca47fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe6602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe660810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41954/0 (socket says 172.21.15.22:41954) 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 -- 172.21.15.22:0/2363808641 learned_addr learned my addr 172.21.15.22:0/2363808641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca4cad8640 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe605b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 -- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 msgr2=0x563bbe605b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe605b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 -- 172.21.15.22:0/2363808641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 msgr2=0x563bbe6602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe6602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.160+0000 7fca477fe640 1 -- 172.21.15.22:0/2363808641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bbe606220 con 0x563bbe5f4960 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca477fe640 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe660810 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005d00 tx=0x7fca3c012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516eb18e2edc572c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca277fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:40.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca277fe640 1 -- 172.21.15.22:0/2363808641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c021770 con 0x563bbe5f4960 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca277fe640 1 -- 172.21.15.22:0/2363808641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fca3c004030 con 0x563bbe5f4960 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca40003e50 con 0x563bbe5f4960 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca277fe640 1 -- 172.21.15.22:0/2363808641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c033050 con 0x563bbe5f4960 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca511f2740 10 client.16734 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563bbe667aa0 con 0x563bbe5f4960 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca277fe640 1 -- 172.21.15.22:0/2363808641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca40003e50 con 0x563bbe5f4960 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca277fe640 1 -- 172.21.15.22:0/2363808641 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca3c028020 con 0x563bbe5f4960 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca511f2740 10 client.16734 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca511f2740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.161+0000 7fca511f2740 20 client.16734 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe6681b0 0x563bbe66a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 4 client.16734 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563bbe55f750 con 0x563bbe6681b0 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca47fff640 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe6681b0 0x563bbe66a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca47fff640 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe6681b0 0x563bbe66a5d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fa78c0bc33088e server_cookie=fd6d1cdb9752c1af in_seq=0 out_seq=0 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca277fe640 10 client.16734 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca277fe640 1 -- 172.21.15.22:0/2363808641 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563bbe55f750 con 0x563bbe6681b0 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca277fe640 10 client.16734 handle_command_reply: tid=0 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 client.16734 shutdown 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 msgr2=0x563bbe660810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe660810 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005d00 tx=0x7fca3c012b20 comp rx=0 tx=0).stop 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 shutdown_connections 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563bbe6681b0 0x563bbe66a5d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563bbe5ff8b0 0x563bbe605b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bbe5f4960 0x563bbe660810 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 --2- 172.21.15.22:0/2363808641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bbe5f3fb0 0x563bbe6602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca277fe640 0 client.16734 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.162+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 >> 172.21.15.22:0/2363808641 conn(0x563bbe5ee8d0 msgr2=0x563bbe5efd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 shutdown_connections 2024-03-08T23:57:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca511f2740 1 -- 172.21.15.22:0/2363808641 wait complete. 2024-03-08T23:57:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca511f2740 20 client.16734 trim_cache size 0 max 16384 2024-03-08T23:57:40.167 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca300a2970 msgr2=0x7fca300a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca300a2970 0x7fca300a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca40049000 tx=0x7fca4004a000 comp rx=0 tx=0).stop 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 msgr2=0x7fca4819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 0x7fca4819c670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9f0 tx=0x7fca3800edf0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 shutdown_connections 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca300a2970 0x7fca300a4e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca48196830 0x7fca48196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca481054b0 0x7fca4819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 --2- 172.21.15.22:0/2305873037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca481003e0 0x7fca4819c670 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 >> 172.21.15.22:0/2305873037 conn(0x7fca480773a0 msgr2=0x7fca480fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 shutdown_connections 2024-03-08T23:57:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:40.163+0000 7fca4e53b640 1 -- 172.21.15.22:0/2305873037 wait complete. 2024-03-08T23:57:40.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 262631 13.28 MB/sec warmup 108 sec latency 96.401 ms 2024-03-08T23:57:40.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:40 smithi022 ceph-mon[31894]: pgmap v98: 129 pgs: 129 active+clean; 49 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 468 op/s 2024-03-08T23:57:40.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:40 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3647141534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:57:40.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:40 smithi062 ceph-mon[36129]: pgmap v98: 129 pgs: 129 active+clean; 49 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 468 op/s 2024-03-08T23:57:40.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:40 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3647141534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:57:40.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:40 smithi192 ceph-mon[35995]: pgmap v98: 129 pgs: 129 active+clean; 49 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 468 op/s 2024-03-08T23:57:40.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:40 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3647141534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:57:41.170 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:57:41.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:57:41.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:57:41.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 -- 172.21.15.22:0/3007223011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08100da0 msgr2=0x7feb08104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/3007223011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08100da0 0x7feb08104de0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009900 tx=0x7feaf002bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 -- 172.21.15.22:0/3007223011 shutdown_connections 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/3007223011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb081054c0 0x7feb0810d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/3007223011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb08100da0 0x7feb08104de0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/3007223011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081003f0 0x7feb081007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 -- 172.21.15.22:0/3007223011 >> 172.21.15.22:0/3007223011 conn(0x7feb08076fa0 msgr2=0x7feb080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 -- 172.21.15.22:0/3007223011 shutdown_connections 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.290+0000 7feb0d6b8640 1 -- 172.21.15.22:0/3007223011 wait complete. 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 Processor -- start 2024-03-08T23:57:41.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 -- start start 2024-03-08T23:57:41.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb081003f0 0x7feb0819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb081054c0 0x7feb0819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb08196940 0x7feb08196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081105a0 con 0x7feb081003f0 2024-03-08T23:57:41.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08110420 con 0x7feb08196940 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb0d6b8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08110720 con 0x7feb081054c0 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb081003f0 0x7feb0819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb08196940 0x7feb08196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb081003f0 0x7feb0819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43198/0 (socket says 172.21.15.22:43198) 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feafffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb081054c0 0x7feb0819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.291+0000 7feb06ffd640 1 -- 172.21.15.22:0/1299959943 learned_addr learned my addr 172.21.15.22:0/1299959943 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb06ffd640 1 -- 172.21.15.22:0/1299959943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb081054c0 msgr2=0x7feb0819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb06ffd640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb081054c0 0x7feb0819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb06ffd640 1 -- 172.21.15.22:0/1299959943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb08196940 msgr2=0x7feb08196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb06ffd640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb08196940 0x7feb08196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb06ffd640 1 -- 172.21.15.22:0/1299959943 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08197460 con 0x7feb081003f0 2024-03-08T23:57:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb077fe640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb08196940 0x7feb08196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb06ffd640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb081003f0 0x7feb0819c740 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9f0 tx=0x7feaf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb04ff9640 1 -- 172.21.15.22:0/1299959943 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf800cd80 con 0x7feb081003f0 2024-03-08T23:57:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb04ff9640 1 -- 172.21.15.22:0/1299959943 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaf8010910 con 0x7feb081003f0 2024-03-08T23:57:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb08197750 con 0x7feb081003f0 2024-03-08T23:57:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb04ff9640 1 -- 172.21.15.22:0/1299959943 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8018990 con 0x7feb081003f0 2024-03-08T23:57:41.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.292+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb081a3950 con 0x7feb081003f0 2024-03-08T23:57:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb04ff9640 1 -- 172.21.15.22:0/1299959943 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7feaf8010ac0 con 0x7feb081003f0 2024-03-08T23:57:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 Processor -- start 2024-03-08T23:57:41.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 -- start start 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd91de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd928f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd92b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcfd881b40 con 0x55fcfd929490 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcfd82ad80 con 0x55fcfd91e390 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.295+0000 7feb10370740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcfd87dbe0 con 0x55fcfd91da70 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd928f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feafffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd92b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb077fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd91de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd928f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:44682/0 (socket says 172.21.15.22:44682) 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 -- 172.21.15.22:0/2066504183 learned_addr learned my addr 172.21.15.22:0/2066504183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 -- 172.21.15.22:0/2066504183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 msgr2=0x55fcfd91de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 --2- 172.21.15.22:0/2066504183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd91de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 -- 172.21.15.22:0/2066504183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 msgr2=0x55fcfd92b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 --2- 172.21.15.22:0/2066504183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd92b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 -- 172.21.15.22:0/2066504183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcfd65e7e0 con 0x55fcfd91e390 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb04ff9640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feaf40a2970 0x7feaf40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb077fe640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feaf40a2970 0x7feaf40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb06ffd640 1 --2- 172.21.15.22:0/2066504183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd928f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cc890 tx=0x7feaf80db050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b01290b0658342f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feb04ff9640 1 -- 172.21.15.22:0/1299959943 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7feaf80cf570 con 0x7feb081003f0 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.296+0000 7feafd7fa640 1 -- 172.21.15.22:0/2066504183 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf809b6b0 con 0x55fcfd91e390 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.297+0000 7feafd7fa640 1 -- 172.21.15.22:0/2066504183 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaf809b8e0 con 0x55fcfd91e390 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.297+0000 7feafd7fa640 1 -- 172.21.15.22:0/2066504183 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf809b240 con 0x55fcfd91e390 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.297+0000 7feb10370740 1 -- 172.21.15.22:0/2066504183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 msgr2=0x55fcfd928f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.297+0000 7feb077fe640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feaf40a2970 0x7feaf40a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb00005fd0 tx=0x7feb00009600 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.297+0000 7feb10370740 1 --2- 172.21.15.22:0/2066504183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd928f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cc890 tx=0x7feaf80db050 comp rx=0 tx=0).stop 2024-03-08T23:57:41.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 -- 172.21.15.22:0/2066504183 shutdown_connections 2024-03-08T23:57:41.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 --2- 172.21.15.22:0/2066504183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd92b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 --2- 172.21.15.22:0/2066504183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd928f50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 --2- 172.21.15.22:0/2066504183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd91de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 -- 172.21.15.22:0/2066504183 >> 172.21.15.22:0/2066504183 conn(0x55fcfd9184b0 msgr2=0x55fcfd918890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 -- 172.21.15.22:0/2066504183 shutdown_connections 2024-03-08T23:57:41.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 -- 172.21.15.22:0/2066504183 wait complete. 2024-03-08T23:57:41.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 -- start start 2024-03-08T23:57:41.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.298+0000 7feb10370740 1 Processor -- start 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb10370740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91da70 0x55fcfd9246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb10370740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91e390 0x55fcfd924c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb10370740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd925180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb10370740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcfd91ef20 con 0x55fcfd91da70 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb10370740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcfd83e1c0 con 0x55fcfd929490 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb10370740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcfd91f0a0 con 0x55fcfd91e390 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb10370740 10 client.0 _ll_register_callbacks cb 0x55fcfd82d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91da70 0x55fcfd9246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91da70 0x55fcfd9246c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43214/0 (socket says 172.21.15.22:43214) 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 -- 172.21.15.22:0/456664560 learned_addr learned my addr 172.21.15.22:0/456664560 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd925180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feafffff640 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91e390 0x55fcfd924c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 -- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91e390 msgr2=0x55fcfd924c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91e390 0x55fcfd924c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 -- 172.21.15.22:0/456664560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 msgr2=0x55fcfd925180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd925180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 -- 172.21.15.22:0/456664560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcfd6bef70 con 0x55fcfd91da70 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb077fe640 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd925180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feb06ffd640 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91da70 0x55fcfd9246c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cc9d0 tx=0x7feaf8097cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404ff4e6016fc50d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.299+0000 7feaee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feaee7fc640 1 -- 172.21.15.22:0/456664560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf809b420 con 0x55fcfd91da70 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feaee7fc640 1 -- 172.21.15.22:0/456664560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7feaf80d90c0 con 0x55fcfd91da70 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaf809b8e0 con 0x55fcfd91da70 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feaee7fc640 1 -- 172.21.15.22:0/456664560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf80b4050 con 0x55fcfd91da70 2024-03-08T23:57:41.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 10 client.16746 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fcfd556170 con 0x55fcfd91da70 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feaee7fc640 1 -- 172.21.15.22:0/456664560 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaf809b8e0 con 0x55fcfd91da70 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feaee7fc640 1 -- 172.21.15.22:0/456664560 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feaf80860d0 con 0x55fcfd91da70 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 10 client.16746 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 20 client.16746 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd937270 0x55fcfd939660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 4 client.16746 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcfd857960 con 0x55fcfd937270 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.300+0000 7feafffff640 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd937270 0x55fcfd939660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.301+0000 7feaee7fc640 10 client.16746 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.301+0000 7feafffff640 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd937270 0x55fcfd939660 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe1641f8d65c0cb server_cookie=d594fb00f9857cdc in_seq=0 out_seq=0 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feaee7fc640 1 -- 172.21.15.22:0/456664560 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fcfd857960 con 0x55fcfd937270 2024-03-08T23:57:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feaee7fc640 10 client.16746 handle_command_reply: tid=0 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 client.16746 shutdown 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91da70 msgr2=0x55fcfd9246c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91da70 0x55fcfd9246c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cc9d0 tx=0x7feaf8097cb0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 shutdown_connections 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd937270 0x55fcfd939660 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd925180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91e390 0x55fcfd924c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 --2- 172.21.15.22:0/456664560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91da70 0x55fcfd9246c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 >> 172.21.15.22:0/456664560 conn(0x55fcfd9184b0 msgr2=0x55fcfd91a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feaee7fc640 0 client.16746 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 shutdown_connections 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 1 -- 172.21.15.22:0/456664560 wait complete. 2024-03-08T23:57:41.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.302+0000 7feb10370740 20 client.16746 trim_cache size 0 max 16384 2024-03-08T23:57:41.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 Processor -- start 2024-03-08T23:57:41.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 -- start start 2024-03-08T23:57:41.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd928e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91e390 0x55fcfd932780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd93b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081ae890 con 0x55fcfd91e390 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081ac430 con 0x55fcfd929490 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.321+0000 7feb10370740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081aa4d0 con 0x55fcfd91da70 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91e390 0x55fcfd932780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91e390 0x55fcfd932780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43220/0 (socket says 172.21.15.22:43220) 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 -- 172.21.15.22:0/3183950406 learned_addr learned my addr 172.21.15.22:0/3183950406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feb06ffd640 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd928e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feb077fe640 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd93b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 -- 172.21.15.22:0/3183950406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 msgr2=0x55fcfd928e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd928e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 -- 172.21.15.22:0/3183950406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 msgr2=0x55fcfd93b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd93b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 -- 172.21.15.22:0/3183950406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcfd920290 con 0x55fcfd91e390 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafffff640 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91e390 0x55fcfd932780 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009850 tx=0x7feaf002df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990a15ea6623bdcf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafcff9640 1 -- 172.21.15.22:0/3183950406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00040c0 con 0x55fcfd91e390 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafcff9640 1 -- 172.21.15.22:0/3183950406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaf0004260 con 0x55fcfd91e390 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.322+0000 7feafcff9640 1 -- 172.21.15.22:0/3183950406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00046c0 con 0x55fcfd91e390 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 -- 172.21.15.22:0/3183950406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91e390 msgr2=0x55fcfd932780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91e390 0x55fcfd932780 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009850 tx=0x7feaf002df90 comp rx=0 tx=0).stop 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 -- 172.21.15.22:0/3183950406 shutdown_connections 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd929490 0x55fcfd93b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd91e390 0x55fcfd932780 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 --2- 172.21.15.22:0/3183950406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd928e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 -- 172.21.15.22:0/3183950406 >> 172.21.15.22:0/3183950406 conn(0x55fcfd9184b0 msgr2=0x55fcfd932070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 -- 172.21.15.22:0/3183950406 shutdown_connections 2024-03-08T23:57:41.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.323+0000 7feb10370740 1 -- 172.21.15.22:0/3183950406 wait complete. 2024-03-08T23:57:41.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 -- start start 2024-03-08T23:57:41.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 Processor -- start 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd9380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd938630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd938b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081aaad0 con 0x55fcfd929490 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081abdd0 con 0x55fcfd91e390 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081ab110 con 0x55fcfd91da70 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb10370740 10 client.0 _ll_register_callbacks cb 0x7feaf80d9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd938b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd938b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43226/0 (socket says 172.21.15.22:43226) 2024-03-08T23:57:41.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 -- 172.21.15.22:0/95685477 learned_addr learned my addr 172.21.15.22:0/95685477 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb06ffd640 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd9380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feafffff640 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd938630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 -- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 msgr2=0x55fcfd9380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd9380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 -- 172.21.15.22:0/95685477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 msgr2=0x55fcfd938630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd938630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 -- 172.21.15.22:0/95685477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcfd98d3c0 con 0x55fcfd929490 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feb077fe640 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd938b70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7feb0001db30 tx=0x7feb00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141b9fd8f2c89601 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.324+0000 7feaee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feaee7fc640 1 -- 172.21.15.22:0/95685477 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00011ea0 con 0x55fcfd929490 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feaee7fc640 1 -- 172.21.15.22:0/95685477 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7feb0001cd70 con 0x55fcfd929490 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaf0004260 con 0x55fcfd929490 2024-03-08T23:57:41.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feaee7fc640 1 -- 172.21.15.22:0/95685477 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb00028400 con 0x55fcfd929490 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 10 client.16752 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fcfd98e740 con 0x55fcfd929490 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feaee7fc640 1 -- 172.21.15.22:0/95685477 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaf0004260 con 0x55fcfd929490 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feaee7fc640 1 -- 172.21.15.22:0/95685477 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feb0001a040 con 0x55fcfd929490 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 10 client.16752 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 20 client.16752 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd98eef0 0x55fcfd991310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 4 client.16752 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fcfd843080 con 0x55fcfd98eef0 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.325+0000 7feb06ffd640 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd98eef0 0x55fcfd991310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.326+0000 7feb06ffd640 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd98eef0 0x55fcfd991310 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757d1fe6fd802e88 server_cookie=b69d07bfa45e045c in_seq=0 out_seq=0 2024-03-08T23:57:41.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.326+0000 7feaee7fc640 10 client.16752 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feaee7fc640 1 -- 172.21.15.22:0/95685477 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fcfd843080 con 0x55fcfd98eef0 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feaee7fc640 10 client.16752 handle_command_reply: tid=0 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 client.16752 shutdown 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 msgr2=0x55fcfd938b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd938b70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7feb0001db30 tx=0x7feb00004a40 comp rx=0 tx=0).stop 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 shutdown_connections 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fcfd98eef0 0x55fcfd991310 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fcfd929490 0x55fcfd938b70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcfd91e390 0x55fcfd938630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 --2- 172.21.15.22:0/95685477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fcfd91da70 0x55fcfd9380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 >> 172.21.15.22:0/95685477 conn(0x55fcfd9184b0 msgr2=0x55fcfd932900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feaee7fc640 0 client.16752 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 shutdown_connections 2024-03-08T23:57:41.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 1 -- 172.21.15.22:0/95685477 wait complete. 2024-03-08T23:57:41.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.327+0000 7feb10370740 20 client.16752 trim_cache size 0 max 16384 2024-03-08T23:57:41.329 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:57:41.329 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:57:41.329 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "bf67ba3d-fef9-4684-8e63-3c819e15c408", 2024-03-08T23:57:41.329 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:57:41.329 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:57:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feaf40a2970 msgr2=0x7feaf40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feaf40a2970 0x7feaf40a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb00005fd0 tx=0x7feb00009600 comp rx=0 tx=0).stop 2024-03-08T23:57:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb081003f0 msgr2=0x7feb0819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb081003f0 0x7feb0819c740 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9f0 tx=0x7feaf800eeb0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 shutdown_connections 2024-03-08T23:57:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feaf40a2970 0x7feaf40a4e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb08196940 0x7feb08196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb081054c0 0x7feb0819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 --2- 172.21.15.22:0/1299959943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb081003f0 0x7feb0819c740 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 >> 172.21.15.22:0/1299959943 conn(0x7feb08076fa0 msgr2=0x7feb080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 shutdown_connections 2024-03-08T23:57:41.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.328+0000 7feb0d6b8640 1 -- 172.21.15.22:0/1299959943 wait complete. 2024-03-08T23:57:41.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:57:41.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 -- 172.21.15.22:0/3069257406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b41065d0 msgr2=0x7f06b410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 --2- 172.21.15.22:0/3069257406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b41065d0 0x7f06b410e1a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009ca0 tx=0x7f06b002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 -- 172.21.15.22:0/3069257406 shutdown_connections 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 --2- 172.21.15.22:0/3069257406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b41065d0 0x7f06b410e1a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 --2- 172.21.15.22:0/3069257406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b4101eb0 0x7f06b4105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 --2- 172.21.15.22:0/3069257406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06b4101500 0x7f06b41018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 -- 172.21.15.22:0/3069257406 >> 172.21.15.22:0/3069257406 conn(0x7f06b40fcd20 msgr2=0x7f06b40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 -- 172.21.15.22:0/3069257406 shutdown_connections 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.465+0000 7f06bb969640 1 -- 172.21.15.22:0/3069257406 wait complete. 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 Processor -- start 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 -- start start 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06b4101500 0x7f06b40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b41065d0 0x7f06b4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b4074070 0x7f06b41a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b4111690 con 0x7f06b4074070 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b4111510 con 0x7f06b41065d0 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06bb969640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b4111810 con 0x7f06b4101500 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b4074070 0x7f06b41a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b4074070 0x7f06b41a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43242/0 (socket says 172.21.15.22:43242) 2024-03-08T23:57:41.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 -- 172.21.15.22:0/3969544863 learned_addr learned my addr 172.21.15.22:0/3969544863 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9705640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06b4101500 0x7f06b40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b41065d0 0x7f06b4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 -- 172.21.15.22:0/3969544863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06b4101500 msgr2=0x7f06b40735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06b4101500 0x7f06b40735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 -- 172.21.15.22:0/3969544863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b41065d0 msgr2=0x7f06b4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b41065d0 0x7f06b4073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 -- 172.21.15.22:0/3969544863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b41a6460 con 0x7f06b4074070 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.466+0000 7f06b9f06640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b4074070 0x7f06b41a5dc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f06b002c700 tx=0x7f06b0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.467+0000 7f06aa7fc640 1 -- 172.21.15.22:0/3969544863 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0041020 con 0x7f06b4074070 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.467+0000 7f06aa7fc640 1 -- 172.21.15.22:0/3969544863 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b0034cd0 con 0x7f06b4074070 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.467+0000 7f06aa7fc640 1 -- 172.21.15.22:0/3969544863 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0039040 con 0x7f06b4074070 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.467+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b41a66f0 con 0x7f06b4074070 2024-03-08T23:57:41.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.467+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b41a6be0 con 0x7f06b4074070 2024-03-08T23:57:41.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.469+0000 7f06aa7fc640 1 -- 172.21.15.22:0/3969544863 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f06b0044020 con 0x7f06b4074070 2024-03-08T23:57:41.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 Processor -- start 2024-03-08T23:57:41.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 -- start start 2024-03-08T23:57:41.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc80b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc89720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06aa7fc640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f06a00a2970 0x7f06a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc89c60 0x5650ecc8c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ecbe3e00 con 0x5650ecc89c60 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ecb8cb10 con 0x5650ecc81080 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06be625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ecbdfea0 con 0x5650ecc80760 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b8f04640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc80b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9f06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc89c60 0x5650ecc8c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9705640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc89720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b8f04640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc80b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38836/0 (socket says 172.21.15.22:38836) 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9705640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc89720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:44720/0 (socket says 172.21.15.22:44720) 2024-03-08T23:57:41.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b8f04640 1 -- 172.21.15.22:0/1082870628 learned_addr learned my addr 172.21.15.22:0/1082870628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06aa7fc640 1 -- 172.21.15.22:0/3969544863 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f06b00f0d30 con 0x7f06b4074070 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f06a00a2970 0x7f06a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9f06640 1 -- 172.21.15.22:0/1082870628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 msgr2=0x5650ecc80b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9f06640 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc80b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9f06640 1 -- 172.21.15.22:0/1082870628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 msgr2=0x5650ecc89720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9f06640 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc89720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b9f06640 1 -- 172.21.15.22:0/1082870628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650ec9c27d0 con 0x5650ecc89c60 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.470+0000 7f06b8f04640 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc80b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:57:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.471+0000 7f06b9f06640 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc89c60 0x5650ecc8c050 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f06b00ee1f0 tx=0x7f06b0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe90b9c6a4ea5fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.471+0000 7f069f7fe640 1 -- 172.21.15.22:0/1082870628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00bcc40 con 0x5650ecc89c60 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.471+0000 7f069f7fe640 1 -- 172.21.15.22:0/1082870628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b0034920 con 0x5650ecc89c60 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.471+0000 7f069f7fe640 1 -- 172.21.15.22:0/1082870628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b008efe0 con 0x5650ecc89c60 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.471+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f06a00a2970 0x7f06a00a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009560 tx=0x7f06a4005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 -- 172.21.15.22:0/1082870628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc89c60 msgr2=0x5650ecc8c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc89c60 0x5650ecc8c050 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f06b00ee1f0 tx=0x7f06b0009a20 comp rx=0 tx=0).stop 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 -- 172.21.15.22:0/1082870628 shutdown_connections 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc89c60 0x5650ecc8c050 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc89720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 --2- 172.21.15.22:0/1082870628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc80b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 -- 172.21.15.22:0/1082870628 >> 172.21.15.22:0/1082870628 conn(0x5650ecc7ae90 msgr2=0x5650ecc7b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 -- 172.21.15.22:0/1082870628 shutdown_connections 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 -- 172.21.15.22:0/1082870628 wait complete. 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 -- start start 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.472+0000 7f06be625740 1 Processor -- start 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06be625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc89360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06be625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06be625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc89c60 0x5650ecc86c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06be625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ecc8c810 con 0x5650ecc81080 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06be625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ecba00a0 con 0x5650ecc89c60 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06be625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ecc8c990 con 0x5650ecc80760 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b8f04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b8f04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc866f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43260/0 (socket says 172.21.15.22:43260) 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b8f04640 1 -- 172.21.15.22:0/4215651546 learned_addr learned my addr 172.21.15.22:0/4215651546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b9705640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc89360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b9f06640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc89c60 0x5650ecc86c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06be625740 10 client.0 _ll_register_callbacks cb 0x5650ecb9fa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b9f06640 1 -- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 msgr2=0x5650ecc89360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b9f06640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc89360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b9f06640 1 -- 172.21.15.22:0/4215651546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 msgr2=0x5650ecc866f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b9f06640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc866f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b9f06640 1 -- 172.21.15.22:0/4215651546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650ec8f83f0 con 0x5650ecc89c60 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.473+0000 7f06b8f04640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f06b9f06640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc89c60 0x5650ecc86c30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06b008f390 tx=0x7f06b00dd0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd197495302b6d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f069cff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f069cff9640 1 -- 172.21.15.22:0/4215651546 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00c2770 con 0x5650ecc89c60 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f069cff9640 1 -- 172.21.15.22:0/4215651546 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f06b00bb1a0 con 0x5650ecc89c60 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06b0034920 con 0x5650ecc89c60 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f069cff9640 1 -- 172.21.15.22:0/4215651546 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b00bca60 con 0x5650ecc89c60 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f06be625740 10 client.26225 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650ec8b80f0 con 0x5650ecc89c60 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.474+0000 7f069cff9640 1 -- 172.21.15.22:0/4215651546 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b0034920 con 0x5650ecc89c60 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f069cff9640 1 -- 172.21.15.22:0/4215651546 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f06b003c030 con 0x5650ecc89c60 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06be625740 10 client.26225 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06be625740 10 client.26225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06be625740 20 client.26225 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06be625740 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650ecc991d0 0x5650ecc9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06be625740 4 client.26225 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:41.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650ecbf1340 con 0x5650ecc991d0 2024-03-08T23:57:41.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06b9705640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650ecc991d0 0x5650ecc9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f069cff9640 10 client.26225 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.475+0000 7f06b9705640 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650ecc991d0 0x5650ecc9b5c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4fd1586735288d server_cookie=330f3fa38ff9c76f in_seq=0 out_seq=0 2024-03-08T23:57:41.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f069cff9640 1 -- 172.21.15.22:0/4215651546 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650ecbf1340 con 0x5650ecc991d0 2024-03-08T23:57:41.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f069cff9640 10 client.26225 handle_command_reply: tid=0 2024-03-08T23:57:41.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 client.26225 shutdown 2024-03-08T23:57:41.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc89c60 msgr2=0x5650ecc86c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc89c60 0x5650ecc86c30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06b008f390 tx=0x7f06b00dd0d0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 shutdown_connections 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650ecc991d0 0x5650ecc9b5c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc89c60 0x5650ecc86c30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f069cff9640 0 client.26225 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 --2- 172.21.15.22:0/4215651546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc80760 0x5650ecc89360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.478+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 >> 172.21.15.22:0/4215651546 conn(0x5650ecc7ae90 msgr2=0x5650ecc7cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.479+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 shutdown_connections 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.479+0000 7f06be625740 1 -- 172.21.15.22:0/4215651546 wait complete. 2024-03-08T23:57:41.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.479+0000 7f06be625740 20 client.26225 trim_cache size 0 max 16384 2024-03-08T23:57:41.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:41 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:57:41.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:41 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:57:41.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:41 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:57:41.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:41 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:57:41.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 Processor -- start 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 -- start start 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc80760 0x5650ecc88db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc84fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41b2c30 con 0x5650ecc81080 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41b07d0 con 0x5650ecc80760 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06be625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41ae870 con 0x5650ecc89c60 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06b8f04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.496+0000 7f06b8f04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc892f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43276/0 (socket says 172.21.15.22:43276) 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b8f04640 1 -- 172.21.15.22:0/3029034226 learned_addr learned my addr 172.21.15.22:0/3029034226 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b9f06640 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc84fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b9705640 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc80760 0x5650ecc88db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b8f04640 1 -- 172.21.15.22:0/3029034226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 msgr2=0x5650ecc84fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc84fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b8f04640 1 -- 172.21.15.22:0/3029034226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc80760 msgr2=0x5650ecc88db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc80760 0x5650ecc88db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b8f04640 1 -- 172.21.15.22:0/3029034226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650ecca06e0 con 0x5650ecc81080 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc892f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f06a400faa0 tx=0x7f06a4008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbecb939b539aad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f069effd640 1 -- 172.21.15.22:0/3029034226 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4020920 con 0x5650ecc81080 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f069effd640 1 -- 172.21.15.22:0/3029034226 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06a4005020 con 0x5650ecc81080 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.497+0000 7f069effd640 1 -- 172.21.15.22:0/3029034226 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4004d00 con 0x5650ecc81080 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 -- 172.21.15.22:0/3029034226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 msgr2=0x5650ecc892f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc892f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f06a400faa0 tx=0x7f06a4008000 comp rx=0 tx=0).stop 2024-03-08T23:57:41.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 -- 172.21.15.22:0/3029034226 shutdown_connections 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc84fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc81080 0x5650ecc892f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 --2- 172.21.15.22:0/3029034226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc80760 0x5650ecc88db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 -- 172.21.15.22:0/3029034226 >> 172.21.15.22:0/3029034226 conn(0x5650ecc7ae90 msgr2=0x5650ecc866b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 -- 172.21.15.22:0/3029034226 shutdown_connections 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 -- 172.21.15.22:0/3029034226 wait complete. 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.498+0000 7f06be625740 1 -- start start 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 1 Processor -- start 2024-03-08T23:57:41.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc80760 0x5650ecc91060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc8d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc8db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41aee70 con 0x5650ecc80760 2024-03-08T23:57:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41b0170 con 0x5650ecc81080 2024-03-08T23:57:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41af4b0 con 0x5650ecc89c60 2024-03-08T23:57:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06be625740 10 client.0 _ll_register_callbacks cb 0x7f06ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:41.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b9705640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc80760 0x5650ecc91060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b9705640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc80760 0x5650ecc91060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43282/0 (socket says 172.21.15.22:43282) 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b9705640 1 -- 172.21.15.22:0/3646510097 learned_addr learned my addr 172.21.15.22:0/3646510097 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc8d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b9f06640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc8db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b8f04640 1 -- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 msgr2=0x5650ecc8db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc8db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b8f04640 1 -- 172.21.15.22:0/3646510097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc80760 msgr2=0x5650ecc91060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc80760 0x5650ecc91060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b8f04640 1 -- 172.21.15.22:0/3646510097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650eccef5d0 con 0x5650ecc81080 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.499+0000 7f06b9705640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc80760 0x5650ecc91060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06b8f04640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc8d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5650ecc899d0 tx=0x7f06a4004c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5251eebab8da0e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:41.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f069cff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f069cff9640 1 -- 172.21.15.22:0/3646510097 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4020920 con 0x5650ecc81080 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f069cff9640 1 -- 172.21.15.22:0/3646510097 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f06a4026040 con 0x5650ecc81080 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06a4005020 con 0x5650ecc81080 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f069cff9640 1 -- 172.21.15.22:0/3646510097 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a4020ac0 con 0x5650ecc81080 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 10 client.26231 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650eccf07a0 con 0x5650ecc81080 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f069cff9640 1 -- 172.21.15.22:0/3646510097 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06a4005020 con 0x5650ecc81080 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f069cff9640 1 -- 172.21.15.22:0/3646510097 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f06a402a9d0 con 0x5650ecc81080 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 10 client.26231 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 20 client.26231 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650eccf0de0 0x5650eccf3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 4 client.26231 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:57:41.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.500+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5650ecbeb690 con 0x5650eccf0de0 2024-03-08T23:57:41.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.501+0000 7f06b9705640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650eccf0de0 0x5650eccf3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:41.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.501+0000 7f069cff9640 10 client.26231 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.501+0000 7f06b9705640 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650eccf0de0 0x5650eccf3200 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f7e311aadb444b server_cookie=1bdbaceb76fcdccd in_seq=0 out_seq=0 2024-03-08T23:57:41.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f069cff9640 1 -- 172.21.15.22:0/3646510097 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5650ecbeb690 con 0x5650eccf0de0 2024-03-08T23:57:41.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f069cff9640 10 client.26231 handle_command_reply: tid=0 2024-03-08T23:57:41.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 client.26231 shutdown 2024-03-08T23:57:41.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 msgr2=0x5650ecc8d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc8d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5650ecc899d0 tx=0x7f06a4004c90 comp rx=0 tx=0).stop 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 shutdown_connections 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5650eccf0de0 0x5650eccf3200 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5650ecc89c60 0x5650ecc8db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f069cff9640 0 client.26231 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650ecc81080 0x5650ecc8d3e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 --2- 172.21.15.22:0/3646510097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5650ecc80760 0x5650ecc91060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 >> 172.21.15.22:0/3646510097 conn(0x5650ecc7ae90 msgr2=0x5650ecc816a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 shutdown_connections 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 1 -- 172.21.15.22:0/3646510097 wait complete. 2024-03-08T23:57:41.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.503+0000 7f06be625740 20 client.26231 trim_cache size 0 max 16384 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout: "bf67ba3d-fef9-4684-8e63-3c819e15c408": { 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "bf67ba3d-fef9-4684-8e63-3c819e15c408", 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:57:41.507 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:57:41.508 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:57:41.508 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f06a00a2970 msgr2=0x7f06a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f06a00a2970 0x7f06a00a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009560 tx=0x7f06a4005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b4074070 msgr2=0x7f06b41a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b4074070 0x7f06b41a5dc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f06b002c700 tx=0x7f06b0032f20 comp rx=0 tx=0).stop 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 shutdown_connections 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f06a00a2970 0x7f06a00a4e30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06b4074070 0x7f06b41a5dc0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b41065d0 0x7f06b4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 --2- 172.21.15.22:0/3969544863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06b4101500 0x7f06b40735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:41.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 >> 172.21.15.22:0/3969544863 conn(0x7f06b40fcd20 msgr2=0x7f06b40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:41.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 shutdown_connections 2024-03-08T23:57:41.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:57:41.504+0000 7f06bb969640 1 -- 172.21.15.22:0/3969544863 wait complete. 2024-03-08T23:57:41.511 INFO:tasks.cephfs.filesystem:scrub status for tag:bf67ba3d-fef9-4684-8e63-3c819e15c408 - {'path': '/', 'tag': 'bf67ba3d-fef9-4684-8e63-3c819e15c408', 'options': 'recursive,force'} 2024-03-08T23:57:41.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 264338 13.22 MB/sec warmup 109 sec latency 19.507 ms 2024-03-08T23:57:41.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:41 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:57:41.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:41 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:57:41.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:41 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:57:41.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:41 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:57:41.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:41 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:57:41.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:41 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:57:41.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:41 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:57:41.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:41 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:57:42.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 266753 13.23 MB/sec warmup 110 sec latency 12.617 ms 2024-03-08T23:57:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:42 smithi022 ceph-mon[31894]: pgmap v99: 129 pgs: 129 active+clean; 52 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 497 op/s 2024-03-08T23:57:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:42 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:57:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:42 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:42 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:57:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:42 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:42 smithi062 ceph-mon[36129]: pgmap v99: 129 pgs: 129 active+clean; 52 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 497 op/s 2024-03-08T23:57:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:42 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:57:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:42 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:42 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:57:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:42 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:42 smithi192 ceph-mon[35995]: pgmap v99: 129 pgs: 129 active+clean; 52 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 497 op/s 2024-03-08T23:57:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:42 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:57:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:42 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:42 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:57:42.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:42 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:57:43.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 269814 13.28 MB/sec warmup 111 sec latency 19.261 ms 2024-03-08T23:57:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 -- 172.21.15.22:0/2178515664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa10069310 msgr2=0x7ffa1010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/2178515664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa10069310 0x7ffa1010e180 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009ca0 tx=0x7ffa0802c180 comp rx=0 tx=0).stop 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 -- 172.21.15.22:0/2178515664 shutdown_connections 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/2178515664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa10069310 0x7ffa1010e180 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/2178515664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa10068970 0x7ffa10068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/2178515664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa101073e0 0x7ffa101077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 -- 172.21.15.22:0/2178515664 >> 172.21.15.22:0/2178515664 conn(0x7ffa10077040 msgr2=0x7ffa10077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 -- 172.21.15.22:0/2178515664 shutdown_connections 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.003+0000 7ffa14b1b640 1 -- 172.21.15.22:0/2178515664 wait complete. 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.004+0000 7ffa14b1b640 1 Processor -- start 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.004+0000 7ffa14b1b640 1 -- start start 2024-03-08T23:57:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa14b1b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa10068970 0x7ffa1019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa14b1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa10069310 0x7ffa1019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa14b1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa101073e0 0x7ffa10199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa14b1b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10111670 con 0x7ffa101073e0 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa14b1b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101114f0 con 0x7ffa10068970 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa14b1b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101117f0 con 0x7ffa10069310 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa0ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa101073e0 0x7ffa10199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa10069310 0x7ffa1019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa0ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa101073e0 0x7ffa10199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43296/0 (socket says 172.21.15.22:43296) 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa10069310 0x7ffa1019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38872/0 (socket says 172.21.15.22:38872) 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa0ed9d640 1 -- 172.21.15.22:0/1867380745 learned_addr learned my addr 172.21.15.22:0/1867380745 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa10068970 0x7ffa1019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 -- 172.21.15.22:0/1867380745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa10068970 msgr2=0x7ffa1019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa10068970 0x7ffa1019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 -- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa101073e0 msgr2=0x7ffa10199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa101073e0 0x7ffa10199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 -- 172.21.15.22:0/1867380745 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1019a190 con 0x7ffa10069310 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa0ed9d640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa101073e0 0x7ffa10199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa10069310 0x7ffa1019fda0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800e9f0 tx=0x7ff9f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.005+0000 7ffa07fff640 1 -- 172.21.15.22:0/1867380745 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800ccc0 con 0x7ffa10069310 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.006+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1019a480 con 0x7ffa10069310 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.006+0000 7ffa07fff640 1 -- 172.21.15.22:0/1867380745 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9f8010910 con 0x7ffa10069310 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.006+0000 7ffa07fff640 1 -- 172.21.15.22:0/1867380745 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8018960 con 0x7ffa10069310 2024-03-08T23:57:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.006+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa101a6720 con 0x7ffa10069310 2024-03-08T23:57:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.008+0000 7ffa07fff640 1 -- 172.21.15.22:0/1867380745 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff9f8010ac0 con 0x7ffa10069310 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.008+0000 7ffa177d5740 1 Processor -- start 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa177d5740 1 -- start start 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd96a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd96fd0 0x55f55fd97450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fda2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa177d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55fcfb5f0 con 0x55f55fd97d40 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa177d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55fca4930 con 0x55f55fda0550 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa177d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55fcf7690 con 0x55f55fd96fd0 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd96a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd96a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43302/0 (socket says 172.21.15.22:43302) 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0ed9d640 1 -- 172.21.15.22:0/788280462 learned_addr learned my addr 172.21.15.22:0/788280462 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd96fd0 0x55f55fd97450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fda2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa07fff640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff9fc0a2a10 0x7ff9fc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0ed9d640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff9fc0a2a10 0x7ff9fc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0e59c640 1 -- 172.21.15.22:0/788280462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 msgr2=0x55f55fda2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fda2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0e59c640 1 -- 172.21.15.22:0/788280462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 msgr2=0x55f55fd96a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd96a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.009+0000 7ffa0e59c640 1 -- 172.21.15.22:0/788280462 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f55fa10210 con 0x55f55fd96fd0 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.010+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fda2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.010+0000 7ffa07fff640 1 -- 172.21.15.22:0/1867380745 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff9f80cf5c0 con 0x7ffa10069310 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.010+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd96fd0 0x55f55fd97450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009bd0 tx=0x7ffa00011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c9ec656c595d83 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.010+0000 7ffa0ed9d640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff9fc0a2a10 0x7ff9fc0a4ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009a60 tx=0x7ffa0802c180 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.010+0000 7ffa04d99640 1 -- 172.21.15.22:0/788280462 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00004230 con 0x55f55fd96fd0 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.010+0000 7ffa04d99640 1 -- 172.21.15.22:0/788280462 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa00009e40 con 0x55f55fd96fd0 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.010+0000 7ffa04d99640 1 -- 172.21.15.22:0/788280462 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00004d90 con 0x55f55fd96fd0 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- 172.21.15.22:0/788280462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd96fd0 msgr2=0x55f55fd97450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd96fd0 0x55f55fd97450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009bd0 tx=0x7ffa00011810 comp rx=0 tx=0).stop 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- 172.21.15.22:0/788280462 shutdown_connections 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fda2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd96fd0 0x55f55fd97450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 --2- 172.21.15.22:0/788280462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd96a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- 172.21.15.22:0/788280462 >> 172.21.15.22:0/788280462 conn(0x55f55fd5e5b0 msgr2=0x55f55fd91cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- 172.21.15.22:0/788280462 shutdown_connections 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- 172.21.15.22:0/788280462 wait complete. 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- start start 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 Processor -- start 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd96fd0 0x55f55fd9d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd9db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fd9e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55fda3280 con 0x55f55fd97d40 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55fcb7d90 con 0x55f55fd96fd0 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55fda3400 con 0x55f55fda0550 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa177d5740 10 client.0 _ll_register_callbacks cb 0x55f55fa0fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.011+0000 7ffa05d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd9db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd9db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43316/0 (socket says 172.21.15.22:43316) 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 -- 172.21.15.22:0/339090038 learned_addr learned my addr 172.21.15.22:0/339090038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa0ed9d640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fd9e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd96fd0 0x55f55fd9d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 -- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 msgr2=0x55f55fd9e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fd9e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 -- 172.21.15.22:0/339090038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd96fd0 msgr2=0x55f55fd9d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd96fd0 0x55f55fd9d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 -- 172.21.15.22:0/339090038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f55fadc620 con 0x55f55fd97d40 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd96fd0 0x55f55fd9d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd9db20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8098300 tx=0x7ff9f80d3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd08142266bdaf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ff9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/339090038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80cf3e0 con 0x55f55fd97d40 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/339090038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ff9f80860e0 con 0x55f55fd97d40 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f55fd08b30 con 0x55f55fd97d40 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/339090038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f8086300 con 0x55f55fd97d40 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa177d5740 10 client.16782 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f55fdaf5c0 con 0x55f55fd97d40 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.012+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/339090038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f55fd08b30 con 0x55f55fd97d40 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/339090038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff9f809ad80 con 0x55f55fd97d40 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa177d5740 10 client.16782 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa177d5740 10 client.16782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa177d5740 20 client.16782 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa177d5740 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fdb0a30 0x55f55fdb2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa177d5740 4 client.16782 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f55fd02e80 con 0x55f55fdb0a30 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fdb0a30 0x55f55fdb2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ff9f5ffb640 10 client.16782 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.013+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fdb0a30 0x55f55fdb2e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b70236630c5468 server_cookie=6724df69aefd430a in_seq=0 out_seq=0 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/339090038 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f55fd02e80 con 0x55f55fdb0a30 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ff9f5ffb640 10 client.16782 handle_command_reply: tid=0 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 client.16782 shutdown 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 msgr2=0x55f55fd9db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd9db20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8098300 tx=0x7ff9f80d3040 comp rx=0 tx=0).stop 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 shutdown_connections 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fdb0a30 0x55f55fdb2e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fd9e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd97d40 0x55f55fd9db20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 --2- 172.21.15.22:0/339090038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd96fd0 0x55f55fd9d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 >> 172.21.15.22:0/339090038 conn(0x55f55fd5e5b0 msgr2=0x55f55fd93a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ff9f5ffb640 0 client.16782 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 shutdown_connections 2024-03-08T23:57:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 1 -- 172.21.15.22:0/339090038 wait complete. 2024-03-08T23:57:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.014+0000 7ffa177d5740 20 client.16782 trim_cache size 0 max 16384 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 Processor -- start 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 -- start start 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fda7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd91f00 0x55f55fda7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101b2b60 con 0x55f55fd96fd0 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101b0700 con 0x55f55fd91f00 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa177d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101ae7a0 con 0x55f55fda0550 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa0e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa0e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43322/0 (socket says 172.21.15.22:43322) 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa05d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fda7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.029+0000 7ffa0e59c640 1 -- 172.21.15.22:0/320619512 learned_addr learned my addr 172.21.15.22:0/320619512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa0ed9d640 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd91f00 0x55f55fda7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa0e59c640 1 -- 172.21.15.22:0/320619512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 msgr2=0x55f55fda7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fda7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa0e59c640 1 -- 172.21.15.22:0/320619512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd91f00 msgr2=0x55f55fda7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd91f00 0x55f55fda7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa0e59c640 1 -- 172.21.15.22:0/320619512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f55fd962a0 con 0x55f55fd96fd0 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fda7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9fd10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffa00011d10 tx=0x7ffa00005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4052d1c8fb200a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ff9f7fff640 1 -- 172.21.15.22:0/320619512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0002b020 con 0x55f55fd96fd0 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ff9f7fff640 1 -- 172.21.15.22:0/320619512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa0001f040 con 0x55f55fd96fd0 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.030+0000 7ff9f7fff640 1 -- 172.21.15.22:0/320619512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0001e040 con 0x55f55fd96fd0 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- 172.21.15.22:0/320619512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 msgr2=0x55f55fd9fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9fd10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffa00011d10 tx=0x7ffa00005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- 172.21.15.22:0/320619512 shutdown_connections 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fd91f00 0x55f55fda7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fda0550 0x55f55fda7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 --2- 172.21.15.22:0/320619512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9fd10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- 172.21.15.22:0/320619512 >> 172.21.15.22:0/320619512 conn(0x55f55fd5e5b0 msgr2=0x55f55fd917f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- 172.21.15.22:0/320619512 shutdown_connections 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- 172.21.15.22:0/320619512 wait complete. 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- start start 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 Processor -- start 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd91f00 0x55f55fd9a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fe06ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101aeda0 con 0x55f55fd96fd0 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101b00a0 con 0x55f55fda0550 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101af3e0 con 0x55f55fd91f00 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.031+0000 7ffa177d5740 10 client.0 _ll_register_callbacks cb 0x7ffa00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd91f00 0x55f55fd9a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa05d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd91f00 0x55f55fd9a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38896/0 (socket says 172.21.15.22:38896) 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fe06ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 -- 172.21.15.22:0/2481554108 learned_addr learned my addr 172.21.15.22:0/2481554108 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 -- 172.21.15.22:0/2481554108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 msgr2=0x55f55fe06ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fe06ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 -- 172.21.15.22:0/2481554108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 msgr2=0x55f55fd9aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 -- 172.21.15.22:0/2481554108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f55fe07460 con 0x55f55fd91f00 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0ed9d640 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fe06ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa0e59c640 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd91f00 0x55f55fd9a9b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa000058c0 tx=0x7ffa00005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6caaa6aabc9f247e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ff9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/2481554108 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0002b030 con 0x55f55fd91f00 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa0001f040 con 0x55f55fd91f00 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.032+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/2481554108 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ffa000270d0 con 0x55f55fd91f00 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/2481554108 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa000272d0 con 0x55f55fd91f00 2024-03-08T23:57:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 10 client.26248 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f55fe08770 con 0x55f55fd91f00 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/2481554108 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa0001f040 con 0x55f55fd91f00 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/2481554108 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa00018070 con 0x55f55fd91f00 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 10 client.26248 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 10 client.26248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 20 client.26248 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fe08c50 0x55f55fe0b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 4 client.26248 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:57:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f55fc81710 con 0x55f55fe08c50 2024-03-08T23:57:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.033+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fe08c50 0x55f55fe0b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.034+0000 7ffa05d9b640 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fe08c50 0x55f55fe0b070 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab73b751c6647e server_cookie=38819c1ba85a315b in_seq=0 out_seq=0 2024-03-08T23:57:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.034+0000 7ff9f5ffb640 10 client.26248 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.326+0000 7ff9f5ffb640 1 -- 172.21.15.22:0/2481554108 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f55fc81710 con 0x55f55fe08c50 2024-03-08T23:57:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.326+0000 7ff9f5ffb640 10 client.26248 handle_command_reply: tid=0 2024-03-08T23:57:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.326+0000 7ffa177d5740 1 client.26248 shutdown 2024-03-08T23:57:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd91f00 msgr2=0x55f55fd9a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd91f00 0x55f55fd9a9b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa000058c0 tx=0x7ffa00005db0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 shutdown_connections 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f55fe08c50 0x55f55fe0b070 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f55fda0550 0x55f55fe06ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f55fd96fd0 0x55f55fd9aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ff9f5ffb640 0 client.26248 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 --2- 172.21.15.22:0/2481554108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f55fd91f00 0x55f55fd9a9b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 >> 172.21.15.22:0/2481554108 conn(0x55f55fd5e5b0 msgr2=0x55f55fd9be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 shutdown_connections 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 1 -- 172.21.15.22:0/2481554108 wait complete. 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa177d5740 20 client.26248 trim_cache size 0 max 16384 2024-03-08T23:57:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:57:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:57:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:57:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff9fc0a2a10 msgr2=0x7ff9fc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff9fc0a2a10 0x7ff9fc0a4ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009a60 tx=0x7ffa0802c180 comp rx=0 tx=0).stop 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa10069310 msgr2=0x7ffa1019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa10069310 0x7ffa1019fda0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800e9f0 tx=0x7ff9f800eeb0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 shutdown_connections 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff9fc0a2a10 0x7ff9fc0a4ed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa101073e0 0x7ffa10199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa10069310 0x7ffa1019fda0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 --2- 172.21.15.22:0/1867380745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa10068970 0x7ffa1019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.327+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 >> 172.21.15.22:0/1867380745 conn(0x7ffa10077040 msgr2=0x7ffa1010c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.328+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 shutdown_connections 2024-03-08T23:57:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:44.328+0000 7ffa14b1b640 1 -- 172.21.15.22:0/1867380745 wait complete. 2024-03-08T23:57:44.708 INFO:tasks.workunit.client.0.smithi022.stdout: 1 271306 13.21 MB/sec warmup 112 sec latency 59.857 ms 2024-03-08T23:57:44.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:44 smithi022 ceph-mon[31894]: pgmap v100: 129 pgs: 129 active+clean; 52 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2024-03-08T23:57:44.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:44 smithi062 ceph-mon[36129]: pgmap v100: 129 pgs: 129 active+clean; 52 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2024-03-08T23:57:44.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:44 smithi192 ceph-mon[35995]: pgmap v100: 129 pgs: 129 active+clean; 52 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2024-03-08T23:57:45.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 273566 13.22 MB/sec warmup 113 sec latency 56.548 ms 2024-03-08T23:57:45.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:45 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:57:45.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:45 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:57:45.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:45 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:57:46.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 276079 13.21 MB/sec warmup 114 sec latency 22.708 ms 2024-03-08T23:57:46.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:46 smithi022 ceph-mon[31894]: pgmap v101: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.3 MiB/s wr, 543 op/s 2024-03-08T23:57:46.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:46 smithi062 ceph-mon[36129]: pgmap v101: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.3 MiB/s wr, 543 op/s 2024-03-08T23:57:46.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:46 smithi192 ceph-mon[35995]: pgmap v101: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.3 MiB/s wr, 543 op/s 2024-03-08T23:57:47.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 278118 13.20 MB/sec warmup 115 sec latency 39.833 ms 2024-03-08T23:57:48.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 280797 13.20 MB/sec warmup 116 sec latency 15.885 ms 2024-03-08T23:57:48.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:48 smithi022 ceph-mon[31894]: pgmap v102: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-03-08T23:57:48.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:48 smithi062 ceph-mon[36129]: pgmap v102: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-03-08T23:57:48.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:48 smithi192 ceph-mon[35995]: pgmap v102: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-03-08T23:57:49.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 283868 13.24 MB/sec warmup 117 sec latency 10.886 ms 2024-03-08T23:57:50.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 285476 13.19 MB/sec warmup 118 sec latency 111.116 ms 2024-03-08T23:57:50.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:50 smithi022 ceph-mon[31894]: pgmap v103: 129 pgs: 129 active+clean; 50 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 364 op/s 2024-03-08T23:57:50.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:50 smithi062 ceph-mon[36129]: pgmap v103: 129 pgs: 129 active+clean; 50 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 364 op/s 2024-03-08T23:57:50.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:50 smithi192 ceph-mon[35995]: pgmap v103: 129 pgs: 129 active+clean; 50 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 364 op/s 2024-03-08T23:57:51.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 287897 13.19 MB/sec warmup 119 sec latency 21.597 ms 2024-03-08T23:57:52.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:52 smithi022 ceph-mon[31894]: pgmap v104: 129 pgs: 129 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 488 op/s 2024-03-08T23:57:52.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:52 smithi062 ceph-mon[36129]: pgmap v104: 129 pgs: 129 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 488 op/s 2024-03-08T23:57:52.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:52 smithi192 ceph-mon[35995]: pgmap v104: 129 pgs: 129 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 488 op/s 2024-03-08T23:57:53.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 293283 8.52 MB/sec execute 1 sec latency 10.606 ms 2024-03-08T23:57:54.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 295298 10.59 MB/sec execute 2 sec latency 13.125 ms 2024-03-08T23:57:54.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:54 smithi022 ceph-mon[31894]: pgmap v105: 129 pgs: 129 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2024-03-08T23:57:54.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:54 smithi062 ceph-mon[36129]: pgmap v105: 129 pgs: 129 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2024-03-08T23:57:54.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:54 smithi192 ceph-mon[35995]: pgmap v105: 129 pgs: 129 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2024-03-08T23:57:55.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 298046 12.74 MB/sec execute 3 sec latency 74.311 ms 2024-03-08T23:57:56.682 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:56 smithi192 ceph-mon[35995]: pgmap v106: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 560 op/s 2024-03-08T23:57:56.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 299665 11.24 MB/sec execute 4 sec latency 72.629 ms 2024-03-08T23:57:56.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:56 smithi022 ceph-mon[31894]: pgmap v106: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 560 op/s 2024-03-08T23:57:56.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:56 smithi062 ceph-mon[36129]: pgmap v106: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 560 op/s 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 -- 172.21.15.22:0/3236331428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100440 msgr2=0x7fd58c1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 --2- 172.21.15.22:0/3236331428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100440 0x7fd58c1008c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009900 tx=0x7fd57c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 -- 172.21.15.22:0/3236331428 shutdown_connections 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 --2- 172.21.15.22:0/3236331428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c100e00 0x7fd58c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 --2- 172.21.15.22:0/3236331428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100440 0x7fd58c1008c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 --2- 172.21.15.22:0/3236331428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd58c105190 0x7fd58c105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 -- 172.21.15.22:0/3236331428 >> 172.21.15.22:0/3236331428 conn(0x7fd58c077040 msgr2=0x7fd58c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 -- 172.21.15.22:0/3236331428 shutdown_connections 2024-03-08T23:57:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.418+0000 7fd5929cd640 1 -- 172.21.15.22:0/3236331428 wait complete. 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 Processor -- start 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 -- start start 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd58c100440 0x7fd58c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100e00 0x7fd58c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c105190 0x7fd58c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c110630 con 0x7fd58c100e00 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1104b0 con 0x7fd58c100440 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd5929cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1107b0 con 0x7fd58c105190 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd58b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100e00 0x7fd58c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd58b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100e00 0x7fd58c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48216/0 (socket says 172.21.15.22:48216) 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd58b7fe640 1 -- 172.21.15.22:0/2875500796 learned_addr learned my addr 172.21.15.22:0/2875500796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd58bfff640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd58c100440 0x7fd58c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.419+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c105190 0x7fd58c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd58b7fe640 1 -- 172.21.15.22:0/2875500796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c105190 msgr2=0x7fd58c1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c105190 0x7fd58c1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd58b7fe640 1 -- 172.21.15.22:0/2875500796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd58c100440 msgr2=0x7fd58c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd58c100440 0x7fd58c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd58b7fe640 1 -- 172.21.15.22:0/2875500796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c197130 con 0x7fd58c100e00 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100e00 0x7fd58c19ccf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd57c005940 tx=0x7fd57c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd5897fa640 1 -- 172.21.15.22:0/2875500796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c041070 con 0x7fd58c100e00 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd5897fa640 1 -- 172.21.15.22:0/2875500796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd57c02ccc0 con 0x7fd58c100e00 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c1973c0 con 0x7fd58c100e00 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd5897fa640 1 -- 172.21.15.22:0/2875500796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c003d80 con 0x7fd58c100e00 2024-03-08T23:57:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.420+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c1a3670 con 0x7fd58c100e00 2024-03-08T23:57:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.423+0000 7fd5955ac740 1 Processor -- start 2024-03-08T23:57:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.423+0000 7fd5955ac740 1 -- start start 2024-03-08T23:57:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251fd40 0x55fce251ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce251f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd5955ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce24835f0 con 0x55fce251fd40 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd5955ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce242c930 con 0x55fce2528550 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd5955ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce247f690 con 0x55fce251efd0 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251fd40 0x55fce251ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251fd40 0x55fce251ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48224/0 (socket says 172.21.15.22:48224) 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 -- 172.21.15.22:0/2927072898 learned_addr learned my addr 172.21.15.22:0/2927072898 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd58bfff640 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce251f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 -- 172.21.15.22:0/2927072898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 msgr2=0x55fce251f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce251f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 -- 172.21.15.22:0/2927072898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 msgr2=0x55fce252a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 -- 172.21.15.22:0/2927072898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce2198210 con 0x55fce251fd40 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd58bfff640 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce251f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.424+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251fd40 0x55fce251ea90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd5840039f0 tx=0x7fd584003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1860c616940c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5727fc640 1 -- 172.21.15.22:0/2927072898 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58401a470 con 0x55fce251fd40 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5727fc640 1 -- 172.21.15.22:0/2927072898 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd584015070 con 0x55fce251fd40 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5727fc640 1 -- 172.21.15.22:0/2927072898 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584010860 con 0x55fce251fd40 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5955ac740 1 -- 172.21.15.22:0/2927072898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251fd40 msgr2=0x55fce251ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251fd40 0x55fce251ea90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd5840039f0 tx=0x7fd584003eb0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5955ac740 1 -- 172.21.15.22:0/2927072898 shutdown_connections 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce251f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2927072898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251fd40 0x55fce251ea90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.425+0000 7fd5955ac740 1 -- 172.21.15.22:0/2927072898 >> 172.21.15.22:0/2927072898 conn(0x55fce24e65b0 msgr2=0x55fce2519cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5897fa640 1 -- 172.21.15.22:0/2875500796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd57c044070 con 0x7fd58c100e00 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5955ac740 1 -- 172.21.15.22:0/2927072898 shutdown_connections 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5955ac740 1 -- 172.21.15.22:0/2927072898 wait complete. 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5955ac740 1 -- start start 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5955ac740 1 Processor -- start 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5897fa640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd5740a2920 0x7fd5740a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2525640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.426+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce25260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd5955ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce252b280 con 0x55fce251efd0 2024-03-08T23:57:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd5955ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce243fd90 con 0x55fce2528550 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd5955ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce252b400 con 0x55fce251fd40 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd5955ac740 10 client.0 _ll_register_callbacks cb 0x55fce2197a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd5740a2920 0x7fd5740a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2525640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd5897fa640 1 -- 172.21.15.22:0/2875500796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd57c0f30c0 con 0x7fd58c100e00 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2525640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48232/0 (socket says 172.21.15.22:48232) 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58bfff640 1 -- 172.21.15.22:0/1391585937 learned_addr learned my addr 172.21.15.22:0/1391585937 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:57368/0 (socket says 172.21.15.22:57368) 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd590f6a640 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce25260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 -- 172.21.15.22:0/1391585937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 msgr2=0x55fce25260c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce25260c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 -- 172.21.15.22:0/1391585937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 msgr2=0x55fce2525640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2525640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 -- 172.21.15.22:0/1391585937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce2264620 con 0x55fce251fd40 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58bfff640 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2525640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd5740a2920 0x7fd5740a4de0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd584011ef0 tx=0x7fd584011f50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0f0580 tx=0x7fd57c0bf010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba2d33a4f28128db server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd56b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd56b7fe640 1 -- 172.21.15.22:0/1391585937 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c04b040 con 0x55fce251fd40 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd584015070 con 0x55fce251fd40 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.427+0000 7fd56b7fe640 1 -- 172.21.15.22:0/1391585937 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd57c0bbb40 con 0x55fce251fd40 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd56b7fe640 1 -- 172.21.15.22:0/1391585937 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57c090d20 con 0x55fce251fd40 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 10 client.26260 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fce252b280 con 0x55fce251fd40 2024-03-08T23:57:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd56b7fe640 1 -- 172.21.15.22:0/1391585937 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd584015070 con 0x55fce251fd40 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd56b7fe640 1 -- 172.21.15.22:0/1391585937 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd57c0abf30 con 0x55fce251fd40 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 10 client.26260 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 10 client.26260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 20 client.26260 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce2538a00 0x55fce253ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 4 client.26260 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fce2490b30 con 0x55fce2538a00 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.428+0000 7fd58bfff640 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce2538a00 0x55fce253ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.429+0000 7fd58bfff640 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce2538a00 0x55fce253ae20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cec4a0fe6704bc server_cookie=de2e1b3f5148d33f in_seq=0 out_seq=0 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.429+0000 7fd56b7fe640 10 client.26260 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd56b7fe640 1 -- 172.21.15.22:0/1391585937 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fce2490b30 con 0x55fce2538a00 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd56b7fe640 10 client.26260 handle_command_reply: tid=0 2024-03-08T23:57:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 client.26260 shutdown 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 msgr2=0x55fce2525b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0f0580 tx=0x7fd57c0bf010 comp rx=0 tx=0).stop 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 shutdown_connections 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce2538a00 0x55fce253ae20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce25260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525b80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd56b7fe640 0 client.26260 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 --2- 172.21.15.22:0/1391585937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2525640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 >> 172.21.15.22:0/1391585937 conn(0x55fce24e65b0 msgr2=0x55fce251ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 shutdown_connections 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 1 -- 172.21.15.22:0/1391585937 wait complete. 2024-03-08T23:57:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.430+0000 7fd5955ac740 20 client.26260 trim_cache size 0 max 16384 2024-03-08T23:57:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 Processor -- start 2024-03-08T23:57:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 -- start start 2024-03-08T23:57:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2527d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1ae7d0 con 0x55fce251efd0 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1ac370 con 0x55fce2528550 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd5955ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1aa410 con 0x55fce251fd40 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd58bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2527d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.443+0000 7fd58bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2527d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48236/0 (socket says 172.21.15.22:48236) 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58bfff640 1 -- 172.21.15.22:0/3769543076 learned_addr learned my addr 172.21.15.22:0/3769543076 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd590f6a640 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58bfff640 1 -- 172.21.15.22:0/3769543076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 msgr2=0x55fce2525ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58bfff640 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58bfff640 1 -- 172.21.15.22:0/3769543076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 msgr2=0x55fce252f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58bfff640 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58bfff640 1 -- 172.21.15.22:0/3769543076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce2515590 con 0x55fce251efd0 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd58bfff640 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2527d70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd580000c00 tx=0x7fd580005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad6dcb0d50c1400 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd571ffb640 1 -- 172.21.15.22:0/3769543076 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580023070 con 0x55fce251efd0 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd571ffb640 1 -- 172.21.15.22:0/3769543076 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd58001f750 con 0x55fce251efd0 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.444+0000 7fd571ffb640 1 -- 172.21.15.22:0/3769543076 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580010e30 con 0x55fce251efd0 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 -- 172.21.15.22:0/3769543076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 msgr2=0x55fce2527d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2527d70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd580000c00 tx=0x7fd580005f90 comp rx=0 tx=0).stop 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 -- 172.21.15.22:0/3769543076 shutdown_connections 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce2528550 0x55fce252f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251fd40 0x55fce2525ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 --2- 172.21.15.22:0/3769543076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce251efd0 0x55fce2527d70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 -- 172.21.15.22:0/3769543076 >> 172.21.15.22:0/3769543076 conn(0x55fce24e65b0 msgr2=0x55fce2523f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 -- 172.21.15.22:0/3769543076 shutdown_connections 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 -- 172.21.15.22:0/3769543076 wait complete. 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 -- start start 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.445+0000 7fd5955ac740 1 Processor -- start 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce252fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce251fd40 0x55fce2522120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce2528550 0x55fce2522860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1aaa10 con 0x55fce2528550 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1abd10 con 0x55fce251fd40 2024-03-08T23:57:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1ab050 con 0x55fce251efd0 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 10 client.0 _ll_register_callbacks cb 0x7fd584009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce2528550 0x55fce2522860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd58b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce251fd40 0x55fce2522120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce2528550 0x55fce2522860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48246/0 (socket says 172.21.15.22:48246) 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 -- 172.21.15.22:0/2485450500 learned_addr learned my addr 172.21.15.22:0/2485450500 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd58b7fe640 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce251fd40 0x55fce2522120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:38576/0 (socket says 172.21.15.22:38576) 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd58bfff640 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce252fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 -- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 msgr2=0x55fce252fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce252fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 -- 172.21.15.22:0/2485450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce251fd40 msgr2=0x55fce2522120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce251fd40 0x55fce2522120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 -- 172.21.15.22:0/2485450500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce258eec0 con 0x55fce2528550 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd590f6a640 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce2528550 0x55fce2522860 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd5840039f0 tx=0x7fd5840095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e73a5e81b55669a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:57:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd56b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd56b7fe640 1 -- 172.21.15.22:0/2485450500 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584009b80 con 0x55fce2528550 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd56b7fe640 1 -- 172.21.15.22:0/2485450500 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd584022040 con 0x55fce2528550 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.446+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd58001f750 con 0x55fce2528550 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd56b7fe640 1 -- 172.21.15.22:0/2485450500 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd584004d40 con 0x55fce2528550 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 10 client.16806 fetch_fsmap learned FSMap version 29 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fce2590010 con 0x55fce2528550 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd56b7fe640 1 -- 172.21.15.22:0/2485450500 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd58001f750 con 0x55fce2528550 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd56b7fe640 1 -- 172.21.15.22:0/2485450500 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd584010900 con 0x55fce2528550 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 10 client.16806 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 20 client.16806 populate_metadata read hostname 'smithi022' 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce25904c0 0x55fce25928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:57:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 4 client.16806 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:57:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fce248ae80 con 0x55fce25904c0 2024-03-08T23:57:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.447+0000 7fd58bfff640 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce25904c0 0x55fce25928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:57:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.448+0000 7fd58bfff640 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce25904c0 0x55fce25928e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910a529bb543d3c4 server_cookie=25400975895a85f9 in_seq=0 out_seq=0 2024-03-08T23:57:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.448+0000 7fd56b7fe640 10 client.16806 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd56b7fe640 1 -- 172.21.15.22:0/2485450500 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fce248ae80 con 0x55fce25904c0 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd56b7fe640 10 client.16806 handle_command_reply: tid=0 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 client.16806 shutdown 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce2528550 msgr2=0x55fce2522860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce2528550 0x55fce2522860 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd5840039f0 tx=0x7fd5840095f0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 shutdown_connections 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fce25904c0 0x55fce25928e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd56b7fe640 0 client.16806 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fce2528550 0x55fce2522860 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce251fd40 0x55fce2522120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 --2- 172.21.15.22:0/2485450500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fce251efd0 0x55fce252fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.703+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 >> 172.21.15.22:0/2485450500 conn(0x55fce24e65b0 msgr2=0x55fce25243a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.704+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 shutdown_connections 2024-03-08T23:57:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.704+0000 7fd5955ac740 1 -- 172.21.15.22:0/2485450500 wait complete. 2024-03-08T23:57:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.704+0000 7fd5955ac740 20 client.16806 trim_cache size 0 max 16384 2024-03-08T23:57:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:57:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:57:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:57:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.704+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd5740a2920 msgr2=0x7fd5740a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.704+0000 7fd5929cd640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd5740a2920 0x7fd5740a4de0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd584011ef0 tx=0x7fd584011f50 comp rx=0 tx=0).stop 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.704+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100e00 msgr2=0x7fd58c19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.704+0000 7fd5929cd640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100e00 0x7fd58c19ccf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd57c005940 tx=0x7fd57c0042f0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 shutdown_connections 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd5740a2920 0x7fd5740a4de0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd58c105190 0x7fd58c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd58c100e00 0x7fd58c19ccf0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 --2- 172.21.15.22:0/2875500796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd58c100440 0x7fd58c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:57:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 >> 172.21.15.22:0/2875500796 conn(0x7fd58c077040 msgr2=0x7fd58c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:57:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 shutdown_connections 2024-03-08T23:57:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:57:57.705+0000 7fd5929cd640 1 -- 172.21.15.22:0/2875500796 wait complete. 2024-03-08T23:57:57.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 301764 11.70 MB/sec execute 5 sec latency 166.841 ms 2024-03-08T23:57:58.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 303922 10.97 MB/sec execute 6 sec latency 25.217 ms 2024-03-08T23:57:58.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:57:58 smithi022 ceph-mon[31894]: pgmap v107: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 421 op/s 2024-03-08T23:57:58.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:57:58 smithi062 ceph-mon[36129]: pgmap v107: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 421 op/s 2024-03-08T23:57:58.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:57:58 smithi192 ceph-mon[35995]: pgmap v107: 129 pgs: 129 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 421 op/s 2024-03-08T23:57:59.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 305871 11.21 MB/sec execute 7 sec latency 9.278 ms 2024-03-08T23:58:00.709 INFO:tasks.workunit.client.0.smithi022.stdout: 1 308755 11.97 MB/sec execute 8 sec latency 132.767 ms 2024-03-08T23:58:00.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:00 smithi022 ceph-mon[31894]: pgmap v108: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 590 op/s 2024-03-08T23:58:00.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:00 smithi062 ceph-mon[36129]: pgmap v108: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 590 op/s 2024-03-08T23:58:00.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:00 smithi192 ceph-mon[35995]: pgmap v108: 129 pgs: 129 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 590 op/s 2024-03-08T23:58:01.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 308755 10.64 MB/sec execute 9 sec latency 1132.834 ms 2024-03-08T23:58:02.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 308755 9.58 MB/sec execute 10 sec latency 2132.936 ms 2024-03-08T23:58:02.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:02 smithi022 ceph-mon[31894]: pgmap v109: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 614 op/s 2024-03-08T23:58:02.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:02 smithi062 ceph-mon[36129]: pgmap v109: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 614 op/s 2024-03-08T23:58:02.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:02 smithi192 ceph-mon[35995]: pgmap v109: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 614 op/s 2024-03-08T23:58:03.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 308755 8.71 MB/sec execute 11 sec latency 3133.047 ms 2024-03-08T23:58:04.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 308781 7.99 MB/sec execute 12 sec latency 4038.769 ms 2024-03-08T23:58:04.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:04 smithi022 ceph-mon[31894]: pgmap v110: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 435 op/s 2024-03-08T23:58:04.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:04 smithi062 ceph-mon[36129]: pgmap v110: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 435 op/s 2024-03-08T23:58:04.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:04 smithi192 ceph-mon[35995]: pgmap v110: 129 pgs: 129 active+clean; 45 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 435 op/s 2024-03-08T23:58:05.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 309075 7.46 MB/sec execute 13 sec latency 109.269 ms 2024-03-08T23:58:06.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 311528 7.83 MB/sec execute 14 sec latency 111.970 ms 2024-03-08T23:58:06.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:06 smithi022 ceph-mon[31894]: pgmap v111: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 482 op/s 2024-03-08T23:58:06.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:06 smithi062 ceph-mon[36129]: pgmap v111: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 482 op/s 2024-03-08T23:58:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:06 smithi192 ceph-mon[35995]: pgmap v111: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 482 op/s 2024-03-08T23:58:07.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 313215 7.87 MB/sec execute 15 sec latency 21.330 ms 2024-03-08T23:58:08.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 316119 8.46 MB/sec execute 16 sec latency 14.062 ms 2024-03-08T23:58:08.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:08 smithi022 ceph-mon[31894]: pgmap v112: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-03-08T23:58:08.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:08 smithi062 ceph-mon[36129]: pgmap v112: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-03-08T23:58:08.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:08 smithi192 ceph-mon[35995]: pgmap v112: 129 pgs: 129 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-03-08T23:58:09.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 318890 8.74 MB/sec execute 17 sec latency 13.909 ms 2024-03-08T23:58:10.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 320804 8.86 MB/sec execute 18 sec latency 18.164 ms 2024-03-08T23:58:10.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:10 smithi022 ceph-mon[31894]: pgmap v113: 129 pgs: 129 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 402 op/s 2024-03-08T23:58:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 -- 172.21.15.22:0/1770039168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 msgr2=0x7f74e8068970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 --2- 172.21.15.22:0/1770039168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e8068970 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009ca0 tx=0x7f74d802c180 comp rx=0 tx=0).stop 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 -- 172.21.15.22:0/1770039168 shutdown_connections 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 --2- 172.21.15.22:0/1770039168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74e810db40 0x7f74e810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 --2- 172.21.15.22:0/1770039168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74e8068eb0 0x7f74e80692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 --2- 172.21.15.22:0/1770039168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e8068970 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 -- 172.21.15.22:0/1770039168 >> 172.21.15.22:0/1770039168 conn(0x7f74e8077040 msgr2=0x7f74e8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 -- 172.21.15.22:0/1770039168 shutdown_connections 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.800+0000 7f74edf7e640 1 -- 172.21.15.22:0/1770039168 wait complete. 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.801+0000 7f74edf7e640 1 Processor -- start 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.801+0000 7f74edf7e640 1 -- start start 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.801+0000 7f74edf7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74e8068eb0 0x7f74e8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.801+0000 7f74edf7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74edf7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74e810db40 0x7f74e819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74edf7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81138c0 con 0x7f74e81021d0 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74edf7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8113740 con 0x7f74e8068eb0 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74edf7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8113a40 con 0x7f74e810db40 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52388/0 (socket says 172.21.15.22:52388) 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 -- 172.21.15.22:0/2745376007 learned_addr learned my addr 172.21.15.22:0/2745376007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e77fe640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74e8068eb0 0x7f74e8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74e810db40 0x7f74e819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 -- 172.21.15.22:0/2745376007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74e810db40 msgr2=0x7f74e819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74e810db40 0x7f74e819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 -- 172.21.15.22:0/2745376007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74e8068eb0 msgr2=0x7f74e8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74e8068eb0 0x7f74e8199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 -- 172.21.15.22:0/2745376007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e819ee70 con 0x7f74e81021d0 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e6ffd640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e819a3c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00e9f0 tx=0x7f74dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e4ff9640 1 -- 172.21.15.22:0/2745376007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc00cd80 con 0x7f74e81021d0 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e4ff9640 1 -- 172.21.15.22:0/2745376007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74dc005500 con 0x7f74e81021d0 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e819f160 con 0x7f74e81021d0 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74e4ff9640 1 -- 172.21.15.22:0/2745376007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc010690 con 0x7f74e81021d0 2024-03-08T23:58:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.802+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e81a69f0 con 0x7f74e81021d0 2024-03-08T23:58:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74e4ff9640 1 -- 172.21.15.22:0/2745376007 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f74dc0046c0 con 0x7f74e81021d0 2024-03-08T23:58:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 Processor -- start 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 -- start start 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62dd40 0x5589eb62ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62cfd0 0x5589eb62d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74e4ff9640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f74d00a2a10 0x7f74d00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb638940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589eb5915f0 con 0x5589eb636550 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589eb53a930 con 0x5589eb62dd40 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74f0b65740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589eb58d690 con 0x5589eb62cfd0 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb638940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74e6ffd640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f74d00a2a10 0x7f74d00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74e77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62cfd0 0x5589eb62d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb638940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52394/0 (socket says 172.21.15.22:52394) 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.805+0000 7f74e7fff640 1 -- 172.21.15.22:0/2130893940 learned_addr learned my addr 172.21.15.22:0/2130893940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e4ff9640 1 -- 172.21.15.22:0/2745376007 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f74dc0cf5f0 con 0x7f74e81021d0 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62dd40 0x5589eb62ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e7fff640 1 -- 172.21.15.22:0/2130893940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62cfd0 msgr2=0x5589eb62d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62cfd0 0x5589eb62d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e7fff640 1 -- 172.21.15.22:0/2130893940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62dd40 msgr2=0x5589eb62ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62dd40 0x5589eb62ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e7fff640 1 -- 172.21.15.22:0/2130893940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589eb2a6210 con 0x5589eb636550 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e77fe640 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62cfd0 0x5589eb62d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb638940 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009f20 tx=0x7f74e0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2cc4344d595c4b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74e6ffd640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f74d00a2a10 0x7f74d00a4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0ccfa0 tx=0x7f74dc086040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74cdffb640 1 -- 172.21.15.22:0/2130893940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0004050 con 0x5589eb636550 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74cdffb640 1 -- 172.21.15.22:0/2130893940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e00041f0 con 0x5589eb636550 2024-03-08T23:58:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.806+0000 7f74cdffb640 1 -- 172.21.15.22:0/2130893940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000f6b0 con 0x5589eb636550 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 -- 172.21.15.22:0/2130893940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 msgr2=0x5589eb638940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb638940 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009f20 tx=0x7f74e0005850 comp rx=0 tx=0).stop 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 -- 172.21.15.22:0/2130893940 shutdown_connections 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb638940 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62cfd0 0x5589eb62d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2130893940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62dd40 0x5589eb62ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 -- 172.21.15.22:0/2130893940 >> 172.21.15.22:0/2130893940 conn(0x5589eb5f45b0 msgr2=0x5589eb627cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 -- 172.21.15.22:0/2130893940 shutdown_connections 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 -- 172.21.15.22:0/2130893940 wait complete. 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.807+0000 7f74f0b65740 1 -- start start 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 1 Processor -- start 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb6335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62dd40 0x5589eb633b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb636550 0x5589eb634060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589eb639280 con 0x5589eb62dd40 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589eb54dd90 con 0x5589eb62cfd0 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589eb639400 con 0x5589eb636550 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74f0b65740 10 client.0 _ll_register_callbacks cb 0x5589eb2a5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62dd40 0x5589eb633b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62dd40 0x5589eb633b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52408/0 (socket says 172.21.15.22:52408) 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 -- 172.21.15.22:0/1210132524 learned_addr learned my addr 172.21.15.22:0/1210132524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e7fff640 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb636550 0x5589eb634060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e7fff640 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb6335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 -- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb636550 msgr2=0x5589eb634060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb636550 0x5589eb634060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 -- 172.21.15.22:0/1210132524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 msgr2=0x5589eb6335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb6335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 -- 172.21.15.22:0/1210132524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589eb372620 con 0x5589eb62dd40 2024-03-08T23:58:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74e77fe640 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62dd40 0x5589eb633b20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f74d8004af0 tx=0x7f74d8032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c291277086790c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.808+0000 7f74c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74c6ffd640 1 -- 172.21.15.22:0/1210132524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8044070 con 0x5589eb62dd40 2024-03-08T23:58:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74c6ffd640 1 -- 172.21.15.22:0/1210132524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f74d8003d80 con 0x5589eb62dd40 2024-03-08T23:58:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74e00041f0 con 0x5589eb62dd40 2024-03-08T23:58:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74c6ffd640 1 -- 172.21.15.22:0/1210132524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d8050050 con 0x5589eb62dd40 2024-03-08T23:58:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 10 client.16818 fetch_fsmap learned FSMap version 29 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5589eb6455c0 con 0x5589eb62dd40 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74c6ffd640 1 -- 172.21.15.22:0/1210132524 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e00041f0 con 0x5589eb62dd40 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74c6ffd640 1 -- 172.21.15.22:0/1210132524 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74d8032a00 con 0x5589eb62dd40 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 20 client.16818 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb646a30 0x5589eb648e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 4 client.16818 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589eb59eb30 con 0x5589eb646a30 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.809+0000 7f74e7fff640 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb646a30 0x5589eb648e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.810+0000 7f74e7fff640 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb646a30 0x5589eb648e20 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c371891097f8693c server_cookie=e97e9e5f49a8261a in_seq=0 out_seq=0 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.810+0000 7f74c6ffd640 10 client.16818 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74c6ffd640 1 -- 172.21.15.22:0/1210132524 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589eb59eb30 con 0x5589eb646a30 2024-03-08T23:58:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74c6ffd640 10 client.16818 handle_command_reply: tid=0 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 client.16818 shutdown 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62dd40 msgr2=0x5589eb633b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62dd40 0x5589eb633b20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f74d8004af0 tx=0x7f74d8032340 comp rx=0 tx=0).stop 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 shutdown_connections 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb646a30 0x5589eb648e20 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb636550 0x5589eb634060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62dd40 0x5589eb633b20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 --2- 172.21.15.22:0/1210132524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb6335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74c6ffd640 0 client.16818 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 >> 172.21.15.22:0/1210132524 conn(0x5589eb5f45b0 msgr2=0x5589eb629a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 shutdown_connections 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 1 -- 172.21.15.22:0/1210132524 wait complete. 2024-03-08T23:58:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.812+0000 7f74f0b65740 20 client.16818 trim_cache size 0 max 16384 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 Processor -- start 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 -- start start 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb635d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 0x5589eb63d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb63dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81b2ca0 con 0x5589eb636550 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81b0840 con 0x5589eb62cfd0 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.827+0000 7f74f0b65740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81ae8e0 con 0x5589eb62dd40 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb63dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb63dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52422/0 (socket says 172.21.15.22:52422) 2024-03-08T23:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 -- 172.21.15.22:0/2536883273 learned_addr learned my addr 172.21.15.22:0/2536883273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb635d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 -- 172.21.15.22:0/2536883273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 msgr2=0x5589eb63d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 0x5589eb63d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 -- 172.21.15.22:0/2536883273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 msgr2=0x5589eb635d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb635d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.828+0000 7f74e7fff640 1 -- 172.21.15.22:0/2536883273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589eb64e030 con 0x5589eb636550 2024-03-08T23:58:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.829+0000 7f74e7fff640 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb63dbb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009ef0 tx=0x7f74e0003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abbb52947f6f792 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.829+0000 7f74cd7fa640 1 -- 172.21.15.22:0/2536883273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0035070 con 0x5589eb636550 2024-03-08T23:58:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.829+0000 7f74cd7fa640 1 -- 172.21.15.22:0/2536883273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e00041f0 con 0x5589eb636550 2024-03-08T23:58:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.829+0000 7f74cd7fa640 1 -- 172.21.15.22:0/2536883273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0003850 con 0x5589eb636550 2024-03-08T23:58:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 -- 172.21.15.22:0/2536883273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 msgr2=0x5589eb63dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb63dbb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009ef0 tx=0x7f74e0003e40 comp rx=0 tx=0).stop 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 -- 172.21.15.22:0/2536883273 shutdown_connections 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb636550 0x5589eb63dbb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 0x5589eb63d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 --2- 172.21.15.22:0/2536883273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb62cfd0 0x5589eb635d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 -- 172.21.15.22:0/2536883273 >> 172.21.15.22:0/2536883273 conn(0x5589eb5f45b0 msgr2=0x5589eb6277f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 -- 172.21.15.22:0/2536883273 shutdown_connections 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.831+0000 7f74f0b65740 1 -- 172.21.15.22:0/2536883273 wait complete. 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.832+0000 7f74f0b65740 1 -- start start 2024-03-08T23:58:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.832+0000 7f74f0b65740 1 Processor -- start 2024-03-08T23:58:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.832+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62cfd0 0x5589eb630460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.832+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 0x5589eb6309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.832+0000 7f74f0b65740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb636550 0x5589eb630ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.832+0000 7f74f0b65740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81aeee0 con 0x5589eb62cfd0 2024-03-08T23:58:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.832+0000 7f74f0b65740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81b01e0 con 0x5589eb636550 2024-03-08T23:58:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.833+0000 7f74f0b65740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81af520 con 0x5589eb62dd40 2024-03-08T23:58:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.833+0000 7f74f0b65740 10 client.0 _ll_register_callbacks cb 0x7f74e00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.833+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62cfd0 0x5589eb630460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.833+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62cfd0 0x5589eb630460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52434/0 (socket says 172.21.15.22:52434) 2024-03-08T23:58:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.833+0000 7f74e7fff640 1 -- 172.21.15.22:0/3434984868 learned_addr learned my addr 172.21.15.22:0/3434984868 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.833+0000 7f74e7fff640 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb636550 0x5589eb630ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74e77fe640 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 0x5589eb6309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74e7fff640 1 -- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 msgr2=0x5589eb6309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74e7fff640 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 0x5589eb6309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74e7fff640 1 -- 172.21.15.22:0/3434984868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb636550 msgr2=0x5589eb630ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74e7fff640 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb636550 0x5589eb630ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74e7fff640 1 -- 172.21.15.22:0/3434984868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589eb6a5900 con 0x5589eb62cfd0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74e7fff640 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62cfd0 0x5589eb630460 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f74e0011de0 tx=0x7f74e002f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6a07ea64b5b338 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74c6ffd640 1 -- 172.21.15.22:0/3434984868 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0040060 con 0x5589eb62cfd0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74c6ffd640 1 -- 172.21.15.22:0/3434984868 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f74e00035f0 con 0x5589eb62cfd0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.834+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74e00041f0 con 0x5589eb62cfd0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.835+0000 7f74c6ffd640 1 -- 172.21.15.22:0/3434984868 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74e0047050 con 0x5589eb62cfd0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.835+0000 7f74f0b65740 10 client.16824 fetch_fsmap learned FSMap version 29 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.835+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5589eb6a6bf0 con 0x5589eb62cfd0 2024-03-08T23:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.835+0000 7f74c6ffd640 1 -- 172.21.15.22:0/3434984868 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e003b420 con 0x5589eb62cfd0 2024-03-08T23:58:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.836+0000 7f74c6ffd640 1 -- 172.21.15.22:0/3434984868 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 29) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74e002f040 con 0x5589eb62cfd0 2024-03-08T23:58:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.836+0000 7f74f0b65740 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2024-03-08T23:58:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.836+0000 7f74f0b65740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.836+0000 7f74f0b65740 20 client.16824 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.836+0000 7f74f0b65740 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb6a7090 0x5589eb6a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.836+0000 7f74f0b65740 4 client.16824 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:58:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.836+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f74e00041f0 con 0x5589eb6a7090 2024-03-08T23:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.837+0000 7f74e77fe640 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb6a7090 0x5589eb6a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.838+0000 7f74e77fe640 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb6a7090 0x5589eb6a94b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b140611f2e54438 server_cookie=59054e4ef29c71bf in_seq=0 out_seq=0 2024-03-08T23:58:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:10.838+0000 7f74c6ffd640 10 client.16824 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:10.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:10 smithi062 ceph-mon[36129]: pgmap v113: 129 pgs: 129 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 402 op/s 2024-03-08T23:58:10.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:10 smithi192 ceph-mon[35995]: pgmap v113: 129 pgs: 129 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 402 op/s 2024-03-08T23:58:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74c6ffd640 1 -- 172.21.15.22:0/3434984868 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f74e00041f0 con 0x5589eb6a7090 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74c6ffd640 10 client.16824 handle_command_reply: tid=0 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 client.16824 shutdown 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62cfd0 msgr2=0x5589eb630460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62cfd0 0x5589eb630460 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f74e0011de0 tx=0x7f74e002f810 comp rx=0 tx=0).stop 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 shutdown_connections 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589eb6a7090 0x5589eb6a94b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589eb636550 0x5589eb630ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589eb62dd40 0x5589eb6309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 --2- 172.21.15.22:0/3434984868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589eb62cfd0 0x5589eb630460 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74c6ffd640 0 client.16824 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 >> 172.21.15.22:0/3434984868 conn(0x5589eb5f45b0 msgr2=0x5589eb631e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 shutdown_connections 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.116+0000 7f74f0b65740 1 -- 172.21.15.22:0/3434984868 wait complete. 2024-03-08T23:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74f0b65740 20 client.16824 trim_cache size 0 max 16384 2024-03-08T23:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f74d00a2a10 msgr2=0x7f74d00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f74d00a2a10 0x7f74d00a4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0ccfa0 tx=0x7f74dc086040 comp rx=0 tx=0).stop 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 msgr2=0x7f74e819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e819a3c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00e9f0 tx=0x7f74dc00edf0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 shutdown_connections 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f74d00a2a10 0x7f74d00a4ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74e810db40 0x7f74e819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74e81021d0 0x7f74e819a3c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 --2- 172.21.15.22:0/2745376007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74e8068eb0 0x7f74e8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 >> 172.21.15.22:0/2745376007 conn(0x7f74e8077040 msgr2=0x7f74e80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 shutdown_connections 2024-03-08T23:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:11.117+0000 7f74edf7e640 1 -- 172.21.15.22:0/2745376007 wait complete. 2024-03-08T23:58:11.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:58:11.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 -- 172.21.15.22:0/1204818590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668103e70 msgr2=0x7f1668110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 --2- 172.21.15.22:0/1204818590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668103e70 0x7f1668110330 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1660009ca0 tx=0x7f166002c0f0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 -- 172.21.15.22:0/1204818590 shutdown_connections 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 --2- 172.21.15.22:0/1204818590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668103e70 0x7f1668110330 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 --2- 172.21.15.22:0/1204818590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16681034b0 0x7f1668103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 --2- 172.21.15.22:0/1204818590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1668109570 0x7f1668109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 -- 172.21.15.22:0/1204818590 >> 172.21.15.22:0/1204818590 conn(0x7f16680fef60 msgr2=0x7f1668101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 -- 172.21.15.22:0/1204818590 shutdown_connections 2024-03-08T23:58:11.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.625+0000 7f166ee29640 1 -- 172.21.15.22:0/1204818590 wait complete. 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 Processor -- start 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 -- start start 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16681034b0 0x7f166819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1668103e70 0x7f166819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668109570 0x7f1668199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1668113830 con 0x7f1668109570 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681136b0 con 0x7f1668103e70 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166ee29640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681139b0 con 0x7f16681034b0 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668109570 0x7f1668199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166cbc5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16681034b0 0x7f166819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668109570 0x7f1668199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52458/0 (socket says 172.21.15.22:52458) 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 -- 172.21.15.22:0/2802554490 learned_addr learned my addr 172.21.15.22:0/2802554490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f1667fff640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1668103e70 0x7f166819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 -- 172.21.15.22:0/2802554490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16681034b0 msgr2=0x7f166819f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16681034b0 0x7f166819f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 -- 172.21.15.22:0/2802554490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1668103e70 msgr2=0x7f166819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1668103e70 0x7f166819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 -- 172.21.15.22:0/2802554490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166819a250 con 0x7f1668109570 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f1667fff640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1668103e70 0x7f166819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.626+0000 7f166d3c6640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668109570 0x7f1668199ad0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1660037040 tx=0x7f1660032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.627+0000 7f1665ffb640 1 -- 172.21.15.22:0/2802554490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660042020 con 0x7f1668109570 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.627+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166819a4e0 con 0x7f1668109570 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.627+0000 7f1665ffb640 1 -- 172.21.15.22:0/2802554490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1660034640 con 0x7f1668109570 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.627+0000 7f1665ffb640 1 -- 172.21.15.22:0/2802554490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660039040 con 0x7f1668109570 2024-03-08T23:58:11.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.627+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16681a6790 con 0x7f1668109570 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.629+0000 7f1665ffb640 1 -- 172.21.15.22:0/2802554490 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1660045020 con 0x7f1668109570 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 Processor -- start 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 -- start start 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec0760 0x5585d3ec0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec9c60 0x5585d3ecc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d3e23e00 con 0x5585d3ec9c60 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d3dccb10 con 0x5585d3ec1080 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1665ffb640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16580a2970 0x7f16580a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1671a05740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d3e1fea0 con 0x5585d3ec0760 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1667fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec0760 0x5585d3ec0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f166d3c6640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16580a2970 0x7f16580a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1667fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec0760 0x5585d3ec0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:56494/0 (socket says 172.21.15.22:56494) 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1667fff640 1 -- 172.21.15.22:0/996681422 learned_addr learned my addr 172.21.15.22:0/996681422 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1667fff640 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec9c60 0x5585d3ecc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f1665ffb640 1 -- 172.21.15.22:0/2802554490 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f16600f10b0 con 0x7f1668109570 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.630+0000 7f166cbc5640 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1667fff640 1 -- 172.21.15.22:0/996681422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 msgr2=0x5585d3ec9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1667fff640 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1667fff640 1 -- 172.21.15.22:0/996681422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec9c60 msgr2=0x5585d3ecc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1667fff640 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec9c60 0x5585d3ecc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1667fff640 1 -- 172.21.15.22:0/996681422 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585d3c027d0 con 0x5585d3ec0760 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f166d3c6640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16580a2970 0x7f16580a4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16600b98c0 tx=0x7f16600885c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1667fff640 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec0760 0x5585d3ec0b40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1654014900 tx=0x7f1654009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e564a080cd43124 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1652ffd640 1 -- 172.21.15.22:0/996681422 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16540142d0 con 0x5585d3ec0760 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1652ffd640 1 -- 172.21.15.22:0/996681422 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f165400f760 con 0x5585d3ec0760 2024-03-08T23:58:11.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.631+0000 7f1652ffd640 1 -- 172.21.15.22:0/996681422 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654003850 con 0x5585d3ec0760 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 -- 172.21.15.22:0/996681422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec0760 msgr2=0x5585d3ec0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec0760 0x5585d3ec0b40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1654014900 tx=0x7f1654009630 comp rx=0 tx=0).stop 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 -- 172.21.15.22:0/996681422 shutdown_connections 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec9c60 0x5585d3ecc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 --2- 172.21.15.22:0/996681422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec0760 0x5585d3ec0b40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 -- 172.21.15.22:0/996681422 >> 172.21.15.22:0/996681422 conn(0x5585d3ebae90 msgr2=0x5585d3ebb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 -- 172.21.15.22:0/996681422 shutdown_connections 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 -- 172.21.15.22:0/996681422 wait complete. 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 -- start start 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.632+0000 7f1671a05740 1 Processor -- start 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1671a05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec0760 0x5585d3ec9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1671a05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1671a05740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1671a05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d3ecc9f0 con 0x5585d3ec0760 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1671a05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d3de00a0 con 0x5585d3ec1080 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1671a05740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585d3eccb70 con 0x5585d3ec9c60 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1671a05740 10 client.0 _ll_register_callbacks cb 0x5585d3ddfa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1667fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec0760 0x5585d3ec9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f166cbc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1667fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec0760 0x5585d3ec9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52478/0 (socket says 172.21.15.22:52478) 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1667fff640 1 -- 172.21.15.22:0/3617340300 learned_addr learned my addr 172.21.15.22:0/3617340300 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1667fff640 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f166cbc5640 1 -- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 msgr2=0x5585d3ec6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f166cbc5640 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f166cbc5640 1 -- 172.21.15.22:0/3617340300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec0760 msgr2=0x5585d3ec9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f166cbc5640 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec0760 0x5585d3ec9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f166cbc5640 1 -- 172.21.15.22:0/3617340300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585d3b383f0 con 0x5585d3ec1080 2024-03-08T23:58:11.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f166cbc5640 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec6690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1668108680 tx=0x7f165c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523551b40c3a14f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.633+0000 7f1647fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1647fff640 1 -- 172.21.15.22:0/3617340300 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c00f8c0 con 0x5585d3ec1080 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1647fff640 1 -- 172.21.15.22:0/3617340300 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f165c00fa60 con 0x5585d3ec1080 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f165400f760 con 0x5585d3ec1080 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1647fff640 1 -- 172.21.15.22:0/3617340300 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f165c00fc60 con 0x5585d3ec1080 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 10 client.26297 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5585d3af80f0 con 0x5585d3ec1080 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1647fff640 1 -- 172.21.15.22:0/3617340300 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f165400f760 con 0x5585d3ec1080 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1647fff640 1 -- 172.21.15.22:0/3617340300 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f165c0104c0 con 0x5585d3ec1080 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 10 client.26297 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 20 client.26297 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3ed9610 0x5585d3edba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 4 client.26297 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:11.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585d3e31340 con 0x5585d3ed9610 2024-03-08T23:58:11.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.634+0000 7f1667fff640 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3ed9610 0x5585d3edba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.635+0000 7f1667fff640 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3ed9610 0x5585d3edba40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4320a009197db42 server_cookie=6548872f6425d0b8 in_seq=0 out_seq=0 2024-03-08T23:58:11.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.635+0000 7f1647fff640 10 client.26297 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1647fff640 1 -- 172.21.15.22:0/3617340300 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585d3e31340 con 0x5585d3ed9610 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1647fff640 10 client.26297 handle_command_reply: tid=0 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 client.26297 shutdown 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 msgr2=0x5585d3ec6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec6690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1668108680 tx=0x7f165c012b60 comp rx=0 tx=0).stop 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 shutdown_connections 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3ed9610 0x5585d3edba40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1647fff640 0 client.26297 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:11.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec1080 0x5585d3ec6690 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 --2- 172.21.15.22:0/3617340300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec0760 0x5585d3ec9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.636+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 >> 172.21.15.22:0/3617340300 conn(0x5585d3ebae90 msgr2=0x5585d3ebcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.637+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 shutdown_connections 2024-03-08T23:58:11.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.637+0000 7f1671a05740 1 -- 172.21.15.22:0/3617340300 wait complete. 2024-03-08T23:58:11.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.637+0000 7f1671a05740 20 client.26297 trim_cache size 0 max 16384 2024-03-08T23:58:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.652+0000 7f1671a05740 1 Processor -- start 2024-03-08T23:58:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.652+0000 7f1671a05740 1 -- start start 2024-03-08T23:58:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1671a05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1671a05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1671a05740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ecb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1671a05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681b2ba0 con 0x5585d3ec1080 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1671a05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681b0740 con 0x5585d3ec0760 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1671a05740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681ae7e0 con 0x5585d3ec9c60 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52488/0 (socket says 172.21.15.22:52488) 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 -- 172.21.15.22:0/3824438591 learned_addr learned my addr 172.21.15.22:0/3824438591 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1667fff640 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f1667fff640 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ecb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 -- 172.21.15.22:0/3824438591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 msgr2=0x5585d3ecb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ecb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 -- 172.21.15.22:0/3824438591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 msgr2=0x5585d3ec8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 -- 172.21.15.22:0/3824438591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585d3ed5e30 con 0x5585d3ec1080 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f166cbc5640 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec9300 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f165c0096c0 tx=0x7f165c0096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580e3128b746225 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f16527fc640 1 -- 172.21.15.22:0/3824438591 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c00fe00 con 0x5585d3ec1080 2024-03-08T23:58:11.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f16527fc640 1 -- 172.21.15.22:0/3824438591 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f165c010430 con 0x5585d3ec1080 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.653+0000 7f16527fc640 1 -- 172.21.15.22:0/3824438591 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c005300 con 0x5585d3ec1080 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 -- 172.21.15.22:0/3824438591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 msgr2=0x5585d3ec9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec9300 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f165c0096c0 tx=0x7f165c0096f0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 -- 172.21.15.22:0/3824438591 shutdown_connections 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ecb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec9300 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 --2- 172.21.15.22:0/3824438591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 -- 172.21.15.22:0/3824438591 >> 172.21.15.22:0/3824438591 conn(0x5585d3ebae90 msgr2=0x5585d3ec4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 -- 172.21.15.22:0/3824438591 shutdown_connections 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 -- 172.21.15.22:0/3824438591 wait complete. 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.654+0000 7f1671a05740 1 -- start start 2024-03-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 1 Processor -- start 2024-03-08T23:58:11.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681aede0 con 0x5585d3ec1080 2024-03-08T23:58:11.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681b00e0 con 0x5585d3ec0760 2024-03-08T23:58:11.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681af420 con 0x5585d3ec9c60 2024-03-08T23:58:11.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1671a05740 10 client.0 _ll_register_callbacks cb 0x7f165c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52496/0 (socket says 172.21.15.22:52496) 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 -- 172.21.15.22:0/550750659 learned_addr learned my addr 172.21.15.22:0/550750659 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1667fff640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1667fff640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 -- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 msgr2=0x5585d3ec3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 -- 172.21.15.22:0/550750659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 msgr2=0x5585d3ec3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f166cbc5640 1 -- 172.21.15.22:0/550750659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585d3f2fb90 con 0x5585d3ec1080 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.655+0000 7f1667fff640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f166cbc5640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec38c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f165c0108b0 tx=0x7f165c0109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398696e88024ac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1647fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1647fff640 1 -- 172.21.15.22:0/550750659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c010ad0 con 0x5585d3ec1080 2024-03-08T23:58:11.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1647fff640 1 -- 172.21.15.22:0/550750659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f165c010c70 con 0x5585d3ec1080 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f165c010430 con 0x5585d3ec1080 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1647fff640 1 -- 172.21.15.22:0/550750659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f165c01c470 con 0x5585d3ec1080 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 10 client.16848 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5585d3f30e80 con 0x5585d3ec1080 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1647fff640 1 -- 172.21.15.22:0/550750659 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f165c010430 con 0x5585d3ec1080 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1647fff640 1 -- 172.21.15.22:0/550750659 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f165c029690 con 0x5585d3ec1080 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 10 client.16848 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 20 client.16848 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3f31440 0x5585d3f33830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 4 client.16848 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:58:11.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.656+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5585d3e2b690 con 0x5585d3f31440 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1667fff640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3f31440 0x5585d3f33830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1647fff640 10 client.16848 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1667fff640 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3f31440 0x5585d3f33830 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dbe85059725ec7e server_cookie=d63a16cb2677e3e9 in_seq=0 out_seq=0 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1647fff640 1 -- 172.21.15.22:0/550750659 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5585d3e2b690 con 0x5585d3f31440 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1647fff640 10 client.16848 handle_command_reply: tid=0 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 client.16848 shutdown 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 msgr2=0x5585d3ec38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec38c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f165c0108b0 tx=0x7f165c0109b0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 shutdown_connections 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5585d3f31440 0x5585d3f33830 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5585d3ec9c60 0x5585d3ec3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5585d3ec1080 0x5585d3ec38c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1647fff640 0 client.16848 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 --2- 172.21.15.22:0/550750659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5585d3ec0760 0x5585d3ec3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.657+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 >> 172.21.15.22:0/550750659 conn(0x5585d3ebae90 msgr2=0x5585d3ec1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 shutdown_connections 2024-03-08T23:58:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f1671a05740 1 -- 172.21.15.22:0/550750659 wait complete. 2024-03-08T23:58:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f1671a05740 20 client.16848 trim_cache size 0 max 16384 2024-03-08T23:58:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:58:11.660 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:58:11.660 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:58:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16580a2970 msgr2=0x7f16580a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16580a2970 0x7f16580a4e30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16600b98c0 tx=0x7f16600885c0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668109570 msgr2=0x7f1668199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668109570 0x7f1668199ad0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1660037040 tx=0x7f1660032ea0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 shutdown_connections 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16580a2970 0x7f16580a4e30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1668109570 0x7f1668199ad0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1668103e70 0x7f166819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 --2- 172.21.15.22:0/2802554490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16681034b0 0x7f166819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 >> 172.21.15.22:0/2802554490 conn(0x7f16680fef60 msgr2=0x7f16681007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 shutdown_connections 2024-03-08T23:58:11.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.658+0000 7f166ee29640 1 -- 172.21.15.22:0/2802554490 wait complete. 2024-03-08T23:58:11.664 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:58:11.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:58:11.688 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:11 smithi022 ceph-mon[31894]: daemon mds.h restarted 2024-03-08T23:58:11.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 323061 9.12 MB/sec execute 19 sec latency 88.138 ms 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 -- 172.21.15.22:0/3971868259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 msgr2=0x7f12981002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3971868259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f12981002d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1288009ca0 tx=0x7f128802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 -- 172.21.15.22:0/3971868259 shutdown_connections 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3971868259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12981066b0 0x7f129810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3971868259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12981008a0 0x7f1298105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3971868259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f12981002d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 -- 172.21.15.22:0/3971868259 >> 172.21.15.22:0/3971868259 conn(0x7f1298075500 msgr2=0x7f1298075910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 -- 172.21.15.22:0/3971868259 shutdown_connections 2024-03-08T23:58:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 -- 172.21.15.22:0/3971868259 wait complete. 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.776+0000 7f129ddc4640 1 Processor -- start 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f129ddc4640 1 -- start start 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f129ddc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f1298199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f129ddc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12981066b0 0x7f129819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f129ddc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129819e6d0 0x7f129819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f129ddc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298111780 con 0x7f12980ffef0 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f129ddc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298111600 con 0x7f129819e6d0 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f129ddc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298111900 con 0x7f12981066b0 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f1298199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f1298199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52510/0 (socket says 172.21.15.22:52510) 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 -- 172.21.15.22:0/3249310493 learned_addr learned my addr 172.21.15.22:0/3249310493 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f1296ffd640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12981066b0 0x7f129819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f1297fff640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129819e6d0 0x7f129819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 -- 172.21.15.22:0/3249310493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12981066b0 msgr2=0x7f129819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12981066b0 0x7f129819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 -- 172.21.15.22:0/3249310493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129819e6d0 msgr2=0x7f129819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129819e6d0 0x7f129819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 -- 172.21.15.22:0/3249310493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129819f1c0 con 0x7f12980ffef0 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.777+0000 7f12977fe640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f1298199e10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1288005e30 tx=0x7f1288032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.778+0000 7f1294ff9640 1 -- 172.21.15.22:0/3249310493 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288039040 con 0x7f12980ffef0 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.778+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129819f450 con 0x7f12980ffef0 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.778+0000 7f1294ff9640 1 -- 172.21.15.22:0/3249310493 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1288044070 con 0x7f12980ffef0 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.778+0000 7f1294ff9640 1 -- 172.21.15.22:0/3249310493 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880042a0 con 0x7f12980ffef0 2024-03-08T23:58:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.778+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12981a6c70 con 0x7f12980ffef0 2024-03-08T23:58:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.779+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1298069420 con 0x7f12980ffef0 2024-03-08T23:58:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.780+0000 7f1294ff9640 1 -- 172.21.15.22:0/3249310493 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1288004440 con 0x7f12980ffef0 2024-03-08T23:58:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.781+0000 7f1294ff9640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12800a2970 0x7f12800a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.781+0000 7f1296ffd640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12800a2970 0x7f12800a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.781+0000 7f1294ff9640 1 -- 172.21.15.22:0/3249310493 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f12880f1e50 con 0x7f12980ffef0 2024-03-08T23:58:11.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.781+0000 7f1296ffd640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12800a2970 0x7f12800a4e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f12980ffb10 tx=0x7f128c00b410 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:11.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:11.785+0000 7f1294ff9640 1 -- 172.21.15.22:0/3249310493 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f12880be920 con 0x7f12980ffef0 2024-03-08T23:58:11.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:11 smithi062 ceph-mon[36129]: daemon mds.h restarted 2024-03-08T23:58:11.861 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:11 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setuser ceph since I am not root 2024-03-08T23:58:11.861 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:11 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setgroup ceph since I am not root 2024-03-08T23:58:11.861 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:11 smithi062 ceph-mds[70095]: -- 172.21.15.62:0/717717799 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b750f53a40 con 0x55b750f26c00 2024-03-08T23:58:11.861 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:11 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: starting mds.h at 2024-03-08T23:58:11.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:11 smithi192 ceph-mon[35995]: daemon mds.h restarted 2024-03-08T23:58:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.111+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f129819b190 con 0x7f12980ffef0 2024-03-08T23:58:12.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.112+0000 7f1294ff9640 1 -- 172.21.15.22:0/3249310493 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f129819b190 con 0x7f12980ffef0 2024-03-08T23:58:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:58:12.108 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:11.445096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26288":{"gid":26288,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2009262827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2009262827},{"type":"v1","addr":"172.21.15.62:6837","nonce":2009262827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:58:12.108 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 31 2024-03-08T23:58:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12800a2970 msgr2=0x7f12800a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12800a2970 0x7f12800a4e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f12980ffb10 tx=0x7f128c00b410 comp rx=0 tx=0).stop 2024-03-08T23:58:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 msgr2=0x7f1298199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f1298199e10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1288005e30 tx=0x7f1288032f20 comp rx=0 tx=0).stop 2024-03-08T23:58:12.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 shutdown_connections 2024-03-08T23:58:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f12800a2970 0x7f12800a4e30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f129819e6d0 0x7f129819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12981066b0 0x7f129819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 --2- 172.21.15.22:0/3249310493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12980ffef0 0x7f1298199e10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 >> 172.21.15.22:0/3249310493 conn(0x7f1298075500 msgr2=0x7f12980fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 shutdown_connections 2024-03-08T23:58:12.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.114+0000 7f129ddc4640 1 -- 172.21.15.22:0/3249310493 wait complete. 2024-03-08T23:58:12.121 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:58:12.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:58:12.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.280+0000 7f081b4fc640 1 -- 172.21.15.22:0/3152351565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141003e0 msgr2=0x7f08141007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.280+0000 7f081b4fc640 1 --2- 172.21.15.22:0/3152351565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141003e0 0x7f08141007c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0810009900 tx=0x7f081002bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:12.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.280+0000 7f081b4fc640 1 -- 172.21.15.22:0/3152351565 shutdown_connections 2024-03-08T23:58:12.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.280+0000 7f081b4fc640 1 --2- 172.21.15.22:0/3152351565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08141054b0 0x7f081410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.280+0000 7f081b4fc640 1 --2- 172.21.15.22:0/3152351565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0814100d90 0x7f0814104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.280+0000 7f081b4fc640 1 --2- 172.21.15.22:0/3152351565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141003e0 0x7f08141007c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 -- 172.21.15.22:0/3152351565 >> 172.21.15.22:0/3152351565 conn(0x7f08140773a0 msgr2=0x7f08140777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 -- 172.21.15.22:0/3152351565 shutdown_connections 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 -- 172.21.15.22:0/3152351565 wait complete. 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 Processor -- start 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 -- start start 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08141003e0 0x7f081419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141054b0 0x7f081419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0814197970 0x7f0814197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814110590 con 0x7f08141054b0 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814110410 con 0x7f08141003e0 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.281+0000 7f081b4fc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814110710 con 0x7f0814197970 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141054b0 0x7f081419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141054b0 0x7f081419dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52532/0 (socket says 172.21.15.22:52532) 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 -- 172.21.15.22:0/4029921361 learned_addr learned my addr 172.21.15.22:0/4029921361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0819298640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08141003e0 0x7f081419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0819a99640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0814197970 0x7f0814197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 -- 172.21.15.22:0/4029921361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0814197970 msgr2=0x7f0814197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0814197970 0x7f0814197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 -- 172.21.15.22:0/4029921361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08141003e0 msgr2=0x7f081419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08141003e0 0x7f081419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 -- 172.21.15.22:0/4029921361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814198490 con 0x7f08141054b0 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f0818a97640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141054b0 0x7f081419dd00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f080400d740 tx=0x7f080400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f080a7fc640 1 -- 172.21.15.22:0/4029921361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804017070 con 0x7f08141054b0 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0814198780 con 0x7f08141054b0 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f080a7fc640 1 -- 172.21.15.22:0/4029921361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f080400de50 con 0x7f08141054b0 2024-03-08T23:58:12.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f080a7fc640 1 -- 172.21.15.22:0/4029921361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804004e00 con 0x7f08141054b0 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.282+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08141a4980 con 0x7f08141054b0 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f080a7fc640 1 -- 172.21.15.22:0/4029921361 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0804004fa0 con 0x7f08141054b0 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 Processor -- start 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 -- start start 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a01a0 0x556a568a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a0ac0 0x556a568a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568aba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56867360 con 0x556a568a0ac0 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56803ec0 con 0x556a568a01a0 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f081e1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a567acd30 con 0x556a568a96a0 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.285+0000 7f080a7fc640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f08000a2970 0x7f08000a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a0ac0 0x556a568a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0818a97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a0ac0 0x556a568a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52542/0 (socket says 172.21.15.22:52542) 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 -- 172.21.15.22:0/2193896019 learned_addr learned my addr 172.21.15.22:0/2193896019 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819a99640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f08000a2970 0x7f08000a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f080a7fc640 1 -- 172.21.15.22:0/4029921361 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f08040ceb00 con 0x7f08141054b0 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819a99640 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a01a0 0x556a568a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 -- 172.21.15.22:0/2193896019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 msgr2=0x556a568aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 -- 172.21.15.22:0/2193896019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a01a0 msgr2=0x556a568a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a01a0 0x556a568a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 -- 172.21.15.22:0/2193896019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a567abbc0 con 0x556a568a0ac0 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819a99640 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a01a0 0x556a568a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819298640 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a0ac0 0x556a568a9160 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f081002c170 tx=0x7f0810009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da93105c6ffb4d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f0819a99640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f08000a2970 0x7f08000a4e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f080c005fd0 tx=0x7f080c009b60 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f07ff7fe640 1 -- 172.21.15.22:0/2193896019 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810042070 con 0x556a568a0ac0 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f07ff7fe640 1 -- 172.21.15.22:0/2193896019 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f081002ca80 con 0x556a568a0ac0 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.286+0000 7f07ff7fe640 1 -- 172.21.15.22:0/2193896019 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810004ec0 con 0x556a568a0ac0 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 -- 172.21.15.22:0/2193896019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a0ac0 msgr2=0x556a568a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a0ac0 0x556a568a9160 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f081002c170 tx=0x7f0810009610 comp rx=0 tx=0).stop 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 -- 172.21.15.22:0/2193896019 shutdown_connections 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a0ac0 0x556a568a9160 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2193896019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a01a0 0x556a568a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 -- 172.21.15.22:0/2193896019 >> 172.21.15.22:0/2193896019 conn(0x556a5689a8d0 msgr2=0x556a5689acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 -- 172.21.15.22:0/2193896019 shutdown_connections 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 -- 172.21.15.22:0/2193896019 wait complete. 2024-03-08T23:58:12.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.287+0000 7f081e1b4740 1 -- start start 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 1 Processor -- start 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a01a0 0x556a568a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a96a0 0x556a568a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a567a24a0 con 0x556a568a96a0 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a567fff60 con 0x556a568a0ac0 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a568ac430 con 0x556a568a01a0 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f081e1b4740 10 client.0 _ll_register_callbacks cb 0x556a567fdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819a99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a96a0 0x556a568a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0818a97640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a01a0 0x556a568a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a01a0 0x556a568a6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:56576/0 (socket says 172.21.15.22:56576) 2024-03-08T23:58:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 -- 172.21.15.22:0/3742725807 learned_addr learned my addr 172.21.15.22:0/3742725807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 -- 172.21.15.22:0/3742725807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 msgr2=0x556a568a6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568a6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 -- 172.21.15.22:0/3742725807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a96a0 msgr2=0x556a568a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a96a0 0x556a568a71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.288+0000 7f0819298640 1 -- 172.21.15.22:0/3742725807 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a565183f0 con 0x556a568a01a0 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f0819298640 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a01a0 0x556a568a6740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0810000c00 tx=0x7f0810009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2200529c00febe server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f07fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f07fcff9640 1 -- 172.21.15.22:0/3742725807 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081004a070 con 0x556a568a01a0 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f081002ca80 con 0x556a568a01a0 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f07fcff9640 1 -- 172.21.15.22:0/3742725807 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f08100050d0 con 0x556a568a01a0 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f07fcff9640 1 -- 172.21.15.22:0/3742725807 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0810056050 con 0x556a568a01a0 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f081e1b4740 10 client.26323 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.289+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556a565e27d0 con 0x556a568a01a0 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f07fcff9640 1 -- 172.21.15.22:0/3742725807 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f081002ca80 con 0x556a568a01a0 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f07fcff9640 1 -- 172.21.15.22:0/3742725807 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f081003e400 con 0x556a568a01a0 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f081e1b4740 10 client.26323 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f081e1b4740 10 client.26323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f081e1b4740 20 client.26323 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f081e1b4740 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a568bbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f081e1b4740 4 client.26323 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a56811400 con 0x556a568b9730 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.290+0000 7f0818a97640 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a568bbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.291+0000 7f0818a97640 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a568bbb20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9c3943ee76876a9 server_cookie=fa4c59baec0073ef in_seq=0 out_seq=0 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.291+0000 7f07fcff9640 10 client.26323 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f07fcff9640 1 -- 172.21.15.22:0/3742725807 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a56811400 con 0x556a568b9730 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f07fcff9640 10 client.26323 handle_command_reply: tid=0 2024-03-08T23:58:12.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 client.26323 shutdown 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a01a0 msgr2=0x556a568a6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a01a0 0x556a568a6740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0810000c00 tx=0x7f0810009680 comp rx=0 tx=0).stop 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 shutdown_connections 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a568bbb20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a96a0 0x556a568a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f07fcff9640 0 client.26323 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568a6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 --2- 172.21.15.22:0/3742725807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a01a0 0x556a568a6740 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 >> 172.21.15.22:0/3742725807 conn(0x556a5689a8d0 msgr2=0x556a5689c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 shutdown_connections 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 1 -- 172.21.15.22:0/3742725807 wait complete. 2024-03-08T23:58:12.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.292+0000 7f081e1b4740 20 client.26323 trim_cache size 0 max 16384 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 Processor -- start 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 -- start start 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a56899810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141ae710 con 0x556a568a01a0 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141ac2b0 con 0x556a568a0ac0 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f081e1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141aa350 con 0x556a568a96a0 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f0819298640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f0819298640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52558/0 (socket says 172.21.15.22:52558) 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f0819298640 1 -- 172.21.15.22:0/2188467906 learned_addr learned my addr 172.21.15.22:0/2188467906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f0818a97640 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a56899810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.306+0000 7f0819a99640 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f0819298640 1 -- 172.21.15.22:0/2188467906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 msgr2=0x556a568a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f0819298640 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f0819298640 1 -- 172.21.15.22:0/2188467906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 msgr2=0x556a56899810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f0819298640 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a56899810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f0819298640 1 -- 172.21.15.22:0/2188467906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a568a08f0 con 0x556a568a01a0 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f0818a97640 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a56899810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f0819298640 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568a8e70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0810009900 tx=0x7f0810004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f79adb7dd12c7b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f07feffd640 1 -- 172.21.15.22:0/2188467906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081004a040 con 0x556a568a01a0 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f07feffd640 1 -- 172.21.15.22:0/2188467906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f081002c5d0 con 0x556a568a01a0 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.307+0000 7f07feffd640 1 -- 172.21.15.22:0/2188467906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810042040 con 0x556a568a01a0 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 -- 172.21.15.22:0/2188467906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 msgr2=0x556a568a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568a8e70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0810009900 tx=0x7f0810004e00 comp rx=0 tx=0).stop 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 -- 172.21.15.22:0/2188467906 shutdown_connections 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568a39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a56899810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 --2- 172.21.15.22:0/2188467906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568a8e70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 -- 172.21.15.22:0/2188467906 >> 172.21.15.22:0/2188467906 conn(0x556a5689a8d0 msgr2=0x556a56899100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 -- 172.21.15.22:0/2188467906 shutdown_connections 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 -- 172.21.15.22:0/2188467906 wait complete. 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 -- start start 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.308+0000 7f081e1b4740 1 Processor -- start 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f081e1b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f081e1b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141aa950 con 0x556a568a01a0 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f081e1b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141abc50 con 0x556a568a0ac0 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f081e1b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141aaf90 con 0x556a568a96a0 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0819298640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f081e1b4740 10 client.0 _ll_register_callbacks cb 0x7f0810009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0819298640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52568/0 (socket says 172.21.15.22:52568) 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0818a97640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0819298640 1 -- 172.21.15.22:0/1488179890 learned_addr learned my addr 172.21.15.22:0/1488179890 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0819a99640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0818a97640 1 -- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 msgr2=0x556a568afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0818a97640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0818a97640 1 -- 172.21.15.22:0/1488179890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 msgr2=0x556a568af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0818a97640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0818a97640 1 -- 172.21.15.22:0/1488179890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a5690f9a0 con 0x556a568a0ac0 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0819a99640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0819298640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f0818a97640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568af5b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f08040cbeb0 tx=0x7f08040048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54030532d489f684 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f07fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.309+0000 7f07fcff9640 1 -- 172.21.15.22:0/1488179890 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804020040 con 0x556a568a0ac0 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f07fcff9640 1 -- 172.21.15.22:0/1488179890 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f08040cc130 con 0x556a568a0ac0 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f081002c5d0 con 0x556a568a0ac0 2024-03-08T23:58:12.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f07fcff9640 1 -- 172.21.15.22:0/1488179890 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08040974e0 con 0x556a568a0ac0 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 10 client.26324 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556a56910d30 con 0x556a568a0ac0 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f07fcff9640 1 -- 172.21.15.22:0/1488179890 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f081002c5d0 con 0x556a568a0ac0 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f07fcff9640 1 -- 172.21.15.22:0/1488179890 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f080409a2c0 con 0x556a568a0ac0 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 10 client.26324 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 10 client.26324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 20 client.26324 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a56913450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 4 client.26324 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.310+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a5680b750 con 0x556a568b9730 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f0819298640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a56913450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f07fcff9640 10 client.26324 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f0819298640 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a56913450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7443d831d33fd3f5 server_cookie=8cc158a120405d76 in_seq=0 out_seq=0 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f07fcff9640 1 -- 172.21.15.22:0/1488179890 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a5680b750 con 0x556a568b9730 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f07fcff9640 10 client.26324 handle_command_reply: tid=0 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 client.26324 shutdown 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 msgr2=0x556a568af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568af5b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f08040cbeb0 tx=0x7f08040048a0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 shutdown_connections 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x556a568b9730 0x556a56913450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a568a96a0 0x556a568afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a568a0ac0 0x556a568af5b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 --2- 172.21.15.22:0/1488179890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a568a01a0 0x556a568af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f07fcff9640 0 client.26324 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.311+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 >> 172.21.15.22:0/1488179890 conn(0x556a5689a8d0 msgr2=0x556a568a5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 shutdown_connections 2024-03-08T23:58:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081e1b4740 1 -- 172.21.15.22:0/1488179890 wait complete. 2024-03-08T23:58:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081e1b4740 20 client.26324 trim_cache size 0 max 16384 2024-03-08T23:58:12.315 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f08000a2970 msgr2=0x7f08000a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f08000a2970 0x7f08000a4e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f080c005fd0 tx=0x7f080c009b60 comp rx=0 tx=0).stop 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141054b0 msgr2=0x7f081419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141054b0 0x7f081419dd00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f080400d740 tx=0x7f080400dc00 comp rx=0 tx=0).stop 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 shutdown_connections 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f08000a2970 0x7f08000a4e30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0814197970 0x7f0814197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08141054b0 0x7f081419dd00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 --2- 172.21.15.22:0/4029921361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08141003e0 0x7f081419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 >> 172.21.15.22:0/4029921361 conn(0x7f08140773a0 msgr2=0x7f08141040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 shutdown_connections 2024-03-08T23:58:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.312+0000 7f081b4fc640 1 -- 172.21.15.22:0/4029921361 wait complete. 2024-03-08T23:58:12.319 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:58:12.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:58:12.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 -- 172.21.15.22:0/2756742260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 msgr2=0x7f91d410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 --2- 172.21.15.22:0/2756742260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d410d080 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009ca0 tx=0x7f91cc02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 -- 172.21.15.22:0/2756742260 shutdown_connections 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 --2- 172.21.15.22:0/2756742260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d410d080 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 --2- 172.21.15.22:0/2756742260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91d4100d90 0x7f91d4104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 --2- 172.21.15.22:0/2756742260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d41003e0 0x7f91d41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 -- 172.21.15.22:0/2756742260 >> 172.21.15.22:0/2756742260 conn(0x7f91d40773a0 msgr2=0x7f91d40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 -- 172.21.15.22:0/2756742260 shutdown_connections 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.436+0000 7f91daa75640 1 -- 172.21.15.22:0/2756742260 wait complete. 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 Processor -- start 2024-03-08T23:58:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 -- start start 2024-03-08T23:58:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d41003e0 0x7f91d419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91d41978b0 0x7f91d4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4110570 con 0x7f91d41054b0 2024-03-08T23:58:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41103f0 con 0x7f91d41003e0 2024-03-08T23:58:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91daa75640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41106f0 con 0x7f91d41978b0 2024-03-08T23:58:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d419dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52582/0 (socket says 172.21.15.22:52582) 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91d3fff640 1 -- 172.21.15.22:0/2715075429 learned_addr learned my addr 172.21.15.22:0/2715075429 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91d9012640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91d41978b0 0x7f91d4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.437+0000 7f91d8811640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d41003e0 0x7f91d419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.438+0000 7f91d9012640 1 -- 172.21.15.22:0/2715075429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d41003e0 msgr2=0x7f91d419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.438+0000 7f91d9012640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d41003e0 0x7f91d419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.438+0000 7f91d9012640 1 -- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 msgr2=0x7f91d419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.438+0000 7f91d9012640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.438+0000 7f91d9012640 1 -- 172.21.15.22:0/2715075429 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d41983d0 con 0x7f91d41978b0 2024-03-08T23:58:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.438+0000 7f91d3fff640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.438+0000 7f91d9012640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91d41978b0 0x7f91d4197d60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91cc02c700 tx=0x7f91cc032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.443+0000 7f91d1ffb640 1 -- 172.21.15.22:0/2715075429 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc039040 con 0x7f91d41978b0 2024-03-08T23:58:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.443+0000 7f91d1ffb640 1 -- 172.21.15.22:0/2715075429 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91cc034cd0 con 0x7f91d41978b0 2024-03-08T23:58:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.443+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d4198660 con 0x7f91d41978b0 2024-03-08T23:58:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.443+0000 7f91d1ffb640 1 -- 172.21.15.22:0/2715075429 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0042a0 con 0x7f91d41978b0 2024-03-08T23:58:12.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.444+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d4102090 con 0x7f91d41978b0 2024-03-08T23:58:12.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.445+0000 7f91d1ffb640 1 -- 172.21.15.22:0/2715075429 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f91cc032590 con 0x7f91d41978b0 2024-03-08T23:58:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.446+0000 7f91d1ffb640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f91c40a29f0 0x7f91c40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.446+0000 7f91d8811640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f91c40a29f0 0x7f91c40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.446+0000 7f91d1ffb640 1 -- 172.21.15.22:0/2715075429 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f91cc0f4dc0 con 0x7f91d41978b0 2024-03-08T23:58:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.446+0000 7f91dd64c740 1 Processor -- start 2024-03-08T23:58:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.446+0000 7f91dd64c740 1 -- start start 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be23b80 0x55cb4be23f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be24530 0x55cb4be35970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91dd64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bdeb360 con 0x55cb4be35eb0 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91dd64c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd87ec0 con 0x55cb4be23b80 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91dd64c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd30d30 con 0x55cb4be24530 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d8811640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f91c40a29f0 0x7f91c40a4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f91d41989f0 tx=0x7f91c8005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d8811640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be23b80 0x55cb4be23f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be382a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52584/0 (socket says 172.21.15.22:52584) 2024-03-08T23:58:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 -- 172.21.15.22:0/715655933 learned_addr learned my addr 172.21.15.22:0/715655933 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d8811640 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be23b80 0x55cb4be23f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45426/0 (socket says 172.21.15.22:45426) 2024-03-08T23:58:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d3fff640 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be24530 0x55cb4be35970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 -- 172.21.15.22:0/715655933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be24530 msgr2=0x55cb4be35970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be24530 0x55cb4be35970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 -- 172.21.15.22:0/715655933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be23b80 msgr2=0x55cb4be23f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be23b80 0x55cb4be23f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 -- 172.21.15.22:0/715655933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4bd2fbc0 con 0x55cb4be35eb0 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d8811640 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be23b80 0x55cb4be23f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91d9012640 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be382a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0c93a0 tx=0x7f91cc0324b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d13b2a2dc1c61c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91beffd640 1 -- 172.21.15.22:0/715655933 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc034920 con 0x55cb4be35eb0 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91beffd640 1 -- 172.21.15.22:0/715655933 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91cc0e10c0 con 0x55cb4be35eb0 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.447+0000 7f91beffd640 1 -- 172.21.15.22:0/715655933 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc003970 con 0x55cb4be35eb0 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 -- 172.21.15.22:0/715655933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 msgr2=0x55cb4be382a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be382a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0c93a0 tx=0x7f91cc0324b0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 -- 172.21.15.22:0/715655933 shutdown_connections 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be382a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be24530 0x55cb4be35970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 --2- 172.21.15.22:0/715655933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be23b80 0x55cb4be23f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 -- 172.21.15.22:0/715655933 >> 172.21.15.22:0/715655933 conn(0x55cb4be1e8d0 msgr2=0x55cb4be1ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 -- 172.21.15.22:0/715655933 shutdown_connections 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 -- 172.21.15.22:0/715655933 wait complete. 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 -- start start 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.448+0000 7f91dd64c740 1 Processor -- start 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be33410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be33950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91dd64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd264a0 con 0x55cb4be24530 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91dd64c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4bd83f60 con 0x55cb4be35eb0 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91dd64c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4be38a30 con 0x55cb4be23b80 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91dd64c740 10 client.0 _ll_register_callbacks cb 0x55cb4bd81d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:12.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be33410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d8811640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be33410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52600/0 (socket says 172.21.15.22:52600) 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d8811640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be32ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:56616/0 (socket says 172.21.15.22:56616) 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 -- 172.21.15.22:0/2270374481 learned_addr learned my addr 172.21.15.22:0/2270374481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d9012640 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be33950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 -- 172.21.15.22:0/2270374481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 msgr2=0x55cb4be32ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be32ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 -- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 msgr2=0x55cb4be33950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be33950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 -- 172.21.15.22:0/2270374481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb4ba9c3f0 con 0x55cb4be24530 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91d3fff640 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be33410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f91c0012d20 tx=0x7f91c000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84652004849fe226 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.449+0000 7f91b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91b3fff640 1 -- 172.21.15.22:0/2270374481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c00120d0 con 0x55cb4be24530 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91b3fff640 1 -- 172.21.15.22:0/2270374481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f91c00047a0 con 0x55cb4be24530 2024-03-08T23:58:12.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91cc0e10c0 con 0x55cb4be24530 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91b3fff640 1 -- 172.21.15.22:0/2270374481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91c0025050 con 0x55cb4be24530 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 10 client.16884 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cb4bb667d0 con 0x55cb4be24530 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91b3fff640 1 -- 172.21.15.22:0/2270374481 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91cc0e10c0 con 0x55cb4be24530 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91b3fff640 1 -- 172.21.15.22:0/2270374481 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f91c00105c0 con 0x55cb4be24530 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 10 client.16884 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 10 client.16884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 20 client.16884 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be45e00 0x55cb4be481f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 4 client.16884 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb4bd95400 con 0x55cb4be45e00 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.450+0000 7f91d8811640 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be45e00 0x55cb4be481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.451+0000 7f91d8811640 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be45e00 0x55cb4be481f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31de9fd35625efb server_cookie=1759f7e949de7f9a in_seq=0 out_seq=0 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.451+0000 7f91b3fff640 10 client.16884 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:12.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.451+0000 7f91b3fff640 1 -- 172.21.15.22:0/2270374481 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb4bd95400 con 0x55cb4be45e00 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.451+0000 7f91b3fff640 10 client.16884 handle_command_reply: tid=0 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.451+0000 7f91dd64c740 1 client.16884 shutdown 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 msgr2=0x55cb4be33410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be33410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f91c0012d20 tx=0x7f91c000f790 comp rx=0 tx=0).stop 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 shutdown_connections 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be45e00 0x55cb4be481f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be33950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be33410 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 --2- 172.21.15.22:0/2270374481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be32ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91b3fff640 0 client.16884 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 >> 172.21.15.22:0/2270374481 conn(0x55cb4be1e8d0 msgr2=0x55cb4be20620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 shutdown_connections 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 1 -- 172.21.15.22:0/2270374481 wait complete. 2024-03-08T23:58:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.452+0000 7f91dd64c740 20 client.16884 trim_cache size 0 max 16384 2024-03-08T23:58:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 Processor -- start 2024-03-08T23:58:12.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 -- start start 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be35600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be24530 0x55cb4be31800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be31d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41ae7d0 con 0x55cb4be35eb0 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41ac370 con 0x55cb4be24530 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91dd64c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41aa410 con 0x55cb4be23b80 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.465+0000 7f91d9012640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be31d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be24530 0x55cb4be31800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be24530 0x55cb4be31800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45442/0 (socket says 172.21.15.22:45442) 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d9012640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be31d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52602/0 (socket says 172.21.15.22:52602) 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d8811640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 -- 172.21.15.22:0/1472646386 learned_addr learned my addr 172.21.15.22:0/1472646386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 -- 172.21.15.22:0/1472646386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 msgr2=0x55cb4be35600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be35600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 -- 172.21.15.22:0/1472646386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 msgr2=0x55cb4be31d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be31d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 -- 172.21.15.22:0/1472646386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4be38610 con 0x55cb4be24530 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d9012640 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be31d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91d3fff640 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be24530 0x55cb4be31800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c0010a10 tx=0x7f91c000fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49750c48211fb6c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91be7fc640 1 -- 172.21.15.22:0/1472646386 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c002e070 con 0x55cb4be24530 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91be7fc640 1 -- 172.21.15.22:0/1472646386 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91c0010b80 con 0x55cb4be24530 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.466+0000 7f91be7fc640 1 -- 172.21.15.22:0/1472646386 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0005330 con 0x55cb4be24530 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 -- 172.21.15.22:0/1472646386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be24530 msgr2=0x55cb4be31800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be24530 0x55cb4be31800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c0010a10 tx=0x7f91c000fd10 comp rx=0 tx=0).stop 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 -- 172.21.15.22:0/1472646386 shutdown_connections 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be35eb0 0x55cb4be31d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be24530 0x55cb4be31800 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 --2- 172.21.15.22:0/1472646386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be35600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 -- 172.21.15.22:0/1472646386 >> 172.21.15.22:0/1472646386 conn(0x55cb4be1e8d0 msgr2=0x55cb4be2e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 -- 172.21.15.22:0/1472646386 shutdown_connections 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 -- 172.21.15.22:0/1472646386 wait complete. 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 -- start start 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.467+0000 7f91dd64c740 1 Processor -- start 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be3b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91dd64c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be3c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91dd64c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41aaa10 con 0x55cb4be24530 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91dd64c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41abd10 con 0x55cb4be35eb0 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91dd64c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41ab050 con 0x55cb4be23b80 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91dd64c740 10 client.0 _ll_register_callbacks cb 0x7f91cc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d9012640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be3c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d8811640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be3b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be3bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52610/0 (socket says 172.21.15.22:52610) 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 -- 172.21.15.22:0/3663747345 learned_addr learned my addr 172.21.15.22:0/3663747345 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 -- 172.21.15.22:0/3663747345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 msgr2=0x55cb4be3b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be3b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 -- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 msgr2=0x55cb4be3c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be3c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 -- 172.21.15.22:0/3663747345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb4be9c300 con 0x55cb4be24530 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d9012640 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be3c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91d3fff640 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be3bb10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f91c000a010 tx=0x7f91c0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ba2786c20b90fd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.468+0000 7f91b3fff640 1 -- 172.21.15.22:0/3663747345 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c002e070 con 0x55cb4be24530 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91b3fff640 1 -- 172.21.15.22:0/3663747345 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f91c0018040 con 0x55cb4be24530 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91c0010b80 con 0x55cb4be24530 2024-03-08T23:58:12.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91b3fff640 1 -- 172.21.15.22:0/3663747345 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91c00122b0 con 0x55cb4be24530 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 10 client.16890 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cb4be9d520 con 0x55cb4be24530 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91b3fff640 1 -- 172.21.15.22:0/3663747345 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91c0010b80 con 0x55cb4be24530 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91b3fff640 1 -- 172.21.15.22:0/3663747345 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f91c0004c80 con 0x55cb4be24530 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 10 client.16890 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 20 client.16890 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be9db30 0x55cb4be9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 4 client.16890 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb4bd8f750 con 0x55cb4be9db30 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.469+0000 7f91d8811640 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be9db30 0x55cb4be9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91d8811640 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be9db30 0x55cb4be9ff50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95884cfaf8cd167b server_cookie=9423086ebf73a095 in_seq=0 out_seq=0 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91b3fff640 10 client.16890 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:12.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91b3fff640 1 -- 172.21.15.22:0/3663747345 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb4bd8f750 con 0x55cb4be9db30 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91b3fff640 10 client.16890 handle_command_reply: tid=0 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 client.16890 shutdown 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 msgr2=0x55cb4be3bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be3bb10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f91c000a010 tx=0x7f91c0005f90 comp rx=0 tx=0).stop 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 shutdown_connections 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55cb4be9db30 0x55cb4be9ff50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4be35eb0 0x55cb4be3c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb4be24530 0x55cb4be3bb10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 --2- 172.21.15.22:0/3663747345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb4be23b80 0x55cb4be3b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91b3fff640 0 client.16890 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 >> 172.21.15.22:0/3663747345 conn(0x55cb4be1e8d0 msgr2=0x55cb4be317a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 shutdown_connections 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 1 -- 172.21.15.22:0/3663747345 wait complete. 2024-03-08T23:58:12.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.470+0000 7f91dd64c740 20 client.16890 trim_cache size 0 max 16384 2024-03-08T23:58:12.474 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f91c40a29f0 msgr2=0x7f91c40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f91c40a29f0 0x7f91c40a4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f91d41989f0 tx=0x7f91c8005f90 comp rx=0 tx=0).stop 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91d41978b0 msgr2=0x7f91d4197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91d41978b0 0x7f91d4197d60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91cc02c700 tx=0x7f91cc032f20 comp rx=0 tx=0).stop 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 shutdown_connections 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f91c40a29f0 0x7f91c40a4eb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91d41978b0 0x7f91d4197d60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d41054b0 0x7f91d419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 --2- 172.21.15.22:0/2715075429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d41003e0 0x7f91d419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 >> 172.21.15.22:0/2715075429 conn(0x7f91d40773a0 msgr2=0x7f91d40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 shutdown_connections 2024-03-08T23:58:12.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.471+0000 7f91daa75640 1 -- 172.21.15.22:0/2715075429 wait complete. 2024-03-08T23:58:12.477 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:58:12.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:58:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 -- 172.21.15.22:0/4232521379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e0006a5c0 msgr2=0x7f7e00105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 --2- 172.21.15.22:0/4232521379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e0006a5c0 0x7f7e00105ec0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009900 tx=0x7f7df002bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 -- 172.21.15.22:0/4232521379 shutdown_connections 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 --2- 172.21.15.22:0/4232521379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e001065a0 0x7f7e0010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 --2- 172.21.15.22:0/4232521379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e0006a5c0 0x7f7e00105ec0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 --2- 172.21.15.22:0/4232521379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e00069c10 0x7f7e00069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 -- 172.21.15.22:0/4232521379 >> 172.21.15.22:0/4232521379 conn(0x7f7e00076f80 msgr2=0x7f7e00077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 -- 172.21.15.22:0/4232521379 shutdown_connections 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 -- 172.21.15.22:0/4232521379 wait complete. 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 Processor -- start 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.594+0000 7f7e07668640 1 -- start start 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e07668640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e00069c10 0x7f7e00100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e07668640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e001065a0 0x7f7e00100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e07668640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e00100f90 0x7f7e001a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e07668640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00111630 con 0x7f7e00100f90 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e07668640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001114b0 con 0x7f7e001065a0 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e07668640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001117b0 con 0x7f7e00069c10 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e00100f90 0x7f7e001a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e00100f90 0x7f7e001a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52630/0 (socket says 172.21.15.22:52630) 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 -- 172.21.15.22:0/2976984211 learned_addr learned my addr 172.21.15.22:0/2976984211 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e04c03640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e001065a0 0x7f7e00100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05404640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e00069c10 0x7f7e00100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 -- 172.21.15.22:0/2976984211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e00069c10 msgr2=0x7f7e00100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e00069c10 0x7f7e00100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 -- 172.21.15.22:0/2976984211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e001065a0 msgr2=0x7f7e00100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e001065a0 0x7f7e00100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 -- 172.21.15.22:0/2976984211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e001a6480 con 0x7f7e00100f90 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e04c03640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e001065a0 0x7f7e00100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7e05c05640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e00100f90 0x7f7e001a5d80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7df800efd0 tx=0x7f7df800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.595+0000 7f7df67fc640 1 -- 172.21.15.22:0/2976984211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8003b70 con 0x7f7e00100f90 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.596+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e001a6770 con 0x7f7e00100f90 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.596+0000 7f7df67fc640 1 -- 172.21.15.22:0/2976984211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df8003d10 con 0x7f7e00100f90 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.596+0000 7f7df67fc640 1 -- 172.21.15.22:0/2976984211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80109f0 con 0x7f7e00100f90 2024-03-08T23:58:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.596+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e001a6c90 con 0x7f7e00100f90 2024-03-08T23:58:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.598+0000 7f7df67fc640 1 -- 172.21.15.22:0/2976984211 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7df8004000 con 0x7f7e00100f90 2024-03-08T23:58:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.598+0000 7f7e0a24a740 1 Processor -- start 2024-03-08T23:58:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.598+0000 7f7e0a24a740 1 -- start start 2024-03-08T23:58:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x558968387580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558968387ac0 0x558968390160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7df67fc640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7dec0a29a0 0x7f7dec0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683906a0 0x558968392a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55896834e360 con 0x5589683871a0 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589682eaec0 con 0x558968387ac0 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558968293d30 con 0x5589683906a0 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683906a0 0x558968392a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683906a0 0x558968392a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:56668/0 (socket says 172.21.15.22:56668) 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 -- 172.21.15.22:0/3674493412 learned_addr learned my addr 172.21.15.22:0/3674493412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e04c03640 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x558968387580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7df67fc640 1 -- 172.21.15.22:0/2976984211 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7df80cf590 con 0x7f7e00100f90 2024-03-08T23:58:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05404640 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558968387ac0 0x558968390160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e04c03640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7dec0a29a0 0x7f7dec0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 -- 172.21.15.22:0/3674493412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558968387ac0 msgr2=0x558968390160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558968387ac0 0x558968390160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 -- 172.21.15.22:0/3674493412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 msgr2=0x558968387580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x558968387580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 -- 172.21.15.22:0/3674493412 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558968292bc0 con 0x5589683906a0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e05c05640 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683906a0 0x558968392a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7df800cba0 tx=0x7f7df8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185d4f504a8f796a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7e04c03640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7dec0a29a0 0x7f7dec0a4e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009230 tx=0x7f7df0005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.599+0000 7f7deb7fe640 1 -- 172.21.15.22:0/3674493412 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80a0ec0 con 0x5589683906a0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7deb7fe640 1 -- 172.21.15.22:0/3674493412 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df8009820 con 0x5589683906a0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7deb7fe640 1 -- 172.21.15.22:0/3674493412 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80a25e0 con 0x5589683906a0 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 -- 172.21.15.22:0/3674493412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683906a0 msgr2=0x558968392a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683906a0 0x558968392a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7df800cba0 tx=0x7f7df8009f90 comp rx=0 tx=0).stop 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 -- 172.21.15.22:0/3674493412 shutdown_connections 2024-03-08T23:58:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683906a0 0x558968392a90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558968387ac0 0x558968390160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/3674493412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x558968387580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 -- 172.21.15.22:0/3674493412 >> 172.21.15.22:0/3674493412 conn(0x5589683818d0 msgr2=0x558968381cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 -- 172.21.15.22:0/3674493412 shutdown_connections 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.600+0000 7f7e0a24a740 1 -- 172.21.15.22:0/3674493412 wait complete. 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 -- start start 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 Processor -- start 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x55896838d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558968387ac0 0x55896838dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589682894a0 con 0x5589683871a0 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589682e6f60 con 0x5589683906a0 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558968393430 con 0x558968387ac0 2024-03-08T23:58:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e0a24a740 10 client.0 _ll_register_callbacks cb 0x5589682e4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e05404640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x55896838d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e05404640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x55896838d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52644/0 (socket says 172.21.15.22:52644) 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e05404640 1 -- 172.21.15.22:0/1485141766 learned_addr learned my addr 172.21.15.22:0/1485141766 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e04c03640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558968387ac0 0x55896838dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.601+0000 7f7e05c05640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e05404640 1 -- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558968387ac0 msgr2=0x55896838dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e05404640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558968387ac0 0x55896838dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e05404640 1 -- 172.21.15.22:0/1485141766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 msgr2=0x55896838e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e05404640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e05404640 1 -- 172.21.15.22:0/1485141766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558967fff3f0 con 0x5589683871a0 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e05c05640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e04c03640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558968387ac0 0x55896838dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e05404640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x55896838d6c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc012740 tx=0x7f7dfc012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2f143372f28eec server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7de8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7de8ff9640 1 -- 172.21.15.22:0/1485141766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00f920 con 0x5589683871a0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7de8ff9640 1 -- 172.21.15.22:0/1485141766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7dfc00fac0 con 0x5589683871a0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7df8009820 con 0x5589683871a0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7de8ff9640 1 -- 172.21.15.22:0/1485141766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dfc00fd00 con 0x5589683871a0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e0a24a740 10 client.16902 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.602+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5589680c97d0 con 0x5589683871a0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7de8ff9640 1 -- 172.21.15.22:0/1485141766 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df8009820 con 0x5589683871a0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7de8ff9640 1 -- 172.21.15.22:0/1485141766 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7dfc01c420 con 0x5589683871a0 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7e0a24a740 10 client.16902 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7e0a24a740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7e0a24a740 20 client.16902 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683a0880 0x5589683a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7e0a24a740 4 client.16902 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589682f8400 con 0x5589683a0880 2024-03-08T23:58:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.603+0000 7f7e04c03640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683a0880 0x5589683a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.604+0000 7f7e04c03640 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683a0880 0x5589683a2cc0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee2cbabfbcb58d5 server_cookie=f200fc2475566e4 in_seq=0 out_seq=0 2024-03-08T23:58:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.604+0000 7f7de8ff9640 10 client.16902 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7de8ff9640 1 -- 172.21.15.22:0/1485141766 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589682f8400 con 0x5589683a0880 2024-03-08T23:58:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7de8ff9640 10 client.16902 handle_command_reply: tid=0 2024-03-08T23:58:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 client.16902 shutdown 2024-03-08T23:58:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 msgr2=0x55896838d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x55896838d6c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc012740 tx=0x7f7dfc012c00 comp rx=0 tx=0).stop 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 shutdown_connections 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683a0880 0x5589683a2cc0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558968387ac0 0x55896838dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/1485141766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589683871a0 0x55896838d6c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7de8ff9640 0 client.16902 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.605+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 >> 172.21.15.22:0/1485141766 conn(0x5589683818d0 msgr2=0x558968383620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.606+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 shutdown_connections 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.606+0000 7f7e0a24a740 1 -- 172.21.15.22:0/1485141766 wait complete. 2024-03-08T23:58:12.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.606+0000 7f7e0a24a740 20 client.16902 trim_cache size 0 max 16384 2024-03-08T23:58:12.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.617+0000 7f7e0a24a740 1 Processor -- start 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e0a24a740 1 -- start start 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x55896838fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x55896838a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001b2c10 con 0x558968387ac0 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001b07b0 con 0x5589683906a0 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001ae850 con 0x5589683871a0 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x55896838a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x55896838a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52658/0 (socket says 172.21.15.22:52658) 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 -- 172.21.15.22:0/627220062 learned_addr learned my addr 172.21.15.22:0/627220062 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e05404640 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x55896838fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e05c05640 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 -- 172.21.15.22:0/627220062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 msgr2=0x55896838fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x55896838fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 -- 172.21.15.22:0/627220062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 msgr2=0x55896838ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 -- 172.21.15.22:0/627220062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589683878f0 con 0x558968387ac0 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7e04c03640 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x55896838a920 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7e00069770 tx=0x7f7df0005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1597686e3835133e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.618+0000 7f7deaffd640 1 -- 172.21.15.22:0/627220062 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0058030 con 0x558968387ac0 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7deaffd640 1 -- 172.21.15.22:0/627220062 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df0053e50 con 0x558968387ac0 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7deaffd640 1 -- 172.21.15.22:0/627220062 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0053580 con 0x558968387ac0 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 -- 172.21.15.22:0/627220062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 msgr2=0x55896838a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x55896838a920 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7e00069770 tx=0x7f7df0005bb0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 -- 172.21.15.22:0/627220062 shutdown_connections 2024-03-08T23:58:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x55896838ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x55896838a920 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/627220062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x55896838fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 -- 172.21.15.22:0/627220062 >> 172.21.15.22:0/627220062 conn(0x5589683818d0 msgr2=0x558968388980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 -- 172.21.15.22:0/627220062 shutdown_connections 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 -- 172.21.15.22:0/627220062 wait complete. 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.619+0000 7f7e0a24a740 1 -- start start 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 1 Processor -- start 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x5589683963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x558968396920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x5589683f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001aee50 con 0x558968387ac0 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001b0150 con 0x5589683906a0 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001af490 con 0x5589683871a0 2024-03-08T23:58:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e0a24a740 10 client.0 _ll_register_callbacks cb 0x7f7df8003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x558968396920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x558968396920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52666/0 (socket says 172.21.15.22:52666) 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e05c05640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x5589683f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e05404640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x5589683963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 -- 172.21.15.22:0/551199163 learned_addr learned my addr 172.21.15.22:0/551199163 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 -- 172.21.15.22:0/551199163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 msgr2=0x5589683963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x5589683963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 -- 172.21.15.22:0/551199163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 msgr2=0x5589683f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x5589683f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e04c03640 1 -- 172.21.15.22:0/551199163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589683971b0 con 0x558968387ac0 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.620+0000 7f7e05c05640 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x5589683f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e04c03640 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x558968396920 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7df00531a0 tx=0x7f7df002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d8e1dace4563d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7de8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:12.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7de8ff9640 1 -- 172.21.15.22:0/551199163 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0058020 con 0x558968387ac0 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7de8ff9640 1 -- 172.21.15.22:0/551199163 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7df0053850 con 0x558968387ac0 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7df0053e50 con 0x558968387ac0 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7de8ff9640 1 -- 172.21.15.22:0/551199163 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7df0004f20 con 0x558968387ac0 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 10 client.16908 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5589683f83c0 con 0x558968387ac0 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7de8ff9640 1 -- 172.21.15.22:0/551199163 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7df0053e50 con 0x558968387ac0 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7de8ff9640 1 -- 172.21.15.22:0/551199163 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7df0034440 con 0x558968387ac0 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 10 client.16908 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 20 client.16908 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683f8a10 0x5589683fae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 4 client.16908 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.621+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5589682f2750 con 0x5589683f8a10 2024-03-08T23:58:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e05404640 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683f8a10 0x5589683fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7de8ff9640 10 client.16908 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e05404640 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683f8a10 0x5589683fae30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d7b116d1fa57d1 server_cookie=8c97205bf940ca59 in_seq=0 out_seq=0 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7de8ff9640 1 -- 172.21.15.22:0/551199163 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5589682f2750 con 0x5589683f8a10 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7de8ff9640 10 client.16908 handle_command_reply: tid=0 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 client.16908 shutdown 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 msgr2=0x558968396920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x558968396920 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7df00531a0 tx=0x7f7df002c7e0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 shutdown_connections 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5589683f8a10 0x5589683fae30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589683906a0 0x5589683f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558968387ac0 0x558968396920 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7de8ff9640 0 client.16908 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 --2- 172.21.15.22:0/551199163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5589683871a0 0x5589683963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 >> 172.21.15.22:0/551199163 conn(0x5589683818d0 msgr2=0x558968380100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 shutdown_connections 2024-03-08T23:58:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 1 -- 172.21.15.22:0/551199163 wait complete. 2024-03-08T23:58:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.622+0000 7f7e0a24a740 20 client.16908 trim_cache size 0 max 16384 2024-03-08T23:58:12.626 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:58:12.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7dec0a29a0 msgr2=0x7f7dec0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7dec0a29a0 0x7f7dec0a4e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009230 tx=0x7f7df0005fd0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e00100f90 msgr2=0x7f7e001a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:12.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e00100f90 0x7f7e001a5d80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7df800efd0 tx=0x7f7df800c6a0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 shutdown_connections 2024-03-08T23:58:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7dec0a29a0 0x7f7dec0a4e60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e00100f90 0x7f7e001a5d80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e001065a0 0x7f7e00100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 --2- 172.21.15.22:0/2976984211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e00069c10 0x7f7e00100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 >> 172.21.15.22:0/2976984211 conn(0x7f7e00076f80 msgr2=0x7f7e000fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 shutdown_connections 2024-03-08T23:58:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:12.623+0000 7f7e07668640 1 -- 172.21.15.22:0/2976984211 wait complete. 2024-03-08T23:58:12.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 325243 9.03 MB/sec execute 20 sec latency 24.157 ms 2024-03-08T23:58:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:12 smithi022 ceph-mon[31894]: pgmap v114: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 346 op/s 2024-03-08T23:58:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:12 smithi022 ceph-mon[31894]: osdmap e97: 12 total, 12 up, 12 in 2024-03-08T23:58:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:12 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6836/2009262827,v1:172.21.15.62:6837/2009262827] up:boot 2024-03-08T23:58:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:12 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:58:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:12 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:58:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:12 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:58:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:12 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3249310493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:58:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:12 smithi062 ceph-mon[36129]: pgmap v114: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 346 op/s 2024-03-08T23:58:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:12 smithi062 ceph-mon[36129]: osdmap e97: 12 total, 12 up, 12 in 2024-03-08T23:58:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:12 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6836/2009262827,v1:172.21.15.62:6837/2009262827] up:boot 2024-03-08T23:58:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:12 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:58:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:12 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:58:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:12 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:58:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:12 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3249310493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:58:12.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:12 smithi192 ceph-mon[35995]: pgmap v114: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 346 op/s 2024-03-08T23:58:12.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:12 smithi192 ceph-mon[35995]: osdmap e97: 12 total, 12 up, 12 in 2024-03-08T23:58:12.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:12 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6836/2009262827,v1:172.21.15.62:6837/2009262827] up:boot 2024-03-08T23:58:12.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:12 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:58:12.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:12 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:58:12.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:12 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:58:12.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:12 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3249310493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:58:13.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:58:13.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:58:13.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:58:13.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 327282 9.21 MB/sec execute 21 sec latency 12.545 ms 2024-03-08T23:58:13.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 -- 172.21.15.22:0/4260282221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa40100d80 msgr2=0x7ffa40104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4260282221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa40100d80 0x7ffa40104dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009900 tx=0x7ffa3402bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 -- 172.21.15.22:0/4260282221 shutdown_connections 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4260282221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa401054a0 0x7ffa4010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4260282221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa40100d80 0x7ffa40104dc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4260282221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 0x7ffa401007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 -- 172.21.15.22:0/4260282221 >> 172.21.15.22:0/4260282221 conn(0x7ffa400772f0 msgr2=0x7ffa40077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 -- 172.21.15.22:0/4260282221 shutdown_connections 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.750+0000 7ffa4682b640 1 -- 172.21.15.22:0/4260282221 wait complete. 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 Processor -- start 2024-03-08T23:58:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 -- start start 2024-03-08T23:58:13.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 0x7ffa4019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa401054a0 0x7ffa4019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa401979b0 0x7ffa40197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40111670 con 0x7ffa401979b0 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401114f0 con 0x7ffa401054a0 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa4682b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401117f0 con 0x7ffa401003d0 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa401979b0 0x7ffa40197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.751+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 0x7ffa4019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 0x7ffa4019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:56718/0 (socket says 172.21.15.22:56718) 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa401979b0 0x7ffa40197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52678/0 (socket says 172.21.15.22:52678) 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 -- 172.21.15.22:0/4238404277 learned_addr learned my addr 172.21.15.22:0/4238404277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa401054a0 0x7ffa4019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 -- 172.21.15.22:0/4238404277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa401054a0 msgr2=0x7ffa4019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa401054a0 0x7ffa4019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 -- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa401979b0 msgr2=0x7ffa40197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa401979b0 0x7ffa40197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 -- 172.21.15.22:0/4238404277 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa401984d0 con 0x7ffa401003d0 2024-03-08T23:58:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 0x7ffa4019d800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e9f0 tx=0x7ffa3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3d7fa640 1 -- 172.21.15.22:0/4238404277 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000cd80 con 0x7ffa401003d0 2024-03-08T23:58:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa401987c0 con 0x7ffa401003d0 2024-03-08T23:58:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3d7fa640 1 -- 172.21.15.22:0/4238404277 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa30010910 con 0x7ffa401003d0 2024-03-08T23:58:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa3d7fa640 1 -- 172.21.15.22:0/4238404277 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300188f0 con 0x7ffa401003d0 2024-03-08T23:58:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.752+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4010dc80 con 0x7ffa401003d0 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.755+0000 7ffa3d7fa640 1 -- 172.21.15.22:0/4238404277 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ffa30010ac0 con 0x7ffa401003d0 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.755+0000 7ffa49413740 1 Processor -- start 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.755+0000 7ffa49413740 1 -- start start 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa49413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6ddfd80 0x557dc6de0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa49413740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de06a0 0x557dc6de8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa49413740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6deb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa49413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc6d43b40 con 0x557dc6ddfd80 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa49413740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc6cecd80 con 0x557dc6de9280 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa49413740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc6d3fbe0 con 0x557dc6de06a0 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa3d7fa640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa280a2970 0x7ffa280a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6ddfd80 0x557dc6de0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6ddfd80 0x557dc6de0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52694/0 (socket says 172.21.15.22:52694) 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 -- 172.21.15.22:0/3773692008 learned_addr learned my addr 172.21.15.22:0/3773692008 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6deb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de06a0 0x557dc6de8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa3d7fa640 1 -- 172.21.15.22:0/4238404277 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ffa300cf5c0 con 0x7ffa401003d0 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa280a2970 0x7ffa280a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 -- 172.21.15.22:0/3773692008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de06a0 msgr2=0x557dc6de8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de06a0 0x557dc6de8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 -- 172.21.15.22:0/3773692008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 msgr2=0x557dc6deb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6deb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 -- 172.21.15.22:0/3773692008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc6b207e0 con 0x557dc6ddfd80 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.756+0000 7ffa44dc8640 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6ddfd80 0x557dc6de0160 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ffa38010b10 tx=0x7ffa3800b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8486ff6de3bd31a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.757+0000 7ffa44dc8640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa280a2970 0x7ffa280a4e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa38010fd0 tx=0x7ffa38023000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.757+0000 7ffa267fc640 1 -- 172.21.15.22:0/3773692008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38023070 con 0x557dc6ddfd80 2024-03-08T23:58:13.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.757+0000 7ffa267fc640 1 -- 172.21.15.22:0/3773692008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa3801fc80 con 0x557dc6ddfd80 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.757+0000 7ffa267fc640 1 -- 172.21.15.22:0/3773692008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38026370 con 0x557dc6ddfd80 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 -- 172.21.15.22:0/3773692008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6ddfd80 msgr2=0x557dc6de0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6ddfd80 0x557dc6de0160 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ffa38010b10 tx=0x7ffa3800b2c0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 -- 172.21.15.22:0/3773692008 shutdown_connections 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6deb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de06a0 0x557dc6de8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 --2- 172.21.15.22:0/3773692008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6ddfd80 0x557dc6de0160 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 -- 172.21.15.22:0/3773692008 >> 172.21.15.22:0/3773692008 conn(0x557dc6dda4b0 msgr2=0x557dc6dda890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 -- 172.21.15.22:0/3773692008 shutdown_connections 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 -- 172.21.15.22:0/3773692008 wait complete. 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 -- start start 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.758+0000 7ffa49413740 1 Processor -- start 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6ddfd80 0x557dc6de8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6de60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de9280 0x557dc6de6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc6dec010 con 0x557dc6de06a0 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc6d001c0 con 0x557dc6ddfd80 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc6dec190 con 0x557dc6de9280 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 10 client.0 _ll_register_callbacks cb 0x557dc6cef7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6de60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de9280 0x557dc6de6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6ddfd80 0x557dc6de8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de9280 0x557dc6de6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:56736/0 (socket says 172.21.15.22:56736) 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6ddfd80 0x557dc6de8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45536/0 (socket says 172.21.15.22:45536) 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa44dc8640 1 -- 172.21.15.22:0/3322673928 learned_addr learned my addr 172.21.15.22:0/3322673928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3f7fe640 1 -- 172.21.15.22:0/3322673928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de9280 msgr2=0x557dc6de6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de9280 0x557dc6de6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3f7fe640 1 -- 172.21.15.22:0/3322673928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6ddfd80 msgr2=0x557dc6de8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6ddfd80 0x557dc6de8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3f7fe640 1 -- 172.21.15.22:0/3322673928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dc6b80f70 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6ddfd80 0x557dc6de8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6de60c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ffa34036040 tx=0x7ffa3402ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb0255a2c09325a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/3322673928 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34004240 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/3322673928 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ffa340043e0 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.759+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa3801fc80 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/3322673928 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa340046c0 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 10 client.16920 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557dc6a18170 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/3322673928 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa3801fc80 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/3322673928 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa340035f0 con 0x557dc6de06a0 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 10 client.16920 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:13.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 20 client.16920 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:13.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6df93b0 0x557dc6dfb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 4 client.16920 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:13.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dc6d19960 con 0x557dc6df93b0 2024-03-08T23:58:13.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.760+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6df93b0 0x557dc6dfb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.761+0000 7ffa1f7fe640 10 client.16920 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.761+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6df93b0 0x557dc6dfb7a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389a0b87307241d server_cookie=d45a311f09387a45 in_seq=0 out_seq=0 2024-03-08T23:58:13.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.761+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/3322673928 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557dc6d19960 con 0x557dc6df93b0 2024-03-08T23:58:13.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.761+0000 7ffa1f7fe640 10 client.16920 handle_command_reply: tid=0 2024-03-08T23:58:13.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 client.16920 shutdown 2024-03-08T23:58:13.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 msgr2=0x557dc6de60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6de60c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ffa34036040 tx=0x7ffa3402ce30 comp rx=0 tx=0).stop 2024-03-08T23:58:13.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 shutdown_connections 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6df93b0 0x557dc6dfb7a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6de9280 0x557dc6de6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6de60c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa1f7fe640 0 client.16920 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 --2- 172.21.15.22:0/3322673928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6ddfd80 0x557dc6de8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 >> 172.21.15.22:0/3322673928 conn(0x557dc6dda4b0 msgr2=0x557dc6ddc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 shutdown_connections 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 1 -- 172.21.15.22:0/3322673928 wait complete. 2024-03-08T23:58:13.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.762+0000 7ffa49413740 20 client.16920 trim_cache size 0 max 16384 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 Processor -- start 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 -- start start 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6df5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de06a0 0x557dc6df03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de9280 0x557dc6df0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401ae850 con 0x557dc6de9280 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401ac3f0 con 0x557dc6de06a0 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa49413740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401aa490 con 0x557dc6ddfd80 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de9280 0x557dc6df0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6df5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa44dc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de9280 0x557dc6df0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52708/0 (socket says 172.21.15.22:52708) 2024-03-08T23:58:13.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6df5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:56752/0 (socket says 172.21.15.22:56752) 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa44dc8640 1 -- 172.21.15.22:0/1392988188 learned_addr learned my addr 172.21.15.22:0/1392988188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de06a0 0x557dc6df03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3ffff640 1 -- 172.21.15.22:0/1392988188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de06a0 msgr2=0x557dc6df03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de06a0 0x557dc6df03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3ffff640 1 -- 172.21.15.22:0/1392988188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de9280 msgr2=0x557dc6df0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de9280 0x557dc6df0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa3ffff640 1 -- 172.21.15.22:0/1392988188 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc6dfbb10 con 0x557dc6ddfd80 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.779+0000 7ffa44dc8640 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de9280 0x557dc6df0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.780+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6df5810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa300cc970 tx=0x7ffa3006d620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777f662c3c1e112f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.780+0000 7ffa25ffb640 1 -- 172.21.15.22:0/1392988188 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300a4020 con 0x557dc6ddfd80 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.780+0000 7ffa25ffb640 1 -- 172.21.15.22:0/1392988188 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa30099c30 con 0x557dc6ddfd80 2024-03-08T23:58:13.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.780+0000 7ffa25ffb640 1 -- 172.21.15.22:0/1392988188 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3009ad80 con 0x557dc6ddfd80 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- 172.21.15.22:0/1392988188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 msgr2=0x557dc6df5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6df5810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa300cc970 tx=0x7ffa3006d620 comp rx=0 tx=0).stop 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- 172.21.15.22:0/1392988188 shutdown_connections 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de9280 0x557dc6df0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de06a0 0x557dc6df03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 --2- 172.21.15.22:0/1392988188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6df5810 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- 172.21.15.22:0/1392988188 >> 172.21.15.22:0/1392988188 conn(0x557dc6dda4b0 msgr2=0x557dc6de13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- 172.21.15.22:0/1392988188 shutdown_connections 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- 172.21.15.22:0/1392988188 wait complete. 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- start start 2024-03-08T23:58:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 Processor -- start 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6e48bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6e49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6def680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401aaa90 con 0x557dc6de06a0 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401abd90 con 0x557dc6de9280 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.781+0000 7ffa49413740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401ab0d0 con 0x557dc6ddfd80 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa49413740 10 client.0 _ll_register_callbacks cb 0x7ffa38009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6e49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6e49130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52712/0 (socket says 172.21.15.22:52712) 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 -- 172.21.15.22:0/2879498629 learned_addr learned my addr 172.21.15.22:0/2879498629 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6e48bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa44dc8640 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6def680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 -- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 msgr2=0x557dc6e48bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6e48bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 -- 172.21.15.22:0/2879498629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 msgr2=0x557dc6def680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6def680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 -- 172.21.15.22:0/2879498629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dc6defe60 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa3f7fe640 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6e49130 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009900 tx=0x7ffa3402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b6a01524b6f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/2879498629 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3405a070 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/2879498629 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7ffa340041b0 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa30099c30 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/2879498629 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa340043b0 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa49413740 10 client.16926 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.782+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557dc6e513e0 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/2879498629 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa30099c30 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/2879498629 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa34034440 con 0x557dc6de06a0 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa49413740 10 client.16926 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:13.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa49413740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa49413740 20 client.16926 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa49413740 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6e51b40 0x557dc6e53f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa49413740 4 client.16926 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557dc6d05080 con 0x557dc6e51b40 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6e51b40 0x557dc6e53f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa1f7fe640 10 client.16926 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.783+0000 7ffa3ffff640 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6e51b40 0x557dc6e53f60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd01f494a2c6866 server_cookie=1aa83412c745237d in_seq=0 out_seq=0 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.784+0000 7ffa1f7fe640 1 -- 172.21.15.22:0/2879498629 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557dc6d05080 con 0x557dc6e51b40 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.784+0000 7ffa1f7fe640 10 client.16926 handle_command_reply: tid=0 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.784+0000 7ffa49413740 1 client.16926 shutdown 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.784+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 msgr2=0x557dc6e49130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.784+0000 7ffa49413740 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6e49130 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009900 tx=0x7ffa3402c330 comp rx=0 tx=0).stop 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 shutdown_connections 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557dc6e51b40 0x557dc6e53f60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc6de9280 0x557dc6def680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557dc6de06a0 0x557dc6e49130 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa1f7fe640 0 client.16926 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 --2- 172.21.15.22:0/2879498629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557dc6ddfd80 0x557dc6e48bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 >> 172.21.15.22:0/2879498629 conn(0x557dc6dda4b0 msgr2=0x557dc6ddb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 shutdown_connections 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 1 -- 172.21.15.22:0/2879498629 wait complete. 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa49413740 20 client.16926 trim_cache size 0 max 16384 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:58:13.786 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "5dbdbc33-de12-460b-9c6c-0bf6e2ce0e15", 2024-03-08T23:58:13.787 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:58:13.787 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:58:13.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa280a2970 msgr2=0x7ffa280a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa280a2970 0x7ffa280a4e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa38010fd0 tx=0x7ffa38023000 comp rx=0 tx=0).stop 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 msgr2=0x7ffa4019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 0x7ffa4019d800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e9f0 tx=0x7ffa3000eeb0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 shutdown_connections 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa280a2970 0x7ffa280a4e30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa401979b0 0x7ffa40197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa401054a0 0x7ffa4019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 --2- 172.21.15.22:0/4238404277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa401003d0 0x7ffa4019d800 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.785+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 >> 172.21.15.22:0/4238404277 conn(0x7ffa400772f0 msgr2=0x7ffa400fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.786+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 shutdown_connections 2024-03-08T23:58:13.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.786+0000 7ffa4682b640 1 -- 172.21.15.22:0/4238404277 wait complete. 2024-03-08T23:58:13.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:58:13.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:58:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 -- 172.21.15.22:0/1129088496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1065c0 msgr2=0x7f63dc10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 --2- 172.21.15.22:0/1129088496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1065c0 0x7f63dc10e190 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009ca0 tx=0x7f63d402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 -- 172.21.15.22:0/1129088496 shutdown_connections 2024-03-08T23:58:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 --2- 172.21.15.22:0/1129088496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1065c0 0x7f63dc10e190 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 --2- 172.21.15.22:0/1129088496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc101ea0 0x7f63dc105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 --2- 172.21.15.22:0/1129088496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63dc1014f0 0x7f63dc1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 -- 172.21.15.22:0/1129088496 >> 172.21.15.22:0/1129088496 conn(0x7f63dc0786f0 msgr2=0x7f63dc078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 -- 172.21.15.22:0/1129088496 shutdown_connections 2024-03-08T23:58:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 -- 172.21.15.22:0/1129088496 wait complete. 2024-03-08T23:58:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.924+0000 7f63e269a640 1 Processor -- start 2024-03-08T23:58:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e269a640 1 -- start start 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e269a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1014f0 0x7f63dc19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e269a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63dc1065c0 0x7f63dc19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e269a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc1979a0 0x7f63dc197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e269a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc111680 con 0x7f63dc1014f0 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e269a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc111500 con 0x7f63dc1979a0 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e269a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc111800 con 0x7f63dc1065c0 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1014f0 0x7f63dc19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1014f0 0x7f63dc19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52732/0 (socket says 172.21.15.22:52732) 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 -- 172.21.15.22:0/168385087 learned_addr learned my addr 172.21.15.22:0/168385087 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63db7fe640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63dc1065c0 0x7f63dc19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63e0c37640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc1979a0 0x7f63dc197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 -- 172.21.15.22:0/168385087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63dc1065c0 msgr2=0x7f63dc19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63dc1065c0 0x7f63dc19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 -- 172.21.15.22:0/168385087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc1979a0 msgr2=0x7f63dc197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc1979a0 0x7f63dc197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.925+0000 7f63dbfff640 1 -- 172.21.15.22:0/168385087 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc1984c0 con 0x7f63dc1014f0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.926+0000 7f63dbfff640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1014f0 0x7f63dc19d7a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e9f0 tx=0x7f63cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.926+0000 7f63d97fa640 1 -- 172.21.15.22:0/168385087 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00cd80 con 0x7f63dc1014f0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.926+0000 7f63d97fa640 1 -- 172.21.15.22:0/168385087 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63cc005500 con 0x7f63dc1014f0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.926+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63dc1987b0 con 0x7f63dc1014f0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.926+0000 7f63d97fa640 1 -- 172.21.15.22:0/168385087 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc010690 con 0x7f63dc1014f0 2024-03-08T23:58:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.926+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63dc1a4960 con 0x7f63dc1014f0 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.928+0000 7f63d97fa640 1 -- 172.21.15.22:0/168385087 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f63cc0046c0 con 0x7f63dc1014f0 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.928+0000 7f63e5283740 1 Processor -- start 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.928+0000 7f63d97fa640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63c40a2970 0x7f63c40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.928+0000 7f63e5283740 1 -- start start 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e0c37640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63c40a2970 0x7f63c40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63d97fa640 1 -- 172.21.15.22:0/168385087 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f63cc0cf6b0 con 0x7f63dc1014f0 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e5283740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976140 0x55c2bc976520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e5283740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976af0 0x55c2bc983b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e5283740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc986460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e5283740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bc8d9e00 con 0x55c2bc976140 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e5283740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bc882b10 con 0x55c2bc984070 2024-03-08T23:58:13.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e5283740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bc8d5ea0 con 0x55c2bc976af0 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976140 0x55c2bc976520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976140 0x55c2bc976520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52740/0 (socket says 172.21.15.22:52740) 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 -- 172.21.15.22:0/686051616 learned_addr learned my addr 172.21.15.22:0/686051616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63db7fe640 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976af0 0x55c2bc983b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e0c37640 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc986460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 -- 172.21.15.22:0/686051616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976af0 msgr2=0x55c2bc983b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976af0 0x55c2bc983b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 -- 172.21.15.22:0/686051616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 msgr2=0x55c2bc986460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc986460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 -- 172.21.15.22:0/686051616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2bc6b87d0 con 0x55c2bc976140 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e0c37640 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc986460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63e0c37640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63c40a2970 0x7f63c40a4e30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f63d4005cc0 tx=0x7f63d4005c30 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.929+0000 7f63dbfff640 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976140 0x55c2bc976520 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009380 tx=0x7f63cc09b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a30b1f25d5e108 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.930+0000 7f63c27fc640 1 -- 172.21.15.22:0/686051616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc09fac0 con 0x55c2bc976140 2024-03-08T23:58:13.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.930+0000 7f63c27fc640 1 -- 172.21.15.22:0/686051616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63cc0a4110 con 0x55c2bc976140 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.930+0000 7f63c27fc640 1 -- 172.21.15.22:0/686051616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0860e0 con 0x55c2bc976140 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- 172.21.15.22:0/686051616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976140 msgr2=0x55c2bc976520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976140 0x55c2bc976520 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009380 tx=0x7f63cc09b2d0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- 172.21.15.22:0/686051616 shutdown_connections 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc986460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976af0 0x55c2bc983b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 --2- 172.21.15.22:0/686051616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976140 0x55c2bc976520 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- 172.21.15.22:0/686051616 >> 172.21.15.22:0/686051616 conn(0x55c2bc970e90 msgr2=0x55c2bc971270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- 172.21.15.22:0/686051616 shutdown_connections 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- 172.21.15.22:0/686051616 wait complete. 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- start start 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 Processor -- start 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc976140 0x55c2bc983800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc980b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc984070 0x55c2bc9810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bc986e30 con 0x55c2bc976af0 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bc8960a0 con 0x55c2bc976140 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.931+0000 7f63e5283740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bc986fb0 con 0x55c2bc984070 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e5283740 10 client.0 _ll_register_callbacks cb 0x55c2bc895a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc980b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc976140 0x55c2bc983800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc980b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52744/0 (socket says 172.21.15.22:52744) 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63db7fe640 1 -- 172.21.15.22:0/3900842371 learned_addr learned my addr 172.21.15.22:0/3900842371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e0c37640 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc984070 0x55c2bc9810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e0c37640 1 -- 172.21.15.22:0/3900842371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc976140 msgr2=0x55c2bc983800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e0c37640 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc976140 0x55c2bc983800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e0c37640 1 -- 172.21.15.22:0/3900842371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 msgr2=0x55c2bc980b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e0c37640 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc980b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e0c37640 1 -- 172.21.15.22:0/3900842371 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2bc5ee3f0 con 0x55c2bc984070 2024-03-08T23:58:13.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63db7fe640 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc980b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e0c37640 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc984070 0x55c2bc9810d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63d402c700 tx=0x7f63d402c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25f776469b4bb4d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3900842371 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4039040 con 0x55c2bc984070 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63cc0a4110 con 0x55c2bc984070 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.932+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3900842371 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f63d4003d80 con 0x55c2bc984070 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3900842371 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63d4058050 con 0x55c2bc984070 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 10 client.26395 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c2bc5ae0f0 con 0x55c2bc984070 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3900842371 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63cc0a4110 con 0x55c2bc984070 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3900842371 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63d403f6c0 con 0x55c2bc984070 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 10 client.26395 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 10 client.26395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 20 client.26395 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc993af0 0x55c2bc995ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 4 client.26395 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:13.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2bc8e7340 con 0x55c2bc993af0 2024-03-08T23:58:13.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.933+0000 7f63dbfff640 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc993af0 0x55c2bc995ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.934+0000 7f63bb7fe640 10 client.26395 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.934+0000 7f63dbfff640 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc993af0 0x55c2bc995ee0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c1f96fbab9e169 server_cookie=3a9398ab75caa375 in_seq=0 out_seq=0 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3900842371 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2bc8e7340 con 0x55c2bc993af0 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63bb7fe640 10 client.26395 handle_command_reply: tid=0 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 client.26395 shutdown 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc984070 msgr2=0x55c2bc9810d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc984070 0x55c2bc9810d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63d402c700 tx=0x7f63d402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 shutdown_connections 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc993af0 0x55c2bc995ee0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63bb7fe640 0 client.26395 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc984070 0x55c2bc9810d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc980b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 --2- 172.21.15.22:0/3900842371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc976140 0x55c2bc983800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 >> 172.21.15.22:0/3900842371 conn(0x55c2bc970e90 msgr2=0x55c2bc972be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 shutdown_connections 2024-03-08T23:58:13.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 1 -- 172.21.15.22:0/3900842371 wait complete. 2024-03-08T23:58:13.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.935+0000 7f63e5283740 20 client.26395 trim_cache size 0 max 16384 2024-03-08T23:58:13.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.955+0000 7f63e5283740 1 Processor -- start 2024-03-08T23:58:13.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.955+0000 7f63e5283740 1 -- start start 2024-03-08T23:58:13.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.955+0000 7f63e5283740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc97f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.955+0000 7f63e5283740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc97f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.955+0000 7f63e5283740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc97bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.955+0000 7f63e5283740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1ae8c0 con 0x55c2bc976af0 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63e5283740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1ac460 con 0x55c2bc984070 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63e5283740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1aa500 con 0x55c2bc976140 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc97f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63e0c37640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc97bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc97f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52752/0 (socket says 172.21.15.22:52752) 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc97f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 -- 172.21.15.22:0/4251382777 learned_addr learned my addr 172.21.15.22:0/4251382777 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 -- 172.21.15.22:0/4251382777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 msgr2=0x55c2bc97f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 --2- 172.21.15.22:0/4251382777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc97f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 -- 172.21.15.22:0/4251382777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 msgr2=0x55c2bc97bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 --2- 172.21.15.22:0/4251382777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc97bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 -- 172.21.15.22:0/4251382777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2bc996250 con 0x55c2bc976af0 2024-03-08T23:58:13.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63db7fe640 1 --2- 172.21.15.22:0/4251382777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc97f9a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63d0012580 tx=0x7f63d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a901dabecdf22d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63c1ffb640 1 -- 172.21.15.22:0/4251382777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0021070 con 0x55c2bc976af0 2024-03-08T23:58:13.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63c1ffb640 1 -- 172.21.15.22:0/4251382777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63d00035f0 con 0x55c2bc976af0 2024-03-08T23:58:13.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.956+0000 7f63c1ffb640 1 -- 172.21.15.22:0/4251382777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000f740 con 0x55c2bc976af0 2024-03-08T23:58:13.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 -- 172.21.15.22:0/4251382777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 msgr2=0x55c2bc97f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 --2- 172.21.15.22:0/4251382777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc97f9a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63d0012580 tx=0x7f63d0005f90 comp rx=0 tx=0).stop 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 -- 172.21.15.22:0/4251382777 shutdown_connections 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 --2- 172.21.15.22:0/4251382777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc97bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 --2- 172.21.15.22:0/4251382777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc97f9a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 --2- 172.21.15.22:0/4251382777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc97f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 -- 172.21.15.22:0/4251382777 >> 172.21.15.22:0/4251382777 conn(0x55c2bc970e90 msgr2=0x55c2bc980b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 -- 172.21.15.22:0/4251382777 shutdown_connections 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 -- 172.21.15.22:0/4251382777 wait complete. 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.957+0000 7f63e5283740 1 -- start start 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 1 Processor -- start 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc98b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc987900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc987e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1aab00 con 0x55c2bc976af0 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1abe00 con 0x55c2bc984070 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1ab140 con 0x55c2bc976140 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e5283740 10 client.0 _ll_register_callbacks cb 0x7f63d4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc987900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc98b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc987900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52758/0 (socket says 172.21.15.22:52758) 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63db7fe640 1 -- 172.21.15.22:0/3651286656 learned_addr learned my addr 172.21.15.22:0/3651286656 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e0c37640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc987e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63dbfff640 1 -- 172.21.15.22:0/3651286656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 msgr2=0x55c2bc987e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63dbfff640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc987e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63dbfff640 1 -- 172.21.15.22:0/3651286656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 msgr2=0x55c2bc987900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63dbfff640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc987900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63dbfff640 1 -- 172.21.15.22:0/3651286656 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2bc988650 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63e0c37640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc987e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.958+0000 7f63db7fe640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc987900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63dbfff640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc98b580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0ccb10 tx=0x7f63cc09b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a89843aea6cc3b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3651286656 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0a4110 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3651286656 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f63cc0ccce0 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63d00035f0 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3651286656 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63cc097fe0 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63e5283740 10 client.26401 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c2bc9eaf80 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3651286656 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63cc0cf2d0 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.959+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3651286656 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63cc0860b0 con 0x55c2bc976140 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63e5283740 10 client.26401 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63e5283740 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63e5283740 20 client.26401 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:13.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63e5283740 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc9eb210 0x55c2bc9ed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63e5283740 4 client.26401 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f63d00035f0 con 0x55c2bc9eb210 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63db7fe640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc9eb210 0x55c2bc9ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63db7fe640 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc9eb210 0x55c2bc9ed600 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3539de02ff566fc3 server_cookie=114290d78d5c7826 in_seq=0 out_seq=0 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.960+0000 7f63bb7fe640 10 client.26401 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63bb7fe640 1 -- 172.21.15.22:0/3651286656 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f63d00035f0 con 0x55c2bc9eb210 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63bb7fe640 10 client.26401 handle_command_reply: tid=0 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 client.26401 shutdown 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 msgr2=0x55c2bc98b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc98b580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0ccb10 tx=0x7f63cc09b400 comp rx=0 tx=0).stop 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 shutdown_connections 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2bc9eb210 0x55c2bc9ed600 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bc984070 0x55c2bc987e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2bc976af0 0x55c2bc987900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 --2- 172.21.15.22:0/3651286656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2bc976140 0x55c2bc98b580 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63bb7fe640 0 client.26401 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 >> 172.21.15.22:0/3651286656 conn(0x55c2bc970e90 msgr2=0x55c2bc97bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 shutdown_connections 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 1 -- 172.21.15.22:0/3651286656 wait complete. 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.961+0000 7f63e5283740 20 client.26401 trim_cache size 0 max 16384 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stdout: "5dbdbc33-de12-460b-9c6c-0bf6e2ce0e15": { 2024-03-08T23:58:13.962 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:58:13.963 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "5dbdbc33-de12-460b-9c6c-0bf6e2ce0e15", 2024-03-08T23:58:13.963 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:58:13.963 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:58:13.963 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:58:13.963 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63c40a2970 msgr2=0x7f63c40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63c40a2970 0x7f63c40a4e30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f63d4005cc0 tx=0x7f63d4005c30 comp rx=0 tx=0).stop 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1014f0 msgr2=0x7f63dc19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1014f0 0x7f63dc19d7a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00e9f0 tx=0x7f63cc00edf0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 shutdown_connections 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63c40a2970 0x7f63c40a4e30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc1979a0 0x7f63dc197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63dc1065c0 0x7f63dc19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 --2- 172.21.15.22:0/168385087 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63dc1014f0 0x7f63dc19d7a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 >> 172.21.15.22:0/168385087 conn(0x7f63dc0786f0 msgr2=0x7f63dc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:13.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 shutdown_connections 2024-03-08T23:58:13.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:13.962+0000 7f63e269a640 1 -- 172.21.15.22:0/168385087 wait complete. 2024-03-08T23:58:13.969 INFO:tasks.cephfs.filesystem:scrub status for tag:5dbdbc33-de12-460b-9c6c-0bf6e2ce0e15 - {'path': '/', 'tag': '5dbdbc33-de12-460b-9c6c-0bf6e2ce0e15', 'options': 'recursive,force'} 2024-03-08T23:58:14.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 330204 9.61 MB/sec execute 22 sec latency 13.134 ms 2024-03-08T23:58:14.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:14 smithi022 ceph-mon[31894]: pgmap v116: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 254 op/s 2024-03-08T23:58:14.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:14 smithi062 ceph-mon[36129]: pgmap v116: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 254 op/s 2024-03-08T23:58:14.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:14 smithi192 ceph-mon[35995]: pgmap v116: 129 pgs: 129 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 254 op/s 2024-03-08T23:58:15.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 332778 9.72 MB/sec execute 23 sec latency 19.208 ms 2024-03-08T23:58:15.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:15 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:58:15.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:15 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:15.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:15 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:58:15.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:15 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:15.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:15 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:58:15.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:15 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:58:15.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:15 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:15.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:15 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:58:15.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:15 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:15.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:15 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:58:15.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:15 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:58:15.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:15 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:15 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:58:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:15 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:15 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:58:16.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 334373 9.65 MB/sec execute 24 sec latency 41.684 ms 2024-03-08T23:58:16.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:16 smithi022 ceph-mon[31894]: pgmap v117: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 502 op/s 2024-03-08T23:58:16.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:16 smithi062 ceph-mon[36129]: pgmap v117: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 502 op/s 2024-03-08T23:58:16.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:16 smithi192 ceph-mon[35995]: pgmap v117: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 502 op/s 2024-03-08T23:58:17.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 337251 9.99 MB/sec execute 25 sec latency 23.392 ms 2024-03-08T23:58:18.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 339743 10.04 MB/sec execute 26 sec latency 16.935 ms 2024-03-08T23:58:18.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:18 smithi022 ceph-mon[31894]: pgmap v118: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 502 op/s 2024-03-08T23:58:18.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:18 smithi062 ceph-mon[36129]: pgmap v118: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 502 op/s 2024-03-08T23:58:18.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:18 smithi192 ceph-mon[35995]: pgmap v118: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 502 op/s 2024-03-08T23:58:19.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 341587 10.04 MB/sec execute 27 sec latency 63.409 ms 2024-03-08T23:58:20.710 INFO:tasks.workunit.client.0.smithi022.stdout: 1 344344 10.28 MB/sec execute 28 sec latency 44.063 ms 2024-03-08T23:58:20.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:20 smithi022 ceph-mon[31894]: pgmap v119: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 477 op/s 2024-03-08T23:58:20.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:20 smithi062 ceph-mon[36129]: pgmap v119: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 477 op/s 2024-03-08T23:58:20.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:20 smithi192 ceph-mon[35995]: pgmap v119: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 477 op/s 2024-03-08T23:58:21.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 346327 10.14 MB/sec execute 29 sec latency 72.093 ms 2024-03-08T23:58:22.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 348199 10.25 MB/sec execute 30 sec latency 22.858 ms 2024-03-08T23:58:22.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:22 smithi022 ceph-mon[31894]: pgmap v120: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 455 op/s 2024-03-08T23:58:22.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:22 smithi062 ceph-mon[36129]: pgmap v120: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 455 op/s 2024-03-08T23:58:22.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:22 smithi192 ceph-mon[35995]: pgmap v120: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 455 op/s 2024-03-08T23:58:23.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 351166 10.48 MB/sec execute 31 sec latency 7.472 ms 2024-03-08T23:58:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 -- 172.21.15.22:0/2367759743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39240ffa60 msgr2=0x7f39240ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2367759743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39240ffa60 0x7f39240ffe40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3914009900 tx=0x7f391402bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 -- 172.21.15.22:0/2367759743 shutdown_connections 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2367759743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924106620 0x7f392410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2367759743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3924100410 0x7f3924105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2367759743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39240ffa60 0x7f39240ffe40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 -- 172.21.15.22:0/2367759743 >> 172.21.15.22:0/2367759743 conn(0x7f3924076740 msgr2=0x7f39240fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 -- 172.21.15.22:0/2367759743 shutdown_connections 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.205+0000 7f3929b7e640 1 -- 172.21.15.22:0/2367759743 wait complete. 2024-03-08T23:58:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 Processor -- start 2024-03-08T23:58:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 -- start start 2024-03-08T23:58:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39240ffa60 0x7f392419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3924106620 0x7f392419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3924197970 0x7f3924197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241116d0 con 0x7f3924106620 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924111550 con 0x7f39240ffa60 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3929b7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924111850 con 0x7f3924197970 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.206+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3924106620 0x7f392419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3924106620 0x7f392419dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56560/0 (socket says 172.21.15.22:56560) 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 -- 172.21.15.22:0/2530348900 learned_addr learned my addr 172.21.15.22:0/2530348900 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3923fff640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3924197970 0x7f3924197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f39237fe640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39240ffa60 0x7f392419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 -- 172.21.15.22:0/2530348900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3924197970 msgr2=0x7f3924197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3924197970 0x7f3924197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 -- 172.21.15.22:0/2530348900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39240ffa60 msgr2=0x7f392419d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39240ffa60 0x7f392419d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 -- 172.21.15.22:0/2530348900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924198490 con 0x7f3924106620 2024-03-08T23:58:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f39237fe640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39240ffa60 0x7f392419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:58:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3922ffd640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3924106620 0x7f392419dcb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f391800e9f0 tx=0x7f391800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3920ff9640 1 -- 172.21.15.22:0/2530348900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800ccc0 con 0x7f3924106620 2024-03-08T23:58:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3920ff9640 1 -- 172.21.15.22:0/2530348900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3918005500 con 0x7f3924106620 2024-03-08T23:58:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3924198780 con 0x7f3924106620 2024-03-08T23:58:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3920ff9640 1 -- 172.21.15.22:0/2530348900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918010690 con 0x7f3924106620 2024-03-08T23:58:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.207+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39241a4930 con 0x7f3924106620 2024-03-08T23:58:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3920ff9640 1 -- 172.21.15.22:0/2530348900 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f39180046c0 con 0x7f3924106620 2024-03-08T23:58:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 Processor -- start 2024-03-08T23:58:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 -- start start 2024-03-08T23:58:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168a5125f0 con 0x56168a5b23e0 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168a4bb930 con 0x56168a5b3150 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f392c837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168a50e690 con 0x56168a5bb960 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3920ff9640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f390c0a2a10 0x7f390c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3923fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56574/0 (socket says 172.21.15.22:56574) 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f39237fe640 1 -- 172.21.15.22:0/2003948572 learned_addr learned my addr 172.21.15.22:0/2003948572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3923fff640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f390c0a2a10 0x7f390c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3920ff9640 1 -- 172.21.15.22:0/2530348900 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f39180cf620 con 0x7f3924106620 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3923fff640 1 -- 172.21.15.22:0/2003948572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 msgr2=0x56168a5bdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3923fff640 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5bdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3923fff640 1 -- 172.21.15.22:0/2003948572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 msgr2=0x56168a5b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3923fff640 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.210+0000 7f3923fff640 1 -- 172.21.15.22:0/2003948572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56168a227210 con 0x56168a5b3150 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.211+0000 7f3922ffd640 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5bdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.211+0000 7f39237fe640 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.211+0000 7f3923fff640 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5b1ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f391c011cc0 tx=0x7f391c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21df0eb7ef02cd95 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.211+0000 7f3923fff640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f390c0a2a10 0x7f390c0a4ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f391c00fc90 tx=0x7f391c018000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.211+0000 7f3909ffb640 1 -- 172.21.15.22:0/2003948572 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c004b20 con 0x56168a5b3150 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.211+0000 7f3909ffb640 1 -- 172.21.15.22:0/2003948572 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f391c009ce0 con 0x56168a5b3150 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.211+0000 7f3909ffb640 1 -- 172.21.15.22:0/2003948572 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c005420 con 0x56168a5b3150 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 -- 172.21.15.22:0/2003948572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 msgr2=0x56168a5b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5b1ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f391c011cc0 tx=0x7f391c00f790 comp rx=0 tx=0).stop 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 -- 172.21.15.22:0/2003948572 shutdown_connections 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5bdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 --2- 172.21.15.22:0/2003948572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5b1ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 -- 172.21.15.22:0/2003948572 >> 172.21.15.22:0/2003948572 conn(0x56168a5755b0 msgr2=0x56168a5ad0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 -- 172.21.15.22:0/2003948572 shutdown_connections 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 -- 172.21.15.22:0/2003948572 wait complete. 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 -- start start 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.212+0000 7f392c837740 1 Processor -- start 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f392c837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f392c837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f392c837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5b9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f392c837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168a5be630 con 0x56168a5b23e0 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f392c837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168a4ced90 con 0x56168a5bb960 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f392c837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168a5be7b0 con 0x56168a5b3150 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f392c837740 10 client.0 _ll_register_callbacks cb 0x56168a226a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56578/0 (socket says 172.21.15.22:56578) 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 -- 172.21.15.22:0/1782115580 learned_addr learned my addr 172.21.15.22:0/1782115580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f3922ffd640 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f3923fff640 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 -- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 msgr2=0x56168a5b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 -- 172.21.15.22:0/1782115580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 msgr2=0x56168a5b9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5b9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 -- 172.21.15.22:0/1782115580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56168a2f3620 con 0x56168a5b23e0 2024-03-08T23:58:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f39237fe640 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b89d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f391402cfd0 tx=0x7f3914009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801930786b17b9a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f3902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.213+0000 7f3902ffd640 1 -- 172.21.15.22:0/1782115580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914004450 con 0x56168a5b23e0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f3902ffd640 1 -- 172.21.15.22:0/1782115580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3914005030 con 0x56168a5b23e0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f391c009ce0 con 0x56168a5b23e0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f3902ffd640 1 -- 172.21.15.22:0/1782115580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f391404b050 con 0x56168a5b23e0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 10 client.16956 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56168a5ca8d0 con 0x56168a5b23e0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f3902ffd640 1 -- 172.21.15.22:0/1782115580 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f391c009ce0 con 0x56168a5b23e0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f3902ffd640 1 -- 172.21.15.22:0/1782115580 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f39140035f0 con 0x56168a5b23e0 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 10 client.16956 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 20 client.16956 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a5cbd90 0x56168a5ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 4 client.16956 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56168a51fb30 con 0x56168a5cbd90 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.214+0000 7f3922ffd640 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a5cbd90 0x56168a5ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.215+0000 7f3902ffd640 10 client.16956 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.215+0000 7f3922ffd640 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a5cbd90 0x56168a5ce180 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae866633b928a598 server_cookie=f230982c5fee8ced in_seq=0 out_seq=0 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f3902ffd640 1 -- 172.21.15.22:0/1782115580 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56168a51fb30 con 0x56168a5cbd90 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f3902ffd640 10 client.16956 handle_command_reply: tid=0 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 client.16956 shutdown 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 msgr2=0x56168a5b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b89d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f391402cfd0 tx=0x7f3914009670 comp rx=0 tx=0).stop 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 shutdown_connections 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a5cbd90 0x56168a5ce180 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5b9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f3902ffd640 0 client.16956 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 --2- 172.21.15.22:0/1782115580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5b89d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 >> 172.21.15.22:0/1782115580 conn(0x56168a5755b0 msgr2=0x56168a5a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 shutdown_connections 2024-03-08T23:58:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 1 -- 172.21.15.22:0/1782115580 wait complete. 2024-03-08T23:58:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.217+0000 7f392c837740 20 client.16956 trim_cache size 0 max 16384 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 Processor -- start 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 -- start start 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5bb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241ae790 con 0x56168a5b23e0 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241ac330 con 0x56168a5bb960 2024-03-08T23:58:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f392c837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241aa3d0 con 0x56168a5b3150 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5bb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f3923fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f3923fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58602/0 (socket says 172.21.15.22:58602) 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.238+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5bb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56584/0 (socket says 172.21.15.22:56584) 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f3923fff640 1 -- 172.21.15.22:0/956373345 learned_addr learned my addr 172.21.15.22:0/956373345 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f3922ffd640 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39237fe640 1 -- 172.21.15.22:0/956373345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 msgr2=0x56168a5c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39237fe640 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39237fe640 1 -- 172.21.15.22:0/956373345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 msgr2=0x56168a5c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39237fe640 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5c2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39237fe640 1 -- 172.21.15.22:0/956373345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56168a5be0c0 con 0x56168a5b23e0 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f3923fff640 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39237fe640 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5bb100 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3914005ab0 tx=0x7f391402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e86a323789a5ff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39097fa640 1 -- 172.21.15.22:0/956373345 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914003bb0 con 0x56168a5b23e0 2024-03-08T23:58:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39097fa640 1 -- 172.21.15.22:0/956373345 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3914005550 con 0x56168a5b23e0 2024-03-08T23:58:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.239+0000 7f39097fa640 1 -- 172.21.15.22:0/956373345 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140046c0 con 0x56168a5b23e0 2024-03-08T23:58:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 -- 172.21.15.22:0/956373345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 msgr2=0x56168a5bb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5bb100 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3914005ab0 tx=0x7f391402c7e0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 -- 172.21.15.22:0/956373345 shutdown_connections 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5bb960 0x56168a5c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5b3150 0x56168a5c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 --2- 172.21.15.22:0/956373345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5bb100 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 -- 172.21.15.22:0/956373345 >> 172.21.15.22:0/956373345 conn(0x56168a5755b0 msgr2=0x56168a5b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 -- 172.21.15.22:0/956373345 shutdown_connections 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 -- 172.21.15.22:0/956373345 wait complete. 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.240+0000 7f392c837740 1 -- start start 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 1 Processor -- start 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241aa9d0 con 0x56168a5b23e0 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241abcd0 con 0x56168a5b3150 2024-03-08T23:58:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241ab010 con 0x56168a5bb960 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f392c837740 10 client.0 _ll_register_callbacks cb 0x7f391c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5c1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56598/0 (socket says 172.21.15.22:56598) 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 -- 172.21.15.22:0/377545076 learned_addr learned my addr 172.21.15.22:0/377545076 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f3923fff640 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 -- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 msgr2=0x56168a5c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 -- 172.21.15.22:0/377545076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 msgr2=0x56168a5c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f39237fe640 1 -- 172.21.15.22:0/377545076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56168a6224b0 con 0x56168a5b23e0 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.241+0000 7f3922ffd640 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f39237fe640 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5c1600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3914003a40 tx=0x7f3914005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5155cc966a7f3f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f3902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f3902ffd640 1 -- 172.21.15.22:0/377545076 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140035f0 con 0x56168a5b23e0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f3902ffd640 1 -- 172.21.15.22:0/377545076 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3914034440 con 0x56168a5b23e0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3914005550 con 0x56168a5b23e0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f3902ffd640 1 -- 172.21.15.22:0/377545076 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3914003790 con 0x56168a5b23e0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 10 client.16962 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56168a6237a0 con 0x56168a5b23e0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f3902ffd640 1 -- 172.21.15.22:0/377545076 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3914005550 con 0x56168a5b23e0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f3902ffd640 1 -- 172.21.15.22:0/377545076 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f391404f5c0 con 0x56168a5b23e0 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 10 client.16962 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 20 client.16962 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a623df0 0x56168a6261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 4 client.16962 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:58:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.242+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56168a519e80 con 0x56168a623df0 2024-03-08T23:58:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.243+0000 7f3922ffd640 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a623df0 0x56168a6261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.243+0000 7f3922ffd640 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a623df0 0x56168a6261e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc2d514395b31e77 server_cookie=a412ea0c1b3e80f5 in_seq=0 out_seq=0 2024-03-08T23:58:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.243+0000 7f3902ffd640 10 client.16962 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.495+0000 7f3902ffd640 1 -- 172.21.15.22:0/377545076 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56168a519e80 con 0x56168a623df0 2024-03-08T23:58:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.495+0000 7f3902ffd640 10 client.16962 handle_command_reply: tid=0 2024-03-08T23:58:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.495+0000 7f392c837740 1 client.16962 shutdown 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 msgr2=0x56168a5c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5c1600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3914003a40 tx=0x7f3914005e50 comp rx=0 tx=0).stop 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 shutdown_connections 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56168a623df0 0x56168a6261e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56168a5bb960 0x56168a5c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168a5b3150 0x56168a5c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 --2- 172.21.15.22:0/377545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56168a5b23e0 0x56168a5c1600 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 >> 172.21.15.22:0/377545076 conn(0x56168a5755b0 msgr2=0x56168a5a75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f3902ffd640 0 client.16962 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 shutdown_connections 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 1 -- 172.21.15.22:0/377545076 wait complete. 2024-03-08T23:58:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.496+0000 7f392c837740 20 client.16962 trim_cache size 0 max 16384 2024-03-08T23:58:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:58:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:58:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:58:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f390c0a2a10 msgr2=0x7f390c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f390c0a2a10 0x7f390c0a4ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f391c00fc90 tx=0x7f391c018000 comp rx=0 tx=0).stop 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3924106620 msgr2=0x7f392419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3924106620 0x7f392419dcb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f391800e9f0 tx=0x7f391800eeb0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 shutdown_connections 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f390c0a2a10 0x7f390c0a4ed0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3924197970 0x7f3924197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3924106620 0x7f392419dcb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 --2- 172.21.15.22:0/2530348900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39240ffa60 0x7f392419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 >> 172.21.15.22:0/2530348900 conn(0x7f3924076740 msgr2=0x7f3924104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 shutdown_connections 2024-03-08T23:58:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:24.497+0000 7f3929b7e640 1 -- 172.21.15.22:0/2530348900 wait complete. 2024-03-08T23:58:24.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 352408 10.27 MB/sec execute 32 sec latency 133.317 ms 2024-03-08T23:58:24.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:24 smithi022 ceph-mon[31894]: pgmap v121: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 396 op/s 2024-03-08T23:58:24.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:24 smithi062 ceph-mon[36129]: pgmap v121: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 396 op/s 2024-03-08T23:58:24.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:24 smithi192 ceph-mon[35995]: pgmap v121: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 396 op/s 2024-03-08T23:58:25.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 355178 10.47 MB/sec execute 33 sec latency 19.723 ms 2024-03-08T23:58:26.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 357485 10.49 MB/sec execute 34 sec latency 72.461 ms 2024-03-08T23:58:26.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:26 smithi022 ceph-mon[31894]: pgmap v122: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 511 op/s 2024-03-08T23:58:26.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:26 smithi062 ceph-mon[36129]: pgmap v122: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 511 op/s 2024-03-08T23:58:26.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:26 smithi192 ceph-mon[35995]: pgmap v122: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 511 op/s 2024-03-08T23:58:27.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 359218 10.45 MB/sec execute 35 sec latency 25.429 ms 2024-03-08T23:58:28.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 362162 10.66 MB/sec execute 36 sec latency 13.167 ms 2024-03-08T23:58:28.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:28 smithi022 ceph-mon[31894]: pgmap v123: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-03-08T23:58:28.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:28 smithi062 ceph-mon[36129]: pgmap v123: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-03-08T23:58:28.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:28 smithi192 ceph-mon[35995]: pgmap v123: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-03-08T23:58:29.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 364782 10.71 MB/sec execute 37 sec latency 24.789 ms 2024-03-08T23:58:30.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 366591 10.66 MB/sec execute 38 sec latency 20.065 ms 2024-03-08T23:58:30.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:30 smithi022 ceph-mon[31894]: pgmap v124: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-08T23:58:30.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:30 smithi062 ceph-mon[36129]: pgmap v124: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-08T23:58:30.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:30 smithi192 ceph-mon[35995]: pgmap v124: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 392 op/s 2024-03-08T23:58:31.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 369270 10.82 MB/sec execute 39 sec latency 55.347 ms 2024-03-08T23:58:32.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 371896 10.87 MB/sec execute 40 sec latency 14.595 ms 2024-03-08T23:58:32.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:32 smithi022 ceph-mon[31894]: pgmap v125: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-03-08T23:58:32.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:32 smithi062 ceph-mon[36129]: pgmap v125: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-03-08T23:58:32.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:32 smithi192 ceph-mon[35995]: pgmap v125: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-03-08T23:58:33.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 373924 10.89 MB/sec execute 41 sec latency 15.058 ms 2024-03-08T23:58:34.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 376532 10.96 MB/sec execute 42 sec latency 13.368 ms 2024-03-08T23:58:34.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:34 smithi022 ceph-mon[31894]: pgmap v126: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 405 op/s 2024-03-08T23:58:34.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:34 smithi062 ceph-mon[36129]: pgmap v126: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 405 op/s 2024-03-08T23:58:34.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:34 smithi192 ceph-mon[35995]: pgmap v126: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 405 op/s 2024-03-08T23:58:35.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 379557 11.11 MB/sec execute 43 sec latency 16.647 ms 2024-03-08T23:58:36.711 INFO:tasks.workunit.client.0.smithi022.stdout: 1 381425 11.02 MB/sec execute 44 sec latency 28.037 ms 2024-03-08T23:58:36.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:36 smithi022 ceph-mon[31894]: pgmap v127: 129 pgs: 129 active+clean; 56 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-03-08T23:58:36.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:36 smithi062 ceph-mon[36129]: pgmap v127: 129 pgs: 129 active+clean; 56 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-03-08T23:58:36.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:36 smithi192 ceph-mon[35995]: pgmap v127: 129 pgs: 129 active+clean; 56 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.7 MiB/s wr, 528 op/s 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 -- 172.21.15.22:0/4097156956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8105190 msgr2=0x7f0de8105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 --2- 172.21.15.22:0/4097156956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8105190 0x7f0de8105570 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009900 tx=0x7f0dd802bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 -- 172.21.15.22:0/4097156956 shutdown_connections 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 --2- 172.21.15.22:0/4097156956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8100e00 0x7f0de810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 --2- 172.21.15.22:0/4097156956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de8100440 0x7f0de81008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 --2- 172.21.15.22:0/4097156956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8105190 0x7f0de8105570 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 -- 172.21.15.22:0/4097156956 >> 172.21.15.22:0/4097156956 conn(0x7f0de8077040 msgr2=0x7f0de8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 -- 172.21.15.22:0/4097156956 shutdown_connections 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.586+0000 7f0decad1640 1 -- 172.21.15.22:0/4097156956 wait complete. 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 Processor -- start 2024-03-08T23:58:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 -- start start 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8100440 0x7f0de819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8100e00 0x7f0de819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de8105190 0x7f0de8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de8110650 con 0x7f0de8100e00 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81104d0 con 0x7f0de8105190 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0decad1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81107d0 con 0x7f0de8100440 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8100e00 0x7f0de819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8100e00 0x7f0de819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54094/0 (socket says 172.21.15.22:54094) 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0ddffff640 1 -- 172.21.15.22:0/2508039213 learned_addr learned my addr 172.21.15.22:0/2508039213 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0de659c640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8100440 0x7f0de819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.587+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de8105190 0x7f0de8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddffff640 1 -- 172.21.15.22:0/2508039213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8100440 msgr2=0x7f0de819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddffff640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8100440 0x7f0de819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddffff640 1 -- 172.21.15.22:0/2508039213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de8105190 msgr2=0x7f0de8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddffff640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de8105190 0x7f0de8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddffff640 1 -- 172.21.15.22:0/2508039213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8198110 con 0x7f0de8100e00 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddffff640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8100e00 0x7f0de819dd20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000d950 tx=0x7f0dd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddf7fe640 1 -- 172.21.15.22:0/2508039213 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0017070 con 0x7f0de8100e00 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddf7fe640 1 -- 172.21.15.22:0/2508039213 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dd0009c30 con 0x7f0de8100e00 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de8198400 con 0x7f0de8100e00 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0ddf7fe640 1 -- 172.21.15.22:0/2508039213 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd000f6b0 con 0x7f0de8100e00 2024-03-08T23:58:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.588+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de81a46a0 con 0x7f0de8100e00 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.590+0000 7f0ddf7fe640 1 -- 172.21.15.22:0/2508039213 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0dd00109d0 con 0x7f0de8100e00 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 Processor -- start 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0ddf7fe640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0dd40a2a90 0x7f0dd40a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 -- start start 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0dd40a2a90 0x7f0dd40a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0ddf7fe640 1 -- 172.21.15.22:0/2508039213 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0dd00ced30 con 0x7f0de8100e00 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ebea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ebf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c530e235f0 con 0x55c530ebfd40 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c530dcc930 con 0x55c530ecc960 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0def78d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c530e1f690 con 0x55c530ebefd0 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0de659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ebea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ebf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0de659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ebea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54110/0 (socket says 172.21.15.22:54110) 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.591+0000 7f0de659c640 1 -- 172.21.15.22:0/28081457 learned_addr learned my addr 172.21.15.22:0/28081457 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddffff640 1 -- 172.21.15.22:0/28081457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 msgr2=0x55c530eced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddffff640 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddffff640 1 -- 172.21.15.22:0/28081457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 msgr2=0x55c530ebea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddffff640 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ebea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddffff640 1 -- 172.21.15.22:0/28081457 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c530b38210 con 0x55c530ebefd0 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0dd40a2a90 0x7f0dd40a4f50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009fd0 tx=0x7f0de0011bc0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddffff640 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ebf450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00cc0e0 tx=0x7f0dd0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aeff7809506f8c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddcff9640 1 -- 172.21.15.22:0/28081457 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0020040 con 0x55c530ebefd0 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddcff9640 1 -- 172.21.15.22:0/28081457 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dd00a3790 con 0x55c530ebefd0 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.592+0000 7f0ddcff9640 1 -- 172.21.15.22:0/28081457 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd009a750 con 0x55c530ebefd0 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 -- 172.21.15.22:0/28081457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 msgr2=0x55c530ebf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ebf450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00cc0e0 tx=0x7f0dd0005f90 comp rx=0 tx=0).stop 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 -- 172.21.15.22:0/28081457 shutdown_connections 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ebf450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 --2- 172.21.15.22:0/28081457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ebea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 -- 172.21.15.22:0/28081457 >> 172.21.15.22:0/28081457 conn(0x55c530e865b0 msgr2=0x55c530eb9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 -- 172.21.15.22:0/28081457 shutdown_connections 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 -- 172.21.15.22:0/28081457 wait complete. 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 -- start start 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.593+0000 7f0def78d740 1 Processor -- start 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ec9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ec9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c530ecf630 con 0x55c530ebfd40 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c530ddfd90 con 0x55c530ecc960 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c530ecf7b0 con 0x55c530ebefd0 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 10 client.0 _ll_register_callbacks cb 0x55c530b37a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ec9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ec9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54118/0 (socket says 172.21.15.22:54118) 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 -- 172.21.15.22:0/1687313253 learned_addr learned my addr 172.21.15.22:0/1687313253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0de659c640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ec9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 -- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 msgr2=0x55c530ec9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ec9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 -- 172.21.15.22:0/1687313253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 msgr2=0x55c530eca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 -- 172.21.15.22:0/1687313253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c530c04620 con 0x55c530ebfd40 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0ddffff640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ec9ed0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009610 tx=0x7f0dd0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d585a1c11a7b607 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1687313253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0020070 con 0x55c530ebfd40 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1687313253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0dd009a6f0 con 0x55c530ebfd40 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.594+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0dd00a3790 con 0x55c530ebfd40 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1687313253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd0082050 con 0x55c530ebfd40 2024-03-08T23:58:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 10 client.16974 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c530edb910 con 0x55c530ebfd40 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1687313253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dd00a3790 con 0x55c530ebfd40 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1687313253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0dd0010460 con 0x55c530ebfd40 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 10 client.16974 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 20 client.16974 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530edcc60 0x55c530edf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 4 client.16974 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c530e30b30 con 0x55c530edcc60 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.595+0000 7f0de659c640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530edcc60 0x55c530edf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.596+0000 7f0de659c640 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530edcc60 0x55c530edf050 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb80846456e97641 server_cookie=e7d26048aa18ae62 in_seq=0 out_seq=0 2024-03-08T23:58:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.596+0000 7f0dcdffb640 10 client.16974 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1687313253 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c530e30b30 con 0x55c530edcc60 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0dcdffb640 10 client.16974 handle_command_reply: tid=0 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 client.16974 shutdown 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 msgr2=0x55c530ec9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ec9ed0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009610 tx=0x7f0dd0000bc0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 shutdown_connections 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530edcc60 0x55c530edf050 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ecc960 0x55c530eca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0dcdffb640 0 client.16974 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebfd40 0x55c530ec9ed0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 --2- 172.21.15.22:0/1687313253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ec9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 >> 172.21.15.22:0/1687313253 conn(0x55c530e865b0 msgr2=0x55c530ebba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 shutdown_connections 2024-03-08T23:58:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.597+0000 7f0def78d740 1 -- 172.21.15.22:0/1687313253 wait complete. 2024-03-08T23:58:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.598+0000 7f0def78d740 20 client.16974 trim_cache size 0 max 16384 2024-03-08T23:58:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 Processor -- start 2024-03-08T23:58:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 -- start start 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ecc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ec6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ecc960 0x55c530ec7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81ae710 con 0x55c530ecc960 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81ac2b0 con 0x55c530ebfd40 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0def78d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81aa350 con 0x55c530ebefd0 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ecc960 0x55c530ec7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ecc960 0x55c530ec7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54120/0 (socket says 172.21.15.22:54120) 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0de6d9d640 1 -- 172.21.15.22:0/3975956112 learned_addr learned my addr 172.21.15.22:0/3975956112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.611+0000 7f0de659c640 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ecc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0ddffff640 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ec6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0de6d9d640 1 -- 172.21.15.22:0/3975956112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 msgr2=0x55c530ecc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ecc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0de6d9d640 1 -- 172.21.15.22:0/3975956112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 msgr2=0x55c530ec6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ec6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0de6d9d640 1 -- 172.21.15.22:0/3975956112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c530ebe2a0 con 0x55c530ecc960 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0ddffff640 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ec6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ecc960 0x55c530ec7150 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0de000fc70 tx=0x7f0de001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57293c78cbdec5fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0dcffff640 1 -- 172.21.15.22:0/3975956112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de001a040 con 0x55c530ecc960 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0dcffff640 1 -- 172.21.15.22:0/3975956112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0de0018040 con 0x55c530ecc960 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.612+0000 7f0dcffff640 1 -- 172.21.15.22:0/3975956112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0028630 con 0x55c530ecc960 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- 172.21.15.22:0/3975956112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ecc960 msgr2=0x55c530ec7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ecc960 0x55c530ec7150 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0de000fc70 tx=0x7f0de001f000 comp rx=0 tx=0).stop 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- 172.21.15.22:0/3975956112 shutdown_connections 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ecc960 0x55c530ec7150 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ec6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 --2- 172.21.15.22:0/3975956112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ebefd0 0x55c530ecc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- 172.21.15.22:0/3975956112 >> 172.21.15.22:0/3975956112 conn(0x55c530e865b0 msgr2=0x55c530ec8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- 172.21.15.22:0/3975956112 shutdown_connections 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- 172.21.15.22:0/3975956112 wait complete. 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- start start 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 Processor -- start 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebefd0 0x55c530ed3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ed23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ecc960 0x55c530ed2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81aa950 con 0x55c530ebefd0 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81abc50 con 0x55c530ebfd40 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81aaf90 con 0x55c530ecc960 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0def78d740 10 client.0 _ll_register_callbacks cb 0x7f0dd0096cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0de659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebefd0 0x55c530ed3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0de659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebefd0 0x55c530ed3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54130/0 (socket says 172.21.15.22:54130) 2024-03-08T23:58:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.613+0000 7f0de659c640 1 -- 172.21.15.22:0/1566402645 learned_addr learned my addr 172.21.15.22:0/1566402645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0de6d9d640 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ecc960 0x55c530ed2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0ddffff640 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ed23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0de659c640 1 -- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ecc960 msgr2=0x55c530ed2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0de659c640 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ecc960 0x55c530ed2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0de659c640 1 -- 172.21.15.22:0/1566402645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 msgr2=0x55c530ed23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0de659c640 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ed23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0de659c640 1 -- 172.21.15.22:0/1566402645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c530f32e00 con 0x55c530ebefd0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0de659c640 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebefd0 0x55c530ed3f50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80096c0 tx=0x7f0dd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685a5474e355fc1b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1566402645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd802ccc0 con 0x55c530ebefd0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1566402645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0dd8049410 con 0x55c530ebefd0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0de0018040 con 0x55c530ebefd0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1566402645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd805b050 con 0x55c530ebefd0 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0def78d740 10 client.16980 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c530f34010 con 0x55c530ebefd0 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.614+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1566402645 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0de0018040 con 0x55c530ebefd0 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1566402645 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0dd8004ec0 con 0x55c530ebefd0 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0def78d740 10 client.16980 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0def78d740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0def78d740 20 client.16980 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0def78d740 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530f34730 0x55c530f36b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0def78d740 4 client.16980 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c530e2ae80 con 0x55c530f34730 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0ddffff640 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530f34730 0x55c530f36b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0dcdffb640 10 client.16980 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.615+0000 7f0ddffff640 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530f34730 0x55c530f36b20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5253cbc52d8010fd server_cookie=e0f47a24212f734a in_seq=0 out_seq=0 2024-03-08T23:58:37.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 383680 11.08 MB/sec execute 45 sec latency 29.998 ms 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0dcdffb640 1 -- 172.21.15.22:0/1566402645 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c530e2ae80 con 0x55c530f34730 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0dcdffb640 10 client.16980 handle_command_reply: tid=0 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 client.16980 shutdown 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebefd0 msgr2=0x55c530ed3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebefd0 0x55c530ed3f50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80096c0 tx=0x7f0dd8004270 comp rx=0 tx=0).stop 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 shutdown_connections 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c530f34730 0x55c530f36b20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c530ecc960 0x55c530ed2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c530ebfd40 0x55c530ed23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 --2- 172.21.15.22:0/1566402645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c530ebefd0 0x55c530ed3f50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 >> 172.21.15.22:0/1566402645 conn(0x55c530e865b0 msgr2=0x55c530eb85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.899+0000 7f0dcdffb640 0 client.16980 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.900+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 shutdown_connections 2024-03-08T23:58:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.900+0000 7f0def78d740 1 -- 172.21.15.22:0/1566402645 wait complete. 2024-03-08T23:58:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.900+0000 7f0def78d740 20 client.16980 trim_cache size 0 max 16384 2024-03-08T23:58:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:58:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:58:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:58:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:58:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.900+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0dd40a2a90 msgr2=0x7f0dd40a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.900+0000 7f0decad1640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0dd40a2a90 0x7f0dd40a4f50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009fd0 tx=0x7f0de0011bc0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.900+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8100e00 msgr2=0x7f0de819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.900+0000 7f0decad1640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8100e00 0x7f0de819dd20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000d950 tx=0x7f0dd000de10 comp rx=0 tx=0).stop 2024-03-08T23:58:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 shutdown_connections 2024-03-08T23:58:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0dd40a2a90 0x7f0dd40a4f50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de8105190 0x7f0de8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0de8100e00 0x7f0de819dd20 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 --2- 172.21.15.22:0/2508039213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8100440 0x7f0de819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 >> 172.21.15.22:0/2508039213 conn(0x7f0de8077040 msgr2=0x7f0de8068bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 shutdown_connections 2024-03-08T23:58:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:37.901+0000 7f0decad1640 1 -- 172.21.15.22:0/2508039213 wait complete. 2024-03-08T23:58:38.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 386323 11.13 MB/sec execute 46 sec latency 28.876 ms 2024-03-08T23:58:38.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:38 smithi022 ceph-mon[31894]: pgmap v128: 129 pgs: 129 active+clean; 56 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 396 op/s 2024-03-08T23:58:38.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:38 smithi062 ceph-mon[36129]: pgmap v128: 129 pgs: 129 active+clean; 56 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 396 op/s 2024-03-08T23:58:38.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:38 smithi192 ceph-mon[35995]: pgmap v128: 129 pgs: 129 active+clean; 56 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 396 op/s 2024-03-08T23:58:39.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 388335 11.13 MB/sec execute 47 sec latency 12.463 ms 2024-03-08T23:58:40.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 390815 11.19 MB/sec execute 48 sec latency 14.561 ms 2024-03-08T23:58:40.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:40 smithi022 ceph-mon[31894]: pgmap v129: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 484 op/s 2024-03-08T23:58:40.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:40 smithi062 ceph-mon[36129]: pgmap v129: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 484 op/s 2024-03-08T23:58:40.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:40 smithi192 ceph-mon[35995]: pgmap v129: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 484 op/s 2024-03-08T23:58:41.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 393842 11.32 MB/sec execute 49 sec latency 18.036 ms 2024-03-08T23:58:42.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 394950 11.16 MB/sec execute 50 sec latency 118.763 ms 2024-03-08T23:58:42.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:42 smithi022 ceph-mon[31894]: pgmap v130: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 522 op/s 2024-03-08T23:58:42.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:42 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:58:42.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:42 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:58:42.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:42 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:58:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:42 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:58:42.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:42 smithi062 ceph-mon[36129]: pgmap v130: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 522 op/s 2024-03-08T23:58:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:42 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:58:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:42 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:58:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:42 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:58:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:42 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:58:42.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:42 smithi192 ceph-mon[35995]: pgmap v130: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 522 op/s 2024-03-08T23:58:42.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:42 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:58:42.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:42 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:58:42.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:42 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:58:42.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:42 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:58:43.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 397744 11.28 MB/sec execute 51 sec latency 26.161 ms 2024-03-08T23:58:43.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:58:43.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 -- 172.21.15.22:0/86414857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784101ea0 msgr2=0x7f0784105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 --2- 172.21.15.22:0/86414857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784101ea0 0x7f0784105ee0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0778009900 tx=0x7f077802bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 -- 172.21.15.22:0/86414857 shutdown_connections 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 --2- 172.21.15.22:0/86414857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07841065c0 0x7f078410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 --2- 172.21.15.22:0/86414857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784101ea0 0x7f0784105ee0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 --2- 172.21.15.22:0/86414857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07841014f0 0x7f07841018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 -- 172.21.15.22:0/86414857 >> 172.21.15.22:0/86414857 conn(0x7f07840fcd90 msgr2=0x7f07840ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 -- 172.21.15.22:0/86414857 shutdown_connections 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.088+0000 7f07891a9640 1 -- 172.21.15.22:0/86414857 wait complete. 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.089+0000 7f07891a9640 1 Processor -- start 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.089+0000 7f07891a9640 1 -- start start 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f07891a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07841014f0 0x7f078419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f07891a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07841065c0 0x7f078419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f07891a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784197a70 0x7f0784197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f07891a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0784111650 con 0x7f0784197a70 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f07891a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841114d0 con 0x7f07841014f0 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f07891a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841117d0 con 0x7f07841065c0 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f078359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784197a70 0x7f0784197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f0782d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07841014f0 0x7f078419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f078359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784197a70 0x7f0784197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53706/0 (socket says 172.21.15.22:53706) 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f078359e640 1 -- 172.21.15.22:0/1223471220 learned_addr learned my addr 172.21.15.22:0/1223471220 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f078259c640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07841065c0 0x7f078419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f0782d9d640 1 -- 172.21.15.22:0/1223471220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07841065c0 msgr2=0x7f078419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f0782d9d640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07841065c0 0x7f078419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f0782d9d640 1 -- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784197a70 msgr2=0x7f0784197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f0782d9d640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784197a70 0x7f0784197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f0782d9d640 1 -- 172.21.15.22:0/1223471220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784198590 con 0x7f07841014f0 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f078359e640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784197a70 0x7f0784197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f0782d9d640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07841014f0 0x7f078419d8c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fd0 tx=0x7f077400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f076bfff640 1 -- 172.21.15.22:0/1223471220 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774004b20 con 0x7f07841014f0 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0784198880 con 0x7f07841014f0 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.090+0000 7f076bfff640 1 -- 172.21.15.22:0/1223471220 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0774004cc0 con 0x7f07841014f0 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.091+0000 7f076bfff640 1 -- 172.21.15.22:0/1223471220 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740107b0 con 0x7f07841014f0 2024-03-08T23:58:44.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.091+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07841a4ad0 con 0x7f07841014f0 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f076bfff640 1 -- 172.21.15.22:0/1223471220 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0774005020 con 0x7f07841014f0 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 Processor -- start 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 -- start start 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f076bfff640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f076c0a2970 0x7f076c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56076372d760 0x56076372db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763736720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078359e640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f076c0a2970 0x7f076c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560763736c60 0x560763739050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560763690e00 con 0x56076372e080 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560763639b10 con 0x560763736c60 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078bd85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076368cea0 con 0x56076372d760 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f076bfff640 1 -- 172.21.15.22:0/1223471220 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f07740cf960 con 0x7f07841014f0 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763736720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763736720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53708/0 (socket says 172.21.15.22:53708) 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078259c640 1 -- 172.21.15.22:0/1940963862 learned_addr learned my addr 172.21.15.22:0/1940963862 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f0782d9d640 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56076372d760 0x56076372db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.093+0000 7f078359e640 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560763736c60 0x560763739050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f078259c640 1 -- 172.21.15.22:0/1940963862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56076372d760 msgr2=0x56076372db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f078259c640 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56076372d760 0x56076372db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f078259c640 1 -- 172.21.15.22:0/1940963862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560763736c60 msgr2=0x560763739050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f078259c640 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560763736c60 0x560763739050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f078259c640 1 -- 172.21.15.22:0/1940963862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076346f7d0 con 0x56076372e080 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f078359e640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f076c0a2970 0x7f076c0a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f077c013860 tx=0x7f077c013c60 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f078259c640 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763736720 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f078406a9c0 tx=0x7f0778005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3116fa7e72d3d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f07697fa640 1 -- 172.21.15.22:0/1940963862 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778042070 con 0x56076372e080 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f07697fa640 1 -- 172.21.15.22:0/1940963862 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0778004030 con 0x56076372e080 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.094+0000 7f07697fa640 1 -- 172.21.15.22:0/1940963862 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778004c70 con 0x56076372e080 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- 172.21.15.22:0/1940963862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 msgr2=0x560763736720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763736720 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f078406a9c0 tx=0x7f0778005be0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- 172.21.15.22:0/1940963862 shutdown_connections 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560763736c60 0x560763739050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763736720 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 --2- 172.21.15.22:0/1940963862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56076372d760 0x56076372db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- 172.21.15.22:0/1940963862 >> 172.21.15.22:0/1940963862 conn(0x560763727e90 msgr2=0x560763728270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- 172.21.15.22:0/1940963862 shutdown_connections 2024-03-08T23:58:44.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- 172.21.15.22:0/1940963862 wait complete. 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- start start 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 Processor -- start 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x5607637363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763733740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x560763733c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607637399f0 con 0x56076372e080 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076364d0a0 con 0x56076372d760 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.095+0000 7f078bd85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560763739b70 con 0x560763736c60 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078bd85740 10 client.0 _ll_register_callbacks cb 0x56076364ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763733740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763733740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53722/0 (socket says 172.21.15.22:53722) 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 -- 172.21.15.22:0/2203723572 learned_addr learned my addr 172.21.15.22:0/2203723572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f0782d9d640 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x5607637363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078359e640 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x560763733c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 -- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 msgr2=0x560763733c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x560763733c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 -- 172.21.15.22:0/2203723572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 msgr2=0x5607637363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x5607637363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 -- 172.21.15.22:0/2203723572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607633a53f0 con 0x56076372e080 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078259c640 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763733740 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0778036040 tx=0x7f0778004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700ee630041da74 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f0766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f0766ff5640 1 -- 172.21.15.22:0/2203723572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077804e060 con 0x56076372e080 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f0766ff5640 1 -- 172.21.15.22:0/2203723572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f077802ca90 con 0x56076372e080 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0778004030 con 0x56076372e080 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f0766ff5640 1 -- 172.21.15.22:0/2203723572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0778055050 con 0x56076372e080 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078bd85740 10 client.16992 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.096+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5607633650f0 con 0x56076372e080 2024-03-08T23:58:44.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f0766ff5640 1 -- 172.21.15.22:0/2203723572 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0778004030 con 0x56076372e080 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f0766ff5640 1 -- 172.21.15.22:0/2203723572 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07780038f0 con 0x56076372e080 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f078bd85740 10 client.16992 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f078bd85740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f078bd85740 20 client.16992 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f078bd85740 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560763746570 0x560763748960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f078bd85740 4 client.16992 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56076369e340 con 0x560763746570 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f0782d9d640 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560763746570 0x560763748960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f0766ff5640 10 client.16992 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.097+0000 7f0782d9d640 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560763746570 0x560763748960 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d8eae998ed2600 server_cookie=1f4474919170c9ee in_seq=0 out_seq=0 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.099+0000 7f0766ff5640 1 -- 172.21.15.22:0/2203723572 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56076369e340 con 0x560763746570 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.099+0000 7f0766ff5640 10 client.16992 handle_command_reply: tid=0 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.099+0000 7f078bd85740 1 client.16992 shutdown 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.099+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 msgr2=0x560763733740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.099+0000 7f078bd85740 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763733740 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0778036040 tx=0x7f0778004490 comp rx=0 tx=0).stop 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 shutdown_connections 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560763746570 0x560763748960 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x560763733c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f0766ff5640 0 client.16992 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x560763733740 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 --2- 172.21.15.22:0/2203723572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x5607637363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 >> 172.21.15.22:0/2203723572 conn(0x560763727e90 msgr2=0x560763729be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 shutdown_connections 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 1 -- 172.21.15.22:0/2203723572 wait complete. 2024-03-08T23:58:44.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.100+0000 7f078bd85740 20 client.16992 trim_cache size 0 max 16384 2024-03-08T23:58:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.115+0000 7f078bd85740 1 Processor -- start 2024-03-08T23:58:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.115+0000 7f078bd85740 1 -- start start 2024-03-08T23:58:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078bd85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x560763735e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078bd85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x5607637363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078bd85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078bd85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841b08c0 con 0x56076372e080 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078bd85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841ae460 con 0x56076372d760 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078bd85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841ac500 con 0x560763736c60 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x5607637363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x5607637363b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53726/0 (socket says 172.21.15.22:53726) 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 -- 172.21.15.22:0/2970041912 learned_addr learned my addr 172.21.15.22:0/2970041912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f0782d9d640 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x560763735e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078359e640 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 -- 172.21.15.22:0/2970041912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 msgr2=0x56076373dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 -- 172.21.15.22:0/2970041912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 msgr2=0x560763735e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x560763735e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 -- 172.21.15.22:0/2970041912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076372deb0 con 0x56076372e080 2024-03-08T23:58:44.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078359e640 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f078259c640 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x5607637363b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f077802bc70 tx=0x7f0778004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d988ce3de322863 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.116+0000 7f0768ff9640 1 -- 172.21.15.22:0/2970041912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077804e030 con 0x56076372e080 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f0768ff9640 1 -- 172.21.15.22:0/2970041912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0778003ad0 con 0x56076372e080 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f0768ff9640 1 -- 172.21.15.22:0/2970041912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778042070 con 0x56076372e080 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 -- 172.21.15.22:0/2970041912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 msgr2=0x5607637363b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x5607637363b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f077802bc70 tx=0x7f0778004800 comp rx=0 tx=0).stop 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 -- 172.21.15.22:0/2970041912 shutdown_connections 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x5607637363b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 --2- 172.21.15.22:0/2970041912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x560763735e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 -- 172.21.15.22:0/2970041912 >> 172.21.15.22:0/2970041912 conn(0x560763727e90 msgr2=0x560763732010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 -- 172.21.15.22:0/2970041912 shutdown_connections 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.117+0000 7f078bd85740 1 -- 172.21.15.22:0/2970041912 wait complete. 2024-03-08T23:58:44.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 -- start start 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 Processor -- start 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x56076373e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x56076373c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841acb00 con 0x56076372e080 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841ade00 con 0x56076372d760 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841ad140 con 0x560763736c60 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078bd85740 10 client.0 _ll_register_callbacks cb 0x7f0774005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x56076373e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x56076373c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x56076373e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:39960/0 (socket says 172.21.15.22:39960) 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x56076373c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53732/0 (socket says 172.21.15.22:53732) 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 -- 172.21.15.22:0/1203181978 learned_addr learned my addr 172.21.15.22:0/1203181978 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078359e640 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 -- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 msgr2=0x56076373cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 -- 172.21.15.22:0/1203181978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 msgr2=0x56076373c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x56076373c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f0782d9d640 1 -- 172.21.15.22:0/1203181978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56076379c710 con 0x56076372d760 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.118+0000 7f078259c640 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x56076373c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f0782d9d640 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x56076373e260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f077409b920 tx=0x7f077409bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b449f4baf59f352d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f0766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f0766ff5640 1 -- 172.21.15.22:0/1203181978 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774005590 con 0x56076372d760 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0778003ad0 con 0x56076372d760 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f0766ff5640 1 -- 172.21.15.22:0/1203181978 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0774098290 con 0x56076372d760 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f0766ff5640 1 -- 172.21.15.22:0/1203181978 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07740512d0 con 0x56076372d760 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 10 client.26447 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56076379d8f0 con 0x56076372d760 2024-03-08T23:58:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f0766ff5640 1 -- 172.21.15.22:0/1203181978 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0778003ad0 con 0x56076372d760 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f0766ff5640 1 -- 172.21.15.22:0/1203181978 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07740b4ce0 con 0x56076372d760 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 10 client.26447 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 20 client.26447 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56076379df70 0x5607637a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 4 client.26447 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560763698690 con 0x56076379df70 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.119+0000 7f078259c640 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56076379df70 0x5607637a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078259c640 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56076379df70 0x5607637a0360 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e03c027b86a746a server_cookie=5bb2f3b1146faa78 in_seq=0 out_seq=0 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f0766ff5640 10 client.26447 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f0766ff5640 1 -- 172.21.15.22:0/1203181978 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560763698690 con 0x56076379df70 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f0766ff5640 10 client.26447 handle_command_reply: tid=0 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 client.26447 shutdown 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 msgr2=0x56076373e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x56076373e260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f077409b920 tx=0x7f077409bce0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 shutdown_connections 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56076379df70 0x5607637a0360 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560763736c60 0x56076373cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56076372e080 0x56076373c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f0766ff5640 0 client.26447 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 --2- 172.21.15.22:0/1203181978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56076372d760 0x56076373e260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.120+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 >> 172.21.15.22:0/1203181978 conn(0x560763727e90 msgr2=0x56076372e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 shutdown_connections 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f078bd85740 1 -- 172.21.15.22:0/1203181978 wait complete. 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f078bd85740 20 client.26447 trim_cache size 0 max 16384 2024-03-08T23:58:44.123 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:58:44.124 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:58:44.124 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:58:44.124 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f076c0a2970 msgr2=0x7f076c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f076c0a2970 0x7f076c0a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f077c013860 tx=0x7f077c013c60 comp rx=0 tx=0).stop 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07841014f0 msgr2=0x7f078419d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07841014f0 0x7f078419d8c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fd0 tx=0x7f077400ef90 comp rx=0 tx=0).stop 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 shutdown_connections 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f076c0a2970 0x7f076c0a4e30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0784197a70 0x7f0784197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07841065c0 0x7f078419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 --2- 172.21.15.22:0/1223471220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07841014f0 0x7f078419d8c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 >> 172.21.15.22:0/1223471220 conn(0x7f07840fcd90 msgr2=0x7f07840fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 shutdown_connections 2024-03-08T23:58:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.121+0000 7f07891a9640 1 -- 172.21.15.22:0/1223471220 wait complete. 2024-03-08T23:58:44.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:58:44.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 -- 172.21.15.22:0/2819757712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100e10 msgr2=0x7fe3f8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2819757712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100e10 0x7fe3f8104e50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009900 tx=0x7fe3ec02bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 -- 172.21.15.22:0/2819757712 shutdown_connections 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2819757712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f8105530 0x7fe3f810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2819757712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100e10 0x7fe3f8104e50 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2819757712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8100460 0x7fe3f8100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 -- 172.21.15.22:0/2819757712 >> 172.21.15.22:0/2819757712 conn(0x7fe3f80772f0 msgr2=0x7fe3f8077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 -- 172.21.15.22:0/2819757712 shutdown_connections 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 -- 172.21.15.22:0/2819757712 wait complete. 2024-03-08T23:58:44.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.239+0000 7fe3fef53640 1 Processor -- start 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fef53640 1 -- start start 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fef53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100460 0x7fe3f8101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fef53640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8105530 0x7fe3f8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fef53640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f8102970 0x7fe3f81a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fef53640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f81105c0 con 0x7fe3f8100460 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fef53640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8110440 con 0x7fe3f8102970 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fef53640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8110740 con 0x7fe3f8105530 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100460 0x7fe3f8101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100460 0x7fe3f8101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53744/0 (socket says 172.21.15.22:53744) 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 -- 172.21.15.22:0/2289704435 learned_addr learned my addr 172.21.15.22:0/2289704435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fd4f0640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f8102970 0x7fe3f81a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3f7fff640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8105530 0x7fe3f8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 -- 172.21.15.22:0/2289704435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8105530 msgr2=0x7fe3f8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8105530 0x7fe3f8102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 -- 172.21.15.22:0/2289704435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f8102970 msgr2=0x7fe3f81a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f8102970 0x7fe3f81a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 -- 172.21.15.22:0/2289704435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f81a64c0 con 0x7fe3f8100460 2024-03-08T23:58:44.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fd4f0640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f8102970 0x7fe3f81a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.240+0000 7fe3fccef640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100460 0x7fe3f8101ef0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000eb90 tx=0x7fe3f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.241+0000 7fe3f5ffb640 1 -- 172.21.15.22:0/2289704435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0004d20 con 0x7fe3f8100460 2024-03-08T23:58:44.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.241+0000 7fe3f5ffb640 1 -- 172.21.15.22:0/2289704435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3f000ee90 con 0x7fe3f8100460 2024-03-08T23:58:44.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.241+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f81a67b0 con 0x7fe3f8100460 2024-03-08T23:58:44.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.241+0000 7fe3f5ffb640 1 -- 172.21.15.22:0/2289704435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0010830 con 0x7fe3f8100460 2024-03-08T23:58:44.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.241+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f81a6cd0 con 0x7fe3f8100460 2024-03-08T23:58:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.242+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f80680e0 con 0x7fe3f8100460 2024-03-08T23:58:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.243+0000 7fe3f5ffb640 1 -- 172.21.15.22:0/2289704435 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe3f0005020 con 0x7fe3f8100460 2024-03-08T23:58:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.243+0000 7fe3f5ffb640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3e00a2a20 0x7fe3e00a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.244+0000 7fe3f7fff640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3e00a2a20 0x7fe3e00a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.244+0000 7fe3f5ffb640 1 -- 172.21.15.22:0/2289704435 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fe3f00cf7e0 con 0x7fe3f8100460 2024-03-08T23:58:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.244+0000 7fe3f7fff640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3e00a2a20 0x7fe3e00a4ee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009230 tx=0x7fe3ec005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.248+0000 7fe3f5ffb640 1 -- 172.21.15.22:0/2289704435 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe3f009c140 con 0x7fe3f8100460 2024-03-08T23:58:44.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.485+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3f8100e10 con 0x7fe3f8100460 2024-03-08T23:58:44.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.486+0000 7fe3f5ffb640 1 -- 172.21.15.22:0/2289704435 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fe3f00afe20 con 0x7fe3f8100460 2024-03-08T23:58:44.481 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:58:44.482 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:11.445096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26288":{"gid":26288,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2009262827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2009262827},{"type":"v1","addr":"172.21.15.62:6837","nonce":2009262827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:58:44.482 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 31 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3e00a2a20 msgr2=0x7fe3e00a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3e00a2a20 0x7fe3e00a4ee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009230 tx=0x7fe3ec005fd0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100460 msgr2=0x7fe3f8101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100460 0x7fe3f8101ef0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000eb90 tx=0x7fe3f000c6a0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 shutdown_connections 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3e00a2a20 0x7fe3e00a4ee0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f8102970 0x7fe3f81a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8105530 0x7fe3f8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 --2- 172.21.15.22:0/2289704435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3f8100460 0x7fe3f8101ef0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 >> 172.21.15.22:0/2289704435 conn(0x7fe3f80772f0 msgr2=0x7fe3f8104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 shutdown_connections 2024-03-08T23:58:44.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.488+0000 7fe3fef53640 1 -- 172.21.15.22:0/2289704435 wait complete. 2024-03-08T23:58:44.495 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:58:44.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 -- 172.21.15.22:0/4158806373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda141062a0 msgr2=0x7fda14106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 --2- 172.21.15.22:0/4158806373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda141062a0 0x7fda14106680 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fda08009900 tx=0x7fda0802bc80 comp rx=0 tx=0).stop 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 -- 172.21.15.22:0/4158806373 shutdown_connections 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 --2- 172.21.15.22:0/4158806373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 0x7fda1410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 --2- 172.21.15.22:0/4158806373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda14100440 0x7fda141008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 --2- 172.21.15.22:0/4158806373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda141062a0 0x7fda14106680 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 -- 172.21.15.22:0/4158806373 >> 172.21.15.22:0/4158806373 conn(0x7fda14077010 msgr2=0x7fda14077420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 -- 172.21.15.22:0/4158806373 shutdown_connections 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 -- 172.21.15.22:0/4158806373 wait complete. 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.606+0000 7fda1ade1640 1 Processor -- start 2024-03-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1ade1640 1 -- start start 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1ade1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14100440 0x7fda1419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1ade1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 0x7fda1419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1ade1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141062a0 0x7fda141979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1ade1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141105a0 con 0x7fda14100440 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1ade1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14110420 con 0x7fda141062a0 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1ade1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14110720 con 0x7fda14100e00 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda18b7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14100440 0x7fda1419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda13fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 0x7fda1419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda13fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 0x7fda1419dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38024/0 (socket says 172.21.15.22:38024) 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda18b7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14100440 0x7fda1419d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53764/0 (socket says 172.21.15.22:53764) 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda13fff640 1 -- 172.21.15.22:0/2374610101 learned_addr learned my addr 172.21.15.22:0/2374610101 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda1937e640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141062a0 0x7fda141979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda18b7d640 1 -- 172.21.15.22:0/2374610101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 msgr2=0x7fda1419dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda18b7d640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 0x7fda1419dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda18b7d640 1 -- 172.21.15.22:0/2374610101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141062a0 msgr2=0x7fda141979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda18b7d640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141062a0 0x7fda141979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda18b7d640 1 -- 172.21.15.22:0/2374610101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda14198150 con 0x7fda14100440 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.607+0000 7fda13fff640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 0x7fda1419dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.608+0000 7fda18b7d640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14100440 0x7fda1419d820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda0802cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.608+0000 7fda11ffb640 1 -- 172.21.15.22:0/2374610101 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08041070 con 0x7fda14100440 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.608+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda141983e0 con 0x7fda14100440 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.608+0000 7fda11ffb640 1 -- 172.21.15.22:0/2374610101 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda08004450 con 0x7fda14100440 2024-03-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.608+0000 7fda11ffb640 1 -- 172.21.15.22:0/2374610101 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08034440 con 0x7fda14100440 2024-03-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.608+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda141a46e0 con 0x7fda14100440 2024-03-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda11ffb640 1 -- 172.21.15.22:0/2374610101 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fda08044070 con 0x7fda14100440 2024-03-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 Processor -- start 2024-03-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 -- start start 2024-03-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0eac0 0x565131c17160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c176a0 0x565131c19a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565131bd5360 con 0x565131c0e1a0 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x565131b71ec0 con 0x565131c0eac0 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.611+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565131b1ad30 con 0x565131c176a0 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c0e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53780/0 (socket says 172.21.15.22:53780) 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 -- 172.21.15.22:0/810193500 learned_addr learned my addr 172.21.15.22:0/810193500 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda13fff640 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c176a0 0x565131c19a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda18b7d640 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0eac0 0x565131c17160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda11ffb640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda040a2970 0x7fda040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 -- 172.21.15.22:0/810193500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c176a0 msgr2=0x565131c19a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c176a0 0x565131c19a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 -- 172.21.15.22:0/810193500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0eac0 msgr2=0x565131c17160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0eac0 0x565131c17160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 -- 172.21.15.22:0/810193500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565131b19bc0 con 0x565131c0e1a0 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda13fff640 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c176a0 0x565131c19a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda18b7d640 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0eac0 0x565131c17160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda040a2970 0x7fda040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda11ffb640 1 -- 172.21.15.22:0/2374610101 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fda080f0a90 con 0x7fda14100440 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fda1937e640 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c0e580 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fda0c010440 tx=0x7fda0c010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9431615c1b1a9c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fd9feffd640 1 -- 172.21.15.22:0/810193500 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c01d400 con 0x565131c0e1a0 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fd9feffd640 1 -- 172.21.15.22:0/810193500 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda0c01da80 con 0x565131c0e1a0 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.612+0000 7fd9feffd640 1 -- 172.21.15.22:0/810193500 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c01c840 con 0x565131c0e1a0 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 -- 172.21.15.22:0/810193500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 msgr2=0x565131c0e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c0e580 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fda0c010440 tx=0x7fda0c010900 comp rx=0 tx=0).stop 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1937e640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda040a2970 0x7fda040a4e30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda0c021000 tx=0x7fda0c01e000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 -- 172.21.15.22:0/810193500 shutdown_connections 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c176a0 0x565131c19a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0eac0 0x565131c17160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/810193500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c0e580 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 -- 172.21.15.22:0/810193500 >> 172.21.15.22:0/810193500 conn(0x565131c088d0 msgr2=0x565131c08cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 -- 172.21.15.22:0/810193500 shutdown_connections 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 -- 172.21.15.22:0/810193500 wait complete. 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.613+0000 7fda1d9bd740 1 -- start start 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 1 Processor -- start 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0e1a0 0x565131c14680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c14bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c176a0 0x565131c15100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565131b104a0 con 0x565131c176a0 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x565131b6df60 con 0x565131c0e1a0 2024-03-08T23:58:44.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565131c1a490 con 0x565131c0eac0 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1d9bd740 10 client.0 _ll_register_callbacks cb 0x565131b6bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda13fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c14bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1937e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c176a0 0x565131c15100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda18b7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0e1a0 0x565131c14680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda13fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c14bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38046/0 (socket says 172.21.15.22:38046) 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda13fff640 1 -- 172.21.15.22:0/732304138 learned_addr learned my addr 172.21.15.22:0/732304138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda18b7d640 1 -- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 msgr2=0x565131c14bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda18b7d640 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c14bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda18b7d640 1 -- 172.21.15.22:0/732304138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c176a0 msgr2=0x565131c15100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda18b7d640 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c176a0 0x565131c15100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda18b7d640 1 -- 172.21.15.22:0/732304138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651318863f0 con 0x565131c0e1a0 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda13fff640 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c14bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.614+0000 7fda1937e640 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c176a0 0x565131c15100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda18b7d640 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0e1a0 0x565131c14680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fda080edf50 tx=0x7fda08005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9635a66edfda220 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fd9f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fd9f3fff640 1 -- 172.21.15.22:0/732304138 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0804b070 con 0x565131c0e1a0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fd9f3fff640 1 -- 172.21.15.22:0/732304138 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fda080bd170 con 0x565131c0e1a0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda0c01da80 con 0x565131c0e1a0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fd9f3fff640 1 -- 172.21.15.22:0/732304138 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda080bcce0 con 0x565131c0e1a0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 10 client.26465 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5651319507d0 con 0x565131c0e1a0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fd9f3fff640 1 -- 172.21.15.22:0/732304138 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda0c01da80 con 0x565131c0e1a0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fd9f3fff640 1 -- 172.21.15.22:0/732304138 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda080c5170 con 0x565131c0e1a0 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 10 client.26465 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 10 client.26465 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 20 client.26465 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c27780 0x565131c29b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 4 client.26465 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.615+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565131b7f400 con 0x565131c27780 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.616+0000 7fda13fff640 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c27780 0x565131c29b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.616+0000 7fda13fff640 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c27780 0x565131c29b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3b85273257c69f6 server_cookie=2f65266a75e64e33 in_seq=0 out_seq=0 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.616+0000 7fd9f3fff640 10 client.26465 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fd9f3fff640 1 -- 172.21.15.22:0/732304138 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565131b7f400 con 0x565131c27780 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fd9f3fff640 10 client.26465 handle_command_reply: tid=0 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 client.26465 shutdown 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0e1a0 msgr2=0x565131c14680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0e1a0 0x565131c14680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fda080edf50 tx=0x7fda08005fd0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 shutdown_connections 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c27780 0x565131c29b90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c176a0 0x565131c15100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c14bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/732304138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c0e1a0 0x565131c14680 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fd9f3fff640 0 client.26465 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 >> 172.21.15.22:0/732304138 conn(0x565131c088d0 msgr2=0x565131c0a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 shutdown_connections 2024-03-08T23:58:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 1 -- 172.21.15.22:0/732304138 wait complete. 2024-03-08T23:58:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.617+0000 7fda1d9bd740 20 client.26465 trim_cache size 0 max 16384 2024-03-08T23:58:44.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 Processor -- start 2024-03-08T23:58:44.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 -- start start 2024-03-08T23:58:44.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c16db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141b0ab0 con 0x565131c0e1a0 2024-03-08T23:58:44.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141ae650 con 0x565131c176a0 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141ac6f0 con 0x565131c0eac0 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c16db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c16db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53786/0 (socket says 172.21.15.22:53786) 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 -- 172.21.15.22:0/1871381922 learned_addr learned my addr 172.21.15.22:0/1871381922 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda1937e640 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda13fff640 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 -- 172.21.15.22:0/1871381922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 msgr2=0x565131c13660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c13660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 -- 172.21.15.22:0/1871381922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 msgr2=0x565131c11900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c11900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda18b7d640 1 -- 172.21.15.22:0/1871381922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565131c08110 con 0x565131c0e1a0 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.629+0000 7fda13fff640 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda18b7d640 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c16db0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fda080bb1c0 tx=0x7fda08005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb76ae57eb906fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fd9fe7fc640 1 -- 172.21.15.22:0/1871381922 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0804b040 con 0x565131c0e1a0 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fd9fe7fc640 1 -- 172.21.15.22:0/1871381922 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda080f8040 con 0x565131c0e1a0 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fd9fe7fc640 1 -- 172.21.15.22:0/1871381922 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080bc870 con 0x565131c0e1a0 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1871381922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 msgr2=0x565131c16db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c16db0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fda080bb1c0 tx=0x7fda08005b70 comp rx=0 tx=0).stop 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1871381922 shutdown_connections 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1871381922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c16db0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1871381922 >> 172.21.15.22:0/1871381922 conn(0x565131c088d0 msgr2=0x565131c12f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.630+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1871381922 shutdown_connections 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1871381922 wait complete. 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 -- start start 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 Processor -- start 2024-03-08T23:58:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c1d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c1d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c1dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141accf0 con 0x565131c0e1a0 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141adff0 con 0x565131c176a0 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141ad330 con 0x565131c0eac0 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1d9bd740 10 client.0 _ll_register_callbacks cb 0x7fda0c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda18b7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c1d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda18b7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c1d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53800/0 (socket says 172.21.15.22:53800) 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda18b7d640 1 -- 172.21.15.22:0/1168866832 learned_addr learned my addr 172.21.15.22:0/1168866832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda1937e640 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c1dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.631+0000 7fda13fff640 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c1d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda18b7d640 1 -- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 msgr2=0x565131c1d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda18b7d640 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c1d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda18b7d640 1 -- 172.21.15.22:0/1168866832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 msgr2=0x565131c1dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda18b7d640 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c1dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda18b7d640 1 -- 172.21.15.22:0/1168866832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565131c7ddb0 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda18b7d640 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c1d280 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fda080b98e0 tx=0x7fda08005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594d43c55a0509ca server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fd9f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fd9f3fff640 1 -- 172.21.15.22:0/1168866832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0804b040 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fd9f3fff640 1 -- 172.21.15.22:0/1168866832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fda080bc360 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda080f8040 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fd9f3fff640 1 -- 172.21.15.22:0/1168866832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda080a3050 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda1d9bd740 10 client.17022 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x565131c7ddb0 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fd9f3fff640 1 -- 172.21.15.22:0/1168866832 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda080f8040 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fd9f3fff640 1 -- 172.21.15.22:0/1168866832 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda080bbdd0 con 0x565131c0e1a0 2024-03-08T23:58:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda1d9bd740 10 client.17022 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda1d9bd740 10 client.17022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.632+0000 7fda1d9bd740 20 client.17022 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c7f270 0x565131c81660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 4 client.17022 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565131b79750 con 0x565131c7f270 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda13fff640 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c7f270 0x565131c81660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda13fff640 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c7f270 0x565131c81660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8729e0ff003fdce6 server_cookie=8d362a89096ec2fe in_seq=0 out_seq=0 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fd9f3fff640 10 client.17022 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fd9f3fff640 1 -- 172.21.15.22:0/1168866832 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565131b79750 con 0x565131c7f270 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fd9f3fff640 10 client.17022 handle_command_reply: tid=0 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 client.17022 shutdown 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 msgr2=0x565131c1d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c1d280 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fda080b98e0 tx=0x7fda08005be0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 shutdown_connections 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x565131c7f270 0x565131c81660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565131c176a0 0x565131c1dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565131c0eac0 0x565131c1d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fd9f3fff640 0 client.17022 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 --2- 172.21.15.22:0/1168866832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565131c0e1a0 0x565131c1d280 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.633+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 >> 172.21.15.22:0/1168866832 conn(0x565131c088d0 msgr2=0x565131c09d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 shutdown_connections 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1d9bd740 1 -- 172.21.15.22:0/1168866832 wait complete. 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1d9bd740 20 client.17022 trim_cache size 0 max 16384 2024-03-08T23:58:44.636 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:58:44.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda040a2970 msgr2=0x7fda040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda040a2970 0x7fda040a4e30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fda0c021000 tx=0x7fda0c01e000 comp rx=0 tx=0).stop 2024-03-08T23:58:44.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14100440 msgr2=0x7fda1419d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14100440 0x7fda1419d820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda0802cbe0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 shutdown_connections 2024-03-08T23:58:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fda040a2970 0x7fda040a4e30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141062a0 0x7fda141979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda14100e00 0x7fda1419dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 --2- 172.21.15.22:0/2374610101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda14100440 0x7fda1419d820 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 >> 172.21.15.22:0/2374610101 conn(0x7fda14077010 msgr2=0x7fda140ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 shutdown_connections 2024-03-08T23:58:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.634+0000 7fda1ade1640 1 -- 172.21.15.22:0/2374610101 wait complete. 2024-03-08T23:58:44.640 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:58:44.641 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:58:44.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 400483 11.31 MB/sec execute 52 sec latency 18.577 ms 2024-03-08T23:58:44.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:44 smithi022 ceph-mon[31894]: pgmap v131: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 382 op/s 2024-03-08T23:58:44.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.756+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/2967492575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e4100d90 msgr2=0x7fb6e4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.756+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/2967492575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e4100d90 0x7fb6e4104dd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009900 tx=0x7fb6d802bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.756+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/2967492575 shutdown_connections 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.756+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/2967492575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e41054b0 0x7fb6e410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.756+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/2967492575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e4100d90 0x7fb6e4104dd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.756+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/2967492575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e41003e0 0x7fb6e41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.756+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/2967492575 >> 172.21.15.22:0/2967492575 conn(0x7fb6e40773a0 msgr2=0x7fb6e40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.757+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/2967492575 shutdown_connections 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.757+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/2967492575 wait complete. 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.757+0000 7fb6ea6b2640 1 Processor -- start 2024-03-08T23:58:44.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.757+0000 7fb6ea6b2640 1 -- start start 2024-03-08T23:58:44.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.757+0000 7fb6ea6b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e41003e0 0x7fb6e419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.757+0000 7fb6ea6b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e41054b0 0x7fb6e419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6ea6b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e4197930 0x7fb6e4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6ea6b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4110590 con 0x7fb6e41003e0 2024-03-08T23:58:44.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6ea6b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4110410 con 0x7fb6e41054b0 2024-03-08T23:58:44.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6ea6b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4110710 con 0x7fb6e4197930 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e41003e0 0x7fb6e419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e41003e0 0x7fb6e419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53810/0 (socket says 172.21.15.22:53810) 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 -- 172.21.15.22:0/428210885 learned_addr learned my addr 172.21.15.22:0/428210885 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e4197930 0x7fb6e4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e41054b0 0x7fb6e419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 -- 172.21.15.22:0/428210885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e4197930 msgr2=0x7fb6e4197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e4197930 0x7fb6e4197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 -- 172.21.15.22:0/428210885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e41054b0 msgr2=0x7fb6e419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e41054b0 0x7fb6e419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 -- 172.21.15.22:0/428210885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4198450 con 0x7fb6e41003e0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e41003e0 0x7fb6e419d780 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e9f0 tx=0x7fb6d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e17fa640 1 -- 172.21.15.22:0/428210885 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400cd80 con 0x7fb6e41003e0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e17fa640 1 -- 172.21.15.22:0/428210885 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4005500 con 0x7fb6e41003e0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e4198740 con 0x7fb6e41003e0 2024-03-08T23:58:44.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.758+0000 7fb6e17fa640 1 -- 172.21.15.22:0/428210885 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4010690 con 0x7fb6e41003e0 2024-03-08T23:58:44.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.759+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e41a4940 con 0x7fb6e41003e0 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.761+0000 7fb6e17fa640 1 -- 172.21.15.22:0/428210885 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb6d40046c0 con 0x7fb6e41003e0 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.761+0000 7fb6e17fa640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6cc0a2970 0x7fb6cc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.761+0000 7fb6e17fa640 1 -- 172.21.15.22:0/428210885 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb6d40cf600 con 0x7fb6e41003e0 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.761+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6cc0a2970 0x7fb6cc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.761+0000 7fb6ed374740 1 Processor -- start 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.761+0000 7fb6ed374740 1 -- start start 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2b1a0 0x55647ef2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2bac0 0x55647ef34160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef36a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6ed374740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647eef2360 con 0x55647ef2bac0 2024-03-08T23:58:44.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6ed374740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647ee8eec0 con 0x55647ef346a0 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6ed374740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647ee37d30 con 0x55647ef2b1a0 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6cc0a2970 0x7fb6cc0a4e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6d802c240 tx=0x7fb6d802df90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2bac0 0x55647ef34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2b1a0 0x55647ef2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2bac0 0x55647ef34160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53814/0 (socket says 172.21.15.22:53814) 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 -- 172.21.15.22:0/2906136962 learned_addr learned my addr 172.21.15.22:0/2906136962 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef36a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 -- 172.21.15.22:0/2906136962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2b1a0 msgr2=0x55647ef2b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2b1a0 0x55647ef2b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 -- 172.21.15.22:0/2906136962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 msgr2=0x55647ef36a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef36a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 -- 172.21.15.22:0/2906136962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55647ee36bc0 con 0x55647ef2bac0 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2b1a0 0x55647ef2b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef36a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2bac0 0x55647ef34160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009900 tx=0x7fb6d803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3735a43ee88e7324 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.762+0000 7fb6ca7fc640 1 -- 172.21.15.22:0/2906136962 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d802c880 con 0x55647ef2bac0 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ca7fc640 1 -- 172.21.15.22:0/2906136962 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6d8050070 con 0x55647ef2bac0 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ca7fc640 1 -- 172.21.15.22:0/2906136962 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8005170 con 0x55647ef2bac0 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 -- 172.21.15.22:0/2906136962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2bac0 msgr2=0x55647ef34160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2bac0 0x55647ef34160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009900 tx=0x7fb6d803f000 comp rx=0 tx=0).stop 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 -- 172.21.15.22:0/2906136962 shutdown_connections 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef36a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2bac0 0x55647ef34160 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 --2- 172.21.15.22:0/2906136962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2b1a0 0x55647ef2b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 -- 172.21.15.22:0/2906136962 >> 172.21.15.22:0/2906136962 conn(0x55647ef258d0 msgr2=0x55647ef25cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 -- 172.21.15.22:0/2906136962 shutdown_connections 2024-03-08T23:58:44.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.763+0000 7fb6ed374740 1 -- 172.21.15.22:0/2906136962 wait complete. 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 -- start start 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 Processor -- start 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef31c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef32170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647ee2d4a0 con 0x55647ef346a0 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647ee8af60 con 0x55647ef2b1a0 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647ef37430 con 0x55647ef2bac0 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6ed374740 10 client.0 _ll_register_callbacks cb 0x55647ee88d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6e8c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef32170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6e37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef31c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6e8c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef32170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53826/0 (socket says 172.21.15.22:53826) 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef316f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:40072/0 (socket says 172.21.15.22:40072) 2024-03-08T23:58:44.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.764+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1547476129 learned_addr learned my addr 172.21.15.22:0/1547476129 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1547476129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 msgr2=0x55647ef31c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef31c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 msgr2=0x55647ef316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef316f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1547476129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55647eba33f0 con 0x55647ef346a0 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef31c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef32170 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0127b0 tx=0x7fb6dc012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86803b05ff47773e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1547476129 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc012e60 con 0x55647ef346a0 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1547476129 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb6dc0109b0 con 0x55647ef346a0 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6d8050070 con 0x55647ef346a0 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1547476129 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6dc003e40 con 0x55647ef346a0 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6ed374740 10 client.17034 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55647ec6d7d0 con 0x55647ef346a0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.765+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1547476129 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6d8050070 con 0x55647ef346a0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1547476129 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb6dc019070 con 0x55647ef346a0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6ed374740 10 client.17034 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6ed374740 10 client.17034 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6ed374740 20 client.17034 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef448c0 0x55647ef46ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6ed374740 4 client.17034 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55647ee9c400 con 0x55647ef448c0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef448c0 0x55647ef46ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef448c0 0x55647ef46ce0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d79249da2c1dff8 server_cookie=c5276f7f2b77926e in_seq=0 out_seq=0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.766+0000 7fb6c37fe640 10 client.17034 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1547476129 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55647ee9c400 con 0x55647ef448c0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6c37fe640 10 client.17034 handle_command_reply: tid=0 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 client.17034 shutdown 2024-03-08T23:58:44.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 msgr2=0x55647ef32170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef32170 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0127b0 tx=0x7fb6dc012c70 comp rx=0 tx=0).stop 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 shutdown_connections 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef448c0 0x55647ef46ce0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef32170 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef31c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1547476129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6c37fe640 0 client.17034 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 >> 172.21.15.22:0/1547476129 conn(0x55647ef258d0 msgr2=0x55647ef27620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 shutdown_connections 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 1 -- 172.21.15.22:0/1547476129 wait complete. 2024-03-08T23:58:44.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.767+0000 7fb6ed374740 20 client.17034 trim_cache size 0 max 16384 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 Processor -- start 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 -- start start 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef33e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef2bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef2ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41b09d0 con 0x55647ef346a0 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41ae570 con 0x55647ef2b1a0 2024-03-08T23:58:44.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6ed374740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41ac610 con 0x55647ef2bac0 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef2ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef2ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53836/0 (socket says 172.21.15.22:53836) 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef33e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1140221825 learned_addr learned my addr 172.21.15.22:0/1140221825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef2bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1140221825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 msgr2=0x55647ef2bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef2bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1140221825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 msgr2=0x55647ef33e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef33e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.779+0000 7fb6e8c4f640 1 -- 172.21.15.22:0/1140221825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55647ef2b8f0 con 0x55647ef346a0 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef2ca20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc01b610 tx=0x7fb6dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5020773ac66700d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6c9ffb640 1 -- 172.21.15.22:0/1140221825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc02e070 con 0x55647ef346a0 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6c9ffb640 1 -- 172.21.15.22:0/1140221825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc010e90 con 0x55647ef346a0 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6c9ffb640 1 -- 172.21.15.22:0/1140221825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc012e60 con 0x55647ef346a0 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 -- 172.21.15.22:0/1140221825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 msgr2=0x55647ef2ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef2ca20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc01b610 tx=0x7fb6dc005f90 comp rx=0 tx=0).stop 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 -- 172.21.15.22:0/1140221825 shutdown_connections 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef346a0 0x55647ef2ca20 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef2bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1140221825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef2b1a0 0x55647ef33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 -- 172.21.15.22:0/1140221825 >> 172.21.15.22:0/1140221825 conn(0x55647ef258d0 msgr2=0x55647ef25cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 -- 172.21.15.22:0/1140221825 shutdown_connections 2024-03-08T23:58:44.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.780+0000 7fb6ed374740 1 -- 172.21.15.22:0/1140221825 wait complete. 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 -- start start 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 Processor -- start 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2b1a0 0x55647ef3b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef39c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef3a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41acc10 con 0x55647ef2b1a0 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41adf10 con 0x55647ef346a0 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41ad250 con 0x55647ef2bac0 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6ed374740 10 client.0 _ll_register_callbacks cb 0x7fb6d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2b1a0 0x55647ef3b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2b1a0 0x55647ef3b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53844/0 (socket says 172.21.15.22:53844) 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6e3fff640 1 -- 172.21.15.22:0/1353053447 learned_addr learned my addr 172.21.15.22:0/1353053447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef39c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.781+0000 7fb6e8c4f640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef3a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6e3fff640 1 -- 172.21.15.22:0/1353053447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 msgr2=0x55647ef39c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef39c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6e3fff640 1 -- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 msgr2=0x55647ef3a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef3a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6e3fff640 1 -- 172.21.15.22:0/1353053447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55647ef3a780 con 0x55647ef2b1a0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef39c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6e3fff640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2b1a0 0x55647ef3b820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4097d20 tx=0x7fb6d4086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4f85bac06e37e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1353053447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40cf040 con 0x55647ef2b1a0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1353053447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb6d4097f30 con 0x55647ef2b1a0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6dc010e90 con 0x55647ef2b1a0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1353053447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6d40860b0 con 0x55647ef2b1a0 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 10 client.17040 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55647ef9bc10 con 0x55647ef2b1a0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1353053447 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc010e90 con 0x55647ef2b1a0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1353053447 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb6d40ae020 con 0x55647ef2b1a0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 10 client.17040 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 10 client.17040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 20 client.17040 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef9c2e0 0x55647ef9e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 4 client.17040 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.782+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55647ee96750 con 0x55647ef9c2e0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef9c2e0 0x55647ef9e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6e37fe640 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef9c2e0 0x55647ef9e700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=940bcde555c8583e server_cookie=a9aeb31f722ba0d9 in_seq=0 out_seq=0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6c37fe640 10 client.17040 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6c37fe640 1 -- 172.21.15.22:0/1353053447 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55647ee96750 con 0x55647ef9c2e0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6c37fe640 10 client.17040 handle_command_reply: tid=0 2024-03-08T23:58:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 client.17040 shutdown 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2b1a0 msgr2=0x55647ef3b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2b1a0 0x55647ef3b820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4097d20 tx=0x7fb6d4086040 comp rx=0 tx=0).stop 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 shutdown_connections 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55647ef9c2e0 0x55647ef9e700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55647ef346a0 0x55647ef3a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55647ef2bac0 0x55647ef39c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 --2- 172.21.15.22:0/1353053447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55647ef2b1a0 0x55647ef3b820 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6c37fe640 0 client.17040 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 >> 172.21.15.22:0/1353053447 conn(0x55647ef258d0 msgr2=0x55647ef2bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.783+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 shutdown_connections 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ed374740 1 -- 172.21.15.22:0/1353053447 wait complete. 2024-03-08T23:58:44.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ed374740 20 client.17040 trim_cache size 0 max 16384 2024-03-08T23:58:44.790 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:58:44.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6cc0a2970 msgr2=0x7fb6cc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6cc0a2970 0x7fb6cc0a4e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6d802c240 tx=0x7fb6d802df90 comp rx=0 tx=0).stop 2024-03-08T23:58:44.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e41003e0 msgr2=0x7fb6e419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e41003e0 0x7fb6e419d780 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e9f0 tx=0x7fb6d400edf0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 shutdown_connections 2024-03-08T23:58:44.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6cc0a2970 0x7fb6cc0a4e30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e4197930 0x7fb6e4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e41054b0 0x7fb6e419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 --2- 172.21.15.22:0/428210885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6e41003e0 0x7fb6e419d780 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 >> 172.21.15.22:0/428210885 conn(0x7fb6e40773a0 msgr2=0x7fb6e4068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 shutdown_connections 2024-03-08T23:58:44.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.784+0000 7fb6ea6b2640 1 -- 172.21.15.22:0/428210885 wait complete. 2024-03-08T23:58:44.792 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:58:44.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:58:44.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:44 smithi062 ceph-mon[36129]: pgmap v131: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 382 op/s 2024-03-08T23:58:44.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:44 smithi192 ceph-mon[35995]: pgmap v131: 129 pgs: 129 active+clean; 48 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 382 op/s 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 -- 172.21.15.22:0/1094616334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4101e90 msgr2=0x7f28f4105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 --2- 172.21.15.22:0/1094616334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4101e90 0x7f28f4105ed0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009900 tx=0x7f28e802bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 -- 172.21.15.22:0/1094616334 shutdown_connections 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 --2- 172.21.15.22:0/1094616334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f41065b0 0x7f28f410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 --2- 172.21.15.22:0/1094616334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4101e90 0x7f28f4105ed0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 --2- 172.21.15.22:0/1094616334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41014e0 0x7f28f41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 -- 172.21.15.22:0/1094616334 >> 172.21.15.22:0/1094616334 conn(0x7f28f40fcda0 msgr2=0x7f28f40ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 -- 172.21.15.22:0/1094616334 shutdown_connections 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.906+0000 7f28fa753640 1 -- 172.21.15.22:0/1094616334 wait complete. 2024-03-08T23:58:44.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 Processor -- start 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 -- start start 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41014e0 0x7f28f419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f41065b0 0x7f28f419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4197960 0x7f28f4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4111620 con 0x7f28f4197960 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41114a0 con 0x7f28f41014e0 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28fa753640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41117a0 con 0x7f28f41065b0 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4197960 0x7f28f4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4197960 0x7f28f4197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53874/0 (socket says 172.21.15.22:53874) 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 -- 172.21.15.22:0/2351092795 learned_addr learned my addr 172.21.15.22:0/2351092795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f41065b0 0x7f28f419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f3fff640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41014e0 0x7f28f419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 -- 172.21.15.22:0/2351092795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f41065b0 msgr2=0x7f28f419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f41065b0 0x7f28f419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 -- 172.21.15.22:0/2351092795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41014e0 msgr2=0x7f28f419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41014e0 0x7f28f419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.907+0000 7f28f8cf0640 1 -- 172.21.15.22:0/2351092795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4198480 con 0x7f28f4197960 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.908+0000 7f28f8cf0640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4197960 0x7f28f4197e10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009fd0 tx=0x7f28ec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.908+0000 7f28f17fa640 1 -- 172.21.15.22:0/2351092795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec017070 con 0x7f28f4197960 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.908+0000 7f28f17fa640 1 -- 172.21.15.22:0/2351092795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28ec003e50 con 0x7f28f4197960 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.908+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f4198770 con 0x7f28f4197960 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.908+0000 7f28f17fa640 1 -- 172.21.15.22:0/2351092795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0106c0 con 0x7f28f4197960 2024-03-08T23:58:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.908+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f41a4920 con 0x7f28f4197960 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f17fa640 1 -- 172.21.15.22:0/2351092795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f28ec003970 con 0x7f28f4197960 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 Processor -- start 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 -- start start 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c92d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c936160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c938a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265c8f4360 con 0x56265c92d1a0 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265c890ec0 con 0x56265c9366a0 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28fd415740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265c839d30 con 0x56265c92dac0 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f17fa640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28dc0a2970 0x7f28dc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c92d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c936160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c92d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53880/0 (socket says 172.21.15.22:53880) 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f37fe640 1 -- 172.21.15.22:0/2077256984 learned_addr learned my addr 172.21.15.22:0/2077256984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c938a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f8cf0640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28dc0a2970 0x7f28dc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.911+0000 7f28f17fa640 1 -- 172.21.15.22:0/2351092795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f28ec0cef50 con 0x7f28f4197960 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f37fe640 1 -- 172.21.15.22:0/2077256984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 msgr2=0x56265c936160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c936160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f37fe640 1 -- 172.21.15.22:0/2077256984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 msgr2=0x56265c938a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c938a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f37fe640 1 -- 172.21.15.22:0/2077256984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56265c838bc0 con 0x56265c92d1a0 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f3fff640 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c936160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c92d580 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f28e802cfd0 tx=0x7f28e8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d37e2880b5f1a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28f8cf0640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28dc0a2970 0x7f28dc0a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28ec09af10 tx=0x7f28ec016b40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28da7fc640 1 -- 172.21.15.22:0/2077256984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004230 con 0x56265c92d1a0 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28da7fc640 1 -- 172.21.15.22:0/2077256984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28e8004ba0 con 0x56265c92d1a0 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.912+0000 7f28da7fc640 1 -- 172.21.15.22:0/2077256984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80053b0 con 0x56265c92d1a0 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- 172.21.15.22:0/2077256984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 msgr2=0x56265c92d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c92d580 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f28e802cfd0 tx=0x7f28e8005b50 comp rx=0 tx=0).stop 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- 172.21.15.22:0/2077256984 shutdown_connections 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c938a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c936160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 --2- 172.21.15.22:0/2077256984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c92d580 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- 172.21.15.22:0/2077256984 >> 172.21.15.22:0/2077256984 conn(0x56265c9278d0 msgr2=0x56265c927cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- 172.21.15.22:0/2077256984 shutdown_connections 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- 172.21.15.22:0/2077256984 wait complete. 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- start start 2024-03-08T23:58:44.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 Processor -- start 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c9336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c933c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c934140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265c82f4a0 con 0x56265c92d1a0 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265c88cf60 con 0x56265c9366a0 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.913+0000 7f28fd415740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56265c939430 con 0x56265c92dac0 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c9336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28fd415740 10 client.0 _ll_register_callbacks cb 0x56265c88ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c9336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53886/0 (socket says 172.21.15.22:53886) 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 -- 172.21.15.22:0/4049019663 learned_addr learned my addr 172.21.15.22:0/4049019663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f3fff640 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c933c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c934140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 -- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 msgr2=0x56265c933c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c933c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 -- 172.21.15.22:0/4049019663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 msgr2=0x56265c934140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c934140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 -- 172.21.15.22:0/4049019663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56265c5a53f0 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c9336c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036040 tx=0x7f28e8005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed41d0d542fa5dc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28d37fe640 1 -- 172.21.15.22:0/4049019663 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8055060 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28d37fe640 1 -- 172.21.15.22:0/4049019663 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f28e8050460 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28e8004ba0 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28d37fe640 1 -- 172.21.15.22:0/4049019663 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e8005640 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28fd415740 10 client.17052 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.914+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56265c66f7d0 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28d37fe640 1 -- 172.21.15.22:0/4049019663 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28e8004ba0 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28d37fe640 1 -- 172.21.15.22:0/4049019663 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f28e8041690 con 0x56265c92d1a0 2024-03-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28fd415740 10 client.17052 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28fd415740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28fd415740 20 client.17052 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28fd415740 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9466b0 0x56265c948aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28fd415740 4 client.17052 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56265c89e400 con 0x56265c9466b0 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28f3fff640 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9466b0 0x56265c948aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28f3fff640 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9466b0 0x56265c948aa0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd0032c0fc31dc9 server_cookie=810779969f6378e6 in_seq=0 out_seq=0 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.915+0000 7f28d37fe640 10 client.17052 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28d37fe640 1 -- 172.21.15.22:0/4049019663 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56265c89e400 con 0x56265c9466b0 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28d37fe640 10 client.17052 handle_command_reply: tid=0 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 client.17052 shutdown 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 msgr2=0x56265c9336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c9336c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036040 tx=0x7f28e8005070 comp rx=0 tx=0).stop 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 shutdown_connections 2024-03-08T23:58:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9466b0 0x56265c948aa0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c934140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c933c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28d37fe640 0 client.17052 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 --2- 172.21.15.22:0/4049019663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c9336c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 >> 172.21.15.22:0/4049019663 conn(0x56265c9278d0 msgr2=0x56265c929620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 shutdown_connections 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 1 -- 172.21.15.22:0/4049019663 wait complete. 2024-03-08T23:58:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.916+0000 7f28fd415740 20 client.17052 trim_cache size 0 max 16384 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 Processor -- start 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 -- start start 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c92d1a0 0x56265c935df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c932840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c9366a0 0x56265c941d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41b09d0 con 0x56265c9366a0 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41ae570 con 0x56265c92d1a0 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28fd415740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41ac610 con 0x56265c92dac0 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c9366a0 0x56265c941d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c9366a0 0x56265c941d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53890/0 (socket says 172.21.15.22:53890) 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 -- 172.21.15.22:0/4225981526 learned_addr learned my addr 172.21.15.22:0/4225981526 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c92d1a0 0x56265c935df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f3fff640 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c932840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 -- 172.21.15.22:0/4225981526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 msgr2=0x56265c932840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c932840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 -- 172.21.15.22:0/4225981526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c92d1a0 msgr2=0x56265c935df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c92d1a0 0x56265c935df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.929+0000 7f28f37fe640 1 -- 172.21.15.22:0/4225981526 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56265c92e510 con 0x56265c9366a0 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28f37fe640 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c9366a0 0x56265c941d90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f28e802bc70 tx=0x7f28e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531c3adcf84976bb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28d9ffb640 1 -- 172.21.15.22:0/4225981526 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004ba0 con 0x56265c9366a0 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28d9ffb640 1 -- 172.21.15.22:0/4225981526 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28e8005440 con 0x56265c9366a0 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28d9ffb640 1 -- 172.21.15.22:0/4225981526 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e804e920 con 0x56265c9366a0 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 -- 172.21.15.22:0/4225981526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c9366a0 msgr2=0x56265c941d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c9366a0 0x56265c941d90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f28e802bc70 tx=0x7f28e8005fd0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 -- 172.21.15.22:0/4225981526 shutdown_connections 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c9366a0 0x56265c941d90 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c932840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 --2- 172.21.15.22:0/4225981526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c92d1a0 0x56265c935df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 -- 172.21.15.22:0/4225981526 >> 172.21.15.22:0/4225981526 conn(0x56265c9278d0 msgr2=0x56265c941680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 -- 172.21.15.22:0/4225981526 shutdown_connections 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.930+0000 7f28fd415740 1 -- 172.21.15.22:0/4225981526 wait complete. 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 -- start start 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 Processor -- start 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c94af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c949380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c949ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41acc10 con 0x56265c92d1a0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41adf10 con 0x56265c9366a0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41ad250 con 0x56265c92dac0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28fd415740 10 client.0 _ll_register_callbacks cb 0x7f28e8005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c94af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c94af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53906/0 (socket says 172.21.15.22:53906) 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 -- 172.21.15.22:0/2597184669 learned_addr learned my addr 172.21.15.22:0/2597184669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f3fff640 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c949380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 -- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 msgr2=0x56265c949380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c949380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 -- 172.21.15.22:0/2597184669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 msgr2=0x56265c949ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c949ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.931+0000 7f28f37fe640 1 -- 172.21.15.22:0/2597184669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56265c9a0ba0 con 0x56265c92d1a0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28f37fe640 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c94af90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f28e8041b80 tx=0x7f28e80419b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c84771b0887a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:44.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28d37fe640 1 -- 172.21.15.22:0/2597184669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8066070 con 0x56265c92d1a0 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28d37fe640 1 -- 172.21.15.22:0/2597184669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f28e802c880 con 0x56265c92d1a0 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28e8005440 con 0x56265c92d1a0 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28d37fe640 1 -- 172.21.15.22:0/2597184669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e8004fa0 con 0x56265c92d1a0 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 10 client.17058 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56265c9a1cd0 con 0x56265c92d1a0 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28d37fe640 1 -- 172.21.15.22:0/2597184669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28e8005440 con 0x56265c92d1a0 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28d37fe640 1 -- 172.21.15.22:0/2597184669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f28e8050640 con 0x56265c92d1a0 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 10 client.17058 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 20 client.17058 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9a2140 0x56265c9a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:44.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 4 client.17058 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.932+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56265c898750 con 0x56265c9a2140 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28f3fff640 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9a2140 0x56265c9a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28d37fe640 10 client.17058 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28f3fff640 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9a2140 0x56265c9a4560 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca3adf8c95b4f4 server_cookie=339e4da96d80e7be in_seq=0 out_seq=0 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28d37fe640 1 -- 172.21.15.22:0/2597184669 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56265c898750 con 0x56265c9a2140 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28d37fe640 10 client.17058 handle_command_reply: tid=0 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 client.17058 shutdown 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 msgr2=0x56265c94af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c94af90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f28e8041b80 tx=0x7f28e80419b0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 shutdown_connections 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56265c9a2140 0x56265c9a4560 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56265c9366a0 0x56265c949ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56265c92dac0 0x56265c949380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 --2- 172.21.15.22:0/2597184669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56265c92d1a0 0x56265c94af90 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28d37fe640 0 client.17058 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:44.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 >> 172.21.15.22:0/2597184669 conn(0x56265c9278d0 msgr2=0x56265c926990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.933+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 shutdown_connections 2024-03-08T23:58:44.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fd415740 1 -- 172.21.15.22:0/2597184669 wait complete. 2024-03-08T23:58:44.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fd415740 20 client.17058 trim_cache size 0 max 16384 2024-03-08T23:58:44.940 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:58:44.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28dc0a2970 msgr2=0x7f28dc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28dc0a2970 0x7f28dc0a4e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28ec09af10 tx=0x7f28ec016b40 comp rx=0 tx=0).stop 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4197960 msgr2=0x7f28f4197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4197960 0x7f28f4197e10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009fd0 tx=0x7f28ec00b260 comp rx=0 tx=0).stop 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 shutdown_connections 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28dc0a2970 0x7f28dc0a4e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28f4197960 0x7f28f4197e10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28f41065b0 0x7f28f419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 --2- 172.21.15.22:0/2351092795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41014e0 0x7f28f419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 >> 172.21.15.22:0/2351092795 conn(0x7f28f40fcda0 msgr2=0x7f28f40ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 shutdown_connections 2024-03-08T23:58:44.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:44.934+0000 7f28fa753640 1 -- 172.21.15.22:0/2351092795 wait complete. 2024-03-08T23:58:45.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 402381 11.31 MB/sec execute 53 sec latency 13.237 ms 2024-03-08T23:58:45.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:45 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2289704435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:58:45.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:45 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2289704435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:58:45.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:45 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2289704435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:58:45.944 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:58:45.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:58:45.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:58:46.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 -- 172.21.15.22:0/3002123893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 msgr2=0x7fb1c01007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 --2- 172.21.15.22:0/3002123893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c01007b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc02bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 -- 172.21.15.22:0/3002123893 shutdown_connections 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 --2- 172.21.15.22:0/3002123893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1c01054a0 0x7fb1c010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 --2- 172.21.15.22:0/3002123893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c0100d80 0x7fb1c0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 --2- 172.21.15.22:0/3002123893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c01007b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 -- 172.21.15.22:0/3002123893 >> 172.21.15.22:0/3002123893 conn(0x7fb1c00772f0 msgr2=0x7fb1c0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 -- 172.21.15.22:0/3002123893 shutdown_connections 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.065+0000 7fb1c771b640 1 -- 172.21.15.22:0/3002123893 wait complete. 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 Processor -- start 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 -- start start 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c01054a0 0x7fb1c019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1c0197980 0x7fb1c0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0110550 con 0x7fb1c01003d0 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01103d0 con 0x7fb1c0197980 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c771b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01106d0 con 0x7fb1c01054a0 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c54b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c54b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53924/0 (socket says 172.21.15.22:53924) 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c54b7640 1 -- 172.21.15.22:0/1537382227 learned_addr learned my addr 172.21.15.22:0/1537382227 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.066+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c01054a0 0x7fb1c019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1c0197980 0x7fb1c0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c54b7640 1 -- 172.21.15.22:0/1537382227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c01054a0 msgr2=0x7fb1c019dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c01054a0 0x7fb1c019dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c54b7640 1 -- 172.21.15.22:0/1537382227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1c0197980 msgr2=0x7fb1c0197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1c0197980 0x7fb1c0197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c54b7640 1 -- 172.21.15.22:0/1537382227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c01984a0 con 0x7fb1c01003d0 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1c0197980 0x7fb1c0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c019d7d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006010 tx=0x7fb1bc02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1537382227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc041070 con 0x7fb1c01003d0 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c0198730 con 0x7fb1c01003d0 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1537382227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1bc004450 con 0x7fb1c01003d0 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1537382227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc003e00 con 0x7fb1c01003d0 2024-03-08T23:58:46.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.067+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c01a4980 con 0x7fb1c01003d0 2024-03-08T23:58:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.069+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1537382227 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb1bc044070 con 0x7fb1c01003d0 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 Processor -- start 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 -- start start 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1b67fc640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1ac0a2970 0x7fb1ac0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c855ab0 0x55f91c855e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c863150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c865a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c7b9b40 con 0x55f91c855ab0 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c762d80 con 0x55f91c8563d0 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c7b5be0 con 0x55f91c863690 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1ac0a2970 0x7fb1ac0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1537382227 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb1bc0f0c30 con 0x7fb1c01003d0 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c54b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c855ab0 0x55f91c855e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c4cb6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c863150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c54b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c855ab0 0x55f91c855e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53938/0 (socket says 172.21.15.22:53938) 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c54b7640 1 -- 172.21.15.22:0/142836782 learned_addr learned my addr 172.21.15.22:0/142836782 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c865a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/142836782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 msgr2=0x55f91c865a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c865a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/142836782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c855ab0 msgr2=0x55f91c855e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c855ab0 0x55f91c855e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.070+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/142836782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91c5967e0 con 0x55f91c8563d0 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.071+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c855ab0 0x55f91c855e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.071+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1ac0a2970 0x7fb1ac0a4e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8005fd0 tx=0x7fb1b8013b00 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.071+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c863150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0199170 tx=0x7fb1b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e761f0e68099559b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.071+0000 7fb1ab7fe640 1 -- 172.21.15.22:0/142836782 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0004250 con 0x55f91c8563d0 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.071+0000 7fb1ab7fe640 1 -- 172.21.15.22:0/142836782 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0009e10 con 0x55f91c8563d0 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.071+0000 7fb1ab7fe640 1 -- 172.21.15.22:0/142836782 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0004da0 con 0x55f91c8563d0 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/142836782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 msgr2=0x55f91c863150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c863150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0199170 tx=0x7fb1b0011810 comp rx=0 tx=0).stop 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/142836782 shutdown_connections 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c865a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c863150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/142836782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c855ab0 0x55f91c855e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/142836782 >> 172.21.15.22:0/142836782 conn(0x55f91c8504b0 msgr2=0x55f91c850890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/142836782 shutdown_connections 2024-03-08T23:58:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/142836782 wait complete. 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- start start 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 Processor -- start 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c863150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c8604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c863690 0x55f91c860c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c866450 con 0x55f91c8563d0 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c7761c0 con 0x55f91c863690 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.072+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c8665d0 con 0x55f91c855ab0 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1ca3d5740 10 client.0 _ll_register_callbacks cb 0x55f91c7657a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c4cb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c8604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c54b7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c863150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c54b7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c863150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38206/0 (socket says 172.21.15.22:38206) 2024-03-08T23:58:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c54b7640 1 -- 172.21.15.22:0/1622371709 learned_addr learned my addr 172.21.15.22:0/1622371709 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c863690 0x55f91c860c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 msgr2=0x55f91c863150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c863150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/1622371709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c863690 msgr2=0x55f91c860c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c863690 0x55f91c860c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/1622371709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91c5f6f70 con 0x55f91c8563d0 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c863150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c8604e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005890 tx=0x7fb1b00098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13df0126dc0da04f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/1622371709 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0029060 con 0x55f91c8563d0 2024-03-08T23:58:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/1622371709 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb1b0003a00 con 0x55f91c8563d0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1b0009e10 con 0x55f91c8563d0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/1622371709 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1b0030050 con 0x55f91c8563d0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1ca3d5740 10 client.17070 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.073+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f91c48e170 con 0x55f91c8563d0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/1622371709 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0009e10 con 0x55f91c8563d0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/1622371709 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb1b0004b80 con 0x55f91c8563d0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1ca3d5740 10 client.17070 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1ca3d5740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1ca3d5740 20 client.17070 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8736c0 0x55f91c875ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1ca3d5740 4 client.17070 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f91c78f960 con 0x55f91c8736c0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8736c0 0x55f91c875ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8736c0 0x55f91c875ab0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a049fd6e321957d server_cookie=7940afca26a934a8 in_seq=0 out_seq=0 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.074+0000 7fb1a8ff9640 10 client.17070 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/1622371709 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f91c78f960 con 0x55f91c8736c0 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1a8ff9640 10 client.17070 handle_command_reply: tid=0 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1ca3d5740 1 client.17070 shutdown 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 msgr2=0x55f91c8604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c8604e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005890 tx=0x7fb1b00098b0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 shutdown_connections 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8736c0 0x55f91c875ab0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.076+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c863690 0x55f91c860c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.077+0000 7fb1a8ff9640 0 client.17070 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.077+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c8604e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.077+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/1622371709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c863150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.077+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 >> 172.21.15.22:0/1622371709 conn(0x55f91c8504b0 msgr2=0x55f91c852200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.077+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 shutdown_connections 2024-03-08T23:58:46.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.077+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/1622371709 wait complete. 2024-03-08T23:58:46.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.077+0000 7fb1ca3d5740 20 client.17070 trim_cache size 0 max 16384 2024-03-08T23:58:46.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.092+0000 7fb1ca3d5740 1 Processor -- start 2024-03-08T23:58:46.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.092+0000 7fb1ca3d5740 1 -- start start 2024-03-08T23:58:46.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c87a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c862c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c863690 0x55f91c863150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01ae890 con 0x55f91c863690 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01ac430 con 0x55f91c8563d0 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01aa4d0 con 0x55f91c855ab0 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c863690 0x55f91c863150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c863690 0x55f91c863150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53950/0 (socket says 172.21.15.22:53950) 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 -- 172.21.15.22:0/2980365950 learned_addr learned my addr 172.21.15.22:0/2980365950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c87a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c862c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 -- 172.21.15.22:0/2980365950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 msgr2=0x55f91c87a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c87a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 -- 172.21.15.22:0/2980365950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 msgr2=0x55f91c862c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c862c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 -- 172.21.15.22:0/2980365950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91c856200 con 0x55f91c863690 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c87a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.093+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c863690 0x55f91c863150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8011ae0 tx=0x7fb1b801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a591956d5ba4bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1aaffd640 1 -- 172.21.15.22:0/2980365950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8020c10 con 0x55f91c863690 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1aaffd640 1 -- 172.21.15.22:0/2980365950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b8020db0 con 0x55f91c863690 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1aaffd640 1 -- 172.21.15.22:0/2980365950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8021770 con 0x55f91c863690 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/2980365950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c863690 msgr2=0x55f91c863150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c863690 0x55f91c863150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8011ae0 tx=0x7fb1b801c000 comp rx=0 tx=0).stop 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/2980365950 shutdown_connections 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c863690 0x55f91c863150 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c8563d0 0x55f91c862c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/2980365950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c855ab0 0x55f91c87a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/2980365950 >> 172.21.15.22:0/2980365950 conn(0x55f91c8504b0 msgr2=0x55f91c875ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/2980365950 shutdown_connections 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.094+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/2980365950 wait complete. 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 -- start start 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 Processor -- start 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c855ab0 0x55f91c869520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c869a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c869fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01aaad0 con 0x55f91c8563d0 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01abdd0 con 0x55f91c855ab0 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01ab110 con 0x55f91c863690 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1ca3d5740 10 client.0 _ll_register_callbacks cb 0x7fb1b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1c4cb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c869a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1c54b7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c855ab0 0x55f91c869520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1c4cb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c869a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53952/0 (socket says 172.21.15.22:53952) 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/4051088319 learned_addr learned my addr 172.21.15.22:0/4051088319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.095+0000 7fb1c5cb8640 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c869fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 msgr2=0x55f91c869fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c869fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/4051088319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c855ab0 msgr2=0x55f91c869520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c855ab0 0x55f91c869520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1c4cb6640 1 -- 172.21.15.22:0/4051088319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91c8c9d00 con 0x55f91c8563d0 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c855ab0 0x55f91c869520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1c4cb6640 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c869a60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0011850 tx=0x7fb1b0005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf91a34906aaf381 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/4051088319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0029040 con 0x55f91c8563d0 2024-03-08T23:58:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/4051088319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb1b0004910 con 0x55f91c8563d0 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1b8020db0 con 0x55f91c8563d0 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/4051088319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1b0005240 con 0x55f91c8563d0 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1ca3d5740 10 client.17076 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f91c8cb000 con 0x55f91c8563d0 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/4051088319 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b8020db0 con 0x55f91c8563d0 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/4051088319 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb1b00194c0 con 0x55f91c8563d0 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1ca3d5740 10 client.17076 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.096+0000 7fb1ca3d5740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.097+0000 7fb1ca3d5740 20 client.17076 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.097+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8cb610 0x55f91c8cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.097+0000 7fb1ca3d5740 4 client.17076 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:58:46.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.097+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f91c77b080 con 0x55f91c8cb610 2024-03-08T23:58:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.097+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8cb610 0x55f91c8cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.097+0000 7fb1c54b7640 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8cb610 0x55f91c8cda30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c96fa73e606bc7 server_cookie=a51c42d2305f7d41 in_seq=0 out_seq=0 2024-03-08T23:58:46.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.097+0000 7fb1a8ff9640 10 client.17076 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1a8ff9640 1 -- 172.21.15.22:0/4051088319 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f91c77b080 con 0x55f91c8cb610 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1a8ff9640 10 client.17076 handle_command_reply: tid=0 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 client.17076 shutdown 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 msgr2=0x55f91c869a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c869a60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0011850 tx=0x7fb1b0005130 comp rx=0 tx=0).stop 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 shutdown_connections 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f91c8cb610 0x55f91c8cda30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f91c863690 0x55f91c869fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f91c8563d0 0x55f91c869a60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1a8ff9640 0 client.17076 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 --2- 172.21.15.22:0/4051088319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91c855ab0 0x55f91c869520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.098+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 >> 172.21.15.22:0/4051088319 conn(0x55f91c8504b0 msgr2=0x55f91c87a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 shutdown_connections 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1ca3d5740 1 -- 172.21.15.22:0/4051088319 wait complete. 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1ca3d5740 20 client.17076 trim_cache size 0 max 16384 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:58:46.101 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "69543582-a3c7-4944-9e44-32dd7d7743b2", 2024-03-08T23:58:46.102 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:58:46.102 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:58:46.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1ac0a2970 msgr2=0x7fb1ac0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1ac0a2970 0x7fb1ac0a4e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8005fd0 tx=0x7fb1b8013b00 comp rx=0 tx=0).stop 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 msgr2=0x7fb1c019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c019d7d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006010 tx=0x7fb1bc02cb30 comp rx=0 tx=0).stop 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 shutdown_connections 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1ac0a2970 0x7fb1ac0a4e30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1c0197980 0x7fb1c0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1c01054a0 0x7fb1c019dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 --2- 172.21.15.22:0/1537382227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1c01003d0 0x7fb1c019d7d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 >> 172.21.15.22:0/1537382227 conn(0x7fb1c00772f0 msgr2=0x7fb1c0068790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 shutdown_connections 2024-03-08T23:58:46.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.099+0000 7fb1c771b640 1 -- 172.21.15.22:0/1537382227 wait complete. 2024-03-08T23:58:46.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:58:46.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 -- 172.21.15.22:0/3611956464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 msgr2=0x7f6c5410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 --2- 172.21.15.22:0/3611956464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5410e1e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009ca0 tx=0x7f6c4c02c0f0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 -- 172.21.15.22:0/3611956464 shutdown_connections 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 --2- 172.21.15.22:0/3611956464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5410e1e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 --2- 172.21.15.22:0/3611956464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54069110 0x7f6c54105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 --2- 172.21.15.22:0/3611956464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c54068760 0x7f6c54068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 -- 172.21.15.22:0/3611956464 >> 172.21.15.22:0/3611956464 conn(0x7f6c540772f0 msgr2=0x7f6c54077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 -- 172.21.15.22:0/3611956464 shutdown_connections 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 -- 172.21.15.22:0/3611956464 wait complete. 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 Processor -- start 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.218+0000 7f6c5a353640 1 -- start start 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c54068760 0x7f6c5419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54197930 0x7f6c54197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54111670 con 0x7f6c54106630 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541114f0 con 0x7f6c54197930 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541117f0 con 0x7f6c54068760 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5419dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53970/0 (socket says 172.21.15.22:53970) 2024-03-08T23:58:46.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 -- 172.21.15.22:0/1434280855 learned_addr learned my addr 172.21.15.22:0/1434280855 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c53fff640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c54068760 0x7f6c5419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c588f0640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54197930 0x7f6c54197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 -- 172.21.15.22:0/1434280855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c54068760 msgr2=0x7f6c5419d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c54068760 0x7f6c5419d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 -- 172.21.15.22:0/1434280855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54197930 msgr2=0x7f6c54197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54197930 0x7f6c54197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 -- 172.21.15.22:0/1434280855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c54198450 con 0x7f6c54106630 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c537fe640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5419dcc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9f0 tx=0x7f6c4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c517fa640 1 -- 172.21.15.22:0/1434280855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800ccc0 con 0x7f6c54106630 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c54198740 con 0x7f6c54106630 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c517fa640 1 -- 172.21.15.22:0/1434280855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c48010910 con 0x7f6c54106630 2024-03-08T23:58:46.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c517fa640 1 -- 172.21.15.22:0/1434280855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48018920 con 0x7f6c54106630 2024-03-08T23:58:46.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.219+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541a49e0 con 0x7f6c54106630 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.221+0000 7f6c517fa640 1 -- 172.21.15.22:0/1434280855 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6c48010ac0 con 0x7f6c54106630 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c517fa640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6c3c0a2970 0x7f6c3c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 Processor -- start 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 -- start start 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c53fff640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6c3c0a2970 0x7f6c3c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c517fa640 1 -- 172.21.15.22:0/1434280855 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6c480cf510 con 0x7f6c54106630 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6410 0x557bdb0e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bdb049e00 con 0x557bdb0e6410 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bdaff2b10 con 0x557bdb0f4070 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c5d011740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bdb045ea0 con 0x557bdb0e6d30 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6410 0x557bdb0e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6410 0x557bdb0e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53982/0 (socket says 172.21.15.22:53982) 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.222+0000 7f6c53fff640 1 -- 172.21.15.22:0/2100195760 learned_addr learned my addr 172.21.15.22:0/2100195760 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c537fe640 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c588f0640 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c53fff640 1 -- 172.21.15.22:0/2100195760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 msgr2=0x557bdb0f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c53fff640 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c53fff640 1 -- 172.21.15.22:0/2100195760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 msgr2=0x557bdb0f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c53fff640 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c53fff640 1 -- 172.21.15.22:0/2100195760 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bdae287d0 con 0x557bdb0e6410 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c53fff640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6c3c0a2970 0x7f6c3c0a4e30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009fd0 tx=0x7f6c440134e0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c53fff640 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6410 0x557bdb0e67f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6c440137a0 tx=0x7f6c4401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b7d7e272a01275 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c3a7fc640 1 -- 172.21.15.22:0/2100195760 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44026070 con 0x557bdb0e6410 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c3a7fc640 1 -- 172.21.15.22:0/2100195760 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c440120d0 con 0x557bdb0e6410 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.223+0000 7f6c3a7fc640 1 -- 172.21.15.22:0/2100195760 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400edf0 con 0x557bdb0e6410 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 -- 172.21.15.22:0/2100195760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6410 msgr2=0x557bdb0e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6410 0x557bdb0e67f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6c440137a0 tx=0x7f6c4401d000 comp rx=0 tx=0).stop 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 -- 172.21.15.22:0/2100195760 shutdown_connections 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2100195760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6410 0x557bdb0e67f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 -- 172.21.15.22:0/2100195760 >> 172.21.15.22:0/2100195760 conn(0x557bdb0e0e90 msgr2=0x557bdb0e1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 -- 172.21.15.22:0/2100195760 shutdown_connections 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 -- 172.21.15.22:0/2100195760 wait complete. 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 -- start start 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.224+0000 7f6c5d011740 1 Processor -- start 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0e6410 0x557bdb0f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0f4070 0x557bdb0f1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bdb0f6e50 con 0x557bdb0f4070 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bdb0060a0 con 0x557bdb0e6410 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bdb0f6fd0 con 0x557bdb0e6d30 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 10 client.0 _ll_register_callbacks cb 0x557bdb005a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:46.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0f4070 0x557bdb0f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0f4070 0x557bdb0f1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53986/0 (socket says 172.21.15.22:53986) 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0e6410 0x557bdb0f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 -- 172.21.15.22:0/2950236121 learned_addr learned my addr 172.21.15.22:0/2950236121 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c537fe640 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 -- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 msgr2=0x557bdb0f0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 -- 172.21.15.22:0/2950236121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0e6410 msgr2=0x557bdb0f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0e6410 0x557bdb0f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 -- 172.21.15.22:0/2950236121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bdad5e3f0 con 0x557bdb0f4070 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c53fff640 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0e6410 0x557bdb0f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c588f0640 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0f4070 0x557bdb0f1050 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009ca0 tx=0x7f6c4c0323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8046627518f3c66 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c337fe640 1 -- 172.21.15.22:0/2950236121 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c044070 con 0x557bdb0f4070 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c337fe640 1 -- 172.21.15.22:0/2950236121 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6c4c032790 con 0x557bdb0f4070 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.225+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c440120d0 con 0x557bdb0f4070 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c337fe640 1 -- 172.21.15.22:0/2950236121 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c4c051050 con 0x557bdb0f4070 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 10 client.17088 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557bdad1e0f0 con 0x557bdb0f4070 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c337fe640 1 -- 172.21.15.22:0/2950236121 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c440120d0 con 0x557bdb0f4070 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c337fe640 1 -- 172.21.15.22:0/2950236121 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c4c004460 con 0x557bdb0f4070 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 10 client.17088 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 20 client.17088 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb103af0 0x557bdb105ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 4 client.17088 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bdb057340 con 0x557bdb103af0 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.226+0000 7f6c53fff640 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb103af0 0x557bdb105ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c53fff640 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb103af0 0x557bdb105ee0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cbb0ace04af382 server_cookie=f5988edd8f387295 in_seq=0 out_seq=0 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c337fe640 10 client.17088 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c337fe640 1 -- 172.21.15.22:0/2950236121 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bdb057340 con 0x557bdb103af0 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c337fe640 10 client.17088 handle_command_reply: tid=0 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 client.17088 shutdown 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0f4070 msgr2=0x557bdb0f1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0f4070 0x557bdb0f1050 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009ca0 tx=0x7f6c4c0323d0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 shutdown_connections 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb103af0 0x557bdb105ee0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0f4070 0x557bdb0f1050 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6d30 0x557bdb0f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 --2- 172.21.15.22:0/2950236121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0e6410 0x557bdb0f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c337fe640 0 client.17088 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 >> 172.21.15.22:0/2950236121 conn(0x557bdb0e0e90 msgr2=0x557bdb0e2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.227+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 shutdown_connections 2024-03-08T23:58:46.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.228+0000 7f6c5d011740 1 -- 172.21.15.22:0/2950236121 wait complete. 2024-03-08T23:58:46.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.228+0000 7f6c5d011740 20 client.17088 trim_cache size 0 max 16384 2024-03-08T23:58:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.244+0000 7f6c5d011740 1 Processor -- start 2024-03-08T23:58:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.244+0000 7f6c5d011740 1 -- start start 2024-03-08T23:58:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c5d011740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541ae8c0 con 0x557bdb0e6d30 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c5d011740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541ac460 con 0x557bdb0f4070 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c5d011740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541aa500 con 0x557bdb0e6410 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54000/0 (socket says 172.21.15.22:54000) 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 -- 172.21.15.22:0/913862103 learned_addr learned my addr 172.21.15.22:0/913862103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c53fff640 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c588f0640 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 -- 172.21.15.22:0/913862103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 msgr2=0x557bdb0ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 -- 172.21.15.22:0/913862103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 msgr2=0x557bdb0f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 -- 172.21.15.22:0/913862103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bdb106250 con 0x557bdb0e6d30 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c537fe640 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3240 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9c0 tx=0x7f6c48012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a6d925b1594812 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.245+0000 7f6c39ffb640 1 -- 172.21.15.22:0/913862103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480b4070 con 0x557bdb0e6d30 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c39ffb640 1 -- 172.21.15.22:0/913862103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c48097e90 con 0x557bdb0e6d30 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c39ffb640 1 -- 172.21.15.22:0/913862103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480afb50 con 0x557bdb0e6d30 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 -- 172.21.15.22:0/913862103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 msgr2=0x557bdb0f3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3240 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9c0 tx=0x7f6c48012040 comp rx=0 tx=0).stop 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 -- 172.21.15.22:0/913862103 shutdown_connections 2024-03-08T23:58:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f3240 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 --2- 172.21.15.22:0/913862103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 -- 172.21.15.22:0/913862103 >> 172.21.15.22:0/913862103 conn(0x557bdb0e0e90 msgr2=0x557bdb0ebad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 -- 172.21.15.22:0/913862103 shutdown_connections 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.246+0000 7f6c5d011740 1 -- 172.21.15.22:0/913862103 wait complete. 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 -- start start 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 Processor -- start 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541aab00 con 0x557bdb0e6d30 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541abe00 con 0x557bdb0f4070 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541ab140 con 0x557bdb0e6410 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c5d011740 10 client.0 _ll_register_callbacks cb 0x7f6c44005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54014/0 (socket says 172.21.15.22:54014) 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c537fe640 1 -- 172.21.15.22:0/527236679 learned_addr learned my addr 172.21.15.22:0/527236679 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c53fff640 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c588f0640 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.247+0000 7f6c537fe640 1 -- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 msgr2=0x557bdb0fb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c537fe640 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0fb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c537fe640 1 -- 172.21.15.22:0/527236679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 msgr2=0x557bdb0fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c537fe640 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c537fe640 1 -- 172.21.15.22:0/527236679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bdb159fd0 con 0x557bdb0e6d30 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c537fe640 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f9b80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6c4809b7b0 tx=0x7f6c480cc800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cd6572104c18da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c337fe640 1 -- 172.21.15.22:0/527236679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480b4070 con 0x557bdb0e6d30 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c337fe640 1 -- 172.21.15.22:0/527236679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6c480afb50 con 0x557bdb0e6d30 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c48097e90 con 0x557bdb0e6d30 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c337fe640 1 -- 172.21.15.22:0/527236679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c480afd50 con 0x557bdb0e6d30 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 10 client.17094 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557bdb15b190 con 0x557bdb0e6d30 2024-03-08T23:58:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c337fe640 1 -- 172.21.15.22:0/527236679 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c48097e90 con 0x557bdb0e6d30 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c337fe640 1 -- 172.21.15.22:0/527236679 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c48097690 con 0x557bdb0e6d30 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 10 client.17094 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 20 client.17094 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb15b730 0x557bdb15db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 4 client.17094 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.248+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557bdb051690 con 0x557bdb15b730 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c53fff640 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb15b730 0x557bdb15db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c53fff640 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb15b730 0x557bdb15db50 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=856ca063a77b9baa server_cookie=2c297956fd1548bf in_seq=0 out_seq=0 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c337fe640 10 client.17094 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c337fe640 1 -- 172.21.15.22:0/527236679 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557bdb051690 con 0x557bdb15b730 2024-03-08T23:58:46.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c337fe640 10 client.17094 handle_command_reply: tid=0 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 client.17094 shutdown 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 msgr2=0x557bdb0f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f9b80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6c4809b7b0 tx=0x7f6c480cc800 comp rx=0 tx=0).stop 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 shutdown_connections 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557bdb15b730 0x557bdb15db50 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bdb0f4070 0x557bdb0fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bdb0e6d30 0x557bdb0f9b80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c337fe640 0 client.17094 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 --2- 172.21.15.22:0/527236679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bdb0e6410 0x557bdb0fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.249+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 >> 172.21.15.22:0/527236679 conn(0x557bdb0e0e90 msgr2=0x557bdb0ef3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 shutdown_connections 2024-03-08T23:58:46.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5d011740 1 -- 172.21.15.22:0/527236679 wait complete. 2024-03-08T23:58:46.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5d011740 20 client.17094 trim_cache size 0 max 16384 2024-03-08T23:58:46.254 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:58:46.254 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:58:46.254 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:58:46.254 INFO:teuthology.orchestra.run.smithi022.stdout: "69543582-a3c7-4944-9e44-32dd7d7743b2": { 2024-03-08T23:58:46.254 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:58:46.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "69543582-a3c7-4944-9e44-32dd7d7743b2", 2024-03-08T23:58:46.255 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:58:46.255 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:58:46.255 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:58:46.255 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:58:46.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6c3c0a2970 msgr2=0x7f6c3c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6c3c0a2970 0x7f6c3c0a4e30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009fd0 tx=0x7f6c440134e0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 msgr2=0x7f6c5419dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5419dcc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9f0 tx=0x7f6c4800eeb0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 shutdown_connections 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6c3c0a2970 0x7f6c3c0a4e30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c54197930 0x7f6c54197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c54106630 0x7f6c5419dcc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 --2- 172.21.15.22:0/1434280855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c54068760 0x7f6c5419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 >> 172.21.15.22:0/1434280855 conn(0x7f6c540772f0 msgr2=0x7f6c540fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 shutdown_connections 2024-03-08T23:58:46.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:58:46.250+0000 7f6c5a353640 1 -- 172.21.15.22:0/1434280855 wait complete. 2024-03-08T23:58:46.258 INFO:tasks.cephfs.filesystem:scrub status for tag:69543582-a3c7-4944-9e44-32dd7d7743b2 - {'path': '/', 'tag': '69543582-a3c7-4944-9e44-32dd7d7743b2', 'options': 'recursive,force'} 2024-03-08T23:58:46.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 404944 11.36 MB/sec execute 54 sec latency 21.227 ms 2024-03-08T23:58:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:46 smithi022 ceph-mon[31894]: pgmap v132: 129 pgs: 129 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 506 op/s 2024-03-08T23:58:46.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:46 smithi062 ceph-mon[36129]: pgmap v132: 129 pgs: 129 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 506 op/s 2024-03-08T23:58:46.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:46 smithi192 ceph-mon[35995]: pgmap v132: 129 pgs: 129 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 506 op/s 2024-03-08T23:58:47.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 407413 11.38 MB/sec execute 55 sec latency 75.808 ms 2024-03-08T23:58:47.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:47 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:58:47.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:47 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:47.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:47 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:58:47.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:47 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:47.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:47 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:58:47.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:47 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:47.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:47 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:58:47.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:47 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:47.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:47 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:58:47.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:47 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:47.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:47 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:58:47.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:47 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:58:48.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 409200 11.33 MB/sec execute 56 sec latency 17.229 ms 2024-03-08T23:58:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:48 smithi022 ceph-mon[31894]: pgmap v133: 129 pgs: 129 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 383 op/s 2024-03-08T23:58:48.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:48 smithi062 ceph-mon[36129]: pgmap v133: 129 pgs: 129 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 383 op/s 2024-03-08T23:58:48.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:48 smithi192 ceph-mon[35995]: pgmap v133: 129 pgs: 129 active+clean; 54 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 383 op/s 2024-03-08T23:58:49.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 411908 11.43 MB/sec execute 57 sec latency 14.065 ms 2024-03-08T23:58:50.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 414831 11.48 MB/sec execute 58 sec latency 16.213 ms 2024-03-08T23:58:50.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:50 smithi022 ceph-mon[31894]: pgmap v134: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-03-08T23:58:50.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:50 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:58:50.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:50 smithi062 ceph-mon[36129]: pgmap v134: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-03-08T23:58:50.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:50 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:58:50.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:50 smithi192 ceph-mon[35995]: pgmap v134: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-03-08T23:58:50.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:50 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 -- 172.21.15.22:0/1452013947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 msgr2=0x7f07901018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 --2- 172.21.15.22:0/1452013947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07901018e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f078c009900 tx=0x7f078c02bc30 comp rx=0 tx=0).stop 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 -- 172.21.15.22:0/1452013947 shutdown_connections 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 --2- 172.21.15.22:0/1452013947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07901065d0 0x7f079010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 --2- 172.21.15.22:0/1452013947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0790101eb0 0x7f0790105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 --2- 172.21.15.22:0/1452013947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07901018e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 -- 172.21.15.22:0/1452013947 >> 172.21.15.22:0/1452013947 conn(0x7f07900fcd80 msgr2=0x7f07900ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 -- 172.21.15.22:0/1452013947 shutdown_connections 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.008+0000 7f0797644640 1 -- 172.21.15.22:0/1452013947 wait complete. 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 Processor -- start 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 -- start start 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07900735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07901065d0 0x7f0790073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0790074070 0x7f07901a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790111680 con 0x7f0790101500 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790111500 con 0x7f0790074070 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0797644640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790111800 con 0x7f07901065d0 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f07953e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07900735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f07953e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07900735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43232/0 (socket says 172.21.15.22:43232) 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f07953e0640 1 -- 172.21.15.22:0/3647637726 learned_addr learned my addr 172.21.15.22:0/3647637726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0794bdf640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07901065d0 0x7f0790073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f07953e0640 1 -- 172.21.15.22:0/3647637726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07901065d0 msgr2=0x7f0790073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.009+0000 7f0795be1640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0790074070 0x7f07901a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07953e0640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07901065d0 0x7f0790073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07953e0640 1 -- 172.21.15.22:0/3647637726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0790074070 msgr2=0x7f07901a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07953e0640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0790074070 0x7f07901a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07953e0640 1 -- 172.21.15.22:0/3647637726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07901a63f0 con 0x7f0790101500 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f0795be1640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0790074070 0x7f07901a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07953e0640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07900735f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07867fc640 1 -- 172.21.15.22:0/3647637726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c041070 con 0x7f0790101500 2024-03-08T23:58:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07901a6680 con 0x7f0790101500 2024-03-08T23:58:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07867fc640 1 -- 172.21.15.22:0/3647637726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f078c004450 con 0x7f0790101500 2024-03-08T23:58:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f07867fc640 1 -- 172.21.15.22:0/3647637726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c003e00 con 0x7f0790101500 2024-03-08T23:58:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.010+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07901a6b70 con 0x7f0790101500 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.012+0000 7f07867fc640 1 -- 172.21.15.22:0/3647637726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f078c044070 con 0x7f0790101500 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 Processor -- start 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 -- start start 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd45a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd45fd0 0x55e49fd46450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd4f550 0x55e49fd51940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f07867fc640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f077c0a2a40 0x7f077c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49fcaa5f0 con 0x55e49fd45fd0 2024-03-08T23:58:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49fc53930 con 0x55e49fd4f550 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f079a228740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49fca6690 con 0x55e49fd46d40 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f07953e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd45fd0 0x55e49fd46450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f0795be1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd45a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f0794bdf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd4f550 0x55e49fd51940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f07953e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd45fd0 0x55e49fd46450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43242/0 (socket says 172.21.15.22:43242) 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f0795be1640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f077c0a2a40 0x7f077c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.013+0000 7f07953e0640 1 -- 172.21.15.22:0/1856888689 learned_addr learned my addr 172.21.15.22:0/1856888689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f07867fc640 1 -- 172.21.15.22:0/3647637726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f078c0f0b80 con 0x7f0790101500 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0794bdf640 1 -- 172.21.15.22:0/1856888689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 msgr2=0x55e49fd45a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0794bdf640 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd45a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0794bdf640 1 -- 172.21.15.22:0/1856888689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd45fd0 msgr2=0x55e49fd46450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0794bdf640 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd45fd0 0x55e49fd46450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0794bdf640 1 -- 172.21.15.22:0/1856888689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e49f9bf210 con 0x55e49fd4f550 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0795be1640 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd45a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0795be1640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f077c0a2a40 0x7f077c0a4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0788005c40 tx=0x7f0788009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f0794bdf640 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd4f550 0x55e49fd51940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f07900685f0 tx=0x7f0780011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314cffeda7393905 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f077b7fe640 1 -- 172.21.15.22:0/1856888689 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780004230 con 0x55e49fd4f550 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f077b7fe640 1 -- 172.21.15.22:0/1856888689 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0780009e10 con 0x55e49fd4f550 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.014+0000 7f077b7fe640 1 -- 172.21.15.22:0/1856888689 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780004d70 con 0x55e49fd4f550 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 -- 172.21.15.22:0/1856888689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd4f550 msgr2=0x55e49fd51940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd4f550 0x55e49fd51940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f07900685f0 tx=0x7f0780011810 comp rx=0 tx=0).stop 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 -- 172.21.15.22:0/1856888689 shutdown_connections 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd4f550 0x55e49fd51940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd45fd0 0x55e49fd46450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 --2- 172.21.15.22:0/1856888689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd45a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 -- 172.21.15.22:0/1856888689 >> 172.21.15.22:0/1856888689 conn(0x55e49fd0d5b0 msgr2=0x55e49fd40cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 -- 172.21.15.22:0/1856888689 shutdown_connections 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 -- 172.21.15.22:0/1856888689 wait complete. 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 -- start start 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.015+0000 7f079a228740 1 Processor -- start 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f079a228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f079a228740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd4cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f079a228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd4f550 0x55e49fd4d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f079a228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49fd52280 con 0x55e49fd4f550 2024-03-08T23:58:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f079a228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49fc66d90 con 0x55e49fd45fd0 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f079a228740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49fd52400 con 0x55e49fd46d40 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f079a228740 10 client.0 _ll_register_callbacks cb 0x55e49f9bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd4cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0795be1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd4f550 0x55e49fd4d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd4cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:43772/0 (socket says 172.21.15.22:43772) 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 -- 172.21.15.22:0/3336509575 learned_addr learned my addr 172.21.15.22:0/3336509575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f07953e0640 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 -- 172.21.15.22:0/3336509575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 msgr2=0x55e49fd4c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 -- 172.21.15.22:0/3336509575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd4f550 msgr2=0x55e49fd4d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd4f550 0x55e49fd4d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0794bdf640 1 -- 172.21.15.22:0/3336509575 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e49fa8b620 con 0x55e49fd46d40 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.016+0000 7f0795be1640 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd4f550 0x55e49fd4d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f0794bdf640 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd4cb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0780005890 tx=0x7f0780003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d035c4ac333243c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f0778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f0778ff9640 1 -- 172.21.15.22:0/3336509575 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780027020 con 0x55e49fd46d40 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f0778ff9640 1 -- 172.21.15.22:0/3336509575 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0780003c10 con 0x55e49fd46d40 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0780009e10 con 0x55e49fd46d40 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f0778ff9640 1 -- 172.21.15.22:0/3336509575 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078001e040 con 0x55e49fd46d40 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f079a228740 10 client.26560 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.017+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e49fd5e5c0 con 0x55e49fd46d40 2024-03-08T23:58:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f0778ff9640 1 -- 172.21.15.22:0/3336509575 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0780009e10 con 0x55e49fd46d40 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f0778ff9640 1 -- 172.21.15.22:0/3336509575 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0780018070 con 0x55e49fd46d40 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f079a228740 10 client.26560 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f079a228740 10 client.26560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f079a228740 20 client.26560 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f079a228740 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fd5f970 0x55e49fd61d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f079a228740 4 client.26560 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e49fcb7b30 con 0x55e49fd5f970 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.018+0000 7f07953e0640 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fd5f970 0x55e49fd61d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.019+0000 7f0778ff9640 10 client.26560 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.019+0000 7f07953e0640 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fd5f970 0x55e49fd61d60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c874d22f39562cb1 server_cookie=ebf575815018be75 in_seq=0 out_seq=0 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f0778ff9640 1 -- 172.21.15.22:0/3336509575 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e49fcb7b30 con 0x55e49fd5f970 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f0778ff9640 10 client.26560 handle_command_reply: tid=0 2024-03-08T23:58:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 client.26560 shutdown 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 msgr2=0x55e49fd4cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd4cb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0780005890 tx=0x7f0780003820 comp rx=0 tx=0).stop 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 shutdown_connections 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fd5f970 0x55e49fd61d60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd4f550 0x55e49fd4d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f0778ff9640 0 client.26560 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd46d40 0x55e49fd4cb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 --2- 172.21.15.22:0/3336509575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.020+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 >> 172.21.15.22:0/3336509575 conn(0x55e49fd0d5b0 msgr2=0x55e49fd42a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.021+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 shutdown_connections 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.021+0000 7f079a228740 1 -- 172.21.15.22:0/3336509575 wait complete. 2024-03-08T23:58:51.024 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.021+0000 7f079a228740 20 client.26560 trim_cache size 0 max 16384 2024-03-08T23:58:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 Processor -- start 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 -- start start 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd56bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901b2bd0 con 0x55e49fd46d40 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901b0770 con 0x55e49fd45fd0 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f079a228740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901ae810 con 0x55e49fd4f550 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43260/0 (socket says 172.21.15.22:43260) 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 -- 172.21.15.22:0/3457154316 learned_addr learned my addr 172.21.15.22:0/3457154316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0795be1640 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd56bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f07953e0640 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 -- 172.21.15.22:0/3457154316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 msgr2=0x55e49fd56bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd56bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 -- 172.21.15.22:0/3457154316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 msgr2=0x55e49fd4ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 -- 172.21.15.22:0/3457154316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e49fd41e30 con 0x55e49fd46d40 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.035+0000 7f0794bdf640 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56670 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0780023d00 tx=0x7f0780005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193d4db42a7c2f38 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f077affd640 1 -- 172.21.15.22:0/3457154316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780027070 con 0x55e49fd46d40 2024-03-08T23:58:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f077affd640 1 -- 172.21.15.22:0/3457154316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0780009d50 con 0x55e49fd46d40 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f077affd640 1 -- 172.21.15.22:0/3457154316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780023410 con 0x55e49fd46d40 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 -- 172.21.15.22:0/3457154316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 msgr2=0x55e49fd56670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56670 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0780023d00 tx=0x7f0780005d30 comp rx=0 tx=0).stop 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 -- 172.21.15.22:0/3457154316 shutdown_connections 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd56bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56670 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 --2- 172.21.15.22:0/3457154316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd4ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 -- 172.21.15.22:0/3457154316 >> 172.21.15.22:0/3457154316 conn(0x55e49fd0d5b0 msgr2=0x55e49fd517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 -- 172.21.15.22:0/3457154316 shutdown_connections 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.036+0000 7f079a228740 1 -- 172.21.15.22:0/3457154316 wait complete. 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.037+0000 7f079a228740 1 -- start start 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.037+0000 7f079a228740 1 Processor -- start 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.037+0000 7f079a228740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.037+0000 7f079a228740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.037+0000 7f079a228740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd52830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f079a228740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901aee10 con 0x55e49fd46d40 2024-03-08T23:58:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f079a228740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901b0110 con 0x55e49fd45fd0 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f079a228740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901af450 con 0x55e49fd4f550 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f079a228740 10 client.0 _ll_register_callbacks cb 0x7f0780009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f0794bdf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57144/0 (socket says 172.21.15.22:57144) 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 -- 172.21.15.22:0/1102747526 learned_addr learned my addr 172.21.15.22:0/1102747526 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f0795be1640 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd52830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 -- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 msgr2=0x55e49fd52830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd52830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 -- 172.21.15.22:0/1102747526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 msgr2=0x55e49fd56910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 -- 172.21.15.22:0/1102747526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e49fd53030 con 0x55e49fd45fd0 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f07953e0640 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd563d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f078c0ee0f0 tx=0x7f078c08ca20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f383ca5dd4aa6518 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f0778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f0778ff9640 1 -- 172.21.15.22:0/1102747526 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c04b040 con 0x55e49fd45fd0 2024-03-08T23:58:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f0778ff9640 1 -- 172.21.15.22:0/1102747526 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f078c02cca0 con 0x55e49fd45fd0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.038+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0780009d50 con 0x55e49fd45fd0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f0778ff9640 1 -- 172.21.15.22:0/1102747526 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078c0c56f0 con 0x55e49fd45fd0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 10 client.26549 fetch_fsmap learned FSMap version 31 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e49fdb6810 con 0x55e49fd45fd0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f0778ff9640 1 -- 172.21.15.22:0/1102747526 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0780009d50 con 0x55e49fd45fd0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f0778ff9640 1 -- 172.21.15.22:0/1102747526 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f078c0b95c0 con 0x55e49fd45fd0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 10 client.26549 fetch_fsmap finished waiting for FSMap version 31 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 10 client.26549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 20 client.26549 populate_metadata read hostname 'smithi022' 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fdb6ee0 0x55e49fdb92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 4 client.26549 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e49fcb1e80 con 0x55e49fdb6ee0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f0794bdf640 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fdb6ee0 0x55e49fdb92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f0794bdf640 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fdb6ee0 0x55e49fdb92d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd59a6661c37436 server_cookie=13772cb81c556ace in_seq=0 out_seq=0 2024-03-08T23:58:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.039+0000 7f0778ff9640 10 client.26549 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.288+0000 7f0778ff9640 1 -- 172.21.15.22:0/1102747526 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e49fcb1e80 con 0x55e49fdb6ee0 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.288+0000 7f0778ff9640 10 client.26549 handle_command_reply: tid=0 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.288+0000 7f079a228740 1 client.26549 shutdown 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.288+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 msgr2=0x55e49fd563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.288+0000 7f079a228740 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd563d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f078c0ee0f0 tx=0x7f078c08ca20 comp rx=0 tx=0).stop 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 shutdown_connections 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e49fdb6ee0 0x55e49fdb92d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e49fd4f550 0x55e49fd52830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e49fd46d40 0x55e49fd56910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f0778ff9640 0 client.26549 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:58:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 --2- 172.21.15.22:0/1102747526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e49fd45fd0 0x55e49fd563d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 >> 172.21.15.22:0/1102747526 conn(0x55e49fd0d5b0 msgr2=0x55e49fd4ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 shutdown_connections 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 1 -- 172.21.15.22:0/1102747526 wait complete. 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.289+0000 7f079a228740 20 client.26549 trim_cache size 0 max 16384 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:58:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:58:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f077c0a2a40 msgr2=0x7f077c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f077c0a2a40 0x7f077c0a4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0788005c40 tx=0x7f0788009500 comp rx=0 tx=0).stop 2024-03-08T23:58:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 msgr2=0x7f07900735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:58:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07900735f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c02cb30 comp rx=0 tx=0).stop 2024-03-08T23:58:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 shutdown_connections 2024-03-08T23:58:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f077c0a2a40 0x7f077c0a4f00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0790074070 0x7f07901a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07901065d0 0x7f0790073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 --2- 172.21.15.22:0/3647637726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0790101500 0x7f07900735f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:58:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 >> 172.21.15.22:0/3647637726 conn(0x7f07900fcd80 msgr2=0x7f07900ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:58:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 shutdown_connections 2024-03-08T23:58:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:58:51.290+0000 7f0797644640 1 -- 172.21.15.22:0/3647637726 wait complete. 2024-03-08T23:58:51.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:51 smithi062 ceph-mon[36129]: daemon mds.h restarted 2024-03-08T23:58:51.611 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setuser ceph since I am not root 2024-03-08T23:58:51.611 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setgroup ceph since I am not root 2024-03-08T23:58:51.611 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:51 smithi062 ceph-mds[70095]: -- 172.21.15.62:0/2674614886 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559a254e7a40 con 0x559a254ba800 2024-03-08T23:58:51.611 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 08 23:58:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: starting mds.h at 2024-03-08T23:58:51.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 415985 11.38 MB/sec execute 59 sec latency 170.668 ms 2024-03-08T23:58:51.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:51 smithi192 ceph-mon[35995]: daemon mds.h restarted 2024-03-08T23:58:51.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:51 smithi022 ceph-mon[31894]: daemon mds.h restarted 2024-03-08T23:58:52.712 INFO:tasks.workunit.client.0.smithi022.stdout: 1 418702 11.47 MB/sec execute 60 sec latency 122.403 ms 2024-03-08T23:58:52.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:52 smithi062 ceph-mon[36129]: pgmap v135: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 431 op/s 2024-03-08T23:58:52.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:52 smithi062 ceph-mon[36129]: osdmap e98: 12 total, 12 up, 12 in 2024-03-08T23:58:52.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:52 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6836/2421521561,v1:172.21.15.62:6837/2421521561] up:boot 2024-03-08T23:58:52.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:52 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:58:52.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:52 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:58:52.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:52 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:58:52.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:52 smithi192 ceph-mon[35995]: pgmap v135: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 431 op/s 2024-03-08T23:58:52.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:52 smithi192 ceph-mon[35995]: osdmap e98: 12 total, 12 up, 12 in 2024-03-08T23:58:52.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:52 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6836/2421521561,v1:172.21.15.62:6837/2421521561] up:boot 2024-03-08T23:58:52.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:52 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:58:52.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:52 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:58:52.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:52 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:58:52.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:52 smithi022 ceph-mon[31894]: pgmap v135: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 431 op/s 2024-03-08T23:58:52.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:52 smithi022 ceph-mon[31894]: osdmap e98: 12 total, 12 up, 12 in 2024-03-08T23:58:52.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:52 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6836/2421521561,v1:172.21.15.62:6837/2421521561] up:boot 2024-03-08T23:58:52.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:52 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-08T23:58:52.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:52 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T23:58:52.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:52 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-08T23:58:53.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 420409 11.40 MB/sec execute 61 sec latency 22.710 ms 2024-03-08T23:58:54.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 422853 11.44 MB/sec execute 62 sec latency 14.883 ms 2024-03-08T23:58:54.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:54 smithi062 ceph-mon[36129]: pgmap v137: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 311 op/s 2024-03-08T23:58:54.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:54 smithi192 ceph-mon[35995]: pgmap v137: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 311 op/s 2024-03-08T23:58:54.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:54 smithi022 ceph-mon[31894]: pgmap v137: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 311 op/s 2024-03-08T23:58:55.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 425838 11.54 MB/sec execute 63 sec latency 14.072 ms 2024-03-08T23:58:56.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 428112 11.49 MB/sec execute 64 sec latency 11.433 ms 2024-03-08T23:58:56.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:56 smithi062 ceph-mon[36129]: pgmap v138: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 493 op/s 2024-03-08T23:58:56.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:56 smithi192 ceph-mon[35995]: pgmap v138: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 493 op/s 2024-03-08T23:58:56.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:56 smithi022 ceph-mon[31894]: pgmap v138: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 493 op/s 2024-03-08T23:58:57.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 429908 11.50 MB/sec execute 65 sec latency 132.033 ms 2024-03-08T23:58:58.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 432838 11.59 MB/sec execute 66 sec latency 21.377 ms 2024-03-08T23:58:58.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:58:58 smithi062 ceph-mon[36129]: pgmap v139: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 493 op/s 2024-03-08T23:58:58.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:58:58 smithi192 ceph-mon[35995]: pgmap v139: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 493 op/s 2024-03-08T23:58:58.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:58:58 smithi022 ceph-mon[31894]: pgmap v139: 129 pgs: 129 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 493 op/s 2024-03-08T23:58:59.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 435094 11.54 MB/sec execute 67 sec latency 16.281 ms 2024-03-08T23:59:00.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 437122 11.56 MB/sec execute 68 sec latency 11.360 ms 2024-03-08T23:59:00.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:00 smithi062 ceph-mon[36129]: pgmap v140: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 532 op/s 2024-03-08T23:59:00.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:00 smithi192 ceph-mon[35995]: pgmap v140: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 532 op/s 2024-03-08T23:59:00.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:00 smithi022 ceph-mon[31894]: pgmap v140: 129 pgs: 129 active+clean; 53 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 532 op/s 2024-03-08T23:59:01.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 437468 11.41 MB/sec execute 69 sec latency 103.628 ms 2024-03-08T23:59:02.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 440059 11.49 MB/sec execute 70 sec latency 113.127 ms 2024-03-08T23:59:02.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:02 smithi062 ceph-mon[36129]: pgmap v141: 129 pgs: 129 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 478 op/s 2024-03-08T23:59:02.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:02 smithi192 ceph-mon[35995]: pgmap v141: 129 pgs: 129 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 478 op/s 2024-03-08T23:59:02.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:02 smithi022 ceph-mon[31894]: pgmap v141: 129 pgs: 129 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 478 op/s 2024-03-08T23:59:03.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 441749 11.41 MB/sec execute 71 sec latency 21.365 ms 2024-03-08T23:59:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 -- 172.21.15.22:0/1295582403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754105190 msgr2=0x7f6754105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 --2- 172.21.15.22:0/1295582403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754105190 0x7f6754105570 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6744009900 tx=0x7f674402bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 -- 172.21.15.22:0/1295582403 shutdown_connections 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 --2- 172.21.15.22:0/1295582403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6754100e00 0x7f675410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 --2- 172.21.15.22:0/1295582403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6754100440 0x7f67541008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 --2- 172.21.15.22:0/1295582403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754105190 0x7f6754105570 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 -- 172.21.15.22:0/1295582403 >> 172.21.15.22:0/1295582403 conn(0x7f6754077040 msgr2=0x7f6754077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 -- 172.21.15.22:0/1295582403 shutdown_connections 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 -- 172.21.15.22:0/1295582403 wait complete. 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 Processor -- start 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 -- start start 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.379+0000 7f675aa43640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754100440 0x7f675419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f675aa43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6754100e00 0x7f675419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f675aa43640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6754105190 0x7f6754196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f675aa43640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6754110680 con 0x7f6754100440 2024-03-08T23:59:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f675aa43640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6754110500 con 0x7f6754100e00 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f675aa43640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6754110800 con 0x7f6754105190 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754100440 0x7f675419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754100440 0x7f675419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59836/0 (socket says 172.21.15.22:59836) 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 -- 172.21.15.22:0/1154959109 learned_addr learned my addr 172.21.15.22:0/1154959109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6758fe0640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6754105190 0x7f6754196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f67537fe640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6754100e00 0x7f675419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 -- 172.21.15.22:0/1154959109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6754105190 msgr2=0x7f6754196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6754105190 0x7f6754196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 -- 172.21.15.22:0/1154959109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6754100e00 msgr2=0x7f675419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6754100e00 0x7f675419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 -- 172.21.15.22:0/1154959109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754197010 con 0x7f6754100440 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f6753fff640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754100440 0x7f675419c6d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f674402efd0 tx=0x7f67440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f67517fa640 1 -- 172.21.15.22:0/1154959109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744041070 con 0x7f6754100440 2024-03-08T23:59:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f67517fa640 1 -- 172.21.15.22:0/1154959109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f674402ccc0 con 0x7f6754100440 2024-03-08T23:59:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67541972a0 con 0x7f6754100440 2024-03-08T23:59:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f67517fa640 1 -- 172.21.15.22:0/1154959109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744003d40 con 0x7f6754100440 2024-03-08T23:59:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.380+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67541a3520 con 0x7f6754100440 2024-03-08T23:59:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.382+0000 7f67517fa640 1 -- 172.21.15.22:0/1154959109 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6744044070 con 0x7f6754100440 2024-03-08T23:59:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 Processor -- start 2024-03-08T23:59:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 -- start start 2024-03-08T23:59:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c63838a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63838fd0 0x563c63839450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63842550 0x563c63844940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f67517fa640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f673c0a2a90 0x7f673c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6379d5f0 con 0x563c63839d40 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c63746930 con 0x563c63842550 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f675d61b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c63799690 con 0x563c63838fd0 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.383+0000 7f6758fe0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c63838a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f67537fe640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f673c0a2a90 0x7f673c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c63838a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59850/0 (socket says 172.21.15.22:59850) 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 -- 172.21.15.22:0/259189985 learned_addr learned my addr 172.21.15.22:0/259189985 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6753fff640 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63838fd0 0x563c63839450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f67517fa640 1 -- 172.21.15.22:0/1154959109 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f67440f0b20 con 0x7f6754100440 2024-03-08T23:59:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63842550 0x563c63844940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 -- 172.21.15.22:0/259189985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63838fd0 msgr2=0x563c63839450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63838fd0 0x563c63839450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 -- 172.21.15.22:0/259189985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63842550 msgr2=0x563c63844940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63842550 0x563c63844940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 -- 172.21.15.22:0/259189985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c634b2210 con 0x563c63839d40 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f67537fe640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f673c0a2a90 0x7f673c0a4f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6754104e00 tx=0x7f6748005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f6758fe0640 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c63838a90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f674c013950 tx=0x7f674c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eb056b21b8b189 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f673a7fc640 1 -- 172.21.15.22:0/259189985 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c004b20 con 0x563c63839d40 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f673a7fc640 1 -- 172.21.15.22:0/259189985 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f674c003e50 con 0x563c63839d40 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.384+0000 7f673a7fc640 1 -- 172.21.15.22:0/259189985 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00fea0 con 0x563c63839d40 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 -- 172.21.15.22:0/259189985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 msgr2=0x563c63838a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c63838a90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f674c013950 tx=0x7f674c009bd0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 -- 172.21.15.22:0/259189985 shutdown_connections 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63842550 0x563c63844940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63838fd0 0x563c63839450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 --2- 172.21.15.22:0/259189985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c63838a90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 -- 172.21.15.22:0/259189985 >> 172.21.15.22:0/259189985 conn(0x563c638005b0 msgr2=0x563c63833cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 -- 172.21.15.22:0/259189985 shutdown_connections 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 -- 172.21.15.22:0/259189985 wait complete. 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 -- start start 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.385+0000 7f675d61b740 1 Processor -- start 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63838fd0 0x563c6383f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c6383faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c63845280 con 0x563c63839d40 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c63759d90 con 0x563c63838fd0 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c63845400 con 0x563c63842550 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 10 client.0 _ll_register_callbacks cb 0x563c634b1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c6383faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f6758fe0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c6383faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59854/0 (socket says 172.21.15.22:59854) 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 -- 172.21.15.22:0/1223201548 learned_addr learned my addr 172.21.15.22:0/1223201548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f6753fff640 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63838fd0 0x563c6383f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 -- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 msgr2=0x563c6383ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 -- 172.21.15.22:0/1223201548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63838fd0 msgr2=0x563c6383f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63838fd0 0x563c6383f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 -- 172.21.15.22:0/1223201548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c6357e620 con 0x563c63839d40 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f6758fe0640 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67537fe640 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c6383faa0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f674801cde0 tx=0x7f6748004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d5755378c0b947 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67337fe640 1 -- 172.21.15.22:0/1223201548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674801c050 con 0x563c63839d40 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f67337fe640 1 -- 172.21.15.22:0/1223201548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f674801c1f0 con 0x563c63839d40 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.386+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f674c003e50 con 0x563c63839d40 2024-03-08T23:59:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f67337fe640 1 -- 172.21.15.22:0/1223201548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6748031050 con 0x563c63839d40 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 10 client.17130 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563c638515a0 con 0x563c63839d40 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f67337fe640 1 -- 172.21.15.22:0/1223201548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f674c003e50 con 0x563c63839d40 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f67337fe640 1 -- 172.21.15.22:0/1223201548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6748026500 con 0x563c63839d40 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 10 client.17130 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 20 client.17130 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c63852ab0 0x563c63854ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 4 client.17130 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c637aab30 con 0x563c63852ab0 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.387+0000 7f6753fff640 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c63852ab0 0x563c63854ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.388+0000 7f6753fff640 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c63852ab0 0x563c63854ea0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5a2b4734398ec3 server_cookie=8a9ae99d5654f68 in_seq=0 out_seq=0 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.388+0000 7f67337fe640 10 client.17130 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.388+0000 7f67337fe640 1 -- 172.21.15.22:0/1223201548 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c637aab30 con 0x563c63852ab0 2024-03-08T23:59:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f67337fe640 10 client.17130 handle_command_reply: tid=0 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 client.17130 shutdown 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 msgr2=0x563c6383faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c6383faa0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f674801cde0 tx=0x7f6748004730 comp rx=0 tx=0).stop 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 shutdown_connections 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c63852ab0 0x563c63854ea0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63839d40 0x563c6383faa0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 --2- 172.21.15.22:0/1223201548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63838fd0 0x563c6383f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 >> 172.21.15.22:0/1223201548 conn(0x563c638005b0 msgr2=0x563c63835a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f67337fe640 0 client.17130 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 shutdown_connections 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 1 -- 172.21.15.22:0/1223201548 wait complete. 2024-03-08T23:59:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.389+0000 7f675d61b740 20 client.17130 trim_cache size 0 max 16384 2024-03-08T23:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 Processor -- start 2024-03-08T23:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 -- start start 2024-03-08T23:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63841c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c6383de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541ae790 con 0x563c63838fd0 2024-03-08T23:59:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541ac330 con 0x563c63839d40 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f675d61b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541aa3d0 con 0x563c63842550 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63841c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63841c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59856/0 (socket says 172.21.15.22:59856) 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 -- 172.21.15.22:0/2859547812 learned_addr learned my addr 172.21.15.22:0/2859547812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6758fe0640 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f67537fe640 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c6383de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 -- 172.21.15.22:0/2859547812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 msgr2=0x563c6383e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 -- 172.21.15.22:0/2859547812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 msgr2=0x563c6383de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c6383de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.403+0000 7f6753fff640 1 -- 172.21.15.22:0/2859547812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c638382a0 con 0x563c63838fd0 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f6753fff640 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63841c90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f67440b93a0 tx=0x7f67440b23d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0b643c19980990 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f6739ffb640 1 -- 172.21.15.22:0/2859547812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674404b040 con 0x563c63838fd0 2024-03-08T23:59:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f6739ffb640 1 -- 172.21.15.22:0/2859547812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67440bcf40 con 0x563c63838fd0 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f6739ffb640 1 -- 172.21.15.22:0/2859547812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744003680 con 0x563c63838fd0 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 -- 172.21.15.22:0/2859547812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 msgr2=0x563c63841c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63841c90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f67440b93a0 tx=0x7f67440b23d0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 -- 172.21.15.22:0/2859547812 shutdown_connections 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c6383e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c6383de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 --2- 172.21.15.22:0/2859547812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63841c90 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 -- 172.21.15.22:0/2859547812 >> 172.21.15.22:0/2859547812 conn(0x563c638005b0 msgr2=0x563c63859840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.404+0000 7f675d61b740 1 -- 172.21.15.22:0/2859547812 shutdown_connections 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 -- 172.21.15.22:0/2859547812 wait complete. 2024-03-08T23:59:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 -- start start 2024-03-08T23:59:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 Processor -- start 2024-03-08T23:59:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63848220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c63848760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c63848ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541aa9d0 con 0x563c63838fd0 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541abcd0 con 0x563c63839d40 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541ab010 con 0x563c63842550 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f675d61b740 10 client.0 _ll_register_callbacks cb 0x7f674c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f6753fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63848220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f6753fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63848220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59858/0 (socket says 172.21.15.22:59858) 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f6753fff640 1 -- 172.21.15.22:0/2758937474 learned_addr learned my addr 172.21.15.22:0/2758937474 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f67537fe640 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c63848760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.405+0000 7f6758fe0640 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c63848ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f6753fff640 1 -- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 msgr2=0x563c63848ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f6753fff640 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c63848ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f6753fff640 1 -- 172.21.15.22:0/2758937474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 msgr2=0x563c63848760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f6753fff640 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c63848760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f6753fff640 1 -- 172.21.15.22:0/2758937474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c6383b140 con 0x563c63838fd0 2024-03-08T23:59:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f6753fff640 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63848220 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f67440b9910 tx=0x7f67440056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39613b3ed76d1b5e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f67337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f67337fe640 1 -- 172.21.15.22:0/2758937474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67440d0030 con 0x563c63838fd0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f67337fe640 1 -- 172.21.15.22:0/2758937474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f67440b95b0 con 0x563c63838fd0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67440bcf40 con 0x563c63838fd0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f67337fe640 1 -- 172.21.15.22:0/2758937474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674404b050 con 0x563c63838fd0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f675d61b740 10 client.17136 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563c638aa310 con 0x563c63838fd0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.406+0000 7f67337fe640 1 -- 172.21.15.22:0/2758937474 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67440bcf40 con 0x563c63838fd0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f67337fe640 1 -- 172.21.15.22:0/2758937474 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f67440c51a0 con 0x563c63838fd0 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f675d61b740 10 client.17136 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f675d61b740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f675d61b740 20 client.17136 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f675d61b740 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c638aa8d0 0x563c638accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f675d61b740 4 client.17136 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:59:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c637a4e80 con 0x563c638aa8d0 2024-03-08T23:59:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f67537fe640 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c638aa8d0 0x563c638accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f67537fe640 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c638aa8d0 0x563c638accc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea736856d3e815f server_cookie=efc629c0b6a95b3a in_seq=0 out_seq=0 2024-03-08T23:59:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.407+0000 7f67337fe640 10 client.17136 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f67337fe640 1 -- 172.21.15.22:0/2758937474 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c637a4e80 con 0x563c638aa8d0 2024-03-08T23:59:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f67337fe640 10 client.17136 handle_command_reply: tid=0 2024-03-08T23:59:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 client.17136 shutdown 2024-03-08T23:59:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 msgr2=0x563c63848220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63848220 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f67440b9910 tx=0x7f67440056f0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 shutdown_connections 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563c638aa8d0 0x563c638accc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563c63842550 0x563c63848ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c63839d40 0x563c63848760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 --2- 172.21.15.22:0/2758937474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563c63838fd0 0x563c63848220 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f67337fe640 0 client.17136 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 >> 172.21.15.22:0/2758937474 conn(0x563c638005b0 msgr2=0x563c6383ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 shutdown_connections 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 1 -- 172.21.15.22:0/2758937474 wait complete. 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.655+0000 7f675d61b740 20 client.17136 trim_cache size 0 max 16384 2024-03-08T23:59:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:59:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:59:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:59:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:59:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f673c0a2a90 msgr2=0x7f673c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f673c0a2a90 0x7f673c0a4f50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6754104e00 tx=0x7f6748005f90 comp rx=0 tx=0).stop 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754100440 msgr2=0x7f675419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754100440 0x7f675419c6d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f674402efd0 tx=0x7f67440042f0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 shutdown_connections 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f673c0a2a90 0x7f673c0a4f50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6754105190 0x7f6754196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6754100e00 0x7f675419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 --2- 172.21.15.22:0/1154959109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6754100440 0x7f675419c6d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 >> 172.21.15.22:0/1154959109 conn(0x7f6754077040 msgr2=0x7f67540fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 shutdown_connections 2024-03-08T23:59:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:04.656+0000 7f675aa43640 1 -- 172.21.15.22:0/1154959109 wait complete. 2024-03-08T23:59:04.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 443788 11.44 MB/sec execute 72 sec latency 132.882 ms 2024-03-08T23:59:04.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:04 smithi062 ceph-mon[36129]: pgmap v142: 129 pgs: 129 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 418 op/s 2024-03-08T23:59:04.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:04 smithi192 ceph-mon[35995]: pgmap v142: 129 pgs: 129 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 418 op/s 2024-03-08T23:59:04.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:04 smithi022 ceph-mon[31894]: pgmap v142: 129 pgs: 129 active+clean; 55 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 418 op/s 2024-03-08T23:59:05.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 446401 11.46 MB/sec execute 73 sec latency 21.017 ms 2024-03-08T23:59:06.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 448381 11.46 MB/sec execute 74 sec latency 17.447 ms 2024-03-08T23:59:06.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:06 smithi062 ceph-mon[36129]: pgmap v143: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 542 op/s 2024-03-08T23:59:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:06 smithi192 ceph-mon[35995]: pgmap v143: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 542 op/s 2024-03-08T23:59:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:06 smithi022 ceph-mon[31894]: pgmap v143: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 542 op/s 2024-03-08T23:59:07.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 450747 11.49 MB/sec execute 75 sec latency 59.490 ms 2024-03-08T23:59:08.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 452321 11.41 MB/sec execute 76 sec latency 32.063 ms 2024-03-08T23:59:08.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:08 smithi062 ceph-mon[36129]: pgmap v144: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-03-08T23:59:08.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:08 smithi192 ceph-mon[35995]: pgmap v144: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-03-08T23:59:08.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:08 smithi022 ceph-mon[31894]: pgmap v144: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-03-08T23:59:09.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 454609 11.44 MB/sec execute 77 sec latency 21.693 ms 2024-03-08T23:59:10.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 457008 11.46 MB/sec execute 78 sec latency 19.464 ms 2024-03-08T23:59:10.787 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:10 smithi022 ceph-mon[31894]: pgmap v145: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 367 op/s 2024-03-08T23:59:10.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:10 smithi062 ceph-mon[36129]: pgmap v145: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 367 op/s 2024-03-08T23:59:10.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:10 smithi192 ceph-mon[35995]: pgmap v145: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 367 op/s 2024-03-08T23:59:11.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 458551 11.50 MB/sec execute 79 sec latency 86.431 ms 2024-03-08T23:59:12.713 INFO:tasks.workunit.client.0.smithi022.stdout: 1 460006 11.58 MB/sec execute 80 sec latency 141.691 ms 2024-03-08T23:59:12.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:12 smithi062 ceph-mon[36129]: pgmap v146: 129 pgs: 129 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 473 op/s 2024-03-08T23:59:12.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:12 smithi192 ceph-mon[35995]: pgmap v146: 129 pgs: 129 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 473 op/s 2024-03-08T23:59:12.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:12 smithi022 ceph-mon[31894]: pgmap v146: 129 pgs: 129 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 473 op/s 2024-03-08T23:59:13.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 462362 11.61 MB/sec execute 81 sec latency 19.262 ms 2024-03-08T23:59:14.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 464468 11.57 MB/sec execute 82 sec latency 21.198 ms 2024-03-08T23:59:14.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:14 smithi062 ceph-mon[36129]: pgmap v147: 129 pgs: 129 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 418 op/s 2024-03-08T23:59:14.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:14 smithi192 ceph-mon[35995]: pgmap v147: 129 pgs: 129 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 418 op/s 2024-03-08T23:59:14.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:14 smithi022 ceph-mon[31894]: pgmap v147: 129 pgs: 129 active+clean; 54 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 418 op/s 2024-03-08T23:59:15.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 466609 11.59 MB/sec execute 83 sec latency 18.069 ms 2024-03-08T23:59:16.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:59:16.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:59:16.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 -- 172.21.15.22:0/2913725065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1065c0 msgr2=0x7fb94c10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2913725065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1065c0 0x7fb94c10e190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb944009ca0 tx=0x7fb94402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 -- 172.21.15.22:0/2913725065 shutdown_connections 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2913725065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1065c0 0x7fb94c10e190 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2913725065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c101ea0 0x7fb94c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2913725065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb94c1014f0 0x7fb94c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 -- 172.21.15.22:0/2913725065 >> 172.21.15.22:0/2913725065 conn(0x7fb94c0fcd90 msgr2=0x7fb94c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 -- 172.21.15.22:0/2913725065 shutdown_connections 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.378+0000 7fb952b3b640 1 -- 172.21.15.22:0/2913725065 wait complete. 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 Processor -- start 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 -- start start 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1014f0 0x7fb94c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb94c1065c0 0x7fb94c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c197a00 0x7fb94c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c111680 con 0x7fb94c1014f0 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c111500 con 0x7fb94c197a00 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb952b3b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c111800 con 0x7fb94c1065c0 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb9508d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1014f0 0x7fb94c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb9508d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1014f0 0x7fb94c19d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43518/0 (socket says 172.21.15.22:43518) 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb9508d7640 1 -- 172.21.15.22:0/2610074374 learned_addr learned my addr 172.21.15.22:0/2610074374 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb94bfff640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb94c1065c0 0x7fb94c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.379+0000 7fb9510d8640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c197a00 0x7fb94c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb9508d7640 1 -- 172.21.15.22:0/2610074374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb94c1065c0 msgr2=0x7fb94c19dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb9508d7640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb94c1065c0 0x7fb94c19dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb9508d7640 1 -- 172.21.15.22:0/2610074374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c197a00 msgr2=0x7fb94c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb9508d7640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c197a00 0x7fb94c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb9508d7640 1 -- 172.21.15.22:0/2610074374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c198520 con 0x7fb94c1014f0 2024-03-08T23:59:16.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb94bfff640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb94c1065c0 0x7fb94c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:16.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb9508d7640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1014f0 0x7fb94c19d850 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb94000e9f0 tx=0x7fb94000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb949ffb640 1 -- 172.21.15.22:0/2610074374 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000cd80 con 0x7fb94c1014f0 2024-03-08T23:59:16.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb949ffb640 1 -- 172.21.15.22:0/2610074374 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb940005500 con 0x7fb94c1014f0 2024-03-08T23:59:16.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c198810 con 0x7fb94c1014f0 2024-03-08T23:59:16.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb949ffb640 1 -- 172.21.15.22:0/2610074374 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940010690 con 0x7fb94c1014f0 2024-03-08T23:59:16.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.380+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c1a4a10 con 0x7fb94c1014f0 2024-03-08T23:59:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.382+0000 7fb949ffb640 1 -- 172.21.15.22:0/2610074374 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb9400046c0 con 0x7fb94c1014f0 2024-03-08T23:59:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.383+0000 7fb9557f6740 1 Processor -- start 2024-03-08T23:59:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.383+0000 7fb949ffb640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb93c0a2970 0x7fb93c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.383+0000 7fb9557f6740 1 -- start start 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.383+0000 7fb9510d8640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb93c0a2970 0x7fb93c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.383+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.383+0000 7fb949ffb640 1 -- 172.21.15.22:0/2610074374 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb9400cf660 con 0x7fb94c1014f0 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.383+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d8080 0x558af54e0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54e3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9557f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af543ae00 con 0x558af54e0c60 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9557f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af53e3b10 con 0x558af54d7760 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9557f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af5436ea0 con 0x558af54d8080 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9510d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9510d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54e3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43530/0 (socket says 172.21.15.22:43530) 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9508d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9510d8640 1 -- 172.21.15.22:0/1847718959 learned_addr learned my addr 172.21.15.22:0/1847718959 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb94bfff640 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d8080 0x558af54e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9508d7640 1 -- 172.21.15.22:0/1847718959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d8080 msgr2=0x558af54e0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9508d7640 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d8080 0x558af54e0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9508d7640 1 -- 172.21.15.22:0/1847718959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 msgr2=0x558af54e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9508d7640 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54e3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9508d7640 1 -- 172.21.15.22:0/1847718959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af52197d0 con 0x558af54d7760 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb94bfff640 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d8080 0x558af54e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9510d8640 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9510d8640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb93c0a2970 0x7fb93c0a4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb944037040 tx=0x7fb9440323c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.384+0000 7fb9508d7640 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54d7b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9400cc9b0 tx=0x7fb94009b220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96b95d486c7c7e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb936ffd640 1 -- 172.21.15.22:0/1847718959 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94009fa10 con 0x558af54d7760 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb936ffd640 1 -- 172.21.15.22:0/1847718959 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9400b4070 con 0x558af54d7760 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb936ffd640 1 -- 172.21.15.22:0/1847718959 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400cf220 con 0x558af54d7760 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 -- 172.21.15.22:0/1847718959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 msgr2=0x558af54d7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54d7b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9400cc9b0 tx=0x7fb94009b220 comp rx=0 tx=0).stop 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 -- 172.21.15.22:0/1847718959 shutdown_connections 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d8080 0x558af54e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 --2- 172.21.15.22:0/1847718959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54d7b40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 -- 172.21.15.22:0/1847718959 >> 172.21.15.22:0/1847718959 conn(0x558af54d1e90 msgr2=0x558af54d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 -- 172.21.15.22:0/1847718959 shutdown_connections 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.385+0000 7fb9557f6740 1 -- 172.21.15.22:0/1847718959 wait complete. 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 -- start start 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 Processor -- start 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af54e7e00 con 0x558af54d8080 2024-03-08T23:59:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af53f70a0 con 0x558af54d7760 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af54e7f80 con 0x558af54e0c60 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9557f6740 10 client.0 _ll_register_callbacks cb 0x558af53f6a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9508d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54dd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43534/0 (socket says 172.21.15.22:43534) 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb94bfff640 1 -- 172.21.15.22:0/4236656240 learned_addr learned my addr 172.21.15.22:0/4236656240 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.386+0000 7fb9510d8640 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb94bfff640 1 -- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 msgr2=0x558af54ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb94bfff640 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb94bfff640 1 -- 172.21.15.22:0/4236656240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 msgr2=0x558af54e03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb94bfff640 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54e03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb94bfff640 1 -- 172.21.15.22:0/4236656240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558af514f3f0 con 0x558af54d8080 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb9508d7640 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54e03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb94bfff640 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54dd750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb9380116c0 tx=0x7fb938011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd24369eb4a82c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb92bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb92bfff640 1 -- 172.21.15.22:0/4236656240 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938010c60 con 0x558af54d8080 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb92bfff640 1 -- 172.21.15.22:0/4236656240 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb938010e00 con 0x558af54d8080 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9400b4070 con 0x558af54d8080 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb92bfff640 1 -- 172.21.15.22:0/4236656240 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb938025050 con 0x558af54d8080 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb9557f6740 10 client.17148 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558af510f0f0 con 0x558af54d8080 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.387+0000 7fb92bfff640 1 -- 172.21.15.22:0/4236656240 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9400b4070 con 0x558af54d8080 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb92bfff640 1 -- 172.21.15.22:0/4236656240 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb93800f500 con 0x558af54d8080 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9557f6740 10 client.17148 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9557f6740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9557f6740 20 client.17148 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9557f6740 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af54f4ac0 0x558af54f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9557f6740 4 client.17148 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558af5448340 con 0x558af54f4ac0 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9508d7640 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af54f4ac0 0x558af54f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb9508d7640 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af54f4ac0 0x558af54f6ed0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b139aa1dc05a93 server_cookie=34c1214313d99fc7 in_seq=0 out_seq=0 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.388+0000 7fb92bfff640 10 client.17148 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb92bfff640 1 -- 172.21.15.22:0/4236656240 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558af5448340 con 0x558af54f4ac0 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb92bfff640 10 client.17148 handle_command_reply: tid=0 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 client.17148 shutdown 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 msgr2=0x558af54dd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54dd750 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb9380116c0 tx=0x7fb938011b80 comp rx=0 tx=0).stop 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 shutdown_connections 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af54f4ac0 0x558af54f6ed0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54dd750 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb92bfff640 0 client.17148 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 --2- 172.21.15.22:0/4236656240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54e03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 >> 172.21.15.22:0/4236656240 conn(0x558af54d1e90 msgr2=0x558af54d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 shutdown_connections 2024-03-08T23:59:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 1 -- 172.21.15.22:0/4236656240 wait complete. 2024-03-08T23:59:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.389+0000 7fb9557f6740 20 client.17148 trim_cache size 0 max 16384 2024-03-08T23:59:16.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.406+0000 7fb9557f6740 1 Processor -- start 2024-03-08T23:59:16.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.406+0000 7fb9557f6740 1 -- start start 2024-03-08T23:59:16.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9557f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1aea00 con 0x558af54d8080 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9557f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1ac5a0 con 0x558af54d7760 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9557f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1aa640 con 0x558af54e0c60 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54e03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43536/0 (socket says 172.21.15.22:43536) 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 -- 172.21.15.22:0/354577186 learned_addr learned my addr 172.21.15.22:0/354577186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9508d7640 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9510d8640 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 -- 172.21.15.22:0/354577186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 msgr2=0x558af54da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 -- 172.21.15.22:0/354577186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 msgr2=0x558af54dfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54dfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 -- 172.21.15.22:0/354577186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af54cc330 con 0x558af54d8080 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9508d7640 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:16.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb94bfff640 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54e03c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb93800b620 tx=0x7fb93800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d1789bd54725b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.407+0000 7fb9367fc640 1 -- 172.21.15.22:0/354577186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93802d030 con 0x558af54d8080 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9367fc640 1 -- 172.21.15.22:0/354577186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb93800f4c0 con 0x558af54d8080 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9367fc640 1 -- 172.21.15.22:0/354577186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938010c80 con 0x558af54d8080 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 -- 172.21.15.22:0/354577186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 msgr2=0x558af54e03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54e03c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb93800b620 tx=0x7fb93800b480 comp rx=0 tx=0).stop 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 -- 172.21.15.22:0/354577186 shutdown_connections 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54e0c60 0x558af54da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54d8080 0x558af54e03c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 --2- 172.21.15.22:0/354577186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d7760 0x558af54dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 -- 172.21.15.22:0/354577186 >> 172.21.15.22:0/354577186 conn(0x558af54d1e90 msgr2=0x558af54dd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 -- 172.21.15.22:0/354577186 shutdown_connections 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.408+0000 7fb9557f6740 1 -- 172.21.15.22:0/354577186 wait complete. 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 -- start start 2024-03-08T23:59:16.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 Processor -- start 2024-03-08T23:59:16.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d7760 0x558af54eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d8080 0x558af54eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1aac40 con 0x558af54e0c60 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1abf40 con 0x558af54d8080 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1ab280 con 0x558af54d7760 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9557f6740 10 client.0 _ll_register_callbacks cb 0x7fb93800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9510d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9510d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54eb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43546/0 (socket says 172.21.15.22:43546) 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9510d8640 1 -- 172.21.15.22:0/3347668630 learned_addr learned my addr 172.21.15.22:0/3347668630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb9508d7640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d7760 0x558af54eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.409+0000 7fb94bfff640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d8080 0x558af54eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9510d8640 1 -- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d7760 msgr2=0x558af54eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9510d8640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d7760 0x558af54eae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9510d8640 1 -- 172.21.15.22:0/3347668630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d8080 msgr2=0x558af54eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9510d8640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d8080 0x558af54eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9510d8640 1 -- 172.21.15.22:0/3347668630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558af554b2a0 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9508d7640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d7760 0x558af54eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9510d8640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54eb880 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb944032530 tx=0x7fb94402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10982d08ab08b0a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb92bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb92bfff640 1 -- 172.21.15.22:0/3347668630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944057060 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb92bfff640 1 -- 172.21.15.22:0/3347668630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb944004230 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb93800f4c0 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb92bfff640 1 -- 172.21.15.22:0/3347668630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb94405e050 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9557f6740 10 client.17154 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558af554c5b0 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.410+0000 7fb92bfff640 1 -- 172.21.15.22:0/3347668630 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb93800f4c0 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb92bfff640 1 -- 172.21.15.22:0/3347668630 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb94403f040 con 0x558af54e0c60 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9557f6740 10 client.17154 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:16.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9557f6740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9557f6740 20 client.17154 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9557f6740 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af554cce0 0x558af554f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9557f6740 4 client.17154 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558af5442690 con 0x558af554cce0 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9508d7640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af554cce0 0x558af554f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb92bfff640 10 client.17154 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.411+0000 7fb9508d7640 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af554cce0 0x558af554f100 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d707b017177af8f server_cookie=29876ef0ceebd0ea in_seq=0 out_seq=0 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb92bfff640 1 -- 172.21.15.22:0/3347668630 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558af5442690 con 0x558af554cce0 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb92bfff640 10 client.17154 handle_command_reply: tid=0 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 client.17154 shutdown 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 msgr2=0x558af54eb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54eb880 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb944032530 tx=0x7fb94402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 shutdown_connections 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558af554cce0 0x558af554f100 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558af54e0c60 0x558af54eb880 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb92bfff640 0 client.17154 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558af54d8080 0x558af54eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 --2- 172.21.15.22:0/3347668630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558af54d7760 0x558af54eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 >> 172.21.15.22:0/3347668630 conn(0x558af54d1e90 msgr2=0x558af54d8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 shutdown_connections 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 1 -- 172.21.15.22:0/3347668630 wait complete. 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.412+0000 7fb9557f6740 20 client.17154 trim_cache size 0 max 16384 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:59:16.414 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:59:16.415 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:59:16.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb93c0a2970 msgr2=0x7fb93c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb93c0a2970 0x7fb93c0a4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb944037040 tx=0x7fb9440323c0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1014f0 msgr2=0x7fb94c19d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1014f0 0x7fb94c19d850 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb94000e9f0 tx=0x7fb94000edf0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 shutdown_connections 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb93c0a2970 0x7fb93c0a4e30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c197a00 0x7fb94c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb94c1065c0 0x7fb94c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 --2- 172.21.15.22:0/2610074374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb94c1014f0 0x7fb94c19d850 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 >> 172.21.15.22:0/2610074374 conn(0x7fb94c0fcd90 msgr2=0x7fb94c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 shutdown_connections 2024-03-08T23:59:16.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.413+0000 7fb952b3b640 1 -- 172.21.15.22:0/2610074374 wait complete. 2024-03-08T23:59:16.420 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:59:16.420 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:59:16.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 -- 172.21.15.22:0/333277626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 msgr2=0x7fc8f810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/333277626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f810d750 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009900 tx=0x7fc8e802bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 -- 172.21.15.22:0/333277626 shutdown_connections 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/333277626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8f810dc90 0x7fc8f8110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/333277626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f810d750 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/333277626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f81000e0 0x7fc8f81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 -- 172.21.15.22:0/333277626 >> 172.21.15.22:0/333277626 conn(0x7fc8f80772f0 msgr2=0x7fc8f8077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 -- 172.21.15.22:0/333277626 shutdown_connections 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.538+0000 7fc8ffe75640 1 -- 172.21.15.22:0/333277626 wait complete. 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 Processor -- start 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 -- start start 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8f81000e0 0x7fc8f8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f810dc90 0x7fc8f819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f81138e0 con 0x7fc8f8100a90 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8113760 con 0x7fc8f810dc90 2024-03-08T23:59:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8ffe75640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8113a60 con 0x7fc8f81000e0 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43570/0 (socket says 172.21.15.22:43570) 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 -- 172.21.15.22:0/2616752774 learned_addr learned my addr 172.21.15.22:0/2616752774 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fe412640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f810dc90 0x7fc8f819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fdc11640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8f81000e0 0x7fc8f8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 -- 172.21.15.22:0/2616752774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8f81000e0 msgr2=0x7fc8f8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8f81000e0 0x7fc8f8199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 -- 172.21.15.22:0/2616752774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f810dc90 msgr2=0x7fc8f819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f810dc90 0x7fc8f819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.539+0000 7fc8fd410640 1 -- 172.21.15.22:0/2616752774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f819ee20 con 0x7fc8f8100a90 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.540+0000 7fc8fd410640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f819a3c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8005940 tx=0x7fc8e802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.540+0000 7fc8eeffd640 1 -- 172.21.15.22:0/2616752774 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8041070 con 0x7fc8f8100a90 2024-03-08T23:59:16.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.540+0000 7fc8eeffd640 1 -- 172.21.15.22:0/2616752774 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8e8004450 con 0x7fc8f8100a90 2024-03-08T23:59:16.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.540+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f819f0b0 con 0x7fc8f8100a90 2024-03-08T23:59:16.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.540+0000 7fc8eeffd640 1 -- 172.21.15.22:0/2616752774 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8003df0 con 0x7fc8f8100a90 2024-03-08T23:59:16.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.540+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f81a69a0 con 0x7fc8f8100a90 2024-03-08T23:59:16.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.541+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f80680e0 con 0x7fc8f8100a90 2024-03-08T23:59:16.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.542+0000 7fc8eeffd640 1 -- 172.21.15.22:0/2616752774 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc8e8044070 con 0x7fc8f8100a90 2024-03-08T23:59:16.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.543+0000 7fc8eeffd640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc8e40a2970 0x7fc8e40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.543+0000 7fc8fdc11640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc8e40a2970 0x7fc8e40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.543+0000 7fc8eeffd640 1 -- 172.21.15.22:0/2616752774 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fc8e80f0de0 con 0x7fc8f8100a90 2024-03-08T23:59:16.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.543+0000 7fc8fdc11640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc8e40a2970 0x7fc8e40a4e30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009b60 tx=0x7fc8f4005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.547+0000 7fc8eeffd640 1 -- 172.21.15.22:0/2616752774 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc8e80bd890 con 0x7fc8f8100a90 2024-03-08T23:59:16.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 469278 11.65 MB/sec execute 84 sec latency 29.712 ms 2024-03-08T23:59:16.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:16 smithi022 ceph-mon[31894]: pgmap v148: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-08T23:59:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.792+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8f8108e20 con 0x7fc8f8100a90 2024-03-08T23:59:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.793+0000 7fc8eeffd640 1 -- 172.21.15.22:0/2616752774 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fc8e80bd190 con 0x7fc8f8100a90 2024-03-08T23:59:16.789 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:59:16.789 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:51.502302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26555":{"gid":26555,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2421521561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2421521561},{"type":"v1","addr":"172.21.15.62:6837","nonce":2421521561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:59:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-03-08T23:59:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.795+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc8e40a2970 msgr2=0x7fc8e40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc8e40a2970 0x7fc8e40a4e30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009b60 tx=0x7fc8f4005f90 comp rx=0 tx=0).stop 2024-03-08T23:59:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 msgr2=0x7fc8f819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f819a3c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8005940 tx=0x7fc8e802cb30 comp rx=0 tx=0).stop 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 shutdown_connections 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc8e40a2970 0x7fc8e40a4e30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f810dc90 0x7fc8f819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8f8100a90 0x7fc8f819a3c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 --2- 172.21.15.22:0/2616752774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8f81000e0 0x7fc8f8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 >> 172.21.15.22:0/2616752774 conn(0x7fc8f80772f0 msgr2=0x7fc8f810f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 shutdown_connections 2024-03-08T23:59:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.796+0000 7fc8ffe75640 1 -- 172.21.15.22:0/2616752774 wait complete. 2024-03-08T23:59:16.802 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:59:16.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:59:16.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:16 smithi062 ceph-mon[36129]: pgmap v148: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-08T23:59:16.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:16 smithi192 ceph-mon[35995]: pgmap v148: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 -- 172.21.15.22:0/1072398987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041014e0 msgr2=0x7f80041018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 --2- 172.21.15.22:0/1072398987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041014e0 0x7f80041018c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f800002bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 -- 172.21.15.22:0/1072398987 shutdown_connections 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 --2- 172.21.15.22:0/1072398987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80041065b0 0x7f800410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 --2- 172.21.15.22:0/1072398987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8004101e90 0x7f8004105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 --2- 172.21.15.22:0/1072398987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041014e0 0x7f80041018c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 -- 172.21.15.22:0/1072398987 >> 172.21.15.22:0/1072398987 conn(0x7f80040fcda0 msgr2=0x7f80040ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 -- 172.21.15.22:0/1072398987 shutdown_connections 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 -- 172.21.15.22:0/1072398987 wait complete. 2024-03-08T23:59:16.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 Processor -- start 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.917+0000 7f800c5a9640 1 -- start start 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800c5a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041014e0 0x7f800419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800c5a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041065b0 0x7f800419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800c5a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8004197a00 0x7f8004197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800c5a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8004111660 con 0x7f80041065b0 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800c5a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041114e0 con 0x7f8004197a00 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800c5a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041117e0 con 0x7f80041014e0 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041065b0 0x7f800419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041065b0 0x7f800419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43582/0 (socket says 172.21.15.22:43582) 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 -- 172.21.15.22:0/2777371419 learned_addr learned my addr 172.21.15.22:0/2777371419 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800a345640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041014e0 0x7f800419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800ab46640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8004197a00 0x7f8004197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 -- 172.21.15.22:0/2777371419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041014e0 msgr2=0x7f800419d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041014e0 0x7f800419d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 -- 172.21.15.22:0/2777371419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8004197a00 msgr2=0x7f8004197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8004197a00 0x7f8004197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 -- 172.21.15.22:0/2777371419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004198520 con 0x7f80041065b0 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f8009b44640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041065b0 0x7f800419dd90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400e9f0 tx=0x7f7ff400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f7ffb7fe640 1 -- 172.21.15.22:0/2777371419 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400ccc0 con 0x7f80041065b0 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f7ffb7fe640 1 -- 172.21.15.22:0/2777371419 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ff4005500 con 0x7f80041065b0 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.918+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8004198810 con 0x7f80041065b0 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.919+0000 7f7ffb7fe640 1 -- 172.21.15.22:0/2777371419 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4010690 con 0x7f80041065b0 2024-03-08T23:59:16.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.919+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80041a4a60 con 0x7f80041065b0 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.921+0000 7f7ffb7fe640 1 -- 172.21.15.22:0/2777371419 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7ff40046c0 con 0x7f80041065b0 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.921+0000 7f800f187740 1 Processor -- start 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800f187740 1 -- start start 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f7ffb7fe640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7ff00a29f0 0x7f7ff00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800f187740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7166021a0 0x55e716602580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800f187740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e716602ac0 0x55e71660b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800f187740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e71660da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800f187740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7165c9360 con 0x55e7166021a0 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800f187740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e716565ec0 con 0x55e71660b6a0 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800f187740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71650ed30 con 0x55e716602ac0 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800ab46640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7ff00a29f0 0x7f7ff00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f7ffb7fe640 1 -- 172.21.15.22:0/2777371419 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7ff40cf460 con 0x7f80041065b0 2024-03-08T23:59:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800a345640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7166021a0 0x55e716602580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800a345640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7166021a0 0x55e716602580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43590/0 (socket says 172.21.15.22:43590) 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800a345640 1 -- 172.21.15.22:0/3828501587 learned_addr learned my addr 172.21.15.22:0/3828501587 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f8009b44640 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e716602ac0 0x55e71660b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.922+0000 7f800ab46640 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e71660da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f800a345640 1 -- 172.21.15.22:0/3828501587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e716602ac0 msgr2=0x55e71660b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f800a345640 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e716602ac0 0x55e71660b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f800a345640 1 -- 172.21.15.22:0/3828501587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 msgr2=0x55e71660da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f800a345640 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e71660da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f800a345640 1 -- 172.21.15.22:0/3828501587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e71650dbc0 con 0x55e7166021a0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f800ab46640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7ff00a29f0 0x7f7ff00a4eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc005fd0 tx=0x7f7ffc013b00 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f800a345640 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7166021a0 0x55e716602580 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f800002c170 tx=0x7f8000005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d5f9643a7bee55 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f7ff8bfe640 1 -- 172.21.15.22:0/3828501587 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000042070 con 0x55e7166021a0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f7ff8bfe640 1 -- 172.21.15.22:0/3828501587 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8000004030 con 0x55e7166021a0 2024-03-08T23:59:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.923+0000 7f7ff8bfe640 1 -- 172.21.15.22:0/3828501587 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000004b50 con 0x55e7166021a0 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- 172.21.15.22:0/3828501587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7166021a0 msgr2=0x55e716602580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7166021a0 0x55e716602580 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f800002c170 tx=0x7f8000005be0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- 172.21.15.22:0/3828501587 shutdown_connections 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e71660da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e716602ac0 0x55e71660b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 --2- 172.21.15.22:0/3828501587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e7166021a0 0x55e716602580 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- 172.21.15.22:0/3828501587 >> 172.21.15.22:0/3828501587 conn(0x55e7165fc8d0 msgr2=0x55e7165fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- 172.21.15.22:0/3828501587 shutdown_connections 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- 172.21.15.22:0/3828501587 wait complete. 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- start start 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 Processor -- start 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7166021a0 0x55e7166086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716608bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71660b6a0 0x55e716609130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7165044a0 con 0x55e716602ac0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e716561f60 con 0x55e7166021a0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.924+0000 7f800f187740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71660e430 con 0x55e71660b6a0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716608bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f800f187740 10 client.0 _ll_register_callbacks cb 0x55e71655fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716608bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43594/0 (socket says 172.21.15.22:43594) 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 -- 172.21.15.22:0/281957780 learned_addr learned my addr 172.21.15.22:0/281957780 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f800ab46640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71660b6a0 0x55e716609130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f800a345640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7166021a0 0x55e7166086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 -- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71660b6a0 msgr2=0x55e716609130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71660b6a0 0x55e716609130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 -- 172.21.15.22:0/281957780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7166021a0 msgr2=0x55e7166086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7166021a0 0x55e7166086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 -- 172.21.15.22:0/281957780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e71627a3f0 con 0x55e716602ac0 2024-03-08T23:59:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f800ab46640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71660b6a0 0x55e716609130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f8009b44640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716608bf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400cf10 tx=0x7f7ff409b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29a69bfa4c35ad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f7fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f7fedffb640 1 -- 172.21.15.22:0/281957780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff409b7c0 con 0x55e716602ac0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f7fedffb640 1 -- 172.21.15.22:0/281957780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7ff40cf160 con 0x55e716602ac0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8000004030 con 0x55e716602ac0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f7fedffb640 1 -- 172.21.15.22:0/281957780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ff409f910 con 0x55e716602ac0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f800f187740 10 client.17172 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e7163447d0 con 0x55e716602ac0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.925+0000 7f7fedffb640 1 -- 172.21.15.22:0/281957780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8000004030 con 0x55e716602ac0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f7fedffb640 1 -- 172.21.15.22:0/281957780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ff4093210 con 0x55e716602ac0 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800f187740 10 client.17172 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800f187740 10 client.17172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800f187740 20 client.17172 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800f187740 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e71661b820 0x55e71661dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800f187740 4 client.17172 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e716573400 con 0x55e71661b820 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800a345640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e71661b820 0x55e71661dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f800a345640 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e71661b820 0x55e71661dc10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50407d1832a8c9da server_cookie=9ca3391ecd6bcc2d in_seq=0 out_seq=0 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.926+0000 7f7fedffb640 10 client.17172 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f7fedffb640 1 -- 172.21.15.22:0/281957780 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e716573400 con 0x55e71661b820 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f7fedffb640 10 client.17172 handle_command_reply: tid=0 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 client.17172 shutdown 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 msgr2=0x55e716608bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716608bf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400cf10 tx=0x7f7ff409b370 comp rx=0 tx=0).stop 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 shutdown_connections 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e71661b820 0x55e71661dc10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71660b6a0 0x55e716609130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716608bf0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f7fedffb640 0 client.17172 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 --2- 172.21.15.22:0/281957780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7166021a0 0x55e7166086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 >> 172.21.15.22:0/281957780 conn(0x55e7165fc8d0 msgr2=0x55e7165fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 shutdown_connections 2024-03-08T23:59:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 1 -- 172.21.15.22:0/281957780 wait complete. 2024-03-08T23:59:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.927+0000 7f800f187740 20 client.17172 trim_cache size 0 max 16384 2024-03-08T23:59:16.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 Processor -- start 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 -- start start 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e71660ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e716602ac0 0x55e716607690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71660b6a0 0x55e716605930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041b0960 con 0x55e71660b6a0 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041ae500 con 0x55e716602ac0 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800f187740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041ac5a0 con 0x55e7166021a0 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71660b6a0 0x55e716605930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71660b6a0 0x55e716605930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43608/0 (socket says 172.21.15.22:43608) 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 -- 172.21.15.22:0/959261574 learned_addr learned my addr 172.21.15.22:0/959261574 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800a345640 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e71660ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f8009b44640 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e716602ac0 0x55e716607690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 -- 172.21.15.22:0/959261574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 msgr2=0x55e71660ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e71660ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 -- 172.21.15.22:0/959261574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e716602ac0 msgr2=0x55e716607690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e716602ac0 0x55e716607690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.940+0000 7f800ab46640 1 -- 172.21.15.22:0/959261574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7165fc700 con 0x55e71660b6a0 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800ab46640 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71660b6a0 0x55e716605930 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc012300 tx=0x7f7ffc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51ae1a1fb3edbf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f7feffff640 1 -- 172.21.15.22:0/959261574 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc003b70 con 0x55e71660b6a0 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f7feffff640 1 -- 172.21.15.22:0/959261574 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ffc003d10 con 0x55e71660b6a0 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f7feffff640 1 -- 172.21.15.22:0/959261574 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc010ac0 con 0x55e71660b6a0 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 -- 172.21.15.22:0/959261574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71660b6a0 msgr2=0x55e716605930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71660b6a0 0x55e716605930 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc012300 tx=0x7f7ffc01f000 comp rx=0 tx=0).stop 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 -- 172.21.15.22:0/959261574 shutdown_connections 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71660b6a0 0x55e716605930 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e716602ac0 0x55e716607690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 --2- 172.21.15.22:0/959261574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e71660ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 -- 172.21.15.22:0/959261574 >> 172.21.15.22:0/959261574 conn(0x55e7165fc8d0 msgr2=0x55e716606f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 -- 172.21.15.22:0/959261574 shutdown_connections 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.941+0000 7f800f187740 1 -- 172.21.15.22:0/959261574 wait complete. 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 -- start start 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 Processor -- start 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e7166116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716611be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e716671d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041acba0 con 0x55e716602ac0 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041adea0 con 0x55e71660b6a0 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041ad1e0 con 0x55e7166021a0 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800f187740 10 client.0 _ll_register_callbacks cb 0x7f7ff4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716611be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716611be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43616/0 (socket says 172.21.15.22:43616) 2024-03-08T23:59:16.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 -- 172.21.15.22:0/4013036023 learned_addr learned my addr 172.21.15.22:0/4013036023 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800a345640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e7166116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800ab46640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e716671d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 -- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 msgr2=0x55e7166116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e7166116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 -- 172.21.15.22:0/4013036023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 msgr2=0x55e716671d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e716671d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f8009b44640 1 -- 172.21.15.22:0/4013036023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e716672240 con 0x55e716602ac0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.942+0000 7f800a345640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e7166116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f8009b44640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716611be0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7ff409b4c0 tx=0x7f7ff409b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a373b532752decf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f7fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f7fedffb640 1 -- 172.21.15.22:0/4013036023 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff409b120 con 0x55e716602ac0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f7fedffb640 1 -- 172.21.15.22:0/4013036023 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f7ff409ac90 con 0x55e716602ac0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ffc003d10 con 0x55e716602ac0 2024-03-08T23:59:16.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f7fedffb640 1 -- 172.21.15.22:0/4013036023 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ff409b7c0 con 0x55e716602ac0 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 10 client.17178 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e716673420 con 0x55e716602ac0 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f7fedffb640 1 -- 172.21.15.22:0/4013036023 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ffc003d10 con 0x55e716602ac0 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f7fedffb640 1 -- 172.21.15.22:0/4013036023 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ff40afba0 con 0x55e716602ac0 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 10 client.17178 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 10 client.17178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 20 client.17178 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e716673900 0x55e716675d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 4 client.17178 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e71656d750 con 0x55e716673900 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.943+0000 7f800a345640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e716673900 0x55e716675d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800a345640 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e716673900 0x55e716675d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b51847d945cbd852 server_cookie=eec34bf27454d2a5 in_seq=0 out_seq=0 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f7fedffb640 10 client.17178 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f7fedffb640 1 -- 172.21.15.22:0/4013036023 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e71656d750 con 0x55e716673900 2024-03-08T23:59:16.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f7fedffb640 10 client.17178 handle_command_reply: tid=0 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 client.17178 shutdown 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 msgr2=0x55e716611be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716611be0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7ff409b4c0 tx=0x7f7ff409b300 comp rx=0 tx=0).stop 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 shutdown_connections 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55e716673900 0x55e716675d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71660b6a0 0x55e716671d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e716602ac0 0x55e716611be0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 --2- 172.21.15.22:0/4013036023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e7166021a0 0x55e7166116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f7fedffb640 0 client.17178 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 >> 172.21.15.22:0/4013036023 conn(0x55e7165fc8d0 msgr2=0x55e7165fb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 shutdown_connections 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 1 -- 172.21.15.22:0/4013036023 wait complete. 2024-03-08T23:59:16.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.944+0000 7f800f187740 20 client.17178 trim_cache size 0 max 16384 2024-03-08T23:59:16.947 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:59:16.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7ff00a29f0 msgr2=0x7f7ff00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7ff00a29f0 0x7f7ff00a4eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc005fd0 tx=0x7f7ffc013b00 comp rx=0 tx=0).stop 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041065b0 msgr2=0x7f800419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041065b0 0x7f800419dd90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400e9f0 tx=0x7f7ff400edf0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 shutdown_connections 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7ff00a29f0 0x7f7ff00a4eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8004197a00 0x7f8004197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80041065b0 0x7f800419dd90 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 --2- 172.21.15.22:0/2777371419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041014e0 0x7f800419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 >> 172.21.15.22:0/2777371419 conn(0x7f80040fcda0 msgr2=0x7f80040fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 shutdown_connections 2024-03-08T23:59:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:16.945+0000 7f800c5a9640 1 -- 172.21.15.22:0/2777371419 wait complete. 2024-03-08T23:59:16.951 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:59:16.951 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:59:17.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 -- 172.21.15.22:0/2096937045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8006a4e0 msgr2=0x7f8c80105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/2096937045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8006a4e0 0x7f8c80105f00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009900 tx=0x7f8c7002bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 -- 172.21.15.22:0/2096937045 shutdown_connections 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/2096937045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c801065e0 0x7f8c8010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/2096937045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c8006a4e0 0x7f8c80105f00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/2096937045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c80069b30 0x7f8c80069f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 -- 172.21.15.22:0/2096937045 >> 172.21.15.22:0/2096937045 conn(0x7f8c800771d0 msgr2=0x7f8c800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 -- 172.21.15.22:0/2096937045 shutdown_connections 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 -- 172.21.15.22:0/2096937045 wait complete. 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.075+0000 7f8c87fc4640 1 Processor -- start 2024-03-08T23:59:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c87fc4640 1 -- start start 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c87fc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c80069b30 0x7f8c80199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c87fc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c801065e0 0x7f8c8019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c87fc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c8019e660 0x7f8c8019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c87fc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80111670 con 0x7f8c80069b30 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c87fc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801114f0 con 0x7f8c801065e0 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c87fc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801117f0 con 0x7f8c8019e660 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c80069b30 0x7f8c80199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c80069b30 0x7f8c80199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43626/0 (socket says 172.21.15.22:43626) 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 -- 172.21.15.22:0/352649466 learned_addr learned my addr 172.21.15.22:0/352649466 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c8555f640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c801065e0 0x7f8c8019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c86561640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c8019e660 0x7f8c8019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 -- 172.21.15.22:0/352649466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c8019e660 msgr2=0x7f8c8019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c8019e660 0x7f8c8019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 -- 172.21.15.22:0/352649466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c801065e0 msgr2=0x7f8c8019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c801065e0 0x7f8c8019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 -- 172.21.15.22:0/352649466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8019f150 con 0x7f8c80069b30 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c85d60640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c80069b30 0x7f8c80199df0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00e9f0 tx=0x7f8c7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.076+0000 7f8c76ffd640 1 -- 172.21.15.22:0/352649466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00cd80 con 0x7f8c80069b30 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.077+0000 7f8c76ffd640 1 -- 172.21.15.22:0/352649466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c005500 con 0x7f8c80069b30 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.077+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8019f440 con 0x7f8c80069b30 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.077+0000 7f8c76ffd640 1 -- 172.21.15.22:0/352649466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c010690 con 0x7f8c80069b30 2024-03-08T23:59:17.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.077+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c801a6bb0 con 0x7f8c80069b30 2024-03-08T23:59:17.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.079+0000 7f8c76ffd640 1 -- 172.21.15.22:0/352649466 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8c7c0046c0 con 0x7f8c80069b30 2024-03-08T23:59:17.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.079+0000 7f8c8aba3740 1 Processor -- start 2024-03-08T23:59:17.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8aba3740 1 -- start start 2024-03-08T23:59:17.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c76ffd640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8c6c0a2970 0x7f8c6c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffab73360 con 0x563ffabb56a0 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffab0fec0 con 0x563ffabac1a0 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffaab8d30 con 0x563ffabacac0 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c76ffd640 1 -- 172.21.15.22:0/352649466 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8c7c0cf600 con 0x7f8c80069b30 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c86561640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8c6c0a2970 0x7f8c6c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44012/0 (socket says 172.21.15.22:44012) 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 -- 172.21.15.22:0/1611215914 learned_addr learned my addr 172.21.15.22:0/1611215914 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51682/0 (socket says 172.21.15.22:51682) 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c86561640 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 -- 172.21.15.22:0/1611215914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 msgr2=0x563ffabac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 -- 172.21.15.22:0/1611215914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 msgr2=0x563ffabb7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c8555f640 1 -- 172.21.15.22:0/1611215914 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ffaab7bc0 con 0x563ffabacac0 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.080+0000 7f8c86561640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8c6c0a2970 0x7f8c6c0a4e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c78005fd0 tx=0x7f8c78009e90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.081+0000 7f8c8555f640 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c7002cfd0 tx=0x7f8c70005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e547cbc00407e2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.081+0000 7f8c6bfff640 1 -- 172.21.15.22:0/1611215914 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70042070 con 0x563ffabacac0 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.081+0000 7f8c6bfff640 1 -- 172.21.15.22:0/1611215914 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c70004030 con 0x563ffabacac0 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.081+0000 7f8c6bfff640 1 -- 172.21.15.22:0/1611215914 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70004ec0 con 0x563ffabacac0 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.081+0000 7f8c8aba3740 1 -- 172.21.15.22:0/1611215914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 msgr2=0x563ffabb5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.081+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c7002cfd0 tx=0x7f8c70005be0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- 172.21.15.22:0/1611215914 shutdown_connections 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb5160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/1611215914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- 172.21.15.22:0/1611215914 >> 172.21.15.22:0/1611215914 conn(0x563ffaba68d0 msgr2=0x563ffaba6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- 172.21.15.22:0/1611215914 shutdown_connections 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- 172.21.15.22:0/1611215914 wait complete. 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- start start 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 Processor -- start 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabac1a0 0x563ffabb26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabacac0 0x563ffabb2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffaaae4a0 con 0x563ffabb56a0 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffab0bf60 con 0x563ffabacac0 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ffabb8430 con 0x563ffabac1a0 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.082+0000 7f8c8aba3740 10 client.0 _ll_register_callbacks cb 0x563ffab09d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabac1a0 0x563ffabb26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43644/0 (socket says 172.21.15.22:43644) 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 -- 172.21.15.22:0/2674176430 learned_addr learned my addr 172.21.15.22:0/2674176430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c8555f640 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabacac0 0x563ffabb2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 -- 172.21.15.22:0/2674176430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabac1a0 msgr2=0x563ffabb26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabac1a0 0x563ffabb26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 -- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabacac0 msgr2=0x563ffabb2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabacac0 0x563ffabb2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 -- 172.21.15.22:0/2674176430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ffa8243f0 con 0x563ffabb56a0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c86561640 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb3140 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8c780095c0 tx=0x7f8c7801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e185d38eac0e710 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c697fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c697fa640 1 -- 172.21.15.22:0/2674176430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78024070 con 0x563ffabb56a0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c697fa640 1 -- 172.21.15.22:0/2674176430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8c780047c0 con 0x563ffabb56a0 2024-03-08T23:59:17.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.083+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c70004030 con 0x563ffabb56a0 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c697fa640 1 -- 172.21.15.22:0/2674176430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c78033050 con 0x563ffabb56a0 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c8aba3740 10 client.17190 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563ffa8ee7d0 con 0x563ffabb56a0 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c697fa640 1 -- 172.21.15.22:0/2674176430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c7801f930 con 0x563ffabb56a0 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c697fa640 1 -- 172.21.15.22:0/2674176430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8c78028660 con 0x563ffabb56a0 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c8aba3740 10 client.17190 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c8aba3740 10 client.17190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.084+0000 7f8c8aba3740 20 client.17190 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.085+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffabc5870 0x563ffabc7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.085+0000 7f8c8aba3740 4 client.17190 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.085+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c70004030 con 0x563ffabc5870 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.085+0000 7f8c85d60640 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffabc5870 0x563ffabc7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.085+0000 7f8c85d60640 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffabc5870 0x563ffabc7c80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcdb4b838f0bd331 server_cookie=9408b654bfd496f6 in_seq=0 out_seq=0 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.085+0000 7f8c697fa640 10 client.17190 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:17.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c697fa640 1 -- 172.21.15.22:0/2674176430 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f8c70004030 con 0x563ffabc5870 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c697fa640 10 client.17190 handle_command_reply: tid=0 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 client.17190 shutdown 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 msgr2=0x563ffabb3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb3140 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8c780095c0 tx=0x7f8c7801d000 comp rx=0 tx=0).stop 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 shutdown_connections 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffabc5870 0x563ffabc7c80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabb3140 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c697fa640 0 client.17190 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabacac0 0x563ffabb2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2674176430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabac1a0 0x563ffabb26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 >> 172.21.15.22:0/2674176430 conn(0x563ffaba68d0 msgr2=0x563ffaba8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 shutdown_connections 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2674176430 wait complete. 2024-03-08T23:59:17.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.086+0000 7f8c8aba3740 20 client.17190 trim_cache size 0 max 16384 2024-03-08T23:59:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c8aba3740 1 Processor -- start 2024-03-08T23:59:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c8aba3740 1 -- start start 2024-03-08T23:59:17.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabb4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabaf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801b2bb0 con 0x563ffabb56a0 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabb4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabb4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51692/0 (socket says 172.21.15.22:51692) 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c85d60640 1 -- 172.21.15.22:0/3343432177 learned_addr learned my addr 172.21.15.22:0/3343432177 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c8555f640 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c8aba3740 1 -- 172.21.15.22:0/3343432177 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801b0750 con 0x563ffabac1a0 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c8aba3740 1 -- 172.21.15.22:0/3343432177 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801ae7f0 con 0x563ffabacac0 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.102+0000 7f8c86561640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabaf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c85d60640 1 -- 172.21.15.22:0/3343432177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 msgr2=0x563ffabb16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c85d60640 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c85d60640 1 -- 172.21.15.22:0/3343432177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 msgr2=0x563ffabaf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c85d60640 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabaf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c85d60640 1 -- 172.21.15.22:0/3343432177 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ffabab290 con 0x563ffabac1a0 2024-03-08T23:59:17.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c85d60640 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabb4df0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0cca60 tx=0x7f8c7c097c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc56bcced3b34f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.103+0000 7f8c6b7fe640 1 -- 172.21.15.22:0/3343432177 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0cf040 con 0x563ffabac1a0 2024-03-08T23:59:17.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c6b7fe640 1 -- 172.21.15.22:0/3343432177 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c096040 con 0x563ffabac1a0 2024-03-08T23:59:17.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c6b7fe640 1 -- 172.21.15.22:0/3343432177 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c097f30 con 0x563ffabac1a0 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 -- 172.21.15.22:0/3343432177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 msgr2=0x563ffabb4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabb4df0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0cca60 tx=0x7f8c7c097c50 comp rx=0 tx=0).stop 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 -- 172.21.15.22:0/3343432177 shutdown_connections 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabb56a0 0x563ffabaf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabb16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/3343432177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabac1a0 0x563ffabb4df0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 -- 172.21.15.22:0/3343432177 >> 172.21.15.22:0/3343432177 conn(0x563ffaba68d0 msgr2=0x563ffabb0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.104+0000 7f8c8aba3740 1 -- 172.21.15.22:0/3343432177 shutdown_connections 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.105+0000 7f8c8aba3740 1 -- 172.21.15.22:0/3343432177 wait complete. 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.105+0000 7f8c8aba3740 1 -- start start 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.105+0000 7f8c8aba3740 1 Processor -- start 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabac1a0 0x563ffabbcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabbaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8aba3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabb56a0 0x563ffabbb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801aedf0 con 0x563ffabac1a0 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801b00f0 con 0x563ffabb56a0 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8aba3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801af430 con 0x563ffabacac0 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8aba3740 10 client.0 _ll_register_callbacks cb 0x7f8c7c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabac1a0 0x563ffabbcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8555f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabbaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabac1a0 0x563ffabbcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43662/0 (socket says 172.21.15.22:43662) 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c86561640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabb56a0 0x563ffabbb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 -- 172.21.15.22:0/2158750491 learned_addr learned my addr 172.21.15.22:0/2158750491 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 -- 172.21.15.22:0/2158750491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 msgr2=0x563ffabbaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabbaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 -- 172.21.15.22:0/2158750491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabb56a0 msgr2=0x563ffabbb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabb56a0 0x563ffabbb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 -- 172.21.15.22:0/2158750491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ffac1bd70 con 0x563ffabac1a0 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c8555f640 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabbaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c85d60640 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabac1a0 0x563ffabbcba0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c09b5c0 tx=0x7f8c7c09b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54d0d12d5fa4e3f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.106+0000 7f8c697fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.107+0000 7f8c697fa640 1 -- 172.21.15.22:0/2158750491 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0cf040 con 0x563ffabac1a0 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.107+0000 7f8c697fa640 1 -- 172.21.15.22:0/2158750491 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8c7c0ccc30 con 0x563ffabac1a0 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.107+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c7c096040 con 0x563ffabac1a0 2024-03-08T23:59:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.107+0000 7f8c697fa640 1 -- 172.21.15.22:0/2158750491 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c7c09fa10 con 0x563ffabac1a0 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.107+0000 7f8c8aba3740 10 client.17196 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.107+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563ffac1cf00 con 0x563ffabac1a0 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.107+0000 7f8c697fa640 1 -- 172.21.15.22:0/2158750491 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c097f30 con 0x563ffabac1a0 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c697fa640 1 -- 172.21.15.22:0/2158750491 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8c7c0a3bd0 con 0x563ffabac1a0 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8aba3740 10 client.17196 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8aba3740 10 client.17196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8aba3740 20 client.17196 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffac1d510 0x563ffac1f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8aba3740 4 client.17196 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8c7c096040 con 0x563ffac1d510 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8555f640 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffac1d510 0x563ffac1f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c697fa640 10 client.17196 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c8555f640 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffac1d510 0x563ffac1f900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5acb9eee7131602c server_cookie=8f047c1ea2d1da8f in_seq=0 out_seq=0 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.108+0000 7f8c697fa640 1 -- 172.21.15.22:0/2158750491 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8c7c096040 con 0x563ffac1d510 2024-03-08T23:59:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c697fa640 10 client.17196 handle_command_reply: tid=0 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 client.17196 shutdown 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabac1a0 msgr2=0x563ffabbcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabac1a0 0x563ffabbcba0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c09b5c0 tx=0x7f8c7c09b8c0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 shutdown_connections 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563ffac1d510 0x563ffac1f900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563ffabb56a0 0x563ffabbb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ffabacac0 0x563ffabbaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 --2- 172.21.15.22:0/2158750491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563ffabac1a0 0x563ffabbcba0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 >> 172.21.15.22:0/2158750491 conn(0x563ffaba68d0 msgr2=0x563ffabb2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c697fa640 0 client.17196 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 shutdown_connections 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 1 -- 172.21.15.22:0/2158750491 wait complete. 2024-03-08T23:59:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c8aba3740 20 client.17196 trim_cache size 0 max 16384 2024-03-08T23:59:17.110 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:59:17.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8c6c0a2970 msgr2=0x7f8c6c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8c6c0a2970 0x7f8c6c0a4e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c78005fd0 tx=0x7f8c78009e90 comp rx=0 tx=0).stop 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c80069b30 msgr2=0x7f8c80199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.109+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c80069b30 0x7f8c80199df0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00e9f0 tx=0x7f8c7c00edf0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 shutdown_connections 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8c6c0a2970 0x7f8c6c0a4e30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c8019e660 0x7f8c8019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c801065e0 0x7f8c8019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 --2- 172.21.15.22:0/352649466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c80069b30 0x7f8c80199df0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 >> 172.21.15.22:0/352649466 conn(0x7f8c800771d0 msgr2=0x7f8c8010c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 shutdown_connections 2024-03-08T23:59:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.110+0000 7f8c87fc4640 1 -- 172.21.15.22:0/352649466 wait complete. 2024-03-08T23:59:17.117 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:59:17.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:59:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 -- 172.21.15.22:0/2711615571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 msgr2=0x7fd2a810c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 --2- 172.21.15.22:0/2711615571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a810c7b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ade0 tx=0x7fd2a002cfd0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 -- 172.21.15.22:0/2711615571 shutdown_connections 2024-03-08T23:59:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 --2- 172.21.15.22:0/2711615571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a810c7b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 --2- 172.21.15.22:0/2711615571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2a80ff8e0 0x7fd2a80ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 --2- 172.21.15.22:0/2711615571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2a81058f0 0x7fd2a8105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 -- 172.21.15.22:0/2711615571 >> 172.21.15.22:0/2711615571 conn(0x7fd2a8076720 msgr2=0x7fd2a80fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 -- 172.21.15.22:0/2711615571 shutdown_connections 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.236+0000 7fd2ad569640 1 -- 172.21.15.22:0/2711615571 wait complete. 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 Processor -- start 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 -- start start 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2a80ff8e0 0x7fd2a819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2a81058f0 0x7fd2a8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a810f480 con 0x7fd2a81002a0 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a810f300 con 0x7fd2a81058f0 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2ad569640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a810f600 con 0x7fd2a80ff8e0 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43682/0 (socket says 172.21.15.22:43682) 2024-03-08T23:59:17.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 -- 172.21.15.22:0/1895939138 learned_addr learned my addr 172.21.15.22:0/1895939138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2a80ff8e0 0x7fd2a819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2a81058f0 0x7fd2a8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 -- 172.21.15.22:0/1895939138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2a80ff8e0 msgr2=0x7fd2a819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2a80ff8e0 0x7fd2a819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 -- 172.21.15.22:0/1895939138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2a81058f0 msgr2=0x7fd2a8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2a81058f0 0x7fd2a8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a67fc640 1 -- 172.21.15.22:0/1895939138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8198110 con 0x7fd2a81002a0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.237+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2a81058f0 0x7fd2a8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.238+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a819dcd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00c890 tx=0x7fd29c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.238+0000 7fd28ffff640 1 -- 172.21.15.22:0/1895939138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c004220 con 0x7fd2a81002a0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.238+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a8198400 con 0x7fd2a81002a0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.238+0000 7fd28ffff640 1 -- 172.21.15.22:0/1895939138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd29c0043c0 con 0x7fd2a81002a0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.238+0000 7fd28ffff640 1 -- 172.21.15.22:0/1895939138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0049e0 con 0x7fd2a81002a0 2024-03-08T23:59:17.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.238+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a81a4650 con 0x7fd2a81002a0 2024-03-08T23:59:17.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.240+0000 7fd28ffff640 1 -- 172.21.15.22:0/1895939138 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd29c0079b0 con 0x7fd2a81002a0 2024-03-08T23:59:17.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 Processor -- start 2024-03-08T23:59:17.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 -- start start 2024-03-08T23:59:17.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x55573723c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd28ffff640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2900a2970 0x7fd2900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x555737245160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557372456a0 0x555737247a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737203360 con 0x55573723cac0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573719fec0 con 0x5557372456a0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2b0221740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555737148d30 con 0x55573723c1a0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x555737245160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2900a2970 0x7fd2900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x55573723c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x55573723c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44060/0 (socket says 172.21.15.22:44060) 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2a77fe640 1 -- 172.21.15.22:0/3602856358 learned_addr learned my addr 172.21.15.22:0/3602856358 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd28ffff640 1 -- 172.21.15.22:0/1895939138 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd29c0cf2d0 con 0x7fd2a81002a0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.241+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557372456a0 0x555737247a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd2a77fe640 1 -- 172.21.15.22:0/3602856358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557372456a0 msgr2=0x555737247a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557372456a0 0x555737247a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd2a77fe640 1 -- 172.21.15.22:0/3602856358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 msgr2=0x555737245160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x555737245160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd2a77fe640 1 -- 172.21.15.22:0/3602856358 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555737147bc0 con 0x55573723c1a0 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2900a2970 0x7fd2900a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd2a81040c0 tx=0x7fd298005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x55573723c580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006010 tx=0x7fd2a0005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba146be23d570c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd28d7fa640 1 -- 172.21.15.22:0/3602856358 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003a070 con 0x55573723c1a0 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd28d7fa640 1 -- 172.21.15.22:0/3602856358 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2a003b040 con 0x55573723c1a0 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.242+0000 7fd28d7fa640 1 -- 172.21.15.22:0/3602856358 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0004270 con 0x55573723c1a0 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 -- 172.21.15.22:0/3602856358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 msgr2=0x55573723c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x55573723c580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006010 tx=0x7fd2a0005ef0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 -- 172.21.15.22:0/3602856358 shutdown_connections 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557372456a0 0x555737247a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x555737245160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3602856358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x55573723c580 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 -- 172.21.15.22:0/3602856358 >> 172.21.15.22:0/3602856358 conn(0x5557372368d0 msgr2=0x555737236cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 -- 172.21.15.22:0/3602856358 shutdown_connections 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 -- 172.21.15.22:0/3602856358 wait complete. 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 -- start start 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 Processor -- start 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x555737242610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723cac0 0x555737242b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.243+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x555737243090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2b0221740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573713e4a0 con 0x5557372456a0 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2b0221740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573719bf60 con 0x55573723cac0 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2b0221740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557372483d0 con 0x55573723c1a0 2024-03-08T23:59:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2b0221740 10 client.0 _ll_register_callbacks cb 0x555737199d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x555737242610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x555737242610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44070/0 (socket says 172.21.15.22:44070) 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723cac0 0x555737242b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a6ffd640 1 -- 172.21.15.22:0/3831303527 learned_addr learned my addr 172.21.15.22:0/3831303527 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x555737243090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a77fe640 1 -- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 msgr2=0x555737242610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x555737242610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a77fe640 1 -- 172.21.15.22:0/3831303527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723cac0 msgr2=0x555737242b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723cac0 0x555737242b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a77fe640 1 -- 172.21.15.22:0/3831303527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555736eb43f0 con 0x5557372456a0 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723cac0 0x555737242b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x555737242610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x555737243090 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd2a002d510 tx=0x7fd2a0033dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6ea97de4bc3b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd28aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd28aff5640 1 -- 172.21.15.22:0/3831303527 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003a040 con 0x5557372456a0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd28aff5640 1 -- 172.21.15.22:0/3831303527 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd2a0003e00 con 0x5557372456a0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a003b040 con 0x5557372456a0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd28aff5640 1 -- 172.21.15.22:0/3831303527 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a005b050 con 0x5557372456a0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.244+0000 7fd2b0221740 10 client.17208 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555736f7e7d0 con 0x5557372456a0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd28aff5640 1 -- 172.21.15.22:0/3831303527 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2a003b040 con 0x5557372456a0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd28aff5640 1 -- 172.21.15.22:0/3831303527 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2a0056030 con 0x5557372456a0 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2b0221740 10 client.17208 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2b0221740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2b0221740 20 client.17208 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372555d0 0x5557372579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2b0221740 4 client.17208 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557371ad400 con 0x5557372555d0 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372555d0 0x5557372579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd28aff5640 10 client.17208 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.245+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372555d0 0x5557372579c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270a2c167d80ed59 server_cookie=b05af79a8702643f in_seq=0 out_seq=0 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.246+0000 7fd28aff5640 1 -- 172.21.15.22:0/3831303527 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557371ad400 con 0x5557372555d0 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.246+0000 7fd28aff5640 10 client.17208 handle_command_reply: tid=0 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.246+0000 7fd2b0221740 1 client.17208 shutdown 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 msgr2=0x555737243090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x555737243090 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd2a002d510 tx=0x7fd2a0033dd0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 shutdown_connections 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372555d0 0x5557372579c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x555737243090 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723cac0 0x555737242b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 --2- 172.21.15.22:0/3831303527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723c1a0 0x555737242610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd28aff5640 0 client.17208 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:17.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 >> 172.21.15.22:0/3831303527 conn(0x5557372368d0 msgr2=0x555737238620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 shutdown_connections 2024-03-08T23:59:17.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 1 -- 172.21.15.22:0/3831303527 wait complete. 2024-03-08T23:59:17.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.247+0000 7fd2b0221740 20 client.17208 trim_cache size 0 max 16384 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 Processor -- start 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 -- start start 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x555737244d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723cac0 0x55573724c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x55573724cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81b0a80 con 0x5557372456a0 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81ae620 con 0x55573723c1a0 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2b0221740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81ac6c0 con 0x55573723cac0 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723cac0 0x55573724c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x55573724cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x55573724cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43692/0 (socket says 172.21.15.22:43692) 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723cac0 0x55573724c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44076/0 (socket says 172.21.15.22:44076) 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a77fe640 1 -- 172.21.15.22:0/4130429034 learned_addr learned my addr 172.21.15.22:0/4130429034 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x555737244d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a67fc640 1 -- 172.21.15.22:0/4130429034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 msgr2=0x555737244d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x555737244d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a67fc640 1 -- 172.21.15.22:0/4130429034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 msgr2=0x55573724cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x55573724cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a67fc640 1 -- 172.21.15.22:0/4130429034 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555737257d30 con 0x55573723cac0 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.260+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x55573724cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.261+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723cac0 0x55573724c630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0cc620 tx=0x7fd29c086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f56460634c5f32d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.261+0000 7fd28cff9640 1 -- 172.21.15.22:0/4130429034 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c01d070 con 0x55573723cac0 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.261+0000 7fd28cff9640 1 -- 172.21.15.22:0/4130429034 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd29c09bc30 con 0x55573723cac0 2024-03-08T23:59:17.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.261+0000 7fd28cff9640 1 -- 172.21.15.22:0/4130429034 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c09ca90 con 0x55573723cac0 2024-03-08T23:59:17.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- 172.21.15.22:0/4130429034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723cac0 msgr2=0x55573724c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723cac0 0x55573724c630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0cc620 tx=0x7fd29c086040 comp rx=0 tx=0).stop 2024-03-08T23:59:17.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- 172.21.15.22:0/4130429034 shutdown_connections 2024-03-08T23:59:17.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5557372456a0 0x55573724cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55573723cac0 0x55573724c630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 --2- 172.21.15.22:0/4130429034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x555737244d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- 172.21.15.22:0/4130429034 >> 172.21.15.22:0/4130429034 conn(0x5557372368d0 msgr2=0x55573725bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- 172.21.15.22:0/4130429034 shutdown_connections 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- 172.21.15.22:0/4130429034 wait complete. 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- start start 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 Processor -- start 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x55573724b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x55573724b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5557372456a0 0x55573724bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81accc0 con 0x55573723cac0 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81adfc0 con 0x55573723c1a0 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81ad300 con 0x5557372456a0 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.262+0000 7fd2b0221740 10 client.0 _ll_register_callbacks cb 0x7fd2a0038070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x55573724b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x55573724b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x55573724b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43696/0 (socket says 172.21.15.22:43696) 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x55573724b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51778/0 (socket says 172.21.15.22:51778) 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 -- 172.21.15.22:0/2087409912 learned_addr learned my addr 172.21.15.22:0/2087409912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a77fe640 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5557372456a0 0x55573724bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 -- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5557372456a0 msgr2=0x55573724bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5557372456a0 0x55573724bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 -- 172.21.15.22:0/2087409912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 msgr2=0x55573724b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x55573724b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 -- 172.21.15.22:0/2087409912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557372ab790 con 0x55573723cac0 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x55573724b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2a67fc640 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x55573724b690 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd29c09b230 tx=0x7fd29c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20f385da241dd8b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd28aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd28aff5640 1 -- 172.21.15.22:0/2087409912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0b8020 con 0x55573723cac0 2024-03-08T23:59:17.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd28aff5640 1 -- 172.21.15.22:0/2087409912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd29c0cc8a0 con 0x55573723cac0 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd29c09bc30 con 0x55573723cac0 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd28aff5640 1 -- 172.21.15.22:0/2087409912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd29c081050 con 0x55573723cac0 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2b0221740 10 client.17214 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.263+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5557372aca90 con 0x55573723cac0 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd28aff5640 1 -- 172.21.15.22:0/2087409912 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd29c09bc30 con 0x55573723cac0 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd28aff5640 1 -- 172.21.15.22:0/2087409912 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd29c0a3840 con 0x55573723cac0 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2b0221740 10 client.17214 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2b0221740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2b0221740 20 client.17214 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2b0221740 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372ad160 0x5557372af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2b0221740 4 client.17214 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5557371a7750 con 0x5557372ad160 2024-03-08T23:59:17.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372ad160 0x5557372af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd28aff5640 10 client.17214 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.264+0000 7fd2a6ffd640 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372ad160 0x5557372af580 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f955bff2f40c8d1 server_cookie=93092d4132d1ba27 in_seq=0 out_seq=0 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd28aff5640 1 -- 172.21.15.22:0/2087409912 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5557371a7750 con 0x5557372ad160 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd28aff5640 10 client.17214 handle_command_reply: tid=0 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 client.17214 shutdown 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 msgr2=0x55573724b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x55573724b690 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd29c09b230 tx=0x7fd29c01b000 comp rx=0 tx=0).stop 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 shutdown_connections 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5557372ad160 0x5557372af580 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5557372456a0 0x55573724bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd28aff5640 0 client.17214 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55573723cac0 0x55573724b690 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 --2- 172.21.15.22:0/2087409912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55573723c1a0 0x55573724b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 >> 172.21.15.22:0/2087409912 conn(0x5557372368d0 msgr2=0x555737235100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 shutdown_connections 2024-03-08T23:59:17.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 1 -- 172.21.15.22:0/2087409912 wait complete. 2024-03-08T23:59:17.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2b0221740 20 client.17214 trim_cache size 0 max 16384 2024-03-08T23:59:17.268 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:59:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2900a2970 msgr2=0x7fd2900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2ad569640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2900a2970 0x7fd2900a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd2a81040c0 tx=0x7fd298005f90 comp rx=0 tx=0).stop 2024-03-08T23:59:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 msgr2=0x7fd2a819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.265+0000 7fd2ad569640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a819dcd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00c890 tx=0x7fd29c00cd50 comp rx=0 tx=0).stop 2024-03-08T23:59:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 shutdown_connections 2024-03-08T23:59:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2900a2970 0x7fd2900a4e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2a81058f0 0x7fd2a8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2a81002a0 0x7fd2a819dcd0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 --2- 172.21.15.22:0/1895939138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2a80ff8e0 0x7fd2a819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 >> 172.21.15.22:0/1895939138 conn(0x7fd2a8076720 msgr2=0x7fd2a8102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 shutdown_connections 2024-03-08T23:59:17.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:17.266+0000 7fd2ad569640 1 -- 172.21.15.22:0/1895939138 wait complete. 2024-03-08T23:59:17.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 470890 11.58 MB/sec execute 85 sec latency 140.193 ms 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.746+0000 7f458c53b640 1 -- 172.21.15.22:0/2052571560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 msgr2=0x7f45841008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.746+0000 7f458c53b640 1 --2- 172.21.15.22:0/2052571560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f45841008c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4574009900 tx=0x7f457402bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- 172.21.15.22:0/2052571560 shutdown_connections 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 --2- 172.21.15.22:0/2052571560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4584100e00 0x7f458410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 --2- 172.21.15.22:0/2052571560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f45841008c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 --2- 172.21.15.22:0/2052571560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4584105190 0x7f4584105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- 172.21.15.22:0/2052571560 >> 172.21.15.22:0/2052571560 conn(0x7f4584077040 msgr2=0x7f4584077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- 172.21.15.22:0/2052571560 shutdown_connections 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- 172.21.15.22:0/2052571560 wait complete. 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 Processor -- start 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- start start 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f4584199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4584100e00 0x7f458419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4584105190 0x7f458419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584110650 con 0x7f4584100440 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841104d0 con 0x7f4584100e00 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458c53b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841107d0 con 0x7f4584105190 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f4584199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f4584199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43708/0 (socket says 172.21.15.22:43708) 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.747+0000 7f458a2d7640 1 -- 172.21.15.22:0/2885546864 learned_addr learned my addr 172.21.15.22:0/2885546864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458aad8640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4584105190 0x7f458419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f4589ad6640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4584100e00 0x7f458419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458a2d7640 1 -- 172.21.15.22:0/2885546864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4584105190 msgr2=0x7f458419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458a2d7640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4584105190 0x7f458419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458a2d7640 1 -- 172.21.15.22:0/2885546864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4584100e00 msgr2=0x7f458419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458a2d7640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4584100e00 0x7f458419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458a2d7640 1 -- 172.21.15.22:0/2885546864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458419ed40 con 0x7f4584100440 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458aad8640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4584105190 0x7f458419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458a2d7640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f4584199da0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f458000e9f0 tx=0x7f458000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f457b7fe640 1 -- 172.21.15.22:0/2885546864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458000cd80 con 0x7f4584100440 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f457b7fe640 1 -- 172.21.15.22:0/2885546864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4580005500 con 0x7f4584100440 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f457b7fe640 1 -- 172.21.15.22:0/2885546864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580010690 con 0x7f4584100440 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458419f030 con 0x7f4584100440 2024-03-08T23:59:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.748+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45841a68c0 con 0x7f4584100440 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.751+0000 7f457b7fe640 1 -- 172.21.15.22:0/2885546864 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f45800046c0 con 0x7f4584100440 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.751+0000 7f458f1f7740 1 Processor -- start 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.751+0000 7f458f1f7740 1 -- start start 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f457b7fe640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f45700a2a10 0x7f45700a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446f6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458f1f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44464f5f0 con 0x55f4446f4550 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458f1f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4445f8930 con 0x55f4446eafd0 2024-03-08T23:59:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458f1f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44464b690 con 0x55f4446ebd40 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458aad8640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f45700a2a10 0x7f45700a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446eb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51808/0 (socket says 172.21.15.22:51808) 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 -- 172.21.15.22:0/1188445539 learned_addr learned my addr 172.21.15.22:0/1188445539 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f457b7fe640 1 -- 172.21.15.22:0/2885546864 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f45800cf710 con 0x7f4584100440 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458aad8640 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 -- 172.21.15.22:0/1188445539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 msgr2=0x55f4446eaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446eaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 -- 172.21.15.22:0/1188445539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 msgr2=0x55f4446f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 -- 172.21.15.22:0/1188445539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f444364210 con 0x55f4446eafd0 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458a2d7640 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446eaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f458aad8640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f45700a2a10 0x7f45700a4ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f457c005fd0 tx=0x7f457c009f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.752+0000 7f4589ad6640 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446eb450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4574000c00 tx=0x7f4574005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2f2cb1b3827174 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f4578bfe640 1 -- 172.21.15.22:0/1188445539 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574042070 con 0x55f4446eafd0 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f4578bfe640 1 -- 172.21.15.22:0/1188445539 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f457402c6d0 con 0x55f4446eafd0 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f4578bfe640 1 -- 172.21.15.22:0/1188445539 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45740046c0 con 0x55f4446eafd0 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 -- 172.21.15.22:0/1188445539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 msgr2=0x55f4446eb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446eb450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4574000c00 tx=0x7f4574005dc0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 -- 172.21.15.22:0/1188445539 shutdown_connections 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446eb450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1188445539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446eaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 -- 172.21.15.22:0/1188445539 >> 172.21.15.22:0/1188445539 conn(0x55f4446b25b0 msgr2=0x55f4446e5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 -- 172.21.15.22:0/1188445539 shutdown_connections 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.753+0000 7f458f1f7740 1 -- 172.21.15.22:0/1188445539 wait complete. 2024-03-08T23:59:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 -- start start 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 Processor -- start 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446f1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446f4550 0x55f4446f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4446f7220 con 0x55f4446eafd0 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44460bd90 con 0x55f4446f4550 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4446f73a0 con 0x55f4446ebd40 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458f1f7740 10 client.0 _ll_register_callbacks cb 0x55f444363a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458aad8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446f4550 0x55f4446f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446f1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43712/0 (socket says 172.21.15.22:43712) 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458a2d7640 1 -- 172.21.15.22:0/205484948 learned_addr learned my addr 172.21.15.22:0/205484948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f4589ad6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458a2d7640 1 -- 172.21.15.22:0/205484948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 msgr2=0x55f4446f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458a2d7640 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458a2d7640 1 -- 172.21.15.22:0/205484948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446f4550 msgr2=0x55f4446f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.754+0000 7f458a2d7640 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446f4550 0x55f4446f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458a2d7640 1 -- 172.21.15.22:0/205484948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f444430620 con 0x55f4446eafd0 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458aad8640 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446f4550 0x55f4446f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458a2d7640 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446f1550 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f458000cfd0 tx=0x7f458009b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c293a9f9d27e71 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f456dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f456dffb640 1 -- 172.21.15.22:0/205484948 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458009c0d0 con 0x55f4446eafd0 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f456dffb640 1 -- 172.21.15.22:0/205484948 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4580098040 con 0x55f4446eafd0 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f457402c6d0 con 0x55f4446eafd0 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f456dffb640 1 -- 172.21.15.22:0/205484948 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f458009b9d0 con 0x55f4446eafd0 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 10 client.17226 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f444703520 con 0x55f4446eafd0 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f456dffb640 1 -- 172.21.15.22:0/205484948 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f457402c6d0 con 0x55f4446eafd0 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f456dffb640 1 -- 172.21.15.22:0/205484948 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f45800ba0e0 con 0x55f4446eafd0 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 10 client.17226 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 10 client.17226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 20 client.17226 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f444704990 0x55f444706d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 4 client.17226 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.755+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f44465cb30 con 0x55f444704990 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.756+0000 7f4589ad6640 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f444704990 0x55f444706d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.756+0000 7f4589ad6640 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f444704990 0x55f444706d80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb9ab8bc47656c4 server_cookie=c842c4aa8df376b1 in_seq=0 out_seq=0 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.756+0000 7f456dffb640 10 client.17226 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f456dffb640 1 -- 172.21.15.22:0/205484948 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f44465cb30 con 0x55f444704990 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f456dffb640 10 client.17226 handle_command_reply: tid=0 2024-03-08T23:59:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 client.17226 shutdown 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 msgr2=0x55f4446f1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446f1550 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f458000cfd0 tx=0x7f458009b5d0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 shutdown_connections 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f444704990 0x55f444706d80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446f4550 0x55f4446f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f456dffb640 0 client.17226 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 --2- 172.21.15.22:0/205484948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446f1550 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 >> 172.21.15.22:0/205484948 conn(0x55f4446b25b0 msgr2=0x55f4446e7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 shutdown_connections 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 1 -- 172.21.15.22:0/205484948 wait complete. 2024-03-08T23:59:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.757+0000 7f458f1f7740 20 client.17226 trim_cache size 0 max 16384 2024-03-08T23:59:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.771+0000 7f458f1f7740 1 Processor -- start 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458f1f7740 1 -- start start 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446fbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458f1f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841b2ac0 con 0x55f4446f4550 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458f1f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841b0660 con 0x55f4446eafd0 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458f1f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841ae700 con 0x55f4446ebd40 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458aad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446fbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51816/0 (socket says 172.21.15.22:51816) 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458aad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446fbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43726/0 (socket says 172.21.15.22:43726) 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 -- 172.21.15.22:0/1731167572 learned_addr learned my addr 172.21.15.22:0/1731167572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f4589ad6640 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 -- 172.21.15.22:0/1731167572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 msgr2=0x55f4446fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 -- 172.21.15.22:0/1731167572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 msgr2=0x55f4446fbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446fbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 -- 172.21.15.22:0/1731167572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4446ea2a0 con 0x55f4446eafd0 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458aad8640 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f458a2d7640 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446f3c80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4580097e30 tx=0x7f458009b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685bb6b69903e78b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.772+0000 7f456ffff640 1 -- 172.21.15.22:0/1731167572 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580063020 con 0x55f4446eafd0 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f456ffff640 1 -- 172.21.15.22:0/1731167572 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f458009c0d0 con 0x55f4446eafd0 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f456ffff640 1 -- 172.21.15.22:0/1731167572 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45800cf300 con 0x55f4446eafd0 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 -- 172.21.15.22:0/1731167572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 msgr2=0x55f4446f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446f3c80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4580097e30 tx=0x7f458009b640 comp rx=0 tx=0).stop 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 -- 172.21.15.22:0/1731167572 shutdown_connections 2024-03-08T23:59:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446f4550 0x55f4446fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446ebd40 0x55f4446fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 --2- 172.21.15.22:0/1731167572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446eafd0 0x55f4446f3c80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 -- 172.21.15.22:0/1731167572 >> 172.21.15.22:0/1731167572 conn(0x55f4446b25b0 msgr2=0x55f4446f6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 -- 172.21.15.22:0/1731167572 shutdown_connections 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.773+0000 7f458f1f7740 1 -- 172.21.15.22:0/1731167572 wait complete. 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 -- start start 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 Processor -- start 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446fb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446ebd40 0x55f4446f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446f4550 0x55f4446f8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841aed00 con 0x55f4446eafd0 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841b0000 con 0x55f4446ebd40 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841af340 con 0x55f4446f4550 2024-03-08T23:59:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458f1f7740 10 client.0 _ll_register_callbacks cb 0x7f4580005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446fb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43728/0 (socket says 172.21.15.22:43728) 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 -- 172.21.15.22:0/3243157731 learned_addr learned my addr 172.21.15.22:0/3243157731 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458aad8640 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446f4550 0x55f4446f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f4589ad6640 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446ebd40 0x55f4446f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 -- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446f4550 msgr2=0x55f4446f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446f4550 0x55f4446f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 -- 172.21.15.22:0/3243157731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446ebd40 msgr2=0x55f4446f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446ebd40 0x55f4446f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.774+0000 7f458a2d7640 1 -- 172.21.15.22:0/3243157731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4446f86f0 con 0x55f4446eafd0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458a2d7640 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446fb740 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f458009b6d0 tx=0x7f458009b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6ff126c0c578c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f456dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f456dffb640 1 -- 172.21.15.22:0/3243157731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580063070 con 0x55f4446eafd0 2024-03-08T23:59:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f456dffb640 1 -- 172.21.15.22:0/3243157731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f45800a3ef0 con 0x55f4446eafd0 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f458009c0d0 con 0x55f4446eafd0 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f456dffb640 1 -- 172.21.15.22:0/3243157731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45800a0a40 con 0x55f4446eafd0 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 10 client.17232 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f4446f86f0 con 0x55f4446eafd0 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f456dffb640 1 -- 172.21.15.22:0/3243157731 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f458009c0d0 con 0x55f4446eafd0 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f456dffb640 1 -- 172.21.15.22:0/3243157731 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f45800861e0 con 0x55f4446eafd0 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 10 client.17232 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 20 client.17232 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f44475be50 0x55f44475e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 4 client.17232 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.775+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f444656e80 con 0x55f44475be50 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.776+0000 7f4589ad6640 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f44475be50 0x55f44475e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.776+0000 7f4589ad6640 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f44475be50 0x55f44475e240 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c80fdf21f46ce6 server_cookie=9d849425e024b21a in_seq=0 out_seq=0 2024-03-08T23:59:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:17.776+0000 7f456dffb640 10 client.17232 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:17.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2616752774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:59:17.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2616752774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:59:17.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2616752774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:59:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.070+0000 7f456dffb640 1 -- 172.21.15.22:0/3243157731 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f444656e80 con 0x55f44475be50 2024-03-08T23:59:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.070+0000 7f456dffb640 10 client.17232 handle_command_reply: tid=0 2024-03-08T23:59:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.070+0000 7f458f1f7740 1 client.17232 shutdown 2024-03-08T23:59:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 msgr2=0x55f4446fb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446fb740 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f458009b6d0 tx=0x7f458009b5d0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 shutdown_connections 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f44475be50 0x55f44475e240 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4446f4550 0x55f4446f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f456dffb640 0 client.17232 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4446ebd40 0x55f4446f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 --2- 172.21.15.22:0/3243157731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4446eafd0 0x55f4446fb740 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 >> 172.21.15.22:0/3243157731 conn(0x55f4446b25b0 msgr2=0x55f4446f1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 shutdown_connections 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 1 -- 172.21.15.22:0/3243157731 wait complete. 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458f1f7740 20 client.17232 trim_cache size 0 max 16384 2024-03-08T23:59:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:59:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:59:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:59:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f45700a2a10 msgr2=0x7f45700a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.071+0000 7f458c53b640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f45700a2a10 0x7f45700a4ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f457c005fd0 tx=0x7f457c009f90 comp rx=0 tx=0).stop 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 msgr2=0x7f4584199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f4584199da0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f458000e9f0 tx=0x7f458000edf0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 shutdown_connections 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f45700a2a10 0x7f45700a4ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4584105190 0x7f458419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4584100e00 0x7f458419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 --2- 172.21.15.22:0/2885546864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4584100440 0x7f4584199da0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 >> 172.21.15.22:0/2885546864 conn(0x7f4584077040 msgr2=0x7f45840fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 shutdown_connections 2024-03-08T23:59:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:18.072+0000 7f458c53b640 1 -- 172.21.15.22:0/2885546864 wait complete. 2024-03-08T23:59:18.273 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:59:18.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:59:18.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:59:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 -- 172.21.15.22:0/2447921365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 msgr2=0x7f1058102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 --2- 172.21.15.22:0/2447921365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058102910 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1048009900 tx=0x7f104802bcd0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 -- 172.21.15.22:0/2447921365 shutdown_connections 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 --2- 172.21.15.22:0/2447921365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f105810b900 0x7f105810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 --2- 172.21.15.22:0/2447921365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1058102e50 0x7f105810b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 --2- 172.21.15.22:0/2447921365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058102910 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 -- 172.21.15.22:0/2447921365 >> 172.21.15.22:0/2447921365 conn(0x7f1058076570 msgr2=0x7f1058076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 -- 172.21.15.22:0/2447921365 shutdown_connections 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 -- 172.21.15.22:0/2447921365 wait complete. 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 Processor -- start 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 -- start start 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.392+0000 7f105eae8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1058102e50 0x7f105819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105eae8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105810b900 0x7f105819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105eae8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1058101930 con 0x7f1058102530 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105eae8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10581017b0 con 0x7f1058102e50 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105eae8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1058101ab0 con 0x7f105810b900 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43740/0 (socket says 172.21.15.22:43740) 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 -- 172.21.15.22:0/1841248857 learned_addr learned my addr 172.21.15.22:0/1841248857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105d085640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105810b900 0x7f105819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f1057fff640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1058102e50 0x7f105819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 -- 172.21.15.22:0/1841248857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105810b900 msgr2=0x7f105819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105810b900 0x7f105819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 -- 172.21.15.22:0/1841248857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1058102e50 msgr2=0x7f105819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1058102e50 0x7f105819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 -- 172.21.15.22:0/1841248857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105819ed90 con 0x7f1058102530 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105c884640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058199df0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1048005940 tx=0x7f104802cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f1055ffb640 1 -- 172.21.15.22:0/1841248857 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048041070 con 0x7f1058102530 2024-03-08T23:59:18.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f1055ffb640 1 -- 172.21.15.22:0/1841248857 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1048004450 con 0x7f1058102530 2024-03-08T23:59:18.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105819f020 con 0x7f1058102530 2024-03-08T23:59:18.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.393+0000 7f1055ffb640 1 -- 172.21.15.22:0/1841248857 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048003d80 con 0x7f1058102530 2024-03-08T23:59:18.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.394+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10581a6910 con 0x7f1058102530 2024-03-08T23:59:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f1055ffb640 1 -- 172.21.15.22:0/1841248857 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1048044070 con 0x7f1058102530 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 Processor -- start 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 -- start start 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f1055ffb640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10400a2970 0x7f10400a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560940860d80 0x560940861160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940869d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x56094086c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609407c4b40 con 0x56094086a280 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094076dd80 con 0x560940860d80 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f106179e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609407c0be0 con 0x5609408616a0 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f1055ffb640 1 -- 172.21.15.22:0/1841248857 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f10480f0ba0 con 0x7f1058102530 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f105d085640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10400a2970 0x7f10400a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f1057fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940869d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f105c884640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560940860d80 0x560940861160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f105d085640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x56094086c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f1057fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940869d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44168/0 (socket says 172.21.15.22:44168) 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f105c884640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560940860d80 0x560940861160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51848/0 (socket says 172.21.15.22:51848) 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.397+0000 7f1057fff640 1 -- 172.21.15.22:0/3836119751 learned_addr learned my addr 172.21.15.22:0/3836119751 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105c884640 1 -- 172.21.15.22:0/3836119751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 msgr2=0x560940869d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105c884640 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940869d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105c884640 1 -- 172.21.15.22:0/3836119751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 msgr2=0x56094086c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105c884640 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x56094086c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105c884640 1 -- 172.21.15.22:0/3836119751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609405a17e0 con 0x560940860d80 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105d085640 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x56094086c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105d085640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10400a2970 0x7f10400a4e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f105806a9a0 tx=0x7f1050009e90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f105c884640 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560940860d80 0x560940861160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10480ee060 tx=0x7f1048005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1481b61c51547e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f103effd640 1 -- 172.21.15.22:0/3836119751 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104804b040 con 0x560940860d80 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f103effd640 1 -- 172.21.15.22:0/3836119751 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1048004c80 con 0x560940860d80 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.398+0000 7f103effd640 1 -- 172.21.15.22:0/3836119751 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10480bd020 con 0x560940860d80 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 -- 172.21.15.22:0/3836119751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560940860d80 msgr2=0x560940861160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560940860d80 0x560940861160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10480ee060 tx=0x7f1048005c00 comp rx=0 tx=0).stop 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 -- 172.21.15.22:0/3836119751 shutdown_connections 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x56094086c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940869d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 --2- 172.21.15.22:0/3836119751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560940860d80 0x560940861160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 -- 172.21.15.22:0/3836119751 >> 172.21.15.22:0/3836119751 conn(0x56094085b4b0 msgr2=0x56094085b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 -- 172.21.15.22:0/3836119751 shutdown_connections 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 -- 172.21.15.22:0/3836119751 wait complete. 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 -- start start 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.399+0000 7f106179e740 1 Processor -- start 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x560940869bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940866f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56094086a280 0x5609408674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094076dd80 con 0x560940860d80 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609407811c0 con 0x56094086a280 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094086ce30 con 0x5609408616a0 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 10 client.0 _ll_register_callbacks cb 0x5609407707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x560940869bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x560940869bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43750/0 (socket says 172.21.15.22:43750) 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 -- 172.21.15.22:0/785596275 learned_addr learned my addr 172.21.15.22:0/785596275 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105d085640 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56094086a280 0x5609408674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f1057fff640 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940866f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 -- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 msgr2=0x560940866f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940866f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 -- 172.21.15.22:0/785596275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56094086a280 msgr2=0x5609408674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56094086a280 0x5609408674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 -- 172.21.15.22:0/785596275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560940601f70 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f105c884640 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x560940869bf0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f10480ee550 tx=0x7f104804cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23144afd063dcb6a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f1037fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f1037fff640 1 -- 172.21.15.22:0/785596275 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104804b050 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f1037fff640 1 -- 172.21.15.22:0/785596275 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f10480bd280 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.400+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1048004c80 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f1037fff640 1 -- 172.21.15.22:0/785596275 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10480a3050 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 10 client.17244 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560940499170 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f1037fff640 1 -- 172.21.15.22:0/785596275 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1048004c80 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f1037fff640 1 -- 172.21.15.22:0/785596275 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f10480c0070 con 0x560940860d80 2024-03-08T23:59:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 10 client.17244 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 20 client.17244 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x56094087c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 4 client.17244 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56094079a960 con 0x560940879dd0 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.401+0000 7f1057fff640 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x56094087c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.402+0000 7f1037fff640 10 client.17244 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.402+0000 7f1057fff640 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x56094087c1c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b73366e442ebc7 server_cookie=899d708a581b7e97 in_seq=0 out_seq=0 2024-03-08T23:59:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f1037fff640 1 -- 172.21.15.22:0/785596275 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56094079a960 con 0x560940879dd0 2024-03-08T23:59:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f1037fff640 10 client.17244 handle_command_reply: tid=0 2024-03-08T23:59:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 client.17244 shutdown 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 msgr2=0x560940869bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x560940869bf0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f10480ee550 tx=0x7f104804cd70 comp rx=0 tx=0).stop 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 shutdown_connections 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x56094087c1c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56094086a280 0x5609408674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f1037fff640 0 client.17244 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5609408616a0 0x560940866f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 --2- 172.21.15.22:0/785596275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x560940869bf0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.403+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 >> 172.21.15.22:0/785596275 conn(0x56094085b4b0 msgr2=0x56094085d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.404+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 shutdown_connections 2024-03-08T23:59:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.404+0000 7f106179e740 1 -- 172.21.15.22:0/785596275 wait complete. 2024-03-08T23:59:18.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.404+0000 7f106179e740 20 client.17244 trim_cache size 0 max 16384 2024-03-08T23:59:18.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 Processor -- start 2024-03-08T23:59:18.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 -- start start 2024-03-08T23:59:18.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560940860d80 0x5609408696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x560940869bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x5609408621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10581b2c30 con 0x56094086a280 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10581b07d0 con 0x5609408616a0 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f106179e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10581ae870 con 0x560940860d80 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x5609408621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x5609408621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43764/0 (socket says 172.21.15.22:43764) 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 -- 172.21.15.22:0/1521634582 learned_addr learned my addr 172.21.15.22:0/1521634582 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105c884640 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560940860d80 0x5609408696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f1057fff640 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x560940869bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 -- 172.21.15.22:0/1521634582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560940860d80 msgr2=0x5609408696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560940860d80 0x5609408696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 -- 172.21.15.22:0/1521634582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 msgr2=0x560940869bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x560940869bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.420+0000 7f105d085640 1 -- 172.21.15.22:0/1521634582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609408614d0 con 0x56094086a280 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.421+0000 7f105d085640 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x5609408621e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f10500116f0 tx=0x7f1050021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c22015208a6a3a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.421+0000 7f103e7fc640 1 -- 172.21.15.22:0/1521634582 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105002d070 con 0x56094086a280 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.421+0000 7f103e7fc640 1 -- 172.21.15.22:0/1521634582 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1050004520 con 0x56094086a280 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.421+0000 7f103e7fc640 1 -- 172.21.15.22:0/1521634582 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105001ea80 con 0x56094086a280 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.421+0000 7f106179e740 1 -- 172.21.15.22:0/1521634582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 msgr2=0x5609408621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.421+0000 7f106179e740 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x5609408621e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f10500116f0 tx=0x7f1050021000 comp rx=0 tx=0).stop 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- 172.21.15.22:0/1521634582 shutdown_connections 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56094086a280 0x5609408621e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x560940869bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 --2- 172.21.15.22:0/1521634582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560940860d80 0x5609408696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- 172.21.15.22:0/1521634582 >> 172.21.15.22:0/1521634582 conn(0x56094085b4b0 msgr2=0x56094087c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- 172.21.15.22:0/1521634582 shutdown_connections 2024-03-08T23:59:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- 172.21.15.22:0/1521634582 wait complete. 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- start start 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 Processor -- start 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x5609408711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x56094086e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56094086a280 0x56094086eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10581aee70 con 0x560940860d80 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10581b0170 con 0x5609408616a0 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10581af4b0 con 0x56094086a280 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.422+0000 7f106179e740 10 client.0 _ll_register_callbacks cb 0x7f1048005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x5609408711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x5609408711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43768/0 (socket says 172.21.15.22:43768) 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 -- 172.21.15.22:0/132248006 learned_addr learned my addr 172.21.15.22:0/132248006 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f1057fff640 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x56094086e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105d085640 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56094086a280 0x56094086eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 -- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56094086a280 msgr2=0x56094086eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56094086a280 0x56094086eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 -- 172.21.15.22:0/132248006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 msgr2=0x56094086e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x56094086e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 -- 172.21.15.22:0/132248006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56094086f060 con 0x560940860d80 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f105c884640 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x5609408711d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1048033e90 tx=0x7f1048009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddb4cce19b4af21 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f1037fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f1037fff640 1 -- 172.21.15.22:0/132248006 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104804b070 con 0x560940860d80 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f1037fff640 1 -- 172.21.15.22:0/132248006 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f10480c5480 con 0x560940860d80 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1050004520 con 0x560940860d80 2024-03-08T23:59:18.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f1037fff640 1 -- 172.21.15.22:0/132248006 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10480c7e60 con 0x560940860d80 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f106179e740 10 client.17250 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56094086f060 con 0x560940860d80 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.423+0000 7f1037fff640 1 -- 172.21.15.22:0/132248006 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1050004520 con 0x560940860d80 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f1037fff640 1 -- 172.21.15.22:0/132248006 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f10480c0040 con 0x560940860d80 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f106179e740 10 client.17250 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f106179e740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f106179e740 20 client.17250 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f106179e740 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x5609408d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f106179e740 4 client.17250 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560940786080 con 0x560940879dd0 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f1057fff640 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x5609408d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f1037fff640 10 client.17250 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.424+0000 7f1057fff640 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x5609408d42d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1939aa81ccec2b1b server_cookie=57cee5b3c15ae2b3 in_seq=0 out_seq=0 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f1037fff640 1 -- 172.21.15.22:0/132248006 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560940786080 con 0x560940879dd0 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f1037fff640 10 client.17250 handle_command_reply: tid=0 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 client.17250 shutdown 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 msgr2=0x5609408711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x5609408711d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1048033e90 tx=0x7f1048009560 comp rx=0 tx=0).stop 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 shutdown_connections 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560940879dd0 0x5609408d42d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f1037fff640 0 client.17250 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56094086a280 0x56094086eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609408616a0 0x56094086e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 --2- 172.21.15.22:0/132248006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560940860d80 0x5609408711d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 >> 172.21.15.22:0/132248006 conn(0x56094085b4b0 msgr2=0x560940865420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 shutdown_connections 2024-03-08T23:59:18.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 1 -- 172.21.15.22:0/132248006 wait complete. 2024-03-08T23:59:18.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.426+0000 7f106179e740 20 client.17250 trim_cache size 0 max 16384 2024-03-08T23:59:18.428 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:59:18.428 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:59:18.428 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "171282c6-1518-4860-a4ac-3de92894d1c7", 2024-03-08T23:59:18.428 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:59:18.428 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10400a2970 msgr2=0x7f10400a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10400a2970 0x7f10400a4e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f105806a9a0 tx=0x7f1050009e90 comp rx=0 tx=0).stop 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 msgr2=0x7f1058199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058199df0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1048005940 tx=0x7f104802cbd0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 shutdown_connections 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10400a2970 0x7f10400a4e30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105810b900 0x7f105819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1058102e50 0x7f105819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 --2- 172.21.15.22:0/1841248857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1058102530 0x7f1058199df0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 >> 172.21.15.22:0/1841248857 conn(0x7f1058076570 msgr2=0x7f10580fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 shutdown_connections 2024-03-08T23:59:18.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.427+0000 7f105eae8640 1 -- 172.21.15.22:0/1841248857 wait complete. 2024-03-08T23:59:18.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:59:18.433 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.545+0000 7f6666243640 1 -- 172.21.15.22:0/3685057865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 msgr2=0x7f666010d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.545+0000 7f6666243640 1 --2- 172.21.15.22:0/3685057865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666010d070 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6658009ca0 tx=0x7f665802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 -- 172.21.15.22:0/3685057865 shutdown_connections 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 --2- 172.21.15.22:0/3685057865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666010d070 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 --2- 172.21.15.22:0/3685057865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6660100d80 0x7f6660104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 --2- 172.21.15.22:0/3685057865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66601003d0 0x7f66601007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 -- 172.21.15.22:0/3685057865 >> 172.21.15.22:0/3685057865 conn(0x7f66600772f0 msgr2=0x7f6660077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 -- 172.21.15.22:0/3685057865 shutdown_connections 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 -- 172.21.15.22:0/3685057865 wait complete. 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 Processor -- start 2024-03-08T23:59:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.546+0000 7f6666243640 1 -- start start 2024-03-08T23:59:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f6666243640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66601003d0 0x7f6660199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f6666243640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f6666243640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666019e660 0x7f666019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f6666243640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660110560 con 0x7f66601054a0 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f6666243640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601103e0 con 0x7f66601003d0 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f6666243640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601106e0 con 0x7f666019e660 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43778/0 (socket says 172.21.15.22:43778) 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 -- 172.21.15.22:0/2793835649 learned_addr learned my addr 172.21.15.22:0/2793835649 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665ffff640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666019e660 0x7f666019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665f7fe640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66601003d0 0x7f6660199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 -- 172.21.15.22:0/2793835649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666019e660 msgr2=0x7f666019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666019e660 0x7f666019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 -- 172.21.15.22:0/2793835649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66601003d0 msgr2=0x7f6660199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66601003d0 0x7f6660199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 -- 172.21.15.22:0/2793835649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666019f150 con 0x7f66601054a0 2024-03-08T23:59:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665ffff640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666019e660 0x7f666019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.547+0000 7f665effd640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666019a330 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f665400e9f0 tx=0x7f665400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.548+0000 7f665cff9640 1 -- 172.21.15.22:0/2793835649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400ccc0 con 0x7f66601054a0 2024-03-08T23:59:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.548+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666019f440 con 0x7f66601054a0 2024-03-08T23:59:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.548+0000 7f665cff9640 1 -- 172.21.15.22:0/2793835649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6654005500 con 0x7f66601054a0 2024-03-08T23:59:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.548+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66601a6c30 con 0x7f66601054a0 2024-03-08T23:59:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.548+0000 7f665cff9640 1 -- 172.21.15.22:0/2793835649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654010690 con 0x7f66601054a0 2024-03-08T23:59:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.550+0000 7f665cff9640 1 -- 172.21.15.22:0/2793835649 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f66540046c0 con 0x7f66601054a0 2024-03-08T23:59:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f665cff9640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f66480a2970 0x7f66480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 Processor -- start 2024-03-08T23:59:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f665f7fe640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f66480a2970 0x7f66480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f665cff9640 1 -- 172.21.15.22:0/2793835649 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f66540cf4e0 con 0x7f66601054a0 2024-03-08T23:59:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 -- start start 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a0db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a19050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c58970e00 con 0x556c58a16c60 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c58919b10 con 0x556c58a0e080 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f6668e1b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c5896cea0 con 0x556c58a0d760 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.551+0000 7f665effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51892/0 (socket says 172.21.15.22:51892) 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 -- 172.21.15.22:0/3179232516 learned_addr learned my addr 172.21.15.22:0/3179232516 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665f7fe640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f66480a2970 0x7f66480a4e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6650009bd0 tx=0x7f6650005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665f7fe640 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a0db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665ffff640 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 -- 172.21.15.22:0/3179232516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 msgr2=0x556c58a0db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a0db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 -- 172.21.15.22:0/3179232516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 msgr2=0x556c58a19050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a19050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 -- 172.21.15.22:0/3179232516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c5874f7d0 con 0x556c58a0e080 2024-03-08T23:59:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665f7fe640 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a0db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f665effd640 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6654097c00 tx=0x7f665409b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48433094c5495925 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f6645ffb640 1 -- 172.21.15.22:0/3179232516 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540b4070 con 0x556c58a0e080 2024-03-08T23:59:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f6645ffb640 1 -- 172.21.15.22:0/3179232516 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f665409b7a0 con 0x556c58a0e080 2024-03-08T23:59:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.552+0000 7f6645ffb640 1 -- 172.21.15.22:0/3179232516 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654097e10 con 0x556c58a0e080 2024-03-08T23:59:18.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 -- 172.21.15.22:0/3179232516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 msgr2=0x556c58a16720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6654097c00 tx=0x7f665409b630 comp rx=0 tx=0).stop 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 -- 172.21.15.22:0/3179232516 shutdown_connections 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a19050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3179232516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a0db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 -- 172.21.15.22:0/3179232516 >> 172.21.15.22:0/3179232516 conn(0x556c58a07e90 msgr2=0x556c58a08270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 -- 172.21.15.22:0/3179232516 shutdown_connections 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.553+0000 7f6668e1b740 1 -- 172.21.15.22:0/3179232516 wait complete. 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 -- start start 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 Processor -- start 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a16370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0e080 0x556c58a13700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a16c60 0x556c58a13c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c58a19a50 con 0x556c58a0d760 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c5892d0a0 con 0x556c58a16c60 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c58a19bd0 con 0x556c58a0e080 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f6668e1b740 10 client.0 _ll_register_callbacks cb 0x556c5892ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f665f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a16370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f665effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0e080 0x556c58a13700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f665f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a16370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43792/0 (socket says 172.21.15.22:43792) 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.554+0000 7f665f7fe640 1 -- 172.21.15.22:0/1160207145 learned_addr learned my addr 172.21.15.22:0/1160207145 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665ffff640 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a16c60 0x556c58a13c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665f7fe640 1 -- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0e080 msgr2=0x556c58a13700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665f7fe640 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0e080 0x556c58a13700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665f7fe640 1 -- 172.21.15.22:0/1160207145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a16c60 msgr2=0x556c58a13c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665f7fe640 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a16c60 0x556c58a13c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665f7fe640 1 -- 172.21.15.22:0/1160207145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c586853f0 con 0x556c58a0d760 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665effd640 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0e080 0x556c58a13700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:18.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f665f7fe640 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a16370 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f665001ed60 tx=0x7f6650004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7358448f029abce5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f663effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f663effd640 1 -- 172.21.15.22:0/1160207145 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665001e340 con 0x556c58a0d760 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f663effd640 1 -- 172.21.15.22:0/1160207145 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6650004020 con 0x556c58a0d760 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f665409b7a0 con 0x556c58a0d760 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f663effd640 1 -- 172.21.15.22:0/1160207145 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f665002f050 con 0x556c58a0d760 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f6668e1b740 10 client.17262 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.555+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556c586450f0 con 0x556c58a0d760 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f663effd640 1 -- 172.21.15.22:0/1160207145 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f665409b7a0 con 0x556c58a0d760 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f663effd640 1 -- 172.21.15.22:0/1160207145 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6650025030 con 0x556c58a0d760 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f6668e1b740 10 client.17262 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f6668e1b740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f6668e1b740 20 client.17262 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a26760 0x556c58a28b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f6668e1b740 4 client.17262 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c5897e340 con 0x556c58a26760 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.556+0000 7f665effd640 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a26760 0x556c58a28b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.557+0000 7f663effd640 10 client.17262 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.557+0000 7f665effd640 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a26760 0x556c58a28b70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d537afd82117c7a2 server_cookie=fbe3c5845dedba50 in_seq=0 out_seq=0 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.557+0000 7f663effd640 1 -- 172.21.15.22:0/1160207145 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c5897e340 con 0x556c58a26760 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.557+0000 7f663effd640 10 client.17262 handle_command_reply: tid=0 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 client.17262 shutdown 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 msgr2=0x556c58a16370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a16370 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f665001ed60 tx=0x7f6650004730 comp rx=0 tx=0).stop 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 shutdown_connections 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a26760 0x556c58a28b70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a16c60 0x556c58a13c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0e080 0x556c58a13700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1160207145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a16370 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 >> 172.21.15.22:0/1160207145 conn(0x556c58a07e90 msgr2=0x556c58a09be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f663effd640 0 client.17262 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 shutdown_connections 2024-03-08T23:59:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 1 -- 172.21.15.22:0/1160207145 wait complete. 2024-03-08T23:59:18.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.558+0000 7f6668e1b740 20 client.17262 trim_cache size 0 max 16384 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 Processor -- start 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 -- start start 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601b2cc0 con 0x556c58a16c60 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601b0860 con 0x556c58a0e080 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f6668e1b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601ae900 con 0x556c58a0d760 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f665ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f665ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43804/0 (socket says 172.21.15.22:43804) 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f665ffff640 1 -- 172.21.15.22:0/3010719602 learned_addr learned my addr 172.21.15.22:0/3010719602 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f665effd640 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.573+0000 7f665f7fe640 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f665ffff640 1 -- 172.21.15.22:0/3010719602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 msgr2=0x556c58a15e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f665ffff640 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a15e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f665ffff640 1 -- 172.21.15.22:0/3010719602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 msgr2=0x556c58a16370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f665ffff640 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f665ffff640 1 -- 172.21.15.22:0/3010719602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c58a0cae0 con 0x556c58a16c60 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f665f7fe640 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f665ffff640 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a136c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6658009ca0 tx=0x7f6658005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6193195cfa819837 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f66457fa640 1 -- 172.21.15.22:0/3010719602 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658039040 con 0x556c58a16c60 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f66457fa640 1 -- 172.21.15.22:0/3010719602 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6658032c90 con 0x556c58a16c60 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.574+0000 7f66457fa640 1 -- 172.21.15.22:0/3010719602 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66580347f0 con 0x556c58a16c60 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 -- 172.21.15.22:0/3010719602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 msgr2=0x556c58a136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a136c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6658009ca0 tx=0x7f6658005c50 comp rx=0 tx=0).stop 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 -- 172.21.15.22:0/3010719602 shutdown_connections 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a16c60 0x556c58a136c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a16370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 --2- 172.21.15.22:0/3010719602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a0d760 0x556c58a15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 -- 172.21.15.22:0/3010719602 >> 172.21.15.22:0/3010719602 conn(0x556c58a07e90 msgr2=0x556c58a2d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 -- 172.21.15.22:0/3010719602 shutdown_connections 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 -- 172.21.15.22:0/3010719602 wait complete. 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 -- start start 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.575+0000 7f6668e1b740 1 Processor -- start 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a19de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a1a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a16c60 0x556c58a1a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601aef00 con 0x556c58a0d760 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601b0200 con 0x556c58a0e080 2024-03-08T23:59:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66601af540 con 0x556c58a16c60 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 10 client.0 _ll_register_callbacks cb 0x7f665001e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a19de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a19de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43808/0 (socket says 172.21.15.22:43808) 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 -- 172.21.15.22:0/1377173186 learned_addr learned my addr 172.21.15.22:0/1377173186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665effd640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a1a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665ffff640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a16c60 0x556c58a1a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 -- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a16c60 msgr2=0x556c58a1a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a16c60 0x556c58a1a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 -- 172.21.15.22:0/1377173186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 msgr2=0x556c58a1a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a1a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 -- 172.21.15.22:0/1377173186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c58a7be90 con 0x556c58a0d760 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665effd640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a1a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f665f7fe640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a19de0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f665001e920 tx=0x7f665000ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e603f4ee837e9684 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f663effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f663effd640 1 -- 172.21.15.22:0/1377173186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665001e340 con 0x556c58a0d760 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f663effd640 1 -- 172.21.15.22:0/1377173186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6650003c50 con 0x556c58a0d760 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.576+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6658032c90 con 0x556c58a0d760 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f663effd640 1 -- 172.21.15.22:0/1377173186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6650041050 con 0x556c58a0d760 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 10 client.17268 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556c58a7d160 con 0x556c58a0d760 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f663effd640 1 -- 172.21.15.22:0/1377173186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6658032c90 con 0x556c58a0d760 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f663effd640 1 -- 172.21.15.22:0/1377173186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6650032070 con 0x556c58a0d760 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 10 client.17268 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 20 client.17268 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a7d850 0x556c58a7fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 4 client.17268 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:59:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c58978690 con 0x556c58a7d850 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.577+0000 7f665effd640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a7d850 0x556c58a7fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f663effd640 10 client.17268 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f665effd640 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a7d850 0x556c58a7fc70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847a1a6e46d4413a server_cookie=80c571e8425ef4b1 in_seq=0 out_seq=0 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f663effd640 1 -- 172.21.15.22:0/1377173186 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556c58978690 con 0x556c58a7d850 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f663effd640 10 client.17268 handle_command_reply: tid=0 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 client.17268 shutdown 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 msgr2=0x556c58a19de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a19de0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f665001e920 tx=0x7f665000ea30 comp rx=0 tx=0).stop 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 shutdown_connections 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556c58a7d850 0x556c58a7fc70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c58a16c60 0x556c58a1a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c58a0e080 0x556c58a1a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f663effd640 0 client.17268 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 --2- 172.21.15.22:0/1377173186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556c58a0d760 0x556c58a19de0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 >> 172.21.15.22:0/1377173186 conn(0x556c58a07e90 msgr2=0x556c58a136c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 shutdown_connections 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.578+0000 7f6668e1b740 1 -- 172.21.15.22:0/1377173186 wait complete. 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6668e1b740 20 client.17268 trim_cache size 0 max 16384 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stdout: "171282c6-1518-4860-a4ac-3de92894d1c7": { 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "171282c6-1518-4860-a4ac-3de92894d1c7", 2024-03-08T23:59:18.581 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:59:18.582 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:59:18.582 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:59:18.582 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f66480a2970 msgr2=0x7f66480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f66480a2970 0x7f66480a4e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6650009bd0 tx=0x7f6650005f90 comp rx=0 tx=0).stop 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 msgr2=0x7f666019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666019a330 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f665400e9f0 tx=0x7f665400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 shutdown_connections 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f66480a2970 0x7f66480a4e30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666019e660 0x7f666019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66601054a0 0x7f666019a330 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 --2- 172.21.15.22:0/2793835649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66601003d0 0x7f6660199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 >> 172.21.15.22:0/2793835649 conn(0x7f66600772f0 msgr2=0x7f6660068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 shutdown_connections 2024-03-08T23:59:18.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:18.579+0000 7f6666243640 1 -- 172.21.15.22:0/2793835649 wait complete. 2024-03-08T23:59:18.584 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:18 smithi022 ceph-mon[31894]: pgmap v149: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 391 op/s 2024-03-08T23:59:18.585 INFO:tasks.cephfs.filesystem:scrub status for tag:171282c6-1518-4860-a4ac-3de92894d1c7 - {'path': '/', 'tag': '171282c6-1518-4860-a4ac-3de92894d1c7', 'options': 'recursive,force'} 2024-03-08T23:59:18.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 472751 11.60 MB/sec execute 86 sec latency 185.184 ms 2024-03-08T23:59:18.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:18 smithi062 ceph-mon[36129]: pgmap v149: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 391 op/s 2024-03-08T23:59:18.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:18 smithi192 ceph-mon[35995]: pgmap v149: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 391 op/s 2024-03-08T23:59:19.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 474454 11.53 MB/sec execute 87 sec latency 16.113 ms 2024-03-08T23:59:19.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:19 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:59:19.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:19 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:19.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:19 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:59:19.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:19 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:19.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:19 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:59:19.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:19 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:19.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:19 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:59:19.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:19 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:19.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:19 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:59:19.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:19 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:19.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:19 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:59:19.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:19 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:20.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 476820 11.56 MB/sec execute 88 sec latency 27.616 ms 2024-03-08T23:59:20.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:20 smithi062 ceph-mon[36129]: pgmap v150: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 509 op/s 2024-03-08T23:59:20.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:20 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:59:20.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:20 smithi192 ceph-mon[35995]: pgmap v150: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 509 op/s 2024-03-08T23:59:20.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:20 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:59:20.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:20 smithi022 ceph-mon[31894]: pgmap v150: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 509 op/s 2024-03-08T23:59:20.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:20 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:59:21.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 479420 11.59 MB/sec execute 89 sec latency 44.221 ms 2024-03-08T23:59:22.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 481121 11.57 MB/sec execute 90 sec latency 76.916 ms 2024-03-08T23:59:22.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:22 smithi062 ceph-mon[36129]: pgmap v151: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 592 op/s 2024-03-08T23:59:22.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:22 smithi192 ceph-mon[35995]: pgmap v151: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 592 op/s 2024-03-08T23:59:22.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:22 smithi022 ceph-mon[31894]: pgmap v151: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.0 MiB/s wr, 592 op/s 2024-03-08T23:59:23.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 483313 11.58 MB/sec execute 91 sec latency 128.403 ms 2024-03-08T23:59:24.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 485198 11.53 MB/sec execute 92 sec latency 17.807 ms 2024-03-08T23:59:24.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:24 smithi062 ceph-mon[36129]: pgmap v152: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 417 op/s 2024-03-08T23:59:24.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:24 smithi192 ceph-mon[35995]: pgmap v152: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 417 op/s 2024-03-08T23:59:24.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:24 smithi022 ceph-mon[31894]: pgmap v152: 129 pgs: 129 active+clean; 49 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 417 op/s 2024-03-08T23:59:25.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 487632 11.56 MB/sec execute 93 sec latency 16.349 ms 2024-03-08T23:59:26.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 490281 11.61 MB/sec execute 94 sec latency 45.078 ms 2024-03-08T23:59:26.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:26 smithi062 ceph-mon[36129]: pgmap v153: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-08T23:59:26.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:26 smithi192 ceph-mon[35995]: pgmap v153: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-08T23:59:26.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:26 smithi022 ceph-mon[31894]: pgmap v153: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-08T23:59:27.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 492134 11.57 MB/sec execute 95 sec latency 17.410 ms 2024-03-08T23:59:28.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 494154 11.59 MB/sec execute 96 sec latency 119.658 ms 2024-03-08T23:59:28.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:28 smithi062 ceph-mon[36129]: pgmap v154: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.1 MiB/s wr, 409 op/s 2024-03-08T23:59:28.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:28 smithi192 ceph-mon[35995]: pgmap v154: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.1 MiB/s wr, 409 op/s 2024-03-08T23:59:28.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:28 smithi022 ceph-mon[31894]: pgmap v154: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.1 MiB/s wr, 409 op/s 2024-03-08T23:59:29.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 496259 11.54 MB/sec execute 97 sec latency 16.727 ms 2024-03-08T23:59:30.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 498380 11.56 MB/sec execute 98 sec latency 15.997 ms 2024-03-08T23:59:30.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:30 smithi062 ceph-mon[36129]: pgmap v155: 129 pgs: 129 active+clean; 55 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 440 op/s 2024-03-08T23:59:30.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:30 smithi192 ceph-mon[35995]: pgmap v155: 129 pgs: 129 active+clean; 55 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 440 op/s 2024-03-08T23:59:30.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:30 smithi022 ceph-mon[31894]: pgmap v155: 129 pgs: 129 active+clean; 55 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 440 op/s 2024-03-08T23:59:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 -- 172.21.15.22:0/1980579869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c069310 msgr2=0x7f4d4c10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 --2- 172.21.15.22:0/1980579869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c069310 0x7f4d4c10e180 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009900 tx=0x7f4d4402bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 -- 172.21.15.22:0/1980579869 shutdown_connections 2024-03-08T23:59:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 --2- 172.21.15.22:0/1980579869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c069310 0x7f4d4c10e180 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 --2- 172.21.15.22:0/1980579869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d4c068970 0x7f4d4c068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 --2- 172.21.15.22:0/1980579869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d4c107400 0x7f4d4c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 -- 172.21.15.22:0/1980579869 >> 172.21.15.22:0/1980579869 conn(0x7f4d4c077040 msgr2=0x7f4d4c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 -- 172.21.15.22:0/1980579869 shutdown_connections 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.162+0000 7f4d50c20640 1 -- 172.21.15.22:0/1980579869 wait complete. 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 Processor -- start 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 -- start start 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c068970 0x7f4d4c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d4c069310 0x7f4d4c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d4c107400 0x7f4d4c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c111640 con 0x7f4d4c068970 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1114c0 con 0x7f4d4c107400 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d50c20640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1117c0 con 0x7f4d4c069310 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c068970 0x7f4d4c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c068970 0x7f4d4c19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60012/0 (socket says 172.21.15.22:60012) 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3614181273 learned_addr learned my addr 172.21.15.22:0/3614181273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d4c069310 0x7f4d4c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4ad9d640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d4c107400 0x7f4d4c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3614181273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d4c069310 msgr2=0x7f4d4c19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d4c069310 0x7f4d4c19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3614181273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d4c107400 msgr2=0x7f4d4c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d4c107400 0x7f4d4c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.163+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3614181273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c197fd0 con 0x7f4d4c068970 2024-03-08T23:59:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.164+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c068970 0x7f4d4c19d690 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00ebc0 tx=0x7f4d3c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.164+0000 7f4d437fe640 1 -- 172.21.15.22:0/3614181273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c010630 con 0x7f4d4c068970 2024-03-08T23:59:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.164+0000 7f4d437fe640 1 -- 172.21.15.22:0/3614181273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c010cb0 con 0x7f4d4c068970 2024-03-08T23:59:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.164+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c1982c0 con 0x7f4d4c068970 2024-03-08T23:59:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.164+0000 7f4d437fe640 1 -- 172.21.15.22:0/3614181273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c018bd0 con 0x7f4d4c068970 2024-03-08T23:59:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.164+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c1a44e0 con 0x7f4d4c068970 2024-03-08T23:59:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.166+0000 7f4d437fe640 1 -- 172.21.15.22:0/3614181273 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4d3c0107d0 con 0x7f4d4c068970 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.166+0000 7f4d53802740 1 Processor -- start 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d53802740 1 -- start start 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d53802740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3bd40 0x558ee1f3aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d53802740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d437fe640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d380a2a10 0x7f4d380a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d53802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f44550 0x558ee1f46940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d53802740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee1e9f5f0 con 0x558ee1f3afd0 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d53802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee1e48930 con 0x558ee1f44550 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d53802740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee1e9b690 con 0x558ee1f3bd40 2024-03-08T23:59:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d380a2a10 0x7f4d380a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3bd40 0x558ee1f3aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f3b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60014/0 (socket says 172.21.15.22:60014) 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3bd40 0x558ee1f3aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:46962/0 (socket says 172.21.15.22:46962) 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3073976867 learned_addr learned my addr 172.21.15.22:0/3073976867 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4ad9d640 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f44550 0x558ee1f46940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d437fe640 1 -- 172.21.15.22:0/3614181273 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4d3c0cf7c0 con 0x7f4d4c068970 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3073976867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3bd40 msgr2=0x558ee1f3aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3bd40 0x558ee1f3aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3073976867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f44550 msgr2=0x558ee1f46940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f44550 0x558ee1f46940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.167+0000 7f4d4a59c640 1 -- 172.21.15.22:0/3073976867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee1bb4210 con 0x558ee1f3afd0 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.168+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f3b450 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000f80 tx=0x7f4d3c097e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8731198b37512b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.168+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d380a2a10 0x7f4d380a4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0775d0 tx=0x7f4d34005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.168+0000 7f4d40ff9640 1 -- 172.21.15.22:0/3073976867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c010e60 con 0x558ee1f3afd0 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.168+0000 7f4d40ff9640 1 -- 172.21.15.22:0/3073976867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c06dc00 con 0x558ee1f3afd0 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.168+0000 7f4d40ff9640 1 -- 172.21.15.22:0/3073976867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0a4220 con 0x558ee1f3afd0 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.168+0000 7f4d53802740 1 -- 172.21.15.22:0/3073976867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 msgr2=0x558ee1f3b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.168+0000 7f4d53802740 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f3b450 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000f80 tx=0x7f4d3c097e90 comp rx=0 tx=0).stop 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- 172.21.15.22:0/3073976867 shutdown_connections 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f44550 0x558ee1f46940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f3b450 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 --2- 172.21.15.22:0/3073976867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3bd40 0x558ee1f3aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- 172.21.15.22:0/3073976867 >> 172.21.15.22:0/3073976867 conn(0x558ee1f025b0 msgr2=0x558ee1f35cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- 172.21.15.22:0/3073976867 shutdown_connections 2024-03-08T23:59:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- 172.21.15.22:0/3073976867 wait complete. 2024-03-08T23:59:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- start start 2024-03-08T23:59:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 Processor -- start 2024-03-08T23:59:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3afd0 0x558ee1f414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3bd40 0x558ee1f41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f41f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee1f47220 con 0x558ee1f3bd40 2024-03-08T23:59:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee1e5bd90 con 0x558ee1f3afd0 2024-03-08T23:59:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee1f473a0 con 0x558ee1f44550 2024-03-08T23:59:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d53802740 10 client.0 _ll_register_callbacks cb 0x558ee1bb3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d49d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3bd40 0x558ee1f41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d49d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3bd40 0x558ee1f41a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60018/0 (socket says 172.21.15.22:60018) 2024-03-08T23:59:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d49d9b640 1 -- 172.21.15.22:0/1546996866 learned_addr learned my addr 172.21.15.22:0/1546996866 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d4ad9d640 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f41f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.169+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3afd0 0x558ee1f414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d49d9b640 1 -- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 msgr2=0x558ee1f41f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f41f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d49d9b640 1 -- 172.21.15.22:0/1546996866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3afd0 msgr2=0x558ee1f414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3afd0 0x558ee1f414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d49d9b640 1 -- 172.21.15.22:0/1546996866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ee1c80620 con 0x558ee1f3bd40 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3bd40 0x558ee1f41a20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400ed00 tx=0x7f4d34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4830104af63c6753 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1546996866 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3401ccb0 con 0x558ee1f3bd40 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1546996866 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4d34004020 con 0x558ee1f3bd40 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d3c06dc00 con 0x558ee1f3bd40 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1546996866 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d34030050 con 0x558ee1f3bd40 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d53802740 10 client.17280 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558ee1f534c0 con 0x558ee1f3bd40 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.170+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1546996866 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c06dc00 con 0x558ee1f3bd40 2024-03-08T23:59:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1546996866 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4d34027430 con 0x558ee1f3bd40 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d53802740 10 client.17280 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d53802740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d53802740 20 client.17280 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d53802740 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1f54980 0x558ee1f56d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d53802740 4 client.17280 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ee1eacb30 con 0x558ee1f54980 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1f54980 0x558ee1f56d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1f54980 0x558ee1f56d70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb855961514d01c server_cookie=29619acde86ed74b in_seq=0 out_seq=0 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.171+0000 7f4d31ffb640 10 client.17280 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1546996866 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ee1eacb30 con 0x558ee1f54980 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d31ffb640 10 client.17280 handle_command_reply: tid=0 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d53802740 1 client.17280 shutdown 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3bd40 msgr2=0x558ee1f41a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d53802740 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3bd40 0x558ee1f41a20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400ed00 tx=0x7f4d34004730 comp rx=0 tx=0).stop 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 shutdown_connections 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d53802740 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1f54980 0x558ee1f56d70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.172+0000 7f4d53802740 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f41f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.173+0000 7f4d53802740 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3bd40 0x558ee1f41a20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.173+0000 7f4d31ffb640 0 client.17280 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.173+0000 7f4d53802740 1 --2- 172.21.15.22:0/1546996866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3afd0 0x558ee1f414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.173+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 >> 172.21.15.22:0/1546996866 conn(0x558ee1f025b0 msgr2=0x558ee1f37a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.173+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 shutdown_connections 2024-03-08T23:59:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.173+0000 7f4d53802740 1 -- 172.21.15.22:0/1546996866 wait complete. 2024-03-08T23:59:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.173+0000 7f4d53802740 20 client.17280 trim_cache size 0 max 16384 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 Processor -- start 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 -- start start 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f43c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f3e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f3ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1b0a70 con 0x558ee1f3afd0 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1ae610 con 0x558ee1f3bd40 2024-03-08T23:59:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d53802740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1ac6b0 con 0x558ee1f44550 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f43c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f43c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60028/0 (socket says 172.21.15.22:60028) 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 -- 172.21.15.22:0/122359546 learned_addr learned my addr 172.21.15.22:0/122359546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f3e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4ad9d640 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f3ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 -- 172.21.15.22:0/122359546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 msgr2=0x558ee1f3ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f3ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 -- 172.21.15.22:0/122359546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 msgr2=0x558ee1f3e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f3e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 -- 172.21.15.22:0/122359546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee1f570e0 con 0x558ee1f3afd0 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f3e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.187+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f43c10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009270 tx=0x7f4d3c097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a523146eb266e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d33fff640 1 -- 172.21.15.22:0/122359546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0b4020 con 0x558ee1f3afd0 2024-03-08T23:59:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d33fff640 1 -- 172.21.15.22:0/122359546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c09bb30 con 0x558ee1f3afd0 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d33fff640 1 -- 172.21.15.22:0/122359546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0afe60 con 0x558ee1f3afd0 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 -- 172.21.15.22:0/122359546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 msgr2=0x558ee1f43c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f43c10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009270 tx=0x7f4d3c097f00 comp rx=0 tx=0).stop 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 -- 172.21.15.22:0/122359546 shutdown_connections 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f44550 0x558ee1f3ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f3e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 --2- 172.21.15.22:0/122359546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f3afd0 0x558ee1f43c10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 -- 172.21.15.22:0/122359546 >> 172.21.15.22:0/122359546 conn(0x558ee1f025b0 msgr2=0x558ee1f3fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 -- 172.21.15.22:0/122359546 shutdown_connections 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.188+0000 7f4d53802740 1 -- 172.21.15.22:0/122359546 wait complete. 2024-03-08T23:59:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 -- start start 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 Processor -- start 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3afd0 0x558ee1f4a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f4a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f44550 0x558ee1f4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1accb0 con 0x558ee1f44550 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1adfb0 con 0x558ee1f3bd40 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1ad2f0 con 0x558ee1f3afd0 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d53802740 10 client.0 _ll_register_callbacks cb 0x7f4d34009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f44550 0x558ee1f4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d49d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f4a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d4a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3afd0 0x558ee1f4a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d4ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f44550 0x558ee1f4aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60030/0 (socket says 172.21.15.22:60030) 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d4ad9d640 1 -- 172.21.15.22:0/1337699277 learned_addr learned my addr 172.21.15.22:0/1337699277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d49d9b640 1 -- 172.21.15.22:0/1337699277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3afd0 msgr2=0x558ee1f4a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3afd0 0x558ee1f4a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d49d9b640 1 -- 172.21.15.22:0/1337699277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f44550 msgr2=0x558ee1f4aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f44550 0x558ee1f4aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d49d9b640 1 -- 172.21.15.22:0/1337699277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ee1faae40 con 0x558ee1f3bd40 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.189+0000 7f4d4ad9d640 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f44550 0x558ee1f4aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d49d9b640 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f4a5b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400ecd0 tx=0x7f4d34011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9a2234440ed5cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1337699277 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34038070 con 0x558ee1f3bd40 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d3c09bb30 con 0x558ee1f3bd40 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1337699277 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4d34009040 con 0x558ee1f3bd40 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1337699277 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d34044050 con 0x558ee1f3bd40 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d53802740 10 client.26720 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558ee1fac130 con 0x558ee1f3bd40 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1337699277 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c09bb30 con 0x558ee1f3bd40 2024-03-08T23:59:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1337699277 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4d340043a0 con 0x558ee1f3bd40 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d53802740 10 client.26720 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d53802740 10 client.26720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d53802740 20 client.26720 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.190+0000 7f4d53802740 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1fac800 0x558ee1faec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.191+0000 7f4d53802740 4 client.26720 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.191+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ee1ea6e80 con 0x558ee1fac800 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.191+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1fac800 0x558ee1faec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.191+0000 7f4d4a59c640 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1fac800 0x558ee1faec20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a453a899cf77fe9e server_cookie=d0fce2f851422c31 in_seq=0 out_seq=0 2024-03-08T23:59:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.191+0000 7f4d31ffb640 10 client.26720 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d31ffb640 1 -- 172.21.15.22:0/1337699277 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ee1ea6e80 con 0x558ee1fac800 2024-03-08T23:59:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d31ffb640 10 client.26720 handle_command_reply: tid=0 2024-03-08T23:59:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 client.26720 shutdown 2024-03-08T23:59:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 msgr2=0x558ee1f4a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f4a5b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400ecd0 tx=0x7f4d34011920 comp rx=0 tx=0).stop 2024-03-08T23:59:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 shutdown_connections 2024-03-08T23:59:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558ee1fac800 0x558ee1faec20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558ee1f44550 0x558ee1f4aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d31ffb640 0 client.26720 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ee1f3bd40 0x558ee1f4a5b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 --2- 172.21.15.22:0/1337699277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ee1f3afd0 0x558ee1f4a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 >> 172.21.15.22:0/1337699277 conn(0x558ee1f025b0 msgr2=0x558ee1f345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 shutdown_connections 2024-03-08T23:59:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 1 -- 172.21.15.22:0/1337699277 wait complete. 2024-03-08T23:59:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.489+0000 7f4d53802740 20 client.26720 trim_cache size 0 max 16384 2024-03-08T23:59:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:59:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:59:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:59:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:59:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.490+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d380a2a10 msgr2=0x7f4d380a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.490+0000 7f4d50c20640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d380a2a10 0x7f4d380a4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0775d0 tx=0x7f4d34005f90 comp rx=0 tx=0).stop 2024-03-08T23:59:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.490+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c068970 msgr2=0x7f4d4c19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.490+0000 7f4d50c20640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c068970 0x7f4d4c19d690 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00ebc0 tx=0x7f4d3c00c6a0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.490+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 shutdown_connections 2024-03-08T23:59:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.490+0000 7f4d50c20640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d380a2a10 0x7f4d380a4ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.491+0000 7f4d50c20640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d4c107400 0x7f4d4c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.491+0000 7f4d50c20640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d4c069310 0x7f4d4c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.491+0000 7f4d50c20640 1 --2- 172.21.15.22:0/3614181273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d4c068970 0x7f4d4c19d690 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.491+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 >> 172.21.15.22:0/3614181273 conn(0x7f4d4c077040 msgr2=0x7f4d4c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.491+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 shutdown_connections 2024-03-08T23:59:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:31.491+0000 7f4d50c20640 1 -- 172.21.15.22:0/3614181273 wait complete. 2024-03-08T23:59:31.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 500410 11.56 MB/sec execute 99 sec latency 147.325 ms 2024-03-08T23:59:32.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 502096 11.53 MB/sec execute 100 sec latency 39.211 ms 2024-03-08T23:59:32.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:32 smithi062 ceph-mon[36129]: pgmap v156: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-03-08T23:59:32.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:32 smithi192 ceph-mon[35995]: pgmap v156: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-03-08T23:59:32.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:32 smithi022 ceph-mon[31894]: pgmap v156: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-03-08T23:59:33.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 504128 11.53 MB/sec execute 101 sec latency 68.809 ms 2024-03-08T23:59:34.714 INFO:tasks.workunit.client.0.smithi022.stdout: 1 505926 11.52 MB/sec execute 102 sec latency 20.385 ms 2024-03-08T23:59:34.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:34 smithi062 ceph-mon[36129]: pgmap v157: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-03-08T23:59:34.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:34 smithi192 ceph-mon[35995]: pgmap v157: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-03-08T23:59:34.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:34 smithi022 ceph-mon[31894]: pgmap v157: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-03-08T23:59:35.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 508372 11.54 MB/sec execute 103 sec latency 17.628 ms 2024-03-08T23:59:36.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 510520 11.51 MB/sec execute 104 sec latency 15.760 ms 2024-03-08T23:59:37.152 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:36 smithi192 ceph-mon[35995]: pgmap v158: 129 pgs: 129 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-08T23:59:37.152 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:36 smithi062 ceph-mon[36129]: pgmap v158: 129 pgs: 129 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-08T23:59:37.152 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:36 smithi022 ceph-mon[31894]: pgmap v158: 129 pgs: 129 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-08T23:59:37.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 512524 11.51 MB/sec execute 105 sec latency 23.349 ms 2024-03-08T23:59:38.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 514957 11.53 MB/sec execute 106 sec latency 126.126 ms 2024-03-08T23:59:38.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:38 smithi062 ceph-mon[36129]: pgmap v159: 129 pgs: 129 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.9 MiB/s wr, 402 op/s 2024-03-08T23:59:38.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:38 smithi192 ceph-mon[35995]: pgmap v159: 129 pgs: 129 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.9 MiB/s wr, 402 op/s 2024-03-08T23:59:38.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:38 smithi022 ceph-mon[31894]: pgmap v159: 129 pgs: 129 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.9 MiB/s wr, 402 op/s 2024-03-08T23:59:39.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 516686 11.52 MB/sec execute 107 sec latency 23.479 ms 2024-03-08T23:59:40.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 519230 11.54 MB/sec execute 108 sec latency 13.848 ms 2024-03-08T23:59:40.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:40 smithi062 ceph-mon[36129]: pgmap v160: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 467 op/s 2024-03-08T23:59:40.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:40 smithi192 ceph-mon[35995]: pgmap v160: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 467 op/s 2024-03-08T23:59:40.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:40 smithi022 ceph-mon[31894]: pgmap v160: 129 pgs: 129 active+clean; 60 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 467 op/s 2024-03-08T23:59:41.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 521940 11.57 MB/sec execute 109 sec latency 16.605 ms 2024-03-08T23:59:42.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 523638 11.55 MB/sec execute 110 sec latency 70.351 ms 2024-03-08T23:59:42.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:42 smithi062 ceph-mon[36129]: pgmap v161: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-03-08T23:59:42.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:42 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:59:42.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:42 smithi192 ceph-mon[35995]: pgmap v161: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-03-08T23:59:42.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:42 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:59:42.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:42 smithi022 ceph-mon[31894]: pgmap v161: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-03-08T23:59:42.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:42 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T23:59:43.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 525833 11.56 MB/sec execute 111 sec latency 125.969 ms 2024-03-08T23:59:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:43 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:59:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:43 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:59:43.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:43 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:59:43.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:43 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:59:43.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:43 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:59:43.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:43 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:59:43.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:43 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T23:59:43.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:43 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T23:59:43.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:43 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-08T23:59:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 -- 172.21.15.22:0/4033694712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ffa00 msgr2=0x7f496c0ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 --2- 172.21.15.22:0/4033694712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ffa00 0x7f496c0ffde0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4968009900 tx=0x7f496802bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 -- 172.21.15.22:0/4033694712 shutdown_connections 2024-03-08T23:59:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 --2- 172.21.15.22:0/4033694712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c106670 0x7f496c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 --2- 172.21.15.22:0/4033694712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c1003b0 0x7f496c105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 --2- 172.21.15.22:0/4033694712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c0ffa00 0x7f496c0ffde0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 -- 172.21.15.22:0/4033694712 >> 172.21.15.22:0/4033694712 conn(0x7f496c076710 msgr2=0x7f496c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 -- 172.21.15.22:0/4033694712 shutdown_connections 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.582+0000 7f4973aab640 1 -- 172.21.15.22:0/4033694712 wait complete. 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 Processor -- start 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 -- start start 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c0ffa00 0x7f496c19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c106670 0x7f496c19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c197930 0x7f496c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1116b0 con 0x7f496c106670 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c111530 con 0x7f496c197930 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4973aab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c111830 con 0x7f496c0ffa00 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c106670 0x7f496c19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971847640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c0ffa00 0x7f496c19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c106670 0x7f496c19dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54726/0 (socket says 172.21.15.22:54726) 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 -- 172.21.15.22:0/350215787 learned_addr learned my addr 172.21.15.22:0/350215787 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4972048640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c197930 0x7f496c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 -- 172.21.15.22:0/350215787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c0ffa00 msgr2=0x7f496c19d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c0ffa00 0x7f496c19d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 -- 172.21.15.22:0/350215787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c197930 msgr2=0x7f496c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c197930 0x7f496c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.583+0000 7f4971046640 1 -- 172.21.15.22:0/350215787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c198450 con 0x7f496c106670 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.584+0000 7f4971046640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c106670 0x7f496c19dcc0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f495c00d950 tx=0x7f495c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.584+0000 7f4962ffd640 1 -- 172.21.15.22:0/350215787 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c017070 con 0x7f496c106670 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.584+0000 7f4962ffd640 1 -- 172.21.15.22:0/350215787 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f495c009c30 con 0x7f496c106670 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.584+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c198740 con 0x7f496c106670 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.584+0000 7f4962ffd640 1 -- 172.21.15.22:0/350215787 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c0047a0 con 0x7f496c106670 2024-03-08T23:59:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.584+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c1a49e0 con 0x7f496c106670 2024-03-08T23:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.586+0000 7f4962ffd640 1 -- 172.21.15.22:0/350215787 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f495c004020 con 0x7f496c106670 2024-03-08T23:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.586+0000 7f497676a740 1 Processor -- start 2024-03-08T23:59:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f497676a740 1 -- start start 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f497676a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 0x561ef781ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f497676a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef781efd0 0x561ef781f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f497676a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef7828550 0x561ef782a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f497676a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef77835f0 con 0x561ef781efd0 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f497676a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef772c930 con 0x561ef7828550 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f497676a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef777f690 con 0x561ef781fd40 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4962ffd640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49580a2a90 0x7f49580a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971847640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef781efd0 0x561ef781f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971847640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef781efd0 0x561ef781f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54730/0 (socket says 172.21.15.22:54730) 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971046640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef7828550 0x561ef782a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971847640 1 -- 172.21.15.22:0/4002861834 learned_addr learned my addr 172.21.15.22:0/4002861834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4972048640 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 0x561ef781ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4972048640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49580a2a90 0x7f49580a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4962ffd640 1 -- 172.21.15.22:0/350215787 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f495c0ced50 con 0x7f496c106670 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971046640 1 -- 172.21.15.22:0/4002861834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 msgr2=0x561ef781ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971046640 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 0x561ef781ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971046640 1 -- 172.21.15.22:0/4002861834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef781efd0 msgr2=0x561ef781f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971046640 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef781efd0 0x561ef781f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971046640 1 -- 172.21.15.22:0/4002861834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef7498210 con 0x561ef7828550 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.587+0000 7f4971847640 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef781efd0 0x561ef781f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f4972048640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49580a2a90 0x7f49580a4f50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f496400a780 tx=0x7f496400a4d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f4971046640 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef7828550 0x561ef782a940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f495c0cc100 tx=0x7f495c066220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d0111052e3da70 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f4957fff640 1 -- 172.21.15.22:0/4002861834 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c09ab50 con 0x561ef7828550 2024-03-08T23:59:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f4957fff640 1 -- 172.21.15.22:0/4002861834 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f495c021d00 con 0x561ef7828550 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f4957fff640 1 -- 172.21.15.22:0/4002861834 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c09a970 con 0x561ef7828550 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f497676a740 1 -- 172.21.15.22:0/4002861834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef7828550 msgr2=0x561ef782a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f497676a740 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef7828550 0x561ef782a940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f495c0cc100 tx=0x7f495c066220 comp rx=0 tx=0).stop 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f497676a740 1 -- 172.21.15.22:0/4002861834 shutdown_connections 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f497676a740 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef7828550 0x561ef782a940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f497676a740 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef781efd0 0x561ef781f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f497676a740 1 --2- 172.21.15.22:0/4002861834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 0x561ef781ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.588+0000 7f497676a740 1 -- 172.21.15.22:0/4002861834 >> 172.21.15.22:0/4002861834 conn(0x561ef77e65b0 msgr2=0x561ef7819cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 -- 172.21.15.22:0/4002861834 shutdown_connections 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 -- 172.21.15.22:0/4002861834 wait complete. 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 -- start start 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 Processor -- start 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef78255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7825b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef7826040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef782b280 con 0x561ef7828550 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef773fd90 con 0x561ef781fd40 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef782b400 con 0x561ef781efd0 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.589+0000 7f497676a740 10 client.0 _ll_register_callbacks cb 0x561ef7497a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef78255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef78255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59984/0 (socket says 172.21.15.22:59984) 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 -- 172.21.15.22:0/4105387768 learned_addr learned my addr 172.21.15.22:0/4105387768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4972048640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef7826040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971046640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7825b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 -- 172.21.15.22:0/4105387768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 msgr2=0x561ef7825b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7825b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 -- 172.21.15.22:0/4105387768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 msgr2=0x561ef7826040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef7826040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 -- 172.21.15.22:0/4105387768 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ef7564620 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4972048640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef7826040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f4971847640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef78255c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4968035000 tx=0x7f496802ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d873b60ac79daeb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f49557fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f49557fa640 1 -- 172.21.15.22:0/4105387768 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496802c690 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ef7790b30 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f49557fa640 1 -- 172.21.15.22:0/4105387768 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f49680046c0 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.590+0000 7f49557fa640 1 -- 172.21.15.22:0/4105387768 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49680048c0 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 10 client.26752 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561ef7837600 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f49557fa640 1 -- 172.21.15.22:0/4105387768 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561ef7790b30 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f49557fa640 1 -- 172.21.15.22:0/4105387768 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f49680037b0 con 0x561ef781efd0 2024-03-08T23:59:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 10 client.26752 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 10 client.26752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 20 client.26752 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7838a70 0x561ef783ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 4 client.26752 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ef778ae80 con 0x561ef7838a70 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f4971046640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7838a70 0x561ef783ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f4971046640 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7838a70 0x561ef783ae60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7c5ea716657df4 server_cookie=de8b803836f36e99 in_seq=0 out_seq=0 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.591+0000 7f49557fa640 10 client.26752 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f49557fa640 1 -- 172.21.15.22:0/4105387768 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ef778ae80 con 0x561ef7838a70 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f49557fa640 10 client.26752 handle_command_reply: tid=0 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 client.26752 shutdown 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 msgr2=0x561ef78255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef78255c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4968035000 tx=0x7f496802ce30 comp rx=0 tx=0).stop 2024-03-08T23:59:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 shutdown_connections 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7838a70 0x561ef783ae60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef7826040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7825b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f497676a740 1 --2- 172.21.15.22:0/4105387768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef78255c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.592+0000 7f49557fa640 0 client.26752 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.593+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 >> 172.21.15.22:0/4105387768 conn(0x561ef77e65b0 msgr2=0x561ef781ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.593+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 shutdown_connections 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.593+0000 7f497676a740 1 -- 172.21.15.22:0/4105387768 wait complete. 2024-03-08T23:59:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.593+0000 7f497676a740 20 client.26752 trim_cache size 0 max 16384 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 Processor -- start 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 -- start start 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef7827cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7823eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef78243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1b0a50 con 0x561ef7828550 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1ae5f0 con 0x561ef781fd40 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.606+0000 7f497676a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1ac690 con 0x561ef781efd0 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef78243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef78243f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54744/0 (socket says 172.21.15.22:54744) 2024-03-08T23:59:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 -- 172.21.15.22:0/4050049084 learned_addr learned my addr 172.21.15.22:0/4050049084 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4971046640 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7823eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 -- 172.21.15.22:0/4050049084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 msgr2=0x561ef7827cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef7827cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 -- 172.21.15.22:0/4050049084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 msgr2=0x561ef7823eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7823eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 -- 172.21.15.22:0/4050049084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef7815360 con 0x561ef7828550 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f4972048640 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef78243f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4964011fd0 tx=0x7f496400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea4bbf0a7cc0e58 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f49577fe640 1 -- 172.21.15.22:0/4050049084 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964003b70 con 0x561ef7828550 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f49577fe640 1 -- 172.21.15.22:0/4050049084 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4964003d10 con 0x561ef7828550 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.607+0000 7f49577fe640 1 -- 172.21.15.22:0/4050049084 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496402b890 con 0x561ef7828550 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 -- 172.21.15.22:0/4050049084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 msgr2=0x561ef78243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef78243f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4964011fd0 tx=0x7f496400a890 comp rx=0 tx=0).stop 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 -- 172.21.15.22:0/4050049084 shutdown_connections 2024-03-08T23:59:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef78243f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781fd40 0x561ef7823eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 --2- 172.21.15.22:0/4050049084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781efd0 0x561ef7827cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 -- 172.21.15.22:0/4050049084 >> 172.21.15.22:0/4050049084 conn(0x561ef77e65b0 msgr2=0x561ef78197f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 -- 172.21.15.22:0/4050049084 shutdown_connections 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 -- 172.21.15.22:0/4050049084 wait complete. 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 -- start start 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.608+0000 7f497676a740 1 Processor -- start 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f497676a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781efd0 0x561ef782f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f497676a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 0x561ef782dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f497676a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef782e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f497676a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1acc90 con 0x561ef7828550 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f497676a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1adf90 con 0x561ef781efd0 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f497676a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1ad2d0 con 0x561ef781fd40 2024-03-08T23:59:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f497676a740 10 client.0 _ll_register_callbacks cb 0x7f495c00bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4971847640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781efd0 0x561ef782f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4971847640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781efd0 0x561ef782f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41648/0 (socket says 172.21.15.22:41648) 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4971847640 1 -- 172.21.15.22:0/3671897790 learned_addr learned my addr 172.21.15.22:0/3671897790 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4972048640 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef782e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4972048640 1 -- 172.21.15.22:0/3671897790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 msgr2=0x561ef782dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4972048640 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 0x561ef782dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4972048640 1 -- 172.21.15.22:0/3671897790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781efd0 msgr2=0x561ef782f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4972048640 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781efd0 0x561ef782f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4972048640 1 -- 172.21.15.22:0/3671897790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ef782e850 con 0x561ef7828550 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4971847640 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781efd0 0x561ef782f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f4972048640 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef782e1c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4964004990 tx=0x7f4964004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ade721638ee02 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f49557fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.609+0000 7f49557fa640 1 -- 172.21.15.22:0/3671897790 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496403a060 con 0x561ef7828550 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f49557fa640 1 -- 172.21.15.22:0/3671897790 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4964003ec0 con 0x561ef7828550 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4964003d10 con 0x561ef7828550 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f49557fa640 1 -- 172.21.15.22:0/3671897790 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4964041050 con 0x561ef7828550 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 10 client.17304 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561ef788fe10 con 0x561ef7828550 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f49557fa640 1 -- 172.21.15.22:0/3671897790 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4964003d10 con 0x561ef7828550 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f49557fa640 1 -- 172.21.15.22:0/3671897790 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4964005290 con 0x561ef7828550 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 10 client.17304 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 20 client.17304 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7890430 0x561ef7892850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 4 client.17304 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:59:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561ef7709710 con 0x561ef7890430 2024-03-08T23:59:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.610+0000 7f4971847640 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7890430 0x561ef7892850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.611+0000 7f4971847640 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7890430 0x561ef7892850 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d734c3b0b5679167 server_cookie=2877e4690c8d820 in_seq=0 out_seq=0 2024-03-08T23:59:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.611+0000 7f49557fa640 10 client.17304 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:44.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 526985 11.49 MB/sec execute 112 sec latency 50.940 ms 2024-03-08T23:59:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f49557fa640 1 -- 172.21.15.22:0/3671897790 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561ef7709710 con 0x561ef7890430 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f49557fa640 10 client.17304 handle_command_reply: tid=0 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 client.17304 shutdown 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 msgr2=0x561ef782e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef782e1c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4964004990 tx=0x7f4964004c70 comp rx=0 tx=0).stop 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 shutdown_connections 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x561ef7890430 0x561ef7892850 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561ef7828550 0x561ef782e1c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f49557fa640 0 client.17304 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ef781fd40 0x561ef782dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 --2- 172.21.15.22:0/3671897790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ef781efd0 0x561ef782f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 >> 172.21.15.22:0/3671897790 conn(0x561ef77e65b0 msgr2=0x561ef7823d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 shutdown_connections 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.870+0000 7f497676a740 1 -- 172.21.15.22:0/3671897790 wait complete. 2024-03-08T23:59:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.871+0000 7f497676a740 20 client.17304 trim_cache size 0 max 16384 2024-03-08T23:59:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:59:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:59:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:59:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.871+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49580a2a90 msgr2=0x7f49580a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.871+0000 7f4973aab640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49580a2a90 0x7f49580a4f50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f496400a780 tx=0x7f496400a4d0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.871+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c106670 msgr2=0x7f496c19dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.871+0000 7f4973aab640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c106670 0x7f496c19dcc0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f495c00d950 tx=0x7f495c00de10 comp rx=0 tx=0).stop 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.871+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 shutdown_connections 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.871+0000 7f4973aab640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49580a2a90 0x7f49580a4f50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.872+0000 7f4973aab640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c197930 0x7f496c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.872+0000 7f4973aab640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f496c106670 0x7f496c19dcc0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.872+0000 7f4973aab640 1 --2- 172.21.15.22:0/350215787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f496c0ffa00 0x7f496c19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.872+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 >> 172.21.15.22:0/350215787 conn(0x7f496c076710 msgr2=0x7f496c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.872+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 shutdown_connections 2024-03-08T23:59:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:44.872+0000 7f4973aab640 1 -- 172.21.15.22:0/350215787 wait complete. 2024-03-08T23:59:45.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:44 smithi192 ceph-mon[35995]: pgmap v162: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.8 MiB/s wr, 374 op/s 2024-03-08T23:59:45.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:44 smithi022 ceph-mon[31894]: pgmap v162: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.8 MiB/s wr, 374 op/s 2024-03-08T23:59:45.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:44 smithi062 ceph-mon[36129]: pgmap v162: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.8 MiB/s wr, 374 op/s 2024-03-08T23:59:45.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 529369 11.53 MB/sec execute 113 sec latency 160.608 ms 2024-03-08T23:59:46.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 530890 11.48 MB/sec execute 114 sec latency 29.992 ms 2024-03-08T23:59:47.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:46 smithi192 ceph-mon[35995]: pgmap v163: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 486 op/s 2024-03-08T23:59:47.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:46 smithi022 ceph-mon[31894]: pgmap v163: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 486 op/s 2024-03-08T23:59:47.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:46 smithi062 ceph-mon[36129]: pgmap v163: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 486 op/s 2024-03-08T23:59:47.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 533260 11.50 MB/sec execute 115 sec latency 43.141 ms 2024-03-08T23:59:48.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:59:48.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:59:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/685351763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b84068760 msgr2=0x7f9b84068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/685351763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b84068760 0x7f9b84068b40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009900 tx=0x7f9b8002bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/685351763 shutdown_connections 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/685351763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84106630 0x7f9b8410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/685351763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84069110 0x7f9b84105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/685351763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b84068760 0x7f9b84068b40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/685351763 >> 172.21.15.22:0/685351763 conn(0x7f9b840772f0 msgr2=0x7f9b84077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/685351763 shutdown_connections 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.701+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/685351763 wait complete. 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 Processor -- start 2024-03-08T23:59:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 -- start start 2024-03-08T23:59:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84068760 0x7f9b8419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84106630 0x7f9b8419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b841979a0 0x7f9b84197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84111660 con 0x7f9b841979a0 2024-03-08T23:59:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841114e0 con 0x7f9b84106630 2024-03-08T23:59:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b8b4dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841117e0 con 0x7f9b84068760 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b89a79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b841979a0 0x7f9b84197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b89a79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b841979a0 0x7f9b84197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54774/0 (socket says 172.21.15.22:54774) 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.702+0000 7f9b89a79640 1 -- 172.21.15.22:0/2549789820 learned_addr learned my addr 172.21.15.22:0/2549789820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b88a77640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84106630 0x7f9b8419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b89278640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84068760 0x7f9b8419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b89a79640 1 -- 172.21.15.22:0/2549789820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84068760 msgr2=0x7f9b8419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b89a79640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84068760 0x7f9b8419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b89a79640 1 -- 172.21.15.22:0/2549789820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84106630 msgr2=0x7f9b8419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b89a79640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84106630 0x7f9b8419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b89a79640 1 -- 172.21.15.22:0/2549789820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b841984c0 con 0x7f9b841979a0 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b88a77640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84106630 0x7f9b8419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b89a79640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b841979a0 0x7f9b84197e50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00efd0 tx=0x7f9b7c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b7a7fc640 1 -- 172.21.15.22:0/2549789820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c003b70 con 0x7f9b841979a0 2024-03-08T23:59:48.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b7a7fc640 1 -- 172.21.15.22:0/2549789820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c003d10 con 0x7f9b841979a0 2024-03-08T23:59:48.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b841987b0 con 0x7f9b841979a0 2024-03-08T23:59:48.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b7a7fc640 1 -- 172.21.15.22:0/2549789820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0109f0 con 0x7f9b841979a0 2024-03-08T23:59:48.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.703+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b841a4a00 con 0x7f9b841979a0 2024-03-08T23:59:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.706+0000 7f9b7a7fc640 1 -- 172.21.15.22:0/2549789820 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9b7c004020 con 0x7f9b841979a0 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.706+0000 7f9b8e198740 1 Processor -- start 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.706+0000 7f9b8e198740 1 -- start start 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b7a7fc640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b700a29f0 0x7f9b700a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b0080 0x560ca56b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b8c60 0x560ca56bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b8e198740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca5612e00 con 0x560ca56af760 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b8e198740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca55bbb10 con 0x560ca56b0080 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b8e198740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca560eea0 con 0x560ca56b8c60 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b88a77640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b700a29f0 0x7f9b700a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b7a7fc640 1 -- 172.21.15.22:0/2549789820 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9b7c0cf600 con 0x7f9b841979a0 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54784/0 (socket says 172.21.15.22:54784) 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 -- 172.21.15.22:0/2342269213 learned_addr learned my addr 172.21.15.22:0/2342269213 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89a79640 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b8c60 0x560ca56bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b88a77640 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b0080 0x560ca56b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 -- 172.21.15.22:0/2342269213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b8c60 msgr2=0x560ca56bb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b8c60 0x560ca56bb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 -- 172.21.15.22:0/2342269213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b0080 msgr2=0x560ca56b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b0080 0x560ca56b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 -- 172.21.15.22:0/2342269213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca53f17d0 con 0x560ca56af760 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b88a77640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b700a29f0 0x7f9b700a4eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009fd0 tx=0x7f9b74009ca0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.707+0000 7f9b89278640 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56afb40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9b8002c170 tx=0x7f9b80005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfdff565b1885ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b6f7fe640 1 -- 172.21.15.22:0/2342269213 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80042070 con 0x560ca56af760 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b6f7fe640 1 -- 172.21.15.22:0/2342269213 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b80004030 con 0x560ca56af760 2024-03-08T23:59:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b6f7fe640 1 -- 172.21.15.22:0/2342269213 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80004c90 con 0x560ca56af760 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 -- 172.21.15.22:0/2342269213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 msgr2=0x560ca56afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56afb40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9b8002c170 tx=0x7f9b80005be0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 -- 172.21.15.22:0/2342269213 shutdown_connections 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b8c60 0x560ca56bb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b0080 0x560ca56b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2342269213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56afb40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 -- 172.21.15.22:0/2342269213 >> 172.21.15.22:0/2342269213 conn(0x560ca56a9e90 msgr2=0x560ca56aa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 -- 172.21.15.22:0/2342269213 shutdown_connections 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.708+0000 7f9b8e198740 1 -- 172.21.15.22:0/2342269213 wait complete. 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 -- start start 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 Processor -- start 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56af760 0x560ca56b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b8c60 0x560ca56b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca56bb9f0 con 0x560ca56b8c60 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca55cf0a0 con 0x560ca56af760 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca56bbb70 con 0x560ca56b0080 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b8e198740 10 client.0 _ll_register_callbacks cb 0x560ca55cea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.709+0000 7f9b89a79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b8c60 0x560ca56b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89278640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56af760 0x560ca56b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b88a77640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b8c60 0x560ca56b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54800/0 (socket says 172.21.15.22:54800) 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 -- 172.21.15.22:0/1718671301 learned_addr learned my addr 172.21.15.22:0/1718671301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 -- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 msgr2=0x560ca56b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 -- 172.21.15.22:0/1718671301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56af760 msgr2=0x560ca56b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56af760 0x560ca56b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 -- 172.21.15.22:0/1718671301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ca53273f0 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b88a77640 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89278640 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56af760 0x560ca56b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b89a79640 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b8c60 0x560ca56b5ca0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c09b5c0 tx=0x7f9b7c09b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32af3f48d40b5d64 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b6cff9640 1 -- 172.21.15.22:0/1718671301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0cf040 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b6cff9640 1 -- 172.21.15.22:0/1718671301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9b7c004310 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b80004030 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b6cff9640 1 -- 172.21.15.22:0/1718671301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b7c09b8c0 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b8e198740 10 client.17316 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.710+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560ca52e70f0 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b6cff9640 1 -- 172.21.15.22:0/1718671301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b80004030 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b6cff9640 1 -- 172.21.15.22:0/1718671301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b7c01f020 con 0x560ca56b8c60 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b8e198740 10 client.17316 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b8e198740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b8e198740 20 client.17316 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b8e198740 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca56c8610 0x560ca56caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b8e198740 4 client.17316 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ca5620340 con 0x560ca56c8610 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b89278640 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca56c8610 0x560ca56caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b89278640 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca56c8610 0x560ca56caa20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06bcff30a972d94 server_cookie=545bfe027594115a in_seq=0 out_seq=0 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.711+0000 7f9b6cff9640 10 client.17316 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b6cff9640 1 -- 172.21.15.22:0/1718671301 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560ca5620340 con 0x560ca56c8610 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b6cff9640 10 client.17316 handle_command_reply: tid=0 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 client.17316 shutdown 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b8c60 msgr2=0x560ca56b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b8c60 0x560ca56b5ca0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c09b5c0 tx=0x7f9b7c09b220 comp rx=0 tx=0).stop 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 shutdown_connections 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca56c8610 0x560ca56caa20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b8c60 0x560ca56b5ca0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b6cff9640 0 client.17316 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 --2- 172.21.15.22:0/1718671301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56af760 0x560ca56b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.712+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 >> 172.21.15.22:0/1718671301 conn(0x560ca56a9e90 msgr2=0x560ca56abbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.713+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 shutdown_connections 2024-03-08T23:59:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.713+0000 7f9b8e198740 1 -- 172.21.15.22:0/1718671301 wait complete. 2024-03-08T23:59:48.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.713+0000 7f9b8e198740 20 client.17316 trim_cache size 0 max 16384 2024-03-08T23:59:48.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 534866 11.45 MB/sec execute 116 sec latency 72.355 ms 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 Processor -- start 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 -- start start 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56af760 0x560ca56b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b0080 0x560ca56b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841b0970 con 0x560ca56b0080 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841ae510 con 0x560ca56b8c60 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b8e198740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841ac5b0 con 0x560ca56af760 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b0080 0x560ca56b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b0080 0x560ca56b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54802/0 (socket says 172.21.15.22:54802) 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 -- 172.21.15.22:0/3973327248 learned_addr learned my addr 172.21.15.22:0/3973327248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b89a79640 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b89278640 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56af760 0x560ca56b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 -- 172.21.15.22:0/3973327248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56af760 msgr2=0x560ca56b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56af760 0x560ca56b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 -- 172.21.15.22:0/3973327248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 msgr2=0x560ca56b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.728+0000 7f9b88a77640 1 -- 172.21.15.22:0/3973327248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca56afeb0 con 0x560ca56b0080 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b88a77640 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b0080 0x560ca56b83d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9b8406aa50 tx=0x7f9b74005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d2bb115ca6169f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b6effd640 1 -- 172.21.15.22:0/3973327248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74013d60 con 0x560ca56b0080 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b6effd640 1 -- 172.21.15.22:0/3973327248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b740130c0 con 0x560ca56b0080 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b6effd640 1 -- 172.21.15.22:0/3973327248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740108a0 con 0x560ca56b0080 2024-03-08T23:59:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b8e198740 1 -- 172.21.15.22:0/3973327248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b0080 msgr2=0x560ca56b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b8e198740 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b0080 0x560ca56b83d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9b8406aa50 tx=0x7f9b74005f90 comp rx=0 tx=0).stop 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b8e198740 1 -- 172.21.15.22:0/3973327248 shutdown_connections 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b8e198740 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b8e198740 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56b0080 0x560ca56b83d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b8e198740 1 --2- 172.21.15.22:0/3973327248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56af760 0x560ca56b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.729+0000 7f9b8e198740 1 -- 172.21.15.22:0/3973327248 >> 172.21.15.22:0/3973327248 conn(0x560ca56a9e90 msgr2=0x560ca56b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 -- 172.21.15.22:0/3973327248 shutdown_connections 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 -- 172.21.15.22:0/3973327248 wait complete. 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 -- start start 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 Processor -- start 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841acbb0 con 0x560ca56af760 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841adeb0 con 0x560ca56b8c60 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841ad1f0 con 0x560ca56b0080 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b8e198740 10 client.0 _ll_register_callbacks cb 0x7f9b7c011fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b89278640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b88a77640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b89278640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56bfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54812/0 (socket says 172.21.15.22:54812) 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b89278640 1 -- 172.21.15.22:0/2361558026 learned_addr learned my addr 172.21.15.22:0/2361558026 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.730+0000 7f9b89a79640 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b89278640 1 -- 172.21.15.22:0/2361558026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 msgr2=0x560ca56c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b89278640 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b89278640 1 -- 172.21.15.22:0/2361558026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 msgr2=0x560ca56bc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b89278640 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56bc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b89278640 1 -- 172.21.15.22:0/2361558026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ca56bca40 con 0x560ca56af760 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b89278640 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56bfcd0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9b80004840 tx=0x7f9b80004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ba5cccf15d0654 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b6cff9640 1 -- 172.21.15.22:0/2361558026 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80042040 con 0x560ca56af760 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b6cff9640 1 -- 172.21.15.22:0/2361558026 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9b8002ca80 con 0x560ca56af760 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b740130c0 con 0x560ca56af760 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b6cff9640 1 -- 172.21.15.22:0/2361558026 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b8005b050 con 0x560ca56af760 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b8e198740 10 client.17322 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560ca5723ae0 con 0x560ca56af760 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b6cff9640 1 -- 172.21.15.22:0/2361558026 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b740130c0 con 0x560ca56af760 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b6cff9640 1 -- 172.21.15.22:0/2361558026 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b80004d20 con 0x560ca56af760 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b8e198740 10 client.17322 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b8e198740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b8e198740 20 client.17322 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.731+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca5723ec0 0x560ca57262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b8e198740 4 client.17322 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ca561a690 con 0x560ca5723ec0 2024-03-08T23:59:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b88a77640 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca5723ec0 0x560ca57262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b88a77640 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca5723ec0 0x560ca57262b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd9a9b438c3d184 server_cookie=d59ae3bd815338df in_seq=0 out_seq=0 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b6cff9640 10 client.17322 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b6cff9640 1 -- 172.21.15.22:0/2361558026 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560ca561a690 con 0x560ca5723ec0 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b6cff9640 10 client.17322 handle_command_reply: tid=0 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.732+0000 7f9b8e198740 1 client.17322 shutdown 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 msgr2=0x560ca56bfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56bfcd0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9b80004840 tx=0x7f9b80004870 comp rx=0 tx=0).stop 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 shutdown_connections 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560ca5723ec0 0x560ca57262b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ca56b8c60 0x560ca56bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ca56b0080 0x560ca56c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 --2- 172.21.15.22:0/2361558026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ca56af760 0x560ca56bfcd0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b6cff9640 0 client.17322 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 >> 172.21.15.22:0/2361558026 conn(0x560ca56a9e90 msgr2=0x560ca56b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 shutdown_connections 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 1 -- 172.21.15.22:0/2361558026 wait complete. 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8e198740 20 client.17322 trim_cache size 0 max 16384 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-08T23:59:48.735 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:59:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b700a29f0 msgr2=0x7f9b700a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b700a29f0 0x7f9b700a4eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009fd0 tx=0x7f9b74009ca0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b841979a0 msgr2=0x7f9b84197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b841979a0 0x7f9b84197e50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00efd0 tx=0x7f9b7c00c6a0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 shutdown_connections 2024-03-08T23:59:48.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b700a29f0 0x7f9b700a4eb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b841979a0 0x7f9b84197e50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84106630 0x7f9b8419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 --2- 172.21.15.22:0/2549789820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84068760 0x7f9b8419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 >> 172.21.15.22:0/2549789820 conn(0x7f9b840772f0 msgr2=0x7f9b840fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 shutdown_connections 2024-03-08T23:59:48.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.733+0000 7f9b8b4dc640 1 -- 172.21.15.22:0/2549789820 wait complete. 2024-03-08T23:59:48.739 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-08T23:59:48.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T23:59:48.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 -- 172.21.15.22:0/2830365024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100e10 msgr2=0x7f79c4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 --2- 172.21.15.22:0/2830365024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100e10 0x7f79c4104e50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009900 tx=0x7f79b402bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:48.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 -- 172.21.15.22:0/2830365024 shutdown_connections 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 --2- 172.21.15.22:0/2830365024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4105530 0x7f79c410d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 --2- 172.21.15.22:0/2830365024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100e10 0x7f79c4104e50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 --2- 172.21.15.22:0/2830365024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c4100460 0x7f79c4100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 -- 172.21.15.22:0/2830365024 >> 172.21.15.22:0/2830365024 conn(0x7f79c40772f0 msgr2=0x7f79c4077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 -- 172.21.15.22:0/2830365024 shutdown_connections 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.850+0000 7f79cc420640 1 -- 172.21.15.22:0/2830365024 wait complete. 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 Processor -- start 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 -- start start 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100460 0x7f79c4101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4105530 0x7f79c4102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c4102970 0x7f79c41a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4110610 con 0x7f79c4100460 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4110490 con 0x7f79c4105530 2024-03-08T23:59:48.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79cc420640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4110790 con 0x7f79c4102970 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79ca1bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100460 0x7f79c4101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79ca1bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100460 0x7f79c4101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54820/0 (socket says 172.21.15.22:54820) 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79ca1bc640 1 -- 172.21.15.22:0/3387471655 learned_addr learned my addr 172.21.15.22:0/3387471655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.851+0000 7f79c99bb640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4105530 0x7f79c4102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca1bc640 1 -- 172.21.15.22:0/3387471655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c4102970 msgr2=0x7f79c41a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca9bd640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c4102970 0x7f79c41a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca1bc640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c4102970 0x7f79c41a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca1bc640 1 -- 172.21.15.22:0/3387471655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4105530 msgr2=0x7f79c4102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca1bc640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4105530 0x7f79c4102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca1bc640 1 -- 172.21.15.22:0/3387471655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c41a64b0 con 0x7f79c4100460 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca9bd640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c4102970 0x7f79c41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79ca1bc640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100460 0x7f79c4101ef0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00eb90 tx=0x7f79bc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79bb7fe640 1 -- 172.21.15.22:0/3387471655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc004d20 con 0x7f79c4100460 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79bb7fe640 1 -- 172.21.15.22:0/3387471655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79bc00ee90 con 0x7f79c4100460 2024-03-08T23:59:48.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c41a67a0 con 0x7f79c4100460 2024-03-08T23:59:48.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79bb7fe640 1 -- 172.21.15.22:0/3387471655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc010830 con 0x7f79c4100460 2024-03-08T23:59:48.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.852+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c41a6c70 con 0x7f79c4100460 2024-03-08T23:59:48.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.853+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c40680e0 con 0x7f79c4100460 2024-03-08T23:59:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.854+0000 7f79bb7fe640 1 -- 172.21.15.22:0/3387471655 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f79bc005000 con 0x7f79c4100460 2024-03-08T23:59:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.855+0000 7f79bb7fe640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79b00a29a0 0x7f79b00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.855+0000 7f79c99bb640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79b00a29a0 0x7f79b00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.855+0000 7f79bb7fe640 1 -- 172.21.15.22:0/3387471655 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f79bc0cfb00 con 0x7f79c4100460 2024-03-08T23:59:48.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.855+0000 7f79c99bb640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79b00a29a0 0x7f79b00a4e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79b402c240 tx=0x7f79b402df90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:48.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:48.858+0000 7f79bb7fe640 1 -- 172.21.15.22:0/3387471655 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f79bc053260 con 0x7f79c4100460 2024-03-08T23:59:49.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.093+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79c4103420 con 0x7f79c4100460 2024-03-08T23:59:49.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.093+0000 7f79bb7fe640 1 -- 172.21.15.22:0/3387471655 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f79bc09c460 con 0x7f79c4100460 2024-03-08T23:59:49.089 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-08T23:59:49.089 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:51.502302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26555":{"gid":26555,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2421521561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2421521561},{"type":"v1","addr":"172.21.15.62:6837","nonce":2421521561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T23:59:49.089 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-03-08T23:59:49.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79b00a29a0 msgr2=0x7f79b00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79b00a29a0 0x7f79b00a4e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79b402c240 tx=0x7f79b402df90 comp rx=0 tx=0).stop 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100460 msgr2=0x7f79c4101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100460 0x7f79c4101ef0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00eb90 tx=0x7f79bc00c6a0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 shutdown_connections 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79b00a29a0 0x7f79b00a4e60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c4102970 0x7f79c41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79c4105530 0x7f79c4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 --2- 172.21.15.22:0/3387471655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79c4100460 0x7f79c4101ef0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 >> 172.21.15.22:0/3387471655 conn(0x7f79c40772f0 msgr2=0x7f79c4068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 shutdown_connections 2024-03-08T23:59:49.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.096+0000 7f79cc420640 1 -- 172.21.15.22:0/3387471655 wait complete. 2024-03-08T23:59:49.102 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:59:49.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-08T23:59:49.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:48 smithi192 ceph-mon[35995]: pgmap v164: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 193 op/s 2024-03-08T23:59:49.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.228+0000 7fdb1c81b640 1 -- 172.21.15.22:0/1048943356 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14032340 con 0x7fdb1810fa10 2024-03-08T23:59:49.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.229+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1048943356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fa10 msgr2=0x7fdb1810d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.229+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1048943356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fa10 0x7fdb1810d630 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009860 tx=0x7fdb1402bb70 comp rx=0 tx=0).stop 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.229+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1048943356 shutdown_connections 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.229+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1048943356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb1811aad0 0x7fdb1811cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.229+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1048943356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb1810fe80 0x7fdb1810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.229+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1048943356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fa10 0x7fdb1810d630 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.229+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1048943356 >> 172.21.15.22:0/1048943356 conn(0x7fdb1806c3e0 msgr2=0x7fdb1806c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.230+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1048943356 shutdown_connections 2024-03-08T23:59:49.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.230+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1048943356 wait complete. 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 Processor -- start 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 -- start start 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb1810fa10 0x7fdb18119ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fe80 0x7fdb1811a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb1811aad0 0x7fdb18113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181108c0 con 0x7fdb1810fe80 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18110740 con 0x7fdb1811aad0 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.231+0000 7fdb1fa81640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18110a40 con 0x7fdb1810fa10 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.232+0000 7fdb1d01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fe80 0x7fdb1811a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.232+0000 7fdb1d01c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fe80 0x7fdb1811a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54840/0 (socket says 172.21.15.22:54840) 2024-03-08T23:59:49.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.232+0000 7fdb1d01c640 1 -- 172.21.15.22:0/1959439379 learned_addr learned my addr 172.21.15.22:0/1959439379 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.233+0000 7fdb1e01e640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb1811aad0 0x7fdb18113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.233+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb1810fa10 0x7fdb18119ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.233+0000 7fdb1d01c640 1 -- 172.21.15.22:0/1959439379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb1810fa10 msgr2=0x7fdb18119ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.233+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb1810fa10 0x7fdb18119ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.233+0000 7fdb1d01c640 1 -- 172.21.15.22:0/1959439379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb1811aad0 msgr2=0x7fdb18113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.233+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb1811aad0 0x7fdb18113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.233+0000 7fdb1d01c640 1 -- 172.21.15.22:0/1959439379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb181147f0 con 0x7fdb1810fe80 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.234+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fe80 0x7fdb1811a3e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800d740 tx=0x7fdb0800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.234+0000 7fdb0effd640 1 -- 172.21.15.22:0/1959439379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08017070 con 0x7fdb1810fe80 2024-03-08T23:59:49.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.234+0000 7fdb0effd640 1 -- 172.21.15.22:0/1959439379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb0800de50 con 0x7fdb1810fe80 2024-03-08T23:59:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.235+0000 7fdb0effd640 1 -- 172.21.15.22:0/1959439379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004e00 con 0x7fdb1810fe80 2024-03-08T23:59:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.235+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb18114ae0 con 0x7fdb1810fe80 2024-03-08T23:59:49.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.235+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb181bbe10 con 0x7fdb1810fe80 2024-03-08T23:59:49.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:48 smithi022 ceph-mon[31894]: pgmap v164: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 193 op/s 2024-03-08T23:59:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.239+0000 7fdb22666740 1 Processor -- start 2024-03-08T23:59:49.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.240+0000 7fdb22666740 1 -- start start 2024-03-08T23:59:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.240+0000 7fdb22666740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.240+0000 7fdb22666740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743cbac0 0x5567743d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.240+0000 7fdb22666740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743d46a0 0x5567743d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.240+0000 7fdb22666740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556774392360 con 0x5567743cb1a0 2024-03-08T23:59:49.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.240+0000 7fdb22666740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677432eec0 con 0x5567743cbac0 2024-03-08T23:59:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.240+0000 7fdb22666740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567742d7d30 con 0x5567743d46a0 2024-03-08T23:59:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.241+0000 7fdb1e01e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.241+0000 7fdb1e01e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54854/0 (socket says 172.21.15.22:54854) 2024-03-08T23:59:49.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.241+0000 7fdb1e01e640 1 -- 172.21.15.22:0/16510775 learned_addr learned my addr 172.21.15.22:0/16510775 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.241+0000 7fdb0effd640 1 -- 172.21.15.22:0/1959439379 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdb08009c30 con 0x7fdb1810fe80 2024-03-08T23:59:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743d46a0 0x5567743d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb1e01e640 1 -- 172.21.15.22:0/16510775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743d46a0 msgr2=0x5567743d6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb1e01e640 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743d46a0 0x5567743d6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb1e01e640 1 -- 172.21.15.22:0/16510775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743cbac0 msgr2=0x5567743d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb1e01e640 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743cbac0 0x5567743d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb1e01e640 1 -- 172.21.15.22:0/16510775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567742d6bc0 con 0x5567743cb1a0 2024-03-08T23:59:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb1e01e640 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743cb580 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdb10005c20 tx=0x7fdb100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d9bba9fbf8f7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb0effd640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb040a2910 0x7fdb040a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.242+0000 7fdb0effd640 1 -- 172.21.15.22:0/1959439379 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fdb080cee10 con 0x7fdb1810fe80 2024-03-08T23:59:49.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.243+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb040a2910 0x7fdb040a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.243+0000 7fdb03fff640 1 -- 172.21.15.22:0/16510775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10003b70 con 0x5567743cb1a0 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.243+0000 7fdb03fff640 1 -- 172.21.15.22:0/16510775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb10003d10 con 0x5567743cb1a0 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.243+0000 7fdb03fff640 1 -- 172.21.15.22:0/16510775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb100047e0 con 0x5567743cb1a0 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.244+0000 7fdb22666740 1 -- 172.21.15.22:0/16510775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 msgr2=0x5567743cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.244+0000 7fdb22666740 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743cb580 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdb10005c20 tx=0x7fdb100095c0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.244+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb040a2910 0x7fdb040a4dd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb14000c00 tx=0x7fdb1402c090 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.245+0000 7fdb22666740 1 -- 172.21.15.22:0/16510775 shutdown_connections 2024-03-08T23:59:49.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.245+0000 7fdb22666740 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743d46a0 0x5567743d6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.245+0000 7fdb22666740 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743cbac0 0x5567743d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.245+0000 7fdb22666740 1 --2- 172.21.15.22:0/16510775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743cb580 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.245+0000 7fdb22666740 1 -- 172.21.15.22:0/16510775 >> 172.21.15.22:0/16510775 conn(0x5567743c58d0 msgr2=0x5567743c5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.246+0000 7fdb22666740 1 -- 172.21.15.22:0/16510775 shutdown_connections 2024-03-08T23:59:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.246+0000 7fdb22666740 1 -- 172.21.15.22:0/16510775 wait complete. 2024-03-08T23:59:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.246+0000 7fdb22666740 1 -- start start 2024-03-08T23:59:49.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.246+0000 7fdb22666740 1 Processor -- start 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb22666740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb22666740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb22666740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb22666740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567742cd4a0 con 0x5567743cbac0 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb22666740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677432af60 con 0x5567743d46a0 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb22666740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567743d7490 con 0x5567743cb1a0 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb22666740 10 client.0 _ll_register_callbacks cb 0x556774328d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb1e01e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb1e01e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743d2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41740/0 (socket says 172.21.15.22:41740) 2024-03-08T23:59:49.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.247+0000 7fdb1e01e640 1 -- 172.21.15.22:0/1424032494 learned_addr learned my addr 172.21.15.22:0/1424032494 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.248+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.248+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.248+0000 7fdb1d01c640 1 -- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 msgr2=0x5567743d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.248+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.248+0000 7fdb1d01c640 1 -- 172.21.15.22:0/1424032494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 msgr2=0x5567743d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.248+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.248+0000 7fdb1d01c640 1 -- 172.21.15.22:0/1424032494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567740433f0 con 0x5567743cbac0 2024-03-08T23:59:49.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.249+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743d1c30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdb080cc190 tx=0x7fdb0809aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956b9cd2c25db2ce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.251+0000 7fdb017fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.251+0000 7fdb017fa640 1 -- 172.21.15.22:0/1424032494 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004660 con 0x5567743cbac0 2024-03-08T23:59:49.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.251+0000 7fdb017fa640 1 -- 172.21.15.22:0/1424032494 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fdb080a33e0 con 0x5567743cbac0 2024-03-08T23:59:49.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.252+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb10003d10 con 0x5567743cbac0 2024-03-08T23:59:49.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.254+0000 7fdb017fa640 1 -- 172.21.15.22:0/1424032494 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb0809b7d0 con 0x5567743cbac0 2024-03-08T23:59:49.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.255+0000 7fdb22666740 10 client.17340 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.255+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55677410d7d0 con 0x5567743cbac0 2024-03-08T23:59:49.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb017fa640 1 -- 172.21.15.22:0/1424032494 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb10003d10 con 0x5567743cbac0 2024-03-08T23:59:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb017fa640 1 -- 172.21.15.22:0/1424032494 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdb0809a520 con 0x5567743cbac0 2024-03-08T23:59:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb22666740 10 client.17340 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb22666740 10 client.17340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:59:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb22666740 20 client.17340 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb22666740 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5567743e4660 0x5567743e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb22666740 4 client.17340 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:49.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.256+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55677433c400 con 0x5567743e4660 2024-03-08T23:59:49.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.257+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5567743e4660 0x5567743e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.258+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5567743e4660 0x5567743e6a50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ab0480c8bd3e833 server_cookie=9d471bd059cc28a4 in_seq=0 out_seq=0 2024-03-08T23:59:49.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.258+0000 7fdb017fa640 10 client.17340 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:49.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.259+0000 7fdb017fa640 1 -- 172.21.15.22:0/1424032494 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55677433c400 con 0x5567743e4660 2024-03-08T23:59:49.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.259+0000 7fdb017fa640 10 client.17340 handle_command_reply: tid=0 2024-03-08T23:59:49.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.259+0000 7fdb22666740 1 client.17340 shutdown 2024-03-08T23:59:49.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.260+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 msgr2=0x5567743d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.260+0000 7fdb22666740 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743d1c30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdb080cc190 tx=0x7fdb0809aea0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.261+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 shutdown_connections 2024-03-08T23:59:49.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.261+0000 7fdb22666740 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5567743e4660 0x5567743e6a50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.261+0000 7fdb22666740 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.261+0000 7fdb22666740 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743d1c30 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.261+0000 7fdb22666740 1 --2- 172.21.15.22:0/1424032494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.261+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 >> 172.21.15.22:0/1424032494 conn(0x5567743c58d0 msgr2=0x5567743c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.262+0000 7fdb017fa640 0 client.17340 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:49.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.262+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 shutdown_connections 2024-03-08T23:59:49.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.262+0000 7fdb22666740 1 -- 172.21.15.22:0/1424032494 wait complete. 2024-03-08T23:59:49.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.262+0000 7fdb22666740 20 client.17340 trim_cache size 0 max 16384 2024-03-08T23:59:49.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.289+0000 7fdb22666740 1 Processor -- start 2024-03-08T23:59:49.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.289+0000 7fdb22666740 1 -- start start 2024-03-08T23:59:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.289+0000 7fdb22666740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.289+0000 7fdb22666740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cbac0 0x5567743d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.289+0000 7fdb22666740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.290+0000 7fdb1d81d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.290+0000 7fdb22666740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181c7f90 con 0x5567743cb1a0 2024-03-08T23:59:49.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.290+0000 7fdb1d81d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743d3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54882/0 (socket says 172.21.15.22:54882) 2024-03-08T23:59:49.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.290+0000 7fdb1d81d640 1 -- 172.21.15.22:0/3190879527 learned_addr learned my addr 172.21.15.22:0/3190879527 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.290+0000 7fdb22666740 1 -- 172.21.15.22:0/3190879527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181c5b30 con 0x5567743d46a0 2024-03-08T23:59:49.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.290+0000 7fdb22666740 1 -- 172.21.15.22:0/3190879527 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181c3bd0 con 0x5567743cbac0 2024-03-08T23:59:49.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.291+0000 7fdb1e01e640 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.291+0000 7fdb1d81d640 1 -- 172.21.15.22:0/3190879527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cbac0 msgr2=0x5567743d1f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T23:59:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.291+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cbac0 0x5567743d1f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.291+0000 7fdb1d81d640 1 -- 172.21.15.22:0/3190879527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 msgr2=0x5567743ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.291+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.291+0000 7fdb1d81d640 1 -- 172.21.15.22:0/3190879527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567743e6dc0 con 0x5567743cb1a0 2024-03-08T23:59:49.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.292+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743d3e20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009780 tx=0x7fdb1403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277e4a030a6199ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.292+0000 7fdb037fe640 1 -- 172.21.15.22:0/3190879527 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14052070 con 0x5567743cb1a0 2024-03-08T23:59:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.293+0000 7fdb037fe640 1 -- 172.21.15.22:0/3190879527 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb14005040 con 0x5567743cb1a0 2024-03-08T23:59:49.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.293+0000 7fdb037fe640 1 -- 172.21.15.22:0/3190879527 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14045040 con 0x5567743cb1a0 2024-03-08T23:59:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.294+0000 7fdb22666740 1 -- 172.21.15.22:0/3190879527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 msgr2=0x5567743d3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.294+0000 7fdb22666740 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743d3e20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009780 tx=0x7fdb1403d000 comp rx=0 tx=0).stop 2024-03-08T23:59:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.294+0000 7fdb22666740 1 -- 172.21.15.22:0/3190879527 shutdown_connections 2024-03-08T23:59:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.294+0000 7fdb22666740 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x5567743ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.294+0000 7fdb22666740 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cbac0 0x5567743d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.294+0000 7fdb22666740 1 --2- 172.21.15.22:0/3190879527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cb1a0 0x5567743d3e20 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.294+0000 7fdb22666740 1 -- 172.21.15.22:0/3190879527 >> 172.21.15.22:0/3190879527 conn(0x5567743c58d0 msgr2=0x5567743cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.295+0000 7fdb22666740 1 -- 172.21.15.22:0/3190879527 shutdown_connections 2024-03-08T23:59:49.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.295+0000 7fdb22666740 1 -- 172.21.15.22:0/3190879527 wait complete. 2024-03-08T23:59:49.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.296+0000 7fdb22666740 1 -- start start 2024-03-08T23:59:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.296+0000 7fdb22666740 1 Processor -- start 2024-03-08T23:59:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb22666740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb22666740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb22666740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x55677443aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb22666740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181c41d0 con 0x5567743cbac0 2024-03-08T23:59:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb22666740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181c54d0 con 0x5567743d46a0 2024-03-08T23:59:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb22666740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181c4810 con 0x5567743cb1a0 2024-03-08T23:59:49.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb22666740 10 client.0 _ll_register_callbacks cb 0x7fdb10009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb1d81d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb1d81d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743da410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60148/0 (socket says 172.21.15.22:60148) 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb1d81d640 1 -- 172.21.15.22:0/133034342 learned_addr learned my addr 172.21.15.22:0/133034342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb1d81d640 1 -- 172.21.15.22:0/133034342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 msgr2=0x55677443aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x55677443aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.297+0000 7fdb1d81d640 1 -- 172.21.15.22:0/133034342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 msgr2=0x5567743da950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.298+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.298+0000 7fdb1d81d640 1 -- 172.21.15.22:0/133034342 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567743db1e0 con 0x5567743cb1a0 2024-03-08T23:59:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.298+0000 7fdb1d81d640 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743da410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb1402c100 tx=0x7fdb14034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6ef8b86d5cb8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.298+0000 7fdb017fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:59:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.298+0000 7fdb017fa640 1 -- 172.21.15.22:0/133034342 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14045070 con 0x5567743cb1a0 2024-03-08T23:59:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.298+0000 7fdb017fa640 1 -- 172.21.15.22:0/133034342 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fdb14034630 con 0x5567743cb1a0 2024-03-08T23:59:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.299+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb14005040 con 0x5567743cb1a0 2024-03-08T23:59:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.299+0000 7fdb017fa640 1 -- 172.21.15.22:0/133034342 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb14052040 con 0x5567743cb1a0 2024-03-08T23:59:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.300+0000 7fdb22666740 10 client.26788 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.300+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55677443c090 con 0x5567743cb1a0 2024-03-08T23:59:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.300+0000 7fdb017fa640 1 -- 172.21.15.22:0/133034342 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb1402c510 con 0x5567743cb1a0 2024-03-08T23:59:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.300+0000 7fdb017fa640 1 -- 172.21.15.22:0/133034342 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdb14004840 con 0x5567743cb1a0 2024-03-08T23:59:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.301+0000 7fdb22666740 10 client.26788 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.301+0000 7fdb22666740 10 client.26788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-08T23:59:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.301+0000 7fdb22666740 20 client.26788 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.301+0000 7fdb22666740 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55677443c740 0x55677443eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.301+0000 7fdb22666740 4 client.26788 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:59:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.301+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fdb14005040 con 0x55677443c740 2024-03-08T23:59:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.301+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55677443c740 0x55677443eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.303+0000 7fdb1d01c640 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55677443c740 0x55677443eb80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3d4af0ed1120cfd server_cookie=335c0b9e728e2f4c in_seq=0 out_seq=0 2024-03-08T23:59:49.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.303+0000 7fdb017fa640 10 client.26788 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:49.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.303+0000 7fdb017fa640 1 -- 172.21.15.22:0/133034342 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fdb14005040 con 0x55677443c740 2024-03-08T23:59:49.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.303+0000 7fdb017fa640 10 client.26788 handle_command_reply: tid=0 2024-03-08T23:59:49.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.303+0000 7fdb22666740 1 client.26788 shutdown 2024-03-08T23:59:49.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 msgr2=0x5567743da410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743da410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb1402c100 tx=0x7fdb14034440 comp rx=0 tx=0).stop 2024-03-08T23:59:49.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 shutdown_connections 2024-03-08T23:59:49.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55677443c740 0x55677443eb80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567743d46a0 0x55677443aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5567743cbac0 0x5567743da950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 --2- 172.21.15.22:0/133034342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5567743cb1a0 0x5567743da410 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb1402c100 tx=0x7fdb14034440 comp rx=0 tx=0).stop 2024-03-08T23:59:49.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 >> 172.21.15.22:0/133034342 conn(0x5567743c58d0 msgr2=0x5567743c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.304+0000 7fdb017fa640 0 client.26788 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-08T23:59:49.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.305+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 shutdown_connections 2024-03-08T23:59:49.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.306+0000 7fdb22666740 1 -- 172.21.15.22:0/133034342 wait complete. 2024-03-08T23:59:49.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.307+0000 7fdb22666740 20 client.26788 trim_cache size 0 max 16384 2024-03-08T23:59:49.303 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb040a2910 msgr2=0x7fdb040a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb040a2910 0x7fdb040a4dd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb14000c00 tx=0x7fdb1402c090 comp rx=0 tx=0).stop 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fe80 msgr2=0x7fdb1811a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fe80 0x7fdb1811a3e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800d740 tx=0x7fdb0800dc00 comp rx=0 tx=0).stop 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 shutdown_connections 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb040a2910 0x7fdb040a4dd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb1811aad0 0x7fdb18113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb1810fe80 0x7fdb1811a3e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 --2- 172.21.15.22:0/1959439379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb1810fa10 0x7fdb18119ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 >> 172.21.15.22:0/1959439379 conn(0x7fdb1806c3e0 msgr2=0x7fdb1811b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 shutdown_connections 2024-03-08T23:59:49.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.309+0000 7fdb1fa81640 1 -- 172.21.15.22:0/1959439379 wait complete. 2024-03-08T23:59:49.322 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:59:49.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-08T23:59:49.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:48 smithi062 ceph-mon[36129]: pgmap v164: 129 pgs: 129 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 193 op/s 2024-03-08T23:59:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.438+0000 7f19bb285640 1 -- 172.21.15.22:0/120468702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 msgr2=0x7f19b410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.438+0000 7f19bb285640 1 --2- 172.21.15.22:0/120468702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b410e1d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009cd0 tx=0x7f19ac02c140 comp rx=0 tx=0).stop 2024-03-08T23:59:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.438+0000 7f19bb285640 1 -- 172.21.15.22:0/120468702 shutdown_connections 2024-03-08T23:59:49.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.438+0000 7f19bb285640 1 --2- 172.21.15.22:0/120468702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b410e1d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.438+0000 7f19bb285640 1 --2- 172.21.15.22:0/120468702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b4100410 0x7f19b4105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.438+0000 7f19bb285640 1 --2- 172.21.15.22:0/120468702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19b40ffa60 0x7f19b40ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.438+0000 7f19bb285640 1 -- 172.21.15.22:0/120468702 >> 172.21.15.22:0/120468702 conn(0x7f19b4076720 msgr2=0x7f19b40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 -- 172.21.15.22:0/120468702 shutdown_connections 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 -- 172.21.15.22:0/120468702 wait complete. 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 Processor -- start 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 -- start start 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b40ffa60 0x7f19b419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19b4197950 0x7f19b4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b4111670 con 0x7f19b4106670 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41114f0 con 0x7f19b40ffa60 2024-03-08T23:59:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19bb285640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41117f0 con 0x7f19b4197950 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19b8820640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19b8820640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b419dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54916/0 (socket says 172.21.15.22:54916) 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19b8820640 1 -- 172.21.15.22:0/1740863931 learned_addr learned my addr 172.21.15.22:0/1740863931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.439+0000 7f19b9822640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19b4197950 0x7f19b4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19b9021640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b40ffa60 0x7f19b419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19b8820640 1 -- 172.21.15.22:0/1740863931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19b4197950 msgr2=0x7f19b4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19b8820640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19b4197950 0x7f19b4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19b8820640 1 -- 172.21.15.22:0/1740863931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b40ffa60 msgr2=0x7f19b419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19b8820640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b40ffa60 0x7f19b419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19b8820640 1 -- 172.21.15.22:0/1740863931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b4198470 con 0x7f19b4106670 2024-03-08T23:59:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19b8820640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b419dce0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e9f0 tx=0x7f19a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19aa7fc640 1 -- 172.21.15.22:0/1740863931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a400cd80 con 0x7f19b4106670 2024-03-08T23:59:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19aa7fc640 1 -- 172.21.15.22:0/1740863931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19a4010910 con 0x7f19b4106670 2024-03-08T23:59:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b4198760 con 0x7f19b4106670 2024-03-08T23:59:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19aa7fc640 1 -- 172.21.15.22:0/1740863931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4018990 con 0x7f19b4106670 2024-03-08T23:59:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.440+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b41a49b0 con 0x7f19b4106670 2024-03-08T23:59:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.442+0000 7f19aa7fc640 1 -- 172.21.15.22:0/1740863931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f19a4010ac0 con 0x7f19b4106670 2024-03-08T23:59:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 Processor -- start 2024-03-08T23:59:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 -- start start 2024-03-08T23:59:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d49160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19aa7fc640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19a00a2970 0x7f19a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25d4ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e25d07360 con 0x556e25d401a0 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e25ca3ec0 con 0x556e25d496a0 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19bde6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e25c4cd30 con 0x556e25d40ac0 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19b9822640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19b9021640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19a00a2970 0x7f19a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19b9822640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d40580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54928/0 (socket says 172.21.15.22:54928) 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19b9822640 1 -- 172.21.15.22:0/3615759136 learned_addr learned my addr 172.21.15.22:0/3615759136 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19b8820640 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d49160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19aa7fc640 1 -- 172.21.15.22:0/1740863931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f19a40cf560 con 0x7f19b4106670 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.443+0000 7f19b9822640 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25d4ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f19b8820640 1 -- 172.21.15.22:0/3615759136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 msgr2=0x556e25d4ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f19b8820640 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25d4ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f19b8820640 1 -- 172.21.15.22:0/3615759136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 msgr2=0x556e25d40580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f19b8820640 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d40580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f19b8820640 1 -- 172.21.15.22:0/3615759136 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e25c4bbc0 con 0x556e25d40ac0 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f19b9021640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19a00a2970 0x7f19a00a4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19b4198a90 tx=0x7f19b0005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f19b8820640 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d49160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f19a40cc910 tx=0x7f19a409b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f6f174c0f0a5a84 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f199f7fe640 1 -- 172.21.15.22:0/3615759136 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a409f970 con 0x556e25d40ac0 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f199f7fe640 1 -- 172.21.15.22:0/3615759136 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19a40a3fc0 con 0x556e25d40ac0 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.444+0000 7f199f7fe640 1 -- 172.21.15.22:0/3615759136 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40cf160 con 0x556e25d40ac0 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 -- 172.21.15.22:0/3615759136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 msgr2=0x556e25d49160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d49160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f19a40cc910 tx=0x7f19a409b760 comp rx=0 tx=0).stop 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 -- 172.21.15.22:0/3615759136 shutdown_connections 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25d4ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d49160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 --2- 172.21.15.22:0/3615759136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d40580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 -- 172.21.15.22:0/3615759136 >> 172.21.15.22:0/3615759136 conn(0x556e25d3a8d0 msgr2=0x556e25d3acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 -- 172.21.15.22:0/3615759136 shutdown_connections 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 -- 172.21.15.22:0/3615759136 wait complete. 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 -- start start 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.445+0000 7f19bde6f740 1 Processor -- start 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d46650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d46b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19bde6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e25c424a0 con 0x556e25d40ac0 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19bde6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e25c9ff60 con 0x556e25d401a0 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19bde6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e25d4c3d0 con 0x556e25d496a0 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19bde6f740 10 client.0 _ll_register_callbacks cb 0x556e25c9dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d46b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b9021640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d46650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d46b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54940/0 (socket says 172.21.15.22:54940) 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 -- 172.21.15.22:0/1867561519 learned_addr learned my addr 172.21.15.22:0/1867561519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b9822640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 -- 172.21.15.22:0/1867561519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 msgr2=0x556e25d470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 -- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 msgr2=0x556e25d46650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d46650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 -- 172.21.15.22:0/1867561519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e259b83f0 con 0x556e25d40ac0 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b9822640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b9021640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d46650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:49.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f19b8820640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d46b90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e9c0 tx=0x7f19a409b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e05318df5e35a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f199cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.446+0000 7f199cff9640 1 -- 172.21.15.22:0/1867561519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40862c0 con 0x556e25d40ac0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f199cff9640 1 -- 172.21.15.22:0/1867561519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f19a40cf160 con 0x556e25d40ac0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a40a3fc0 con 0x556e25d40ac0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f199cff9640 1 -- 172.21.15.22:0/1867561519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a405a070 con 0x556e25d40ac0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 10 client.17355 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556e25a827d0 con 0x556e25d40ac0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f199cff9640 1 -- 172.21.15.22:0/1867561519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19a40a3fc0 con 0x556e25d40ac0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f199cff9640 1 -- 172.21.15.22:0/1867561519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f19a4018b30 con 0x556e25d40ac0 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 10 client.17355 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 10 client.17355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 20 client.17355 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25d595f0 0x556e25d5b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 4 client.17355 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e25cb1400 con 0x556e25d595f0 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.447+0000 7f19b9021640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25d595f0 0x556e25d5b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.448+0000 7f19b9021640 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25d595f0 0x556e25d5b9e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3c2d9b48bbe0770 server_cookie=a692a7713215f1b7 in_seq=0 out_seq=0 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.448+0000 7f199cff9640 10 client.17355 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f199cff9640 1 -- 172.21.15.22:0/1867561519 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e25cb1400 con 0x556e25d595f0 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f199cff9640 10 client.17355 handle_command_reply: tid=0 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 client.17355 shutdown 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 msgr2=0x556e25d46b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d46b90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e9c0 tx=0x7f19a409b7d0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 shutdown_connections 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25d595f0 0x556e25d5b9e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d46b90 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1867561519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d46650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f199cff9640 0 client.17355 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 >> 172.21.15.22:0/1867561519 conn(0x556e25d3a8d0 msgr2=0x556e25d3c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 shutdown_connections 2024-03-08T23:59:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 1 -- 172.21.15.22:0/1867561519 wait complete. 2024-03-08T23:59:49.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.449+0000 7f19bde6f740 20 client.17355 trim_cache size 0 max 16384 2024-03-08T23:59:49.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 Processor -- start 2024-03-08T23:59:49.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 -- start start 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d48d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d41a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d60030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41b0b50 con 0x556e25d40ac0 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41ae6f0 con 0x556e25d401a0 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19bde6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41ac790 con 0x556e25d496a0 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19b8820640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d41a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19b8820640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d41a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54954/0 (socket says 172.21.15.22:54954) 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.461+0000 7f19b8820640 1 -- 172.21.15.22:0/1609343009 learned_addr learned my addr 172.21.15.22:0/1609343009 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b9021640 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d48d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b9822640 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d60030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b8820640 1 -- 172.21.15.22:0/1609343009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 msgr2=0x556e25d60030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b8820640 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d60030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b8820640 1 -- 172.21.15.22:0/1609343009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 msgr2=0x556e25d48d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b8820640 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d48d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b8820640 1 -- 172.21.15.22:0/1609343009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e25d4c370 con 0x556e25d40ac0 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f19b8820640 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d41a80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f19a40ccf10 tx=0x7f19a4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfaf8ac43cdeec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f199effd640 1 -- 172.21.15.22:0/1609343009 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4097b70 con 0x556e25d40ac0 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f199effd640 1 -- 172.21.15.22:0/1609343009 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19a40862c0 con 0x556e25d40ac0 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.462+0000 7f199effd640 1 -- 172.21.15.22:0/1609343009 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40cf160 con 0x556e25d40ac0 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- 172.21.15.22:0/1609343009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 msgr2=0x556e25d41a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d41a80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f19a40ccf10 tx=0x7f19a4005e30 comp rx=0 tx=0).stop 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- 172.21.15.22:0/1609343009 shutdown_connections 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d496a0 0x556e25d60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d40ac0 0x556e25d41a80 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 --2- 172.21.15.22:0/1609343009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d401a0 0x556e25d48d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- 172.21.15.22:0/1609343009 >> 172.21.15.22:0/1609343009 conn(0x556e25d3a8d0 msgr2=0x556e25d3acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- 172.21.15.22:0/1609343009 shutdown_connections 2024-03-08T23:59:49.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- 172.21.15.22:0/1609343009 wait complete. 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- start start 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 Processor -- start 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d4e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d4ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25dae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41acd90 con 0x556e25d401a0 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41ae090 con 0x556e25d496a0 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41ad3d0 con 0x556e25d40ac0 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.463+0000 7f19bde6f740 10 client.0 _ll_register_callbacks cb 0x7f19a4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d4e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d4e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54962/0 (socket says 172.21.15.22:54962) 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 -- 172.21.15.22:0/2800613592 learned_addr learned my addr 172.21.15.22:0/2800613592 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9822640 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25dae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b8820640 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d4ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 -- 172.21.15.22:0/2800613592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 msgr2=0x556e25d4ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d4ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 -- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 msgr2=0x556e25dae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25dae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 -- 172.21.15.22:0/2800613592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e25daecf0 con 0x556e25d401a0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19b9021640 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d4e7e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f19b00041c0 tx=0x7f19b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43dc3f31523b97f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f199cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f199cff9640 1 -- 172.21.15.22:0/2800613592 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0026130 con 0x556e25d401a0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f199cff9640 1 -- 172.21.15.22:0/2800613592 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f19b00262d0 con 0x556e25d401a0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a40862c0 con 0x556e25d401a0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f199cff9640 1 -- 172.21.15.22:0/2800613592 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b0037050 con 0x556e25d401a0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.464+0000 7f19bde6f740 10 client.17361 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x556e25dafff0 con 0x556e25d401a0 2024-03-08T23:59:49.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f199cff9640 1 -- 172.21.15.22:0/2800613592 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19a40862c0 con 0x556e25d401a0 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f199cff9640 1 -- 172.21.15.22:0/2800613592 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f19b002fbf0 con 0x556e25d401a0 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19bde6f740 10 client.17361 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19bde6f740 10 client.17361 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19bde6f740 20 client.17361 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19bde6f740 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25db0720 0x556e25db2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19bde6f740 4 client.17361 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556e25cab750 con 0x556e25db0720 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19b8820640 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25db0720 0x556e25db2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f199cff9640 10 client.17361 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.465+0000 7f19b8820640 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25db0720 0x556e25db2ba0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68765c63c37f52a8 server_cookie=cd52925a1ac5bf1d in_seq=0 out_seq=0 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f199cff9640 1 -- 172.21.15.22:0/2800613592 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556e25cab750 con 0x556e25db0720 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f199cff9640 10 client.17361 handle_command_reply: tid=0 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 client.17361 shutdown 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 msgr2=0x556e25d4e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d4e7e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f19b00041c0 tx=0x7f19b0014000 comp rx=0 tx=0).stop 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 shutdown_connections 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556e25db0720 0x556e25db2ba0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e25d496a0 0x556e25dae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e25d40ac0 0x556e25d4ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f199cff9640 0 client.17361 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 --2- 172.21.15.22:0/2800613592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e25d401a0 0x556e25d4e7e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 >> 172.21.15.22:0/2800613592 conn(0x556e25d3a8d0 msgr2=0x556e25d41a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 shutdown_connections 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 1 -- 172.21.15.22:0/2800613592 wait complete. 2024-03-08T23:59:49.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bde6f740 20 client.17361 trim_cache size 0 max 16384 2024-03-08T23:59:49.470 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:59:49.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19a00a2970 msgr2=0x7f19a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bb285640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19a00a2970 0x7f19a00a4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19b4198a90 tx=0x7f19b0005f90 comp rx=0 tx=0).stop 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 msgr2=0x7f19b419dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bb285640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b419dce0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e9f0 tx=0x7f19a400eeb0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 shutdown_connections 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.466+0000 7f19bb285640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19a00a2970 0x7f19a00a4e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.467+0000 7f19bb285640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19b4197950 0x7f19b4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.467+0000 7f19bb285640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b4106670 0x7f19b419dce0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.467+0000 7f19bb285640 1 --2- 172.21.15.22:0/1740863931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19b40ffa60 0x7f19b419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.467+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 >> 172.21.15.22:0/1740863931 conn(0x7f19b4076720 msgr2=0x7f19b40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.467+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 shutdown_connections 2024-03-08T23:59:49.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.467+0000 7f19bb285640 1 -- 172.21.15.22:0/1740863931 wait complete. 2024-03-08T23:59:49.472 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T23:59:49.473 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-08T23:59:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 -- 172.21.15.22:0/888232324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 msgr2=0x7fdfac10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 --2- 172.21.15.22:0/888232324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac10e130 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009cd0 tx=0x7fdfa402c0f0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 -- 172.21.15.22:0/888232324 shutdown_connections 2024-03-08T23:59:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 --2- 172.21.15.22:0/888232324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac10e130 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 --2- 172.21.15.22:0/888232324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac069920 0x7fdfac069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 --2- 172.21.15.22:0/888232324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1073b0 0x7fdfac107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 -- 172.21.15.22:0/888232324 >> 172.21.15.22:0/888232324 conn(0x7fdfac076f90 msgr2=0x7fdfac0773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 -- 172.21.15.22:0/888232324 shutdown_connections 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.582+0000 7fdfb179e640 1 -- 172.21.15.22:0/888232324 wait complete. 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 Processor -- start 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 -- start start 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac069920 0x7fdfac19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1073b0 0x7fdfac197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac111620 con 0x7fdfac06a2e0 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1114a0 con 0x7fdfac1073b0 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfb179e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1117a0 con 0x7fdfac069920 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfa27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfaaffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac069920 0x7fdfac19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfaaffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac069920 0x7fdfac19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60218/0 (socket says 172.21.15.22:60218) 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfaaffd640 1 -- 172.21.15.22:0/1153615251 learned_addr learned my addr 172.21.15.22:0/1153615251 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfa27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac19dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54988/0 (socket says 172.21.15.22:54988) 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1073b0 0x7fdfac197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfa27fc640 1 -- 172.21.15.22:0/1153615251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac069920 msgr2=0x7fdfac19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac069920 0x7fdfac19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfa27fc640 1 -- 172.21.15.22:0/1153615251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1073b0 msgr2=0x7fdfac197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1073b0 0x7fdfac197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.583+0000 7fdfa27fc640 1 -- 172.21.15.22:0/1153615251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac198000 con 0x7fdfac06a2e0 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.584+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac19dc10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdf94005fd0 tx=0x7fdf9400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.584+0000 7fdfa8ff9640 1 -- 172.21.15.22:0/1153615251 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94009c30 con 0x7fdfac06a2e0 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.584+0000 7fdfa8ff9640 1 -- 172.21.15.22:0/1153615251 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf94009dd0 con 0x7fdfac06a2e0 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.584+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac1982f0 con 0x7fdfac06a2e0 2024-03-08T23:59:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.584+0000 7fdfa8ff9640 1 -- 172.21.15.22:0/1153615251 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94010460 con 0x7fdfac06a2e0 2024-03-08T23:59:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.584+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac1a4590 con 0x7fdfac06a2e0 2024-03-08T23:59:49.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.586+0000 7fdfa8ff9640 1 -- 172.21.15.22:0/1153615251 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdf94003f50 con 0x7fdfac06a2e0 2024-03-08T23:59:49.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 Processor -- start 2024-03-08T23:59:49.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 -- start start 2024-03-08T23:59:49.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcbd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcc66a0 0x558abbcc8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558abbc84360 con 0x558abbcbdac0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558abbc20ec0 con 0x558abbcbd1a0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfb4380740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558abbbc9d30 con 0x558abbcc66a0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfa8ff9640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfab7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcbd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfa27fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcc66a0 0x558abbcc8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfaaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfab7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcbd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41818/0 (socket says 172.21.15.22:41818) 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfab7fe640 1 -- 172.21.15.22:0/2580648885 learned_addr learned my addr 172.21.15.22:0/2580648885 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.587+0000 7fdfa8ff9640 1 -- 172.21.15.22:0/1153615251 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fdf940cedc0 con 0x7fdfac06a2e0 2024-03-08T23:59:49.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2580648885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 msgr2=0x558abbcbd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcbd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2580648885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 msgr2=0x558abbcc6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa27fc640 1 -- 172.21.15.22:0/2580648885 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558abbbc8bc0 con 0x558abbcc66a0 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcc66a0 0x558abbcc8a90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf940cc170 tx=0x7fdf9406de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f86203c0f444a9b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009a60 tx=0x7fdfa4005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa17fa640 1 -- 172.21.15.22:0/2580648885 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94020070 con 0x558abbcc66a0 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa17fa640 1 -- 172.21.15.22:0/2580648885 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf940a3820 con 0x558abbcc66a0 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.588+0000 7fdfa17fa640 1 -- 172.21.15.22:0/2580648885 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9409b5f0 con 0x558abbcc66a0 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 -- 172.21.15.22:0/2580648885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcc66a0 msgr2=0x558abbcc8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcc66a0 0x558abbcc8a90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf940cc170 tx=0x7fdf9406de10 comp rx=0 tx=0).stop 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 -- 172.21.15.22:0/2580648885 shutdown_connections 2024-03-08T23:59:49.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcc66a0 0x558abbcc8a90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 --2- 172.21.15.22:0/2580648885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcbd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 -- 172.21.15.22:0/2580648885 >> 172.21.15.22:0/2580648885 conn(0x558abbcb78d0 msgr2=0x558abbcb7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 -- 172.21.15.22:0/2580648885 shutdown_connections 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 -- 172.21.15.22:0/2580648885 wait complete. 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 -- start start 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 Processor -- start 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.589+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcc36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcc66a0 0x558abbcc4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558abbbbf4a0 con 0x558abbcbdac0 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558abbc1cf60 con 0x558abbcc66a0 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558abbcc9490 con 0x558abbcbd1a0 2024-03-08T23:59:49.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 10 client.0 _ll_register_callbacks cb 0x558abbc1ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55016/0 (socket says 172.21.15.22:55016) 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfab7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcc66a0 0x558abbcc4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcc36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4103883572 learned_addr learned my addr 172.21.15.22:0/4103883572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 msgr2=0x558abbcc36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcc36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4103883572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcc66a0 msgr2=0x558abbcc4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcc66a0 0x558abbcc4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4103883572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558abb9353f0 con 0x558abbcbdac0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcc66a0 0x558abbcc4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc3c30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdf940cc220 tx=0x7fdf94097830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65da21f610a67fb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdf927fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdf927fc640 1 -- 172.21.15.22:0/4103883572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94020070 con 0x558abbcbdac0 2024-03-08T23:59:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdf927fc640 1 -- 172.21.15.22:0/4103883572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fdf940a3390 con 0x558abbcbdac0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf940a3820 con 0x558abbcbdac0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdf927fc640 1 -- 172.21.15.22:0/4103883572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf940a3590 con 0x558abbcbdac0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 10 client.17373 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.590+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558abb9ff7d0 con 0x558abbcbdac0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdf927fc640 1 -- 172.21.15.22:0/4103883572 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf940a3820 con 0x558abbcbdac0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdf927fc640 1 -- 172.21.15.22:0/4103883572 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf94099ff0 con 0x558abbcbdac0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfb4380740 10 client.17373 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfb4380740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfb4380740 20 client.17373 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfb4380740 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbcd8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfb4380740 4 client.17373 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558abbc2e400 con 0x558abbcd6750 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbcd8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbcd8b40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f93e2ffce04707 server_cookie=42c1ec93081ee691 in_seq=0 out_seq=0 2024-03-08T23:59:49.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.591+0000 7fdf927fc640 10 client.17373 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdf927fc640 1 -- 172.21.15.22:0/4103883572 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558abbc2e400 con 0x558abbcd6750 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdf927fc640 10 client.17373 handle_command_reply: tid=0 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 client.17373 shutdown 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 msgr2=0x558abbcc3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc3c30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdf940cc220 tx=0x7fdf94097830 comp rx=0 tx=0).stop 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 shutdown_connections 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbcd8b40 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcc66a0 0x558abbcc4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcbdac0 0x558abbcc3c30 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdf927fc640 0 client.17373 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 --2- 172.21.15.22:0/4103883572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcc36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 >> 172.21.15.22:0/4103883572 conn(0x558abbcb78d0 msgr2=0x558abbcb9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 shutdown_connections 2024-03-08T23:59:49.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 1 -- 172.21.15.22:0/4103883572 wait complete. 2024-03-08T23:59:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.592+0000 7fdfb4380740 20 client.17373 trim_cache size 0 max 16384 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 Processor -- start 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 -- start start 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcc5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbdac0 0x558abbcdab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcdb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1b0970 con 0x558abbcc66a0 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1ae510 con 0x558abbcbd1a0 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfb4380740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1ac5b0 con 0x558abbcbdac0 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcdb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcdb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55024/0 (socket says 172.21.15.22:55024) 2024-03-08T23:59:49.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3542365659 learned_addr learned my addr 172.21.15.22:0/3542365659 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbdac0 0x558abbcdab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.605+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcc5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3542365659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbdac0 msgr2=0x558abbcdab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbdac0 0x558abbcdab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3542365659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 msgr2=0x558abbcc5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcc5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3542365659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558abbcc1c40 con 0x558abbcc66a0 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcdb090 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdfa402c700 tx=0x7fdfa402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c37a23e801d3cf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfa0ff9640 1 -- 172.21.15.22:0/3542365659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4039040 con 0x558abbcc66a0 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfa0ff9640 1 -- 172.21.15.22:0/3542365659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa4055070 con 0x558abbcc66a0 2024-03-08T23:59:49.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.606+0000 7fdfa0ff9640 1 -- 172.21.15.22:0/3542365659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40508f0 con 0x558abbcc66a0 2024-03-08T23:59:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 -- 172.21.15.22:0/3542365659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 msgr2=0x558abbcdb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcdb090 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdfa402c700 tx=0x7fdfa402e040 comp rx=0 tx=0).stop 2024-03-08T23:59:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 -- 172.21.15.22:0/3542365659 shutdown_connections 2024-03-08T23:59:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcdb090 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbdac0 0x558abbcdab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3542365659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbd1a0 0x558abbcc5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 -- 172.21.15.22:0/3542365659 >> 172.21.15.22:0/3542365659 conn(0x558abbcb78d0 msgr2=0x558abbcb7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 -- 172.21.15.22:0/3542365659 shutdown_connections 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 -- 172.21.15.22:0/3542365659 wait complete. 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 -- start start 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.607+0000 7fdfb4380740 1 Processor -- start 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcdad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbdac0 0x558abbcd9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcd96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1acbb0 con 0x558abbcc66a0 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1adeb0 con 0x558abbcbdac0 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1ad1f0 con 0x558abbcbd1a0 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 10 client.0 _ll_register_callbacks cb 0x7fdf940038f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcd96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcd96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55034/0 (socket says 172.21.15.22:55034) 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3939559277 learned_addr learned my addr 172.21.15.22:0/3939559277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:49.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcdad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbdac0 0x558abbcd9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 msgr2=0x558abbcdad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcdad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3939559277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbdac0 msgr2=0x558abbcd9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbdac0 0x558abbcd9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 -- 172.21.15.22:0/3939559277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558abbcd9e00 con 0x558abbcc66a0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfab7fe640 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcd96c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4005cc0 tx=0x7fdfa4050a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe2a54f921dd69d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdf927fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdf927fc640 1 -- 172.21.15.22:0/3939559277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4039070 con 0x558abbcc66a0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdf927fc640 1 -- 172.21.15.22:0/3939559277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fdfa4050c00 con 0x558abbcc66a0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfa4055070 con 0x558abbcc66a0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdf927fc640 1 -- 172.21.15.22:0/3939559277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfa4066050 con 0x558abbcc66a0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 10 client.17379 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.608+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558abbd2d760 con 0x558abbcc66a0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdf927fc640 1 -- 172.21.15.22:0/3939559277 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa4055070 con 0x558abbcc66a0 2024-03-08T23:59:49.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdf927fc640 1 -- 172.21.15.22:0/3939559277 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdfa4059450 con 0x558abbcc66a0 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfb4380740 10 client.17379 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfb4380740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfb4380740 20 client.17379 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbd2fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfb4380740 4 client.17379 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558abbc28750 con 0x558abbcd6750 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbd2fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdf927fc640 10 client.17379 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.609+0000 7fdfaaffd640 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbd2fe30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d777727ae81f760f server_cookie=9249e31e9896a3ea in_seq=0 out_seq=0 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdf927fc640 1 -- 172.21.15.22:0/3939559277 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558abbc28750 con 0x558abbcd6750 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdf927fc640 10 client.17379 handle_command_reply: tid=0 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 client.17379 shutdown 2024-03-08T23:59:49.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 msgr2=0x558abbcd96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcd96c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4005cc0 tx=0x7fdfa4050a90 comp rx=0 tx=0).stop 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 shutdown_connections 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558abbcd6750 0x558abbd2fe30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558abbcc66a0 0x558abbcd96c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558abbcbdac0 0x558abbcd9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 --2- 172.21.15.22:0/3939559277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558abbcbd1a0 0x558abbcdad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdf927fc640 0 client.17379 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 >> 172.21.15.22:0/3939559277 conn(0x558abbcb78d0 msgr2=0x558abbcb8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 shutdown_connections 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 1 -- 172.21.15.22:0/3939559277 wait complete. 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.611+0000 7fdfb4380740 20 client.17379 trim_cache size 0 max 16384 2024-03-08T23:59:49.611 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-08T23:59:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 msgr2=0x7fdf980a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009a60 tx=0x7fdfa4005c50 comp rx=0 tx=0).stop 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 msgr2=0x7fdfac19dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac19dc10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdf94005fd0 tx=0x7fdf9400dde0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 shutdown_connections 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1073b0 0x7fdfac197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac06a2e0 0x7fdfac19dc10 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 --2- 172.21.15.22:0/1153615251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac069920 0x7fdfac19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 >> 172.21.15.22:0/1153615251 conn(0x7fdfac076f90 msgr2=0x7fdfac0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 shutdown_connections 2024-03-08T23:59:49.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:49.612+0000 7fdfb179e640 1 -- 172.21.15.22:0/1153615251 wait complete. 2024-03-08T23:59:49.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 536982 11.47 MB/sec execute 117 sec latency 26.751 ms 2024-03-08T23:59:50.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:49 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3387471655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:59:50.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:49 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3387471655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:59:50.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:49 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3387471655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T23:59:50.619 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-08T23:59:50.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T23:59:50.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-08T23:59:50.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 539038 11.44 MB/sec execute 118 sec latency 20.850 ms 2024-03-08T23:59:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 -- 172.21.15.22:0/2093428694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941003d0 msgr2=0x7f9f941007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/2093428694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941003d0 0x7f9f941007b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009900 tx=0x7f9f8402bc30 comp rx=0 tx=0).stop 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 -- 172.21.15.22:0/2093428694 shutdown_connections 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/2093428694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f941054a0 0x7f9f9410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/2093428694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f94100d80 0x7f9f94104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/2093428694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941003d0 0x7f9f941007b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 -- 172.21.15.22:0/2093428694 >> 172.21.15.22:0/2093428694 conn(0x7f9f940772f0 msgr2=0x7f9f94077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 -- 172.21.15.22:0/2093428694 shutdown_connections 2024-03-08T23:59:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.740+0000 7f9f98fc4640 1 -- 172.21.15.22:0/2093428694 wait complete. 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 Processor -- start 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 -- start start 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f941003d0 0x7f9f9419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941054a0 0x7f9f9419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f941979a0 0x7f9f94197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f94110530 con 0x7f9f941054a0 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941103b0 con 0x7f9f941003d0 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f98fc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941106b0 con 0x7f9f941979a0 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941054a0 0x7f9f9419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941054a0 0x7f9f9419dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59404/0 (socket says 172.21.15.22:59404) 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3056821940 learned_addr learned my addr 172.21.15.22:0/3056821940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f941979a0 0x7f9f94197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f9259c640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f941003d0 0x7f9f9419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3056821940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f941979a0 msgr2=0x7f9f94197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f941979a0 0x7f9f94197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3056821940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f941003d0 msgr2=0x7f9f9419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f941003d0 0x7f9f9419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3056821940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f941984c0 con 0x7f9f941054a0 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f9259c640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f941003d0 0x7f9f9419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.741+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f941979a0 0x7f9f94197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.742+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941054a0 0x7f9f9419dd30 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800e9c0 tx=0x7f9f8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.742+0000 7f9f837fe640 1 -- 172.21.15.22:0/3056821940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800cd80 con 0x7f9f941054a0 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.742+0000 7f9f837fe640 1 -- 172.21.15.22:0/3056821940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f88005500 con 0x7f9f941054a0 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.742+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f941987b0 con 0x7f9f941054a0 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.742+0000 7f9f837fe640 1 -- 172.21.15.22:0/3056821940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88010690 con 0x7f9f941054a0 2024-03-08T23:59:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.742+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f941a4a00 con 0x7f9f941054a0 2024-03-08T23:59:50.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.744+0000 7f9f837fe640 1 -- 172.21.15.22:0/3056821940 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9f880046c0 con 0x7f9f941054a0 2024-03-08T23:59:50.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 Processor -- start 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 -- start start 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f837fe640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f7c0a2970 0x7f9f7c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d249e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d257150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d257690 0x558e1d259a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1d1adb40 con 0x558e1d257690 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1d156d80 con 0x558e1d249a30 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1d1a9be0 con 0x558e1d24a350 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f7c0a2970 0x7f9f7c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f91d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d257150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f837fe640 1 -- 172.21.15.22:0/3056821940 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9f880cf620 con 0x7f9f941054a0 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d249e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f92d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d257690 0x558e1d259a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f91d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d257150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48708/0 (socket says 172.21.15.22:48708) 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f91d9b640 1 -- 172.21.15.22:0/2249322717 learned_addr learned my addr 172.21.15.22:0/2249322717 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.745+0000 7f9f9259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d249e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:47594/0 (socket says 172.21.15.22:47594) 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f91d9b640 1 -- 172.21.15.22:0/2249322717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 msgr2=0x558e1d249e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d249e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f91d9b640 1 -- 172.21.15.22:0/2249322717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d257690 msgr2=0x558e1d259a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d257690 0x558e1d259a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f91d9b640 1 -- 172.21.15.22:0/2249322717 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e1cf8a7e0 con 0x558e1d24a350 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d257690 0x558e1d259a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f7c0a2970 0x7f9f7c0a4e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009fd0 tx=0x7f9f8c0116b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d257150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9f880cca30 tx=0x7f9f8809b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2889eb2c4dcd53 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f80ff9640 1 -- 172.21.15.22:0/2249322717 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8807e710 con 0x558e1d24a350 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f80ff9640 1 -- 172.21.15.22:0/2249322717 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f8809bfe0 con 0x558e1d24a350 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.746+0000 7f9f80ff9640 1 -- 172.21.15.22:0/2249322717 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88097f50 con 0x558e1d24a350 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2249322717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 msgr2=0x558e1d257150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d257150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9f880cca30 tx=0x7f9f8809b420 comp rx=0 tx=0).stop 2024-03-08T23:59:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2249322717 shutdown_connections 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d257690 0x558e1d259a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d257150 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2249322717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d249e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2249322717 >> 172.21.15.22:0/2249322717 conn(0x558e1d2444b0 msgr2=0x558e1d244890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2249322717 shutdown_connections 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.747+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2249322717 wait complete. 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 -- start start 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 Processor -- start 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d257040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d2543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d257690 0x558e1d254970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1d25a4d0 con 0x558e1d249a30 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1d16a1c0 con 0x558e1d257690 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1d25a650 con 0x558e1d24a350 2024-03-08T23:59:50.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9bba4740 10 client.0 _ll_register_callbacks cb 0x558e1d1597a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d257040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d257040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59424/0 (socket says 172.21.15.22:59424) 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f9259c640 1 -- 172.21.15.22:0/2121142616 learned_addr learned my addr 172.21.15.22:0/2121142616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d2543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.748+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d257690 0x558e1d254970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f91d9b640 1 -- 172.21.15.22:0/2121142616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d257690 msgr2=0x558e1d254970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d257690 0x558e1d254970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f91d9b640 1 -- 172.21.15.22:0/2121142616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 msgr2=0x558e1d257040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d257040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f91d9b640 1 -- 172.21.15.22:0/2121142616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e1cfeaf70 con 0x558e1d24a350 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f9259c640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d257040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d2543d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f880cca30 tx=0x7f9f88099de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28db593a0e4c212 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-08T23:59:50.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f79ffb640 1 -- 172.21.15.22:0/2121142616 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8809b8e0 con 0x558e1d24a350 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f8809bfe0 con 0x558e1d24a350 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f79ffb640 1 -- 172.21.15.22:0/2121142616 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9f88086110 con 0x558e1d24a350 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.749+0000 7f9f79ffb640 1 -- 172.21.15.22:0/2121142616 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f8809fa30 con 0x558e1d24a350 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 10 client.26836 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558e1ce82170 con 0x558e1d24a350 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f79ffb640 1 -- 172.21.15.22:0/2121142616 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f8809bfe0 con 0x558e1d24a350 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f79ffb640 1 -- 172.21.15.22:0/2121142616 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9f880cf260 con 0x558e1d24a350 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 10 client.26836 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 10 client.26836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 20 client.26836 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d267640 0x558e1d269a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 4 client.26836 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e1d183960 con 0x558e1d267640 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.750+0000 7f9f9259c640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d267640 0x558e1d269a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.751+0000 7f9f9259c640 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d267640 0x558e1d269a30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c93cf46a5cd078e server_cookie=3113b558c57fac5e in_seq=0 out_seq=0 2024-03-08T23:59:50.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.751+0000 7f9f79ffb640 10 client.26836 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f79ffb640 1 -- 172.21.15.22:0/2121142616 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e1d183960 con 0x558e1d267640 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f79ffb640 10 client.26836 handle_command_reply: tid=0 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 client.26836 shutdown 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 msgr2=0x558e1d2543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d2543d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f880cca30 tx=0x7f9f88099de0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 shutdown_connections 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d267640 0x558e1d269a30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d257690 0x558e1d254970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f79ffb640 0 client.26836 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d24a350 0x558e1d2543d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/2121142616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d257040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.752+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 >> 172.21.15.22:0/2121142616 conn(0x558e1d2444b0 msgr2=0x558e1d246200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.753+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 shutdown_connections 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.753+0000 7f9f9bba4740 1 -- 172.21.15.22:0/2121142616 wait complete. 2024-03-08T23:59:50.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.753+0000 7f9f9bba4740 20 client.26836 trim_cache size 0 max 16384 2024-03-08T23:59:50.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 Processor -- start 2024-03-08T23:59:50.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 -- start start 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d262ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d24a350 0x558e1d256b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d257040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941ae890 con 0x558e1d24a350 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941ac430 con 0x558e1d249a30 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941aa4d0 con 0x558e1d257690 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f91d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d24a350 0x558e1d256b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f91d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d24a350 0x558e1d256b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59438/0 (socket says 172.21.15.22:59438) 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3210613451 learned_addr learned my addr 172.21.15.22:0/3210613451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.770+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d257040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f9259c640 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d262ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3210613451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 msgr2=0x558e1d257040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d257040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3210613451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 msgr2=0x558e1d262ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d262ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f91d9b640 1 -- 172.21.15.22:0/3210613451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e1d254ce0 con 0x558e1d24a350 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d24a350 0x558e1d256b00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9f8809b5e0 tx=0x7f9f88097c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0c2675e733edc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f7bfff640 1 -- 172.21.15.22:0/3210613451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8809b240 con 0x558e1d24a350 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f7bfff640 1 -- 172.21.15.22:0/3210613451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f880b4070 con 0x558e1d24a350 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.771+0000 7f9f7bfff640 1 -- 172.21.15.22:0/3210613451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8807a700 con 0x558e1d24a350 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 -- 172.21.15.22:0/3210613451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d24a350 msgr2=0x558e1d256b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d24a350 0x558e1d256b00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9f8809b5e0 tx=0x7f9f88097c90 comp rx=0 tx=0).stop 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 -- 172.21.15.22:0/3210613451 shutdown_connections 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d257040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d24a350 0x558e1d256b00 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/3210613451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d249a30 0x558e1d262ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 -- 172.21.15.22:0/3210613451 >> 172.21.15.22:0/3210613451 conn(0x558e1d2444b0 msgr2=0x558e1d2628e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 -- 172.21.15.22:0/3210613451 shutdown_connections 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 -- 172.21.15.22:0/3210613451 wait complete. 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 -- start start 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.772+0000 7f9f9bba4740 1 Processor -- start 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d26bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d24a350 0x558e1d269030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9bba4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d269570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941aaad0 con 0x558e1d249a30 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941abdd0 con 0x558e1d24a350 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9bba4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941ab110 con 0x558e1d257690 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9bba4740 10 client.0 _ll_register_callbacks cb 0x7f9f88009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d26bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d26bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59440/0 (socket says 172.21.15.22:59440) 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 -- 172.21.15.22:0/1615944281 learned_addr learned my addr 172.21.15.22:0/1615944281 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f92d9d640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d269570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d24a350 0x558e1d269030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 -- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 msgr2=0x558e1d269570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d269570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 -- 172.21.15.22:0/1615944281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d24a350 msgr2=0x558e1d269030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d24a350 0x558e1d269030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 -- 172.21.15.22:0/1615944281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e1d24a180 con 0x558e1d249a30 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d24a350 0x558e1d269030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f9259c640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d26bca0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9f84036040 tx=0x7f9f8402cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf42f9d78f3891d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.773+0000 7f9f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f79ffb640 1 -- 172.21.15.22:0/1615944281 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84049070 con 0x558e1d249a30 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f79ffb640 1 -- 172.21.15.22:0/1615944281 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f9f84004040 con 0x558e1d249a30 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f880b4070 con 0x558e1d249a30 2024-03-08T23:59:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f79ffb640 1 -- 172.21.15.22:0/1615944281 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f84054050 con 0x558e1d249a30 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 10 client.17397 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558e1d2bf4b0 con 0x558e1d249a30 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f79ffb640 1 -- 172.21.15.22:0/1615944281 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f880b4070 con 0x558e1d249a30 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f79ffb640 1 -- 172.21.15.22:0/1615944281 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9f84005310 con 0x558e1d249a30 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 10 client.17397 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 20 client.17397 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d2bfbe0 0x558e1d2c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 4 client.17397 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558e1d16f080 con 0x558e1d2bfbe0 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.774+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d2bfbe0 0x558e1d2c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.775+0000 7f9f79ffb640 10 client.17397 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.775+0000 7f9f91d9b640 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d2bfbe0 0x558e1d2c2000 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa6ca69f5cae849c server_cookie=dff865b9d9df0bfe in_seq=0 out_seq=0 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.776+0000 7f9f79ffb640 1 -- 172.21.15.22:0/1615944281 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558e1d16f080 con 0x558e1d2bfbe0 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.776+0000 7f9f79ffb640 10 client.17397 handle_command_reply: tid=0 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.776+0000 7f9f9bba4740 1 client.17397 shutdown 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 msgr2=0x558e1d26bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d26bca0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9f84036040 tx=0x7f9f8402cd30 comp rx=0 tx=0).stop 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 shutdown_connections 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558e1d2bfbe0 0x558e1d2c2000 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e1d257690 0x558e1d269570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f79ffb640 0 client.17397 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e1d24a350 0x558e1d269030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 --2- 172.21.15.22:0/1615944281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558e1d249a30 0x558e1d26bca0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 >> 172.21.15.22:0/1615944281 conn(0x558e1d2444b0 msgr2=0x558e1d246190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 shutdown_connections 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 1 -- 172.21.15.22:0/1615944281 wait complete. 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f9bba4740 20 client.17397 trim_cache size 0 max 16384 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "2bc728b6-87a5-40dd-8aaf-1501ebfb929f", 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-08T23:59:50.778 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:59:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f7c0a2970 msgr2=0x7f9f7c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f7c0a2970 0x7f9f7c0a4e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009fd0 tx=0x7f9f8c0116b0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941054a0 msgr2=0x7f9f9419dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941054a0 0x7f9f9419dd30 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800e9c0 tx=0x7f9f8800ee80 comp rx=0 tx=0).stop 2024-03-08T23:59:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 shutdown_connections 2024-03-08T23:59:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f7c0a2970 0x7f9f7c0a4e30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f941979a0 0x7f9f94197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f941054a0 0x7f9f9419dd30 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 --2- 172.21.15.22:0/3056821940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f941003d0 0x7f9f9419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.777+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 >> 172.21.15.22:0/3056821940 conn(0x7f9f940772f0 msgr2=0x7f9f94068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.778+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 shutdown_connections 2024-03-08T23:59:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.778+0000 7f9f98fc4640 1 -- 172.21.15.22:0/3056821940 wait complete. 2024-03-08T23:59:50.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-08T23:59:50.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 -- 172.21.15.22:0/1319810140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 msgr2=0x7f642410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 --2- 172.21.15.22:0/1319810140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 0x7f642410dd30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f641c009be0 tx=0x7f641c02bf80 comp rx=0 tx=0).stop 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 -- 172.21.15.22:0/1319810140 shutdown_connections 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 --2- 172.21.15.22:0/1319810140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 0x7f642410dd30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 --2- 172.21.15.22:0/1319810140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6424102e50 0x7f642410b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 --2- 172.21.15.22:0/1319810140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 0x7f6424102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 -- 172.21.15.22:0/1319810140 >> 172.21.15.22:0/1319810140 conn(0x7f6424076570 msgr2=0x7f6424076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 -- 172.21.15.22:0/1319810140 shutdown_connections 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 -- 172.21.15.22:0/1319810140 wait complete. 2024-03-08T23:59:50.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.909+0000 7f6429d16640 1 Processor -- start 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6429d16640 1 -- start start 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6429d16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 0x7f6424100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6429d16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6424102e50 0x7f64241006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6429d16640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 0x7f6424100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6429d16640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6424111670 con 0x7f6424102530 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6429d16640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241114f0 con 0x7f6424102e50 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6429d16640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241117f0 con 0x7f642410b900 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 0x7f6424100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 0x7f6424100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59452/0 (socket says 172.21.15.22:59452) 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 -- 172.21.15.22:0/3349561821 learned_addr learned my addr 172.21.15.22:0/3349561821 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6422ffd640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6424102e50 0x7f64241006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f6423fff640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 0x7f6424100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 -- 172.21.15.22:0/3349561821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 msgr2=0x7f6424100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 0x7f6424100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 -- 172.21.15.22:0/3349561821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6424102e50 msgr2=0x7f64241006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6424102e50 0x7f64241006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 -- 172.21.15.22:0/3349561821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64241a6070 con 0x7f6424102530 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.910+0000 7f64237fe640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 0x7f6424100180 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9c0 tx=0x7f641400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.911+0000 7f6420ff9640 1 -- 172.21.15.22:0/3349561821 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641400cde0 con 0x7f6424102530 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.911+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64241a6360 con 0x7f6424102530 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.911+0000 7f6420ff9640 1 -- 172.21.15.22:0/3349561821 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64140035f0 con 0x7f6424102530 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.911+0000 7f6420ff9640 1 -- 172.21.15.22:0/3349561821 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414010650 con 0x7f6424102530 2024-03-08T23:59:50.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.911+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a6900 con 0x7f6424102530 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.913+0000 7f6420ff9640 1 -- 172.21.15.22:0/3349561821 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f64140037a0 con 0x7f6424102530 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 Processor -- start 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 -- start start 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6a760 0x55eea2c6ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c76050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f6420ff9640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f640c0a2970 0x7f640c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea2bcde00 con 0x55eea2c73c60 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea2b76b10 con 0x55eea2c6b080 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f642c9d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea2bc9ea0 con 0x55eea2c6a760 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f6423fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6a760 0x55eea2c6ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f6420ff9640 1 -- 172.21.15.22:0/3349561821 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f64140cf570 con 0x7f6424102530 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f6423fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6a760 0x55eea2c6ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48746/0 (socket says 172.21.15.22:48746) 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f6423fff640 1 -- 172.21.15.22:0/3682282072 learned_addr learned my addr 172.21.15.22:0/3682282072 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f64237fe640 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.914+0000 7f6423fff640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f640c0a2970 0x7f640c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f64237fe640 1 -- 172.21.15.22:0/3682282072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6a760 msgr2=0x55eea2c6ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f64237fe640 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6a760 0x55eea2c6ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f64237fe640 1 -- 172.21.15.22:0/3682282072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 msgr2=0x55eea2c76050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T23:59:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f64237fe640 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c76050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f64237fe640 1 -- 172.21.15.22:0/3682282072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea29ac7d0 con 0x55eea2c6b080 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f6423fff640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f640c0a2970 0x7f640c0a4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f641c0099a0 tx=0x7f641c009280 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f64237fe640 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c73720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64140cc8c0 tx=0x7f641409b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9757784101d46c92 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f6409ffb640 1 -- 172.21.15.22:0/3682282072 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64140b4070 con 0x55eea2c6b080 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f6409ffb640 1 -- 172.21.15.22:0/3682282072 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641409b7d0 con 0x55eea2c6b080 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.915+0000 7f6409ffb640 1 -- 172.21.15.22:0/3682282072 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64140cf120 con 0x55eea2c6b080 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 -- 172.21.15.22:0/3682282072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 msgr2=0x55eea2c73720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c73720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64140cc8c0 tx=0x7f641409b660 comp rx=0 tx=0).stop 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 -- 172.21.15.22:0/3682282072 shutdown_connections 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c76050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c73720 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3682282072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6a760 0x55eea2c6ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 -- 172.21.15.22:0/3682282072 >> 172.21.15.22:0/3682282072 conn(0x55eea2c64e90 msgr2=0x55eea2c65270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 -- 172.21.15.22:0/3682282072 shutdown_connections 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 -- 172.21.15.22:0/3682282072 wait complete. 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.916+0000 7f642c9d0740 1 -- start start 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 1 Processor -- start 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c73380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6b080 0x55eea2c70710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c70c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea2c769f0 con 0x55eea2c6b080 2024-03-08T23:59:50.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea2b8a0a0 con 0x55eea2c6a760 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea2c76b70 con 0x55eea2c73c60 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f642c9d0740 10 client.0 _ll_register_callbacks cb 0x55eea2b89a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6b080 0x55eea2c70710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6b080 0x55eea2c70710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59460/0 (socket says 172.21.15.22:59460) 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 -- 172.21.15.22:0/1821238001 learned_addr learned my addr 172.21.15.22:0/1821238001 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6423fff640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c70c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f64237fe640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c73380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 -- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 msgr2=0x55eea2c70c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c70c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 -- 172.21.15.22:0/1821238001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 msgr2=0x55eea2c73380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c73380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6422ffd640 1 -- 172.21.15.22:0/1821238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eea28e23f0 con 0x55eea2c6b080 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.917+0000 7f6423fff640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c70c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f6422ffd640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6b080 0x55eea2c70710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f641800b8f0 tx=0x7f641800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760ce90e3aff8b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f6402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f6402ffd640 1 -- 172.21.15.22:0/1821238001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418005020 con 0x55eea2c6b080 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f6402ffd640 1 -- 172.21.15.22:0/1821238001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f64180051c0 con 0x55eea2c6b080 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f641409b7d0 con 0x55eea2c6b080 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f6402ffd640 1 -- 172.21.15.22:0/1821238001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6418022050 con 0x55eea2c6b080 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 10 client.17406 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55eea28a20f0 con 0x55eea2c6b080 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f6402ffd640 1 -- 172.21.15.22:0/1821238001 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641409b7d0 con 0x55eea2c6b080 2024-03-08T23:59:50.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f6402ffd640 1 -- 172.21.15.22:0/1821238001 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6418011440 con 0x55eea2c6b080 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 10 client.17406 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 20 client.17406 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2c836d0 0x55eea2c85ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 4 client.17406 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.918+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eea2bdb340 con 0x55eea2c836d0 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.919+0000 7f64237fe640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2c836d0 0x55eea2c85ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.919+0000 7f6402ffd640 10 client.17406 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.919+0000 7f64237fe640 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2c836d0 0x55eea2c85ac0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9453da95c9550e9 server_cookie=39c1283a896c4ef3 in_seq=0 out_seq=0 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f6402ffd640 1 -- 172.21.15.22:0/1821238001 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eea2bdb340 con 0x55eea2c836d0 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f6402ffd640 10 client.17406 handle_command_reply: tid=0 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 client.17406 shutdown 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6b080 msgr2=0x55eea2c70710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6b080 0x55eea2c70710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f641800b8f0 tx=0x7f641800bcf0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 shutdown_connections 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2c836d0 0x55eea2c85ac0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c70c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6b080 0x55eea2c70710 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f6402ffd640 0 client.17406 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1821238001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c73380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 >> 172.21.15.22:0/1821238001 conn(0x55eea2c64e90 msgr2=0x55eea2c66be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 shutdown_connections 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 1 -- 172.21.15.22:0/1821238001 wait complete. 2024-03-08T23:59:50.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.920+0000 7f642c9d0740 20 client.17406 trim_cache size 0 max 16384 2024-03-08T23:59:50.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 Processor -- start 2024-03-08T23:59:50.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 -- start start 2024-03-08T23:59:50.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c72e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6b080 0x55eea2c73380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241b2d40 con 0x55eea2c73c60 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241b08e0 con 0x55eea2c6a760 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f642c9d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241ae980 con 0x55eea2c6b080 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6423fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6422ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6b080 0x55eea2c73380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6423fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c6efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59470/0 (socket says 172.21.15.22:59470) 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6423fff640 1 -- 172.21.15.22:0/3141705412 learned_addr learned my addr 172.21.15.22:0/3141705412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f64237fe640 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c72e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6423fff640 1 -- 172.21.15.22:0/3141705412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6b080 msgr2=0x55eea2c73380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6423fff640 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6b080 0x55eea2c73380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6423fff640 1 -- 172.21.15.22:0/3141705412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 msgr2=0x55eea2c72e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.937+0000 7f6423fff640 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c72e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.938+0000 7f6423fff640 1 -- 172.21.15.22:0/3141705412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea2c6aeb0 con 0x55eea2c73c60 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.938+0000 7f6423fff640 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c6efe0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f642406a9a0 tx=0x7f641c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e84a951b4fe48a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.938+0000 7f64097fa640 1 -- 172.21.15.22:0/3141705412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c054070 con 0x55eea2c73c60 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.938+0000 7f64097fa640 1 -- 172.21.15.22:0/3141705412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641c02cdb0 con 0x55eea2c73c60 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.938+0000 7f64097fa640 1 -- 172.21.15.22:0/3141705412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c034690 con 0x55eea2c73c60 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 -- 172.21.15.22:0/3141705412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 msgr2=0x55eea2c6efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c6efe0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f642406a9a0 tx=0x7f641c005e30 comp rx=0 tx=0).stop 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 -- 172.21.15.22:0/3141705412 shutdown_connections 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c73c60 0x55eea2c6efe0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c6b080 0x55eea2c73380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 --2- 172.21.15.22:0/3141705412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6a760 0x55eea2c72e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 -- 172.21.15.22:0/3141705412 >> 172.21.15.22:0/3141705412 conn(0x55eea2c64e90 msgr2=0x55eea2c706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 -- 172.21.15.22:0/3141705412 shutdown_connections 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 -- 172.21.15.22:0/3141705412 wait complete. 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 -- start start 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.939+0000 7f642c9d0740 1 Processor -- start 2024-03-08T23:59:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6a760 0x55eea2c7ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241aef80 con 0x55eea2c6a760 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241b0280 con 0x55eea2c6b080 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241af5c0 con 0x55eea2c73c60 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 10 client.0 _ll_register_callbacks cb 0x7f6418004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6a760 0x55eea2c7ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6a760 0x55eea2c7ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59484/0 (socket says 172.21.15.22:59484) 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 -- 172.21.15.22:0/1738009294 learned_addr learned my addr 172.21.15.22:0/1738009294 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f6423fff640 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f6422ffd640 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 -- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 msgr2=0x55eea2c778e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c778e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 -- 172.21.15.22:0/1738009294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 msgr2=0x55eea2c773a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c773a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 -- 172.21.15.22:0/1738009294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eea2c77e20 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f64237fe640 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6a760 0x55eea2c7ae30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6414097c30 tx=0x7f641400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c5482792f4d504 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f6402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f6402ffd640 1 -- 172.21.15.22:0/1738009294 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64140b9060 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f6402ffd640 1 -- 172.21.15.22:0/1738009294 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f64140ccb40 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f641c02cdb0 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f6402ffd640 1 -- 172.21.15.22:0/1738009294 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64140b4040 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 10 client.17412 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.940+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55eea2cda930 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f6402ffd640 1 -- 172.21.15.22:0/1738009294 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641c02cdb0 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f6402ffd640 1 -- 172.21.15.22:0/1738009294 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64140cf120 con 0x55eea2c6a760 2024-03-08T23:59:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f642c9d0740 10 client.17412 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f642c9d0740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f642c9d0740 20 client.17412 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2cdac20 0x55eea2cdd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f642c9d0740 4 client.17412 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eea2bd5690 con 0x55eea2cdac20 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f6422ffd640 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2cdac20 0x55eea2cdd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f6422ffd640 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2cdac20 0x55eea2cdd010 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cdb4a466e6cc74 server_cookie=39ef52bfb686ea3a in_seq=0 out_seq=0 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.941+0000 7f6402ffd640 10 client.17412 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.942+0000 7f6402ffd640 1 -- 172.21.15.22:0/1738009294 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eea2bd5690 con 0x55eea2cdac20 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.942+0000 7f6402ffd640 10 client.17412 handle_command_reply: tid=0 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.942+0000 7f642c9d0740 1 client.17412 shutdown 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6a760 msgr2=0x55eea2c7ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6a760 0x55eea2c7ae30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6414097c30 tx=0x7f641400e940 comp rx=0 tx=0).stop 2024-03-08T23:59:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 shutdown_connections 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eea2cdac20 0x55eea2cdd010 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eea2c73c60 0x55eea2c778e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eea2c6b080 0x55eea2c773a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6402ffd640 0 client.17412 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 --2- 172.21.15.22:0/1738009294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eea2c6a760 0x55eea2c7ae30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 >> 172.21.15.22:0/1738009294 conn(0x55eea2c64e90 msgr2=0x55eea2c6ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 shutdown_connections 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 1 -- 172.21.15.22:0/1738009294 wait complete. 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f642c9d0740 20 client.17412 trim_cache size 0 max 16384 2024-03-08T23:59:50.945 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: "2bc728b6-87a5-40dd-8aaf-1501ebfb929f": { 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "2bc728b6-87a5-40dd-8aaf-1501ebfb929f", 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-08T23:59:50.946 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-08T23:59:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f640c0a2970 msgr2=0x7f640c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f640c0a2970 0x7f640c0a4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f641c0099a0 tx=0x7f641c009280 comp rx=0 tx=0).stop 2024-03-08T23:59:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 msgr2=0x7f6424100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 0x7f6424100180 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9c0 tx=0x7f641400ee80 comp rx=0 tx=0).stop 2024-03-08T23:59:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 shutdown_connections 2024-03-08T23:59:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f640c0a2970 0x7f640c0a4e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f642410b900 0x7f6424100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6424102e50 0x7f64241006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 --2- 172.21.15.22:0/3349561821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6424102530 0x7f6424100180 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.943+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 >> 172.21.15.22:0/3349561821 conn(0x7f6424076570 msgr2=0x7f64240fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.944+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 shutdown_connections 2024-03-08T23:59:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-08T23:59:50.944+0000 7f6429d16640 1 -- 172.21.15.22:0/3349561821 wait complete. 2024-03-08T23:59:50.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:50 smithi022 ceph-mon[31894]: pgmap v165: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 300 op/s 2024-03-08T23:59:50.950 INFO:tasks.cephfs.filesystem:scrub status for tag:2bc728b6-87a5-40dd-8aaf-1501ebfb929f - {'path': '/', 'tag': '2bc728b6-87a5-40dd-8aaf-1501ebfb929f', 'options': 'recursive,force'} 2024-03-08T23:59:51.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:50 smithi062 ceph-mon[36129]: pgmap v165: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 300 op/s 2024-03-08T23:59:51.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:50 smithi192 ceph-mon[35995]: pgmap v165: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 300 op/s 2024-03-08T23:59:51.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 540913 11.45 MB/sec execute 119 sec latency 16.752 ms 2024-03-08T23:59:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:51 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-08T23:59:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:51 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:52.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:51 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-08T23:59:52.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:51 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:52.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:51 smithi022 ceph-mon[31894]: pgmap v166: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 378 op/s 2024-03-08T23:59:52.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:51 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-08T23:59:52.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:51 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:52.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:51 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-08T23:59:52.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:51 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:52.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:51 smithi062 ceph-mon[36129]: pgmap v166: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 378 op/s 2024-03-08T23:59:52.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:51 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-08T23:59:52.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:51 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:52.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:51 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-08T23:59:52.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:51 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-08T23:59:52.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:51 smithi192 ceph-mon[35995]: pgmap v166: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 378 op/s 2024-03-08T23:59:52.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 543594 11.49 MB/sec execute 120 sec latency 32.941 ms 2024-03-08T23:59:53.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 544992 11.45 MB/sec execute 121 sec latency 40.736 ms 2024-03-08T23:59:54.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:54 smithi062 ceph-mon[36129]: pgmap v167: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 362 op/s 2024-03-08T23:59:54.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:54 smithi192 ceph-mon[35995]: pgmap v167: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 362 op/s 2024-03-08T23:59:54.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:54 smithi022 ceph-mon[31894]: pgmap v167: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 362 op/s 2024-03-08T23:59:54.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 547354 11.47 MB/sec execute 122 sec latency 23.018 ms 2024-03-08T23:59:55.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:55 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-08T23:59:55.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:55 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-08T23:59:55.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:55 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-08T23:59:55.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 549429 11.43 MB/sec execute 123 sec latency 16.738 ms 2024-03-08T23:59:56.281 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:56 smithi192 ceph-mon[35995]: pgmap v168: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 481 op/s 2024-03-08T23:59:56.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:56 smithi062 ceph-mon[36129]: pgmap v168: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 481 op/s 2024-03-08T23:59:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:56 smithi022 ceph-mon[31894]: pgmap v168: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 481 op/s 2024-03-08T23:59:56.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 551419 11.44 MB/sec execute 124 sec latency 18.098 ms 2024-03-08T23:59:57.715 INFO:tasks.workunit.client.0.smithi022.stdout: 1 554152 11.49 MB/sec execute 125 sec latency 32.994 ms 2024-03-08T23:59:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.964+0000 7f5bc82b2640 1 -- 172.21.15.22:0/1254006289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100e00 msgr2=0x7f5bc010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.964+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/1254006289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100e00 0x7f5bc010d020 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009ca0 tx=0x7f5bb802c0f0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- 172.21.15.22:0/1254006289 shutdown_connections 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/1254006289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100e00 0x7f5bc010d020 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/1254006289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc0100440 0x7f5bc01008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/1254006289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bc0105190 0x7f5bc0105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- 172.21.15.22:0/1254006289 >> 172.21.15.22:0/1254006289 conn(0x7f5bc0077040 msgr2=0x7f5bc0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- 172.21.15.22:0/1254006289 shutdown_connections 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- 172.21.15.22:0/1254006289 wait complete. 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 Processor -- start 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- start start 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100440 0x7f5bc019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc0100e00 0x7f5bc019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bc0105190 0x7f5bc0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0110650 con 0x7f5bc0100440 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01104d0 con 0x7f5bc0105190 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.965+0000 7f5bc82b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01107d0 con 0x7f5bc0100e00 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc604e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100440 0x7f5bc019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc604e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100440 0x7f5bc019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59504/0 (socket says 172.21.15.22:59504) 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc604e640 1 -- 172.21.15.22:0/3126668431 learned_addr learned my addr 172.21.15.22:0/3126668431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc584d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc0100e00 0x7f5bc019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bc0105190 0x7f5bc0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc584d640 1 -- 172.21.15.22:0/3126668431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bc0105190 msgr2=0x7f5bc0197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc584d640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bc0105190 0x7f5bc0197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc584d640 1 -- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100440 msgr2=0x7f5bc019d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc584d640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100440 0x7f5bc019d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc584d640 1 -- 172.21.15.22:0/3126668431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc01980e0 con 0x7f5bc0100e00 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bc0105190 0x7f5bc0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc604e640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100440 0x7f5bc019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc584d640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc0100e00 0x7f5bc019dca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e9f0 tx=0x7f5bb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bb77fe640 1 -- 172.21.15.22:0/3126668431 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000ccc0 con 0x7f5bc0100e00 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.966+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc01983d0 con 0x7f5bc0100e00 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.967+0000 7f5bb77fe640 1 -- 172.21.15.22:0/3126668431 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bb0005020 con 0x7f5bc0100e00 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.967+0000 7f5bb77fe640 1 -- 172.21.15.22:0/3126668431 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00049e0 con 0x7f5bc0100e00 2024-03-08T23:59:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.967+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc01a4620 con 0x7f5bc0100e00 2024-03-08T23:59:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.969+0000 7f5bb77fe640 1 -- 172.21.15.22:0/3126668431 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5bb0005560 con 0x7f5bc0100e00 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.969+0000 7f5bcaf73740 1 Processor -- start 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.969+0000 7f5bcaf73740 1 -- start start 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e19a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e1a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e25940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d9d7e5f0 con 0x55b2d9e1ad40 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d9d27930 con 0x55b2d9e23550 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d9d7a690 con 0x55b2d9e19fd0 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bb77fe640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bac0a2a10 0x7f5bac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc684f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e19a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e1a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc684f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e19a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59516/0 (socket says 172.21.15.22:59516) 2024-03-08T23:59:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc684f640 1 -- 172.21.15.22:0/3578366034 learned_addr learned my addr 172.21.15.22:0/3578366034 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e1a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48782/0 (socket says 172.21.15.22:48782) 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc584d640 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e25940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bac0a2a10 0x7f5bac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bb77fe640 1 -- 172.21.15.22:0/3126668431 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5bb00cf280 con 0x7f5bc0100e00 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 -- 172.21.15.22:0/3578366034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 msgr2=0x55b2d9e25940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e25940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 -- 172.21.15.22:0/3578366034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 msgr2=0x55b2d9e19a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e19a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 -- 172.21.15.22:0/3578366034 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2d9a93210 con 0x55b2d9e19fd0 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e19a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bac0a2a10 0x7f5bac0a4ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009430 tx=0x7f5bb80095c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.970+0000 7f5bc604e640 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e1a450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc005fd0 tx=0x7f5bbc010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb1e309309cf34 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bb4bfe640 1 -- 172.21.15.22:0/3578366034 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc004230 con 0x55b2d9e19fd0 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bb4bfe640 1 -- 172.21.15.22:0/3578366034 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bbc0043d0 con 0x55b2d9e19fd0 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bb4bfe640 1 -- 172.21.15.22:0/3578366034 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0035f0 con 0x55b2d9e19fd0 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3578366034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 msgr2=0x55b2d9e1a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e1a450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc005fd0 tx=0x7f5bbc010960 comp rx=0 tx=0).stop 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3578366034 shutdown_connections 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e25940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e1a450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3578366034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e19a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3578366034 >> 172.21.15.22:0/3578366034 conn(0x55b2d9de15b0 msgr2=0x55b2d9e14cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3578366034 shutdown_connections 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.971+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3578366034 wait complete. 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 -- start start 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 Processor -- start 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e21040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d9e26280 con 0x55b2d9e1ad40 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d9d3ad90 con 0x55b2d9e23550 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d9e26400 con 0x55b2d9e19fd0 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bcaf73740 10 client.0 _ll_register_callbacks cb 0x55b2d9a92a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bc584d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bc584d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e20b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59522/0 (socket says 172.21.15.22:59522) 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bc584d640 1 -- 172.21.15.22:0/1123736945 learned_addr learned my addr 172.21.15.22:0/1123736945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bc604e640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.972+0000 7f5bc684f640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e21040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bc584d640 1 -- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 msgr2=0x55b2d9e205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bc584d640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bc584d640 1 -- 172.21.15.22:0/1123736945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 msgr2=0x55b2d9e21040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bc584d640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e21040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bc584d640 1 -- 172.21.15.22:0/1123736945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2d9b5f620 con 0x55b2d9e1ad40 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bc684f640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e21040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bc584d640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e20b00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5bb009b240 tx=0x7f5bb001ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3cd472364c5d2a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/1123736945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00ccac0 con 0x55b2d9e1ad40 2024-03-08T23:59:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/1123736945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5bb009b7a0 con 0x55b2d9e1ad40 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bbc0043d0 con 0x55b2d9e1ad40 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/1123736945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb009bc40 con 0x55b2d9e1ad40 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bcaf73740 10 client.17424 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.973+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b2d9e32610 con 0x55b2d9e1ad40 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/1123736945 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bbc0043d0 con 0x55b2d9e1ad40 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/1123736945 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5bb00a3850 con 0x55b2d9e1ad40 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bcaf73740 10 client.17424 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bcaf73740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bcaf73740 20 client.17424 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e33a80 0x55b2d9e35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bcaf73740 4 client.17424 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2d9d8bb30 con 0x55b2d9e33a80 2024-03-08T23:59:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bc604e640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e33a80 0x55b2d9e35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5ba9ffb640 10 client.17424 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.974+0000 7f5bc604e640 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e33a80 0x55b2d9e35e70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757a5a0791719d6a server_cookie=1b34720d33863df6 in_seq=0 out_seq=0 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/1123736945 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2d9d8bb30 con 0x55b2d9e33a80 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5ba9ffb640 10 client.17424 handle_command_reply: tid=0 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 client.17424 shutdown 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 msgr2=0x55b2d9e20b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e20b00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5bb009b240 tx=0x7f5bb001ce60 comp rx=0 tx=0).stop 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 shutdown_connections 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e33a80 0x55b2d9e35e70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e23550 0x55b2d9e21040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e20b00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/1123736945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5ba9ffb640 0 client.17424 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 >> 172.21.15.22:0/1123736945 conn(0x55b2d9de15b0 msgr2=0x55b2d9e16a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 shutdown_connections 2024-03-08T23:59:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 1 -- 172.21.15.22:0/1123736945 wait complete. 2024-03-08T23:59:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.976+0000 7f5bcaf73740 20 client.17424 trim_cache size 0 max 16384 2024-03-08T23:59:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 Processor -- start 2024-03-08T23:59:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 -- start start 2024-03-08T23:59:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e2a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01b0980 con 0x55b2d9e23550 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01ae520 con 0x55b2d9e19fd0 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.991+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01ac5c0 con 0x55b2d9e1ad40 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e2abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59526/0 (socket says 172.21.15.22:59526) 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 -- 172.21.15.22:0/118342461 learned_addr learned my addr 172.21.15.22:0/118342461 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc584d640 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e2a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc604e640 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e22cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 -- 172.21.15.22:0/118342461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e1ad40 msgr2=0x55b2d9e2a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e2a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 -- 172.21.15.22:0/118342461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e19fd0 msgr2=0x55b2d9e22cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e22cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 -- 172.21.15.22:0/118342461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2d9e192a0 con 0x55b2d9e23550 2024-03-08T23:59:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5bc684f640 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e2abb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005fe0 tx=0x7f5bb803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb26b0854586d8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5babfff640 1 -- 172.21.15.22:0/118342461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8052070 con 0x55b2d9e23550 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5babfff640 1 -- 172.21.15.22:0/118342461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bb8004520 con 0x55b2d9e23550 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.992+0000 7f5babfff640 1 -- 172.21.15.22:0/118342461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8039040 con 0x55b2d9e23550 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 -- 172.21.15.22:0/118342461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 msgr2=0x55b2d9e2abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e2abb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005fe0 tx=0x7f5bb803f000 comp rx=0 tx=0).stop 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 -- 172.21.15.22:0/118342461 shutdown_connections 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e2abb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e2a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/118342461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 -- 172.21.15.22:0/118342461 >> 172.21.15.22:0/118342461 conn(0x55b2d9de15b0 msgr2=0x55b2d9e147f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 -- 172.21.15.22:0/118342461 shutdown_connections 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 -- 172.21.15.22:0/118342461 wait complete. 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 -- start start 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.993+0000 7f5bcaf73740 1 Processor -- start 2024-03-08T23:59:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e276e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bcaf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e27c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01acbc0 con 0x55b2d9e23550 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01adec0 con 0x55b2d9e1ad40 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bcaf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc01ad200 con 0x55b2d9e19fd0 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bcaf73740 10 client.0 _ll_register_callbacks cb 0x7f5bbc00bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e27c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e27c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59534/0 (socket says 172.21.15.22:59534) 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 -- 172.21.15.22:0/3950567238 learned_addr learned my addr 172.21.15.22:0/3950567238 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc604e640 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc584d640 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 -- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 msgr2=0x55b2d9e271a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e271a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 -- 172.21.15.22:0/3950567238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e1ad40 msgr2=0x55b2d9e276e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e276e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 -- 172.21.15.22:0/3950567238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2d9e8a050 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5bc684f640 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e27c20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8037040 tx=0x7f5bb80324b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1f4c16c2fe94e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.994+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/3950567238 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb805d060 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/3950567238 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f5bb8004180 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bb8004520 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/3950567238 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb8052070 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 10 client.17430 fetch_fsmap learned FSMap version 33 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b2d9e8b340 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/3950567238 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bb8004520 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/3950567238 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5bb803f070 con 0x55b2d9e23550 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 10 client.17430 fetch_fsmap finished waiting for FSMap version 33 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-08T23:59:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 20 client.17430 populate_metadata read hostname 'smithi022' 2024-03-08T23:59:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e8b990 0x55b2d9e8ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T23:59:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 4 client.17430 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T23:59:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b2d9d85e80 con 0x55b2d9e8b990 2024-03-08T23:59:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.995+0000 7f5bc604e640 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e8b990 0x55b2d9e8ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T23:59:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.996+0000 7f5bc604e640 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e8b990 0x55b2d9e8ddb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d8ccc19f73d7d0 server_cookie=29565ac6a36a7e94 in_seq=0 out_seq=0 2024-03-08T23:59:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:57.996+0000 7f5ba9ffb640 10 client.17430 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5ba9ffb640 1 -- 172.21.15.22:0/3950567238 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b2d9d85e80 con 0x55b2d9e8b990 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5ba9ffb640 10 client.17430 handle_command_reply: tid=0 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5bcaf73740 1 client.17430 shutdown 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 msgr2=0x55b2d9e27c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e27c20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8037040 tx=0x7f5bb80324b0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 shutdown_connections 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2d9e8b990 0x55b2d9e8ddb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5ba9ffb640 0 client.17430 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.270+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2d9e23550 0x55b2d9e27c20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.271+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2d9e1ad40 0x55b2d9e276e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.271+0000 7f5bcaf73740 1 --2- 172.21.15.22:0/3950567238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2d9e19fd0 0x55b2d9e271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.271+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 >> 172.21.15.22:0/3950567238 conn(0x55b2d9de15b0 msgr2=0x55b2d9e1edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.271+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 shutdown_connections 2024-03-08T23:59:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.271+0000 7f5bcaf73740 1 -- 172.21.15.22:0/3950567238 wait complete. 2024-03-08T23:59:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.271+0000 7f5bcaf73740 20 client.17430 trim_cache size 0 max 16384 2024-03-08T23:59:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-08T23:59:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-08T23:59:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-08T23:59:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-08T23:59:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bac0a2a10 msgr2=0x7f5bac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bac0a2a10 0x7f5bac0a4ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009430 tx=0x7f5bb80095c0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc0100e00 msgr2=0x7f5bc019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T23:59:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc0100e00 0x7f5bc019dca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e9f0 tx=0x7f5bb000eeb0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 shutdown_connections 2024-03-08T23:59:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bac0a2a10 0x7f5bac0a4ed0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bc0105190 0x7f5bc0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc0100e00 0x7f5bc019dca0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 --2- 172.21.15.22:0/3126668431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bc0100440 0x7f5bc019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T23:59:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 >> 172.21.15.22:0/3126668431 conn(0x7f5bc0077040 msgr2=0x7f5bc0068b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T23:59:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 shutdown_connections 2024-03-08T23:59:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-08T23:59:58.272+0000 7f5bc82b2640 1 -- 172.21.15.22:0/3126668431 wait complete. 2024-03-08T23:59:58.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 08 23:59:58 smithi062 ceph-mon[36129]: pgmap v169: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-03-08T23:59:58.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 08 23:59:58 smithi192 ceph-mon[35995]: pgmap v169: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-03-08T23:59:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 08 23:59:58 smithi022 ceph-mon[31894]: pgmap v169: 129 pgs: 129 active+clean; 86 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-03-08T23:59:58.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 554973 11.41 MB/sec execute 126 sec latency 143.335 ms 2024-03-08T23:59:59.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 557334 11.42 MB/sec execute 127 sec latency 57.020 ms 2024-03-09T00:00:00.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:00 smithi062 ceph-mon[36129]: pgmap v170: 129 pgs: 129 active+clean; 84 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 475 op/s 2024-03-09T00:00:00.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:00 smithi062 ceph-mon[36129]: overall HEALTH_OK 2024-03-09T00:00:00.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:00 smithi192 ceph-mon[35995]: pgmap v170: 129 pgs: 129 active+clean; 84 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 475 op/s 2024-03-09T00:00:00.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:00 smithi192 ceph-mon[35995]: overall HEALTH_OK 2024-03-09T00:00:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:00 smithi022 ceph-mon[31894]: pgmap v170: 129 pgs: 129 active+clean; 84 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 475 op/s 2024-03-09T00:00:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:00 smithi022 ceph-mon[31894]: overall HEALTH_OK 2024-03-09T00:00:00.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 559140 11.42 MB/sec execute 128 sec latency 16.546 ms 2024-03-09T00:00:01.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 559996 11.35 MB/sec execute 129 sec latency 55.942 ms 2024-03-09T00:00:02.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:02 smithi062 ceph-mon[36129]: pgmap v171: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-03-09T00:00:02.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:02 smithi192 ceph-mon[35995]: pgmap v171: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-03-09T00:00:02.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:02 smithi022 ceph-mon[31894]: pgmap v171: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-03-09T00:00:02.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 561848 11.35 MB/sec execute 130 sec latency 55.672 ms 2024-03-09T00:00:03.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 564013 11.35 MB/sec execute 131 sec latency 51.703 ms 2024-03-09T00:00:04.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:04 smithi062 ceph-mon[36129]: pgmap v172: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-03-09T00:00:04.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:04 smithi192 ceph-mon[35995]: pgmap v172: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-03-09T00:00:04.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:04 smithi022 ceph-mon[31894]: pgmap v172: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-03-09T00:00:04.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 565580 11.33 MB/sec execute 132 sec latency 98.036 ms 2024-03-09T00:00:05.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 568286 11.37 MB/sec execute 133 sec latency 32.848 ms 2024-03-09T00:00:06.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:06 smithi062 ceph-mon[36129]: pgmap v173: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.4 MiB/s wr, 530 op/s 2024-03-09T00:00:06.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:06 smithi192 ceph-mon[35995]: pgmap v173: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.4 MiB/s wr, 530 op/s 2024-03-09T00:00:06.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:06 smithi022 ceph-mon[31894]: pgmap v173: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.4 MiB/s wr, 530 op/s 2024-03-09T00:00:06.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 569875 11.34 MB/sec execute 134 sec latency 29.720 ms 2024-03-09T00:00:07.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 572287 11.36 MB/sec execute 135 sec latency 21.514 ms 2024-03-09T00:00:08.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:08 smithi062 ceph-mon[36129]: pgmap v174: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 411 op/s 2024-03-09T00:00:08.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:08 smithi192 ceph-mon[35995]: pgmap v174: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 411 op/s 2024-03-09T00:00:08.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:08 smithi022 ceph-mon[31894]: pgmap v174: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 411 op/s 2024-03-09T00:00:08.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 574355 11.33 MB/sec execute 136 sec latency 34.600 ms 2024-03-09T00:00:09.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 575960 11.33 MB/sec execute 137 sec latency 47.320 ms 2024-03-09T00:00:10.285 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:10 smithi022 ceph-mon[31894]: pgmap v175: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 441 op/s 2024-03-09T00:00:10.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:10 smithi062 ceph-mon[36129]: pgmap v175: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 441 op/s 2024-03-09T00:00:10.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:10 smithi192 ceph-mon[35995]: pgmap v175: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 441 op/s 2024-03-09T00:00:10.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 578030 11.31 MB/sec execute 138 sec latency 27.709 ms 2024-03-09T00:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.366+0000 7fa81c64b640 1 -- 172.21.15.22:0/3438886106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 msgr2=0x7fa8141008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.366+0000 7fa81c64b640 1 --2- 172.21.15.22:0/3438886106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa8141008c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa804009900 tx=0x7fa80402bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.366+0000 7fa81c64b640 1 -- 172.21.15.22:0/3438886106 shutdown_connections 2024-03-09T00:00:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.366+0000 7fa81c64b640 1 --2- 172.21.15.22:0/3438886106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa814100e00 0x7fa81410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 --2- 172.21.15.22:0/3438886106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa8141008c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 --2- 172.21.15.22:0/3438886106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa814105190 0x7fa814105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 -- 172.21.15.22:0/3438886106 >> 172.21.15.22:0/3438886106 conn(0x7fa814077040 msgr2=0x7fa814077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 -- 172.21.15.22:0/3438886106 shutdown_connections 2024-03-09T00:00:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 -- 172.21.15.22:0/3438886106 wait complete. 2024-03-09T00:00:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 Processor -- start 2024-03-09T00:00:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 -- start start 2024-03-09T00:00:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa81419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.367+0000 7fa81c64b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa814100e00 0x7fa81419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81c64b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa814105190 0x7fa814197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81c64b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141117a0 con 0x7fa814100440 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81c64b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814111620 con 0x7fa814105190 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81c64b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814111920 con 0x7fa814100e00 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa81419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa81419d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34846/0 (socket says 172.21.15.22:34846) 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 -- 172.21.15.22:0/153138369 learned_addr learned my addr 172.21.15.22:0/153138369 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa819be6640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa814100e00 0x7fa81419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81abe8640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa814105190 0x7fa814197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 -- 172.21.15.22:0/153138369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa814100e00 msgr2=0x7fa81419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa814100e00 0x7fa81419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 -- 172.21.15.22:0/153138369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa814105190 msgr2=0x7fa814197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa814105190 0x7fa814197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 -- 172.21.15.22:0/153138369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814198180 con 0x7fa814100440 2024-03-09T00:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa819be6640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa814100e00 0x7fa81419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa81419d800 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e9f0 tx=0x7fa81000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa80b7fe640 1 -- 172.21.15.22:0/153138369 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81000cd80 con 0x7fa814100440 2024-03-09T00:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa80b7fe640 1 -- 172.21.15.22:0/153138369 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa810010910 con 0x7fa814100440 2024-03-09T00:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa814198470 con 0x7fa814100440 2024-03-09T00:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa80b7fe640 1 -- 172.21.15.22:0/153138369 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810018990 con 0x7fa814100440 2024-03-09T00:00:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.368+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8141a46c0 con 0x7fa814100440 2024-03-09T00:00:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.371+0000 7fa80b7fe640 1 -- 172.21.15.22:0/153138369 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa810010ac0 con 0x7fa814100440 2024-03-09T00:00:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 Processor -- start 2024-03-09T00:00:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 -- start start 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca9425d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca94268940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca941c15f0 con 0x55ca94266550 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9416a930 con 0x55ca9425dd40 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81f22b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca941bd690 con 0x55ca9425cfd0 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa80b7fe640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8000a2a10 0x7fa8000a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa819be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca94268940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa819be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca94268940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34850/0 (socket says 172.21.15.22:34850) 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81a3e7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca9425d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa819be6640 1 -- 172.21.15.22:0/2458082713 learned_addr learned my addr 172.21.15.22:0/2458082713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81abe8640 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81abe8640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8000a2a10 0x7fa8000a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa80b7fe640 1 -- 172.21.15.22:0/153138369 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa8100cf4d0 con 0x7fa814100440 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81abe8640 1 -- 172.21.15.22:0/2458082713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 msgr2=0x55ca9425d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.372+0000 7fa81abe8640 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca9425d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa81abe8640 1 -- 172.21.15.22:0/2458082713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 msgr2=0x55ca94268940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa81abe8640 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca94268940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa81abe8640 1 -- 172.21.15.22:0/2458082713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca93ed6210 con 0x55ca9425dd40 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa819be6640 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca94268940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa81abe8640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8000a2a10 0x7fa8000a4ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005c40 tx=0x7fa80c016000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa81abe8640 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425ca90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa80c014e40 tx=0x7fa80c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19203884c03cc0df server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa808bfe640 1 -- 172.21.15.22:0/2458082713 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c007bf0 con 0x55ca9425dd40 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa808bfe640 1 -- 172.21.15.22:0/2458082713 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa80c00cbf0 con 0x55ca9425dd40 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.373+0000 7fa808bfe640 1 -- 172.21.15.22:0/2458082713 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c004280 con 0x55ca9425dd40 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 -- 172.21.15.22:0/2458082713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 msgr2=0x55ca9425ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425ca90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa80c014e40 tx=0x7fa80c0117b0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 -- 172.21.15.22:0/2458082713 shutdown_connections 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca94268940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca9425d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 --2- 172.21.15.22:0/2458082713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425ca90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 -- 172.21.15.22:0/2458082713 >> 172.21.15.22:0/2458082713 conn(0x55ca942245b0 msgr2=0x55ca94257cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 -- 172.21.15.22:0/2458082713 shutdown_connections 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 -- 172.21.15.22:0/2458082713 wait complete. 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 -- start start 2024-03-09T00:00:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 Processor -- start 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.374+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca942635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca94263b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca94264070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81f22b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca94269220 con 0x55ca9425cfd0 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81f22b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9417dd90 con 0x55ca94266550 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81f22b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca942693a0 con 0x55ca9425dd40 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca942635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81f22b740 10 client.0 _ll_register_callbacks cb 0x55ca93ed5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca942635f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34852/0 (socket says 172.21.15.22:34852) 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 -- 172.21.15.22:0/1020442163 learned_addr learned my addr 172.21.15.22:0/1020442163 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa819be6640 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca94263b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81abe8640 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca94264070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 -- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 msgr2=0x55ca94263b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca94263b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 -- 172.21.15.22:0/1020442163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 msgr2=0x55ca94264070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca94264070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 -- 172.21.15.22:0/1020442163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca93fa2620 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca942635f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa8100cc930 tx=0x7fa81006a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ec390843783c75 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa7fdffb640 1 -- 172.21.15.22:0/1020442163 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81009f8e0 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa7fdffb640 1 -- 172.21.15.22:0/1020442163 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa8100cf170 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.375+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa80c00cbf0 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa7fdffb640 1 -- 172.21.15.22:0/1020442163 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8100a3f30 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 10 client.17442 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ca94275560 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa7fdffb640 1 -- 172.21.15.22:0/1020442163 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa80c00cbf0 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa7fdffb640 1 -- 172.21.15.22:0/1020442163 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa8100afb70 con 0x55ca9425cfd0 2024-03-09T00:00:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 10 client.17442 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 10 client.17442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 20 client.17442 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942769d0 0x55ca94278dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 4 client.17442 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca941ceb30 con 0x55ca942769d0 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.376+0000 7fa819be6640 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942769d0 0x55ca94278dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa7fdffb640 10 client.17442 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa819be6640 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942769d0 0x55ca94278dc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277bfd14dc595139 server_cookie=a309927406b865f5 in_seq=0 out_seq=0 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa7fdffb640 1 -- 172.21.15.22:0/1020442163 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca941ceb30 con 0x55ca942769d0 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa7fdffb640 10 client.17442 handle_command_reply: tid=0 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 client.17442 shutdown 2024-03-09T00:00:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 msgr2=0x55ca942635f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca942635f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa8100cc930 tx=0x7fa81006a710 comp rx=0 tx=0).stop 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 shutdown_connections 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942769d0 0x55ca94278dc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca94264070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca94263b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 --2- 172.21.15.22:0/1020442163 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca942635f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa7fdffb640 0 client.17442 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.377+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 >> 172.21.15.22:0/1020442163 conn(0x55ca942245b0 msgr2=0x55ca94259a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.378+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 shutdown_connections 2024-03-09T00:00:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.378+0000 7fa81f22b740 1 -- 172.21.15.22:0/1020442163 wait complete. 2024-03-09T00:00:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.378+0000 7fa81f22b740 20 client.17442 trim_cache size 0 max 16384 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 Processor -- start 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 -- start start 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca94265d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca9425edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141b0a10 con 0x55ca94266550 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141ae5b0 con 0x55ca9425dd40 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81f22b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141ac650 con 0x55ca9425cfd0 2024-03-09T00:00:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81abe8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca9425edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81abe8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca9425edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34868/0 (socket says 172.21.15.22:34868) 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81abe8640 1 -- 172.21.15.22:0/3290250810 learned_addr learned my addr 172.21.15.22:0/3290250810 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca94265d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.392+0000 7fa819be6640 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa81a3e7640 1 -- 172.21.15.22:0/3290250810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 msgr2=0x55ca9425e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa81a3e7640 1 -- 172.21.15.22:0/3290250810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 msgr2=0x55ca9425edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca9425edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa81a3e7640 1 -- 172.21.15.22:0/3290250810 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca94253340 con 0x55ca9425cfd0 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa81abe8640 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca9425edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa819be6640 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca94265d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa81006e5b0 tx=0x7fa810000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d1e5a537337e28 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa7fffff640 1 -- 172.21.15.22:0/3290250810 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81009b1a0 con 0x55ca9425cfd0 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa7fffff640 1 -- 172.21.15.22:0/3290250810 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa810097eb0 con 0x55ca9425cfd0 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.393+0000 7fa7fffff640 1 -- 172.21.15.22:0/3290250810 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810018b30 con 0x55ca9425cfd0 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- 172.21.15.22:0/3290250810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 msgr2=0x55ca94265d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca94265d20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa81006e5b0 tx=0x7fa810000bc0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- 172.21.15.22:0/3290250810 shutdown_connections 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca94266550 0x55ca9425edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca9425dd40 0x55ca9425e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3290250810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425cfd0 0x55ca94265d20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- 172.21.15.22:0/3290250810 >> 172.21.15.22:0/3290250810 conn(0x55ca942245b0 msgr2=0x55ca9427d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- 172.21.15.22:0/3290250810 shutdown_connections 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- 172.21.15.22:0/3290250810 wait complete. 2024-03-09T00:00:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- start start 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 Processor -- start 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca9426d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca9426bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca9426c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141acc50 con 0x55ca9425cfd0 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141adf50 con 0x55ca94266550 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141ad290 con 0x55ca9425dd40 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81f22b740 10 client.0 _ll_register_callbacks cb 0x7fa80c007e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81a3e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca9426d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81a3e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca9426d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34874/0 (socket says 172.21.15.22:34874) 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81a3e7640 1 -- 172.21.15.22:0/3991929696 learned_addr learned my addr 172.21.15.22:0/3991929696 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa819be6640 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca9426bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.394+0000 7fa81abe8640 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca9426c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81a3e7640 1 -- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 msgr2=0x55ca9426bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca9426bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81a3e7640 1 -- 172.21.15.22:0/3991929696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 msgr2=0x55ca9426c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca9426c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81a3e7640 1 -- 172.21.15.22:0/3991929696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca9426c790 con 0x55ca9425cfd0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81a3e7640 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca9426d810 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa8100a3a00 tx=0x7fa8100afea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405efbbdb64805e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa7fdffb640 1 -- 172.21.15.22:0/3991929696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8100b4060 con 0x55ca9425cfd0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa810097eb0 con 0x55ca9425cfd0 2024-03-09T00:00:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa7fdffb640 1 -- 172.21.15.22:0/3991929696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fa81009ac90 con 0x55ca9425cfd0 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa7fdffb640 1 -- 172.21.15.22:0/3991929696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81006c070 con 0x55ca9425cfd0 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81f22b740 10 client.17448 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ca942cdca0 con 0x55ca9425cfd0 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa7fdffb640 1 -- 172.21.15.22:0/3991929696 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa810097eb0 con 0x55ca9425cfd0 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa7fdffb640 1 -- 172.21.15.22:0/3991929696 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa810097770 con 0x55ca9425cfd0 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81f22b740 10 client.17448 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81f22b740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.395+0000 7fa81f22b740 20 client.17448 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.396+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942ce100 0x55ca942d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.396+0000 7fa81f22b740 4 client.17448 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.396+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca941c8e80 con 0x55ca942ce100 2024-03-09T00:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.396+0000 7fa819be6640 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942ce100 0x55ca942d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.396+0000 7fa7fdffb640 10 client.17448 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.396+0000 7fa819be6640 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942ce100 0x55ca942d0520 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff88446d986bea0c server_cookie=d07ee96536218c8c in_seq=0 out_seq=0 2024-03-09T00:00:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa7fdffb640 1 -- 172.21.15.22:0/3991929696 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca941c8e80 con 0x55ca942ce100 2024-03-09T00:00:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa7fdffb640 10 client.17448 handle_command_reply: tid=0 2024-03-09T00:00:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa81f22b740 1 client.17448 shutdown 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 msgr2=0x55ca9426d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca9426d810 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa8100a3a00 tx=0x7fa8100afea0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 shutdown_connections 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ca942ce100 0x55ca942d0520 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca94266550 0x55ca9426c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca9425dd40 0x55ca9426bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.677+0000 7fa7fdffb640 0 client.17448 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.678+0000 7fa81f22b740 1 --2- 172.21.15.22:0/3991929696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ca9425cfd0 0x55ca9426d810 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.678+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 >> 172.21.15.22:0/3991929696 conn(0x55ca942245b0 msgr2=0x55ca9425e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.678+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 shutdown_connections 2024-03-09T00:00:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.678+0000 7fa81f22b740 1 -- 172.21.15.22:0/3991929696 wait complete. 2024-03-09T00:00:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.678+0000 7fa81f22b740 20 client.17448 trim_cache size 0 max 16384 2024-03-09T00:00:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:00:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:00:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:00:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:00:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8000a2a10 msgr2=0x7fa8000a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8000a2a10 0x7fa8000a4ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005c40 tx=0x7fa80c016000 comp rx=0 tx=0).stop 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 msgr2=0x7fa81419d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa81419d800 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa81000e9f0 tx=0x7fa81000eeb0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 shutdown_connections 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8000a2a10 0x7fa8000a4ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa814105190 0x7fa814197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa814100e00 0x7fa81419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 --2- 172.21.15.22:0/153138369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa814100440 0x7fa81419d800 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 >> 172.21.15.22:0/153138369 conn(0x7fa814077040 msgr2=0x7fa8140ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 shutdown_connections 2024-03-09T00:00:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:11.679+0000 7fa81c64b640 1 -- 172.21.15.22:0/153138369 wait complete. 2024-03-09T00:00:11.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 579553 11.31 MB/sec execute 139 sec latency 155.177 ms 2024-03-09T00:00:12.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:12 smithi062 ceph-mon[36129]: pgmap v176: 129 pgs: 129 active+clean; 89 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-03-09T00:00:12.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:12 smithi192 ceph-mon[35995]: pgmap v176: 129 pgs: 129 active+clean; 89 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-03-09T00:00:12.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:12 smithi022 ceph-mon[31894]: pgmap v176: 129 pgs: 129 active+clean; 89 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-03-09T00:00:12.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 581692 11.30 MB/sec execute 140 sec latency 34.461 ms 2024-03-09T00:00:13.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 583598 11.30 MB/sec execute 141 sec latency 18.596 ms 2024-03-09T00:00:14.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:14 smithi062 ceph-mon[36129]: pgmap v177: 129 pgs: 129 active+clean; 89 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-03-09T00:00:14.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:14 smithi192 ceph-mon[35995]: pgmap v177: 129 pgs: 129 active+clean; 89 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-03-09T00:00:14.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:14 smithi022 ceph-mon[31894]: pgmap v177: 129 pgs: 129 active+clean; 89 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-03-09T00:00:14.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 585520 11.29 MB/sec execute 142 sec latency 89.357 ms 2024-03-09T00:00:15.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 587481 11.29 MB/sec execute 143 sec latency 16.983 ms 2024-03-09T00:00:16.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:16 smithi062 ceph-mon[36129]: pgmap v178: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-03-09T00:00:16.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:16 smithi192 ceph-mon[35995]: pgmap v178: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-03-09T00:00:16.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:16 smithi022 ceph-mon[31894]: pgmap v178: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-03-09T00:00:16.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 589862 11.31 MB/sec execute 144 sec latency 15.860 ms 2024-03-09T00:00:17.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 591600 11.27 MB/sec execute 145 sec latency 39.357 ms 2024-03-09T00:00:18.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:18 smithi062 ceph-mon[36129]: pgmap v179: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-03-09T00:00:18.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:18 smithi192 ceph-mon[35995]: pgmap v179: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-03-09T00:00:18.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:18 smithi022 ceph-mon[31894]: pgmap v179: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-03-09T00:00:18.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 593784 11.29 MB/sec execute 146 sec latency 18.438 ms 2024-03-09T00:00:19.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 595600 11.26 MB/sec execute 147 sec latency 67.753 ms 2024-03-09T00:00:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:20 smithi062 ceph-mon[36129]: pgmap v180: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-03-09T00:00:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:20 smithi192 ceph-mon[35995]: pgmap v180: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-03-09T00:00:20.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:20 smithi022 ceph-mon[31894]: pgmap v180: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-03-09T00:00:20.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 597512 11.27 MB/sec execute 148 sec latency 16.558 ms 2024-03-09T00:00:20.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:00:20.951 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:00:21.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 -- 172.21.15.22:0/4121768204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 msgr2=0x7f51081007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 --2- 172.21.15.22:0/4121768204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f51081007b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009860 tx=0x7f50f802bb70 comp rx=0 tx=0).stop 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 -- 172.21.15.22:0/4121768204 shutdown_connections 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 --2- 172.21.15.22:0/4121768204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51081054a0 0x7f510810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 --2- 172.21.15.22:0/4121768204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5108100d80 0x7f5108104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 --2- 172.21.15.22:0/4121768204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f51081007b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 -- 172.21.15.22:0/4121768204 >> 172.21.15.22:0/4121768204 conn(0x7f51080772f0 msgr2=0x7f5108077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 -- 172.21.15.22:0/4121768204 shutdown_connections 2024-03-09T00:00:21.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.073+0000 7f510e659640 1 -- 172.21.15.22:0/4121768204 wait complete. 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.074+0000 7f510e659640 1 Processor -- start 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.074+0000 7f510e659640 1 -- start start 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f510e659640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f5108101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f510e659640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51081054a0 0x7f51081023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f510e659640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51081028e0 0x7f51081a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f510e659640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5108110560 con 0x7f51081028e0 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f510e659640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081103e0 con 0x7f51081003d0 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f510e659640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081106e0 con 0x7f51081054a0 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f5108101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f5108101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:44824/0 (socket says 172.21.15.22:44824) 2024-03-09T00:00:21.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 -- 172.21.15.22:0/3284657639 learned_addr learned my addr 172.21.15.22:0/3284657639 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f51077fe640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51081054a0 0x7f51081023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f510cbf6640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51081028e0 0x7f51081a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 -- 172.21.15.22:0/3284657639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51081054a0 msgr2=0x7f51081023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51081054a0 0x7f51081023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 -- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51081028e0 msgr2=0x7f51081a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51081028e0 0x7f51081a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 -- 172.21.15.22:0/3284657639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51081a6430 con 0x7f51081003d0 2024-03-09T00:00:21.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.075+0000 7f5107fff640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f5108101e60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50f8037040 tx=0x7f50f80045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.076+0000 7f51057fa640 1 -- 172.21.15.22:0/3284657639 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8041020 con 0x7f51081003d0 2024-03-09T00:00:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.076+0000 7f51057fa640 1 -- 172.21.15.22:0/3284657639 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f802c9f0 con 0x7f51081003d0 2024-03-09T00:00:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.076+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51081a66c0 con 0x7f51081003d0 2024-03-09T00:00:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.076+0000 7f51057fa640 1 -- 172.21.15.22:0/3284657639 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8039040 con 0x7f51081003d0 2024-03-09T00:00:21.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.076+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081a6bb0 con 0x7f51081003d0 2024-03-09T00:00:21.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.079+0000 7f51057fa640 1 -- 172.21.15.22:0/3284657639 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f50f8044020 con 0x7f51081003d0 2024-03-09T00:00:21.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.079+0000 7f511123b740 1 Processor -- start 2024-03-09T00:00:21.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.079+0000 7f511123b740 1 -- start start 2024-03-09T00:00:21.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f511123b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3318b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f511123b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f3321720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f511123b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3324050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f511123b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8f327be00 con 0x55e8f3321c60 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f511123b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8f3224b10 con 0x55e8f3318760 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f511123b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8f3277ea0 con 0x55e8f3319080 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f51057fa640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50f00a2af0 0x7f50f00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f51077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3324050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3318b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3318b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:44828/0 (socket says 172.21.15.22:44828) 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 -- 172.21.15.22:0/2157496186 learned_addr learned my addr 172.21.15.22:0/2157496186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f51057fa640 1 -- 172.21.15.22:0/3284657639 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f50f80f07e0 con 0x7f51081003d0 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50f00a2af0 0x7f50f00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f5107fff640 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f3321720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 -- 172.21.15.22:0/2157496186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 msgr2=0x55e8f3321720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f3321720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 -- 172.21.15.22:0/2157496186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 msgr2=0x55e8f3324050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3324050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.080+0000 7f510cbf6640 1 -- 172.21.15.22:0/2157496186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8f305a7d0 con 0x55e8f3318760 2024-03-09T00:00:21.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.081+0000 7f510cbf6640 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3318b40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f51000099e0 tx=0x7f5100011d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc635a51ec2533b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.081+0000 7f50ee7fc640 1 -- 172.21.15.22:0/2157496186 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100022020 con 0x55e8f3318760 2024-03-09T00:00:21.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.081+0000 7f50ee7fc640 1 -- 172.21.15.22:0/2157496186 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5100003e50 con 0x55e8f3318760 2024-03-09T00:00:21.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.081+0000 7f50ee7fc640 1 -- 172.21.15.22:0/2157496186 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100004320 con 0x55e8f3318760 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 -- 172.21.15.22:0/2157496186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 msgr2=0x55e8f3318b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3318b40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f51000099e0 tx=0x7f5100011d80 comp rx=0 tx=0).stop 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f510cbf6640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50f00a2af0 0x7f50f00a4fb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5100023000 tx=0x7f510001f000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 -- 172.21.15.22:0/2157496186 shutdown_connections 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3324050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f3321720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 --2- 172.21.15.22:0/2157496186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3318b40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 -- 172.21.15.22:0/2157496186 >> 172.21.15.22:0/2157496186 conn(0x55e8f3312e90 msgr2=0x55e8f3313270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 -- 172.21.15.22:0/2157496186 shutdown_connections 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 -- 172.21.15.22:0/2157496186 wait complete. 2024-03-09T00:00:21.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.082+0000 7f511123b740 1 -- start start 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 1 Processor -- start 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f33212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f331e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f5107fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f33212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f5107fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f33212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:44844/0 (socket says 172.21.15.22:44844) 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f331eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8f3324af0 con 0x55e8f3321c60 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8f32380a0 con 0x55e8f3318760 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8f3324c70 con 0x55e8f3319080 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f51077fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f331e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f51077fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f331e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44980/0 (socket says 172.21.15.22:44980) 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f51077fe640 1 -- 172.21.15.22:0/3152179757 learned_addr learned my addr 172.21.15.22:0/3152179757 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.083+0000 7f511123b740 10 client.0 _ll_register_callbacks cb 0x55e8f3237a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f510cbf6640 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f331eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f51077fe640 1 -- 172.21.15.22:0/3152179757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 msgr2=0x55e8f33212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f51077fe640 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f33212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f51077fe640 1 -- 172.21.15.22:0/3152179757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 msgr2=0x55e8f331eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f51077fe640 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f331eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f51077fe640 1 -- 172.21.15.22:0/3152179757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8f2f903f0 con 0x55e8f3319080 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f51077fe640 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f331e650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0126e0 tx=0x7f50fc012ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb50ed91aef98199 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f50e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f50e77fe640 1 -- 172.21.15.22:0/3152179757 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00f670 con 0x55e8f3319080 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f50e77fe640 1 -- 172.21.15.22:0/3152179757 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f50fc00f810 con 0x55e8f3319080 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.084+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5100003e50 con 0x55e8f3319080 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f50e77fe640 1 -- 172.21.15.22:0/3152179757 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50fc00fa10 con 0x55e8f3319080 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 10 client.26905 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e8f2f500f0 con 0x55e8f3319080 2024-03-09T00:00:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f50e77fe640 1 -- 172.21.15.22:0/3152179757 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50fc00fbb0 con 0x55e8f3319080 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f50e77fe640 1 -- 172.21.15.22:0/3152179757 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f50fc01b3c0 con 0x55e8f3319080 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 10 client.26905 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 10 client.26905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 20 client.26905 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3331750 0x55e8f3333ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 4 client.26905 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5100003e50 con 0x55e8f3331750 2024-03-09T00:00:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.085+0000 7f5107fff640 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3331750 0x55e8f3333ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.086+0000 7f5107fff640 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3331750 0x55e8f3333ba0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6289523240c305c5 server_cookie=b8b770e2c3cd9548 in_seq=0 out_seq=0 2024-03-09T00:00:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.086+0000 7f50e77fe640 10 client.26905 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.086+0000 7f50e77fe640 1 -- 172.21.15.22:0/3152179757 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f5100003e50 con 0x55e8f3331750 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.086+0000 7f50e77fe640 10 client.26905 handle_command_reply: tid=0 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.086+0000 7f511123b740 1 client.26905 shutdown 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 msgr2=0x55e8f331e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f331e650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0126e0 tx=0x7f50fc012ae0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 shutdown_connections 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3331750 0x55e8f3333ba0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f331eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3319080 0x55e8f331e650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f50e77fe640 0 client.26905 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 --2- 172.21.15.22:0/3152179757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f33212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 >> 172.21.15.22:0/3152179757 conn(0x55e8f3312e90 msgr2=0x55e8f3314be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 shutdown_connections 2024-03-09T00:00:21.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 1 -- 172.21.15.22:0/3152179757 wait complete. 2024-03-09T00:00:21.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.087+0000 7f511123b740 20 client.26905 trim_cache size 0 max 16384 2024-03-09T00:00:21.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 Processor -- start 2024-03-09T00:00:21.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 -- start start 2024-03-09T00:00:21.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3320d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3319080 0x55e8f33212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3321c60 0x55e8f33196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081b2c60 con 0x55e8f3319080 2024-03-09T00:00:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081b0800 con 0x55e8f3318760 2024-03-09T00:00:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f511123b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081ae8a0 con 0x55e8f3321c60 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f51077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3319080 0x55e8f33212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f51077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3319080 0x55e8f33212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50448/0 (socket says 172.21.15.22:50448) 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f51077fe640 1 -- 172.21.15.22:0/3653546117 learned_addr learned my addr 172.21.15.22:0/3653546117 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f5107fff640 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3320d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.105+0000 7f510cbf6640 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3321c60 0x55e8f33196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f5107fff640 1 -- 172.21.15.22:0/3653546117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3321c60 msgr2=0x55e8f33196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f5107fff640 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3321c60 0x55e8f33196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f5107fff640 1 -- 172.21.15.22:0/3653546117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3319080 msgr2=0x55e8f33212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f5107fff640 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3319080 0x55e8f33212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f5107fff640 1 -- 172.21.15.22:0/3653546117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8f3338de0 con 0x55e8f3318760 2024-03-09T00:00:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f5107fff640 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3320d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f50f80edca0 tx=0x7f50f8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c173ed823e2698e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f50edffb640 1 -- 172.21.15.22:0/3653546117 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80bd2b0 con 0x55e8f3318760 2024-03-09T00:00:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f50edffb640 1 -- 172.21.15.22:0/3653546117 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f802cbc0 con 0x55e8f3318760 2024-03-09T00:00:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.106+0000 7f50edffb640 1 -- 172.21.15.22:0/3653546117 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80ede10 con 0x55e8f3318760 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 -- 172.21.15.22:0/3653546117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 msgr2=0x55e8f3320d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3320d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f50f80edca0 tx=0x7f50f8005f90 comp rx=0 tx=0).stop 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 -- 172.21.15.22:0/3653546117 shutdown_connections 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3321c60 0x55e8f33196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3319080 0x55e8f33212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 --2- 172.21.15.22:0/3653546117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3318760 0x55e8f3320d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 -- 172.21.15.22:0/3653546117 >> 172.21.15.22:0/3653546117 conn(0x55e8f3312e90 msgr2=0x55e8f331cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 -- 172.21.15.22:0/3653546117 shutdown_connections 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 -- 172.21.15.22:0/3653546117 wait complete. 2024-03-09T00:00:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.107+0000 7f511123b740 1 -- start start 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 1 Processor -- start 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3318760 0x55e8f3328f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3319080 0x55e8f3329450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3326ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081aeea0 con 0x55e8f3321c60 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081b01a0 con 0x55e8f3319080 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081af4e0 con 0x55e8f3318760 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f511123b740 10 client.0 _ll_register_callbacks cb 0x7f50fc009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f510cbf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3326ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f5107fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3318760 0x55e8f3328f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f51077fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3319080 0x55e8f3329450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f510cbf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3326ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50452/0 (socket says 172.21.15.22:50452) 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f510cbf6640 1 -- 172.21.15.22:0/2859708704 learned_addr learned my addr 172.21.15.22:0/2859708704 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f51077fe640 1 -- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3318760 msgr2=0x55e8f3328f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f51077fe640 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3318760 0x55e8f3328f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f51077fe640 1 -- 172.21.15.22:0/2859708704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 msgr2=0x55e8f3326ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f51077fe640 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3326ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f51077fe640 1 -- 172.21.15.22:0/2859708704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8f3327530 con 0x55e8f3319080 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.108+0000 7f5107fff640 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3318760 0x55e8f3328f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f51077fe640 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3319080 0x55e8f3329450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50fc01b830 tx=0x7f50fc00f8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c864c97d8054108 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f50e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f50e77fe640 1 -- 172.21.15.22:0/2859708704 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc009e10 con 0x55e8f3319080 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f50e77fe640 1 -- 172.21.15.22:0/2859708704 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f50fc00fdc0 con 0x55e8f3319080 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50f802cbc0 con 0x55e8f3319080 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f50e77fe640 1 -- 172.21.15.22:0/2859708704 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50fc038050 con 0x55e8f3319080 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 10 client.26888 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e8f3389600 con 0x55e8f3319080 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f50e77fe640 1 -- 172.21.15.22:0/2859708704 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f802cbc0 con 0x55e8f3319080 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f50e77fe640 1 -- 172.21.15.22:0/2859708704 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f50fc0035f0 con 0x55e8f3319080 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 10 client.26888 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 10 client.26888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 20 client.26888 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3389d50 0x55e8f338c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 4 client.26888 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.109+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e8f3289340 con 0x55e8f3389d50 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.110+0000 7f5107fff640 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3389d50 0x55e8f338c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.110+0000 7f5107fff640 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3389d50 0x55e8f338c170 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74790b481bc4da87 server_cookie=b40d8c2afaa74b4f in_seq=0 out_seq=0 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.110+0000 7f50e77fe640 10 client.26888 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f50e77fe640 1 -- 172.21.15.22:0/2859708704 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e8f3289340 con 0x55e8f3389d50 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f50e77fe640 10 client.26888 handle_command_reply: tid=0 2024-03-09T00:00:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 client.26888 shutdown 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3319080 msgr2=0x55e8f3329450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3319080 0x55e8f3329450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50fc01b830 tx=0x7f50fc00f8a0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 shutdown_connections 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8f3389d50 0x55e8f338c170 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e8f3321c60 0x55e8f3326ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8f3319080 0x55e8f3329450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 --2- 172.21.15.22:0/2859708704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8f3318760 0x55e8f3328f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f50e77fe640 0 client.26888 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 >> 172.21.15.22:0/2859708704 conn(0x55e8f3312e90 msgr2=0x55e8f331cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 shutdown_connections 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.111+0000 7f511123b740 1 -- 172.21.15.22:0/2859708704 wait complete. 2024-03-09T00:00:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f511123b740 20 client.26888 trim_cache size 0 max 16384 2024-03-09T00:00:21.113 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:00:21.113 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:00:21.113 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:00:21.113 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50f00a2af0 msgr2=0x7f50f00a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50f00a2af0 0x7f50f00a4fb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5100023000 tx=0x7f510001f000 comp rx=0 tx=0).stop 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 msgr2=0x7f5108101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f5108101e60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f50f8037040 tx=0x7f50f80045b0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 shutdown_connections 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50f00a2af0 0x7f50f00a4fb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51081028e0 0x7f51081a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51081054a0 0x7f51081023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 --2- 172.21.15.22:0/3284657639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51081003d0 0x7f5108101e60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 >> 172.21.15.22:0/3284657639 conn(0x7f51080772f0 msgr2=0x7f51080687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 shutdown_connections 2024-03-09T00:00:21.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.112+0000 7f510e659640 1 -- 172.21.15.22:0/3284657639 wait complete. 2024-03-09T00:00:21.119 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:00:21.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:00:21.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 -- 172.21.15.22:0/405942856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918105530 msgr2=0x7f191810d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 --2- 172.21.15.22:0/405942856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918105530 0x7f191810d100 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1910009ca0 tx=0x7f191002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 -- 172.21.15.22:0/405942856 shutdown_connections 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 --2- 172.21.15.22:0/405942856 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918105530 0x7f191810d100 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 --2- 172.21.15.22:0/405942856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1918100e10 0x7f1918104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 --2- 172.21.15.22:0/405942856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1918100460 0x7f1918100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 -- 172.21.15.22:0/405942856 >> 172.21.15.22:0/405942856 conn(0x7f19180772f0 msgr2=0x7f1918077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 -- 172.21.15.22:0/405942856 shutdown_connections 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.236+0000 7f191d8b2640 1 -- 172.21.15.22:0/405942856 wait complete. 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 Processor -- start 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 -- start start 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1918100460 0x7f1918101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1918105530 0x7f1918102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918102970 0x7f19181a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19181105f0 con 0x7f1918102970 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918110470 con 0x7f1918100460 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f191d8b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918110770 con 0x7f1918105530 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918102970 0x7f19181a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918102970 0x7f19181a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50484/0 (socket says 172.21.15.22:50484) 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 -- 172.21.15.22:0/4272292732 learned_addr learned my addr 172.21.15.22:0/4272292732 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f1916ffd640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1918100460 0x7f1918101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f190e7fc640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1918105530 0x7f1918102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 -- 172.21.15.22:0/4272292732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1918105530 msgr2=0x7f1918102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1918105530 0x7f1918102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 -- 172.21.15.22:0/4272292732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1918100460 msgr2=0x7f1918101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1918100460 0x7f1918101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.237+0000 7f19177fe640 1 -- 172.21.15.22:0/4272292732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19181a64d0 con 0x7f1918102970 2024-03-09T00:00:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.238+0000 7f19177fe640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918102970 0x7f19181a5dd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1910037040 tx=0x7f1910032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.238+0000 7f191c8b0640 1 -- 172.21.15.22:0/4272292732 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910041020 con 0x7f1918102970 2024-03-09T00:00:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.238+0000 7f191c8b0640 1 -- 172.21.15.22:0/4272292732 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1910032e80 con 0x7f1918102970 2024-03-09T00:00:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.238+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19181a6760 con 0x7f1918102970 2024-03-09T00:00:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.238+0000 7f191c8b0640 1 -- 172.21.15.22:0/4272292732 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910039040 con 0x7f1918102970 2024-03-09T00:00:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.238+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19181a6ca0 con 0x7f1918102970 2024-03-09T00:00:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.239+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19180680e0 con 0x7f1918102970 2024-03-09T00:00:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.240+0000 7f191c8b0640 1 -- 172.21.15.22:0/4272292732 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1910044020 con 0x7f1918102970 2024-03-09T00:00:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.241+0000 7f191c8b0640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19040a2a20 0x7f19040a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.241+0000 7f1916ffd640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19040a2a20 0x7f19040a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.241+0000 7f191c8b0640 1 -- 172.21.15.22:0/4272292732 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f19100f0cb0 con 0x7f1918102970 2024-03-09T00:00:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.241+0000 7f1916ffd640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19040a2a20 0x7f19040a4ee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1908009bd0 tx=0x7f1908005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.246+0000 7f191c8b0640 1 -- 172.21.15.22:0/4272292732 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f191004b020 con 0x7f1918102970 2024-03-09T00:00:21.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.510+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19181097d0 con 0x7f1918102970 2024-03-09T00:00:21.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.511+0000 7f191c8b0640 1 -- 172.21.15.22:0/4272292732 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f1910003e20 con 0x7f1918102970 2024-03-09T00:00:21.506 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:00:21.507 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:51.502302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26555":{"gid":26555,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2421521561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2421521561},{"type":"v1","addr":"172.21.15.62:6837","nonce":2421521561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:00:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-03-09T00:00:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19040a2a20 msgr2=0x7f19040a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19040a2a20 0x7f19040a4ee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1908009bd0 tx=0x7f1908005f90 comp rx=0 tx=0).stop 2024-03-09T00:00:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918102970 msgr2=0x7f19181a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918102970 0x7f19181a5dd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1910037040 tx=0x7f1910032c90 comp rx=0 tx=0).stop 2024-03-09T00:00:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 shutdown_connections 2024-03-09T00:00:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f19040a2a20 0x7f19040a4ee0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1918102970 0x7f19181a5dd0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1918105530 0x7f1918102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 --2- 172.21.15.22:0/4272292732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1918100460 0x7f1918101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 >> 172.21.15.22:0/4272292732 conn(0x7f19180772f0 msgr2=0x7f1918068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 shutdown_connections 2024-03-09T00:00:21.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.513+0000 7f191d8b2640 1 -- 172.21.15.22:0/4272292732 wait complete. 2024-03-09T00:00:21.521 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:00:21.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:00:21.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 -- 172.21.15.22:0/921994015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf1806a540 msgr2=0x7faf18105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/921994015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf1806a540 0x7faf18105ec0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7faf00009900 tx=0x7faf0002bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:21.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 -- 172.21.15.22:0/921994015 shutdown_connections 2024-03-09T00:00:21.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/921994015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf181065a0 0x7faf1810e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/921994015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf1806a540 0x7faf18105ec0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/921994015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf18069b90 0x7faf18069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 -- 172.21.15.22:0/921994015 >> 172.21.15.22:0/921994015 conn(0x7faf18076f80 msgr2=0x7faf18077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 -- 172.21.15.22:0/921994015 shutdown_connections 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.643+0000 7faf1cc0c640 1 -- 172.21.15.22:0/921994015 wait complete. 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 Processor -- start 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 -- start start 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf18069b90 0x7faf18100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf181065a0 0x7faf18100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf18100f90 0x7faf181a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf18111630 con 0x7faf18100f90 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181114b0 con 0x7faf18069b90 2024-03-09T00:00:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1cc0c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181117b0 con 0x7faf181065a0 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf16d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf18100f90 0x7faf181a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf16d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf18100f90 0x7faf181a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50508/0 (socket says 172.21.15.22:50508) 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf16d9d640 1 -- 172.21.15.22:0/2694569403 learned_addr learned my addr 172.21.15.22:0/2694569403 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf0ffff640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf181065a0 0x7faf18100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.644+0000 7faf1659c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf18069b90 0x7faf18100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf16d9d640 1 -- 172.21.15.22:0/2694569403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf181065a0 msgr2=0x7faf18100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf16d9d640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf181065a0 0x7faf18100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf16d9d640 1 -- 172.21.15.22:0/2694569403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf18069b90 msgr2=0x7faf18100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf16d9d640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf18069b90 0x7faf18100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf16d9d640 1 -- 172.21.15.22:0/2694569403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf181a63a0 con 0x7faf18100f90 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf16d9d640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf18100f90 0x7faf181a5d00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7faf1000eb10 tx=0x7faf1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf0f7fe640 1 -- 172.21.15.22:0/2694569403 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10004b20 con 0x7faf18100f90 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf0f7fe640 1 -- 172.21.15.22:0/2694569403 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf10004cc0 con 0x7faf18100f90 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf181a6690 con 0x7faf18100f90 2024-03-09T00:00:21.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf0f7fe640 1 -- 172.21.15.22:0/2694569403 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10010760 con 0x7faf18100f90 2024-03-09T00:00:21.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.645+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf181a6b60 con 0x7faf18100f90 2024-03-09T00:00:21.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.647+0000 7faf0f7fe640 1 -- 172.21.15.22:0/2694569403 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7faf10010900 con 0x7faf18100f90 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.648+0000 7faf0f7fe640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faf040a2970 0x7faf040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.648+0000 7faf1659c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faf040a2970 0x7faf040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.648+0000 7faf0f7fe640 1 -- 172.21.15.22:0/2694569403 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7faf100cf7f0 con 0x7faf18100f90 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1659c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faf040a2970 0x7faf040a4e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf08005e10 tx=0x7faf08005da0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 Processor -- start 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 -- start start 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697301a0 0x55d869730580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869739160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697396a0 0x55d86973ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8696f7360 con 0x55d869730ac0 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d869693ec0 con 0x55d8697301a0 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86963cd30 con 0x55d8697396a0 2024-03-09T00:00:21.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869739160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf16d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697396a0 0x55d86973ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869739160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50522/0 (socket says 172.21.15.22:50522) 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 -- 172.21.15.22:0/4073565733 learned_addr learned my addr 172.21.15.22:0/4073565733 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf1659c640 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697301a0 0x55d869730580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 -- 172.21.15.22:0/4073565733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697396a0 msgr2=0x55d86973ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697396a0 0x55d86973ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 -- 172.21.15.22:0/4073565733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697301a0 msgr2=0x55d869730580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697301a0 0x55d869730580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf0ffff640 1 -- 172.21.15.22:0/4073565733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86963bbc0 con 0x55d869730ac0 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.649+0000 7faf16d9d640 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697396a0 0x55d86973ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.650+0000 7faf0ffff640 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869739160 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7faf180696f0 tx=0x7faf0002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d305325328446def server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.650+0000 7faf0cff9640 1 -- 172.21.15.22:0/4073565733 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00042070 con 0x55d869730ac0 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.650+0000 7faf0cff9640 1 -- 172.21.15.22:0/4073565733 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf0002ca80 con 0x55d869730ac0 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.650+0000 7faf0cff9640 1 -- 172.21.15.22:0/4073565733 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00004c90 con 0x55d869730ac0 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4073565733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 msgr2=0x55d869739160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869739160 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7faf180696f0 tx=0x7faf0002c5d0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4073565733 shutdown_connections 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697396a0 0x55d86973ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869739160 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4073565733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697301a0 0x55d869730580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4073565733 >> 172.21.15.22:0/4073565733 conn(0x55d86972a8d0 msgr2=0x55d86972acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4073565733 shutdown_connections 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4073565733 wait complete. 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- start start 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 Processor -- start 2024-03-09T00:00:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697301a0 0x55d8697366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d869730ac0 0x55d869736bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869737130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8696324a0 con 0x55d8697301a0 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86968ff60 con 0x55d8697396a0 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86973c430 con 0x55d869730ac0 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1f7e0740 10 client.0 _ll_register_callbacks cb 0x55d86968dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.651+0000 7faf1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697301a0 0x55d8697366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697301a0 0x55d8697366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50528/0 (socket says 172.21.15.22:50528) 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 -- 172.21.15.22:0/4078706530 learned_addr learned my addr 172.21.15.22:0/4078706530 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf0ffff640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d869730ac0 0x55d869736bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf16d9d640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869737130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 -- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d869730ac0 msgr2=0x55d869736bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d869730ac0 0x55d869736bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 -- 172.21.15.22:0/4078706530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 msgr2=0x55d869737130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869737130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 -- 172.21.15.22:0/4078706530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8693a83f0 con 0x55d8697301a0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf0ffff640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d869730ac0 0x55d869736bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1659c640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697301a0 0x55d8697366b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7faf0800f010 tx=0x7faf080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1b522a1af4f20d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faefdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faefdffb640 1 -- 172.21.15.22:0/4078706530 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0801de10 con 0x55d8697301a0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faefdffb640 1 -- 172.21.15.22:0/4078706530 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7faf0801e430 con 0x55d8697301a0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf0002ca80 con 0x55d8697301a0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faefdffb640 1 -- 172.21.15.22:0/4078706530 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf08031050 con 0x55d8697301a0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1f7e0740 10 client.17484 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.652+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d8694727d0 con 0x55d8697301a0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faefdffb640 1 -- 172.21.15.22:0/4078706530 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf0002ca80 con 0x55d8697301a0 2024-03-09T00:00:21.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faefdffb640 1 -- 172.21.15.22:0/4078706530 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faf08026790 con 0x55d8697301a0 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf1f7e0740 10 client.17484 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf1f7e0740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf1f7e0740 20 client.17484 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697498d0 0x55d86974bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf1f7e0740 4 client.17484 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8696a1400 con 0x55d8697498d0 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf0ffff640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697498d0 0x55d86974bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faefdffb640 10 client.17484 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.653+0000 7faf0ffff640 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697498d0 0x55d86974bd00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4532b4adf31e91e0 server_cookie=e6a83fd7c5517986 in_seq=0 out_seq=0 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faefdffb640 1 -- 172.21.15.22:0/4078706530 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8696a1400 con 0x55d8697498d0 2024-03-09T00:00:21.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faefdffb640 10 client.17484 handle_command_reply: tid=0 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 client.17484 shutdown 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697301a0 msgr2=0x55d8697366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697301a0 0x55d8697366b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7faf0800f010 tx=0x7faf080046c0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 shutdown_connections 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697498d0 0x55d86974bd00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869737130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d869730ac0 0x55d869736bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/4078706530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697301a0 0x55d8697366b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faefdffb640 0 client.17484 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 >> 172.21.15.22:0/4078706530 conn(0x55d86972a8d0 msgr2=0x55d86972c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 shutdown_connections 2024-03-09T00:00:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.654+0000 7faf1f7e0740 1 -- 172.21.15.22:0/4078706530 wait complete. 2024-03-09T00:00:21.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.655+0000 7faf1f7e0740 20 client.17484 trim_cache size 0 max 16384 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.667+0000 7faf1f7e0740 1 Processor -- start 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.667+0000 7faf1f7e0740 1 -- start start 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.667+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869738de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.667+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869735560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.667+0000 7faf0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869735560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.667+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869733930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.667+0000 7faf0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869735560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50532/0 (socket says 172.21.15.22:50532) 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf0ffff640 1 -- 172.21.15.22:0/3639541079 learned_addr learned my addr 172.21.15.22:0/3639541079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181b2bb0 con 0x55d869730ac0 2024-03-09T00:00:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181b0750 con 0x55d8697396a0 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181ae7f0 con 0x55d8697301a0 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf1659c640 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869738de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf16d9d640 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869733930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf0ffff640 1 -- 172.21.15.22:0/3639541079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 msgr2=0x55d869738de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf0ffff640 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869738de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf0ffff640 1 -- 172.21.15.22:0/3639541079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 msgr2=0x55d869733930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf0ffff640 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869733930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf0ffff640 1 -- 172.21.15.22:0/3639541079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86972f550 con 0x55d869730ac0 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faf0ffff640 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869735560 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x55d86972fc70 tx=0x7faf0002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25608d56b63cc824 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faefffff640 1 -- 172.21.15.22:0/3639541079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00042040 con 0x55d869730ac0 2024-03-09T00:00:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faefffff640 1 -- 172.21.15.22:0/3639541079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf00004b40 con 0x55d869730ac0 2024-03-09T00:00:21.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.668+0000 7faefffff640 1 -- 172.21.15.22:0/3639541079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0004d4c0 con 0x55d869730ac0 2024-03-09T00:00:21.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 msgr2=0x55d869735560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869735560 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x55d86972fc70 tx=0x7faf0002c5d0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 shutdown_connections 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8697396a0 0x55d869733930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d869730ac0 0x55d869735560 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/3639541079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869738de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 >> 172.21.15.22:0/3639541079 conn(0x55d86972a8d0 msgr2=0x55d869734e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 shutdown_connections 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 -- 172.21.15.22:0/3639541079 wait complete. 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.669+0000 7faf1f7e0740 1 -- start start 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 1 Processor -- start 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869740b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d869730ac0 0x55d86973ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697396a0 0x55d86973f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181aedf0 con 0x55d8697396a0 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181b00f0 con 0x55d869730ac0 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf181af430 con 0x55d8697301a0 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1f7e0740 10 client.0 _ll_register_callbacks cb 0x7faf00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697396a0 0x55d86973f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697396a0 0x55d86973f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50542/0 (socket says 172.21.15.22:50542) 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf0ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d869730ac0 0x55d86973ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 -- 172.21.15.22:0/2852277766 learned_addr learned my addr 172.21.15.22:0/2852277766 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf1659c640 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869740b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 -- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 msgr2=0x55d869740b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869740b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 -- 172.21.15.22:0/2852277766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d869730ac0 msgr2=0x55d86973ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d869730ac0 0x55d86973ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 -- 172.21.15.22:0/2852277766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d86979fdf0 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faf16d9d640 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697396a0 0x55d86973f6c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7faf10005880 tx=0x7faf1009c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4efece39396b1a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faefdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.670+0000 7faefdffb640 1 -- 172.21.15.22:0/2852277766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf100a4250 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faefdffb640 1 -- 172.21.15.22:0/2852277766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7faf100a3dc0 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf00004b40 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faefdffb640 1 -- 172.21.15.22:0/2852277766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf100a3fc0 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 10 client.17490 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d8697a0f80 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faefdffb640 1 -- 172.21.15.22:0/2852277766 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf00004b40 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faefdffb640 1 -- 172.21.15.22:0/2852277766 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faf100ce040 con 0x55d8697396a0 2024-03-09T00:00:21.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 10 client.17490 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 10 client.17490 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 20 client.17490 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697a16d0 0x55d8697a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 4 client.17490 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d86969b750 con 0x55d8697a16d0 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.671+0000 7faf1659c640 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697a16d0 0x55d8697a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1659c640 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697a16d0 0x55d8697a3af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecd84343a0f2c117 server_cookie=44caca3e281d7d15 in_seq=0 out_seq=0 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faefdffb640 10 client.17490 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faefdffb640 1 -- 172.21.15.22:0/2852277766 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d86969b750 con 0x55d8697a16d0 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faefdffb640 10 client.17490 handle_command_reply: tid=0 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 client.17490 shutdown 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697396a0 msgr2=0x55d86973f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697396a0 0x55d86973f6c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7faf10005880 tx=0x7faf1009c270 comp rx=0 tx=0).stop 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 shutdown_connections 2024-03-09T00:00:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8697a16d0 0x55d8697a3af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8697396a0 0x55d86973f6c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d869730ac0 0x55d86973ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faefdffb640 0 client.17490 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 --2- 172.21.15.22:0/2852277766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8697301a0 0x55d869740b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 >> 172.21.15.22:0/2852277766 conn(0x55d86972a8d0 msgr2=0x55d8697356e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 shutdown_connections 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 1 -- 172.21.15.22:0/2852277766 wait complete. 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.672+0000 7faf1f7e0740 20 client.17490 trim_cache size 0 max 16384 2024-03-09T00:00:21.676 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:00:21.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faf040a2970 msgr2=0x7faf040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faf040a2970 0x7faf040a4e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf08005e10 tx=0x7faf08005da0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf18100f90 msgr2=0x7faf181a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf18100f90 0x7faf181a5d00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7faf1000eb10 tx=0x7faf1000c2d0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 shutdown_connections 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faf040a2970 0x7faf040a4e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf18100f90 0x7faf181a5d00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf181065a0 0x7faf18100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 --2- 172.21.15.22:0/2694569403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf18069b90 0x7faf18100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 >> 172.21.15.22:0/2694569403 conn(0x7faf18076f80 msgr2=0x7faf180fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 shutdown_connections 2024-03-09T00:00:21.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.673+0000 7faf1cc0c640 1 -- 172.21.15.22:0/2694569403 wait complete. 2024-03-09T00:00:21.682 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:00:21.682 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:00:21.716 INFO:tasks.workunit.client.0.smithi022.stdout: 1 600086 11.29 MB/sec execute 149 sec latency 29.760 ms 2024-03-09T00:00:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.799+0000 7fbcc9459640 1 -- 172.21.15.22:0/2019676628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc4100d90 msgr2=0x7fbcc4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.799+0000 7fbcc9459640 1 --2- 172.21.15.22:0/2019676628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc4100d90 0x7fbcc4104dd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009900 tx=0x7fbcb802bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.799+0000 7fbcc9459640 1 -- 172.21.15.22:0/2019676628 shutdown_connections 2024-03-09T00:00:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 --2- 172.21.15.22:0/2019676628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc41054b0 0x7fbcc410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 --2- 172.21.15.22:0/2019676628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc4100d90 0x7fbcc4104dd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 --2- 172.21.15.22:0/2019676628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc41003e0 0x7fbcc41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 -- 172.21.15.22:0/2019676628 >> 172.21.15.22:0/2019676628 conn(0x7fbcc40773a0 msgr2=0x7fbcc40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 -- 172.21.15.22:0/2019676628 shutdown_connections 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 -- 172.21.15.22:0/2019676628 wait complete. 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 Processor -- start 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 -- start start 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc41003e0 0x7fbcc4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41054b0 0x7fbcc4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc4199460 0x7fbcc41a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc4110590 con 0x7fbcc41054b0 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc4110410 con 0x7fbcc4199460 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.800+0000 7fbcc9459640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc4110710 con 0x7fbcc41003e0 2024-03-09T00:00:21.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41054b0 0x7fbcc4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41054b0 0x7fbcc4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50572/0 (socket says 172.21.15.22:50572) 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 -- 172.21.15.22:0/3690467024 learned_addr learned my addr 172.21.15.22:0/3690467024 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc41003e0 0x7fbcc4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc4199460 0x7fbcc41a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 -- 172.21.15.22:0/3690467024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc41003e0 msgr2=0x7fbcc4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc41003e0 0x7fbcc4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 -- 172.21.15.22:0/3690467024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc4199460 msgr2=0x7fbcc41a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc4199460 0x7fbcc41a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 -- 172.21.15.22:0/3690467024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc41a4210 con 0x7fbcc41054b0 2024-03-09T00:00:21.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41054b0 0x7fbcc4198ed0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbcb802c240 tx=0x7fbcb8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcabfff640 1 -- 172.21.15.22:0/3690467024 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8043020 con 0x7fbcc41054b0 2024-03-09T00:00:21.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc41a44a0 con 0x7fbcc41054b0 2024-03-09T00:00:21.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcabfff640 1 -- 172.21.15.22:0/3690467024 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb80054e0 con 0x7fbcc41054b0 2024-03-09T00:00:21.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcabfff640 1 -- 172.21.15.22:0/3690467024 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8003b50 con 0x7fbcc41054b0 2024-03-09T00:00:21.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.801+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc41a4a40 con 0x7fbcc41054b0 2024-03-09T00:00:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.803+0000 7fbcabfff640 1 -- 172.21.15.22:0/3690467024 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbcb8046020 con 0x7fbcc41054b0 2024-03-09T00:00:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 Processor -- start 2024-03-09T00:00:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 -- start start 2024-03-09T00:00:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcabfff640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbcac0a2970 0x7fbcac0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f2461a0 0x557f2f246580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbcac0a2970 0x7fbcac0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f24f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f251a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcabfff640 1 -- 172.21.15.22:0/3690467024 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fbcb80f10d0 con 0x7fbcc41054b0 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2f20d360 con 0x557f2f24f6a0 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2f1a9ec0 con 0x557f2f246ac0 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbccc038740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2f152d30 con 0x557f2f2461a0 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcc37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f251a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcc37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f251a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50576/0 (socket says 172.21.15.22:50576) 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcc37fe640 1 -- 172.21.15.22:0/2655759836 learned_addr learned my addr 172.21.15.22:0/2655759836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f2461a0 0x557f2f246580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.804+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f24f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/2655759836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 msgr2=0x557f2f24f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f24f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/2655759836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 msgr2=0x557f2f251a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f251a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbcac0a2970 0x7fbcac0a4e30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0137d0 tx=0x7fbcbc013bd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/2655759836 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f2f151bc0 con 0x557f2f2461a0 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f2461a0 0x557f2f246580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009c00 tx=0x7fbcb4011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbc25e12bc66a52 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbca97fa640 1 -- 172.21.15.22:0/2655759836 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4004310 con 0x557f2f2461a0 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbca97fa640 1 -- 172.21.15.22:0/2655759836 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb40044b0 con 0x557f2f2461a0 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.805+0000 7fbca97fa640 1 -- 172.21.15.22:0/2655759836 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40035f0 con 0x557f2f2461a0 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- 172.21.15.22:0/2655759836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f2461a0 msgr2=0x557f2f246580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f2461a0 0x557f2f246580 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009c00 tx=0x7fbcb4011440 comp rx=0 tx=0).stop 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- 172.21.15.22:0/2655759836 shutdown_connections 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f251a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f24f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 --2- 172.21.15.22:0/2655759836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f2461a0 0x557f2f246580 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- 172.21.15.22:0/2655759836 >> 172.21.15.22:0/2655759836 conn(0x557f2f2408d0 msgr2=0x557f2f240cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- 172.21.15.22:0/2655759836 shutdown_connections 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- 172.21.15.22:0/2655759836 wait complete. 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- start start 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 Processor -- start 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f24c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f246ac0 0x557f2f24cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f24d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2f1484a0 con 0x557f2f246ac0 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2f1a5f60 con 0x557f2f2461a0 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2f252490 con 0x557f2f24f6a0 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.806+0000 7fbccc038740 10 client.0 _ll_register_callbacks cb 0x557f2f1a3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f246ac0 0x557f2f24cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f246ac0 0x557f2f24cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50590/0 (socket says 172.21.15.22:50590) 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc27fc640 1 -- 172.21.15.22:0/2714787311 learned_addr learned my addr 172.21.15.22:0/2714787311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f24d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f24c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc37fe640 1 -- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 msgr2=0x557f2f24c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f24c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc37fe640 1 -- 172.21.15.22:0/2714787311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f246ac0 msgr2=0x557f2f24cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f246ac0 0x557f2f24cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc37fe640 1 -- 172.21.15.22:0/2714787311 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f2eebe3f0 con 0x557f2f24f6a0 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f246ac0 0x557f2f24cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f24d090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005fd0 tx=0x7fbcbc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ec19c3dd6f15c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbca6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbca6ff5640 1 -- 172.21.15.22:0/2714787311 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc004710 con 0x557f2f24f6a0 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcb40044b0 con 0x557f2f24f6a0 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbca6ff5640 1 -- 172.21.15.22:0/2714787311 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fbcbc010430 con 0x557f2f24f6a0 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbca6ff5640 1 -- 172.21.15.22:0/2714787311 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcbc031050 con 0x557f2f24f6a0 2024-03-09T00:00:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbccc038740 10 client.26947 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.807+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557f2ef887d0 con 0x557f2f24f6a0 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbca6ff5640 1 -- 172.21.15.22:0/2714787311 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb40044b0 con 0x557f2f24f6a0 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbca6ff5640 1 -- 172.21.15.22:0/2714787311 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbcbc01f070 con 0x557f2f24f6a0 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbccc038740 10 client.26947 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbccc038740 10 client.26947 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbccc038740 20 client.26947 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbccc038740 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f261bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbccc038740 4 client.26947 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f2f1b7400 con 0x557f2f25f7d0 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f261bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f261bc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb281ddf00e582cf server_cookie=a263759ad5f0c1eb in_seq=0 out_seq=0 2024-03-09T00:00:21.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.808+0000 7fbca6ff5640 10 client.26947 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbca6ff5640 1 -- 172.21.15.22:0/2714787311 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f2f1b7400 con 0x557f2f25f7d0 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbca6ff5640 10 client.26947 handle_command_reply: tid=0 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 client.26947 shutdown 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 msgr2=0x557f2f24d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f24d090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005fd0 tx=0x7fbcbc004ab0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 shutdown_connections 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f261bc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f24d090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbca6ff5640 0 client.26947 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f246ac0 0x557f2f24cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 --2- 172.21.15.22:0/2714787311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f24c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 >> 172.21.15.22:0/2714787311 conn(0x557f2f2408d0 msgr2=0x557f2f242620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 shutdown_connections 2024-03-09T00:00:21.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 1 -- 172.21.15.22:0/2714787311 wait complete. 2024-03-09T00:00:21.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.809+0000 7fbccc038740 20 client.26947 trim_cache size 0 max 16384 2024-03-09T00:00:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.821+0000 7fbccc038740 1 Processor -- start 2024-03-09T00:00:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbccc038740 1 -- start start 2024-03-09T00:00:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbccc038740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f2461a0 0x557f2f24ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbccc038740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f249890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbccc038740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f249dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbccc038740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41b0a00 con 0x557f2f2461a0 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbccc038740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41ae5a0 con 0x557f2f246ac0 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbccc038740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41ac640 con 0x557f2f24f6a0 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f2461a0 0x557f2f24ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f249dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f2461a0 0x557f2f24ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50596/0 (socket says 172.21.15.22:50596) 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/516831385 learned_addr learned my addr 172.21.15.22:0/516831385 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f249890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/516831385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 msgr2=0x557f2f249dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f249dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/516831385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 msgr2=0x557f2f249890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f249890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/516831385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f2f261f30 con 0x557f2f2461a0 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f249dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f249890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.822+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f2461a0 0x557f2f24ed40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4014040 tx=0x7fbcb4004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae4ed69ebac4911 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbca8ff9640 1 -- 172.21.15.22:0/516831385 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4005000 con 0x557f2f2461a0 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbca8ff9640 1 -- 172.21.15.22:0/516831385 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb401f040 con 0x557f2f2461a0 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbca8ff9640 1 -- 172.21.15.22:0/516831385 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4004760 con 0x557f2f2461a0 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 -- 172.21.15.22:0/516831385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f2461a0 msgr2=0x557f2f24ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f2461a0 0x557f2f24ed40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4014040 tx=0x7fbcb4004c10 comp rx=0 tx=0).stop 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 -- 172.21.15.22:0/516831385 shutdown_connections 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f24f6a0 0x557f2f249dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f246ac0 0x557f2f249890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 --2- 172.21.15.22:0/516831385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f2461a0 0x557f2f24ed40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 -- 172.21.15.22:0/516831385 >> 172.21.15.22:0/516831385 conn(0x557f2f2408d0 msgr2=0x557f2f24aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 -- 172.21.15.22:0/516831385 shutdown_connections 2024-03-09T00:00:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.823+0000 7fbccc038740 1 -- 172.21.15.22:0/516831385 wait complete. 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 -- start start 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 Processor -- start 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f255750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f246ac0 0x557f2f255c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f2b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41acc40 con 0x557f2f24f6a0 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41adf40 con 0x557f2f2461a0 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc41ad280 con 0x557f2f246ac0 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbccc038740 10 client.0 _ll_register_callbacks cb 0x7fbcb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f255750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc27fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f246ac0 0x557f2f255c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f2b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f255750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:44982/0 (socket says 172.21.15.22:44982) 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc27fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f246ac0 0x557f2f255c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45138/0 (socket says 172.21.15.22:45138) 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc2ffd640 1 -- 172.21.15.22:0/1299132600 learned_addr learned my addr 172.21.15.22:0/1299132600 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc27fc640 1 -- 172.21.15.22:0/1299132600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 msgr2=0x557f2f255750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f255750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc27fc640 1 -- 172.21.15.22:0/1299132600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 msgr2=0x557f2f2b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.824+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f2b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbcc27fc640 1 -- 172.21.15.22:0/1299132600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f2f2b61a0 con 0x557f2f246ac0 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbcc37fe640 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f2b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbcc27fc640 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f246ac0 0x557f2f255c90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80b9900 tx=0x7fbcb802ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e068dbac06bc3c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbca6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbca6ff5640 1 -- 172.21.15.22:0/1299132600 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80da060 con 0x557f2f246ac0 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbca6ff5640 1 -- 172.21.15.22:0/1299132600 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fbcb80c1170 con 0x557f2f246ac0 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcb401f040 con 0x557f2f246ac0 2024-03-09T00:00:21.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbca6ff5640 1 -- 172.21.15.22:0/1299132600 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb802cd20 con 0x557f2f246ac0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbccc038740 10 client.26953 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557f2f2b73c0 con 0x557f2f246ac0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.825+0000 7fbca6ff5640 1 -- 172.21.15.22:0/1299132600 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb401f040 con 0x557f2f246ac0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbca6ff5640 1 -- 172.21.15.22:0/1299132600 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbcb80eebd0 con 0x557f2f246ac0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbccc038740 10 client.26953 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbccc038740 10 client.26953 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbccc038740 20 client.26953 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbccc038740 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f2b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbccc038740 4 client.26953 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f2f1b1750 con 0x557f2f25f7d0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f2b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbca6ff5640 10 client.26953 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.826+0000 7fbcc2ffd640 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f2b9b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1841bf04a7f9cd5 server_cookie=c37071558882dc2d in_seq=0 out_seq=0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbca6ff5640 1 -- 172.21.15.22:0/1299132600 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f2f1b1750 con 0x557f2f25f7d0 2024-03-09T00:00:21.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbca6ff5640 10 client.26953 handle_command_reply: tid=0 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 client.26953 shutdown 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f246ac0 msgr2=0x557f2f255c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f246ac0 0x557f2f255c90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80b9900 tx=0x7fbcb802ca90 comp rx=0 tx=0).stop 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 shutdown_connections 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x557f2f25f7d0 0x557f2f2b9b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f2f24f6a0 0x557f2f2b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f2f246ac0 0x557f2f255c90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 --2- 172.21.15.22:0/1299132600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f2f2461a0 0x557f2f255750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbca6ff5640 0 client.26953 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 >> 172.21.15.22:0/1299132600 conn(0x557f2f2408d0 msgr2=0x557f2f24c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 shutdown_connections 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 1 -- 172.21.15.22:0/1299132600 wait complete. 2024-03-09T00:00:21.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbccc038740 20 client.26953 trim_cache size 0 max 16384 2024-03-09T00:00:21.830 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbcac0a2970 msgr2=0x7fbcac0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbcac0a2970 0x7fbcac0a4e30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0137d0 tx=0x7fbcbc013bd0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41054b0 msgr2=0x7fbcc4198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41054b0 0x7fbcc4198ed0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbcb802c240 tx=0x7fbcb8004360 comp rx=0 tx=0).stop 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 shutdown_connections 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbcac0a2970 0x7fbcac0a4e30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc4199460 0x7fbcc41a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbcc41054b0 0x7fbcc4198ed0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 --2- 172.21.15.22:0/3690467024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc41003e0 0x7fbcc4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.827+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 >> 172.21.15.22:0/3690467024 conn(0x7fbcc40773a0 msgr2=0x7fbcc40fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:21.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.828+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 shutdown_connections 2024-03-09T00:00:21.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.828+0000 7fbcc9459640 1 -- 172.21.15.22:0/3690467024 wait complete. 2024-03-09T00:00:21.834 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:00:21.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:00:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 -- 172.21.15.22:0/1462430772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 msgr2=0x7fd0201018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 --2- 172.21.15.22:0/1462430772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0201018b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009900 tx=0x7fd01c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 -- 172.21.15.22:0/1462430772 shutdown_connections 2024-03-09T00:00:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 --2- 172.21.15.22:0/1462430772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0201065a0 0x7fd02010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 --2- 172.21.15.22:0/1462430772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd020101e80 0x7fd020105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 --2- 172.21.15.22:0/1462430772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0201018b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 -- 172.21.15.22:0/1462430772 >> 172.21.15.22:0/1462430772 conn(0x7fd0200fcd70 msgr2=0x7fd0200ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 -- 172.21.15.22:0/1462430772 shutdown_connections 2024-03-09T00:00:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.993+0000 7fd0276dc640 1 -- 172.21.15.22:0/1462430772 wait complete. 2024-03-09T00:00:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 Processor -- start 2024-03-09T00:00:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 -- start start 2024-03-09T00:00:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0200735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0201065a0 0x7fd020073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd020074070 0x7fd0201a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd020111650 con 0x7fd0201014d0 2024-03-09T00:00:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201114d0 con 0x7fd0201065a0 2024-03-09T00:00:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd0276dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201117d0 con 0x7fd020074070 2024-03-09T00:00:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd025478640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0200735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd025478640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0200735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50616/0 (socket says 172.21.15.22:50616) 2024-03-09T00:00:22.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd025478640 1 -- 172.21.15.22:0/4052225551 learned_addr learned my addr 172.21.15.22:0/4052225551 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.994+0000 7fd025c79640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd020074070 0x7fd0201a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd024c77640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0201065a0 0x7fd020073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd025478640 1 -- 172.21.15.22:0/4052225551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd020074070 msgr2=0x7fd0201a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd025478640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd020074070 0x7fd0201a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd025478640 1 -- 172.21.15.22:0/4052225551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0201065a0 msgr2=0x7fd020073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd025478640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0201065a0 0x7fd020073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd025478640 1 -- 172.21.15.22:0/4052225551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0201a6440 con 0x7fd0201014d0 2024-03-09T00:00:22.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd024c77640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0201065a0 0x7fd020073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:00:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd025478640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0200735f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd01c006010 tx=0x7fd01c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd0167fc640 1 -- 172.21.15.22:0/4052225551 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c041070 con 0x7fd0201014d0 2024-03-09T00:00:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd0167fc640 1 -- 172.21.15.22:0/4052225551 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd01c004450 con 0x7fd0201014d0 2024-03-09T00:00:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0201a66d0 con 0x7fd0201014d0 2024-03-09T00:00:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd0167fc640 1 -- 172.21.15.22:0/4052225551 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c003e00 con 0x7fd0201014d0 2024-03-09T00:00:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.995+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0201a6bc0 con 0x7fd0201014d0 2024-03-09T00:00:22.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 Processor -- start 2024-03-09T00:00:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 -- start start 2024-03-09T00:00:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd0167fc640 1 -- 172.21.15.22:0/4052225551 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd01c044070 con 0x7fd0201014d0 2024-03-09T00:00:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b1ac0 0x55697c8ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8ba6a0 0x55697c8bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697c878360 con 0x55697c8ba6a0 2024-03-09T00:00:22.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697c814ec0 con 0x55697c8b1ac0 2024-03-09T00:00:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.998+0000 7fd02a399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697c7bdd30 con 0x55697c8b11a0 2024-03-09T00:00:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd025478640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b1ac0 0x55697c8ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8ba6a0 0x55697c8bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd025c79640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd025478640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b1ac0 0x55697c8ba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45004/0 (socket says 172.21.15.22:45004) 2024-03-09T00:00:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8ba6a0 0x55697c8bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50630/0 (socket says 172.21.15.22:50630) 2024-03-09T00:00:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd025c79640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45164/0 (socket says 172.21.15.22:45164) 2024-03-09T00:00:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 -- 172.21.15.22:0/317824493 learned_addr learned my addr 172.21.15.22:0/317824493 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 -- 172.21.15.22:0/317824493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 msgr2=0x55697c8b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 -- 172.21.15.22:0/317824493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b1ac0 msgr2=0x55697c8ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b1ac0 0x55697c8ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 -- 172.21.15.22:0/317824493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55697c7bcbc0 con 0x55697c8ba6a0 2024-03-09T00:00:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd025478640 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b1ac0 0x55697c8ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd025c79640 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd0167fc640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd00c0a2970 0x7fd00c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd025c79640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd00c0a2970 0x7fd00c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd024c77640 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8ba6a0 0x55697c8bca90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd010009cb0 tx=0x7fd0100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeb9670d619441 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd0167fc640 1 -- 172.21.15.22:0/4052225551 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd01c0f0c30 con 0x7fd0201014d0 2024-03-09T00:00:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd00b7fe640 1 -- 172.21.15.22:0/317824493 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010012450 con 0x55697c8ba6a0 2024-03-09T00:00:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd00b7fe640 1 -- 172.21.15.22:0/317824493 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd010012ad0 con 0x55697c8ba6a0 2024-03-09T00:00:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:21.999+0000 7fd00b7fe640 1 -- 172.21.15.22:0/317824493 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000fc70 con 0x55697c8ba6a0 2024-03-09T00:00:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 -- 172.21.15.22:0/317824493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8ba6a0 msgr2=0x55697c8bca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd025c79640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd00c0a2970 0x7fd00c0a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd018009fd0 tx=0x7fd018009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8ba6a0 0x55697c8bca90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd010009cb0 tx=0x7fd0100117b0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 -- 172.21.15.22:0/317824493 shutdown_connections 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8ba6a0 0x55697c8bca90 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b1ac0 0x55697c8ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 --2- 172.21.15.22:0/317824493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 -- 172.21.15.22:0/317824493 >> 172.21.15.22:0/317824493 conn(0x55697c8ab8d0 msgr2=0x55697c8abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 -- 172.21.15.22:0/317824493 shutdown_connections 2024-03-09T00:00:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.000+0000 7fd02a399740 1 -- 172.21.15.22:0/317824493 wait complete. 2024-03-09T00:00:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 -- start start 2024-03-09T00:00:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 Processor -- start 2024-03-09T00:00:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b11a0 0x55697c8b7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b1ac0 0x55697c8b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697c7b34a0 con 0x55697c8b11a0 2024-03-09T00:00:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697c810f60 con 0x55697c8ba6a0 2024-03-09T00:00:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697c8bd490 con 0x55697c8b1ac0 2024-03-09T00:00:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd02a399740 10 client.0 _ll_register_callbacks cb 0x55697c80ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b11a0 0x55697c8b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b11a0 0x55697c8b7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50636/0 (socket says 172.21.15.22:50636) 2024-03-09T00:00:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 -- 172.21.15.22:0/4034445023 learned_addr learned my addr 172.21.15.22:0/4034445023 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd024c77640 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b1ac0 0x55697c8b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025c79640 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 -- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b1ac0 msgr2=0x55697c8b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b1ac0 0x55697c8b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 -- 172.21.15.22:0/4034445023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 msgr2=0x55697c8b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.001+0000 7fd025478640 1 -- 172.21.15.22:0/4034445023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55697c5293f0 con 0x55697c8b11a0 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd025478640 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b11a0 0x55697c8b7640 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd01c02cfd0 tx=0x7fd01c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc3ebb1e841783 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd008ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd008ff9640 1 -- 172.21.15.22:0/4034445023 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c04b070 con 0x55697c8b11a0 2024-03-09T00:00:22.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd008ff9640 1 -- 172.21.15.22:0/4034445023 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd01c0b9670 con 0x55697c8b11a0 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd010012ad0 con 0x55697c8b11a0 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd008ff9640 1 -- 172.21.15.22:0/4034445023 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01c0bcdd0 con 0x55697c8b11a0 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 10 client.17520 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55697c5f37d0 con 0x55697c8b11a0 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd008ff9640 1 -- 172.21.15.22:0/4034445023 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd010012ad0 con 0x55697c8b11a0 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd008ff9640 1 -- 172.21.15.22:0/4034445023 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd01c0bd260 con 0x55697c8b11a0 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 10 client.17520 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:22.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 20 client.17520 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c8ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 4 client.17520 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.002+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55697c822400 con 0x55697c8ca7f0 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.003+0000 7fd024c77640 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c8ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.003+0000 7fd008ff9640 10 client.17520 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.003+0000 7fd024c77640 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c8ccc20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260e82506ce1871b server_cookie=1750ef7c27f2490f in_seq=0 out_seq=0 2024-03-09T00:00:22.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd008ff9640 1 -- 172.21.15.22:0/4034445023 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55697c822400 con 0x55697c8ca7f0 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd008ff9640 10 client.17520 handle_command_reply: tid=0 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 client.17520 shutdown 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b11a0 msgr2=0x55697c8b7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b11a0 0x55697c8b7640 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd01c02cfd0 tx=0x7fd01c005770 comp rx=0 tx=0).stop 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 shutdown_connections 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c8ccc20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd008ff9640 0 client.17520 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:22.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b1ac0 0x55697c8b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 --2- 172.21.15.22:0/4034445023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b11a0 0x55697c8b7640 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 >> 172.21.15.22:0/4034445023 conn(0x55697c8ab8d0 msgr2=0x55697c8ad620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:22.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 shutdown_connections 2024-03-09T00:00:22.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 1 -- 172.21.15.22:0/4034445023 wait complete. 2024-03-09T00:00:22.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.004+0000 7fd02a399740 20 client.17520 trim_cache size 0 max 16384 2024-03-09T00:00:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 Processor -- start 2024-03-09T00:00:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 -- start start 2024-03-09T00:00:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201b2c60 con 0x55697c8b1ac0 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201b0800 con 0x55697c8ba6a0 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd02a399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201ae8a0 con 0x55697c8b11a0 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd025478640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd024c77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd025478640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45196/0 (socket says 172.21.15.22:45196) 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd025478640 1 -- 172.21.15.22:0/414090656 learned_addr learned my addr 172.21.15.22:0/414090656 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.018+0000 7fd025c79640 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd025478640 1 -- 172.21.15.22:0/414090656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 msgr2=0x55697c8b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd025478640 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd025478640 1 -- 172.21.15.22:0/414090656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 msgr2=0x55697c8b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd025478640 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd025478640 1 -- 172.21.15.22:0/414090656 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55697c8b18f0 con 0x55697c8b11a0 2024-03-09T00:00:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd025c79640 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd025478640 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b9d00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0b99d0 tx=0x7fd01c0bb510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7b00b382eabe49 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd00affd640 1 -- 172.21.15.22:0/414090656 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c04b050 con 0x55697c8b11a0 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd00affd640 1 -- 172.21.15.22:0/414090656 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd01c0bd700 con 0x55697c8b11a0 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.019+0000 7fd00affd640 1 -- 172.21.15.22:0/414090656 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0bc960 con 0x55697c8b11a0 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 -- 172.21.15.22:0/414090656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 msgr2=0x55697c8b9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b9d00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0b99d0 tx=0x7fd01c0bb510 comp rx=0 tx=0).stop 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 -- 172.21.15.22:0/414090656 shutdown_connections 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8ba6a0 0x55697c8b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 --2- 172.21.15.22:0/414090656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8b11a0 0x55697c8b9d00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 -- 172.21.15.22:0/414090656 >> 172.21.15.22:0/414090656 conn(0x55697c8ab8d0 msgr2=0x55697c8b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 -- 172.21.15.22:0/414090656 shutdown_connections 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 -- 172.21.15.22:0/414090656 wait complete. 2024-03-09T00:00:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 -- start start 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.020+0000 7fd02a399740 1 Processor -- start 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd02a399740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b11a0 0x55697c8c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd02a399740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd02a399740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8ba6a0 0x55697c920ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd02a399740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201aeea0 con 0x55697c8b1ac0 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd02a399740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201b01a0 con 0x55697c8b11a0 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd02a399740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0201af4e0 con 0x55697c8ba6a0 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd02a399740 10 client.0 _ll_register_callbacks cb 0x7fd010009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd025c79640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8ba6a0 0x55697c920ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8c0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50656/0 (socket says 172.21.15.22:50656) 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 -- 172.21.15.22:0/1090149727 learned_addr learned my addr 172.21.15.22:0/1090149727 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd025c79640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8ba6a0 0x55697c920ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45198/0 (socket says 172.21.15.22:45198) 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd025478640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b11a0 0x55697c8c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 -- 172.21.15.22:0/1090149727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8ba6a0 msgr2=0x55697c920ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8ba6a0 0x55697c920ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 -- 172.21.15.22:0/1090149727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b11a0 msgr2=0x55697c8c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b11a0 0x55697c8c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd024c77640 1 -- 172.21.15.22:0/1090149727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55697c921220 con 0x55697c8b1ac0 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.021+0000 7fd025478640 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b11a0 0x55697c8c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd024c77640 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8c0940 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd010014010 tx=0x7fd010009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4def9ef9a59ef729 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd008ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd008ff9640 1 -- 172.21.15.22:0/1090149727 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010036070 con 0x55697c8b1ac0 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd008ff9640 1 -- 172.21.15.22:0/1090149727 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fd010012660 con 0x55697c8b1ac0 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd01c0bd700 con 0x55697c8b1ac0 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd008ff9640 1 -- 172.21.15.22:0/1090149727 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01003f050 con 0x55697c8b1ac0 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 10 client.17526 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55697c922350 con 0x55697c8b1ac0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd008ff9640 1 -- 172.21.15.22:0/1090149727 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd01c0bd700 con 0x55697c8b1ac0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd008ff9640 1 -- 172.21.15.22:0/1090149727 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd010039020 con 0x55697c8b1ac0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 10 client.17526 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 20 client.17526 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c924780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 4 client.17526 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55697c81c750 con 0x55697c8ca7f0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.022+0000 7fd025478640 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c924780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.023+0000 7fd008ff9640 10 client.17526 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.023+0000 7fd025478640 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c924780 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2727e3acbb3c45a server_cookie=487c25c2fb370451 in_seq=0 out_seq=0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.023+0000 7fd008ff9640 1 -- 172.21.15.22:0/1090149727 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55697c81c750 con 0x55697c8ca7f0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.023+0000 7fd008ff9640 10 client.17526 handle_command_reply: tid=0 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.023+0000 7fd02a399740 1 client.17526 shutdown 2024-03-09T00:00:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 msgr2=0x55697c8c0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8c0940 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd010014010 tx=0x7fd010009370 comp rx=0 tx=0).stop 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 shutdown_connections 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55697c8ca7f0 0x55697c924780 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697c8ba6a0 0x55697c920ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd008ff9640 0 client.17526 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55697c8b1ac0 0x55697c8c0940 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 --2- 172.21.15.22:0/1090149727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55697c8b11a0 0x55697c8c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 >> 172.21.15.22:0/1090149727 conn(0x55697c8ab8d0 msgr2=0x55697c8b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 shutdown_connections 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 1 -- 172.21.15.22:0/1090149727 wait complete. 2024-03-09T00:00:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd02a399740 20 client.17526 trim_cache size 0 max 16384 2024-03-09T00:00:22.035 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:00:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd00c0a2970 msgr2=0x7fd00c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd00c0a2970 0x7fd00c0a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd018009fd0 tx=0x7fd018009500 comp rx=0 tx=0).stop 2024-03-09T00:00:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 msgr2=0x7fd0200735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0200735f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd01c006010 tx=0x7fd01c02cb30 comp rx=0 tx=0).stop 2024-03-09T00:00:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 shutdown_connections 2024-03-09T00:00:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd00c0a2970 0x7fd00c0a4e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd020074070 0x7fd0201a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0201065a0 0x7fd020073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 --2- 172.21.15.22:0/4052225551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0201014d0 0x7fd0200735f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.024+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 >> 172.21.15.22:0/4052225551 conn(0x7fd0200fcd70 msgr2=0x7fd0200fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.025+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 shutdown_connections 2024-03-09T00:00:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:22.025+0000 7fd0276dc640 1 -- 172.21.15.22:0/4052225551 wait complete. 2024-03-09T00:00:22.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:22 smithi062 ceph-mon[36129]: pgmap v181: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 427 op/s 2024-03-09T00:00:22.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:22 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4272292732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:00:22.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:22 smithi192 ceph-mon[35995]: pgmap v181: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 427 op/s 2024-03-09T00:00:22.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:22 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4272292732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:00:22.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:22 smithi022 ceph-mon[31894]: pgmap v181: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 427 op/s 2024-03-09T00:00:22.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:22 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4272292732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:00:22.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 601680 11.28 MB/sec execute 150 sec latency 49.351 ms 2024-03-09T00:00:23.038 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:00:23.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:00:23.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:00:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.157+0000 7f14c3873640 1 -- 172.21.15.22:0/2824979897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc069fd0 msgr2=0x7f14bc10e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.157+0000 7f14c3873640 1 --2- 172.21.15.22:0/2824979897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc069fd0 0x7f14bc10e100 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009ca0 tx=0x7f14b402c140 comp rx=0 tx=0).stop 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.157+0000 7f14c3873640 1 -- 172.21.15.22:0/2824979897 shutdown_connections 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.157+0000 7f14c3873640 1 --2- 172.21.15.22:0/2824979897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc069fd0 0x7f14bc10e100 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.157+0000 7f14c3873640 1 --2- 172.21.15.22:0/2824979897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14bc069630 0x7f14bc069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.157+0000 7f14c3873640 1 --2- 172.21.15.22:0/2824979897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14bc106290 0x7f14bc106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.157+0000 7f14c3873640 1 -- 172.21.15.22:0/2824979897 >> 172.21.15.22:0/2824979897 conn(0x7f14bc077280 msgr2=0x7f14bc077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 -- 172.21.15.22:0/2824979897 shutdown_connections 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 -- 172.21.15.22:0/2824979897 wait complete. 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 Processor -- start 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 -- start start 2024-03-09T00:00:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14bc069630 0x7f14bc19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14bc069fd0 0x7f14bc19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc106290 0x7f14bc197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc111730 con 0x7f14bc106290 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1115b0 con 0x7f14bc069fd0 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c3873640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1118b0 con 0x7f14bc069630 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c1e10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc106290 0x7f14bc197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c160f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14bc069630 0x7f14bc19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c1e10640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc106290 0x7f14bc197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50670/0 (socket says 172.21.15.22:50670) 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c1e10640 1 -- 172.21.15.22:0/4247592795 learned_addr learned my addr 172.21.15.22:0/4247592795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.158+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14bc069fd0 0x7f14bc19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c1e10640 1 -- 172.21.15.22:0/4247592795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14bc069630 msgr2=0x7f14bc19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c1e10640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14bc069630 0x7f14bc19d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c1e10640 1 -- 172.21.15.22:0/4247592795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14bc069fd0 msgr2=0x7f14bc19dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c1e10640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14bc069fd0 0x7f14bc19dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c1e10640 1 -- 172.21.15.22:0/4247592795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc198190 con 0x7f14bc106290 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c160f640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14bc069630 0x7f14bc19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14bc069fd0 0x7f14bc19dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c1e10640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc106290 0x7f14bc197a10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f14b402c750 tx=0x7f14b40346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14b27fc640 1 -- 172.21.15.22:0/4247592795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4039050 con 0x7f14bc106290 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14b27fc640 1 -- 172.21.15.22:0/4247592795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14b4045070 con 0x7f14bc106290 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc198420 con 0x7f14bc106290 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14b27fc640 1 -- 172.21.15.22:0/4247592795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40045a0 con 0x7f14bc106290 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.159+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14bc1a4720 con 0x7f14bc106290 2024-03-09T00:00:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.161+0000 7f14b27fc640 1 -- 172.21.15.22:0/4247592795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f14b4003970 con 0x7f14bc106290 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.162+0000 7f14c6455740 1 Processor -- start 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.162+0000 7f14c6455740 1 -- start start 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.162+0000 7f14c6455740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.162+0000 7f14c6455740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c26a0 0x55f4402cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c6455740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c6455740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f440225b40 con 0x55f4402cb280 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c6455740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4401ced80 con 0x55f4402c26a0 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c6455740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f440221be0 con 0x55f4402c1d80 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14b27fc640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14a80a29f0 0x7f14a80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c160f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c26a0 0x55f4402cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402c2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45206/0 (socket says 172.21.15.22:45206) 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c160f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c26a0 0x55f4402cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45058/0 (socket says 172.21.15.22:45058) 2024-03-09T00:00:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3684435312 learned_addr learned my addr 172.21.15.22:0/3684435312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c1e10640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14a80a29f0 0x7f14a80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14b27fc640 1 -- 172.21.15.22:0/4247592795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f14b40f1910 con 0x7f14bc106290 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3684435312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c26a0 msgr2=0x55f4402cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c26a0 0x55f4402cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3684435312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 msgr2=0x55f4402cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3684435312 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4400027e0 con 0x55f4402c1d80 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402c2160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14ac011bb0 tx=0x7f14ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c832d0eddfb18d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14c1e10640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14a80a29f0 0x7f14a80a4eb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f14b4005e30 tx=0x7f14b40095c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.163+0000 7f14a77fe640 1 -- 172.21.15.22:0/3684435312 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00fc50 con 0x55f4402c1d80 2024-03-09T00:00:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14a77fe640 1 -- 172.21.15.22:0/3684435312 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14ac00fdf0 con 0x55f4402c1d80 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14a77fe640 1 -- 172.21.15.22:0/3684435312 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00e3e0 con 0x55f4402c1d80 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 -- 172.21.15.22:0/3684435312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 msgr2=0x55f4402c2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402c2160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14ac011bb0 tx=0x7f14ac0046c0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 -- 172.21.15.22:0/3684435312 shutdown_connections 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c26a0 0x55f4402cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 --2- 172.21.15.22:0/3684435312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402c2160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 -- 172.21.15.22:0/3684435312 >> 172.21.15.22:0/3684435312 conn(0x55f4402bc4b0 msgr2=0x55f4402bc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 -- 172.21.15.22:0/3684435312 shutdown_connections 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.164+0000 7f14c6455740 1 -- 172.21.15.22:0/3684435312 wait complete. 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 -- start start 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 Processor -- start 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c26a0 0x55f4402c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4402ce070 con 0x55f4402cb280 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4401e21c0 con 0x55f4402c1d80 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4402ce1f0 con 0x55f4402c26a0 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c6455740 10 client.0 _ll_register_callbacks cb 0x55f4401d17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c0e0e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c160f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c26a0 0x55f4402c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c0e0e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50680/0 (socket says 172.21.15.22:50680) 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3643072746 learned_addr learned my addr 172.21.15.22:0/3643072746 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.165+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3643072746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c26a0 msgr2=0x55f4402c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c26a0 0x55f4402c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3643072746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 msgr2=0x55f4402cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c0e0e640 1 -- 172.21.15.22:0/3643072746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f440062f70 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402c8560 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005780 tx=0x7f14ac003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7983556654abe804 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3643072746 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac005020 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3643072746 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f14ac0051c0 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14ac00fdf0 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3643072746 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ac033050 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c6455740 10 client.17538 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f43fefa170 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3643072746 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14ac00fdf0 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.166+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3643072746 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f14ac003de0 con 0x55f4402cb280 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c6455740 10 client.17538 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:23.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c6455740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c6455740 20 client.17538 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c6455740 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f4402dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c6455740 4 client.17538 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4401fb960 con 0x55f4402db2c0 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c160f640 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f4402dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14a4ff9640 10 client.17538 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.167+0000 7f14c160f640 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f4402dd6b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab77f09ff0d02b53 server_cookie=31347faf96846d00 in_seq=0 out_seq=0 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3643072746 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4401fb960 con 0x55f4402db2c0 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14a4ff9640 10 client.17538 handle_command_reply: tid=0 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 client.17538 shutdown 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 msgr2=0x55f4402c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402c8560 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005780 tx=0x7f14ac003c30 comp rx=0 tx=0).stop 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 shutdown_connections 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f4402dd6b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402cb280 0x55f4402c8560 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c26a0 0x55f4402c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 --2- 172.21.15.22:0/3643072746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14a4ff9640 0 client.17538 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 >> 172.21.15.22:0/3643072746 conn(0x55f4402bc4b0 msgr2=0x55f4402be200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 shutdown_connections 2024-03-09T00:00:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 1 -- 172.21.15.22:0/3643072746 wait complete. 2024-03-09T00:00:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.168+0000 7f14c6455740 20 client.17538 trim_cache size 0 max 16384 2024-03-09T00:00:23.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 Processor -- start 2024-03-09T00:00:23.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 -- start start 2024-03-09T00:00:23.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402cb280 0x55f4402d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1b0930 con 0x55f4402c26a0 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1ae4d0 con 0x55f4402cb280 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c6455740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1ac570 con 0x55f4402c1d80 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50688/0 (socket says 172.21.15.22:50688) 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 -- 172.21.15.22:0/1540917553 learned_addr learned my addr 172.21.15.22:0/1540917553 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402cb280 0x55f4402d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 -- 172.21.15.22:0/1540917553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 msgr2=0x55f4402ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 -- 172.21.15.22:0/1540917553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402cb280 msgr2=0x55f4402d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402cb280 0x55f4402d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c160f640 1 -- 172.21.15.22:0/1540917553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4402c24d0 con 0x55f4402c26a0 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.184+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402cb280 0x55f4402d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.185+0000 7f14c160f640 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402cac30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x55f4402c7820 tx=0x7f14b8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12031376c7d9d987 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.185+0000 7f14a6ffd640 1 -- 172.21.15.22:0/1540917553 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b801f6c0 con 0x55f4402c26a0 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.185+0000 7f14a6ffd640 1 -- 172.21.15.22:0/1540917553 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14b801f860 con 0x55f4402c26a0 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.185+0000 7f14a6ffd640 1 -- 172.21.15.22:0/1540917553 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8010430 con 0x55f4402c26a0 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.185+0000 7f14c6455740 1 -- 172.21.15.22:0/1540917553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 msgr2=0x55f4402cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.185+0000 7f14c6455740 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402cac30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x55f4402c7820 tx=0x7f14b8005f90 comp rx=0 tx=0).stop 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- 172.21.15.22:0/1540917553 shutdown_connections 2024-03-09T00:00:23.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402cb280 0x55f4402d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402cac30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 --2- 172.21.15.22:0/1540917553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402c1d80 0x55f4402ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- 172.21.15.22:0/1540917553 >> 172.21.15.22:0/1540917553 conn(0x55f4402bc4b0 msgr2=0x55f4402c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- 172.21.15.22:0/1540917553 shutdown_connections 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- 172.21.15.22:0/1540917553 wait complete. 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- start start 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 Processor -- start 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402cb280 0x55f4402c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1acb70 con 0x55f4402c26a0 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1ade70 con 0x55f4402c1d80 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1ad1b0 con 0x55f4402cb280 2024-03-09T00:00:23.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.186+0000 7f14c6455740 10 client.0 _ll_register_callbacks cb 0x7f14ac009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c0e0e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50690/0 (socket says 172.21.15.22:50690) 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 -- 172.21.15.22:0/3814934601 learned_addr learned my addr 172.21.15.22:0/3814934601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402cb280 0x55f4402c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 -- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402cb280 msgr2=0x55f4402c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402cb280 0x55f4402c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 -- 172.21.15.22:0/3814934601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 msgr2=0x55f4402d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 -- 172.21.15.22:0/3814934601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f440335a60 con 0x55f4402c26a0 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c160f640 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402c4b20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f14b8025000 tx=0x7f14b8012050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb99b86be3fd7c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:23.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3814934601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8036060 con 0x55f4402c26a0 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14b801f860 con 0x55f4402c26a0 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3814934601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f14b8004700 con 0x55f4402c26a0 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3814934601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14b803d050 con 0x55f4402c26a0 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c6455740 10 client.17544 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.187+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f440336c00 con 0x55f4402c26a0 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3814934601 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14b801f860 con 0x55f4402c26a0 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3814934601 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f14b8010550 con 0x55f4402c26a0 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c6455740 10 client.17544 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:23.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c6455740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c6455740 20 client.17544 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c6455740 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f440339290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c6455740 4 client.17544 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f4401e7080 con 0x55f4402db2c0 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f440339290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14c0e0e640 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f440339290 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5081abe82074c23d server_cookie=17a44de0e958fff1 in_seq=0 out_seq=0 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.188+0000 7f14a4ff9640 10 client.17544 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.189+0000 7f14a4ff9640 1 -- 172.21.15.22:0/3814934601 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f4401e7080 con 0x55f4402db2c0 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14a4ff9640 10 client.17544 handle_command_reply: tid=0 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 client.17544 shutdown 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 msgr2=0x55f4402c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402c4b20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f14b8025000 tx=0x7f14b8012050 comp rx=0 tx=0).stop 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 shutdown_connections 2024-03-09T00:00:23.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4402db2c0 0x55f440339290 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4402cb280 0x55f4402c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14a4ff9640 0 client.17544 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4402c26a0 0x55f4402c4b20 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 --2- 172.21.15.22:0/3814934601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4402c1d80 0x55f4402d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 >> 172.21.15.22:0/3814934601 conn(0x55f4402bc4b0 msgr2=0x55f4402bd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 shutdown_connections 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 1 -- 172.21.15.22:0/3814934601 wait complete. 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.190+0000 7f14c6455740 20 client.17544 trim_cache size 0 max 16384 2024-03-09T00:00:23.207 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:00:23.208 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:00:23.208 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "6d1a8151-82f4-4fe5-8d33-a0c427fae481", 2024-03-09T00:00:23.208 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:00:23.208 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:00:23.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14a80a29f0 msgr2=0x7f14a80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14a80a29f0 0x7f14a80a4eb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f14b4005e30 tx=0x7f14b40095c0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc106290 msgr2=0x7f14bc197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc106290 0x7f14bc197a10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f14b402c750 tx=0x7f14b40346f0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 shutdown_connections 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f14a80a29f0 0x7f14a80a4eb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14bc106290 0x7f14bc197a10 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14bc069fd0 0x7f14bc19dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 --2- 172.21.15.22:0/4247592795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14bc069630 0x7f14bc19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 >> 172.21.15.22:0/4247592795 conn(0x7f14bc077280 msgr2=0x7f14bc0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 shutdown_connections 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.191+0000 7f14c3873640 1 -- 172.21.15.22:0/4247592795 wait complete. 2024-03-09T00:00:23.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:00:23.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:00:23.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 -- 172.21.15.22:0/2271148402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39481003d0 msgr2=0x7f39481007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 --2- 172.21.15.22:0/2271148402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39481003d0 0x7f39481007b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3938009900 tx=0x7f393802bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:23.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 -- 172.21.15.22:0/2271148402 shutdown_connections 2024-03-09T00:00:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 --2- 172.21.15.22:0/2271148402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39481054a0 0x7f394810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 --2- 172.21.15.22:0/2271148402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3948100d80 0x7f3948104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 --2- 172.21.15.22:0/2271148402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f39481003d0 0x7f39481007b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 -- 172.21.15.22:0/2271148402 >> 172.21.15.22:0/2271148402 conn(0x7f39480772f0 msgr2=0x7f3948077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 -- 172.21.15.22:0/2271148402 shutdown_connections 2024-03-09T00:00:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.338+0000 7f394db53640 1 -- 172.21.15.22:0/2271148402 wait complete. 2024-03-09T00:00:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 Processor -- start 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 -- start start 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39481003d0 0x7f3948199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39481054a0 0x7f394819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394819e670 0x7f394819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948110550 con 0x7f394819e670 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481103d0 con 0x7f39481003d0 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f394db53640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481106d0 con 0x7f39481054a0 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f3947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394819e670 0x7f394819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39481054a0 0x7f394819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f39477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39481003d0 0x7f3948199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39481054a0 0x7f394819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45244/0 (socket says 172.21.15.22:45244) 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f3947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394819e670 0x7f394819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50702/0 (socket says 172.21.15.22:50702) 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.339+0000 7f3946ffd640 1 -- 172.21.15.22:0/1625017180 learned_addr learned my addr 172.21.15.22:0/1625017180 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3946ffd640 1 -- 172.21.15.22:0/1625017180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39481003d0 msgr2=0x7f3948199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3946ffd640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39481003d0 0x7f3948199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3946ffd640 1 -- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394819e670 msgr2=0x7f394819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3946ffd640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394819e670 0x7f394819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3946ffd640 1 -- 172.21.15.22:0/1625017180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394819f160 con 0x7f39481054a0 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f39477fe640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39481003d0 0x7f3948199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3946ffd640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39481054a0 0x7f394819a2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e9f0 tx=0x7f393c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3944ff9640 1 -- 172.21.15.22:0/1625017180 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00ccc0 con 0x7f39481054a0 2024-03-09T00:00:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394819f450 con 0x7f39481054a0 2024-03-09T00:00:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3944ff9640 1 -- 172.21.15.22:0/1625017180 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f393c005500 con 0x7f39481054a0 2024-03-09T00:00:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f3944ff9640 1 -- 172.21.15.22:0/1625017180 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c010690 con 0x7f39481054a0 2024-03-09T00:00:23.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.340+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39481a6c40 con 0x7f39481054a0 2024-03-09T00:00:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 Processor -- start 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 -- start start 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3944ff9640 1 -- 172.21.15.22:0/1625017180 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f393c0046c0 con 0x7f39481054a0 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b468546b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468547080 0x55b46854f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b468552050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4684a9e00 con 0x55b46854fc60 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b468452b10 con 0x55b468546760 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.343+0000 7f3950813740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4684a5ea0 con 0x55b468547080 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b468546b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f39477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468547080 0x55b46854f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b468552050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b468546b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45118/0 (socket says 172.21.15.22:45118) 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f39477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468547080 0x55b46854f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45260/0 (socket says 172.21.15.22:45260) 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 -- 172.21.15.22:0/1066264568 learned_addr learned my addr 172.21.15.22:0/1066264568 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3944ff9640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39300a2970 0x7f39300a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 -- 172.21.15.22:0/1066264568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468547080 msgr2=0x55b46854f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 --2- 172.21.15.22:0/1066264568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468547080 0x55b46854f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 -- 172.21.15.22:0/1066264568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 msgr2=0x55b468552050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 --2- 172.21.15.22:0/1066264568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b468552050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 -- 172.21.15.22:0/1066264568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4682887d0 con 0x55b468546760 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3944ff9640 1 -- 172.21.15.22:0/1625017180 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f393c0cf570 con 0x7f39481054a0 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39300a2970 0x7f39300a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f3947fff640 1 --2- 172.21.15.22:0/1066264568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b468546b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3940009f50 tx=0x7f39400118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51ef0390fe31da server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f392dffb640 1 -- 172.21.15.22:0/1066264568 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000f680 con 0x55b468546760 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f392dffb640 1 -- 172.21.15.22:0/1066264568 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3940003cb0 con 0x55b468546760 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.344+0000 7f392dffb640 1 -- 172.21.15.22:0/1066264568 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394001bb30 con 0x55b468546760 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 -- 172.21.15.22:0/1066264568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 msgr2=0x55b468546b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3947fff640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39300a2970 0x7f39300a4e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3940012fd0 tx=0x7f3940021000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 --2- 172.21.15.22:0/1066264568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b468546b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3940009f50 tx=0x7f39400118d0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 -- 172.21.15.22:0/1066264568 shutdown_connections 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 --2- 172.21.15.22:0/1066264568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b468552050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 --2- 172.21.15.22:0/1066264568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468547080 0x55b46854f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 --2- 172.21.15.22:0/1066264568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b468546b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 -- 172.21.15.22:0/1066264568 >> 172.21.15.22:0/1066264568 conn(0x55b468540e90 msgr2=0x55b468541270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 -- 172.21.15.22:0/1066264568 shutdown_connections 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.345+0000 7f3950813740 1 -- 172.21.15.22:0/1066264568 wait complete. 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 -- start start 2024-03-09T00:00:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 Processor -- start 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b46854f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b468547080 0x55b46854c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b46854fc60 0x55b46854cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b468552a50 con 0x55b468547080 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4684660a0 con 0x55b468546760 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b468552bd0 con 0x55b46854fc60 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3950813740 10 client.0 _ll_register_callbacks cb 0x55b468465a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f39477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b46854f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b468547080 0x55b46854c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f3947fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b46854fc60 0x55b46854cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f39477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b46854f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45128/0 (socket says 172.21.15.22:45128) 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.346+0000 7f39477fe640 1 -- 172.21.15.22:0/2490116095 learned_addr learned my addr 172.21.15.22:0/2490116095 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3946ffd640 1 -- 172.21.15.22:0/2490116095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b46854fc60 msgr2=0x55b46854cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3946ffd640 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b46854fc60 0x55b46854cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3946ffd640 1 -- 172.21.15.22:0/2490116095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 msgr2=0x55b46854f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3946ffd640 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b46854f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3946ffd640 1 -- 172.21.15.22:0/2490116095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4681be3f0 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3946ffd640 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b468547080 0x55b46854c6f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f393c0cc920 tx=0x7f393c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eee7d2edb019abe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3926ffd640 1 -- 172.21.15.22:0/2490116095 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c09b600 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3926ffd640 1 -- 172.21.15.22:0/2490116095 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f393c0cf170 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4684b7340 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3926ffd640 1 -- 172.21.15.22:0/2490116095 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f393c0b4070 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3950813740 10 client.17556 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b46817e0f0 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.347+0000 7f3926ffd640 1 -- 172.21.15.22:0/2490116095 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b4684b7340 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3926ffd640 1 -- 172.21.15.22:0/2490116095 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f393c09ba90 con 0x55b468547080 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3950813740 10 client.17556 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3950813740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3950813740 20 client.17556 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3950813740 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b46855f710 0x55b468561b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3950813740 4 client.17556 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4684b1690 con 0x55b46855f710 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f39477fe640 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b46855f710 0x55b468561b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f39477fe640 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b46855f710 0x55b468561b00 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0db67a1457609ba server_cookie=fca92d5b1b20c441 in_seq=0 out_seq=0 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.348+0000 7f3926ffd640 10 client.17556 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3926ffd640 1 -- 172.21.15.22:0/2490116095 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4684b1690 con 0x55b46855f710 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3926ffd640 10 client.17556 handle_command_reply: tid=0 2024-03-09T00:00:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 client.17556 shutdown 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b468547080 msgr2=0x55b46854c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b468547080 0x55b46854c6f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f393c0cc920 tx=0x7f393c0057b0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 shutdown_connections 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b46855f710 0x55b468561b00 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b46854fc60 0x55b46854cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b468547080 0x55b46854c6f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3926ffd640 0 client.17556 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 --2- 172.21.15.22:0/2490116095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b468546760 0x55b46854f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 >> 172.21.15.22:0/2490116095 conn(0x55b468540e90 msgr2=0x55b468542be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 shutdown_connections 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 1 -- 172.21.15.22:0/2490116095 wait complete. 2024-03-09T00:00:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.349+0000 7f3950813740 20 client.17556 trim_cache size 0 max 16384 2024-03-09T00:00:23.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 Processor -- start 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 -- start start 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b46854edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854fc60 0x55b46854f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854cdc0 0x55b468549970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481b2d20 con 0x55b46854cdc0 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481b08c0 con 0x55b46854fc60 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3950813740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481ae960 con 0x55b468546760 2024-03-09T00:00:23.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854cdc0 0x55b468549970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f39477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b46854edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3947fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854cdc0 0x55b468549970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50722/0 (socket says 172.21.15.22:50722) 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854fc60 0x55b46854f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.366+0000 7f3947fff640 1 -- 172.21.15.22:0/3622887503 learned_addr learned my addr 172.21.15.22:0/3622887503 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f3946ffd640 1 -- 172.21.15.22:0/3622887503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 msgr2=0x55b46854edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f3946ffd640 1 --2- 172.21.15.22:0/3622887503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b46854edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f3946ffd640 1 -- 172.21.15.22:0/3622887503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854cdc0 msgr2=0x55b468549970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f3946ffd640 1 --2- 172.21.15.22:0/3622887503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854cdc0 0x55b468549970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f3946ffd640 1 -- 172.21.15.22:0/3622887503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46854ac40 con 0x55b46854fc60 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f3946ffd640 1 --2- 172.21.15.22:0/3622887503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854fc60 0x55b46854f2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55b46854d780 tx=0x7f393c09be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b7c8d9beeb486 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f392d7fa640 1 -- 172.21.15.22:0/3622887503 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c09b190 con 0x55b46854fc60 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f392d7fa640 1 -- 172.21.15.22:0/3622887503 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f393c09bf30 con 0x55b46854fc60 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.367+0000 7f392d7fa640 1 -- 172.21.15.22:0/3622887503 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c09b370 con 0x55b46854fc60 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 -- 172.21.15.22:0/3622887503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854fc60 msgr2=0x55b46854f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 --2- 172.21.15.22:0/3622887503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854fc60 0x55b46854f2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55b46854d780 tx=0x7f393c09be50 comp rx=0 tx=0).stop 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 -- 172.21.15.22:0/3622887503 shutdown_connections 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 --2- 172.21.15.22:0/3622887503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854cdc0 0x55b468549970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 --2- 172.21.15.22:0/3622887503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854fc60 0x55b46854f2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 --2- 172.21.15.22:0/3622887503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b46854edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 -- 172.21.15.22:0/3622887503 >> 172.21.15.22:0/3622887503 conn(0x55b468540e90 msgr2=0x55b46854c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 -- 172.21.15.22:0/3622887503 shutdown_connections 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.368+0000 7f3950813740 1 -- 172.21.15.22:0/3622887503 wait complete. 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 -- start start 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 Processor -- start 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b4685572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854cdc0 0x55b468554df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b4685553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481aef60 con 0x55b46854fc60 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481b0260 con 0x55b46854cdc0 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481af5a0 con 0x55b468546760 2024-03-09T00:00:23.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3950813740 10 client.0 _ll_register_callbacks cb 0x7f393c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854cdc0 0x55b468554df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f39477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b4685572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854cdc0 0x55b468554df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45148/0 (socket says 172.21.15.22:45148) 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3946ffd640 1 -- 172.21.15.22:0/2057832905 learned_addr learned my addr 172.21.15.22:0/2057832905 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.369+0000 7f3947fff640 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b4685553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f39477fe640 1 -- 172.21.15.22:0/2057832905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854cdc0 msgr2=0x55b468554df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f39477fe640 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854cdc0 0x55b468554df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f39477fe640 1 -- 172.21.15.22:0/2057832905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 msgr2=0x55b4685553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f39477fe640 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b4685553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f39477fe640 1 -- 172.21.15.22:0/2057832905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b468555be0 con 0x55b468546760 2024-03-09T00:00:23.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3946ffd640 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854cdc0 0x55b468554df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f39477fe640 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b4685572a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3938005c40 tx=0x7f3938004d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5770b983f7f015 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3926ffd640 1 -- 172.21.15.22:0/2057832905 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393802c880 con 0x55b468546760 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3926ffd640 1 -- 172.21.15.22:0/2057832905 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f393802ca20 con 0x55b468546760 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f393c09bf30 con 0x55b468546760 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3926ffd640 1 -- 172.21.15.22:0/2057832905 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f393805a050 con 0x55b468546760 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3950813740 10 client.27007 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.370+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b4685b7c80 con 0x55b468546760 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3926ffd640 1 -- 172.21.15.22:0/2057832905 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f393c09bf30 con 0x55b468546760 2024-03-09T00:00:23.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3926ffd640 1 -- 172.21.15.22:0/2057832905 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3938005150 con 0x55b468546760 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3950813740 10 client.27007 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3950813740 10 client.27007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3950813740 20 client.27007 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3950813740 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b4685b83d0 0x55b4685ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3950813740 4 client.27007 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b46842f4e0 con 0x55b4685b83d0 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3946ffd640 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b4685b83d0 0x55b4685ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3926ffd640 10 client.27007 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.371+0000 7f3946ffd640 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b4685b83d0 0x55b4685ba7f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e3bcbfc96b258e server_cookie=6525144eb61dba2f in_seq=0 out_seq=0 2024-03-09T00:00:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3926ffd640 1 -- 172.21.15.22:0/2057832905 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b46842f4e0 con 0x55b4685b83d0 2024-03-09T00:00:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3926ffd640 10 client.27007 handle_command_reply: tid=0 2024-03-09T00:00:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 client.27007 shutdown 2024-03-09T00:00:23.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 msgr2=0x55b4685572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b4685572a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3938005c40 tx=0x7f3938004d90 comp rx=0 tx=0).stop 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 shutdown_connections 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b4685b83d0 0x55b4685ba7f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b46854fc60 0x55b4685553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3926ffd640 0 client.27007 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b46854cdc0 0x55b468554df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 --2- 172.21.15.22:0/2057832905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b468546760 0x55b4685572a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 >> 172.21.15.22:0/2057832905 conn(0x55b468540e90 msgr2=0x55b4685475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 shutdown_connections 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 1 -- 172.21.15.22:0/2057832905 wait complete. 2024-03-09T00:00:23.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.373+0000 7f3950813740 20 client.27007 trim_cache size 0 max 16384 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: "6d1a8151-82f4-4fe5-8d33-a0c427fae481": { 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "6d1a8151-82f4-4fe5-8d33-a0c427fae481", 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:00:23.378 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:00:23.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39300a2970 msgr2=0x7f39300a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39300a2970 0x7f39300a4e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3940012fd0 tx=0x7f3940021000 comp rx=0 tx=0).stop 2024-03-09T00:00:23.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39481054a0 msgr2=0x7f394819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39481054a0 0x7f394819a2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e9f0 tx=0x7f393c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 shutdown_connections 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39300a2970 0x7f39300a4e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394819e670 0x7f394819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39481054a0 0x7f394819a2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 --2- 172.21.15.22:0/1625017180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39481003d0 0x7f3948199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 >> 172.21.15.22:0/1625017180 conn(0x7f39480772f0 msgr2=0x7f39480fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 shutdown_connections 2024-03-09T00:00:23.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:23.374+0000 7f394db53640 1 -- 172.21.15.22:0/1625017180 wait complete. 2024-03-09T00:00:23.381 INFO:tasks.cephfs.filesystem:scrub status for tag:6d1a8151-82f4-4fe5-8d33-a0c427fae481 - {'path': '/', 'tag': '6d1a8151-82f4-4fe5-8d33-a0c427fae481', 'options': 'recursive,force'} 2024-03-09T00:00:23.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 604139 11.29 MB/sec execute 151 sec latency 12.495 ms 2024-03-09T00:00:24.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:24 smithi062 ceph-mon[36129]: pgmap v182: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-03-09T00:00:24.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:24 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:00:24.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:24 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:24.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:24 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:00:24.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:24 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:24 smithi192 ceph-mon[35995]: pgmap v182: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-03-09T00:00:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:24 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:00:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:24 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:24 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:00:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:24 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:24 smithi022 ceph-mon[31894]: pgmap v182: 129 pgs: 129 active+clean; 79 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-03-09T00:00:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:24 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:00:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:24 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:24 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:00:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:24 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:24.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 605495 11.25 MB/sec execute 152 sec latency 96.463 ms 2024-03-09T00:00:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 -- 172.21.15.22:0/3060718437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 msgr2=0x7f6ba810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 --2- 172.21.15.22:0/3060718437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba810d020 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009ca0 tx=0x7f6ba002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 -- 172.21.15.22:0/3060718437 shutdown_connections 2024-03-09T00:00:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 --2- 172.21.15.22:0/3060718437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba810d020 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 --2- 172.21.15.22:0/3060718437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ba8100440 0x7f6ba81008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 --2- 172.21.15.22:0/3060718437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba8105190 0x7f6ba8105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 -- 172.21.15.22:0/3060718437 >> 172.21.15.22:0/3060718437 conn(0x7f6ba8077040 msgr2=0x7f6ba8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 -- 172.21.15.22:0/3060718437 shutdown_connections 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.766+0000 7f6baf612640 1 -- 172.21.15.22:0/3060718437 wait complete. 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 Processor -- start 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 -- start start 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba8100440 0x7f6ba8101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ba8105190 0x7f6ba81028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8110650 con 0x7f6ba8100e00 2024-03-09T00:00:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81104d0 con 0x7f6ba8105190 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6baf612640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81107d0 con 0x7f6ba8100440 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6bacbad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6bacbad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba81023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50734/0 (socket says 172.21.15.22:50734) 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba8100440 0x7f6ba8101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.767+0000 7f6bacbad640 1 -- 172.21.15.22:0/3062322343 learned_addr learned my addr 172.21.15.22:0/3062322343 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6badbaf640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ba8105190 0x7f6ba81028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6bacbad640 1 -- 172.21.15.22:0/3062322343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba8100440 msgr2=0x7f6ba8101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6bacbad640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba8100440 0x7f6ba8101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6bacbad640 1 -- 172.21.15.22:0/3062322343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ba8105190 msgr2=0x7f6ba81028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6bacbad640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ba8105190 0x7f6ba81028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6bacbad640 1 -- 172.21.15.22:0/3062322343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba81a60b0 con 0x7f6ba8100e00 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba8100440 0x7f6ba8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6badbaf640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ba8105190 0x7f6ba81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6bacbad640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba81023a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9f0 tx=0x7f6b9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6b9e7fc640 1 -- 172.21.15.22:0/3062322343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800cd80 con 0x7f6ba8100e00 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6b9e7fc640 1 -- 172.21.15.22:0/3062322343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b98010910 con 0x7f6ba8100e00 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba81a63a0 con 0x7f6ba8100e00 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6b9e7fc640 1 -- 172.21.15.22:0/3062322343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98018990 con 0x7f6ba8100e00 2024-03-09T00:00:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.768+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba81a6940 con 0x7f6ba8100e00 2024-03-09T00:00:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.770+0000 7f6b9e7fc640 1 -- 172.21.15.22:0/3062322343 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6b98010ac0 con 0x7f6ba8100e00 2024-03-09T00:00:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 Processor -- start 2024-03-09T00:00:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6b9e7fc640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b940a2a90 0x7f6b940a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 -- start start 2024-03-09T00:00:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6b9e7fc640 1 -- 172.21.15.22:0/3062322343 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6b980cf570 con 0x7f6ba8100e00 2024-03-09T00:00:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6badbaf640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b940a2a90 0x7f6b940a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909fd40 0x55cd3909ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd3909f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd390035f0 con 0x55cd390a8550 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd38fac930 con 0x55cd3909efd0 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd38fff690 con 0x55cd3909fd40 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909fd40 0x55cd3909ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6badbaf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bacbad640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd3909f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909fd40 0x55cd3909ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45324/0 (socket says 172.21.15.22:45324) 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.771+0000 7f6bad3ae640 1 -- 172.21.15.22:0/1396817876 learned_addr learned my addr 172.21.15.22:0/1396817876 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6bacbad640 1 -- 172.21.15.22:0/1396817876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909fd40 msgr2=0x55cd3909ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6bacbad640 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909fd40 0x55cd3909ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6bacbad640 1 -- 172.21.15.22:0/1396817876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 msgr2=0x55cd390aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6bacbad640 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6bacbad640 1 -- 172.21.15.22:0/1396817876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd38d18210 con 0x55cd3909efd0 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909fd40 0x55cd3909ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6badbaf640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b940a2a90 0x7f6b940a4f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006010 tx=0x7f6ba0032e60 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6bacbad640 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd3909f450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b980cc920 tx=0x7f6b9809bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbddd3ced184f4bb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6b937fe640 1 -- 172.21.15.22:0/1396817876 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980a3fd0 con 0x55cd3909efd0 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6b937fe640 1 -- 172.21.15.22:0/1396817876 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b980860b0 con 0x55cd3909efd0 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.772+0000 7f6b937fe640 1 -- 172.21.15.22:0/1396817876 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980cf3d0 con 0x55cd3909efd0 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1396817876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 msgr2=0x55cd3909f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd3909f450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b980cc920 tx=0x7f6b9809bff0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1396817876 shutdown_connections 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd3909f450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1396817876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909fd40 0x55cd3909ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1396817876 >> 172.21.15.22:0/1396817876 conn(0x55cd390665b0 msgr2=0x55cd39099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1396817876 shutdown_connections 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1396817876 wait complete. 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 -- start start 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.773+0000 7f6bb21fa740 1 Processor -- start 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909efd0 0x55cd390a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd390a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd390ab280 con 0x55cd3909efd0 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd38fbfd90 con 0x55cd3909fd40 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd390ab400 con 0x55cd390a8550 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 10 client.0 _ll_register_callbacks cb 0x55cd38d17a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909efd0 0x55cd390a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909efd0 0x55cd390a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50758/0 (socket says 172.21.15.22:50758) 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 -- 172.21.15.22:0/1570994512 learned_addr learned my addr 172.21.15.22:0/1570994512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bacbad640 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6badbaf640 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd390a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 -- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 msgr2=0x55cd390a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd390a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 -- 172.21.15.22:0/1570994512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 msgr2=0x55cd390a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 -- 172.21.15.22:0/1570994512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd38de4620 con 0x55cd3909efd0 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909efd0 0x55cd390a56f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4012d10 tx=0x7f6ba400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232b433a4de8c423 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6b90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6b90ff9640 1 -- 172.21.15.22:0/1570994512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4012110 con 0x55cd3909efd0 2024-03-09T00:00:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6b90ff9640 1 -- 172.21.15.22:0/1570994512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6ba4012e00 con 0x55cd3909efd0 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.774+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b980860b0 con 0x55cd3909efd0 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6b90ff9640 1 -- 172.21.15.22:0/1570994512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba4025050 con 0x55cd3909efd0 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 10 client.17574 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cd390b7790 con 0x55cd3909efd0 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6b90ff9640 1 -- 172.21.15.22:0/1570994512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b980860b0 con 0x55cd3909efd0 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6b90ff9640 1 -- 172.21.15.22:0/1570994512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ba4010430 con 0x55cd3909efd0 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 10 client.17574 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 10 client.17574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 20 client.17574 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd390b8c50 0x55cd390bb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 4 client.17574 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd39010b30 con 0x55cd390b8c50 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bacbad640 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd390b8c50 0x55cd390bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6b90ff9640 10 client.17574 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.775+0000 7f6bacbad640 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd390b8c50 0x55cd390bb040 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd2ca2d70671f51 server_cookie=38e5339cbbab000b in_seq=0 out_seq=0 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6b90ff9640 1 -- 172.21.15.22:0/1570994512 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd39010b30 con 0x55cd390b8c50 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6b90ff9640 10 client.17574 handle_command_reply: tid=0 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 client.17574 shutdown 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909efd0 msgr2=0x55cd390a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909efd0 0x55cd390a56f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4012d10 tx=0x7f6ba400f790 comp rx=0 tx=0).stop 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 shutdown_connections 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd390b8c50 0x55cd390bb040 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd390a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/1570994512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909efd0 0x55cd390a56f0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6b90ff9640 0 client.17574 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 >> 172.21.15.22:0/1570994512 conn(0x55cd390665b0 msgr2=0x55cd3909ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 shutdown_connections 2024-03-09T00:00:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 1 -- 172.21.15.22:0/1570994512 wait complete. 2024-03-09T00:00:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.777+0000 7f6bb21fa740 20 client.17574 trim_cache size 0 max 16384 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 Processor -- start 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 -- start start 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909efd0 0x55cd390a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390a4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81b2be0 con 0x55cd390a8550 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81b0780 con 0x55cd3909fd40 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81ae820 con 0x55cd3909efd0 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909efd0 0x55cd390a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6badbaf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909efd0 0x55cd390a7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:45338/0 (socket says 172.21.15.22:45338) 2024-03-09T00:00:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.791+0000 7f6bad3ae640 1 -- 172.21.15.22:0/2447214442 learned_addr learned my addr 172.21.15.22:0/2447214442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6bacbad640 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6bad3ae640 1 -- 172.21.15.22:0/2447214442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 msgr2=0x55cd390a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6bad3ae640 1 -- 172.21.15.22:0/2447214442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 msgr2=0x55cd390a4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390a4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6bad3ae640 1 -- 172.21.15.22:0/2447214442 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd390a12f0 con 0x55cd3909efd0 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909efd0 0x55cd390a7e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4000c00 tx=0x7f6ba4012850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0221cebbeab9005 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6b92ffd640 1 -- 172.21.15.22:0/2447214442 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4012110 con 0x55cd3909efd0 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6b92ffd640 1 -- 172.21.15.22:0/2447214442 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ba4010430 con 0x55cd3909efd0 2024-03-09T00:00:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.792+0000 7f6b92ffd640 1 -- 172.21.15.22:0/2447214442 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4003720 con 0x55cd3909efd0 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 -- 172.21.15.22:0/2447214442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909efd0 msgr2=0x55cd390a7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909efd0 0x55cd390a7e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4000c00 tx=0x7f6ba4012850 comp rx=0 tx=0).stop 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 -- 172.21.15.22:0/2447214442 shutdown_connections 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd390a8550 0x55cd390a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909fd40 0x55cd390a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/2447214442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd3909efd0 0x55cd390a7e20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 -- 172.21.15.22:0/2447214442 >> 172.21.15.22:0/2447214442 conn(0x55cd390665b0 msgr2=0x55cd390997f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 -- 172.21.15.22:0/2447214442 shutdown_connections 2024-03-09T00:00:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.793+0000 7f6bb21fa740 1 -- 172.21.15.22:0/2447214442 wait complete. 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 -- start start 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 Processor -- start 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd390a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909fd40 0x55cd390a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd3910f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81aee20 con 0x55cd3909fd40 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81b0120 con 0x55cd3909efd0 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81af460 con 0x55cd390a8550 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bb21fa740 10 client.0 _ll_register_callbacks cb 0x7f6b980973b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bacbad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909fd40 0x55cd390a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6badbaf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd3910f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bad3ae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd390a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bacbad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909fd40 0x55cd390a3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50776/0 (socket says 172.21.15.22:50776) 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.794+0000 7f6bacbad640 1 -- 172.21.15.22:0/3337122477 learned_addr learned my addr 172.21.15.22:0/3337122477 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6badbaf640 1 -- 172.21.15.22:0/3337122477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 msgr2=0x55cd390a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6badbaf640 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd390a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6badbaf640 1 -- 172.21.15.22:0/3337122477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909fd40 msgr2=0x55cd390a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6badbaf640 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909fd40 0x55cd390a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6badbaf640 1 -- 172.21.15.22:0/3337122477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd3910f5e0 con 0x55cd390a8550 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6bacbad640 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909fd40 0x55cd390a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6badbaf640 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd3910f0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0037040 tx=0x7f6ba002c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea57f367befe14ee server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6b90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6b90ff9640 1 -- 172.21.15.22:0/3337122477 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0039030 con 0x55cd390a8550 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6b90ff9640 1 -- 172.21.15.22:0/3337122477 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6ba0004510 con 0x55cd390a8550 2024-03-09T00:00:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ba4010430 con 0x55cd390a8550 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6b90ff9640 1 -- 172.21.15.22:0/3337122477 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba0004710 con 0x55cd390a8550 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.795+0000 7f6bb21fa740 10 client.27025 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cd391108c0 con 0x55cd390a8550 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6b90ff9640 1 -- 172.21.15.22:0/3337122477 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ba4010430 con 0x55cd390a8550 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6b90ff9640 1 -- 172.21.15.22:0/3337122477 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ba0043430 con 0x55cd390a8550 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bb21fa740 10 client.27025 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bb21fa740 10 client.27025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bb21fa740 20 client.27025 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd39110fc0 0x55cd391133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bb21fa740 4 client.27025 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd3900ae80 con 0x55cd39110fc0 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.796+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd39110fc0 0x55cd391133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.797+0000 7f6bad3ae640 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd39110fc0 0x55cd391133e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26515a06b49603fb server_cookie=9e67428ca0e9903 in_seq=0 out_seq=0 2024-03-09T00:00:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:24.797+0000 7f6b90ff9640 10 client.27025 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6b90ff9640 1 -- 172.21.15.22:0/3337122477 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd3900ae80 con 0x55cd39110fc0 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6b90ff9640 10 client.27025 handle_command_reply: tid=0 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 client.27025 shutdown 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 msgr2=0x55cd3910f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd3910f0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0037040 tx=0x7f6ba002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 shutdown_connections 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cd39110fc0 0x55cd391133e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd390a8550 0x55cd3910f0a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cd3909fd40 0x55cd390a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 --2- 172.21.15.22:0/3337122477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd3909efd0 0x55cd390a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 >> 172.21.15.22:0/3337122477 conn(0x55cd390665b0 msgr2=0x55cd390a3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6b90ff9640 0 client.27025 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 shutdown_connections 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.045+0000 7f6bb21fa740 1 -- 172.21.15.22:0/3337122477 wait complete. 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6bb21fa740 20 client.27025 trim_cache size 0 max 16384 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:00:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:00:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:00:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b940a2a90 msgr2=0x7f6b940a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b940a2a90 0x7f6b940a4f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006010 tx=0x7f6ba0032e60 comp rx=0 tx=0).stop 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 msgr2=0x7f6ba81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba81023a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9f0 tx=0x7f6b9800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 shutdown_connections 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b940a2a90 0x7f6b940a4f50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ba8105190 0x7f6ba81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba81023a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 --2- 172.21.15.22:0/3062322343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba8100440 0x7f6ba8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.046+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 >> 172.21.15.22:0/3062322343 conn(0x7f6ba8077040 msgr2=0x7f6ba8068cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.047+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 shutdown_connections 2024-03-09T00:00:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:25.047+0000 7f6baf612640 1 -- 172.21.15.22:0/3062322343 wait complete. 2024-03-09T00:00:25.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:25 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:00:25.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:25 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:00:25.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:25 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:00:25.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 607649 11.27 MB/sec execute 153 sec latency 102.641 ms 2024-03-09T00:00:26.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:26 smithi062 ceph-mon[36129]: pgmap v183: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.7 MiB/s wr, 461 op/s 2024-03-09T00:00:26.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:26 smithi192 ceph-mon[35995]: pgmap v183: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.7 MiB/s wr, 461 op/s 2024-03-09T00:00:26.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:26 smithi022 ceph-mon[31894]: pgmap v183: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.7 MiB/s wr, 461 op/s 2024-03-09T00:00:26.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 609435 11.24 MB/sec execute 154 sec latency 21.941 ms 2024-03-09T00:00:27.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 611447 11.25 MB/sec execute 155 sec latency 21.636 ms 2024-03-09T00:00:28.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:28 smithi192 ceph-mon[35995]: pgmap v184: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-03-09T00:00:28.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:28 smithi062 ceph-mon[36129]: pgmap v184: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-03-09T00:00:28.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:28 smithi022 ceph-mon[31894]: pgmap v184: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 221 op/s 2024-03-09T00:00:28.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 613788 11.25 MB/sec execute 156 sec latency 28.129 ms 2024-03-09T00:00:29.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 615242 11.23 MB/sec execute 157 sec latency 105.812 ms 2024-03-09T00:00:30.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:30 smithi062 ceph-mon[36129]: pgmap v185: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-03-09T00:00:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:30 smithi192 ceph-mon[35995]: pgmap v185: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-03-09T00:00:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:30 smithi022 ceph-mon[31894]: pgmap v185: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-03-09T00:00:30.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 617731 11.25 MB/sec execute 158 sec latency 24.137 ms 2024-03-09T00:00:31.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 619408 11.24 MB/sec execute 159 sec latency 18.433 ms 2024-03-09T00:00:32.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:32 smithi062 ceph-mon[36129]: pgmap v186: 129 pgs: 129 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 425 op/s 2024-03-09T00:00:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:32 smithi192 ceph-mon[35995]: pgmap v186: 129 pgs: 129 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 425 op/s 2024-03-09T00:00:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:32 smithi022 ceph-mon[31894]: pgmap v186: 129 pgs: 129 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 425 op/s 2024-03-09T00:00:32.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 621807 11.25 MB/sec execute 160 sec latency 16.046 ms 2024-03-09T00:00:33.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 623668 11.22 MB/sec execute 161 sec latency 17.088 ms 2024-03-09T00:00:34.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:34 smithi062 ceph-mon[36129]: pgmap v187: 129 pgs: 129 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 344 op/s 2024-03-09T00:00:34.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:34 smithi192 ceph-mon[35995]: pgmap v187: 129 pgs: 129 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 344 op/s 2024-03-09T00:00:34.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:34 smithi022 ceph-mon[31894]: pgmap v187: 129 pgs: 129 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 344 op/s 2024-03-09T00:00:34.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 625572 11.24 MB/sec execute 162 sec latency 83.304 ms 2024-03-09T00:00:35.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 627544 11.21 MB/sec execute 163 sec latency 34.869 ms 2024-03-09T00:00:36.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:36 smithi062 ceph-mon[36129]: pgmap v188: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-09T00:00:36.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:36 smithi192 ceph-mon[35995]: pgmap v188: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-09T00:00:36.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:36 smithi022 ceph-mon[31894]: pgmap v188: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-09T00:00:36.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 629529 11.22 MB/sec execute 164 sec latency 19.846 ms 2024-03-09T00:00:37.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 632281 11.25 MB/sec execute 165 sec latency 13.723 ms 2024-03-09T00:00:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 -- 172.21.15.22:0/1459407776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 msgr2=0x7f4bc00ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 --2- 172.21.15.22:0/1459407776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00ffdc0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009930 tx=0x7f4ba802bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 -- 172.21.15.22:0/1459407776 shutdown_connections 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 --2- 172.21.15.22:0/1459407776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc0100300 0x7f4bc010c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 --2- 172.21.15.22:0/1459407776 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00ffdc0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 --2- 172.21.15.22:0/1459407776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0105950 0x7f4bc0105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 -- 172.21.15.22:0/1459407776 >> 172.21.15.22:0/1459407776 conn(0x7f4bc00767b0 msgr2=0x7f4bc00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 -- 172.21.15.22:0/1459407776 shutdown_connections 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.137+0000 7f4bc5337640 1 -- 172.21.15.22:0/1459407776 wait complete. 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 Processor -- start 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 -- start start 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc0100300 0x7f4bc00fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0105950 0x7f4bc00fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc010f4e0 con 0x7f4bc00ff940 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc010f360 con 0x7f4bc0100300 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bc5337640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc010f660 con 0x7f4bc0105950 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41900/0 (socket says 172.21.15.22:41900) 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bbeffd640 1 -- 172.21.15.22:0/2716105003 learned_addr learned my addr 172.21.15.22:0/2716105003 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc0100300 0x7f4bc00fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.138+0000 7f4bbf7fe640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0105950 0x7f4bc00fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbeffd640 1 -- 172.21.15.22:0/2716105003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0105950 msgr2=0x7f4bc00fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0105950 0x7f4bc00fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbeffd640 1 -- 172.21.15.22:0/2716105003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc0100300 msgr2=0x7f4bc00fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc0100300 0x7f4bc00fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbeffd640 1 -- 172.21.15.22:0/2716105003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc01a6060 con 0x7f4bc00ff940 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00fbbc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000ea40 tx=0x7f4bb000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbcff9640 1 -- 172.21.15.22:0/2716105003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000cd40 con 0x7f4bc00ff940 2024-03-09T00:00:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc01a6350 con 0x7f4bc00ff940 2024-03-09T00:00:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbcff9640 1 -- 172.21.15.22:0/2716105003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bb0005500 con 0x7f4bc00ff940 2024-03-09T00:00:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bbcff9640 1 -- 172.21.15.22:0/2716105003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0016670 con 0x7f4bc00ff940 2024-03-09T00:00:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.139+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc01a6870 con 0x7f4bc00ff940 2024-03-09T00:00:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.141+0000 7f4bbcff9640 1 -- 172.21.15.22:0/2716105003 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4bb00046c0 con 0x7f4bc00ff940 2024-03-09T00:00:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 Processor -- start 2024-03-09T00:00:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 -- start start 2024-03-09T00:00:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bbcff9640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4bac0a2a10 0x7f4bac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bbcff9640 1 -- 172.21.15.22:0/2716105003 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4bb00cef90 con 0x7f4bc00ff940 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bbf7fe640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4bac0a2a10 0x7f4bac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3be7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3be8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3a3b4c5f0 con 0x55d3a3be8d40 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3a3af5930 con 0x55d3a3be7fd0 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3a3b48690 con 0x55d3a3bf1550 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3be7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3be7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41914/0 (socket says 172.21.15.22:41914) 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1825231923 learned_addr learned my addr 172.21.15.22:0/1825231923 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3be8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.142+0000 7f4bbf7fe640 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1825231923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 msgr2=0x55d3a3bf3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1825231923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 msgr2=0x55d3a3be8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3be8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bbf7fe640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4bac0a2a10 0x7f4bac0a4ed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007ea0 tx=0x7f4bb800f790 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1825231923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3a3861210 con 0x55d3a3be8d40 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3be7a90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0097cd0 tx=0x7f4bb00056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27936b78e09c6639 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bb57fa640 1 -- 172.21.15.22:0/1825231923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00a39f0 con 0x55d3a3be8d40 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bb57fa640 1 -- 172.21.15.22:0/1825231923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bb0099600 con 0x55d3a3be8d40 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.143+0000 7f4bb57fa640 1 -- 172.21.15.22:0/1825231923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb009a750 con 0x55d3a3be8d40 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1825231923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 msgr2=0x55d3a3be7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3be7a90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0097cd0 tx=0x7f4bb00056b0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1825231923 shutdown_connections 2024-03-09T00:00:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3be8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1825231923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3be7a90 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1825231923 >> 172.21.15.22:0/1825231923 conn(0x55d3a3baf5b0 msgr2=0x55d3a3be2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1825231923 shutdown_connections 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1825231923 wait complete. 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 -- start start 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 Processor -- start 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.144+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3bee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3be8d40 0x55d3a3beeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3bef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3a3bf41c0 con 0x55d3a3bf1550 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3a3b08d90 con 0x55d3a3be7fd0 2024-03-09T00:00:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3a3bf4340 con 0x55d3a3be8d40 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bc7feb740 10 client.0 _ll_register_callbacks cb 0x55d3a3860a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3bef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3bee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bb67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3be8d40 0x55d3a3beeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3bee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57436/0 (socket says 172.21.15.22:57436) 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1986412165 learned_addr learned my addr 172.21.15.22:0/1986412165 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1986412165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3be8d40 msgr2=0x55d3a3beeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3be8d40 0x55d3a3beeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1986412165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 msgr2=0x55d3a3bef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3bef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 -- 172.21.15.22:0/1986412165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3a392d620 con 0x55d3a3be7fd0 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbf7fe640 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3bef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3bee640 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00cc3f0 tx=0x7f4bb009b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc34cbf60ba53910 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4ba67fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:00:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4ba67fc640 1 -- 172.21.15.22:0/1986412165 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00b3070 con 0x55d3a3be7fd0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4ba67fc640 1 -- 172.21.15.22:0/1986412165 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4bb009a750 con 0x55d3a3be7fd0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.145+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bb0099600 con 0x55d3a3be7fd0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4ba67fc640 1 -- 172.21.15.22:0/1986412165 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bb009a950 con 0x55d3a3be7fd0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 10 client.26999 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d3a3bf41c0 con 0x55d3a3be7fd0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4ba67fc640 1 -- 172.21.15.22:0/1986412165 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bb0099600 con 0x55d3a3be7fd0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4ba67fc640 1 -- 172.21.15.22:0/1986412165 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4bb007f070 con 0x55d3a3be7fd0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 10 client.26999 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 10 client.26999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 20 client.26999 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c016e0 0x55d3a3c03ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 4 client.26999 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3a3b59b30 con 0x55d3a3c016e0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c016e0 0x55d3a3c03ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c016e0 0x55d3a3c03ad0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff89fd5d7a907e5 server_cookie=a92f91ecbf6eee98 in_seq=0 out_seq=0 2024-03-09T00:00:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.146+0000 7f4ba67fc640 10 client.26999 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4ba67fc640 1 -- 172.21.15.22:0/1986412165 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3a3b59b30 con 0x55d3a3c016e0 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4ba67fc640 10 client.26999 handle_command_reply: tid=0 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 client.26999 shutdown 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 msgr2=0x55d3a3bee640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3bee640 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00cc3f0 tx=0x7f4bb009b400 comp rx=0 tx=0).stop 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 shutdown_connections 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c016e0 0x55d3a3c03ad0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3bef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3be8d40 0x55d3a3beeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/1986412165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be7fd0 0x55d3a3bee640 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 >> 172.21.15.22:0/1986412165 conn(0x55d3a3baf5b0 msgr2=0x55d3a3be4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4ba67fc640 0 client.26999 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 shutdown_connections 2024-03-09T00:00:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 1 -- 172.21.15.22:0/1986412165 wait complete. 2024-03-09T00:00:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.149+0000 7f4bc7feb740 20 client.26999 trim_cache size 0 max 16384 2024-03-09T00:00:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 Processor -- start 2024-03-09T00:00:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 -- start start 2024-03-09T00:00:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3be79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01b2bc0 con 0x55d3a3bf1550 2024-03-09T00:00:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01b0760 con 0x55d3a3be8d40 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.163+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01ae800 con 0x55d3a3bf85b0 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3be79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3be79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41932/0 (socket says 172.21.15.22:41932) 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 -- 172.21.15.22:0/3083245059 learned_addr learned my addr 172.21.15.22:0/3083245059 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bbf7fe640 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 -- 172.21.15.22:0/3083245059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf85b0 msgr2=0x55d3a3bf8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 -- 172.21.15.22:0/3083245059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be8d40 msgr2=0x55d3a3bf0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 -- 172.21.15.22:0/3083245059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3a3c086d0 con 0x55d3a3bf1550 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3be79b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8005ab0 tx=0x7f4ba802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadfc6776742481a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb4ff9640 1 -- 172.21.15.22:0/3083245059 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba804c030 con 0x55d3a3bf1550 2024-03-09T00:00:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb4ff9640 1 -- 172.21.15.22:0/3083245059 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba8048be0 con 0x55d3a3bf1550 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.164+0000 7f4bb4ff9640 1 -- 172.21.15.22:0/3083245059 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8048590 con 0x55d3a3bf1550 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 -- 172.21.15.22:0/3083245059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 msgr2=0x55d3a3be79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3be79b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8005ab0 tx=0x7f4ba802e040 comp rx=0 tx=0).stop 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 -- 172.21.15.22:0/3083245059 shutdown_connections 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3bf1550 0x55d3a3be79b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/3083245059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 -- 172.21.15.22:0/3083245059 >> 172.21.15.22:0/3083245059 conn(0x55d3a3baf5b0 msgr2=0x55d3a3be72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 -- 172.21.15.22:0/3083245059 shutdown_connections 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 -- 172.21.15.22:0/3083245059 wait complete. 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.165+0000 7f4bc7feb740 1 -- start start 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 Processor -- start 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01aee00 con 0x55d3a3be8d40 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01b0100 con 0x55d3a3bf85b0 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01af440 con 0x55d3a3bf1550 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 10 client.0 _ll_register_callbacks cb 0x7f4bb000cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41940/0 (socket says 172.21.15.22:41940) 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 -- 172.21.15.22:0/4128971061 learned_addr learned my addr 172.21.15.22:0/4128971061 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbf7fe640 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 -- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 msgr2=0x55d3a3bf7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 -- 172.21.15.22:0/4128971061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3bf85b0 msgr2=0x55d3a3bf8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 -- 172.21.15.22:0/4128971061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3a3beab30 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bbeffd640 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf75b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00995b0 tx=0x7f4bb00a0610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3ce85f32846bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4ba67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4ba67fc640 1 -- 172.21.15.22:0/4128971061 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00b3040 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4ba67fc640 1 -- 172.21.15.22:0/4128971061 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4bb009a3d0 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.166+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ba8048be0 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4ba67fc640 1 -- 172.21.15.22:0/4128971061 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bb009ae40 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 10 client.17598 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d3a3c58e80 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4ba67fc640 1 -- 172.21.15.22:0/4128971061 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba8048be0 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4ba67fc640 1 -- 172.21.15.22:0/4128971061 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4bb007f040 con 0x55d3a3be8d40 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 10 client.17598 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 20 client.17598 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c59300 0x55d3a3c5b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 4 client.17598 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:00:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3a3b53e80 con 0x55d3a3c59300 2024-03-09T00:00:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c59300 0x55d3a3c5b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4bb67fc640 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c59300 0x55d3a3c5b6f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4350afe7bf8d33a server_cookie=e2223ec464082abf in_seq=0 out_seq=0 2024-03-09T00:00:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.167+0000 7f4ba67fc640 10 client.17598 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:38.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:38 smithi062 ceph-mon[36129]: pgmap v189: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 335 op/s 2024-03-09T00:00:38.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:38 smithi192 ceph-mon[35995]: pgmap v189: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 335 op/s 2024-03-09T00:00:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4ba67fc640 1 -- 172.21.15.22:0/4128971061 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3a3b53e80 con 0x55d3a3c59300 2024-03-09T00:00:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4ba67fc640 10 client.17598 handle_command_reply: tid=0 2024-03-09T00:00:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 client.17598 shutdown 2024-03-09T00:00:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 msgr2=0x55d3a3bf75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf75b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00995b0 tx=0x7f4bb00a0610 comp rx=0 tx=0).stop 2024-03-09T00:00:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 shutdown_connections 2024-03-09T00:00:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d3a3c59300 0x55d3a3c5b6f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a3bf85b0 0x55d3a3bf8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d3a3bf1550 0x55d3a3bf7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 --2- 172.21.15.22:0/4128971061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d3a3be8d40 0x55d3a3bf75b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 >> 172.21.15.22:0/4128971061 conn(0x55d3a3baf5b0 msgr2=0x55d3a3becf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4ba67fc640 0 client.17598 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 shutdown_connections 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 1 -- 172.21.15.22:0/4128971061 wait complete. 2024-03-09T00:00:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.468+0000 7f4bc7feb740 20 client.17598 trim_cache size 0 max 16384 2024-03-09T00:00:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:00:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:00:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:00:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4bac0a2a10 msgr2=0x7f4bac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4bac0a2a10 0x7f4bac0a4ed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007ea0 tx=0x7f4bb800f790 comp rx=0 tx=0).stop 2024-03-09T00:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 msgr2=0x7f4bc00fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00fbbc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000ea40 tx=0x7f4bb000ef00 comp rx=0 tx=0).stop 2024-03-09T00:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 shutdown_connections 2024-03-09T00:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4bac0a2a10 0x7f4bac0a4ed0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0105950 0x7f4bc00fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bc0100300 0x7f4bc00fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 --2- 172.21.15.22:0/2716105003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00ff940 0x7f4bc00fbbc0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 >> 172.21.15.22:0/2716105003 conn(0x7f4bc00767b0 msgr2=0x7f4bc0102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 shutdown_connections 2024-03-09T00:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:38.469+0000 7f4bc5337640 1 -- 172.21.15.22:0/2716105003 wait complete. 2024-03-09T00:00:38.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:38 smithi022 ceph-mon[31894]: pgmap v189: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 335 op/s 2024-03-09T00:00:38.875 INFO:tasks.workunit.client.0.smithi022.stdout: 1 633346 11.21 MB/sec execute 166 sec latency 185.874 ms 2024-03-09T00:00:39.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 635527 11.21 MB/sec execute 167 sec latency 63.975 ms 2024-03-09T00:00:40.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:40 smithi062 ceph-mon[36129]: pgmap v190: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 486 op/s 2024-03-09T00:00:40.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:40 smithi192 ceph-mon[35995]: pgmap v190: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 486 op/s 2024-03-09T00:00:40.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:40 smithi022 ceph-mon[31894]: pgmap v190: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 486 op/s 2024-03-09T00:00:40.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 636991 11.19 MB/sec execute 168 sec latency 32.509 ms 2024-03-09T00:00:41.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 639566 11.22 MB/sec execute 169 sec latency 14.159 ms 2024-03-09T00:00:42.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:42 smithi062 ceph-mon[36129]: pgmap v191: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-03-09T00:00:42.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:42 smithi192 ceph-mon[35995]: pgmap v191: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-03-09T00:00:42.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:42 smithi022 ceph-mon[31894]: pgmap v191: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-03-09T00:00:42.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 641585 11.20 MB/sec execute 170 sec latency 27.420 ms 2024-03-09T00:00:43.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 643485 11.20 MB/sec execute 171 sec latency 22.555 ms 2024-03-09T00:00:44.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:44 smithi062 ceph-mon[36129]: pgmap v192: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-03-09T00:00:44.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:44 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:00:44.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:44 smithi192 ceph-mon[35995]: pgmap v192: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-03-09T00:00:44.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:44 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:00:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:44 smithi022 ceph-mon[31894]: pgmap v192: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-03-09T00:00:44.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:44 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:00:44.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 645566 11.20 MB/sec execute 172 sec latency 85.443 ms 2024-03-09T00:00:45.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:00:45.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:00:45.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:00:45.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:00:45.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:00:45.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:00:45.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:00:45.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:00:45.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:00:45.717 INFO:tasks.workunit.client.0.smithi022.stdout: 1 647257 11.19 MB/sec execute 173 sec latency 39.940 ms 2024-03-09T00:00:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:46 smithi022 ceph-mon[31894]: pgmap v193: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 482 op/s 2024-03-09T00:00:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:46 smithi062 ceph-mon[36129]: pgmap v193: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 482 op/s 2024-03-09T00:00:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:46 smithi192 ceph-mon[35995]: pgmap v193: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.3 MiB/s wr, 482 op/s 2024-03-09T00:00:46.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 649989 11.22 MB/sec execute 174 sec latency 17.070 ms 2024-03-09T00:00:47.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 651583 11.20 MB/sec execute 175 sec latency 18.233 ms 2024-03-09T00:00:48.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:48 smithi022 ceph-mon[31894]: pgmap v194: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 378 op/s 2024-03-09T00:00:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:48 smithi062 ceph-mon[36129]: pgmap v194: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 378 op/s 2024-03-09T00:00:48.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:48 smithi192 ceph-mon[35995]: pgmap v194: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 378 op/s 2024-03-09T00:00:48.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 653903 11.21 MB/sec execute 176 sec latency 18.188 ms 2024-03-09T00:00:49.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 656021 11.19 MB/sec execute 177 sec latency 31.995 ms 2024-03-09T00:00:50.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:50 smithi022 ceph-mon[31894]: pgmap v195: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 445 op/s 2024-03-09T00:00:50.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:50 smithi062 ceph-mon[36129]: pgmap v195: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 445 op/s 2024-03-09T00:00:50.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:50 smithi192 ceph-mon[35995]: pgmap v195: 129 pgs: 129 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 445 op/s 2024-03-09T00:00:50.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 657533 11.19 MB/sec execute 178 sec latency 56.686 ms 2024-03-09T00:00:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 -- 172.21.15.22:0/44594096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21601027a0 msgr2=0x7f216010d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 --2- 172.21.15.22:0/44594096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21601027a0 0x7f216010d1a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2158009cd0 tx=0x7f215802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 -- 172.21.15.22:0/44594096 shutdown_connections 2024-03-09T00:00:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 --2- 172.21.15.22:0/44594096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21601027a0 0x7f216010d1a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 --2- 172.21.15.22:0/44594096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2160101e00 0x7f2160102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 --2- 172.21.15.22:0/44594096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2160100520 0x7f2160100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 -- 172.21.15.22:0/44594096 >> 172.21.15.22:0/44594096 conn(0x7f2160077310 msgr2=0x7f2160077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 -- 172.21.15.22:0/44594096 shutdown_connections 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.558+0000 7f216707e640 1 -- 172.21.15.22:0/44594096 wait complete. 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 Processor -- start 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 -- start start 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2160100520 0x7f2160197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160101e00 0x7f21601980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21601027a0 0x7f216019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160110670 con 0x7f2160101e00 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601104f0 con 0x7f21601027a0 2024-03-09T00:00:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216707e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601107f0 con 0x7f2160100520 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f215ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160101e00 0x7f21601980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f2164e1a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2160100520 0x7f2160197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f215ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160101e00 0x7f21601980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49530/0 (socket says 172.21.15.22:49530) 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f215ffff640 1 -- 172.21.15.22:0/3645571965 learned_addr learned my addr 172.21.15.22:0/3645571965 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f216561b640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21601027a0 0x7f216019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f215ffff640 1 -- 172.21.15.22:0/3645571965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2160100520 msgr2=0x7f2160197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f215ffff640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2160100520 0x7f2160197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.559+0000 7f215ffff640 1 -- 172.21.15.22:0/3645571965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21601027a0 msgr2=0x7f216019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f215ffff640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21601027a0 0x7f216019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f215ffff640 1 -- 172.21.15.22:0/3645571965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216019cb60 con 0x7f2160101e00 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f215ffff640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160101e00 0x7f21601980d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f214c00e9f0 tx=0x7f214c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f215dffb640 1 -- 172.21.15.22:0/3645571965 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c00cd80 con 0x7f2160101e00 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f215dffb640 1 -- 172.21.15.22:0/3645571965 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f214c010910 con 0x7f2160101e00 2024-03-09T00:00:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216019ce50 con 0x7f2160101e00 2024-03-09T00:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f215dffb640 1 -- 172.21.15.22:0/3645571965 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c018990 con 0x7f2160101e00 2024-03-09T00:00:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.560+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21601a46e0 con 0x7f2160101e00 2024-03-09T00:00:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.562+0000 7f215dffb640 1 -- 172.21.15.22:0/3645571965 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f214c010ac0 con 0x7f2160101e00 2024-03-09T00:00:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 Processor -- start 2024-03-09T00:00:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 -- start start 2024-03-09T00:00:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f215dffb640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f21500a2a10 0x7f21500a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80bd40 0x55ef3b80aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b80b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b814550 0x55ef3b816940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3b76f5f0 con 0x55ef3b80bd40 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3b718930 con 0x55ef3b80afd0 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2169c58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3b76b690 con 0x55ef3b814550 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f216561b640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f21500a2a10 0x7f21500a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f215dffb640 1 -- 172.21.15.22:0/3645571965 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f214c0cf520 con 0x7f2160101e00 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80bd40 0x55ef3b80aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80bd40 0x55ef3b80aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49542/0 (socket says 172.21.15.22:49542) 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 -- 172.21.15.22:0/4216374079 learned_addr learned my addr 172.21.15.22:0/4216374079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f215ffff640 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b80b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f216561b640 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b814550 0x55ef3b816940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 -- 172.21.15.22:0/4216374079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b814550 msgr2=0x55ef3b816940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b814550 0x55ef3b816940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 -- 172.21.15.22:0/4216374079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 msgr2=0x55ef3b80b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b80b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.563+0000 7f2164e1a640 1 -- 172.21.15.22:0/4216374079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3b484210 con 0x55ef3b80bd40 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.564+0000 7f216561b640 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b814550 0x55ef3b816940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.564+0000 7f2164e1a640 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80bd40 0x55ef3b80aa90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2154009cf0 tx=0x7f2154011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93904c2d7eb32d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.564+0000 7f216561b640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f21500a2a10 0x7f21500a4ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2158005cc0 tx=0x7f2158009820 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.564+0000 7f214affd640 1 -- 172.21.15.22:0/4216374079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154004230 con 0x55ef3b80bd40 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.564+0000 7f214affd640 1 -- 172.21.15.22:0/4216374079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21540043d0 con 0x55ef3b80bd40 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.564+0000 7f214affd640 1 -- 172.21.15.22:0/4216374079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154019920 con 0x55ef3b80bd40 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- 172.21.15.22:0/4216374079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80bd40 msgr2=0x55ef3b80aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80bd40 0x55ef3b80aa90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2154009cf0 tx=0x7f2154011810 comp rx=0 tx=0).stop 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- 172.21.15.22:0/4216374079 shutdown_connections 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b814550 0x55ef3b816940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b80b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 --2- 172.21.15.22:0/4216374079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80bd40 0x55ef3b80aa90 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- 172.21.15.22:0/4216374079 >> 172.21.15.22:0/4216374079 conn(0x55ef3b7d25b0 msgr2=0x55ef3b805cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- 172.21.15.22:0/4216374079 shutdown_connections 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- 172.21.15.22:0/4216374079 wait complete. 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- start start 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 Processor -- start 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b8114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b811a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b811f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3b817280 con 0x55ef3b814550 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3b72bd90 con 0x55ef3b80afd0 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3b817400 con 0x55ef3b80bd40 2024-03-09T00:00:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f2169c58740 10 client.0 _ll_register_callbacks cb 0x55ef3b483a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.565+0000 7f216561b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b811f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b811f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49556/0 (socket says 172.21.15.22:49556) 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 -- 172.21.15.22:0/2937534999 learned_addr learned my addr 172.21.15.22:0/2937534999 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f2164e1a640 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b8114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f215ffff640 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b811a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 -- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 msgr2=0x55ef3b811a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b811a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 -- 172.21.15.22:0/2937534999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 msgr2=0x55ef3b8114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b8114e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 -- 172.21.15.22:0/2937534999 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef3b550620 con 0x55ef3b814550 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f216561b640 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b811f60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2158032b60 tx=0x7f2158005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b92b029a88ddf15 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f213ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f213ffff640 1 -- 172.21.15.22:0/2937534999 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158053060 con 0x55ef3b814550 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f213ffff640 1 -- 172.21.15.22:0/2937534999 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2158004040 con 0x55ef3b814550 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21540043d0 con 0x55ef3b814550 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f213ffff640 1 -- 172.21.15.22:0/2937534999 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f215805a050 con 0x55ef3b814550 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f2169c58740 10 client.17610 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.566+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ef3b823510 con 0x55ef3b814550 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f213ffff640 1 -- 172.21.15.22:0/2937534999 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21540043d0 con 0x55ef3b814550 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f213ffff640 1 -- 172.21.15.22:0/2937534999 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f215803f030 con 0x55ef3b814550 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2169c58740 10 client.17610 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2169c58740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2169c58740 20 client.17610 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2169c58740 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b8249d0 0x55ef3b826dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2169c58740 4 client.17610 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef3b77cb30 con 0x55ef3b8249d0 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2164e1a640 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b8249d0 0x55ef3b826dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f213ffff640 10 client.17610 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.567+0000 7f2164e1a640 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b8249d0 0x55ef3b826dc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f530bdcad1398b52 server_cookie=4e1f4836970019fc in_seq=0 out_seq=0 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f213ffff640 1 -- 172.21.15.22:0/2937534999 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef3b77cb30 con 0x55ef3b8249d0 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f213ffff640 10 client.17610 handle_command_reply: tid=0 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 client.17610 shutdown 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 msgr2=0x55ef3b811f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b811f60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2158032b60 tx=0x7f2158005c50 comp rx=0 tx=0).stop 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 shutdown_connections 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b8249d0 0x55ef3b826dc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b811f60 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b811a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f213ffff640 0 client.17610 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 --2- 172.21.15.22:0/2937534999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b8114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 >> 172.21.15.22:0/2937534999 conn(0x55ef3b7d25b0 msgr2=0x55ef3b807a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 shutdown_connections 2024-03-09T00:00:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 1 -- 172.21.15.22:0/2937534999 wait complete. 2024-03-09T00:00:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.569+0000 7f2169c58740 20 client.17610 trim_cache size 0 max 16384 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 Processor -- start 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 -- start start 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b813c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b80fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b8102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601b0990 con 0x55ef3b814550 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601ae530 con 0x55ef3b80afd0 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f2169c58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601ac5d0 con 0x55ef3b80bd40 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.582+0000 7f216561b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b8102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f215ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b80fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b8102f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49558/0 (socket says 172.21.15.22:49558) 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 -- 172.21.15.22:0/677023376 learned_addr learned my addr 172.21.15.22:0/677023376 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f2164e1a640 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b813c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 -- 172.21.15.22:0/677023376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 msgr2=0x55ef3b80fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b80fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 -- 172.21.15.22:0/677023376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 msgr2=0x55ef3b813c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b813c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 -- 172.21.15.22:0/677023376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3b80a2a0 con 0x55ef3b814550 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f216561b640 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b8102f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2158009620 tx=0x7f2158003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd34f3a249fd63be server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f214a7fc640 1 -- 172.21.15.22:0/677023376 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158047020 con 0x55ef3b814550 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f214a7fc640 1 -- 172.21.15.22:0/677023376 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2158009b20 con 0x55ef3b814550 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.583+0000 7f214a7fc640 1 -- 172.21.15.22:0/677023376 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158053040 con 0x55ef3b814550 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 -- 172.21.15.22:0/677023376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 msgr2=0x55ef3b8102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b8102f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2158009620 tx=0x7f2158003d40 comp rx=0 tx=0).stop 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 -- 172.21.15.22:0/677023376 shutdown_connections 2024-03-09T00:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b814550 0x55ef3b8102f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b80fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 --2- 172.21.15.22:0/677023376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b80afd0 0x55ef3b813c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 -- 172.21.15.22:0/677023376 >> 172.21.15.22:0/677023376 conn(0x55ef3b7d25b0 msgr2=0x55ef3b82b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 -- 172.21.15.22:0/677023376 shutdown_connections 2024-03-09T00:00:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 -- 172.21.15.22:0/677023376 wait complete. 2024-03-09T00:00:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 -- start start 2024-03-09T00:00:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.584+0000 7f2169c58740 1 Processor -- start 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2169c58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80afd0 0x55ef3b81b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2169c58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b819b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2169c58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b814550 0x55ef3b81a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2169c58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601acbd0 con 0x55ef3b80afd0 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2169c58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601aded0 con 0x55ef3b814550 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2169c58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601ad210 con 0x55ef3b80bd40 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2169c58740 10 client.0 _ll_register_callbacks cb 0x7f21540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80afd0 0x55ef3b81b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80afd0 0x55ef3b81b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49566/0 (socket says 172.21.15.22:49566) 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 -- 172.21.15.22:0/2936271446 learned_addr learned my addr 172.21.15.22:0/2936271446 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f216561b640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b814550 0x55ef3b81a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f215ffff640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b819b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 -- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 msgr2=0x55ef3b819b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b819b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 -- 172.21.15.22:0/2936271446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b814550 msgr2=0x55ef3b81a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b814550 0x55ef3b81a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f2164e1a640 1 -- 172.21.15.22:0/2936271446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef3b81a600 con 0x55ef3b80afd0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.585+0000 7f216561b640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b814550 0x55ef3b81a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f2164e1a640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80afd0 0x55ef3b81b6d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2154009ab0 tx=0x7f21540194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc17711d1dac3d47 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f213ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f213ffff640 1 -- 172.21.15.22:0/2936271446 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154031070 con 0x55ef3b80afd0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f213ffff640 1 -- 172.21.15.22:0/2936271446 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f2154005040 con 0x55ef3b80afd0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2158009b20 con 0x55ef3b80afd0 2024-03-09T00:00:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f213ffff640 1 -- 172.21.15.22:0/2936271446 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f215403a050 con 0x55ef3b80afd0 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f2169c58740 10 client.17616 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ef3b87bbc0 con 0x55ef3b80afd0 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.586+0000 7f213ffff640 1 -- 172.21.15.22:0/2936271446 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2158009b20 con 0x55ef3b80afd0 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f213ffff640 1 -- 172.21.15.22:0/2936271446 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2154034020 con 0x55ef3b80afd0 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f2169c58740 10 client.17616 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f2169c58740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f2169c58740 20 client.17616 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f2169c58740 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b87c180 0x55ef3b87e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f2169c58740 4 client.17616 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef3b776e80 con 0x55ef3b87c180 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f215ffff640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b87c180 0x55ef3b87e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f215ffff640 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b87c180 0x55ef3b87e5a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=771d10664dccf38e server_cookie=48753e4470c25e39 in_seq=0 out_seq=0 2024-03-09T00:00:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.587+0000 7f213ffff640 10 client.17616 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:51.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 659413 11.17 MB/sec execute 179 sec latency 102.386 ms 2024-03-09T00:00:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.847+0000 7f213ffff640 1 -- 172.21.15.22:0/2936271446 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef3b776e80 con 0x55ef3b87c180 2024-03-09T00:00:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.847+0000 7f213ffff640 10 client.17616 handle_command_reply: tid=0 2024-03-09T00:00:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.847+0000 7f2169c58740 1 client.17616 shutdown 2024-03-09T00:00:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80afd0 msgr2=0x55ef3b81b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80afd0 0x55ef3b81b6d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2154009ab0 tx=0x7f21540194c0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 shutdown_connections 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3b87c180 0x55ef3b87e5a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f213ffff640 0 client.17616 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef3b814550 0x55ef3b81a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef3b80bd40 0x55ef3b819b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 --2- 172.21.15.22:0/2936271446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef3b80afd0 0x55ef3b81b6d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.848+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 >> 172.21.15.22:0/2936271446 conn(0x55ef3b7d25b0 msgr2=0x55ef3b80fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.849+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 shutdown_connections 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.849+0000 7f2169c58740 1 -- 172.21.15.22:0/2936271446 wait complete. 2024-03-09T00:00:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.849+0000 7f2169c58740 20 client.17616 trim_cache size 0 max 16384 2024-03-09T00:00:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:00:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:00:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:00:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f21500a2a10 msgr2=0x7f21500a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f21500a2a10 0x7f21500a4ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2158005cc0 tx=0x7f2158009820 comp rx=0 tx=0).stop 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160101e00 msgr2=0x7f21601980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160101e00 0x7f21601980d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f214c00e9f0 tx=0x7f214c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 shutdown_connections 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f21500a2a10 0x7f21500a4ed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21601027a0 0x7f216019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2160101e00 0x7f21601980d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 --2- 172.21.15.22:0/3645571965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2160100520 0x7f2160197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 >> 172.21.15.22:0/3645571965 conn(0x7f2160077310 msgr2=0x7f2160069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 shutdown_connections 2024-03-09T00:00:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:00:51.850+0000 7f216707e640 1 -- 172.21.15.22:0/3645571965 wait complete. 2024-03-09T00:00:52.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:52 smithi022 ceph-mon[31894]: pgmap v196: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-03-09T00:00:52.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:52 smithi062 ceph-mon[36129]: pgmap v196: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-03-09T00:00:52.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:52 smithi192 ceph-mon[35995]: pgmap v196: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-03-09T00:00:52.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 661131 11.17 MB/sec execute 180 sec latency 195.180 ms 2024-03-09T00:00:53.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:00:53.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:00:53.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 -- 172.21.15.22:0/488845867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81003d0 msgr2=0x7f81e81007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 --2- 172.21.15.22:0/488845867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81003d0 0x7f81e81007b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009ca0 tx=0x7f81d802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 -- 172.21.15.22:0/488845867 shutdown_connections 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 --2- 172.21.15.22:0/488845867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81054a0 0x7f81e810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 --2- 172.21.15.22:0/488845867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e8100d80 0x7f81e8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 --2- 172.21.15.22:0/488845867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81003d0 0x7f81e81007b0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 -- 172.21.15.22:0/488845867 >> 172.21.15.22:0/488845867 conn(0x7f81e80772f0 msgr2=0x7f81e8077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 -- 172.21.15.22:0/488845867 shutdown_connections 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 -- 172.21.15.22:0/488845867 wait complete. 2024-03-09T00:00:53.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 Processor -- start 2024-03-09T00:00:53.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.499+0000 7f81ed659640 1 -- start start 2024-03-09T00:00:53.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81ed659640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81003d0 0x7f81e8101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81ed659640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81054a0 0x7f81e81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81ed659640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e81028e0 0x7f81e81a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81ed659640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8110560 con 0x7f81e81054a0 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81ed659640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81103e0 con 0x7f81e81028e0 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81ed659640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81106e0 con 0x7f81e81003d0 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81054a0 0x7f81e81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81054a0 0x7f81e81023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49582/0 (socket says 172.21.15.22:49582) 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 -- 172.21.15.22:0/2180503737 learned_addr learned my addr 172.21.15.22:0/2180503737 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81003d0 0x7f81e8101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e77fe640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e81028e0 0x7f81e81a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 -- 172.21.15.22:0/2180503737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81003d0 msgr2=0x7f81e8101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81003d0 0x7f81e8101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 -- 172.21.15.22:0/2180503737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e81028e0 msgr2=0x7f81e81a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e81028e0 0x7f81e81a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 -- 172.21.15.22:0/2180503737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e81a6440 con 0x7f81e81054a0 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e77fe640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e81028e0 0x7f81e81a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.500+0000 7f81e67fc640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81054a0 0x7f81e81023a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9c0 tx=0x7f81dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.501+0000 7f81cffff640 1 -- 172.21.15.22:0/2180503737 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00cd80 con 0x7f81e81054a0 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.501+0000 7f81cffff640 1 -- 172.21.15.22:0/2180503737 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81dc005500 con 0x7f81e81054a0 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.501+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e81a6730 con 0x7f81e81054a0 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.501+0000 7f81cffff640 1 -- 172.21.15.22:0/2180503737 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc010690 con 0x7f81e81054a0 2024-03-09T00:00:53.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.501+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e81a6c80 con 0x7f81e81054a0 2024-03-09T00:00:53.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81cffff640 1 -- 172.21.15.22:0/2180503737 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f81dc0046c0 con 0x7f81e81054a0 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 Processor -- start 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 -- start start 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab47760 0x55a1bab47b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1bab50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1baaaae00 con 0x55a1bab47760 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1baa53b10 con 0x55a1bab48080 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.503+0000 7f81f0236740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1baaa6ea0 con 0x55a1bab50c60 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81cffff640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f81d00a29a0 0x7f81d00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1bab50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1bab50720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36398/0 (socket says 172.21.15.22:36398) 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 -- 172.21.15.22:0/3684477791 learned_addr learned my addr 172.21.15.22:0/3684477791 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:53.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e77fe640 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab47760 0x55a1bab47b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e77fe640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f81d00a29a0 0x7f81d00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81cffff640 1 -- 172.21.15.22:0/2180503737 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f81dc0cf5c0 con 0x7f81e81054a0 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 -- 172.21.15.22:0/3684477791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 msgr2=0x55a1bab53050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab53050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 -- 172.21.15.22:0/3684477791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab47760 msgr2=0x55a1bab47b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab47760 0x55a1bab47b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 -- 172.21.15.22:0/3684477791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ba8897d0 con 0x55a1bab48080 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e77fe640 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab47760 0x55a1bab47b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e77fe640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f81d00a29a0 0x7f81d00a4e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009fd0 tx=0x7f81e0005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1bab50720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81d802c700 tx=0x7f81d80098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271d1cac3416f923 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81cd7fa640 1 -- 172.21.15.22:0/3684477791 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8043070 con 0x55a1bab48080 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81cd7fa640 1 -- 172.21.15.22:0/3684477791 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81d8032c90 con 0x55a1bab48080 2024-03-09T00:00:53.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.504+0000 7f81cd7fa640 1 -- 172.21.15.22:0/3684477791 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8039040 con 0x55a1bab48080 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 -- 172.21.15.22:0/3684477791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 msgr2=0x55a1bab50720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1bab50720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81d802c700 tx=0x7f81d80098c0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 -- 172.21.15.22:0/3684477791 shutdown_connections 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab53050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1bab50720 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 --2- 172.21.15.22:0/3684477791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab47760 0x55a1bab47b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 -- 172.21.15.22:0/3684477791 >> 172.21.15.22:0/3684477791 conn(0x55a1bab41e90 msgr2=0x55a1bab42270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 -- 172.21.15.22:0/3684477791 shutdown_connections 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 -- 172.21.15.22:0/3684477791 wait complete. 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.505+0000 7f81f0236740 1 -- start start 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 1 Processor -- start 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1bab50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab4d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab50c60 0x55a1bab4dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bab539f0 con 0x55a1bab48080 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1baa670a0 con 0x55a1bab50c60 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bab53b70 con 0x55a1bab47760 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81f0236740 10 client.0 _ll_register_callbacks cb 0x55a1baa66a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab4d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab4d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49596/0 (socket says 172.21.15.22:49596) 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1bab50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 -- 172.21.15.22:0/214237360 learned_addr learned my addr 172.21.15.22:0/214237360 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e77fe640 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab50c60 0x55a1bab4dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 -- 172.21.15.22:0/214237360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 msgr2=0x55a1bab50310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1bab50310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 -- 172.21.15.22:0/214237360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab50c60 msgr2=0x55a1bab4dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab50c60 0x55a1bab4dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.506+0000 7f81e67fc640 1 -- 172.21.15.22:0/214237360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1ba7bf3f0 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81caff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81e67fc640 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab4d6a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f81dc097c80 tx=0x7f81dc06a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126a6eedc9882f57 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81caff5640 1 -- 172.21.15.22:0/214237360 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0ccd90 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81caff5640 1 -- 172.21.15.22:0/214237360 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f81dc0ccb90 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81d8032c90 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81caff5640 1 -- 172.21.15.22:0/214237360 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81dc09b820 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81f0236740 10 client.17628 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a1ba77f0f0 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.507+0000 7f81caff5640 1 -- 172.21.15.22:0/214237360 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81d8032c90 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81caff5640 1 -- 172.21.15.22:0/214237360 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f81dc0a3b30 con 0x55a1bab48080 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81f0236740 10 client.17628 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81f0236740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:53.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81f0236740 20 client.17628 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81f0236740 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1bab60680 0x55a1bab62ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81f0236740 4 client.17628 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1baab8340 con 0x55a1bab60680 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1bab60680 0x55a1bab62ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1bab60680 0x55a1bab62ab0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3433150bba89f250 server_cookie=1c5e8fee16fd0532 in_seq=0 out_seq=0 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.508+0000 7f81caff5640 10 client.17628 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81caff5640 1 -- 172.21.15.22:0/214237360 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a1baab8340 con 0x55a1bab60680 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81caff5640 10 client.17628 handle_command_reply: tid=0 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 client.17628 shutdown 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 msgr2=0x55a1bab4d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab4d6a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f81dc097c80 tx=0x7f81dc06a630 comp rx=0 tx=0).stop 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 shutdown_connections 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1bab60680 0x55a1bab62ab0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab50c60 0x55a1bab4dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab4d6a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 --2- 172.21.15.22:0/214237360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1bab50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 >> 172.21.15.22:0/214237360 conn(0x55a1bab41e90 msgr2=0x55a1bab43be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81caff5640 0 client.17628 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 shutdown_connections 2024-03-09T00:00:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 1 -- 172.21.15.22:0/214237360 wait complete. 2024-03-09T00:00:53.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.509+0000 7f81f0236740 20 client.17628 trim_cache size 0 max 16384 2024-03-09T00:00:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.524+0000 7f81f0236740 1 Processor -- start 2024-03-09T00:00:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.524+0000 7f81f0236740 1 -- start start 2024-03-09T00:00:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81f0236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab47760 0x55a1bab4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81f0236740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81f0236740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab4d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81f0236740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81b2c70 con 0x55a1bab48080 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81f0236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81b0810 con 0x55a1bab47760 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81f0236740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81ae8b0 con 0x55a1bab50c60 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab50310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49608/0 (socket says 172.21.15.22:49608) 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 -- 172.21.15.22:0/3497317624 learned_addr learned my addr 172.21.15.22:0/3497317624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab47760 0x55a1bab4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e77fe640 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab4d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 -- 172.21.15.22:0/3497317624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 msgr2=0x55a1bab4d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab4d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 -- 172.21.15.22:0/3497317624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab47760 msgr2=0x55a1bab4fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab47760 0x55a1bab4fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 -- 172.21.15.22:0/3497317624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1bab41670 con 0x55a1bab48080 2024-03-09T00:00:53.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81e67fc640 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab50310 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c6a0 tx=0x7f81dc0a3ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793d080bc6c43c13 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81ccff9640 1 -- 172.21.15.22:0/3497317624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc09f970 con 0x55a1bab48080 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81ccff9640 1 -- 172.21.15.22:0/3497317624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81dc0ccd90 con 0x55a1bab48080 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.525+0000 7f81ccff9640 1 -- 172.21.15.22:0/3497317624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc06a1c0 con 0x55a1bab48080 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 -- 172.21.15.22:0/3497317624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 msgr2=0x55a1bab50310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab50310 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c6a0 tx=0x7f81dc0a3ef0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 -- 172.21.15.22:0/3497317624 shutdown_connections 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab50c60 0x55a1bab4d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab48080 0x55a1bab50310 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 --2- 172.21.15.22:0/3497317624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab47760 0x55a1bab4fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 -- 172.21.15.22:0/3497317624 >> 172.21.15.22:0/3497317624 conn(0x55a1bab41e90 msgr2=0x55a1bab67410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 -- 172.21.15.22:0/3497317624 shutdown_connections 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 -- 172.21.15.22:0/3497317624 wait complete. 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 -- start start 2024-03-09T00:00:53.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.526+0000 7f81f0236740 1 Processor -- start 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1babaee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1babaf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab50c60 0x55a1bab541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81aeeb0 con 0x55a1bab50c60 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81b01b0 con 0x55a1bab48080 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81af4f0 con 0x55a1bab47760 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 10 client.0 _ll_register_callbacks cb 0x7f81dc097b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab50c60 0x55a1bab541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab50c60 0x55a1bab541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49614/0 (socket says 172.21.15.22:49614) 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e77fe640 1 -- 172.21.15.22:0/120356790 learned_addr learned my addr 172.21.15.22:0/120356790 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1babaee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e67fc640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1babaf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e6ffd640 1 -- 172.21.15.22:0/120356790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 msgr2=0x55a1babaf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1babaf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e6ffd640 1 -- 172.21.15.22:0/120356790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab50c60 msgr2=0x55a1bab541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab50c60 0x55a1bab541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e6ffd640 1 -- 172.21.15.22:0/120356790 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1bab54890 con 0x55a1bab47760 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e67fc640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1babaf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e77fe640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab50c60 0x55a1bab541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81e6ffd640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1babaee80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81d8037040 tx=0x7f81d802c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c70863c5d37e443c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81caff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81caff5640 1 -- 172.21.15.22:0/120356790 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8054070 con 0x55a1bab47760 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81caff5640 1 -- 172.21.15.22:0/120356790 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f81d8050be0 con 0x55a1bab47760 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.527+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81dc0ccd90 con 0x55a1bab47760 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81caff5640 1 -- 172.21.15.22:0/120356790 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81d8039040 con 0x55a1bab47760 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 10 client.27079 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a1babb7760 con 0x55a1bab47760 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81caff5640 1 -- 172.21.15.22:0/120356790 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81dc0ccd90 con 0x55a1bab47760 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81caff5640 1 -- 172.21.15.22:0/120356790 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f81d8050390 con 0x55a1bab47760 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 10 client.27079 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 10 client.27079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 20 client.27079 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1babb7a50 0x55a1babb9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 4 client.27079 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a1baab2690 con 0x55a1babb7a50 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.528+0000 7f81e67fc640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1babb7a50 0x55a1babb9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81caff5640 10 client.27079 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81e67fc640 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1babb7a50 0x55a1babb9e40 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=516d7c2a9030fde8 server_cookie=ee253703e1ecb992 in_seq=0 out_seq=0 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81caff5640 1 -- 172.21.15.22:0/120356790 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a1baab2690 con 0x55a1babb7a50 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81caff5640 10 client.27079 handle_command_reply: tid=0 2024-03-09T00:00:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 client.27079 shutdown 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 msgr2=0x55a1babaee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1babaee80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81d8037040 tx=0x7f81d802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 shutdown_connections 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a1babb7a50 0x55a1babb9e40 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1bab50c60 0x55a1bab541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a1bab48080 0x55a1babaf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81caff5640 0 client.27079 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 --2- 172.21.15.22:0/120356790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1bab47760 0x55a1babaee80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 >> 172.21.15.22:0/120356790 conn(0x55a1bab41e90 msgr2=0x55a1bab4d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 shutdown_connections 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 1 -- 172.21.15.22:0/120356790 wait complete. 2024-03-09T00:00:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81f0236740 20 client.27079 trim_cache size 0 max 16384 2024-03-09T00:00:53.532 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:00:53.532 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:00:53.532 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:00:53.532 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f81d00a29a0 msgr2=0x7f81d00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81ed659640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f81d00a29a0 0x7f81d00a4e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009fd0 tx=0x7f81e0005f90 comp rx=0 tx=0).stop 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81054a0 msgr2=0x7f81e81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.529+0000 7f81ed659640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81054a0 0x7f81e81023a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9c0 tx=0x7f81dc00ee80 comp rx=0 tx=0).stop 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 shutdown_connections 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f81d00a29a0 0x7f81d00a4e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e81028e0 0x7f81e81a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e81054a0 0x7f81e81023a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 --2- 172.21.15.22:0/2180503737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81e81003d0 0x7f81e8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 >> 172.21.15.22:0/2180503737 conn(0x7f81e80772f0 msgr2=0x7f81e80687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 shutdown_connections 2024-03-09T00:00:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.530+0000 7f81ed659640 1 -- 172.21.15.22:0/2180503737 wait complete. 2024-03-09T00:00:53.536 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:00:53.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:00:53.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3305898578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01001f0 msgr2=0x7f3ce0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3305898578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01001f0 0x7f3ce0100650 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009900 tx=0x7f3ccc02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3305898578 shutdown_connections 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3305898578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce0100b90 0x7f3ce010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3305898578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01001f0 0x7f3ce0100650 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3305898578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ce01062b0 0x7f3ce0106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3305898578 >> 172.21.15.22:0/3305898578 conn(0x7f3ce0077320 msgr2=0x7f3ce0077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3305898578 shutdown_connections 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.649+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3305898578 wait complete. 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 Processor -- start 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 -- start start 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce01001f0 0x7f3ce019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ce0100b90 0x7f3ce019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01062b0 0x7f3ce0197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0110570 con 0x7f3ce01062b0 2024-03-09T00:00:53.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce01103f0 con 0x7f3ce01001f0 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce6fc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce01106f0 con 0x7f3ce0100b90 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce5561640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01062b0 0x7f3ce0197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce5561640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01062b0 0x7f3ce0197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49634/0 (socket says 172.21.15.22:49634) 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce5561640 1 -- 172.21.15.22:0/3950752202 learned_addr learned my addr 172.21.15.22:0/3950752202 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3cdffff640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ce0100b90 0x7f3ce019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.650+0000 7f3ce4d60640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce01001f0 0x7f3ce019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce5561640 1 -- 172.21.15.22:0/3950752202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ce0100b90 msgr2=0x7f3ce019dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce5561640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ce0100b90 0x7f3ce019dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce5561640 1 -- 172.21.15.22:0/3950752202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce01001f0 msgr2=0x7f3ce019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce5561640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce01001f0 0x7f3ce019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce5561640 1 -- 172.21.15.22:0/3950752202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0198100 con 0x7f3ce01062b0 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce4d60640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce01001f0 0x7f3ce019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:00:53.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce5561640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01062b0 0x7f3ce0197980 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800ea60 tx=0x7f3cd800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3cddffb640 1 -- 172.21.15.22:0/3950752202 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800ce40 con 0x7f3ce01062b0 2024-03-09T00:00:53.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3cddffb640 1 -- 172.21.15.22:0/3950752202 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cd8005020 con 0x7f3ce01062b0 2024-03-09T00:00:53.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce01983f0 con 0x7f3ce01062b0 2024-03-09T00:00:53.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3cddffb640 1 -- 172.21.15.22:0/3950752202 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8010690 con 0x7f3ce01062b0 2024-03-09T00:00:53.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.651+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce01a4640 con 0x7f3ce01062b0 2024-03-09T00:00:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.652+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce0067850 con 0x7f3ce01062b0 2024-03-09T00:00:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.654+0000 7f3cddffb640 1 -- 172.21.15.22:0/3950752202 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3cd8005520 con 0x7f3ce01062b0 2024-03-09T00:00:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.654+0000 7f3cddffb640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cd00a29f0 0x7f3cd00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.654+0000 7f3cddffb640 1 -- 172.21.15.22:0/3950752202 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f3cd80cf700 con 0x7f3ce01062b0 2024-03-09T00:00:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.654+0000 7f3ce4d60640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cd00a29f0 0x7f3cd00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.655+0000 7f3ce4d60640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cd00a29f0 0x7f3cd00a4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0105a60 tx=0x7f3cd4005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:53.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.659+0000 7f3cddffb640 1 -- 172.21.15.22:0/3950752202 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3cd809c0c0 con 0x7f3ce01062b0 2024-03-09T00:00:53.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 663518 11.18 MB/sec execute 181 sec latency 17.068 ms 2024-03-09T00:00:53.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.897+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ce0198e60 con 0x7f3ce01062b0 2024-03-09T00:00:53.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.898+0000 7f3cddffb640 1 -- 172.21.15.22:0/3950752202 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f3cd809b9c0 con 0x7f3ce01062b0 2024-03-09T00:00:53.893 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:00:53.893 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:51.502302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26555":{"gid":26555,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2421521561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2421521561},{"type":"v1","addr":"172.21.15.62:6837","nonce":2421521561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:00:53.894 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-03-09T00:00:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cd00a29f0 msgr2=0x7f3cd00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cd00a29f0 0x7f3cd00a4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0105a60 tx=0x7f3cd4005f90 comp rx=0 tx=0).stop 2024-03-09T00:00:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01062b0 msgr2=0x7f3ce0197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01062b0 0x7f3ce0197980 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800ea60 tx=0x7f3cd800ef20 comp rx=0 tx=0).stop 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 shutdown_connections 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cd00a29f0 0x7f3cd00a4eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ce01062b0 0x7f3ce0197980 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ce0100b90 0x7f3ce019dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 --2- 172.21.15.22:0/3950752202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ce01001f0 0x7f3ce019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 >> 172.21.15.22:0/3950752202 conn(0x7f3ce0077320 msgr2=0x7f3ce00ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 shutdown_connections 2024-03-09T00:00:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:53.900+0000 7f3ce6fc4640 1 -- 172.21.15.22:0/3950752202 wait complete. 2024-03-09T00:00:53.906 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:00:53.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:00:54.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 -- 172.21.15.22:0/3589375536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 msgr2=0x7fbd5010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 --2- 172.21.15.22:0/3589375536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5010d080 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009ca0 tx=0x7fbd4802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 -- 172.21.15.22:0/3589375536 shutdown_connections 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 --2- 172.21.15.22:0/3589375536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5010d080 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 --2- 172.21.15.22:0/3589375536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd50100d90 0x7fbd50104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 --2- 172.21.15.22:0/3589375536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd501003e0 0x7fbd501007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 -- 172.21.15.22:0/3589375536 >> 172.21.15.22:0/3589375536 conn(0x7fbd500773a0 msgr2=0x7fbd500777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 -- 172.21.15.22:0/3589375536 shutdown_connections 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.020+0000 7fbd56d59640 1 -- 172.21.15.22:0/3589375536 wait complete. 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 Processor -- start 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 -- start start 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd501003e0 0x7fbd5019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd50197940 0x7fbd50197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd50110570 con 0x7fbd501054b0 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501103f0 con 0x7fbd501003e0 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd56d59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501106f0 con 0x7fbd50197940 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd47fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd501003e0 0x7fbd5019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd552f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd50197940 0x7fbd50197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd552f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd50197940 0x7fbd50197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35494/0 (socket says 172.21.15.22:35494) 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd47fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5019dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49642/0 (socket says 172.21.15.22:49642) 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd552f6640 1 -- 172.21.15.22:0/2199388873 learned_addr learned my addr 172.21.15.22:0/2199388873 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.021+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd501003e0 0x7fbd5019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36466/0 (socket says 172.21.15.22:36466) 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd47fff640 1 -- 172.21.15.22:0/2199388873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd50197940 msgr2=0x7fbd50197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd47fff640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd50197940 0x7fbd50197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd47fff640 1 -- 172.21.15.22:0/2199388873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd501003e0 msgr2=0x7fbd5019d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd47fff640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd501003e0 0x7fbd5019d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd47fff640 1 -- 172.21.15.22:0/2199388873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd50198460 con 0x7fbd501054b0 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd552f6640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd50197940 0x7fbd50197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd54af5640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd501003e0 0x7fbd5019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd47fff640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5019dcd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000e9c0 tx=0x7fbd4000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd45ffb640 1 -- 172.21.15.22:0/2199388873 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000ccc0 con 0x7fbd501054b0 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd45ffb640 1 -- 172.21.15.22:0/2199388873 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd40010910 con 0x7fbd501054b0 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd50198750 con 0x7fbd501054b0 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd45ffb640 1 -- 172.21.15.22:0/2199388873 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd400188c0 con 0x7fbd501054b0 2024-03-09T00:00:54.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.022+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd501a49a0 con 0x7fbd501054b0 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.024+0000 7fbd45ffb640 1 -- 172.21.15.22:0/2199388873 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbd40010ac0 con 0x7fbd501054b0 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 Processor -- start 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 -- start start 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3001a0 0x55d8de300580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de300ac0 0x55d8de309160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de30ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de2c7360 con 0x55d8de300ac0 2024-03-09T00:00:54.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de263ec0 con 0x55d8de3001a0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de20cd30 con 0x55d8de3096a0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de300ac0 0x55d8de309160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd552f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3001a0 0x55d8de300580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de300ac0 0x55d8de309160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49646/0 (socket says 172.21.15.22:49646) 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd54af5640 1 -- 172.21.15.22:0/1348763513 learned_addr learned my addr 172.21.15.22:0/1348763513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd45ffb640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbd380a2970 0x7fbd380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd47fff640 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de30ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd552f6640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbd380a2970 0x7fbd380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd552f6640 1 -- 172.21.15.22:0/1348763513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 msgr2=0x55d8de30ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd552f6640 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de30ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd552f6640 1 -- 172.21.15.22:0/1348763513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de300ac0 msgr2=0x55d8de309160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd45ffb640 1 -- 172.21.15.22:0/2199388873 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fbd400cf480 con 0x7fbd501054b0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.025+0000 7fbd552f6640 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de300ac0 0x55d8de309160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.026+0000 7fbd552f6640 1 -- 172.21.15.22:0/1348763513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8de20bbc0 con 0x55d8de3001a0 2024-03-09T00:00:54.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.026+0000 7fbd54af5640 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de300ac0 0x55d8de309160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.026+0000 7fbd552f6640 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3001a0 0x55d8de300580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005e30 tx=0x7fbd4802cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8bc4dfd025a70f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.026+0000 7fbd552f6640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbd380a2970 0x7fbd380a4e30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd4804b000 tx=0x7fbd4804c000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.026+0000 7fbd36ffd640 1 -- 172.21.15.22:0/1348763513 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48054070 con 0x55d8de3001a0 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.026+0000 7fbd36ffd640 1 -- 172.21.15.22:0/1348763513 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd48032a20 con 0x55d8de3001a0 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.026+0000 7fbd36ffd640 1 -- 172.21.15.22:0/1348763513 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48039040 con 0x55d8de3001a0 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 -- 172.21.15.22:0/1348763513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3001a0 msgr2=0x55d8de300580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3001a0 0x55d8de300580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005e30 tx=0x7fbd4802cf30 comp rx=0 tx=0).stop 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 -- 172.21.15.22:0/1348763513 shutdown_connections 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de30ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de300ac0 0x55d8de309160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/1348763513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3001a0 0x55d8de300580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 -- 172.21.15.22:0/1348763513 >> 172.21.15.22:0/1348763513 conn(0x55d8de2fa8d0 msgr2=0x55d8de2facb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 -- 172.21.15.22:0/1348763513 shutdown_connections 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 -- 172.21.15.22:0/1348763513 wait complete. 2024-03-09T00:00:54.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 -- start start 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.027+0000 7fbd59a0a740 1 Processor -- start 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de306750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de300ac0 0x55d8de306c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de3071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de2024a0 con 0x55d8de3001a0 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de25ff60 con 0x55d8de300ac0 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de30c430 con 0x55d8de3096a0 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 10 client.0 _ll_register_callbacks cb 0x55d8de25dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de306750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de306750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49650/0 (socket says 172.21.15.22:49650) 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 -- 172.21.15.22:0/106653854 learned_addr learned my addr 172.21.15.22:0/106653854 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd552f6640 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de3071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd47fff640 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de300ac0 0x55d8de306c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 -- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 msgr2=0x55d8de3071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de3071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 -- 172.21.15.22:0/106653854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de300ac0 msgr2=0x55d8de306c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de300ac0 0x55d8de306c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 -- 172.21.15.22:0/106653854 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8ddf783f0 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd54af5640 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de306750 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c012d20 tx=0x7fbd4c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bddd527d26ed17 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd2ffff640 1 -- 172.21.15.22:0/106653854 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c012110 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd2ffff640 1 -- 172.21.15.22:0/106653854 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fbd4c004050 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.028+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd48032a20 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd2ffff640 1 -- 172.21.15.22:0/106653854 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd4c0122b0 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 10 client.17652 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d8de0427d0 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd2ffff640 1 -- 172.21.15.22:0/106653854 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd48032a20 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd2ffff640 1 -- 172.21.15.22:0/106653854 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd4c0104c0 con 0x55d8de3001a0 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 10 client.17652 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:54.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 10 client.17652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 20 client.17652 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de319930 0x55d8de31bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 4 client.17652 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8de271400 con 0x55d8de319930 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.029+0000 7fbd47fff640 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de319930 0x55d8de31bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd47fff640 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de319930 0x55d8de31bd20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc6bd647e5d63b66 server_cookie=57833ebff50dc3dd in_seq=0 out_seq=0 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd2ffff640 10 client.17652 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd2ffff640 1 -- 172.21.15.22:0/106653854 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8de271400 con 0x55d8de319930 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd2ffff640 10 client.17652 handle_command_reply: tid=0 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 client.17652 shutdown 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 msgr2=0x55d8de306750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de306750 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c012d20 tx=0x7fbd4c00f790 comp rx=0 tx=0).stop 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 shutdown_connections 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de319930 0x55d8de31bd20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de3096a0 0x55d8de3071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd2ffff640 0 client.17652 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de300ac0 0x55d8de306c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/106653854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de306750 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.030+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 >> 172.21.15.22:0/106653854 conn(0x55d8de2fa8d0 msgr2=0x55d8de2fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.031+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 shutdown_connections 2024-03-09T00:00:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.031+0000 7fbd59a0a740 1 -- 172.21.15.22:0/106653854 wait complete. 2024-03-09T00:00:54.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.031+0000 7fbd59a0a740 20 client.17652 trim_cache size 0 max 16384 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.042+0000 7fbd59a0a740 1 Processor -- start 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.042+0000 7fbd59a0a740 1 -- start start 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de308e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de301a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de301fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501ae870 con 0x55d8de3001a0 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501ac410 con 0x55d8de3096a0 2024-03-09T00:00:54.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501aa4b0 con 0x55d8de300ac0 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de308e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de308e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49666/0 (socket says 172.21.15.22:49666) 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd552f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de301fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd54af5640 1 -- 172.21.15.22:0/293212099 learned_addr learned my addr 172.21.15.22:0/293212099 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd47fff640 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de301a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd552f6640 1 -- 172.21.15.22:0/293212099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 msgr2=0x55d8de301a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd552f6640 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de301a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd552f6640 1 -- 172.21.15.22:0/293212099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 msgr2=0x55d8de308e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd552f6640 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de308e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd552f6640 1 -- 172.21.15.22:0/293212099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8de30be00 con 0x55d8de3096a0 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd54af5640 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de308e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd552f6640 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de301fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd4802c700 tx=0x7fbd48003d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6bb126c4c84459d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd367fc640 1 -- 172.21.15.22:0/293212099 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4805c070 con 0x55d8de3096a0 2024-03-09T00:00:54.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd367fc640 1 -- 172.21.15.22:0/293212099 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd48003b70 con 0x55d8de3096a0 2024-03-09T00:00:54.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.043+0000 7fbd367fc640 1 -- 172.21.15.22:0/293212099 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48039050 con 0x55d8de3096a0 2024-03-09T00:00:54.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 -- 172.21.15.22:0/293212099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 msgr2=0x55d8de301fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de301fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd4802c700 tx=0x7fbd48003d80 comp rx=0 tx=0).stop 2024-03-09T00:00:54.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 -- 172.21.15.22:0/293212099 shutdown_connections 2024-03-09T00:00:54.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de301fc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de301a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/293212099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de308e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 -- 172.21.15.22:0/293212099 >> 172.21.15.22:0/293212099 conn(0x55d8de2fa8d0 msgr2=0x55d8de320660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 -- 172.21.15.22:0/293212099 shutdown_connections 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 -- 172.21.15.22:0/293212099 wait complete. 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 -- start start 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.044+0000 7fbd59a0a740 1 Processor -- start 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de310760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de30ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd59a0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de30f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501aaab0 con 0x55d8de3001a0 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501abdb0 con 0x55d8de3096a0 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd59a0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501ab0f0 con 0x55d8de300ac0 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd59a0a740 10 client.0 _ll_register_callbacks cb 0x7fbd40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:54.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de310760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de310760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49682/0 (socket says 172.21.15.22:49682) 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 -- 172.21.15.22:0/54044307 learned_addr learned my addr 172.21.15.22:0/54044307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd552f6640 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de30f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd47fff640 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de30ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 -- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 msgr2=0x55d8de30ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de30ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 -- 172.21.15.22:0/54044307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 msgr2=0x55d8de30f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de30f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.045+0000 7fbd54af5640 1 -- 172.21.15.22:0/54044307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8de30f6d0 con 0x55d8de3001a0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd54af5640 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de310760 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c010860 tx=0x7fbd4c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b905545935d3f3a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd2ffff640 1 -- 172.21.15.22:0/54044307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c012110 con 0x55d8de3001a0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd2ffff640 1 -- 172.21.15.22:0/54044307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fbd4c005250 con 0x55d8de3001a0 2024-03-09T00:00:54.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd48003b70 con 0x55d8de3001a0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd2ffff640 1 -- 172.21.15.22:0/54044307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd4c03a050 con 0x55d8de3001a0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 10 client.17658 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d8de370b00 con 0x55d8de3001a0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd2ffff640 1 -- 172.21.15.22:0/54044307 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd48003b70 con 0x55d8de3001a0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd2ffff640 1 -- 172.21.15.22:0/54044307 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd4c003ce0 con 0x55d8de3001a0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 10 client.17658 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 10 client.17658 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 20 client.17658 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de3711d0 0x55d8de3735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 4 client.17658 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8de26b750 con 0x55d8de3711d0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.046+0000 7fbd47fff640 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de3711d0 0x55d8de3735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd47fff640 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de3711d0 0x55d8de3735f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b63733aceb8eafd server_cookie=3b388d530eab1309 in_seq=0 out_seq=0 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd2ffff640 10 client.17658 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:54.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd2ffff640 1 -- 172.21.15.22:0/54044307 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8de26b750 con 0x55d8de3711d0 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd2ffff640 10 client.17658 handle_command_reply: tid=0 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 client.17658 shutdown 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 msgr2=0x55d8de310760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de310760 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c010860 tx=0x7fbd4c010ce0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 shutdown_connections 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d8de3711d0 0x55d8de3735f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8de3096a0 0x55d8de30f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8de300ac0 0x55d8de30ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd2ffff640 0 client.17658 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 --2- 172.21.15.22:0/54044307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8de3001a0 0x55d8de310760 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 >> 172.21.15.22:0/54044307 conn(0x55d8de2fa8d0 msgr2=0x55d8de301a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 shutdown_connections 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 1 -- 172.21.15.22:0/54044307 wait complete. 2024-03-09T00:00:54.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.047+0000 7fbd59a0a740 20 client.17658 trim_cache size 0 max 16384 2024-03-09T00:00:54.050 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:00:54.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbd380a2970 msgr2=0x7fbd380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbd380a2970 0x7fbd380a4e30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd4804b000 tx=0x7fbd4804c000 comp rx=0 tx=0).stop 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 msgr2=0x7fbd5019dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5019dcd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000e9c0 tx=0x7fbd4000ee80 comp rx=0 tx=0).stop 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 shutdown_connections 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbd380a2970 0x7fbd380a4e30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd50197940 0x7fbd50197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd501054b0 0x7fbd5019dcd0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 --2- 172.21.15.22:0/2199388873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd501003e0 0x7fbd5019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 >> 172.21.15.22:0/2199388873 conn(0x7fbd500773a0 msgr2=0x7fbd500686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 shutdown_connections 2024-03-09T00:00:54.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.048+0000 7fbd56d59640 1 -- 172.21.15.22:0/2199388873 wait complete. 2024-03-09T00:00:54.054 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:00:54.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 -- 172.21.15.22:0/3178616648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1003e0 msgr2=0x7fb1cc1007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/3178616648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1003e0 0x7fb1cc1007c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009900 tx=0x7fb1c002bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 -- 172.21.15.22:0/3178616648 shutdown_connections 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/3178616648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc1054b0 0x7fb1cc10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/3178616648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc100d90 0x7fb1cc104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/3178616648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1003e0 0x7fb1cc1007c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 -- 172.21.15.22:0/3178616648 >> 172.21.15.22:0/3178616648 conn(0x7fb1cc0773a0 msgr2=0x7fb1cc0777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 -- 172.21.15.22:0/3178616648 shutdown_connections 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.169+0000 7fb1d2e81640 1 -- 172.21.15.22:0/3178616648 wait complete. 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 Processor -- start 2024-03-09T00:00:54.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 -- start start 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc1003e0 0x7fb1cc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc1054b0 0x7fb1cc19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1979c0 0x7fb1cc197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc111680 con 0x7fb1cc1979c0 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc111500 con 0x7fb1cc1054b0 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d2e81640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc111800 con 0x7fb1cc1003e0 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1979c0 0x7fb1cc197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1979c0 0x7fb1cc197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49706/0 (socket says 172.21.15.22:49706) 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1cbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc1054b0 0x7fb1cc19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 -- 172.21.15.22:0/2961965472 learned_addr learned my addr 172.21.15.22:0/2961965472 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc1003e0 0x7fb1cc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 -- 172.21.15.22:0/2961965472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc1003e0 msgr2=0x7fb1cc19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc1003e0 0x7fb1cc19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 -- 172.21.15.22:0/2961965472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc1054b0 msgr2=0x7fb1cc19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc1054b0 0x7fb1cc19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 -- 172.21.15.22:0/2961965472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc1984e0 con 0x7fb1cc1979c0 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc1054b0 0x7fb1cc19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.170+0000 7fb1d141e640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1979c0 0x7fb1cc197e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400eb10 tx=0x7fb1c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.171+0000 7fb1c9ffb640 1 -- 172.21.15.22:0/2961965472 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4004b20 con 0x7fb1cc1979c0 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.171+0000 7fb1c9ffb640 1 -- 172.21.15.22:0/2961965472 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1c4004cc0 con 0x7fb1cc1979c0 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.171+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc1987d0 con 0x7fb1cc1979c0 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.171+0000 7fb1c9ffb640 1 -- 172.21.15.22:0/2961965472 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4010760 con 0x7fb1cc1979c0 2024-03-09T00:00:54.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.171+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc1a49d0 con 0x7fb1cc1979c0 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.173+0000 7fb1c9ffb640 1 -- 172.21.15.22:0/2961965472 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb1c4005020 con 0x7fb1cc1979c0 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 Processor -- start 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 -- start start 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b4fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4feac0 0x55828b507160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b509a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828b4c5360 con 0x55828b4feac0 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828b461ec0 con 0x55828b5076a0 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1c9ffb640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1bc0a29f0 0x7fb1bc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828b40ad30 con 0x55828b4fe1a0 2024-03-09T00:00:54.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d0c1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4feac0 0x55828b507160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d0c1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4feac0 0x55828b507160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49720/0 (socket says 172.21.15.22:49720) 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d0c1d640 1 -- 172.21.15.22:0/2574741126 learned_addr learned my addr 172.21.15.22:0/2574741126 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b4fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1d141e640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1bc0a29f0 0x7fb1bc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b509a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.174+0000 7fb1c9ffb640 1 -- 172.21.15.22:0/2961965472 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb1c40cf6b0 con 0x7fb1cc1979c0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1cbfff640 1 -- 172.21.15.22:0/2574741126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 msgr2=0x55828b509a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b509a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1cbfff640 1 -- 172.21.15.22:0/2574741126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4feac0 msgr2=0x55828b507160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4feac0 0x55828b507160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1cbfff640 1 -- 172.21.15.22:0/2574741126 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55828b409bc0 con 0x55828b4fe1a0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4feac0 0x55828b507160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1d141e640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1bc0a29f0 0x7fb1bc0a4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40ccb10 tx=0x7fb1c4005b40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b4fe580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0ffcd0 tx=0x7fb1b8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=623e775c5d24419c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1b6ffd640 1 -- 172.21.15.22:0/2574741126 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8005020 con 0x55828b4fe1a0 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1b6ffd640 1 -- 172.21.15.22:0/2574741126 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b800f850 con 0x55828b4fe1a0 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.175+0000 7fb1b6ffd640 1 -- 172.21.15.22:0/2574741126 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8004aa0 con 0x55828b4fe1a0 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/2574741126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 msgr2=0x55828b4fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b4fe580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0ffcd0 tx=0x7fb1b8005850 comp rx=0 tx=0).stop 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/2574741126 shutdown_connections 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b509a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4feac0 0x55828b507160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/2574741126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b4fe580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/2574741126 >> 172.21.15.22:0/2574741126 conn(0x55828b4f88d0 msgr2=0x55828b4f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/2574741126 shutdown_connections 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/2574741126 wait complete. 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 -- start start 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.176+0000 7fb1d5a5d740 1 Processor -- start 2024-03-09T00:00:54.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4fe1a0 0x55828b504770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b504cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b5051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828b4004a0 con 0x55828b4fe1a0 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828b45df60 con 0x55828b5076a0 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828b50e8a0 con 0x55828b4feac0 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4fe1a0 0x55828b504770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d5a5d740 10 client.0 _ll_register_callbacks cb 0x55828b45bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b504cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4fe1a0 0x55828b504770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49724/0 (socket says 172.21.15.22:49724) 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1125237248 learned_addr learned my addr 172.21.15.22:0/1125237248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b504cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35560/0 (socket says 172.21.15.22:35560) 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b5051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 -- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 msgr2=0x55828b5051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b5051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 -- 172.21.15.22:0/1125237248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4fe1a0 msgr2=0x55828b504770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4fe1a0 0x55828b504770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 -- 172.21.15.22:0/1125237248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55828b1763f0 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b504cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55828b4fdc70 tx=0x7fb1c0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd0a11640c035f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.177+0000 7fb1abfff640 1 -- 172.21.15.22:0/1125237248 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0004240 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1abfff640 1 -- 172.21.15.22:0/1125237248 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb1c00043e0 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1b800f850 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1abfff640 1 -- 172.21.15.22:0/1125237248 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c00046c0 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 10 client.27115 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55828b2407d0 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1abfff640 1 -- 172.21.15.22:0/1125237248 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b800f850 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1abfff640 1 -- 172.21.15.22:0/1125237248 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb1c00035f0 con 0x55828b4feac0 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 10 client.27115 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 10 client.27115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 20 client.27115 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b51bd40 0x55828b51e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 4 client.27115 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55828b46f400 con 0x55828b51bd40 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.178+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b51bd40 0x55828b51e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b51bd40 0x55828b51e130 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a433da89ffdbadb server_cookie=449366e84045c400 in_seq=0 out_seq=0 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1abfff640 10 client.27115 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1abfff640 1 -- 172.21.15.22:0/1125237248 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55828b46f400 con 0x55828b51bd40 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1abfff640 10 client.27115 handle_command_reply: tid=0 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 client.27115 shutdown 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 msgr2=0x55828b504cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b504cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55828b4fdc70 tx=0x7fb1c0005b50 comp rx=0 tx=0).stop 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 shutdown_connections 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b51bd40 0x55828b51e130 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b5076a0 0x55828b5051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b504cb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1abfff640 0 client.27115 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:54.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1125237248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b4fe1a0 0x55828b504770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.179+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 >> 172.21.15.22:0/1125237248 conn(0x55828b4f88d0 msgr2=0x55828b4fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.180+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 shutdown_connections 2024-03-09T00:00:54.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.180+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1125237248 wait complete. 2024-03-09T00:00:54.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.180+0000 7fb1d5a5d740 20 client.27115 trim_cache size 0 max 16384 2024-03-09T00:00:54.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 Processor -- start 2024-03-09T00:00:54.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 -- start start 2024-03-09T00:00:54.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4fe1a0 0x55828b506ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b5019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b501f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc1ae930 con 0x55828b5076a0 2024-03-09T00:00:54.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc1ac4d0 con 0x55828b4fe1a0 2024-03-09T00:00:54.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc1aa570 con 0x55828b4feac0 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b501f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d0c1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b5019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b501f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49730/0 (socket says 172.21.15.22:49730) 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1642479862 learned_addr learned my addr 172.21.15.22:0/1642479862 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b5019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35574/0 (socket says 172.21.15.22:35574) 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.192+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4fe1a0 0x55828b506ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1642479862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 msgr2=0x55828b5019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b5019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1642479862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4fe1a0 msgr2=0x55828b506ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4fe1a0 0x55828b506ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1642479862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55828b4fd430 con 0x55828b5076a0 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b5019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b501f10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009b50 tx=0x7fb1b8023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d853a8af578cf95 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1642479862 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800fda0 con 0x55828b5076a0 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1642479862 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b8011740 con 0x55828b5076a0 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.193+0000 7fb1b67fc640 1 -- 172.21.15.22:0/1642479862 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80035f0 con 0x55828b5076a0 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1642479862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 msgr2=0x55828b501f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b501f10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009b50 tx=0x7fb1b8023040 comp rx=0 tx=0).stop 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1642479862 shutdown_connections 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b501f10 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4feac0 0x55828b5019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1642479862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4fe1a0 0x55828b506ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1642479862 >> 172.21.15.22:0/1642479862 conn(0x55828b4f88d0 msgr2=0x55828b503040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1642479862 shutdown_connections 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1642479862 wait complete. 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 -- start start 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 Processor -- start 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b511b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4feac0 0x55828b512080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b572200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.194+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc1aab70 con 0x55828b5076a0 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc1abe70 con 0x55828b4feac0 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1d5a5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc1ab1b0 con 0x55828b4fe1a0 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1d5a5d740 10 client.0 _ll_register_callbacks cb 0x7fb1b8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b511b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1d0c1d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4feac0 0x55828b512080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b511b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35584/0 (socket says 172.21.15.22:35584) 2024-03-09T00:00:54.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1302831711 learned_addr learned my addr 172.21.15.22:0/1302831711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b572200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4feac0 msgr2=0x55828b512080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4feac0 0x55828b512080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1302831711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 msgr2=0x55828b572200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b572200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 -- 172.21.15.22:0/1302831711 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55828b572740 con 0x55828b4fe1a0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4feac0 0x55828b512080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1cbfff640 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b511b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802e8c0 tx=0x7fb1b802e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144c6b9df9294c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1abfff640 1 -- 172.21.15.22:0/1302831711 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8000e60 con 0x55828b4fe1a0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1b8011740 con 0x55828b4fe1a0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.195+0000 7fb1abfff640 1 -- 172.21.15.22:0/1302831711 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7fb1b802e2e0 con 0x55828b4fe1a0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1abfff640 1 -- 172.21.15.22:0/1302831711 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1b802e4e0 con 0x55828b4fe1a0 2024-03-09T00:00:54.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 10 client.27121 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55828b5739e0 con 0x55828b4fe1a0 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1abfff640 1 -- 172.21.15.22:0/1302831711 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1b8011740 con 0x55828b4fe1a0 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1abfff640 1 -- 172.21.15.22:0/1302831711 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb1b802d030 con 0x55828b4fe1a0 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 10 client.27121 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 10 client.27121 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 20 client.27121 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b573e10 0x55828b576200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 4 client.27121 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55828b469750 con 0x55828b573e10 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b573e10 0x55828b576200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1abfff640 10 client.27121 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.196+0000 7fb1d0c1d640 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b573e10 0x55828b576200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5c7e2e5b4e6c4ac server_cookie=af2e68dbe315a47a in_seq=0 out_seq=0 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1abfff640 1 -- 172.21.15.22:0/1302831711 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55828b469750 con 0x55828b573e10 2024-03-09T00:00:54.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1abfff640 10 client.27121 handle_command_reply: tid=0 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 client.27121 shutdown 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 msgr2=0x55828b511b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b511b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802e8c0 tx=0x7fb1b802e7c0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 shutdown_connections 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55828b573e10 0x55828b576200 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55828b5076a0 0x55828b572200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55828b4feac0 0x55828b512080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 --2- 172.21.15.22:0/1302831711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55828b4fe1a0 0x55828b511b40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1abfff640 0 client.27121 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 >> 172.21.15.22:0/1302831711 conn(0x55828b4f88d0 msgr2=0x55828b504710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 shutdown_connections 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 1 -- 172.21.15.22:0/1302831711 wait complete. 2024-03-09T00:00:54.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.197+0000 7fb1d5a5d740 20 client.27121 trim_cache size 0 max 16384 2024-03-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1bc0a29f0 msgr2=0x7fb1bc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1bc0a29f0 0x7fb1bc0a4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40ccb10 tx=0x7fb1c4005b40 comp rx=0 tx=0).stop 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1979c0 msgr2=0x7fb1cc197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1979c0 0x7fb1cc197e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400eb10 tx=0x7fb1c400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 shutdown_connections 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb1bc0a29f0 0x7fb1bc0a4eb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1cc1979c0 0x7fb1cc197e70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1cc1054b0 0x7fb1cc19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 --2- 172.21.15.22:0/2961965472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc1003e0 0x7fb1cc19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 >> 172.21.15.22:0/2961965472 conn(0x7fb1cc0773a0 msgr2=0x7fb1cc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 shutdown_connections 2024-03-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.198+0000 7fb1d2e81640 1 -- 172.21.15.22:0/2961965472 wait complete. 2024-03-09T00:00:54.205 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:00:54.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:00:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 -- 172.21.15.22:0/2386615683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f50100d90 msgr2=0x7f6f50104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 --2- 172.21.15.22:0/2386615683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f50100d90 0x7f6f50104dd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009900 tx=0x7f6f3802bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 -- 172.21.15.22:0/2386615683 shutdown_connections 2024-03-09T00:00:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 --2- 172.21.15.22:0/2386615683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f501054b0 0x7f6f5010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 --2- 172.21.15.22:0/2386615683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f50100d90 0x7f6f50104dd0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 --2- 172.21.15.22:0/2386615683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f501003e0 0x7f6f501007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 -- 172.21.15.22:0/2386615683 >> 172.21.15.22:0/2386615683 conn(0x7f6f500773a0 msgr2=0x7f6f500777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 -- 172.21.15.22:0/2386615683 shutdown_connections 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 -- 172.21.15.22:0/2386615683 wait complete. 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.317+0000 7f6f55524640 1 Processor -- start 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f55524640 1 -- start start 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f55524640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f501003e0 0x7f6f50101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f55524640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f501054b0 0x7f6f501023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f55524640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f501028f0 0x7f6f501a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f55524640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50110540 con 0x7f6f501054b0 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f55524640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501103c0 con 0x7f6f501028f0 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f55524640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501106c0 con 0x7f6f501003e0 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f501054b0 0x7f6f501023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f501054b0 0x7f6f501023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49758/0 (socket says 172.21.15.22:49758) 2024-03-09T00:00:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/1640934028 learned_addr learned my addr 172.21.15.22:0/1640934028 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4effd640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f501003e0 0x7f6f50101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f501028f0 0x7f6f501a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/1640934028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f501003e0 msgr2=0x7f6f50101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f501003e0 0x7f6f50101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/1640934028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f501028f0 msgr2=0x7f6f501a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f501028f0 0x7f6f501a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.318+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/1640934028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f501a62f0 con 0x7f6f501054b0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.319+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f501054b0 0x7f6f501023b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6f38036040 tx=0x7f6f3802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.319+0000 7f6f3ffff640 1 -- 172.21.15.22:0/1640934028 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38041070 con 0x7f6f501054b0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.319+0000 7f6f3ffff640 1 -- 172.21.15.22:0/1640934028 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f38004450 con 0x7f6f501054b0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.319+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f501a6580 con 0x7f6f501054b0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.319+0000 7f6f3ffff640 1 -- 172.21.15.22:0/1640934028 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38003dc0 con 0x7f6f501054b0 2024-03-09T00:00:54.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.319+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f501a6ac0 con 0x7f6f501054b0 2024-03-09T00:00:54.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.321+0000 7f6f3ffff640 1 -- 172.21.15.22:0/1640934028 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6f38044070 con 0x7f6f501054b0 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 Processor -- start 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 -- start start 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f3ffff640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6f440a29f0 0x7f6f440a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583221221a0 0x558322122580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558322122ac0 0x55832212b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x55832212da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583220e9360 con 0x5583221221a0 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558322085ec0 con 0x55832212b6a0 2024-03-09T00:00:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f581d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832202ed30 con 0x558322122ac0 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6f440a29f0 0x7f6f440a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f4effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583221221a0 0x558322122580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f4effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583221221a0 0x558322122580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49764/0 (socket says 172.21.15.22:49764) 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f4effd640 1 -- 172.21.15.22:0/2337597547 learned_addr learned my addr 172.21.15.22:0/2337597547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558322122ac0 0x55832212b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f3ffff640 1 -- 172.21.15.22:0/1640934028 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6f380f0f70 con 0x7f6f501054b0 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.322+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x55832212da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f4effd640 1 -- 172.21.15.22:0/2337597547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558322122ac0 msgr2=0x55832212b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558322122ac0 0x55832212b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f4effd640 1 -- 172.21.15.22:0/2337597547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 msgr2=0x55832212da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x55832212da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f4effd640 1 -- 172.21.15.22:0/2337597547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55832202dbc0 con 0x5583221221a0 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6f440a29f0 0x7f6f440a4eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005fd0 tx=0x7f6f48013880 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583221221a0 0x558322122580 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000ad00 tx=0x7f6f4000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5de32ea264b2cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f3d7fa640 1 -- 172.21.15.22:0/2337597547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4001d070 con 0x5583221221a0 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f3d7fa640 1 -- 172.21.15.22:0/2337597547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f40018b20 con 0x5583221221a0 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.323+0000 7f6f3d7fa640 1 -- 172.21.15.22:0/2337597547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f40010bc0 con 0x5583221221a0 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 -- 172.21.15.22:0/2337597547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583221221a0 msgr2=0x558322122580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583221221a0 0x558322122580 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000ad00 tx=0x7f6f4000f790 comp rx=0 tx=0).stop 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 -- 172.21.15.22:0/2337597547 shutdown_connections 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x55832212da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558322122ac0 0x55832212b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2337597547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5583221221a0 0x558322122580 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 -- 172.21.15.22:0/2337597547 >> 172.21.15.22:0/2337597547 conn(0x55832211c8d0 msgr2=0x55832211ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 -- 172.21.15.22:0/2337597547 shutdown_connections 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 -- 172.21.15.22:0/2337597547 wait complete. 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 -- start start 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 Processor -- start 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322128bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.324+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x558322129130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f581d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583220244a0 con 0x558322122ac0 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f581d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558322081f60 con 0x55832212b6a0 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f581d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832212e490 con 0x5583221221a0 2024-03-09T00:00:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f581d7740 10 client.0 _ll_register_callbacks cb 0x55832207fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322128bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322128bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49766/0 (socket says 172.21.15.22:49766) 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2998828559 learned_addr learned my addr 172.21.15.22:0/2998828559 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x558322129130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 msgr2=0x5583221286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2998828559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 msgr2=0x558322129130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x558322129130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2998828559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558321d9a3f0 con 0x558322122ac0 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322128bf0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802cec0 tx=0x7f6f3804ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56710d44d67a079 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f367fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f367fc640 1 -- 172.21.15.22:0/2998828559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3804b050 con 0x558322122ac0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f367fc640 1 -- 172.21.15.22:0/2998828559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6f380b9990 con 0x558322122ac0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f40018b20 con 0x558322122ac0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f367fc640 1 -- 172.21.15.22:0/2998828559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f380bda20 con 0x558322122ac0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f581d7740 10 client.17688 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.325+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558321e647d0 con 0x558322122ac0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f367fc640 1 -- 172.21.15.22:0/2998828559 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f40018b20 con 0x558322122ac0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f367fc640 1 -- 172.21.15.22:0/2998828559 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f380ee580 con 0x558322122ac0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f581d7740 10 client.17688 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f581d7740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f581d7740 20 client.17688 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x55832213dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f581d7740 4 client.17688 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558322093400 con 0x55832213b7d0 2024-03-09T00:00:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x55832213dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x55832213dc20 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aadbf2faf0ed6a01 server_cookie=7a422d1cb438a709 in_seq=0 out_seq=0 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.326+0000 7f6f367fc640 10 client.17688 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f367fc640 1 -- 172.21.15.22:0/2998828559 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558322093400 con 0x55832213b7d0 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f367fc640 10 client.17688 handle_command_reply: tid=0 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 client.17688 shutdown 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 msgr2=0x558322128bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322128bf0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802cec0 tx=0x7f6f3804ef90 comp rx=0 tx=0).stop 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 shutdown_connections 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x55832213dc20 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f367fc640 0 client.17688 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x558322129130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322128bf0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2998828559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 >> 172.21.15.22:0/2998828559 conn(0x55832211c8d0 msgr2=0x55832211e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 shutdown_connections 2024-03-09T00:00:54.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 1 -- 172.21.15.22:0/2998828559 wait complete. 2024-03-09T00:00:54.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.328+0000 7f6f581d7740 20 client.17688 trim_cache size 0 max 16384 2024-03-09T00:00:54.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.340+0000 7f6f581d7740 1 Processor -- start 2024-03-09T00:00:54.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.340+0000 7f6f581d7740 1 -- start start 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x55832212ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558322122ac0 0x5583221239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55832212b6a0 0x558322123f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f581d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501b2b40 con 0x55832212b6a0 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f581d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501b06e0 con 0x558322122ac0 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f581d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501ae780 con 0x5583221221a0 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55832212b6a0 0x558322123f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55832212b6a0 0x558322123f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49780/0 (socket says 172.21.15.22:49780) 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 -- 172.21.15.22:0/3993541664 learned_addr learned my addr 172.21.15.22:0/3993541664 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4effd640 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x55832212ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558322122ac0 0x5583221239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 -- 172.21.15.22:0/3993541664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 msgr2=0x55832212ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x55832212ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 -- 172.21.15.22:0/3993541664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558322122ac0 msgr2=0x5583221239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558322122ac0 0x5583221239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 -- 172.21.15.22:0/3993541664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55832211c700 con 0x55832212b6a0 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.341+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55832212b6a0 0x558322123f20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6f48003cc0 tx=0x7f6f4801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c3fb94f2645320 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f3cff9640 1 -- 172.21.15.22:0/3993541664 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48024070 con 0x55832212b6a0 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f3cff9640 1 -- 172.21.15.22:0/3993541664 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f4800f9c0 con 0x55832212b6a0 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f3cff9640 1 -- 172.21.15.22:0/3993541664 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48010800 con 0x55832212b6a0 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 -- 172.21.15.22:0/3993541664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55832212b6a0 msgr2=0x558322123f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55832212b6a0 0x558322123f20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6f48003cc0 tx=0x7f6f4801b000 comp rx=0 tx=0).stop 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 -- 172.21.15.22:0/3993541664 shutdown_connections 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55832212b6a0 0x558322123f20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558322122ac0 0x5583221239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 --2- 172.21.15.22:0/3993541664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x55832212ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 -- 172.21.15.22:0/3993541664 >> 172.21.15.22:0/3993541664 conn(0x55832211c8d0 msgr2=0x558322122ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.342+0000 7f6f581d7740 1 -- 172.21.15.22:0/3993541664 shutdown_connections 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 -- 172.21.15.22:0/3993541664 wait complete. 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 -- start start 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 Processor -- start 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322130cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x5583221311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501aed80 con 0x558322122ac0 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501b0080 con 0x55832212b6a0 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501af3c0 con 0x5583221221a0 2024-03-09T00:00:54.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f581d7740 10 client.0 _ll_register_callbacks cb 0x7f6f4000a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322130cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322130cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49786/0 (socket says 172.21.15.22:49786) 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2622884154 learned_addr learned my addr 172.21.15.22:0/2622884154 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4f7fe640 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x5583221311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2622884154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 msgr2=0x5583221327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2622884154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 msgr2=0x5583221311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.343+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x5583221311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f4e7fc640 1 -- 172.21.15.22:0/2622884154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583221317b0 con 0x558322122ac0 2024-03-09T00:00:54.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f4e7fc640 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322130cb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6f38005820 tx=0x7f6f380bcd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6103f0c00f578342 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f367fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f367fc640 1 -- 172.21.15.22:0/2622884154 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3804b070 con 0x558322122ac0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f367fc640 1 -- 172.21.15.22:0/2622884154 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f6f380b9990 con 0x558322122ac0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f4800f9c0 con 0x558322122ac0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f367fc640 1 -- 172.21.15.22:0/2622884154 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f380bd0f0 con 0x558322122ac0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 10 client.17694 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558322192b00 con 0x558322122ac0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f367fc640 1 -- 172.21.15.22:0/2622884154 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f4800f9c0 con 0x558322122ac0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f367fc640 1 -- 172.21.15.22:0/2622884154 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f380c5630 con 0x558322122ac0 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 10 client.17694 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 20 client.17694 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x558322195180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:54.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 4 client.17694 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.344+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55832208d750 con 0x55832213b7d0 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.345+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x558322195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.345+0000 7f6f367fc640 10 client.17694 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.345+0000 7f6f4effd640 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x558322195180 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd500be42f3d3386 server_cookie=464efb3e3d66a8a3 in_seq=0 out_seq=0 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f367fc640 1 -- 172.21.15.22:0/2622884154 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55832208d750 con 0x55832213b7d0 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f367fc640 10 client.17694 handle_command_reply: tid=0 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 client.17694 shutdown 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 msgr2=0x558322130cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322130cb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6f38005820 tx=0x7f6f380bcd40 comp rx=0 tx=0).stop 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 shutdown_connections 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55832213b7d0 0x558322195180 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55832212b6a0 0x5583221311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558322122ac0 0x558322130cb0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 --2- 172.21.15.22:0/2622884154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583221221a0 0x5583221327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 >> 172.21.15.22:0/2622884154 conn(0x55832211c8d0 msgr2=0x5583221239e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f367fc640 0 client.17694 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:54.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 shutdown_connections 2024-03-09T00:00:54.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 1 -- 172.21.15.22:0/2622884154 wait complete. 2024-03-09T00:00:54.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f581d7740 20 client.17694 trim_cache size 0 max 16384 2024-03-09T00:00:54.349 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:00:54.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6f440a29f0 msgr2=0x7f6f440a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f55524640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6f440a29f0 0x7f6f440a4eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005fd0 tx=0x7f6f48013880 comp rx=0 tx=0).stop 2024-03-09T00:00:54.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f501054b0 msgr2=0x7f6f501023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:54.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f55524640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f501054b0 0x7f6f501023b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6f38036040 tx=0x7f6f3802cbf0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.346+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 shutdown_connections 2024-03-09T00:00:54.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.347+0000 7f6f55524640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6f440a29f0 0x7f6f440a4eb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.347+0000 7f6f55524640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f501028f0 0x7f6f501a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.347+0000 7f6f55524640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f501054b0 0x7f6f501023b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.347+0000 7f6f55524640 1 --2- 172.21.15.22:0/1640934028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f501003e0 0x7f6f50101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:54.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.347+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 >> 172.21.15.22:0/1640934028 conn(0x7f6f500773a0 msgr2=0x7f6f50068630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:54.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.347+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 shutdown_connections 2024-03-09T00:00:54.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:54.347+0000 7f6f55524640 1 -- 172.21.15.22:0/1640934028 wait complete. 2024-03-09T00:00:54.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:54 smithi022 ceph-mon[31894]: pgmap v197: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-03-09T00:00:54.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:54 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3950752202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:00:54.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:54 smithi062 ceph-mon[36129]: pgmap v197: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-03-09T00:00:54.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:54 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3950752202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:00:54.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:54 smithi192 ceph-mon[35995]: pgmap v197: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-03-09T00:00:54.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:54 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3950752202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:00:54.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 665228 11.17 MB/sec execute 182 sec latency 22.042 ms 2024-03-09T00:00:55.353 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:00:55.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:00:55.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:00:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.518+0000 7f7802762640 1 -- 172.21.15.22:0/2179806780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4099320 msgr2=0x7f77f4099700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.518+0000 7f7802762640 1 --2- 172.21.15.22:0/2179806780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4099320 0x7f77f4099700 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f77fc066fc0 tx=0x7f77fc096fc0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.518+0000 7f7802762640 1 -- 172.21.15.22:0/2179806780 shutdown_connections 2024-03-09T00:00:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.518+0000 7f7802762640 1 --2- 172.21.15.22:0/2179806780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f4094f50 0x7f77f40a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.518+0000 7f7802762640 1 --2- 172.21.15.22:0/2179806780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f40945b0 0x7f77f4094a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.518+0000 7f7802762640 1 --2- 172.21.15.22:0/2179806780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4099320 0x7f77f4099700 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.518+0000 7f7802762640 1 -- 172.21.15.22:0/2179806780 >> 172.21.15.22:0/2179806780 conn(0x7f77f4009ea0 msgr2=0x7f77f400a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.520+0000 7f7802762640 1 -- 172.21.15.22:0/2179806780 shutdown_connections 2024-03-09T00:00:55.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.520+0000 7f7802762640 1 -- 172.21.15.22:0/2179806780 wait complete. 2024-03-09T00:00:55.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 Processor -- start 2024-03-09T00:00:55.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 -- start start 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f40945b0 0x7f77f413b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f4094f50 0x7f77f413b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f4099320 0x7f77f413fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40a3e80 con 0x7f77f40945b0 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40a3d00 con 0x7f77f4094f50 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.521+0000 7f7802762640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40a4000 con 0x7f77f4099320 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f4099320 0x7f77f413fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f4099320 0x7f77f413fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35664/0 (socket says 172.21.15.22:35664) 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 -- 172.21.15.22:0/2127593824 learned_addr learned my addr 172.21.15.22:0/2127593824 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801760640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f40945b0 0x7f77f413b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 -- 172.21.15.22:0/2127593824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f4094f50 msgr2=0x7f77f413b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f4094f50 0x7f77f413b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 -- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f40945b0 msgr2=0x7f77f413b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f40945b0 0x7f77f413b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.522+0000 7f7801f61640 1 -- 172.21.15.22:0/2127593824 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4140270 con 0x7f77f4099320 2024-03-09T00:00:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.523+0000 7f7801f61640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f4099320 0x7f77f413fb40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e9c0 tx=0x7f77f800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.523+0000 7f77f27fc640 1 -- 172.21.15.22:0/2127593824 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f800cce0 con 0x7f77f4099320 2024-03-09T00:00:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.523+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f4140560 con 0x7f77f4099320 2024-03-09T00:00:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.524+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f4143dd0 con 0x7f77f4099320 2024-03-09T00:00:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.525+0000 7f77f27fc640 1 -- 172.21.15.22:0/2127593824 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77f8005500 con 0x7f77f4099320 2024-03-09T00:00:55.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.525+0000 7f77f27fc640 1 -- 172.21.15.22:0/2127593824 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8016670 con 0x7f77f4099320 2024-03-09T00:00:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.527+0000 7f78065a2740 1 Processor -- start 2024-03-09T00:00:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f77f27fc640 1 -- 172.21.15.22:0/2127593824 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f77f801d020 con 0x7f77f4099320 2024-03-09T00:00:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f78065a2740 1 -- start start 2024-03-09T00:00:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f78065a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbcb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f78065a2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbcb8a0 0x56403dbd3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f78065a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd4480 0x56403dbd6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f78065a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403db2eb40 con 0x56403dbd4480 2024-03-09T00:00:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f78065a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403dad7d80 con 0x56403dbcaf80 2024-03-09T00:00:55.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.529+0000 7f78065a2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403db2abe0 con 0x56403dbcb8a0 2024-03-09T00:00:55.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbcb8a0 0x56403dbd3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbcb8a0 0x56403dbd3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35678/0 (socket says 172.21.15.22:35678) 2024-03-09T00:00:55.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 -- 172.21.15.22:0/2159641596 learned_addr learned my addr 172.21.15.22:0/2159641596 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7800f5f640 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd4480 0x56403dbd6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7800f5f640 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbcb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f77f27fc640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77e80a2a70 0x7f77e80a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 -- 172.21.15.22:0/2159641596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 msgr2=0x56403dbcb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbcb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 -- 172.21.15.22:0/2159641596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd4480 msgr2=0x56403dbd6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd4480 0x56403dbd6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801760640 1 -- 172.21.15.22:0/2159641596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403d90b7e0 con 0x56403dbcb8a0 2024-03-09T00:00:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.530+0000 7f7801f61640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77e80a2a70 0x7f77e80a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.531+0000 7f77f27fc640 1 -- 172.21.15.22:0/2127593824 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f77f80cedd0 con 0x7f77f4099320 2024-03-09T00:00:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.531+0000 7f7801760640 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbcb8a0 0x56403dbd3f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77fc02d430 tx=0x7f77fc067da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a31aacd04c27189e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.531+0000 7f7801f61640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77e80a2a70 0x7f77e80a4f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c7a0 tx=0x7f77f80c5a80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.532+0000 7f77e77fe640 1 -- 172.21.15.22:0/2159641596 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0b6070 con 0x56403dbcb8a0 2024-03-09T00:00:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.532+0000 7f77e77fe640 1 -- 172.21.15.22:0/2159641596 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77fc09b040 con 0x56403dbcb8a0 2024-03-09T00:00:55.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.532+0000 7f77e77fe640 1 -- 172.21.15.22:0/2159641596 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc069320 con 0x56403dbcb8a0 2024-03-09T00:00:55.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.533+0000 7f78065a2740 1 -- 172.21.15.22:0/2159641596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbcb8a0 msgr2=0x56403dbd3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.533+0000 7f78065a2740 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbcb8a0 0x56403dbd3f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77fc02d430 tx=0x7f77fc067da0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.533+0000 7f78065a2740 1 -- 172.21.15.22:0/2159641596 shutdown_connections 2024-03-09T00:00:55.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.533+0000 7f78065a2740 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd4480 0x56403dbd6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.533+0000 7f78065a2740 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbcb8a0 0x56403dbd3f40 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77fc02d430 tx=0x7f77fc067da0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.533+0000 7f78065a2740 1 --2- 172.21.15.22:0/2159641596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbcb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.533+0000 7f78065a2740 1 -- 172.21.15.22:0/2159641596 >> 172.21.15.22:0/2159641596 conn(0x56403dbc56b0 msgr2=0x56403dbc5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.534+0000 7f78065a2740 1 -- 172.21.15.22:0/2159641596 shutdown_connections 2024-03-09T00:00:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.535+0000 7f78065a2740 1 -- 172.21.15.22:0/2159641596 wait complete. 2024-03-09T00:00:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.535+0000 7f78065a2740 1 -- start start 2024-03-09T00:00:55.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.535+0000 7f78065a2740 1 Processor -- start 2024-03-09T00:00:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.536+0000 7f78065a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbcaf80 0x56403dbd12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.536+0000 7f78065a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbd4480 0x56403dbd1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f78065a2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd1d60 0x56403dbe06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f7801760640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbd4480 0x56403dbd1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f7801760640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbd4480 0x56403dbd1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36682/0 (socket says 172.21.15.22:36682) 2024-03-09T00:00:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f7800f5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbcaf80 0x56403dbd12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f78065a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403dbd7210 con 0x56403dbcaf80 2024-03-09T00:00:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f7800f5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbcaf80 0x56403dbd12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49816/0 (socket says 172.21.15.22:49816) 2024-03-09T00:00:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f78065a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403daeb1c0 con 0x56403dbd4480 2024-03-09T00:00:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f7800f5f640 1 -- 172.21.15.22:0/3564215635 learned_addr learned my addr 172.21.15.22:0/3564215635 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403dbd7390 con 0x56403dbd1d60 2024-03-09T00:00:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.537+0000 7f78065a2740 10 client.0 _ll_register_callbacks cb 0x56403dada7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f7800f5f640 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd1d60 0x56403dbe06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f7801760640 1 -- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd1d60 msgr2=0x56403dbe06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f7801760640 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd1d60 0x56403dbe06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f7801760640 1 -- 172.21.15.22:0/3564215635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbcaf80 msgr2=0x56403dbd12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f7801760640 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbcaf80 0x56403dbd12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f7801760640 1 -- 172.21.15.22:0/3564215635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56403d96bf70 con 0x56403dbd4480 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f7801760640 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbd4480 0x56403dbd1820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x56403dbcbdc0 tx=0x7f77fc076510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84b21f7281970241 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.538+0000 7f77e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:00:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.539+0000 7f77e4ff9640 1 -- 172.21.15.22:0/3564215635 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0be020 con 0x56403dbd4480 2024-03-09T00:00:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.540+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77fc09b040 con 0x56403dbd4480 2024-03-09T00:00:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.540+0000 7f77e4ff9640 1 -- 172.21.15.22:0/3564215635 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f77fc097960 con 0x56403dbd4480 2024-03-09T00:00:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.540+0000 7f77e4ff9640 1 -- 172.21.15.22:0/3564215635 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77fc0b6040 con 0x56403dbd4480 2024-03-09T00:00:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.540+0000 7f78065a2740 10 client.27101 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.540+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56403d803170 con 0x56403dbd4480 2024-03-09T00:00:55.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.541+0000 7f77e4ff9640 1 -- 172.21.15.22:0/3564215635 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77fc04f8a0 con 0x56403dbd4480 2024-03-09T00:00:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.542+0000 7f77e4ff9640 1 -- 172.21.15.22:0/3564215635 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f77fc0b9070 con 0x56403dbd4480 2024-03-09T00:00:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.542+0000 7f78065a2740 10 client.27101 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.542+0000 7f78065a2740 10 client.27101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.542+0000 7f78065a2740 20 client.27101 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.542+0000 7f78065a2740 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dbe4a50 0x56403dbe6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.542+0000 7f78065a2740 4 client.27101 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:55.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.542+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77fc09b040 con 0x56403dbe4a50 2024-03-09T00:00:55.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.546+0000 7f7800f5f640 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dbe4a50 0x56403dbe6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.547+0000 7f7800f5f640 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dbe4a50 0x56403dbe6e60 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb4ae854b8b92df server_cookie=b9a219871bee8346 in_seq=0 out_seq=0 2024-03-09T00:00:55.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.547+0000 7f77e4ff9640 10 client.27101 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.549+0000 7f77e4ff9640 1 -- 172.21.15.22:0/3564215635 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f77fc09b040 con 0x56403dbe4a50 2024-03-09T00:00:55.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f77e4ff9640 10 client.27101 handle_command_reply: tid=0 2024-03-09T00:00:55.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 client.27101 shutdown 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbd4480 msgr2=0x56403dbd1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbd4480 0x56403dbd1820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x56403dbcbdc0 tx=0x7f77fc076510 comp rx=0 tx=0).stop 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 shutdown_connections 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dbe4a50 0x56403dbe6e60 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd1d60 0x56403dbe06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbd4480 0x56403dbd1820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 --2- 172.21.15.22:0/3564215635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbcaf80 0x56403dbd12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 >> 172.21.15.22:0/3564215635 conn(0x56403dbc56b0 msgr2=0x56403dbc7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f77e4ff9640 0 client.27101 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 shutdown_connections 2024-03-09T00:00:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 1 -- 172.21.15.22:0/3564215635 wait complete. 2024-03-09T00:00:55.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.550+0000 7f78065a2740 20 client.27101 trim_cache size 0 max 16384 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.568+0000 7f78065a2740 1 Processor -- start 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.568+0000 7f78065a2740 1 -- start start 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f78065a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f78065a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f78065a2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbc3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49828/0 (socket says 172.21.15.22:49828) 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 -- 172.21.15.22:0/455067613 learned_addr learned my addr 172.21.15.22:0/455067613 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e0012280 con 0x56403dbd1d60 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000fe20 con 0x56403dbcaf80 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000dec0 con 0x56403dbd4480 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7800f5f640 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7800f5f640 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbc3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 -- 172.21.15.22:0/455067613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 msgr2=0x56403dbc3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbc3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 -- 172.21.15.22:0/455067613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 msgr2=0x56403dbce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 -- 172.21.15.22:0/455067613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403dbd7390 con 0x56403dbd1d60 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f7801760640 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbceae0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0647e0 tx=0x7f77fc066b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc52235677faf4a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f77e6ffd640 1 -- 172.21.15.22:0/455067613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0be040 con 0x56403dbd1d60 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.569+0000 7f77e6ffd640 1 -- 172.21.15.22:0/455067613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77fc09b070 con 0x56403dbd1d60 2024-03-09T00:00:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f77e6ffd640 1 -- 172.21.15.22:0/455067613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc097ba0 con 0x56403dbd1d60 2024-03-09T00:00:55.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 msgr2=0x56403dbceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbceae0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0647e0 tx=0x7f77fc066b70 comp rx=0 tx=0).stop 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 shutdown_connections 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbc3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbceae0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 --2- 172.21.15.22:0/455067613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 >> 172.21.15.22:0/455067613 conn(0x56403dbc56b0 msgr2=0x56403dbe00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 shutdown_connections 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.570+0000 7f78065a2740 1 -- 172.21.15.22:0/455067613 wait complete. 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 -- start start 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 Processor -- start 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbdb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbd9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000e4c0 con 0x56403dbd1d60 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000f7c0 con 0x56403dbcaf80 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000eb00 con 0x56403dbd4480 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f7801760640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbd9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f78065a2740 10 client.0 _ll_register_callbacks cb 0x7f77fc069790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f7800f5f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbdb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f7801760640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbd9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49834/0 (socket says 172.21.15.22:49834) 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f7801760640 1 -- 172.21.15.22:0/1257962360 learned_addr learned my addr 172.21.15.22:0/1257962360 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.571+0000 7f7800f5f640 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f7801760640 1 -- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 msgr2=0x56403dbda0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f7801760640 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbda0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f7801760640 1 -- 172.21.15.22:0/1257962360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 msgr2=0x56403dbdb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f7801760640 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbdb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f7801760640 1 -- 172.21.15.22:0/1257962360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56403dbda720 con 0x56403dbd1d60 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f7800f5f640 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbda0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:00:55.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f7801760640 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbd9b70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f77fc064310 tx=0x7f77fc067da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c98bd7fad200875 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f77e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f77e4ff9640 1 -- 172.21.15.22:0/1257962360 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0be070 con 0x56403dbd1d60 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f77e4ff9640 1 -- 172.21.15.22:0/1257962360 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f77fc0694d0 con 0x56403dbd1d60 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77fc09b070 con 0x56403dbd1d60 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f77e4ff9640 1 -- 172.21.15.22:0/1257962360 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77fc04f8a0 con 0x56403dbd1d60 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f78065a2740 10 client.17712 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56403dc3be60 con 0x56403dbd1d60 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f77e4ff9640 1 -- 172.21.15.22:0/1257962360 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77fc097e40 con 0x56403dbd1d60 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f77e4ff9640 1 -- 172.21.15.22:0/1257962360 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f77fc0b9040 con 0x56403dbd1d60 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f78065a2740 10 client.17712 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f78065a2740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.572+0000 7f78065a2740 20 client.17712 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:55.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.573+0000 7f78065a2740 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dc3c2e0 0x56403dc3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.573+0000 7f78065a2740 4 client.17712 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.573+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f77fc09b070 con 0x56403dc3c2e0 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.573+0000 7f7800f5f640 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dc3c2e0 0x56403dc3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.573+0000 7f77e4ff9640 10 client.17712 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.573+0000 7f7800f5f640 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dc3c2e0 0x56403dc3e700 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b73ccb591677c8 server_cookie=e28dd4cf10e92b5c in_seq=0 out_seq=0 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f77e4ff9640 1 -- 172.21.15.22:0/1257962360 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f77fc09b070 con 0x56403dc3c2e0 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f77e4ff9640 10 client.17712 handle_command_reply: tid=0 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 client.17712 shutdown 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 msgr2=0x56403dbd9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbd9b70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f77fc064310 tx=0x7f77fc067da0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 shutdown_connections 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56403dc3c2e0 0x56403dc3e700 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403dbd4480 0x56403dbda0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56403dbd1d60 0x56403dbd9b70 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 --2- 172.21.15.22:0/1257962360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56403dbcaf80 0x56403dbdb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f77e4ff9640 0 client.17712 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.574+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 >> 172.21.15.22:0/1257962360 conn(0x56403dbc56b0 msgr2=0x56403dbd0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 shutdown_connections 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f78065a2740 1 -- 172.21.15.22:0/1257962360 wait complete. 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f78065a2740 20 client.17712 trim_cache size 0 max 16384 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "8ca5c136-c7f9-407f-b46d-f6edc5eb2369", 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:00:55.576 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:00:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77e80a2a70 msgr2=0x7f77e80a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77e80a2a70 0x7f77e80a4f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c7a0 tx=0x7f77f80c5a80 comp rx=0 tx=0).stop 2024-03-09T00:00:55.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f4099320 msgr2=0x7f77f413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f4099320 0x7f77f413fb40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e9c0 tx=0x7f77f800ee80 comp rx=0 tx=0).stop 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 shutdown_connections 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77e80a2a70 0x7f77e80a4f30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77f4099320 0x7f77f413fb40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f4094f50 0x7f77f413b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 --2- 172.21.15.22:0/2127593824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f40945b0 0x7f77f413b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 >> 172.21.15.22:0/2127593824 conn(0x7f77f4009ea0 msgr2=0x7f77f40926a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 shutdown_connections 2024-03-09T00:00:55.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.575+0000 7f7802762640 1 -- 172.21.15.22:0/2127593824 wait complete. 2024-03-09T00:00:55.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:00:55.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 -- 172.21.15.22:0/2172649540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1003d0 msgr2=0x7f8f4c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2172649540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1003d0 0x7f8f4c1007b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009900 tx=0x7f8f3c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 -- 172.21.15.22:0/2172649540 shutdown_connections 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2172649540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4c1054a0 0x7f8f4c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2172649540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c100d80 0x7f8f4c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2172649540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1003d0 0x7f8f4c1007b0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 -- 172.21.15.22:0/2172649540 >> 172.21.15.22:0/2172649540 conn(0x7f8f4c0772f0 msgr2=0x7f8f4c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 -- 172.21.15.22:0/2172649540 shutdown_connections 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.696+0000 7f8f531a9640 1 -- 172.21.15.22:0/2172649540 wait complete. 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 Processor -- start 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 -- start start 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c1003d0 0x7f8f4c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1054a0 0x7f8f4c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4c197990 0x7f8f4c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c110530 con 0x7f8f4c1054a0 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1103b0 con 0x7f8f4c197990 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f531a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1106b0 con 0x7f8f4c1003d0 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1054a0 0x7f8f4c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1054a0 0x7f8f4c19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49842/0 (socket says 172.21.15.22:49842) 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 -- 172.21.15.22:0/2956939359 learned_addr learned my addr 172.21.15.22:0/2956939359 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f50f45640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c1003d0 0x7f8f4c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f51746640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4c197990 0x7f8f4c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 -- 172.21.15.22:0/2956939359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c1003d0 msgr2=0x7f8f4c19d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c1003d0 0x7f8f4c19d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 -- 172.21.15.22:0/2956939359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4c197990 msgr2=0x7f8f4c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4c197990 0x7f8f4c197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.697+0000 7f8f4bfff640 1 -- 172.21.15.22:0/2956939359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c1984b0 con 0x7f8f4c1054a0 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.698+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1054a0 0x7f8f4c19dd20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400e9c0 tx=0x7f8f3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.698+0000 7f8f49ffb640 1 -- 172.21.15.22:0/2956939359 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400cd80 con 0x7f8f4c1054a0 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.698+0000 7f8f49ffb640 1 -- 172.21.15.22:0/2956939359 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f34005500 con 0x7f8f4c1054a0 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.698+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c1987a0 con 0x7f8f4c1054a0 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.698+0000 7f8f49ffb640 1 -- 172.21.15.22:0/2956939359 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34010690 con 0x7f8f4c1054a0 2024-03-09T00:00:55.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.698+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c1a49f0 con 0x7f8f4c1054a0 2024-03-09T00:00:55.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.700+0000 7f8f49ffb640 1 -- 172.21.15.22:0/2956939359 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8f340046c0 con 0x7f8f4c1054a0 2024-03-09T00:00:55.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.700+0000 7f8f49ffb640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f380a2970 0x7f8f380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f49ffb640 1 -- 172.21.15.22:0/2956939359 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8f340cf620 con 0x7f8f4c1054a0 2024-03-09T00:00:55.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f50f45640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f380a2970 0x7f8f380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 Processor -- start 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 -- start start 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368526ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56368526b080 0x563685273720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563685273c60 0x563685276050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636851cde00 con 0x56368526b080 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563685176b10 con 0x563685273c60 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f55d83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636851c9ea0 con 0x56368526a760 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f50f45640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f380a2970 0x7f8f380a4e30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009230 tx=0x7f8f3c005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56368526b080 0x563685273720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f50f45640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368526ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56368526b080 0x563685273720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49858/0 (socket says 172.21.15.22:49858) 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f4bfff640 1 -- 172.21.15.22:0/1578029053 learned_addr learned my addr 172.21.15.22:0/1578029053 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f51746640 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563685273c60 0x563685276050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f4bfff640 1 -- 172.21.15.22:0/1578029053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 msgr2=0x56368526ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368526ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f4bfff640 1 -- 172.21.15.22:0/1578029053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563685273c60 msgr2=0x563685276050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.701+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563685273c60 0x563685276050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.702+0000 7f8f4bfff640 1 -- 172.21.15.22:0/1578029053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563684fac7d0 con 0x56368526b080 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.702+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56368526b080 0x563685273720 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8f340982b0 tx=0x7f8f34097cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9079bf138ffa95e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.702+0000 7f8f42ffd640 1 -- 172.21.15.22:0/1578029053 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f340b4070 con 0x56368526b080 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.702+0000 7f8f42ffd640 1 -- 172.21.15.22:0/1578029053 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f3409fa30 con 0x56368526b080 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.702+0000 7f8f42ffd640 1 -- 172.21.15.22:0/1578029053 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f340cf260 con 0x56368526b080 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- 172.21.15.22:0/1578029053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56368526b080 msgr2=0x563685273720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56368526b080 0x563685273720 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8f340982b0 tx=0x7f8f34097cb0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- 172.21.15.22:0/1578029053 shutdown_connections 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563685273c60 0x563685276050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56368526b080 0x563685273720 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 --2- 172.21.15.22:0/1578029053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368526ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- 172.21.15.22:0/1578029053 >> 172.21.15.22:0/1578029053 conn(0x563685264e90 msgr2=0x563685265270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- 172.21.15.22:0/1578029053 shutdown_connections 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- 172.21.15.22:0/1578029053 wait complete. 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- start start 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 Processor -- start 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x5636852733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685270780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x563685270cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636852769f0 con 0x563685273c60 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368518a0a0 con 0x56368526b080 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563685276b70 con 0x56368526a760 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.703+0000 7f8f55d83740 10 client.0 _ll_register_callbacks cb 0x563685189a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x563685270cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685270780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f50f45640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x5636852733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f50f45640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x5636852733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35768/0 (socket says 172.21.15.22:35768) 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x563685270cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49874/0 (socket says 172.21.15.22:49874) 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f50f45640 1 -- 172.21.15.22:0/3727926198 learned_addr learned my addr 172.21.15.22:0/3727926198 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 -- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 msgr2=0x5636852733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x5636852733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 -- 172.21.15.22:0/3727926198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 msgr2=0x563685270780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685270780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 -- 172.21.15.22:0/3727926198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563684ee23f0 con 0x563685273c60 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x5636852733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685270780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f51746640 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x563685270cc0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005fd0 tx=0x7f8f44012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c65625a96a5eea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3727926198 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44010e00 con 0x563685273c60 2024-03-09T00:00:55.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3727926198 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8f4401a3e0 con 0x563685273c60 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f3409fa30 con 0x563685273c60 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3727926198 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f4401a5e0 con 0x563685273c60 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f55d83740 10 client.17724 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.704+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563684ea20f0 con 0x563685273c60 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3727926198 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f3409fa30 con 0x563685273c60 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3727926198 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8f44019070 con 0x563685273c60 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f55d83740 10 client.17724 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f55d83740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f55d83740 20 client.17724 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x563685285cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f55d83740 4 client.17724 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636851db340 con 0x563685283890 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x563685285cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f2bfff640 10 client.17724 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.705+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x563685285cb0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0f103c6fad7ec0 server_cookie=5689d3908b1b78a2 in_seq=0 out_seq=0 2024-03-09T00:00:55.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3727926198 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636851db340 con 0x563685283890 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f2bfff640 10 client.17724 handle_command_reply: tid=0 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 client.17724 shutdown 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 msgr2=0x563685270cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x563685270cc0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005fd0 tx=0x7f8f44012b60 comp rx=0 tx=0).stop 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 shutdown_connections 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x563685285cb0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f2bfff640 0 client.17724 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x563685270cc0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685270780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3727926198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x5636852733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.706+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 >> 172.21.15.22:0/3727926198 conn(0x563685264e90 msgr2=0x563685266be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.707+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 shutdown_connections 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.707+0000 7f8f55d83740 1 -- 172.21.15.22:0/3727926198 wait complete. 2024-03-09T00:00:55.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.707+0000 7f8f55d83740 20 client.17724 trim_cache size 0 max 16384 2024-03-09T00:00:55.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 667071 11.16 MB/sec execute 183 sec latency 130.067 ms 2024-03-09T00:00:55.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 Processor -- start 2024-03-09T00:00:55.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 -- start start 2024-03-09T00:00:55.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526a760 0x563685272eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526b080 0x5636852733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x56368526f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1b0ba0 con 0x563685273c60 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1ae740 con 0x56368526a760 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f55d83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1ac7e0 con 0x56368526b080 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f51746640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x56368526f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526b080 0x5636852733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f50f45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526a760 0x563685272eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f51746640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x56368526f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49884/0 (socket says 172.21.15.22:49884) 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f51746640 1 -- 172.21.15.22:0/3270131112 learned_addr learned my addr 172.21.15.22:0/3270131112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f50f45640 1 -- 172.21.15.22:0/3270131112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526b080 msgr2=0x5636852733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526b080 0x5636852733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f50f45640 1 -- 172.21.15.22:0/3270131112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 msgr2=0x56368526f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x56368526f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f50f45640 1 -- 172.21.15.22:0/3270131112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56368528ae80 con 0x56368526a760 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f51746640 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x56368526f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f4bfff640 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526b080 0x5636852733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.724+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526a760 0x563685272eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c02c820 tx=0x7f8f3c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a77c0d42e4c40cb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f427fc640 1 -- 172.21.15.22:0/3270131112 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c05d070 con 0x56368526a760 2024-03-09T00:00:55.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f427fc640 1 -- 172.21.15.22:0/3270131112 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c004980 con 0x56368526a760 2024-03-09T00:00:55.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f427fc640 1 -- 172.21.15.22:0/3270131112 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c003970 con 0x56368526a760 2024-03-09T00:00:55.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f55d83740 1 -- 172.21.15.22:0/3270131112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526a760 msgr2=0x563685272eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526a760 0x563685272eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c02c820 tx=0x7f8f3c03f000 comp rx=0 tx=0).stop 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f55d83740 1 -- 172.21.15.22:0/3270131112 shutdown_connections 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x56368526f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526b080 0x5636852733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3270131112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526a760 0x563685272eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.725+0000 7f8f55d83740 1 -- 172.21.15.22:0/3270131112 >> 172.21.15.22:0/3270131112 conn(0x563685264e90 msgr2=0x563685270740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 -- 172.21.15.22:0/3270131112 shutdown_connections 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 -- 172.21.15.22:0/3270131112 wait complete. 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 -- start start 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 Processor -- start 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368527ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685277380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x5636852778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1acde0 con 0x563685273c60 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1ae0e0 con 0x56368526b080 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1ad420 con 0x56368526a760 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.726+0000 7f8f55d83740 10 client.0 _ll_register_callbacks cb 0x7f8f44009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x5636852778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685277380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x5636852778c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49900/0 (socket says 172.21.15.22:49900) 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f50f45640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368527ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 -- 172.21.15.22:0/3179008095 learned_addr learned my addr 172.21.15.22:0/3179008095 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685277380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36766/0 (socket says 172.21.15.22:36766) 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 -- 172.21.15.22:0/3179008095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 msgr2=0x56368527ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368527ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 -- 172.21.15.22:0/3179008095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 msgr2=0x563685277380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685277380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 -- 172.21.15.22:0/3179008095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563685277e00 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f51746640 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x5636852778c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8f4401ba50 tx=0x7f8f44010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab7c07b809d95ba server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3179008095 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44009c80 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f3c004980 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3179008095 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f8f4401b650 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3179008095 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f44037050 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f55d83740 10 client.17730 fetch_fsmap learned FSMap version 33 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.727+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5636852daad0 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3179008095 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c004980 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3179008095 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8f44019040 con 0x563685273c60 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f55d83740 10 client.17730 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f55d83740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f55d83740 20 client.17730 populate_metadata read hostname 'smithi022' 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x5636852dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:00:55.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f55d83740 4 client.17730 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:00:55.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5636851d5690 con 0x563685283890 2024-03-09T00:00:55.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x5636852dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:00:55.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f2bfff640 10 client.17730 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.728+0000 7f8f50f45640 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x5636852dd230 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba07f14fd6dcf3e server_cookie=2417d1c41f3a96fa in_seq=0 out_seq=0 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f2bfff640 1 -- 172.21.15.22:0/3179008095 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5636851d5690 con 0x563685283890 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f2bfff640 10 client.17730 handle_command_reply: tid=0 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 client.17730 shutdown 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 msgr2=0x5636852778c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x5636852778c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8f4401ba50 tx=0x7f8f44010e30 comp rx=0 tx=0).stop 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 shutdown_connections 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563685283890 0x5636852dd230 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563685273c60 0x5636852778c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56368526b080 0x563685277380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 --2- 172.21.15.22:0/3179008095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56368526a760 0x56368527ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f2bfff640 0 client.17730 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 >> 172.21.15.22:0/3179008095 conn(0x563685264e90 msgr2=0x56368526eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 shutdown_connections 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 1 -- 172.21.15.22:0/3179008095 wait complete. 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.730+0000 7f8f55d83740 20 client.17730 trim_cache size 0 max 16384 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:00:55.730 INFO:teuthology.orchestra.run.smithi022.stdout: "8ca5c136-c7f9-407f-b46d-f6edc5eb2369": { 2024-03-09T00:00:55.731 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:00:55.731 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "8ca5c136-c7f9-407f-b46d-f6edc5eb2369", 2024-03-09T00:00:55.731 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:00:55.731 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:00:55.731 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:00:55.731 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f380a2970 msgr2=0x7f8f380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f380a2970 0x7f8f380a4e30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009230 tx=0x7f8f3c005fd0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1054a0 msgr2=0x7f8f4c19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1054a0 0x7f8f4c19dd20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400e9c0 tx=0x7f8f3400ee80 comp rx=0 tx=0).stop 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 shutdown_connections 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f380a2970 0x7f8f380a4e30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4c197990 0x7f8f4c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4c1054a0 0x7f8f4c19dd20 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 --2- 172.21.15.22:0/2956939359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c1003d0 0x7f8f4c19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 >> 172.21.15.22:0/2956939359 conn(0x7f8f4c0772f0 msgr2=0x7f8f4c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:00:55.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 shutdown_connections 2024-03-09T00:00:55.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:00:55.731+0000 7f8f531a9640 1 -- 172.21.15.22:0/2956939359 wait complete. 2024-03-09T00:00:55.737 INFO:tasks.cephfs.filesystem:scrub status for tag:8ca5c136-c7f9-407f-b46d-f6edc5eb2369 - {'path': '/', 'tag': '8ca5c136-c7f9-407f-b46d-f6edc5eb2369', 'options': 'recursive,force'} 2024-03-09T00:00:56.432 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:56 smithi192 ceph-mon[35995]: pgmap v198: 129 pgs: 129 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-03-09T00:00:56.432 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:56 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:00:56.432 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:56 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:56.432 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:56 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:00:56.432 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:56 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:56 smithi022 ceph-mon[31894]: pgmap v198: 129 pgs: 129 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-03-09T00:00:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:56 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:00:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:56 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:56.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:56 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:00:56.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:56 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:56 smithi062 ceph-mon[36129]: pgmap v198: 129 pgs: 129 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-03-09T00:00:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:56 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:00:56.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:56 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:56.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:56 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:00:56.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:56 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:00:56.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 669048 11.17 MB/sec execute 184 sec latency 16.463 ms 2024-03-09T00:00:57.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 671527 11.18 MB/sec execute 185 sec latency 14.555 ms 2024-03-09T00:00:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:00:58 smithi022 ceph-mon[31894]: pgmap v199: 129 pgs: 129 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 353 op/s 2024-03-09T00:00:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:00:58 smithi062 ceph-mon[36129]: pgmap v199: 129 pgs: 129 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 353 op/s 2024-03-09T00:00:58.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:00:58 smithi192 ceph-mon[35995]: pgmap v199: 129 pgs: 129 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 353 op/s 2024-03-09T00:00:58.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 673620 11.16 MB/sec execute 186 sec latency 24.307 ms 2024-03-09T00:00:59.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 675567 11.17 MB/sec execute 187 sec latency 22.561 ms 2024-03-09T00:01:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:00 smithi022 ceph-mon[31894]: pgmap v200: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 402 op/s 2024-03-09T00:01:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:00 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:01:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:00 smithi062 ceph-mon[36129]: pgmap v200: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 402 op/s 2024-03-09T00:01:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:00 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:01:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:00 smithi192 ceph-mon[35995]: pgmap v200: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 402 op/s 2024-03-09T00:01:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:00 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:01:00.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 677349 11.15 MB/sec execute 188 sec latency 103.045 ms 2024-03-09T00:01:01.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 678284 11.14 MB/sec execute 189 sec latency 63.220 ms 2024-03-09T00:01:02.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:02 smithi022 ceph-mon[31894]: pgmap v201: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 403 op/s 2024-03-09T00:01:02.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:02 smithi062 ceph-mon[36129]: pgmap v201: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 403 op/s 2024-03-09T00:01:02.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:02 smithi192 ceph-mon[35995]: pgmap v201: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 403 op/s 2024-03-09T00:01:02.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 679719 11.12 MB/sec execute 190 sec latency 45.619 ms 2024-03-09T00:01:03.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 682118 11.13 MB/sec execute 191 sec latency 15.675 ms 2024-03-09T00:01:04.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:04 smithi022 ceph-mon[31894]: pgmap v202: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-03-09T00:01:04.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:04 smithi062 ceph-mon[36129]: pgmap v202: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-03-09T00:01:04.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:04 smithi192 ceph-mon[35995]: pgmap v202: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-03-09T00:01:04.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 683948 11.10 MB/sec execute 192 sec latency 38.449 ms 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.937+0000 7f0d0d93b640 1 -- 172.21.15.22:0/1716406254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d080ffa60 msgr2=0x7f0d080ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.937+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/1716406254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d080ffa60 0x7f0d080ffe40 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009930 tx=0x7f0d0002bc80 comp rx=0 tx=0).stop 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 -- 172.21.15.22:0/1716406254 shutdown_connections 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/1716406254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d08106670 0x7f0d0810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/1716406254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d08100410 0x7f0d08105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/1716406254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d080ffa60 0x7f0d080ffe40 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 -- 172.21.15.22:0/1716406254 >> 172.21.15.22:0/1716406254 conn(0x7f0d08076780 msgr2=0x7f0d080fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 -- 172.21.15.22:0/1716406254 shutdown_connections 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 -- 172.21.15.22:0/1716406254 wait complete. 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 Processor -- start 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 -- start start 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d080ffa60 0x7f0d0819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d08106670 0x7f0d0819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.938+0000 7f0d0d93b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d08197850 0x7f0d08197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0d93b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08111650 con 0x7f0d08197850 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0d93b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081114d0 con 0x7f0d08106670 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0d93b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081117d0 con 0x7f0d080ffa60 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d08197850 0x7f0d08197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d08197850 0x7f0d08197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52506/0 (socket says 172.21.15.22:52506) 2024-03-09T00:01:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 -- 172.21.15.22:0/557447025 learned_addr learned my addr 172.21.15.22:0/557447025 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d080ffa60 0x7f0d0819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d067fc640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d08106670 0x7f0d0819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 -- 172.21.15.22:0/557447025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d080ffa60 msgr2=0x7f0d0819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d080ffa60 0x7f0d0819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 -- 172.21.15.22:0/557447025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d08106670 msgr2=0x7f0d0819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d08106670 0x7f0d0819dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 -- 172.21.15.22:0/557447025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d081983e0 con 0x7f0d08197850 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d077fe640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d08197850 0x7f0d08197d00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800eb10 tx=0x7f0cf800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0c939640 1 -- 172.21.15.22:0/557447025 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800ce20 con 0x7f0d08197850 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d081986d0 con 0x7f0d08197850 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0c939640 1 -- 172.21.15.22:0/557447025 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8005020 con 0x7f0d08197850 2024-03-09T00:01:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0c939640 1 -- 172.21.15.22:0/557447025 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8010690 con 0x7f0d08197850 2024-03-09T00:01:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.939+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d081a4860 con 0x7f0d08197850 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.941+0000 7f0d0c939640 1 -- 172.21.15.22:0/557447025 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0cf8005540 con 0x7f0d08197850 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 Processor -- start 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 -- start start 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be9776ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776bfd0 0x55be9776c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d0c939640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cf40a2a10 0x7f0cf40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be97779960 0x55be9777bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be976d05f0 con 0x55be9776cd40 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be97679930 con 0x55be97779960 2024-03-09T00:01:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d105f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be976cc690 con 0x55be9776bfd0 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be9776ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776bfd0 0x55be9776c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be9776ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52508/0 (socket says 172.21.15.22:52508) 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d067fc640 1 -- 172.21.15.22:0/3503707399 learned_addr learned my addr 172.21.15.22:0/3503707399 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776bfd0 0x55be9776c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36848/0 (socket says 172.21.15.22:36848) 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d0c939640 1 -- 172.21.15.22:0/557447025 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0cf80cf820 con 0x7f0d08197850 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.942+0000 7f0d067fc640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cf40a2a10 0x7f0cf40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d077fe640 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be97779960 0x55be9777bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d06ffd640 1 -- 172.21.15.22:0/3503707399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be97779960 msgr2=0x55be9777bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be97779960 0x55be9777bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d06ffd640 1 -- 172.21.15.22:0/3503707399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 msgr2=0x55be9776ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be9776ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d06ffd640 1 -- 172.21.15.22:0/3503707399 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be973e5210 con 0x55be9776bfd0 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d067fc640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cf40a2a10 0x7f0cf40a4ed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00058c0 tx=0x7f0cf000a2d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776bfd0 0x55be9776c450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009230 tx=0x7f0d00005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d2d0d423ce298d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0cfdffb640 1 -- 172.21.15.22:0/3503707399 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0002ca70 con 0x55be9776bfd0 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0cfdffb640 1 -- 172.21.15.22:0/3503707399 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d0002cc10 con 0x55be9776bfd0 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.943+0000 7f0cfdffb640 1 -- 172.21.15.22:0/3503707399 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000050d0 con 0x55be9776bfd0 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 -- 172.21.15.22:0/3503707399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776bfd0 msgr2=0x55be9776c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776bfd0 0x55be9776c450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009230 tx=0x7f0d00005e80 comp rx=0 tx=0).stop 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 -- 172.21.15.22:0/3503707399 shutdown_connections 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be97779960 0x55be9777bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776bfd0 0x55be9776c450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3503707399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be9776ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 -- 172.21.15.22:0/3503707399 >> 172.21.15.22:0/3503707399 conn(0x55be977335b0 msgr2=0x55be97766cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 -- 172.21.15.22:0/3503707399 shutdown_connections 2024-03-09T00:01:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 -- 172.21.15.22:0/3503707399 wait complete. 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.944+0000 7f0d105f9740 1 -- start start 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 1 Processor -- start 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be97776ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be97779960 0x55be97777420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9777c630 con 0x55be9776cd40 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9768cd90 con 0x55be9776bfd0 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9777c7b0 con 0x55be97779960 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d105f9740 10 client.0 _ll_register_callbacks cb 0x55be973e4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be97776ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be97776ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52514/0 (socket says 172.21.15.22:52514) 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 -- 172.21.15.22:0/2262748256 learned_addr learned my addr 172.21.15.22:0/2262748256 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d077fe640 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be97779960 0x55be97777420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 -- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be97779960 msgr2=0x55be97777420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be97779960 0x55be97777420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 -- 172.21.15.22:0/2262748256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 msgr2=0x55be977769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977769a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 -- 172.21.15.22:0/2262748256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be974b1620 con 0x55be9776cd40 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0d067fc640 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be97776ee0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000f700 tx=0x7f0cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa35c3653689e5fa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.945+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2262748256 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf001e610 con 0x55be9776cd40 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2262748256 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0cf001e7b0 con 0x55be9776cd40 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d0002cc10 con 0x55be9776cd40 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2262748256 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf001e9b0 con 0x55be9776cd40 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 10 client.17742 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55be97788930 con 0x55be9776cd40 2024-03-09T00:01:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2262748256 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d0002cc10 con 0x55be9776cd40 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2262748256 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cf0026070 con 0x55be9776cd40 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 10 client.17742 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 10 client.17742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 20 client.17742 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be97789df0 0x55be9778c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 4 client.17742 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be976ddb30 con 0x55be97789df0 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.946+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be97789df0 0x55be9778c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.947+0000 7f0ce6ffd640 10 client.17742 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.947+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be97789df0 0x55be9778c200 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf47597e4d6e6b76 server_cookie=4839d4197841ac8e in_seq=0 out_seq=0 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.948+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2262748256 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be976ddb30 con 0x55be97789df0 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.948+0000 7f0ce6ffd640 10 client.17742 handle_command_reply: tid=0 2024-03-09T00:01:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.948+0000 7f0d105f9740 1 client.17742 shutdown 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 msgr2=0x55be97776ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be97776ee0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000f700 tx=0x7f0cf00046c0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 shutdown_connections 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be97789df0 0x55be9778c200 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be97779960 0x55be97777420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be9776cd40 0x55be97776ee0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2262748256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0ce6ffd640 0 client.17742 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 >> 172.21.15.22:0/2262748256 conn(0x55be977335b0 msgr2=0x55be97768a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 shutdown_connections 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 1 -- 172.21.15.22:0/2262748256 wait complete. 2024-03-09T00:01:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.949+0000 7f0d105f9740 20 client.17742 trim_cache size 0 max 16384 2024-03-09T00:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 Processor -- start 2024-03-09T00:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 -- start start 2024-03-09T00:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9778c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be97775270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081ae710 con 0x55be97779960 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081ac2b0 con 0x55be9776bfd0 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d105f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081aa350 con 0x55be9776cd40 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be97775270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be97775270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52528/0 (socket says 172.21.15.22:52528) 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.962+0000 7f0d077fe640 1 -- 172.21.15.22:0/3146227422 learned_addr learned my addr 172.21.15.22:0/3146227422 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d067fc640 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9778c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d077fe640 1 -- 172.21.15.22:0/3146227422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 msgr2=0x55be9778c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d077fe640 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9778c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d077fe640 1 -- 172.21.15.22:0/3146227422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 msgr2=0x55be977790d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d077fe640 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977790d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d077fe640 1 -- 172.21.15.22:0/3146227422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be97774ef0 con 0x55be97779960 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0d077fe640 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be97775270 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80ccbd0 tx=0x7f0cf8005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0ff0e250e2865 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0cfd7fa640 1 -- 172.21.15.22:0/3146227422 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf809fbd0 con 0x55be97779960 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0cfd7fa640 1 -- 172.21.15.22:0/3146227422 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf80a4220 con 0x55be97779960 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.963+0000 7f0cfd7fa640 1 -- 172.21.15.22:0/3146227422 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80cf3d0 con 0x55be97779960 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- 172.21.15.22:0/3146227422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 msgr2=0x55be97775270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be97775270 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80ccbd0 tx=0x7f0cf8005290 comp rx=0 tx=0).stop 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- 172.21.15.22:0/3146227422 shutdown_connections 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be97775270 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9778c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 --2- 172.21.15.22:0/3146227422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be977790d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- 172.21.15.22:0/3146227422 >> 172.21.15.22:0/3146227422 conn(0x55be977335b0 msgr2=0x55be977667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- 172.21.15.22:0/3146227422 shutdown_connections 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- 172.21.15.22:0/3146227422 wait complete. 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- start start 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 Processor -- start 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be9777f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9777fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be9777ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081aa950 con 0x55be97779960 2024-03-09T00:01:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081abc50 con 0x55be9776bfd0 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.964+0000 7f0d105f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081aaf90 con 0x55be9776cd40 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d105f9740 10 client.0 _ll_register_callbacks cb 0x7f0d00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be9777ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be9777f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be9777ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52540/0 (socket says 172.21.15.22:52540) 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 -- 172.21.15.22:0/2931476930 learned_addr learned my addr 172.21.15.22:0/2931476930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d067fc640 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9777fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 -- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 msgr2=0x55be9777fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9777fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 -- 172.21.15.22:0/2931476930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 msgr2=0x55be9777f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be9777f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 -- 172.21.15.22:0/2931476930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be977e0400 con 0x55be97779960 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be9777f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d077fe640 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be9777ff90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8097ee0 tx=0x7f0cf809bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e3f825cfe2f035 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2931476930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80afe60 con 0x55be97779960 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2931476930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f0cf80cf3d0 con 0x55be97779960 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cf80a4220 con 0x55be97779960 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2931476930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf80ccff0 con 0x55be97779960 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d105f9740 10 client.17748 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.965+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55be977e1710 con 0x55be97779960 2024-03-09T00:01:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2931476930 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf80a4220 con 0x55be97779960 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2931476930 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cf809aed0 con 0x55be97779960 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d105f9740 10 client.17748 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d105f9740 10 client.17748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d105f9740 20 client.17748 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be977e1d60 0x55be977e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d105f9740 4 client.17748 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be976d7e80 con 0x55be977e1d60 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be977e1d60 0x55be977e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0ce6ffd640 10 client.17748 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:04.966+0000 7f0d06ffd640 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be977e1d60 0x55be977e4180 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a494b606ac24a763 server_cookie=f9c2c8925ef4c42d in_seq=0 out_seq=0 2024-03-09T00:01:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0ce6ffd640 1 -- 172.21.15.22:0/2931476930 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be976d7e80 con 0x55be977e1d60 2024-03-09T00:01:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0ce6ffd640 10 client.17748 handle_command_reply: tid=0 2024-03-09T00:01:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 client.17748 shutdown 2024-03-09T00:01:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 msgr2=0x55be9777ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be9777ff90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8097ee0 tx=0x7f0cf809bc00 comp rx=0 tx=0).stop 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 shutdown_connections 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55be977e1d60 0x55be977e4180 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0ce6ffd640 0 client.17748 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55be97779960 0x55be9777ff90 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be9776cd40 0x55be9777fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 --2- 172.21.15.22:0/2931476930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be9776bfd0 0x55be9777f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 >> 172.21.15.22:0/2931476930 conn(0x55be977335b0 msgr2=0x55be977751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 shutdown_connections 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.464+0000 7f0d105f9740 1 -- 172.21.15.22:0/2931476930 wait complete. 2024-03-09T00:01:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d105f9740 20 client.17748 trim_cache size 0 max 16384 2024-03-09T00:01:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:01:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:01:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:01:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cf40a2a10 msgr2=0x7f0cf40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cf40a2a10 0x7f0cf40a4ed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00058c0 tx=0x7f0cf000a2d0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d08197850 msgr2=0x7f0d08197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d08197850 0x7f0d08197d00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800eb10 tx=0x7f0cf800ef10 comp rx=0 tx=0).stop 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 shutdown_connections 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0cf40a2a10 0x7f0cf40a4ed0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d08197850 0x7f0d08197d00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d08106670 0x7f0d0819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 --2- 172.21.15.22:0/557447025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d080ffa60 0x7f0d0819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.465+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 >> 172.21.15.22:0/557447025 conn(0x7f0d08076780 msgr2=0x7f0d08104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.466+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 shutdown_connections 2024-03-09T00:01:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:05.466+0000 7f0d0d93b640 1 -- 172.21.15.22:0/557447025 wait complete. 2024-03-09T00:01:05.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 684837 11.08 MB/sec execute 193 sec latency 140.496 ms 2024-03-09T00:01:06.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:06 smithi022 ceph-mon[31894]: pgmap v203: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-03-09T00:01:06.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:06 smithi022 ceph-mon[31894]: osdmap e99: 12 total, 12 up, 12 in 2024-03-09T00:01:06.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:06 smithi062 ceph-mon[36129]: pgmap v203: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-03-09T00:01:06.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:06 smithi062 ceph-mon[36129]: osdmap e99: 12 total, 12 up, 12 in 2024-03-09T00:01:06.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:06 smithi192 ceph-mon[35995]: pgmap v203: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-03-09T00:01:06.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:06 smithi192 ceph-mon[35995]: osdmap e99: 12 total, 12 up, 12 in 2024-03-09T00:01:06.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 685546 11.05 MB/sec execute 194 sec latency 63.847 ms 2024-03-09T00:01:07.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 685951 11.00 MB/sec execute 195 sec latency 112.667 ms 2024-03-09T00:01:08.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:08 smithi022 ceph-mon[31894]: pgmap v205: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-03-09T00:01:08.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:08 smithi062 ceph-mon[36129]: pgmap v205: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-03-09T00:01:08.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:08 smithi192 ceph-mon[35995]: pgmap v205: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-03-09T00:01:08.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 686459 10.95 MB/sec execute 196 sec latency 78.554 ms 2024-03-09T00:01:09.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 688404 10.95 MB/sec execute 197 sec latency 18.770 ms 2024-03-09T00:01:10.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:10 smithi022 ceph-mon[31894]: pgmap v206: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-03-09T00:01:10.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:10 smithi062 ceph-mon[36129]: pgmap v206: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-03-09T00:01:10.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:10 smithi192 ceph-mon[35995]: pgmap v206: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-03-09T00:01:10.718 INFO:tasks.workunit.client.0.smithi022.stdout: 1 689630 10.93 MB/sec execute 198 sec latency 97.362 ms 2024-03-09T00:01:11.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 692156 10.94 MB/sec execute 199 sec latency 12.962 ms 2024-03-09T00:01:12.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:12 smithi022 ceph-mon[31894]: pgmap v207: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 406 op/s 2024-03-09T00:01:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:12 smithi062 ceph-mon[36129]: pgmap v207: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 406 op/s 2024-03-09T00:01:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:12 smithi192 ceph-mon[35995]: pgmap v207: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 406 op/s 2024-03-09T00:01:12.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 693926 10.94 MB/sec execute 200 sec latency 12.144 ms 2024-03-09T00:01:13.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 696341 10.96 MB/sec execute 201 sec latency 15.734 ms 2024-03-09T00:01:14.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:14 smithi022 ceph-mon[31894]: pgmap v208: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 406 op/s 2024-03-09T00:01:14.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:14 smithi062 ceph-mon[36129]: pgmap v208: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 406 op/s 2024-03-09T00:01:14.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:14 smithi192 ceph-mon[35995]: pgmap v208: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 406 op/s 2024-03-09T00:01:14.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 697930 10.93 MB/sec execute 202 sec latency 47.790 ms 2024-03-09T00:01:15.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 699964 10.94 MB/sec execute 203 sec latency 68.484 ms 2024-03-09T00:01:16.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:16 smithi022 ceph-mon[31894]: pgmap v209: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-03-09T00:01:16.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:16 smithi062 ceph-mon[36129]: pgmap v209: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-03-09T00:01:16.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:16 smithi192 ceph-mon[35995]: pgmap v209: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-03-09T00:01:16.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 701724 10.92 MB/sec execute 204 sec latency 46.409 ms 2024-03-09T00:01:17.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 703816 10.93 MB/sec execute 205 sec latency 20.026 ms 2024-03-09T00:01:18.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:18 smithi022 ceph-mon[31894]: pgmap v210: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 -- 172.21.15.22:0/851653128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 msgr2=0x7f60e41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 --2- 172.21.15.22:0/851653128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e41077e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d402bcd0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 -- 172.21.15.22:0/851653128 shutdown_connections 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 --2- 172.21.15.22:0/851653128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e4069330 0x7f60e410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 --2- 172.21.15.22:0/851653128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60e4068970 0x7f60e4068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 --2- 172.21.15.22:0/851653128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e41077e0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 -- 172.21.15.22:0/851653128 >> 172.21.15.22:0/851653128 conn(0x7f60e4077040 msgr2=0x7f60e4077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 -- 172.21.15.22:0/851653128 shutdown_connections 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.565+0000 7f60e928c640 1 -- 172.21.15.22:0/851653128 wait complete. 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 Processor -- start 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 -- start start 2024-03-09T00:01:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e4068970 0x7f60e419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60e4069330 0x7f60e419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4111660 con 0x7f60e4107400 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41114e0 con 0x7f60e4068970 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e928c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41117e0 con 0x7f60e4069330 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43374/0 (socket says 172.21.15.22:43374) 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e37fe640 1 -- 172.21.15.22:0/816902726 learned_addr learned my addr 172.21.15.22:0/816902726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e4068970 0x7f60e419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60da7fc640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60e4069330 0x7f60e419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e37fe640 1 -- 172.21.15.22:0/816902726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60e4069330 msgr2=0x7f60e419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.566+0000 7f60e37fe640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60e4069330 0x7f60e419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e37fe640 1 -- 172.21.15.22:0/816902726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e4068970 msgr2=0x7f60e419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e37fe640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e4068970 0x7f60e419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e37fe640 1 -- 172.21.15.22:0/816902726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4197ff0 con 0x7f60e4107400 2024-03-09T00:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e37fe640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e4197820 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00cab0 tx=0x7f60dc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e0ff9640 1 -- 172.21.15.22:0/816902726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00ae20 con 0x7f60e4107400 2024-03-09T00:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e41982e0 con 0x7f60e4107400 2024-03-09T00:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e0ff9640 1 -- 172.21.15.22:0/816902726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60dc00f930 con 0x7f60e4107400 2024-03-09T00:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e0ff9640 1 -- 172.21.15.22:0/816902726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc018b00 con 0x7f60e4107400 2024-03-09T00:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.567+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e41a4500 con 0x7f60e4107400 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.569+0000 7f60e0ff9640 1 -- 172.21.15.22:0/816902726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f60dc00fae0 con 0x7f60e4107400 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 Processor -- start 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 -- start start 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa60a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa61450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa70d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dcf9c55f0 con 0x558dcfa61d40 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dcf96e930 con 0x558dcfa60fd0 2024-03-09T00:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60ebe61740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dcf9c1690 con 0x558dcfa6e960 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60e0ff9640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60d00a2a10 0x7f60d00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60e37fe640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60d00a2a10 0x7f60d00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa60a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa60a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43386/0 (socket says 172.21.15.22:43386) 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 -- 172.21.15.22:0/2403749667 learned_addr learned my addr 172.21.15.22:0/2403749667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa70d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60e0ff9640 1 -- 172.21.15.22:0/816902726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f60dc0cf9c0 con 0x7f60e4107400 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa61450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 -- 172.21.15.22:0/2403749667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 msgr2=0x558dcfa70d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa70d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 -- 172.21.15.22:0/2403749667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 msgr2=0x558dcfa61450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa61450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.570+0000 7f60da7fc640 1 -- 172.21.15.22:0/2403749667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dcf6da210 con 0x558dcfa61d40 2024-03-09T00:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.571+0000 7f60e37fe640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60d00a2a10 0x7f60d00a4ed0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f60dc09b870 tx=0x7f60dc018400 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.571+0000 7f60da7fc640 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa60a90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009710 tx=0x7f60cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc31cedee344b7b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.571+0000 7f60d97fa640 1 -- 172.21.15.22:0/2403749667 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc004050 con 0x558dcfa61d40 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.571+0000 7f60d97fa640 1 -- 172.21.15.22:0/2403749667 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60cc00fca0 con 0x558dcfa61d40 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.571+0000 7f60d97fa640 1 -- 172.21.15.22:0/2403749667 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc003930 con 0x558dcfa61d40 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 -- 172.21.15.22:0/2403749667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 msgr2=0x558dcfa60a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa60a90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009710 tx=0x7f60cc009780 comp rx=0 tx=0).stop 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 -- 172.21.15.22:0/2403749667 shutdown_connections 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa70d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa61450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 --2- 172.21.15.22:0/2403749667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa60a90 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 -- 172.21.15.22:0/2403749667 >> 172.21.15.22:0/2403749667 conn(0x558dcfa285b0 msgr2=0x558dcfa5bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 -- 172.21.15.22:0/2403749667 shutdown_connections 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 -- 172.21.15.22:0/2403749667 wait complete. 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 -- start start 2024-03-09T00:01:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.572+0000 7f60ebe61740 1 Processor -- start 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa60fd0 0x558dcfa6b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa6c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dcfa71690 con 0x558dcfa61d40 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dcf981d90 con 0x558dcfa6e960 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dcfa71810 con 0x558dcfa60fd0 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 10 client.0 _ll_register_callbacks cb 0x558dcf6d9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43394/0 (socket says 172.21.15.22:43394) 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60da7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa60fd0 0x558dcfa6b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 -- 172.21.15.22:0/736421248 learned_addr learned my addr 172.21.15.22:0/736421248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60da7fc640 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa6c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 -- 172.21.15.22:0/736421248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa60fd0 msgr2=0x558dcfa6b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa60fd0 0x558dcfa6b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 -- 172.21.15.22:0/736421248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 msgr2=0x558dcfa6c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa6c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 -- 172.21.15.22:0/736421248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dcf7a6620 con 0x558dcfa61d40 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6be60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f60d4035000 tx=0x7f60d402c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01b4d5fe07f3f95 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ca7fc640 1 -- 172.21.15.22:0/736421248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4044070 con 0x558dcfa61d40 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ca7fc640 1 -- 172.21.15.22:0/736421248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f60d40046c0 con 0x558dcfa61d40 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.573+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60cc00fca0 con 0x558dcfa61d40 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ca7fc640 1 -- 172.21.15.22:0/736421248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d4051050 con 0x558dcfa61d40 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 10 client.17760 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558dcfa7d910 con 0x558dcfa61d40 2024-03-09T00:01:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ca7fc640 1 -- 172.21.15.22:0/736421248 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60cc00fca0 con 0x558dcfa61d40 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ca7fc640 1 -- 172.21.15.22:0/736421248 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f60d4003680 con 0x558dcfa61d40 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 10 client.17760 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 10 client.17760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 20 client.17760 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfa7edd0 0x558dcfa811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 4 client.17760 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dcf9d2b30 con 0x558dcfa7edd0 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60da7fc640 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfa7edd0 0x558dcfa811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60ca7fc640 10 client.17760 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.574+0000 7f60da7fc640 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfa7edd0 0x558dcfa811c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9317e8e0c116d677 server_cookie=7642a507c9b88ddd in_seq=0 out_seq=0 2024-03-09T00:01:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ca7fc640 1 -- 172.21.15.22:0/736421248 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558dcf9d2b30 con 0x558dcfa7edd0 2024-03-09T00:01:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ca7fc640 10 client.17760 handle_command_reply: tid=0 2024-03-09T00:01:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 client.17760 shutdown 2024-03-09T00:01:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 msgr2=0x558dcfa6be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6be60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f60d4035000 tx=0x7f60d402c8a0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 shutdown_connections 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfa7edd0 0x558dcfa811c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa6c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6be60 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 --2- 172.21.15.22:0/736421248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa60fd0 0x558dcfa6b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ca7fc640 0 client.17760 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 >> 172.21.15.22:0/736421248 conn(0x558dcfa285b0 msgr2=0x558dcfa5da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 shutdown_connections 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 1 -- 172.21.15.22:0/736421248 wait complete. 2024-03-09T00:01:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.576+0000 7f60ebe61740 20 client.17760 trim_cache size 0 max 16384 2024-03-09T00:01:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.590+0000 7f60ebe61740 1 Processor -- start 2024-03-09T00:01:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.590+0000 7f60ebe61740 1 -- start start 2024-03-09T00:01:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa6e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa6a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60ebe61740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41ae8b0 con 0x558dcfa61d40 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60ebe61740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41ac450 con 0x558dcfa60fd0 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60ebe61740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41aa510 con 0x558dcfa6e960 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60da7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa6e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43402/0 (socket says 172.21.15.22:43402) 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 -- 172.21.15.22:0/1215176324 learned_addr learned my addr 172.21.15.22:0/1215176324 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60da7fc640 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa6a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 -- 172.21.15.22:0/1215176324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 msgr2=0x558dcfa6a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa6a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 -- 172.21.15.22:0/1215176324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 msgr2=0x558dcfa6e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa6e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 -- 172.21.15.22:0/1215176324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dcfa602a0 con 0x558dcfa61d40 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60da7fc640 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa6e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.591+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6a1f0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f60d402c210 tx=0x7f60d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc23054fcddb4de4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60d8ff9640 1 -- 172.21.15.22:0/1215176324 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4044040 con 0x558dcfa61d40 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60d8ff9640 1 -- 172.21.15.22:0/1215176324 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d4009520 con 0x558dcfa61d40 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60d8ff9640 1 -- 172.21.15.22:0/1215176324 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4003850 con 0x558dcfa61d40 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60ebe61740 1 -- 172.21.15.22:0/1215176324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 msgr2=0x558dcfa6a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60ebe61740 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6a1f0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f60d402c210 tx=0x7f60d4005fd0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60ebe61740 1 -- 172.21.15.22:0/1215176324 shutdown_connections 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60ebe61740 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa6e960 0x558dcfa6a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60ebe61740 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa61d40 0x558dcfa6a1f0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60ebe61740 1 --2- 172.21.15.22:0/1215176324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa60fd0 0x558dcfa6e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.592+0000 7f60ebe61740 1 -- 172.21.15.22:0/1215176324 >> 172.21.15.22:0/1215176324 conn(0x558dcfa285b0 msgr2=0x558dcfa66c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 -- 172.21.15.22:0/1215176324 shutdown_connections 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 -- 172.21.15.22:0/1215176324 wait complete. 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 -- start start 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 Processor -- start 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa60fd0 0x558dcfa74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa61d40 0x558dcfa74bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41aab10 con 0x558dcfa60fd0 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41abdf0 con 0x558dcfa6e960 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41ab130 con 0x558dcfa61d40 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60ebe61740 10 client.0 _ll_register_callbacks cb 0x7f60cc009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60da7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa60fd0 0x558dcfa74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60da7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa60fd0 0x558dcfa74670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43418/0 (socket says 172.21.15.22:43418) 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60da7fc640 1 -- 172.21.15.22:0/4100730931 learned_addr learned my addr 172.21.15.22:0/4100730931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa61d40 0x558dcfa74bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.593+0000 7f60da7fc640 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60da7fc640 1 -- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa61d40 msgr2=0x558dcfa74bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60da7fc640 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa61d40 0x558dcfa74bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60da7fc640 1 -- 172.21.15.22:0/4100730931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 msgr2=0x558dcfa750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60da7fc640 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa750f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60da7fc640 1 -- 172.21.15.22:0/4100730931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dcfad54c0 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60da7fc640 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa60fd0 0x558dcfa74670 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009e50 tx=0x7f60cc021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359afa71ccf5753d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ca7fc640 1 -- 172.21.15.22:0/4100730931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc033020 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ca7fc640 1 -- 172.21.15.22:0/4100730931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f60cc005280 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60d4009520 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ca7fc640 1 -- 172.21.15.22:0/4100730931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60cc005480 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ebe61740 10 client.17766 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558dcfad67b0 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ca7fc640 1 -- 172.21.15.22:0/4100730931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d4009520 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ca7fc640 1 -- 172.21.15.22:0/4100730931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f60cc003af0 con 0x558dcfa60fd0 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ebe61740 10 client.17766 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.594+0000 7f60ebe61740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.595+0000 7f60ebe61740 20 client.17766 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.595+0000 7f60ebe61740 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfad6c30 0x558dcfad9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.595+0000 7f60ebe61740 4 client.17766 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:01:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.595+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558dcf9cce80 con 0x558dcfad6c30 2024-03-09T00:01:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.595+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfad6c30 0x558dcfad9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.595+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfad6c30 0x558dcfad9020 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f957e3f80d25516 server_cookie=893e22a477b2895f in_seq=0 out_seq=0 2024-03-09T00:01:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.595+0000 7f60ca7fc640 10 client.17766 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:18.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:18 smithi062 ceph-mon[36129]: pgmap v210: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-03-09T00:01:18.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:18 smithi192 ceph-mon[35995]: pgmap v210: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 320 op/s 2024-03-09T00:01:18.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 706009 10.93 MB/sec execute 206 sec latency 60.258 ms 2024-03-09T00:01:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.862+0000 7f60ca7fc640 1 -- 172.21.15.22:0/4100730931 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558dcf9cce80 con 0x558dcfad6c30 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.862+0000 7f60ca7fc640 10 client.17766 handle_command_reply: tid=0 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.862+0000 7f60ebe61740 1 client.17766 shutdown 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.862+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa60fd0 msgr2=0x558dcfa74670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.862+0000 7f60ebe61740 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa60fd0 0x558dcfa74670 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009e50 tx=0x7f60cc021040 comp rx=0 tx=0).stop 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 shutdown_connections 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dcfad6c30 0x558dcfad9020 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dcfa6e960 0x558dcfa750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcfa61d40 0x558dcfa74bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 --2- 172.21.15.22:0/4100730931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dcfa60fd0 0x558dcfa74670 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ca7fc640 0 client.17766 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 >> 172.21.15.22:0/4100730931 conn(0x558dcfa285b0 msgr2=0x558dcfa6a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 shutdown_connections 2024-03-09T00:01:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 1 -- 172.21.15.22:0/4100730931 wait complete. 2024-03-09T00:01:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60ebe61740 20 client.17766 trim_cache size 0 max 16384 2024-03-09T00:01:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:01:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:01:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:01:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60d00a2a10 msgr2=0x7f60d00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60d00a2a10 0x7f60d00a4ed0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f60dc09b870 tx=0x7f60dc018400 comp rx=0 tx=0).stop 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 msgr2=0x7f60e4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e4197820 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00cab0 tx=0x7f60dc00cf70 comp rx=0 tx=0).stop 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 shutdown_connections 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60d00a2a10 0x7f60d00a4ed0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e4107400 0x7f60e4197820 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60e4069330 0x7f60e419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 --2- 172.21.15.22:0/816902726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e4068970 0x7f60e419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.863+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 >> 172.21.15.22:0/816902726 conn(0x7f60e4077040 msgr2=0x7f60e40fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.864+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 shutdown_connections 2024-03-09T00:01:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:18.864+0000 7f60e928c640 1 -- 172.21.15.22:0/816902726 wait complete. 2024-03-09T00:01:19.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 707413 10.92 MB/sec execute 207 sec latency 140.828 ms 2024-03-09T00:01:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:19 smithi022 ceph-mon[31894]: osdmap e100: 12 total, 12 up, 12 in 2024-03-09T00:01:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:19 smithi022 ceph-mon[31894]: pgmap v212: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-03-09T00:01:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:19 smithi062 ceph-mon[36129]: osdmap e100: 12 total, 12 up, 12 in 2024-03-09T00:01:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:19 smithi062 ceph-mon[36129]: pgmap v212: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-03-09T00:01:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:19 smithi192 ceph-mon[35995]: osdmap e100: 12 total, 12 up, 12 in 2024-03-09T00:01:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:19 smithi192 ceph-mon[35995]: pgmap v212: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-03-09T00:01:20.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 709467 10.92 MB/sec execute 208 sec latency 99.887 ms 2024-03-09T00:01:21.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 711046 10.91 MB/sec execute 209 sec latency 17.290 ms 2024-03-09T00:01:22.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:22 smithi062 ceph-mon[36129]: pgmap v213: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2024-03-09T00:01:22.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:22 smithi192 ceph-mon[35995]: pgmap v213: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2024-03-09T00:01:22.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:22 smithi022 ceph-mon[31894]: pgmap v213: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2024-03-09T00:01:22.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 713727 10.94 MB/sec execute 210 sec latency 15.979 ms 2024-03-09T00:01:23.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 715253 10.92 MB/sec execute 211 sec latency 55.022 ms 2024-03-09T00:01:24.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:24 smithi062 ceph-mon[36129]: pgmap v214: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2024-03-09T00:01:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:24 smithi192 ceph-mon[35995]: pgmap v214: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2024-03-09T00:01:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:24 smithi022 ceph-mon[31894]: pgmap v214: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2024-03-09T00:01:24.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 717638 10.93 MB/sec execute 212 sec latency 17.140 ms 2024-03-09T00:01:25.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 719139 10.90 MB/sec execute 213 sec latency 138.804 ms 2024-03-09T00:01:25.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:01:25.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 -- 172.21.15.22:0/1342279871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881054b0 msgr2=0x7fe48810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 --2- 172.21.15.22:0/1342279871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881054b0 0x7fe48810d080 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe480009ca0 tx=0x7fe48002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 -- 172.21.15.22:0/1342279871 shutdown_connections 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 --2- 172.21.15.22:0/1342279871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881054b0 0x7fe48810d080 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 --2- 172.21.15.22:0/1342279871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe488100d90 0x7fe488104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 --2- 172.21.15.22:0/1342279871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4881003e0 0x7fe4881007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 -- 172.21.15.22:0/1342279871 >> 172.21.15.22:0/1342279871 conn(0x7fe488076570 msgr2=0x7fe488076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 -- 172.21.15.22:0/1342279871 shutdown_connections 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.855+0000 7fe48fc38640 1 -- 172.21.15.22:0/1342279871 wait complete. 2024-03-09T00:01:25.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 Processor -- start 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 -- start start 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881003e0 0x7fe48819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4881054b0 0x7fe48819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4881979d0 0x7fe488197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488110570 con 0x7fe4881003e0 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881103f0 con 0x7fe4881054b0 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48fc38640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881106f0 con 0x7fe4881979d0 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881003e0 0x7fe48819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881003e0 0x7fe48819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55266/0 (socket says 172.21.15.22:55266) 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48d9d4640 1 -- 172.21.15.22:0/3489308799 learned_addr learned my addr 172.21.15.22:0/3489308799 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48e1d5640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4881979d0 0x7fe488197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4881054b0 0x7fe48819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.856+0000 7fe48d9d4640 1 -- 172.21.15.22:0/3489308799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4881979d0 msgr2=0x7fe488197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4881979d0 0x7fe488197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48d9d4640 1 -- 172.21.15.22:0/3489308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4881054b0 msgr2=0x7fe48819dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4881054b0 0x7fe48819dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48d9d4640 1 -- 172.21.15.22:0/3489308799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4881984f0 con 0x7fe4881003e0 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4881054b0 0x7fe48819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881003e0 0x7fe48819d7d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe48400e9f0 tx=0x7fe48400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe47effd640 1 -- 172.21.15.22:0/3489308799 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48400cd80 con 0x7fe4881003e0 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4881987e0 con 0x7fe4881003e0 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe47effd640 1 -- 172.21.15.22:0/3489308799 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe484010910 con 0x7fe4881003e0 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe47effd640 1 -- 172.21.15.22:0/3489308799 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4840188f0 con 0x7fe4881003e0 2024-03-09T00:01:25.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.857+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4881a4a30 con 0x7fe4881003e0 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.859+0000 7fe47effd640 1 -- 172.21.15.22:0/3489308799 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe484010ac0 con 0x7fe4881003e0 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 Processor -- start 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 -- start start 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fb4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe47effd640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4740a2970 0x7fe4740a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fbdc60 0x563fc9fc0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc9f17e00 con 0x563fc9fbdc60 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc9ec0b10 con 0x563fc9fb4760 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe492817740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc9f13ea0 con 0x563fc9fb5080 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe48e1d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fb4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4740a2970 0x7fe4740a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe48e1d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fb4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51288/0 (socket says 172.21.15.22:51288) 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe48e1d5640 1 -- 172.21.15.22:0/2330608597 learned_addr learned my addr 172.21.15.22:0/2330608597 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe48e1d5640 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fbdc60 0x563fc9fc0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.860+0000 7fe47effd640 1 -- 172.21.15.22:0/3489308799 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe4840cf630 con 0x7fe4881003e0 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe48d9d4640 1 -- 172.21.15.22:0/2330608597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 msgr2=0x563fc9fb4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fb4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe48d9d4640 1 -- 172.21.15.22:0/2330608597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fbdc60 msgr2=0x563fc9fc0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fbdc60 0x563fc9fc0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe48d9d4640 1 -- 172.21.15.22:0/2330608597 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc9cf67d0 con 0x563fc9fb5080 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4740a2970 0x7fe4740a4e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe48806a9e0 tx=0x7fe478005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe4840cc980 tx=0x7fe48401ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc438f627de9a846 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe473fff640 1 -- 172.21.15.22:0/2330608597 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484026040 con 0x563fc9fb5080 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe473fff640 1 -- 172.21.15.22:0/2330608597 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe48409b940 con 0x563fc9fb5080 2024-03-09T00:01:25.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.861+0000 7fe473fff640 1 -- 172.21.15.22:0/2330608597 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4840cce00 con 0x563fc9fb5080 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 -- 172.21.15.22:0/2330608597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 msgr2=0x563fc9fbd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe4840cc980 tx=0x7fe48401ccc0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 -- 172.21.15.22:0/2330608597 shutdown_connections 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fbdc60 0x563fc9fc0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 --2- 172.21.15.22:0/2330608597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fb4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 -- 172.21.15.22:0/2330608597 >> 172.21.15.22:0/2330608597 conn(0x563fc9faee90 msgr2=0x563fc9faf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 -- 172.21.15.22:0/2330608597 shutdown_connections 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 -- 172.21.15.22:0/2330608597 wait complete. 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.862+0000 7fe492817740 1 -- start start 2024-03-09T00:01:25.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 1 Processor -- start 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fbd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fbad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc9fc09f0 con 0x563fc9fb5080 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc9ed40a0 con 0x563fc9fb4760 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc9fc0b70 con 0x563fc9fbdc60 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe492817740 10 client.0 _ll_register_callbacks cb 0x563fc9ed3a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fbd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48e1d5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fbad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fba830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55278/0 (socket says 172.21.15.22:55278) 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 -- 172.21.15.22:0/716019011 learned_addr learned my addr 172.21.15.22:0/716019011 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 -- 172.21.15.22:0/716019011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 msgr2=0x563fc9fbad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fbad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 -- 172.21.15.22:0/716019011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 msgr2=0x563fc9fbd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fbd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d1d3640 1 -- 172.21.15.22:0/716019011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fc9c2c3f0 con 0x563fc9fb5080 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.863+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fbd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fba830 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe478004650 tx=0x7fe478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2603627d3861196c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe4717fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe4717fa640 1 -- 172.21.15.22:0/716019011 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47801caa0 con 0x563fc9fb5080 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe4717fa640 1 -- 172.21.15.22:0/716019011 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe47801cc40 con 0x563fc9fb5080 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe48409b940 con 0x563fc9fb5080 2024-03-09T00:01:25.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe4717fa640 1 -- 172.21.15.22:0/716019011 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe47801ce40 con 0x563fc9fb5080 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe492817740 10 client.17778 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563fc9bec0f0 con 0x563fc9fb5080 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe4717fa640 1 -- 172.21.15.22:0/716019011 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe48409b940 con 0x563fc9fb5080 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.864+0000 7fe4717fa640 1 -- 172.21.15.22:0/716019011 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe47801d780 con 0x563fc9fb5080 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe492817740 10 client.17778 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe492817740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe492817740 20 client.17778 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe492817740 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fc9fcd7f0 0x563fc9fcfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe492817740 4 client.17778 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fc9f25340 con 0x563fc9fcd7f0 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fc9fcd7f0 0x563fc9fcfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fc9fcd7f0 0x563fc9fcfbe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6091a61af21b3e75 server_cookie=42bfca8e7018821c in_seq=0 out_seq=0 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.865+0000 7fe4717fa640 10 client.17778 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe4717fa640 1 -- 172.21.15.22:0/716019011 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fc9f25340 con 0x563fc9fcd7f0 2024-03-09T00:01:25.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe4717fa640 10 client.17778 handle_command_reply: tid=0 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 client.17778 shutdown 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 msgr2=0x563fc9fba830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fba830 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe478004650 tx=0x7fe478004730 comp rx=0 tx=0).stop 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 shutdown_connections 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fc9fcd7f0 0x563fc9fcfbe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fbad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fba830 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 --2- 172.21.15.22:0/716019011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fbd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 >> 172.21.15.22:0/716019011 conn(0x563fc9faee90 msgr2=0x563fc9fb0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe4717fa640 0 client.17778 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 shutdown_connections 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 1 -- 172.21.15.22:0/716019011 wait complete. 2024-03-09T00:01:25.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.866+0000 7fe492817740 20 client.17778 trim_cache size 0 max 16384 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 Processor -- start 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 -- start start 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb4760 0x563fc9fbcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fbdc60 0x563fc9fb9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881b0cc0 con 0x563fc9fb4760 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881ae860 con 0x563fc9fbdc60 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe492817740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881ac920 con 0x563fc9fb5080 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb4760 0x563fc9fbcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb4760 0x563fc9fbcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55286/0 (socket says 172.21.15.22:55286) 2024-03-09T00:01:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1608075966 learned_addr learned my addr 172.21.15.22:0/1608075966 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48e1d5640 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fbdc60 0x563fc9fb9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1608075966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 msgr2=0x563fc9fbd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1608075966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fbdc60 msgr2=0x563fc9fb9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fbdc60 0x563fc9fb9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.883+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1608075966 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc9fd4d80 con 0x563fc9fb4760 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb4760 0x563fc9fbcf60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe484098310 tx=0x7fe484086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1180dc700283f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe4737fe640 1 -- 172.21.15.22:0/1608075966 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48409b940 con 0x563fc9fb4760 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe4737fe640 1 -- 172.21.15.22:0/1608075966 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe484097fb0 con 0x563fc9fb4760 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe4737fe640 1 -- 172.21.15.22:0/1608075966 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4840cf430 con 0x563fc9fb4760 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe492817740 1 -- 172.21.15.22:0/1608075966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb4760 msgr2=0x563fc9fbcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe492817740 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb4760 0x563fc9fbcf60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe484098310 tx=0x7fe484086040 comp rx=0 tx=0).stop 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe492817740 1 -- 172.21.15.22:0/1608075966 shutdown_connections 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe492817740 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fbdc60 0x563fc9fb9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe492817740 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fb5080 0x563fc9fbd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.884+0000 7fe492817740 1 --2- 172.21.15.22:0/1608075966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb4760 0x563fc9fbcf60 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 -- 172.21.15.22:0/1608075966 >> 172.21.15.22:0/1608075966 conn(0x563fc9faee90 msgr2=0x563fc9fba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 -- 172.21.15.22:0/1608075966 shutdown_connections 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 -- 172.21.15.22:0/1608075966 wait complete. 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 -- start start 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 Processor -- start 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fc2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fc3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881acf20 con 0x563fc9fb5080 2024-03-09T00:01:25.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881ae200 con 0x563fc9fb4760 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881ad540 con 0x563fc9fbdc60 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe492817740 10 client.0 _ll_register_callbacks cb 0x7fe478009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe48d1d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fc2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe48d9d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fc2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51318/0 (socket says 172.21.15.22:51318) 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe48d1d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fc32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55296/0 (socket says 172.21.15.22:55296) 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1771770196 learned_addr learned my addr 172.21.15.22:0/1771770196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.885+0000 7fe48e1d5640 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fc3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 msgr2=0x563fc9fc3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fc3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1771770196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 msgr2=0x563fc9fc32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fc32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe48d9d4640 1 -- 172.21.15.22:0/1771770196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fca022ec0 con 0x563fc9fb4760 2024-03-09T00:01:25.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe48d9d4640 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fc2d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe484097da0 tx=0x7fe48409b520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=157fc7e51a3a4ee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe4717fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe4717fa640 1 -- 172.21.15.22:0/1771770196 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4840862b0 con 0x563fc9fb4760 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe4717fa640 1 -- 172.21.15.22:0/1771770196 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe4840cf1d0 con 0x563fc9fb4760 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe484097fb0 con 0x563fc9fb4760 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe4717fa640 1 -- 172.21.15.22:0/1771770196 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe48409b940 con 0x563fc9fb4760 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe492817740 10 client.27167 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563fca0241b0 con 0x563fc9fb4760 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.886+0000 7fe4717fa640 1 -- 172.21.15.22:0/1771770196 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe484097fb0 con 0x563fc9fb4760 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe4717fa640 1 -- 172.21.15.22:0/1771770196 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe4840b9fc0 con 0x563fc9fb4760 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe492817740 10 client.27167 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe492817740 10 client.27167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:25.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe492817740 20 client.27167 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe492817740 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fca024430 0x563fca026820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe492817740 4 client.27167 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563fc9f1f690 con 0x563fca024430 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fca024430 0x563fca026820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe48d1d3640 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fca024430 0x563fca026820 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd646797200b47c1 server_cookie=f00b144b8e9a48af in_seq=0 out_seq=0 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe4717fa640 10 client.27167 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe4717fa640 1 -- 172.21.15.22:0/1771770196 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563fc9f1f690 con 0x563fca024430 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe4717fa640 10 client.27167 handle_command_reply: tid=0 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.887+0000 7fe492817740 1 client.27167 shutdown 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 msgr2=0x563fc9fc2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fc2d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe484097da0 tx=0x7fe48409b520 comp rx=0 tx=0).stop 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 shutdown_connections 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fca024430 0x563fca026820 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fc9fbdc60 0x563fc9fc3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe4717fa640 0 client.27167 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fc9fb5080 0x563fc9fc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 --2- 172.21.15.22:0/1771770196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc9fb4760 0x563fc9fc2d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 >> 172.21.15.22:0/1771770196 conn(0x563fc9faee90 msgr2=0x563fc9fb8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 shutdown_connections 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 1 -- 172.21.15.22:0/1771770196 wait complete. 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe492817740 20 client.27167 trim_cache size 0 max 16384 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:01:25.891 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:01:25.892 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:01:25.892 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:01:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4740a2970 msgr2=0x7fe4740a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe48fc38640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4740a2970 0x7fe4740a4e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe48806a9e0 tx=0x7fe478005f90 comp rx=0 tx=0).stop 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881003e0 msgr2=0x7fe48819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.888+0000 7fe48fc38640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881003e0 0x7fe48819d7d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe48400e9f0 tx=0x7fe48400eeb0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 shutdown_connections 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4740a2970 0x7fe4740a4e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4881979d0 0x7fe488197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4881054b0 0x7fe48819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 --2- 172.21.15.22:0/3489308799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4881003e0 0x7fe48819d7d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 >> 172.21.15.22:0/3489308799 conn(0x7fe488076570 msgr2=0x7fe4881040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 shutdown_connections 2024-03-09T00:01:25.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:25.889+0000 7fe48fc38640 1 -- 172.21.15.22:0/3489308799 wait complete. 2024-03-09T00:01:25.895 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:01:25.896 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:01:26.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 -- 172.21.15.22:0/1821214349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e8101ed0 msgr2=0x7fa2e8105f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 --2- 172.21.15.22:0/1821214349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e8101ed0 0x7fa2e8105f10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009900 tx=0x7fa2d402bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 -- 172.21.15.22:0/1821214349 shutdown_connections 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 --2- 172.21.15.22:0/1821214349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e81065f0 0x7fa2e810e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 --2- 172.21.15.22:0/1821214349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e8101ed0 0x7fa2e8105f10 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 --2- 172.21.15.22:0/1821214349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2e8101520 0x7fa2e8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 -- 172.21.15.22:0/1821214349 >> 172.21.15.22:0/1821214349 conn(0x7fa2e80fcd60 msgr2=0x7fa2e80ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.007+0000 7fa2eec85640 1 -- 172.21.15.22:0/1821214349 shutdown_connections 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 -- 172.21.15.22:0/1821214349 wait complete. 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 Processor -- start 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 -- start start 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e8101520 0x7fa2e819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e81065f0 0x7fa2e819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2e8197a80 0x7fa2e8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8111680 con 0x7fa2e81065f0 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8111500 con 0x7fa2e8197a80 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eec85640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8111800 con 0x7fa2e8101520 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e81065f0 0x7fa2e819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e81065f0 0x7fa2e819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55312/0 (socket says 172.21.15.22:55312) 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2e7fff640 1 -- 172.21.15.22:0/3544860161 learned_addr learned my addr 172.21.15.22:0/3544860161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2eca21640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e8101520 0x7fa2e819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.008+0000 7fa2ed222640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2e8197a80 0x7fa2e8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e7fff640 1 -- 172.21.15.22:0/3544860161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e8101520 msgr2=0x7fa2e819d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e7fff640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e8101520 0x7fa2e819d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e7fff640 1 -- 172.21.15.22:0/3544860161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2e8197a80 msgr2=0x7fa2e8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e7fff640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2e8197a80 0x7fa2e8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e7fff640 1 -- 172.21.15.22:0/3544860161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e81985a0 con 0x7fa2e81065f0 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e7fff640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e81065f0 0x7fa2e819de10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4036040 tx=0x7fa2d402cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e5ffb640 1 -- 172.21.15.22:0/3544860161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4041070 con 0x7fa2e81065f0 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e5ffb640 1 -- 172.21.15.22:0/3544860161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2d4004450 con 0x7fa2e81065f0 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e8198830 con 0x7fa2e81065f0 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2e5ffb640 1 -- 172.21.15.22:0/3544860161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4003dc0 con 0x7fa2e81065f0 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.009+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e81a4a20 con 0x7fa2e81065f0 2024-03-09T00:01:26.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.010+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e80680e0 con 0x7fa2e81065f0 2024-03-09T00:01:26.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.012+0000 7fa2e5ffb640 1 -- 172.21.15.22:0/3544860161 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa2d4044070 con 0x7fa2e81065f0 2024-03-09T00:01:26.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.013+0000 7fa2e5ffb640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2d80a2970 0x7fa2d80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.013+0000 7fa2e5ffb640 1 -- 172.21.15.22:0/3544860161 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa2d40f0e70 con 0x7fa2e81065f0 2024-03-09T00:01:26.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.013+0000 7fa2eca21640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2d80a2970 0x7fa2d80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.013+0000 7fa2eca21640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2d80a2970 0x7fa2d80a4e30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8198bc0 tx=0x7fa2dc005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.016+0000 7fa2e5ffb640 1 -- 172.21.15.22:0/3544860161 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa2d40bd230 con 0x7fa2e81065f0 2024-03-09T00:01:26.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.253+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2e8101ed0 con 0x7fa2e81065f0 2024-03-09T00:01:26.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.254+0000 7fa2e5ffb640 1 -- 172.21.15.22:0/3544860161 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fa2d40c12d0 con 0x7fa2e81065f0 2024-03-09T00:01:26.249 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:01:26.249 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:51.502302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26555":{"gid":26555,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2421521561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2421521561},{"type":"v1","addr":"172.21.15.62:6837","nonce":2421521561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:01:26.250 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2d80a2970 msgr2=0x7fa2d80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2d80a2970 0x7fa2d80a4e30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8198bc0 tx=0x7fa2dc005f90 comp rx=0 tx=0).stop 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e81065f0 msgr2=0x7fa2e819de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e81065f0 0x7fa2e819de10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4036040 tx=0x7fa2d402cb20 comp rx=0 tx=0).stop 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 shutdown_connections 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2d80a2970 0x7fa2d80a4e30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2e8197a80 0x7fa2e8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2e81065f0 0x7fa2e819de10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 --2- 172.21.15.22:0/3544860161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e8101520 0x7fa2e819d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 >> 172.21.15.22:0/3544860161 conn(0x7fa2e80fcd60 msgr2=0x7fa2e80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 shutdown_connections 2024-03-09T00:01:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.256+0000 7fa2eec85640 1 -- 172.21.15.22:0/3544860161 wait complete. 2024-03-09T00:01:26.263 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:01:26.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:01:26.287 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:26 smithi022 ceph-mon[31894]: pgmap v215: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-09T00:01:26.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:26 smithi062 ceph-mon[36129]: pgmap v215: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.375+0000 7f53f38b2640 1 -- 172.21.15.22:0/2898455988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 msgr2=0x7f53ec109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.375+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2898455988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec109930 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009900 tx=0x7f53e802bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- 172.21.15.22:0/2898455988 shutdown_connections 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2898455988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ec103e50 0x7f53ec110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2898455988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53ec103490 0x7f53ec103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2898455988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec109930 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- 172.21.15.22:0/2898455988 >> 172.21.15.22:0/2898455988 conn(0x7f53ec0fefa0 msgr2=0x7f53ec1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- 172.21.15.22:0/2898455988 shutdown_connections 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- 172.21.15.22:0/2898455988 wait complete. 2024-03-09T00:01:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 Processor -- start 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- start start 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ec103490 0x7f53ec199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53ec103e50 0x7f53ec19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec113830 con 0x7f53ec109550 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1136b0 con 0x7f53ec103490 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.376+0000 7f53f38b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1139b0 con 0x7f53ec103e50 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f164e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ec103490 0x7f53ec199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55336/0 (socket says 172.21.15.22:55336) 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53ec103e50 0x7f53ec19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 -- 172.21.15.22:0/2664440718 learned_addr learned my addr 172.21.15.22:0/2664440718 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 -- 172.21.15.22:0/2664440718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53ec103e50 msgr2=0x7f53ec19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53ec103e50 0x7f53ec19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 -- 172.21.15.22:0/2664440718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ec103490 msgr2=0x7f53ec199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ec103490 0x7f53ec199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 -- 172.21.15.22:0/2664440718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec19ed50 con 0x7f53ec109550 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f164e640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ec103490 0x7f53ec199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f1e4f640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec19e620 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f53e400eb10 tx=0x7f53e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53e27fc640 1 -- 172.21.15.22:0/2664440718 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4004b20 con 0x7f53ec109550 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec19f040 con 0x7f53ec109550 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53e27fc640 1 -- 172.21.15.22:0/2664440718 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53e4004cc0 con 0x7f53ec109550 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53e27fc640 1 -- 172.21.15.22:0/2664440718 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4010760 con 0x7f53ec109550 2024-03-09T00:01:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.377+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec1a68d0 con 0x7f53ec109550 2024-03-09T00:01:26.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:26 smithi192 ceph-mon[35995]: pgmap v215: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-09T00:01:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.379+0000 7f53e27fc640 1 -- 172.21.15.22:0/2664440718 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f53e4005020 con 0x7f53ec109550 2024-03-09T00:01:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 Processor -- start 2024-03-09T00:01:26.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 -- start start 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53e27fc640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f53d80a2970 0x7f53d80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f15580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c5edc360 con 0x5566c5f15ac0 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c5e78ec0 con 0x5566c5f151a0 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f6571740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c5e21d30 con 0x5566c5f1e6a0 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53e27fc640 1 -- 172.21.15.22:0/2664440718 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f53e40cf7d0 con 0x7f53ec109550 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f53d80a2970 0x7f53d80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.380+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f164e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f15580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f1e4f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55346/0 (socket says 172.21.15.22:55346) 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f0e4d640 1 -- 172.21.15.22:0/62491102 learned_addr learned my addr 172.21.15.22:0/62491102 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f164e640 1 -- 172.21.15.22:0/62491102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 msgr2=0x5566c5f20a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f164e640 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f20a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f164e640 1 -- 172.21.15.22:0/62491102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 msgr2=0x5566c5f1e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f164e640 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f164e640 1 -- 172.21.15.22:0/62491102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566c5e20bc0 con 0x5566c5f151a0 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f53d80a2970 0x7f53d80a4e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0119d0 tx=0x7f53dc009c00 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53f164e640 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f15580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53e8000c00 tx=0x7f53e8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9d126c45d75577 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53d77fe640 1 -- 172.21.15.22:0/62491102 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8042070 con 0x5566c5f151a0 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53d77fe640 1 -- 172.21.15.22:0/62491102 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53e8004a70 con 0x5566c5f151a0 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.381+0000 7f53d77fe640 1 -- 172.21.15.22:0/62491102 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8005180 con 0x5566c5f151a0 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 -- 172.21.15.22:0/62491102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 msgr2=0x5566c5f15580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f15580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53e8000c00 tx=0x7f53e8005e80 comp rx=0 tx=0).stop 2024-03-09T00:01:26.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 -- 172.21.15.22:0/62491102 shutdown_connections 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f20a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 --2- 172.21.15.22:0/62491102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f15580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 -- 172.21.15.22:0/62491102 >> 172.21.15.22:0/62491102 conn(0x5566c5f0f8d0 msgr2=0x5566c5f0fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 -- 172.21.15.22:0/62491102 shutdown_connections 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 -- 172.21.15.22:0/62491102 wait complete. 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 -- start start 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.382+0000 7f53f6571740 1 Processor -- start 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f6571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f6571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f6571740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f6571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c5e174a0 con 0x5566c5f15ac0 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f6571740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c5e74f60 con 0x5566c5f151a0 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f6571740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c5f21430 con 0x5566c5f1e6a0 2024-03-09T00:01:26.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f6571740 10 client.0 _ll_register_callbacks cb 0x5566c5e72d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55362/0 (socket says 172.21.15.22:55362) 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2105558210 learned_addr learned my addr 172.21.15.22:0/2105558210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f164e640 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f1e4f640 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 msgr2=0x5566c5f1c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2105558210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 msgr2=0x5566c5f1b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2105558210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566c5b8d3f0 con 0x5566c5f15ac0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1bbe0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005880 tx=0x7f53dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cbc93c22e54601 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.383+0000 7f53d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2105558210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00fe50 con 0x5566c5f15ac0 2024-03-09T00:01:26.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2105558210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f53dc003800 con 0x5566c5f15ac0 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e8004a70 con 0x5566c5f15ac0 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2105558210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53dc003a00 con 0x5566c5f15ac0 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 10 client.17802 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5566c5c577d0 con 0x5566c5f15ac0 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2105558210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53e8004a70 con 0x5566c5f15ac0 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2105558210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53dc028400 con 0x5566c5f15ac0 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 10 client.17802 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 10 client.17802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 20 client.17802 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f2e800 0x5566c5f30c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 4 client.17802 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566c5e86400 con 0x5566c5f2e800 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.384+0000 7f53f164e640 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f2e800 0x5566c5f30c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.385+0000 7f53d4ff9640 10 client.17802 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:26.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.385+0000 7f53f164e640 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f2e800 0x5566c5f30c40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70d561eb4ecb8c8f server_cookie=fc5fdc42816f87bb in_seq=0 out_seq=0 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.385+0000 7f53d4ff9640 1 -- 172.21.15.22:0/2105558210 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566c5e86400 con 0x5566c5f2e800 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.385+0000 7f53d4ff9640 10 client.17802 handle_command_reply: tid=0 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.385+0000 7f53f6571740 1 client.17802 shutdown 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 msgr2=0x5566c5f1bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1bbe0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005880 tx=0x7f53dc005f90 comp rx=0 tx=0).stop 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 shutdown_connections 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f2e800 0x5566c5f30c40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f1bbe0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 --2- 172.21.15.22:0/2105558210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53d4ff9640 0 client.17802 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 >> 172.21.15.22:0/2105558210 conn(0x5566c5f0f8d0 msgr2=0x5566c5f11620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 shutdown_connections 2024-03-09T00:01:26.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 1 -- 172.21.15.22:0/2105558210 wait complete. 2024-03-09T00:01:26.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.386+0000 7f53f6571740 20 client.17802 trim_cache size 0 max 16384 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 Processor -- start 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 -- start start 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1b2de0 con 0x5566c5f15ac0 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1b0980 con 0x5566c5f151a0 2024-03-09T00:01:26.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f6571740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1aea40 con 0x5566c5f1e6a0 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f19f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55368/0 (socket says 172.21.15.22:55368) 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2482999598 learned_addr learned my addr 172.21.15.22:0/2482999598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f1e4f640 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f164e640 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2482999598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 msgr2=0x5566c5f1a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2482999598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 msgr2=0x5566c5f1ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 -- 172.21.15.22:0/2482999598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566c5f172b0 con 0x5566c5f15ac0 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.399+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f19f90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009800 tx=0x7f53dc0091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d4573640059998 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53d6ffd640 1 -- 172.21.15.22:0/2482999598 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc039070 con 0x5566c5f15ac0 2024-03-09T00:01:26.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53d6ffd640 1 -- 172.21.15.22:0/2482999598 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc011350 con 0x5566c5f15ac0 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53d6ffd640 1 -- 172.21.15.22:0/2482999598 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc003ba0 con 0x5566c5f15ac0 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 -- 172.21.15.22:0/2482999598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 msgr2=0x5566c5f19f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f19f90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009800 tx=0x7f53dc0091e0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 -- 172.21.15.22:0/2482999598 shutdown_connections 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f1e6a0 0x5566c5f1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f15ac0 0x5566c5f19f90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 --2- 172.21.15.22:0/2482999598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f151a0 0x5566c5f1ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 -- 172.21.15.22:0/2482999598 >> 172.21.15.22:0/2482999598 conn(0x5566c5f0f8d0 msgr2=0x5566c5f35630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 -- 172.21.15.22:0/2482999598 shutdown_connections 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.400+0000 7f53f6571740 1 -- 172.21.15.22:0/2482999598 wait complete. 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 -- start start 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 Processor -- start 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f151a0 0x5566c5f24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f15ac0 0x5566c5f24c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f1e6a0 0x5566c5f84dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1af040 con 0x5566c5f151a0 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1b0320 con 0x5566c5f1e6a0 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1af660 con 0x5566c5f15ac0 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f6571740 10 client.0 _ll_register_callbacks cb 0x7f53dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f164e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f151a0 0x5566c5f24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f15ac0 0x5566c5f24c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f164e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f151a0 0x5566c5f24740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55384/0 (socket says 172.21.15.22:55384) 2024-03-09T00:01:26.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f0e4d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f15ac0 0x5566c5f24c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:53584/0 (socket says 172.21.15.22:53584) 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f164e640 1 -- 172.21.15.22:0/4273733022 learned_addr learned my addr 172.21.15.22:0/4273733022 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.401+0000 7f53f1e4f640 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f1e6a0 0x5566c5f84dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f164e640 1 -- 172.21.15.22:0/4273733022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f15ac0 msgr2=0x5566c5f24c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f164e640 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f15ac0 0x5566c5f24c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f164e640 1 -- 172.21.15.22:0/4273733022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f1e6a0 msgr2=0x5566c5f84dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f164e640 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f1e6a0 0x5566c5f84dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f164e640 1 -- 172.21.15.22:0/4273733022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566c5f85300 con 0x5566c5f151a0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f164e640 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f151a0 0x5566c5f24740 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f53e802c850 tx=0x7f53e80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861e638051b7e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53d4ff9640 1 -- 172.21.15.22:0/4273733022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8042040 con 0x5566c5f151a0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53d4ff9640 1 -- 172.21.15.22:0/4273733022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f53e804eb70 con 0x5566c5f151a0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53dc011350 con 0x5566c5f151a0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53d4ff9640 1 -- 172.21.15.22:0/4273733022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e805c050 con 0x5566c5f151a0 2024-03-09T00:01:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f6571740 10 client.17808 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5566c5f86600 con 0x5566c5f151a0 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.402+0000 7f53d4ff9640 1 -- 172.21.15.22:0/4273733022 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc011350 con 0x5566c5f151a0 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53d4ff9640 1 -- 172.21.15.22:0/4273733022 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53e8004f20 con 0x5566c5f151a0 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 10 client.17808 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 10 client.17808 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 20 client.17808 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f86940 0x5566c5f88d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 4 client.17808 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566c5e80750 con 0x5566c5f86940 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f86940 0x5566c5f88d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53d4ff9640 10 client.17808 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f0e4d640 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f86940 0x5566c5f88d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b6833e91eb0904c server_cookie=4b6c2416a3bb45f7 in_seq=0 out_seq=0 2024-03-09T00:01:26.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53d4ff9640 1 -- 172.21.15.22:0/4273733022 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566c5e80750 con 0x5566c5f86940 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53d4ff9640 10 client.17808 handle_command_reply: tid=0 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 1 client.17808 shutdown 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.403+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f151a0 msgr2=0x5566c5f24740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f151a0 0x5566c5f24740 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f53e802c850 tx=0x7f53e80058d0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 shutdown_connections 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566c5f86940 0x5566c5f88d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5566c5f1e6a0 0x5566c5f84dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566c5f15ac0 0x5566c5f24c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 --2- 172.21.15.22:0/4273733022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566c5f151a0 0x5566c5f24740 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53d4ff9640 0 client.17808 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 >> 172.21.15.22:0/4273733022 conn(0x5566c5f0f8d0 msgr2=0x5566c5f19ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 shutdown_connections 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 1 -- 172.21.15.22:0/4273733022 wait complete. 2024-03-09T00:01:26.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f6571740 20 client.17808 trim_cache size 0 max 16384 2024-03-09T00:01:26.407 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:01:26.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f53d80a2970 msgr2=0x7f53d80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f53d80a2970 0x7f53d80a4e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0119d0 tx=0x7f53dc009c00 comp rx=0 tx=0).stop 2024-03-09T00:01:26.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 msgr2=0x7f53ec19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec19e620 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f53e400eb10 tx=0x7f53e400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 shutdown_connections 2024-03-09T00:01:26.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f53d80a2970 0x7f53d80a4e30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53ec109550 0x7f53ec19e620 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53ec103e50 0x7f53ec19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 --2- 172.21.15.22:0/2664440718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ec103490 0x7f53ec199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 >> 172.21.15.22:0/2664440718 conn(0x7f53ec0fefa0 msgr2=0x7f53ec1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 shutdown_connections 2024-03-09T00:01:26.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.404+0000 7f53f38b2640 1 -- 172.21.15.22:0/2664440718 wait complete. 2024-03-09T00:01:26.411 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:01:26.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:01:26.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.599+0000 7f051227e640 1 -- 172.21.15.22:0/2708757905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c106380 msgr2=0x7f050c106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.599+0000 7f051227e640 1 --2- 172.21.15.22:0/2708757905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c106380 0x7f050c106760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009900 tx=0x7f04fc02de70 comp rx=0 tx=0).stop 2024-03-09T00:01:26.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.600+0000 7f051227e640 1 -- 172.21.15.22:0/2708757905 shutdown_connections 2024-03-09T00:01:26.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.600+0000 7f051227e640 1 --2- 172.21.15.22:0/2708757905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f050c100e00 0x7f050c10cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.600+0000 7f051227e640 1 --2- 172.21.15.22:0/2708757905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100440 0x7f050c1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.600+0000 7f051227e640 1 --2- 172.21.15.22:0/2708757905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c106380 0x7f050c106760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.600+0000 7f051227e640 1 -- 172.21.15.22:0/2708757905 >> 172.21.15.22:0/2708757905 conn(0x7f050c076fb0 msgr2=0x7f050c0773c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.602+0000 7f051227e640 1 -- 172.21.15.22:0/2708757905 shutdown_connections 2024-03-09T00:01:26.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.604+0000 7f051227e640 1 -- 172.21.15.22:0/2708757905 wait complete. 2024-03-09T00:01:26.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.606+0000 7f051227e640 1 Processor -- start 2024-03-09T00:01:26.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.606+0000 7f051227e640 1 -- start start 2024-03-09T00:01:26.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f051227e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c100440 0x7f050c10b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f051227e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100e00 0x7f050c10b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f051227e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f050c106380 0x7f050c10bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f051227e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c10eb90 con 0x7f050c106380 2024-03-09T00:01:26.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f051227e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c10ea10 con 0x7f050c100440 2024-03-09T00:01:26.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f051227e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c10ed10 con 0x7f050c100e00 2024-03-09T00:01:26.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f050b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100e00 0x7f050c10b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f050b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100e00 0x7f050c10b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:53594/0 (socket says 172.21.15.22:53594) 2024-03-09T00:01:26.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f050b7fe640 1 -- 172.21.15.22:0/2098936992 learned_addr learned my addr 172.21.15.22:0/2098936992 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.608+0000 7f051081b640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f050c106380 0x7f050c10bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.607+0000 7f050bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c100440 0x7f050c10b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.608+0000 7f050b7fe640 1 -- 172.21.15.22:0/2098936992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c100440 msgr2=0x7f050c10b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.608+0000 7f050b7fe640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c100440 0x7f050c10b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.608+0000 7f050b7fe640 1 -- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f050c106380 msgr2=0x7f050c10bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.608+0000 7f050b7fe640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f050c106380 0x7f050c10bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.608+0000 7f050b7fe640 1 -- 172.21.15.22:0/2098936992 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c1aa4c0 con 0x7f050c100e00 2024-03-09T00:01:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.608+0000 7f050b7fe640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100e00 0x7f050c10b960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f050000b450 tx=0x7f050000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.609+0000 7f05097fa640 1 -- 172.21.15.22:0/2098936992 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500011690 con 0x7f050c100e00 2024-03-09T00:01:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.609+0000 7f05097fa640 1 -- 172.21.15.22:0/2098936992 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0500026020 con 0x7f050c100e00 2024-03-09T00:01:26.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.609+0000 7f05097fa640 1 -- 172.21.15.22:0/2098936992 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500010dc0 con 0x7f050c100e00 2024-03-09T00:01:26.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.610+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c06ad70 con 0x7f050c100e00 2024-03-09T00:01:26.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.610+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c06b1e0 con 0x7f050c100e00 2024-03-09T00:01:26.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.614+0000 7f0514e58740 1 Processor -- start 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.616+0000 7f0514e58740 1 -- start start 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.616+0000 7f0514e58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5a9fb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.616+0000 7f0514e58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa04160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.616+0000 7f0514e58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.616+0000 7f0514e58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5a9c2360 con 0x561a5aa046a0 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.616+0000 7f0514e58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5a95eec0 con 0x561a5a9fb1a0 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.616+0000 7f0514e58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5a907d30 con 0x561a5a9fbac0 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f050bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa04160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5a9fb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5a9fb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51396/0 (socket says 172.21.15.22:51396) 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 -- 172.21.15.22:0/2140198940 learned_addr learned my addr 172.21.15.22:0/2140198940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f05097fa640 1 -- 172.21.15.22:0/2098936992 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f050000f050 con 0x7f050c100e00 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 -- 172.21.15.22:0/2140198940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 msgr2=0x561a5aa04160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 --2- 172.21.15.22:0/2140198940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa04160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 -- 172.21.15.22:0/2140198940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 msgr2=0x561a5aa06a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 --2- 172.21.15.22:0/2140198940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.617+0000 7f051081b640 1 -- 172.21.15.22:0/2140198940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a5a906bc0 con 0x561a5a9fb1a0 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.618+0000 7f051081b640 1 --2- 172.21.15.22:0/2140198940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5a9fb580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0504009f80 tx=0x7f0504010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8ad08a66d7ec03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.618+0000 7f05097fa640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04f40a29f0 0x7f04f40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.618+0000 7f04f27fc640 1 -- 172.21.15.22:0/2140198940 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504003b70 con 0x561a5a9fb1a0 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.618+0000 7f04f27fc640 1 -- 172.21.15.22:0/2140198940 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0504003d10 con 0x561a5a9fb1a0 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.618+0000 7f050bfff640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04f40a29f0 0x7f04f40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.618+0000 7f04f27fc640 1 -- 172.21.15.22:0/2140198940 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504033b00 con 0x561a5a9fb1a0 2024-03-09T00:01:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.619+0000 7f05097fa640 1 -- 172.21.15.22:0/2098936992 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f05000dfaf0 con 0x7f050c100e00 2024-03-09T00:01:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.619+0000 7f0514e58740 1 -- 172.21.15.22:0/2140198940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 msgr2=0x561a5a9fb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.619+0000 7f0514e58740 1 --2- 172.21.15.22:0/2140198940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5a9fb580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0504009f80 tx=0x7f0504010860 comp rx=0 tx=0).stop 2024-03-09T00:01:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.619+0000 7f050bfff640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04f40a29f0 0x7f04f40a4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04fc006010 tx=0x7f04fc005b50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.623+0000 7f0514e58740 1 -- 172.21.15.22:0/2140198940 shutdown_connections 2024-03-09T00:01:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.623+0000 7f0514e58740 1 --2- 172.21.15.22:0/2140198940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa06a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.623+0000 7f0514e58740 1 --2- 172.21.15.22:0/2140198940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa04160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.623+0000 7f0514e58740 1 --2- 172.21.15.22:0/2140198940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5a9fb580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.623+0000 7f0514e58740 1 -- 172.21.15.22:0/2140198940 >> 172.21.15.22:0/2140198940 conn(0x561a5a9f58d0 msgr2=0x561a5a9f5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.623+0000 7f0514e58740 1 -- 172.21.15.22:0/2140198940 shutdown_connections 2024-03-09T00:01:26.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.623+0000 7f0514e58740 1 -- 172.21.15.22:0/2140198940 wait complete. 2024-03-09T00:01:26.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 -- start start 2024-03-09T00:01:26.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 Processor -- start 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa02170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5a8fd4a0 con 0x561a5aa046a0 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5a95af60 con 0x561a5a9fb1a0 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5aa07490 con 0x561a5a9fbac0 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.624+0000 7f0514e58740 10 client.0 _ll_register_callbacks cb 0x561a5a958d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa02170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f050bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa02170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55440/0 (socket says 172.21.15.22:55440) 2024-03-09T00:01:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 -- 172.21.15.22:0/2906302635 learned_addr learned my addr 172.21.15.22:0/2906302635 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 -- 172.21.15.22:0/2906302635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 msgr2=0x561a5aa01c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 -- 172.21.15.22:0/2906302635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 msgr2=0x561a5aa016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.625+0000 7f051081b640 1 -- 172.21.15.22:0/2906302635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a5a6733f0 con 0x561a5aa046a0 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f051081b640 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa02170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0504011010 tx=0x7f0504004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d387cb3ba2642c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f04eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f04eb7fe640 1 -- 172.21.15.22:0/2906302635 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050404c060 con 0x561a5aa046a0 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f04eb7fe640 1 -- 172.21.15.22:0/2906302635 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0504033a90 con 0x561a5aa046a0 2024-03-09T00:01:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0504003d10 con 0x561a5aa046a0 2024-03-09T00:01:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f04eb7fe640 1 -- 172.21.15.22:0/2906302635 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0504004e60 con 0x561a5aa046a0 2024-03-09T00:01:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f0514e58740 10 client.17817 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.626+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561a5a73d7d0 con 0x561a5aa046a0 2024-03-09T00:01:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f04eb7fe640 1 -- 172.21.15.22:0/2906302635 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0504003d10 con 0x561a5aa046a0 2024-03-09T00:01:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f04eb7fe640 1 -- 172.21.15.22:0/2906302635 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05040050b0 con 0x561a5aa046a0 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f0514e58740 10 client.17817 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f0514e58740 10 client.17817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f0514e58740 20 client.17817 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f0514e58740 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa147e0 0x561a5aa16bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f0514e58740 4 client.17817 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.627+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a5a96c400 con 0x561a5aa147e0 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.628+0000 7f050bfff640 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa147e0 0x561a5aa16bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.629+0000 7f050bfff640 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa147e0 0x561a5aa16bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f434ce279ce8fe1 server_cookie=e94b956b1d52b35e in_seq=0 out_seq=0 2024-03-09T00:01:26.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.629+0000 7f04eb7fe640 10 client.17817 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f04eb7fe640 1 -- 172.21.15.22:0/2906302635 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a5a96c400 con 0x561a5aa147e0 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f04eb7fe640 10 client.17817 handle_command_reply: tid=0 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 client.17817 shutdown 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 msgr2=0x561a5aa02170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa02170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0504011010 tx=0x7f0504004c70 comp rx=0 tx=0).stop 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 shutdown_connections 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa147e0 0x561a5aa16bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa02170 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa01c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f04eb7fe640 0 client.17817 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 --2- 172.21.15.22:0/2906302635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.630+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 >> 172.21.15.22:0/2906302635 conn(0x561a5a9f58d0 msgr2=0x561a5a9f7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.631+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 shutdown_connections 2024-03-09T00:01:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.631+0000 7f0514e58740 1 -- 172.21.15.22:0/2906302635 wait complete. 2024-03-09T00:01:26.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.631+0000 7f0514e58740 20 client.17817 trim_cache size 0 max 16384 2024-03-09T00:01:26.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.651+0000 7f0514e58740 1 Processor -- start 2024-03-09T00:01:26.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f0514e58740 1 -- start start 2024-03-09T00:01:26.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f0514e58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5a9fb1a0 0x561a5aa03e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f0514e58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fbac0 0x561a5a9fca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f0514e58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5aa046a0 0x561a5a9fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f0514e58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c1aea40 con 0x561a5a9fb1a0 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f0514e58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c1ac5e0 con 0x561a5a9fbac0 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f0514e58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c1aa6d0 con 0x561a5aa046a0 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f050bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5a9fb1a0 0x561a5aa03e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f050b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fbac0 0x561a5a9fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f050bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5a9fb1a0 0x561a5aa03e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55450/0 (socket says 172.21.15.22:55450) 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f050b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fbac0 0x561a5a9fca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51410/0 (socket says 172.21.15.22:51410) 2024-03-09T00:01:26.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.652+0000 7f050bfff640 1 -- 172.21.15.22:0/3920798115 learned_addr learned my addr 172.21.15.22:0/3920798115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f050bfff640 1 -- 172.21.15.22:0/3920798115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5aa046a0 msgr2=0x561a5a9fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f050bfff640 1 --2- 172.21.15.22:0/3920798115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5aa046a0 0x561a5a9fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f050bfff640 1 -- 172.21.15.22:0/3920798115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fbac0 msgr2=0x561a5a9fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f050bfff640 1 --2- 172.21.15.22:0/3920798115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fbac0 0x561a5a9fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f050bfff640 1 -- 172.21.15.22:0/3920798115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a5a9fab70 con 0x561a5a9fb1a0 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f050bfff640 1 --2- 172.21.15.22:0/3920798115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5a9fb1a0 0x561a5aa03e20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009900 tx=0x7f04fc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94532fdcf18c5092 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f04f1ffb640 1 -- 172.21.15.22:0/3920798115 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc02e810 con 0x561a5a9fb1a0 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f04f1ffb640 1 -- 172.21.15.22:0/3920798115 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04fc058070 con 0x561a5a9fb1a0 2024-03-09T00:01:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.653+0000 7f04f1ffb640 1 -- 172.21.15.22:0/3920798115 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0035f0 con 0x561a5a9fb1a0 2024-03-09T00:01:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 -- 172.21.15.22:0/3920798115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5a9fb1a0 msgr2=0x561a5aa03e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 --2- 172.21.15.22:0/3920798115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5a9fb1a0 0x561a5aa03e20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009900 tx=0x7f04fc041000 comp rx=0 tx=0).stop 2024-03-09T00:01:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 -- 172.21.15.22:0/3920798115 shutdown_connections 2024-03-09T00:01:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 --2- 172.21.15.22:0/3920798115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5aa046a0 0x561a5a9fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 --2- 172.21.15.22:0/3920798115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fbac0 0x561a5a9fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 --2- 172.21.15.22:0/3920798115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5a9fb1a0 0x561a5aa03e20 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 -- 172.21.15.22:0/3920798115 >> 172.21.15.22:0/3920798115 conn(0x561a5a9f58d0 msgr2=0x561a5a9fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.655+0000 7f0514e58740 1 -- 172.21.15.22:0/3920798115 shutdown_connections 2024-03-09T00:01:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.656+0000 7f0514e58740 1 -- 172.21.15.22:0/3920798115 wait complete. 2024-03-09T00:01:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.656+0000 7f0514e58740 1 -- start start 2024-03-09T00:01:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.656+0000 7f0514e58740 1 Processor -- start 2024-03-09T00:01:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f0514e58740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa0a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f0514e58740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa0aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f0514e58740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa6a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f0514e58740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c1aaca0 con 0x561a5aa046a0 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f0514e58740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c1abf80 con 0x561a5a9fb1a0 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f0514e58740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c1ab2c0 con 0x561a5a9fbac0 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f0514e58740 10 client.0 _ll_register_callbacks cb 0x7f0504009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa0a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa0a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51426/0 (socket says 172.21.15.22:51426) 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 -- 172.21.15.22:0/3606565586 learned_addr learned my addr 172.21.15.22:0/3606565586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f051081b640 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa6a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050b7fe640 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa0aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 -- 172.21.15.22:0/3606565586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 msgr2=0x561a5aa0aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa0aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 -- 172.21.15.22:0/3606565586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 msgr2=0x561a5aa6a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa6a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.657+0000 7f050bfff640 1 -- 172.21.15.22:0/3606565586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a5aa6ae60 con 0x561a5a9fb1a0 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.658+0000 7f050bfff640 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa0a510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04fc053380 tx=0x7f04fc0533b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857fb8447238a5c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.658+0000 7f04eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:01:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.658+0000 7f04eb7fe640 1 -- 172.21.15.22:0/3606565586 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc02ec20 con 0x561a5a9fb1a0 2024-03-09T00:01:26.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.658+0000 7f04eb7fe640 1 -- 172.21.15.22:0/3606565586 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f04fc02edc0 con 0x561a5a9fb1a0 2024-03-09T00:01:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.658+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04fc058070 con 0x561a5a9fb1a0 2024-03-09T00:01:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.659+0000 7f04eb7fe640 1 -- 172.21.15.22:0/3606565586 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04fc0365a0 con 0x561a5a9fb1a0 2024-03-09T00:01:26.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.660+0000 7f0514e58740 10 client.27209 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:26.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.660+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561a5aa6bfd0 con 0x561a5a9fb1a0 2024-03-09T00:01:26.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f04eb7fe640 1 -- 172.21.15.22:0/3606565586 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04fc058070 con 0x561a5a9fb1a0 2024-03-09T00:01:26.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f04eb7fe640 1 -- 172.21.15.22:0/3606565586 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04fc003a60 con 0x561a5a9fb1a0 2024-03-09T00:01:26.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f0514e58740 10 client.27209 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:26.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f0514e58740 10 client.27209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:01:26.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f0514e58740 20 client.27209 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f0514e58740 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa6c6b0 0x561a5aa6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f0514e58740 4 client.27209 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a5a966750 con 0x561a5aa6c6b0 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.661+0000 7f050b7fe640 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa6c6b0 0x561a5aa6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.662+0000 7f04eb7fe640 10 client.27209 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.662+0000 7f050b7fe640 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa6c6b0 0x561a5aa6ead0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2757fca165f08d1 server_cookie=7d2c37874f342c4f in_seq=0 out_seq=0 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.662+0000 7f04eb7fe640 1 -- 172.21.15.22:0/3606565586 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a5a966750 con 0x561a5aa6c6b0 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.662+0000 7f04eb7fe640 10 client.27209 handle_command_reply: tid=0 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.662+0000 7f0514e58740 1 client.27209 shutdown 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 msgr2=0x561a5aa0a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa0a510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04fc053380 tx=0x7f04fc0533b0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 shutdown_connections 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561a5aa6c6b0 0x561a5aa6ead0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a5aa046a0 0x561a5aa6a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561a5a9fbac0 0x561a5aa0aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 --2- 172.21.15.22:0/3606565586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a5a9fb1a0 0x561a5aa0a510 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04fc053380 tx=0x7f04fc0533b0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 >> 172.21.15.22:0/3606565586 conn(0x561a5a9f58d0 msgr2=0x561a5a9fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f04eb7fe640 0 client.27209 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 shutdown_connections 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.663+0000 7f0514e58740 1 -- 172.21.15.22:0/3606565586 wait complete. 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.664+0000 7f0514e58740 20 client.27209 trim_cache size 0 max 16384 2024-03-09T00:01:26.662 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:01:26.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.665+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04f40a29f0 msgr2=0x7f04f40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.665+0000 7f051227e640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04f40a29f0 0x7f04f40a4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04fc006010 tx=0x7f04fc005b50 comp rx=0 tx=0).stop 2024-03-09T00:01:26.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.665+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100e00 msgr2=0x7f050c10b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.665+0000 7f051227e640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100e00 0x7f050c10b960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f050000b450 tx=0x7f050000b910 comp rx=0 tx=0).stop 2024-03-09T00:01:26.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.666+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 shutdown_connections 2024-03-09T00:01:26.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.666+0000 7f051227e640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f04f40a29f0 0x7f04f40a4eb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.666+0000 7f051227e640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f050c106380 0x7f050c10bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.666+0000 7f051227e640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f050c100e00 0x7f050c10b960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.666+0000 7f051227e640 1 --2- 172.21.15.22:0/2098936992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c100440 0x7f050c10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.666+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 >> 172.21.15.22:0/2098936992 conn(0x7f050c076fb0 msgr2=0x7f050c1110c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.666+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 shutdown_connections 2024-03-09T00:01:26.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.668+0000 7f051227e640 1 -- 172.21.15.22:0/2098936992 wait complete. 2024-03-09T00:01:26.678 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:01:26.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:01:26.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 721321 10.92 MB/sec execute 214 sec latency 20.559 ms 2024-03-09T00:01:26.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 -- 172.21.15.22:0/2661435114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c0ff9f0 msgr2=0x7f2c2c0ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 --2- 172.21.15.22:0/2661435114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c0ff9f0 0x7f2c2c0ffdd0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009900 tx=0x7f2c1c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:26.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 -- 172.21.15.22:0/2661435114 shutdown_connections 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 --2- 172.21.15.22:0/2661435114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c2c106660 0x7f2c2c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 --2- 172.21.15.22:0/2661435114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c2c1003a0 0x7f2c2c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 --2- 172.21.15.22:0/2661435114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c0ff9f0 0x7f2c2c0ffdd0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 -- 172.21.15.22:0/2661435114 >> 172.21.15.22:0/2661435114 conn(0x7f2c2c076700 msgr2=0x7f2c2c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 -- 172.21.15.22:0/2661435114 shutdown_connections 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 -- 172.21.15.22:0/2661435114 wait complete. 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 Processor -- start 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 -- start start 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.803+0000 7f2c32643640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c2c0ff9f0 0x7f2c2c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c32643640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c2c106660 0x7f2c2c19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c32643640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c197a20 0x7f2c2c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c32643640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1116c0 con 0x7f2c2c197a20 2024-03-09T00:01:26.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c32643640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c111540 con 0x7f2c2c0ff9f0 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c32643640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c111840 con 0x7f2c2c106660 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c197a20 0x7f2c2c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c197a20 0x7f2c2c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55480/0 (socket says 172.21.15.22:55480) 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 -- 172.21.15.22:0/1033079566 learned_addr learned my addr 172.21.15.22:0/1033079566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c2c0ff9f0 0x7f2c2c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c2c106660 0x7f2c2c19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 -- 172.21.15.22:0/1033079566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c2c106660 msgr2=0x7f2c2c19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c2c106660 0x7f2c2c19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 -- 172.21.15.22:0/1033079566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c2c0ff9f0 msgr2=0x7f2c2c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c2c0ff9f0 0x7f2c2c19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 -- 172.21.15.22:0/1033079566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c198540 con 0x7f2c2c197a20 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c30be0640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c197a20 0x7f2c2c197ed0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009fd0 tx=0x7f2c2400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c297fa640 1 -- 172.21.15.22:0/1033079566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24017070 con 0x7f2c2c197a20 2024-03-09T00:01:26.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c297fa640 1 -- 172.21.15.22:0/1033079566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c24003e50 con 0x7f2c2c197a20 2024-03-09T00:01:26.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c198830 con 0x7f2c2c197a20 2024-03-09T00:01:26.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c297fa640 1 -- 172.21.15.22:0/1033079566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c240106c0 con 0x7f2c2c197a20 2024-03-09T00:01:26.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.804+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c1a4a30 con 0x7f2c2c197a20 2024-03-09T00:01:26.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c297fa640 1 -- 172.21.15.22:0/1033079566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2c24003970 con 0x7f2c2c197a20 2024-03-09T00:01:26.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 Processor -- start 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 -- start start 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b71ac0 0x55eda3b7a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b7ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda3b38360 con 0x55eda3b71ac0 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda3ad4ec0 con 0x55eda3b711a0 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c3521c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda3a7dd30 con 0x55eda3b7a6a0 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.807+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b71ac0 0x55eda3b7a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b71ac0 0x55eda3b7a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55486/0 (socket says 172.21.15.22:55486) 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 -- 172.21.15.22:0/3573662740 learned_addr learned my addr 172.21.15.22:0/3573662740 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b7ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c297fa640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c140a2970 0x7f2c140a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 -- 172.21.15.22:0/3573662740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 msgr2=0x55eda3b7ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b7ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 -- 172.21.15.22:0/3573662740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 msgr2=0x55eda3b71580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b71580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 -- 172.21.15.22:0/3573662740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda3a7cbc0 con 0x55eda3b71ac0 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c30be0640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c140a2970 0x7f2c140a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c297fa640 1 -- 172.21.15.22:0/1033079566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2c240cef00 con 0x7f2c2c197a20 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b71ac0 0x55eda3b7a160 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c02c170 tx=0x7f2c1c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d6d0615ebc82d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c127fc640 1 -- 172.21.15.22:0/3573662740 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c042070 con 0x55eda3b71ac0 2024-03-09T00:01:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c127fc640 1 -- 172.21.15.22:0/3573662740 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c1c02ca80 con 0x55eda3b71ac0 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c30be0640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c140a2970 0x7f2c140a4e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c24097be0 tx=0x7f2c2406def0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.808+0000 7f2c127fc640 1 -- 172.21.15.22:0/3573662740 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c004ec0 con 0x55eda3b71ac0 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 -- 172.21.15.22:0/3573662740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b71ac0 msgr2=0x55eda3b7a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b71ac0 0x55eda3b7a160 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c02c170 tx=0x7f2c1c005e80 comp rx=0 tx=0).stop 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 -- 172.21.15.22:0/3573662740 shutdown_connections 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b7ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b71ac0 0x55eda3b7a160 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 --2- 172.21.15.22:0/3573662740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b71580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 -- 172.21.15.22:0/3573662740 >> 172.21.15.22:0/3573662740 conn(0x55eda3b6b8d0 msgr2=0x55eda3b6bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 -- 172.21.15.22:0/3573662740 shutdown_connections 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 -- 172.21.15.22:0/3573662740 wait complete. 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.809+0000 7f2c3521c740 1 -- start start 2024-03-09T00:01:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 1 Processor -- start 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b71ac0 0x55eda3b77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda3a734a0 con 0x55eda3b7a6a0 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda3ad0f60 con 0x55eda3b711a0 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda3b7d430 con 0x55eda3b71ac0 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c3521c740 10 client.0 _ll_register_callbacks cb 0x55eda3aced60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b780b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55490/0 (socket says 172.21.15.22:55490) 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2258431180 learned_addr learned my addr 172.21.15.22:0/2258431180 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b71ac0 0x55eda3b77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b71ac0 msgr2=0x55eda3b77b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b71ac0 0x55eda3b77b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2258431180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 msgr2=0x55eda3b77630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b77630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2258431180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eda37e93f0 con 0x55eda3b7a6a0 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b780b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c069600 tx=0x7f2c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21571a78b5b9f5d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.810+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2258431180 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000ee80 con 0x55eda3b7a6a0 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2258431180 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2c20004030 con 0x55eda3b7a6a0 2024-03-09T00:01:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c1c02ca80 con 0x55eda3b7a6a0 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2258431180 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c20004230 con 0x55eda3b7a6a0 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 10 client.17835 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55eda38b37d0 con 0x55eda3b7a6a0 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2258431180 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c1c02ca80 con 0x55eda3b7a6a0 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2258431180 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c20010430 con 0x55eda3b7a6a0 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 10 client.17835 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 20 client.17835 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3b8a800 0x55eda3b8cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 4 client.17835 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eda3ae2400 con 0x55eda3b8a800 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.811+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3b8a800 0x55eda3b8cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.812+0000 7f2c0b7fe640 10 client.17835 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.812+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3b8a800 0x55eda3b8cbf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7e6500163932f2 server_cookie=c65267dd7efbba72 in_seq=0 out_seq=0 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.812+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2258431180 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eda3ae2400 con 0x55eda3b8a800 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.812+0000 7f2c0b7fe640 10 client.17835 handle_command_reply: tid=0 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 client.17835 shutdown 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 msgr2=0x55eda3b780b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b780b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c069600 tx=0x7f2c200046c0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 shutdown_connections 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3b8a800 0x55eda3b8cbf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b780b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b71ac0 0x55eda3b77b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2258431180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b711a0 0x55eda3b77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 >> 172.21.15.22:0/2258431180 conn(0x55eda3b6b8d0 msgr2=0x55eda3b6d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c0b7fe640 0 client.17835 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 shutdown_connections 2024-03-09T00:01:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 1 -- 172.21.15.22:0/2258431180 wait complete. 2024-03-09T00:01:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.813+0000 7f2c3521c740 20 client.17835 trim_cache size 0 max 16384 2024-03-09T00:01:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 Processor -- start 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 -- start start 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b711a0 0x55eda3b79d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b76610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b74890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1b0c00 con 0x55eda3b711a0 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1ae7a0 con 0x55eda3b71ac0 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c3521c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1ac860 con 0x55eda3b7a6a0 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b711a0 0x55eda3b79d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b711a0 0x55eda3b79d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55492/0 (socket says 172.21.15.22:55492) 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/4049246094 learned_addr learned my addr 172.21.15.22:0/4049246094 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.825+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b74890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b76610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/4049246094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 msgr2=0x55eda3b74890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b74890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/4049246094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 msgr2=0x55eda3b76610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b76610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/4049246094 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda3b7ca90 con 0x55eda3b711a0 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b711a0 0x55eda3b79d60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x55eda3b7a470 tx=0x7f2c2000b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bdfcc2ff4593a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c11ffb640 1 -- 172.21.15.22:0/4049246094 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200043d0 con 0x55eda3b711a0 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c11ffb640 1 -- 172.21.15.22:0/4049246094 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c20016030 con 0x55eda3b711a0 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c11ffb640 1 -- 172.21.15.22:0/4049246094 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20004c60 con 0x55eda3b711a0 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c3521c740 1 -- 172.21.15.22:0/4049246094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b711a0 msgr2=0x55eda3b79d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c3521c740 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b711a0 0x55eda3b79d60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x55eda3b7a470 tx=0x7f2c2000b510 comp rx=0 tx=0).stop 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c3521c740 1 -- 172.21.15.22:0/4049246094 shutdown_connections 2024-03-09T00:01:26.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c3521c740 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b7a6a0 0x55eda3b74890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c3521c740 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b76610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c3521c740 1 --2- 172.21.15.22:0/4049246094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b711a0 0x55eda3b79d60 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.826+0000 7f2c3521c740 1 -- 172.21.15.22:0/4049246094 >> 172.21.15.22:0/4049246094 conn(0x55eda3b6b8d0 msgr2=0x55eda3b75f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 -- 172.21.15.22:0/4049246094 shutdown_connections 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 -- 172.21.15.22:0/4049246094 wait complete. 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 -- start start 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 Processor -- start 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b711a0 0x55eda3b81a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b7fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1ace60 con 0x55eda3b7a6a0 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1ae140 con 0x55eda3b71ac0 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1ad480 con 0x55eda3b711a0 2024-03-09T00:01:26.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c3521c740 10 client.0 _ll_register_callbacks cb 0x7f2c1c005dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55500/0 (socket says 172.21.15.22:55500) 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2656509934 learned_addr learned my addr 172.21.15.22:0/2656509934 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.827+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b711a0 0x55eda3b81a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b7fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b711a0 msgr2=0x55eda3b81a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b711a0 0x55eda3b81a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2656509934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 msgr2=0x55eda3b7fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b7fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c2b7fe640 1 -- 172.21.15.22:0/2656509934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eda3b80bf0 con 0x55eda3b7a6a0 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c2b7fe640 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b803c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2c20010740 tx=0x7f2c20004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4c2bd89e7950a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2656509934 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20044060 con 0x55eda3b7a6a0 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2656509934 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2c200045d0 con 0x55eda3b7a6a0 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c20016030 con 0x55eda3b7a6a0 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2656509934 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c20004e10 con 0x55eda3b7a6a0 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c3521c740 10 client.17841 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55eda3be1e10 con 0x55eda3b7a6a0 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.828+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2656509934 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c20016030 con 0x55eda3b7a6a0 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2656509934 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2c2002f450 con 0x55eda3b7a6a0 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c3521c740 10 client.17841 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c3521c740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c3521c740 20 client.17841 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3be2460 0x55eda3be4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c3521c740 4 client.17841 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:01:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eda3adc750 con 0x55eda3be2460 2024-03-09T00:01:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3be2460 0x55eda3be4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c2bfff640 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3be2460 0x55eda3be4880 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3145dbb6fda0ddbd server_cookie=141ef994aaab78b7 in_seq=0 out_seq=0 2024-03-09T00:01:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.829+0000 7f2c0b7fe640 10 client.17841 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c0b7fe640 1 -- 172.21.15.22:0/2656509934 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eda3adc750 con 0x55eda3be2460 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c0b7fe640 10 client.17841 handle_command_reply: tid=0 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 client.17841 shutdown 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 msgr2=0x55eda3b803c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b803c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2c20010740 tx=0x7f2c20004840 comp rx=0 tx=0).stop 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 shutdown_connections 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eda3be2460 0x55eda3be4880 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eda3b7a6a0 0x55eda3b803c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eda3b71ac0 0x55eda3b7fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c0b7fe640 0 client.17841 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 --2- 172.21.15.22:0/2656509934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eda3b711a0 0x55eda3b81a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 >> 172.21.15.22:0/2656509934 conn(0x55eda3b6b8d0 msgr2=0x55eda3b76790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 shutdown_connections 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 1 -- 172.21.15.22:0/2656509934 wait complete. 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.832+0000 7f2c3521c740 20 client.17841 trim_cache size 0 max 16384 2024-03-09T00:01:26.833 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:01:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c140a2970 msgr2=0x7f2c140a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c140a2970 0x7f2c140a4e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c24097be0 tx=0x7f2c2406def0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c197a20 msgr2=0x7f2c2c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c197a20 0x7f2c2c197ed0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009fd0 tx=0x7f2c2400b260 comp rx=0 tx=0).stop 2024-03-09T00:01:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 shutdown_connections 2024-03-09T00:01:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2c140a2970 0x7f2c140a4e30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c2c197a20 0x7f2c2c197ed0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c2c106660 0x7f2c2c19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 --2- 172.21.15.22:0/1033079566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c2c0ff9f0 0x7f2c2c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 >> 172.21.15.22:0/1033079566 conn(0x7f2c2c076700 msgr2=0x7f2c2c104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 shutdown_connections 2024-03-09T00:01:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:26.833+0000 7f2c32643640 1 -- 172.21.15.22:0/1033079566 wait complete. 2024-03-09T00:01:27.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:27 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3544860161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:01:27.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:27 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3544860161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:01:27.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:27 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3544860161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:01:27.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 723446 10.90 MB/sec execute 215 sec latency 19.230 ms 2024-03-09T00:01:27.840 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:01:27.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:01:27.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:01:27.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 -- 172.21.15.22:0/3280500590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac0690f0 msgr2=0x7fe2ac105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 --2- 172.21.15.22:0/3280500590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac0690f0 0x7fe2ac105ec0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009900 tx=0x7fe2a002bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:27.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 -- 172.21.15.22:0/3280500590 shutdown_connections 2024-03-09T00:01:27.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 --2- 172.21.15.22:0/3280500590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ac1065a0 0x7fe2ac10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 --2- 172.21.15.22:0/3280500590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac0690f0 0x7fe2ac105ec0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 --2- 172.21.15.22:0/3280500590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2ac068740 0x7fe2ac068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 -- 172.21.15.22:0/3280500590 >> 172.21.15.22:0/3280500590 conn(0x7fe2ac0772f0 msgr2=0x7fe2ac077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:27.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 -- 172.21.15.22:0/3280500590 shutdown_connections 2024-03-09T00:01:27.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.958+0000 7fe2b186f640 1 -- 172.21.15.22:0/3280500590 wait complete. 2024-03-09T00:01:27.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 Processor -- start 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 -- start start 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2ac068740 0x7fe2ac19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac1065a0 0x7fe2ac19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ac1979b0 0x7fe2ac197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac111630 con 0x7fe2ac1065a0 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1114b0 con 0x7fe2ac068740 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.959+0000 7fe2b186f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1117b0 con 0x7fe2ac1979b0 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac1065a0 0x7fe2ac19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac1065a0 0x7fe2ac19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55518/0 (socket says 172.21.15.22:55518) 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/527261598 learned_addr learned my addr 172.21.15.22:0/527261598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2ac068740 0x7fe2ac19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ac1979b0 0x7fe2ac197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/527261598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ac1979b0 msgr2=0x7fe2ac197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ac1979b0 0x7fe2ac197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/527261598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2ac068740 msgr2=0x7fe2ac19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2ac068740 0x7fe2ac19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/527261598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac1984d0 con 0x7fe2ac1065a0 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac1065a0 0x7fe2ac19dd40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0036040 tx=0x7fe2a002cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2b086d640 1 -- 172.21.15.22:0/527261598 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0041070 con 0x7fe2ac1065a0 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2b086d640 1 -- 172.21.15.22:0/527261598 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2a0004450 con 0x7fe2ac1065a0 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac198760 con 0x7fe2ac1065a0 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2b086d640 1 -- 172.21.15.22:0/527261598 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0003dc0 con 0x7fe2ac1065a0 2024-03-09T00:01:27.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.960+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac1a4950 con 0x7fe2ac1065a0 2024-03-09T00:01:27.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.962+0000 7fe2b086d640 1 -- 172.21.15.22:0/527261598 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe2a0044070 con 0x7fe2ac1065a0 2024-03-09T00:01:27.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 Processor -- start 2024-03-09T00:01:27.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 -- start start 2024-03-09T00:01:27.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b086d640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2940a2970 0x7fe2940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b01e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2940a2970 0x7fe2940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b11a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b086d640 1 -- 172.21.15.22:0/527261598 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe2a00f0cb0 con 0x7fe2ac1065a0 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe8a65b40 con 0x55bbe8b023d0 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe8a0ed80 con 0x55bbe8b01ab0 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2b444d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe8a61be0 con 0x55bbe8b0f690 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2aaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b01e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2aa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2aaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b01e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51496/0 (socket says 172.21.15.22:51496) 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2aaffd640 1 -- 172.21.15.22:0/3251741316 learned_addr learned my addr 172.21.15.22:0/3251741316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:27.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.963+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b11a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/3251741316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 msgr2=0x55bbe8b11a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b11a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/3251741316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 msgr2=0x55bbe8b01e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b01e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/3251741316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbe88427e0 con 0x55bbe8b023d0 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b11a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2940a2970 0x7fe2940a4e30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005fd0 tx=0x7fe2a4011b40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0f150 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00ee0b0 tx=0x7fe2a00bcf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245b8cd9db8aabf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe291ffb640 1 -- 172.21.15.22:0/3251741316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a004b040 con 0x55bbe8b023d0 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe291ffb640 1 -- 172.21.15.22:0/3251741316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2a00c1110 con 0x55bbe8b023d0 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.964+0000 7fe291ffb640 1 -- 172.21.15.22:0/3251741316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b96e0 con 0x55bbe8b023d0 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- 172.21.15.22:0/3251741316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 msgr2=0x55bbe8b0f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0f150 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00ee0b0 tx=0x7fe2a00bcf00 comp rx=0 tx=0).stop 2024-03-09T00:01:27.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- 172.21.15.22:0/3251741316 shutdown_connections 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b11a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0f150 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 --2- 172.21.15.22:0/3251741316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b01e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- 172.21.15.22:0/3251741316 >> 172.21.15.22:0/3251741316 conn(0x55bbe8afc4b0 msgr2=0x55bbe8afc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- 172.21.15.22:0/3251741316 shutdown_connections 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- 172.21.15.22:0/3251741316 wait complete. 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- start start 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 Processor -- start 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe8b12210 con 0x55bbe8b023d0 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe8a221c0 con 0x55bbe8b01ab0 2024-03-09T00:01:27.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.965+0000 7fe2b444d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbe8b12390 con 0x55bbe8b0f690 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2b444d740 10 client.0 _ll_register_callbacks cb 0x55bbe8a117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55544/0 (socket says 172.21.15.22:55544) 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/461769413 learned_addr learned my addr 172.21.15.22:0/461769413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 msgr2=0x55bbe8b0cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/461769413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 msgr2=0x55bbe8b0f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 -- 172.21.15.22:0/461769413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbe88a2f70 con 0x55bbe8b023d0 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0c4e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00b94d0 tx=0x7fe2a0004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ec808be4261ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe28affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:27.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe28affd640 1 -- 172.21.15.22:0/461769413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a004b030 con 0x55bbe8b023d0 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe28affd640 1 -- 172.21.15.22:0/461769413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe2a00b96e0 con 0x55bbe8b023d0 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2a00c1110 con 0x55bbe8b023d0 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe28affd640 1 -- 172.21.15.22:0/461769413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a00c52d0 con 0x55bbe8b023d0 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2b444d740 10 client.17853 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bbe873a170 con 0x55bbe8b023d0 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.966+0000 7fe28affd640 1 -- 172.21.15.22:0/461769413 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2a00c1110 con 0x55bbe8b023d0 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe28affd640 1 -- 172.21.15.22:0/461769413 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe2a00bf070 con 0x55bbe8b023d0 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2b444d740 10 client.17853 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2b444d740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2b444d740 20 client.17853 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2b444d740 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b1f3c0 0x55bbe8b217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2b444d740 4 client.17853 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbe8a3b960 con 0x55bbe8b1f3c0 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b1f3c0 0x55bbe8b217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe28affd640 10 client.17853 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.967+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b1f3c0 0x55bbe8b217b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b66108f2dfa3b782 server_cookie=f6a8c5e9aa13c2ca in_seq=0 out_seq=0 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe28affd640 1 -- 172.21.15.22:0/461769413 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bbe8a3b960 con 0x55bbe8b1f3c0 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe28affd640 10 client.17853 handle_command_reply: tid=0 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 client.17853 shutdown 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 msgr2=0x55bbe8b0c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0c4e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00b94d0 tx=0x7fe2a0004ee0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 shutdown_connections 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b1f3c0 0x55bbe8b217b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0c4e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 --2- 172.21.15.22:0/461769413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe28affd640 0 client.17853 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 >> 172.21.15.22:0/461769413 conn(0x55bbe8afc4b0 msgr2=0x55bbe8afe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 shutdown_connections 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.968+0000 7fe2b444d740 1 -- 172.21.15.22:0/461769413 wait complete. 2024-03-09T00:01:27.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.969+0000 7fe2b444d740 20 client.17853 trim_cache size 0 max 16384 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 Processor -- start 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 -- start start 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1aea20 con 0x55bbe8b01ab0 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1ac5c0 con 0x55bbe8b0f690 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.984+0000 7fe2b444d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1aa680 con 0x55bbe8b023d0 2024-03-09T00:01:27.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55560/0 (socket says 172.21.15.22:55560) 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 -- 172.21.15.22:0/1097797637 learned_addr learned my addr 172.21.15.22:0/1097797637 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 -- 172.21.15.22:0/1097797637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 msgr2=0x55bbe8b0ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 -- 172.21.15.22:0/1097797637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b0f690 msgr2=0x55bbe8b0f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 -- 172.21.15.22:0/1097797637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbe8b02200 con 0x55bbe8b01ab0 2024-03-09T00:01:27.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:27.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0c4a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x55bbe8b21db0 tx=0x7fe29c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190777158e18c80a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:27.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2917fa640 1 -- 172.21.15.22:0/1097797637 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c023020 con 0x55bbe8b01ab0 2024-03-09T00:01:27.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2917fa640 1 -- 172.21.15.22:0/1097797637 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe29c003d80 con 0x55bbe8b01ab0 2024-03-09T00:01:27.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.985+0000 7fe2917fa640 1 -- 172.21.15.22:0/1097797637 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c010990 con 0x55bbe8b01ab0 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 -- 172.21.15.22:0/1097797637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b01ab0 msgr2=0x55bbe8b0c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0c4a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x55bbe8b21db0 tx=0x7fe29c005f90 comp rx=0 tx=0).stop 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 -- 172.21.15.22:0/1097797637 shutdown_connections 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b0f690 0x55bbe8b0f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b0ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1097797637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b0c4a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 -- 172.21.15.22:0/1097797637 >> 172.21.15.22:0/1097797637 conn(0x55bbe8afc4b0 msgr2=0x55bbe8b077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 -- 172.21.15.22:0/1097797637 shutdown_connections 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 -- 172.21.15.22:0/1097797637 wait complete. 2024-03-09T00:01:27.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.986+0000 7fe2b444d740 1 -- start start 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 1 Processor -- start 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b15400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b15940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b0f690 0x55bbe8b75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1aac80 con 0x55bbe8b0f690 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1abf60 con 0x55bbe8b01ab0 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1ab2a0 con 0x55bbe8b023d0 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2b444d740 10 client.0 _ll_register_callbacks cb 0x7fe2a0005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b0f690 0x55bbe8b75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b0f690 0x55bbe8b75590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55574/0 (socket says 172.21.15.22:55574) 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 -- 172.21.15.22:0/1881735342 learned_addr learned my addr 172.21.15.22:0/1881735342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2aa7fc640 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b15940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b15400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 -- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 msgr2=0x55bbe8b15940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b15940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 -- 172.21.15.22:0/1881735342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 msgr2=0x55bbe8b15400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b15400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 -- 172.21.15.22:0/1881735342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbe8b75ad0 con 0x55bbe8b0f690 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe2ab7fe640 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b0f690 0x55bbe8b75590 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400fea0 tx=0x7fe2a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0536308a3cabc03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.987+0000 7fe28affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe28affd640 1 -- 172.21.15.22:0/1881735342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a401a040 con 0x55bbe8b0f690 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe28affd640 1 -- 172.21.15.22:0/1881735342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe2a4004260 con 0x55bbe8b0f690 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe29c003d80 con 0x55bbe8b0f690 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe28affd640 1 -- 172.21.15.22:0/1881735342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a4034050 con 0x55bbe8b0f690 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 10 client.17859 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bbe8b76c90 con 0x55bbe8b0f690 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe28affd640 1 -- 172.21.15.22:0/1881735342 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe29c003d80 con 0x55bbe8b0f690 2024-03-09T00:01:27.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe28affd640 1 -- 172.21.15.22:0/1881735342 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe2a4021b50 con 0x55bbe8b0f690 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 10 client.17859 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 20 client.17859 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b773c0 0x55bbe8b797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 4 client.17859 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bbe8a27080 con 0x55bbe8b773c0 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.988+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b773c0 0x55bbe8b797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.989+0000 7fe2aaffd640 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b773c0 0x55bbe8b797e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eefa3d6821c5bf54 server_cookie=2a1b8d804563aca in_seq=0 out_seq=0 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.989+0000 7fe28affd640 10 client.17859 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe28affd640 1 -- 172.21.15.22:0/1881735342 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bbe8a27080 con 0x55bbe8b773c0 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe28affd640 10 client.17859 handle_command_reply: tid=0 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 client.17859 shutdown 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b0f690 msgr2=0x55bbe8b75590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b0f690 0x55bbe8b75590 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400fea0 tx=0x7fe2a4004ab0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 shutdown_connections 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbe8b773c0 0x55bbe8b797e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbe8b0f690 0x55bbe8b75590 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbe8b023d0 0x55bbe8b15940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 --2- 172.21.15.22:0/1881735342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbe8b01ab0 0x55bbe8b15400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe28affd640 0 client.17859 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 >> 172.21.15.22:0/1881735342 conn(0x55bbe8afc4b0 msgr2=0x55bbe8afabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 shutdown_connections 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 1 -- 172.21.15.22:0/1881735342 wait complete. 2024-03-09T00:01:27.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.990+0000 7fe2b444d740 20 client.17859 trim_cache size 0 max 16384 2024-03-09T00:01:27.993 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:01:27.993 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:01:27.993 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "dcd07942-24ae-43d7-b820-cde287aa687c", 2024-03-09T00:01:27.993 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:01:27.993 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2940a2970 msgr2=0x7fe2940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2940a2970 0x7fe2940a4e30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005fd0 tx=0x7fe2a4011b40 comp rx=0 tx=0).stop 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac1065a0 msgr2=0x7fe2ac19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac1065a0 0x7fe2ac19dd40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0036040 tx=0x7fe2a002cb20 comp rx=0 tx=0).stop 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 shutdown_connections 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2940a2970 0x7fe2940a4e30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ac1979b0 0x7fe2ac197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2ac1065a0 0x7fe2ac19dd40 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 --2- 172.21.15.22:0/527261598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2ac068740 0x7fe2ac19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:27.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 >> 172.21.15.22:0/527261598 conn(0x7fe2ac0772f0 msgr2=0x7fe2ac0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:27.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 shutdown_connections 2024-03-09T00:01:27.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:27.991+0000 7fe2b186f640 1 -- 172.21.15.22:0/527261598 wait complete. 2024-03-09T00:01:27.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:01:27.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 -- 172.21.15.22:0/180403171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac0690f0 msgr2=0x7f8fac105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/180403171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac0690f0 0x7f8fac105ec0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009900 tx=0x7f8f9c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 -- 172.21.15.22:0/180403171 shutdown_connections 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/180403171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fac1065a0 0x7f8fac10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/180403171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac0690f0 0x7f8fac105ec0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/180403171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fac068740 0x7f8fac068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 -- 172.21.15.22:0/180403171 >> 172.21.15.22:0/180403171 conn(0x7f8fac0772f0 msgr2=0x7f8fac077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:28.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 -- 172.21.15.22:0/180403171 shutdown_connections 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.112+0000 7f8fb41c4640 1 -- 172.21.15.22:0/180403171 wait complete. 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 Processor -- start 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 -- start start 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fac068740 0x7f8fac1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac1065a0 0x7f8fac1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fac100c30 0x7f8fac1a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac111680 con 0x7f8fac1065a0 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac111500 con 0x7f8fac068740 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb41c4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac111800 con 0x7f8fac100c30 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac1065a0 0x7f8fac1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac1065a0 0x7f8fac1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55586/0 (socket says 172.21.15.22:55586) 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 -- 172.21.15.22:0/1320334677 learned_addr learned my addr 172.21.15.22:0/1320334677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb2761640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fac100c30 0x7f8fac1a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fac068740 0x7f8fac1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 -- 172.21.15.22:0/1320334677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fac100c30 msgr2=0x7f8fac1a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fac100c30 0x7f8fac1a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 -- 172.21.15.22:0/1320334677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fac068740 msgr2=0x7f8fac1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fac068740 0x7f8fac1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 -- 172.21.15.22:0/1320334677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac1a6470 con 0x7f8fac1065a0 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fac068740 0x7f8fac1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.113+0000 7f8fb175f640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac1065a0 0x7f8fac1006f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c02c240 tx=0x7f8f9c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.114+0000 7f8fa2ffd640 1 -- 172.21.15.22:0/1320334677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c043020 con 0x7f8fac1065a0 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.114+0000 7f8fa2ffd640 1 -- 172.21.15.22:0/1320334677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f9c0054e0 con 0x7f8fac1065a0 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.114+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac1a6700 con 0x7f8fac1065a0 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.114+0000 7f8fa2ffd640 1 -- 172.21.15.22:0/1320334677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c003bb0 con 0x7f8fac1065a0 2024-03-09T00:01:28.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.114+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac1a6c50 con 0x7f8fac1065a0 2024-03-09T00:01:28.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.116+0000 7f8fa2ffd640 1 -- 172.21.15.22:0/1320334677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8f9c046020 con 0x7f8fac1065a0 2024-03-09T00:01:28.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.116+0000 7f8fb6d9e740 1 Processor -- start 2024-03-09T00:01:28.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb6d9e740 1 -- start start 2024-03-09T00:01:28.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fa2ffd640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f980a29a0 0x7f8f980a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edeb2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb3080 0x5561edebb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edebe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561ede15e00 con 0x5561edeb3080 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561eddbeb10 con 0x5561edebbc60 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561ede11ea0 con 0x5561edeb2760 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fa2ffd640 1 -- 172.21.15.22:0/1320334677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8f9c0f1190 con 0x7f8fac1065a0 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb2761640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f980a29a0 0x7f8f980a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb2761640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edebe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb2761640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edebe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51550/0 (socket says 172.21.15.22:51550) 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb2761640 1 -- 172.21.15.22:0/218799253 learned_addr learned my addr 172.21.15.22:0/218799253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb175f640 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb3080 0x5561edebb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edeb2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb175f640 1 -- 172.21.15.22:0/218799253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 msgr2=0x5561edeb2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb175f640 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edeb2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb175f640 1 -- 172.21.15.22:0/218799253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 msgr2=0x5561edebe050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb175f640 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edebe050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb175f640 1 -- 172.21.15.22:0/218799253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561edbf47d0 con 0x5561edeb3080 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.117+0000 7f8fb2761640 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edebe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.118+0000 7f8fb2761640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f980a29a0 0x7f8f980a4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009fd0 tx=0x7f8fa4009f40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.118+0000 7f8fb175f640 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb3080 0x5561edebb720 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0ee5f0 tx=0x7f8f9c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd31aa685a922f16 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.118+0000 7f8f97fff640 1 -- 172.21.15.22:0/218799253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c004c80 con 0x5561edeb3080 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.118+0000 7f8f97fff640 1 -- 172.21.15.22:0/218799253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f9c005690 con 0x5561edeb3080 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.118+0000 7f8f97fff640 1 -- 172.21.15.22:0/218799253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0ee760 con 0x5561edeb3080 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/218799253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb3080 msgr2=0x5561edebb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb3080 0x5561edebb720 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0ee5f0 tx=0x7f8f9c004ee0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/218799253 shutdown_connections 2024-03-09T00:01:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edebe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb3080 0x5561edebb720 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/218799253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edeb2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/218799253 >> 172.21.15.22:0/218799253 conn(0x5561edeace90 msgr2=0x5561edead270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/218799253 shutdown_connections 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/218799253 wait complete. 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 -- start start 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.119+0000 7f8fb6d9e740 1 Processor -- start 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edebb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb3080 0x5561edeb8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edeb8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561edebe810 con 0x5561edeb2760 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561eddd20a0 con 0x5561edebbc60 2024-03-09T00:01:28.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561edebe990 con 0x5561edeb3080 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 10 client.0 _ll_register_callbacks cb 0x5561eddd1a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edebb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edebb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55598/0 (socket says 172.21.15.22:55598) 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 -- 172.21.15.22:0/1176578272 learned_addr learned my addr 172.21.15.22:0/1176578272 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb175f640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb3080 0x5561edeb8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb2761640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edeb8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 -- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb3080 msgr2=0x5561edeb8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb3080 0x5561edeb8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 -- 172.21.15.22:0/1176578272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 msgr2=0x5561edeb8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edeb8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 -- 172.21.15.22:0/1176578272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561edb2a3f0 con 0x5561edeb2760 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb2761640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edeb8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:28.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edebb3d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80137e0 tx=0x7f8fa8013ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffa2472def616c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8f957fa640 1 -- 172.21.15.22:0/1176578272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8020060 con 0x5561edeb2760 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8f957fa640 1 -- 172.21.15.22:0/1176578272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8fa8011e30 con 0x5561edeb2760 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561ede23340 con 0x5561edeb2760 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.120+0000 7f8f957fa640 1 -- 172.21.15.22:0/1176578272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fa8027050 con 0x5561edeb2760 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 10 client.17871 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5561edaea0f0 con 0x5561edeb2760 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8f957fa640 1 -- 172.21.15.22:0/1176578272 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5561ede23340 con 0x5561edeb2760 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8f957fa640 1 -- 172.21.15.22:0/1176578272 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8fa801a030 con 0x5561edeb2760 2024-03-09T00:01:28.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 10 client.17871 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 20 client.17871 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edecb430 0x5561edecd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 4 client.17871 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561ede1d690 con 0x5561edecb430 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.121+0000 7f8fb175f640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edecb430 0x5561edecd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.122+0000 7f8f957fa640 10 client.17871 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.122+0000 7f8fb175f640 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edecb430 0x5561edecd860 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89619cc2a971302f server_cookie=f807b6e324d518d1 in_seq=0 out_seq=0 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.122+0000 7f8f957fa640 1 -- 172.21.15.22:0/1176578272 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561ede1d690 con 0x5561edecb430 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.122+0000 7f8f957fa640 10 client.17871 handle_command_reply: tid=0 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.122+0000 7f8fb6d9e740 1 client.17871 shutdown 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.122+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 msgr2=0x5561edebb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.122+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edebb3d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80137e0 tx=0x7f8fa8013ca0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 shutdown_connections 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edecb430 0x5561edecd860 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edebbc60 0x5561edeb8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb3080 0x5561edeb8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/1176578272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edebb3d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8f957fa640 0 client.17871 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 >> 172.21.15.22:0/1176578272 conn(0x5561edeace90 msgr2=0x5561edeaebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 shutdown_connections 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/1176578272 wait complete. 2024-03-09T00:01:28.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.123+0000 7f8fb6d9e740 20 client.17871 trim_cache size 0 max 16384 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.138+0000 7f8fb6d9e740 1 Processor -- start 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.138+0000 7f8fb6d9e740 1 -- start start 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edebae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edebb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edebbc60 0x5561edeb59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1b2d90 con 0x5561edebbc60 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1b0930 con 0x5561edeb3080 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1ae9f0 con 0x5561edeb2760 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edebbc60 0x5561edeb59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edebbc60 0x5561edeb59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55602/0 (socket says 172.21.15.22:55602) 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 -- 172.21.15.22:0/4226581991 learned_addr learned my addr 172.21.15.22:0/4226581991 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:28.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edebae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb175f640 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edebb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 -- 172.21.15.22:0/4226581991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 msgr2=0x5561edebae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edebae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 -- 172.21.15.22:0/4226581991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 msgr2=0x5561edebb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edebb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 -- 172.21.15.22:0/4226581991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561eded2a50 con 0x5561edebbc60 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8fb2761640 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edebbc60 0x5561edeb59e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400ffa0 tx=0x7f8fa4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119e9c3fc1ce3e3a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8f977fe640 1 -- 172.21.15.22:0/4226581991 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4011cb0 con 0x5561edebbc60 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8f977fe640 1 -- 172.21.15.22:0/4226581991 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fa4024070 con 0x5561edebbc60 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.139+0000 7f8f977fe640 1 -- 172.21.15.22:0/4226581991 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4028bc0 con 0x5561edebbc60 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/4226581991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edebbc60 msgr2=0x5561edeb59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edebbc60 0x5561edeb59e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400ffa0 tx=0x7f8fa4021000 comp rx=0 tx=0).stop 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/4226581991 shutdown_connections 2024-03-09T00:01:28.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edebbc60 0x5561edeb59e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edebb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/4226581991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edeb2760 0x5561edebae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/4226581991 >> 172.21.15.22:0/4226581991 conn(0x5561edeace90 msgr2=0x5561edeb8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:28.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/4226581991 shutdown_connections 2024-03-09T00:01:28.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/4226581991 wait complete. 2024-03-09T00:01:28.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 -- start start 2024-03-09T00:01:28.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.140+0000 7f8fb6d9e740 1 Processor -- start 2024-03-09T00:01:28.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edec2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edec3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb6d9e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edebbc60 0x5561edeb4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1aeff0 con 0x5561edeb2760 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1b02d0 con 0x5561edeb3080 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb6d9e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1af610 con 0x5561edebbc60 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb6d9e740 10 client.0 _ll_register_callbacks cb 0x7f8fa80098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edec2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edec2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55612/0 (socket says 172.21.15.22:55612) 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 -- 172.21.15.22:0/2816908449 learned_addr learned my addr 172.21.15.22:0/2816908449 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb175f640 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edec3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb2761640 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edebbc60 0x5561edeb4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 -- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edebbc60 msgr2=0x5561edeb4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edebbc60 0x5561edeb4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 -- 172.21.15.22:0/2816908449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 msgr2=0x5561edec3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edec3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 -- 172.21.15.22:0/2816908449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561edeb55f0 con 0x5561edeb2760 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8fb1f60640 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edec2af0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009ed0 tx=0x7f8fa8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addc0d6cdc7d6809 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.141+0000 7f8f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8f957fa640 1 -- 172.21.15.22:0/2816908449 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa800e520 con 0x5561edeb2760 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8f957fa640 1 -- 172.21.15.22:0/2816908449 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8fa8011c30 con 0x5561edeb2760 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fa4024070 con 0x5561edeb2760 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8f957fa640 1 -- 172.21.15.22:0/2816908449 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fa8039050 con 0x5561edeb2760 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 10 client.17877 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:28.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5561edf22470 con 0x5561edeb2760 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8f957fa640 1 -- 172.21.15.22:0/2816908449 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fa4024070 con 0x5561edeb2760 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8f957fa640 1 -- 172.21.15.22:0/2816908449 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8fa801a070 con 0x5561edeb2760 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 10 client.17877 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 20 client.17877 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edf22b90 0x5561edf24fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 4 client.17877 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5561edd9b4e0 con 0x5561edf22b90 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.142+0000 7f8fb175f640 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edf22b90 0x5561edf24fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8f957fa640 10 client.17877 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb175f640 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edf22b90 0x5561edf24fb0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9198f92c5dbc2d41 server_cookie=49674aa5130f5496 in_seq=0 out_seq=0 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8f957fa640 1 -- 172.21.15.22:0/2816908449 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5561edd9b4e0 con 0x5561edf22b90 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8f957fa640 10 client.17877 handle_command_reply: tid=0 2024-03-09T00:01:28.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 client.17877 shutdown 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 msgr2=0x5561edec2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edec2af0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009ed0 tx=0x7f8fa8009f00 comp rx=0 tx=0).stop 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 shutdown_connections 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5561edf22b90 0x5561edf24fb0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5561edebbc60 0x5561edeb4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561edeb3080 0x5561edec3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 --2- 172.21.15.22:0/2816908449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561edeb2760 0x5561edec2af0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8f957fa640 0 client.17877 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 >> 172.21.15.22:0/2816908449 conn(0x5561edeace90 msgr2=0x5561edeb35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 shutdown_connections 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 1 -- 172.21.15.22:0/2816908449 wait complete. 2024-03-09T00:01:28.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.143+0000 7f8fb6d9e740 20 client.17877 trim_cache size 0 max 16384 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: "dcd07942-24ae-43d7-b820-cde287aa687c": { 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "dcd07942-24ae-43d7-b820-cde287aa687c", 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:01:28.146 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:01:28.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f980a29a0 msgr2=0x7f8f980a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f980a29a0 0x7f8f980a4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009fd0 tx=0x7f8fa4009f40 comp rx=0 tx=0).stop 2024-03-09T00:01:28.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac1065a0 msgr2=0x7f8fac1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac1065a0 0x7f8fac1006f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c02c240 tx=0x7f8f9c004360 comp rx=0 tx=0).stop 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 shutdown_connections 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8f980a29a0 0x7f8f980a4e60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fac100c30 0x7f8fac1a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fac1065a0 0x7f8fac1006f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 --2- 172.21.15.22:0/1320334677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fac068740 0x7f8fac1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 >> 172.21.15.22:0/1320334677 conn(0x7f8fac0772f0 msgr2=0x7f8fac0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 shutdown_connections 2024-03-09T00:01:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:28.144+0000 7f8fb41c4640 1 -- 172.21.15.22:0/1320334677 wait complete. 2024-03-09T00:01:28.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:28 smithi022 ceph-mon[31894]: pgmap v216: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-09T00:01:28.150 INFO:tasks.cephfs.filesystem:scrub status for tag:dcd07942-24ae-43d7-b820-cde287aa687c - {'path': '/', 'tag': 'dcd07942-24ae-43d7-b820-cde287aa687c', 'options': 'recursive,force'} 2024-03-09T00:01:28.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:28 smithi062 ceph-mon[36129]: pgmap v216: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-09T00:01:28.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:28 smithi192 ceph-mon[35995]: pgmap v216: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-09T00:01:28.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 725328 10.91 MB/sec execute 216 sec latency 18.016 ms 2024-03-09T00:01:29.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:29 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:01:29.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:29 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:01:29.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:29 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:01:29.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:29 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:01:29.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:29 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:01:29.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:29 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:01:29.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:29 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:01:29.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:29 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:01:29.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:29 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:01:29.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:29 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:01:29.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:29 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:01:29.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:29 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:01:29.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 727955 10.94 MB/sec execute 217 sec latency 16.312 ms 2024-03-09T00:01:30.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:30 smithi062 ceph-mon[36129]: pgmap v217: 129 pgs: 129 active+clean; 83 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 490 op/s 2024-03-09T00:01:30.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:30 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:01:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:30 smithi192 ceph-mon[35995]: pgmap v217: 129 pgs: 129 active+clean; 83 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 490 op/s 2024-03-09T00:01:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:30 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:01:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:30 smithi022 ceph-mon[31894]: pgmap v217: 129 pgs: 129 active+clean; 83 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 490 op/s 2024-03-09T00:01:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:30 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:01:30.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 729476 10.92 MB/sec execute 218 sec latency 59.256 ms 2024-03-09T00:01:31.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 731630 10.92 MB/sec execute 219 sec latency 139.558 ms 2024-03-09T00:01:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.954+0000 7f601c97e640 1 -- 172.21.15.22:0/2446059503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018100440 msgr2=0x7f60181008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.954+0000 7f601c97e640 1 --2- 172.21.15.22:0/2446059503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018100440 0x7f60181008a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 -- 172.21.15.22:0/2446059503 shutdown_connections 2024-03-09T00:01:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 --2- 172.21.15.22:0/2446059503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6018100de0 0x7f601810cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 --2- 172.21.15.22:0/2446059503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018100440 0x7f60181008a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 --2- 172.21.15.22:0/2446059503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6018106220 0x7f6018106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 -- 172.21.15.22:0/2446059503 >> 172.21.15.22:0/2446059503 conn(0x7f6018077010 msgr2=0x7f6018077420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 -- 172.21.15.22:0/2446059503 shutdown_connections 2024-03-09T00:01:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 -- 172.21.15.22:0/2446059503 wait complete. 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 Processor -- start 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.955+0000 7f601c97e640 1 -- start start 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6018100440 0x7f60181020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6018100de0 0x7f6018102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018106220 0x7f6018102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6018110550 con 0x7f6018106220 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181103d0 con 0x7f6018100440 2024-03-09T00:01:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181106d0 con 0x7f6018100de0 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018106220 0x7f6018102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018106220 0x7f6018102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41992/0 (socket says 172.21.15.22:41992) 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 -- 172.21.15.22:0/2778979850 learned_addr learned my addr 172.21.15.22:0/2778979850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6015d9b640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6018100de0 0x7f6018102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601659c640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6018100440 0x7f60181020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 -- 172.21.15.22:0/2778979850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6018100de0 msgr2=0x7f6018102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6018100de0 0x7f6018102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 -- 172.21.15.22:0/2778979850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6018100440 msgr2=0x7f60181020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6018100440 0x7f60181020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 -- 172.21.15.22:0/2778979850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60181a5fa0 con 0x7f6018106220 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601659c640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6018100440 0x7f60181020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f6016d9d640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018106220 0x7f6018102b50 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f600800eb10 tx=0x7f600800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f600f7fe640 1 -- 172.21.15.22:0/2778979850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008004b20 con 0x7f6018106220 2024-03-09T00:01:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60181a6290 con 0x7f6018106220 2024-03-09T00:01:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f600f7fe640 1 -- 172.21.15.22:0/2778979850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6008004cc0 con 0x7f6018106220 2024-03-09T00:01:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f600f7fe640 1 -- 172.21.15.22:0/2778979850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008010760 con 0x7f6018106220 2024-03-09T00:01:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.956+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60181a6800 con 0x7f6018106220 2024-03-09T00:01:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.958+0000 7f600f7fe640 1 -- 172.21.15.22:0/2778979850 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6008005020 con 0x7f6018106220 2024-03-09T00:01:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 Processor -- start 2024-03-09T00:01:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 -- start start 2024-03-09T00:01:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f600f7fe640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60040a2a10 0x7f60040a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571e1d40 0x5556571e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571ea550 0x5556571ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556571455f0 con 0x5556571ea550 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556570ee930 con 0x5556571e0fd0 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601f636740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555657141690 con 0x5556571e1d40 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f6015d9b640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60040a2a10 0x7f60040a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f600f7fe640 1 -- 172.21.15.22:0/2778979850 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f60080cfa00 con 0x7f6018106220 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571e1d40 0x5556571e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f6016d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571ea550 0x5556571ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571e1d40 0x5556571e0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:43014/0 (socket says 172.21.15.22:43014) 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.959+0000 7f601659c640 1 -- 172.21.15.22:0/1767436010 learned_addr learned my addr 172.21.15.22:0/1767436010 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6015d9b640 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6016d9d640 1 -- 172.21.15.22:0/1767436010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571e1d40 msgr2=0x5556571e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6016d9d640 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571e1d40 0x5556571e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6016d9d640 1 -- 172.21.15.22:0/1767436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 msgr2=0x5556571e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6016d9d640 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6016d9d640 1 -- 172.21.15.22:0/1767436010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555656e5a210 con 0x5556571ea550 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6015d9b640 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6015d9b640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60040a2a10 0x7f60040a4ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6000006010 tx=0x7f600002ca40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f6016d9d640 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571ea550 0x5556571ec940 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f60080ccd50 tx=0x7f600809b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b919e0bdbf820fd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f600cff9640 1 -- 172.21.15.22:0/1767436010 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600809bc60 con 0x5556571ea550 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f600cff9640 1 -- 172.21.15.22:0/1767436010 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f600809fdb0 con 0x5556571ea550 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.960+0000 7f600cff9640 1 -- 172.21.15.22:0/1767436010 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008005510 con 0x5556571ea550 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 -- 172.21.15.22:0/1767436010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571ea550 msgr2=0x5556571ec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571ea550 0x5556571ec940 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f60080ccd50 tx=0x7f600809b7a0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 -- 172.21.15.22:0/1767436010 shutdown_connections 2024-03-09T00:01:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571ea550 0x5556571ec940 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 --2- 172.21.15.22:0/1767436010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571e1d40 0x5556571e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 -- 172.21.15.22:0/1767436010 >> 172.21.15.22:0/1767436010 conn(0x5556571a85b0 msgr2=0x5556571dbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 -- 172.21.15.22:0/1767436010 shutdown_connections 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 -- 172.21.15.22:0/1767436010 wait complete. 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 -- start start 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.961+0000 7f601f636740 1 Processor -- start 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e0fd0 0x5556571e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e1d40 0x5556571e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556571ed220 con 0x5556571e0fd0 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555657101d90 con 0x5556571e1d40 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556571ed3a0 con 0x5556571ea550 2024-03-09T00:01:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 10 client.0 _ll_register_callbacks cb 0x555656e59a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e0fd0 0x5556571e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e0fd0 0x5556571e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42006/0 (socket says 172.21.15.22:42006) 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 -- 172.21.15.22:0/561614297 learned_addr learned my addr 172.21.15.22:0/561614297 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f6015d9b640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e1d40 0x5556571e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f6016d9d640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 -- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 msgr2=0x5556571e7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 -- 172.21.15.22:0/561614297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e1d40 msgr2=0x5556571e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e1d40 0x5556571e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 -- 172.21.15.22:0/561614297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555656f26620 con 0x5556571e0fd0 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f6016d9d640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601659c640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e0fd0 0x5556571e74f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f60100127a0 tx=0x7f6010012c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2f8c167a706dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f5ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f5ffdffb640 1 -- 172.21.15.22:0/561614297 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010011bb0 con 0x5556571e0fd0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f600809fdb0 con 0x5556571e0fd0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.962+0000 7f5ffdffb640 1 -- 172.21.15.22:0/561614297 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6010011d50 con 0x5556571e0fd0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f5ffdffb640 1 -- 172.21.15.22:0/561614297 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f601001c420 con 0x5556571e0fd0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 10 client.17889 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5556571f94c0 con 0x5556571e0fd0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f5ffdffb640 1 -- 172.21.15.22:0/561614297 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f600809fdb0 con 0x5556571e0fd0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f5ffdffb640 1 -- 172.21.15.22:0/561614297 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f601000f7c0 con 0x5556571e0fd0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 10 client.17889 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 10 client.17889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 20 client.17889 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556571fa980 0x5556571fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 4 client.17889 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555657152b30 con 0x5556571fa980 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.963+0000 7f6015d9b640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556571fa980 0x5556571fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.964+0000 7f5ffdffb640 10 client.17889 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.964+0000 7f6015d9b640 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556571fa980 0x5556571fcd70 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcaead9dad597c87 server_cookie=938e4853f626472e in_seq=0 out_seq=0 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f5ffdffb640 1 -- 172.21.15.22:0/561614297 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555657152b30 con 0x5556571fa980 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f5ffdffb640 10 client.17889 handle_command_reply: tid=0 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 client.17889 shutdown 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e0fd0 msgr2=0x5556571e74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e0fd0 0x5556571e74f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f60100127a0 tx=0x7f6010012c60 comp rx=0 tx=0).stop 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 shutdown_connections 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556571fa980 0x5556571fcd70 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e1d40 0x5556571e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f5ffdffb640 0 client.17889 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 --2- 172.21.15.22:0/561614297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e0fd0 0x5556571e74f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 >> 172.21.15.22:0/561614297 conn(0x5556571a85b0 msgr2=0x5556571dda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 shutdown_connections 2024-03-09T00:01:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 1 -- 172.21.15.22:0/561614297 wait complete. 2024-03-09T00:01:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.965+0000 7f601f636740 20 client.17889 trim_cache size 0 max 16384 2024-03-09T00:01:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 Processor -- start 2024-03-09T00:01:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 -- start start 2024-03-09T00:01:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181b2dc0 con 0x5556571e1d40 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181b0960 con 0x5556571e0fd0 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.979+0000 7f601f636740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181aea20 con 0x5556571ea550 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571e5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42010/0 (socket says 172.21.15.22:42010) 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 -- 172.21.15.22:0/693232299 learned_addr learned my addr 172.21.15.22:0/693232299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6016d9d640 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f601659c640 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 -- 172.21.15.22:0/693232299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 msgr2=0x5556571e6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 -- 172.21.15.22:0/693232299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 msgr2=0x5556571e9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 -- 172.21.15.22:0/693232299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556571e02a0 con 0x5556571e1d40 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f601659c640 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f6015d9b640 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571e5dc0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f600002cfd0 tx=0x7f600002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe787caeb6a56ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f5ffffff640 1 -- 172.21.15.22:0/693232299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000004980 con 0x5556571e1d40 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f5ffffff640 1 -- 172.21.15.22:0/693232299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6000059070 con 0x5556571e1d40 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.980+0000 7f5ffffff640 1 -- 172.21.15.22:0/693232299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60000533c0 con 0x5556571e1d40 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 -- 172.21.15.22:0/693232299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 msgr2=0x5556571e5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571e5dc0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f600002cfd0 tx=0x7f600002e050 comp rx=0 tx=0).stop 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 -- 172.21.15.22:0/693232299 shutdown_connections 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x5556571e6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571e5dc0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 --2- 172.21.15.22:0/693232299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571e9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 -- 172.21.15.22:0/693232299 >> 172.21.15.22:0/693232299 conn(0x5556571a85b0 msgr2=0x5556572016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 -- 172.21.15.22:0/693232299 shutdown_connections 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 -- 172.21.15.22:0/693232299 wait complete. 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 -- start start 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 Processor -- start 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.981+0000 7f601f636740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x555657250e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601f636740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181af020 con 0x5556571e1d40 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601f636740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181b0300 con 0x5556571e0fd0 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601f636740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181af640 con 0x5556571ea550 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601f636740 10 client.0 _ll_register_callbacks cb 0x7f6010009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42016/0 (socket says 172.21.15.22:42016) 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 -- 172.21.15.22:0/2929974208 learned_addr learned my addr 172.21.15.22:0/2929974208 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601659c640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6016d9d640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x555657250e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 -- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 msgr2=0x555657250e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x555657250e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 -- 172.21.15.22:0/2929974208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 msgr2=0x5556571f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 -- 172.21.15.22:0/2929974208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556572513f0 con 0x5556571e1d40 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601659c640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f6015d9b640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571f0c70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f600002cd90 tx=0x7f600002c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca461dddd9909c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f5ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f5ffdffb640 1 -- 172.21.15.22:0/2929974208 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000004ee0 con 0x5556571e1d40 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f5ffdffb640 1 -- 172.21.15.22:0/2929974208 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6000005080 con 0x5556571e1d40 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6000059070 con 0x5556571e1d40 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f5ffdffb640 1 -- 172.21.15.22:0/2929974208 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6000068050 con 0x5556571e1d40 2024-03-09T00:01:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601f636740 10 client.17895 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.982+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555657252670 con 0x5556571e1d40 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f5ffdffb640 1 -- 172.21.15.22:0/2929974208 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6000059070 con 0x5556571e1d40 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f5ffdffb640 1 -- 172.21.15.22:0/2929974208 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f600004d040 con 0x5556571e1d40 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601f636740 10 client.17895 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601f636740 10 client.17895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601f636740 20 client.17895 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601f636740 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555657252d90 0x5556572551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601f636740 4 client.17895 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55565714ce80 con 0x555657252d90 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601659c640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555657252d90 0x5556572551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f5ffdffb640 10 client.17895 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:31.983+0000 7f601659c640 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555657252d90 0x5556572551b0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45425c22fa55d8c2 server_cookie=e2410f13b189105a in_seq=0 out_seq=0 2024-03-09T00:01:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:32 smithi062 ceph-mon[36129]: pgmap v218: 129 pgs: 129 active+clean; 85 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f5ffdffb640 1 -- 172.21.15.22:0/2929974208 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55565714ce80 con 0x555657252d90 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f5ffdffb640 10 client.17895 handle_command_reply: tid=0 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 client.17895 shutdown 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 msgr2=0x5556571f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571f0c70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f600002cd90 tx=0x7f600002c330 comp rx=0 tx=0).stop 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 shutdown_connections 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555657252d90 0x5556572551b0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f5ffdffb640 0 client.17895 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556571ea550 0x555657250e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556571e1d40 0x5556571f0c70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 --2- 172.21.15.22:0/2929974208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556571e0fd0 0x5556571f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.376+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 >> 172.21.15.22:0/2929974208 conn(0x5556571a85b0 msgr2=0x5556571e5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.377+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 shutdown_connections 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.377+0000 7f601f636740 1 -- 172.21.15.22:0/2929974208 wait complete. 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.377+0000 7f601f636740 20 client.17895 trim_cache size 0 max 16384 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:01:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:01:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.377+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60040a2a10 msgr2=0x7f60040a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.377+0000 7f601c97e640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60040a2a10 0x7f60040a4ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6000006010 tx=0x7f600002ca40 comp rx=0 tx=0).stop 2024-03-09T00:01:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.377+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018106220 msgr2=0x7f6018102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.377+0000 7f601c97e640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018106220 0x7f6018102b50 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f600800eb10 tx=0x7f600800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 shutdown_connections 2024-03-09T00:01:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f60040a2a10 0x7f60040a4ed0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6018106220 0x7f6018102b50 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6018100de0 0x7f6018102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 --2- 172.21.15.22:0/2778979850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6018100440 0x7f60181020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 >> 172.21.15.22:0/2778979850 conn(0x7f6018077010 msgr2=0x7f6018104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 shutdown_connections 2024-03-09T00:01:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:32.378+0000 7f601c97e640 1 -- 172.21.15.22:0/2778979850 wait complete. 2024-03-09T00:01:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:32 smithi192 ceph-mon[35995]: pgmap v218: 129 pgs: 129 active+clean; 85 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-09T00:01:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:32 smithi022 ceph-mon[31894]: pgmap v218: 129 pgs: 129 active+clean; 85 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-09T00:01:32.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 732475 10.88 MB/sec execute 220 sec latency 189.294 ms 2024-03-09T00:01:33.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 735189 10.91 MB/sec execute 221 sec latency 16.097 ms 2024-03-09T00:01:34.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:34 smithi062 ceph-mon[36129]: pgmap v219: 129 pgs: 129 active+clean; 85 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2024-03-09T00:01:34.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:34 smithi192 ceph-mon[35995]: pgmap v219: 129 pgs: 129 active+clean; 85 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2024-03-09T00:01:34.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:34 smithi022 ceph-mon[31894]: pgmap v219: 129 pgs: 129 active+clean; 85 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2024-03-09T00:01:34.719 INFO:tasks.workunit.client.0.smithi022.stdout: 1 736813 10.89 MB/sec execute 222 sec latency 25.021 ms 2024-03-09T00:01:35.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 739017 10.91 MB/sec execute 223 sec latency 17.795 ms 2024-03-09T00:01:36.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:36 smithi062 ceph-mon[36129]: pgmap v220: 129 pgs: 129 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 469 op/s 2024-03-09T00:01:36.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:36 smithi192 ceph-mon[35995]: pgmap v220: 129 pgs: 129 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 469 op/s 2024-03-09T00:01:36.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:36 smithi022 ceph-mon[31894]: pgmap v220: 129 pgs: 129 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 469 op/s 2024-03-09T00:01:36.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 740447 10.88 MB/sec execute 224 sec latency 48.112 ms 2024-03-09T00:01:37.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 742654 10.89 MB/sec execute 225 sec latency 28.578 ms 2024-03-09T00:01:38.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:38 smithi062 ceph-mon[36129]: pgmap v221: 129 pgs: 129 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2024-03-09T00:01:38.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:38 smithi192 ceph-mon[35995]: pgmap v221: 129 pgs: 129 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2024-03-09T00:01:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:38 smithi022 ceph-mon[31894]: pgmap v221: 129 pgs: 129 active+clean; 71 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2024-03-09T00:01:38.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 744730 10.88 MB/sec execute 226 sec latency 16.805 ms 2024-03-09T00:01:39.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 746421 10.88 MB/sec execute 227 sec latency 28.945 ms 2024-03-09T00:01:40.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:40 smithi062 ceph-mon[36129]: pgmap v222: 129 pgs: 129 active+clean; 78 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-03-09T00:01:40.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:40 smithi192 ceph-mon[35995]: pgmap v222: 129 pgs: 129 active+clean; 78 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-03-09T00:01:40.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:40 smithi022 ceph-mon[31894]: pgmap v222: 129 pgs: 129 active+clean; 78 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 429 op/s 2024-03-09T00:01:40.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 749003 10.90 MB/sec execute 228 sec latency 15.708 ms 2024-03-09T00:01:41.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 750327 10.88 MB/sec execute 229 sec latency 46.525 ms 2024-03-09T00:01:42.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:42 smithi062 ceph-mon[36129]: pgmap v223: 129 pgs: 129 active+clean; 79 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 380 op/s 2024-03-09T00:01:42.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:42 smithi192 ceph-mon[35995]: pgmap v223: 129 pgs: 129 active+clean; 79 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 380 op/s 2024-03-09T00:01:42.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:42 smithi022 ceph-mon[31894]: pgmap v223: 129 pgs: 129 active+clean; 79 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 380 op/s 2024-03-09T00:01:42.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 752978 10.91 MB/sec execute 230 sec latency 13.220 ms 2024-03-09T00:01:43.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 754464 10.88 MB/sec execute 231 sec latency 32.064 ms 2024-03-09T00:01:44.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:44 smithi062 ceph-mon[36129]: pgmap v224: 129 pgs: 129 active+clean; 79 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-03-09T00:01:44.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:44 smithi192 ceph-mon[35995]: pgmap v224: 129 pgs: 129 active+clean; 79 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-03-09T00:01:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:44 smithi022 ceph-mon[31894]: pgmap v224: 129 pgs: 129 active+clean; 79 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-03-09T00:01:44.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 756753 10.89 MB/sec execute 232 sec latency 22.895 ms 2024-03-09T00:01:45.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:01:45.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 -- 172.21.15.22:0/3102427049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c81062d0 msgr2=0x7f95c81066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 --2- 172.21.15.22:0/3102427049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c81062d0 0x7f95c81066b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009cd0 tx=0x7f95c402c0f0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 -- 172.21.15.22:0/3102427049 shutdown_connections 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 --2- 172.21.15.22:0/3102427049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c81001b0 0x7f95c810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 --2- 172.21.15.22:0/3102427049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95c80ff7f0 0x7f95c80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 --2- 172.21.15.22:0/3102427049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c81062d0 0x7f95c81066b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 -- 172.21.15.22:0/3102427049 >> 172.21.15.22:0/3102427049 conn(0x7f95c80767b0 msgr2=0x7f95c80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 -- 172.21.15.22:0/3102427049 shutdown_connections 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 -- 172.21.15.22:0/3102427049 wait complete. 2024-03-09T00:01:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.469+0000 7f95cf980640 1 Processor -- start 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 -- start start 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c80ff7f0 0x7f95c819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c81001b0 0x7f95c819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95c81062d0 0x7f95c81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c8111780 con 0x7f95c80ff7f0 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c8111600 con 0x7f95c81001b0 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c8111900 con 0x7f95c81062d0 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c80ff7f0 0x7f95c819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c80ff7f0 0x7f95c819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41272/0 (socket says 172.21.15.22:41272) 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 -- 172.21.15.22:0/4083200847 learned_addr learned my addr 172.21.15.22:0/4083200847 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c81001b0 0x7f95c819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95c81062d0 0x7f95c81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 -- 172.21.15.22:0/4083200847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95c81062d0 msgr2=0x7f95c81979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95c81062d0 0x7f95c81979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 -- 172.21.15.22:0/4083200847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c81001b0 msgr2=0x7f95c819dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c81001b0 0x7f95c819dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 -- 172.21.15.22:0/4083200847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8198140 con 0x7f95c80ff7f0 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95c81062d0 0x7f95c81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cd71c640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c80ff7f0 0x7f95c819d810 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f95c4004af0 tx=0x7f95c4032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95be7fc640 1 -- 172.21.15.22:0/4083200847 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4041020 con 0x7f95c80ff7f0 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95be7fc640 1 -- 172.21.15.22:0/4083200847 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c4034cd0 con 0x7f95c80ff7f0 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c81983d0 con 0x7f95c80ff7f0 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.470+0000 7f95be7fc640 1 -- 172.21.15.22:0/4083200847 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4039040 con 0x7f95c80ff7f0 2024-03-09T00:01:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.471+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c81a46d0 con 0x7f95c80ff7f0 2024-03-09T00:01:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.473+0000 7f95be7fc640 1 -- 172.21.15.22:0/4083200847 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f95c4044020 con 0x7f95c80ff7f0 2024-03-09T00:01:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.473+0000 7f95d2634740 1 Processor -- start 2024-03-09T00:01:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.473+0000 7f95d2634740 1 -- start start 2024-03-09T00:01:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.473+0000 7f95be7fc640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95b40a2a10 0x7f95b40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.473+0000 7f95d2634740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8551a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.473+0000 7f95d2634740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8552450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.473+0000 7f95d2634740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a855d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95d2634740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a84b65f0 con 0x5629a855b550 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95d2634740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a845f930 con 0x5629a8552d40 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95d2634740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a84b2690 con 0x5629a8551fd0 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95be7fc640 1 -- 172.21.15.22:0/4083200847 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f95c40f0e90 con 0x7f95c80ff7f0 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8552450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95b40a2a10 0x7f95b40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8552450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60106/0 (socket says 172.21.15.22:60106) 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 -- 172.21.15.22:0/4102804961 learned_addr learned my addr 172.21.15.22:0/4102804961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a855d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95cd71c640 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8551a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 -- 172.21.15.22:0/4102804961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 msgr2=0x5629a8551a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8551a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 -- 172.21.15.22:0/4102804961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 msgr2=0x5629a855d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a855d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 -- 172.21.15.22:0/4102804961 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629a81cb210 con 0x5629a8551fd0 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95cd71c640 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8551a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8552450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009c60 tx=0x7f95b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a5121ae5fdd244 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95b40a2a10 0x7f95b40a4ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f95c0005fd0 tx=0x7f95c000cc40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95b37fe640 1 -- 172.21.15.22:0/4102804961 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8004230 con 0x5629a8551fd0 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95b37fe640 1 -- 172.21.15.22:0/4102804961 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95b80043d0 con 0x5629a8551fd0 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.474+0000 7f95b37fe640 1 -- 172.21.15.22:0/4102804961 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80197c0 con 0x5629a8551fd0 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 -- 172.21.15.22:0/4102804961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 msgr2=0x5629a8552450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8552450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009c60 tx=0x7f95b8011810 comp rx=0 tx=0).stop 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 -- 172.21.15.22:0/4102804961 shutdown_connections 2024-03-09T00:01:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a855d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8552450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 --2- 172.21.15.22:0/4102804961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8551a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 -- 172.21.15.22:0/4102804961 >> 172.21.15.22:0/4102804961 conn(0x5629a85195b0 msgr2=0x5629a854ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 -- 172.21.15.22:0/4102804961 shutdown_connections 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 -- 172.21.15.22:0/4102804961 wait complete. 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.475+0000 7f95d2634740 1 -- start start 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 1 Processor -- start 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8551fd0 0x5629a8558530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8552d40 0x5629a8558a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8558fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a855e280 con 0x5629a8552d40 2024-03-09T00:01:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a8472d90 con 0x5629a8551fd0 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629a855e400 con 0x5629a855b550 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95d2634740 10 client.0 _ll_register_callbacks cb 0x5629a81caa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8552d40 0x5629a8558a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8552d40 0x5629a8558a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41304/0 (socket says 172.21.15.22:41304) 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 -- 172.21.15.22:0/968031484 learned_addr learned my addr 172.21.15.22:0/968031484 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8558fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95cd71c640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8551fd0 0x5629a8558530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 -- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 msgr2=0x5629a8558fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8558fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 -- 172.21.15.22:0/968031484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8551fd0 msgr2=0x5629a8558530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8551fd0 0x5629a8558530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95ccf1b640 1 -- 172.21.15.22:0/968031484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629a8297620 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8558fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.476+0000 7f95cd71c640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8551fd0 0x5629a8558530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8552d40 0x5629a8558a70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f95b8003f90 tx=0x7f95b80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627448f676f4e7fd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95b0ff9640 1 -- 172.21.15.22:0/968031484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8028070 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95b0ff9640 1 -- 172.21.15.22:0/968031484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f95b8005040 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95b80043d0 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95b0ff9640 1 -- 172.21.15.22:0/968031484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95b8034050 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 10 client.17907 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5629a856a4a0 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95b0ff9640 1 -- 172.21.15.22:0/968031484 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95b80043d0 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95b0ff9640 1 -- 172.21.15.22:0/968031484 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95b8004b10 con 0x5629a8552d40 2024-03-09T00:01:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 10 client.17907 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 10 client.17907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 20 client.17907 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a856b8b0 0x5629a856dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 4 client.17907 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629a84c3b30 con 0x5629a856b8b0 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.477+0000 7f95cd71c640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a856b8b0 0x5629a856dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.478+0000 7f95b0ff9640 10 client.17907 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.478+0000 7f95cd71c640 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a856b8b0 0x5629a856dca0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a07e6e2d16fa571 server_cookie=151f0e0eff2c14a7 in_seq=0 out_seq=0 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.478+0000 7f95b0ff9640 1 -- 172.21.15.22:0/968031484 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629a84c3b30 con 0x5629a856b8b0 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.478+0000 7f95b0ff9640 10 client.17907 handle_command_reply: tid=0 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 client.17907 shutdown 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8552d40 msgr2=0x5629a8558a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8552d40 0x5629a8558a70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f95b8003f90 tx=0x7f95b80038a0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 shutdown_connections 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a856b8b0 0x5629a856dca0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8558fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95b0ff9640 0 client.17907 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8552d40 0x5629a8558a70 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 --2- 172.21.15.22:0/968031484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8551fd0 0x5629a8558530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 >> 172.21.15.22:0/968031484 conn(0x5629a85195b0 msgr2=0x5629a854ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 shutdown_connections 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 1 -- 172.21.15.22:0/968031484 wait complete. 2024-03-09T00:01:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.479+0000 7f95d2634740 20 client.17907 trim_cache size 0 max 16384 2024-03-09T00:01:45.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:01:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 Processor -- start 2024-03-09T00:01:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 -- start start 2024-03-09T00:01:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8556e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8557340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a85557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81b0ac0 con 0x5629a855b550 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81ae660 con 0x5629a8552d40 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.493+0000 7f95d2634740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81ac720 con 0x5629a8551fd0 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cdf1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a85557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95ccf1b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8557340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8556e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cdf1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a85557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41318/0 (socket says 172.21.15.22:41318) 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95ccf1b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8557340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41028/0 (socket says 172.21.15.22:41028) 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8556e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60120/0 (socket says 172.21.15.22:60120) 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cdf1d640 1 -- 172.21.15.22:0/1684232015 learned_addr learned my addr 172.21.15.22:0/1684232015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 -- 172.21.15.22:0/1684232015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 msgr2=0x5629a8557340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8557340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 -- 172.21.15.22:0/1684232015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 msgr2=0x5629a85557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a85557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 -- 172.21.15.22:0/1684232015 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629a85540c0 con 0x5629a8551fd0 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a85557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95cd71c640 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8556e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95c40ee2f0 tx=0x7f95c4009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7049b74c2e1c9b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95b2ffd640 1 -- 172.21.15.22:0/1684232015 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4034920 con 0x5629a8551fd0 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95b2ffd640 1 -- 172.21.15.22:0/1684232015 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c40c1120 con 0x5629a8551fd0 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.494+0000 7f95b2ffd640 1 -- 172.21.15.22:0/1684232015 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40ee460 con 0x5629a8551fd0 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 -- 172.21.15.22:0/1684232015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 msgr2=0x5629a8556e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8556e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95c40ee2f0 tx=0x7f95c4009800 comp rx=0 tx=0).stop 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 -- 172.21.15.22:0/1684232015 shutdown_connections 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a855b550 0x5629a85557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8557340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 --2- 172.21.15.22:0/1684232015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a8551fd0 0x5629a8556e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 -- 172.21.15.22:0/1684232015 >> 172.21.15.22:0/1684232015 conn(0x5629a85195b0 msgr2=0x5629a854c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 -- 172.21.15.22:0/1684232015 shutdown_connections 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 -- 172.21.15.22:0/1684232015 wait complete. 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 -- start start 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.495+0000 7f95d2634740 1 Processor -- start 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95d2634740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8551fd0 0x5629a8562ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95d2634740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8560e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95d2634740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8561480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95d2634740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81acd20 con 0x5629a8551fd0 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95d2634740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81ae000 con 0x5629a8552d40 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95d2634740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81ad340 con 0x5629a855b550 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95d2634740 10 client.0 _ll_register_callbacks cb 0x7f95b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8551fd0 0x5629a8562ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8551fd0 0x5629a8562ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41334/0 (socket says 172.21.15.22:41334) 2024-03-09T00:01:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 -- 172.21.15.22:0/24020048 learned_addr learned my addr 172.21.15.22:0/24020048 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cdf1d640 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8561480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8560e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 -- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 msgr2=0x5629a8561480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8561480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 -- 172.21.15.22:0/24020048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 msgr2=0x5629a8560e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8560e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 -- 172.21.15.22:0/24020048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629a8561c80 con 0x5629a8551fd0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.496+0000 7f95cd71c640 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8551fd0 0x5629a8562ab0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f95c40b9bd0 tx=0x7f95c4003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1d31c93ced4cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95b0ff9640 1 -- 172.21.15.22:0/24020048 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40bd900 con 0x5629a8551fd0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95b0ff9640 1 -- 172.21.15.22:0/24020048 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f95c40ee460 con 0x5629a8551fd0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95c40c1120 con 0x5629a8551fd0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95b0ff9640 1 -- 172.21.15.22:0/24020048 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c40c1350 con 0x5629a8551fd0 2024-03-09T00:01:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 10 client.17913 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5629a85c2cc0 con 0x5629a8551fd0 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95b0ff9640 1 -- 172.21.15.22:0/24020048 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c40c1120 con 0x5629a8551fd0 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95b0ff9640 1 -- 172.21.15.22:0/24020048 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95c403c030 con 0x5629a8551fd0 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 10 client.17913 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 10 client.17913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 20 client.17913 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a85c32f0 0x5629a85c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 4 client.17913 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629a84bde80 con 0x5629a85c32f0 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.497+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a85c32f0 0x5629a85c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.498+0000 7f95b0ff9640 10 client.17913 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.498+0000 7f95ccf1b640 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a85c32f0 0x5629a85c5710 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148711228ccff22e server_cookie=f1cf6e938b8f3b4f in_seq=0 out_seq=0 2024-03-09T00:01:45.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 758143 10.87 MB/sec execute 233 sec latency 198.307 ms 2024-03-09T00:01:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.795+0000 7f95b0ff9640 1 -- 172.21.15.22:0/24020048 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629a84bde80 con 0x5629a85c32f0 2024-03-09T00:01:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.795+0000 7f95b0ff9640 10 client.17913 handle_command_reply: tid=0 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.795+0000 7f95d2634740 1 client.17913 shutdown 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8551fd0 msgr2=0x5629a8562ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8551fd0 0x5629a8562ab0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f95c40b9bd0 tx=0x7f95c4003c30 comp rx=0 tx=0).stop 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 shutdown_connections 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629a85c32f0 0x5629a85c5710 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629a855b550 0x5629a8561480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95b0ff9640 0 client.17913 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629a8552d40 0x5629a8560e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 --2- 172.21.15.22:0/24020048 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a8551fd0 0x5629a8562ab0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 >> 172.21.15.22:0/24020048 conn(0x5629a85195b0 msgr2=0x5629a8556dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 shutdown_connections 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 1 -- 172.21.15.22:0/24020048 wait complete. 2024-03-09T00:01:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95d2634740 20 client.17913 trim_cache size 0 max 16384 2024-03-09T00:01:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:01:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:01:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:01:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95b40a2a10 msgr2=0x7f95b40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95b40a2a10 0x7f95b40a4ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f95c0005fd0 tx=0x7f95c000cc40 comp rx=0 tx=0).stop 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c80ff7f0 msgr2=0x7f95c819d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c80ff7f0 0x7f95c819d810 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f95c4004af0 tx=0x7f95c4032ea0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 shutdown_connections 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95b40a2a10 0x7f95b40a4ed0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95c81062d0 0x7f95c81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c81001b0 0x7f95c819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 --2- 172.21.15.22:0/4083200847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95c80ff7f0 0x7f95c819d810 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.796+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 >> 172.21.15.22:0/4083200847 conn(0x7f95c80767b0 msgr2=0x7f95c8104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.797+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 shutdown_connections 2024-03-09T00:01:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:45.797+0000 7f95cf980640 1 -- 172.21.15.22:0/4083200847 wait complete. 2024-03-09T00:01:46.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:46 smithi062 ceph-mon[36129]: pgmap v225: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-03-09T00:01:46.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:46 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:01:46.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:46 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:01:46.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:46 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:01:46.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:46 smithi192 ceph-mon[35995]: pgmap v225: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-03-09T00:01:46.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:46 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:01:46.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:46 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:01:46.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:46 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:01:46.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:46 smithi022 ceph-mon[31894]: pgmap v225: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-03-09T00:01:46.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:46 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:01:46.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:46 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:01:46.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:46 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:01:46.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 760099 10.88 MB/sec execute 234 sec latency 229.438 ms 2024-03-09T00:01:47.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 761515 10.86 MB/sec execute 235 sec latency 42.678 ms 2024-03-09T00:01:48.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:48 smithi062 ceph-mon[36129]: pgmap v226: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-09T00:01:48.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:48 smithi192 ceph-mon[35995]: pgmap v226: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-09T00:01:48.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:48 smithi022 ceph-mon[31894]: pgmap v226: 129 pgs: 129 active+clean; 86 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-03-09T00:01:48.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 763858 10.87 MB/sec execute 236 sec latency 14.515 ms 2024-03-09T00:01:49.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 765968 10.86 MB/sec execute 237 sec latency 17.673 ms 2024-03-09T00:01:50.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:50 smithi062 ceph-mon[36129]: pgmap v227: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-03-09T00:01:50.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:50 smithi192 ceph-mon[35995]: pgmap v227: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-03-09T00:01:50.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:50 smithi022 ceph-mon[31894]: pgmap v227: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-03-09T00:01:50.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 767871 10.86 MB/sec execute 238 sec latency 17.460 ms 2024-03-09T00:01:51.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 769955 10.87 MB/sec execute 239 sec latency 66.439 ms 2024-03-09T00:01:52.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:52 smithi062 ceph-mon[36129]: pgmap v228: 129 pgs: 129 active+clean; 76 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-03-09T00:01:52.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:52 smithi192 ceph-mon[35995]: pgmap v228: 129 pgs: 129 active+clean; 76 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-03-09T00:01:52.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:52 smithi022 ceph-mon[31894]: pgmap v228: 129 pgs: 129 active+clean; 76 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-03-09T00:01:52.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 771750 10.86 MB/sec execute 240 sec latency 16.867 ms 2024-03-09T00:01:53.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 774324 10.88 MB/sec execute 241 sec latency 15.749 ms 2024-03-09T00:01:54.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:54 smithi062 ceph-mon[36129]: pgmap v229: 129 pgs: 129 active+clean; 76 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-03-09T00:01:54.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:54 smithi192 ceph-mon[35995]: pgmap v229: 129 pgs: 129 active+clean; 76 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-03-09T00:01:54.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:54 smithi022 ceph-mon[31894]: pgmap v229: 129 pgs: 129 active+clean; 76 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-03-09T00:01:54.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 776233 10.86 MB/sec execute 242 sec latency 30.514 ms 2024-03-09T00:01:55.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 778252 10.87 MB/sec execute 243 sec latency 16.195 ms 2024-03-09T00:01:56.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:56 smithi062 ceph-mon[36129]: pgmap v230: 129 pgs: 129 active+clean; 83 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 430 op/s 2024-03-09T00:01:56.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:56 smithi192 ceph-mon[35995]: pgmap v230: 129 pgs: 129 active+clean; 83 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 430 op/s 2024-03-09T00:01:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:56 smithi022 ceph-mon[31894]: pgmap v230: 129 pgs: 129 active+clean; 83 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 430 op/s 2024-03-09T00:01:56.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 779968 10.86 MB/sec execute 244 sec latency 151.263 ms 2024-03-09T00:01:57.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 781860 10.86 MB/sec execute 245 sec latency 29.049 ms 2024-03-09T00:01:58.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:01:58.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:01:58.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 -- 172.21.15.22:0/3510047453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c100410 msgr2=0x7fa86c105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 --2- 172.21.15.22:0/3510047453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c100410 0x7fa86c105f40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa858009900 tx=0x7fa85802bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 -- 172.21.15.22:0/3510047453 shutdown_connections 2024-03-09T00:01:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 --2- 172.21.15.22:0/3510047453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c106670 0x7fa86c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 --2- 172.21.15.22:0/3510047453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c100410 0x7fa86c105f40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 --2- 172.21.15.22:0/3510047453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c0ffa60 0x7fa86c0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 -- 172.21.15.22:0/3510047453 >> 172.21.15.22:0/3510047453 conn(0x7fa86c0767b0 msgr2=0x7fa86c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 -- 172.21.15.22:0/3510047453 shutdown_connections 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 -- 172.21.15.22:0/3510047453 wait complete. 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 Processor -- start 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.264+0000 7fa872b0a640 1 -- start start 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa872b0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c0ffa60 0x7fa86c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa872b0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c106670 0x7fa86c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa872b0a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c197a00 0x7fa86c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa872b0a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1116c0 con 0x7fa86c106670 2024-03-09T00:01:58.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa872b0a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c111540 con 0x7fa86c0ffa60 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa872b0a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c111840 con 0x7fa86c197a00 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c106670 0x7fa86c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c106670 0x7fa86c19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38266/0 (socket says 172.21.15.22:38266) 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 -- 172.21.15.22:0/2909160658 learned_addr learned my addr 172.21.15.22:0/2909160658 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa8708a6640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c0ffa60 0x7fa86c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa8710a7640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c197a00 0x7fa86c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 -- 172.21.15.22:0/2909160658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c197a00 msgr2=0x7fa86c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c197a00 0x7fa86c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 -- 172.21.15.22:0/2909160658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c0ffa60 msgr2=0x7fa86c19d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c0ffa60 0x7fa86c19d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 -- 172.21.15.22:0/2909160658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c198520 con 0x7fa86c106670 2024-03-09T00:01:58.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa86bfff640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c106670 0x7fa86c19dd90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa85802c240 tx=0x7fa858004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa869ffb640 1 -- 172.21.15.22:0/2909160658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858043020 con 0x7fa86c106670 2024-03-09T00:01:58.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa869ffb640 1 -- 172.21.15.22:0/2909160658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8580054e0 con 0x7fa86c106670 2024-03-09T00:01:58.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c1987b0 con 0x7fa86c106670 2024-03-09T00:01:58.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.265+0000 7fa869ffb640 1 -- 172.21.15.22:0/2909160658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858003bb0 con 0x7fa86c106670 2024-03-09T00:01:58.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.266+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c1a4a00 con 0x7fa86c106670 2024-03-09T00:01:58.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.268+0000 7fa869ffb640 1 -- 172.21.15.22:0/2909160658 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa858046020 con 0x7fa86c106670 2024-03-09T00:01:58.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.268+0000 7fa8757c0740 1 Processor -- start 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.268+0000 7fa8757c0740 1 -- start start 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b13b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b1c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8757c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906a76e00 con 0x560906b14080 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8757c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906a1fb10 con 0x560906b13760 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8757c0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906a72ea0 con 0x560906b1cc60 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa869ffb640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa85c0a2970 0x7fa85c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b1c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b1c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38278/0 (socket says 172.21.15.22:38278) 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 -- 172.21.15.22:0/4154070628 learned_addr learned my addr 172.21.15.22:0/4154070628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8710a7640 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b13b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8710a7640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa85c0a2970 0x7fa85c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa869ffb640 1 -- 172.21.15.22:0/2909160658 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa8580f10d0 con 0x7fa86c106670 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 -- 172.21.15.22:0/4154070628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 msgr2=0x560906b1f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 -- 172.21.15.22:0/4154070628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 msgr2=0x560906b13b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b13b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 -- 172.21.15.22:0/4154070628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609068557d0 con 0x560906b14080 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8710a7640 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b13b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8708a6640 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b1c720 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa860009c90 tx=0x7fa860011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7f823c9a82a7a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa8710a7640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa85c0a2970 0x7fa85c0a4e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa864005c40 tx=0x7fa864009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa856ffd640 1 -- 172.21.15.22:0/4154070628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860004230 con 0x560906b14080 2024-03-09T00:01:58.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.269+0000 7fa856ffd640 1 -- 172.21.15.22:0/4154070628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8600043d0 con 0x560906b14080 2024-03-09T00:01:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa856ffd640 1 -- 172.21.15.22:0/4154070628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860019780 con 0x560906b14080 2024-03-09T00:01:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 -- 172.21.15.22:0/4154070628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 msgr2=0x560906b1c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b1c720 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa860009c90 tx=0x7fa860011810 comp rx=0 tx=0).stop 2024-03-09T00:01:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 -- 172.21.15.22:0/4154070628 shutdown_connections 2024-03-09T00:01:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b1c720 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 --2- 172.21.15.22:0/4154070628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b13b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 -- 172.21.15.22:0/4154070628 >> 172.21.15.22:0/4154070628 conn(0x560906b0de90 msgr2=0x560906b0e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 -- 172.21.15.22:0/4154070628 shutdown_connections 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.270+0000 7fa8757c0740 1 -- 172.21.15.22:0/4154070628 wait complete. 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 -- start start 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 Processor -- start 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b13760 0x560906b1c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b14080 0x560906b19750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b1cc60 0x560906b19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906b1fa50 con 0x560906b13760 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906a330a0 con 0x560906b1cc60 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906b1fbd0 con 0x560906b14080 2024-03-09T00:01:58.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8757c0740 10 client.0 _ll_register_callbacks cb 0x560906a32a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b13760 0x560906b1c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b13760 0x560906b1c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38284/0 (socket says 172.21.15.22:38284) 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 -- 172.21.15.22:0/576476378 learned_addr learned my addr 172.21.15.22:0/576476378 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa86bfff640 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b14080 0x560906b19750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8710a7640 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b1cc60 0x560906b19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 -- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b14080 msgr2=0x560906b19750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b14080 0x560906b19750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 -- 172.21.15.22:0/576476378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b1cc60 msgr2=0x560906b19c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b1cc60 0x560906b19c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.271+0000 7fa8708a6640 1 -- 172.21.15.22:0/576476378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56090678b3f0 con 0x560906b13760 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8708a6640 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b13760 0x560906b1c3c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa8600093f0 tx=0x7fa860005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa72c612008b333b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa84bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa84bfff640 1 -- 172.21.15.22:0/576476378 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860028020 con 0x560906b13760 2024-03-09T00:01:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa84bfff640 1 -- 172.21.15.22:0/576476378 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa8600037f0 con 0x560906b13760 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8600043d0 con 0x560906b13760 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa84bfff640 1 -- 172.21.15.22:0/576476378 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa860033050 con 0x560906b13760 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 10 client.17925 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56090674b0f0 con 0x560906b13760 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa84bfff640 1 -- 172.21.15.22:0/576476378 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8600043d0 con 0x560906b13760 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa84bfff640 1 -- 172.21.15.22:0/576476378 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa860018070 con 0x560906b13760 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 10 client.17925 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 20 client.17925 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b2c6b0 0x560906b2eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 4 client.17925 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.272+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560906a84340 con 0x560906b2c6b0 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.273+0000 7fa86bfff640 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b2c6b0 0x560906b2eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.273+0000 7fa86bfff640 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b2c6b0 0x560906b2eaa0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae7bde55d29b217 server_cookie=3d391bbc6bb43c16 in_seq=0 out_seq=0 2024-03-09T00:01:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.273+0000 7fa84bfff640 10 client.17925 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa84bfff640 1 -- 172.21.15.22:0/576476378 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560906a84340 con 0x560906b2c6b0 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa84bfff640 10 client.17925 handle_command_reply: tid=0 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 client.17925 shutdown 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b13760 msgr2=0x560906b1c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b13760 0x560906b1c3c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa8600093f0 tx=0x7fa860005da0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 shutdown_connections 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b2c6b0 0x560906b2eaa0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b1cc60 0x560906b19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b14080 0x560906b19750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 --2- 172.21.15.22:0/576476378 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b13760 0x560906b1c3c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa84bfff640 0 client.17925 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 >> 172.21.15.22:0/576476378 conn(0x560906b0de90 msgr2=0x560906b0fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 shutdown_connections 2024-03-09T00:01:58.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 1 -- 172.21.15.22:0/576476378 wait complete. 2024-03-09T00:01:58.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.274+0000 7fa8757c0740 20 client.17925 trim_cache size 0 max 16384 2024-03-09T00:01:58.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 Processor -- start 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 -- start start 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b19030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b14cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1b0c70 con 0x560906b14080 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1ae810 con 0x560906b13760 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8757c0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1ac8d0 con 0x560906b1cc60 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b14cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b14cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38290/0 (socket says 172.21.15.22:38290) 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 -- 172.21.15.22:0/1676030562 learned_addr learned my addr 172.21.15.22:0/1676030562 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8708a6640 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b19030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8710a7640 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 -- 172.21.15.22:0/1676030562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 msgr2=0x560906b1be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 -- 172.21.15.22:0/1676030562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 msgr2=0x560906b19030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b19030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 -- 172.21.15.22:0/1676030562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560906b17ca0 con 0x560906b14080 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8710a7640 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa86bfff640 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b14cb0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa85802c210 tx=0x7fa858090120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9d5879a9c41b66 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.290+0000 7fa8567fc640 1 -- 172.21.15.22:0/1676030562 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580bcda0 con 0x560906b14080 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8567fc640 1 -- 172.21.15.22:0/1676030562 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa858004c80 con 0x560906b14080 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8567fc640 1 -- 172.21.15.22:0/1676030562 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580c5be0 con 0x560906b14080 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 -- 172.21.15.22:0/1676030562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 msgr2=0x560906b14cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b14cb0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa85802c210 tx=0x7fa858090120 comp rx=0 tx=0).stop 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 -- 172.21.15.22:0/1676030562 shutdown_connections 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b1be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b14cb0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 --2- 172.21.15.22:0/1676030562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b19030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 -- 172.21.15.22:0/1676030562 >> 172.21.15.22:0/1676030562 conn(0x560906b0de90 msgr2=0x560906b145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 -- 172.21.15.22:0/1676030562 shutdown_connections 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.291+0000 7fa8757c0740 1 -- 172.21.15.22:0/1676030562 wait complete. 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 -- start start 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 Processor -- start 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b23ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b22290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1aced0 con 0x560906b14080 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1ae1b0 con 0x560906b13760 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1ad4f0 con 0x560906b1cc60 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8757c0740 10 client.0 _ll_register_callbacks cb 0x7fa8580b97d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b22290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b22290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38306/0 (socket says 172.21.15.22:38306) 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa86bfff640 1 -- 172.21.15.22:0/2885352252 learned_addr learned my addr 172.21.15.22:0/2885352252 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8708a6640 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b23ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa8710a7640 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa86bfff640 1 -- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 msgr2=0x560906b227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa86bfff640 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.292+0000 7fa86bfff640 1 -- 172.21.15.22:0/2885352252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 msgr2=0x560906b23ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa86bfff640 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b23ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa86bfff640 1 -- 172.21.15.22:0/2885352252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560906b22d10 con 0x560906b14080 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa86bfff640 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b22290 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa8580ee5e0 tx=0x7fa858003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ff86b7ddceda78 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa84bfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa84bfff640 1 -- 172.21.15.22:0/2885352252 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580bd440 con 0x560906b14080 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa84bfff640 1 -- 172.21.15.22:0/2885352252 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa85802cb90 con 0x560906b14080 2024-03-09T00:01:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa858004c80 con 0x560906b14080 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa84bfff640 1 -- 172.21.15.22:0/2885352252 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8580bdb40 con 0x560906b14080 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 10 client.17931 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560906b837a0 con 0x560906b14080 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa84bfff640 1 -- 172.21.15.22:0/2885352252 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa858004c80 con 0x560906b14080 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa84bfff640 1 -- 172.21.15.22:0/2885352252 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa8580ee6a0 con 0x560906b14080 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 10 client.17931 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 10 client.17931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 20 client.17931 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b83e30 0x560906b86250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 4 client.17931 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.293+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560906a7e690 con 0x560906b83e30 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa8708a6640 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b83e30 0x560906b86250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa84bfff640 10 client.17931 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa8708a6640 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b83e30 0x560906b86250 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5704b3cf0f6046b1 server_cookie=96c504bd6662bd79 in_seq=0 out_seq=0 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa84bfff640 1 -- 172.21.15.22:0/2885352252 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560906a7e690 con 0x560906b83e30 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa84bfff640 10 client.17931 handle_command_reply: tid=0 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa8757c0740 1 client.17931 shutdown 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 msgr2=0x560906b22290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.294+0000 7fa8757c0740 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b22290 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa8580ee5e0 tx=0x7fa858003680 comp rx=0 tx=0).stop 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 shutdown_connections 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560906b83e30 0x560906b86250 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560906b1cc60 0x560906b227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560906b14080 0x560906b22290 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 --2- 172.21.15.22:0/2885352252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560906b13760 0x560906b23ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa84bfff640 0 client.17931 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 >> 172.21.15.22:0/2885352252 conn(0x560906b0de90 msgr2=0x560906b0c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 shutdown_connections 2024-03-09T00:01:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 1 -- 172.21.15.22:0/2885352252 wait complete. 2024-03-09T00:01:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa8757c0740 20 client.17931 trim_cache size 0 max 16384 2024-03-09T00:01:58.300 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:01:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:01:58.300 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:01:58.300 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:01:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa85c0a2970 msgr2=0x7fa85c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa872b0a640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa85c0a2970 0x7fa85c0a4e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa864005c40 tx=0x7fa864009500 comp rx=0 tx=0).stop 2024-03-09T00:01:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c106670 msgr2=0x7fa86c19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.295+0000 7fa872b0a640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c106670 0x7fa86c19dd90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa85802c240 tx=0x7fa858004360 comp rx=0 tx=0).stop 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 shutdown_connections 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa85c0a2970 0x7fa85c0a4e30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa86c197a00 0x7fa86c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa86c106670 0x7fa86c19dd90 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 --2- 172.21.15.22:0/2909160658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86c0ffa60 0x7fa86c19d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 >> 172.21.15.22:0/2909160658 conn(0x7fa86c0767b0 msgr2=0x7fa86c1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 shutdown_connections 2024-03-09T00:01:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.296+0000 7fa872b0a640 1 -- 172.21.15.22:0/2909160658 wait complete. 2024-03-09T00:01:58.304 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:01:58.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:01:58.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:58 smithi022 ceph-mon[31894]: pgmap v231: 129 pgs: 129 active+clean; 83 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-03-09T00:01:58.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:58 smithi062 ceph-mon[36129]: pgmap v231: 129 pgs: 129 active+clean; 83 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-03-09T00:01:58.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:58 smithi192 ceph-mon[35995]: pgmap v231: 129 pgs: 129 active+clean; 83 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.413+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3112513693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70100510 msgr2=0x7f4d701008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.413+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3112513693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70100510 0x7f4d701008f0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009900 tx=0x7f4d6802bc30 comp rx=0 tx=0).stop 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3112513693 shutdown_connections 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3112513693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70105330 0x7f4d7010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3112513693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70100ec0 0x7f4d70104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3112513693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70100510 0x7f4d701008f0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3112513693 >> 172.21.15.22:0/3112513693 conn(0x7f4d70075c00 msgr2=0x7f4d70076010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3112513693 shutdown_connections 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3112513693 wait complete. 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 Processor -- start 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.414+0000 7f4d6f59e640 1 -- start start 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70100510 0x7f4d70198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70105330 0x7f4d70198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70199310 0x7f4d701a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d701103e0 con 0x7f4d70199310 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70110260 con 0x7f4d70105330 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70110560 con 0x7f4d70100510 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70199310 0x7f4d701a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70199310 0x7f4d701a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38328/0 (socket says 172.21.15.22:38328) 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70100510 0x7f4d70198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 -- 172.21.15.22:0/3321041545 learned_addr learned my addr 172.21.15.22:0/3321041545 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6dd9b640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70105330 0x7f4d70198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 -- 172.21.15.22:0/3321041545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70100510 msgr2=0x7f4d70198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70100510 0x7f4d70198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 -- 172.21.15.22:0/3321041545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70105330 msgr2=0x7f4d70198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70105330 0x7f4d70198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 -- 172.21.15.22:0/3321041545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d701a4090 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6ed9d640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70199310 0x7f4d701a38c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000eb10 tx=0x7f4d6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d677fe640 1 -- 172.21.15.22:0/3321041545 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004b20 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d677fe640 1 -- 172.21.15.22:0/3321041545 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d60004cc0 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d701a4380 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d677fe640 1 -- 172.21.15.22:0/3321041545 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60010760 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.415+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d701a4880 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.416+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d70068000 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.418+0000 7f4d677fe640 1 -- 172.21.15.22:0/3321041545 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4d60005000 con 0x7f4d70199310 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.418+0000 7f4d677fe640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d5c0a2970 0x7f4d5c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.418+0000 7f4d6e59c640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d5c0a2970 0x7f4d5c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.418+0000 7f4d677fe640 1 -- 172.21.15.22:0/3321041545 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4d600cf990 con 0x7f4d70199310 2024-03-09T00:01:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.419+0000 7f4d6e59c640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d5c0a2970 0x7f4d5c0a4e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006010 tx=0x7f4d68005dd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.423+0000 7f4d677fe640 1 -- 172.21.15.22:0/3321041545 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4d60053260 con 0x7f4d70199310 2024-03-09T00:01:58.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.657+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d70100ec0 con 0x7f4d70199310 2024-03-09T00:01:58.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.658+0000 7f4d677fe640 1 -- 172.21.15.22:0/3321041545 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f4d6009c2f0 con 0x7f4d70199310 2024-03-09T00:01:58.653 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:01:58.653 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-08T23:58:51.502302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26555":{"gid":26555,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2421521561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2421521561},{"type":"v1","addr":"172.21.15.62:6837","nonce":2421521561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:01:58.654 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2024-03-09T00:01:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d5c0a2970 msgr2=0x7f4d5c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d5c0a2970 0x7f4d5c0a4e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006010 tx=0x7f4d68005dd0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70199310 msgr2=0x7f4d701a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70199310 0x7f4d701a38c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000eb10 tx=0x7f4d6000c2d0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 shutdown_connections 2024-03-09T00:01:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4d5c0a2970 0x7f4d5c0a4e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4d70199310 0x7f4d701a38c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70105330 0x7f4d70198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 --2- 172.21.15.22:0/3321041545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70100510 0x7f4d70198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 >> 172.21.15.22:0/3321041545 conn(0x7f4d70075c00 msgr2=0x7f4d70106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 shutdown_connections 2024-03-09T00:01:58.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.660+0000 7f4d6f59e640 1 -- 172.21.15.22:0/3321041545 wait complete. 2024-03-09T00:01:58.667 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:01:58.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:01:58.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 784406 10.88 MB/sec execute 246 sec latency 13.225 ms 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 -- 172.21.15.22:0/3360960143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200106600 msgr2=0x7fa20010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 --2- 172.21.15.22:0/3360960143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200106600 0x7fa20010e1d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009ca0 tx=0x7fa1f802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 -- 172.21.15.22:0/3360960143 shutdown_connections 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 --2- 172.21.15.22:0/3360960143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa200106600 0x7fa20010e1d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 --2- 172.21.15.22:0/3360960143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa200101ee0 0x7fa200105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 --2- 172.21.15.22:0/3360960143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa200101530 0x7fa200101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 -- 172.21.15.22:0/3360960143 >> 172.21.15.22:0/3360960143 conn(0x7fa2000fcd90 msgr2=0x7fa2000ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 -- 172.21.15.22:0/3360960143 shutdown_connections 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.824+0000 7fa204a7e640 1 -- 172.21.15.22:0/3360960143 wait complete. 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 Processor -- start 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 -- start start 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa200101530 0x7fa20019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa200106600 0x7fa20019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2001979c0 0x7fa200197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001116c0 con 0x7fa2001979c0 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa200111540 con 0x7fa200106600 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa204a7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa200111840 con 0x7fa200101530 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa1fed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2001979c0 0x7fa200197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa1fed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2001979c0 0x7fa200197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38346/0 (socket says 172.21.15.22:38346) 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa1fed9d640 1 -- 172.21.15.22:0/2534302859 learned_addr learned my addr 172.21.15.22:0/2534302859 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa200101530 0x7fa20019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.825+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa200106600 0x7fa20019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1fed9d640 1 -- 172.21.15.22:0/2534302859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa200101530 msgr2=0x7fa20019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1fed9d640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa200101530 0x7fa20019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1fed9d640 1 -- 172.21.15.22:0/2534302859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa200106600 msgr2=0x7fa20019dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1fed9d640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa200106600 0x7fa20019dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1fed9d640 1 -- 172.21.15.22:0/2534302859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2001984e0 con 0x7fa2001979c0 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1fed9d640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2001979c0 0x7fa200197e70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa1f802c700 tx=0x7fa1f8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1f77fe640 1 -- 172.21.15.22:0/2534302859 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8041020 con 0x7fa2001979c0 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa200198770 con 0x7fa2001979c0 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1f77fe640 1 -- 172.21.15.22:0/2534302859 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8034cd0 con 0x7fa2001979c0 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2001a4970 con 0x7fa2001979c0 2024-03-09T00:01:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.826+0000 7fa1f77fe640 1 -- 172.21.15.22:0/2534302859 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8039040 con 0x7fa2001979c0 2024-03-09T00:01:58.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.828+0000 7fa1f77fe640 1 -- 172.21.15.22:0/2534302859 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa1f8044020 con 0x7fa2001979c0 2024-03-09T00:01:58.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.828+0000 7fa207658740 1 Processor -- start 2024-03-09T00:01:58.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa207658740 1 -- start start 2024-03-09T00:01:58.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa207658740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00051580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa207658740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec00051ac0 0x55ec0005a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa207658740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec0005a6a0 0x55ec0005ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa207658740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec00018360 con 0x55ec0005a6a0 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa207658740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfffb4ec0 con 0x55ec00051ac0 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa207658740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfff5dd30 con 0x55ec000511a0 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00051580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00051580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35920/0 (socket says 172.21.15.22:35920) 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 -- 172.21.15.22:0/2269528267 learned_addr learned my addr 172.21.15.22:0/2269528267 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec0005a6a0 0x55ec0005ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec00051ac0 0x55ec0005a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f77fe640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa1ec0a2970 0x7fa1ec0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 -- 172.21.15.22:0/2269528267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec00051ac0 msgr2=0x55ec0005a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec00051ac0 0x55ec0005a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 -- 172.21.15.22:0/2269528267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec0005a6a0 msgr2=0x55ec0005ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.829+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec0005a6a0 0x55ec0005ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1f7fff640 1 -- 172.21.15.22:0/2269528267 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebfff5cbc0 con 0x55ec000511a0 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec00051ac0 0x55ec0005a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1f77fe640 1 -- 172.21.15.22:0/2534302859 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa1f80f0dd0 con 0x7fa2001979c0 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1fed9d640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa1ec0a2970 0x7fa1ec0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00051580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009710 tx=0x7fa1e80096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634070d3aa5e85b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1f4ff9640 1 -- 172.21.15.22:0/2269528267 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8004050 con 0x55ec000511a0 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1f4ff9640 1 -- 172.21.15.22:0/2269528267 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8005500 con 0x55ec000511a0 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.830+0000 7fa1f4ff9640 1 -- 172.21.15.22:0/2269528267 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8004d10 con 0x55ec000511a0 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- 172.21.15.22:0/2269528267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 msgr2=0x55ec00051580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00051580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009710 tx=0x7fa1e80096a0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa1fed9d640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa1ec0a2970 0x7fa1ec0a4e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80ee2e0 tx=0x7fa1f80324b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- 172.21.15.22:0/2269528267 shutdown_connections 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec0005a6a0 0x55ec0005ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec00051ac0 0x55ec0005a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 --2- 172.21.15.22:0/2269528267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00051580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- 172.21.15.22:0/2269528267 >> 172.21.15.22:0/2269528267 conn(0x55ec0004b8d0 msgr2=0x55ec0004bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- 172.21.15.22:0/2269528267 shutdown_connections 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- 172.21.15.22:0/2269528267 wait complete. 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- start start 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 Processor -- start 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec000511a0 0x55ec00057790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec00051ac0 0x55ec00057cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00058210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfff534a0 con 0x55ec000511a0 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebfffb0f60 con 0x55ec0005a6a0 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.831+0000 7fa207658740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec0005d490 con 0x55ec00051ac0 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa207658740 10 client.0 _ll_register_callbacks cb 0x55ebfffaed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec000511a0 0x55ec00057790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec000511a0 0x55ec00057790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38360/0 (socket says 172.21.15.22:38360) 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 -- 172.21.15.22:0/382205637 learned_addr learned my addr 172.21.15.22:0/382205637 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec00051ac0 0x55ec00057cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00058210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 -- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec00051ac0 msgr2=0x55ec00057cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec00051ac0 0x55ec00057cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 -- 172.21.15.22:0/382205637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 msgr2=0x55ec00058210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00058210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 -- 172.21.15.22:0/382205637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebffcc93f0 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec00051ac0 0x55ec00057cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec000511a0 0x55ec00057790 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009950 tx=0x7fa1e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2dcc84bee74828 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/382205637 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800f7c0 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/382205637 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa1e800f960 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1e8005500 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/382205637 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1e8034050 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa207658740 10 client.17949 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.832+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ebffd937d0 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/382205637 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8005500 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/382205637 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa1e801f8c0 con 0x55ec000511a0 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa207658740 10 client.17949 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa207658740 10 client.17949 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:01:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa207658740 20 client.17949 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa207658740 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec0006a7f0 0x55ec0006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa207658740 4 client.17949 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebfffc2400 con 0x55ec0006a7f0 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec0006a7f0 0x55ec0006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa1e5ffb640 10 client.17949 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.833+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec0006a7f0 0x55ec0006cbe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e97919f8f54fae99 server_cookie=ad97169ffe432980 in_seq=0 out_seq=0 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/382205637 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ebfffc2400 con 0x55ec0006a7f0 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa1e5ffb640 10 client.17949 handle_command_reply: tid=0 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 client.17949 shutdown 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec000511a0 msgr2=0x55ec00057790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec000511a0 0x55ec00057790 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009950 tx=0x7fa1e8005fd0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 shutdown_connections 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec0006a7f0 0x55ec0006cbe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00058210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec00051ac0 0x55ec00057cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa1e5ffb640 0 client.17949 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:58.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 --2- 172.21.15.22:0/382205637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec000511a0 0x55ec00057790 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 >> 172.21.15.22:0/382205637 conn(0x55ec0004b8d0 msgr2=0x55ec0004d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 shutdown_connections 2024-03-09T00:01:58.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 1 -- 172.21.15.22:0/382205637 wait complete. 2024-03-09T00:01:58.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.834+0000 7fa207658740 20 client.17949 trim_cache size 0 max 16384 2024-03-09T00:01:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 Processor -- start 2024-03-09T00:01:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 -- start start 2024-03-09T00:01:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00059ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec000549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00054f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001b0c30 con 0x55ec00051ac0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001ae7d0 con 0x55ec0005a6a0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa207658740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001ac890 con 0x55ec000511a0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec000549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00059ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec000549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38374/0 (socket says 172.21.15.22:38374) 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 -- 172.21.15.22:0/2005942338 learned_addr learned my addr 172.21.15.22:0/2005942338 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00054f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 -- 172.21.15.22:0/2005942338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 msgr2=0x55ec00059ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00059ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 -- 172.21.15.22:0/2005942338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 msgr2=0x55ec00054f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00054f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 -- 172.21.15.22:0/2005942338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec0004b110 con 0x55ec00051ac0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00059ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.847+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec000549f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000c00 tx=0x7fa1f0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6395f5728626ff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa1e7fff640 1 -- 172.21.15.22:0/2005942338 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0004050 con 0x55ec00051ac0 2024-03-09T00:01:58.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa1e7fff640 1 -- 172.21.15.22:0/2005942338 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1f000fdd0 con 0x55ec00051ac0 2024-03-09T00:01:58.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa1e7fff640 1 -- 172.21.15.22:0/2005942338 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0021670 con 0x55ec00051ac0 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 -- 172.21.15.22:0/2005942338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 msgr2=0x55ec000549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec000549f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000c00 tx=0x7fa1f0005f90 comp rx=0 tx=0).stop 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 -- 172.21.15.22:0/2005942338 shutdown_connections 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec00054f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec000549f0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 --2- 172.21.15.22:0/2005942338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec00059ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 -- 172.21.15.22:0/2005942338 >> 172.21.15.22:0/2005942338 conn(0x55ec0004b8d0 msgr2=0x55ec00052a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.848+0000 7fa207658740 1 -- 172.21.15.22:0/2005942338 shutdown_connections 2024-03-09T00:01:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 -- 172.21.15.22:0/2005942338 wait complete. 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 -- start start 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 Processor -- start 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec000616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec00061be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec0005fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001ace90 con 0x55ec00051ac0 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001ae170 con 0x55ec0005a6a0 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001ad4b0 con 0x55ec000511a0 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa207658740 10 client.0 _ll_register_callbacks cb 0x7fa1e80099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec00061be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec00061be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38378/0 (socket says 172.21.15.22:38378) 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1fe59c640 1 -- 172.21.15.22:0/1629582158 learned_addr learned my addr 172.21.15.22:0/1629582158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec000616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec0005fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1fe59c640 1 -- 172.21.15.22:0/1629582158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 msgr2=0x55ec000616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec000616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1fe59c640 1 -- 172.21.15.22:0/1629582158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 msgr2=0x55ec0005fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.849+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec0005fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1fe59c640 1 -- 172.21.15.22:0/1629582158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec00060440 con 0x55ec00051ac0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1fe59c640 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec00061be0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009e20 tx=0x7fa1f0009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f96023b04818a2a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/1629582158 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f002e070 con 0x55ec00051ac0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/1629582158 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa1f000faf0 con 0x55ec00051ac0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1f000fdd0 con 0x55ec00051ac0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/1629582158 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1f0004020 con 0x55ec00051ac0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 10 client.17955 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ec000c18e0 con 0x55ec00051ac0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/1629582158 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1f000fdd0 con 0x55ec00051ac0 2024-03-09T00:01:58.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/1629582158 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa1f0003710 con 0x55ec00051ac0 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 10 client.17955 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 10 client.17955 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 20 client.17955 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec000c1f50 0x55ec000c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 4 client.17955 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.850+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ebfffbc750 con 0x55ec000c1f50 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec000c1f50 0x55ec000c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa1e5ffb640 10 client.17955 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa1f7fff640 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec000c1f50 0x55ec000c4370 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62365d913599ebe6 server_cookie=daaeca64a81ba100 in_seq=0 out_seq=0 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa1e5ffb640 1 -- 172.21.15.22:0/1629582158 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ebfffbc750 con 0x55ec000c1f50 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa1e5ffb640 10 client.17955 handle_command_reply: tid=0 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 client.17955 shutdown 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 msgr2=0x55ec00061be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec00061be0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009e20 tx=0x7fa1f0009e50 comp rx=0 tx=0).stop 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 shutdown_connections 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec000c1f50 0x55ec000c4370 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec0005a6a0 0x55ec0005fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec00051ac0 0x55ec00061be0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa1e5ffb640 0 client.17955 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 --2- 172.21.15.22:0/1629582158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec000511a0 0x55ec000616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 >> 172.21.15.22:0/1629582158 conn(0x55ec0004b8d0 msgr2=0x55ec00056060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 shutdown_connections 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.851+0000 7fa207658740 1 -- 172.21.15.22:0/1629582158 wait complete. 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa207658740 20 client.17955 trim_cache size 0 max 16384 2024-03-09T00:01:58.853 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:01:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa1ec0a2970 msgr2=0x7fa1ec0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa1ec0a2970 0x7fa1ec0a4e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80ee2e0 tx=0x7fa1f80324b0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2001979c0 msgr2=0x7fa200197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2001979c0 0x7fa200197e70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa1f802c700 tx=0x7fa1f8032f20 comp rx=0 tx=0).stop 2024-03-09T00:01:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 shutdown_connections 2024-03-09T00:01:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa1ec0a2970 0x7fa1ec0a4e30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2001979c0 0x7fa200197e70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa200106600 0x7fa20019dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 --2- 172.21.15.22:0/2534302859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa200101530 0x7fa20019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 >> 172.21.15.22:0/2534302859 conn(0x7fa2000fcd90 msgr2=0x7fa2000fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 shutdown_connections 2024-03-09T00:01:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:58.852+0000 7fa204a7e640 1 -- 172.21.15.22:0/2534302859 wait complete. 2024-03-09T00:01:58.859 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:01:58.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:01:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 -- 172.21.15.22:0/3487972999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 msgr2=0x7f726810ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 --2- 172.21.15.22:0/3487972999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f726810ff20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7260009ca0 tx=0x7f726002c180 comp rx=0 tx=0).stop 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 -- 172.21.15.22:0/3487972999 shutdown_connections 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 --2- 172.21.15.22:0/3487972999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f726810ff20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 --2- 172.21.15.22:0/3487972999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7268074040 0x7f726810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 --2- 172.21.15.22:0/3487972999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7268073720 0x7f7268073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 -- 172.21.15.22:0/3487972999 >> 172.21.15.22:0/3487972999 conn(0x7f72680fcd80 msgr2=0x7f72680ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 -- 172.21.15.22:0/3487972999 shutdown_connections 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 -- 172.21.15.22:0/3487972999 wait complete. 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.890+0000 7f726f7c4640 1 Processor -- start 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726f7c4640 1 -- start start 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726f7c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7268073720 0x7f726819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726f7c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7268074040 0x7f726819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726f7c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f7268199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726f7c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7268113850 con 0x7f726810dad0 2024-03-09T00:01:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726f7c4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681136d0 con 0x7f7268073720 2024-03-09T00:01:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726f7c4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681139d0 con 0x7f7268074040 2024-03-09T00:01:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f7268199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f7268199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38394/0 (socket says 172.21.15.22:38394) 2024-03-09T00:01:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 -- 172.21.15.22:0/966701249 learned_addr learned my addr 172.21.15.22:0/966701249 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726d560640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7268073720 0x7f726819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726cd5f640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7268074040 0x7f726819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 -- 172.21.15.22:0/966701249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7268074040 msgr2=0x7f726819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7268074040 0x7f726819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 -- 172.21.15.22:0/966701249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7268073720 msgr2=0x7f726819f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7268073720 0x7f726819f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 -- 172.21.15.22:0/966701249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726819a250 con 0x7f726810dad0 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726d560640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7268073720 0x7f726819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.891+0000 7f726dd61640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f7268199ad0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f726002c790 tx=0x7f7260035e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.892+0000 7f725e7fc640 1 -- 172.21.15.22:0/966701249 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260039040 con 0x7f726810dad0 2024-03-09T00:01:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.892+0000 7f725e7fc640 1 -- 172.21.15.22:0/966701249 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7260044070 con 0x7f726810dad0 2024-03-09T00:01:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.892+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726819a4e0 con 0x7f726810dad0 2024-03-09T00:01:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.892+0000 7f725e7fc640 1 -- 172.21.15.22:0/966701249 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260032940 con 0x7f726810dad0 2024-03-09T00:01:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.892+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72681a67e0 con 0x7f726810dad0 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.894+0000 7f72723a3740 1 Processor -- start 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.894+0000 7f725e7fc640 1 -- 172.21.15.22:0/966701249 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7260032ae0 con 0x7f726810dad0 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.894+0000 7f72723a3740 1 -- start start 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f72723a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f72723a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f72723a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f72723a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae023195f0 con 0x55ae023be550 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f72723a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae022c2930 con 0x55ae023b4fd0 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f72723a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae02315690 con 0x55ae023b5d40 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35960/0 (socket says 172.21.15.22:35960) 2024-03-09T00:01:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 -- 172.21.15.22:0/2555818568 learned_addr learned my addr 172.21.15.22:0/2555818568 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726d560640 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f725e7fc640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f72540a2a10 0x7f72540a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 -- 172.21.15.22:0/2555818568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 msgr2=0x55ae023b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 -- 172.21.15.22:0/2555818568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 msgr2=0x55ae023c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 -- 172.21.15.22:0/2555818568 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae0202e210 con 0x55ae023b5d40 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726dd61640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f72540a2a10 0x7f72540a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f725e7fc640 1 -- 172.21.15.22:0/966701249 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f72600f14d0 con 0x7f726810dad0 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.895+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023b4a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72580058c0 tx=0x7f7258009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27318a8fae766ae server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72537fe640 1 -- 172.21.15.22:0/2555818568 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258004030 con 0x55ae023b5d40 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72537fe640 1 -- 172.21.15.22:0/2555818568 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f725800fca0 con 0x55ae023b5d40 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72537fe640 1 -- 172.21.15.22:0/2555818568 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258003880 con 0x55ae023b5d40 2024-03-09T00:01:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 -- 172.21.15.22:0/2555818568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 msgr2=0x55ae023b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f726dd61640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f72540a2a10 0x7f72540a4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72600eea80 tx=0x7f72600b9610 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023b4a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72580058c0 tx=0x7f7258009770 comp rx=0 tx=0).stop 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 -- 172.21.15.22:0/2555818568 shutdown_connections 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 --2- 172.21.15.22:0/2555818568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023b4a90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 -- 172.21.15.22:0/2555818568 >> 172.21.15.22:0/2555818568 conn(0x55ae0237c5b0 msgr2=0x55ae023afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 -- 172.21.15.22:0/2555818568 shutdown_connections 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.896+0000 7f72723a3740 1 -- 172.21.15.22:0/2555818568 wait complete. 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 -- start start 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 Processor -- start 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b5d40 0x55ae023bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023be550 0x55ae023bbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae023c1280 con 0x55ae023b5d40 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae022d5d90 con 0x55ae023b4fd0 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae023c1400 con 0x55ae023be550 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f72723a3740 10 client.0 _ll_register_callbacks cb 0x55ae0202da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f726cd5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023be550 0x55ae023bbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f726d560640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b5d40 0x55ae023bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f726cd5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023be550 0x55ae023bbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35962/0 (socket says 172.21.15.22:35962) 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.897+0000 7f726cd5f640 1 -- 172.21.15.22:0/2077076101 learned_addr learned my addr 172.21.15.22:0/2077076101 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726cd5f640 1 -- 172.21.15.22:0/2077076101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 msgr2=0x55ae023bb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023bb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726cd5f640 1 -- 172.21.15.22:0/2077076101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b5d40 msgr2=0x55ae023bba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b5d40 0x55ae023bba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726cd5f640 1 -- 172.21.15.22:0/2077076101 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae020fa620 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726d560640 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b5d40 0x55ae023bba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f726cd5f640 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023be550 0x55ae023bbfd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72580094b0 tx=0x7f7258009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d781fdf1d090566 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f7250ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f7250ff9640 1 -- 172.21.15.22:0/2077076101 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258020490 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f7250ff9640 1 -- 172.21.15.22:0/2077076101 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7258020630 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f725800fca0 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f7250ff9640 1 -- 172.21.15.22:0/2077076101 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7258035050 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f72723a3740 10 client.27406 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.898+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ae023cd530 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f7250ff9640 1 -- 172.21.15.22:0/2077076101 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f725800fca0 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f7250ff9640 1 -- 172.21.15.22:0/2077076101 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f725800e850 con 0x55ae023be550 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f72723a3740 10 client.27406 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f72723a3740 10 client.27406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f72723a3740 20 client.27406 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f72723a3740 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae023d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f72723a3740 4 client.27406 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae02326b30 con 0x55ae023ce880 2024-03-09T00:01:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f726d560640 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae023d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f726d560640 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae023d0c70 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=279fe539c6855b10 server_cookie=e415d06897f773cd in_seq=0 out_seq=0 2024-03-09T00:01:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.899+0000 7f7250ff9640 10 client.27406 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.902+0000 7f7250ff9640 1 -- 172.21.15.22:0/2077076101 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae02326b30 con 0x55ae023ce880 2024-03-09T00:01:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.902+0000 7f7250ff9640 10 client.27406 handle_command_reply: tid=0 2024-03-09T00:01:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.902+0000 7f72723a3740 1 client.27406 shutdown 2024-03-09T00:01:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023be550 msgr2=0x55ae023bbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023be550 0x55ae023bbfd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72580094b0 tx=0x7f7258009770 comp rx=0 tx=0).stop 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 shutdown_connections 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae023d0c70 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f7250ff9640 0 client.27406 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023be550 0x55ae023bbfd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b5d40 0x55ae023bba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 --2- 172.21.15.22:0/2077076101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b4fd0 0x55ae023bb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 >> 172.21.15.22:0/2077076101 conn(0x55ae0237c5b0 msgr2=0x55ae023b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 shutdown_connections 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 1 -- 172.21.15.22:0/2077076101 wait complete. 2024-03-09T00:01:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.903+0000 7f72723a3740 20 client.27406 trim_cache size 0 max 16384 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 Processor -- start 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 -- start start 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b4fd0 0x55ae023bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b5d40 0x55ae023b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681b2d40 con 0x55ae023be550 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681b08e0 con 0x55ae023b5d40 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.917+0000 7f72723a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681ae9a0 con 0x55ae023b4fd0 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b5d40 0x55ae023b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b5d40 0x55ae023b9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:46756/0 (socket says 172.21.15.22:46756) 2024-03-09T00:01:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726cd5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 -- 172.21.15.22:0/4265066693 learned_addr learned my addr 172.21.15.22:0/4265066693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726cd5f640 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b4fd0 0x55ae023bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 -- 172.21.15.22:0/4265066693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b4fd0 msgr2=0x55ae023bdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b4fd0 0x55ae023bdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 -- 172.21.15.22:0/4265066693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 msgr2=0x55ae023ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023ba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 -- 172.21.15.22:0/4265066693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae023b42a0 con 0x55ae023b5d40 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726cd5f640 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f726d560640 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b5d40 0x55ae023b9e40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55ae023badb0 tx=0x7f7264005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec2b000ac57726eb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f7252ffd640 1 -- 172.21.15.22:0/4265066693 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264012110 con 0x55ae023b5d40 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f7252ffd640 1 -- 172.21.15.22:0/4265066693 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7264025030 con 0x55ae023b5d40 2024-03-09T00:01:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.918+0000 7f7252ffd640 1 -- 172.21.15.22:0/4265066693 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72640047a0 con 0x55ae023b5d40 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 -- 172.21.15.22:0/4265066693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b5d40 msgr2=0x55ae023b9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b5d40 0x55ae023b9e40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55ae023badb0 tx=0x7f7264005f90 comp rx=0 tx=0).stop 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 -- 172.21.15.22:0/4265066693 shutdown_connections 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023be550 0x55ae023ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023b5d40 0x55ae023b9e40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 --2- 172.21.15.22:0/4265066693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b4fd0 0x55ae023bdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 -- 172.21.15.22:0/4265066693 >> 172.21.15.22:0/4265066693 conn(0x55ae0237c5b0 msgr2=0x55ae023d54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 -- 172.21.15.22:0/4265066693 shutdown_connections 2024-03-09T00:01:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 -- 172.21.15.22:0/4265066693 wait complete. 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 -- start start 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.919+0000 7f72723a3740 1 Processor -- start 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f72723a3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b4fd0 0x55ae023c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f72723a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f72723a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023be550 0x55ae023c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f72723a3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681aefa0 con 0x55ae023b4fd0 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f72723a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681b0280 con 0x55ae023be550 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f72723a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681af5c0 con 0x55ae023b5d40 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f72723a3740 10 client.0 _ll_register_callbacks cb 0x7f7258009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b4fd0 0x55ae023c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726d560640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b4fd0 0x55ae023c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38432/0 (socket says 172.21.15.22:38432) 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 -- 172.21.15.22:0/3656579013 learned_addr learned my addr 172.21.15.22:0/3656579013 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023be550 0x55ae023c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 -- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 msgr2=0x55ae023c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 -- 172.21.15.22:0/3656579013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023be550 msgr2=0x55ae023c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023be550 0x55ae023c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726cd5f640 1 -- 172.21.15.22:0/3656579013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae023c4810 con 0x55ae023b4fd0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.920+0000 7f726d560640 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f726cd5f640 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b4fd0 0x55ae023c58a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7258006010 tx=0x7f72580045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b633034ac576c11 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f7250ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f7250ff9640 1 -- 172.21.15.22:0/3656579013 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725803f060 con 0x55ae023b4fd0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f7250ff9640 1 -- 172.21.15.22:0/3656579013 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7258005030 con 0x55ae023b4fd0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7264025030 con 0x55ae023b4fd0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f7250ff9640 1 -- 172.21.15.22:0/3656579013 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7258009850 con 0x55ae023b4fd0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f72723a3740 10 client.17973 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ae02425ac0 con 0x55ae023b4fd0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f7250ff9640 1 -- 172.21.15.22:0/3656579013 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7264025030 con 0x55ae023b4fd0 2024-03-09T00:01:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f7250ff9640 1 -- 172.21.15.22:0/3656579013 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f725800e550 con 0x55ae023b4fd0 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f72723a3740 10 client.17973 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f72723a3740 10 client.17973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.921+0000 7f72723a3740 20 client.17973 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.922+0000 7f72723a3740 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae024281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.922+0000 7f72723a3740 4 client.17973 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.922+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae02320e80 con 0x55ae023ce880 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.922+0000 7f726d560640 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae024281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.922+0000 7f7250ff9640 10 client.17973 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:58.922+0000 7f726d560640 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae024281d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=244e1b48768aa4ff server_cookie=a9813cbca172bf0c in_seq=0 out_seq=0 2024-03-09T00:01:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.042+0000 7f5195ffb640 1 -- 172.21.15.22:0/303251674 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190033880 con 0x7f5198100300 2024-03-09T00:01:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.043+0000 7f519d2a7640 1 -- 172.21.15.22:0/303251674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5198100300 msgr2=0x7f51981006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.043+0000 7f519d2a7640 1 --2- 172.21.15.22:0/303251674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5198100300 0x7f51981006e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5190009be0 tx=0x7f519002bfa0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.047+0000 7f519d2a7640 1 -- 172.21.15.22:0/303251674 shutdown_connections 2024-03-09T00:01:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.047+0000 7f519d2a7640 1 --2- 172.21.15.22:0/303251674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5198105620 0x7f5198105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.047+0000 7f519d2a7640 1 --2- 172.21.15.22:0/303251674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 0x7f51981050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.047+0000 7f519d2a7640 1 --2- 172.21.15.22:0/303251674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5198100300 0x7f51981006e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.047+0000 7f519d2a7640 1 -- 172.21.15.22:0/303251674 >> 172.21.15.22:0/303251674 conn(0x7f51980773a0 msgr2=0x7f51980777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.053+0000 7f519d2a7640 1 -- 172.21.15.22:0/303251674 shutdown_connections 2024-03-09T00:01:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.056+0000 7f519d2a7640 1 -- 172.21.15.22:0/303251674 wait complete. 2024-03-09T00:01:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.057+0000 7f519d2a7640 1 Processor -- start 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f519d2a7640 1 -- start start 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f519d2a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5198100300 0x7f519810ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f519d2a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 0x7f519810d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f519d2a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5198105620 0x7f5198071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f519d2a7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5198109050 con 0x7f5198105620 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f519d2a7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5198108ed0 con 0x7f5198100300 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f519d2a7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981091d0 con 0x7f5198100c20 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f51967fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 0x7f519810d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f51967fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 0x7f519810d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35996/0 (socket says 172.21.15.22:35996) 2024-03-09T00:01:59.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.060+0000 7f51967fc640 1 -- 172.21.15.22:0/3290076532 learned_addr learned my addr 172.21.15.22:0/3290076532 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.061+0000 7f51967fc640 1 -- 172.21.15.22:0/3290076532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5198100300 msgr2=0x7f519810ccf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:01:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.061+0000 7f51967fc640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5198100300 0x7f519810ccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.061+0000 7f51967fc640 1 -- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5198105620 msgr2=0x7f5198071ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:01:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.061+0000 7f51967fc640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5198105620 0x7f5198071ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.061+0000 7f51967fc640 1 -- 172.21.15.22:0/3290076532 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980720e0 con 0x7f5198100c20 2024-03-09T00:01:59.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.063+0000 7f51967fc640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 0x7f519810d230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9f0 tx=0x7f518c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.063+0000 7f517ffff640 1 -- 172.21.15.22:0/3290076532 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00cd80 con 0x7f5198100c20 2024-03-09T00:01:59.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.063+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51980723d0 con 0x7f5198100c20 2024-03-09T00:01:59.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.063+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5198072920 con 0x7f5198100c20 2024-03-09T00:01:59.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.064+0000 7f517ffff640 1 -- 172.21.15.22:0/3290076532 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f518c010910 con 0x7f5198100c20 2024-03-09T00:01:59.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.064+0000 7f517ffff640 1 -- 172.21.15.22:0/3290076532 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0188b0 con 0x7f5198100c20 2024-03-09T00:01:59.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.070+0000 7f519fe85740 1 Processor -- start 2024-03-09T00:01:59.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.070+0000 7f519fe85740 1 -- start start 2024-03-09T00:01:59.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.071+0000 7f519fe85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e819d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.071+0000 7f519fe85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.071+0000 7f519fe85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 0x55c6e81a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.071+0000 7f519fe85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8164360 con 0x55c6e81a66a0 2024-03-09T00:01:59.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.071+0000 7f519fe85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e8100ec0 con 0x55c6e819dac0 2024-03-09T00:01:59.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.071+0000 7f519fe85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e80a9d30 con 0x55c6e819d1a0 2024-03-09T00:01:59.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.071+0000 7f517ffff640 1 -- 172.21.15.22:0/3290076532 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f518c010ac0 con 0x7f5198100c20 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.072+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.072+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81a6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:46790/0 (socket says 172.21.15.22:46790) 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.072+0000 7f5196ffd640 1 -- 172.21.15.22:0/244155907 learned_addr learned my addr 172.21.15.22:0/244155907 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.072+0000 7f517ffff640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51800a2b70 0x7f51800a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.073+0000 7f517ffff640 1 -- 172.21.15.22:0/3290076532 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f518c0cf590 con 0x7f5198100c20 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.073+0000 7f5196ffd640 1 -- 172.21.15.22:0/244155907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 msgr2=0x55c6e819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.073+0000 7f5196ffd640 1 --2- 172.21.15.22:0/244155907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.073+0000 7f5196ffd640 1 -- 172.21.15.22:0/244155907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 msgr2=0x55c6e81a8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.073+0000 7f5196ffd640 1 --2- 172.21.15.22:0/244155907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 0x55c6e81a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.073+0000 7f5196ffd640 1 -- 172.21.15.22:0/244155907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e80a8bc0 con 0x55c6e819dac0 2024-03-09T00:01:59.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.074+0000 7f5196ffd640 1 --2- 172.21.15.22:0/244155907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81a6160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f519002c5b0 tx=0x7f5190009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a4d68c49736108 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.075+0000 7f51977fe640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51800a2b70 0x7f51800a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.076+0000 7f517d7fa640 1 -- 172.21.15.22:0/244155907 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190042020 con 0x55c6e819dac0 2024-03-09T00:01:59.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.076+0000 7f517d7fa640 1 -- 172.21.15.22:0/244155907 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5190003e50 con 0x55c6e819dac0 2024-03-09T00:01:59.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.077+0000 7f519fe85740 1 -- 172.21.15.22:0/244155907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 msgr2=0x55c6e81a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.077+0000 7f519fe85740 1 --2- 172.21.15.22:0/244155907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81a6160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f519002c5b0 tx=0x7f5190009870 comp rx=0 tx=0).stop 2024-03-09T00:01:59.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.077+0000 7f51977fe640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51800a2b70 0x7f51800a5030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5198077be0 tx=0x7f518801a650 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.081+0000 7f519fe85740 1 -- 172.21.15.22:0/244155907 shutdown_connections 2024-03-09T00:01:59.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.081+0000 7f519fe85740 1 --2- 172.21.15.22:0/244155907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 0x55c6e81a8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.081+0000 7f519fe85740 1 --2- 172.21.15.22:0/244155907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81a6160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.081+0000 7f519fe85740 1 --2- 172.21.15.22:0/244155907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.081+0000 7f519fe85740 1 -- 172.21.15.22:0/244155907 >> 172.21.15.22:0/244155907 conn(0x55c6e81978d0 msgr2=0x55c6e8197cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.090+0000 7f519fe85740 1 -- 172.21.15.22:0/244155907 shutdown_connections 2024-03-09T00:01:59.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.092+0000 7f519fe85740 1 -- 172.21.15.22:0/244155907 wait complete. 2024-03-09T00:01:59.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.092+0000 7f519fe85740 1 -- start start 2024-03-09T00:01:59.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.092+0000 7f519fe85740 1 Processor -- start 2024-03-09T00:01:59.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.093+0000 7f519fe85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.093+0000 7f519fe85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.093+0000 7f519fe85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e81a66a0 0x55c6e81a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.094+0000 7f519fe85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e809f4a0 con 0x55c6e819dac0 2024-03-09T00:01:59.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.094+0000 7f519fe85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e80fcf60 con 0x55c6e81a66a0 2024-03-09T00:01:59.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.094+0000 7f519fe85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e81a9480 con 0x55c6e819d1a0 2024-03-09T00:01:59.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.094+0000 7f519fe85740 10 client.0 _ll_register_callbacks cb 0x55c6e80fad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.094+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.096+0000 7f5196ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81a35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36012/0 (socket says 172.21.15.22:36012) 2024-03-09T00:01:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.096+0000 7f5196ffd640 1 -- 172.21.15.22:0/1042213142 learned_addr learned my addr 172.21.15.22:0/1042213142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.098+0000 7f51977fe640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e81a66a0 0x55c6e81a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.099+0000 7f51977fe640 1 -- 172.21.15.22:0/1042213142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 msgr2=0x55c6e81a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.099+0000 7f51977fe640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.099+0000 7f51967fc640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.099+0000 7f51977fe640 1 -- 172.21.15.22:0/1042213142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 msgr2=0x55c6e81a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.100+0000 7f51977fe640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.100+0000 7f51977fe640 1 -- 172.21.15.22:0/1042213142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6e7e153f0 con 0x55c6e81a66a0 2024-03-09T00:01:59.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.100+0000 7f51977fe640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e81a66a0 0x55c6e81a4040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5188004610 tx=0x7f51880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d066332889de04c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.101+0000 7f517aff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:01:59.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.101+0000 7f517aff5640 1 -- 172.21.15.22:0/1042213142 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518801fea0 con 0x55c6e81a66a0 2024-03-09T00:01:59.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.101+0000 7f517aff5640 1 -- 172.21.15.22:0/1042213142 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5188003d80 con 0x55c6e81a66a0 2024-03-09T00:01:59.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.103+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5190003e50 con 0x55c6e81a66a0 2024-03-09T00:01:59.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.106+0000 7f51967fc640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:01:59.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.108+0000 7f517aff5640 1 -- 172.21.15.22:0/1042213142 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518802f460 con 0x55c6e81a66a0 2024-03-09T00:01:59.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.108+0000 7f519fe85740 10 client.27365 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:59.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.108+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c6e7edf7d0 con 0x55c6e81a66a0 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f517aff5640 1 -- 172.21.15.22:0/1042213142 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5190003e50 con 0x55c6e81a66a0 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f517aff5640 1 -- 172.21.15.22:0/1042213142 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5188021930 con 0x55c6e81a66a0 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f519fe85740 10 client.27365 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f519fe85740 10 client.27365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f519fe85740 20 client.27365 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f519fe85740 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e81b6d10 0x55c6e81b9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f519fe85740 4 client.27365 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:59.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.111+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6e810e400 con 0x55c6e81b6d10 2024-03-09T00:01:59.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.116+0000 7f5196ffd640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e81b6d10 0x55c6e81b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.120+0000 7f5196ffd640 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e81b6d10 0x55c6e81b9170 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7a41f42b1736dfe server_cookie=b841f5f203eeb825 in_seq=0 out_seq=0 2024-03-09T00:01:59.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.121+0000 7f517aff5640 10 client.27365 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:59.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.122+0000 7f517aff5640 1 -- 172.21.15.22:0/1042213142 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6e810e400 con 0x55c6e81b6d10 2024-03-09T00:01:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.122+0000 7f517aff5640 10 client.27365 handle_command_reply: tid=0 2024-03-09T00:01:59.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.122+0000 7f519fe85740 1 client.27365 shutdown 2024-03-09T00:01:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e81a66a0 msgr2=0x55c6e81a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e81a66a0 0x55c6e81a4040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5188004610 tx=0x7f51880046c0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 shutdown_connections 2024-03-09T00:01:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e81b6d10 0x55c6e81b9170 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e81a66a0 0x55c6e81a4040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 --2- 172.21.15.22:0/1042213142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.124+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 >> 172.21.15.22:0/1042213142 conn(0x55c6e81978d0 msgr2=0x55c6e8199620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.125+0000 7f517aff5640 0 client.27365 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.125+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 shutdown_connections 2024-03-09T00:01:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.125+0000 7f519fe85740 1 -- 172.21.15.22:0/1042213142 wait complete. 2024-03-09T00:01:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.125+0000 7f519fe85740 20 client.27365 trim_cache size 0 max 16384 2024-03-09T00:01:59.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.157+0000 7f519fe85740 1 Processor -- start 2024-03-09T00:01:59.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.159+0000 7f519fe85740 1 -- start start 2024-03-09T00:01:59.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.160+0000 7f519fe85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819d1a0 0x55c6e81a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.160+0000 7f519fe85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81bdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.160+0000 7f519fe85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e81a66a0 0x55c6e81a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.160+0000 7f519fe85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184012490 con 0x55c6e819dac0 2024-03-09T00:01:59.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.160+0000 7f519fe85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184010030 con 0x55c6e819d1a0 2024-03-09T00:01:59.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.161+0000 7f519fe85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518400e0f0 con 0x55c6e81a66a0 2024-03-09T00:01:59.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.162+0000 7f51967fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81bdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.162+0000 7f51967fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81bdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38464/0 (socket says 172.21.15.22:38464) 2024-03-09T00:01:59.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.162+0000 7f51967fc640 1 -- 172.21.15.22:0/432079290 learned_addr learned my addr 172.21.15.22:0/432079290 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.162+0000 7f51977fe640 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e81a66a0 0x55c6e81a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.162+0000 7f5196ffd640 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819d1a0 0x55c6e81a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.164+0000 7f51977fe640 1 -- 172.21.15.22:0/432079290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819d1a0 msgr2=0x55c6e81a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.164+0000 7f51977fe640 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819d1a0 0x55c6e81a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.164+0000 7f51977fe640 1 -- 172.21.15.22:0/432079290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 msgr2=0x55c6e81bdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.164+0000 7f51977fe640 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81bdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.164+0000 7f51977fe640 1 -- 172.21.15.22:0/432079290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e8197700 con 0x55c6e81a66a0 2024-03-09T00:01:59.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.168+0000 7f51977fe640 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e81a66a0 0x55c6e81a0860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f518801b010 tx=0x7f518801a9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779e713827be66c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.173+0000 7f517cff9640 1 -- 172.21.15.22:0/432079290 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188048020 con 0x55c6e81a66a0 2024-03-09T00:01:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.173+0000 7f517cff9640 1 -- 172.21.15.22:0/432079290 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5188034070 con 0x55c6e81a66a0 2024-03-09T00:01:59.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.173+0000 7f517cff9640 1 -- 172.21.15.22:0/432079290 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188020930 con 0x55c6e81a66a0 2024-03-09T00:01:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.175+0000 7f519fe85740 1 -- 172.21.15.22:0/432079290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e81a66a0 msgr2=0x55c6e81a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.175+0000 7f519fe85740 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e81a66a0 0x55c6e81a0860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f518801b010 tx=0x7f518801a9e0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.175+0000 7f519fe85740 1 -- 172.21.15.22:0/432079290 shutdown_connections 2024-03-09T00:01:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.175+0000 7f519fe85740 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e81a66a0 0x55c6e81a0860 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.175+0000 7f519fe85740 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e819dac0 0x55c6e81bdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.175+0000 7f519fe85740 1 --2- 172.21.15.22:0/432079290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819d1a0 0x55c6e81a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.175+0000 7f519fe85740 1 -- 172.21.15.22:0/432079290 >> 172.21.15.22:0/432079290 conn(0x55c6e81978d0 msgr2=0x55c6e819e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.182+0000 7f519fe85740 1 -- 172.21.15.22:0/432079290 shutdown_connections 2024-03-09T00:01:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.182+0000 7f519fe85740 1 -- 172.21.15.22:0/432079290 wait complete. 2024-03-09T00:01:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.183+0000 7f519fe85740 1 -- start start 2024-03-09T00:01:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.183+0000 7f519fe85740 1 Processor -- start 2024-03-09T00:01:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.185+0000 7f519fe85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81abed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.185+0000 7f519fe85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81ac410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.185+0000 7f519fe85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 0x55c6e81ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.185+0000 7f519fe85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518400e6f0 con 0x55c6e81a66a0 2024-03-09T00:01:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.185+0000 7f519fe85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518400f9d0 con 0x55c6e819dac0 2024-03-09T00:01:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.185+0000 7f519fe85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518400ed10 con 0x55c6e819d1a0 2024-03-09T00:01:59.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.185+0000 7f519fe85740 10 client.0 _ll_register_callbacks cb 0x7f5188005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.186+0000 7f51967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.186+0000 7f51967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81ac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:46832/0 (socket says 172.21.15.22:46832) 2024-03-09T00:01:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.186+0000 7f51967fc640 1 -- 172.21.15.22:0/2864320774 learned_addr learned my addr 172.21.15.22:0/2864320774 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.186+0000 7f51977fe640 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 0x55c6e81ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.186+0000 7f5196ffd640 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.189+0000 7f51967fc640 1 -- 172.21.15.22:0/2864320774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 msgr2=0x55c6e81abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.189+0000 7f51967fc640 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.189+0000 7f51967fc640 1 -- 172.21.15.22:0/2864320774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 msgr2=0x55c6e81ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.189+0000 7f51967fc640 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 0x55c6e81ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.189+0000 7f51967fc640 1 -- 172.21.15.22:0/2864320774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6e8197110 con 0x55c6e819dac0 2024-03-09T00:01:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.192+0000 7f51967fc640 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81ac410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f518c0ccfd0 tx=0x7f518c09b220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e287ce15f9ba8edc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.192+0000 7f517aff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:01:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.200+0000 7f517aff5640 1 -- 172.21.15.22:0/2864320774 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0ccd00 con 0x55c6e819dac0 2024-03-09T00:01:59.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.200+0000 7f517aff5640 1 -- 172.21.15.22:0/2864320774 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f518c0cf0c0 con 0x55c6e819dac0 2024-03-09T00:01:59.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.201+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5188034070 con 0x55c6e819dac0 2024-03-09T00:01:59.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.206+0000 7f517aff5640 1 -- 172.21.15.22:0/2864320774 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518c09f8e0 con 0x55c6e819dac0 2024-03-09T00:01:59.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.207+0000 7f519fe85740 10 client.27371 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:59.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.207+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c6e820dc30 con 0x55c6e819dac0 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f517aff5640 1 -- 172.21.15.22:0/2864320774 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f518c0a5080 con 0x55c6e819dac0 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f517aff5640 1 -- 172.21.15.22:0/2864320774 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f518c09abe0 con 0x55c6e819dac0 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f519fe85740 10 client.27371 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f519fe85740 10 client.27371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f519fe85740 20 client.27371 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f519fe85740 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e820e310 0x55c6e8210730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f519fe85740 4 client.27371 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.209+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6e8108750 con 0x55c6e820e310 2024-03-09T00:01:59.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.210+0000 7f5196ffd640 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e820e310 0x55c6e8210730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.212+0000 7f5196ffd640 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e820e310 0x55c6e8210730 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=42189f4bb4a2f8b8 server_cookie=6925d00cbe66c09 in_seq=0 out_seq=0 2024-03-09T00:01:59.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.212+0000 7f517aff5640 10 client.27371 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.212+0000 7f517aff5640 1 -- 172.21.15.22:0/2864320774 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6e8108750 con 0x55c6e820e310 2024-03-09T00:01:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.212+0000 7f517aff5640 10 client.27371 handle_command_reply: tid=0 2024-03-09T00:01:59.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.212+0000 7f519fe85740 1 client.27371 shutdown 2024-03-09T00:01:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.214+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 msgr2=0x55c6e81ac410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.214+0000 7f519fe85740 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81ac410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f518c0ccfd0 tx=0x7f518c09b220 comp rx=0 tx=0).stop 2024-03-09T00:01:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 shutdown_connections 2024-03-09T00:01:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c6e820e310 0x55c6e8210730 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c6e81a66a0 0x55c6e81ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6e819dac0 0x55c6e81ac410 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 --2- 172.21.15.22:0/2864320774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6e819d1a0 0x55c6e81abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f517aff5640 0 client.27371 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:01:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 >> 172.21.15.22:0/2864320774 conn(0x55c6e81978d0 msgr2=0x55c6e81bdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 shutdown_connections 2024-03-09T00:01:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 1 -- 172.21.15.22:0/2864320774 wait complete. 2024-03-09T00:01:59.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.215+0000 7f519fe85740 20 client.27371 trim_cache size 0 max 16384 2024-03-09T00:01:59.211 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.216+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51800a2b70 msgr2=0x7f51800a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.216+0000 7f519d2a7640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51800a2b70 0x7f51800a5030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5198077be0 tx=0x7f518801a650 comp rx=0 tx=0).stop 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.216+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 msgr2=0x7f519810d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.216+0000 7f519d2a7640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 0x7f519810d230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9f0 tx=0x7f518c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 shutdown_connections 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51800a2b70 0x7f51800a5030 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5198105620 0x7f5198071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5198100c20 0x7f519810d230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 --2- 172.21.15.22:0/3290076532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5198100300 0x7f519810ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 >> 172.21.15.22:0/3290076532 conn(0x7f51980773a0 msgr2=0x7f51981030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 shutdown_connections 2024-03-09T00:01:59.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.217+0000 7f519d2a7640 1 -- 172.21.15.22:0/3290076532 wait complete. 2024-03-09T00:01:59.226 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:01:59.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:01:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.232+0000 7f7250ff9640 1 -- 172.21.15.22:0/3656579013 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae02320e80 con 0x55ae023ce880 2024-03-09T00:01:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.232+0000 7f7250ff9640 10 client.17973 handle_command_reply: tid=0 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.232+0000 7f72723a3740 1 client.17973 shutdown 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b4fd0 msgr2=0x55ae023c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b4fd0 0x55ae023c58a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7258006010 tx=0x7f72580045b0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 shutdown_connections 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ae023ce880 0x55ae024281d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae023be550 0x55ae023c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae023b5d40 0x55ae023c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 --2- 172.21.15.22:0/3656579013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ae023b4fd0 0x55ae023c58a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 >> 172.21.15.22:0/3656579013 conn(0x55ae0237c5b0 msgr2=0x55ae023b9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f7250ff9640 0 client.17973 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 shutdown_connections 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 1 -- 172.21.15.22:0/3656579013 wait complete. 2024-03-09T00:01:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.233+0000 7f72723a3740 20 client.17973 trim_cache size 0 max 16384 2024-03-09T00:01:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:01:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:01:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:01:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:01:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f72540a2a10 msgr2=0x7f72540a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f72540a2a10 0x7f72540a4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72600eea80 tx=0x7f72600b9610 comp rx=0 tx=0).stop 2024-03-09T00:01:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 msgr2=0x7f7268199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f7268199ad0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f726002c790 tx=0x7f7260035e00 comp rx=0 tx=0).stop 2024-03-09T00:01:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 shutdown_connections 2024-03-09T00:01:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f72540a2a10 0x7f72540a4ed0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f726810dad0 0x7f7268199ad0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7268074040 0x7f726819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 --2- 172.21.15.22:0/966701249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7268073720 0x7f726819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.234+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 >> 172.21.15.22:0/966701249 conn(0x7f72680fcd80 msgr2=0x7f72680fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.235+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 shutdown_connections 2024-03-09T00:01:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:01:59.235+0000 7f726f7c4640 1 -- 172.21.15.22:0/966701249 wait complete. 2024-03-09T00:01:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 -- 172.21.15.22:0/2600746966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e081065c0 msgr2=0x7f0e0810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 --2- 172.21.15.22:0/2600746966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e081065c0 0x7f0e0810e170 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009ca0 tx=0x7f0e0002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 -- 172.21.15.22:0/2600746966 shutdown_connections 2024-03-09T00:01:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 --2- 172.21.15.22:0/2600746966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e081065c0 0x7f0e0810e170 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 --2- 172.21.15.22:0/2600746966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e0806a580 0x7f0e08105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 --2- 172.21.15.22:0/2600746966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e08069bd0 0x7f0e08069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 -- 172.21.15.22:0/2600746966 >> 172.21.15.22:0/2600746966 conn(0x7f0e08076f90 msgr2=0x7f0e080773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 -- 172.21.15.22:0/2600746966 shutdown_connections 2024-03-09T00:01:59.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.341+0000 7f0e0d012640 1 -- 172.21.15.22:0/2600746966 wait complete. 2024-03-09T00:01:59.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 Processor -- start 2024-03-09T00:01:59.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 -- start start 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e08069bd0 0x7f0e0819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e081065c0 0x7f0e0819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e081978b0 0x7f0e08197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08111670 con 0x7f0e08069bd0 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081114f0 con 0x7f0e081065c0 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e0d012640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081117f0 con 0x7f0e081978b0 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e08069bd0 0x7f0e0819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e08069bd0 0x7f0e0819d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38492/0 (socket says 172.21.15.22:38492) 2024-03-09T00:01:59.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.342+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1317933513 learned_addr learned my addr 172.21.15.22:0/1317933513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e081978b0 0x7f0e08197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0dfffff640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e081065c0 0x7f0e0819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1317933513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e081978b0 msgr2=0x7f0e08197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e081978b0 0x7f0e08197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1317933513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e081065c0 msgr2=0x7f0e0819dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e081065c0 0x7f0e0819dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1317933513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e081983d0 con 0x7f0e08069bd0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0dfffff640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e081065c0 0x7f0e0819dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e08069bd0 0x7f0e0819d700 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e9f0 tx=0x7f0df800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e04d99640 1 -- 172.21.15.22:0/1317933513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800cd80 con 0x7f0e08069bd0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e081986c0 con 0x7f0e08069bd0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e081a4960 con 0x7f0e08069bd0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e04d99640 1 -- 172.21.15.22:0/1317933513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0df8010910 con 0x7f0e08069bd0 2024-03-09T00:01:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.343+0000 7f0e04d99640 1 -- 172.21.15.22:0/1317933513 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df80188f0 con 0x7f0e08069bd0 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.345+0000 7f0e04d99640 1 -- 172.21.15.22:0/1317933513 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0df8010ac0 con 0x7f0e08069bd0 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 Processor -- start 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 -- start start 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7b1a0 0x5631f1f7b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7bac0 0x5631f1f84160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f86a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f1f42360 con 0x5631f1f7bac0 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f1edeec0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f1e87d30 con 0x5631f1f846a0 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e04d99640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0df40a2970 0x7f0df40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7bac0 0x5631f1f84160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f86a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7bac0 0x5631f1f84160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38498/0 (socket says 172.21.15.22:38498) 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f86a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36052/0 (socket says 172.21.15.22:36052) 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e06d9d640 1 -- 172.21.15.22:0/17501523 learned_addr learned my addr 172.21.15.22:0/17501523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0759e640 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7b1a0 0x5631f1f7b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.346+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0df40a2970 0x7f0df40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0e04d99640 1 -- 172.21.15.22:0/1317933513 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0df80cf690 con 0x7f0e08069bd0 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfffff640 1 -- 172.21.15.22:0/17501523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7b1a0 msgr2=0x5631f1f7b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfffff640 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7b1a0 0x5631f1f7b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfffff640 1 -- 172.21.15.22:0/17501523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7bac0 msgr2=0x5631f1f84160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfffff640 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7bac0 0x5631f1f84160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfffff640 1 -- 172.21.15.22:0/17501523 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631f1e86bc0 con 0x5631f1f846a0 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7bac0 0x5631f1f84160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0e0759e640 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7b1a0 0x5631f1f7b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0df40a2970 0x7f0df40a4e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e00004af0 tx=0x7f0e00005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfffff640 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f86a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e081990a0 tx=0x7f0df00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b07d9f10ccb87024 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfd7fa640 1 -- 172.21.15.22:0/17501523 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0005020 con 0x5631f1f846a0 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfd7fa640 1 -- 172.21.15.22:0/17501523 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0df00047a0 con 0x5631f1f846a0 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.347+0000 7f0dfd7fa640 1 -- 172.21.15.22:0/17501523 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00126b0 con 0x5631f1f846a0 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/17501523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 msgr2=0x5631f1f86a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f86a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e081990a0 tx=0x7f0df00117b0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/17501523 shutdown_connections 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f86a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7bac0 0x5631f1f84160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/17501523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7b1a0 0x5631f1f7b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/17501523 >> 172.21.15.22:0/17501523 conn(0x5631f1f758d0 msgr2=0x5631f1f75cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/17501523 shutdown_connections 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/17501523 wait complete. 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 -- start start 2024-03-09T00:01:59.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.348+0000 7f0e0fbfb740 1 Processor -- start 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f81c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f1e7d4a0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f1edaf60 con 0x5631f1f7bac0 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631f1f87490 con 0x5631f1f846a0 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 10 client.0 _ll_register_callbacks cb 0x5631f1ed8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f816c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38508/0 (socket says 172.21.15.22:38508) 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 -- 172.21.15.22:0/3718460643 learned_addr learned my addr 172.21.15.22:0/3718460643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0759e640 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0dfffff640 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f81c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 -- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 msgr2=0x5631f1f82140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f82140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 -- 172.21.15.22:0/3718460643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 msgr2=0x5631f1f81c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f81c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 -- 172.21.15.22:0/3718460643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631f1bf33f0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f816c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0df8097d50 tx=0x7f0df809b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abb06492e4b412 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0dee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0dee7fc640 1 -- 172.21.15.22:0/3718460643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df809b8f0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0dee7fc640 1 -- 172.21.15.22:0/3718460643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0df80ccc60 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0df00047a0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0dee7fc640 1 -- 172.21.15.22:0/3718460643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0df80cce60 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.349+0000 7f0e0fbfb740 10 client.17994 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5631f1cbd7d0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0dee7fc640 1 -- 172.21.15.22:0/3718460643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0df00047a0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0dee7fc640 1 -- 172.21.15.22:0/3718460643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0df80cf330 con 0x5631f1f7b1a0 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0e0fbfb740 10 client.17994 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:59.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0e0fbfb740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0e0fbfb740 20 client.17994 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1f947d0 0x5631f1f96be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0e0fbfb740 4 client.17994 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631f1eec400 con 0x5631f1f947d0 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0dfffff640 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1f947d0 0x5631f1f96be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0dee7fc640 10 client.17994 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.350+0000 7f0dfffff640 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1f947d0 0x5631f1f96be0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db01e18b44fde55 server_cookie=210d7e66f595a254 in_seq=0 out_seq=0 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0dee7fc640 1 -- 172.21.15.22:0/3718460643 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631f1eec400 con 0x5631f1f947d0 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0dee7fc640 10 client.17994 handle_command_reply: tid=0 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 client.17994 shutdown 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 msgr2=0x5631f1f816c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f816c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0df8097d50 tx=0x7f0df809b340 comp rx=0 tx=0).stop 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 shutdown_connections 2024-03-09T00:01:59.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1f947d0 0x5631f1f96be0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f846a0 0x5631f1f82140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f81c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/3718460643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f816c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0dee7fc640 0 client.17994 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 >> 172.21.15.22:0/3718460643 conn(0x5631f1f758d0 msgr2=0x5631f1f77620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 shutdown_connections 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/3718460643 wait complete. 2024-03-09T00:01:59.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.351+0000 7f0e0fbfb740 20 client.17994 trim_cache size 0 max 16384 2024-03-09T00:01:59.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 Processor -- start 2024-03-09T00:01:59.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 -- start start 2024-03-09T00:01:59.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f83df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7bac0 0x5631f1f806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f846a0 0x5631f1f7e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081b0b50 con 0x5631f1f7b1a0 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081ae6f0 con 0x5631f1f846a0 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081ac7b0 con 0x5631f1f7bac0 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f83df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f83df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38522/0 (socket says 172.21.15.22:38522) 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1866841990 learned_addr learned my addr 172.21.15.22:0/1866841990 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0dfffff640 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7bac0 0x5631f1f806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.363+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f846a0 0x5631f1f7e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1866841990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7bac0 msgr2=0x5631f1f806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7bac0 0x5631f1f806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1866841990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f846a0 msgr2=0x5631f1f7e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f846a0 0x5631f1f7e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1866841990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631f1f7b8f0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f83df0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0df800cfd0 tx=0x7f0df800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231aa57bc862af81 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0dfcff9640 1 -- 172.21.15.22:0/1866841990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df809bff0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0dfcff9640 1 -- 172.21.15.22:0/1866841990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0df809fa40 con 0x5631f1f7b1a0 2024-03-09T00:01:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.364+0000 7f0dfcff9640 1 -- 172.21.15.22:0/1866841990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df80cf240 con 0x5631f1f7b1a0 2024-03-09T00:01:59.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:01:59 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3321041545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:01:59.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1866841990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 msgr2=0x5631f1f83df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f83df0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0df800cfd0 tx=0x7f0df800c7d0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1866841990 shutdown_connections 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f846a0 0x5631f1f7e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7bac0 0x5631f1f806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1866841990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f7b1a0 0x5631f1f83df0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1866841990 >> 172.21.15.22:0/1866841990 conn(0x5631f1f758d0 msgr2=0x5631f1f7ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1866841990 shutdown_connections 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1866841990 wait complete. 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 -- start start 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 Processor -- start 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7b1a0 0x5631f1f8bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f8a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.365+0000 7f0e0fbfb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f846a0 0x5631f1f8a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081acdb0 con 0x5631f1f846a0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081ae090 con 0x5631f1f7bac0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0fbfb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081ad3d0 con 0x5631f1f7b1a0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7b1a0 0x5631f1f8bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0fbfb740 10 client.0 _ll_register_callbacks cb 0x7f0df00095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e06d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7b1a0 0x5631f1f8bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36084/0 (socket says 172.21.15.22:36084) 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e06d9d640 1 -- 172.21.15.22:0/1734168353 learned_addr learned my addr 172.21.15.22:0/1734168353 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f846a0 0x5631f1f8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0dfffff640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f8a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0759e640 1 -- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7b1a0 msgr2=0x5631f1f8bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7b1a0 0x5631f1f8bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0759e640 1 -- 172.21.15.22:0/1734168353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 msgr2=0x5631f1f8a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f8a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0759e640 1 -- 172.21.15.22:0/1734168353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631f1f8aff0 con 0x5631f1f846a0 2024-03-09T00:01:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7b1a0 0x5631f1f8bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0759e640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f846a0 0x5631f1f8a850 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0e0002c700 tx=0x7f0e000095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e84cf7d289e55c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0dee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0dee7fc640 1 -- 172.21.15.22:0/1734168353 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00054070 con 0x5631f1f846a0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0dee7fc640 1 -- 172.21.15.22:0/1734168353 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0e00004450 con 0x5631f1f846a0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0df809fa40 con 0x5631f1f846a0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0dee7fc640 1 -- 172.21.15.22:0/1734168353 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e00039040 con 0x5631f1f846a0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0fbfb740 10 client.18000 fetch_fsmap learned FSMap version 33 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.366+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5631f1ff4980 con 0x5631f1f846a0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0dee7fc640 1 -- 172.21.15.22:0/1734168353 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0df809fa40 con 0x5631f1f846a0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0dee7fc640 1 -- 172.21.15.22:0/1734168353 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0e00034b80 con 0x5631f1f846a0 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e0fbfb740 10 client.18000 fetch_fsmap finished waiting for FSMap version 33 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e0fbfb740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e0fbfb740 20 client.18000 populate_metadata read hostname 'smithi022' 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1ff5080 0x5631f1ff74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:01:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e0fbfb740 4 client.18000 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5631f1ee6750 con 0x5631f1ff5080 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1ff5080 0x5631f1ff74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0dee7fc640 10 client.18000 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.367+0000 7f0e06d9d640 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1ff5080 0x5631f1ff74a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7774c99ccaef6129 server_cookie=5a53bc7375a4f723 in_seq=0 out_seq=0 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0dee7fc640 1 -- 172.21.15.22:0/1734168353 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5631f1ee6750 con 0x5631f1ff5080 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0dee7fc640 10 client.18000 handle_command_reply: tid=0 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 client.18000 shutdown 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f846a0 msgr2=0x5631f1f8a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f846a0 0x5631f1f8a850 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0e0002c700 tx=0x7f0e000095c0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 shutdown_connections 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5631f1ff5080 0x5631f1ff74a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631f1f846a0 0x5631f1f8a850 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631f1f7bac0 0x5631f1f8a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0dee7fc640 0 client.18000 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 --2- 172.21.15.22:0/1734168353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631f1f7b1a0 0x5631f1f8bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.368+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 >> 172.21.15.22:0/1734168353 conn(0x5631f1f758d0 msgr2=0x5631f1f80820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 shutdown_connections 2024-03-09T00:01:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0fbfb740 1 -- 172.21.15.22:0/1734168353 wait complete. 2024-03-09T00:01:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0fbfb740 20 client.18000 trim_cache size 0 max 16384 2024-03-09T00:01:59.370 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0df40a2970 msgr2=0x7f0df40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0df40a2970 0x7f0df40a4e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e00004af0 tx=0x7f0e00005c50 comp rx=0 tx=0).stop 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e08069bd0 msgr2=0x7f0e0819d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e08069bd0 0x7f0e0819d700 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e9f0 tx=0x7f0df800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 shutdown_connections 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0df40a2970 0x7f0df40a4e30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e081978b0 0x7f0e08197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e081065c0 0x7f0e0819dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 --2- 172.21.15.22:0/1317933513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e08069bd0 0x7f0e0819d700 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:01:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 >> 172.21.15.22:0/1317933513 conn(0x7f0e08076f90 msgr2=0x7f0e080fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:01:59.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 shutdown_connections 2024-03-09T00:01:59.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:01:59.369+0000 7f0e0d012640 1 -- 172.21.15.22:0/1317933513 wait complete. 2024-03-09T00:01:59.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:01:59 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3321041545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:01:59.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:01:59 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3321041545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:01:59.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 785494 10.85 MB/sec execute 247 sec latency 130.075 ms 2024-03-09T00:02:00.093 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:01:59 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setuser ceph since I am not root 2024-03-09T00:02:00.093 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:01:59 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setgroup ceph since I am not root 2024-03-09T00:02:00.093 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:01:59 smithi062 ceph-mds[70095]: -- 172.21.15.62:0/342545009 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c781ba7a40 con 0x55c781b7ac00 2024-03-09T00:02:00.093 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:01:59 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: starting mds.h at 2024-03-09T00:02:00.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:00 smithi062 ceph-mon[36129]: pgmap v232: 129 pgs: 129 active+clean; 88 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-03-09T00:02:00.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:00 smithi062 ceph-mon[36129]: daemon mds.h restarted 2024-03-09T00:02:00.376 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:02:00.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:02:00.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:02:00.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:00 smithi192 ceph-mon[35995]: pgmap v232: 129 pgs: 129 active+clean; 88 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-03-09T00:02:00.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:00 smithi192 ceph-mon[35995]: daemon mds.h restarted 2024-03-09T00:02:00.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:00 smithi022 ceph-mon[31894]: pgmap v232: 129 pgs: 129 active+clean; 88 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-03-09T00:02:00.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:00 smithi022 ceph-mon[31894]: daemon mds.h restarted 2024-03-09T00:02:00.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 -- 172.21.15.22:0/2145795542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 msgr2=0x7f00841007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 --2- 172.21.15.22:0/2145795542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f00841007b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f0074009900 tx=0x7f007402bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:00.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 -- 172.21.15.22:0/2145795542 shutdown_connections 2024-03-09T00:02:00.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 --2- 172.21.15.22:0/2145795542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00841054a0 0x7f008410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 --2- 172.21.15.22:0/2145795542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0084100d80 0x7f0084104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 --2- 172.21.15.22:0/2145795542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f00841007b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 -- 172.21.15.22:0/2145795542 >> 172.21.15.22:0/2145795542 conn(0x7f00840772f0 msgr2=0x7f0084077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 -- 172.21.15.22:0/2145795542 shutdown_connections 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 -- 172.21.15.22:0/2145795542 wait complete. 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 Processor -- start 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 -- start start 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.491+0000 7f0089f80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f008419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0089f80640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00841054a0 0x7f008419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0089f80640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0084197990 0x7f0084197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0089f80640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0084110560 con 0x7f00841003d0 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0089f80640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841103e0 con 0x7f00841054a0 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0089f80640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841106e0 con 0x7f0084197990 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f008419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f008419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50636/0 (socket says 172.21.15.22:50636) 2024-03-09T00:02:00.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 -- 172.21.15.22:0/3362782039 learned_addr learned my addr 172.21.15.22:0/3362782039 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0083fff640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0084197990 0x7f0084197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0082ffd640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00841054a0 0x7f008419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 -- 172.21.15.22:0/3362782039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0084197990 msgr2=0x7f0084197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0084197990 0x7f0084197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 -- 172.21.15.22:0/3362782039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00841054a0 msgr2=0x7f008419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00841054a0 0x7f008419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 -- 172.21.15.22:0/3362782039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00841984b0 con 0x7f00841003d0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f00837fe640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f008419d7e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0074006010 tx=0x7f007402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0080ff9640 1 -- 172.21.15.22:0/3362782039 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074041070 con 0x7f00841003d0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0080ff9640 1 -- 172.21.15.22:0/3362782039 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0074004450 con 0x7f00841003d0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0084198740 con 0x7f00841003d0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0080ff9640 1 -- 172.21.15.22:0/3362782039 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074003e00 con 0x7f00841003d0 2024-03-09T00:02:00.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.492+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00841a4930 con 0x7f00841003d0 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f0080ff9640 1 -- 172.21.15.22:0/3362782039 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0074044070 con 0x7f00841003d0 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 Processor -- start 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 -- start start 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f987160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f992670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1f8eab40 con 0x55ad1f990280 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1f893d80 con 0x55ad1f986d80 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f008cc35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1f8e6be0 con 0x55ad1f9876a0 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f0080ff9640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00700a2970 0x7f00700a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f0083fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f987160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.495+0000 7f0082ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f992670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f987160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57592/0 (socket says 172.21.15.22:57592) 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 -- 172.21.15.22:0/1561883996 learned_addr learned my addr 172.21.15.22:0/1561883996 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0082ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f992670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50644/0 (socket says 172.21.15.22:50644) 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f00837fe640 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00700a2970 0x7f00700a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0080ff9640 1 -- 172.21.15.22:0/3362782039 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f00740f0b70 con 0x7f00841003d0 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 -- 172.21.15.22:0/1561883996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 msgr2=0x55ad1f98fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 -- 172.21.15.22:0/1561883996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 msgr2=0x55ad1f992670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f992670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 -- 172.21.15.22:0/1561883996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad1f6c77e0 con 0x55ad1f986d80 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0082ffd640 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f992670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f00837fe640 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f987160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f007c005c40 tx=0x7f007c013f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de966da3519fb68 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0083fff640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00700a2970 0x7f00700a4e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f007c00f850 tx=0x7f007c023000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0079ffb640 1 -- 172.21.15.22:0/1561883996 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c02a070 con 0x55ad1f986d80 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0079ffb640 1 -- 172.21.15.22:0/1561883996 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f007c021030 con 0x55ad1f986d80 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.496+0000 7f0079ffb640 1 -- 172.21.15.22:0/1561883996 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c025af0 con 0x55ad1f986d80 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 -- 172.21.15.22:0/1561883996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 msgr2=0x55ad1f987160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f987160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f007c005c40 tx=0x7f007c013f50 comp rx=0 tx=0).stop 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 -- 172.21.15.22:0/1561883996 shutdown_connections 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f992670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 --2- 172.21.15.22:0/1561883996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f987160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 -- 172.21.15.22:0/1561883996 >> 172.21.15.22:0/1561883996 conn(0x55ad1f9814b0 msgr2=0x55ad1f981890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 -- 172.21.15.22:0/1561883996 shutdown_connections 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.497+0000 7f008cc35740 1 -- 172.21.15.22:0/1561883996 wait complete. 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 -- start start 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 Processor -- start 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f98d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1f993010 con 0x55ad1f990280 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1f8a71c0 con 0x55ad1f986d80 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1f993190 con 0x55ad1f9876a0 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f008cc35740 10 client.0 _ll_register_callbacks cb 0x55ad1f8967a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f98d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f00837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f98d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50650/0 (socket says 172.21.15.22:50650) 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 -- 172.21.15.22:0/967165389 learned_addr learned my addr 172.21.15.22:0/967165389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0082ffd640 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 -- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 msgr2=0x55ad1f98d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 -- 172.21.15.22:0/967165389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 msgr2=0x55ad1f98fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.498+0000 7f0083fff640 1 -- 172.21.15.22:0/967165389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad1f727f70 con 0x55ad1f990280 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f0083fff640 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f98d7a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f007c000f80 tx=0x7f007c01ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871e5cfd985eefeb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f0062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f0062ffd640 1 -- 172.21.15.22:0/967165389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c032070 con 0x55ad1f990280 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f0062ffd640 1 -- 172.21.15.22:0/967165389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f007c025a00 con 0x55ad1f990280 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f007c021030 con 0x55ad1f990280 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f0062ffd640 1 -- 172.21.15.22:0/967165389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f007c03e050 con 0x55ad1f990280 2024-03-09T00:02:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 10 client.18018 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ad1f5bf170 con 0x55ad1f990280 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f0062ffd640 1 -- 172.21.15.22:0/967165389 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f007c021030 con 0x55ad1f990280 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f0062ffd640 1 -- 172.21.15.22:0/967165389 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f007c037920 con 0x55ad1f990280 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 10 client.18018 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 20 client.18018 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9a0240 0x55ad1f9a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 4 client.18018 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.499+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad1f8c0960 con 0x55ad1f9a0240 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.500+0000 7f00837fe640 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9a0240 0x55ad1f9a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.500+0000 7f00837fe640 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9a0240 0x55ad1f9a2630 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ac9ba5438e7a6d server_cookie=dfd89c9c0cc4d0d7 in_seq=0 out_seq=0 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.500+0000 7f0062ffd640 10 client.18018 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f0062ffd640 1 -- 172.21.15.22:0/967165389 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad1f8c0960 con 0x55ad1f9a0240 2024-03-09T00:02:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f0062ffd640 10 client.18018 handle_command_reply: tid=0 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 client.18018 shutdown 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 msgr2=0x55ad1f98d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f98d7a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f007c000f80 tx=0x7f007c01ffb0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 shutdown_connections 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9a0240 0x55ad1f9a2630 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f98d7a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f9876a0 0x55ad1f98d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f0062ffd640 0 client.18018 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 --2- 172.21.15.22:0/967165389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 >> 172.21.15.22:0/967165389 conn(0x55ad1f9814b0 msgr2=0x55ad1f983200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 shutdown_connections 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 1 -- 172.21.15.22:0/967165389 wait complete. 2024-03-09T00:02:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.501+0000 7f008cc35740 20 client.18018 trim_cache size 0 max 16384 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.517+0000 7f008cc35740 1 Processor -- start 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.517+0000 7f008cc35740 1 -- start start 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.517+0000 7f008cc35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f008cc35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f9876a0 0x55ad1f98f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f008cc35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f990280 0x55ad1f98fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f008cc35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841ae8b0 con 0x55ad1f9876a0 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f008cc35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841ac450 con 0x55ad1f986d80 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f008cc35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841aa4f0 con 0x55ad1f990280 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f9876a0 0x55ad1f98f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f9876a0 0x55ad1f98f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50662/0 (socket says 172.21.15.22:50662) 2024-03-09T00:02:00.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 -- 172.21.15.22:0/3946211044 learned_addr learned my addr 172.21.15.22:0/3946211044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0083fff640 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f990280 0x55ad1f98fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f00837fe640 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 -- 172.21.15.22:0/3946211044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f990280 msgr2=0x55ad1f98fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f990280 0x55ad1f98fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 -- 172.21.15.22:0/3946211044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 msgr2=0x55ad1f98bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 -- 172.21.15.22:0/3946211044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad1f986750 con 0x55ad1f9876a0 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f0082ffd640 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f9876a0 0x55ad1f98f790 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f006c009d00 tx=0x7f006c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214f48bbb3239e2a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f00797fa640 1 -- 172.21.15.22:0/3946211044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c009df0 con 0x55ad1f9876a0 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f00797fa640 1 -- 172.21.15.22:0/3946211044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f006c005500 con 0x55ad1f9876a0 2024-03-09T00:02:00.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.518+0000 7f00797fa640 1 -- 172.21.15.22:0/3946211044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c004d00 con 0x55ad1f9876a0 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 -- 172.21.15.22:0/3946211044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f9876a0 msgr2=0x55ad1f98f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f9876a0 0x55ad1f98f790 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f006c009d00 tx=0x7f006c005f90 comp rx=0 tx=0).stop 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 -- 172.21.15.22:0/3946211044 shutdown_connections 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f990280 0x55ad1f98fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f9876a0 0x55ad1f98f790 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 --2- 172.21.15.22:0/3946211044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f986d80 0x55ad1f98bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 -- 172.21.15.22:0/3946211044 >> 172.21.15.22:0/3946211044 conn(0x55ad1f9814b0 msgr2=0x55ad1f98b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 -- 172.21.15.22:0/3946211044 shutdown_connections 2024-03-09T00:02:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 -- 172.21.15.22:0/3946211044 wait complete. 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.519+0000 7f008cc35740 1 -- start start 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 1 Processor -- start 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f986d80 0x55ad1f9f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f9876a0 0x55ad1f9f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f994690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841aaaf0 con 0x55ad1f990280 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841abdf0 con 0x55ad1f9876a0 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841ab130 con 0x55ad1f986d80 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f008cc35740 10 client.0 _ll_register_callbacks cb 0x7f007c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f994690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f994690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50672/0 (socket says 172.21.15.22:50672) 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 -- 172.21.15.22:0/657937175 learned_addr learned my addr 172.21.15.22:0/657937175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f00837fe640 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f986d80 0x55ad1f9f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0082ffd640 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f9876a0 0x55ad1f9f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 -- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f986d80 msgr2=0x55ad1f9f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f986d80 0x55ad1f9f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 -- 172.21.15.22:0/657937175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f9876a0 msgr2=0x55ad1f9f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f9876a0 0x55ad1f9f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 -- 172.21.15.22:0/657937175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad1f994d80 con 0x55ad1f990280 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0083fff640 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f994690 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f007c013990 tx=0x7f007c013560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac95331e3c6583c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.520+0000 7f0062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f0062ffd640 1 -- 172.21.15.22:0/657937175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c03b020 con 0x55ad1f990280 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f0062ffd640 1 -- 172.21.15.22:0/657937175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f007c025c00 con 0x55ad1f990280 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f006c005500 con 0x55ad1f990280 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f0062ffd640 1 -- 172.21.15.22:0/657937175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f007c032070 con 0x55ad1f990280 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 10 client.18024 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ad1f9f8080 con 0x55ad1f990280 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f0062ffd640 1 -- 172.21.15.22:0/657937175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f006c005500 con 0x55ad1f990280 2024-03-09T00:02:00.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f0062ffd640 1 -- 172.21.15.22:0/657937175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f007c037540 con 0x55ad1f990280 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 10 client.18024 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 20 client.18024 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9f8680 0x55ad1f9faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 4 client.18024 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ad1f8ac080 con 0x55ad1f9f8680 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.521+0000 7f00837fe640 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9f8680 0x55ad1f9faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.522+0000 7f0062ffd640 10 client.18024 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.522+0000 7f00837fe640 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9f8680 0x55ad1f9faa70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1990efae8ec2d0a server_cookie=f86f724ed7b948ce in_seq=0 out_seq=0 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f0062ffd640 1 -- 172.21.15.22:0/657937175 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ad1f8ac080 con 0x55ad1f9f8680 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f0062ffd640 10 client.18024 handle_command_reply: tid=0 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 client.18024 shutdown 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 msgr2=0x55ad1f994690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f994690 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f007c013990 tx=0x7f007c013560 comp rx=0 tx=0).stop 2024-03-09T00:02:00.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 shutdown_connections 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ad1f9f8680 0x55ad1f9faa70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ad1f990280 0x55ad1f994690 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ad1f9876a0 0x55ad1f9f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 --2- 172.21.15.22:0/657937175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad1f986d80 0x55ad1f9f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 >> 172.21.15.22:0/657937175 conn(0x55ad1f9814b0 msgr2=0x55ad1f98c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f0062ffd640 0 client.18024 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 shutdown_connections 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.523+0000 7f008cc35740 1 -- 172.21.15.22:0/657937175 wait complete. 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f008cc35740 20 client.18024 trim_cache size 0 max 16384 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:02:00.524 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "695beb26-e0f1-40b7-9a78-da461d72b8d5", 2024-03-09T00:02:00.525 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:02:00.525 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:02:00.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00700a2970 msgr2=0x7f00700a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00700a2970 0x7f00700a4e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f007c00f850 tx=0x7f007c023000 comp rx=0 tx=0).stop 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 msgr2=0x7f008419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f008419d7e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0074006010 tx=0x7f007402cb30 comp rx=0 tx=0).stop 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 shutdown_connections 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00700a2970 0x7f00700a4e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0084197990 0x7f0084197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00841054a0 0x7f008419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 --2- 172.21.15.22:0/3362782039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00841003d0 0x7f008419d7e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 >> 172.21.15.22:0/3362782039 conn(0x7f00840772f0 msgr2=0x7f00840687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 shutdown_connections 2024-03-09T00:02:00.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.524+0000 7f0089f80640 1 -- 172.21.15.22:0/3362782039 wait complete. 2024-03-09T00:02:00.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:02:00.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:02:00.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/75935833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141014f0 msgr2=0x7f4a141018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/75935833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141014f0 0x7f4a141018d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009900 tx=0x7f4a0402bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/75935833 shutdown_connections 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/75935833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a141065c0 0x7f4a1410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/75935833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a14101ea0 0x7f4a14105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/75935833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141014f0 0x7f4a141018d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/75935833 >> 172.21.15.22:0/75935833 conn(0x7f4a140fcd90 msgr2=0x7f4a140ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/75935833 shutdown_connections 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.685+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/75935833 wait complete. 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 Processor -- start 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 -- start start 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a141014f0 0x7f4a1419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141065c0 0x7f4a1419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a14197990 0x7f4a14197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a14111650 con 0x7f4a141065c0 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141114d0 con 0x7f4a14197990 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a1a9bf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141117d0 con 0x7f4a141014f0 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141065c0 0x7f4a1419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a18f5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a14197990 0x7f4a14197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141065c0 0x7f4a1419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50694/0 (socket says 172.21.15.22:50694) 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a137fe640 1 -- 172.21.15.22:0/4255536762 learned_addr learned my addr 172.21.15.22:0/4255536762 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.686+0000 7f4a13fff640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a141014f0 0x7f4a1419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a18f5c640 1 -- 172.21.15.22:0/4255536762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a141014f0 msgr2=0x7f4a1419d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a141014f0 0x7f4a1419d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a18f5c640 1 -- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141065c0 msgr2=0x7f4a1419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141065c0 0x7f4a1419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a18f5c640 1 -- 172.21.15.22:0/4255536762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a141984b0 con 0x7f4a14197990 2024-03-09T00:02:00.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a137fe640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141065c0 0x7f4a1419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:00.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a14197990 0x7f4a14197e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009fd0 tx=0x7f4a0c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a117fa640 1 -- 172.21.15.22:0/4255536762 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00ce20 con 0x7f4a14197990 2024-03-09T00:02:00.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a117fa640 1 -- 172.21.15.22:0/4255536762 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c005020 con 0x7f4a14197990 2024-03-09T00:02:00.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a141987a0 con 0x7f4a14197990 2024-03-09T00:02:00.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a117fa640 1 -- 172.21.15.22:0/4255536762 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c010720 con 0x7f4a14197990 2024-03-09T00:02:00.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.687+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a141a49a0 con 0x7f4a14197990 2024-03-09T00:02:00.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.689+0000 7f4a1d595740 1 Processor -- start 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.689+0000 7f4a117fa640 1 -- 172.21.15.22:0/4255536762 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4a0c005540 con 0x7f4a14197990 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.689+0000 7f4a1d595740 1 -- start start 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601410 0x558dc86017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc860f070 0x558dc8611460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a1d595740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc8564e00 con 0x558dc8601d30 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a1d595740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc850db10 con 0x558dc8601410 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a1d595740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc8560ea0 con 0x558dc860f070 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50710/0 (socket says 172.21.15.22:50710) 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 -- 172.21.15.22:0/2560304466 learned_addr learned my addr 172.21.15.22:0/2560304466 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a137fe640 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601410 0x558dc86017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a137fe640 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc860f070 0x558dc8611460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 -- 172.21.15.22:0/2560304466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc860f070 msgr2=0x558dc8611460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc860f070 0x558dc8611460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 -- 172.21.15.22:0/2560304466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601410 msgr2=0x558dc86017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601410 0x558dc86017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 -- 172.21.15.22:0/2560304466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc83437d0 con 0x558dc8601d30 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a117fa640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4a000a2970 0x7f4a000a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a137fe640 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601410 0x558dc86017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a13fff640 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860eb30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4a0402c170 tx=0x7f4a04005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5eef07f3c60344 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a137fe640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4a000a2970 0x7f4a000a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a117fa640 1 -- 172.21.15.22:0/4255536762 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4a0c0cf870 con 0x7f4a14197990 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a0a7fc640 1 -- 172.21.15.22:0/2560304466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04042070 con 0x558dc8601d30 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a0a7fc640 1 -- 172.21.15.22:0/2560304466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a04004030 con 0x558dc8601d30 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.690+0000 7f4a0a7fc640 1 -- 172.21.15.22:0/2560304466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004ec0 con 0x558dc8601d30 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 -- 172.21.15.22:0/2560304466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 msgr2=0x558dc860eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a137fe640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4a000a2970 0x7f4a000a4e30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x558dc8601100 tx=0x7f49fc009530 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860eb30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4a0402c170 tx=0x7f4a04005be0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 -- 172.21.15.22:0/2560304466 shutdown_connections 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc860f070 0x558dc8611460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860eb30 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 --2- 172.21.15.22:0/2560304466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601410 0x558dc86017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 -- 172.21.15.22:0/2560304466 >> 172.21.15.22:0/2560304466 conn(0x558dc85fbe90 msgr2=0x558dc85fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 -- 172.21.15.22:0/2560304466 shutdown_connections 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 -- 172.21.15.22:0/2560304466 wait complete. 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.691+0000 7f4a1d595740 1 -- start start 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 1 Processor -- start 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc8611e50 con 0x558dc8601d30 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc85210a0 con 0x558dc860f070 2024-03-09T00:02:00.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc8611fd0 con 0x558dc8601410 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a1d595740 10 client.0 _ll_register_callbacks cb 0x558dc8520a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50716/0 (socket says 172.21.15.22:50716) 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 -- 172.21.15.22:0/3613133787 learned_addr learned my addr 172.21.15.22:0/3613133787 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a13fff640 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 -- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 msgr2=0x558dc860e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 -- 172.21.15.22:0/3613133787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 msgr2=0x558dc860bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.692+0000 7f4a137fe640 1 -- 172.21.15.22:0/3613133787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dc82793f0 con 0x558dc8601d30 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a137fe640 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860baa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0109c0 tx=0x7f49fc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c34286297c01d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f49f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:00.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f49f37fe640 1 -- 172.21.15.22:0/3613133787 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc021420 con 0x558dc8601d30 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f49f37fe640 1 -- 172.21.15.22:0/3613133787 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f49fc0215c0 con 0x558dc8601d30 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a04004030 con 0x558dc8601d30 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f49f37fe640 1 -- 172.21.15.22:0/3613133787 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49fc0217c0 con 0x558dc8601d30 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 10 client.18036 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558dc82390f0 con 0x558dc8601d30 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f49f37fe640 1 -- 172.21.15.22:0/3613133787 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a04004030 con 0x558dc8601d30 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f49f37fe640 1 -- 172.21.15.22:0/3613133787 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f49fc01fb00 con 0x558dc8601d30 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 10 client.18036 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 20 client.18036 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc861eb20 0x558dc8620f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 4 client.18036 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.693+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dc8572340 con 0x558dc861eb20 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.694+0000 7f4a13fff640 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc861eb20 0x558dc8620f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.694+0000 7f49f37fe640 10 client.18036 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.694+0000 7f4a13fff640 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc861eb20 0x558dc8620f70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdf6e35edfe4c89 server_cookie=9d68442cdb092903 in_seq=0 out_seq=0 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.694+0000 7f49f37fe640 1 -- 172.21.15.22:0/3613133787 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558dc8572340 con 0x558dc861eb20 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.694+0000 7f49f37fe640 10 client.18036 handle_command_reply: tid=0 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 client.18036 shutdown 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 msgr2=0x558dc860baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860baa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0109c0 tx=0x7f49fc005f90 comp rx=0 tx=0).stop 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 shutdown_connections 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc861eb20 0x558dc8620f70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860baa0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f49f37fe640 0 client.18036 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 --2- 172.21.15.22:0/3613133787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 >> 172.21.15.22:0/3613133787 conn(0x558dc85fbe90 msgr2=0x558dc85fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 shutdown_connections 2024-03-09T00:02:00.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 1 -- 172.21.15.22:0/3613133787 wait complete. 2024-03-09T00:02:00.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.695+0000 7f4a1d595740 20 client.18036 trim_cache size 0 max 16384 2024-03-09T00:02:00.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 Processor -- start 2024-03-09T00:02:00.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 -- start start 2024-03-09T00:02:00.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141ae810 con 0x558dc8601d30 2024-03-09T00:02:00.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141ac3b0 con 0x558dc860f070 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a1d595740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141aa450 con 0x558dc8601410 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50720/0 (socket says 172.21.15.22:50720) 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 -- 172.21.15.22:0/797121007 learned_addr learned my addr 172.21.15.22:0/797121007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 -- 172.21.15.22:0/797121007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 msgr2=0x558dc860e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 -- 172.21.15.22:0/797121007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 msgr2=0x558dc860ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 -- 172.21.15.22:0/797121007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc8601b60 con 0x558dc8601d30 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a13fff640 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.711+0000 7f4a137fe640 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860e710 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0059f0 tx=0x7f49fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab90ddc6b973ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.712+0000 7f4a09ffb640 1 -- 172.21.15.22:0/797121007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0043e0 con 0x558dc8601d30 2024-03-09T00:02:00.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.712+0000 7f4a09ffb640 1 -- 172.21.15.22:0/797121007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49fc0039c0 con 0x558dc8601d30 2024-03-09T00:02:00.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.712+0000 7f4a09ffb640 1 -- 172.21.15.22:0/797121007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc005030 con 0x558dc8601d30 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.712+0000 7f4a1d595740 1 -- 172.21.15.22:0/797121007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 msgr2=0x558dc860e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.712+0000 7f4a1d595740 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860e710 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0059f0 tx=0x7f49fc000bc0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- 172.21.15.22:0/797121007 shutdown_connections 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc860f070 0x558dc860ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc8601d30 0x558dc860e710 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 --2- 172.21.15.22:0/797121007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc860e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- 172.21.15.22:0/797121007 >> 172.21.15.22:0/797121007 conn(0x558dc85fbe90 msgr2=0x558dc85fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- 172.21.15.22:0/797121007 shutdown_connections 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- 172.21.15.22:0/797121007 wait complete. 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- start start 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 Processor -- start 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc866e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601d30 0x558dc866e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc860f070 0x558dc8612620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141aaa50 con 0x558dc860f070 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141abd50 con 0x558dc8601d30 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141ab090 con 0x558dc8601410 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a1d595740 10 client.0 _ll_register_callbacks cb 0x7f49fc009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a18f5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc860f070 0x558dc8612620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a18f5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc860f070 0x558dc8612620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50724/0 (socket says 172.21.15.22:50724) 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a18f5c640 1 -- 172.21.15.22:0/1217749969 learned_addr learned my addr 172.21.15.22:0/1217749969 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.713+0000 7f4a13fff640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc866e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a137fe640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601d30 0x558dc866e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a18f5c640 1 -- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 msgr2=0x558dc866e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc866e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a18f5c640 1 -- 172.21.15.22:0/1217749969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601d30 msgr2=0x558dc866e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601d30 0x558dc866e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a18f5c640 1 -- 172.21.15.22:0/1217749969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dc8612e50 con 0x558dc860f070 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a13fff640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc866e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a18f5c640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc860f070 0x558dc8612620 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c097f90 tx=0x7f4a0c09bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b05b8ccfd4a824 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f49f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f49f37fe640 1 -- 172.21.15.22:0/1217749969 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0b9060 con 0x558dc860f070 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f49f37fe640 1 -- 172.21.15.22:0/1217749969 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9451+0+0 (secure 0 0 0) 0x7f4a0c09bd90 con 0x558dc860f070 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49fc0039c0 con 0x558dc860f070 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f49f37fe640 1 -- 172.21.15.22:0/1217749969 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a0c09ce60 con 0x558dc860f070 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a1d595740 10 client.18042 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558dc8675b80 con 0x558dc860f070 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.714+0000 7f49f37fe640 1 -- 172.21.15.22:0/1217749969 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49fc0039c0 con 0x558dc860f070 2024-03-09T00:02:00.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f49f37fe640 1 -- 172.21.15.22:0/1217749969 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a0c0a3e40 con 0x558dc860f070 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a1d595740 10 client.18042 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a1d595740 10 client.18042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a1d595740 20 client.18042 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a1d595740 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc8676200 0x558dc8678620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a1d595740 4 client.18042 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558dc856c690 con 0x558dc8676200 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a13fff640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc8676200 0x558dc8678620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f4a13fff640 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc8676200 0x558dc8678620 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511173235cf7cc84 server_cookie=e0cdf5db605bdb08 in_seq=0 out_seq=0 2024-03-09T00:02:00.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.715+0000 7f49f37fe640 10 client.18042 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f49f37fe640 1 -- 172.21.15.22:0/1217749969 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558dc856c690 con 0x558dc8676200 2024-03-09T00:02:00.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f49f37fe640 10 client.18042 handle_command_reply: tid=0 2024-03-09T00:02:00.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 client.18042 shutdown 2024-03-09T00:02:00.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc860f070 msgr2=0x558dc8612620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc860f070 0x558dc8612620 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c097f90 tx=0x7f4a0c09bbf0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 shutdown_connections 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558dc8676200 0x558dc8678620 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558dc860f070 0x558dc8612620 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dc8601d30 0x558dc866e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 --2- 172.21.15.22:0/1217749969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc8601410 0x558dc866e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f49f37fe640 0 client.18042 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 >> 172.21.15.22:0/1217749969 conn(0x558dc85fbe90 msgr2=0x558dc860ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 shutdown_connections 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 1 -- 172.21.15.22:0/1217749969 wait complete. 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1d595740 20 client.18042 trim_cache size 0 max 16384 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:02:00.717 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: "695beb26-e0f1-40b7-9a78-da461d72b8d5": { 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "695beb26-e0f1-40b7-9a78-da461d72b8d5", 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:02:00.718 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4a000a2970 msgr2=0x7f4a000a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4a000a2970 0x7f4a000a4e30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x558dc8601100 tx=0x7f49fc009530 comp rx=0 tx=0).stop 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a14197990 msgr2=0x7f4a14197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a14197990 0x7f4a14197e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009fd0 tx=0x7f4a0c00ef10 comp rx=0 tx=0).stop 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 shutdown_connections 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4a000a2970 0x7f4a000a4e30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a14197990 0x7f4a14197e40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a141065c0 0x7f4a1419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.718+0000 7f4a1a9bf640 1 --2- 172.21.15.22:0/4255536762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a141014f0 0x7f4a1419d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.719+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 >> 172.21.15.22:0/4255536762 conn(0x7f4a140fcd90 msgr2=0x7f4a140ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.719+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 shutdown_connections 2024-03-09T00:02:00.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:00.719+0000 7f4a1a9bf640 1 -- 172.21.15.22:0/4255536762 wait complete. 2024-03-09T00:02:00.720 INFO:tasks.workunit.client.0.smithi022.stdout: 1 788209 10.88 MB/sec execute 248 sec latency 15.763 ms 2024-03-09T00:02:00.724 INFO:tasks.cephfs.filesystem:scrub status for tag:695beb26-e0f1-40b7-9a78-da461d72b8d5 - {'path': '/', 'tag': '695beb26-e0f1-40b7-9a78-da461d72b8d5', 'options': 'recursive,force'} 2024-03-09T00:02:01.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: osdmap e101: 12 total, 12 up, 12 in 2024-03-09T00:02:01.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6836/2391693225,v1:172.21.15.62:6837/2391693225] up:boot 2024-03-09T00:02:01.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-09T00:02:01.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-09T00:02:01.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:02:01.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:02:01.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:01.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:02:01.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:01 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: osdmap e101: 12 total, 12 up, 12 in 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6836/2391693225,v1:172.21.15.62:6837/2391693225] up:boot 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:02:01.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:01 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:01.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: osdmap e101: 12 total, 12 up, 12 in 2024-03-09T00:02:01.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6836/2391693225,v1:172.21.15.62:6837/2391693225] up:boot 2024-03-09T00:02:01.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-03-09T00:02:01.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-09T00:02:01.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:02:01.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:02:01.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:01.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:02:01.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:01 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:01.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 789179 10.85 MB/sec execute 249 sec latency 103.140 ms 2024-03-09T00:02:02.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:02 smithi062 ceph-mon[36129]: pgmap v234: 129 pgs: 129 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 300 op/s 2024-03-09T00:02:02.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:02 smithi192 ceph-mon[35995]: pgmap v234: 129 pgs: 129 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 300 op/s 2024-03-09T00:02:02.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:02 smithi022 ceph-mon[31894]: pgmap v234: 129 pgs: 129 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 300 op/s 2024-03-09T00:02:02.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 789632 10.82 MB/sec execute 250 sec latency 101.510 ms 2024-03-09T00:02:03.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 792011 10.83 MB/sec execute 251 sec latency 12.860 ms 2024-03-09T00:02:04.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:04 smithi062 ceph-mon[36129]: pgmap v235: 129 pgs: 129 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 300 op/s 2024-03-09T00:02:04.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:04 smithi192 ceph-mon[35995]: pgmap v235: 129 pgs: 129 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 300 op/s 2024-03-09T00:02:04.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:04 smithi022 ceph-mon[31894]: pgmap v235: 129 pgs: 129 active+clean; 90 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 300 op/s 2024-03-09T00:02:04.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 793687 10.81 MB/sec execute 252 sec latency 29.906 ms 2024-03-09T00:02:05.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:05 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:02:05.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:05 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:02:05.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:05 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:02:05.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 795147 10.80 MB/sec execute 253 sec latency 93.136 ms 2024-03-09T00:02:06.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:06 smithi192 ceph-mon[35995]: pgmap v236: 129 pgs: 129 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 457 op/s 2024-03-09T00:02:06.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:06 smithi192 ceph-mon[35995]: osdmap e102: 12 total, 12 up, 12 in 2024-03-09T00:02:06.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:06 smithi022 ceph-mon[31894]: pgmap v236: 129 pgs: 129 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 457 op/s 2024-03-09T00:02:06.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:06 smithi022 ceph-mon[31894]: osdmap e102: 12 total, 12 up, 12 in 2024-03-09T00:02:06.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:06 smithi062 ceph-mon[36129]: pgmap v236: 129 pgs: 129 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 457 op/s 2024-03-09T00:02:06.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:06 smithi062 ceph-mon[36129]: osdmap e102: 12 total, 12 up, 12 in 2024-03-09T00:02:06.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 795545 10.78 MB/sec execute 254 sec latency 76.645 ms 2024-03-09T00:02:07.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 795743 10.74 MB/sec execute 255 sec latency 119.392 ms 2024-03-09T00:02:08.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:08 smithi192 ceph-mon[35995]: pgmap v238: 129 pgs: 129 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 483 op/s 2024-03-09T00:02:08.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:08 smithi022 ceph-mon[31894]: pgmap v238: 129 pgs: 129 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 483 op/s 2024-03-09T00:02:08.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:08 smithi062 ceph-mon[36129]: pgmap v238: 129 pgs: 129 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 483 op/s 2024-03-09T00:02:08.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 796123 10.70 MB/sec execute 256 sec latency 46.035 ms 2024-03-09T00:02:09.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 797097 10.67 MB/sec execute 257 sec latency 29.793 ms 2024-03-09T00:02:10.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:09 smithi022 ceph-mon[31894]: pgmap v239: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-03-09T00:02:10.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:09 smithi062 ceph-mon[36129]: pgmap v239: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-03-09T00:02:10.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:09 smithi192 ceph-mon[35995]: pgmap v239: 129 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 119 active+clean; 76 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-03-09T00:02:10.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 799023 10.68 MB/sec execute 258 sec latency 28.905 ms 2024-03-09T00:02:11.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 800421 10.66 MB/sec execute 259 sec latency 108.235 ms 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.333+0000 7f9b76bcd640 1 -- 172.21.15.22:0/381996745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 msgr2=0x7f9b701077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.333+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/381996745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b701077c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009930 tx=0x7f9b6402bc20 comp rx=0 tx=0).stop 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.333+0000 7f9b76bcd640 1 -- 172.21.15.22:0/381996745 shutdown_connections 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.333+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/381996745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b70069330 0x7f9b7010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.333+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/381996745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b70068970 0x7f9b70068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.333+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/381996745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b701077c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.333+0000 7f9b76bcd640 1 -- 172.21.15.22:0/381996745 >> 172.21.15.22:0/381996745 conn(0x7f9b70077040 msgr2=0x7f9b70077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 -- 172.21.15.22:0/381996745 shutdown_connections 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 -- 172.21.15.22:0/381996745 wait complete. 2024-03-09T00:02:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 Processor -- start 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 -- start start 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b70068970 0x7f9b70100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b70069330 0x7f9b70100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b70100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701116b0 con 0x7f9b701073e0 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b70111530 con 0x7f9b70069330 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.334+0000 7f9b76bcd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b70111830 con 0x7f9b70068970 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b70100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b70100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47842/0 (socket says 172.21.15.22:47842) 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 -- 172.21.15.22:0/679384344 learned_addr learned my addr 172.21.15.22:0/679384344 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b74969640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b70068970 0x7f9b70100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b70069330 0x7f9b70100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 -- 172.21.15.22:0/679384344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b70068970 msgr2=0x7f9b70100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b70068970 0x7f9b70100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 -- 172.21.15.22:0/679384344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b70069330 msgr2=0x7f9b70100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b70069330 0x7f9b70100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 -- 172.21.15.22:0/679384344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b701a5fb0 con 0x7f9b701073e0 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b7516a640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b70100f90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800ea60 tx=0x7f9b6800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b6dffb640 1 -- 172.21.15.22:0/679384344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800ce40 con 0x7f9b701073e0 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b701a62a0 con 0x7f9b701073e0 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b6dffb640 1 -- 172.21.15.22:0/679384344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b68005020 con 0x7f9b701073e0 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b6dffb640 1 -- 172.21.15.22:0/679384344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68010690 con 0x7f9b701073e0 2024-03-09T00:02:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.335+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b701a67f0 con 0x7f9b701073e0 2024-03-09T00:02:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.337+0000 7f9b6dffb640 1 -- 172.21.15.22:0/679384344 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9b680051d0 con 0x7f9b701073e0 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b6dffb640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b600a2a10 0x7f9b600a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 Processor -- start 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 -- start start 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b6dffb640 1 -- 172.21.15.22:0/679384344 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f9b680cf790 con 0x7f9b701073e0 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b74969640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b600a2a10 0x7f9b600a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c039ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c039d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c03015f0 con 0x55f4c039cfd0 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c02aa930 con 0x55f4c039dd40 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b797ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c02fd690 con 0x55f4c03a6550 2024-03-09T00:02:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c039d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c039d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47852/0 (socket says 172.21.15.22:47852) 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.338+0000 7f9b6ffff640 1 -- 172.21.15.22:0/1652476044 learned_addr learned my addr 172.21.15.22:0/1652476044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b74969640 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c039ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b7516a640 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b74969640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b600a2a10 0x7f9b600a4ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009900 tx=0x7f9b64005a20 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b6ffff640 1 -- 172.21.15.22:0/1652476044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 msgr2=0x55f4c03a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b6ffff640 1 -- 172.21.15.22:0/1652476044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 msgr2=0x55f4c039ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c039ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b6ffff640 1 -- 172.21.15.22:0/1652476044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4c0016210 con 0x55f4c039cfd0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c039d450 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009ca0 tx=0x7f9b5c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b4fa1eb050aac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b5affd640 1 -- 172.21.15.22:0/1652476044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c01e070 con 0x55f4c039cfd0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b5affd640 1 -- 172.21.15.22:0/1652476044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b5c019b10 con 0x55f4c039cfd0 2024-03-09T00:02:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.339+0000 7f9b5affd640 1 -- 172.21.15.22:0/1652476044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c00ea20 con 0x55f4c039cfd0 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 -- 172.21.15.22:0/1652476044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 msgr2=0x55f4c039d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c039d450 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009ca0 tx=0x7f9b5c011810 comp rx=0 tx=0).stop 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 -- 172.21.15.22:0/1652476044 shutdown_connections 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c039d450 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 --2- 172.21.15.22:0/1652476044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c039ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 -- 172.21.15.22:0/1652476044 >> 172.21.15.22:0/1652476044 conn(0x55f4c03645b0 msgr2=0x55f4c0397cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 -- 172.21.15.22:0/1652476044 shutdown_connections 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 -- 172.21.15.22:0/1652476044 wait complete. 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 -- start start 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.340+0000 7f9b797ac740 1 Processor -- start 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039dd40 0x55f4c03a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b797ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c03a9280 con 0x55f4c039dd40 2024-03-09T00:02:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b797ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c02bdd90 con 0x55f4c039cfd0 2024-03-09T00:02:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b797ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4c03a9400 con 0x55f4c03a6550 2024-03-09T00:02:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b797ac740 10 client.0 _ll_register_callbacks cb 0x55f4c0015a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039dd40 0x55f4c03a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039dd40 0x55f4c03a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47868/0 (socket says 172.21.15.22:47868) 2024-03-09T00:02:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 -- 172.21.15.22:0/814707918 learned_addr learned my addr 172.21.15.22:0/814707918 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b74969640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b7516a640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 -- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 msgr2=0x55f4c03a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 -- 172.21.15.22:0/814707918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 msgr2=0x55f4c03a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 -- 172.21.15.22:0/814707918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4c00e2620 con 0x55f4c039dd40 2024-03-09T00:02:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b74969640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b7516a640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:02:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039dd40 0x55f4c03a3b30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009c70 tx=0x7f9b5c019a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e4d92e9b4a4b78 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b4ffff640 1 -- 172.21.15.22:0/814707918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c027030 con 0x55f4c039dd40 2024-03-09T00:02:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.341+0000 7f9b4ffff640 1 -- 172.21.15.22:0/814707918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f9b5c005260 con 0x55f4c039dd40 2024-03-09T00:02:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b5c019b10 con 0x55f4c039dd40 2024-03-09T00:02:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b4ffff640 1 -- 172.21.15.22:0/814707918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b5c031050 con 0x55f4c039dd40 2024-03-09T00:02:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 10 client.18054 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f4c03b55c0 con 0x55f4c039dd40 2024-03-09T00:02:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b4ffff640 1 -- 172.21.15.22:0/814707918 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b5c019b10 con 0x55f4c039dd40 2024-03-09T00:02:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b4ffff640 1 -- 172.21.15.22:0/814707918 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b5c0047e0 con 0x55f4c039dd40 2024-03-09T00:02:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 10 client.18054 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 20 client.18054 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c03b69c0 0x55f4c03b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 4 client.18054 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.342+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4c030eb30 con 0x55f4c03b69c0 2024-03-09T00:02:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.359+0000 7f9b74969640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c03b69c0 0x55f4c03b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.359+0000 7f9b74969640 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c03b69c0 0x55f4c03b8db0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18b6720e81519f8 server_cookie=8d17933f592c6b20 in_seq=0 out_seq=0 2024-03-09T00:02:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.359+0000 7f9b4ffff640 10 client.18054 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b4ffff640 1 -- 172.21.15.22:0/814707918 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4c030eb30 con 0x55f4c03b69c0 2024-03-09T00:02:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b4ffff640 10 client.18054 handle_command_reply: tid=0 2024-03-09T00:02:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b797ac740 1 client.18054 shutdown 2024-03-09T00:02:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039dd40 msgr2=0x55f4c03a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b797ac740 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039dd40 0x55f4c03a3b30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009c70 tx=0x7f9b5c019a90 comp rx=0 tx=0).stop 2024-03-09T00:02:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 shutdown_connections 2024-03-09T00:02:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b797ac740 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c03b69c0 0x55f4c03b8db0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.360+0000 7f9b797ac740 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.361+0000 7f9b4ffff640 0 client.18054 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.361+0000 7f9b797ac740 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039dd40 0x55f4c03a3b30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.361+0000 7f9b797ac740 1 --2- 172.21.15.22:0/814707918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.361+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 >> 172.21.15.22:0/814707918 conn(0x55f4c03645b0 msgr2=0x55f4c0399a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.361+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 shutdown_connections 2024-03-09T00:02:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.361+0000 7f9b797ac740 1 -- 172.21.15.22:0/814707918 wait complete. 2024-03-09T00:02:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.361+0000 7f9b797ac740 20 client.18054 trim_cache size 0 max 16384 2024-03-09T00:02:12.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:12 smithi062 ceph-mon[36129]: pgmap v240: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 368 op/s 2024-03-09T00:02:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 Processor -- start 2024-03-09T00:02:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 -- start start 2024-03-09T00:02:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c039dd40 0x55f4c03ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c03a6550 0x55f4c03adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701b2e50 con 0x55f4c03a6550 2024-03-09T00:02:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701b09f0 con 0x55f4c039cfd0 2024-03-09T00:02:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b797ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701aeab0 con 0x55f4c039dd40 2024-03-09T00:02:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c03a6550 0x55f4c03adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b74969640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c03a6550 0x55f4c03adbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47872/0 (socket says 172.21.15.22:47872) 2024-03-09T00:02:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 -- 172.21.15.22:0/781735743 learned_addr learned my addr 172.21.15.22:0/781735743 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c039dd40 0x55f4c03ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 -- 172.21.15.22:0/781735743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c039dd40 msgr2=0x55f4c03ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c039dd40 0x55f4c03ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 -- 172.21.15.22:0/781735743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 msgr2=0x55f4c03a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b7516a640 1 -- 172.21.15.22:0/781735743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4c03a9160 con 0x55f4c03a6550 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.375+0000 7f9b74969640 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.376+0000 7f9b7516a640 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c03a6550 0x55f4c03adbb0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9b680ccaf0 tx=0x7f9b6809b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b749068f5efbd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.376+0000 7f9b5a7fc640 1 -- 172.21.15.22:0/781735743 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68099ca0 con 0x55f4c03a6550 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.376+0000 7f9b5a7fc640 1 -- 172.21.15.22:0/781735743 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b680ccd10 con 0x55f4c03a6550 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.376+0000 7f9b5a7fc640 1 -- 172.21.15.22:0/781735743 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b680a3cb0 con 0x55f4c03a6550 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.376+0000 7f9b797ac740 1 -- 172.21.15.22:0/781735743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c03a6550 msgr2=0x55f4c03adbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.376+0000 7f9b797ac740 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c03a6550 0x55f4c03adbb0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9b680ccaf0 tx=0x7f9b6809b7f0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- 172.21.15.22:0/781735743 shutdown_connections 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c03a6550 0x55f4c03adbb0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c039dd40 0x55f4c03ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 --2- 172.21.15.22:0/781735743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039cfd0 0x55f4c03a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- 172.21.15.22:0/781735743 >> 172.21.15.22:0/781735743 conn(0x55f4c03645b0 msgr2=0x55f4c03977f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- 172.21.15.22:0/781735743 shutdown_connections 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- 172.21.15.22:0/781735743 wait complete. 2024-03-09T00:02:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- start start 2024-03-09T00:02:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 Processor -- start 2024-03-09T00:02:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c03a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c03a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701af0b0 con 0x55f4c039cfd0 2024-03-09T00:02:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701b0390 con 0x55f4c039dd40 2024-03-09T00:02:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.377+0000 7f9b797ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701af6d0 con 0x55f4c03a6550 2024-03-09T00:02:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b797ac740 10 client.0 _ll_register_callbacks cb 0x7f9b5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c03a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c03a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c03a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47878/0 (socket says 172.21.15.22:47878) 2024-03-09T00:02:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b7516a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 -- 172.21.15.22:0/821228201 learned_addr learned my addr 172.21.15.22:0/821228201 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 -- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 msgr2=0x55f4c03a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 -- 172.21.15.22:0/821228201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 msgr2=0x55f4c03a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c03a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 -- 172.21.15.22:0/821228201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4c040ce00 con 0x55f4c039cfd0 2024-03-09T00:02:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c03a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b7516a640 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b74969640 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c03a02c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009930 tx=0x7f9b6402bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad5be4042267aec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b4ffff640 1 -- 172.21.15.22:0/821228201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64059060 con 0x55f4c039cfd0 2024-03-09T00:02:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b4ffff640 1 -- 172.21.15.22:0/821228201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f9b6403a040 con 0x55f4c039cfd0 2024-03-09T00:02:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b680ccd10 con 0x55f4c039cfd0 2024-03-09T00:02:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b4ffff640 1 -- 172.21.15.22:0/821228201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b64060050 con 0x55f4c039cfd0 2024-03-09T00:02:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.378+0000 7f9b797ac740 10 client.18060 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f4c040e170 con 0x55f4c039cfd0 2024-03-09T00:02:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b4ffff640 1 -- 172.21.15.22:0/821228201 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b680ccd10 con 0x55f4c039cfd0 2024-03-09T00:02:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b4ffff640 1 -- 172.21.15.22:0/821228201 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9b640544a0 con 0x55f4c039cfd0 2024-03-09T00:02:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b797ac740 10 client.18060 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b797ac740 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b797ac740 20 client.18060 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b797ac740 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c040e7f0 0x55f4c0410c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b797ac740 4 client.18060 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:02:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4c0308e80 con 0x55f4c040e7f0 2024-03-09T00:02:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.379+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c040e7f0 0x55f4c0410c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.380+0000 7f9b4ffff640 10 client.18060 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.380+0000 7f9b6ffff640 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c040e7f0 0x55f4c0410c10 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4650ec88bc6f87ae server_cookie=3f6dfc779dfd270d in_seq=0 out_seq=0 2024-03-09T00:02:12.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:12 smithi192 ceph-mon[35995]: pgmap v240: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 368 op/s 2024-03-09T00:02:12.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:12 smithi022 ceph-mon[31894]: pgmap v240: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 368 op/s 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.669+0000 7f9b4ffff640 1 -- 172.21.15.22:0/821228201 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4c0308e80 con 0x55f4c040e7f0 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.669+0000 7f9b4ffff640 10 client.18060 handle_command_reply: tid=0 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 client.18060 shutdown 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 msgr2=0x55f4c03a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c03a02c0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009930 tx=0x7f9b6402bc20 comp rx=0 tx=0).stop 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 shutdown_connections 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4c040e7f0 0x55f4c0410c10 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4c03a6550 0x55f4c03a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4c039dd40 0x55f4c03a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b4ffff640 0 client.18060 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 --2- 172.21.15.22:0/821228201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4c039cfd0 0x55f4c03a02c0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 >> 172.21.15.22:0/821228201 conn(0x55f4c03645b0 msgr2=0x55f4c03a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 shutdown_connections 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 1 -- 172.21.15.22:0/821228201 wait complete. 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.670+0000 7f9b797ac740 20 client.18060 trim_cache size 0 max 16384 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:02:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b600a2a10 msgr2=0x7f9b600a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b600a2a10 0x7f9b600a4ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009900 tx=0x7f9b64005a20 comp rx=0 tx=0).stop 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 msgr2=0x7f9b70100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b70100f90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800ea60 tx=0x7f9b6800ef20 comp rx=0 tx=0).stop 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 shutdown_connections 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9b600a2a10 0x7f9b600a4ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b701073e0 0x7f9b70100f90 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b70069330 0x7f9b70100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 --2- 172.21.15.22:0/679384344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b70068970 0x7f9b70100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.671+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 >> 172.21.15.22:0/679384344 conn(0x7f9b70077040 msgr2=0x7f9b700ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.674+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 shutdown_connections 2024-03-09T00:02:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:12.674+0000 7f9b76bcd640 1 -- 172.21.15.22:0/679384344 wait complete. 2024-03-09T00:02:12.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 802353 10.67 MB/sec execute 260 sec latency 111.499 ms 2024-03-09T00:02:13.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 803846 10.66 MB/sec execute 261 sec latency 35.176 ms 2024-03-09T00:02:14.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:14 smithi062 ceph-mon[36129]: pgmap v241: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 368 op/s 2024-03-09T00:02:14.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:14 smithi062 ceph-mon[36129]: osdmap e103: 12 total, 12 up, 12 in 2024-03-09T00:02:14.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:14 smithi192 ceph-mon[35995]: pgmap v241: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 368 op/s 2024-03-09T00:02:14.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:14 smithi192 ceph-mon[35995]: osdmap e103: 12 total, 12 up, 12 in 2024-03-09T00:02:14.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:14 smithi022 ceph-mon[31894]: pgmap v241: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 368 op/s 2024-03-09T00:02:14.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:14 smithi022 ceph-mon[31894]: osdmap e103: 12 total, 12 up, 12 in 2024-03-09T00:02:14.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 806061 10.66 MB/sec execute 262 sec latency 35.223 ms 2024-03-09T00:02:15.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 807531 10.65 MB/sec execute 263 sec latency 28.137 ms 2024-03-09T00:02:16.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:16 smithi062 ceph-mon[36129]: pgmap v243: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-03-09T00:02:16.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:16 smithi192 ceph-mon[35995]: pgmap v243: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-03-09T00:02:16.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:16 smithi022 ceph-mon[31894]: pgmap v243: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-03-09T00:02:16.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 809802 10.66 MB/sec execute 264 sec latency 15.765 ms 2024-03-09T00:02:17.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 811151 10.64 MB/sec execute 265 sec latency 90.797 ms 2024-03-09T00:02:18.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:18 smithi062 ceph-mon[36129]: pgmap v244: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-03-09T00:02:18.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:18 smithi192 ceph-mon[35995]: pgmap v244: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-03-09T00:02:18.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:18 smithi022 ceph-mon[31894]: pgmap v244: 129 pgs: 129 active+clean; 69 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-03-09T00:02:18.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 813408 10.65 MB/sec execute 266 sec latency 34.775 ms 2024-03-09T00:02:19.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 815244 10.63 MB/sec execute 267 sec latency 18.198 ms 2024-03-09T00:02:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:19 smithi022 ceph-mon[31894]: osdmap e104: 12 total, 12 up, 12 in 2024-03-09T00:02:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:19 smithi022 ceph-mon[31894]: pgmap v246: 129 pgs: 129 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-03-09T00:02:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:19 smithi062 ceph-mon[36129]: osdmap e104: 12 total, 12 up, 12 in 2024-03-09T00:02:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:19 smithi062 ceph-mon[36129]: pgmap v246: 129 pgs: 129 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-03-09T00:02:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:19 smithi192 ceph-mon[35995]: osdmap e104: 12 total, 12 up, 12 in 2024-03-09T00:02:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:19 smithi192 ceph-mon[35995]: pgmap v246: 129 pgs: 129 active+clean; 72 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-03-09T00:02:20.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 817196 10.64 MB/sec execute 268 sec latency 14.987 ms 2024-03-09T00:02:21.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 819512 10.65 MB/sec execute 269 sec latency 16.008 ms 2024-03-09T00:02:22.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:22 smithi062 ceph-mon[36129]: pgmap v247: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 439 op/s 2024-03-09T00:02:22.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:22 smithi192 ceph-mon[35995]: pgmap v247: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 439 op/s 2024-03-09T00:02:22.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:22 smithi022 ceph-mon[31894]: pgmap v247: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 439 op/s 2024-03-09T00:02:22.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 820868 10.64 MB/sec execute 270 sec latency 96.651 ms 2024-03-09T00:02:23.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 821914 10.62 MB/sec execute 271 sec latency 23.422 ms 2024-03-09T00:02:24.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:24 smithi062 ceph-mon[36129]: pgmap v248: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 387 op/s 2024-03-09T00:02:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:24 smithi192 ceph-mon[35995]: pgmap v248: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 387 op/s 2024-03-09T00:02:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:24 smithi022 ceph-mon[31894]: pgmap v248: 129 pgs: 129 active+clean; 79 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 387 op/s 2024-03-09T00:02:24.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 824190 10.63 MB/sec execute 272 sec latency 15.413 ms 2024-03-09T00:02:25.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 826279 10.62 MB/sec execute 273 sec latency 14.917 ms 2024-03-09T00:02:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.772+0000 7fe90909a640 1 -- 172.21.15.22:0/3909296657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904105210 msgr2=0x7fe9041055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.772+0000 7fe90909a640 1 --2- 172.21.15.22:0/3909296657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904105210 0x7fe9041055f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009930 tx=0x7fe8f402bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.772+0000 7fe90909a640 1 -- 172.21.15.22:0/3909296657 shutdown_connections 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.772+0000 7fe90909a640 1 --2- 172.21.15.22:0/3909296657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe904100e70 0x7fe90410d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.772+0000 7fe90909a640 1 --2- 172.21.15.22:0/3909296657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9041004d0 0x7fe904100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.772+0000 7fe90909a640 1 --2- 172.21.15.22:0/3909296657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904105210 0x7fe9041055f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.772+0000 7fe90909a640 1 -- 172.21.15.22:0/3909296657 >> 172.21.15.22:0/3909296657 conn(0x7fe9040771d0 msgr2=0x7fe9040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.773+0000 7fe90909a640 1 -- 172.21.15.22:0/3909296657 shutdown_connections 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.773+0000 7fe90909a640 1 -- 172.21.15.22:0/3909296657 wait complete. 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.773+0000 7fe90909a640 1 Processor -- start 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90909a640 1 -- start start 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90909a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9041004d0 0x7fe904198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90909a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904100e70 0x7fe904198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90909a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe904105210 0x7fe9041a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90909a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041105e0 con 0x7fe904100e70 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90909a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe904110460 con 0x7fe9041004d0 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90909a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe904110760 con 0x7fe904105210 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904100e70 0x7fe904198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904100e70 0x7fe904198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35028/0 (socket says 172.21.15.22:35028) 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 -- 172.21.15.22:0/2198182993 learned_addr learned my addr 172.21.15.22:0/2198182993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90359e640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe904105210 0x7fe9041a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe902d9d640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9041004d0 0x7fe904198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 -- 172.21.15.22:0/2198182993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe904105210 msgr2=0x7fe9041a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe904105210 0x7fe9041a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 -- 172.21.15.22:0/2198182993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9041004d0 msgr2=0x7fe904198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9041004d0 0x7fe904198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe8fa59c640 1 -- 172.21.15.22:0/2198182993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9041a3ec0 con 0x7fe904100e70 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe90359e640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe904105210 0x7fe9041a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.774+0000 7fe902d9d640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9041004d0 0x7fe904198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.775+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904100e70 0x7fe904198e40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d950 tx=0x7fe8ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.775+0000 7fe900d99640 1 -- 172.21.15.22:0/2198182993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec017070 con 0x7fe904100e70 2024-03-09T00:02:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.775+0000 7fe900d99640 1 -- 172.21.15.22:0/2198182993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8ec009c30 con 0x7fe904100e70 2024-03-09T00:02:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.775+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9041a4090 con 0x7fe904100e70 2024-03-09T00:02:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.775+0000 7fe900d99640 1 -- 172.21.15.22:0/2198182993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00f6b0 con 0x7fe904100e70 2024-03-09T00:02:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.775+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9041a4630 con 0x7fe904100e70 2024-03-09T00:02:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.777+0000 7fe900d99640 1 -- 172.21.15.22:0/2198182993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe8ec0109d0 con 0x7fe904100e70 2024-03-09T00:02:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.777+0000 7fe90bc7f740 1 Processor -- start 2024-03-09T00:02:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.777+0000 7fe90bc7f740 1 -- start start 2024-03-09T00:02:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303f1d40 0x55ef303f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303fa550 0x55ef303fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef303555f0 con 0x55ef303f0fd0 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef302fe930 con 0x55ef303fa550 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef30351690 con 0x55ef303f1d40 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35030/0 (socket says 172.21.15.22:35030) 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 -- 172.21.15.22:0/795751019 learned_addr learned my addr 172.21.15.22:0/795751019 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303f1d40 0x55ef303f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe900d99640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8f00a2a90 0x7fe8f00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe90359e640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8f00a2a90 0x7fe8f00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 -- 172.21.15.22:0/795751019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303f1d40 msgr2=0x55ef303f0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 --2- 172.21.15.22:0/795751019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303f1d40 0x55ef303f0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 -- 172.21.15.22:0/795751019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303fa550 msgr2=0x55ef303fc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 --2- 172.21.15.22:0/795751019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303fa550 0x55ef303fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe902d9d640 1 -- 172.21.15.22:0/795751019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3006a210 con 0x55ef303f0fd0 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.778+0000 7fe900d99640 1 -- 172.21.15.22:0/2198182993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fe8ec0ceda0 con 0x7fe904100e70 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.779+0000 7fe902d9d640 1 --2- 172.21.15.22:0/795751019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f1450 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009930 tx=0x7fe8f4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314644b8337abb07 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.779+0000 7fe90359e640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8f00a2a90 0x7fe8f00a4f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0094d0 tx=0x7fe8fc004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.779+0000 7fe8f959a640 1 -- 172.21.15.22:0/795751019 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004c70 con 0x55ef303f0fd0 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.779+0000 7fe8f959a640 1 -- 172.21.15.22:0/795751019 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8f402c6d0 con 0x55ef303f0fd0 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.779+0000 7fe8f959a640 1 -- 172.21.15.22:0/795751019 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4041a90 con 0x55ef303f0fd0 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 -- 172.21.15.22:0/795751019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 msgr2=0x55ef303f1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/795751019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f1450 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009930 tx=0x7fe8f4005dd0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 -- 172.21.15.22:0/795751019 shutdown_connections 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/795751019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303fa550 0x55ef303fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/795751019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f1450 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/795751019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303f1d40 0x55ef303f0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 -- 172.21.15.22:0/795751019 >> 172.21.15.22:0/795751019 conn(0x55ef303b85b0 msgr2=0x55ef303ebcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 -- 172.21.15.22:0/795751019 shutdown_connections 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 -- 172.21.15.22:0/795751019 wait complete. 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 -- start start 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.780+0000 7fe90bc7f740 1 Processor -- start 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef303f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303f8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef303fd220 con 0x55ef303f0fd0 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef30311d90 con 0x55ef303f1d40 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef303fd3a0 con 0x55ef303fa550 2024-03-09T00:02:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 10 client.0 _ll_register_callbacks cb 0x55ef30069a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35042/0 (socket says 172.21.15.22:35042) 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 -- 172.21.15.22:0/2883889477 learned_addr learned my addr 172.21.15.22:0/2883889477 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef303f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90359e640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 -- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 msgr2=0x55ef303f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 -- 172.21.15.22:0/2883889477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 msgr2=0x55ef303f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef303f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 -- 172.21.15.22:0/2883889477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef30136620 con 0x55ef303f0fd0 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef303f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe902d9d640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f7600 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005ab0 tx=0x7fe8f4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88b3d0d18811dd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe8ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/2883889477 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4041810 con 0x55ef303f0fd0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/2883889477 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fe8f40419b0 con 0x55ef303f0fd0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.781+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8f402c6d0 con 0x55ef303f0fd0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/2883889477 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f405d050 con 0x55ef303f0fd0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 10 client.18072 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ef30409590 con 0x55ef303f0fd0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/2883889477 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8f402c6d0 con 0x55ef303f0fd0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/2883889477 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe8f4034440 con 0x55ef303f0fd0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 10 client.18072 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 20 client.18072 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3040a8e0 0x55ef3040ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 4 client.18072 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef30362b30 con 0x55ef3040a8e0 2024-03-09T00:02:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.782+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3040a8e0 0x55ef3040ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.783+0000 7fe8ea7fc640 10 client.18072 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.783+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3040a8e0 0x55ef3040ccd0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a84d200a1dc9abd server_cookie=3448d0d0dfe80858 in_seq=0 out_seq=0 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.783+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/2883889477 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef30362b30 con 0x55ef3040a8e0 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.783+0000 7fe8ea7fc640 10 client.18072 handle_command_reply: tid=0 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.783+0000 7fe90bc7f740 1 client.18072 shutdown 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.783+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 msgr2=0x55ef303f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.783+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f7600 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005ab0 tx=0x7fe8f4004d10 comp rx=0 tx=0).stop 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 shutdown_connections 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef3040a8e0 0x55ef3040ccd0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef303f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe8ea7fc640 0 client.18072 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/2883889477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f7600 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 >> 172.21.15.22:0/2883889477 conn(0x55ef303b85b0 msgr2=0x55ef303eda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 shutdown_connections 2024-03-09T00:02:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 1 -- 172.21.15.22:0/2883889477 wait complete. 2024-03-09T00:02:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.784+0000 7fe90bc7f740 20 client.18072 trim_cache size 0 max 16384 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.798+0000 7fe90bc7f740 1 Processor -- start 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.798+0000 7fe90bc7f740 1 -- start start 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.798+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.798+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef30401b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041ae7c0 con 0x55ef303f0fd0 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041ac360 con 0x55ef303f1d40 2024-03-09T00:02:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041aa420 con 0x55ef303fa550 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35052/0 (socket says 172.21.15.22:35052) 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 -- 172.21.15.22:0/3051637794 learned_addr learned my addr 172.21.15.22:0/3051637794 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe90359e640 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef30401b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 -- 172.21.15.22:0/3051637794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 msgr2=0x55ef30401b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef30401b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 -- 172.21.15.22:0/3051637794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 msgr2=0x55ef30401610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 -- 172.21.15.22:0/3051637794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef303f2440 con 0x55ef303f0fd0 2024-03-09T00:02:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe902d9d640 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f9d30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402bc70 tx=0x7fe8f402c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a43c23944247f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe8f8d99640 1 -- 172.21.15.22:0/3051637794 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4034a00 con 0x55ef303f0fd0 2024-03-09T00:02:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe8f8d99640 1 -- 172.21.15.22:0/3051637794 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8f4034ba0 con 0x55ef303f0fd0 2024-03-09T00:02:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.799+0000 7fe8f8d99640 1 -- 172.21.15.22:0/3051637794 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004ef0 con 0x55ef303f0fd0 2024-03-09T00:02:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 -- 172.21.15.22:0/3051637794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 msgr2=0x55ef303f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f9d30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402bc70 tx=0x7fe8f402c510 comp rx=0 tx=0).stop 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 -- 172.21.15.22:0/3051637794 shutdown_connections 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef30401b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/3051637794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef303f9d30 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 -- 172.21.15.22:0/3051637794 >> 172.21.15.22:0/3051637794 conn(0x55ef303b85b0 msgr2=0x55ef303f2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 -- 172.21.15.22:0/3051637794 shutdown_connections 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.800+0000 7fe90bc7f740 1 -- 172.21.15.22:0/3051637794 wait complete. 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 -- start start 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 Processor -- start 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef30401540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041aaa20 con 0x55ef303f0fd0 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041abd00 con 0x55ef303f1d40 2024-03-09T00:02:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041ab040 con 0x55ef303fa550 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90bc7f740 10 client.0 _ll_register_callbacks cb 0x7fe8f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef30401540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef30401540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35058/0 (socket says 172.21.15.22:35058) 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 -- 172.21.15.22:0/1005506142 learned_addr learned my addr 172.21.15.22:0/1005506142 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe90359e640 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 -- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 msgr2=0x55ef303fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 -- 172.21.15.22:0/1005506142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 msgr2=0x55ef30401a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.801+0000 7fe902d9d640 1 -- 172.21.15.22:0/1005506142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef303f02a0 con 0x55ef303f0fd0 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe902d9d640 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef30401540 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402c170 tx=0x7fe8f4034890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39aa4a62cf7e31d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe8ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/1005506142 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4066070 con 0x55ef303f0fd0 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/1005506142 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fe8f4034440 con 0x55ef303f0fd0 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef3035ce80 con 0x55ef303f0fd0 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/1005506142 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f4034640 con 0x55ef303f0fd0 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe90bc7f740 10 client.18078 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ef30461450 con 0x55ef303f0fd0 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.802+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/1005506142 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ef3035ce80 con 0x55ef303f0fd0 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/1005506142 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe8f4005260 con 0x55ef303f0fd0 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe90bc7f740 10 client.18078 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe90bc7f740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe90bc7f740 20 client.18078 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef30461900 0x55ef30463d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe90bc7f740 4 client.18078 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:02:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef302db710 con 0x55ef30461900 2024-03-09T00:02:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef30461900 0x55ef30463d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe8fa59c640 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef30461900 0x55ef30463d20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667e87375fc5ec46 server_cookie=eba5bd4d6dcb24c1 in_seq=0 out_seq=0 2024-03-09T00:02:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:25.803+0000 7fe8ea7fc640 10 client.18078 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe8ea7fc640 1 -- 172.21.15.22:0/1005506142 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef302db710 con 0x55ef30461900 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe8ea7fc640 10 client.18078 handle_command_reply: tid=0 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 client.18078 shutdown 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 msgr2=0x55ef30401540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef30401540 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe8f402c170 tx=0x7fe8f4034890 comp rx=0 tx=0).stop 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 shutdown_connections 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ef30461900 0x55ef30463d20 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef303fa550 0x55ef303fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ef303f1d40 0x55ef30401a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 --2- 172.21.15.22:0/1005506142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef303f0fd0 0x55ef30401540 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe8ea7fc640 0 client.18078 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 >> 172.21.15.22:0/1005506142 conn(0x55ef303b85b0 msgr2=0x55ef303eb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 shutdown_connections 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 1 -- 172.21.15.22:0/1005506142 wait complete. 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.124+0000 7fe90bc7f740 20 client.18078 trim_cache size 0 max 16384 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:02:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:02:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:02:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8f00a2a90 msgr2=0x7fe8f00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8f00a2a90 0x7fe8f00a4f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0094d0 tx=0x7fe8fc004ab0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904100e70 msgr2=0x7fe904198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904100e70 0x7fe904198e40 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d950 tx=0x7fe8ec00de10 comp rx=0 tx=0).stop 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 shutdown_connections 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8f00a2a90 0x7fe8f00a4f50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe904105210 0x7fe9041a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe904100e70 0x7fe904198e40 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 --2- 172.21.15.22:0/2198182993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9041004d0 0x7fe904198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 >> 172.21.15.22:0/2198182993 conn(0x7fe9040771d0 msgr2=0x7fe9040fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 shutdown_connections 2024-03-09T00:02:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:26.125+0000 7fe90909a640 1 -- 172.21.15.22:0/2198182993 wait complete. 2024-03-09T00:02:26.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:26 smithi062 ceph-mon[36129]: pgmap v249: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-03-09T00:02:26.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:26 smithi192 ceph-mon[35995]: pgmap v249: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-03-09T00:02:26.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:26 smithi022 ceph-mon[31894]: pgmap v249: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-03-09T00:02:26.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 827683 10.62 MB/sec execute 274 sec latency 248.603 ms 2024-03-09T00:02:27.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 829089 10.60 MB/sec execute 275 sec latency 75.711 ms 2024-03-09T00:02:28.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:28 smithi062 ceph-mon[36129]: pgmap v250: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-03-09T00:02:28.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:28 smithi192 ceph-mon[35995]: pgmap v250: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-03-09T00:02:28.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:28 smithi022 ceph-mon[31894]: pgmap v250: 129 pgs: 129 active+clean; 85 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 255 op/s 2024-03-09T00:02:28.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 831168 10.61 MB/sec execute 276 sec latency 46.531 ms 2024-03-09T00:02:29.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 833200 10.60 MB/sec execute 277 sec latency 20.554 ms 2024-03-09T00:02:30.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:30 smithi062 ceph-mon[36129]: pgmap v251: 129 pgs: 129 active+clean; 81 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 361 op/s 2024-03-09T00:02:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:30 smithi192 ceph-mon[35995]: pgmap v251: 129 pgs: 129 active+clean; 81 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 361 op/s 2024-03-09T00:02:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:30 smithi022 ceph-mon[31894]: pgmap v251: 129 pgs: 129 active+clean; 81 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 361 op/s 2024-03-09T00:02:30.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 835124 10.61 MB/sec execute 278 sec latency 20.581 ms 2024-03-09T00:02:30.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:02:30.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:02:30.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 -- 172.21.15.22:0/510770478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 msgr2=0x7ff7601078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 --2- 172.21.15.22:0/510770478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601078d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff754009930 tx=0x7ff75402bcd0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 -- 172.21.15.22:0/510770478 shutdown_connections 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 --2- 172.21.15.22:0/510770478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff760100570 0x7ff76010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 --2- 172.21.15.22:0/510770478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7600ffbb0 0x7ff760100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 --2- 172.21.15.22:0/510770478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601078d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 -- 172.21.15.22:0/510770478 >> 172.21.15.22:0/510770478 conn(0x7ff760075530 msgr2=0x7ff760075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 -- 172.21.15.22:0/510770478 shutdown_connections 2024-03-09T00:02:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.958+0000 7ff7671fa640 1 -- 172.21.15.22:0/510770478 wait complete. 2024-03-09T00:02:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 Processor -- start 2024-03-09T00:02:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 -- start start 2024-03-09T00:02:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7600ffbb0 0x7ff76019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff760100570 0x7ff76019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760111710 con 0x7ff7601074f0 2024-03-09T00:02:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760111590 con 0x7ff7600ffbb0 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff7671fa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760111890 con 0x7ff760100570 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36202/0 (socket says 172.21.15.22:36202) 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 -- 172.21.15.22:0/3976961413 learned_addr learned my addr 172.21.15.22:0/3976961413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff764f96640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7600ffbb0 0x7ff76019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff75ffff640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff760100570 0x7ff76019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 -- 172.21.15.22:0/3976961413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff760100570 msgr2=0x7ff76019dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff760100570 0x7ff76019dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 -- 172.21.15.22:0/3976961413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7600ffbb0 msgr2=0x7ff76019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7600ffbb0 0x7ff76019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 -- 172.21.15.22:0/3976961413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff760198170 con 0x7ff7601074f0 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff765797640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601979f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff75800eb10 tx=0x7ff75800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.959+0000 7ff75dffb640 1 -- 172.21.15.22:0/3976961413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758004b20 con 0x7ff7601074f0 2024-03-09T00:02:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.960+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff760198460 con 0x7ff7601074f0 2024-03-09T00:02:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.960+0000 7ff75dffb640 1 -- 172.21.15.22:0/3976961413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff758004cc0 con 0x7ff7601074f0 2024-03-09T00:02:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.960+0000 7ff75dffb640 1 -- 172.21.15.22:0/3976961413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758010760 con 0x7ff7601074f0 2024-03-09T00:02:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.960+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7601a4700 con 0x7ff7601074f0 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.962+0000 7ff75dffb640 1 -- 172.21.15.22:0/3976961413 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff758005020 con 0x7ff7601074f0 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.962+0000 7ff769dd7740 1 Processor -- start 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.962+0000 7ff769dd7740 1 -- start start 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.962+0000 7ff75dffb640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff7500a2970 0x7ff7500a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbb490 0x55a721bbb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bcb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75dffb640 1 -- 172.21.15.22:0/3976961413 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ff7580cf830 con 0x7ff7601074f0 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff769dd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a721b1ee00 con 0x55a721bbbdb0 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff769dd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a721ac7b10 con 0x55a721bc9070 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff769dd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a721b1aea0 con 0x55a721bbb490 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36210/0 (socket says 172.21.15.22:36210) 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 -- 172.21.15.22:0/1135714433 learned_addr learned my addr 172.21.15.22:0/1135714433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff764f96640 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbb490 0x55a721bbb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff7500a2970 0x7ff7500a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff765797640 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bcb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 -- 172.21.15.22:0/1135714433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbb490 msgr2=0x55a721bbb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbb490 0x55a721bbb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 -- 172.21.15.22:0/1135714433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 msgr2=0x55a721bcb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bcb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 -- 172.21.15.22:0/1135714433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7218fd7d0 con 0x55a721bbbdb0 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff764f96640 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbb490 0x55a721bbb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc8b30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0119f0 tx=0x7ff74c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b981d5e00ffc3e39 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.963+0000 7ff75ffff640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff7500a2970 0x7ff7500a4e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00f790 tx=0x7ff74c016000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff74affd640 1 -- 172.21.15.22:0/1135714433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c023070 con 0x55a721bbbdb0 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff74affd640 1 -- 172.21.15.22:0/1135714433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff74c00a970 con 0x55a721bbbdb0 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff74affd640 1 -- 172.21.15.22:0/1135714433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c003850 con 0x55a721bbbdb0 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 -- 172.21.15.22:0/1135714433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 msgr2=0x55a721bc8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc8b30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0119f0 tx=0x7ff74c011df0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 -- 172.21.15.22:0/1135714433 shutdown_connections 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bcb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc8b30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 --2- 172.21.15.22:0/1135714433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbb490 0x55a721bbb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 -- 172.21.15.22:0/1135714433 >> 172.21.15.22:0/1135714433 conn(0x55a721bb5e90 msgr2=0x55a721bb6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 -- 172.21.15.22:0/1135714433 shutdown_connections 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.964+0000 7ff769dd7740 1 -- 172.21.15.22:0/1135714433 wait complete. 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 -- start start 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 Processor -- start 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbb490 0x55a721bc8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bc6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a721bcbdd0 con 0x55a721bbbdb0 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a721adb0a0 con 0x55a721bbb490 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a721bcbf50 con 0x55a721bc9070 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff769dd7740 10 client.0 _ll_register_callbacks cb 0x55a721adaa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff75ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff764f96640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbb490 0x55a721bc8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff75ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36226/0 (socket says 172.21.15.22:36226) 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff75ffff640 1 -- 172.21.15.22:0/3296135174 learned_addr learned my addr 172.21.15.22:0/3296135174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.965+0000 7ff765797640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bc6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff764f96640 1 -- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 msgr2=0x55a721bc6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff764f96640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bc6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff764f96640 1 -- 172.21.15.22:0/3296135174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 msgr2=0x55a721bc5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff764f96640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff764f96640 1 -- 172.21.15.22:0/3296135174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7218333f0 con 0x55a721bbb490 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff75ffff640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff765797640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bc6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff764f96640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbb490 0x55a721bc8870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff754009230 tx=0x7ff75402c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a93566263cb7b3bd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff73ffff640 1 -- 172.21.15.22:0/3296135174 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754043070 con 0x55a721bbb490 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff74c00a970 con 0x55a721bbb490 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff73ffff640 1 -- 172.21.15.22:0/3296135174 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ff7540046c0 con 0x55a721bbb490 2024-03-09T00:02:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff73ffff640 1 -- 172.21.15.22:0/3296135174 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff754050050 con 0x55a721bbb490 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff769dd7740 10 client.27473 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a7217f30f0 con 0x55a721bbb490 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.966+0000 7ff73ffff640 1 -- 172.21.15.22:0/3296135174 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff74c00a970 con 0x55a721bbb490 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff73ffff640 1 -- 172.21.15.22:0/3296135174 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff754003680 con 0x55a721bbb490 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff769dd7740 10 client.27473 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff769dd7740 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff769dd7740 20 client.27473 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721bd8c40 0x55a721bdb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff769dd7740 4 client.27473 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a721b2c340 con 0x55a721bd8c40 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff75ffff640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721bd8c40 0x55a721bdb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff75ffff640 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721bd8c40 0x55a721bdb030 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d989980c4a97c8be server_cookie=2aef0d02bcc2b98f in_seq=0 out_seq=0 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.967+0000 7ff73ffff640 10 client.27473 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff73ffff640 1 -- 172.21.15.22:0/3296135174 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a721b2c340 con 0x55a721bd8c40 2024-03-09T00:02:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff73ffff640 10 client.27473 handle_command_reply: tid=0 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 client.27473 shutdown 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbb490 msgr2=0x55a721bc8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbb490 0x55a721bc8870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff754009230 tx=0x7ff75402c8a0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 shutdown_connections 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721bd8c40 0x55a721bdb030 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bc6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbbdb0 0x55a721bc5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3296135174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbb490 0x55a721bc8870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff73ffff640 0 client.27473 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 >> 172.21.15.22:0/3296135174 conn(0x55a721bb5e90 msgr2=0x55a721bb7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:30.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 shutdown_connections 2024-03-09T00:02:30.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 1 -- 172.21.15.22:0/3296135174 wait complete. 2024-03-09T00:02:30.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.969+0000 7ff769dd7740 20 client.27473 trim_cache size 0 max 16384 2024-03-09T00:02:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 Processor -- start 2024-03-09T00:02:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 -- start start 2024-03-09T00:02:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bc8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbbdb0 0x55a721bc8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bd01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601ae980 con 0x55a721bbb490 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601ac520 con 0x55a721bbbdb0 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff769dd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601aa5e0 con 0x55a721bc9070 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff764f96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bc8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff764f96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bc8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36230/0 (socket says 172.21.15.22:36230) 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff764f96640 1 -- 172.21.15.22:0/3944913221 learned_addr learned my addr 172.21.15.22:0/3944913221 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff765797640 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bd01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.985+0000 7ff75ffff640 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbbdb0 0x55a721bc8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff764f96640 1 -- 172.21.15.22:0/3944913221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 msgr2=0x55a721bd01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff764f96640 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bd01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff764f96640 1 -- 172.21.15.22:0/3944913221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbbdb0 msgr2=0x55a721bc8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff764f96640 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbbdb0 0x55a721bc8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff764f96640 1 -- 172.21.15.22:0/3944913221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a721bbbbe0 con 0x55a721bbb490 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff764f96640 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bc8330 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff754009a50 tx=0x7ff754009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc052db1a0166d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff74a7fc640 1 -- 172.21.15.22:0/3944913221 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754043070 con 0x55a721bbb490 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff74a7fc640 1 -- 172.21.15.22:0/3944913221 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff754009520 con 0x55a721bbb490 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.986+0000 7ff74a7fc640 1 -- 172.21.15.22:0/3944913221 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754004920 con 0x55a721bbb490 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 -- 172.21.15.22:0/3944913221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 msgr2=0x55a721bc8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bc8330 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff754009a50 tx=0x7ff754009260 comp rx=0 tx=0).stop 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 -- 172.21.15.22:0/3944913221 shutdown_connections 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bc9070 0x55a721bd01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bbbdb0 0x55a721bc8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 --2- 172.21.15.22:0/3944913221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bc8330 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 -- 172.21.15.22:0/3944913221 >> 172.21.15.22:0/3944913221 conn(0x55a721bb5e90 msgr2=0x55a721bc5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 -- 172.21.15.22:0/3944913221 shutdown_connections 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 -- 172.21.15.22:0/3944913221 wait complete. 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 -- start start 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.987+0000 7ff769dd7740 1 Processor -- start 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bcfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbbdb0 0x55a721bd0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601aabe0 con 0x55a721bbb490 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601abec0 con 0x55a721bc9070 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601ab200 con 0x55a721bbbdb0 2024-03-09T00:02:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bcfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 10 client.0 _ll_register_callbacks cb 0x7ff754009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bcfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36238/0 (socket says 172.21.15.22:36238) 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 -- 172.21.15.22:0/682658119 learned_addr learned my addr 172.21.15.22:0/682658119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff75ffff640 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbbdb0 0x55a721bd0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff765797640 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 -- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbbdb0 msgr2=0x55a721bd0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbbdb0 0x55a721bd0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 -- 172.21.15.22:0/682658119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 msgr2=0x55a721bce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 -- 172.21.15.22:0/682658119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a721bcea10 con 0x55a721bbb490 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff764f96640 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bcfdc0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff754005c90 tx=0x7ff754003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571bb1fcda174a15 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff73ffff640 1 -- 172.21.15.22:0/682658119 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754043040 con 0x55a721bbb490 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff73ffff640 1 -- 172.21.15.22:0/682658119 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ff754005150 con 0x55a721bbb490 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.988+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff754009520 con 0x55a721bbb490 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff73ffff640 1 -- 172.21.15.22:0/682658119 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff754067050 con 0x55a721bbb490 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 10 client.18096 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a721c2f9d0 con 0x55a721bbb490 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff73ffff640 1 -- 172.21.15.22:0/682658119 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff754009520 con 0x55a721bbb490 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff73ffff640 1 -- 172.21.15.22:0/682658119 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff75405e570 con 0x55a721bbb490 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 10 client.18096 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 20 client.18096 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721c2ff30 0x55a721c32320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 4 client.18096 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a721b26690 con 0x55a721c2ff30 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.989+0000 7ff75ffff640 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721c2ff30 0x55a721c32320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.990+0000 7ff75ffff640 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721c2ff30 0x55a721c32320 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9087434efe9013d server_cookie=ff51c77126b69357 in_seq=0 out_seq=0 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.990+0000 7ff73ffff640 10 client.18096 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.990+0000 7ff73ffff640 1 -- 172.21.15.22:0/682658119 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a721b26690 con 0x55a721c2ff30 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff73ffff640 10 client.18096 handle_command_reply: tid=0 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 client.18096 shutdown 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 msgr2=0x55a721bcfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bcfdc0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff754005c90 tx=0x7ff754003fd0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 shutdown_connections 2024-03-09T00:02:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a721c2ff30 0x55a721c32320 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a721bc9070 0x55a721bce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a721bbbdb0 0x55a721bd0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff73ffff640 0 client.18096 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 --2- 172.21.15.22:0/682658119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a721bbb490 0x55a721bcfdc0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 >> 172.21.15.22:0/682658119 conn(0x55a721bb5e90 msgr2=0x55a721bc0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 shutdown_connections 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 1 -- 172.21.15.22:0/682658119 wait complete. 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff769dd7740 20 client.18096 trim_cache size 0 max 16384 2024-03-09T00:02:30.994 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:02:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:02:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:02:30.995 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff7500a2970 msgr2=0x7ff7500a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff7671fa640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff7500a2970 0x7ff7500a4e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00f790 tx=0x7ff74c016000 comp rx=0 tx=0).stop 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 msgr2=0x7ff7601979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.991+0000 7ff7671fa640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601979f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff75800eb10 tx=0x7ff75800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 shutdown_connections 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff7500a2970 0x7ff7500a4e30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff7601074f0 0x7ff7601979f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff760100570 0x7ff76019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 --2- 172.21.15.22:0/3976961413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7600ffbb0 0x7ff76019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 >> 172.21.15.22:0/3976961413 conn(0x7ff760075530 msgr2=0x7ff7600fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 shutdown_connections 2024-03-09T00:02:30.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:30.992+0000 7ff7671fa640 1 -- 172.21.15.22:0/3976961413 wait complete. 2024-03-09T00:02:30.998 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:02:30.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:02:31.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 -- 172.21.15.22:0/1150246504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c100460 msgr2=0x7ffa7c100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/1150246504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c100460 0x7ffa7c100840 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009900 tx=0x7ffa7002bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 -- 172.21.15.22:0/1150246504 shutdown_connections 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/1150246504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa7c105530 0x7ffa7c10d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/1150246504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c100e10 0x7ffa7c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/1150246504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c100460 0x7ffa7c100840 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 -- 172.21.15.22:0/1150246504 >> 172.21.15.22:0/1150246504 conn(0x7ffa7c0772f0 msgr2=0x7ffa7c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 -- 172.21.15.22:0/1150246504 shutdown_connections 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.152+0000 7ffa82cc0640 1 -- 172.21.15.22:0/1150246504 wait complete. 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 Processor -- start 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 -- start start 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c100460 0x7ffa7c196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa7c105530 0x7ffa7c196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c197300 0x7ffa7c1a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c1105f0 con 0x7ffa7c197300 2024-03-09T00:02:31.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c110470 con 0x7ffa7c100460 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa82cc0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c110770 con 0x7ffa7c105530 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c197300 0x7ffa7c1a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c197300 0x7ffa7c1a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36244/0 (socket says 172.21.15.22:36244) 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 -- 172.21.15.22:0/935790455 learned_addr learned my addr 172.21.15.22:0/935790455 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa80a5c640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c100460 0x7ffa7c196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa7bfff640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa7c105530 0x7ffa7c196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 -- 172.21.15.22:0/935790455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa7c105530 msgr2=0x7ffa7c196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa7c105530 0x7ffa7c196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 -- 172.21.15.22:0/935790455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c100460 msgr2=0x7ffa7c196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c100460 0x7ffa7c196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 -- 172.21.15.22:0/935790455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c1a64c0 con 0x7ffa7c197300 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.153+0000 7ffa8125d640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c197300 0x7ffa7c1a5f00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400eb40 tx=0x7ffa7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.154+0000 7ffa79ffb640 1 -- 172.21.15.22:0/935790455 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74004b20 con 0x7ffa7c197300 2024-03-09T00:02:31.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.154+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c1a67b0 con 0x7ffa7c197300 2024-03-09T00:02:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.154+0000 7ffa79ffb640 1 -- 172.21.15.22:0/935790455 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa74004cc0 con 0x7ffa7c197300 2024-03-09T00:02:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.154+0000 7ffa79ffb640 1 -- 172.21.15.22:0/935790455 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74010690 con 0x7ffa7c197300 2024-03-09T00:02:31.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.154+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c1a6d00 con 0x7ffa7c197300 2024-03-09T00:02:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.155+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c0680e0 con 0x7ffa7c197300 2024-03-09T00:02:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.156+0000 7ffa79ffb640 1 -- 172.21.15.22:0/935790455 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ffa74005020 con 0x7ffa7c197300 2024-03-09T00:02:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.156+0000 7ffa79ffb640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa6c0a2970 0x7ffa6c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.156+0000 7ffa80a5c640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa6c0a2970 0x7ffa6c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.157+0000 7ffa79ffb640 1 -- 172.21.15.22:0/935790455 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ffa740cf700 con 0x7ffa7c197300 2024-03-09T00:02:31.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.157+0000 7ffa80a5c640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa6c0a2970 0x7ffa6c0a4e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa70006010 tx=0x7ffa70005dd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.161+0000 7ffa79ffb640 1 -- 172.21.15.22:0/935790455 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ffa7409c070 con 0x7ffa7c197300 2024-03-09T00:02:31.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.403+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa7c100e10 con 0x7ffa7c197300 2024-03-09T00:02:31.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.404+0000 7ffa79ffb640 1 -- 172.21.15.22:0/935790455 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ffa7409b970 con 0x7ffa7c197300 2024-03-09T00:02:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:02:31.399 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:02:31.399 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:02:31.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa6c0a2970 msgr2=0x7ffa6c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa6c0a2970 0x7ffa6c0a4e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa70006010 tx=0x7ffa70005dd0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c197300 msgr2=0x7ffa7c1a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c197300 0x7ffa7c1a5f00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400eb40 tx=0x7ffa7400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 shutdown_connections 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffa6c0a2970 0x7ffa6c0a4e30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa7c197300 0x7ffa7c1a5f00 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa7c105530 0x7ffa7c196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 --2- 172.21.15.22:0/935790455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c100460 0x7ffa7c196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 >> 172.21.15.22:0/935790455 conn(0x7ffa7c0772f0 msgr2=0x7ffa7c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 shutdown_connections 2024-03-09T00:02:31.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.406+0000 7ffa82cc0640 1 -- 172.21.15.22:0/935790455 wait complete. 2024-03-09T00:02:31.413 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:02:31.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:02:31.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 -- 172.21.15.22:0/440867180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254105620 msgr2=0x7fd254105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/440867180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254105620 0x7fd254105ab0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd244009900 tx=0x7fd24402bcd0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 -- 172.21.15.22:0/440867180 shutdown_connections 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/440867180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254105620 0x7fd254105ab0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/440867180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd254100c20 0x7fd2541050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/440867180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd254100300 0x7fd2541006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 -- 172.21.15.22:0/440867180 >> 172.21.15.22:0/440867180 conn(0x7fd2540773a0 msgr2=0x7fd2540777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 -- 172.21.15.22:0/440867180 shutdown_connections 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 -- 172.21.15.22:0/440867180 wait complete. 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 Processor -- start 2024-03-09T00:02:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.525+0000 7fd25a2b2640 1 -- start start 2024-03-09T00:02:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd25a2b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd254100300 0x7fd25419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd25a2b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254100c20 0x7fd25419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd25a2b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd254105620 0x7fd2541a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd25a2b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254117c50 con 0x7fd254100c20 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd25a2b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254117ad0 con 0x7fd254100300 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd25a2b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254117dd0 con 0x7fd254105620 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254100c20 0x7fd25419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254100c20 0x7fd25419e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36266/0 (socket says 172.21.15.22:36266) 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 -- 172.21.15.22:0/71001155 learned_addr learned my addr 172.21.15.22:0/71001155 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd253fff640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd254100300 0x7fd25419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd25884f640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd254105620 0x7fd2541a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 -- 172.21.15.22:0/71001155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd254105620 msgr2=0x7fd2541a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd254105620 0x7fd2541a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 -- 172.21.15.22:0/71001155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd254100300 msgr2=0x7fd25419e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd254100300 0x7fd25419e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 -- 172.21.15.22:0/71001155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2541a3000 con 0x7fd254100c20 2024-03-09T00:02:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.526+0000 7fd2537fe640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254100c20 0x7fd25419e550 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd24800e9f0 tx=0x7fd24800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.527+0000 7fd2517fa640 1 -- 172.21.15.22:0/71001155 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800cd80 con 0x7fd254100c20 2024-03-09T00:02:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.527+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2541a32f0 con 0x7fd254100c20 2024-03-09T00:02:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.527+0000 7fd2517fa640 1 -- 172.21.15.22:0/71001155 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd248010910 con 0x7fd254100c20 2024-03-09T00:02:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.527+0000 7fd2517fa640 1 -- 172.21.15.22:0/71001155 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2480188f0 con 0x7fd254100c20 2024-03-09T00:02:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.527+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2541aab80 con 0x7fd254100c20 2024-03-09T00:02:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.529+0000 7fd2517fa640 1 -- 172.21.15.22:0/71001155 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd248010ac0 con 0x7fd254100c20 2024-03-09T00:02:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.529+0000 7fd25cf73740 1 Processor -- start 2024-03-09T00:02:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.529+0000 7fd2517fa640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd23c0a2970 0x7fd23c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.529+0000 7fd25cf73740 1 -- start start 2024-03-09T00:02:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.529+0000 7fd25884f640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd23c0a2970 0x7fd23c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.529+0000 7fd2517fa640 1 -- 172.21.15.22:0/71001155 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd2480cf4f0 con 0x7fd254100c20 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.529+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577591a0 0x565257759580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565257759ac0 0x565257762160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257764a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd25cf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565257720360 con 0x5652577591a0 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd25cf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652576bcec0 con 0x565257759ac0 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd25cf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565257665d30 con 0x5652577626a0 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577591a0 0x565257759580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577591a0 0x565257759580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36272/0 (socket says 172.21.15.22:36272) 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 -- 172.21.15.22:0/3018671457 learned_addr learned my addr 172.21.15.22:0/3018671457 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd2537fe640 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565257759ac0 0x565257762160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd25884f640 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257764a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 -- 172.21.15.22:0/3018671457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 msgr2=0x565257764a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257764a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 -- 172.21.15.22:0/3018671457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565257759ac0 msgr2=0x565257762160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565257759ac0 0x565257762160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 -- 172.21.15.22:0/3018671457 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565257664bc0 con 0x5652577591a0 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd25884f640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd23c0a2970 0x7fd23c0a4e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd244005ab0 tx=0x7fd24402c770 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd2537fe640 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565257759ac0 0x565257762160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd253fff640 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577591a0 0x565257759580 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd24c007f90 tx=0x7fd24c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdfd0a69e789a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd23a7fc640 1 -- 172.21.15.22:0/3018671457 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c004d20 con 0x5652577591a0 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd23a7fc640 1 -- 172.21.15.22:0/3018671457 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd24c010be0 con 0x5652577591a0 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.530+0000 7fd23a7fc640 1 -- 172.21.15.22:0/3018671457 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c005780 con 0x5652577591a0 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 -- 172.21.15.22:0/3018671457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577591a0 msgr2=0x565257759580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577591a0 0x565257759580 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd24c007f90 tx=0x7fd24c00f850 comp rx=0 tx=0).stop 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 -- 172.21.15.22:0/3018671457 shutdown_connections 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257764a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565257759ac0 0x565257762160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 --2- 172.21.15.22:0/3018671457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577591a0 0x565257759580 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 -- 172.21.15.22:0/3018671457 >> 172.21.15.22:0/3018671457 conn(0x5652577538d0 msgr2=0x565257753cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 -- 172.21.15.22:0/3018671457 shutdown_connections 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.531+0000 7fd25cf73740 1 -- 172.21.15.22:0/3018671457 wait complete. 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 -- start start 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 Processor -- start 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x56525775f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565257759ac0 0x56525775faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577626a0 0x565257760030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525765b4a0 con 0x5652577626a0 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652576b8f60 con 0x5652577591a0 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565257765490 con 0x565257759ac0 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25cf73740 10 client.0 _ll_register_callbacks cb 0x5652576b6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577626a0 0x565257760030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577626a0 0x565257760030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36274/0 (socket says 172.21.15.22:36274) 2024-03-09T00:02:31.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 -- 172.21.15.22:0/4167016268 learned_addr learned my addr 172.21.15.22:0/4167016268 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd2537fe640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565257759ac0 0x56525775faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd253fff640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x56525775f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 -- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565257759ac0 msgr2=0x56525775faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565257759ac0 0x56525775faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 -- 172.21.15.22:0/4167016268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 msgr2=0x56525775f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x56525775f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd25884f640 1 -- 172.21.15.22:0/4167016268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652573d13f0 con 0x5652577626a0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd253fff640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x56525775f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.532+0000 7fd2537fe640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565257759ac0 0x56525775faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd25884f640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577626a0 0x565257760030 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd24402cd50 tx=0x7fd24402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b53270bf3228a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd2337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd2337fe640 1 -- 172.21.15.22:0/4167016268 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244050070 con 0x5652577626a0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd2337fe640 1 -- 172.21.15.22:0/4167016268 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd2440046c0 con 0x5652577626a0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd24c010be0 con 0x5652577626a0 2024-03-09T00:02:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd2337fe640 1 -- 172.21.15.22:0/4167016268 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd24405c050 con 0x5652577626a0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd25cf73740 10 client.18114 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56525749b7d0 con 0x5652577626a0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd2337fe640 1 -- 172.21.15.22:0/4167016268 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd24c010be0 con 0x5652577626a0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd2337fe640 1 -- 172.21.15.22:0/4167016268 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2440035f0 con 0x5652577626a0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd25cf73740 10 client.18114 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd25cf73740 10 client.18114 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.533+0000 7fd25cf73740 20 client.18114 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.534+0000 7fd25cf73740 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577727d0 0x565257774c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.534+0000 7fd25cf73740 4 client.18114 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.534+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652576ca400 con 0x5652577727d0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.534+0000 7fd253fff640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577727d0 0x565257774c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.534+0000 7fd253fff640 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577727d0 0x565257774c30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f795b3842c2681e6 server_cookie=e52bb0d69a229eb6 in_seq=0 out_seq=0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.534+0000 7fd2337fe640 10 client.18114 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd2337fe640 1 -- 172.21.15.22:0/4167016268 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652576ca400 con 0x5652577727d0 2024-03-09T00:02:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd2337fe640 10 client.18114 handle_command_reply: tid=0 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 client.18114 shutdown 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577626a0 msgr2=0x565257760030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577626a0 0x565257760030 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd24402cd50 tx=0x7fd24402bcd0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 shutdown_connections 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577727d0 0x565257774c30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652577626a0 0x565257760030 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565257759ac0 0x56525775faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 --2- 172.21.15.22:0/4167016268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x56525775f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 >> 172.21.15.22:0/4167016268 conn(0x5652577538d0 msgr2=0x565257755620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd2337fe640 0 client.18114 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 shutdown_connections 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 1 -- 172.21.15.22:0/4167016268 wait complete. 2024-03-09T00:02:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.535+0000 7fd25cf73740 20 client.18114 trim_cache size 0 max 16384 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.548+0000 7fd25cf73740 1 Processor -- start 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.548+0000 7fd25cf73740 1 -- start start 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257761ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x56525775e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x56525775c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd25cf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541b7090 con 0x565257759ac0 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd25cf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541b4c30 con 0x5652577591a0 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd25cf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541b2cf0 con 0x5652577626a0 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x56525775e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd253fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257761ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x56525775e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36278/0 (socket says 172.21.15.22:36278) 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd2537fe640 1 -- 172.21.15.22:0/1622693742 learned_addr learned my addr 172.21.15.22:0/1622693742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd25884f640 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x56525775c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd253fff640 1 -- 172.21.15.22:0/1622693742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 msgr2=0x56525775c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd253fff640 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x56525775c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd253fff640 1 -- 172.21.15.22:0/1622693742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 msgr2=0x56525775e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd253fff640 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x56525775e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd253fff640 1 -- 172.21.15.22:0/1622693742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565257753700 con 0x5652577591a0 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd2537fe640 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x56525775e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd253fff640 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257761ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00fd50 tx=0x7fd24c007380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b49c947c55474de server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.549+0000 7fd239ffb640 1 -- 172.21.15.22:0/1622693742 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c01d040 con 0x5652577591a0 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd239ffb640 1 -- 172.21.15.22:0/1622693742 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd24c010db0 con 0x5652577591a0 2024-03-09T00:02:31.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd239ffb640 1 -- 172.21.15.22:0/1622693742 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c004c40 con 0x5652577591a0 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 -- 172.21.15.22:0/1622693742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 msgr2=0x565257761ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257761ce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00fd50 tx=0x7fd24c007380 comp rx=0 tx=0).stop 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 -- 172.21.15.22:0/1622693742 shutdown_connections 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x56525775c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x56525775e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1622693742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257761ce0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 -- 172.21.15.22:0/1622693742 >> 172.21.15.22:0/1622693742 conn(0x5652577538d0 msgr2=0x56525775de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 -- 172.21.15.22:0/1622693742 shutdown_connections 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.550+0000 7fd25cf73740 1 -- 172.21.15.22:0/1622693742 wait complete. 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 -- start start 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 Processor -- start 2024-03-09T00:02:31.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257769890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x565257767c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257768170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541b32f0 con 0x565257759ac0 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541b45d0 con 0x5652577591a0 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541b3910 con 0x5652577626a0 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25cf73740 10 client.0 _ll_register_callbacks cb 0x7fd24c0079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x565257767c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x565257767c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36292/0 (socket says 172.21.15.22:36292) 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd253fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257769890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 -- 172.21.15.22:0/1986218228 learned_addr learned my addr 172.21.15.22:0/1986218228 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd25884f640 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257768170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 -- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 msgr2=0x565257768170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257768170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 -- 172.21.15.22:0/1986218228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 msgr2=0x565257769890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257769890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd2537fe640 1 -- 172.21.15.22:0/1986218228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652577687e0 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.551+0000 7fd253fff640 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257769890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd2537fe640 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x565257767c30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd248097d10 tx=0x7fd2480a3fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecf8c61dcd57e0e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd2337fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd2337fe640 1 -- 172.21.15.22:0/1986218228 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2480cf320 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd2337fe640 1 -- 172.21.15.22:0/1986218228 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd2480afbe0 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652576c4750 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd2337fe640 1 -- 172.21.15.22:0/1986218228 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2480ba0d0 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 10 client.18120 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5652577c9c30 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd2337fe640 1 -- 172.21.15.22:0/1986218228 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5652576c4750 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd2337fe640 1 -- 172.21.15.22:0/1986218228 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd24809cb30 con 0x565257759ac0 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 10 client.18120 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 10 client.18120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:02:31.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 20 client.18120 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577ca390 0x5652577cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 4 client.18120 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.552+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565257642640 con 0x5652577ca390 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd253fff640 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577ca390 0x5652577cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd253fff640 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577ca390 0x5652577cc7b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=963a7319d5cdf928 server_cookie=b74a590b07112761 in_seq=0 out_seq=0 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd2337fe640 10 client.18120 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd2337fe640 1 -- 172.21.15.22:0/1986218228 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565257642640 con 0x5652577ca390 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd2337fe640 10 client.18120 handle_command_reply: tid=0 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 client.18120 shutdown 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 msgr2=0x565257767c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x565257767c30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd248097d10 tx=0x7fd2480a3fa0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 shutdown_connections 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5652577ca390 0x5652577cc7b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652577626a0 0x565257768170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565257759ac0 0x565257767c30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 --2- 172.21.15.22:0/1986218228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652577591a0 0x565257769890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd2337fe640 0 client.18120 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:02:31.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 >> 172.21.15.22:0/1986218228 conn(0x5652577538d0 msgr2=0x56525775e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 shutdown_connections 2024-03-09T00:02:31.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.553+0000 7fd25cf73740 1 -- 172.21.15.22:0/1986218228 wait complete. 2024-03-09T00:02:31.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25cf73740 20 client.18120 trim_cache size 0 max 16384 2024-03-09T00:02:31.555 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd23c0a2970 msgr2=0x7fd23c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd23c0a2970 0x7fd23c0a4e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd244005ab0 tx=0x7fd24402c770 comp rx=0 tx=0).stop 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254100c20 msgr2=0x7fd25419e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254100c20 0x7fd25419e550 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd24800e9f0 tx=0x7fd24800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 shutdown_connections 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd23c0a2970 0x7fd23c0a4e30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd254105620 0x7fd2541a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd254100c20 0x7fd25419e550 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 --2- 172.21.15.22:0/71001155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd254100300 0x7fd25419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 >> 172.21.15.22:0/71001155 conn(0x7fd2540773a0 msgr2=0x7fd2541030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 shutdown_connections 2024-03-09T00:02:31.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.554+0000 7fd25a2b2640 1 -- 172.21.15.22:0/71001155 wait complete. 2024-03-09T00:02:31.561 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:02:31.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:02:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.675+0000 7f361f64d640 1 -- 172.21.15.22:0/1172374910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100dd0 msgr2=0x7f3618104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.675+0000 7f361f64d640 1 --2- 172.21.15.22:0/1172374910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100dd0 0x7f3618104e10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3608009900 tx=0x7f360802bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.675+0000 7f361f64d640 1 -- 172.21.15.22:0/1172374910 shutdown_connections 2024-03-09T00:02:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 --2- 172.21.15.22:0/1172374910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36181054f0 0x7f361810d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 --2- 172.21.15.22:0/1172374910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100dd0 0x7f3618104e10 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 --2- 172.21.15.22:0/1172374910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3618100420 0x7f3618100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 -- 172.21.15.22:0/1172374910 >> 172.21.15.22:0/1172374910 conn(0x7f3618077300 msgr2=0x7f3618077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 -- 172.21.15.22:0/1172374910 shutdown_connections 2024-03-09T00:02:31.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 -- 172.21.15.22:0/1172374910 wait complete. 2024-03-09T00:02:31.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 Processor -- start 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.676+0000 7f361f64d640 1 -- start start 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361f64d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100420 0x7f361819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361f64d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36181054f0 0x7f361819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361f64d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3618197a30 0x7f3618197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361f64d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618110580 con 0x7f3618100420 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361f64d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618110400 con 0x7f36181054f0 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361f64d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618110700 con 0x7f3618197a30 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100420 0x7f361819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100420 0x7f361819d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36300/0 (socket says 172.21.15.22:36300) 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 -- 172.21.15.22:0/199098453 learned_addr learned my addr 172.21.15.22:0/199098453 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361dbea640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3618197a30 0x7f3618197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361cbe8640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36181054f0 0x7f361819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 -- 172.21.15.22:0/199098453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3618197a30 msgr2=0x7f3618197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3618197a30 0x7f3618197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 -- 172.21.15.22:0/199098453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36181054f0 msgr2=0x7f361819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36181054f0 0x7f361819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 -- 172.21.15.22:0/199098453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618198550 con 0x7f3618100420 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.677+0000 7f361d3e9640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100420 0x7f361819d880 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f361400e9f0 tx=0x7f361400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.678+0000 7f360e7fc640 1 -- 172.21.15.22:0/199098453 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400cd80 con 0x7f3618100420 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.678+0000 7f360e7fc640 1 -- 172.21.15.22:0/199098453 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3614005500 con 0x7f3618100420 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.678+0000 7f360e7fc640 1 -- 172.21.15.22:0/199098453 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614010690 con 0x7f3618100420 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.678+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3618198840 con 0x7f3618100420 2024-03-09T00:02:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.678+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36181a4a40 con 0x7f3618100420 2024-03-09T00:02:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.680+0000 7f360e7fc640 1 -- 172.21.15.22:0/199098453 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f36140046c0 con 0x7f3618100420 2024-03-09T00:02:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 Processor -- start 2024-03-09T00:02:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 -- start start 2024-03-09T00:02:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f360e7fc640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f36040a2970 0x7f36040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead4e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead5b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead5bab0 0x55abead5dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abead15360 con 0x55abead4ded0 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeacb1ec0 con 0x55abead4e7f0 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f362222a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeac5ad30 con 0x55abead5bab0 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f360e7fc640 1 -- 172.21.15.22:0/199098453 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f36140cf650 con 0x7f3618100420 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361dbea640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f36040a2970 0x7f36040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead4e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead4e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36304/0 (socket says 172.21.15.22:36304) 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 -- 172.21.15.22:0/2486757632 learned_addr learned my addr 172.21.15.22:0/2486757632 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361cbe8640 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead5b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361dbea640 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead5bab0 0x55abead5dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 -- 172.21.15.22:0/2486757632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead5bab0 msgr2=0x55abead5dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead5bab0 0x55abead5dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 -- 172.21.15.22:0/2486757632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 msgr2=0x55abead5b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead5b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.681+0000 7f361d3e9640 1 -- 172.21.15.22:0/2486757632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abeac59bc0 con 0x55abead4ded0 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.682+0000 7f361d3e9640 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead4e2b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3614097d20 tx=0x7f361409b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb02ba4a7420c60d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.682+0000 7f361dbea640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f36040a2970 0x7f36040a4e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3610009fd0 tx=0x7f3610009d80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.682+0000 7f36037fe640 1 -- 172.21.15.22:0/2486757632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36140cf070 con 0x55abead4ded0 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.682+0000 7f36037fe640 1 -- 172.21.15.22:0/2486757632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f361409b8c0 con 0x55abead4ded0 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.682+0000 7f36037fe640 1 -- 172.21.15.22:0/2486757632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614097f30 con 0x55abead4ded0 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.682+0000 7f362222a740 1 -- 172.21.15.22:0/2486757632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 msgr2=0x55abead4e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.682+0000 7f362222a740 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead4e2b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3614097d20 tx=0x7f361409b400 comp rx=0 tx=0).stop 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- 172.21.15.22:0/2486757632 shutdown_connections 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead5bab0 0x55abead5dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead5b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 --2- 172.21.15.22:0/2486757632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead4e2b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- 172.21.15.22:0/2486757632 >> 172.21.15.22:0/2486757632 conn(0x55abead488d0 msgr2=0x55abead48cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- 172.21.15.22:0/2486757632 shutdown_connections 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- 172.21.15.22:0/2486757632 wait complete. 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- start start 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 Processor -- start 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead58aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4e7f0 0x55abead58fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead59520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeac504a0 con 0x55abead4e7f0 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abeacadf60 con 0x55abead5bab0 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abead5e870 con 0x55abead4ded0 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.683+0000 7f362222a740 10 client.0 _ll_register_callbacks cb 0x55abeacabd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4e7f0 0x55abead58fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead58aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4e7f0 0x55abead58fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36306/0 (socket says 172.21.15.22:36306) 2024-03-09T00:02:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 -- 172.21.15.22:0/509183895 learned_addr learned my addr 172.21.15.22:0/509183895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361dbea640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead59520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 -- 172.21.15.22:0/509183895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 msgr2=0x55abead58aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead58aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 -- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 msgr2=0x55abead59520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead59520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 -- 172.21.15.22:0/509183895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abea9c63f0 con 0x55abead4e7f0 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361d3e9640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead58aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361dbea640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead59520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f361cbe8640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4e7f0 0x55abead58fe0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f36180697d0 tx=0x7f360802ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ce81e8b82ee52c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f3600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f3600ff9640 1 -- 172.21.15.22:0/509183895 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608004240 con 0x55abead4e7f0 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f3600ff9640 1 -- 172.21.15.22:0/509183895 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f36080043e0 con 0x55abead4e7f0 2024-03-09T00:02:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f361409b8c0 con 0x55abead4e7f0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f3600ff9640 1 -- 172.21.15.22:0/509183895 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36080046c0 con 0x55abead4e7f0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f362222a740 10 client.18132 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.684+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55abeaa907d0 con 0x55abead4e7f0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f3600ff9640 1 -- 172.21.15.22:0/509183895 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f361409b8c0 con 0x55abead4e7f0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f3600ff9640 1 -- 172.21.15.22:0/509183895 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f36080035f0 con 0x55abead4e7f0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f362222a740 10 client.18132 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f362222a740 10 client.18132 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f362222a740 20 client.18132 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f362222a740 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abead6bcc0 0x55abead6e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f362222a740 4 client.18132 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abeacbf400 con 0x55abead6bcc0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f361d3e9640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abead6bcc0 0x55abead6e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f3600ff9640 10 client.18132 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.685+0000 7f361d3e9640 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abead6bcc0 0x55abead6e0b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=499202d87baaee25 server_cookie=c93bb5b4394100ef in_seq=0 out_seq=0 2024-03-09T00:02:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f3600ff9640 1 -- 172.21.15.22:0/509183895 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abeacbf400 con 0x55abead6bcc0 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f3600ff9640 10 client.18132 handle_command_reply: tid=0 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 client.18132 shutdown 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4e7f0 msgr2=0x55abead58fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4e7f0 0x55abead58fe0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f36180697d0 tx=0x7f360802ce30 comp rx=0 tx=0).stop 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 shutdown_connections 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abead6bcc0 0x55abead6e0b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead59520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4e7f0 0x55abead58fe0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 --2- 172.21.15.22:0/509183895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead58aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f3600ff9640 0 client.18132 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 >> 172.21.15.22:0/509183895 conn(0x55abead488d0 msgr2=0x55abead4a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 shutdown_connections 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 1 -- 172.21.15.22:0/509183895 wait complete. 2024-03-09T00:02:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.686+0000 7f362222a740 20 client.18132 trim_cache size 0 max 16384 2024-03-09T00:02:31.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 Processor -- start 2024-03-09T00:02:31.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 -- start start 2024-03-09T00:02:31.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead5b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead53d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead5bab0 0x55abead542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181aea00 con 0x55abead5bab0 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181ac5a0 con 0x55abead4e7f0 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f362222a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181aa660 con 0x55abead4ded0 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f361cbe8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead53d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead5b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f361dbea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead5bab0 0x55abead542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f361cbe8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead53d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55114/0 (socket says 172.21.15.22:55114) 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f361cbe8640 1 -- 172.21.15.22:0/2705765752 learned_addr learned my addr 172.21.15.22:0/2705765752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.699+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead5b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:40386/0 (socket says 172.21.15.22:40386) 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f361cbe8640 1 -- 172.21.15.22:0/2705765752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 msgr2=0x55abead5b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f361cbe8640 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead5b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f361cbe8640 1 -- 172.21.15.22:0/2705765752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead5bab0 msgr2=0x55abead542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f361cbe8640 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead5bab0 0x55abead542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f361cbe8640 1 -- 172.21.15.22:0/2705765752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abead6e420 con 0x55abead4e7f0 2024-03-09T00:02:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f361dbea640 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead5bab0 0x55abead542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f361cbe8640 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead53d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f360802c170 tx=0x7f360802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a3bd38e928f6c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f3602ffd640 1 -- 172.21.15.22:0/2705765752 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608009520 con 0x55abead4e7f0 2024-03-09T00:02:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f3602ffd640 1 -- 172.21.15.22:0/2705765752 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f360805b070 con 0x55abead4e7f0 2024-03-09T00:02:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.700+0000 7f3602ffd640 1 -- 172.21.15.22:0/2705765752 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608034840 con 0x55abead4e7f0 2024-03-09T00:02:31.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 -- 172.21.15.22:0/2705765752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 msgr2=0x55abead53d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead53d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f360802c170 tx=0x7f360802bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 -- 172.21.15.22:0/2705765752 shutdown_connections 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead5bab0 0x55abead542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead4e7f0 0x55abead53d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 --2- 172.21.15.22:0/2705765752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4ded0 0x55abead5b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 -- 172.21.15.22:0/2705765752 >> 172.21.15.22:0/2705765752 conn(0x55abead488d0 msgr2=0x55abead48cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 -- 172.21.15.22:0/2705765752 shutdown_connections 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 -- 172.21.15.22:0/2705765752 wait complete. 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.701+0000 7f362222a740 1 -- start start 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 1 Processor -- start 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4e7f0 0x55abead60fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead61500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181aac60 con 0x55abead4ded0 2024-03-09T00:02:31.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181abf40 con 0x55abead5bab0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181ab280 con 0x55abead4e7f0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f362222a740 10 client.0 _ll_register_callbacks cb 0x7f3614009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f361d3e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead62c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36334/0 (socket says 172.21.15.22:36334) 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f361d3e9640 1 -- 172.21.15.22:0/527791065 learned_addr learned my addr 172.21.15.22:0/527791065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f361dbea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead61500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.702+0000 7f361cbe8640 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4e7f0 0x55abead60fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f361dbea640 1 -- 172.21.15.22:0/527791065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4e7f0 msgr2=0x55abead60fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f361dbea640 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4e7f0 0x55abead60fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f361dbea640 1 -- 172.21.15.22:0/527791065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 msgr2=0x55abead62c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f361dbea640 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead62c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f361dbea640 1 -- 172.21.15.22:0/527791065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abead61a70 con 0x55abead5bab0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f361d3e9640 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead62c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f361dbea640 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead61500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3610017010 tx=0x7f3610004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54715e5eec1141ad server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f3600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f3600ff9640 1 -- 172.21.15.22:0/527791065 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610004b20 con 0x55abead5bab0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f3600ff9640 1 -- 172.21.15.22:0/527791065 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f361001c040 con 0x55abead5bab0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f360805b070 con 0x55abead5bab0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f3600ff9640 1 -- 172.21.15.22:0/527791065 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3610004cc0 con 0x55abead5bab0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f362222a740 10 client.27518 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.703+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55abeadc2f80 con 0x55abead5bab0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f3600ff9640 1 -- 172.21.15.22:0/527791065 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f360805b070 con 0x55abead5bab0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f3600ff9640 1 -- 172.21.15.22:0/527791065 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3610020050 con 0x55abead5bab0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f362222a740 10 client.27518 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f362222a740 10 client.27518 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f362222a740 20 client.27518 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f362222a740 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abeadc36f0 0x55abeadc5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f362222a740 4 client.27518 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55abeacb9750 con 0x55abeadc36f0 2024-03-09T00:02:31.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f361d3e9640 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abeadc36f0 0x55abeadc5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.704+0000 7f3600ff9640 10 client.27518 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f361d3e9640 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abeadc36f0 0x55abeadc5b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4de14f9e4ee075cd server_cookie=1101e13760cad3a4 in_seq=0 out_seq=0 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f3600ff9640 1 -- 172.21.15.22:0/527791065 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55abeacb9750 con 0x55abeadc36f0 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f3600ff9640 10 client.27518 handle_command_reply: tid=0 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 client.27518 shutdown 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 msgr2=0x55abead61500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead61500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3610017010 tx=0x7f3610004ab0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 shutdown_connections 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55abeadc36f0 0x55abeadc5b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abead5bab0 0x55abead61500 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abead4e7f0 0x55abead60fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f3600ff9640 0 client.27518 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 --2- 172.21.15.22:0/527791065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abead4ded0 0x55abead62c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 >> 172.21.15.22:0/527791065 conn(0x55abead488d0 msgr2=0x55abead53d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.705+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 shutdown_connections 2024-03-09T00:02:31.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f362222a740 1 -- 172.21.15.22:0/527791065 wait complete. 2024-03-09T00:02:31.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f362222a740 20 client.27518 trim_cache size 0 max 16384 2024-03-09T00:02:31.706 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f36040a2970 msgr2=0x7f36040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f36040a2970 0x7f36040a4e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3610009fd0 tx=0x7f3610009d80 comp rx=0 tx=0).stop 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100420 msgr2=0x7f361819d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100420 0x7f361819d880 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f361400e9f0 tx=0x7f361400edf0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 shutdown_connections 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f36040a2970 0x7f36040a4e30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3618197a30 0x7f3618197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36181054f0 0x7f361819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 --2- 172.21.15.22:0/199098453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3618100420 0x7f361819d880 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 >> 172.21.15.22:0/199098453 conn(0x7f3618077300 msgr2=0x7f3618069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 shutdown_connections 2024-03-09T00:02:31.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.706+0000 7f361f64d640 1 -- 172.21.15.22:0/199098453 wait complete. 2024-03-09T00:02:31.714 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:02:31.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:02:31.721 INFO:tasks.workunit.client.0.smithi022.stdout: 1 837714 10.62 MB/sec execute 279 sec latency 20.180 ms 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 -- 172.21.15.22:0/1798315407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350100440 msgr2=0x7f33501008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 --2- 172.21.15.22:0/1798315407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350100440 0x7f33501008a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f3344009900 tx=0x7f334402bc80 comp rx=0 tx=0).stop 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 -- 172.21.15.22:0/1798315407 shutdown_connections 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 --2- 172.21.15.22:0/1798315407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350100de0 0x7f335010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 --2- 172.21.15.22:0/1798315407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350100440 0x7f33501008a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 --2- 172.21.15.22:0/1798315407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3350105180 0x7f3350105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 -- 172.21.15.22:0/1798315407 >> 172.21.15.22:0/1798315407 conn(0x7f3350077050 msgr2=0x7f3350077460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 -- 172.21.15.22:0/1798315407 shutdown_connections 2024-03-09T00:02:31.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.828+0000 7f3354c85640 1 -- 172.21.15.22:0/1798315407 wait complete. 2024-03-09T00:02:31.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 Processor -- start 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 -- start start 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3350100440 0x7f335019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350100de0 0x7f335019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350105180 0x7f3350197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3350110640 con 0x7f3350105180 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501104c0 con 0x7f3350100de0 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.829+0000 7f3354c85640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501107c0 con 0x7f3350100440 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350105180 0x7f3350197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3350100440 0x7f335019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350105180 0x7f3350197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36366/0 (socket says 172.21.15.22:36366) 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3350100440 0x7f335019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:40406/0 (socket says 172.21.15.22:40406) 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 -- 172.21.15.22:0/483806720 learned_addr learned my addr 172.21.15.22:0/483806720 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334dd9b640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350100de0 0x7f335019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 -- 172.21.15.22:0/483806720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3350100440 msgr2=0x7f335019d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3350100440 0x7f335019d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 -- 172.21.15.22:0/483806720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350100de0 msgr2=0x7f335019dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350100de0 0x7f335019dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 -- 172.21.15.22:0/483806720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33501981d0 con 0x7f3350105180 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f334ed9d640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350105180 0x7f3350197a50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f334800eb10 tx=0x7f334800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f333f7fe640 1 -- 172.21.15.22:0/483806720 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348004b20 con 0x7f3350105180 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f333f7fe640 1 -- 172.21.15.22:0/483806720 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3348004cc0 con 0x7f3350105180 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f333f7fe640 1 -- 172.21.15.22:0/483806720 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348010760 con 0x7f3350105180 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33501984c0 con 0x7f3350105180 2024-03-09T00:02:31.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.830+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33501a4760 con 0x7f3350105180 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.832+0000 7f333f7fe640 1 -- 172.21.15.22:0/483806720 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3348005020 con 0x7f3350105180 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 Processor -- start 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 -- start start 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd3698ed0 0x55dfd36992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd36997f0 0x55dfd36a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd3660360 con 0x55dfd3698ed0 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd35fcec0 con 0x55dfd36997f0 2024-03-09T00:02:31.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f335786f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd35a5d30 con 0x55dfd36a6ab0 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f333f7fe640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33380a2970 0x7f33380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd3698ed0 0x55dfd36992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd36997f0 0x55dfd36a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd3698ed0 0x55dfd36992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36376/0 (socket says 172.21.15.22:36376) 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd36997f0 0x55dfd36a6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55152/0 (socket says 172.21.15.22:55152) 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334dd9b640 1 -- 172.21.15.22:0/3234577843 learned_addr learned my addr 172.21.15.22:0/3234577843 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334ed9d640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33380a2970 0x7f33380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334dd9b640 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f333f7fe640 1 -- 172.21.15.22:0/483806720 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f33480cf7d0 con 0x7f3350105180 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334e59c640 1 -- 172.21.15.22:0/3234577843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 msgr2=0x55dfd36a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334e59c640 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334e59c640 1 -- 172.21.15.22:0/3234577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd3698ed0 msgr2=0x55dfd36992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334e59c640 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd3698ed0 0x55dfd36992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.833+0000 7f334e59c640 1 -- 172.21.15.22:0/3234577843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfd35a4bc0 con 0x55dfd36997f0 2024-03-09T00:02:31.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.834+0000 7f334ed9d640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33380a2970 0x7f33380a4e30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f334809b740 tx=0x7f334806d670 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.834+0000 7f334e59c640 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd36997f0 0x55dfd36a6570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3340009bd0 tx=0x7f3340011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13a7f9c2a387a13 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.834+0000 7f333cff9640 1 -- 172.21.15.22:0/3234577843 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340004050 con 0x55dfd36997f0 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.834+0000 7f333cff9640 1 -- 172.21.15.22:0/3234577843 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3340005500 con 0x55dfd36997f0 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.834+0000 7f333cff9640 1 -- 172.21.15.22:0/3234577843 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340004d40 con 0x55dfd36997f0 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- 172.21.15.22:0/3234577843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd36997f0 msgr2=0x55dfd36a6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd36997f0 0x55dfd36a6570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3340009bd0 tx=0x7f3340011810 comp rx=0 tx=0).stop 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- 172.21.15.22:0/3234577843 shutdown_connections 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd36997f0 0x55dfd36a6570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 --2- 172.21.15.22:0/3234577843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd3698ed0 0x55dfd36992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- 172.21.15.22:0/3234577843 >> 172.21.15.22:0/3234577843 conn(0x55dfd36938d0 msgr2=0x55dfd3693cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- 172.21.15.22:0/3234577843 shutdown_connections 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- 172.21.15.22:0/3234577843 wait complete. 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- start start 2024-03-09T00:02:31.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 Processor -- start 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd359b4a0 con 0x55dfd36997f0 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd35f8f60 con 0x55dfd3698ed0 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd36a9870 con 0x55dfd36a6ab0 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f335786f740 10 client.0 _ll_register_callbacks cb 0x55dfd35f6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.835+0000 7f334e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36380/0 (socket says 172.21.15.22:36380) 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 -- 172.21.15.22:0/1532701949 learned_addr learned my addr 172.21.15.22:0/1532701949 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334dd9b640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334dd9b640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 -- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 msgr2=0x55dfd36a44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 -- 172.21.15.22:0/1532701949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 msgr2=0x55dfd36a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 -- 172.21.15.22:0/1532701949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfd33113f0 con 0x55dfd36997f0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334dd9b640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f334e59c640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36a3fa0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3340005890 tx=0x7f33400096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf03e905978c3cd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f3335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f3335ffb640 1 -- 172.21.15.22:0/1532701949 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340029070 con 0x55dfd36997f0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3340005500 con 0x55dfd36997f0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f3335ffb640 1 -- 172.21.15.22:0/1532701949 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f3340003800 con 0x55dfd36997f0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f3335ffb640 1 -- 172.21.15.22:0/1532701949 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3340035050 con 0x55dfd36997f0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f335786f740 10 client.18150 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.836+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55dfd33db7d0 con 0x55dfd36997f0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f3335ffb640 1 -- 172.21.15.22:0/1532701949 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3340005500 con 0x55dfd36997f0 2024-03-09T00:02:31.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f3335ffb640 1 -- 172.21.15.22:0/1532701949 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3340005040 con 0x55dfd36997f0 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f335786f740 10 client.18150 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f335786f740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f335786f740 20 client.18150 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f335786f740 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd36b6b90 0x55dfd36b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f335786f740 4 client.18150 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfd360a400 con 0x55dfd36b6b90 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f334dd9b640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd36b6b90 0x55dfd36b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f3335ffb640 10 client.18150 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.837+0000 7f334dd9b640 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd36b6b90 0x55dfd36b8f80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a589446817b1b50d server_cookie=d552bd2500b80700 in_seq=0 out_seq=0 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f3335ffb640 1 -- 172.21.15.22:0/1532701949 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfd360a400 con 0x55dfd36b6b90 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f3335ffb640 10 client.18150 handle_command_reply: tid=0 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 client.18150 shutdown 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 msgr2=0x55dfd36a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36a3fa0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3340005890 tx=0x7f33400096a0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 shutdown_connections 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd36b6b90 0x55dfd36b8f80 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36a3fa0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f3335ffb640 0 client.18150 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 --2- 172.21.15.22:0/1532701949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 >> 172.21.15.22:0/1532701949 conn(0x55dfd36938d0 msgr2=0x55dfd3695620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 shutdown_connections 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 1 -- 172.21.15.22:0/1532701949 wait complete. 2024-03-09T00:02:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.838+0000 7f335786f740 20 client.18150 trim_cache size 0 max 16384 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 Processor -- start 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 -- start start 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36997f0 0x55dfd36bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501ae910 con 0x55dfd36a6ab0 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501ac4b0 con 0x55dfd3698ed0 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.851+0000 7f335786f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501aa570 con 0x55dfd36997f0 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36997f0 0x55dfd36bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36386/0 (socket says 172.21.15.22:36386) 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 -- 172.21.15.22:0/2899789864 learned_addr learned my addr 172.21.15.22:0/2899789864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 -- 172.21.15.22:0/2899789864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36997f0 msgr2=0x55dfd36bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36997f0 0x55dfd36bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 -- 172.21.15.22:0/2899789864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 msgr2=0x55dfd36a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 -- 172.21.15.22:0/2899789864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfd369f680 con 0x55dfd36a6ab0 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f334dd9b640 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a2330 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f334402c1c0 tx=0x7f3344009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d344ff8260b50c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f3337fff640 1 -- 172.21.15.22:0/2899789864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344050540 con 0x55dfd36a6ab0 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f3337fff640 1 -- 172.21.15.22:0/2899789864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f334402cd40 con 0x55dfd36a6ab0 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.852+0000 7f3337fff640 1 -- 172.21.15.22:0/2899789864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344005310 con 0x55dfd36a6ab0 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 -- 172.21.15.22:0/2899789864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36a6ab0 msgr2=0x55dfd36a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a2330 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f334402c1c0 tx=0x7f3344009260 comp rx=0 tx=0).stop 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 -- 172.21.15.22:0/2899789864 shutdown_connections 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36a6ab0 0x55dfd36a2330 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36997f0 0x55dfd36bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 --2- 172.21.15.22:0/2899789864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 -- 172.21.15.22:0/2899789864 >> 172.21.15.22:0/2899789864 conn(0x55dfd36938d0 msgr2=0x55dfd3693cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 -- 172.21.15.22:0/2899789864 shutdown_connections 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 -- 172.21.15.22:0/2899789864 wait complete. 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 -- start start 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.853+0000 7f335786f740 1 Processor -- start 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501aab70 con 0x55dfd36997f0 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501abe50 con 0x55dfd3698ed0 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33501ab190 con 0x55dfd36a6ab0 2024-03-09T00:02:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 10 client.0 _ll_register_callbacks cb 0x7f3340009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36398/0 (socket says 172.21.15.22:36398) 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 -- 172.21.15.22:0/373561309 learned_addr learned my addr 172.21.15.22:0/373561309 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334dd9b640 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334dd9b640 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 -- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 msgr2=0x55dfd36abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 -- 172.21.15.22:0/373561309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 msgr2=0x55dfd36ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 -- 172.21.15.22:0/373561309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfd36ac3e0 con 0x55dfd36997f0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f334e59c640 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36adcb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f33400058c0 tx=0x7f3340004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96715199f985ca60 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f3335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f3335ffb640 1 -- 172.21.15.22:0/373561309 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340029040 con 0x55dfd36997f0 2024-03-09T00:02:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f3335ffb640 1 -- 172.21.15.22:0/373561309 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f3340023520 con 0x55dfd36997f0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.854+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfd3604750 con 0x55dfd36997f0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f3335ffb640 1 -- 172.21.15.22:0/373561309 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3340040050 con 0x55dfd36997f0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 10 client.18156 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55dfd370d9c0 con 0x55dfd36997f0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f3335ffb640 1 -- 172.21.15.22:0/373561309 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dfd3604750 con 0x55dfd36997f0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f3335ffb640 1 -- 172.21.15.22:0/373561309 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3340023970 con 0x55dfd36997f0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 10 client.18156 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 20 client.18156 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd370e030 0x55dfd3710450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 4 client.18156 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dfd3582640 con 0x55dfd370e030 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.855+0000 7f334dd9b640 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd370e030 0x55dfd3710450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.856+0000 7f334dd9b640 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd370e030 0x55dfd3710450 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62f3d9fd6347ee9 server_cookie=c470bacf78318878 in_seq=0 out_seq=0 2024-03-09T00:02:31.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.856+0000 7f3335ffb640 10 client.18156 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f3335ffb640 1 -- 172.21.15.22:0/373561309 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dfd3582640 con 0x55dfd370e030 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f3335ffb640 10 client.18156 handle_command_reply: tid=0 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 client.18156 shutdown 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 msgr2=0x55dfd36adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36adcb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f33400058c0 tx=0x7f3340004590 comp rx=0 tx=0).stop 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 shutdown_connections 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dfd370e030 0x55dfd3710450 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dfd36a6ab0 0x55dfd36abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dfd36997f0 0x55dfd36adcb0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f3335ffb640 0 client.18156 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 --2- 172.21.15.22:0/373561309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfd3698ed0 0x55dfd36ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 >> 172.21.15.22:0/373561309 conn(0x55dfd36938d0 msgr2=0x55dfd36bd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 shutdown_connections 2024-03-09T00:02:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 1 -- 172.21.15.22:0/373561309 wait complete. 2024-03-09T00:02:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.857+0000 7f335786f740 20 client.18156 trim_cache size 0 max 16384 2024-03-09T00:02:31.858 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33380a2970 msgr2=0x7f33380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33380a2970 0x7f33380a4e30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f334809b740 tx=0x7f334806d670 comp rx=0 tx=0).stop 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350105180 msgr2=0x7f3350197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350105180 0x7f3350197a50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f334800eb10 tx=0x7f334800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 shutdown_connections 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33380a2970 0x7f33380a4e30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3350105180 0x7f3350197a50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350100de0 0x7f335019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 --2- 172.21.15.22:0/483806720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3350100440 0x7f335019d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 >> 172.21.15.22:0/483806720 conn(0x7f3350077050 msgr2=0x7f33500698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 shutdown_connections 2024-03-09T00:02:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:31.858+0000 7f3354c85640 1 -- 172.21.15.22:0/483806720 wait complete. 2024-03-09T00:02:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:32 smithi062 ceph-mon[36129]: pgmap v252: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-03-09T00:02:32.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:32 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/935790455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:02:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:32 smithi192 ceph-mon[35995]: pgmap v252: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-03-09T00:02:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:32 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/935790455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:02:32.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:32 smithi022 ceph-mon[31894]: pgmap v252: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-03-09T00:02:32.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:32 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/935790455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:02:32.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 839071 10.61 MB/sec execute 280 sec latency 66.599 ms 2024-03-09T00:02:32.864 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:02:32.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:02:32.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:02:32.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 -- 172.21.15.22:0/1433203981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01065c0 msgr2=0x7f87f010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 --2- 172.21.15.22:0/1433203981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01065c0 0x7f87f010e190 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009ca0 tx=0x7f87e802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 -- 172.21.15.22:0/1433203981 shutdown_connections 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 --2- 172.21.15.22:0/1433203981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01065c0 0x7f87f010e190 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 --2- 172.21.15.22:0/1433203981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f0101ea0 0x7f87f0105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 --2- 172.21.15.22:0/1433203981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87f01014f0 0x7f87f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 -- 172.21.15.22:0/1433203981 >> 172.21.15.22:0/1433203981 conn(0x7f87f00786f0 msgr2=0x7f87f0078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 -- 172.21.15.22:0/1433203981 shutdown_connections 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:32.999+0000 7f87f571b640 1 -- 172.21.15.22:0/1433203981 wait complete. 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 Processor -- start 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 -- start start 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01014f0 0x7f87f019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f01065c0 0x7f87f019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87f0197920 0x7f87f0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0111680 con 0x7f87f01014f0 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0111500 con 0x7f87f01065c0 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87f571b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0111800 con 0x7f87f0197920 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01014f0 0x7f87f019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f01065c0 0x7f87f019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01014f0 0x7f87f019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36410/0 (socket says 172.21.15.22:36410) 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87eeffd640 1 -- 172.21.15.22:0/3222577213 learned_addr learned my addr 172.21.15.22:0/3222577213 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87f0197920 0x7f87f0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87eeffd640 1 -- 172.21.15.22:0/3222577213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87f0197920 msgr2=0x7f87f0197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.000+0000 7f87eeffd640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87f0197920 0x7f87f0197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87eeffd640 1 -- 172.21.15.22:0/3222577213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f01065c0 msgr2=0x7f87f019dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87eeffd640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f01065c0 0x7f87f019dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87eeffd640 1 -- 172.21.15.22:0/3222577213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0198440 con 0x7f87f01014f0 2024-03-09T00:02:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f01065c0 0x7f87f019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:32.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87eeffd640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01014f0 0x7f87f019d770 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e9f0 tx=0x7f87e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:32.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87e7fff640 1 -- 172.21.15.22:0/3222577213 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e000cd80 con 0x7f87f01014f0 2024-03-09T00:02:32.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f0198730 con 0x7f87f01014f0 2024-03-09T00:02:32.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87e7fff640 1 -- 172.21.15.22:0/3222577213 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87e0005500 con 0x7f87f01014f0 2024-03-09T00:02:32.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87e7fff640 1 -- 172.21.15.22:0/3222577213 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0010690 con 0x7f87f01014f0 2024-03-09T00:02:32.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.001+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f01a4980 con 0x7f87f01014f0 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.003+0000 7f87e7fff640 1 -- 172.21.15.22:0/3222577213 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f87e00046c0 con 0x7f87f01014f0 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.003+0000 7f87f83da740 1 Processor -- start 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.003+0000 7f87f83da740 1 -- start start 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87f83da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2107d80 0x55b8c2108160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87f83da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c2110d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87f83da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2111280 0x55b8c2113670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87f83da740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c206bb40 con 0x55b8c21086a0 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87f83da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c2014d80 con 0x55b8c2107d80 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87f83da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c2067be0 con 0x55b8c2111280 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87e7fff640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87dc0a2970 0x7f87dc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2107d80 0x55b8c2108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2111280 0x55b8c2113670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c2110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2107d80 0x55b8c2108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55228/0 (socket says 172.21.15.22:55228) 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2111280 0x55b8c2113670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:40482/0 (socket says 172.21.15.22:40482) 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ef7fe640 1 -- 172.21.15.22:0/94969014 learned_addr learned my addr 172.21.15.22:0/94969014 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87dc0a2970 0x7f87dc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87e7fff640 1 -- 172.21.15.22:0/3222577213 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f87e00cf650 con 0x7f87f01014f0 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ee7fc640 1 -- 172.21.15.22:0/94969014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2107d80 msgr2=0x55b8c2108160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2107d80 0x55b8c2108160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ee7fc640 1 -- 172.21.15.22:0/94969014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 msgr2=0x55b8c2110d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c2110d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ee7fc640 1 -- 172.21.15.22:0/94969014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8c1e487e0 con 0x55b8c2111280 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.004+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2107d80 0x55b8c2108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:33.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.005+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87dc0a2970 0x7f87dc0a4e30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87e8032530 tx=0x7f87e8005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.005+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2111280 0x55b8c2113670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87f0199110 tx=0x7f87d80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223b3390b0807fae server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.005+0000 7f87e57fa640 1 -- 172.21.15.22:0/94969014 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8011700 con 0x55b8c2111280 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.005+0000 7f87e57fa640 1 -- 172.21.15.22:0/94969014 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d8011d80 con 0x55b8c2111280 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.005+0000 7f87e57fa640 1 -- 172.21.15.22:0/94969014 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d801ad10 con 0x55b8c2111280 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- 172.21.15.22:0/94969014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2111280 msgr2=0x55b8c2113670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2111280 0x55b8c2113670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87f0199110 tx=0x7f87d80107a0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- 172.21.15.22:0/94969014 shutdown_connections 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2111280 0x55b8c2113670 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c2110d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 --2- 172.21.15.22:0/94969014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2107d80 0x55b8c2108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- 172.21.15.22:0/94969014 >> 172.21.15.22:0/94969014 conn(0x55b8c21024b0 msgr2=0x55b8c2102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- 172.21.15.22:0/94969014 shutdown_connections 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- 172.21.15.22:0/94969014 wait complete. 2024-03-09T00:02:33.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- start start 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 Processor -- start 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2110c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c21086a0 0x55b8c210df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c2111280 0x55b8c210e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c2114070 con 0x55b8c2111280 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c20281c0 con 0x55b8c21086a0 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8c21141f0 con 0x55b8c2107d80 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.006+0000 7f87f83da740 10 client.0 _ll_register_callbacks cb 0x55b8c20177a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c2111280 0x55b8c210e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c21086a0 0x55b8c210df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2110c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c21086a0 0x55b8c210df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55238/0 (socket says 172.21.15.22:55238) 2024-03-09T00:02:33.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ee7fc640 1 -- 172.21.15.22:0/1431103083 learned_addr learned my addr 172.21.15.22:0/1431103083 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ef7fe640 1 -- 172.21.15.22:0/1431103083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 msgr2=0x55b8c2110c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2110c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ef7fe640 1 -- 172.21.15.22:0/1431103083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c21086a0 msgr2=0x55b8c210df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c21086a0 0x55b8c210df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ef7fe640 1 -- 172.21.15.22:0/1431103083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8c1ea8f70 con 0x55b8c2111280 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c21086a0 0x55b8c210df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c2111280 0x55b8c210e510 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009a60 tx=0x7f87e802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8a5929eb9f9940 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87d67fc640 1 -- 172.21.15.22:0/1431103083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8053060 con 0x55b8c2111280 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87d67fc640 1 -- 172.21.15.22:0/1431103083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f87e8004040 con 0x55b8c2111280 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87d8011d80 con 0x55b8c2111280 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87d67fc640 1 -- 172.21.15.22:0/1431103083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e8039040 con 0x55b8c2111280 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87f83da740 10 client.18168 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:33.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.007+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b8c1d40170 con 0x55b8c2111280 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87d67fc640 1 -- 172.21.15.22:0/1431103083 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d8011d80 con 0x55b8c2111280 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87d67fc640 1 -- 172.21.15.22:0/1431103083 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87e804e960 con 0x55b8c2111280 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87f83da740 10 client.18168 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87f83da740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87f83da740 20 client.18168 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87f83da740 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2121320 0x55b8c2123710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87f83da740 4 client.18168 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8c2041960 con 0x55b8c2121320 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87eeffd640 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2121320 0x55b8c2123710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87d67fc640 10 client.18168 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.008+0000 7f87eeffd640 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2121320 0x55b8c2123710 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc38c7c616ab0a5 server_cookie=136b12afe783cc72 in_seq=0 out_seq=0 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.009+0000 7f87d67fc640 1 -- 172.21.15.22:0/1431103083 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8c2041960 con 0x55b8c2121320 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.009+0000 7f87d67fc640 10 client.18168 handle_command_reply: tid=0 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.009+0000 7f87f83da740 1 client.18168 shutdown 2024-03-09T00:02:33.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.009+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c2111280 msgr2=0x55b8c210e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.009+0000 7f87f83da740 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c2111280 0x55b8c210e510 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009a60 tx=0x7f87e802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 shutdown_connections 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2121320 0x55b8c2123710 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c2111280 0x55b8c210e510 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c21086a0 0x55b8c210df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87d67fc640 0 client.18168 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 --2- 172.21.15.22:0/1431103083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 >> 172.21.15.22:0/1431103083 conn(0x55b8c21024b0 msgr2=0x55b8c2104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 shutdown_connections 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 1 -- 172.21.15.22:0/1431103083 wait complete. 2024-03-09T00:02:33.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.010+0000 7f87f83da740 20 client.18168 trim_cache size 0 max 16384 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.026+0000 7f87f83da740 1 Processor -- start 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.026+0000 7f87f83da740 1 -- start start 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87f83da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c21135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87f83da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87f83da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2110c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87f83da740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01b0bc0 con 0x55b8c21086a0 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87f83da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01ae760 con 0x55b8c2111280 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87f83da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01ac820 con 0x55b8c2107d80 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36446/0 (socket says 172.21.15.22:36446) 2024-03-09T00:02:33.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 -- 172.21.15.22:0/1745607508 learned_addr learned my addr 172.21.15.22:0/1745607508 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87eeffd640 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c21135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2110c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 -- 172.21.15.22:0/1745607508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 msgr2=0x55b8c21135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c21135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 -- 172.21.15.22:0/1745607508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 msgr2=0x55b8c2110c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2110c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 -- 172.21.15.22:0/1745607508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8c2123a80 con 0x55b8c21086a0 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21106c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f87d800bb30 tx=0x7f87d801a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf53853c0b554a6f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87e4ff9640 1 -- 172.21.15.22:0/1745607508 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d801a500 con 0x55b8c21086a0 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.027+0000 7f87e4ff9640 1 -- 172.21.15.22:0/1745607508 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d801a6a0 con 0x55b8c21086a0 2024-03-09T00:02:33.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87e4ff9640 1 -- 172.21.15.22:0/1745607508 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80053a0 con 0x55b8c21086a0 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 -- 172.21.15.22:0/1745607508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 msgr2=0x55b8c21106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21106c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f87d800bb30 tx=0x7f87d801a8f0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 -- 172.21.15.22:0/1745607508 shutdown_connections 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21106c0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 --2- 172.21.15.22:0/1745607508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c21135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 -- 172.21.15.22:0/1745607508 >> 172.21.15.22:0/1745607508 conn(0x55b8c21024b0 msgr2=0x55b8c2109240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 -- 172.21.15.22:0/1745607508 shutdown_connections 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 -- 172.21.15.22:0/1745607508 wait complete. 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.028+0000 7f87f83da740 1 -- start start 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 1 Processor -- start 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2114e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2115910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01ace20 con 0x55b8c21086a0 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01ae100 con 0x55b8c2111280 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01ad440 con 0x55b8c2107d80 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87f83da740 10 client.0 _ll_register_callbacks cb 0x7f87d800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21153d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36448/0 (socket says 172.21.15.22:36448) 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2114e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 -- 172.21.15.22:0/3774191220 learned_addr learned my addr 172.21.15.22:0/3774191220 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ef7fe640 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 -- 172.21.15.22:0/3774191220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 msgr2=0x55b8c2114e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2114e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 -- 172.21.15.22:0/3774191220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 msgr2=0x55b8c2115910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2115910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 -- 172.21.15.22:0/3774191220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8c2177840 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87ee7fc640 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21153d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f87d800be80 tx=0x7f87d801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0208b7490c7af07 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.029+0000 7f87d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87d67fc640 1 -- 172.21.15.22:0/3774191220 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8034070 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87d67fc640 1 -- 172.21.15.22:0/3774191220 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f87d8005250 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87d801a6a0 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87d67fc640 1 -- 172.21.15.22:0/3774191220 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d8040050 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 10 client.18174 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b8c2178b60 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87d67fc640 1 -- 172.21.15.22:0/3774191220 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d801a6a0 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87d67fc640 1 -- 172.21.15.22:0/3774191220 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87d8004b40 con 0x55b8c21086a0 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 10 client.18174 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 20 client.18174 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:33.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2179040 0x55b8c217b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 4 client.18174 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b8c202d080 con 0x55b8c2179040 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.030+0000 7f87eeffd640 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2179040 0x55b8c217b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.031+0000 7f87d67fc640 10 client.18174 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.031+0000 7f87eeffd640 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2179040 0x55b8c217b460 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da29bfefed4cbba5 server_cookie=e6dc95db9c7662e4 in_seq=0 out_seq=0 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.032+0000 7f87d67fc640 1 -- 172.21.15.22:0/3774191220 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b8c202d080 con 0x55b8c2179040 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.032+0000 7f87d67fc640 10 client.18174 handle_command_reply: tid=0 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.032+0000 7f87f83da740 1 client.18174 shutdown 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 msgr2=0x55b8c21153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21153d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f87d800be80 tx=0x7f87d801ac60 comp rx=0 tx=0).stop 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 shutdown_connections 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b8c2179040 0x55b8c217b460 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87d67fc640 0 client.18174 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8c2111280 0x55b8c2115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8c21086a0 0x55b8c21153d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 --2- 172.21.15.22:0/3774191220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b8c2107d80 0x55b8c2114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 >> 172.21.15.22:0/3774191220 conn(0x55b8c21024b0 msgr2=0x55b8c21038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 shutdown_connections 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 1 -- 172.21.15.22:0/3774191220 wait complete. 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f83da740 20 client.18174 trim_cache size 0 max 16384 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "acd87508-f9ee-49a3-9ae2-b6145f3d59e7", 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:02:33.033 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:02:33.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87dc0a2970 msgr2=0x7f87dc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f571b640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87dc0a2970 0x7f87dc0a4e30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87e8032530 tx=0x7f87e8005c50 comp rx=0 tx=0).stop 2024-03-09T00:02:33.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01014f0 msgr2=0x7f87f019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f571b640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01014f0 0x7f87f019d770 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e9f0 tx=0x7f87e000edf0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.033+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 shutdown_connections 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.034+0000 7f87f571b640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87dc0a2970 0x7f87dc0a4e30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.034+0000 7f87f571b640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87f0197920 0x7f87f0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.034+0000 7f87f571b640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f01065c0 0x7f87f019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.034+0000 7f87f571b640 1 --2- 172.21.15.22:0/3222577213 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87f01014f0 0x7f87f019d770 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.034+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 >> 172.21.15.22:0/3222577213 conn(0x7f87f00786f0 msgr2=0x7f87f00ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.034+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 shutdown_connections 2024-03-09T00:02:33.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.034+0000 7f87f571b640 1 -- 172.21.15.22:0/3222577213 wait complete. 2024-03-09T00:02:33.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:02:33.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:02:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 -- 172.21.15.22:0/1418397711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 msgr2=0x7f30341007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 --2- 172.21.15.22:0/1418397711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f30341007b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f302c009900 tx=0x7f302c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 -- 172.21.15.22:0/1418397711 shutdown_connections 2024-03-09T00:02:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 --2- 172.21.15.22:0/1418397711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30341054a0 0x7f303410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 --2- 172.21.15.22:0/1418397711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3034100d80 0x7f3034104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 --2- 172.21.15.22:0/1418397711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f30341007b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 -- 172.21.15.22:0/1418397711 >> 172.21.15.22:0/1418397711 conn(0x7f30340772f0 msgr2=0x7f3034077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 -- 172.21.15.22:0/1418397711 shutdown_connections 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 -- 172.21.15.22:0/1418397711 wait complete. 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 Processor -- start 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 -- start start 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f303419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.154+0000 7f303c6fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30341054a0 0x7f303419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303c6fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3034197970 0x7f3034197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303c6fc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034110550 con 0x7f30341003d0 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303c6fc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341103d0 con 0x7f30341054a0 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303c6fc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341106d0 con 0x7f3034197970 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f303419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f303419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36466/0 (socket says 172.21.15.22:36466) 2024-03-09T00:02:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 -- 172.21.15.22:0/2817162957 learned_addr learned my addr 172.21.15.22:0/2817162957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f3039c97640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30341054a0 0x7f303419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303ac99640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3034197970 0x7f3034197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 -- 172.21.15.22:0/2817162957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3034197970 msgr2=0x7f3034197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3034197970 0x7f3034197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 -- 172.21.15.22:0/2817162957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30341054a0 msgr2=0x7f303419dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30341054a0 0x7f303419dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 -- 172.21.15.22:0/2817162957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3034198490 con 0x7f30341003d0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303a498640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f303419d7c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f302c006010 tx=0x7f302c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f302b7fe640 1 -- 172.21.15.22:0/2817162957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c041070 con 0x7f30341003d0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f302b7fe640 1 -- 172.21.15.22:0/2817162957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f302c004450 con 0x7f30341003d0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3034198720 con 0x7f30341003d0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f302b7fe640 1 -- 172.21.15.22:0/2817162957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c003d00 con 0x7f30341003d0 2024-03-09T00:02:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.155+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341a4910 con 0x7f30341003d0 2024-03-09T00:02:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.157+0000 7f302b7fe640 1 -- 172.21.15.22:0/2817162957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f302c044070 con 0x7f30341003d0 2024-03-09T00:02:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.158+0000 7f302b7fe640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30200a2970 0x7f30200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.158+0000 7f303f3b4740 1 Processor -- start 2024-03-09T00:02:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.158+0000 7f3039c97640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30200a2970 0x7f30200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.158+0000 7f302b7fe640 1 -- 172.21.15.22:0/2817162957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f302c0f0c30 con 0x7f30341003d0 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.158+0000 7f303f3b4740 1 -- start start 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6a760 0x562225e6ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e76050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303f3b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562225dcde00 con 0x562225e73c60 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303f3b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562225d76b10 con 0x562225e6a760 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303f3b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562225dc9ea0 con 0x562225e6b080 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f3039c97640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30200a2970 0x7f30200a4e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f30340fdc70 tx=0x7f3024005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e76050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303a498640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6a760 0x562225e6ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e76050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36470/0 (socket says 172.21.15.22:36470) 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 -- 172.21.15.22:0/734671007 learned_addr learned my addr 172.21.15.22:0/734671007 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f3039c97640 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 -- 172.21.15.22:0/734671007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 msgr2=0x562225e73720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 -- 172.21.15.22:0/734671007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6a760 msgr2=0x562225e6ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6a760 0x562225e6ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 -- 172.21.15.22:0/734671007 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562225bac7d0 con 0x562225e73c60 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303a498640 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6a760 0x562225e6ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f303ac99640 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e76050 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3030009f60 tx=0x7f3030009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd81d70c38bd17a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f3028ff9640 1 -- 172.21.15.22:0/734671007 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303001e070 con 0x562225e73c60 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f3028ff9640 1 -- 172.21.15.22:0/734671007 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3030003c30 con 0x562225e73c60 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.159+0000 7f3028ff9640 1 -- 172.21.15.22:0/734671007 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303000f9c0 con 0x562225e73c60 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 -- 172.21.15.22:0/734671007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 msgr2=0x562225e76050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e76050 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3030009f60 tx=0x7f3030009f90 comp rx=0 tx=0).stop 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 -- 172.21.15.22:0/734671007 shutdown_connections 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e76050 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 --2- 172.21.15.22:0/734671007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6a760 0x562225e6ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 -- 172.21.15.22:0/734671007 >> 172.21.15.22:0/734671007 conn(0x562225e64e90 msgr2=0x562225e65270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 -- 172.21.15.22:0/734671007 shutdown_connections 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.160+0000 7f303f3b4740 1 -- 172.21.15.22:0/734671007 wait complete. 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 -- start start 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 Processor -- start 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6a760 0x562225e73370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6b080 0x562225e70700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e70c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562225e76a50 con 0x562225e73c60 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562225d8a0a0 con 0x562225e6b080 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562225e76bd0 con 0x562225e6a760 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303f3b4740 10 client.0 _ll_register_callbacks cb 0x562225d89a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e70c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303a498640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6a760 0x562225e73370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e70c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36476/0 (socket says 172.21.15.22:36476) 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 -- 172.21.15.22:0/897546341 learned_addr learned my addr 172.21.15.22:0/897546341 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f3039c97640 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6b080 0x562225e70700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 -- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6a760 msgr2=0x562225e73370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6a760 0x562225e73370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 -- 172.21.15.22:0/897546341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6b080 msgr2=0x562225e70700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6b080 0x562225e70700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303ac99640 1 -- 172.21.15.22:0/897546341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562225ae23f0 con 0x562225e73c60 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.161+0000 7f303a498640 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6a760 0x562225e73370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303ac99640 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e70c40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f303001c040 tx=0x7f30300048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc594f34655c3530 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f301dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f301dffb640 1 -- 172.21.15.22:0/897546341 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030027020 con 0x562225e73c60 2024-03-09T00:02:33.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f301dffb640 1 -- 172.21.15.22:0/897546341 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f3030004050 con 0x562225e73c60 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3030003c30 con 0x562225e73c60 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f301dffb640 1 -- 172.21.15.22:0/897546341 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3030032050 con 0x562225e73c60 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 10 client.18186 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x562225aa20f0 con 0x562225e73c60 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f301dffb640 1 -- 172.21.15.22:0/897546341 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3030003c30 con 0x562225e73c60 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f301dffb640 1 -- 172.21.15.22:0/897546341 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3030005200 con 0x562225e73c60 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 10 client.18186 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 20 client.18186 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225e83650 0x562225e85a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 4 client.18186 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.162+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562225ddb340 con 0x562225e83650 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.163+0000 7f303a498640 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225e83650 0x562225e85a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.163+0000 7f303a498640 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225e83650 0x562225e85a40 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f994099265b48dfd server_cookie=17f5ef4046858460 in_seq=0 out_seq=0 2024-03-09T00:02:33.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.163+0000 7f301dffb640 10 client.18186 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f301dffb640 1 -- 172.21.15.22:0/897546341 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562225ddb340 con 0x562225e83650 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f301dffb640 10 client.18186 handle_command_reply: tid=0 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 client.18186 shutdown 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 msgr2=0x562225e70c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e70c40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f303001c040 tx=0x7f30300048a0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 shutdown_connections 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225e83650 0x562225e85a40 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e73c60 0x562225e70c40 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e6b080 0x562225e70700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 --2- 172.21.15.22:0/897546341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6a760 0x562225e73370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f301dffb640 0 client.18186 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 >> 172.21.15.22:0/897546341 conn(0x562225e64e90 msgr2=0x562225e66be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 shutdown_connections 2024-03-09T00:02:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 1 -- 172.21.15.22:0/897546341 wait complete. 2024-03-09T00:02:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.165+0000 7f303f3b4740 20 client.18186 trim_cache size 0 max 16384 2024-03-09T00:02:33.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 Processor -- start 2024-03-09T00:02:33.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 -- start start 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e72e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225e7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341b0bb0 con 0x562225e6a760 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341ae750 con 0x562225e73c60 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303f3b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341ac810 con 0x562225e6b080 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e72e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e72e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36480/0 (socket says 172.21.15.22:36480) 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 -- 172.21.15.22:0/3431257217 learned_addr learned my addr 172.21.15.22:0/3431257217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f3039c97640 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303ac99640 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225e7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 -- 172.21.15.22:0/3431257217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 msgr2=0x562225e73370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 -- 172.21.15.22:0/3431257217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 msgr2=0x562225e7ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225e7ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 -- 172.21.15.22:0/3431257217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562225e85db0 con 0x562225e6a760 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303ac99640 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225e7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f3039c97640 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.180+0000 7f303a498640 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e72e30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f302c000c00 tx=0x7f302c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c08ae5f8bbda483 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f301ffff640 1 -- 172.21.15.22:0/3431257217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c04b040 con 0x562225e6a760 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f301ffff640 1 -- 172.21.15.22:0/3431257217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f302c005570 con 0x562225e6a760 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f301ffff640 1 -- 172.21.15.22:0/3431257217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0b9620 con 0x562225e6a760 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 -- 172.21.15.22:0/3431257217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 msgr2=0x562225e72e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e72e30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f302c000c00 tx=0x7f302c005c90 comp rx=0 tx=0).stop 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 -- 172.21.15.22:0/3431257217 shutdown_connections 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225e7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e73370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3431257217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e72e30 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 -- 172.21.15.22:0/3431257217 >> 172.21.15.22:0/3431257217 conn(0x562225e64e90 msgr2=0x562225e706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 -- 172.21.15.22:0/3431257217 shutdown_connections 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 -- 172.21.15.22:0/3431257217 wait complete. 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.181+0000 7f303f3b4740 1 -- start start 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 1 Processor -- start 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e77030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e77570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225ed29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341ace10 con 0x562225e6a760 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341ae0f0 con 0x562225e73c60 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341ad430 con 0x562225e6b080 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303f3b4740 10 client.0 _ll_register_callbacks cb 0x7f303001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:33.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e77030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e77030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36484/0 (socket says 172.21.15.22:36484) 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 -- 172.21.15.22:0/3117472190 learned_addr learned my addr 172.21.15.22:0/3117472190 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f3039c97640 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e77570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303ac99640 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225ed29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 -- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 msgr2=0x562225e77570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e77570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 -- 172.21.15.22:0/3117472190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 msgr2=0x562225ed29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225ed29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 -- 172.21.15.22:0/3117472190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562225ed2fc0 con 0x562225e6a760 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f303a498640 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e77030 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f302c005940 tx=0x7f302c0bcfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a96172a3d67bc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f301dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.182+0000 7f301dffb640 1 -- 172.21.15.22:0/3117472190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0d9070 con 0x562225e6a760 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f301dffb640 1 -- 172.21.15.22:0/3117472190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f302c0c52c0 con 0x562225e6a760 2024-03-09T00:02:33.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562225dd5690 con 0x562225e6a760 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f301dffb640 1 -- 172.21.15.22:0/3117472190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f302c04b040 con 0x562225e6a760 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 10 client.18192 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x562225eda350 con 0x562225e6a760 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f301dffb640 1 -- 172.21.15.22:0/3117472190 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562225dd5690 con 0x562225e6a760 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f301dffb640 1 -- 172.21.15.22:0/3117472190 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f302c0b8d00 con 0x562225e6a760 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 10 client.18192 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 20 client.18192 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225eda880 0x562225edcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 4 client.18192 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562225d534e0 con 0x562225eda880 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.183+0000 7f3039c97640 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225eda880 0x562225edcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.184+0000 7f3039c97640 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225eda880 0x562225edcc70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8ea9d040233a34 server_cookie=85c94f7cafe00443 in_seq=0 out_seq=0 2024-03-09T00:02:33.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.184+0000 7f301dffb640 10 client.18192 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f301dffb640 1 -- 172.21.15.22:0/3117472190 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562225d534e0 con 0x562225eda880 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f301dffb640 10 client.18192 handle_command_reply: tid=0 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 client.18192 shutdown 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 msgr2=0x562225e77030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e77030 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f302c005940 tx=0x7f302c0bcfb0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 shutdown_connections 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562225eda880 0x562225edcc70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562225e73c60 0x562225ed29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f301dffb640 0 client.18192 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:33.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562225e6b080 0x562225e77570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 --2- 172.21.15.22:0/3117472190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562225e6a760 0x562225e77030 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 >> 172.21.15.22:0/3117472190 conn(0x562225e64e90 msgr2=0x562225e701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 shutdown_connections 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.186+0000 7f303f3b4740 1 -- 172.21.15.22:0/3117472190 wait complete. 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303f3b4740 20 client.18192 trim_cache size 0 max 16384 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stdout: "acd87508-f9ee-49a3-9ae2-b6145f3d59e7": { 2024-03-09T00:02:33.187 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:02:33.188 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "acd87508-f9ee-49a3-9ae2-b6145f3d59e7", 2024-03-09T00:02:33.188 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:02:33.188 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:02:33.188 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:02:33.188 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30200a2970 msgr2=0x7f30200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30200a2970 0x7f30200a4e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f30340fdc70 tx=0x7f3024005f90 comp rx=0 tx=0).stop 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 msgr2=0x7f303419d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f303419d7c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f302c006010 tx=0x7f302c02cb30 comp rx=0 tx=0).stop 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 shutdown_connections 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30200a2970 0x7f30200a4e30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3034197970 0x7f3034197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30341054a0 0x7f303419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 --2- 172.21.15.22:0/2817162957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30341003d0 0x7f303419d7c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 >> 172.21.15.22:0/2817162957 conn(0x7f30340772f0 msgr2=0x7f3034068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 shutdown_connections 2024-03-09T00:02:33.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:02:33.187+0000 7f303c6fc640 1 -- 172.21.15.22:0/2817162957 wait complete. 2024-03-09T00:02:33.193 INFO:tasks.cephfs.filesystem:scrub status for tag:acd87508-f9ee-49a3-9ae2-b6145f3d59e7 - {'path': '/', 'tag': 'acd87508-f9ee-49a3-9ae2-b6145f3d59e7', 'options': 'recursive,force'} 2024-03-09T00:02:33.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 841411 10.62 MB/sec execute 281 sec latency 27.044 ms 2024-03-09T00:02:34.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:34 smithi062 ceph-mon[36129]: pgmap v253: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-03-09T00:02:34.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:34 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:02:34.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:34 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:34.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:34 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:02:34.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:34 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:34.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:34 smithi192 ceph-mon[35995]: pgmap v253: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-03-09T00:02:34.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:34 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:02:34.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:34 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:34.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:34 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:02:34.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:34 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:34.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:34 smithi022 ceph-mon[31894]: pgmap v253: 129 pgs: 129 active+clean; 75 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-03-09T00:02:34.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:34 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:02:34.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:34 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:34.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:34 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:02:34.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:34 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:02:34.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 842322 10.59 MB/sec execute 282 sec latency 36.041 ms 2024-03-09T00:02:35.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:35 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:02:35.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:35 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:02:35.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:35 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:02:35.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 844880 10.61 MB/sec execute 283 sec latency 21.146 ms 2024-03-09T00:02:36.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:36 smithi062 ceph-mon[36129]: pgmap v254: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-03-09T00:02:36.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:36 smithi192 ceph-mon[35995]: pgmap v254: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-03-09T00:02:36.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:36 smithi022 ceph-mon[31894]: pgmap v254: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-03-09T00:02:36.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 845843 10.59 MB/sec execute 284 sec latency 27.150 ms 2024-03-09T00:02:37.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 847920 10.59 MB/sec execute 285 sec latency 110.269 ms 2024-03-09T00:02:38.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:38 smithi062 ceph-mon[36129]: pgmap v255: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 320 op/s 2024-03-09T00:02:38.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:38 smithi192 ceph-mon[35995]: pgmap v255: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 320 op/s 2024-03-09T00:02:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:38 smithi022 ceph-mon[31894]: pgmap v255: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 320 op/s 2024-03-09T00:02:38.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 849115 10.58 MB/sec execute 286 sec latency 24.452 ms 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 -- 172.21.15.22:0/522431135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590101530 msgr2=0x7f0590101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 --2- 172.21.15.22:0/522431135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590101530 0x7f0590101910 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0588009900 tx=0x7f058802bc30 comp rx=0 tx=0).stop 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 -- 172.21.15.22:0/522431135 shutdown_connections 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 --2- 172.21.15.22:0/522431135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0590106600 0x7f059010e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 --2- 172.21.15.22:0/522431135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0590101ee0 0x7f0590105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 --2- 172.21.15.22:0/522431135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590101530 0x7f0590101910 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 -- 172.21.15.22:0/522431135 >> 172.21.15.22:0/522431135 conn(0x7f05900fcdd0 msgr2=0x7f05900ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 -- 172.21.15.22:0/522431135 shutdown_connections 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.214+0000 7f05983cd640 1 -- 172.21.15.22:0/522431135 wait complete. 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 Processor -- start 2024-03-09T00:02:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 -- start start 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0590101530 0x7f059019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590106600 0x7f059019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0590197950 0x7f0590197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0590111690 con 0x7f0590106600 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0590111510 con 0x7f0590197950 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f05983cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0590111810 con 0x7f0590101530 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f0595968640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590106600 0x7f059019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f0595968640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590106600 0x7f059019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36492/0 (socket says 172.21.15.22:36492) 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f0595968640 1 -- 172.21.15.22:0/3238227938 learned_addr learned my addr 172.21.15.22:0/3238227938 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f0596169640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0590101530 0x7f059019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.215+0000 7f059696a640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0590197950 0x7f0590197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f0595968640 1 -- 172.21.15.22:0/3238227938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0590101530 msgr2=0x7f059019d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f0595968640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0590101530 0x7f059019d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f0595968640 1 -- 172.21.15.22:0/3238227938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0590197950 msgr2=0x7f0590197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f0595968640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0590197950 0x7f0590197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f0595968640 1 -- 172.21.15.22:0/3238227938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590198470 con 0x7f0590106600 2024-03-09T00:02:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f0595968640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590106600 0x7f059019dce0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f058000d770 tx=0x7f058000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f05877fe640 1 -- 172.21.15.22:0/3238227938 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580017070 con 0x7f0590106600 2024-03-09T00:02:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f05877fe640 1 -- 172.21.15.22:0/3238227938 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0580009c30 con 0x7f0590106600 2024-03-09T00:02:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0590198760 con 0x7f0590106600 2024-03-09T00:02:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f05877fe640 1 -- 172.21.15.22:0/3238227938 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058000f620 con 0x7f0590106600 2024-03-09T00:02:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.216+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05901a4a00 con 0x7f0590106600 2024-03-09T00:02:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 Processor -- start 2024-03-09T00:02:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 -- start start 2024-03-09T00:02:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f05877fe640 1 -- 172.21.15.22:0/3238227938 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f058001a070 con 0x7f0590106600 2024-03-09T00:02:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba064d40 0x5592ba063a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba064450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba071960 0x5592ba073d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b9fc85f0 con 0x5592ba071960 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b9f71930 con 0x5592ba064d40 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059afa9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b9fc4690 con 0x5592ba063fd0 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f0595968640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba071960 0x5592ba073d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f0596169640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba064450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059696a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba064d40 0x5592ba063a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f0596169640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba064450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:40582/0 (socket says 172.21.15.22:40582) 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f059696a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba064d40 0x5592ba063a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55344/0 (socket says 172.21.15.22:55344) 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.219+0000 7f0596169640 1 -- 172.21.15.22:0/2227232094 learned_addr learned my addr 172.21.15.22:0/2227232094 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0595968640 1 -- 172.21.15.22:0/2227232094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 msgr2=0x5592ba064450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0595968640 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba064450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0595968640 1 -- 172.21.15.22:0/2227232094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba064d40 msgr2=0x5592ba063a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0595968640 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba064d40 0x5592ba063a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f05877fe640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f057c0a2a90 0x7f057c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0595968640 1 -- 172.21.15.22:0/2227232094 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592b9cdd210 con 0x5592ba071960 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f059696a640 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba064d40 0x5592ba063a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0596169640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f057c0a2a90 0x7f057c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f05877fe640 1 -- 172.21.15.22:0/3238227938 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f05800cec00 con 0x7f0590106600 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0595968640 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba071960 0x5592ba073d50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f05800cbf60 tx=0x7f05800d7740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f817bf40ddbf5c68 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0584bfe640 1 -- 172.21.15.22:0/2227232094 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580021030 con 0x5592ba071960 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0584bfe640 1 -- 172.21.15.22:0/2227232094 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0580009de0 con 0x5592ba071960 2024-03-09T00:02:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.220+0000 7f0584bfe640 1 -- 172.21.15.22:0/2227232094 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058009aa60 con 0x5592ba071960 2024-03-09T00:02:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 -- 172.21.15.22:0/2227232094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba071960 msgr2=0x5592ba073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f0596169640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f057c0a2a90 0x7f057c0a4f50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5592ba0654a0 tx=0x7f058802e040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba071960 0x5592ba073d50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f05800cbf60 tx=0x7f05800d7740 comp rx=0 tx=0).stop 2024-03-09T00:02:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 -- 172.21.15.22:0/2227232094 shutdown_connections 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba071960 0x5592ba073d50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba064450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 --2- 172.21.15.22:0/2227232094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba064d40 0x5592ba063a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 -- 172.21.15.22:0/2227232094 >> 172.21.15.22:0/2227232094 conn(0x5592ba02b5b0 msgr2=0x5592ba05ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 -- 172.21.15.22:0/2227232094 shutdown_connections 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 -- 172.21.15.22:0/2227232094 wait complete. 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 -- start start 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.221+0000 7f059afa9740 1 Processor -- start 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba06ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba06f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ba074690 con 0x5592ba064d40 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592b9f84d90 con 0x5592ba071960 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ba074810 con 0x5592ba063fd0 2024-03-09T00:02:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 10 client.0 _ll_register_callbacks cb 0x5592b9cdca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba06ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0596169640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba06ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36508/0 (socket says 172.21.15.22:36508) 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 -- 172.21.15.22:0/3079352642 learned_addr learned my addr 172.21.15.22:0/3079352642 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059696a640 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba06f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 -- 172.21.15.22:0/3079352642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 msgr2=0x5592ba06e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba06e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 -- 172.21.15.22:0/3079352642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 msgr2=0x5592ba06f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba06f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 -- 172.21.15.22:0/3079352642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592b9da9620 con 0x5592ba064d40 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0595968640 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba06ef10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f05800cc560 tx=0x7f05800cef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7368f31190b9f62f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0579ffb640 1 -- 172.21.15.22:0/3079352642 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058009a880 con 0x5592ba064d40 2024-03-09T00:02:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f0579ffb640 1 -- 172.21.15.22:0/3079352642 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f05800cc1e0 con 0x5592ba064d40 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.222+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0580009de0 con 0x5592ba064d40 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f0579ffb640 1 -- 172.21.15.22:0/3079352642 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f058009aa60 con 0x5592ba064d40 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 10 client.18204 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5592ba0809c0 con 0x5592ba064d40 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f0579ffb640 1 -- 172.21.15.22:0/3079352642 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0580009de0 con 0x5592ba064d40 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f0579ffb640 1 -- 172.21.15.22:0/3079352642 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05800d6040 con 0x5592ba064d40 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 10 client.18204 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 10 client.18204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 20 client.18204 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba081d10 0x5592ba084100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 4 client.18204 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592b9fd5b30 con 0x5592ba081d10 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.223+0000 7f0596169640 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba081d10 0x5592ba084100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f0579ffb640 10 client.18204 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f0596169640 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba081d10 0x5592ba084100 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8840da6495ff4f server_cookie=62a55996a0a5966f in_seq=0 out_seq=0 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f0579ffb640 1 -- 172.21.15.22:0/3079352642 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592b9fd5b30 con 0x5592ba081d10 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f0579ffb640 10 client.18204 handle_command_reply: tid=0 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 client.18204 shutdown 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 msgr2=0x5592ba06ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba06ef10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f05800cc560 tx=0x7f05800cef00 comp rx=0 tx=0).stop 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 shutdown_connections 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba081d10 0x5592ba084100 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba06f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba06ef10 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f0579ffb640 0 client.18204 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 --2- 172.21.15.22:0/3079352642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba06e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 >> 172.21.15.22:0/3079352642 conn(0x5592ba02b5b0 msgr2=0x5592ba060a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 shutdown_connections 2024-03-09T00:02:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 1 -- 172.21.15.22:0/3079352642 wait complete. 2024-03-09T00:02:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.224+0000 7f059afa9740 20 client.18204 trim_cache size 0 max 16384 2024-03-09T00:02:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.237+0000 7f059afa9740 1 Processor -- start 2024-03-09T00:02:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f059afa9740 1 -- start start 2024-03-09T00:02:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f059afa9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba063fd0 0x5592ba071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f059afa9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba064d40 0x5592ba06db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f059afa9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba07d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f059afa9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05901aea20 con 0x5592ba063fd0 2024-03-09T00:02:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f059afa9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05901ac5c0 con 0x5592ba071960 2024-03-09T00:02:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f059afa9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05901aa680 con 0x5592ba064d40 2024-03-09T00:02:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba063fd0 0x5592ba071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba063fd0 0x5592ba071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36520/0 (socket says 172.21.15.22:36520) 2024-03-09T00:02:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 -- 172.21.15.22:0/1882177031 learned_addr learned my addr 172.21.15.22:0/1882177031 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0595968640 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba064d40 0x5592ba06db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 -- 172.21.15.22:0/1882177031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba064d40 msgr2=0x5592ba06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba064d40 0x5592ba06db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 -- 172.21.15.22:0/1882177031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 msgr2=0x5592ba07d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba07d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 -- 172.21.15.22:0/1882177031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ba063e00 con 0x5592ba063fd0 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0595968640 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba064d40 0x5592ba06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f0596169640 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba063fd0 0x5592ba071100 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0588036040 tx=0x7f0588009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c783b062121cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f057bfff640 1 -- 172.21.15.22:0/1882177031 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588004240 con 0x5592ba063fd0 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f057bfff640 1 -- 172.21.15.22:0/1882177031 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0588058030 con 0x5592ba063fd0 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.238+0000 7f057bfff640 1 -- 172.21.15.22:0/1882177031 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588004b80 con 0x5592ba063fd0 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 -- 172.21.15.22:0/1882177031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba063fd0 msgr2=0x5592ba071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba063fd0 0x5592ba071100 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0588036040 tx=0x7f0588009260 comp rx=0 tx=0).stop 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 -- 172.21.15.22:0/1882177031 shutdown_connections 2024-03-09T00:02:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba07d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba064d40 0x5592ba06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 --2- 172.21.15.22:0/1882177031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba063fd0 0x5592ba071100 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 -- 172.21.15.22:0/1882177031 >> 172.21.15.22:0/1882177031 conn(0x5592ba02b5b0 msgr2=0x5592ba07caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 -- 172.21.15.22:0/1882177031 shutdown_connections 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 -- 172.21.15.22:0/1882177031 wait complete. 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 -- start start 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.239+0000 7f059afa9740 1 Processor -- start 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba084d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba085240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba085780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05901aac80 con 0x5592ba064d40 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05901abf60 con 0x5592ba071960 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05901ab2a0 con 0x5592ba063fd0 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 10 client.0 _ll_register_callbacks cb 0x7f05800d2620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba085240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba085240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36526/0 (socket says 172.21.15.22:36526) 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 -- 172.21.15.22:0/1342579944 learned_addr learned my addr 172.21.15.22:0/1342579944 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059696a640 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba085780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0596169640 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba084d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 -- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 msgr2=0x5592ba084d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba084d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 -- 172.21.15.22:0/1342579944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 msgr2=0x5592ba085780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba085780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 -- 172.21.15.22:0/1342579944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592ba0d7fc0 con 0x5592ba064d40 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0595968640 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba085240 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0580009610 tx=0x7f05800b9c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e9e7ec4f4c831a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0579ffb640 1 -- 172.21.15.22:0/1342579944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580021040 con 0x5592ba064d40 2024-03-09T00:02:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0579ffb640 1 -- 172.21.15.22:0/1342579944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f058009b180 con 0x5592ba064d40 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0588058030 con 0x5592ba064d40 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f0579ffb640 1 -- 172.21.15.22:0/1342579944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f058009a880 con 0x5592ba064d40 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 10 client.18210 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.240+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5592ba0d7fc0 con 0x5592ba064d40 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f0579ffb640 1 -- 172.21.15.22:0/1342579944 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0588058030 con 0x5592ba064d40 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f0579ffb640 1 -- 172.21.15.22:0/1342579944 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05800a3180 con 0x5592ba064d40 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f059afa9740 10 client.18210 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f059afa9740 10 client.18210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f059afa9740 20 client.18210 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f059afa9740 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba0d9560 0x5592ba0db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f059afa9740 4 client.18210 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5592b9fcfe80 con 0x5592ba0d9560 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f0596169640 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba0d9560 0x5592ba0db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f0579ffb640 10 client.18210 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.241+0000 7f0596169640 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba0d9560 0x5592ba0db980 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3648a5e4dfb89d server_cookie=6d444f7c8a6d67d0 in_seq=0 out_seq=0 2024-03-09T00:02:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f0579ffb640 1 -- 172.21.15.22:0/1342579944 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5592b9fcfe80 con 0x5592ba0d9560 2024-03-09T00:02:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f0579ffb640 10 client.18210 handle_command_reply: tid=0 2024-03-09T00:02:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 client.18210 shutdown 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 msgr2=0x5592ba085240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba085240 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0580009610 tx=0x7f05800b9c20 comp rx=0 tx=0).stop 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 shutdown_connections 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592ba0d9560 0x5592ba0db980 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592ba071960 0x5592ba085780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592ba064d40 0x5592ba085240 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 --2- 172.21.15.22:0/1342579944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592ba063fd0 0x5592ba084d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 >> 172.21.15.22:0/1342579944 conn(0x5592ba02b5b0 msgr2=0x5592ba060930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f0579ffb640 0 client.18210 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 shutdown_connections 2024-03-09T00:02:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 1 -- 172.21.15.22:0/1342579944 wait complete. 2024-03-09T00:02:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.534+0000 7f059afa9740 20 client.18210 trim_cache size 0 max 16384 2024-03-09T00:02:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:02:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:02:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:02:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:02:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f057c0a2a90 msgr2=0x7f057c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f057c0a2a90 0x7f057c0a4f50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5592ba0654a0 tx=0x7f058802e040 comp rx=0 tx=0).stop 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590106600 msgr2=0x7f059019dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590106600 0x7f059019dce0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f058000d770 tx=0x7f058000dc30 comp rx=0 tx=0).stop 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 shutdown_connections 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f057c0a2a90 0x7f057c0a4f50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0590197950 0x7f0590197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0590106600 0x7f059019dce0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 --2- 172.21.15.22:0/3238227938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0590101530 0x7f059019d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 >> 172.21.15.22:0/3238227938 conn(0x7f05900fcdd0 msgr2=0x7f05900ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 shutdown_connections 2024-03-09T00:02:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:39.535+0000 7f05983cd640 1 -- 172.21.15.22:0/3238227938 wait complete. 2024-03-09T00:02:39.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 850255 10.56 MB/sec execute 287 sec latency 164.072 ms 2024-03-09T00:02:40.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:40 smithi062 ceph-mon[36129]: pgmap v256: 129 pgs: 129 active+clean; 84 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 363 op/s 2024-03-09T00:02:40.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:40 smithi192 ceph-mon[35995]: pgmap v256: 129 pgs: 129 active+clean; 84 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 363 op/s 2024-03-09T00:02:40.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:40 smithi022 ceph-mon[31894]: pgmap v256: 129 pgs: 129 active+clean; 84 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 363 op/s 2024-03-09T00:02:40.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 852253 10.56 MB/sec execute 288 sec latency 25.790 ms 2024-03-09T00:02:41.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 853779 10.55 MB/sec execute 289 sec latency 29.477 ms 2024-03-09T00:02:42.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:42 smithi062 ceph-mon[36129]: pgmap v257: 129 pgs: 129 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-03-09T00:02:42.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:42 smithi192 ceph-mon[35995]: pgmap v257: 129 pgs: 129 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-03-09T00:02:42.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:42 smithi022 ceph-mon[31894]: pgmap v257: 129 pgs: 129 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-03-09T00:02:42.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 855794 10.56 MB/sec execute 290 sec latency 109.034 ms 2024-03-09T00:02:43.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 857146 10.54 MB/sec execute 291 sec latency 30.765 ms 2024-03-09T00:02:44.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:44 smithi062 ceph-mon[36129]: pgmap v258: 129 pgs: 129 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-03-09T00:02:44.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:44 smithi192 ceph-mon[35995]: pgmap v258: 129 pgs: 129 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-03-09T00:02:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:44 smithi022 ceph-mon[31894]: pgmap v258: 129 pgs: 129 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-03-09T00:02:44.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 859294 10.55 MB/sec execute 292 sec latency 17.665 ms 2024-03-09T00:02:45.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 860612 10.54 MB/sec execute 293 sec latency 30.418 ms 2024-03-09T00:02:46.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:46 smithi062 ceph-mon[36129]: pgmap v259: 129 pgs: 129 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-03-09T00:02:46.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:46 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:02:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:46 smithi022 ceph-mon[31894]: pgmap v259: 129 pgs: 129 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-03-09T00:02:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:46 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:02:46.499 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:46 smithi192 ceph-mon[35995]: pgmap v259: 129 pgs: 129 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-03-09T00:02:46.499 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:46 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:02:46.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 862751 10.54 MB/sec execute 294 sec latency 55.576 ms 2024-03-09T00:02:47.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:47 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:02:47.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:47 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:02:47.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:47 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:02:47.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:47 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:02:47.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:47 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:02:47.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:47 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:02:47.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:47 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:02:47.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:47 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:02:47.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:47 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:02:47.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 863571 10.51 MB/sec execute 295 sec latency 37.021 ms 2024-03-09T00:02:48.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 865673 10.52 MB/sec execute 296 sec latency 40.325 ms 2024-03-09T00:02:48.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:48 smithi022 ceph-mon[31894]: pgmap v260: 129 pgs: 129 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-09T00:02:49.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:48 smithi062 ceph-mon[36129]: pgmap v260: 129 pgs: 129 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-09T00:02:49.163 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:48 smithi192 ceph-mon[35995]: pgmap v260: 129 pgs: 129 active+clean; 77 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-09T00:02:49.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 866969 10.51 MB/sec execute 297 sec latency 18.892 ms 2024-03-09T00:02:50.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 869265 10.51 MB/sec execute 298 sec latency 15.106 ms 2024-03-09T00:02:50.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:50 smithi022 ceph-mon[31894]: pgmap v261: 129 pgs: 129 active+clean; 80 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-03-09T00:02:51.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:50 smithi062 ceph-mon[36129]: pgmap v261: 129 pgs: 129 active+clean; 80 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-03-09T00:02:51.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:50 smithi192 ceph-mon[35995]: pgmap v261: 129 pgs: 129 active+clean; 80 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-03-09T00:02:51.722 INFO:tasks.workunit.client.0.smithi022.stdout: 1 870502 10.50 MB/sec execute 299 sec latency 17.654 ms 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 -- 172.21.15.22:0/2515744538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100e00 msgr2=0x7f13a0068970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/2515744538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100e00 0x7f13a0068970 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1398009ca0 tx=0x7f139802c140 comp rx=0 tx=0).stop 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 -- 172.21.15.22:0/2515744538 shutdown_connections 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/2515744538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100e00 0x7f13a0068970 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/2515744538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13a0100440 0x7f13a01008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/2515744538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a0105190 0x7f13a0105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 -- 172.21.15.22:0/2515744538 >> 172.21.15.22:0/2515744538 conn(0x7f13a0077040 msgr2=0x7f13a0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 -- 172.21.15.22:0/2515744538 shutdown_connections 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.658+0000 7f13a7b7e640 1 -- 172.21.15.22:0/2515744538 wait complete. 2024-03-09T00:02:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 Processor -- start 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 -- start start 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100440 0x7f13a010f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13a0100e00 0x7f13a010f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a0105190 0x7f13a010fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a01139d0 con 0x7f13a0100e00 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0113850 con 0x7f13a0105190 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.659+0000 7f13a7b7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0113b50 con 0x7f13a0100440 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.660+0000 7f13a5119640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13a0100e00 0x7f13a010f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.660+0000 7f13a5119640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13a0100e00 0x7f13a010f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36182/0 (socket says 172.21.15.22:36182) 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.660+0000 7f13a5119640 1 -- 172.21.15.22:0/1325878957 learned_addr learned my addr 172.21.15.22:0/1325878957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.660+0000 7f13a611b640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a0105190 0x7f13a010fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.660+0000 7f13a591a640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100440 0x7f13a010f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.661+0000 7f13a611b640 1 -- 172.21.15.22:0/1325878957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100440 msgr2=0x7f13a010f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.661+0000 7f13a611b640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100440 0x7f13a010f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.661+0000 7f13a611b640 1 -- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13a0100e00 msgr2=0x7f13a010f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.661+0000 7f13a611b640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13a0100e00 0x7f13a010f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.661+0000 7f13a611b640 1 -- 172.21.15.22:0/1325878957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0071bd0 con 0x7f13a0105190 2024-03-09T00:02:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.662+0000 7f13a611b640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a0105190 0x7f13a010fd80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1398037040 tx=0x7f1398032f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.662+0000 7f1396ffd640 1 -- 172.21.15.22:0/1325878957 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398039040 con 0x7f13a0105190 2024-03-09T00:02:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.662+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a0071e00 con 0x7f13a0105190 2024-03-09T00:02:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.662+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a0072340 con 0x7f13a0105190 2024-03-09T00:02:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.664+0000 7f1396ffd640 1 -- 172.21.15.22:0/1325878957 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1398044070 con 0x7f13a0105190 2024-03-09T00:02:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.664+0000 7f1396ffd640 1 -- 172.21.15.22:0/1325878957 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398003de0 con 0x7f13a0105190 2024-03-09T00:02:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.666+0000 7f13aa765740 1 Processor -- start 2024-03-09T00:02:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.666+0000 7f1396ffd640 1 -- 172.21.15.22:0/1325878957 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1398032600 con 0x7f13a0105190 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.667+0000 7f13aa765740 1 -- start start 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.667+0000 7f13aa765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da3945da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.667+0000 7f13aa765740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da3945e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.667+0000 7f13aa765740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da39467550 0x55da39469940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.667+0000 7f13aa765740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da393c25f0 con 0x55da39467550 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.667+0000 7f13aa765740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3936b930 con 0x55da3945ed40 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.667+0000 7f13aa765740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da393be690 con 0x55da3945dfd0 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.668+0000 7f13a591a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da3945e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.668+0000 7f1396ffd640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f138c0a2b60 0x7f138c0a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.668+0000 7f13a591a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da3945e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58406/0 (socket says 172.21.15.22:58406) 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.668+0000 7f13a591a640 1 -- 172.21.15.22:0/1636971820 learned_addr learned my addr 172.21.15.22:0/1636971820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.668+0000 7f13a5119640 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da3945da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.668+0000 7f1396ffd640 1 -- 172.21.15.22:0/1325878957 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f13980f0ee0 con 0x7f13a0105190 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.669+0000 7f13a5119640 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da39467550 0x55da39469940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.669+0000 7f13a591a640 1 -- 172.21.15.22:0/1636971820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 msgr2=0x55da3945da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.669+0000 7f13a591a640 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da3945da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.669+0000 7f13a591a640 1 -- 172.21.15.22:0/1636971820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da39467550 msgr2=0x55da39469940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.669+0000 7f13a591a640 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da39467550 0x55da39469940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.669+0000 7f13a591a640 1 -- 172.21.15.22:0/1636971820 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da390d7210 con 0x55da3945dfd0 2024-03-09T00:02:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.670+0000 7f13a591a640 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da3945e450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b820 tx=0x7f139c00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a30bdfce34f7cffb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.670+0000 7f138bfff640 1 -- 172.21.15.22:0/1636971820 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c00d950 con 0x55da3945dfd0 2024-03-09T00:02:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.670+0000 7f138bfff640 1 -- 172.21.15.22:0/1636971820 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f139c019070 con 0x55da3945dfd0 2024-03-09T00:02:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.670+0000 7f138bfff640 1 -- 172.21.15.22:0/1636971820 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c0123e0 con 0x55da3945dfd0 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.671+0000 7f13a611b640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f138c0a2b60 0x7f138c0a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.672+0000 7f13aa765740 1 -- 172.21.15.22:0/1636971820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 msgr2=0x55da3945e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.673+0000 7f13aa765740 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da3945e450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b820 tx=0x7f139c00bce0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.673+0000 7f13a611b640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f138c0a2b60 0x7f138c0a5020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13980b9b30 tx=0x7f139802c140 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.673+0000 7f13aa765740 1 -- 172.21.15.22:0/1636971820 shutdown_connections 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.673+0000 7f13aa765740 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da39467550 0x55da39469940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.673+0000 7f13aa765740 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da3945e450 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.673+0000 7f13aa765740 1 --2- 172.21.15.22:0/1636971820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da3945da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.673+0000 7f13aa765740 1 -- 172.21.15.22:0/1636971820 >> 172.21.15.22:0/1636971820 conn(0x55da394255b0 msgr2=0x55da39458cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.674+0000 7f13aa765740 1 -- 172.21.15.22:0/1636971820 shutdown_connections 2024-03-09T00:02:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.675+0000 7f13aa765740 1 -- 172.21.15.22:0/1636971820 wait complete. 2024-03-09T00:02:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.676+0000 7f13aa765740 1 -- start start 2024-03-09T00:02:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.676+0000 7f13aa765740 1 Processor -- start 2024-03-09T00:02:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.677+0000 7f13aa765740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39464560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.677+0000 7f13aa765740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da39464aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.677+0000 7f13aa765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da39464fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13a5119640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39464560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13a591a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da39464aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13a591a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da39464aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36192/0 (socket says 172.21.15.22:36192) 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13a5119640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39464560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58420/0 (socket says 172.21.15.22:58420) 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13a591a640 1 -- 172.21.15.22:0/4156021595 learned_addr learned my addr 172.21.15.22:0/4156021595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3946a280 con 0x55da3945ed40 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3937ed90 con 0x55da39467550 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3946a400 con 0x55da3945dfd0 2024-03-09T00:02:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.678+0000 7f13a5119640 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da39464fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.679+0000 7f13a5119640 1 -- 172.21.15.22:0/4156021595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 msgr2=0x55da39464fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.679+0000 7f13a5119640 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da39464fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.679+0000 7f13a5119640 1 -- 172.21.15.22:0/4156021595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 msgr2=0x55da39464aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.679+0000 7f13a5119640 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da39464aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.679+0000 7f13a5119640 1 -- 172.21.15.22:0/4156021595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da391a3620 con 0x55da3945dfd0 2024-03-09T00:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.679+0000 7f13aa765740 10 client.0 _ll_register_callbacks cb 0x55da390d6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.679+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f139c019070 con 0x55da3945dfd0 2024-03-09T00:02:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.680+0000 7f13a5119640 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39464560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f13a0110ac0 tx=0x7f1390005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b9d67ef87cba9c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.680+0000 7f13897fa640 10 client.27667 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:02:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.682+0000 7f13897fa640 1 -- 172.21.15.22:0/4156021595 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139003b040 con 0x55da3945dfd0 2024-03-09T00:02:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.682+0000 7f13897fa640 1 -- 172.21.15.22:0/4156021595 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1390004030 con 0x55da3945dfd0 2024-03-09T00:02:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.683+0000 7f13897fa640 1 -- 172.21.15.22:0/4156021595 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1390004270 con 0x55da3945dfd0 2024-03-09T00:02:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.683+0000 7f13aa765740 10 client.27667 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.683+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x55da394762b0 con 0x55da3945dfd0 2024-03-09T00:02:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.684+0000 7f13897fa640 1 -- 172.21.15.22:0/4156021595 <== mon.1 v2:172.21.15.192:3300/0 4 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1390041030 con 0x55da3945dfd0 2024-03-09T00:02:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.685+0000 7f13aa765740 10 client.27667 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.686+0000 7f13aa765740 10 client.27667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.686+0000 7f13aa765740 20 client.27667 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.686+0000 7f13aa765740 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da39477660 0x55da39479a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.686+0000 7f13aa765740 4 client.27667 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:02:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.686+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da393cfb30 con 0x55da39477660 2024-03-09T00:02:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.686+0000 7f13a591a640 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da39477660 0x55da39479a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.690+0000 7f13897fa640 10 client.27667 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.690+0000 7f13a591a640 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da39477660 0x55da39479a50 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ff90f0c50b8dce server_cookie=aa9bd7cf10809b9 in_seq=0 out_seq=0 2024-03-09T00:02:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.691+0000 7f13897fa640 1 -- 172.21.15.22:0/4156021595 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da393cfb30 con 0x55da39477660 2024-03-09T00:02:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.691+0000 7f13897fa640 10 client.27667 handle_command_reply: tid=0 2024-03-09T00:02:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.692+0000 7f13aa765740 1 client.27667 shutdown 2024-03-09T00:02:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.692+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 msgr2=0x55da39464560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.692+0000 7f13aa765740 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39464560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f13a0110ac0 tx=0x7f1390005f90 comp rx=0 tx=0).stop 2024-03-09T00:02:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.692+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 shutdown_connections 2024-03-09T00:02:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.692+0000 7f13aa765740 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da39477660 0x55da39479a50 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13aa765740 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da39464fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13aa765740 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da39464aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13aa765740 1 --2- 172.21.15.22:0/4156021595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39464560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 >> 172.21.15.22:0/4156021595 conn(0x55da394255b0 msgr2=0x55da3945aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13897fa640 0 client.27667 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 shutdown_connections 2024-03-09T00:02:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13aa765740 1 -- 172.21.15.22:0/4156021595 wait complete. 2024-03-09T00:02:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.693+0000 7f13aa765740 20 client.27667 trim_cache size 0 max 16384 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 Processor -- start 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 -- start start 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39466c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da3946e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da3946ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13a591a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da3946e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13a591a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da3946e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36200/0 (socket says 172.21.15.22:36200) 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13a591a640 1 -- 172.21.15.22:0/2828385453 learned_addr learned my addr 172.21.15.22:0/2828385453 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13840123e0 con 0x55da3945ed40 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13a5119640 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39466c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138400ff80 con 0x55da39467550 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.719+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138400e040 con 0x55da3945dfd0 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a5119640 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da3946ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a5119640 1 -- 172.21.15.22:0/2828385453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 msgr2=0x55da3946ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a5119640 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da3946ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a5119640 1 -- 172.21.15.22:0/2828385453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 msgr2=0x55da3946e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a5119640 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da3946e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a5119640 1 -- 172.21.15.22:0/2828385453 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da39462ab0 con 0x55da3945dfd0 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a591a640 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da3946e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f13a5119640 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39466c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13900098e0 tx=0x7f1390008000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23c99a2f822d883d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f138b7fe640 1 -- 172.21.15.22:0/2828385453 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13900044b0 con 0x55da3945dfd0 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f138b7fe640 1 -- 172.21.15.22:0/2828385453 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1390054040 con 0x55da3945dfd0 2024-03-09T00:02:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.720+0000 7f138b7fe640 1 -- 172.21.15.22:0/2828385453 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390004990 con 0x55da3945dfd0 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.721+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 msgr2=0x55da39466c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.721+0000 7f13aa765740 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39466c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13900098e0 tx=0x7f1390008000 comp rx=0 tx=0).stop 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.721+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 shutdown_connections 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.721+0000 7f13aa765740 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da39467550 0x55da3946ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.722+0000 7f13aa765740 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945ed40 0x55da3946e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.722+0000 7f13aa765740 1 --2- 172.21.15.22:0/2828385453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da3945dfd0 0x55da39466c90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.722+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 >> 172.21.15.22:0/2828385453 conn(0x55da394255b0 msgr2=0x55da3947e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.722+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 shutdown_connections 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.722+0000 7f13aa765740 1 -- 172.21.15.22:0/2828385453 wait complete. 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.722+0000 7f13aa765740 1 -- start start 2024-03-09T00:02:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.723+0000 7f13aa765740 1 Processor -- start 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.723+0000 7f13aa765740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945dfd0 0x55da3946e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.723+0000 7f13aa765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da39460dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.723+0000 7f13aa765740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da39467550 0x55da39461310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13aa765740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138400e640 con 0x55da3945dfd0 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13aa765740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138400f920 con 0x55da3945ed40 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13aa765740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138400ec60 con 0x55da39467550 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13aa765740 10 client.0 _ll_register_callbacks cb 0x7f139c00b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da39460dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da39460dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50002/0 (socket says 172.21.15.22:50002) 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 -- 172.21.15.22:0/4138921479 learned_addr learned my addr 172.21.15.22:0/4138921479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a5119640 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945dfd0 0x55da3946e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 -- 172.21.15.22:0/4138921479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da39467550 msgr2=0x55da39461310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da39467550 0x55da39461310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 -- 172.21.15.22:0/4138921479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945dfd0 msgr2=0x55da3946e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945dfd0 0x55da3946e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 -- 172.21.15.22:0/4138921479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da39461950 con 0x55da3945ed40 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13a591a640 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da39460dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b7f0 tx=0x7f139c00f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a39340a9a651e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13897fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13897fa640 1 -- 172.21.15.22:0/4138921479 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c014b40 con 0x55da3945ed40 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13897fa640 1 -- 172.21.15.22:0/4138921479 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f139c014ce0 con 0x55da3945ed40 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.724+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1390054040 con 0x55da3945ed40 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13897fa640 1 -- 172.21.15.22:0/4138921479 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f139c036050 con 0x55da3945ed40 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 10 client.27602 fetch_fsmap learned FSMap version 35 2024-03-09T00:02:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55da394ce950 con 0x55da3945ed40 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13897fa640 1 -- 172.21.15.22:0/4138921479 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1390054040 con 0x55da3945ed40 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13897fa640 1 -- 172.21.15.22:0/4138921479 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f139c013070 con 0x55da3945ed40 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 10 client.27602 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 10 client.27602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 20 client.27602 populate_metadata read hostname 'smithi022' 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da394cee70 0x55da394d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 4 client.27602 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da393c9e80 con 0x55da394cee70 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.725+0000 7f13a5119640 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da394cee70 0x55da394d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.726+0000 7f13a5119640 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da394cee70 0x55da394d1290 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d32574c933ab765 server_cookie=a15ac124951434c0 in_seq=0 out_seq=0 2024-03-09T00:02:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:52.726+0000 7f13897fa640 10 client.27602 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:52.726 INFO:tasks.workunit.client.0.smithi022.stdout: 1 872328 10.49 MB/sec execute 300 sec latency 105.676 ms 2024-03-09T00:02:52.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:52 smithi022 ceph-mon[31894]: pgmap v262: 129 pgs: 129 active+clean; 82 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-09T00:02:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.088+0000 7f13897fa640 1 -- 172.21.15.22:0/4138921479 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da393c9e80 con 0x55da394cee70 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.088+0000 7f13897fa640 10 client.27602 handle_command_reply: tid=0 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.088+0000 7f13aa765740 1 client.27602 shutdown 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 msgr2=0x55da39460dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da39460dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b7f0 tx=0x7f139c00f040 comp rx=0 tx=0).stop 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 shutdown_connections 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55da394cee70 0x55da394d1290 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13897fa640 0 client.27602 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:02:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da39467550 0x55da39461310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da3945ed40 0x55da39460dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 --2- 172.21.15.22:0/4138921479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da3945dfd0 0x55da3946e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 >> 172.21.15.22:0/4138921479 conn(0x55da394255b0 msgr2=0x55da394587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 shutdown_connections 2024-03-09T00:02:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.089+0000 7f13aa765740 1 -- 172.21.15.22:0/4138921479 wait complete. 2024-03-09T00:02:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.090+0000 7f13aa765740 20 client.27602 trim_cache size 0 max 16384 2024-03-09T00:02:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:02:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:02:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:02:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:02:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.090+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f138c0a2b60 msgr2=0x7f138c0a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.090+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f138c0a2b60 0x7f138c0a5020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13980b9b30 tx=0x7f139802c140 comp rx=0 tx=0).stop 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.090+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a0105190 msgr2=0x7f13a010fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.090+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a0105190 0x7f13a010fd80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1398037040 tx=0x7f1398032f80 comp rx=0 tx=0).stop 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 shutdown_connections 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f138c0a2b60 0x7f138c0a5020 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a0105190 0x7f13a010fd80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13a0100e00 0x7f13a010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 --2- 172.21.15.22:0/1325878957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13a0100440 0x7f13a010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 >> 172.21.15.22:0/1325878957 conn(0x7f13a0077040 msgr2=0x7f13a0103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 shutdown_connections 2024-03-09T00:02:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:02:53.091+0000 7f13a7b7e640 1 -- 172.21.15.22:0/1325878957 wait complete. 2024-03-09T00:02:53.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:52 smithi062 ceph-mon[36129]: pgmap v262: 129 pgs: 129 active+clean; 82 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-09T00:02:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:52 smithi192 ceph-mon[35995]: pgmap v262: 129 pgs: 129 active+clean; 82 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-09T00:02:53.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 873521 10.49 MB/sec execute 301 sec latency 122.809 ms 2024-03-09T00:02:54.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 875024 10.48 MB/sec execute 302 sec latency 21.244 ms 2024-03-09T00:02:54.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:54 smithi022 ceph-mon[31894]: pgmap v263: 129 pgs: 129 active+clean; 82 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-03-09T00:02:55.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:54 smithi062 ceph-mon[36129]: pgmap v263: 129 pgs: 129 active+clean; 82 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-03-09T00:02:55.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:54 smithi192 ceph-mon[35995]: pgmap v263: 129 pgs: 129 active+clean; 82 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-03-09T00:02:55.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 877067 10.48 MB/sec execute 303 sec latency 31.824 ms 2024-03-09T00:02:56.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 878536 10.47 MB/sec execute 304 sec latency 21.571 ms 2024-03-09T00:02:56.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:56 smithi022 ceph-mon[31894]: pgmap v264: 129 pgs: 129 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 475 op/s 2024-03-09T00:02:57.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:56 smithi062 ceph-mon[36129]: pgmap v264: 129 pgs: 129 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 475 op/s 2024-03-09T00:02:57.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:56 smithi192 ceph-mon[35995]: pgmap v264: 129 pgs: 129 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 475 op/s 2024-03-09T00:02:57.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 880622 10.48 MB/sec execute 305 sec latency 37.656 ms 2024-03-09T00:02:58.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 881982 10.46 MB/sec execute 306 sec latency 133.329 ms 2024-03-09T00:02:58.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:02:58 smithi022 ceph-mon[31894]: pgmap v265: 129 pgs: 129 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-03-09T00:02:59.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:02:58 smithi062 ceph-mon[36129]: pgmap v265: 129 pgs: 129 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-03-09T00:02:59.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:02:58 smithi192 ceph-mon[35995]: pgmap v265: 129 pgs: 129 active+clean; 71 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-03-09T00:02:59.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 884107 10.47 MB/sec execute 307 sec latency 49.500 ms 2024-03-09T00:03:00.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 885249 10.45 MB/sec execute 308 sec latency 61.848 ms 2024-03-09T00:03:00.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:00 smithi022 ceph-mon[31894]: pgmap v266: 129 pgs: 129 active+clean; 74 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-03-09T00:03:01.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:00 smithi062 ceph-mon[36129]: pgmap v266: 129 pgs: 129 active+clean; 74 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-03-09T00:03:01.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:00 smithi192 ceph-mon[35995]: pgmap v266: 129 pgs: 129 active+clean; 74 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-03-09T00:03:01.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 886304 10.43 MB/sec execute 309 sec latency 65.820 ms 2024-03-09T00:03:02.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 887661 10.43 MB/sec execute 310 sec latency 69.627 ms 2024-03-09T00:03:02.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:02 smithi022 ceph-mon[31894]: pgmap v267: 129 pgs: 129 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-03-09T00:03:03.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:02 smithi062 ceph-mon[36129]: pgmap v267: 129 pgs: 129 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-03-09T00:03:03.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:02 smithi192 ceph-mon[35995]: pgmap v267: 129 pgs: 129 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-03-09T00:03:03.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:03:03.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 -- 172.21.15.22:0/1595811100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 msgr2=0x7f2d940ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/1595811100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d940ff840 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009900 tx=0x7f2d8802bc30 comp rx=0 tx=0).stop 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 -- 172.21.15.22:0/1595811100 shutdown_connections 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/1595811100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d940ffd80 0x7f2d9410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/1595811100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d940ff840 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/1595811100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d941073d0 0x7f2d941077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 -- 172.21.15.22:0/1595811100 >> 172.21.15.22:0/1595811100 conn(0x7f2d94076340 msgr2=0x7f2d940fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 -- 172.21.15.22:0/1595811100 shutdown_connections 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.307+0000 7f2d9aa59640 1 -- 172.21.15.22:0/1595811100 wait complete. 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 Processor -- start 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 -- start start 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d94199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d940ffd80 0x7f2d9419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d941073d0 0x7f2d9419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d94111620 con 0x7f2d940ff3c0 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941114a0 con 0x7f2d940ffd80 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.308+0000 7f2d9aa59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941117a0 con 0x7f2d941073d0 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d94199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d94199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42056/0 (socket says 172.21.15.22:42056) 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 -- 172.21.15.22:0/2289758406 learned_addr learned my addr 172.21.15.22:0/2289758406 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d937fe640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d940ffd80 0x7f2d9419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d98ff6640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d941073d0 0x7f2d9419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 -- 172.21.15.22:0/2289758406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d941073d0 msgr2=0x7f2d9419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d941073d0 0x7f2d9419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 -- 172.21.15.22:0/2289758406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d940ffd80 msgr2=0x7f2d9419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d940ffd80 0x7f2d9419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 -- 172.21.15.22:0/2289758406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9419edd0 con 0x7f2d940ff3c0 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d937fe640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d940ffd80 0x7f2d9419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d93fff640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d94199de0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400e9f0 tx=0x7f2d8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d917fa640 1 -- 172.21.15.22:0/2289758406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400ccc0 con 0x7f2d940ff3c0 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d917fa640 1 -- 172.21.15.22:0/2289758406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d84010910 con 0x7f2d940ff3c0 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d917fa640 1 -- 172.21.15.22:0/2289758406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840188a0 con 0x7f2d940ff3c0 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9419f0c0 con 0x7f2d940ff3c0 2024-03-09T00:03:03.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.309+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d941a6950 con 0x7f2d940ff3c0 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.311+0000 7f2d917fa640 1 -- 172.21.15.22:0/2289758406 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2d84010ac0 con 0x7f2d940ff3c0 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d917fa640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2d7c0a29f0 0x7f2d7c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 Processor -- start 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 -- start start 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d937fe640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2d7c0a29f0 0x7f2d7c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d917fa640 1 -- 172.21.15.22:0/2289758406 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f2d840cf540 con 0x7f2d940ff3c0 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba48ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba493720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba493c60 0x5652ba496050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ba3ede00 con 0x5652ba48b080 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ba396b10 con 0x5652ba493c60 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.312+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ba3e9ea0 con 0x5652ba48a760 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba48ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba48ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38956/0 (socket says 172.21.15.22:38956) 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 -- 172.21.15.22:0/2214535543 learned_addr learned my addr 172.21.15.22:0/2214535543 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d98ff6640 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba493c60 0x5652ba496050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d937fe640 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba493720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 -- 172.21.15.22:0/2214535543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba493c60 msgr2=0x5652ba496050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba493c60 0x5652ba496050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 -- 172.21.15.22:0/2214535543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 msgr2=0x5652ba493720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba493720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 -- 172.21.15.22:0/2214535543 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652ba1cc7d0 con 0x5652ba48a760 2024-03-09T00:03:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d937fe640 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba493720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d937fe640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2d7c0a29f0 0x7f2d7c0a4eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006010 tx=0x7f2d8802c6d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d93fff640 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba48ab40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d840a3a20 tx=0x7f2d840097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898fdf2062c6798b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d7a7fc640 1 -- 172.21.15.22:0/2214535543 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840a3f50 con 0x5652ba48a760 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d7a7fc640 1 -- 172.21.15.22:0/2214535543 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d84086070 con 0x5652ba48a760 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.313+0000 7f2d7a7fc640 1 -- 172.21.15.22:0/2214535543 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840cf330 con 0x5652ba48a760 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 -- 172.21.15.22:0/2214535543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 msgr2=0x5652ba48ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba48ab40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d840a3a20 tx=0x7f2d840097a0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 -- 172.21.15.22:0/2214535543 shutdown_connections 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba493c60 0x5652ba496050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba493720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/2214535543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba48ab40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 -- 172.21.15.22:0/2214535543 >> 172.21.15.22:0/2214535543 conn(0x5652ba484e90 msgr2=0x5652ba485270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 -- 172.21.15.22:0/2214535543 shutdown_connections 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 -- 172.21.15.22:0/2214535543 wait complete. 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.314+0000 7f2d9d63b740 1 -- start start 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 1 Processor -- start 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba4933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48b080 0x5652ba490770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba490cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ba4969f0 con 0x5652ba493c60 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ba3aa0a0 con 0x5652ba48a760 2024-03-09T00:03:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ba496b70 con 0x5652ba48b080 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d9d63b740 10 client.0 _ll_register_callbacks cb 0x5652ba3a9a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d98ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba490cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba4933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d98ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba490cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42064/0 (socket says 172.21.15.22:42064) 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48b080 0x5652ba490770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d98ff6640 1 -- 172.21.15.22:0/1429789073 learned_addr learned my addr 172.21.15.22:0/1429789073 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d93fff640 1 -- 172.21.15.22:0/1429789073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48b080 msgr2=0x5652ba490770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d93fff640 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48b080 0x5652ba490770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d93fff640 1 -- 172.21.15.22:0/1429789073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 msgr2=0x5652ba490cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d93fff640 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba490cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d93fff640 1 -- 172.21.15.22:0/1429789073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652ba1023f0 con 0x5652ba48a760 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d93fff640 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba4933e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5652ba496720 tx=0x7f2d84009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=561d2ce1d26a0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d737fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.315+0000 7f2d737fe640 1 -- 172.21.15.22:0/1429789073 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8407f060 con 0x5652ba48a760 2024-03-09T00:03:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d84086070 con 0x5652ba48a760 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d737fe640 1 -- 172.21.15.22:0/1429789073 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f2d840afb90 con 0x5652ba48a760 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d737fe640 1 -- 172.21.15.22:0/1429789073 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d840b9070 con 0x5652ba48a760 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 10 client.27614 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5652ba0c20f0 con 0x5652ba48a760 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d737fe640 1 -- 172.21.15.22:0/1429789073 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d84086070 con 0x5652ba48a760 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d737fe640 1 -- 172.21.15.22:0/1429789073 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d8409ac00 con 0x5652ba48a760 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 10 client.27614 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 10 client.27614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 20 client.27614 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4a3550 0x5652ba4a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 4 client.27614 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652ba3fb340 con 0x5652ba4a3550 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.316+0000 7f2d937fe640 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4a3550 0x5652ba4a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.317+0000 7f2d737fe640 10 client.27614 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:03.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.317+0000 7f2d937fe640 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4a3550 0x5652ba4a5940 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d90fbd094c4bf063 server_cookie=3d940bd58edab053 in_seq=0 out_seq=0 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.317+0000 7f2d737fe640 1 -- 172.21.15.22:0/1429789073 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652ba3fb340 con 0x5652ba4a3550 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.317+0000 7f2d737fe640 10 client.27614 handle_command_reply: tid=0 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.317+0000 7f2d9d63b740 1 client.27614 shutdown 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 msgr2=0x5652ba4933e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba4933e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5652ba496720 tx=0x7f2d84009700 comp rx=0 tx=0).stop 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 shutdown_connections 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4a3550 0x5652ba4a5940 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba490cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48b080 0x5652ba490770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1429789073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba4933e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d737fe640 0 client.27614 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 >> 172.21.15.22:0/1429789073 conn(0x5652ba484e90 msgr2=0x5652ba486be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 shutdown_connections 2024-03-09T00:03:03.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1429789073 wait complete. 2024-03-09T00:03:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.318+0000 7f2d9d63b740 20 client.27614 trim_cache size 0 max 16384 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.333+0000 7f2d9d63b740 1 Processor -- start 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.333+0000 7f2d9d63b740 1 -- start start 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba48bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba492ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba493c60 0x5652ba4933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941b2e40 con 0x5652ba48b080 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941b09e0 con 0x5652ba48a760 2024-03-09T00:03:03.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941aeaa0 con 0x5652ba493c60 2024-03-09T00:03:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba492ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba492ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42070/0 (socket says 172.21.15.22:42070) 2024-03-09T00:03:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 -- 172.21.15.22:0/3525032030 learned_addr learned my addr 172.21.15.22:0/3525032030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d98ff6640 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba493c60 0x5652ba4933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d93fff640 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba48bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 -- 172.21.15.22:0/3525032030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba493c60 msgr2=0x5652ba4933e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba493c60 0x5652ba4933e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 -- 172.21.15.22:0/3525032030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 msgr2=0x5652ba48bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba48bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 -- 172.21.15.22:0/3525032030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652ba48aeb0 con 0x5652ba48b080 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d937fe640 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba492ea0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2d88004e00 tx=0x7f2d88041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9690fc45baf2b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d79ffb640 1 -- 172.21.15.22:0/3525032030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8802cb00 con 0x5652ba48b080 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d79ffb640 1 -- 172.21.15.22:0/3525032030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d8805b070 con 0x5652ba48b080 2024-03-09T00:03:03.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.334+0000 7f2d79ffb640 1 -- 172.21.15.22:0/3525032030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004f20 con 0x5652ba48b080 2024-03-09T00:03:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 -- 172.21.15.22:0/3525032030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 msgr2=0x5652ba492ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba492ea0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2d88004e00 tx=0x7f2d88041000 comp rx=0 tx=0).stop 2024-03-09T00:03:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 -- 172.21.15.22:0/3525032030 shutdown_connections 2024-03-09T00:03:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba493c60 0x5652ba4933e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba48b080 0x5652ba492ea0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/3525032030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48a760 0x5652ba48bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 -- 172.21.15.22:0/3525032030 >> 172.21.15.22:0/3525032030 conn(0x5652ba484e90 msgr2=0x5652ba48b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 -- 172.21.15.22:0/3525032030 shutdown_connections 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.335+0000 7f2d9d63b740 1 -- 172.21.15.22:0/3525032030 wait complete. 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 -- start start 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 Processor -- start 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba497ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48b080 0x5652ba498000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba498560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941af0a0 con 0x5652ba493c60 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941b0380 con 0x5652ba48b080 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941af6c0 con 0x5652ba48a760 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d9d63b740 10 client.0 _ll_register_callbacks cb 0x7f2d8c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d98ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba498560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba497ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d98ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba498560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42086/0 (socket says 172.21.15.22:42086) 2024-03-09T00:03:03.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d98ff6640 1 -- 172.21.15.22:0/1570067084 learned_addr learned my addr 172.21.15.22:0/1570067084 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d937fe640 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48b080 0x5652ba498000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d98ff6640 1 -- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 msgr2=0x5652ba497ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d98ff6640 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba497ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d98ff6640 1 -- 172.21.15.22:0/1570067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48b080 msgr2=0x5652ba498000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.336+0000 7f2d98ff6640 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48b080 0x5652ba498000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d98ff6640 1 -- 172.21.15.22:0/1570067084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652ba4f9950 con 0x5652ba493c60 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d93fff640 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba497ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d98ff6640 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba498560 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c012cb0 tx=0x7f2d8c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7fd80a5485a59f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d737fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d737fe640 1 -- 172.21.15.22:0/1570067084 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c023060 con 0x5652ba493c60 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d737fe640 1 -- 172.21.15.22:0/1570067084 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f2d8c003b70 con 0x5652ba493c60 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d8805b070 con 0x5652ba493c60 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d737fe640 1 -- 172.21.15.22:0/1570067084 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d8c02a050 con 0x5652ba493c60 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d9d63b740 10 client.18246 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5652ba4facc0 con 0x5652ba493c60 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d737fe640 1 -- 172.21.15.22:0/1570067084 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d8805b070 con 0x5652ba493c60 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.337+0000 7f2d737fe640 1 -- 172.21.15.22:0/1570067084 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d8c010430 con 0x5652ba493c60 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d9d63b740 10 client.18246 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d9d63b740 10 client.18246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d9d63b740 20 client.18246 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:03.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4fb420 0x5652ba4fd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d9d63b740 4 client.18246 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:03:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652ba3f5690 con 0x5652ba4fb420 2024-03-09T00:03:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d93fff640 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4fb420 0x5652ba4fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d737fe640 10 client.18246 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:03.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.338+0000 7f2d93fff640 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4fb420 0x5652ba4fd840 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f53a355dae8d05 server_cookie=afaae2ebb986dbf4 in_seq=0 out_seq=0 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d737fe640 1 -- 172.21.15.22:0/1570067084 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5652ba3f5690 con 0x5652ba4fb420 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d737fe640 10 client.18246 handle_command_reply: tid=0 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 client.18246 shutdown 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 msgr2=0x5652ba498560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba498560 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c012cb0 tx=0x7f2d8c00f790 comp rx=0 tx=0).stop 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 shutdown_connections 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652ba4fb420 0x5652ba4fd840 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652ba493c60 0x5652ba498560 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652ba48b080 0x5652ba498000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d737fe640 0 client.18246 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 --2- 172.21.15.22:0/1570067084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652ba48a760 0x5652ba497ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.341+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 >> 172.21.15.22:0/1570067084 conn(0x5652ba484e90 msgr2=0x5652ba48ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 shutdown_connections 2024-03-09T00:03:03.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9d63b740 1 -- 172.21.15.22:0/1570067084 wait complete. 2024-03-09T00:03:03.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9d63b740 20 client.18246 trim_cache size 0 max 16384 2024-03-09T00:03:03.347 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:03:03.347 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:03:03.347 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:03:03.348 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2d7c0a29f0 msgr2=0x7f2d7c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2d7c0a29f0 0x7f2d7c0a4eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006010 tx=0x7f2d8802c6d0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 msgr2=0x7f2d94199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d94199de0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400e9f0 tx=0x7f2d8400edf0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 shutdown_connections 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2d7c0a29f0 0x7f2d7c0a4eb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d941073d0 0x7f2d9419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d940ffd80 0x7f2d9419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 --2- 172.21.15.22:0/2289758406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d940ff3c0 0x7f2d94199de0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 >> 172.21.15.22:0/2289758406 conn(0x7f2d94076340 msgr2=0x7f2d940fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 shutdown_connections 2024-03-09T00:03:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.342+0000 7f2d9aa59640 1 -- 172.21.15.22:0/2289758406 wait complete. 2024-03-09T00:03:03.350 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:03:03.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:03:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 -- 172.21.15.22:0/2763258631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb406a260 msgr2=0x7f8bb410e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2763258631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb406a260 0x7f8bb410e0d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009ca0 tx=0x7f8bac02c0f0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 -- 172.21.15.22:0/2763258631 shutdown_connections 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2763258631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb406a260 0x7f8bb410e0d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2763258631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb40698c0 0x7f8bb4069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2763258631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb4107420 0x7f8bb4107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 -- 172.21.15.22:0/2763258631 >> 172.21.15.22:0/2763258631 conn(0x7f8bb40771d0 msgr2=0x7f8bb40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 -- 172.21.15.22:0/2763258631 shutdown_connections 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 -- 172.21.15.22:0/2763258631 wait complete. 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.461+0000 7f8bbc553640 1 Processor -- start 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbc553640 1 -- start start 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbc553640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb40698c0 0x7f8bb4199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbc553640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb406a260 0x7f8bb419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbc553640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb4107420 0x7f8bb419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbc553640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41116d0 con 0x7f8bb40698c0 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbc553640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4111550 con 0x7f8bb406a260 2024-03-09T00:03:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbc553640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4111850 con 0x7f8bb4107420 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb40698c0 0x7f8bb4199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb40698c0 0x7f8bb4199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42106/0 (socket says 172.21.15.22:42106) 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 -- 172.21.15.22:0/2206900729 learned_addr learned my addr 172.21.15.22:0/2206900729 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bb9aee640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb406a260 0x7f8bb419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bbaaf0640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb4107420 0x7f8bb419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 -- 172.21.15.22:0/2206900729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb4107420 msgr2=0x7f8bb419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb4107420 0x7f8bb419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 -- 172.21.15.22:0/2206900729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb406a260 msgr2=0x7f8bb419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb406a260 0x7f8bb419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 -- 172.21.15.22:0/2206900729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb419edd0 con 0x7f8bb40698c0 2024-03-09T00:03:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.462+0000 7f8bba2ef640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb40698c0 0x7f8bb4199e30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000e9f0 tx=0x7f8bb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.463+0000 7f8bab7fe640 1 -- 172.21.15.22:0/2206900729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000cd80 con 0x7f8bb40698c0 2024-03-09T00:03:03.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.463+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb419f0c0 con 0x7f8bb40698c0 2024-03-09T00:03:03.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.463+0000 7f8bab7fe640 1 -- 172.21.15.22:0/2206900729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bb0005500 con 0x7f8bb40698c0 2024-03-09T00:03:03.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.463+0000 7f8bab7fe640 1 -- 172.21.15.22:0/2206900729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0010690 con 0x7f8bb40698c0 2024-03-09T00:03:03.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.463+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb41a6950 con 0x7f8bb40698c0 2024-03-09T00:03:03.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.464+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb4102a00 con 0x7f8bb40698c0 2024-03-09T00:03:03.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.465+0000 7f8bab7fe640 1 -- 172.21.15.22:0/2206900729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8bb00046c0 con 0x7f8bb40698c0 2024-03-09T00:03:03.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.465+0000 7f8bab7fe640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8ba00a2970 0x7f8ba00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.466+0000 7f8bb9aee640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8ba00a2970 0x7f8ba00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.466+0000 7f8bab7fe640 1 -- 172.21.15.22:0/2206900729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f8bb00cf5d0 con 0x7f8bb40698c0 2024-03-09T00:03:03.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.466+0000 7f8bb9aee640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8ba00a2970 0x7f8ba00a4e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40fe9c0 tx=0x7f8ba4005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.470+0000 7f8bab7fe640 1 -- 172.21.15.22:0/2206900729 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8bb009bf40 con 0x7f8bb40698c0 2024-03-09T00:03:03.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.722+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bb41a6be0 con 0x7f8bb40698c0 2024-03-09T00:03:03.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.723+0000 7f8bab7fe640 1 -- 172.21.15.22:0/2206900729 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f8bb009b840 con 0x7f8bb40698c0 2024-03-09T00:03:03.718 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:03:03.718 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:03:03.719 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:03:03.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8ba00a2970 msgr2=0x7f8ba00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8ba00a2970 0x7f8ba00a4e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40fe9c0 tx=0x7f8ba4005f90 comp rx=0 tx=0).stop 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb40698c0 msgr2=0x7f8bb4199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb40698c0 0x7f8bb4199e30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000e9f0 tx=0x7f8bb000edf0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 shutdown_connections 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8ba00a2970 0x7f8ba00a4e30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bb4107420 0x7f8bb419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb406a260 0x7f8bb419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 --2- 172.21.15.22:0/2206900729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8bb40698c0 0x7f8bb4199e30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 >> 172.21.15.22:0/2206900729 conn(0x7f8bb40771d0 msgr2=0x7f8bb410c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 shutdown_connections 2024-03-09T00:03:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.726+0000 7f8bbc553640 1 -- 172.21.15.22:0/2206900729 wait complete. 2024-03-09T00:03:03.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 888750 10.41 MB/sec execute 311 sec latency 141.338 ms 2024-03-09T00:03:03.733 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:03:03.733 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:03:03.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.866+0000 7ff686e6c640 1 -- 172.21.15.22:0/484108362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6800ffd80 msgr2=0x7ff6801001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.866+0000 7ff686e6c640 1 --2- 172.21.15.22:0/484108362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6800ffd80 0x7ff6801001e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff678009ca0 tx=0x7ff67802c120 comp rx=0 tx=0).stop 2024-03-09T00:03:03.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.866+0000 7ff686e6c640 1 -- 172.21.15.22:0/484108362 shutdown_connections 2024-03-09T00:03:03.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.867+0000 7ff686e6c640 1 --2- 172.21.15.22:0/484108362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6800ffd80 0x7ff6801001e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.867+0000 7ff686e6c640 1 --2- 172.21.15.22:0/484108362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6800ff3c0 0x7ff6800ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.867+0000 7ff686e6c640 1 --2- 172.21.15.22:0/484108362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6801058f0 0x7ff680105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.867+0000 7ff686e6c640 1 -- 172.21.15.22:0/484108362 >> 172.21.15.22:0/484108362 conn(0x7ff680076380 msgr2=0x7ff6800fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.867+0000 7ff686e6c640 1 -- 172.21.15.22:0/484108362 shutdown_connections 2024-03-09T00:03:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.868+0000 7ff686e6c640 1 -- 172.21.15.22:0/484108362 wait complete. 2024-03-09T00:03:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.868+0000 7ff686e6c640 1 Processor -- start 2024-03-09T00:03:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.868+0000 7ff686e6c640 1 -- start start 2024-03-09T00:03:03.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff686e6c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6800ff3c0 0x7ff68010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff686e6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6800ffd80 0x7ff6801100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff686e6c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6801058f0 0x7ff680110620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff686e6c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68010d490 con 0x7ff6801058f0 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff686e6c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68010d310 con 0x7ff6800ffd80 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff686e6c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68010d610 con 0x7ff6800ff3c0 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff685409640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6801058f0 0x7ff680110620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff685409640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6801058f0 0x7ff680110620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42128/0 (socket says 172.21.15.22:42128) 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6800ffd80 0x7ff6801100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff685409640 1 -- 172.21.15.22:0/50359771 learned_addr learned my addr 172.21.15.22:0/50359771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff684c08640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6800ff3c0 0x7ff68010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff67ffff640 1 -- 172.21.15.22:0/50359771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6800ff3c0 msgr2=0x7ff68010fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff67ffff640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6800ff3c0 0x7ff68010fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff67ffff640 1 -- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6801058f0 msgr2=0x7ff680110620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff67ffff640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6801058f0 0x7ff680110620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.869+0000 7ff67ffff640 1 -- 172.21.15.22:0/50359771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6801aa470 con 0x7ff6800ffd80 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.870+0000 7ff685409640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6801058f0 0x7ff680110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.870+0000 7ff67ffff640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6800ffd80 0x7ff6801100e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff67000b3e0 tx=0x7ff67000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.870+0000 7ff67dffb640 1 -- 172.21.15.22:0/50359771 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670015020 con 0x7ff6800ffd80 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.870+0000 7ff67dffb640 1 -- 172.21.15.22:0/50359771 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff670011d10 con 0x7ff6800ffd80 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.870+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6801aa760 con 0x7ff6800ffd80 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.870+0000 7ff67dffb640 1 -- 172.21.15.22:0/50359771 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67000f2e0 con 0x7ff6800ffd80 2024-03-09T00:03:03.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.871+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6801aad00 con 0x7ff6800ffd80 2024-03-09T00:03:03.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.875+0000 7ff67dffb640 1 -- 172.21.15.22:0/50359771 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff670018020 con 0x7ff6800ffd80 2024-03-09T00:03:03.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.875+0000 7ff67dffb640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff6680a2a70 0x7ff6680a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.876+0000 7ff67dffb640 1 -- 172.21.15.22:0/50359771 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ff6700ce930 con 0x7ff6800ffd80 2024-03-09T00:03:03.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.876+0000 7ff684c08640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff6680a2a70 0x7ff6680a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.877+0000 7ff684c08640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff6680a2a70 0x7ff6680a4f30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff674009b60 tx=0x7ff674005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 Processor -- start 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 -- start start 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec28025580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec2802e6a0 0x55ec28030a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec27fec360 con 0x55ec2802e6a0 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec27f88ec0 con 0x55ec280251a0 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.883+0000 7ff689a4e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec27f31d30 con 0x55ec28025ac0 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff685409640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec2802e6a0 0x55ec28030a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff685409640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec2802e6a0 0x55ec28030a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42142/0 (socket says 172.21.15.22:42142) 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff685409640 1 -- 172.21.15.22:0/1748363563 learned_addr learned my addr 172.21.15.22:0/1748363563 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff684c08640 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec28025580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff67ffff640 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff684c08640 1 -- 172.21.15.22:0/1748363563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 msgr2=0x55ec2802e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff684c08640 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff684c08640 1 -- 172.21.15.22:0/1748363563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec2802e6a0 msgr2=0x55ec28030a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff684c08640 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec2802e6a0 0x55ec28030a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.884+0000 7ff684c08640 1 -- 172.21.15.22:0/1748363563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec27f30bc0 con 0x55ec280251a0 2024-03-09T00:03:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.885+0000 7ff684c08640 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec28025580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff674009a20 tx=0x7ff674014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d44bd1ac2a3eae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.886+0000 7ff666ffd640 1 -- 172.21.15.22:0/1748363563 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674012e60 con 0x55ec280251a0 2024-03-09T00:03:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.886+0000 7ff666ffd640 1 -- 172.21.15.22:0/1748363563 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff674004220 con 0x55ec280251a0 2024-03-09T00:03:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.886+0000 7ff666ffd640 1 -- 172.21.15.22:0/1748363563 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67401f7c0 con 0x55ec280251a0 2024-03-09T00:03:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.888+0000 7ff689a4e740 1 -- 172.21.15.22:0/1748363563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 msgr2=0x55ec28025580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.888+0000 7ff689a4e740 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec28025580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff674009a20 tx=0x7ff674014000 comp rx=0 tx=0).stop 2024-03-09T00:03:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.890+0000 7ff689a4e740 1 -- 172.21.15.22:0/1748363563 shutdown_connections 2024-03-09T00:03:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.891+0000 7ff689a4e740 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec2802e6a0 0x55ec28030a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.891+0000 7ff689a4e740 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.891+0000 7ff689a4e740 1 --2- 172.21.15.22:0/1748363563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec28025580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.891+0000 7ff689a4e740 1 -- 172.21.15.22:0/1748363563 >> 172.21.15.22:0/1748363563 conn(0x55ec2801f8d0 msgr2=0x55ec2801fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.892+0000 7ff689a4e740 1 -- 172.21.15.22:0/1748363563 shutdown_connections 2024-03-09T00:03:03.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.893+0000 7ff689a4e740 1 -- 172.21.15.22:0/1748363563 wait complete. 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.893+0000 7ff689a4e740 1 -- start start 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.893+0000 7ff689a4e740 1 Processor -- start 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec2802b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec2802bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec2802e6a0 0x55ec2802c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff689a4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec27f274a0 con 0x55ec28025ac0 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff689a4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec27f84f60 con 0x55ec280251a0 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff689a4e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec28031490 con 0x55ec2802e6a0 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff689a4e740 10 client.0 _ll_register_callbacks cb 0x55ec27f82d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff685409640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec2802e6a0 0x55ec2802c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff685409640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec2802e6a0 0x55ec2802c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39040/0 (socket says 172.21.15.22:39040) 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff685409640 1 -- 172.21.15.22:0/3211377603 learned_addr learned my addr 172.21.15.22:0/3211377603 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.894+0000 7ff67ffff640 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec2802bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.895+0000 7ff685409640 1 -- 172.21.15.22:0/3211377603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 msgr2=0x55ec2802b620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:03:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.895+0000 7ff685409640 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec2802b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.895+0000 7ff685409640 1 -- 172.21.15.22:0/3211377603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 msgr2=0x55ec2802bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.895+0000 7ff685409640 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec2802bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.895+0000 7ff685409640 1 -- 172.21.15.22:0/3211377603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec27c9d3f0 con 0x55ec2802e6a0 2024-03-09T00:03:03.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.897+0000 7ff685409640 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec2802e6a0 0x55ec2802c0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff678037040 tx=0x7ff6780323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a236f959059561 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.898+0000 7ff65ffff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.898+0000 7ff65ffff640 1 -- 172.21.15.22:0/3211377603 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678044070 con 0x55ec2802e6a0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.898+0000 7ff65ffff640 1 -- 172.21.15.22:0/3211377603 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ff678032800 con 0x55ec2802e6a0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.898+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff674004220 con 0x55ec2802e6a0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.899+0000 7ff65ffff640 1 -- 172.21.15.22:0/3211377603 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff678050050 con 0x55ec2802e6a0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.899+0000 7ff689a4e740 10 client.27706 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.899+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ec27d677d0 con 0x55ec2802e6a0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff65ffff640 1 -- 172.21.15.22:0/3211377603 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff674004220 con 0x55ec2802e6a0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff65ffff640 1 -- 172.21.15.22:0/3211377603 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff678004040 con 0x55ec2802e6a0 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff689a4e740 10 client.27706 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff689a4e740 10 client.27706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff689a4e740 20 client.27706 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff689a4e740 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec2804f750 0x55ec28051b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff689a4e740 4 client.27706 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:03.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.900+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec27f96400 con 0x55ec2804f750 2024-03-09T00:03:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.903+0000 7ff684c08640 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec2804f750 0x55ec28051b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.903+0000 7ff65ffff640 10 client.27706 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:03.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.904+0000 7ff684c08640 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec2804f750 0x55ec28051b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f2fc98a97f2d53c server_cookie=a613584be7c92183 in_seq=0 out_seq=0 2024-03-09T00:03:03.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.906+0000 7ff65ffff640 1 -- 172.21.15.22:0/3211377603 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec27f96400 con 0x55ec2804f750 2024-03-09T00:03:03.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.906+0000 7ff65ffff640 10 client.27706 handle_command_reply: tid=0 2024-03-09T00:03:03.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.906+0000 7ff689a4e740 1 client.27706 shutdown 2024-03-09T00:03:03.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.907+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec2802e6a0 msgr2=0x55ec2802c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.907+0000 7ff689a4e740 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec2802e6a0 0x55ec2802c0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff678037040 tx=0x7ff6780323d0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 shutdown_connections 2024-03-09T00:03:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec2804f750 0x55ec28051b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec2802e6a0 0x55ec2802c0a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec2802bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 --2- 172.21.15.22:0/3211377603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec280251a0 0x55ec2802b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 >> 172.21.15.22:0/3211377603 conn(0x55ec2801f8d0 msgr2=0x55ec28021620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff65ffff640 0 client.27706 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 shutdown_connections 2024-03-09T00:03:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 1 -- 172.21.15.22:0/3211377603 wait complete. 2024-03-09T00:03:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.908+0000 7ff689a4e740 20 client.27706 trim_cache size 0 max 16384 2024-03-09T00:03:03.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.933+0000 7ff689a4e740 1 Processor -- start 2024-03-09T00:03:03.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.933+0000 7ff689a4e740 1 -- start start 2024-03-09T00:03:03.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec280251a0 0x55ec2802a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec280356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff684c08640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec280251a0 0x55ec2802a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff689a4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801aead0 con 0x55ec280251a0 2024-03-09T00:03:03.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff689a4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801ac670 con 0x55ec2802e6a0 2024-03-09T00:03:03.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff689a4e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff680068530 con 0x55ec28025ac0 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff685409640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec280356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff685409640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec280356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50506/0 (socket says 172.21.15.22:50506) 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.934+0000 7ff685409640 1 -- 172.21.15.22:0/672909968 learned_addr learned my addr 172.21.15.22:0/672909968 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.935+0000 7ff67ffff640 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.935+0000 7ff685409640 1 -- 172.21.15.22:0/672909968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 msgr2=0x55ec2802bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.935+0000 7ff685409640 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.935+0000 7ff685409640 1 -- 172.21.15.22:0/672909968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec280251a0 msgr2=0x55ec2802a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.935+0000 7ff685409640 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec280251a0 0x55ec2802a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.935+0000 7ff685409640 1 -- 172.21.15.22:0/672909968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2801f700 con 0x55ec2802e6a0 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.935+0000 7ff685409640 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec280356d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff67802e010 tx=0x7ff678032360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96682573c547955b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.936+0000 7ff6667fc640 1 -- 172.21.15.22:0/672909968 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678059070 con 0x55ec2802e6a0 2024-03-09T00:03:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.936+0000 7ff6667fc640 1 -- 172.21.15.22:0/672909968 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff67804d070 con 0x55ec2802e6a0 2024-03-09T00:03:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff6667fc640 1 -- 172.21.15.22:0/672909968 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678044040 con 0x55ec2802e6a0 2024-03-09T00:03:03.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 -- 172.21.15.22:0/672909968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 msgr2=0x55ec280356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec280356d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff67802e010 tx=0x7ff678032360 comp rx=0 tx=0).stop 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 -- 172.21.15.22:0/672909968 shutdown_connections 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec280356d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec28025ac0 0x55ec2802bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 --2- 172.21.15.22:0/672909968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec280251a0 0x55ec2802a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 -- 172.21.15.22:0/672909968 >> 172.21.15.22:0/672909968 conn(0x55ec2801f8d0 msgr2=0x55ec28020bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 -- 172.21.15.22:0/672909968 shutdown_connections 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 -- 172.21.15.22:0/672909968 wait complete. 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.937+0000 7ff689a4e740 1 -- start start 2024-03-09T00:03:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 1 Processor -- start 2024-03-09T00:03:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec280251a0 0x55ec28034920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec28034e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec2809fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801a2140 con 0x55ec28025ac0 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801ac010 con 0x55ec2802e6a0 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801ab350 con 0x55ec280251a0 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff689a4e740 10 client.0 _ll_register_callbacks cb 0x7ff674009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec28034e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec28034e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42184/0 (socket says 172.21.15.22:42184) 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff67ffff640 1 -- 172.21.15.22:0/4044090612 learned_addr learned my addr 172.21.15.22:0/4044090612 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff684c08640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec280251a0 0x55ec28034920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.938+0000 7ff685409640 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec2809fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff67ffff640 1 -- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec280251a0 msgr2=0x55ec28034920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff67ffff640 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec280251a0 0x55ec28034920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff67ffff640 1 -- 172.21.15.22:0/4044090612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 msgr2=0x55ec2809fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff67ffff640 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec2809fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff67ffff640 1 -- 172.21.15.22:0/4044090612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec28030a90 con 0x55ec28025ac0 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff684c08640 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec280251a0 0x55ec28034920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff67ffff640 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec28034e60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff67009abe0 tx=0x7ff67009b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b669aa1eccafd2e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff65ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:03.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff65ffff640 1 -- 172.21.15.22:0/4044090612 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670011830 con 0x55ec28025ac0 2024-03-09T00:03:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff65ffff640 1 -- 172.21.15.22:0/4044090612 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ff6700972c0 con 0x55ec28025ac0 2024-03-09T00:03:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.939+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff67804d070 con 0x55ec28025ac0 2024-03-09T00:03:03.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.940+0000 7ff65ffff640 1 -- 172.21.15.22:0/4044090612 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6700cbeb0 con 0x55ec28025ac0 2024-03-09T00:03:03.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.940+0000 7ff689a4e740 10 client.18270 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:03.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.940+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ec280a73b0 con 0x55ec28025ac0 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff65ffff640 1 -- 172.21.15.22:0/4044090612 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff67804d070 con 0x55ec28025ac0 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff65ffff640 1 -- 172.21.15.22:0/4044090612 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff67009a1c0 con 0x55ec28025ac0 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff689a4e740 10 client.18270 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff689a4e740 10 client.18270 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff689a4e740 20 client.18270 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff689a4e740 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec280a7ae0 0x55ec280a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff689a4e740 4 client.18270 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.941+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec27f90750 con 0x55ec280a7ae0 2024-03-09T00:03:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.943+0000 7ff684c08640 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec280a7ae0 0x55ec280a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.944+0000 7ff684c08640 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec280a7ae0 0x55ec280a9f00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=651ea803ead9122 server_cookie=a4686df3767618b8 in_seq=0 out_seq=0 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.944+0000 7ff65ffff640 10 client.18270 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.944+0000 7ff65ffff640 1 -- 172.21.15.22:0/4044090612 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec27f90750 con 0x55ec280a7ae0 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.944+0000 7ff65ffff640 10 client.18270 handle_command_reply: tid=0 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.944+0000 7ff689a4e740 1 client.18270 shutdown 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 msgr2=0x55ec28034e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec28034e60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff67009abe0 tx=0x7ff67009b490 comp rx=0 tx=0).stop 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 shutdown_connections 2024-03-09T00:03:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55ec280a7ae0 0x55ec280a9f00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec2802e6a0 0x55ec2809fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec28025ac0 0x55ec28034e60 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 --2- 172.21.15.22:0/4044090612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec280251a0 0x55ec28034920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 >> 172.21.15.22:0/4044090612 conn(0x55ec2801f8d0 msgr2=0x55ec2801e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff65ffff640 0 client.18270 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 shutdown_connections 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 1 -- 172.21.15.22:0/4044090612 wait complete. 2024-03-09T00:03:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.945+0000 7ff689a4e740 20 client.18270 trim_cache size 0 max 16384 2024-03-09T00:03:03.943 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff6680a2a70 msgr2=0x7ff6680a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff6680a2a70 0x7ff6680a4f30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff674009b60 tx=0x7ff674005f90 comp rx=0 tx=0).stop 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6800ffd80 msgr2=0x7ff6801100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6800ffd80 0x7ff6801100e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff67000b3e0 tx=0x7ff67000b8a0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 shutdown_connections 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff6680a2a70 0x7ff6680a4f30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6801058f0 0x7ff680110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6800ffd80 0x7ff6801100e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 --2- 172.21.15.22:0/50359771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6800ff3c0 0x7ff68010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 >> 172.21.15.22:0/50359771 conn(0x7ff680076380 msgr2=0x7ff68010e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:03.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 shutdown_connections 2024-03-09T00:03:03.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:03.946+0000 7ff686e6c640 1 -- 172.21.15.22:0/50359771 wait complete. 2024-03-09T00:03:03.958 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:03:03.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:03:03.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:03 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2206900729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:03:04.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.091+0000 7f1d65eea640 1 -- 172.21.15.22:0/1995551030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 msgr2=0x7f1d601007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.091+0000 7f1d65eea640 1 --2- 172.21.15.22:0/1995551030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d601007c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009900 tx=0x7f1d5002bc30 comp rx=0 tx=0).stop 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.091+0000 7f1d65eea640 1 -- 172.21.15.22:0/1995551030 shutdown_connections 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.091+0000 7f1d65eea640 1 --2- 172.21.15.22:0/1995551030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d601054b0 0x7f1d6010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.091+0000 7f1d65eea640 1 --2- 172.21.15.22:0/1995551030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d60100d90 0x7f1d60104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.091+0000 7f1d65eea640 1 --2- 172.21.15.22:0/1995551030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d601007c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.091+0000 7f1d65eea640 1 -- 172.21.15.22:0/1995551030 >> 172.21.15.22:0/1995551030 conn(0x7f1d600773a0 msgr2=0x7f1d600777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.092+0000 7f1d65eea640 1 -- 172.21.15.22:0/1995551030 shutdown_connections 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.092+0000 7f1d65eea640 1 -- 172.21.15.22:0/1995551030 wait complete. 2024-03-09T00:03:04.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.092+0000 7f1d65eea640 1 Processor -- start 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.092+0000 7f1d65eea640 1 -- start start 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.092+0000 7f1d65eea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d6019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.092+0000 7f1d65eea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d601054b0 0x7f1d6019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.092+0000 7f1d65eea640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d60196840 0x7f1d60196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d65eea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60110560 con 0x7f1d601003e0 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d65eea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601103e0 con 0x7f1d601054b0 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d65eea640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601106e0 con 0x7f1d60196840 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d6019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d6019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42202/0 (socket says 172.21.15.22:42202) 2024-03-09T00:03:04.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3105580236 learned_addr learned my addr 172.21.15.22:0/3105580236 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5effd640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d601054b0 0x7f1d6019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d60196840 0x7f1d60196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3105580236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d60196840 msgr2=0x7f1d60196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d60196840 0x7f1d60196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3105580236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d601054b0 msgr2=0x7f1d6019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d601054b0 0x7f1d6019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3105580236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d601973d0 con 0x7f1d601003e0 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d6019c6d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006010 tx=0x7f1d5002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5cff9640 1 -- 172.21.15.22:0/3105580236 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50041070 con 0x7f1d601003e0 2024-03-09T00:03:04.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d60197660 con 0x7f1d601003e0 2024-03-09T00:03:04.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5cff9640 1 -- 172.21.15.22:0/3105580236 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d50004450 con 0x7f1d601003e0 2024-03-09T00:03:04.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d5cff9640 1 -- 172.21.15.22:0/3105580236 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50003e00 con 0x7f1d601003e0 2024-03-09T00:03:04.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.093+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601a3840 con 0x7f1d601003e0 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d5cff9640 1 -- 172.21.15.22:0/3105580236 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1d50044070 con 0x7f1d601003e0 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 Processor -- start 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 -- start start 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25726e50 0x555a25727230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25727770 0x555a25732370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25734ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a256ee360 con 0x555a25727770 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2568aec0 con 0x555a25726e50 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a25633d30 con 0x555a257328b0 2024-03-09T00:03:04.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.096+0000 7f1d5cff9640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1d480a2970 0x7f1d480a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25727770 0x555a25732370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25734ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1d480a2970 0x7f1d480a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25734ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39110/0 (socket says 172.21.15.22:39110) 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 -- 172.21.15.22:0/932376881 learned_addr learned my addr 172.21.15.22:0/932376881 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25726e50 0x555a25727230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5cff9640 1 -- 172.21.15.22:0/3105580236 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1d500f0ba0 con 0x7f1d601003e0 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 -- 172.21.15.22:0/932376881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25726e50 msgr2=0x555a25727230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25726e50 0x555a25727230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 -- 172.21.15.22:0/932376881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25727770 msgr2=0x555a25732370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25727770 0x555a25732370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 -- 172.21.15.22:0/932376881 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a25632bc0 con 0x555a257328b0 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25727770 0x555a25732370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1d480a2970 0x7f1d480a4e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d58005fd0 tx=0x7f1d58009e70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d5effd640 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25734ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009c20 tx=0x7f1d54011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f498e450f10c047 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d45ffb640 1 -- 172.21.15.22:0/932376881 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004230 con 0x555a257328b0 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d45ffb640 1 -- 172.21.15.22:0/932376881 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d540043d0 con 0x555a257328b0 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.097+0000 7f1d45ffb640 1 -- 172.21.15.22:0/932376881 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004e00 con 0x555a257328b0 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 -- 172.21.15.22:0/932376881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 msgr2=0x555a25734ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25734ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009c20 tx=0x7f1d54011810 comp rx=0 tx=0).stop 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 -- 172.21.15.22:0/932376881 shutdown_connections 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25734ca0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25727770 0x555a25732370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/932376881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25726e50 0x555a25727230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 -- 172.21.15.22:0/932376881 >> 172.21.15.22:0/932376881 conn(0x555a257218d0 msgr2=0x555a25721cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 -- 172.21.15.22:0/932376881 shutdown_connections 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 -- 172.21.15.22:0/932376881 wait complete. 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.098+0000 7f1d68ba5740 1 -- start start 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 1 Processor -- start 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a25726e50 0x555a2572f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a257328b0 0x555a257302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a256294a0 con 0x555a257328b0 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a25686f60 con 0x555a25727770 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a257356f0 con 0x555a25726e50 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d68ba5740 10 client.0 _ll_register_callbacks cb 0x555a25684d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a257328b0 0x555a257302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a25726e50 0x555a2572f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a257328b0 0x555a257302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42228/0 (socket says 172.21.15.22:42228) 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 -- 172.21.15.22:0/137427177 learned_addr learned my addr 172.21.15.22:0/137427177 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a25726e50 0x555a2572f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39116/0 (socket says 172.21.15.22:39116) 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5effd640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50570/0 (socket says 172.21.15.22:50570) 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 -- 172.21.15.22:0/137427177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a25726e50 msgr2=0x555a2572f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a25726e50 0x555a2572f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 -- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 msgr2=0x555a2572fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5ffff640 1 -- 172.21.15.22:0/137427177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a2539f3f0 con 0x555a257328b0 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a25726e50 0x555a2572f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.099+0000 7f1d5effd640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:04.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a257328b0 0x555a257302c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1d580115c0 tx=0x7f1d58009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9250a4898f844ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d3effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d3effd640 1 -- 172.21.15.22:0/137427177 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004530 con 0x555a257328b0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d540043d0 con 0x555a257328b0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d3effd640 1 -- 172.21.15.22:0/137427177 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1d580046d0 con 0x555a257328b0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d3effd640 1 -- 172.21.15.22:0/137427177 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d58029430 con 0x555a257328b0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 10 client.18282 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555a254697d0 con 0x555a257328b0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d3effd640 1 -- 172.21.15.22:0/137427177 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d540043d0 con 0x555a257328b0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d3effd640 1 -- 172.21.15.22:0/137427177 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1d58020480 con 0x555a257328b0 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 10 client.18282 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 10 client.18282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:03:04.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 20 client.18282 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a25742ad0 0x555a25744f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 4 client.18282 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.100+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a25698400 con 0x555a25742ad0 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.101+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a25742ad0 0x555a25744f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.101+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a25742ad0 0x555a25744f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ccf43f363b0a4af server_cookie=7fbb012c1da9e7a1 in_seq=0 out_seq=0 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.101+0000 7f1d3effd640 10 client.18282 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.101+0000 7f1d3effd640 1 -- 172.21.15.22:0/137427177 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a25698400 con 0x555a25742ad0 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.101+0000 7f1d3effd640 10 client.18282 handle_command_reply: tid=0 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.101+0000 7f1d68ba5740 1 client.18282 shutdown 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a257328b0 msgr2=0x555a257302c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a257328b0 0x555a257302c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1d580115c0 tx=0x7f1d58009500 comp rx=0 tx=0).stop 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 shutdown_connections 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a25742ad0 0x555a25744f30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a257328b0 0x555a257302c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d3effd640 0 client.18282 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:04.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/137427177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a25726e50 0x555a2572f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 >> 172.21.15.22:0/137427177 conn(0x555a257218d0 msgr2=0x555a25723620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 shutdown_connections 2024-03-09T00:03:04.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 1 -- 172.21.15.22:0/137427177 wait complete. 2024-03-09T00:03:04.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.102+0000 7f1d68ba5740 20 client.18282 trim_cache size 0 max 16384 2024-03-09T00:03:04.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:03 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2206900729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 Processor -- start 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 -- start start 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25731f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a2572e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601ae860 con 0x555a25726e50 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601ac400 con 0x555a25727770 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601aa4c0 con 0x555a257328b0 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25731f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25731f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42234/0 (socket says 172.21.15.22:42234) 2024-03-09T00:03:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3816001889 learned_addr learned my addr 172.21.15.22:0/3816001889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a2572e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5effd640 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3816001889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 msgr2=0x555a2572e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a2572e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3816001889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 msgr2=0x555a2572e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/3816001889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a25721700 con 0x555a25726e50 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a2572e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5effd640 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25731f70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f1d5002cfd0 tx=0x7f1d5008fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a5e2c79d02572b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.114+0000 7f1d457fa640 1 -- 172.21.15.22:0/3816001889 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5004b040 con 0x555a25726e50 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d457fa640 1 -- 172.21.15.22:0/3816001889 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d500be010 con 0x555a25726e50 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d457fa640 1 -- 172.21.15.22:0/3816001889 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d500c2270 con 0x555a25726e50 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 -- 172.21.15.22:0/3816001889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 msgr2=0x555a25731f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25731f70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f1d5002cfd0 tx=0x7f1d5008fcb0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 -- 172.21.15.22:0/3816001889 shutdown_connections 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a2572e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a2572e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/3816001889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25731f70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 -- 172.21.15.22:0/3816001889 >> 172.21.15.22:0/3816001889 conn(0x555a257218d0 msgr2=0x555a25749890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 -- 172.21.15.22:0/3816001889 shutdown_connections 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 -- 172.21.15.22:0/3816001889 wait complete. 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.115+0000 7f1d68ba5740 1 -- start start 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 1 Processor -- start 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25738390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a257388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25738e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601aaac0 con 0x555a25726e50 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601abda0 con 0x555a25727770 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601ab0e0 con 0x555a257328b0 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d68ba5740 10 client.0 _ll_register_callbacks cb 0x7f1d54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25738390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25738390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42238/0 (socket says 172.21.15.22:42238) 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/1696206812 learned_addr learned my addr 172.21.15.22:0/1696206812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5ffff640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25738e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5effd640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a257388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/1696206812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 msgr2=0x555a25738e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25738e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 msgr2=0x555a257388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a257388d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 -- 172.21.15.22:0/1696206812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a2572b420 con 0x555a25726e50 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5effd640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a257388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d5f7fe640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25738390 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1d500b99a0 tx=0x7f1d5002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a256612ba0b40cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.116+0000 7f1d3effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d3effd640 1 -- 172.21.15.22:0/1696206812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5004b040 con 0x555a25726e50 2024-03-09T00:03:04.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d3effd640 1 -- 172.21.15.22:0/1696206812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1d500b9640 con 0x555a25726e50 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d500be010 con 0x555a25726e50 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d3effd640 1 -- 172.21.15.22:0/1696206812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d500c56c0 con 0x555a25726e50 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 10 client.18288 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555a2573e8b0 con 0x555a25726e50 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d3effd640 1 -- 172.21.15.22:0/1696206812 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d500be010 con 0x555a25726e50 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d3effd640 1 -- 172.21.15.22:0/1696206812 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1d500c0070 con 0x555a25726e50 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 10 client.18288 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 10 client.18288 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 20 client.18288 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a2579a550 0x555a2579c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 4 client.18288 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555a25692750 con 0x555a2579a550 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.117+0000 7f1d5effd640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a2579a550 0x555a2579c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d5effd640 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a2579a550 0x555a2579c970 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47983dfc99bb7086 server_cookie=836c1f8e51691b65 in_seq=0 out_seq=0 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d3effd640 10 client.18288 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d3effd640 1 -- 172.21.15.22:0/1696206812 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555a25692750 con 0x555a2579a550 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d3effd640 10 client.18288 handle_command_reply: tid=0 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 client.18288 shutdown 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 msgr2=0x555a25738390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25738390 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1d500b99a0 tx=0x7f1d5002bc30 comp rx=0 tx=0).stop 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 shutdown_connections 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a2579a550 0x555a2579c970 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a257328b0 0x555a25738e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a25727770 0x555a257388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d3effd640 0 client.18288 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 --2- 172.21.15.22:0/1696206812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a25726e50 0x555a25738390 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 >> 172.21.15.22:0/1696206812 conn(0x555a257218d0 msgr2=0x555a2572e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 shutdown_connections 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 1 -- 172.21.15.22:0/1696206812 wait complete. 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.118+0000 7f1d68ba5740 20 client.18288 trim_cache size 0 max 16384 2024-03-09T00:03:04.124 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:03:04.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1d480a2970 msgr2=0x7f1d480a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1d480a2970 0x7f1d480a4e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d58005fd0 tx=0x7f1d58009e70 comp rx=0 tx=0).stop 2024-03-09T00:03:04.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 msgr2=0x7f1d6019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d6019c6d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006010 tx=0x7f1d5002cb30 comp rx=0 tx=0).stop 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 shutdown_connections 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1d480a2970 0x7f1d480a4e30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d60196840 0x7f1d60196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d601054b0 0x7f1d6019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 --2- 172.21.15.22:0/3105580236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d601003e0 0x7f1d6019c6d0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 >> 172.21.15.22:0/3105580236 conn(0x7f1d600773a0 msgr2=0x7f1d600fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 shutdown_connections 2024-03-09T00:03:04.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.119+0000 7f1d65eea640 1 -- 172.21.15.22:0/3105580236 wait complete. 2024-03-09T00:03:04.128 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:03:04.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:03:04.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:03 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2206900729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:03:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 -- 172.21.15.22:0/2282020923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 msgr2=0x7fa840106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 --2- 172.21.15.22:0/2282020923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa840106710 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009930 tx=0x7fa83c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:03:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 -- 172.21.15.22:0/2282020923 shutdown_connections 2024-03-09T00:03:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 --2- 172.21.15.22:0/2282020923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa840101250 0x7fa84010d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 --2- 172.21.15.22:0/2282020923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840100890 0x7fa840100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 --2- 172.21.15.22:0/2282020923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa840106710 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 -- 172.21.15.22:0/2282020923 >> 172.21.15.22:0/2282020923 conn(0x7fa840076700 msgr2=0x7fa8400fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 -- 172.21.15.22:0/2282020923 shutdown_connections 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 -- 172.21.15.22:0/2282020923 wait complete. 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 Processor -- start 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.283+0000 7fa8484cd640 1 -- start start 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840100890 0x7fa840198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa840101250 0x7fa840199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa8401a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401106b0 con 0x7fa840106330 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa840110530 con 0x7fa840101250 2024-03-09T00:03:04.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa840110830 con 0x7fa840100890 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa8401a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846269640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840100890 0x7fa840198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa8401a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42258/0 (socket says 172.21.15.22:42258) 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846269640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840100890 0x7fa840198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39142/0 (socket says 172.21.15.22:39142) 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 -- 172.21.15.22:0/3103419424 learned_addr learned my addr 172.21.15.22:0/3103419424 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa845a68640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa840101250 0x7fa840199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 -- 172.21.15.22:0/3103419424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840100890 msgr2=0x7fa840198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840100890 0x7fa840198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 -- 172.21.15.22:0/3103419424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa840101250 msgr2=0x7fa840199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa840101250 0x7fa840199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 -- 172.21.15.22:0/3103419424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8401a40f0 con 0x7fa840106330 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa846a6a640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa8401a3bb0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa83800eb10 tx=0x7fa83800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8377fe640 1 -- 172.21.15.22:0/3103419424 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838004b20 con 0x7fa840106330 2024-03-09T00:03:04.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8401a42c0 con 0x7fa840106330 2024-03-09T00:03:04.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8377fe640 1 -- 172.21.15.22:0/3103419424 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa838004cc0 con 0x7fa840106330 2024-03-09T00:03:04.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8377fe640 1 -- 172.21.15.22:0/3103419424 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838010760 con 0x7fa840106330 2024-03-09T00:03:04.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.284+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8401a4760 con 0x7fa840106330 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.287+0000 7fa8377fe640 1 -- 172.21.15.22:0/3103419424 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa838005020 con 0x7fa840106330 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 Processor -- start 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 -- start start 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebab8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adeba7b360 con 0x55adebac1ab0 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adeba17ec0 con 0x55adebab8ed0 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.288+0000 7fa84b182740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adeb9c0d30 con 0x55adebab85b0 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa845a68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa845a68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42268/0 (socket says 172.21.15.22:42268) 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa845a68640 1 -- 172.21.15.22:0/1199006623 learned_addr learned my addr 172.21.15.22:0/1199006623 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846269640 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa845a68640 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebab8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa8377fe640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa82c0a2970 0x7fa82c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846a6a640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa82c0a2970 0x7fa82c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846269640 1 -- 172.21.15.22:0/1199006623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 msgr2=0x55adebab8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846269640 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebab8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846269640 1 -- 172.21.15.22:0/1199006623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 msgr2=0x55adebac3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846269640 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846269640 1 -- 172.21.15.22:0/1199006623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adeb9bfbc0 con 0x55adebab8ed0 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa845a68640 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebab8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa8377fe640 1 -- 172.21.15.22:0/3103419424 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa8380cf6e0 con 0x7fa840106330 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.289+0000 7fa846269640 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac1570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa83c036040 tx=0x7fa83c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4547e49bd0ed37f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.290+0000 7fa846a6a640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa82c0a2970 0x7fa82c0a4e30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8380cd040 tx=0x7fa83800efd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.290+0000 7fa834bfe640 1 -- 172.21.15.22:0/1199006623 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c045070 con 0x55adebab8ed0 2024-03-09T00:03:04.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.290+0000 7fa834bfe640 1 -- 172.21.15.22:0/1199006623 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa83c02c650 con 0x55adebab8ed0 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.290+0000 7fa834bfe640 1 -- 172.21.15.22:0/1199006623 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0049b0 con 0x55adebab8ed0 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 -- 172.21.15.22:0/1199006623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 msgr2=0x55adebac1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac1570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa83c036040 tx=0x7fa83c009560 comp rx=0 tx=0).stop 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 -- 172.21.15.22:0/1199006623 shutdown_connections 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac1570 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 --2- 172.21.15.22:0/1199006623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebab8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 -- 172.21.15.22:0/1199006623 >> 172.21.15.22:0/1199006623 conn(0x55adebab2ce0 msgr2=0x55adebab30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 -- 172.21.15.22:0/1199006623 shutdown_connections 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 -- 172.21.15.22:0/1199006623 wait complete. 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 -- start start 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.291+0000 7fa84b182740 1 Processor -- start 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebabe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab8ed0 0x55adebabef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebabf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adeb9b64a0 con 0x55adebac1ab0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adeba13f60 con 0x55adebab85b0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adebac4840 con 0x55adebab8ed0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 10 client.0 _ll_register_callbacks cb 0x55adeba11d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebabe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebabe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50622/0 (socket says 172.21.15.22:50622) 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 -- 172.21.15.22:0/2100240843 learned_addr learned my addr 172.21.15.22:0/2100240843 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa846269640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab8ed0 0x55adebabef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebabf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 -- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab8ed0 msgr2=0x55adebabef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab8ed0 0x55adebabef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 -- 172.21.15.22:0/2100240843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 msgr2=0x55adebabf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebabf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 -- 172.21.15.22:0/2100240843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adeb72c3f0 con 0x55adebab85b0 2024-03-09T00:03:04.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa846269640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab8ed0 0x55adebabef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa845a68640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebabe9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa83000f4d0 tx=0x7fa830004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6129765a4d8defec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa829ffb640 1 -- 172.21.15.22:0/2100240843 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830004450 con 0x55adebab85b0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa83c02c650 con 0x55adebab85b0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.292+0000 7fa829ffb640 1 -- 172.21.15.22:0/2100240843 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa830013110 con 0x55adebab85b0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa829ffb640 1 -- 172.21.15.22:0/2100240843 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa83000fe50 con 0x55adebab85b0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 10 client.27665 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55adeb7f67d0 con 0x55adebab85b0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa829ffb640 1 -- 172.21.15.22:0/2100240843 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa83c02c650 con 0x55adebab85b0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa829ffb640 1 -- 172.21.15.22:0/2100240843 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa8300115b0 con 0x55adebab85b0 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 10 client.27665 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 10 client.27665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 20 client.27665 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebad1b70 0x55adebad3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 4 client.27665 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adeba25400 con 0x55adebad1b70 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.293+0000 7fa846269640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebad1b70 0x55adebad3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.294+0000 7fa846269640 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebad1b70 0x55adebad3f90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78285b3c625c187c server_cookie=864404bb9e17a5f7 in_seq=0 out_seq=0 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.294+0000 7fa829ffb640 10 client.27665 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa829ffb640 1 -- 172.21.15.22:0/2100240843 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55adeba25400 con 0x55adebad1b70 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa829ffb640 10 client.27665 handle_command_reply: tid=0 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 client.27665 shutdown 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 msgr2=0x55adebabe9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebabe9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa83000f4d0 tx=0x7fa830004730 comp rx=0 tx=0).stop 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 shutdown_connections 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebad1b70 0x55adebad3f90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebabf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab8ed0 0x55adebabef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa829ffb640 0 client.27665 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:04.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 --2- 172.21.15.22:0/2100240843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebabe9f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 >> 172.21.15.22:0/2100240843 conn(0x55adebab2ce0 msgr2=0x55adebab4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 shutdown_connections 2024-03-09T00:03:04.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 1 -- 172.21.15.22:0/2100240843 wait complete. 2024-03-09T00:03:04.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.295+0000 7fa84b182740 20 client.27665 trim_cache size 0 max 16384 2024-03-09T00:03:04.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 Processor -- start 2024-03-09T00:03:04.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 -- start start 2024-03-09T00:03:04.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebac1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401ae910 con 0x55adebac1ab0 2024-03-09T00:03:04.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401ac4b0 con 0x55adebab8ed0 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.307+0000 7fa84b182740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401aa570 con 0x55adebab85b0 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa845a68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa846269640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa845a68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42282/0 (socket says 172.21.15.22:42282) 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa845a68640 1 -- 172.21.15.22:0/1230392046 learned_addr learned my addr 172.21.15.22:0/1230392046 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa845a68640 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebac1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa846269640 1 -- 172.21.15.22:0/1230392046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 msgr2=0x55adebac1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa846269640 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebac1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa846269640 1 -- 172.21.15.22:0/1230392046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 msgr2=0x55adebac8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa846269640 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa846269640 1 -- 172.21.15.22:0/1230392046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adebad9180 con 0x55adebab8ed0 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa845a68640 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa846269640 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac8a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa83c02c240 tx=0x7fa83c041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f04326f9d074c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa82bfff640 1 -- 172.21.15.22:0/1230392046 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c051070 con 0x55adebab8ed0 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa82bfff640 1 -- 172.21.15.22:0/1230392046 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa83c004ee0 con 0x55adebab8ed0 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.308+0000 7fa82bfff640 1 -- 172.21.15.22:0/1230392046 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c045040 con 0x55adebab8ed0 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 -- 172.21.15.22:0/1230392046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 msgr2=0x55adebac8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac8a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa83c02c240 tx=0x7fa83c041040 comp rx=0 tx=0).stop 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 -- 172.21.15.22:0/1230392046 shutdown_connections 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebac1ab0 0x55adebac8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab8ed0 0x55adebac8a70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 --2- 172.21.15.22:0/1230392046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebab85b0 0x55adebac1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 -- 172.21.15.22:0/1230392046 >> 172.21.15.22:0/1230392046 conn(0x55adebab2ce0 msgr2=0x55adebab9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 -- 172.21.15.22:0/1230392046 shutdown_connections 2024-03-09T00:03:04.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 -- 172.21.15.22:0/1230392046 wait complete. 2024-03-09T00:03:04.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 -- start start 2024-03-09T00:03:04.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 Processor -- start 2024-03-09T00:03:04.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebac89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.309+0000 7fa84b182740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebab8ed0 0x55adebac8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa84b182740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebac1ab0 0x55adebac6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa84b182740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401aab70 con 0x55adebab8ed0 2024-03-09T00:03:04.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa84b182740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401abe50 con 0x55adebab85b0 2024-03-09T00:03:04.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa84b182740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401ab190 con 0x55adebac1ab0 2024-03-09T00:03:04.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa84b182740 10 client.0 _ll_register_callbacks cb 0x7fa830005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:04.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa845a68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebac89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebab8ed0 0x55adebac8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebab8ed0 0x55adebac8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42292/0 (socket says 172.21.15.22:42292) 2024-03-09T00:03:04.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 -- 172.21.15.22:0/3314830519 learned_addr learned my addr 172.21.15.22:0/3314830519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa845a68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebac89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50630/0 (socket says 172.21.15.22:50630) 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa845a68640 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebac1ab0 0x55adebac6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 -- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebac1ab0 msgr2=0x55adebac6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebac1ab0 0x55adebac6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 -- 172.21.15.22:0/3314830519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 msgr2=0x55adebac89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebac89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 -- 172.21.15.22:0/3314830519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adebac77e0 con 0x55adebab8ed0 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa845a68640 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebac89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa846269640 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebab8ed0 0x55adebac8f30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009900 tx=0x7fa83c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce49bc035c99af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:04.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa829ffb640 1 -- 172.21.15.22:0/3314830519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c051050 con 0x55adebab8ed0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa83c004ee0 con 0x55adebab8ed0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa829ffb640 1 -- 172.21.15.22:0/3314830519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa83c004ac0 con 0x55adebab8ed0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.310+0000 7fa829ffb640 1 -- 172.21.15.22:0/3314830519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa83c045040 con 0x55adebab8ed0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 10 client.18306 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55adebb28c70 con 0x55adebab8ed0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa829ffb640 1 -- 172.21.15.22:0/3314830519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa83c004ee0 con 0x55adebab8ed0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa829ffb640 1 -- 172.21.15.22:0/3314830519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa83c0035f0 con 0x55adebab8ed0 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 10 client.18306 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 10 client.18306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 20 client.18306 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebb28f40 0x55adebb2b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 4 client.18306 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:03:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55adeba1f750 con 0x55adebb28f40 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa845a68640 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebb28f40 0x55adebb2b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa845a68640 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebb28f40 0x55adebb2b330 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de3a34e2060cdb1 server_cookie=2cd5080378aecccc in_seq=0 out_seq=0 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.311+0000 7fa829ffb640 10 client.18306 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.312+0000 7fa829ffb640 1 -- 172.21.15.22:0/3314830519 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55adeba1f750 con 0x55adebb28f40 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.312+0000 7fa829ffb640 10 client.18306 handle_command_reply: tid=0 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.312+0000 7fa84b182740 1 client.18306 shutdown 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.312+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebab8ed0 msgr2=0x55adebac8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.312+0000 7fa84b182740 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebab8ed0 0x55adebac8f30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009900 tx=0x7fa83c0058d0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 shutdown_connections 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55adebb28f40 0x55adebb2b330 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55adebac1ab0 0x55adebac6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa829ffb640 0 client.18306 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55adebab8ed0 0x55adebac8f30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 --2- 172.21.15.22:0/3314830519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adebab85b0 0x55adebac89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 >> 172.21.15.22:0/3314830519 conn(0x55adebab2ce0 msgr2=0x55adebabd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 shutdown_connections 2024-03-09T00:03:04.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 1 -- 172.21.15.22:0/3314830519 wait complete. 2024-03-09T00:03:04.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa84b182740 20 client.18306 trim_cache size 0 max 16384 2024-03-09T00:03:04.318 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:03:04.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa82c0a2970 msgr2=0x7fa82c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa82c0a2970 0x7fa82c0a4e30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8380cd040 tx=0x7fa83800efd0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 msgr2=0x7fa8401a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa8401a3bb0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa83800eb10 tx=0x7fa83800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 shutdown_connections 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa82c0a2970 0x7fa82c0a4e30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa840106330 0x7fa8401a3bb0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa840101250 0x7fa840199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 --2- 172.21.15.22:0/3103419424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840100890 0x7fa840198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.313+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 >> 172.21.15.22:0/3103419424 conn(0x7fa840076700 msgr2=0x7fa8401031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.314+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 shutdown_connections 2024-03-09T00:03:04.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:04.314+0000 7fa8484cd640 1 -- 172.21.15.22:0/3103419424 wait complete. 2024-03-09T00:03:04.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 891109 10.43 MB/sec execute 312 sec latency 37.263 ms 2024-03-09T00:03:05.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:04 smithi062 ceph-mon[36129]: pgmap v268: 129 pgs: 129 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 281 op/s 2024-03-09T00:03:05.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:04 smithi192 ceph-mon[35995]: pgmap v268: 129 pgs: 129 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 281 op/s 2024-03-09T00:03:05.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:04 smithi022 ceph-mon[31894]: pgmap v268: 129 pgs: 129 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 281 op/s 2024-03-09T00:03:05.321 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:03:05.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:03:05.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:03:05.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.519+0000 7fa644916640 1 -- 172.21.15.22:0/2557527664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640101ea0 msgr2=0x7fa640105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.519+0000 7fa644916640 1 --2- 172.21.15.22:0/2557527664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640101ea0 0x7fa640105ee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa630009900 tx=0x7fa63002bc60 comp rx=0 tx=0).stop 2024-03-09T00:03:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.521+0000 7fa644916640 1 -- 172.21.15.22:0/2557527664 shutdown_connections 2024-03-09T00:03:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.521+0000 7fa644916640 1 --2- 172.21.15.22:0/2557527664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401065c0 0x7fa64010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.521+0000 7fa644916640 1 --2- 172.21.15.22:0/2557527664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640101ea0 0x7fa640105ee0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa630009900 tx=0x7fa63002bc60 comp rx=0 tx=0).stop 2024-03-09T00:03:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.521+0000 7fa644916640 1 --2- 172.21.15.22:0/2557527664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6401014f0 0x7fa6401018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.521+0000 7fa644916640 1 -- 172.21.15.22:0/2557527664 >> 172.21.15.22:0/2557527664 conn(0x7fa6400786b0 msgr2=0x7fa640078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.522+0000 7fa644916640 1 -- 172.21.15.22:0/2557527664 shutdown_connections 2024-03-09T00:03:05.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.524+0000 7fa644916640 1 -- 172.21.15.22:0/2557527664 wait complete. 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 Processor -- start 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 -- start start 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401014f0 0x7fa640072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6401065c0 0x7fa64006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa64006c280 0x7fa64006c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64010f590 con 0x7fa6401065c0 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64010f410 con 0x7fa64006c280 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa644916640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64010f710 con 0x7fa6401014f0 2024-03-09T00:03:05.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa63e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401014f0 0x7fa640072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa63e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401014f0 0x7fa640072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39190/0 (socket says 172.21.15.22:39190) 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa63dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6401065c0 0x7fa64006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.525+0000 7fa63e59c640 1 -- 172.21.15.22:0/1148019223 learned_addr learned my addr 172.21.15.22:0/1148019223 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.526+0000 7fa63ed9d640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa64006c280 0x7fa64006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.526+0000 7fa63e59c640 1 -- 172.21.15.22:0/1148019223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa64006c280 msgr2=0x7fa64006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.526+0000 7fa63e59c640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa64006c280 0x7fa64006c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.526+0000 7fa63e59c640 1 -- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6401065c0 msgr2=0x7fa64006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.526+0000 7fa63e59c640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6401065c0 0x7fa64006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.526+0000 7fa63e59c640 1 -- 172.21.15.22:0/1148019223 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6400708e0 con 0x7fa6401014f0 2024-03-09T00:03:05.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.530+0000 7fa63e59c640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401014f0 0x7fa640072cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa62800b4d0 tx=0x7fa62800b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.531+0000 7fa6377fe640 1 -- 172.21.15.22:0/1148019223 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628014070 con 0x7fa6401014f0 2024-03-09T00:03:05.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.531+0000 7fa6377fe640 1 -- 172.21.15.22:0/1148019223 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa628010d50 con 0x7fa6401014f0 2024-03-09T00:03:05.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.531+0000 7fa6377fe640 1 -- 172.21.15.22:0/1148019223 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628017360 con 0x7fa6401014f0 2024-03-09T00:03:05.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.533+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa640070bd0 con 0x7fa6401014f0 2024-03-09T00:03:05.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.533+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa640071040 con 0x7fa6401014f0 2024-03-09T00:03:05.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.537+0000 7fa6475cf740 1 Processor -- start 2024-03-09T00:03:05.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.537+0000 7fa6377fe640 1 -- 172.21.15.22:0/1148019223 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa628017070 con 0x7fa6401014f0 2024-03-09T00:03:05.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.544+0000 7fa6475cf740 1 -- start start 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.544+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c03160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.544+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.544+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.544+0000 7fa6475cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f8b66b40 con 0x55c2f8c0c280 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.544+0000 7fa6475cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f8b0fd80 con 0x55c2f8c036a0 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.544+0000 7fa6475cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f8b62be0 con 0x55c2f8c02d80 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.545+0000 7fa6377fe640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa62c0a2b80 0x7fa62c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.545+0000 7fa6377fe640 1 -- 172.21.15.22:0/1148019223 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa6280ce830 con 0x7fa6401014f0 2024-03-09T00:03:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.546+0000 7fa63ed9d640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa62c0a2b80 0x7fa62c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.546+0000 7fa63ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c03160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.546+0000 7fa63ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c03160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39192/0 (socket says 172.21.15.22:39192) 2024-03-09T00:03:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.546+0000 7fa63ed9d640 1 -- 172.21.15.22:0/76276864 learned_addr learned my addr 172.21.15.22:0/76276864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.546+0000 7fa63e59c640 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.547+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.548+0000 7fa63dd9b640 1 -- 172.21.15.22:0/76276864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 msgr2=0x55c2f8c03160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.548+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c03160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.548+0000 7fa63dd9b640 1 -- 172.21.15.22:0/76276864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 msgr2=0x55c2f8c0bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.548+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.548+0000 7fa63dd9b640 1 -- 172.21.15.22:0/76276864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f89437e0 con 0x55c2f8c0c280 2024-03-09T00:03:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.548+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0e670 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa6401023f0 tx=0x7fa63002c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0e510fd9a15d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.549+0000 7fa634ff9640 1 -- 172.21.15.22:0/76276864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630042070 con 0x55c2f8c0c280 2024-03-09T00:03:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.550+0000 7fa634ff9640 1 -- 172.21.15.22:0/76276864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6300042c0 con 0x55c2f8c0c280 2024-03-09T00:03:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.551+0000 7fa6475cf740 1 -- 172.21.15.22:0/76276864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c0c280 msgr2=0x55c2f8c0e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.551+0000 7fa6475cf740 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0e670 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa6401023f0 tx=0x7fa63002c6d0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.551+0000 7fa63ed9d640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa62c0a2b80 0x7fa62c0a5040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa638009fd0 tx=0x7fa63800b540 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.552+0000 7fa6475cf740 1 -- 172.21.15.22:0/76276864 shutdown_connections 2024-03-09T00:03:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.552+0000 7fa6475cf740 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0e670 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.552+0000 7fa6475cf740 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.552+0000 7fa6475cf740 1 --2- 172.21.15.22:0/76276864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c03160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.552+0000 7fa6475cf740 1 -- 172.21.15.22:0/76276864 >> 172.21.15.22:0/76276864 conn(0x55c2f8bfd4b0 msgr2=0x55c2f8bfd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.553+0000 7fa6475cf740 1 -- 172.21.15.22:0/76276864 shutdown_connections 2024-03-09T00:03:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.555+0000 7fa6475cf740 1 -- 172.21.15.22:0/76276864 wait complete. 2024-03-09T00:03:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.555+0000 7fa6475cf740 1 -- start start 2024-03-09T00:03:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.555+0000 7fa6475cf740 1 Processor -- start 2024-03-09T00:03:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c0bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c036a0 0x55c2f8c08f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa6475cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f8c0f010 con 0x55c2f8c02d80 2024-03-09T00:03:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa6475cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f8b231c0 con 0x55c2f8c0c280 2024-03-09T00:03:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa6475cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f8c0f190 con 0x55c2f8c036a0 2024-03-09T00:03:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa6475cf740 10 client.0 _ll_register_callbacks cb 0x55c2f8b127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.557+0000 7fa63dd9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c036a0 0x55c2f8c08f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.558+0000 7fa63dd9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c036a0 0x55c2f8c08f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39204/0 (socket says 172.21.15.22:39204) 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.558+0000 7fa63dd9b640 1 -- 172.21.15.22:0/1761521185 learned_addr learned my addr 172.21.15.22:0/1761521185 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.558+0000 7fa63dd9b640 1 -- 172.21.15.22:0/1761521185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 msgr2=0x55c2f8c094c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.558+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c094c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.558+0000 7fa63dd9b640 1 -- 172.21.15.22:0/1761521185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 msgr2=0x55c2f8c0bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.558+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c0bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.558+0000 7fa63dd9b640 1 -- 172.21.15.22:0/1761521185 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2f89a3f70 con 0x55c2f8c036a0 2024-03-09T00:03:05.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.559+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c036a0 0x55c2f8c08f60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa6300048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6096c5d012d04972 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.559+0000 7fa625ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:03:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.559+0000 7fa625ffb640 1 -- 172.21.15.22:0/1761521185 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63004f060 con 0x55c2f8c036a0 2024-03-09T00:03:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.559+0000 7fa625ffb640 1 -- 172.21.15.22:0/1761521185 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa63002cac0 con 0x55c2f8c036a0 2024-03-09T00:03:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.560+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6300042c0 con 0x55c2f8c036a0 2024-03-09T00:03:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.561+0000 7fa625ffb640 1 -- 172.21.15.22:0/1761521185 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa630042070 con 0x55c2f8c036a0 2024-03-09T00:03:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.564+0000 7fa6475cf740 10 client.27757 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.564+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c2f883b170 con 0x55c2f8c036a0 2024-03-09T00:03:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.564+0000 7fa625ffb640 1 -- 172.21.15.22:0/1761521185 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6300042c0 con 0x55c2f8c036a0 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.564+0000 7fa625ffb640 1 -- 172.21.15.22:0/1761521185 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa630047460 con 0x55c2f8c036a0 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa6475cf740 10 client.27757 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa6475cf740 10 client.27757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa6475cf740 20 client.27757 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa6475cf740 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c1c150 0x55c2f8c1e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa6475cf740 4 client.27757 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2f8b3c960 con 0x55c2f8c1c150 2024-03-09T00:03:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa63e59c640 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c1c150 0x55c2f8c1e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.565+0000 7fa625ffb640 10 client.27757 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.566+0000 7fa63e59c640 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c1c150 0x55c2f8c1e540 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b840adaa5b933cf9 server_cookie=9fa85fd9cfbab254 in_seq=0 out_seq=0 2024-03-09T00:03:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.567+0000 7fa625ffb640 1 -- 172.21.15.22:0/1761521185 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2f8b3c960 con 0x55c2f8c1c150 2024-03-09T00:03:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.567+0000 7fa625ffb640 10 client.27757 handle_command_reply: tid=0 2024-03-09T00:03:05.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.567+0000 7fa6475cf740 1 client.27757 shutdown 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c036a0 msgr2=0x55c2f8c08f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c036a0 0x55c2f8c08f60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa630006010 tx=0x7fa6300048f0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 shutdown_connections 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c1c150 0x55c2f8c1e540 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c036a0 0x55c2f8c08f60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 --2- 172.21.15.22:0/1761521185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c0bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.569+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 >> 172.21.15.22:0/1761521185 conn(0x55c2f8bfd4b0 msgr2=0x55c2f8bff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.570+0000 7fa625ffb640 0 client.27757 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.570+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 shutdown_connections 2024-03-09T00:03:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.570+0000 7fa6475cf740 1 -- 172.21.15.22:0/1761521185 wait complete. 2024-03-09T00:03:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.571+0000 7fa6475cf740 20 client.27757 trim_cache size 0 max 16384 2024-03-09T00:03:05.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.606+0000 7fa6475cf740 1 Processor -- start 2024-03-09T00:03:05.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.608+0000 7fa6475cf740 1 -- start start 2024-03-09T00:03:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.609+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c22b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.609+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.609+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.609+0000 7fa6475cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401c88f0 con 0x55c2f8c036a0 2024-03-09T00:03:05.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.609+0000 7fa6475cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401c6490 con 0x55c2f8c0c280 2024-03-09T00:03:05.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.609+0000 7fa6475cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401c4550 con 0x55c2f8c02d80 2024-03-09T00:03:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.611+0000 7fa63dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42338/0 (socket says 172.21.15.22:42338) 2024-03-09T00:03:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63dd9b640 1 -- 172.21.15.22:0/25060702 learned_addr learned my addr 172.21.15.22:0/25060702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63e59c640 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c22b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63ed9d640 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63dd9b640 1 -- 172.21.15.22:0/25060702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 msgr2=0x55c2f8c22b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c22b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63dd9b640 1 -- 172.21.15.22:0/25060702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 msgr2=0x55c2f8c0bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.612+0000 7fa63dd9b640 1 -- 172.21.15.22:0/25060702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f8c02750 con 0x55c2f8c036a0 2024-03-09T00:03:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.613+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0b690 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa63002c1a0 tx=0x7fa630000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5854f4e081df0b99 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.613+0000 7fa63e59c640 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c22b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.613+0000 7fa63ed9d640 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:03:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.613+0000 7fa627fff640 1 -- 172.21.15.22:0/25060702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63004f040 con 0x55c2f8c036a0 2024-03-09T00:03:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.613+0000 7fa627fff640 1 -- 172.21.15.22:0/25060702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa630004960 con 0x55c2f8c036a0 2024-03-09T00:03:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.614+0000 7fa6475cf740 1 -- 172.21.15.22:0/25060702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c036a0 msgr2=0x55c2f8c0b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.614+0000 7fa6475cf740 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0b690 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa63002c1a0 tx=0x7fa630000bc0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.615+0000 7fa6475cf740 1 -- 172.21.15.22:0/25060702 shutdown_connections 2024-03-09T00:03:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.615+0000 7fa6475cf740 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c0c280 0x55c2f8c0bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.615+0000 7fa6475cf740 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c036a0 0x55c2f8c0b690 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.615+0000 7fa6475cf740 1 --2- 172.21.15.22:0/25060702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c02d80 0x55c2f8c22b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.615+0000 7fa6475cf740 1 -- 172.21.15.22:0/25060702 >> 172.21.15.22:0/25060702 conn(0x55c2f8bfd4b0 msgr2=0x55c2f8c0e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.616+0000 7fa6475cf740 1 -- 172.21.15.22:0/25060702 shutdown_connections 2024-03-09T00:03:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.616+0000 7fa6475cf740 1 -- 172.21.15.22:0/25060702 wait complete. 2024-03-09T00:03:05.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 -- start start 2024-03-09T00:03:05.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 Processor -- start 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c6c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c6cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c0c280 0x55c2f8c10690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401c4b50 con 0x55c2f8c02d80 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401c5e30 con 0x55c2f8c036a0 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401c5170 con 0x55c2f8c0c280 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.617+0000 7fa6475cf740 10 client.0 _ll_register_callbacks cb 0x7fa630009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c6c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c6cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c6cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50656/0 (socket says 172.21.15.22:50656) 2024-03-09T00:03:05.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63dd9b640 1 -- 172.21.15.22:0/114972826 learned_addr learned my addr 172.21.15.22:0/114972826 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63ed9d640 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c0c280 0x55c2f8c10690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63e59c640 1 -- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c0c280 msgr2=0x55c2f8c10690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63e59c640 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c0c280 0x55c2f8c10690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63e59c640 1 -- 172.21.15.22:0/114972826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 msgr2=0x55c2f8c6cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63e59c640 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c6cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.618+0000 7fa63e59c640 1 -- 172.21.15.22:0/114972826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2f8c10d80 con 0x55c2f8c02d80 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.619+0000 7fa63e59c640 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c6c650 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa6280cbbe0 tx=0x7fa62800b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dd4d47da8487eb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.619+0000 7fa625ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.619+0000 7fa625ffb640 1 -- 172.21.15.22:0/114972826 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6280b7070 con 0x55c2f8c02d80 2024-03-09T00:03:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.619+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2f8b28080 con 0x55c2f8c02d80 2024-03-09T00:03:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.619+0000 7fa625ffb640 1 -- 172.21.15.22:0/114972826 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa62800f530 con 0x55c2f8c02d80 2024-03-09T00:03:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.620+0000 7fa625ffb640 1 -- 172.21.15.22:0/114972826 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6280aa050 con 0x55c2f8c02d80 2024-03-09T00:03:05.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.622+0000 7fa6475cf740 10 client.18318 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:05.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.622+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c2f8c73e30 con 0x55c2f8c02d80 2024-03-09T00:03:05.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.623+0000 7fa625ffb640 1 -- 172.21.15.22:0/114972826 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c2f8b28080 con 0x55c2f8c02d80 2024-03-09T00:03:05.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.623+0000 7fa625ffb640 1 -- 172.21.15.22:0/114972826 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa6280cbd50 con 0x55c2f8c02d80 2024-03-09T00:03:05.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.625+0000 7fa6475cf740 10 client.18318 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:05.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.625+0000 7fa6475cf740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:05.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.625+0000 7fa6475cf740 20 client.18318 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:05.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.625+0000 7fa6475cf740 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c744b0 0x55c2f8c768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.625+0000 7fa6475cf740 4 client.18318 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:03:05.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.625+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c2f8b74080 con 0x55c2f8c744b0 2024-03-09T00:03:05.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.627+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c744b0 0x55c2f8c768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.635+0000 7fa63dd9b640 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c744b0 0x55c2f8c768d0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7533ba6359568bba server_cookie=61bf6fa13779a2f in_seq=0 out_seq=0 2024-03-09T00:03:05.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.636+0000 7fa625ffb640 10 client.18318 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.639+0000 7fa625ffb640 1 -- 172.21.15.22:0/114972826 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c2f8b74080 con 0x55c2f8c744b0 2024-03-09T00:03:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.639+0000 7fa625ffb640 10 client.18318 handle_command_reply: tid=0 2024-03-09T00:03:05.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.640+0000 7fa6475cf740 1 client.18318 shutdown 2024-03-09T00:03:05.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.648+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 msgr2=0x55c2f8c6c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.648+0000 7fa6475cf740 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c6c650 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa6280cbbe0 tx=0x7fa62800b3f0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.649+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 shutdown_connections 2024-03-09T00:03:05.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.649+0000 7fa6475cf740 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c2f8c744b0 0x55c2f8c768d0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.649+0000 7fa6475cf740 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2f8c0c280 0x55c2f8c10690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.649+0000 7fa6475cf740 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2f8c036a0 0x55c2f8c6cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.649+0000 7fa6475cf740 1 --2- 172.21.15.22:0/114972826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f8c02d80 0x55c2f8c6c650 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.649+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 >> 172.21.15.22:0/114972826 conn(0x55c2f8bfd4b0 msgr2=0x55c2f8bfbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.649+0000 7fa625ffb640 0 client.18318 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.651+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 shutdown_connections 2024-03-09T00:03:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.652+0000 7fa6475cf740 1 -- 172.21.15.22:0/114972826 wait complete. 2024-03-09T00:03:05.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.652+0000 7fa6475cf740 20 client.18318 trim_cache size 0 max 16384 2024-03-09T00:03:05.649 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:03:05.649 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:03:05.649 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "35da6cf8-a61b-4a66-bc13-309dc90003ee", 2024-03-09T00:03:05.649 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:03:05.649 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa62c0a2b80 msgr2=0x7fa62c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa62c0a2b80 0x7fa62c0a5040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa638009fd0 tx=0x7fa63800b540 comp rx=0 tx=0).stop 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401014f0 msgr2=0x7fa640072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401014f0 0x7fa640072cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa62800b4d0 tx=0x7fa62800b990 comp rx=0 tx=0).stop 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 shutdown_connections 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa62c0a2b80 0x7fa62c0a5040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa64006c280 0x7fa64006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6401065c0 0x7fa64006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 --2- 172.21.15.22:0/1148019223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6401014f0 0x7fa640072cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 >> 172.21.15.22:0/1148019223 conn(0x7fa6400786b0 msgr2=0x7fa640103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 shutdown_connections 2024-03-09T00:03:05.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.656+0000 7fa644916640 1 -- 172.21.15.22:0/1148019223 wait complete. 2024-03-09T00:03:05.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:03:05.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:03:05.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 891325 10.40 MB/sec execute 313 sec latency 55.856 ms 2024-03-09T00:03:05.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.858+0000 7fc64a759640 1 -- 172.21.15.22:0/2539773834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644068a30 msgr2=0x7fc644068e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.858+0000 7fc64a759640 1 --2- 172.21.15.22:0/2539773834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644068a30 0x7fc644068e90 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc634009930 tx=0x7fc63402bc20 comp rx=0 tx=0).stop 2024-03-09T00:03:05.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.859+0000 7fc64a759640 1 -- 172.21.15.22:0/2539773834 shutdown_connections 2024-03-09T00:03:05.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.859+0000 7fc64a759640 1 --2- 172.21.15.22:0/2539773834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644103db0 0x7fc644104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.859+0000 7fc64a759640 1 --2- 172.21.15.22:0/2539773834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644068a30 0x7fc644068e90 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.859+0000 7fc64a759640 1 --2- 172.21.15.22:0/2539773834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc644106240 0x7fc6440684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.859+0000 7fc64a759640 1 -- 172.21.15.22:0/2539773834 >> 172.21.15.22:0/2539773834 conn(0x7fc6440772f0 msgr2=0x7fc644077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.862+0000 7fc64a759640 1 -- 172.21.15.22:0/2539773834 shutdown_connections 2024-03-09T00:03:05.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.864+0000 7fc64a759640 1 -- 172.21.15.22:0/2539773834 wait complete. 2024-03-09T00:03:05.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.865+0000 7fc64a759640 1 Processor -- start 2024-03-09T00:03:05.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc64a759640 1 -- start start 2024-03-09T00:03:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc64a759640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc644068a30 0x7fc644071db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc64a759640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644103db0 0x7fc6440722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc64a759640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644106240 0x7fc6441098b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc64a759640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc644117ca0 con 0x7fc644106240 2024-03-09T00:03:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc64a759640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc644117b20 con 0x7fc644103db0 2024-03-09T00:03:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc64a759640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc644117e20 con 0x7fc644068a30 2024-03-09T00:03:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.866+0000 7fc648cf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644106240 0x7fc6441098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc648cf6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644106240 0x7fc6441098b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42376/0 (socket says 172.21.15.22:42376) 2024-03-09T00:03:05.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc648cf6640 1 -- 172.21.15.22:0/2577806520 learned_addr learned my addr 172.21.15.22:0/2577806520 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644103db0 0x7fc6440722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc6437fe640 1 -- 172.21.15.22:0/2577806520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc644068a30 msgr2=0x7fc644071db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:03:05.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc644068a30 0x7fc644071db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc6437fe640 1 -- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644106240 msgr2=0x7fc6441098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644106240 0x7fc6441098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc6437fe640 1 -- 172.21.15.22:0/2577806520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644109fe0 con 0x7fc644103db0 2024-03-09T00:03:05.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.867+0000 7fc648cf6640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644106240 0x7fc6441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:05.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.871+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644103db0 0x7fc6440722f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc63402c160 tx=0x7fc63402de90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.871+0000 7fc6417fa640 1 -- 172.21.15.22:0/2577806520 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634043020 con 0x7fc644103db0 2024-03-09T00:03:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.872+0000 7fc6417fa640 1 -- 172.21.15.22:0/2577806520 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc634004a10 con 0x7fc644103db0 2024-03-09T00:03:05.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.872+0000 7fc6417fa640 1 -- 172.21.15.22:0/2577806520 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634005220 con 0x7fc644103db0 2024-03-09T00:03:05.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.878+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64410a270 con 0x7fc644103db0 2024-03-09T00:03:05.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.878+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc644111b80 con 0x7fc644103db0 2024-03-09T00:03:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.888+0000 7fc64d40a740 1 Processor -- start 2024-03-09T00:03:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.892+0000 7fc64d40a740 1 -- start start 2024-03-09T00:03:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.893+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1e760 0x56500eb1eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.893+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1f080 0x56500eb27720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.893+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb2a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.893+0000 7fc64d40a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500ea81e00 con 0x56500eb1e760 2024-03-09T00:03:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.893+0000 7fc64d40a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500ea2ab10 con 0x56500eb27c60 2024-03-09T00:03:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.893+0000 7fc64d40a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500ea7dea0 con 0x56500eb1f080 2024-03-09T00:03:05.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.893+0000 7fc6417fa640 1 -- 172.21.15.22:0/2577806520 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc634046020 con 0x7fc644103db0 2024-03-09T00:03:05.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.894+0000 7fc643fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1f080 0x56500eb27720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.894+0000 7fc643fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1f080 0x56500eb27720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39236/0 (socket says 172.21.15.22:39236) 2024-03-09T00:03:05.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.894+0000 7fc643fff640 1 -- 172.21.15.22:0/2155163861 learned_addr learned my addr 172.21.15.22:0/2155163861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.894+0000 7fc648cf6640 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1e760 0x56500eb1eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.896+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb2a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.897+0000 7fc6417fa640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc6300a2b40 0x7fc6300a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.897+0000 7fc6417fa640 1 -- 172.21.15.22:0/2577806520 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc6340f0b60 con 0x7fc644103db0 2024-03-09T00:03:05.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.901+0000 7fc6437fe640 1 -- 172.21.15.22:0/2155163861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1f080 msgr2=0x56500eb27720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.901+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1f080 0x56500eb27720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.901+0000 7fc6437fe640 1 -- 172.21.15.22:0/2155163861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1e760 msgr2=0x56500eb1eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.901+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1e760 0x56500eb1eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.901+0000 7fc6437fe640 1 -- 172.21.15.22:0/2155163861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56500e8607d0 con 0x56500eb27c60 2024-03-09T00:03:05.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.901+0000 7fc648cf6640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc6300a2b40 0x7fc6300a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.904+0000 7fc648cf6640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc6300a2b40 0x7fc6300a5000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc62c006fd0 tx=0x7fc62c0046c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.904+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb2a050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6340edf10 tx=0x7fc634003990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c61eb234fa18187 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.904+0000 7fc63a7fc640 1 -- 172.21.15.22:0/2155163861 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340bd520 con 0x56500eb27c60 2024-03-09T00:03:05.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.904+0000 7fc63a7fc640 1 -- 172.21.15.22:0/2155163861 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6340bce20 con 0x56500eb27c60 2024-03-09T00:03:05.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.905+0000 7fc63a7fc640 1 -- 172.21.15.22:0/2155163861 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340b9490 con 0x56500eb27c60 2024-03-09T00:03:05.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.906+0000 7fc64d40a740 1 -- 172.21.15.22:0/2155163861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 msgr2=0x56500eb2a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.906+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb2a050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6340edf10 tx=0x7fc634003990 comp rx=0 tx=0).stop 2024-03-09T00:03:05.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.906+0000 7fc64d40a740 1 -- 172.21.15.22:0/2155163861 shutdown_connections 2024-03-09T00:03:05.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.906+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb2a050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.906+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1f080 0x56500eb27720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.906+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2155163861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1e760 0x56500eb1eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.906+0000 7fc64d40a740 1 -- 172.21.15.22:0/2155163861 >> 172.21.15.22:0/2155163861 conn(0x56500eb18e90 msgr2=0x56500eb19270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.907+0000 7fc64d40a740 1 -- 172.21.15.22:0/2155163861 shutdown_connections 2024-03-09T00:03:05.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.907+0000 7fc64d40a740 1 -- 172.21.15.22:0/2155163861 wait complete. 2024-03-09T00:03:05.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.908+0000 7fc64d40a740 1 -- start start 2024-03-09T00:03:05.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.908+0000 7fc64d40a740 1 Processor -- start 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.909+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1e760 0x56500eb273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.909+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb24730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.909+0000 7fc643fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1e760 0x56500eb273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.909+0000 7fc643fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1e760 0x56500eb273a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39248/0 (socket says 172.21.15.22:39248) 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.909+0000 7fc643fff640 1 -- 172.21.15.22:0/2807149488 learned_addr learned my addr 172.21.15.22:0/2807149488 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.909+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb24c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.909+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500ea2ab10 con 0x56500eb1f080 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.910+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb24730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.910+0000 7fc648cf6640 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb24c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.910+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500ea3e0a0 con 0x56500eb27c60 2024-03-09T00:03:05.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.910+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56500eb2a810 con 0x56500eb1e760 2024-03-09T00:03:05.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.910+0000 7fc64d40a740 10 client.0 _ll_register_callbacks cb 0x56500ea3da70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.912+0000 7fc643fff640 1 -- 172.21.15.22:0/2807149488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 msgr2=0x56500eb24c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.912+0000 7fc643fff640 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb24c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.912+0000 7fc643fff640 1 -- 172.21.15.22:0/2807149488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 msgr2=0x56500eb24730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.912+0000 7fc643fff640 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb24730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.912+0000 7fc643fff640 1 -- 172.21.15.22:0/2807149488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56500e7963f0 con 0x56500eb1e760 2024-03-09T00:03:05.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.913+0000 7fc643fff640 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1e760 0x56500eb273a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc63c011930 tx=0x7fc63c011df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac516a3878e65bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.913+0000 7fc6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:03:05.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.915+0000 7fc6237fe640 1 -- 172.21.15.22:0/2807149488 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00fdf0 con 0x56500eb1e760 2024-03-09T00:03:05.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.915+0000 7fc6237fe640 1 -- 172.21.15.22:0/2807149488 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc63c004b20 con 0x56500eb1e760 2024-03-09T00:03:05.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.915+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6340bce20 con 0x56500eb1e760 2024-03-09T00:03:05.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.916+0000 7fc6237fe640 1 -- 172.21.15.22:0/2807149488 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc63c024050 con 0x56500eb1e760 2024-03-09T00:03:05.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.917+0000 7fc64d40a740 10 client.27772 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:05.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.917+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56500e7560f0 con 0x56500eb1e760 2024-03-09T00:03:05.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.920+0000 7fc6237fe640 1 -- 172.21.15.22:0/2807149488 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6340bce20 con 0x56500eb1e760 2024-03-09T00:03:05.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.921+0000 7fc6237fe640 1 -- 172.21.15.22:0/2807149488 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc63c00e5e0 con 0x56500eb1e760 2024-03-09T00:03:05.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.921+0000 7fc64d40a740 10 client.27772 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:05.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.921+0000 7fc64d40a740 10 client.27772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:05.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.921+0000 7fc64d40a740 20 client.27772 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:05.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.921+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb373e0 0x56500eb397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.921+0000 7fc64d40a740 4 client.27772 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:05.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.921+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56500ea8f340 con 0x56500eb373e0 2024-03-09T00:03:05.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.922+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb373e0 0x56500eb397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.930+0000 7fc6437fe640 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb373e0 0x56500eb397d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=341630e3e96f86a2 server_cookie=eaf4375962ee8f83 in_seq=0 out_seq=0 2024-03-09T00:03:05.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.930+0000 7fc6237fe640 10 client.27772 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:05.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.933+0000 7fc6237fe640 1 -- 172.21.15.22:0/2807149488 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56500ea8f340 con 0x56500eb373e0 2024-03-09T00:03:05.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.933+0000 7fc6237fe640 10 client.27772 handle_command_reply: tid=0 2024-03-09T00:03:05.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.933+0000 7fc64d40a740 1 client.27772 shutdown 2024-03-09T00:03:05.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1e760 msgr2=0x56500eb273a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1e760 0x56500eb273a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc63c011930 tx=0x7fc63c011df0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 shutdown_connections 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb373e0 0x56500eb397d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb27c60 0x56500eb24c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb24730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 --2- 172.21.15.22:0/2807149488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb1e760 0x56500eb273a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.936+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 >> 172.21.15.22:0/2807149488 conn(0x56500eb18e90 msgr2=0x56500eb1abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.937+0000 7fc6237fe640 0 client.27772 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.937+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 shutdown_connections 2024-03-09T00:03:05.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.937+0000 7fc64d40a740 1 -- 172.21.15.22:0/2807149488 wait complete. 2024-03-09T00:03:05.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.938+0000 7fc64d40a740 20 client.27772 trim_cache size 0 max 16384 2024-03-09T00:03:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.967+0000 7fc64d40a740 1 Processor -- start 2024-03-09T00:03:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc64d40a740 1 -- start start 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 0x56500eb1fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb17d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb2ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc64d40a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441c8410 con 0x56500eb1f080 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc64d40a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441c5fb0 con 0x56500eb1e760 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc64d40a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441c4090 con 0x56500eb27c60 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc6437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb17d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc6437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb17d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42394/0 (socket says 172.21.15.22:42394) 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc6437fe640 1 -- 172.21.15.22:0/3761077198 learned_addr learned my addr 172.21.15.22:0/3761077198 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.968+0000 7fc648cf6640 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb2ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.969+0000 7fc643fff640 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 0x56500eb1fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.969+0000 7fc6437fe640 1 -- 172.21.15.22:0/3761077198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 msgr2=0x56500eb2ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.969+0000 7fc6437fe640 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb2ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.969+0000 7fc6437fe640 1 -- 172.21.15.22:0/3761077198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 msgr2=0x56500eb1fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.969+0000 7fc6437fe640 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 0x56500eb1fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.969+0000 7fc6437fe640 1 -- 172.21.15.22:0/3761077198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56500eb1eeb0 con 0x56500eb1f080 2024-03-09T00:03:05.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.970+0000 7fc6437fe640 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb17d70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc6340bcdb0 tx=0x7fc63402ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bed9a6de8dc25e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.971+0000 7fc639ffb640 1 -- 172.21.15.22:0/3761077198 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340bce20 con 0x56500eb1f080 2024-03-09T00:03:05.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.971+0000 7fc639ffb640 1 -- 172.21.15.22:0/3761077198 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6340bd520 con 0x56500eb1f080 2024-03-09T00:03:05.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.971+0000 7fc639ffb640 1 -- 172.21.15.22:0/3761077198 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340b9490 con 0x56500eb1f080 2024-03-09T00:03:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.973+0000 7fc64d40a740 1 -- 172.21.15.22:0/3761077198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 msgr2=0x56500eb17d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.973+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb17d70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc6340bcdb0 tx=0x7fc63402ca50 comp rx=0 tx=0).stop 2024-03-09T00:03:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.975+0000 7fc64d40a740 1 -- 172.21.15.22:0/3761077198 shutdown_connections 2024-03-09T00:03:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.975+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb2ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.975+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb17d70 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.975+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3761077198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 0x56500eb1fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.975+0000 7fc64d40a740 1 -- 172.21.15.22:0/3761077198 >> 172.21.15.22:0/3761077198 conn(0x56500eb18e90 msgr2=0x56500eb17660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:05.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.978+0000 7fc64d40a740 1 -- 172.21.15.22:0/3761077198 shutdown_connections 2024-03-09T00:03:05.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.979+0000 7fc64d40a740 1 -- 172.21.15.22:0/3761077198 wait complete. 2024-03-09T00:03:05.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.979+0000 7fc64d40a740 1 -- start start 2024-03-09T00:03:05.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 1 Processor -- start 2024-03-09T00:03:05.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 0x56500eb85e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb86360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb81820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441c4690 con 0x56500eb1f080 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441c5950 con 0x56500eb1e760 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441c4c90 con 0x56500eb27c60 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc64d40a740 10 client.0 _ll_register_callbacks cb 0x7fc63c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc648cf6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb81820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc648cf6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb81820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39280/0 (socket says 172.21.15.22:39280) 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.980+0000 7fc648cf6640 1 -- 172.21.15.22:0/3711639482 learned_addr learned my addr 172.21.15.22:0/3711639482 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:05.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.981+0000 7fc6437fe640 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb86360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.981+0000 7fc648cf6640 1 -- 172.21.15.22:0/3711639482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 msgr2=0x56500eb85e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.981+0000 7fc648cf6640 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 0x56500eb85e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.981+0000 7fc648cf6640 1 -- 172.21.15.22:0/3711639482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 msgr2=0x56500eb86360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:05.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.981+0000 7fc648cf6640 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb86360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:05.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.981+0000 7fc648cf6640 1 -- 172.21.15.22:0/3711639482 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56500eb82050 con 0x56500eb27c60 2024-03-09T00:03:05.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.984+0000 7fc648cf6640 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb81820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc62c012dd0 tx=0x7fc62c0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd71756a71e4d382 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:05.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.984+0000 7fc6237fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:03:05.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.987+0000 7fc6237fe640 1 -- 172.21.15.22:0/3711639482 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c029070 con 0x56500eb27c60 2024-03-09T00:03:05.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.987+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6340bd520 con 0x56500eb27c60 2024-03-09T00:03:05.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.987+0000 7fc6237fe640 1 -- 172.21.15.22:0/3711639482 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc62c025410 con 0x56500eb27c60 2024-03-09T00:03:05.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:05 smithi022 ceph-mon[31894]: osdmap e105: 12 total, 12 up, 12 in 2024-03-09T00:03:05.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:05 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:03:05.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:05 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:05.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:05 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:03:05.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.990+0000 7fc6237fe640 1 -- 172.21.15.22:0/3711639482 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc62c035050 con 0x56500eb27c60 2024-03-09T00:03:05.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.990+0000 7fc64d40a740 10 client.27778 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:05.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.990+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56500eb8e760 con 0x56500eb27c60 2024-03-09T00:03:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.991+0000 7fc6237fe640 1 -- 172.21.15.22:0/3711639482 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6340bd520 con 0x56500eb27c60 2024-03-09T00:03:05.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.991+0000 7fc6237fe640 1 -- 172.21.15.22:0/3711639482 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc62c015050 con 0x56500eb27c60 2024-03-09T00:03:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.991+0000 7fc64d40a740 10 client.27778 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.991+0000 7fc64d40a740 10 client.27778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.991+0000 7fc64d40a740 20 client.27778 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.992+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb8ef10 0x56500eb91350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.992+0000 7fc64d40a740 4 client.27778 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:03:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.992+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56500ea89690 con 0x56500eb8ef10 2024-03-09T00:03:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.993+0000 7fc643fff640 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb8ef10 0x56500eb91350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.998+0000 7fc643fff640 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb8ef10 0x56500eb91350 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659d13014022b1c3 server_cookie=a4a813781a8ebec6 in_seq=0 out_seq=0 2024-03-09T00:03:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:05.998+0000 7fc6237fe640 10 client.27778 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.000+0000 7fc6237fe640 1 -- 172.21.15.22:0/3711639482 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56500ea89690 con 0x56500eb8ef10 2024-03-09T00:03:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.000+0000 7fc6237fe640 10 client.27778 handle_command_reply: tid=0 2024-03-09T00:03:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.000+0000 7fc64d40a740 1 client.27778 shutdown 2024-03-09T00:03:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.003+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 msgr2=0x56500eb81820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:05.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.003+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb81820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc62c012dd0 tx=0x7fc62c0047d0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.004+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 shutdown_connections 2024-03-09T00:03:06.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.004+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56500eb8ef10 0x56500eb91350 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.004+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56500eb27c60 0x56500eb81820 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.004+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56500eb1f080 0x56500eb86360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.004+0000 7fc64d40a740 1 --2- 172.21.15.22:0/3711639482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56500eb1e760 0x56500eb85e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.004+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 >> 172.21.15.22:0/3711639482 conn(0x56500eb18e90 msgr2=0x56500eb1aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.005+0000 7fc6237fe640 0 client.27778 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:06.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.006+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 shutdown_connections 2024-03-09T00:03:06.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.006+0000 7fc64d40a740 1 -- 172.21.15.22:0/3711639482 wait complete. 2024-03-09T00:03:06.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.006+0000 7fc64d40a740 20 client.27778 trim_cache size 0 max 16384 2024-03-09T00:03:06.006 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:03:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:03:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:03:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "35da6cf8-a61b-4a66-bc13-309dc90003ee": { 2024-03-09T00:03:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:03:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35da6cf8-a61b-4a66-bc13-309dc90003ee", 2024-03-09T00:03:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:03:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:03:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:03:06.008 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:03:06.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.010+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc6300a2b40 msgr2=0x7fc6300a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.010+0000 7fc64a759640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc6300a2b40 0x7fc6300a5000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc62c006fd0 tx=0x7fc62c0046c0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.010+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644103db0 msgr2=0x7fc6440722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.010+0000 7fc64a759640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644103db0 0x7fc6440722f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc63402c160 tx=0x7fc63402de90 comp rx=0 tx=0).stop 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.011+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 shutdown_connections 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.011+0000 7fc64a759640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc6300a2b40 0x7fc6300a5000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.011+0000 7fc64a759640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc644106240 0x7fc6441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.011+0000 7fc64a759640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644103db0 0x7fc6440722f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.011+0000 7fc64a759640 1 --2- 172.21.15.22:0/2577806520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc644068a30 0x7fc644071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.011+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 >> 172.21.15.22:0/2577806520 conn(0x7fc6440772f0 msgr2=0x7fc644101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.012+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 shutdown_connections 2024-03-09T00:03:06.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:06.012+0000 7fc64a759640 1 -- 172.21.15.22:0/2577806520 wait complete. 2024-03-09T00:03:06.027 INFO:tasks.cephfs.filesystem:scrub status for tag:35da6cf8-a61b-4a66-bc13-309dc90003ee - {'path': '/', 'tag': '35da6cf8-a61b-4a66-bc13-309dc90003ee', 'options': 'recursive,force'} 2024-03-09T00:03:06.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:05 smithi062 ceph-mon[36129]: osdmap e105: 12 total, 12 up, 12 in 2024-03-09T00:03:06.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:05 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:03:06.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:05 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:06.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:05 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:03:06.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:05 smithi192 ceph-mon[35995]: osdmap e105: 12 total, 12 up, 12 in 2024-03-09T00:03:06.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:05 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:03:06.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:05 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:06.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:05 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:03:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.244+0000 7f30140c0640 1 -- 172.21.15.22:0/3088979591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c10ccd0 msgr2=0x7f300c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.244+0000 7f30140c0640 1 --2- 172.21.15.22:0/3088979591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c10ccd0 0x7f300c10d0b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0074e0 tx=0x7f2ffc02c930 comp rx=0 tx=0).stop 2024-03-09T00:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.245+0000 7f30140c0640 1 -- 172.21.15.22:0/3088979591 shutdown_connections 2024-03-09T00:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.245+0000 7f30140c0640 1 --2- 172.21.15.22:0/3088979591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300c108c90 0x7f300c114b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.245+0000 7f30140c0640 1 --2- 172.21.15.22:0/3088979591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f300c1082f0 0x7f300c108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.245+0000 7f30140c0640 1 --2- 172.21.15.22:0/3088979591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c10ccd0 0x7f300c10d0b0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.245+0000 7f30140c0640 1 -- 172.21.15.22:0/3088979591 >> 172.21.15.22:0/3088979591 conn(0x7f300c06c820 msgr2=0x7f300c06cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.245+0000 7f30140c0640 1 -- 172.21.15.22:0/3088979591 shutdown_connections 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.245+0000 7f30140c0640 1 -- 172.21.15.22:0/3088979591 wait complete. 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 Processor -- start 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 -- start start 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c1082f0 0x7f300c11def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f300c108c90 0x7f300c117fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300c10ccd0 0x7f300c1184f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c116a70 con 0x7f300c1082f0 2024-03-09T00:03:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c1168f0 con 0x7f300c108c90 2024-03-09T00:03:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.246+0000 7f30140c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c116bf0 con 0x7f300c10ccd0 2024-03-09T00:03:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.247+0000 7f301265d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300c10ccd0 0x7f300c1184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.247+0000 7f3011e5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c1082f0 0x7f300c11def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.247+0000 7f3011e5c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c1082f0 0x7f300c11def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42430/0 (socket says 172.21.15.22:42430) 2024-03-09T00:03:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.247+0000 7f3011e5c640 1 -- 172.21.15.22:0/1717511073 learned_addr learned my addr 172.21.15.22:0/1717511073 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.247+0000 7f301165b640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f300c108c90 0x7f300c117fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.248+0000 7f301265d640 1 -- 172.21.15.22:0/1717511073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f300c108c90 msgr2=0x7f300c117fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.248+0000 7f301265d640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f300c108c90 0x7f300c117fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.248+0000 7f301265d640 1 -- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c1082f0 msgr2=0x7f300c11def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.248+0000 7f301265d640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c1082f0 0x7f300c11def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.248+0000 7f301265d640 1 -- 172.21.15.22:0/1717511073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c118c20 con 0x7f300c10ccd0 2024-03-09T00:03:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.249+0000 7f301265d640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300c10ccd0 0x7f300c1184f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f300400ca30 tx=0x7f300400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.249+0000 7f301165b640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f300c108c90 0x7f300c117fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:03:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.252+0000 7f3002ffd640 1 -- 172.21.15.22:0/1717511073 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004007d90 con 0x7f300c10ccd0 2024-03-09T00:03:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.252+0000 7f3002ffd640 1 -- 172.21.15.22:0/1717511073 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3004005020 con 0x7f300c10ccd0 2024-03-09T00:03:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.252+0000 7f3002ffd640 1 -- 172.21.15.22:0/1717511073 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300400f740 con 0x7f300c10ccd0 2024-03-09T00:03:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.252+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c1bb890 con 0x7f300c10ccd0 2024-03-09T00:03:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.252+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c1bbca0 con 0x7f300c10ccd0 2024-03-09T00:03:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.256+0000 7f3016d79740 1 Processor -- start 2024-03-09T00:03:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.262+0000 7f3016d79740 1 -- start start 2024-03-09T00:03:06.258 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3002ffd640 1 -- 172.21.15.22:0/1717511073 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3004004a40 con 0x7f300c10ccd0 2024-03-09T00:03:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3016d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5697d40 0x556df5696a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3016d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df5697450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3016d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df56a0550 0x556df56a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3016d79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556df55fb5f0 con 0x556df5696fd0 2024-03-09T00:03:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3016d79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556df55a4930 con 0x556df56a0550 2024-03-09T00:03:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3016d79740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556df55f7690 con 0x556df5697d40 2024-03-09T00:03:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f301165b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5697d40 0x556df5696a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f301165b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5697d40 0x556df5696a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39310/0 (socket says 172.21.15.22:39310) 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f301165b640 1 -- 172.21.15.22:0/1609944448 learned_addr learned my addr 172.21.15.22:0/1609944448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f301165b640 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df56a0550 0x556df56a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.263+0000 7f3002ffd640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2ff80a2c40 0x7f2ff80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.264+0000 7f3002ffd640 1 -- 172.21.15.22:0/1717511073 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f30040cf2f0 con 0x7f300c10ccd0 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.264+0000 7f301165b640 1 -- 172.21.15.22:0/1609944448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df56a0550 msgr2=0x556df56a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.264+0000 7f301165b640 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df56a0550 0x556df56a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.264+0000 7f301165b640 1 -- 172.21.15.22:0/1609944448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 msgr2=0x556df5697450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.265+0000 7f3011e5c640 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df5697450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.265+0000 7f301165b640 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df5697450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.265+0000 7f301165b640 1 -- 172.21.15.22:0/1609944448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556df5310210 con 0x556df5697d40 2024-03-09T00:03:06.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.266+0000 7f301265d640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2ff80a2c40 0x7f2ff80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.266+0000 7f3011e5c640 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df5697450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:03:06.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.266+0000 7f301165b640 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5697d40 0x556df5696a90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3008013860 tx=0x7f300800b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ab3d71bbf0fdf23 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.267+0000 7f2ff7fff640 1 -- 172.21.15.22:0/1609944448 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300800e8f0 con 0x556df5697d40 2024-03-09T00:03:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.267+0000 7f301265d640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2ff80a2c40 0x7f2ff80a5100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3004097e70 tx=0x7f3004097080 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.271+0000 7f2ff7fff640 1 -- 172.21.15.22:0/1609944448 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f300801e430 con 0x556df5697d40 2024-03-09T00:03:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.271+0000 7f2ff7fff640 1 -- 172.21.15.22:0/1609944448 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008011640 con 0x556df5697d40 2024-03-09T00:03:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.272+0000 7f3016d79740 1 -- 172.21.15.22:0/1609944448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5697d40 msgr2=0x556df5696a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.272+0000 7f3016d79740 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5697d40 0x556df5696a90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3008013860 tx=0x7f300800b400 comp rx=0 tx=0).stop 2024-03-09T00:03:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.272+0000 7f3016d79740 1 -- 172.21.15.22:0/1609944448 shutdown_connections 2024-03-09T00:03:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.272+0000 7f3016d79740 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df56a0550 0x556df56a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.272+0000 7f3016d79740 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df5697450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.272+0000 7f3016d79740 1 --2- 172.21.15.22:0/1609944448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5697d40 0x556df5696a90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.272+0000 7f3016d79740 1 -- 172.21.15.22:0/1609944448 >> 172.21.15.22:0/1609944448 conn(0x556df565e5b0 msgr2=0x556df5691cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.273+0000 7f3016d79740 1 -- 172.21.15.22:0/1609944448 shutdown_connections 2024-03-09T00:03:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.275+0000 7f3016d79740 1 -- 172.21.15.22:0/1609944448 wait complete. 2024-03-09T00:03:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.275+0000 7f3016d79740 1 -- start start 2024-03-09T00:03:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.275+0000 7f3016d79740 1 Processor -- start 2024-03-09T00:03:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.276+0000 7f3016d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df569d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.276+0000 7f3016d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.276+0000 7f3016d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df569df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.276+0000 7f3016d79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556df56a3320 con 0x556df5696fd0 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.276+0000 7f3016d79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556df55b7d90 con 0x556df5697d40 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.276+0000 7f3016d79740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556df56a34a0 con 0x556df56a0550 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.276+0000 7f3016d79740 10 client.0 _ll_register_callbacks cb 0x556df530fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df569d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df569d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42434/0 (socket says 172.21.15.22:42434) 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 -- 172.21.15.22:0/3942638844 learned_addr learned my addr 172.21.15.22:0/3942638844 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df569df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f3011e5c640 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 -- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 msgr2=0x556df569df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df569df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 -- 172.21.15.22:0/3942638844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 msgr2=0x556df569da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.277+0000 7f301165b640 1 -- 172.21.15.22:0/3942638844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556df53dc620 con 0x556df5696fd0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.278+0000 7f301165b640 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df569d500 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f300800b990 tx=0x7f300801ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29e8b1ba307030a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.278+0000 7f2ff57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.278+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3942638844 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300801e9f0 con 0x556df5696fd0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.278+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f300801e430 con 0x556df5696fd0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.278+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3942638844 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f3008004020 con 0x556df5696fd0 2024-03-09T00:03:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.278+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3942638844 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f300801eb90 con 0x556df5696fd0 2024-03-09T00:03:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.281+0000 7f3016d79740 10 client.18345 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.281+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556df56d1590 con 0x556df5696fd0 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3942638844 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f300801e430 con 0x556df5696fd0 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3942638844 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f300800f040 con 0x556df5696fd0 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f3016d79740 10 client.18345 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f3016d79740 10 client.18345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f3016d79740 20 client.18345 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f3016d79740 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df56d2d70 0x556df56d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f3016d79740 4 client.18345 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556df5608b30 con 0x556df56d2d70 2024-03-09T00:03:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.282+0000 7f3011e5c640 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df56d2d70 0x556df56d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.283+0000 7f3011e5c640 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df56d2d70 0x556df56d51b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3757291f997a011 server_cookie=45bcc7b742c5ae07 in_seq=0 out_seq=0 2024-03-09T00:03:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.283+0000 7f2ff57fa640 10 client.18345 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.284+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3942638844 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556df5608b30 con 0x556df56d2d70 2024-03-09T00:03:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.284+0000 7f2ff57fa640 10 client.18345 handle_command_reply: tid=0 2024-03-09T00:03:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.284+0000 7f3016d79740 1 client.18345 shutdown 2024-03-09T00:03:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.286+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 msgr2=0x556df569d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.286+0000 7f3016d79740 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df569d500 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f300800b990 tx=0x7f300801ee30 comp rx=0 tx=0).stop 2024-03-09T00:03:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.287+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 shutdown_connections 2024-03-09T00:03:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.287+0000 7f3016d79740 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df56d2d70 0x556df56d51b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.287+0000 7f3016d79740 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df569df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.287+0000 7f3016d79740 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.287+0000 7f3016d79740 1 --2- 172.21.15.22:0/3942638844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5696fd0 0x556df569d500 secure :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f300800b990 tx=0x7f300801ee30 comp rx=0 tx=0).stop 2024-03-09T00:03:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.287+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 >> 172.21.15.22:0/3942638844 conn(0x556df565e5b0 msgr2=0x556df5693a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.287+0000 7f2ff57fa640 0 client.18345 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.290+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 shutdown_connections 2024-03-09T00:03:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.290+0000 7f3016d79740 1 -- 172.21.15.22:0/3942638844 wait complete. 2024-03-09T00:03:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.292+0000 7f3016d79740 20 client.18345 trim_cache size 0 max 16384 2024-03-09T00:03:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.319+0000 7f3016d79740 1 Processor -- start 2024-03-09T00:03:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.319+0000 7f3016d79740 1 -- start start 2024-03-09T00:03:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.319+0000 7f3016d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.319+0000 7f3016d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df5698830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.319+0000 7f3016d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5698d70 0x556df569bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.320+0000 7f301165b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.320+0000 7f301165b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:50750/0 (socket says 172.21.15.22:50750) 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.320+0000 7f3011e5c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df5698830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.320+0000 7f301165b640 1 -- 172.21.15.22:0/1996275448 learned_addr learned my addr 172.21.15.22:0/1996275448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.320+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00124a0 con 0x556df5698d70 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.320+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0010040 con 0x556df5697d40 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.320+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff000e100 con 0x556df56a0550 2024-03-09T00:03:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.321+0000 7f301165b640 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5698d70 0x556df569bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.322+0000 7f3011e5c640 1 -- 172.21.15.22:0/1996275448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 msgr2=0x556df569fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.322+0000 7f3011e5c640 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.322+0000 7f3011e5c640 1 -- 172.21.15.22:0/1996275448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5698d70 msgr2=0x556df569bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.322+0000 7f3011e5c640 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5698d70 0x556df569bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.322+0000 7f3011e5c640 1 -- 172.21.15.22:0/1996275448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556df56962a0 con 0x556df56a0550 2024-03-09T00:03:06.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.324+0000 7f3011e5c640 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df5698830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000c00 tx=0x7f2ffc005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e32440e135b88bb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:06.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.324+0000 7f2ff77fe640 1 -- 172.21.15.22:0/1996275448 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc045be0 con 0x556df56a0550 2024-03-09T00:03:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.326+0000 7f2ff77fe640 1 -- 172.21.15.22:0/1996275448 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc049070 con 0x556df56a0550 2024-03-09T00:03:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.326+0000 7f2ff77fe640 1 -- 172.21.15.22:0/1996275448 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0046c0 con 0x556df56a0550 2024-03-09T00:03:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.328+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 msgr2=0x556df5698830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.328+0000 7f3016d79740 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df5698830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000c00 tx=0x7f2ffc005a00 comp rx=0 tx=0).stop 2024-03-09T00:03:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.330+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 shutdown_connections 2024-03-09T00:03:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.330+0000 7f3016d79740 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df5698d70 0x556df569bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.330+0000 7f3016d79740 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df56a0550 0x556df5698830 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.330+0000 7f3016d79740 1 --2- 172.21.15.22:0/1996275448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df569fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.330+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 >> 172.21.15.22:0/1996275448 conn(0x556df565e5b0 msgr2=0x556df56d9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.334+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 shutdown_connections 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.339+0000 7f3016d79740 1 -- 172.21.15.22:0/1996275448 wait complete. 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.339+0000 7f3016d79740 1 -- start start 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.339+0000 7f3016d79740 1 Processor -- start 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f3016d79740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df56a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f3016d79740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5698d70 0x556df56a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f3016d79740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df56a0550 0x556df56a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f3016d79740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff000e700 con 0x556df56a0550 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f3016d79740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff000f9e0 con 0x556df5697d40 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f3016d79740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff000ed20 con 0x556df5698d70 2024-03-09T00:03:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f3016d79740 10 client.0 _ll_register_callbacks cb 0x7f300800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f301165b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df56a0550 0x556df56a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f301165b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df56a0550 0x556df56a6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42454/0 (socket says 172.21.15.22:42454) 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f301165b640 1 -- 172.21.15.22:0/3616922665 learned_addr learned my addr 172.21.15.22:0/3616922665 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f301165b640 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df56a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f301165b640 1 -- 172.21.15.22:0/3616922665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5698d70 msgr2=0x556df56a5cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f301165b640 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5698d70 0x556df56a5cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.340+0000 7f301165b640 1 -- 172.21.15.22:0/3616922665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 msgr2=0x556df56a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.341+0000 7f301165b640 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df56a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.341+0000 7f301165b640 1 -- 172.21.15.22:0/3616922665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556df57284e0 con 0x556df56a0550 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.341+0000 7f301165b640 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df56a0550 0x556df56a6210 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3008001eb0 tx=0x7f300801ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebdbd8a9ad75cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.341+0000 7f2ff57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.341+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3616922665 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300801e430 con 0x556df56a0550 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.341+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ffc049070 con 0x556df56a0550 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.342+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3616922665 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f300800e6c0 con 0x556df56a0550 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.342+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3616922665 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f300800e8c0 con 0x556df56a0550 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.342+0000 7f3016d79740 10 client.18351 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.342+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556df574b800 con 0x556df56a0550 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3616922665 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc049070 con 0x556df56a0550 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3616922665 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3008005490 con 0x556df56a0550 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f3016d79740 10 client.18351 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f3016d79740 10 client.18351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f3016d79740 20 client.18351 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f3016d79740 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df574be80 0x556df574e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f3016d79740 4 client.18351 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:03:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.343+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556df5602e80 con 0x556df574be80 2024-03-09T00:03:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.344+0000 7f3011e5c640 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df574be80 0x556df574e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.347+0000 7f3011e5c640 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df574be80 0x556df574e2a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa8ee3d58bf1ae0 server_cookie=20330b4aed89075c in_seq=0 out_seq=0 2024-03-09T00:03:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.347+0000 7f2ff57fa640 10 client.18351 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:06.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 891504 10.36 MB/sec execute 314 sec latency 183.360 ms 2024-03-09T00:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.785+0000 7f2ff57fa640 1 -- 172.21.15.22:0/3616922665 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556df5602e80 con 0x556df574be80 2024-03-09T00:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.785+0000 7f2ff57fa640 10 client.18351 handle_command_reply: tid=0 2024-03-09T00:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.785+0000 7f3016d79740 1 client.18351 shutdown 2024-03-09T00:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.785+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df56a0550 msgr2=0x556df56a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.785+0000 7f3016d79740 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df56a0550 0x556df56a6210 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3008001eb0 tx=0x7f300801ee30 comp rx=0 tx=0).stop 2024-03-09T00:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 shutdown_connections 2024-03-09T00:03:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556df574be80 0x556df574e2a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556df56a0550 0x556df56a6210 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556df5698d70 0x556df56a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 --2- 172.21.15.22:0/3616922665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df5697d40 0x556df56a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 >> 172.21.15.22:0/3616922665 conn(0x556df565e5b0 msgr2=0x556df569bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f2ff57fa640 0 client.18351 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 shutdown_connections 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 1 -- 172.21.15.22:0/3616922665 wait complete. 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.786+0000 7f3016d79740 20 client.18351 trim_cache size 0 max 16384 2024-03-09T00:03:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2ff80a2c40 msgr2=0x7f2ff80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2ff80a2c40 0x7f2ff80a5100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3004097e70 tx=0x7f3004097080 comp rx=0 tx=0).stop 2024-03-09T00:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300c10ccd0 msgr2=0x7f300c1184f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300c10ccd0 0x7f300c1184f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f300400ca30 tx=0x7f300400cef0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 shutdown_connections 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f2ff80a2c40 0x7f2ff80a5100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300c10ccd0 0x7f300c1184f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f300c108c90 0x7f300c117fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 --2- 172.21.15.22:0/1717511073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f300c1082f0 0x7f300c11def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.787+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 >> 172.21.15.22:0/1717511073 conn(0x7f300c06c820 msgr2=0x7f300c070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.788+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 shutdown_connections 2024-03-09T00:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:06.788+0000 7f30140c0640 1 -- 172.21.15.22:0/1717511073 wait complete. 2024-03-09T00:03:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:06 smithi062 ceph-mon[36129]: pgmap v270: 129 pgs: 129 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-09T00:03:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:06 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:07.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:06 smithi192 ceph-mon[35995]: pgmap v270: 129 pgs: 129 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-09T00:03:07.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:06 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:07.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:06 smithi022 ceph-mon[31894]: pgmap v270: 129 pgs: 129 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-09T00:03:07.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:06 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:07.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 891847 10.33 MB/sec execute 315 sec latency 55.286 ms 2024-03-09T00:03:08.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 892288 10.30 MB/sec execute 316 sec latency 74.468 ms 2024-03-09T00:03:09.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:08 smithi062 ceph-mon[36129]: pgmap v271: 129 pgs: 129 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-09T00:03:09.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:08 smithi192 ceph-mon[35995]: pgmap v271: 129 pgs: 129 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-09T00:03:09.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:08 smithi022 ceph-mon[31894]: pgmap v271: 129 pgs: 129 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-09T00:03:09.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 892814 10.28 MB/sec execute 317 sec latency 51.785 ms 2024-03-09T00:03:10.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:09 smithi022 ceph-mon[31894]: pgmap v272: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 73 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 283 op/s 2024-03-09T00:03:10.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:09 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:03:10.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:09 smithi062 ceph-mon[36129]: pgmap v272: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 73 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 283 op/s 2024-03-09T00:03:10.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:09 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:03:10.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:09 smithi192 ceph-mon[35995]: pgmap v272: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 73 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 283 op/s 2024-03-09T00:03:10.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:09 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:03:10.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 894121 10.27 MB/sec execute 318 sec latency 42.885 ms 2024-03-09T00:03:11.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 895258 10.26 MB/sec execute 319 sec latency 17.259 ms 2024-03-09T00:03:12.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:12 smithi062 ceph-mon[36129]: pgmap v273: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-03-09T00:03:12.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:12 smithi022 ceph-mon[31894]: pgmap v273: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-03-09T00:03:12.515 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:12 smithi192 ceph-mon[35995]: pgmap v273: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-03-09T00:03:12.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 897577 10.27 MB/sec execute 320 sec latency 16.466 ms 2024-03-09T00:03:13.723 INFO:tasks.workunit.client.0.smithi022.stdout: 1 898746 10.26 MB/sec execute 321 sec latency 85.149 ms 2024-03-09T00:03:14.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:14 smithi062 ceph-mon[36129]: pgmap v274: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-03-09T00:03:14.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:14 smithi192 ceph-mon[35995]: pgmap v274: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-03-09T00:03:14.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:14 smithi022 ceph-mon[31894]: pgmap v274: 129 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 95 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-03-09T00:03:14.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 900608 10.25 MB/sec execute 322 sec latency 44.587 ms 2024-03-09T00:03:15.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 902072 10.25 MB/sec execute 323 sec latency 73.590 ms 2024-03-09T00:03:16.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:16 smithi062 ceph-mon[36129]: pgmap v275: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-03-09T00:03:16.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:16 smithi192 ceph-mon[35995]: pgmap v275: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-03-09T00:03:16.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:16 smithi022 ceph-mon[31894]: pgmap v275: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-03-09T00:03:16.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 904132 10.24 MB/sec execute 324 sec latency 16.892 ms 2024-03-09T00:03:17.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 905693 10.25 MB/sec execute 325 sec latency 14.529 ms 2024-03-09T00:03:18.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:18 smithi062 ceph-mon[36129]: pgmap v276: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-03-09T00:03:18.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:18 smithi192 ceph-mon[35995]: pgmap v276: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-03-09T00:03:18.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:18 smithi022 ceph-mon[31894]: pgmap v276: 129 pgs: 129 active+clean; 71 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-03-09T00:03:18.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 907182 10.23 MB/sec execute 326 sec latency 80.296 ms 2024-03-09T00:03:19.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 909071 10.24 MB/sec execute 327 sec latency 19.616 ms 2024-03-09T00:03:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 -- 172.21.15.22:0/3460412418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 msgr2=0x7f01380ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 --2- 172.21.15.22:0/3460412418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 0x7f01380ffc70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0120009890 tx=0x7f012002bb70 comp rx=0 tx=0).stop 2024-03-09T00:03:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 -- 172.21.15.22:0/3460412418 shutdown_connections 2024-03-09T00:03:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 --2- 172.21.15.22:0/3460412418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01381001b0 0x7f013810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 --2- 172.21.15.22:0/3460412418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 0x7f01380ffc70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 --2- 172.21.15.22:0/3460412418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381062d0 0x7f01381066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 -- 172.21.15.22:0/3460412418 >> 172.21.15.22:0/3460412418 conn(0x7f01380767b0 msgr2=0x7f01380fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 -- 172.21.15.22:0/3460412418 shutdown_connections 2024-03-09T00:03:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 -- 172.21.15.22:0/3460412418 wait complete. 2024-03-09T00:03:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 Processor -- start 2024-03-09T00:03:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.887+0000 7f013c89a640 1 -- start start 2024-03-09T00:03:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013c89a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 0x7f01381009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013c89a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381001b0 0x7f0138100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013c89a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01381062d0 0x7f0138101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013c89a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138111790 con 0x7f01381001b0 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013c89a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138111610 con 0x7f01381062d0 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013c89a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138111910 con 0x7f01380ff7f0 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381001b0 0x7f0138100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 0x7f01381009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381001b0 0x7f0138100f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34900/0 (socket says 172.21.15.22:34900) 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 -- 172.21.15.22:0/3691364304 learned_addr learned my addr 172.21.15.22:0/3691364304 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f0136d9d640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01381062d0 0x7f0138101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 -- 172.21.15.22:0/3691364304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 msgr2=0x7f01381009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 0x7f01381009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 -- 172.21.15.22:0/3691364304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01381062d0 msgr2=0x7f0138101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01381062d0 0x7f0138101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 -- 172.21.15.22:0/3691364304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01381a60b0 con 0x7f01381001b0 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f0136d9d640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01381062d0 0x7f0138101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012ffff640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381001b0 0x7f0138100f10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f012002c180 tx=0x7f0120004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012f7fe640 1 -- 172.21.15.22:0/3691364304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120041020 con 0x7f01381001b0 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01381a6340 con 0x7f01381001b0 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012f7fe640 1 -- 172.21.15.22:0/3691364304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f012002c9f0 con 0x7f01381001b0 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.888+0000 7f012f7fe640 1 -- 172.21.15.22:0/3691364304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120039040 con 0x7f01381001b0 2024-03-09T00:03:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.889+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01381a6880 con 0x7f01381001b0 2024-03-09T00:03:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.890+0000 7f012f7fe640 1 -- 172.21.15.22:0/3691364304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0120044020 con 0x7f01381001b0 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.891+0000 7f012f7fe640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01240a2a10 0x7f01240a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.891+0000 7f013f483740 1 Processor -- start 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.891+0000 7f012f7fe640 1 -- 172.21.15.22:0/3691364304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f01200f09a0 con 0x7f01381001b0 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.891+0000 7f013659c640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01240a2a10 0x7f01240a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.891+0000 7f013f483740 1 -- start start 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013f483740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff8d40 0x559155ff7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013f483740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff7fd0 0x559155ff8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013f483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559156001550 0x559156003940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013f483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559155f5c5f0 con 0x559156001550 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013f483740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559155f05930 con 0x559155ff7fd0 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013f483740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559155f58690 con 0x559155ff8d40 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013659c640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01240a2a10 0x7f01240a4ed0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01280096c0 tx=0x7f0128009260 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f0136d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559156001550 0x559156003940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f012ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff7fd0 0x559155ff8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f0136d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559156001550 0x559156003940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34914/0 (socket says 172.21.15.22:34914) 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f0136d9d640 1 -- 172.21.15.22:0/1490727091 learned_addr learned my addr 172.21.15.22:0/1490727091 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f013659c640 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff8d40 0x559155ff7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f012ffff640 1 -- 172.21.15.22:0/1490727091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff8d40 msgr2=0x559155ff7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f012ffff640 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff8d40 0x559155ff7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f012ffff640 1 -- 172.21.15.22:0/1490727091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559156001550 msgr2=0x559156003940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f012ffff640 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559156001550 0x559156003940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f012ffff640 1 -- 172.21.15.22:0/1490727091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559155c71210 con 0x559155ff7fd0 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.892+0000 7f012ffff640 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff7fd0 0x559155ff8450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01200ede00 tx=0x7f01200327b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727cc100eed50af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f012cff9640 1 -- 172.21.15.22:0/1490727091 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120034640 con 0x559155ff7fd0 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f012cff9640 1 -- 172.21.15.22:0/1490727091 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01200bd410 con 0x559155ff7fd0 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f012cff9640 1 -- 172.21.15.22:0/1490727091 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200edf70 con 0x559155ff7fd0 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f013f483740 1 -- 172.21.15.22:0/1490727091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff7fd0 msgr2=0x559155ff8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f013f483740 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff7fd0 0x559155ff8450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01200ede00 tx=0x7f01200327b0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f013f483740 1 -- 172.21.15.22:0/1490727091 shutdown_connections 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f013f483740 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559156001550 0x559156003940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f013f483740 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff7fd0 0x559155ff8450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f013f483740 1 --2- 172.21.15.22:0/1490727091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff8d40 0x559155ff7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.893+0000 7f013f483740 1 -- 172.21.15.22:0/1490727091 >> 172.21.15.22:0/1490727091 conn(0x559155fbf5b0 msgr2=0x559155ff2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 -- 172.21.15.22:0/1490727091 shutdown_connections 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 -- 172.21.15.22:0/1490727091 wait complete. 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 -- start start 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 Processor -- start 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff7fd0 0x559155ffe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff8d40 0x559155ffeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559156001550 0x559155fff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559156004040 con 0x559155ff7fd0 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559155f18d90 con 0x559155ff8d40 2024-03-09T00:03:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591560041c0 con 0x559156001550 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013f483740 10 client.0 _ll_register_callbacks cb 0x559155c70a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff7fd0 0x559155ffe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff7fd0 0x559155ffe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34920/0 (socket says 172.21.15.22:34920) 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013659c640 1 -- 172.21.15.22:0/1215131686 learned_addr learned my addr 172.21.15.22:0/1215131686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f0136d9d640 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559156001550 0x559155fff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f012ffff640 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff8d40 0x559155ffeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013659c640 1 -- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559156001550 msgr2=0x559155fff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.894+0000 7f013659c640 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559156001550 0x559155fff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013659c640 1 -- 172.21.15.22:0/1215131686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff8d40 msgr2=0x559155ffeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013659c640 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff8d40 0x559155ffeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013659c640 1 -- 172.21.15.22:0/1215131686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559155d3d620 con 0x559155ff7fd0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013659c640 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff7fd0 0x559155ffe5e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f012800ebe0 tx=0x7f0128004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342880b04d903d52 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f011dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f011dffb640 1 -- 172.21.15.22:0/1215131686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128028440 con 0x559155ff7fd0 2024-03-09T00:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f011dffb640 1 -- 172.21.15.22:0/1215131686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f01280273e0 con 0x559155ff7fd0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01200bd410 con 0x559155ff7fd0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f011dffb640 1 -- 172.21.15.22:0/1215131686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0128027620 con 0x559155ff7fd0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 10 client.18363 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559156010320 con 0x559155ff7fd0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f011dffb640 1 -- 172.21.15.22:0/1215131686 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01200bd410 con 0x559155ff7fd0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f011dffb640 1 -- 172.21.15.22:0/1215131686 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0128026070 con 0x559155ff7fd0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 10 client.18363 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 10 client.18363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 20 client.18363 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591560117e0 0x559156013c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.895+0000 7f013f483740 4 client.18363 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.896+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559155f69b30 con 0x5591560117e0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.896+0000 7f012ffff640 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591560117e0 0x559156013c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.896+0000 7f012ffff640 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591560117e0 0x559156013c10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eefa31761ba57560 server_cookie=522849eadd79f9cd in_seq=0 out_seq=0 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.896+0000 7f011dffb640 10 client.18363 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.897+0000 7f011dffb640 1 -- 172.21.15.22:0/1215131686 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559155f69b30 con 0x5591560117e0 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.897+0000 7f011dffb640 10 client.18363 handle_command_reply: tid=0 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.897+0000 7f013f483740 1 client.18363 shutdown 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff7fd0 msgr2=0x559155ffe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff7fd0 0x559155ffe5e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f012800ebe0 tx=0x7f0128004730 comp rx=0 tx=0).stop 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 shutdown_connections 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591560117e0 0x559156013c10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559156001550 0x559155fff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559155ff8d40 0x559155ffeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f011dffb640 0 client.18363 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 --2- 172.21.15.22:0/1215131686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff7fd0 0x559155ffe5e0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 >> 172.21.15.22:0/1215131686 conn(0x559155fbf5b0 msgr2=0x559155ff4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 shutdown_connections 2024-03-09T00:03:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 1 -- 172.21.15.22:0/1215131686 wait complete. 2024-03-09T00:03:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.898+0000 7f013f483740 20 client.18363 trim_cache size 0 max 16384 2024-03-09T00:03:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 Processor -- start 2024-03-09T00:03:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 -- start start 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156000d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x5591560083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559156008930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381b2d90 con 0x559155ff8d40 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381b0930 con 0x559156001550 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f013f483740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381ae9f0 con 0x559155ff7fd0 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f012ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x5591560083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f012ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x5591560083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34934/0 (socket says 172.21.15.22:34934) 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.912+0000 7f012ffff640 1 -- 172.21.15.22:0/3432136938 learned_addr learned my addr 172.21.15.22:0/3432136938 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f013659c640 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156000d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f0136d9d640 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559156008930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f012ffff640 1 -- 172.21.15.22:0/3432136938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 msgr2=0x559156000d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f012ffff640 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156000d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f012ffff640 1 -- 172.21.15.22:0/3432136938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 msgr2=0x559156008930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f012ffff640 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559156008930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f012ffff640 1 -- 172.21.15.22:0/3432136938 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559155ff72a0 con 0x559155ff8d40 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f012ffff640 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x5591560083f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f01200097b0 tx=0x7f01200324b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aafa78af983e263 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f011ffff640 1 -- 172.21.15.22:0/3432136938 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200d9070 con 0x559155ff8d40 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f011ffff640 1 -- 172.21.15.22:0/3432136938 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01200bd410 con 0x559155ff8d40 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.913+0000 7f011ffff640 1 -- 172.21.15.22:0/3432136938 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200b9380 con 0x559155ff8d40 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 -- 172.21.15.22:0/3432136938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 msgr2=0x5591560083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x5591560083f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f01200097b0 tx=0x7f01200324b0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 -- 172.21.15.22:0/3432136938 shutdown_connections 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559156008930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x5591560083f0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 --2- 172.21.15.22:0/3432136938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156000d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 -- 172.21.15.22:0/3432136938 >> 172.21.15.22:0/3432136938 conn(0x559155fbf5b0 msgr2=0x559155ff27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 -- 172.21.15.22:0/3432136938 shutdown_connections 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 -- 172.21.15.22:0/3432136938 wait complete. 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 -- start start 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.914+0000 7f013f483740 1 Processor -- start 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013f483740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156008920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013f483740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x559155ffb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013f483740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559155ffb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013f483740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381aeff0 con 0x559155ff8d40 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013f483740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381b02d0 con 0x559156001550 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013f483740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381af610 con 0x559155ff7fd0 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013f483740 10 client.0 _ll_register_callbacks cb 0x7f0120032640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156008920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f012ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x559155ffb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f0136d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559155ffb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156008920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:54492/0 (socket says 172.21.15.22:54492) 2024-03-09T00:03:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f012ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x559155ffb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34936/0 (socket says 172.21.15.22:34936) 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f013659c640 1 -- 172.21.15.22:0/3887207835 learned_addr learned my addr 172.21.15.22:0/3887207835 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f0136d9d640 1 -- 172.21.15.22:0/3887207835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 msgr2=0x559156008920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f0136d9d640 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156008920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f0136d9d640 1 -- 172.21.15.22:0/3887207835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 msgr2=0x559155ffb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f0136d9d640 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x559155ffb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f0136d9d640 1 -- 172.21.15.22:0/3887207835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559156067d30 con 0x559156001550 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f0136d9d640 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559155ffb7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0130005fd0 tx=0x7f01300108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6763002d8bb7080 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.915+0000 7f011dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f011dffb640 1 -- 172.21.15.22:0/3887207835 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013001c8a0 con 0x559156001550 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01200bd410 con 0x559156001550 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f011dffb640 1 -- 172.21.15.22:0/3887207835 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f013001ca40 con 0x559156001550 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f011dffb640 1 -- 172.21.15.22:0/3887207835 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f013002b050 con 0x559156001550 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 10 client.27725 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559156068f30 con 0x559156001550 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f011dffb640 1 -- 172.21.15.22:0/3887207835 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01200bd410 con 0x559156001550 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f011dffb640 1 -- 172.21.15.22:0/3887207835 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f01300263b0 con 0x559156001550 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 10 client.27725 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 10 client.27725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 20 client.27725 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559156069630 0x55915606ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 4 client.27725 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559155f63e80 con 0x559156069630 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.916+0000 7f013659c640 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559156069630 0x55915606ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.917+0000 7f011dffb640 10 client.27725 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:19.917+0000 7f013659c640 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559156069630 0x55915606ba50 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05d3a0e0fadb79 server_cookie=ac5101dd43615ed3 in_seq=0 out_seq=0 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f011dffb640 1 -- 172.21.15.22:0/3887207835 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559155f63e80 con 0x559156069630 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f011dffb640 10 client.27725 handle_command_reply: tid=0 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 client.27725 shutdown 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 msgr2=0x559155ffb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559155ffb7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0130005fd0 tx=0x7f01300108c0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 shutdown_connections 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559156069630 0x55915606ba50 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559156001550 0x559155ffb7f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559155ff8d40 0x559155ffb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 --2- 172.21.15.22:0/3887207835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559155ff7fd0 0x559156008920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 >> 172.21.15.22:0/3887207835 conn(0x559155fbf5b0 msgr2=0x559155ffce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f011dffb640 0 client.27725 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 shutdown_connections 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.223+0000 7f013f483740 1 -- 172.21.15.22:0/3887207835 wait complete. 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013f483740 20 client.27725 trim_cache size 0 max 16384 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:03:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:03:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01240a2a10 msgr2=0x7f01240a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01240a2a10 0x7f01240a4ed0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01280096c0 tx=0x7f0128009260 comp rx=0 tx=0).stop 2024-03-09T00:03:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381001b0 msgr2=0x7f0138100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381001b0 0x7f0138100f10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f012002c180 tx=0x7f0120004550 comp rx=0 tx=0).stop 2024-03-09T00:03:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 shutdown_connections 2024-03-09T00:03:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01240a2a10 0x7f01240a4ed0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01381062d0 0x7f0138101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01381001b0 0x7f0138100f10 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 --2- 172.21.15.22:0/3691364304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01380ff7f0 0x7f01381009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 >> 172.21.15.22:0/3691364304 conn(0x7f01380767b0 msgr2=0x7f01381041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 shutdown_connections 2024-03-09T00:03:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:20.224+0000 7f013c89a640 1 -- 172.21.15.22:0/3691364304 wait complete. 2024-03-09T00:03:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:19 smithi022 ceph-mon[31894]: osdmap e106: 12 total, 12 up, 12 in 2024-03-09T00:03:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:19 smithi022 ceph-mon[31894]: pgmap v278: 129 pgs: 129 active+clean; 76 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 271 op/s 2024-03-09T00:03:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:19 smithi062 ceph-mon[36129]: osdmap e106: 12 total, 12 up, 12 in 2024-03-09T00:03:20.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:19 smithi062 ceph-mon[36129]: pgmap v278: 129 pgs: 129 active+clean; 76 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 271 op/s 2024-03-09T00:03:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:19 smithi192 ceph-mon[35995]: osdmap e106: 12 total, 12 up, 12 in 2024-03-09T00:03:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:19 smithi192 ceph-mon[35995]: pgmap v278: 129 pgs: 129 active+clean; 76 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 271 op/s 2024-03-09T00:03:20.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 910224 10.23 MB/sec execute 328 sec latency 216.309 ms 2024-03-09T00:03:21.726 INFO:tasks.workunit.client.0.smithi022.stdout: 1 912471 10.24 MB/sec execute 329 sec latency 35.688 ms 2024-03-09T00:03:22.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:22 smithi062 ceph-mon[36129]: pgmap v279: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-03-09T00:03:22.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:22 smithi192 ceph-mon[35995]: pgmap v279: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-03-09T00:03:22.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:22 smithi022 ceph-mon[31894]: pgmap v279: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-03-09T00:03:22.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 913791 10.22 MB/sec execute 330 sec latency 18.243 ms 2024-03-09T00:03:23.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 915891 10.23 MB/sec execute 331 sec latency 66.575 ms 2024-03-09T00:03:24.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:24 smithi062 ceph-mon[36129]: pgmap v280: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-03-09T00:03:24.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:24 smithi192 ceph-mon[35995]: pgmap v280: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-03-09T00:03:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:24 smithi022 ceph-mon[31894]: pgmap v280: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 214 op/s 2024-03-09T00:03:24.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 916652 10.21 MB/sec execute 332 sec latency 47.116 ms 2024-03-09T00:03:25.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 917816 10.24 MB/sec execute 333 sec latency 99.983 ms 2024-03-09T00:03:26.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:26 smithi062 ceph-mon[36129]: pgmap v281: 129 pgs: 129 active+clean; 73 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-03-09T00:03:26.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:26 smithi192 ceph-mon[35995]: pgmap v281: 129 pgs: 129 active+clean; 73 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-03-09T00:03:26.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:26 smithi022 ceph-mon[31894]: pgmap v281: 129 pgs: 129 active+clean; 73 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-03-09T00:03:26.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 920321 10.26 MB/sec execute 334 sec latency 17.747 ms 2024-03-09T00:03:27.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 921777 10.25 MB/sec execute 335 sec latency 15.009 ms 2024-03-09T00:03:28.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:28 smithi062 ceph-mon[36129]: pgmap v282: 129 pgs: 129 active+clean; 73 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-03-09T00:03:28.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:28 smithi192 ceph-mon[35995]: pgmap v282: 129 pgs: 129 active+clean; 73 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-03-09T00:03:28.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:28 smithi022 ceph-mon[31894]: pgmap v282: 129 pgs: 129 active+clean; 73 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-03-09T00:03:28.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 923922 10.26 MB/sec execute 336 sec latency 133.917 ms 2024-03-09T00:03:29.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 925175 10.25 MB/sec execute 337 sec latency 45.924 ms 2024-03-09T00:03:30.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:30 smithi062 ceph-mon[36129]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-03-09T00:03:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:30 smithi192 ceph-mon[35995]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-03-09T00:03:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:30 smithi022 ceph-mon[31894]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-03-09T00:03:30.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 927558 10.26 MB/sec execute 338 sec latency 27.200 ms 2024-03-09T00:03:31.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 929334 10.24 MB/sec execute 339 sec latency 18.289 ms 2024-03-09T00:03:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:32 smithi062 ceph-mon[36129]: pgmap v284: 129 pgs: 129 active+clean; 81 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-03-09T00:03:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:32 smithi192 ceph-mon[35995]: pgmap v284: 129 pgs: 129 active+clean; 81 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-03-09T00:03:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:32 smithi022 ceph-mon[31894]: pgmap v284: 129 pgs: 129 active+clean; 81 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-03-09T00:03:32.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 931477 10.25 MB/sec execute 340 sec latency 16.771 ms 2024-03-09T00:03:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 -- 172.21.15.22:0/672787475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498069310 msgr2=0x7ff49810e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/672787475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498069310 0x7ff49810e180 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff490009ca0 tx=0x7ff49002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 -- 172.21.15.22:0/672787475 shutdown_connections 2024-03-09T00:03:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/672787475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498069310 0x7ff49810e180 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/672787475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff498068970 0x7ff498068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/672787475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff498107400 0x7ff4981077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 -- 172.21.15.22:0/672787475 >> 172.21.15.22:0/672787475 conn(0x7ff498077040 msgr2=0x7ff498077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 -- 172.21.15.22:0/672787475 shutdown_connections 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.314+0000 7ff49eaa4640 1 -- 172.21.15.22:0/672787475 wait complete. 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 Processor -- start 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 -- start start 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498068970 0x7ff49819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff498069310 0x7ff49819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff498107400 0x7ff498199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498111660 con 0x7ff498068970 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981114e0 con 0x7ff498069310 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981117e0 con 0x7ff498107400 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498068970 0x7ff49819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498068970 0x7ff49819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44298/0 (socket says 172.21.15.22:44298) 2024-03-09T00:03:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 -- 172.21.15.22:0/1035254700 learned_addr learned my addr 172.21.15.22:0/1035254700 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff497fff640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff498069310 0x7ff49819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49d041640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff498107400 0x7ff498199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 -- 172.21.15.22:0/1035254700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff498107400 msgr2=0x7ff498199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff498107400 0x7ff498199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 -- 172.21.15.22:0/1035254700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff498069310 msgr2=0x7ff49819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff498069310 0x7ff49819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 -- 172.21.15.22:0/1035254700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49819a180 con 0x7ff498068970 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49c840640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498068970 0x7ff49819f850 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00d950 tx=0x7ff48c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff495ffb640 1 -- 172.21.15.22:0/1035254700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c017030 con 0x7ff498068970 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff495ffb640 1 -- 172.21.15.22:0/1035254700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff48c004020 con 0x7ff498068970 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49819a470 con 0x7ff498068970 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.315+0000 7ff495ffb640 1 -- 172.21.15.22:0/1035254700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0047a0 con 0x7ff498068970 2024-03-09T00:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.316+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4981a6710 con 0x7ff498068970 2024-03-09T00:03:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.318+0000 7ff495ffb640 1 -- 172.21.15.22:0/1035254700 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff48c0041d0 con 0x7ff498068970 2024-03-09T00:03:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff495ffb640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4880a2a90 0x7ff4880a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 Processor -- start 2024-03-09T00:03:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 -- start start 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff497fff640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4880a2a90 0x7ff4880a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff495ffb640 1 -- 172.21.15.22:0/1035254700 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff48c0ceb00 con 0x7ff498068970 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d6d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d78940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd1cd15f0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd1c7a930 con 0x55dbd1d6dd40 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff4a1764740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd1ccd690 con 0x55dbd1d76550 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d78940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d78940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:55316/0 (socket says 172.21.15.22:55316) 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49c840640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 -- 172.21.15.22:0/2200293029 learned_addr learned my addr 172.21.15.22:0/2200293029 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff497fff640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4880a2a90 0x7ff4880a4f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff498102410 tx=0x7ff484005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff497fff640 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d6d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 -- 172.21.15.22:0/2200293029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 msgr2=0x55dbd1d6ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d6ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 -- 172.21.15.22:0/2200293029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 msgr2=0x55dbd1d6d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d6d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49d041640 1 -- 172.21.15.22:0/2200293029 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd19e6210 con 0x55dbd1d76550 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff497fff640 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d6d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.319+0000 7ff49c840640 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.320+0000 7ff49d041640 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d78940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff490009ca0 tx=0x7ff490009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e41cad749d23faa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.320+0000 7ff482ffd640 1 -- 172.21.15.22:0/2200293029 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490043070 con 0x55dbd1d76550 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.320+0000 7ff482ffd640 1 -- 172.21.15.22:0/2200293029 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff490032c90 con 0x55dbd1d76550 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.320+0000 7ff482ffd640 1 -- 172.21.15.22:0/2200293029 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490039040 con 0x55dbd1d76550 2024-03-09T00:03:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.320+0000 7ff4a1764740 1 -- 172.21.15.22:0/2200293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 msgr2=0x55dbd1d78940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.320+0000 7ff4a1764740 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d78940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff490009ca0 tx=0x7ff490009900 comp rx=0 tx=0).stop 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- 172.21.15.22:0/2200293029 shutdown_connections 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d78940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d6d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 --2- 172.21.15.22:0/2200293029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- 172.21.15.22:0/2200293029 >> 172.21.15.22:0/2200293029 conn(0x55dbd1d345b0 msgr2=0x55dbd1d67cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- 172.21.15.22:0/2200293029 shutdown_connections 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- 172.21.15.22:0/2200293029 wait complete. 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- start start 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 Processor -- start 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d73500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d73a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d76550 0x55dbd1d73f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd1d79220 con 0x55dbd1d76550 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd1c8dd90 con 0x55dbd1d6dd40 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd1d793a0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff4a1764740 10 client.0 _ll_register_callbacks cb 0x55dbd19e5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff49c840640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d73500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff49d041640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d76550 0x55dbd1d73f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff497fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d73a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff49c840640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d73500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:55324/0 (socket says 172.21.15.22:55324) 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff497fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d73a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:52270/0 (socket says 172.21.15.22:52270) 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49c840640 1 -- 172.21.15.22:0/1695163557 learned_addr learned my addr 172.21.15.22:0/1695163557 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.321+0000 7ff49d041640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d76550 0x55dbd1d73f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44310/0 (socket says 172.21.15.22:44310) 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49c840640 1 -- 172.21.15.22:0/1695163557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 msgr2=0x55dbd1d73a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49c840640 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d73a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49c840640 1 -- 172.21.15.22:0/1695163557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d76550 msgr2=0x55dbd1d73f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49c840640 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d76550 0x55dbd1d73f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49c840640 1 -- 172.21.15.22:0/1695163557 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbd1ab2620 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff497fff640 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d73a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49d041640 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d76550 0x55dbd1d73f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff49c840640 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d73500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff48c09acf0 tx=0x7ff48c09b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33631ad811630272 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff477fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:03:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff477fff640 1 -- 172.21.15.22:0/1695163557 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0a3500 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff477fff640 1 -- 172.21.15.22:0/1695163557 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff48c09a1b0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff490032c90 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff477fff640 1 -- 172.21.15.22:0/1695163557 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff48c09a3b0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff4a1764740 10 client.27820 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55dbd1d85540 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.322+0000 7ff477fff640 1 -- 172.21.15.22:0/1695163557 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff490032c90 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff477fff640 1 -- 172.21.15.22:0/1695163557 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff48c09fdd0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff4a1764740 10 client.27820 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff4a1764740 10 client.27820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff4a1764740 20 client.27820 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1d869b0 0x55dbd1d88dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff4a1764740 4 client.27820 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbd1cdeb30 con 0x55dbd1d869b0 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff497fff640 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1d869b0 0x55dbd1d88dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff477fff640 10 client.27820 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.323+0000 7ff497fff640 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1d869b0 0x55dbd1d88dc0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b60dc8e7af41ed server_cookie=37f8c9a2fd196ed4 in_seq=0 out_seq=0 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.324+0000 7ff477fff640 1 -- 172.21.15.22:0/1695163557 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbd1cdeb30 con 0x55dbd1d869b0 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.324+0000 7ff477fff640 10 client.27820 handle_command_reply: tid=0 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.324+0000 7ff4a1764740 1 client.27820 shutdown 2024-03-09T00:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6cfd0 msgr2=0x55dbd1d73500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d73500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff48c09acf0 tx=0x7ff48c09b590 comp rx=0 tx=0).stop 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 shutdown_connections 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1d869b0 0x55dbd1d88dc0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d76550 0x55dbd1d73f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff477fff640 0 client.27820 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d73a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1695163557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d73500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 >> 172.21.15.22:0/1695163557 conn(0x55dbd1d345b0 msgr2=0x55dbd1d69a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 shutdown_connections 2024-03-09T00:03:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 1 -- 172.21.15.22:0/1695163557 wait complete. 2024-03-09T00:03:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.325+0000 7ff4a1764740 20 client.27820 trim_cache size 0 max 16384 2024-03-09T00:03:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.338+0000 7ff4a1764740 1 Processor -- start 2024-03-09T00:03:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.338+0000 7ff4a1764740 1 -- start start 2024-03-09T00:03:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d75c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d71dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d72310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4a1764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981b2cc0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4a1764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981b0860 con 0x55dbd1d6dd40 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4a1764740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981ae920 con 0x55dbd1d76550 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d75c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d75c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44326/0 (socket says 172.21.15.22:44326) 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 -- 172.21.15.22:0/1942738811 learned_addr learned my addr 172.21.15.22:0/1942738811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49d041640 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d72310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff497fff640 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d71dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 -- 172.21.15.22:0/1942738811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 msgr2=0x55dbd1d72310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d72310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 -- 172.21.15.22:0/1942738811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 msgr2=0x55dbd1d71dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d71dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 -- 172.21.15.22:0/1942738811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd1d6c2a0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff497fff640 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d71dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:33.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff49c840640 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d75c30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff48c097730 tx=0x7ff48c004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a031de9bd503a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4827fc640 1 -- 172.21.15.22:0/1942738811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0a3500 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4827fc640 1 -- 172.21.15.22:0/1942738811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff48c09fdd0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.339+0000 7ff4827fc640 1 -- 172.21.15.22:0/1942738811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0cc2e0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 -- 172.21.15.22:0/1942738811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 msgr2=0x55dbd1d75c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d75c30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff48c097730 tx=0x7ff48c004560 comp rx=0 tx=0).stop 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 -- 172.21.15.22:0/1942738811 shutdown_connections 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d76550 0x55dbd1d72310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d71dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1942738811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d75c30 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 -- 172.21.15.22:0/1942738811 >> 172.21.15.22:0/1942738811 conn(0x55dbd1d345b0 msgr2=0x55dbd1d6e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 -- 172.21.15.22:0/1942738811 shutdown_connections 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 -- 172.21.15.22:0/1942738811 wait complete. 2024-03-09T00:03:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 -- start start 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.340+0000 7ff4a1764740 1 Processor -- start 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d7b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d7bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff4a1764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d76550 0x55dbd1dd5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff4a1764740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981aef20 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff4a1764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981b0200 con 0x55dbd1d76550 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff4a1764740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981af540 con 0x55dbd1d6dd40 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff4a1764740 10 client.0 _ll_register_callbacks cb 0x7ff490037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d7b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d7b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44332/0 (socket says 172.21.15.22:44332) 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 -- 172.21.15.22:0/1051048800 learned_addr learned my addr 172.21.15.22:0/1051048800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff497fff640 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d7bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49d041640 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d76550 0x55dbd1dd5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 -- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6dd40 msgr2=0x55dbd1d7bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d7bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 -- 172.21.15.22:0/1051048800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d76550 msgr2=0x55dbd1dd5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d76550 0x55dbd1dd5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.341+0000 7ff49c840640 1 -- 172.21.15.22:0/1051048800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbd1dd6590 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff49c840640 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d7b9b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0045f0 tx=0x7ff48c09b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ffc01ecf3dc03b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff477fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff477fff640 1 -- 172.21.15.22:0/1051048800 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c09a6c0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff477fff640 1 -- 172.21.15.22:0/1051048800 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff48c0b94c0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff48c09fdd0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff477fff640 1 -- 172.21.15.22:0/1051048800 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff48c0cc2e0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 10 client.18387 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55dbd1dd6590 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff477fff640 1 -- 172.21.15.22:0/1051048800 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff48c09fdd0 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff477fff640 1 -- 172.21.15.22:0/1051048800 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff48c080070 con 0x55dbd1d6cfd0 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 10 client.18387 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 10 client.18387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 20 client.18387 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1dddb90 0x55dbd1ddff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 4 client.18387 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dbd1cd8e80 con 0x55dbd1dddb90 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.342+0000 7ff497fff640 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1dddb90 0x55dbd1ddff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.343+0000 7ff497fff640 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1dddb90 0x55dbd1ddff80 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a20b32689213b8d server_cookie=a7e4047fb63944c2 in_seq=0 out_seq=0 2024-03-09T00:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.343+0000 7ff477fff640 10 client.18387 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:33.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 932659 10.24 MB/sec execute 341 sec latency 267.219 ms 2024-03-09T00:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.893+0000 7ff477fff640 1 -- 172.21.15.22:0/1051048800 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dbd1cd8e80 con 0x55dbd1dddb90 2024-03-09T00:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.893+0000 7ff477fff640 10 client.18387 handle_command_reply: tid=0 2024-03-09T00:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.893+0000 7ff4a1764740 1 client.18387 shutdown 2024-03-09T00:03:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 msgr2=0x55dbd1d7b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d7b9b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0045f0 tx=0x7ff48c09b590 comp rx=0 tx=0).stop 2024-03-09T00:03:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 shutdown_connections 2024-03-09T00:03:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dbd1dddb90 0x55dbd1ddff80 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbd1d76550 0x55dbd1dd5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbd1d6dd40 0x55dbd1d7bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff477fff640 0 client.18387 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 --2- 172.21.15.22:0/1051048800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbd1d6cfd0 0x55dbd1d7b9b0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 >> 172.21.15.22:0/1051048800 conn(0x55dbd1d345b0 msgr2=0x55dbd1d71d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 shutdown_connections 2024-03-09T00:03:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 1 -- 172.21.15.22:0/1051048800 wait complete. 2024-03-09T00:03:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.894+0000 7ff4a1764740 20 client.18387 trim_cache size 0 max 16384 2024-03-09T00:03:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:03:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:03:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:03:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:03:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.895+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4880a2a90 msgr2=0x7ff4880a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.895+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4880a2a90 0x7ff4880a4f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff498102410 tx=0x7ff484005f90 comp rx=0 tx=0).stop 2024-03-09T00:03:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.895+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498068970 msgr2=0x7ff49819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.895+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498068970 0x7ff49819f850 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00d950 tx=0x7ff48c00de10 comp rx=0 tx=0).stop 2024-03-09T00:03:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 shutdown_connections 2024-03-09T00:03:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff4880a2a90 0x7ff4880a4f50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff498107400 0x7ff498199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff498069310 0x7ff49819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 --2- 172.21.15.22:0/1035254700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498068970 0x7ff49819f850 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 >> 172.21.15.22:0/1035254700 conn(0x7ff498077040 msgr2=0x7ff49810c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 shutdown_connections 2024-03-09T00:03:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:33.896+0000 7ff49eaa4640 1 -- 172.21.15.22:0/1035254700 wait complete. 2024-03-09T00:03:34.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:34 smithi062 ceph-mon[36129]: pgmap v285: 129 pgs: 129 active+clean; 81 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-03-09T00:03:34.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:34 smithi192 ceph-mon[35995]: pgmap v285: 129 pgs: 129 active+clean; 81 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-03-09T00:03:34.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:34 smithi022 ceph-mon[31894]: pgmap v285: 129 pgs: 129 active+clean; 81 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-03-09T00:03:34.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 934707 10.25 MB/sec execute 342 sec latency 437.302 ms 2024-03-09T00:03:35.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 936081 10.23 MB/sec execute 343 sec latency 29.725 ms 2024-03-09T00:03:36.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:03:36.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 -- 172.21.15.22:0/283254337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041054a0 msgr2=0x7f660410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 --2- 172.21.15.22:0/283254337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041054a0 0x7f660410d070 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009ca0 tx=0x7f65fc02c0f0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 -- 172.21.15.22:0/283254337 shutdown_connections 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 --2- 172.21.15.22:0/283254337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041054a0 0x7f660410d070 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 --2- 172.21.15.22:0/283254337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6604100d80 0x7f6604104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 --2- 172.21.15.22:0/283254337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66041003d0 0x7f66041007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 -- 172.21.15.22:0/283254337 >> 172.21.15.22:0/283254337 conn(0x7f66040772f0 msgr2=0x7f6604077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 -- 172.21.15.22:0/283254337 shutdown_connections 2024-03-09T00:03:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.144+0000 7f660c6a4640 1 -- 172.21.15.22:0/283254337 wait complete. 2024-03-09T00:03:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 Processor -- start 2024-03-09T00:03:36.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 -- start start 2024-03-09T00:03:36.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041003d0 0x7f660419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66041054a0 0x7f660419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6604196880 0x7f6604196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6604110540 con 0x7f66041003d0 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041103c0 con 0x7f66041054a0 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660c6a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041106c0 con 0x7f6604196880 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660a440640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041003d0 0x7f660419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660a440640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041003d0 0x7f660419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44352/0 (socket says 172.21.15.22:44352) 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f660a440640 1 -- 172.21.15.22:0/3313505236 learned_addr learned my addr 172.21.15.22:0/3313505236 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.145+0000 7f6609c3f640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66041054a0 0x7f660419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660a440640 1 -- 172.21.15.22:0/3313505236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6604196880 msgr2=0x7f6604196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660a440640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6604196880 0x7f6604196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660a440640 1 -- 172.21.15.22:0/3313505236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66041054a0 msgr2=0x7f660419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660a440640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66041054a0 0x7f660419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660a440640 1 -- 172.21.15.22:0/3313505236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604197410 con 0x7f66041003d0 2024-03-09T00:03:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660a440640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041003d0 0x7f660419c6c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f660000d950 tx=0x7f660000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f65fb7fe640 1 -- 172.21.15.22:0/3313505236 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600017070 con 0x7f66041003d0 2024-03-09T00:03:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6604197700 con 0x7f66041003d0 2024-03-09T00:03:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f65fb7fe640 1 -- 172.21.15.22:0/3313505236 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6600009c30 con 0x7f66041003d0 2024-03-09T00:03:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f65fb7fe640 1 -- 172.21.15.22:0/3313505236 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000f6b0 con 0x7f66041003d0 2024-03-09T00:03:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.146+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6604068630 con 0x7f66041003d0 2024-03-09T00:03:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.148+0000 7f65fb7fe640 1 -- 172.21.15.22:0/3313505236 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f66000109d0 con 0x7f66041003d0 2024-03-09T00:03:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 Processor -- start 2024-03-09T00:03:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 -- start start 2024-03-09T00:03:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f65fb7fe640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65f00a4b80 0x7f65f00a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d49087bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087c080 0x55d490884720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d490887050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4907dee00 con 0x55d49087c080 2024-03-09T00:03:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d490787b10 con 0x55d490884c60 2024-03-09T00:03:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660f363740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4907daea0 con 0x55d49087b760 2024-03-09T00:03:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f660ac41640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65f00a4b80 0x7f65f00a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.149+0000 7f6609c3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087c080 0x55d490884720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f660a440640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d49087bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087c080 0x55d490884720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44356/0 (socket says 172.21.15.22:44356) 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 -- 172.21.15.22:0/3029932821 learned_addr learned my addr 172.21.15.22:0/3029932821 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f65fb7fe640 1 -- 172.21.15.22:0/3313505236 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f66000ceee0 con 0x7f66041003d0 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f660ac41640 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d490887050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 -- 172.21.15.22:0/3029932821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 msgr2=0x55d49087bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d49087bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 -- 172.21.15.22:0/3029932821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 msgr2=0x55d490887050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d490887050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 -- 172.21.15.22:0/3029932821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4905bd7d0 con 0x55d49087c080 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f660a440640 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d49087bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f660ac41640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65f00a4b80 0x7f65f00a7040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65fc02c630 tx=0x7f65fc032430 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f6609c3f640 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087c080 0x55d490884720 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f65f400bbb0 tx=0x7f65f40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff34012f61b0363 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f65f8ff9640 1 -- 172.21.15.22:0/3029932821 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4004030 con 0x55d49087c080 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f65f8ff9640 1 -- 172.21.15.22:0/3029932821 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65f4005500 con 0x55d49087c080 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.150+0000 7f65f8ff9640 1 -- 172.21.15.22:0/3029932821 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4004c40 con 0x55d49087c080 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 -- 172.21.15.22:0/3029932821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087c080 msgr2=0x55d490884720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087c080 0x55d490884720 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f65f400bbb0 tx=0x7f65f40107a0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 -- 172.21.15.22:0/3029932821 shutdown_connections 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d490887050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087c080 0x55d490884720 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 --2- 172.21.15.22:0/3029932821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d49087bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 -- 172.21.15.22:0/3029932821 >> 172.21.15.22:0/3029932821 conn(0x55d490875e90 msgr2=0x55d490876270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 -- 172.21.15.22:0/3029932821 shutdown_connections 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.151+0000 7f660f363740 1 -- 172.21.15.22:0/3029932821 wait complete. 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 -- start start 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 Processor -- start 2024-03-09T00:03:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087b760 0x55d4908843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d490881730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d490881c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d490887a50 con 0x55d490884c60 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49079b0a0 con 0x55d49087b760 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d490887bd0 con 0x55d49087c080 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660f363740 10 client.0 _ll_register_callbacks cb 0x55d49079aa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d490881c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d490881c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44360/0 (socket says 172.21.15.22:44360) 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 -- 172.21.15.22:0/2848621450 learned_addr learned my addr 172.21.15.22:0/2848621450 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f6609c3f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d490881730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660a440640 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087b760 0x55d4908843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 -- 172.21.15.22:0/2848621450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 msgr2=0x55d490881730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d490881730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 -- 172.21.15.22:0/2848621450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087b760 msgr2=0x55d4908843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087b760 0x55d4908843a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.152+0000 7f660ac41640 1 -- 172.21.15.22:0/2848621450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4904f33f0 con 0x55d490884c60 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660ac41640 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d490881c70 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f65fc02efd0 tx=0x7f65fc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f576c5b6bc0cbf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f65edffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f65edffb640 1 -- 172.21.15.22:0/2848621450 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc039040 con 0x55d490884c60 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f65edffb640 1 -- 172.21.15.22:0/2848621450 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f65fc04c640 con 0x55d490884c60 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65f4005500 con 0x55d490884c60 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f65edffb640 1 -- 172.21.15.22:0/2848621450 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65fc05a050 con 0x55d490884c60 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 10 client.18399 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d4904b30f0 con 0x55d490884c60 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f65edffb640 1 -- 172.21.15.22:0/2848621450 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65f4005500 con 0x55d490884c60 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f65edffb640 1 -- 172.21.15.22:0/2848621450 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f65fc034ad0 con 0x55d490884c60 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 10 client.18399 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 20 client.18399 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908946f0 0x55d490896b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 4 client.18399 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:36.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.153+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4907ec340 con 0x55d4908946f0 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.154+0000 7f660a440640 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908946f0 0x55d490896b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.154+0000 7f65edffb640 10 client.18399 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.154+0000 7f660a440640 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908946f0 0x55d490896b00 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f76e7e11108e49e server_cookie=c2c9b8fd87ff5c4 in_seq=0 out_seq=0 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f65edffb640 1 -- 172.21.15.22:0/2848621450 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d4907ec340 con 0x55d4908946f0 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f65edffb640 10 client.18399 handle_command_reply: tid=0 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 client.18399 shutdown 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 msgr2=0x55d490881c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d490881c70 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f65fc02efd0 tx=0x7f65fc005c50 comp rx=0 tx=0).stop 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 shutdown_connections 2024-03-09T00:03:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908946f0 0x55d490896b00 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d490881c70 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d490881730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 --2- 172.21.15.22:0/2848621450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087b760 0x55d4908843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f65edffb640 0 client.18399 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 >> 172.21.15.22:0/2848621450 conn(0x55d490875e90 msgr2=0x55d490877be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 shutdown_connections 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 1 -- 172.21.15.22:0/2848621450 wait complete. 2024-03-09T00:03:36.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.155+0000 7f660f363740 20 client.18399 trim_cache size 0 max 16384 2024-03-09T00:03:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 Processor -- start 2024-03-09T00:03:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 -- start start 2024-03-09T00:03:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d490883e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087c080 0x55d4908843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d49088be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041ae960 con 0x55d490884c60 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041ac500 con 0x55d49087c080 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.170+0000 7f660f363740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041aa5c0 con 0x55d49087b760 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d49088be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d49088be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44368/0 (socket says 172.21.15.22:44368) 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 -- 172.21.15.22:0/488067609 learned_addr learned my addr 172.21.15.22:0/488067609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660a440640 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d490883e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f6609c3f640 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087c080 0x55d4908843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 -- 172.21.15.22:0/488067609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 msgr2=0x55d490883e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d490883e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 -- 172.21.15.22:0/488067609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087c080 msgr2=0x55d4908843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087c080 0x55d4908843a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 -- 172.21.15.22:0/488067609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d49089bcf0 con 0x55d490884c60 2024-03-09T00:03:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f660ac41640 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d49088be00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f65fc02f020 tx=0x7f65fc02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9704cd1d1c5b87d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f65effff640 1 -- 172.21.15.22:0/488067609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc039040 con 0x55d490884c60 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f65effff640 1 -- 172.21.15.22:0/488067609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65fc051070 con 0x55d490884c60 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.171+0000 7f65effff640 1 -- 172.21.15.22:0/488067609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0347a0 con 0x55d490884c60 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- 172.21.15.22:0/488067609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 msgr2=0x55d49088be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d49088be00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f65fc02f020 tx=0x7f65fc02c8b0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- 172.21.15.22:0/488067609 shutdown_connections 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d490884c60 0x55d49088be00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d49087c080 0x55d4908843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 --2- 172.21.15.22:0/488067609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087b760 0x55d490883e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- 172.21.15.22:0/488067609 >> 172.21.15.22:0/488067609 conn(0x55d490875e90 msgr2=0x55d4908816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- 172.21.15.22:0/488067609 shutdown_connections 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- 172.21.15.22:0/488067609 wait complete. 2024-03-09T00:03:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- start start 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 Processor -- start 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087b760 0x55d49088be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d4908883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d4908888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041aabc0 con 0x55d49087b760 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041abea0 con 0x55d490884c60 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.172+0000 7f660f363740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041ab1e0 con 0x55d49087c080 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660f363740 10 client.0 _ll_register_callbacks cb 0x7f65fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087b760 0x55d49088be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087b760 0x55d49088be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44378/0 (socket says 172.21.15.22:44378) 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 -- 172.21.15.22:0/945309109 learned_addr learned my addr 172.21.15.22:0/945309109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f6609c3f640 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d4908883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660ac41640 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d4908888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 -- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 msgr2=0x55d4908883a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d4908883a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 -- 172.21.15.22:0/945309109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 msgr2=0x55d4908888e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d4908888e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 -- 172.21.15.22:0/945309109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d490888e20 con 0x55d49087b760 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660a440640 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087b760 0x55d49088be80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6600097b10 tx=0x7f6600005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c14f5277783cb07 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f65edffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f65edffb640 1 -- 172.21.15.22:0/945309109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600020040 con 0x55d49087b760 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f65edffb640 1 -- 172.21.15.22:0/945309109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f66000bc430 con 0x55d49087b760 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65fc051070 con 0x55d49087b760 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f65edffb640 1 -- 172.21.15.22:0/945309109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660009b140 con 0x55d49087b760 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660f363740 10 client.18405 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d4908eb960 con 0x55d49087b760 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.173+0000 7f65edffb640 1 -- 172.21.15.22:0/945309109 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65fc051070 con 0x55d49087b760 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f65edffb640 1 -- 172.21.15.22:0/945309109 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f66000b9050 con 0x55d49087b760 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f660f363740 10 client.18405 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f660f363740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f660f363740 20 client.18405 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f660f363740 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908ebfc0 0x55d4908ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f660f363740 4 client.18405 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:03:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4907e6690 con 0x55d4908ebfc0 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f6609c3f640 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908ebfc0 0x55d4908ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f65edffb640 10 client.18405 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.174+0000 7f6609c3f640 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908ebfc0 0x55d4908ee3e0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6735aabecbe488ad server_cookie=b7d3605a4d6af369 in_seq=0 out_seq=0 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f65edffb640 1 -- 172.21.15.22:0/945309109 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d4907e6690 con 0x55d4908ebfc0 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f65edffb640 10 client.18405 handle_command_reply: tid=0 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 client.18405 shutdown 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087b760 msgr2=0x55d49088be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087b760 0x55d49088be80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6600097b10 tx=0x7f6600005880 comp rx=0 tx=0).stop 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 shutdown_connections 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d4908ebfc0 0x55d4908ee3e0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d490884c60 0x55d4908888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49087c080 0x55d4908883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f65edffb640 0 client.18405 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 --2- 172.21.15.22:0/945309109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d49087b760 0x55d49088be80 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 >> 172.21.15.22:0/945309109 conn(0x55d490875e90 msgr2=0x55d49087ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 shutdown_connections 2024-03-09T00:03:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 1 -- 172.21.15.22:0/945309109 wait complete. 2024-03-09T00:03:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.175+0000 7f660f363740 20 client.18405 trim_cache size 0 max 16384 2024-03-09T00:03:36.179 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:03:36.179 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:03:36.179 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:03:36.179 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:03:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65f00a4b80 msgr2=0x7f65f00a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65f00a4b80 0x7f65f00a7040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65fc02c630 tx=0x7f65fc032430 comp rx=0 tx=0).stop 2024-03-09T00:03:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041003d0 msgr2=0x7f660419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041003d0 0x7f660419c6c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f660000d950 tx=0x7f660000de10 comp rx=0 tx=0).stop 2024-03-09T00:03:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 shutdown_connections 2024-03-09T00:03:36.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65f00a4b80 0x7f65f00a7040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6604196880 0x7f6604196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66041054a0 0x7f660419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 --2- 172.21.15.22:0/3313505236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66041003d0 0x7f660419c6c0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 >> 172.21.15.22:0/3313505236 conn(0x7f66040772f0 msgr2=0x7f66040fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 shutdown_connections 2024-03-09T00:03:36.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.176+0000 7f660c6a4640 1 -- 172.21.15.22:0/3313505236 wait complete. 2024-03-09T00:03:36.182 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:03:36.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:03:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 -- 172.21.15.22:0/496019507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 msgr2=0x7f1270100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 --2- 172.21.15.22:0/496019507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f1270100840 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1260009900 tx=0x7f126002bc30 comp rx=0 tx=0).stop 2024-03-09T00:03:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 -- 172.21.15.22:0/496019507 shutdown_connections 2024-03-09T00:03:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 --2- 172.21.15.22:0/496019507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1270105530 0x7f127010d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 --2- 172.21.15.22:0/496019507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1270100e10 0x7f1270104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 --2- 172.21.15.22:0/496019507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f1270100840 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 -- 172.21.15.22:0/496019507 >> 172.21.15.22:0/496019507 conn(0x7f12700772f0 msgr2=0x7f1270077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 -- 172.21.15.22:0/496019507 shutdown_connections 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 -- 172.21.15.22:0/496019507 wait complete. 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 Processor -- start 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.299+0000 7f1274b53640 1 -- start start 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f1274b53640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f127019d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f1274b53640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1270105530 0x7f127019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f1274b53640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1270197a90 0x7f1270197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f1274b53640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701105c0 con 0x7f1270100460 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f1274b53640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270110440 con 0x7f1270105530 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f1274b53640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270110740 con 0x7f1270197a90 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f127019d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f127019d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44388/0 (socket says 172.21.15.22:44388) 2024-03-09T00:03:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 -- 172.21.15.22:0/2340967624 learned_addr learned my addr 172.21.15.22:0/2340967624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f1267fff640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1270105530 0x7f127019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126ed9d640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1270197a90 0x7f1270197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 -- 172.21.15.22:0/2340967624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1270197a90 msgr2=0x7f1270197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1270197a90 0x7f1270197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 -- 172.21.15.22:0/2340967624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1270105530 msgr2=0x7f127019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1270105530 0x7f127019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 -- 172.21.15.22:0/2340967624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12701985b0 con 0x7f1270100460 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f126e59c640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f127019d8e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1260006010 tx=0x7f126002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f12677fe640 1 -- 172.21.15.22:0/2340967624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260041070 con 0x7f1270100460 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.300+0000 7f12677fe640 1 -- 172.21.15.22:0/2340967624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1260004450 con 0x7f1270100460 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.301+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1270198840 con 0x7f1270100460 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.301+0000 7f12677fe640 1 -- 172.21.15.22:0/2340967624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260003e00 con 0x7f1270100460 2024-03-09T00:03:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.301+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12701a4a30 con 0x7f1270100460 2024-03-09T00:03:36.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.301+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12700680e0 con 0x7f1270100460 2024-03-09T00:03:36.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.303+0000 7f12677fe640 1 -- 172.21.15.22:0/2340967624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1260044070 con 0x7f1270100460 2024-03-09T00:03:36.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.303+0000 7f12677fe640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f125c0a2970 0x7f125c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.303+0000 7f1267fff640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f125c0a2970 0x7f125c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.303+0000 7f12677fe640 1 -- 172.21.15.22:0/2340967624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f12600f0e40 con 0x7f1270100460 2024-03-09T00:03:36.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.304+0000 7f1267fff640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f125c0a2970 0x7f125c0a4e30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1270068d40 tx=0x7f125800b410 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.307+0000 7f12677fe640 1 -- 172.21.15.22:0/2340967624 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f12600bd260 con 0x7f1270100460 2024-03-09T00:03:36.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:36 smithi062 ceph-mon[36129]: pgmap v286: 129 pgs: 129 active+clean; 72 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-03-09T00:03:36.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:36 smithi192 ceph-mon[35995]: pgmap v286: 129 pgs: 129 active+clean; 72 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-03-09T00:03:36.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:36 smithi022 ceph-mon[31894]: pgmap v286: 129 pgs: 129 active+clean; 72 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-03-09T00:03:36.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.530+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1270109770 con 0x7f1270100460 2024-03-09T00:03:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.530+0000 7f12677fe640 1 -- 172.21.15.22:0/2340967624 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f12600c1300 con 0x7f1270100460 2024-03-09T00:03:36.526 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:03:36.526 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:03:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:03:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f125c0a2970 msgr2=0x7f125c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f125c0a2970 0x7f125c0a4e30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1270068d40 tx=0x7f125800b410 comp rx=0 tx=0).stop 2024-03-09T00:03:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 msgr2=0x7f127019d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f127019d8e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1260006010 tx=0x7f126002cb30 comp rx=0 tx=0).stop 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 shutdown_connections 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f125c0a2970 0x7f125c0a4e30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1270197a90 0x7f1270197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1270105530 0x7f127019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 --2- 172.21.15.22:0/2340967624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1270100460 0x7f127019d8e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 >> 172.21.15.22:0/2340967624 conn(0x7f12700772f0 msgr2=0x7f12700ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 shutdown_connections 2024-03-09T00:03:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.533+0000 7f1274b53640 1 -- 172.21.15.22:0/2340967624 wait complete. 2024-03-09T00:03:36.539 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:03:36.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:03:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 -- 172.21.15.22:0/1115332290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c106220 msgr2=0x7f326c106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 --2- 172.21.15.22:0/1115332290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c106220 0x7f326c106600 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f325c009900 tx=0x7f325c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:03:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 -- 172.21.15.22:0/1115332290 shutdown_connections 2024-03-09T00:03:36.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 --2- 172.21.15.22:0/1115332290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c100de0 0x7f326c10cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 --2- 172.21.15.22:0/1115332290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c100440 0x7f326c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 --2- 172.21.15.22:0/1115332290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c106220 0x7f326c106600 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 -- 172.21.15.22:0/1115332290 >> 172.21.15.22:0/1115332290 conn(0x7f326c076f90 msgr2=0x7f326c0773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 -- 172.21.15.22:0/1115332290 shutdown_connections 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 -- 172.21.15.22:0/1115332290 wait complete. 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 Processor -- start 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.652+0000 7f32724a4640 1 -- start start 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32724a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c100440 0x7f326c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32724a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c100de0 0x7f326c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32724a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c106220 0x7f326c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32724a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c110570 con 0x7f326c100440 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32724a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1103f0 con 0x7f326c106220 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32724a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1106f0 con 0x7f326c100de0 2024-03-09T00:03:36.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c100440 0x7f326c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c100440 0x7f326c19d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44414/0 (socket says 172.21.15.22:44414) 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c100de0 0x7f326c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 -- 172.21.15.22:0/1779756262 learned_addr learned my addr 172.21.15.22:0/1779756262 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f3270a41640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c106220 0x7f326c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 -- 172.21.15.22:0/1779756262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c100de0 msgr2=0x7f326c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c100de0 0x7f326c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 -- 172.21.15.22:0/1779756262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c106220 msgr2=0x7f326c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c106220 0x7f326c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 -- 172.21.15.22:0/1779756262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c1980c0 con 0x7f326c100440 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f3270a41640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c106220 0x7f326c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f326bfff640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c100440 0x7f326c19d740 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f325c02c240 tx=0x7f325c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32697fa640 1 -- 172.21.15.22:0/1779756262 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c043020 con 0x7f326c100440 2024-03-09T00:03:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c198350 con 0x7f326c100440 2024-03-09T00:03:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32697fa640 1 -- 172.21.15.22:0/1779756262 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f325c0054e0 con 0x7f326c100440 2024-03-09T00:03:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.653+0000 7f32697fa640 1 -- 172.21.15.22:0/1779756262 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c003b30 con 0x7f326c100440 2024-03-09T00:03:36.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.654+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c1a4600 con 0x7f326c100440 2024-03-09T00:03:36.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f32697fa640 1 -- 172.21.15.22:0/1779756262 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f325c046020 con 0x7f326c100440 2024-03-09T00:03:36.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 Processor -- start 2024-03-09T00:03:36.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 -- start start 2024-03-09T00:03:36.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e38160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e3aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb8df6360 con 0x55efb8e2fac0 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb8d92ec0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3275165740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb8d3bd30 con 0x55efb8e386a0 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f326bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e38160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e3aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e3aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:55440/0 (socket says 172.21.15.22:55440) 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f32697fa640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32540a2970 0x7f32540a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f326bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e38160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44420/0 (socket says 172.21.15.22:44420) 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f326b7fe640 1 -- 172.21.15.22:0/1013005561 learned_addr learned my addr 172.21.15.22:0/1013005561 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.656+0000 7f3270a41640 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f3270a41640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32540a2970 0x7f32540a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f326b7fe640 1 -- 172.21.15.22:0/1013005561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 msgr2=0x55efb8e2f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f326b7fe640 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e2f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f32697fa640 1 -- 172.21.15.22:0/1779756262 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f325c0f1160 con 0x7f326c100440 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f326b7fe640 1 -- 172.21.15.22:0/1013005561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 msgr2=0x55efb8e38160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f326b7fe640 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e38160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f326b7fe640 1 -- 172.21.15.22:0/1013005561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb8d3abc0 con 0x55efb8e386a0 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f326b7fe640 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e3aa90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3260009c60 tx=0x7f3260011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9271cc321064fe36 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f3270a41640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32540a2970 0x7f32540a4e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3264009370 tx=0x7f3264009240 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f32527fc640 1 -- 172.21.15.22:0/1013005561 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260004230 con 0x55efb8e386a0 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f32527fc640 1 -- 172.21.15.22:0/1013005561 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32600043d0 con 0x55efb8e386a0 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.657+0000 7f32527fc640 1 -- 172.21.15.22:0/1013005561 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260004d90 con 0x55efb8e386a0 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 -- 172.21.15.22:0/1013005561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 msgr2=0x55efb8e3aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e3aa90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3260009c60 tx=0x7f3260011810 comp rx=0 tx=0).stop 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 -- 172.21.15.22:0/1013005561 shutdown_connections 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e3aa90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e38160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 --2- 172.21.15.22:0/1013005561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e2f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 -- 172.21.15.22:0/1013005561 >> 172.21.15.22:0/1013005561 conn(0x55efb8e298d0 msgr2=0x55efb8e29cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 -- 172.21.15.22:0/1013005561 shutdown_connections 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 -- 172.21.15.22:0/1013005561 wait complete. 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 -- start start 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.658+0000 7f3275165740 1 Processor -- start 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2f1a0 0x55efb8e356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2fac0 0x55efb8e35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e386a0 0x55efb8e36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb8d314a0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb8d8ef60 con 0x55efb8e386a0 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb8e3b490 con 0x55efb8e2fac0 2024-03-09T00:03:36.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 10 client.0 _ll_register_callbacks cb 0x55efb8d8cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2f1a0 0x55efb8e356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2f1a0 0x55efb8e356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44436/0 (socket says 172.21.15.22:44436) 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2fac0 0x55efb8e35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 -- 172.21.15.22:0/4057589393 learned_addr learned my addr 172.21.15.22:0/4057589393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3270a41640 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e386a0 0x55efb8e36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 -- 172.21.15.22:0/4057589393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2fac0 msgr2=0x55efb8e35c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2fac0 0x55efb8e35c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 -- 172.21.15.22:0/4057589393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e386a0 msgr2=0x55efb8e36150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e386a0 0x55efb8e36150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 -- 172.21.15.22:0/4057589393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efb8aa73f0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f326bfff640 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2f1a0 0x55efb8e356d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f325c0dbfd0 tx=0x7f325c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4185e2bc76abf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f324b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f324b7fe640 1 -- 172.21.15.22:0/4057589393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c004c80 con 0x55efb8e2f1a0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f324b7fe640 1 -- 172.21.15.22:0/4057589393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f325c02cb20 con 0x55efb8e2f1a0 2024-03-09T00:03:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.659+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32600043d0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f324b7fe640 1 -- 172.21.15.22:0/4057589393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325c005690 con 0x55efb8e2f1a0 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 10 client.18423 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55efb8b717d0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f324b7fe640 1 -- 172.21.15.22:0/4057589393 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32600043d0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f324b7fe640 1 -- 172.21.15.22:0/4057589393 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f325c0c57e0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 10 client.18423 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 10 client.18423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 20 client.18423 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8e48840 0x55efb8e4ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 4 client.18423 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efb8da0400 con 0x55efb8e48840 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.660+0000 7f326b7fe640 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8e48840 0x55efb8e4ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f324b7fe640 10 client.18423 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f326b7fe640 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8e48840 0x55efb8e4ac70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=541ed51997f702c1 server_cookie=ee8e3e8f3f25092b in_seq=0 out_seq=0 2024-03-09T00:03:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f324b7fe640 1 -- 172.21.15.22:0/4057589393 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55efb8da0400 con 0x55efb8e48840 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f324b7fe640 10 client.18423 handle_command_reply: tid=0 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 client.18423 shutdown 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2f1a0 msgr2=0x55efb8e356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2f1a0 0x55efb8e356d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f325c0dbfd0 tx=0x7f325c009620 comp rx=0 tx=0).stop 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 shutdown_connections 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8e48840 0x55efb8e4ac70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e386a0 0x55efb8e36150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2fac0 0x55efb8e35c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 --2- 172.21.15.22:0/4057589393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2f1a0 0x55efb8e356d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f324b7fe640 0 client.18423 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:36.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 >> 172.21.15.22:0/4057589393 conn(0x55efb8e298d0 msgr2=0x55efb8e2b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 shutdown_connections 2024-03-09T00:03:36.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.661+0000 7f3275165740 1 -- 172.21.15.22:0/4057589393 wait complete. 2024-03-09T00:03:36.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.662+0000 7f3275165740 20 client.18423 trim_cache size 0 max 16384 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 Processor -- start 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 -- start start 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e37e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e32930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e32e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1b0b00 con 0x55efb8e2fac0 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1ae6a0 con 0x55efb8e2f1a0 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3275165740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1ac760 con 0x55efb8e386a0 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e32930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e32930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44438/0 (socket says 172.21.15.22:44438) 2024-03-09T00:03:36.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 -- 172.21.15.22:0/3362208719 learned_addr learned my addr 172.21.15.22:0/3362208719 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f3270a41640 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e32e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326bfff640 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e37e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 -- 172.21.15.22:0/3362208719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 msgr2=0x55efb8e32e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e32e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 -- 172.21.15.22:0/3362208719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 msgr2=0x55efb8e37e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e37e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.674+0000 7f326b7fe640 1 -- 172.21.15.22:0/3362208719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb8e3aa90 con 0x55efb8e2fac0 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f326b7fe640 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e32930 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x55efb8e2aa60 tx=0x7f3260004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca028539aeeb13d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3251ffb640 1 -- 172.21.15.22:0/3362208719 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326001e040 con 0x55efb8e2fac0 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3251ffb640 1 -- 172.21.15.22:0/3362208719 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f326001f040 con 0x55efb8e2fac0 2024-03-09T00:03:36.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3251ffb640 1 -- 172.21.15.22:0/3362208719 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260026a00 con 0x55efb8e2fac0 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 -- 172.21.15.22:0/3362208719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 msgr2=0x55efb8e32930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e32930 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x55efb8e2aa60 tx=0x7f3260004a10 comp rx=0 tx=0).stop 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 -- 172.21.15.22:0/3362208719 shutdown_connections 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e386a0 0x55efb8e32e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e2fac0 0x55efb8e32930 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 --2- 172.21.15.22:0/3362208719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2f1a0 0x55efb8e37e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 -- 172.21.15.22:0/3362208719 >> 172.21.15.22:0/3362208719 conn(0x55efb8e298d0 msgr2=0x55efb8e309e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 -- 172.21.15.22:0/3362208719 shutdown_connections 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.675+0000 7f3275165740 1 -- 172.21.15.22:0/3362208719 wait complete. 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 -- start start 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 Processor -- start 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2f1a0 0x55efb8e3f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2fac0 0x55efb8e3f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e386a0 0x55efb8e3d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1acd60 con 0x55efb8e386a0 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1ae040 con 0x55efb8e2fac0 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1ad380 con 0x55efb8e2f1a0 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3275165740 10 client.0 _ll_register_callbacks cb 0x7f32600097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3270a41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e386a0 0x55efb8e3d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f326bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2f1a0 0x55efb8e3f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3270a41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e386a0 0x55efb8e3d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44448/0 (socket says 172.21.15.22:44448) 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3270a41640 1 -- 172.21.15.22:0/2828793846 learned_addr learned my addr 172.21.15.22:0/2828793846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f326b7fe640 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2fac0 0x55efb8e3f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.676+0000 7f3270a41640 1 -- 172.21.15.22:0/2828793846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2f1a0 msgr2=0x55efb8e3f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3270a41640 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2f1a0 0x55efb8e3f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3270a41640 1 -- 172.21.15.22:0/2828793846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2fac0 msgr2=0x55efb8e3f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3270a41640 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2fac0 0x55efb8e3f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3270a41640 1 -- 172.21.15.22:0/2828793846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efb8e3e0a0 con 0x55efb8e386a0 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f326b7fe640 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2fac0 0x55efb8e3f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3270a41640 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e386a0 0x55efb8e3d9b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3264011c00 tx=0x7f3264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f24323f462dc8ec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f324b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f324b7fe640 1 -- 172.21.15.22:0/2828793846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264004b20 con 0x55efb8e386a0 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f324b7fe640 1 -- 172.21.15.22:0/2828793846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3264004cc0 con 0x55efb8e386a0 2024-03-09T00:03:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f326001f040 con 0x55efb8e386a0 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f324b7fe640 1 -- 172.21.15.22:0/2828793846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3264036050 con 0x55efb8e386a0 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 10 client.18429 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55efb8ea8130 con 0x55efb8e386a0 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f324b7fe640 1 -- 172.21.15.22:0/2828793846 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f326001f040 con 0x55efb8e386a0 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f324b7fe640 1 -- 172.21.15.22:0/2828793846 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3264020040 con 0x55efb8e386a0 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 10 client.18429 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 10 client.18429 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 20 client.18429 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8ea88c0 0x55efb8eaace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 4 client.18429 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.677+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55efb8d9a750 con 0x55efb8ea88c0 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f326bfff640 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8ea88c0 0x55efb8eaace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f326bfff640 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8ea88c0 0x55efb8eaace0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=163d381fc16eca8 server_cookie=290752511b3debcc in_seq=0 out_seq=0 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f324b7fe640 10 client.18429 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f324b7fe640 1 -- 172.21.15.22:0/2828793846 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55efb8d9a750 con 0x55efb8ea88c0 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f324b7fe640 10 client.18429 handle_command_reply: tid=0 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 client.18429 shutdown 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e386a0 msgr2=0x55efb8e3d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e386a0 0x55efb8e3d9b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3264011c00 tx=0x7f3264004a40 comp rx=0 tx=0).stop 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 shutdown_connections 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55efb8ea88c0 0x55efb8eaace0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55efb8e386a0 0x55efb8e3d9b0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efb8e2fac0 0x55efb8e3f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f324b7fe640 0 client.18429 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 --2- 172.21.15.22:0/2828793846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55efb8e2f1a0 0x55efb8e3f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 >> 172.21.15.22:0/2828793846 conn(0x55efb8e298d0 msgr2=0x55efb8e33fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 shutdown_connections 2024-03-09T00:03:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.678+0000 7f3275165740 1 -- 172.21.15.22:0/2828793846 wait complete. 2024-03-09T00:03:36.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f3275165740 20 client.18429 trim_cache size 0 max 16384 2024-03-09T00:03:36.682 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:03:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32540a2970 msgr2=0x7f32540a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32540a2970 0x7f32540a4e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3264009370 tx=0x7f3264009240 comp rx=0 tx=0).stop 2024-03-09T00:03:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c100440 msgr2=0x7f326c19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c100440 0x7f326c19d740 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f325c02c240 tx=0x7f325c0042f0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 shutdown_connections 2024-03-09T00:03:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32540a2970 0x7f32540a4e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c106220 0x7f326c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c100de0 0x7f326c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 --2- 172.21.15.22:0/1779756262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f326c100440 0x7f326c19d740 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 >> 172.21.15.22:0/1779756262 conn(0x7f326c076f90 msgr2=0x7f326c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 shutdown_connections 2024-03-09T00:03:36.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.679+0000 7f32724a4640 1 -- 172.21.15.22:0/1779756262 wait complete. 2024-03-09T00:03:36.686 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:03:36.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:03:36.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 938416 10.24 MB/sec execute 344 sec latency 18.918 ms 2024-03-09T00:03:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.795+0000 7f3e540a1640 1 -- 172.21.15.22:0/2163460596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c102e30 msgr2=0x7f3e4c10b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.795+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2163460596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c102e30 0x7f3e4c10b310 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009900 tx=0x7f3e4002bcd0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- 172.21.15.22:0/2163460596 shutdown_connections 2024-03-09T00:03:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2163460596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e4c10b850 0x7f3e4c10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2163460596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c102e30 0x7f3e4c10b310 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2163460596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e4c102510 0x7f3e4c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- 172.21.15.22:0/2163460596 >> 172.21.15.22:0/2163460596 conn(0x7f3e4c0773a0 msgr2=0x7f3e4c0777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- 172.21.15.22:0/2163460596 shutdown_connections 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- 172.21.15.22:0/2163460596 wait complete. 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 Processor -- start 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- start start 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e4c102510 0x7f3e4c100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e4c102e30 0x7f3e4c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c10b850 0x7f3e4c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c111680 con 0x7f3e4c10b850 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c111500 con 0x7f3e4c102510 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.796+0000 7f3e540a1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c111800 con 0x7f3e4c102e30 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c10b850 0x7f3e4c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c10b850 0x7f3e4c100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44466/0 (socket says 172.21.15.22:44466) 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 -- 172.21.15.22:0/2546116682 learned_addr learned my addr 172.21.15.22:0/2546116682 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e51e3d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e4c102510 0x7f3e4c100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5163c640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e4c102e30 0x7f3e4c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 -- 172.21.15.22:0/2546116682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e4c102e30 msgr2=0x7f3e4c1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e4c102e30 0x7f3e4c1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 -- 172.21.15.22:0/2546116682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e4c102510 msgr2=0x7f3e4c100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e4c102510 0x7f3e4c100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 -- 172.21.15.22:0/2546116682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c1a6020 con 0x7f3e4c10b850 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e5263e640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c10b850 0x7f3e4c100be0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00e970 tx=0x7f3e3c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e4affd640 1 -- 172.21.15.22:0/2546116682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c00ccc0 con 0x7f3e4c10b850 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c1a6310 con 0x7f3e4c10b850 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e4affd640 1 -- 172.21.15.22:0/2546116682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c00ce60 con 0x7f3e4c10b850 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e4affd640 1 -- 172.21.15.22:0/2546116682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c010490 con 0x7f3e4c10b850 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.797+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c1a6830 con 0x7f3e4c10b850 2024-03-09T00:03:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e4affd640 1 -- 172.21.15.22:0/2546116682 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3e3c005020 con 0x7f3e4c10b850 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 Processor -- start 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 -- start start 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada951580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada95a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ada918360 con 0x556ada951ac0 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ada8b4ec0 con 0x556ada9511a0 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.800+0000 7f3e56922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ada85dd30 con 0x556ada95a6a0 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada95a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada95a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44468/0 (socket says 172.21.15.22:44468) 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 -- 172.21.15.22:0/3940595044 learned_addr learned my addr 172.21.15.22:0/3940595044 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e5163c640 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e4affd640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e340a2970 0x7f3e340a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e5163c640 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada951580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e5263e640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e340a2970 0x7f3e340a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 -- 172.21.15.22:0/3940595044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 msgr2=0x556ada95ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 -- 172.21.15.22:0/3940595044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 msgr2=0x556ada951580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada951580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 -- 172.21.15.22:0/3940595044 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ada85cbc0 con 0x556ada951ac0 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e4affd640 1 -- 172.21.15.22:0/2546116682 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3e3c0cf610 con 0x7f3e4c10b850 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada95a160 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3e440114f0 tx=0x7f3e440119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba584b850c54271 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e5263e640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e340a2970 0x7f3e340a4e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0cca10 tx=0x7f3e3c06a680 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e33fff640 1 -- 172.21.15.22:0/3940595044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4401e070 con 0x556ada951ac0 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e33fff640 1 -- 172.21.15.22:0/3940595044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4400fa80 con 0x556ada951ac0 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.801+0000 7f3e33fff640 1 -- 172.21.15.22:0/3940595044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44019b20 con 0x556ada951ac0 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 -- 172.21.15.22:0/3940595044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 msgr2=0x556ada95a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada95a160 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3e440114f0 tx=0x7f3e440119b0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 -- 172.21.15.22:0/3940595044 shutdown_connections 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada95a160 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 --2- 172.21.15.22:0/3940595044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada951580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 -- 172.21.15.22:0/3940595044 >> 172.21.15.22:0/3940595044 conn(0x556ada94b8d0 msgr2=0x556ada94bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 -- 172.21.15.22:0/3940595044 shutdown_connections 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.802+0000 7f3e56922740 1 -- 172.21.15.22:0/3940595044 wait complete. 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 -- start start 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 Processor -- start 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada9511a0 0x556ada9576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada951ac0 0x556ada957c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada95a6a0 0x556ada958170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ada8534a0 con 0x556ada9511a0 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ada8b0f60 con 0x556ada95a6a0 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ada95d490 con 0x556ada951ac0 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e56922740 10 client.0 _ll_register_callbacks cb 0x556ada8aed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada9511a0 0x556ada9576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada9511a0 0x556ada9576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44478/0 (socket says 172.21.15.22:44478) 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e51e3d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada951ac0 0x556ada957c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 -- 172.21.15.22:0/2921558850 learned_addr learned my addr 172.21.15.22:0/2921558850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada95a6a0 0x556ada958170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 -- 172.21.15.22:0/2921558850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada951ac0 msgr2=0x556ada957c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada951ac0 0x556ada957c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 -- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada95a6a0 msgr2=0x556ada958170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.803+0000 7f3e5163c640 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada95a6a0 0x556ada958170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e5163c640 1 -- 172.21.15.22:0/2921558850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ada5c93f0 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada951ac0 0x556ada957c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e5163c640 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada9511a0 0x556ada9576f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3e40000c00 tx=0x7f3e40005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eca1f7d548ab403 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e317fa640 1 -- 172.21.15.22:0/2921558850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4002c920 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e317fa640 1 -- 172.21.15.22:0/2921558850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3e4002cac0 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e4400fa80 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e317fa640 1 -- 172.21.15.22:0/2921558850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e40056050 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 10 client.18441 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556ada6937d0 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e317fa640 1 -- 172.21.15.22:0/2921558850 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4400fa80 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e317fa640 1 -- 172.21.15.22:0/2921558850 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e4002e350 con 0x556ada9511a0 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 10 client.18441 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 10 client.18441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 20 client.18441 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada96a810 0x556ada96cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 4 client.18441 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.804+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ada8c2400 con 0x556ada96a810 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.805+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada96a810 0x556ada96cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.805+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada96a810 0x556ada96cc00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de553106dcba3592 server_cookie=102cb2100c09fade in_seq=0 out_seq=0 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.805+0000 7f3e317fa640 10 client.18441 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e317fa640 1 -- 172.21.15.22:0/2921558850 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ada8c2400 con 0x556ada96a810 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e317fa640 10 client.18441 handle_command_reply: tid=0 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 client.18441 shutdown 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada9511a0 msgr2=0x556ada9576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada9511a0 0x556ada9576f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3e40000c00 tx=0x7f3e40005a20 comp rx=0 tx=0).stop 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 shutdown_connections 2024-03-09T00:03:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada96a810 0x556ada96cc00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada95a6a0 0x556ada958170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada951ac0 0x556ada957c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 --2- 172.21.15.22:0/2921558850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada9511a0 0x556ada9576f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e317fa640 0 client.18441 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 >> 172.21.15.22:0/2921558850 conn(0x556ada94b8d0 msgr2=0x556ada94d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 shutdown_connections 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 1 -- 172.21.15.22:0/2921558850 wait complete. 2024-03-09T00:03:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.806+0000 7f3e56922740 20 client.18441 trim_cache size 0 max 16384 2024-03-09T00:03:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 Processor -- start 2024-03-09T00:03:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 -- start start 2024-03-09T00:03:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada9511a0 0x556ada959e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada951ac0 0x556ada9616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada95a6a0 0x556ada961c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1b2da0 con 0x556ada95a6a0 2024-03-09T00:03:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1b0940 con 0x556ada951ac0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e56922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1aea00 con 0x556ada9511a0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada95a6a0 0x556ada961c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e51e3d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada951ac0 0x556ada9616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada95a6a0 0x556ada961c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44494/0 (socket says 172.21.15.22:44494) 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 -- 172.21.15.22:0/1406151057 learned_addr learned my addr 172.21.15.22:0/1406151057 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada9511a0 0x556ada959e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 -- 172.21.15.22:0/1406151057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada9511a0 msgr2=0x556ada959e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada9511a0 0x556ada959e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 -- 172.21.15.22:0/1406151057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada951ac0 msgr2=0x556ada9616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada951ac0 0x556ada9616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 -- 172.21.15.22:0/1406151057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ada9518f0 con 0x556ada95a6a0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.818+0000 7f3e5163c640 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada95a6a0 0x556ada961c00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005ab0 tx=0x7f3e40004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2958524f25a6e58b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e337fe640 1 -- 172.21.15.22:0/1406151057 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4002e2f0 con 0x556ada95a6a0 2024-03-09T00:03:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e337fe640 1 -- 172.21.15.22:0/1406151057 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e400043e0 con 0x556ada95a6a0 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e337fe640 1 -- 172.21.15.22:0/1406151057 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004920 con 0x556ada95a6a0 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 -- 172.21.15.22:0/1406151057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada95a6a0 msgr2=0x556ada961c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada95a6a0 0x556ada961c00 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005ab0 tx=0x7f3e40004610 comp rx=0 tx=0).stop 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 -- 172.21.15.22:0/1406151057 shutdown_connections 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada95a6a0 0x556ada961c00 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada951ac0 0x556ada9616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 --2- 172.21.15.22:0/1406151057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada9511a0 0x556ada959e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 -- 172.21.15.22:0/1406151057 >> 172.21.15.22:0/1406151057 conn(0x556ada94b8d0 msgr2=0x556ada951ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 -- 172.21.15.22:0/1406151057 shutdown_connections 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.819+0000 7f3e56922740 1 -- 172.21.15.22:0/1406151057 wait complete. 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 -- start start 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 Processor -- start 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada961450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada961990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1af000 con 0x556ada951ac0 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1b02e0 con 0x556ada9511a0 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1af620 con 0x556ada95a6a0 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e56922740 10 client.0 _ll_register_callbacks cb 0x7f3e44009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e5163c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada961450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e51e3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada961990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e5163c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada961450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:52462/0 (socket says 172.21.15.22:52462) 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e5163c640 1 -- 172.21.15.22:0/360160902 learned_addr learned my addr 172.21.15.22:0/360160902 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.820+0000 7f3e5163c640 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e51e3d640 1 -- 172.21.15.22:0/360160902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 msgr2=0x556ada95fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e51e3d640 1 -- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 msgr2=0x556ada961450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada961450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e51e3d640 1 -- 172.21.15.22:0/360160902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ada9602c0 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e5163c640 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada961450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e51e3d640 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada961990 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3e44011eb0 tx=0x7f3e44019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30001b976e2734d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e317fa640 1 -- 172.21.15.22:0/360160902 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4401e040 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e317fa640 1 -- 172.21.15.22:0/360160902 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3e44016040 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ada8bc750 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e317fa640 1 -- 172.21.15.22:0/360160902 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e44035050 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 10 client.18447 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556ada9c1920 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e317fa640 1 -- 172.21.15.22:0/360160902 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556ada8bc750 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e317fa640 1 -- 172.21.15.22:0/360160902 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e440197b0 con 0x556ada951ac0 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 10 client.18447 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 10 client.18447 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:03:36.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 20 client.18447 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada9c1e10 0x556ada9c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 4 client.18447 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.821+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ada83a640 con 0x556ada9c1e10 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e5163c640 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada9c1e10 0x556ada9c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e5163c640 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada9c1e10 0x556ada9c4230 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b00b7b754381923c server_cookie=e9c2bebf39eaa220 in_seq=0 out_seq=0 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e317fa640 10 client.18447 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e317fa640 1 -- 172.21.15.22:0/360160902 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ada83a640 con 0x556ada9c1e10 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e317fa640 10 client.18447 handle_command_reply: tid=0 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 client.18447 shutdown 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 msgr2=0x556ada961990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada961990 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3e44011eb0 tx=0x7f3e44019600 comp rx=0 tx=0).stop 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 shutdown_connections 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x556ada9c1e10 0x556ada9c4230 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ada95a6a0 0x556ada95fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556ada951ac0 0x556ada961990 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e317fa640 0 client.18447 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 --2- 172.21.15.22:0/360160902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ada9511a0 0x556ada961450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 >> 172.21.15.22:0/360160902 conn(0x556ada94b8d0 msgr2=0x556ada94cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 shutdown_connections 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 1 -- 172.21.15.22:0/360160902 wait complete. 2024-03-09T00:03:36.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.822+0000 7f3e56922740 20 client.18447 trim_cache size 0 max 16384 2024-03-09T00:03:36.826 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:03:36.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e340a2970 msgr2=0x7f3e340a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e340a2970 0x7f3e340a4e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0cca10 tx=0x7f3e3c06a680 comp rx=0 tx=0).stop 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c10b850 msgr2=0x7f3e4c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c10b850 0x7f3e4c100be0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00e970 tx=0x7f3e3c00ed70 comp rx=0 tx=0).stop 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 shutdown_connections 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e340a2970 0x7f3e340a4e30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e4c10b850 0x7f3e4c100be0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e4c102e30 0x7f3e4c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 --2- 172.21.15.22:0/2546116682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e4c102510 0x7f3e4c100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 >> 172.21.15.22:0/2546116682 conn(0x7f3e4c0773a0 msgr2=0x7f3e4c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 shutdown_connections 2024-03-09T00:03:36.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.823+0000 7f3e540a1640 1 -- 172.21.15.22:0/2546116682 wait complete. 2024-03-09T00:03:36.828 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:03:36.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.986+0000 7f96566d1640 1 -- 172.21.15.22:0/2151949595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 msgr2=0x7f96501008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.986+0000 7f96566d1640 1 --2- 172.21.15.22:0/2151949595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 0x7f96501008c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9644009900 tx=0x7f964402bc80 comp rx=0 tx=0).stop 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 -- 172.21.15.22:0/2151949595 shutdown_connections 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 --2- 172.21.15.22:0/2151949595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9650100e00 0x7f965010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 --2- 172.21.15.22:0/2151949595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 0x7f96501008c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 --2- 172.21.15.22:0/2151949595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96501062d0 0x7f96501066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 -- 172.21.15.22:0/2151949595 >> 172.21.15.22:0/2151949595 conn(0x7f9650076fd0 msgr2=0x7f96500773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 -- 172.21.15.22:0/2151949595 shutdown_connections 2024-03-09T00:03:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 -- 172.21.15.22:0/2151949595 wait complete. 2024-03-09T00:03:36.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 Processor -- start 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.987+0000 7f96566d1640 1 -- start start 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f96566d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 0x7f96501021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f96566d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9650100e00 0x7f9650102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f96566d1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96501062d0 0x7f9650102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f96566d1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501105b0 con 0x7f9650100440 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f96566d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650110430 con 0x7f9650100e00 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f96566d1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650110730 con 0x7f96501062d0 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 0x7f96501021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 0x7f96501021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44512/0 (socket says 172.21.15.22:44512) 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9650100e00 0x7f9650102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964ffff640 1 -- 172.21.15.22:0/1071533695 learned_addr learned my addr 172.21.15.22:0/1071533695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f9654c6e640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96501062d0 0x7f9650102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964f7fe640 1 -- 172.21.15.22:0/1071533695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96501062d0 msgr2=0x7f9650102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964f7fe640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96501062d0 0x7f9650102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964f7fe640 1 -- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 msgr2=0x7f96501021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964f7fe640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 0x7f96501021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f964f7fe640 1 -- 172.21.15.22:0/1071533695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96501a6060 con 0x7f9650100e00 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.988+0000 7f9654c6e640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96501062d0 0x7f9650102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.989+0000 7f964f7fe640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9650100e00 0x7f9650102710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9644000c00 tx=0x7f964402cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.989+0000 7f964d7fa640 1 -- 172.21.15.22:0/1071533695 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644041070 con 0x7f9650100e00 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.989+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96501a62f0 con 0x7f9650100e00 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.989+0000 7f964d7fa640 1 -- 172.21.15.22:0/1071533695 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9644004450 con 0x7f9650100e00 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.989+0000 7f964d7fa640 1 -- 172.21.15.22:0/1071533695 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644003d70 con 0x7f9650100e00 2024-03-09T00:03:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.989+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501a67e0 con 0x7f9650100e00 2024-03-09T00:03:36.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.991+0000 7f964d7fa640 1 -- 172.21.15.22:0/1071533695 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9644044070 con 0x7f9650100e00 2024-03-09T00:03:36.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 Processor -- start 2024-03-09T00:03:36.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 -- start start 2024-03-09T00:03:36.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9a1a0 0x55e265a9a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9aac0 0x55e265aa3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265aa36a0 0x55e265aa5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e265a61360 con 0x55e265aa36a0 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2659fdec0 con 0x55e265a9aac0 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f965938c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2659a6d30 con 0x55e265a9a1a0 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f964d7fa640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f96380a2970 0x7f96380a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f9654c6e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9a1a0 0x55e265a9a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f964f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265aa36a0 0x55e265aa5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f9654c6e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9a1a0 0x55e265a9a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:55542/0 (socket says 172.21.15.22:55542) 2024-03-09T00:03:36.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f9654c6e640 1 -- 172.21.15.22:0/3780042683 learned_addr learned my addr 172.21.15.22:0/3780042683 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:36.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f964ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9aac0 0x55e265aa3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.992+0000 7f964d7fa640 1 -- 172.21.15.22:0/1071533695 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f96440f0a10 con 0x7f9650100e00 2024-03-09T00:03:36.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f9654c6e640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f96380a2970 0x7f96380a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:36.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f964f7fe640 1 -- 172.21.15.22:0/3780042683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9a1a0 msgr2=0x55e265a9a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f964f7fe640 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9a1a0 0x55e265a9a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:36.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f964f7fe640 1 -- 172.21.15.22:0/3780042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9aac0 msgr2=0x55e265aa3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:36.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f964f7fe640 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9aac0 0x55e265aa3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f964f7fe640 1 -- 172.21.15.22:0/3780042683 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2659a5bc0 con 0x55e265aa36a0 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f964ffff640 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9aac0 0x55e265aa3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f964f7fe640 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265aa36a0 0x55e265aa5a90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f96440ede70 tx=0x7f964408fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d947a48cf7ed271f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f9654c6e640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f96380a2970 0x7f96380a4e30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9648009fd0 tx=0x7f9648004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f96367fc640 1 -- 172.21.15.22:0/3780042683 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964402cd00 con 0x55e265aa36a0 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f96367fc640 1 -- 172.21.15.22:0/3780042683 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96440bce30 con 0x55e265aa36a0 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.993+0000 7f96367fc640 1 -- 172.21.15.22:0/3780042683 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96440edfe0 con 0x55e265aa36a0 2024-03-09T00:03:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 -- 172.21.15.22:0/3780042683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265aa36a0 msgr2=0x55e265aa5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265aa36a0 0x55e265aa5a90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f96440ede70 tx=0x7f964408fb10 comp rx=0 tx=0).stop 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 -- 172.21.15.22:0/3780042683 shutdown_connections 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265aa36a0 0x55e265aa5a90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9aac0 0x55e265aa3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 --2- 172.21.15.22:0/3780042683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9a1a0 0x55e265a9a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 -- 172.21.15.22:0/3780042683 >> 172.21.15.22:0/3780042683 conn(0x55e265a948d0 msgr2=0x55e265a94cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 -- 172.21.15.22:0/3780042683 shutdown_connections 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.994+0000 7f965938c740 1 -- 172.21.15.22:0/3780042683 wait complete. 2024-03-09T00:03:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 -- start start 2024-03-09T00:03:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 Processor -- start 2024-03-09T00:03:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aa0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aa12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26599c4a0 con 0x55e265a9a1a0 2024-03-09T00:03:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2659f9f60 con 0x55e265aa36a0 2024-03-09T00:03:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e265aa6250 con 0x55e265a9aac0 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f965938c740 10 client.0 _ll_register_callbacks cb 0x55e2659f7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f964ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f964ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44530/0 (socket says 172.21.15.22:44530) 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f964ffff640 1 -- 172.21.15.22:0/2956594414 learned_addr learned my addr 172.21.15.22:0/2956594414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f964f7fe640 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aa0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.995+0000 7f9654c6e640 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aa12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f964ffff640 1 -- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 msgr2=0x55e265aa0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:37.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f964ffff640 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aa0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f964ffff640 1 -- 172.21.15.22:0/2956594414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 msgr2=0x55e265aa12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f964ffff640 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aa12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f964ffff640 1 -- 172.21.15.22:0/2956594414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2657123f0 con 0x55e265a9a1a0 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f964ffff640 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa0840 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9640012790 tx=0x7f9640012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53d880520674baa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f962f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f962f7fe640 1 -- 172.21.15.22:0/2956594414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640004030 con 0x55e265a9a1a0 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f962f7fe640 1 -- 172.21.15.22:0/2956594414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f96400041d0 con 0x55e265a9a1a0 2024-03-09T00:03:37.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96440bce30 con 0x55e265a9a1a0 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f962f7fe640 1 -- 172.21.15.22:0/2956594414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96400043d0 con 0x55e265a9a1a0 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f965938c740 10 client.18459 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e2657dc7d0 con 0x55e265a9a1a0 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f962f7fe640 1 -- 172.21.15.22:0/2956594414 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96440bce30 con 0x55e265a9a1a0 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.996+0000 7f962f7fe640 1 -- 172.21.15.22:0/2956594414 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9640003c70 con 0x55e265a9a1a0 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f965938c740 10 client.18459 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f965938c740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:37.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f965938c740 20 client.18459 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f965938c740 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265ab37b0 0x55e265ab5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f965938c740 4 client.18459 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e265a0b400 con 0x55e265ab37b0 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f964f7fe640 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265ab37b0 0x55e265ab5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f962f7fe640 10 client.18459 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.997+0000 7f964f7fe640 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265ab37b0 0x55e265ab5ba0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c34cd692c7126 server_cookie=355ffed08d2513d1 in_seq=0 out_seq=0 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f962f7fe640 1 -- 172.21.15.22:0/2956594414 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e265a0b400 con 0x55e265ab37b0 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f962f7fe640 10 client.18459 handle_command_reply: tid=0 2024-03-09T00:03:37.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 client.18459 shutdown 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 msgr2=0x55e265aa0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa0840 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9640012790 tx=0x7f9640012b90 comp rx=0 tx=0).stop 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 shutdown_connections 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265ab37b0 0x55e265ab5ba0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aa12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aa0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 --2- 172.21.15.22:0/2956594414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa0840 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 >> 172.21.15.22:0/2956594414 conn(0x55e265a948d0 msgr2=0x55e265a96620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:37.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f962f7fe640 0 client.18459 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:37.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 shutdown_connections 2024-03-09T00:03:37.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 1 -- 172.21.15.22:0/2956594414 wait complete. 2024-03-09T00:03:37.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:36.998+0000 7f965938c740 20 client.18459 trim_cache size 0 max 16384 2024-03-09T00:03:37.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 Processor -- start 2024-03-09T00:03:37.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 -- start start 2024-03-09T00:03:37.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aaa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aaa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501b2db0 con 0x55e265a9a1a0 2024-03-09T00:03:37.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501b0950 con 0x55e265aa36a0 2024-03-09T00:03:37.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f965938c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501aea10 con 0x55e265a9aac0 2024-03-09T00:03:37.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44538/0 (socket says 172.21.15.22:44538) 2024-03-09T00:03:37.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 -- 172.21.15.22:0/2747264695 learned_addr learned my addr 172.21.15.22:0/2747264695 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:37.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964f7fe640 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aaa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f9654c6e640 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aaa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 -- 172.21.15.22:0/2747264695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 msgr2=0x55e265aaa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aaa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 -- 172.21.15.22:0/2747264695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 msgr2=0x55e265aaa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aaa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.011+0000 7f964ffff640 1 -- 172.21.15.22:0/2747264695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e265abad90 con 0x55e265a9a1a0 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.012+0000 7f964ffff640 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa2f70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9640009fb0 tx=0x7f964000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d7ace7a3d63eb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.012+0000 7f9635ffb640 1 -- 172.21.15.22:0/2747264695 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964002e070 con 0x55e265a9a1a0 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.012+0000 7f9635ffb640 1 -- 172.21.15.22:0/2747264695 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9640010620 con 0x55e265a9a1a0 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.012+0000 7f9635ffb640 1 -- 172.21.15.22:0/2747264695 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964000fc50 con 0x55e265a9a1a0 2024-03-09T00:03:37.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- 172.21.15.22:0/2747264695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 msgr2=0x55e265aa2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa2f70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9640009fb0 tx=0x7f964000f790 comp rx=0 tx=0).stop 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- 172.21.15.22:0/2747264695 shutdown_connections 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265aa36a0 0x55e265aaa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265a9aac0 0x55e265aaa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 --2- 172.21.15.22:0/2747264695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9a1a0 0x55e265aa2f70 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- 172.21.15.22:0/2747264695 >> 172.21.15.22:0/2747264695 conn(0x55e265a948d0 msgr2=0x55e265a9bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- 172.21.15.22:0/2747264695 shutdown_connections 2024-03-09T00:03:37.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- 172.21.15.22:0/2747264695 wait complete. 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- start start 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 Processor -- start 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9a1a0 0x55e265aaa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9aac0 0x55e265aa8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265aa36a0 0x55e265aa9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501af010 con 0x55e265a9aac0 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501b02f0 con 0x55e265a9a1a0 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.013+0000 7f965938c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501af630 con 0x55e265aa36a0 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f965938c740 10 client.0 _ll_register_callbacks cb 0x7f9644005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9aac0 0x55e265aa8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9aac0 0x55e265aa8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44546/0 (socket says 172.21.15.22:44546) 2024-03-09T00:03:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9a1a0 0x55e265aaa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 -- 172.21.15.22:0/1618555689 learned_addr learned my addr 172.21.15.22:0/1618555689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f9654c6e640 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265aa36a0 0x55e265aa9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 -- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265aa36a0 msgr2=0x55e265aa9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265aa36a0 0x55e265aa9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 -- 172.21.15.22:0/1618555689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9a1a0 msgr2=0x55e265aaa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9a1a0 0x55e265aaa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 -- 172.21.15.22:0/1618555689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e265aa99f0 con 0x55e265a9aac0 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f964f7fe640 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9aac0 0x55e265aa8bf0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f96440edf20 tx=0x7f9644005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693811281afe21bc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f962f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f962f7fe640 1 -- 172.21.15.22:0/1618555689 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964404b040 con 0x55e265a9aac0 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f962f7fe640 1 -- 172.21.15.22:0/1618555689 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f96440b94a0 con 0x55e265a9aac0 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.014+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9640010620 con 0x55e265a9aac0 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f962f7fe640 1 -- 172.21.15.22:0/1618555689 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96440bc790 con 0x55e265a9aac0 2024-03-09T00:03:37.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 10 client.18465 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e265b13630 con 0x55e265a9aac0 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f962f7fe640 1 -- 172.21.15.22:0/1618555689 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9640010620 con 0x55e265a9aac0 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f962f7fe640 1 -- 172.21.15.22:0/1618555689 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f96440bd090 con 0x55e265a9aac0 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 10 client.18465 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 10 client.18465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 20 client.18465 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265b13bd0 0x55e265b15ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 4 client.18465 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e265a05750 con 0x55e265b13bd0 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.015+0000 7f964ffff640 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265b13bd0 0x55e265b15ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.016+0000 7f962f7fe640 10 client.18465 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.016+0000 7f964ffff640 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265b13bd0 0x55e265b15ff0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60eb359360696126 server_cookie=9daad7ffffecdd45 in_seq=0 out_seq=0 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f962f7fe640 1 -- 172.21.15.22:0/1618555689 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e265a05750 con 0x55e265b13bd0 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f962f7fe640 10 client.18465 handle_command_reply: tid=0 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 client.18465 shutdown 2024-03-09T00:03:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9aac0 msgr2=0x55e265aa8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9aac0 0x55e265aa8bf0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f96440edf20 tx=0x7f9644005c00 comp rx=0 tx=0).stop 2024-03-09T00:03:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 shutdown_connections 2024-03-09T00:03:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e265b13bd0 0x55e265b15ff0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e265aa36a0 0x55e265aa9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e265a9aac0 0x55e265aa8bf0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f962f7fe640 0 client.18465 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:37.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 --2- 172.21.15.22:0/1618555689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e265a9a1a0 0x55e265aaa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 >> 172.21.15.22:0/1618555689 conn(0x55e265a948d0 msgr2=0x55e265a9efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:37.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 shutdown_connections 2024-03-09T00:03:37.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 1 -- 172.21.15.22:0/1618555689 wait complete. 2024-03-09T00:03:37.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f965938c740 20 client.18465 trim_cache size 0 max 16384 2024-03-09T00:03:37.031 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:03:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f96380a2970 msgr2=0x7f96380a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f96566d1640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f96380a2970 0x7f96380a4e30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9648009fd0 tx=0x7f9648004ab0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9650100e00 msgr2=0x7f9650102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.018+0000 7f96566d1640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9650100e00 0x7f9650102710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9644000c00 tx=0x7f964402cb10 comp rx=0 tx=0).stop 2024-03-09T00:03:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 shutdown_connections 2024-03-09T00:03:37.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f96380a2970 0x7f96380a4e30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96501062d0 0x7f9650102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9650100e00 0x7f9650102710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 --2- 172.21.15.22:0/1071533695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9650100440 0x7f96501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 >> 172.21.15.22:0/1071533695 conn(0x7f9650076fd0 msgr2=0x7f9650104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 shutdown_connections 2024-03-09T00:03:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:37.019+0000 7f96566d1640 1 -- 172.21.15.22:0/1071533695 wait complete. 2024-03-09T00:03:37.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:37 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2340967624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:03:37.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:37 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2340967624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:03:37.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:37 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2340967624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:03:37.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 940491 10.24 MB/sec execute 345 sec latency 15.688 ms 2024-03-09T00:03:38.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:03:38.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:03:38.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:03:38.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 -- 172.21.15.22:0/2543522791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e40100d80 msgr2=0x7f9e40104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 --2- 172.21.15.22:0/2543522791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e40100d80 0x7f9e40104dc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009900 tx=0x7f9e3402bc30 comp rx=0 tx=0).stop 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 -- 172.21.15.22:0/2543522791 shutdown_connections 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 --2- 172.21.15.22:0/2543522791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e401054a0 0x7f9e4010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 --2- 172.21.15.22:0/2543522791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e40100d80 0x7f9e40104dc0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 --2- 172.21.15.22:0/2543522791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e401003d0 0x7f9e401007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 -- 172.21.15.22:0/2543522791 >> 172.21.15.22:0/2543522791 conn(0x7f9e400772f0 msgr2=0x7f9e40077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 -- 172.21.15.22:0/2543522791 shutdown_connections 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.151+0000 7f9e45d69640 1 -- 172.21.15.22:0/2543522791 wait complete. 2024-03-09T00:03:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 Processor -- start 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 -- start start 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e401003d0 0x7f9e4019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e401054a0 0x7f9e4019dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e401979a0 0x7f9e40197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40110530 con 0x7f9e401054a0 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401103b0 con 0x7f9e401003d0 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e45d69640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401106b0 con 0x7f9e401979a0 2024-03-09T00:03:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e401054a0 0x7f9e4019dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e401054a0 0x7f9e4019dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44564/0 (socket says 172.21.15.22:44564) 2024-03-09T00:03:38.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e3effd640 1 -- 172.21.15.22:0/3061217862 learned_addr learned my addr 172.21.15.22:0/3061217862 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e401003d0 0x7f9e4019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.152+0000 7f9e3ffff640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e401979a0 0x7f9e40197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/3061217862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e401979a0 msgr2=0x7f9e40197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e401979a0 0x7f9e40197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e401054a0 msgr2=0x7f9e4019dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e401054a0 0x7f9e4019dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/3061217862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e401984c0 con 0x7f9e401003d0 2024-03-09T00:03:38.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e401054a0 0x7f9e4019dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:38.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e401003d0 0x7f9e4019d7f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000e9f0 tx=0x7f9e3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3cff9640 1 -- 172.21.15.22:0/3061217862 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000ccc0 con 0x7f9e401003d0 2024-03-09T00:03:38.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3cff9640 1 -- 172.21.15.22:0/3061217862 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e30005020 con 0x7f9e401003d0 2024-03-09T00:03:38.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e401987b0 con 0x7f9e401003d0 2024-03-09T00:03:38.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e3cff9640 1 -- 172.21.15.22:0/3061217862 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300049e0 con 0x7f9e401003d0 2024-03-09T00:03:38.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.153+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e401a49b0 con 0x7f9e401003d0 2024-03-09T00:03:38.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.155+0000 7f9e3cff9640 1 -- 172.21.15.22:0/3061217862 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9e30005560 con 0x7f9e401003d0 2024-03-09T00:03:38.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3cff9640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9e280a2970 0x7f9e280a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 Processor -- start 2024-03-09T00:03:38.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9e280a2970 0x7f9e280a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 -- start start 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3cff9640 1 -- 172.21.15.22:0/3061217862 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9e300cf280 con 0x7f9e401003d0 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab22c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab22c6a0 0x55d0ab234d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab235280 0x55d0ab237670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ab18fb40 con 0x55d0ab235280 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ab138d80 con 0x55d0ab22bd80 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e48a28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ab18bbe0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab22c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab235280 0x55d0ab237670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9e280a2970 0x7f9e280a4e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e3402cfd0 tx=0x7f9e34005b50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab22c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:52556/0 (socket says 172.21.15.22:52556) 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/1501302817 learned_addr learned my addr 172.21.15.22:0/1501302817 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3effd640 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab22c6a0 0x55d0ab234d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/1501302817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab22c6a0 msgr2=0x55d0ab234d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab22c6a0 0x55d0ab234d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/1501302817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab235280 msgr2=0x55d0ab237670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab235280 0x55d0ab237670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.156+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/1501302817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0aaf6c7e0 con 0x55d0ab22bd80 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab22c160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9e300a36f0 tx=0x7f9e30005290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260e21753b353527 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e25ffb640 1 -- 172.21.15.22:0/1501302817 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3001c040 con 0x55d0ab22bd80 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e25ffb640 1 -- 172.21.15.22:0/1501302817 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e300a3c20 con 0x55d0ab22bd80 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e25ffb640 1 -- 172.21.15.22:0/1501302817 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3009f3d0 con 0x55d0ab22bd80 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e48a28740 1 -- 172.21.15.22:0/1501302817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 msgr2=0x55d0ab22c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e48a28740 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab22c160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9e300a36f0 tx=0x7f9e30005290 comp rx=0 tx=0).stop 2024-03-09T00:03:38.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e48a28740 1 -- 172.21.15.22:0/1501302817 shutdown_connections 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e48a28740 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab235280 0x55d0ab237670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e48a28740 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab22c6a0 0x55d0ab234d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.157+0000 7f9e48a28740 1 --2- 172.21.15.22:0/1501302817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab22c160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 -- 172.21.15.22:0/1501302817 >> 172.21.15.22:0/1501302817 conn(0x55d0ab2264b0 msgr2=0x55d0ab226890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 -- 172.21.15.22:0/1501302817 shutdown_connections 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 -- 172.21.15.22:0/1501302817 wait complete. 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 -- start start 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 Processor -- start 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab234d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab232090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ab238070 con 0x55d0ab22c6a0 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ab14c1c0 con 0x55d0ab22bd80 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0ab2381f0 con 0x55d0ab235280 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e48a28740 10 client.0 _ll_register_callbacks cb 0x55d0ab13b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab232090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab232090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44582/0 (socket says 172.21.15.22:44582) 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e3f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab234d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e3effd640 1 -- 172.21.15.22:0/3188540860 learned_addr learned my addr 172.21.15.22:0/3188540860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.158+0000 7f9e3ffff640 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 msgr2=0x55d0ab2327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/3188540860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 msgr2=0x55d0ab232090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab232090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e3f7fe640 1 -- 172.21.15.22:0/3188540860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0aafccf70 con 0x55d0ab22bd80 2024-03-09T00:03:38.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab232090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab234d00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9e300a36f0 tx=0x7f9e3009b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ca1c1354ca0fb75 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e1effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e1effd640 1 -- 172.21.15.22:0/3188540860 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3001c070 con 0x55d0ab22bd80 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e1effd640 1 -- 172.21.15.22:0/3188540860 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9e3009f170 con 0x55d0ab22bd80 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e300a3c20 con 0x55d0ab22bd80 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e1effd640 1 -- 172.21.15.22:0/3188540860 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e3009f370 con 0x55d0ab22bd80 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e48a28740 10 client.27830 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d0aae64170 con 0x55d0ab22bd80 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.159+0000 7f9e1effd640 1 -- 172.21.15.22:0/3188540860 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e300a3c20 con 0x55d0ab22bd80 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e1effd640 1 -- 172.21.15.22:0/3188540860 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9e3007c030 con 0x55d0ab22bd80 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e48a28740 10 client.27830 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e48a28740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:38.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e48a28740 20 client.27830 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab245280 0x55d0ab247670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e48a28740 4 client.27830 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0ab165960 con 0x55d0ab245280 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab245280 0x55d0ab247670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e1effd640 10 client.27830 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.160+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab245280 0x55d0ab247670 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cf98b9412a1a94 server_cookie=20965623d989b57c in_seq=0 out_seq=0 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.161+0000 7f9e1effd640 1 -- 172.21.15.22:0/3188540860 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0ab165960 con 0x55d0ab245280 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.161+0000 7f9e1effd640 10 client.27830 handle_command_reply: tid=0 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.161+0000 7f9e48a28740 1 client.27830 shutdown 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.161+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 msgr2=0x55d0ab234d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.161+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab234d00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9e300a36f0 tx=0x7f9e3009b600 comp rx=0 tx=0).stop 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 shutdown_connections 2024-03-09T00:03:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab245280 0x55d0ab247670 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab232090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3188540860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab234d00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e1effd640 0 client.27830 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 >> 172.21.15.22:0/3188540860 conn(0x55d0ab2264b0 msgr2=0x55d0ab228200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 shutdown_connections 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 1 -- 172.21.15.22:0/3188540860 wait complete. 2024-03-09T00:03:38.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.162+0000 7f9e48a28740 20 client.27830 trim_cache size 0 max 16384 2024-03-09T00:03:38.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 Processor -- start 2024-03-09T00:03:38.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 -- start start 2024-03-09T00:03:38.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab2311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab240b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401b0c70 con 0x55d0ab22c6a0 2024-03-09T00:03:38.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401ae810 con 0x55d0ab22bd80 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e48a28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401ac8d0 con 0x55d0ab235280 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab240b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab240b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44586/0 (socket says 172.21.15.22:44586) 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.177+0000 7f9e3effd640 1 -- 172.21.15.22:0/3404861895 learned_addr learned my addr 172.21.15.22:0/3404861895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3ffff640 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab2311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3effd640 1 -- 172.21.15.22:0/3404861895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 msgr2=0x55d0ab2347c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2347c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3effd640 1 -- 172.21.15.22:0/3404861895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 msgr2=0x55d0ab2311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab2311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3effd640 1 -- 172.21.15.22:0/3404861895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0ab22c4d0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab2311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab240b50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009230 tx=0x7f9e3403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89097fb3f91a5e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e257fa640 1 -- 172.21.15.22:0/3404861895 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34004410 con 0x55d0ab22c6a0 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e257fa640 1 -- 172.21.15.22:0/3404861895 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e34004af0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.178+0000 7f9e257fa640 1 -- 172.21.15.22:0/3404861895 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340035f0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 -- 172.21.15.22:0/3404861895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 msgr2=0x55d0ab240b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab240b50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009230 tx=0x7f9e3403f000 comp rx=0 tx=0).stop 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 -- 172.21.15.22:0/3404861895 shutdown_connections 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab2347c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab240b50 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3404861895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab2311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 -- 172.21.15.22:0/3404861895 >> 172.21.15.22:0/3404861895 conn(0x55d0ab2264b0 msgr2=0x55d0ab240440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 -- 172.21.15.22:0/3404861895 shutdown_connections 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 -- 172.21.15.22:0/3404861895 wait complete. 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 -- start start 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 Processor -- start 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.179+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab248850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab248d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab29b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401aced0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401ae1b0 con 0x55d0ab22bd80 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401ad4f0 con 0x55d0ab235280 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 10 client.0 _ll_register_callbacks cb 0x7f9e30097060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab248d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab248d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44602/0 (socket says 172.21.15.22:44602) 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 -- 172.21.15.22:0/3484489546 learned_addr learned my addr 172.21.15.22:0/3484489546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3ffff640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab29b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab248850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 -- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 msgr2=0x55d0ab29b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab29b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 -- 172.21.15.22:0/3484489546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 msgr2=0x55d0ab248850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab248850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 -- 172.21.15.22:0/3484489546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0ab29b920 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab248850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e3effd640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab248d90 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005c40 tx=0x7f9e34009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfeb5b1ae9c62e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e1effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e1effd640 1 -- 172.21.15.22:0/3484489546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34004490 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e1effd640 1 -- 172.21.15.22:0/3484489546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9e340054a0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e34004af0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e1effd640 1 -- 172.21.15.22:0/3484489546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e3406c050 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 10 client.18483 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.180+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d0ab29cc60 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e1effd640 1 -- 172.21.15.22:0/3484489546 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e34004af0 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e1effd640 1 -- 172.21.15.22:0/3484489546 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9e34003b60 con 0x55d0ab22c6a0 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e48a28740 10 client.18483 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e48a28740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e48a28740 20 client.18483 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:38.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab29d3b0 0x55d0ab29f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e48a28740 4 client.18483 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:03:38.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d0ab151080 con 0x55d0ab29d3b0 2024-03-09T00:03:38.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab29d3b0 0x55d0ab29f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e3f7fe640 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab29d3b0 0x55d0ab29f7d0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb337377512c10e server_cookie=3e068e7ac8cee37a in_seq=0 out_seq=0 2024-03-09T00:03:38.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.181+0000 7f9e1effd640 10 client.18483 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.183+0000 7f9e1effd640 1 -- 172.21.15.22:0/3484489546 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d0ab151080 con 0x55d0ab29d3b0 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.183+0000 7f9e1effd640 10 client.18483 handle_command_reply: tid=0 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.183+0000 7f9e48a28740 1 client.18483 shutdown 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 msgr2=0x55d0ab248d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab248d90 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005c40 tx=0x7f9e34009260 comp rx=0 tx=0).stop 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 shutdown_connections 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d0ab29d3b0 0x55d0ab29f7d0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0ab235280 0x55d0ab29b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d0ab22c6a0 0x55d0ab248d90 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 --2- 172.21.15.22:0/3484489546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0ab22bd80 0x55d0ab248850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e1effd640 0 client.18483 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 >> 172.21.15.22:0/3484489546 conn(0x55d0ab2264b0 msgr2=0x55d0ab240440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 shutdown_connections 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 1 -- 172.21.15.22:0/3484489546 wait complete. 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e48a28740 20 client.18483 trim_cache size 0 max 16384 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "3ae5a417-e193-4936-897f-762443d7c54a", 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:03:38.185 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:03:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9e280a2970 msgr2=0x7f9e280a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e45d69640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9e280a2970 0x7f9e280a4e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e3402cfd0 tx=0x7f9e34005b50 comp rx=0 tx=0).stop 2024-03-09T00:03:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e401003d0 msgr2=0x7f9e4019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e45d69640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e401003d0 0x7f9e4019d7f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000e9f0 tx=0x7f9e3000eeb0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.184+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 shutdown_connections 2024-03-09T00:03:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.185+0000 7f9e45d69640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9e280a2970 0x7f9e280a4e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.185+0000 7f9e45d69640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e401979a0 0x7f9e40197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.185+0000 7f9e45d69640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e401054a0 0x7f9e4019dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.185+0000 7f9e45d69640 1 --2- 172.21.15.22:0/3061217862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e401003d0 0x7f9e4019d7f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.185+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 >> 172.21.15.22:0/3061217862 conn(0x7f9e400772f0 msgr2=0x7f9e400687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.185+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 shutdown_connections 2024-03-09T00:03:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.185+0000 7f9e45d69640 1 -- 172.21.15.22:0/3061217862 wait complete. 2024-03-09T00:03:38.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:03:38.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:03:38.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.303+0000 7febc777e640 1 -- 172.21.15.22:0/435912595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0106630 msgr2=0x7febc010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.303+0000 7febc777e640 1 --2- 172.21.15.22:0/435912595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0106630 0x7febc010e1e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7febb8009ca0 tx=0x7febb802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.303+0000 7febc777e640 1 -- 172.21.15.22:0/435912595 shutdown_connections 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.303+0000 7febc777e640 1 --2- 172.21.15.22:0/435912595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0106630 0x7febc010e1e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.303+0000 7febc777e640 1 --2- 172.21.15.22:0/435912595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febc0069110 0x7febc0105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.303+0000 7febc777e640 1 --2- 172.21.15.22:0/435912595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc0068760 0x7febc0068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 -- 172.21.15.22:0/435912595 >> 172.21.15.22:0/435912595 conn(0x7febc00772f0 msgr2=0x7febc0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 -- 172.21.15.22:0/435912595 shutdown_connections 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 -- 172.21.15.22:0/435912595 wait complete. 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 Processor -- start 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 -- start start 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febc0068760 0x7febc01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc0106630 0x7febc01006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0100c30 0x7febc01a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc0111690 con 0x7febc0100c30 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc0111510 con 0x7febc0068760 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc777e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc0111810 con 0x7febc0106630 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc5d1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0100c30 0x7febc01a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.304+0000 7febc551a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febc0068760 0x7febc01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0100c30 0x7febc01a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44620/0 (socket says 172.21.15.22:44620) 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 -- 172.21.15.22:0/2609994764 learned_addr learned my addr 172.21.15.22:0/2609994764 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc4d19640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc0106630 0x7febc01006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 -- 172.21.15.22:0/2609994764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc0106630 msgr2=0x7febc01006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc0106630 0x7febc01006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 -- 172.21.15.22:0/2609994764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febc0068760 msgr2=0x7febc01001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febc0068760 0x7febc01001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 -- 172.21.15.22:0/2609994764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc01a6400 con 0x7febc0100c30 2024-03-09T00:03:38.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc551a640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febc0068760 0x7febc01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc5d1b640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0100c30 0x7febc01a5d00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7febb802c700 tx=0x7febb8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febb67fc640 1 -- 172.21.15.22:0/2609994764 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8041020 con 0x7febc0100c30 2024-03-09T00:03:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febb67fc640 1 -- 172.21.15.22:0/2609994764 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febb8034cd0 con 0x7febc0100c30 2024-03-09T00:03:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc01a6690 con 0x7febc0100c30 2024-03-09T00:03:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febb67fc640 1 -- 172.21.15.22:0/2609994764 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8039040 con 0x7febc0100c30 2024-03-09T00:03:38.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.305+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc01a6bd0 con 0x7febc0100c30 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.307+0000 7febb67fc640 1 -- 172.21.15.22:0/2609994764 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7febb8044020 con 0x7febc0100c30 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 Processor -- start 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febb67fc640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febac0a2a20 0x7febac0a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 -- start start 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc4d19640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febac0a2a20 0x7febac0a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dc98b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febb67fc640 1 -- 172.21.15.22:0/2609994764 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7febb80f0f00 con 0x7febc0100c30 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dc99080 0x564f2dca1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dca1c60 0x564f2dca4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2dbfbe00 con 0x564f2dc98760 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2dba4b10 con 0x564f2dca1c60 2024-03-09T00:03:38.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc9ffd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2dbf7ea0 con 0x564f2dc99080 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc551a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dc98b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc551a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dc98b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44628/0 (socket says 172.21.15.22:44628) 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.308+0000 7febc551a640 1 -- 172.21.15.22:0/2989803259 learned_addr learned my addr 172.21.15.22:0/2989803259 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc5d1b640 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dca1c60 0x564f2dca4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc4d19640 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dc99080 0x564f2dca1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc551a640 1 -- 172.21.15.22:0/2989803259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dc99080 msgr2=0x564f2dca1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc551a640 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dc99080 0x564f2dca1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc4d19640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febac0a2a20 0x7febac0a4ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febc0101c40 tx=0x7febb0014950 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc551a640 1 -- 172.21.15.22:0/2989803259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dca1c60 msgr2=0x564f2dca4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc551a640 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dca1c60 0x564f2dca4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc551a640 1 -- 172.21.15.22:0/2989803259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2d9da7d0 con 0x564f2dc98760 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc5d1b640 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dca1c60 0x564f2dca4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febc551a640 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dc98b40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7febbc0115c0 tx=0x7febbc0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b6ba75883ece0c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febab7fe640 1 -- 172.21.15.22:0/2989803259 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc01e070 con 0x564f2dc98760 2024-03-09T00:03:38.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febab7fe640 1 -- 172.21.15.22:0/2989803259 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc00fa20 con 0x564f2dc98760 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.309+0000 7febab7fe640 1 -- 172.21.15.22:0/2989803259 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc019b50 con 0x564f2dc98760 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 -- 172.21.15.22:0/2989803259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 msgr2=0x564f2dc98b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dc98b40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7febbc0115c0 tx=0x7febbc0119c0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 -- 172.21.15.22:0/2989803259 shutdown_connections 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dca1c60 0x564f2dca4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dc99080 0x564f2dca1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 --2- 172.21.15.22:0/2989803259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dc98b40 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 -- 172.21.15.22:0/2989803259 >> 172.21.15.22:0/2989803259 conn(0x564f2dc92e90 msgr2=0x564f2dc93270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 -- 172.21.15.22:0/2989803259 shutdown_connections 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 -- 172.21.15.22:0/2989803259 wait complete. 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 -- start start 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.310+0000 7febc9ffd740 1 Processor -- start 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dc9e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dc9ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc9ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2dca4a50 con 0x564f2dc99080 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc9ffd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2dbb80a0 con 0x564f2dc98760 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc9ffd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2dca4bd0 con 0x564f2dca1c60 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc9ffd740 10 client.0 _ll_register_callbacks cb 0x564f2dbb7a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc4d19640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dc9e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc551a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc4d19640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dc9e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44638/0 (socket says 172.21.15.22:44638) 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc4d19640 1 -- 172.21.15.22:0/1035893604 learned_addr learned my addr 172.21.15.22:0/1035893604 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc5d1b640 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dc9ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc551a640 1 -- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 msgr2=0x564f2dc9ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc551a640 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dc9ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc551a640 1 -- 172.21.15.22:0/1035893604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 msgr2=0x564f2dc9e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc551a640 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dc9e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc551a640 1 -- 172.21.15.22:0/1035893604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f2d9103f0 con 0x564f2dc98760 2024-03-09T00:03:38.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc4d19640 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dc9e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7febc551a640 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca1380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febbc0115c0 tx=0x7febbc00ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0eb61ced74ee9f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.311+0000 7feba8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7feba8ff9640 1 -- 172.21.15.22:0/1035893604 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc027060 con 0x564f2dc98760 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7feba8ff9640 1 -- 172.21.15.22:0/1035893604 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7febbc00f600 con 0x564f2dc98760 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febbc00fa20 con 0x564f2dc98760 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7feba8ff9640 1 -- 172.21.15.22:0/1035893604 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7febbc02e050 con 0x564f2dc98760 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 10 client.27848 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564f2d8d00f0 con 0x564f2dc98760 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7feba8ff9640 1 -- 172.21.15.22:0/1035893604 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc00fa20 con 0x564f2dc98760 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7feba8ff9640 1 -- 172.21.15.22:0/1035893604 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7febbc021070 con 0x564f2dc98760 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 10 client.27848 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 10 client.27848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 20 client.27848 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dcb1670 0x564f2dcb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 4 client.27848 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f2dc09340 con 0x564f2dcb1670 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.312+0000 7febc4d19640 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dcb1670 0x564f2dcb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.313+0000 7febc4d19640 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dcb1670 0x564f2dcb3a60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28920710741a2c server_cookie=4cc031ac88fbbbb in_seq=0 out_seq=0 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.313+0000 7feba8ff9640 10 client.27848 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.313+0000 7feba8ff9640 1 -- 172.21.15.22:0/1035893604 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f2dc09340 con 0x564f2dcb1670 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.313+0000 7feba8ff9640 10 client.27848 handle_command_reply: tid=0 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.313+0000 7febc9ffd740 1 client.27848 shutdown 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 msgr2=0x564f2dca1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca1380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febbc0115c0 tx=0x7febbc00ff20 comp rx=0 tx=0).stop 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 shutdown_connections 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dcb1670 0x564f2dcb3a60 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dc9ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7feba8ff9640 0 client.27848 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dc9e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 --2- 172.21.15.22:0/1035893604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca1380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 >> 172.21.15.22:0/1035893604 conn(0x564f2dc92e90 msgr2=0x564f2dc94be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 shutdown_connections 2024-03-09T00:03:38.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 1 -- 172.21.15.22:0/1035893604 wait complete. 2024-03-09T00:03:38.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.314+0000 7febc9ffd740 20 client.27848 trim_cache size 0 max 16384 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 Processor -- start 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 -- start start 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dca1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01b2dc0 con 0x564f2dc99080 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01b0960 con 0x564f2dc98760 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc9ffd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01aea20 con 0x564f2dca1c60 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc4d19640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dca1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:52628/0 (socket says 172.21.15.22:52628) 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc4d19640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dca1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44654/0 (socket says 172.21.15.22:44654) 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 -- 172.21.15.22:0/3715235883 learned_addr learned my addr 172.21.15.22:0/3715235883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc5d1b640 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 -- 172.21.15.22:0/3715235883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 msgr2=0x564f2dca8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 -- 172.21.15.22:0/3715235883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 msgr2=0x564f2dca1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dca1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc551a640 1 -- 172.21.15.22:0/3715235883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2dcb3dd0 con 0x564f2dc98760 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc5d1b640 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.329+0000 7febc4d19640 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dca1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.330+0000 7febc551a640 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca0e40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7febbc0057a0 tx=0x7febbc009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ca4ce985d8714c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.330+0000 7febaaffd640 1 -- 172.21.15.22:0/3715235883 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc027070 con 0x564f2dc98760 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.330+0000 7febaaffd640 1 -- 172.21.15.22:0/3715235883 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc011070 con 0x564f2dc98760 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.330+0000 7febaaffd640 1 -- 172.21.15.22:0/3715235883 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc01e040 con 0x564f2dc98760 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.330+0000 7febc9ffd740 1 -- 172.21.15.22:0/3715235883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 msgr2=0x564f2dca0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.330+0000 7febc9ffd740 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca0e40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7febbc0057a0 tx=0x7febbc009be0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- 172.21.15.22:0/3715235883 shutdown_connections 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc99080 0x564f2dca1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 --2- 172.21.15.22:0/3715235883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc98760 0x564f2dca0e40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- 172.21.15.22:0/3715235883 >> 172.21.15.22:0/3715235883 conn(0x564f2dc92e90 msgr2=0x564f2dcad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- 172.21.15.22:0/3715235883 shutdown_connections 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- 172.21.15.22:0/3715235883 wait complete. 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- start start 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 Processor -- start 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dca4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc99080 0x564f2dca5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01af020 con 0x564f2dc98760 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01b0300 con 0x564f2dc99080 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01af640 con 0x564f2dca1c60 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc9ffd740 10 client.0 _ll_register_callbacks cb 0x7febbc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc551a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dca4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc551a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dca4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44666/0 (socket says 172.21.15.22:44666) 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.331+0000 7febc551a640 1 -- 172.21.15.22:0/772924877 learned_addr learned my addr 172.21.15.22:0/772924877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc5d1b640 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc4d19640 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc99080 0x564f2dca5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc551a640 1 -- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 msgr2=0x564f2dca5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc551a640 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc551a640 1 -- 172.21.15.22:0/772924877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc99080 msgr2=0x564f2dca5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc551a640 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc99080 0x564f2dca5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc551a640 1 -- 172.21.15.22:0/772924877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f2dd06ba0 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc551a640 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dca4de0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7febbc000c00 tx=0x7febbc005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c855d711d88950 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7feba8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7feba8ff9640 1 -- 172.21.15.22:0/772924877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc01e030 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7feba8ff9640 1 -- 172.21.15.22:0/772924877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7febbc00f600 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febbc011070 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7feba8ff9640 1 -- 172.21.15.22:0/772924877 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7febbc027070 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc9ffd740 10 client.18501 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564f2dd07e90 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.332+0000 7feba8ff9640 1 -- 172.21.15.22:0/772924877 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc011070 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7feba8ff9640 1 -- 172.21.15.22:0/772924877 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7febbc021040 con 0x564f2dc98760 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc9ffd740 10 client.18501 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc9ffd740 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc9ffd740 20 client.18501 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc9ffd740 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dd08300 0x564f2dd0a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc9ffd740 4 client.18501 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:03:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f2dc9f6a0 con 0x564f2dd08300 2024-03-09T00:03:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc4d19640 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dd08300 0x564f2dd0a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7febc4d19640 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dd08300 0x564f2dd0a790 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51b0d2b71927c28 server_cookie=3ef1aa5c5c09581a in_seq=0 out_seq=0 2024-03-09T00:03:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.333+0000 7feba8ff9640 10 client.18501 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.334+0000 7feba8ff9640 1 -- 172.21.15.22:0/772924877 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564f2dc9f6a0 con 0x564f2dd08300 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7feba8ff9640 10 client.18501 handle_command_reply: tid=0 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 client.18501 shutdown 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 msgr2=0x564f2dca4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dca4de0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7febbc000c00 tx=0x7febbc005bb0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 shutdown_connections 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564f2dd08300 0x564f2dd0a790 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f2dca1c60 0x564f2dca5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2dc99080 0x564f2dca5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 --2- 172.21.15.22:0/772924877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564f2dc98760 0x564f2dca4de0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7feba8ff9640 0 client.18501 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 >> 172.21.15.22:0/772924877 conn(0x564f2dc92e90 msgr2=0x564f2dc9e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 shutdown_connections 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 1 -- 172.21.15.22:0/772924877 wait complete. 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc9ffd740 20 client.18501 trim_cache size 0 max 16384 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stdout: "3ae5a417-e193-4936-897f-762443d7c54a": { 2024-03-09T00:03:38.336 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:03:38.337 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "3ae5a417-e193-4936-897f-762443d7c54a", 2024-03-09T00:03:38.337 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:03:38.337 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:03:38.337 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:03:38.337 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febac0a2a20 msgr2=0x7febac0a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febac0a2a20 0x7febac0a4ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febc0101c40 tx=0x7febb0014950 comp rx=0 tx=0).stop 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0100c30 msgr2=0x7febc01a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0100c30 0x7febc01a5d00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7febb802c700 tx=0x7febb8032f20 comp rx=0 tx=0).stop 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 shutdown_connections 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febac0a2a20 0x7febac0a4ee0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc0100c30 0x7febc01a5d00 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc0106630 0x7febc01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 --2- 172.21.15.22:0/2609994764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febc0068760 0x7febc01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 >> 172.21.15.22:0/2609994764 conn(0x7febc00772f0 msgr2=0x7febc00fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 shutdown_connections 2024-03-09T00:03:38.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:03:38.335+0000 7febc777e640 1 -- 172.21.15.22:0/2609994764 wait complete. 2024-03-09T00:03:38.340 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:38 smithi022 ceph-mon[31894]: pgmap v287: 129 pgs: 129 active+clean; 72 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-03-09T00:03:38.341 INFO:tasks.cephfs.filesystem:scrub status for tag:3ae5a417-e193-4936-897f-762443d7c54a - {'path': '/', 'tag': '3ae5a417-e193-4936-897f-762443d7c54a', 'options': 'recursive,force'} 2024-03-09T00:03:38.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:38 smithi062 ceph-mon[36129]: pgmap v287: 129 pgs: 129 active+clean; 72 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-03-09T00:03:38.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:38 smithi192 ceph-mon[35995]: pgmap v287: 129 pgs: 129 active+clean; 72 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-03-09T00:03:38.724 INFO:tasks.workunit.client.0.smithi022.stdout: 1 942227 10.24 MB/sec execute 346 sec latency 20.215 ms 2024-03-09T00:03:39.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:39 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:03:39.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:39 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:39.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:39 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:03:39.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:39 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:39.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:39 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:03:39.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:39 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:39.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:39 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:03:39.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:39 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:39 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:03:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:39 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:39 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:03:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:39 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:03:39.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 943980 10.23 MB/sec execute 347 sec latency 31.182 ms 2024-03-09T00:03:40.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:40 smithi062 ceph-mon[36129]: pgmap v288: 129 pgs: 129 active+clean; 77 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 397 op/s 2024-03-09T00:03:40.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:40 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:03:40.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:40 smithi192 ceph-mon[35995]: pgmap v288: 129 pgs: 129 active+clean; 77 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 397 op/s 2024-03-09T00:03:40.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:40 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:03:40.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:40 smithi022 ceph-mon[31894]: pgmap v288: 129 pgs: 129 active+clean; 77 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 397 op/s 2024-03-09T00:03:40.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:40 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:03:40.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 945827 10.24 MB/sec execute 348 sec latency 15.109 ms 2024-03-09T00:03:41.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 948272 10.24 MB/sec execute 349 sec latency 16.706 ms 2024-03-09T00:03:42.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:42 smithi062 ceph-mon[36129]: pgmap v289: 129 pgs: 129 active+clean; 75 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 391 op/s 2024-03-09T00:03:42.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:42 smithi192 ceph-mon[35995]: pgmap v289: 129 pgs: 129 active+clean; 75 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 391 op/s 2024-03-09T00:03:42.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:42 smithi022 ceph-mon[31894]: pgmap v289: 129 pgs: 129 active+clean; 75 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 391 op/s 2024-03-09T00:03:42.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 949721 10.24 MB/sec execute 350 sec latency 19.857 ms 2024-03-09T00:03:43.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 952314 10.25 MB/sec execute 351 sec latency 18.291 ms 2024-03-09T00:03:44.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:44 smithi062 ceph-mon[36129]: pgmap v290: 129 pgs: 129 active+clean; 75 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 352 op/s 2024-03-09T00:03:44.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:44 smithi192 ceph-mon[35995]: pgmap v290: 129 pgs: 129 active+clean; 75 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 352 op/s 2024-03-09T00:03:44.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:44 smithi022 ceph-mon[31894]: pgmap v290: 129 pgs: 129 active+clean; 75 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 352 op/s 2024-03-09T00:03:44.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 953447 10.23 MB/sec execute 352 sec latency 45.092 ms 2024-03-09T00:03:45.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 955872 10.25 MB/sec execute 353 sec latency 32.284 ms 2024-03-09T00:03:46.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:46 smithi062 ceph-mon[36129]: pgmap v291: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 430 op/s 2024-03-09T00:03:46.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:46 smithi192 ceph-mon[35995]: pgmap v291: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 430 op/s 2024-03-09T00:03:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:46 smithi022 ceph-mon[31894]: pgmap v291: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 430 op/s 2024-03-09T00:03:46.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 957571 10.24 MB/sec execute 354 sec latency 16.451 ms 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 -- 172.21.15.22:0/812870428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe801073e0 msgr2=0x7fbe801077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 --2- 172.21.15.22:0/812870428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe801073e0 0x7fbe801077c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009900 tx=0x7fbe7002bcd0 comp rx=0 tx=0).stop 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 -- 172.21.15.22:0/812870428 shutdown_connections 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 --2- 172.21.15.22:0/812870428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe80069310 0x7fbe8010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 --2- 172.21.15.22:0/812870428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80068970 0x7fbe80068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 --2- 172.21.15.22:0/812870428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe801073e0 0x7fbe801077c0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 -- 172.21.15.22:0/812870428 >> 172.21.15.22:0/812870428 conn(0x7fbe80077040 msgr2=0x7fbe80077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 -- 172.21.15.22:0/812870428 shutdown_connections 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 -- 172.21.15.22:0/812870428 wait complete. 2024-03-09T00:03:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 Processor -- start 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.991+0000 7fbe869b2640 1 -- start start 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe869b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe80068970 0x7fbe8019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe869b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe80069310 0x7fbe8019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe869b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe801073e0 0x7fbe80197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe869b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80111660 con 0x7fbe80068970 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe869b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801114e0 con 0x7fbe80069310 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe869b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801117e0 con 0x7fbe801073e0 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe80068970 0x7fbe8019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe80068970 0x7fbe8019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44082/0 (socket says 172.21.15.22:44082) 2024-03-09T00:03:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 -- 172.21.15.22:0/1924465841 learned_addr learned my addr 172.21.15.22:0/1924465841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe801073e0 0x7fbe80197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe80069310 0x7fbe8019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 -- 172.21.15.22:0/1924465841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe801073e0 msgr2=0x7fbe80197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe801073e0 0x7fbe80197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 -- 172.21.15.22:0/1924465841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe80069310 msgr2=0x7fbe8019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe80069310 0x7fbe8019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 -- 172.21.15.22:0/1924465841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe801980c0 con 0x7fbe80068970 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe80068970 0x7fbe8019d740 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbe70005940 tx=0x7fbe7002cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7d7fa640 1 -- 172.21.15.22:0/1924465841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70041070 con 0x7fbe80068970 2024-03-09T00:03:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe7d7fa640 1 -- 172.21.15.22:0/1924465841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe70004450 con 0x7fbe80068970 2024-03-09T00:03:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.992+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe80198350 con 0x7fbe80068970 2024-03-09T00:03:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.993+0000 7fbe7d7fa640 1 -- 172.21.15.22:0/1924465841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70003df0 con 0x7fbe80068970 2024-03-09T00:03:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.993+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe801a4600 con 0x7fbe80068970 2024-03-09T00:03:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe7d7fa640 1 -- 172.21.15.22:0/1924465841 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbe70044070 con 0x7fbe80068970 2024-03-09T00:03:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 Processor -- start 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 -- start start 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe7d7fa640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbe6c0a2a90 0x7fbe6c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x559748687a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748688450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748693940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597485ec5f0 con 0x559748687fd0 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559748595930 con 0x559748691550 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.995+0000 7fbe8958a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597485e8690 con 0x559748688d40 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbe6c0a2a90 0x7fbe6c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748688450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x559748687a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748688450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44084/0 (socket says 172.21.15.22:44084) 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/3999835516 learned_addr learned my addr 172.21.15.22:0/3999835516 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7d7fa640 1 -- 172.21.15.22:0/1924465841 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fbe700f0c80 con 0x7fbe80068970 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748693940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/3999835516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 msgr2=0x559748687a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x559748687a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/3999835516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 msgr2=0x559748693940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748693940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/3999835516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559748301210 con 0x559748687fd0 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbe6c0a2a90 0x7fbe6c0a4f50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005fd0 tx=0x7fbe78009e60 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748688450 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009bf0 tx=0x7fbe680117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e37a55d95c1c83 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe767fc640 1 -- 172.21.15.22:0/3999835516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68012450 con 0x559748687fd0 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe767fc640 1 -- 172.21.15.22:0/3999835516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe68012ad0 con 0x559748687fd0 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.996+0000 7fbe767fc640 1 -- 172.21.15.22:0/3999835516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800fac0 con 0x559748687fd0 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 -- 172.21.15.22:0/3999835516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 msgr2=0x559748688450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748688450 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009bf0 tx=0x7fbe680117b0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 -- 172.21.15.22:0/3999835516 shutdown_connections 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748693940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748688450 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3999835516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x559748687a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 -- 172.21.15.22:0/3999835516 >> 172.21.15.22:0/3999835516 conn(0x55974864f5b0 msgr2=0x559748682cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 -- 172.21.15.22:0/3999835516 shutdown_connections 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 -- 172.21.15.22:0/3999835516 wait complete. 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.997+0000 7fbe8958a740 1 -- start start 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 1 Processor -- start 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x55974868e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748688d40 0x55974868ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748691550 0x55974868efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559748694280 con 0x559748691550 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597485a8d90 con 0x559748688d40 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559748694400 con 0x559748687fd0 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe8958a740 10 client.0 _ll_register_callbacks cb 0x559748300a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748691550 0x55974868efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748691550 0x55974868efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44100/0 (socket says 172.21.15.22:44100) 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 -- 172.21.15.22:0/1513185334 learned_addr learned my addr 172.21.15.22:0/1513185334 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748688d40 0x55974868ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x55974868e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 -- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 msgr2=0x55974868e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x55974868e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 -- 172.21.15.22:0/1513185334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748688d40 msgr2=0x55974868ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748688d40 0x55974868ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 -- 172.21.15.22:0/1513185334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597483cd620 con 0x559748691550 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748691550 0x55974868efa0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fbe78013a00 tx=0x7fbe78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aea8cc24dd6a22c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.998+0000 7fbe5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.999+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/1513185334 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7801e9a0 con 0x559748691550 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.999+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/1513185334 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fbe7801eb40 con 0x559748691550 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.999+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe68012ad0 con 0x559748691550 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.999+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/1513185334 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe78031050 con 0x559748691550 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.999+0000 7fbe8958a740 10 client.18513 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.999+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5597486a0530 con 0x559748691550 2024-03-09T00:03:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:46.999+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/1513185334 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe68012ad0 con 0x559748691550 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/1513185334 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbe7801f070 con 0x559748691550 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe8958a740 10 client.18513 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe8958a740 10 client.18513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe8958a740 20 client.18513 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe8958a740 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486a19a0 0x5597486a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe8958a740 4 client.18513 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597485f9b30 con 0x5597486a19a0 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486a19a0 0x5597486a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe5f7fe640 10 client.18513 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.000+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486a19a0 0x5597486a3d90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976911d908839a94 server_cookie=1b64d88c85f865d in_seq=0 out_seq=0 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.001+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/1513185334 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597485f9b30 con 0x5597486a19a0 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.001+0000 7fbe5f7fe640 10 client.18513 handle_command_reply: tid=0 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.001+0000 7fbe8958a740 1 client.18513 shutdown 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748691550 msgr2=0x55974868efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748691550 0x55974868efa0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fbe78013a00 tx=0x7fbe78004ab0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 shutdown_connections 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486a19a0 0x5597486a3d90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748691550 0x55974868efa0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748688d40 0x55974868ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe5f7fe640 0 client.18513 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 --2- 172.21.15.22:0/1513185334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x55974868e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 >> 172.21.15.22:0/1513185334 conn(0x55974864f5b0 msgr2=0x559748684a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 shutdown_connections 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 1 -- 172.21.15.22:0/1513185334 wait complete. 2024-03-09T00:03:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.002+0000 7fbe8958a740 20 client.18513 trim_cache size 0 max 16384 2024-03-09T00:03:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.015+0000 7fbe8958a740 1 Processor -- start 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.015+0000 7fbe8958a740 1 -- start start 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748690c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x5597486986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748698be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe8958a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801b0a70 con 0x559748687fd0 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe8958a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801ae610 con 0x559748691550 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe8958a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801ac6d0 con 0x559748688d40 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748690c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748690c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44116/0 (socket says 172.21.15.22:44116) 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 -- 172.21.15.22:0/3277511928 learned_addr learned my addr 172.21.15.22:0/3277511928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x5597486986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748698be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 -- 172.21.15.22:0/3277511928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 msgr2=0x5597486986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x5597486986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 -- 172.21.15.22:0/3277511928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 msgr2=0x559748698be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748698be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 -- 172.21.15.22:0/3277511928 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597486872a0 con 0x559748687fd0 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748690c50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbe700ee140 tx=0x7fbe700dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9dd0e171837e5e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe75ffb640 1 -- 172.21.15.22:0/3277511928 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7004b040 con 0x559748687fd0 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe75ffb640 1 -- 172.21.15.22:0/3277511928 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe7002ce30 con 0x559748687fd0 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.016+0000 7fbe75ffb640 1 -- 172.21.15.22:0/3277511928 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700c5790 con 0x559748687fd0 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 -- 172.21.15.22:0/3277511928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 msgr2=0x559748690c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748690c50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbe700ee140 tx=0x7fbe700dd040 comp rx=0 tx=0).stop 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 -- 172.21.15.22:0/3277511928 shutdown_connections 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748698be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748688d40 0x5597486986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 --2- 172.21.15.22:0/3277511928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748687fd0 0x559748690c50 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 -- 172.21.15.22:0/3277511928 >> 172.21.15.22:0/3277511928 conn(0x55974864f5b0 msgr2=0x5597486a8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 -- 172.21.15.22:0/3277511928 shutdown_connections 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.017+0000 7fbe8958a740 1 -- 172.21.15.22:0/3277511928 wait complete. 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 -- start start 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 Processor -- start 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x559748698940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748688d40 0x559748694cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748695200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801accd0 con 0x559748688d40 2024-03-09T00:03:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801adfb0 con 0x559748691550 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801ad2f0 con 0x559748687fd0 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe8958a740 10 client.0 _ll_register_callbacks cb 0x7fbe68009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748688d40 0x559748694cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748688d40 0x559748694cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44130/0 (socket says 172.21.15.22:44130) 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/188309796 learned_addr learned my addr 172.21.15.22:0/188309796 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x559748698940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe84f4f640 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748695200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 msgr2=0x559748698940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.018+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x559748698940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/188309796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 msgr2=0x559748695200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:03:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748695200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe7f7fe640 1 -- 172.21.15.22:0/188309796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597486958f0 con 0x559748688d40 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe7f7fe640 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748688d40 0x559748694cc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009bf0 tx=0x7fbe6800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e485247c31743d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/188309796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800fd10 con 0x559748688d40 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/188309796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fbe68005040 con 0x559748688d40 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe7002ce30 con 0x559748688d40 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/188309796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe68037050 con 0x559748688d40 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe8958a740 10 client.18519 fetch_fsmap learned FSMap version 35 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5597486f8dc0 con 0x559748688d40 2024-03-09T00:03:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.019+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/188309796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe7002ce30 con 0x559748688d40 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/188309796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbe68003b60 con 0x559748688d40 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe8958a740 10 client.18519 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe8958a740 10 client.18519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe8958a740 20 client.18519 populate_metadata read hostname 'smithi022' 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe8958a740 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486f9230 0x5597486fb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe8958a740 4 client.18519 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5597485f3e80 con 0x5597486f9230 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486f9230 0x5597486fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe5f7fe640 10 client.18519 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.020+0000 7fbe7ffff640 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486f9230 0x5597486fb650 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3485be929f2ec705 server_cookie=3dd9f438e8fa8638 in_seq=0 out_seq=0 2024-03-09T00:03:47.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:47 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:03:47.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:47 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe5f7fe640 1 -- 172.21.15.22:0/188309796 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5597485f3e80 con 0x5597486f9230 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe5f7fe640 10 client.18519 handle_command_reply: tid=0 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 client.18519 shutdown 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748688d40 msgr2=0x559748694cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748688d40 0x559748694cc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009bf0 tx=0x7fbe6800f790 comp rx=0 tx=0).stop 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 shutdown_connections 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597486f9230 0x5597486fb650 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559748691550 0x559748695200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559748688d40 0x559748694cc0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe5f7fe640 0 client.18519 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 --2- 172.21.15.22:0/188309796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559748687fd0 0x559748698940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 >> 172.21.15.22:0/188309796 conn(0x55974864f5b0 msgr2=0x559748689830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 shutdown_connections 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 1 -- 172.21.15.22:0/188309796 wait complete. 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.449+0000 7fbe8958a740 20 client.18519 trim_cache size 0 max 16384 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:03:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:03:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbe6c0a2a90 msgr2=0x7fbe6c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbe6c0a2a90 0x7fbe6c0a4f50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005fd0 tx=0x7fbe78009e60 comp rx=0 tx=0).stop 2024-03-09T00:03:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe80068970 msgr2=0x7fbe8019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe80068970 0x7fbe8019d740 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbe70005940 tx=0x7fbe7002cc40 comp rx=0 tx=0).stop 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 shutdown_connections 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbe6c0a2a90 0x7fbe6c0a4f50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe801073e0 0x7fbe80197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe80069310 0x7fbe8019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 --2- 172.21.15.22:0/1924465841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe80068970 0x7fbe8019d740 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 >> 172.21.15.22:0/1924465841 conn(0x7fbe80077040 msgr2=0x7fbe800ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 shutdown_connections 2024-03-09T00:03:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:03:47.450+0000 7fbe869b2640 1 -- 172.21.15.22:0/1924465841 wait complete. 2024-03-09T00:03:47.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:47 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:03:47.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 959366 10.25 MB/sec execute 355 sec latency 199.119 ms 2024-03-09T00:03:48.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:48 smithi062 ceph-mon[36129]: pgmap v292: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-09T00:03:48.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:48 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:03:48.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:48 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:03:48.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:48 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:03:48.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:48 smithi192 ceph-mon[35995]: pgmap v292: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-09T00:03:48.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:48 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:03:48.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:48 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:03:48.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:48 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:03:48.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:48 smithi022 ceph-mon[31894]: pgmap v292: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-03-09T00:03:48.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:48 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:03:48.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:48 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:03:48.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:48 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:03:48.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 960844 10.24 MB/sec execute 356 sec latency 16.537 ms 2024-03-09T00:03:49.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 963044 10.25 MB/sec execute 357 sec latency 141.027 ms 2024-03-09T00:03:50.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:50 smithi062 ceph-mon[36129]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 267 op/s 2024-03-09T00:03:50.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:50 smithi192 ceph-mon[35995]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 267 op/s 2024-03-09T00:03:50.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:50 smithi022 ceph-mon[31894]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 267 op/s 2024-03-09T00:03:50.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 964260 10.23 MB/sec execute 358 sec latency 52.436 ms 2024-03-09T00:03:51.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 966616 10.24 MB/sec execute 359 sec latency 15.288 ms 2024-03-09T00:03:52.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:52 smithi192 ceph-mon[35995]: pgmap v294: 129 pgs: 129 active+clean; 76 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-03-09T00:03:52.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:52 smithi022 ceph-mon[31894]: pgmap v294: 129 pgs: 129 active+clean; 76 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-03-09T00:03:52.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:52 smithi062 ceph-mon[36129]: pgmap v294: 129 pgs: 129 active+clean; 76 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-03-09T00:03:52.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 968244 10.23 MB/sec execute 360 sec latency 21.444 ms 2024-03-09T00:03:53.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 970451 10.24 MB/sec execute 361 sec latency 15.652 ms 2024-03-09T00:03:54.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:54 smithi192 ceph-mon[35995]: pgmap v295: 129 pgs: 129 active+clean; 76 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.3 MiB/s wr, 288 op/s 2024-03-09T00:03:54.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:54 smithi022 ceph-mon[31894]: pgmap v295: 129 pgs: 129 active+clean; 76 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.3 MiB/s wr, 288 op/s 2024-03-09T00:03:54.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:54 smithi062 ceph-mon[36129]: pgmap v295: 129 pgs: 129 active+clean; 76 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.3 MiB/s wr, 288 op/s 2024-03-09T00:03:54.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 971700 10.23 MB/sec execute 362 sec latency 230.990 ms 2024-03-09T00:03:55.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 973713 10.23 MB/sec execute 363 sec latency 25.254 ms 2024-03-09T00:03:56.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:56 smithi192 ceph-mon[35995]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 390 op/s 2024-03-09T00:03:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:56 smithi022 ceph-mon[31894]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 390 op/s 2024-03-09T00:03:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:56 smithi062 ceph-mon[36129]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 390 op/s 2024-03-09T00:03:56.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 975360 10.22 MB/sec execute 364 sec latency 21.522 ms 2024-03-09T00:03:57.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 977447 10.23 MB/sec execute 365 sec latency 17.245 ms 2024-03-09T00:03:58.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:03:58 smithi192 ceph-mon[35995]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-03-09T00:03:58.412 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:03:58 smithi062 ceph-mon[36129]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-03-09T00:03:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:03:58 smithi022 ceph-mon[31894]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-03-09T00:03:58.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 979441 10.22 MB/sec execute 366 sec latency 18.140 ms 2024-03-09T00:03:59.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 980925 10.23 MB/sec execute 367 sec latency 174.243 ms 2024-03-09T00:04:00.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:00 smithi192 ceph-mon[35995]: pgmap v298: 129 pgs: 129 active+clean; 81 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 353 op/s 2024-03-09T00:04:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:00 smithi022 ceph-mon[31894]: pgmap v298: 129 pgs: 129 active+clean; 81 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 353 op/s 2024-03-09T00:04:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 -- 172.21.15.22:0/3588521013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8100e00 msgr2=0x7f20e810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 --2- 172.21.15.22:0/3588521013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8100e00 0x7f20e810d020 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ca0 tx=0x7f20e002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 -- 172.21.15.22:0/3588521013 shutdown_connections 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 --2- 172.21.15.22:0/3588521013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8100e00 0x7f20e810d020 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 --2- 172.21.15.22:0/3588521013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20e8100440 0x7f20e81008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 --2- 172.21.15.22:0/3588521013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20e8105190 0x7f20e8105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 -- 172.21.15.22:0/3588521013 >> 172.21.15.22:0/3588521013 conn(0x7f20e8077040 msgr2=0x7f20e8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 -- 172.21.15.22:0/3588521013 shutdown_connections 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 -- 172.21.15.22:0/3588521013 wait complete. 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.551+0000 7f20ed57e640 1 Processor -- start 2024-03-09T00:04:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20ed57e640 1 -- start start 2024-03-09T00:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20ed57e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20e8100440 0x7f20e819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20ed57e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20e8100e00 0x7f20e819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20ed57e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8105190 0x7f20e8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20ed57e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e8110650 con 0x7f20e8105190 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20ed57e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81104d0 con 0x7f20e8100e00 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20ed57e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81107d0 con 0x7f20e8100440 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8105190 0x7f20e8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20e8100440 0x7f20e819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8105190 0x7f20e8197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43134/0 (socket says 172.21.15.22:43134) 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e77fe640 1 -- 172.21.15.22:0/2071669014 learned_addr learned my addr 172.21.15.22:0/2071669014 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e67fc640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20e8100e00 0x7f20e819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e6ffd640 1 -- 172.21.15.22:0/2071669014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20e8100e00 msgr2=0x7f20e819dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20e8100e00 0x7f20e819dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e6ffd640 1 -- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8105190 msgr2=0x7f20e8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8105190 0x7f20e8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e6ffd640 1 -- 172.21.15.22:0/2071669014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e81980e0 con 0x7f20e8100440 2024-03-09T00:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.552+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8105190 0x7f20e8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.553+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20e8100440 0x7f20e819d760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20d8005fd0 tx=0x7f20d800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.553+0000 7f20cffff640 1 -- 172.21.15.22:0/2071669014 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8009c30 con 0x7f20e8100440 2024-03-09T00:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.553+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e81983d0 con 0x7f20e8100440 2024-03-09T00:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.553+0000 7f20cffff640 1 -- 172.21.15.22:0/2071669014 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20d8009dd0 con 0x7f20e8100440 2024-03-09T00:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.553+0000 7f20cffff640 1 -- 172.21.15.22:0/2071669014 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8010460 con 0x7f20e8100440 2024-03-09T00:04:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.553+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e81a4620 con 0x7f20e8100440 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.555+0000 7f20cffff640 1 -- 172.21.15.22:0/2071669014 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f20d8003ef0 con 0x7f20e8100440 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.555+0000 7f20efdfd740 1 Processor -- start 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.555+0000 7f20efdfd740 1 -- start start 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116407a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x559116408450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116413940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20efdfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55911636c5f0 con 0x559116408d40 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20efdfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559116315930 con 0x559116407fd0 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20efdfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559116368690 con 0x559116411550 2024-03-09T00:04:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116407a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116407a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43144/0 (socket says 172.21.15.22:43144) 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 -- 172.21.15.22:0/2472847645 learned_addr learned my addr 172.21.15.22:0/2472847645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e67fc640 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116413940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x559116408450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20cffff640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f20d00a2a10 0x7f20d00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 -- 172.21.15.22:0/2472847645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 msgr2=0x559116413940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116413940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 -- 172.21.15.22:0/2472847645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 msgr2=0x559116408450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x559116408450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 -- 172.21.15.22:0/2472847645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559116081210 con 0x559116408d40 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20cffff640 1 -- 172.21.15.22:0/2071669014 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f20d80ced20 con 0x7f20e8100440 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f20d00a2a10 0x7f20d00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.556+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116407a90 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009930 tx=0x7f20e0009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16f4db43ef296df server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20e67fc640 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116413940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x559116408450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20cd7fa640 1 -- 172.21.15.22:0/2472847645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0039040 con 0x559116408d40 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20cd7fa640 1 -- 172.21.15.22:0/2472847645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20e0003e50 con 0x559116408d40 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20cd7fa640 1 -- 172.21.15.22:0/2472847645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00045b0 con 0x559116408d40 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f20d00a2a10 0x7f20d00a4ed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20e004c000 tx=0x7f20e0045000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20efdfd740 1 -- 172.21.15.22:0/2472847645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 msgr2=0x559116407a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.557+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116407a90 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009930 tx=0x7f20e0009960 comp rx=0 tx=0).stop 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- 172.21.15.22:0/2472847645 shutdown_connections 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116413940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x559116408450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2472847645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116407a90 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- 172.21.15.22:0/2472847645 >> 172.21.15.22:0/2472847645 conn(0x5591163cf5b0 msgr2=0x559116402cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- 172.21.15.22:0/2472847645 shutdown_connections 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- 172.21.15.22:0/2472847645 wait complete. 2024-03-09T00:04:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- start start 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 Processor -- start 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116407fd0 0x55911640e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116408d40 0x55911640eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116411550 0x55911640f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559116414280 con 0x559116411550 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559116328d90 con 0x559116408d40 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559116414400 con 0x559116407fd0 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20efdfd740 10 client.0 _ll_register_callbacks cb 0x559116080a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116411550 0x55911640f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116411550 0x55911640f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43146/0 (socket says 172.21.15.22:43146) 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20e77fe640 1 -- 172.21.15.22:0/2732969690 learned_addr learned my addr 172.21.15.22:0/2732969690 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.558+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116407fd0 0x55911640e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e67fc640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116408d40 0x55911640eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e77fe640 1 -- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116407fd0 msgr2=0x55911640e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116407fd0 0x55911640e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e77fe640 1 -- 172.21.15.22:0/2732969690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116408d40 msgr2=0x55911640eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116408d40 0x55911640eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e77fe640 1 -- 172.21.15.22:0/2732969690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55911614d620 con 0x559116411550 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e67fc640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116408d40 0x55911640eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20e77fe640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116411550 0x55911640f030 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004af0 tx=0x7f20e0034a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce50ae9c6caff2db server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20caff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20caff5640 1 -- 172.21.15.22:0/2732969690 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0034700 con 0x559116411550 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20caff5640 1 -- 172.21.15.22:0/2732969690 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f20e0003970 con 0x559116411550 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20e0003e50 con 0x559116411550 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20caff5640 1 -- 172.21.15.22:0/2732969690 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20e0066050 con 0x559116411550 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20efdfd740 10 client.18531 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.559+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559116420590 con 0x559116411550 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20caff5640 1 -- 172.21.15.22:0/2732969690 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20e0003e50 con 0x559116411550 2024-03-09T00:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20caff5640 1 -- 172.21.15.22:0/2732969690 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f20e003f040 con 0x559116411550 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20efdfd740 10 client.18531 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20efdfd740 10 client.18531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20efdfd740 20 client.18531 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591164218e0 0x559116423cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20efdfd740 4 client.18531 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559116379b30 con 0x5591164218e0 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591164218e0 0x559116423cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591164218e0 0x559116423cd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21dfd3de541e7e37 server_cookie=e209356f8d053ab9 in_seq=0 out_seq=0 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.560+0000 7f20caff5640 10 client.18531 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20caff5640 1 -- 172.21.15.22:0/2732969690 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f20d8079e50 con 0x5591164218e0 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20caff5640 10 client.18531 handle_command_reply: tid=0 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 client.18531 shutdown 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116411550 msgr2=0x55911640f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116411550 0x55911640f030 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004af0 tx=0x7f20e0034a00 comp rx=0 tx=0).stop 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 shutdown_connections 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5591164218e0 0x559116423cd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116411550 0x55911640f030 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116408d40 0x55911640eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 --2- 172.21.15.22:0/2732969690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116407fd0 0x55911640e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 >> 172.21.15.22:0/2732969690 conn(0x5591163cf5b0 msgr2=0x559116404a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20caff5640 0 client.18531 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 shutdown_connections 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 1 -- 172.21.15.22:0/2732969690 wait complete. 2024-03-09T00:04:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.561+0000 7f20efdfd740 20 client.18531 trim_cache size 0 max 16384 2024-03-09T00:04:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 Processor -- start 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 -- start start 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116407fd0 0x559116410ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116408d40 0x55911640ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116411550 0x55911640d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81ae830 con 0x559116407fd0 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81ac3d0 con 0x559116411550 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20efdfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81aa490 con 0x559116408d40 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116407fd0 0x559116410ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116407fd0 0x559116410ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43150/0 (socket says 172.21.15.22:43150) 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 -- 172.21.15.22:0/3448121846 learned_addr learned my addr 172.21.15.22:0/3448121846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e77fe640 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116411550 0x55911640d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e67fc640 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116408d40 0x55911640ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 -- 172.21.15.22:0/3448121846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116408d40 msgr2=0x55911640ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116408d40 0x55911640ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 -- 172.21.15.22:0/3448121846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116411550 msgr2=0x55911640d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116411550 0x55911640d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 -- 172.21.15.22:0/3448121846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591164072a0 con 0x559116407fd0 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.575+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116407fd0 0x559116410ce0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f20d80bb0b0 tx=0x7f20d8003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ee32f166ee0d78 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20ccff9640 1 -- 172.21.15.22:0/3448121846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8020040 con 0x559116407fd0 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20ccff9640 1 -- 172.21.15.22:0/3448121846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559116379b30 con 0x559116407fd0 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20ccff9640 1 -- 172.21.15.22:0/3448121846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80976a0 con 0x559116407fd0 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 -- 172.21.15.22:0/3448121846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116407fd0 msgr2=0x559116410ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116407fd0 0x559116410ce0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f20d80bb0b0 tx=0x7f20d8003950 comp rx=0 tx=0).stop 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 -- 172.21.15.22:0/3448121846 shutdown_connections 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116411550 0x55911640d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116408d40 0x55911640ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 --2- 172.21.15.22:0/3448121846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116407fd0 0x559116410ce0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 -- 172.21.15.22:0/3448121846 >> 172.21.15.22:0/3448121846 conn(0x5591163cf5b0 msgr2=0x5591164027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 -- 172.21.15.22:0/3448121846 shutdown_connections 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.576+0000 7f20efdfd740 1 -- 172.21.15.22:0/3448121846 wait complete. 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 -- start start 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 Processor -- start 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x5591164171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116417710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116417c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81aaa90 con 0x559116408d40 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81abd70 con 0x559116407fd0 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81ab0b0 con 0x559116411550 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20efdfd740 10 client.0 _ll_register_callbacks cb 0x7f20e0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116417710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116417710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43158/0 (socket says 172.21.15.22:43158) 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 -- 172.21.15.22:0/1541199559 learned_addr learned my addr 172.21.15.22:0/1541199559 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e77fe640 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116417c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x5591164171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 -- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 msgr2=0x559116417c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116417c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 -- 172.21.15.22:0/1541199559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 msgr2=0x5591164171d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x5591164171d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.577+0000 7f20e67fc640 1 -- 172.21.15.22:0/1541199559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559116404fc0 con 0x559116408d40 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20e67fc640 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116417710 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f20e80775d0 tx=0x7f20dc010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3137701498619935 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20caff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20caff5640 1 -- 172.21.15.22:0/1541199559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc004050 con 0x559116408d40 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20caff5640 1 -- 172.21.15.22:0/1541199559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f20dc019040 con 0x559116408d40 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559116379b30 con 0x559116408d40 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20caff5640 1 -- 172.21.15.22:0/1541199559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20dc029050 con 0x559116408d40 2024-03-09T00:04:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 10 client.18537 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559116478e40 con 0x559116408d40 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20caff5640 1 -- 172.21.15.22:0/1541199559 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559116379b30 con 0x559116408d40 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20caff5640 1 -- 172.21.15.22:0/1541199559 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f20dc0037b0 con 0x559116408d40 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 10 client.18537 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 10 client.18537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 20 client.18537 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559116479620 0x55911647ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 4 client.18537 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.578+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559116373e80 con 0x559116479620 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.579+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559116479620 0x55911647ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.579+0000 7f20e6ffd640 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559116479620 0x55911647ba40 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132a222499a6e15c server_cookie=e7573fce04e29afb in_seq=0 out_seq=0 2024-03-09T00:04:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.579+0000 7f20caff5640 10 client.18537 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:00 smithi062 ceph-mon[36129]: pgmap v298: 129 pgs: 129 active+clean; 81 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 353 op/s 2024-03-09T00:04:00.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 981572 10.20 MB/sec execute 368 sec latency 195.792 ms 2024-03-09T00:04:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20caff5640 1 -- 172.21.15.22:0/1541199559 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559116373e80 con 0x559116479620 2024-03-09T00:04:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20caff5640 10 client.18537 handle_command_reply: tid=0 2024-03-09T00:04:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 client.18537 shutdown 2024-03-09T00:04:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 msgr2=0x559116417710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116417710 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f20e80775d0 tx=0x7f20dc010b40 comp rx=0 tx=0).stop 2024-03-09T00:04:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 shutdown_connections 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559116479620 0x55911647ba40 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559116411550 0x559116417c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559116408d40 0x559116417710 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 --2- 172.21.15.22:0/1541199559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559116407fd0 0x5591164171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20caff5640 0 client.18537 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.921+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 >> 172.21.15.22:0/1541199559 conn(0x5591163cf5b0 msgr2=0x55911640ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 shutdown_connections 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20efdfd740 1 -- 172.21.15.22:0/1541199559 wait complete. 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20efdfd740 20 client.18537 trim_cache size 0 max 16384 2024-03-09T00:04:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:04:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:04:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:04:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f20d00a2a10 msgr2=0x7f20d00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f20d00a2a10 0x7f20d00a4ed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20e004c000 tx=0x7f20e0045000 comp rx=0 tx=0).stop 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20e8100440 msgr2=0x7f20e819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20e8100440 0x7f20e819d760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20d8005fd0 tx=0x7f20d800dde0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 shutdown_connections 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f20d00a2a10 0x7f20d00a4ed0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20e8105190 0x7f20e8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20e8100e00 0x7f20e819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 --2- 172.21.15.22:0/2071669014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20e8100440 0x7f20e819d760 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 >> 172.21.15.22:0/2071669014 conn(0x7f20e8077040 msgr2=0x7f20e8068b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 shutdown_connections 2024-03-09T00:04:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:00.922+0000 7f20ed57e640 1 -- 172.21.15.22:0/2071669014 wait complete. 2024-03-09T00:04:01.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 981592 10.18 MB/sec execute 369 sec latency 313.656 ms 2024-03-09T00:04:02.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:02 smithi192 ceph-mon[35995]: pgmap v299: 129 pgs: 129 active+clean; 87 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 326 op/s 2024-03-09T00:04:02.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:02 smithi022 ceph-mon[31894]: pgmap v299: 129 pgs: 129 active+clean; 87 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 326 op/s 2024-03-09T00:04:02.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:02 smithi062 ceph-mon[36129]: pgmap v299: 129 pgs: 129 active+clean; 87 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 326 op/s 2024-03-09T00:04:02.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 983933 10.18 MB/sec execute 370 sec latency 90.948 ms 2024-03-09T00:04:03.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 985014 10.17 MB/sec execute 371 sec latency 18.362 ms 2024-03-09T00:04:04.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:04 smithi192 ceph-mon[35995]: pgmap v300: 129 pgs: 129 active+clean; 87 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-03-09T00:04:04.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:04 smithi022 ceph-mon[31894]: pgmap v300: 129 pgs: 129 active+clean; 87 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-03-09T00:04:04.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:04 smithi062 ceph-mon[36129]: pgmap v300: 129 pgs: 129 active+clean; 87 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-03-09T00:04:04.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 987087 10.18 MB/sec execute 372 sec latency 164.603 ms 2024-03-09T00:04:05.725 INFO:tasks.workunit.client.0.smithi022.stdout: 1 987791 10.16 MB/sec execute 373 sec latency 54.017 ms 2024-03-09T00:04:06.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:06 smithi022 ceph-mon[31894]: pgmap v301: 129 pgs: 129 active+clean; 91 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 415 op/s 2024-03-09T00:04:06.586 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:06 smithi022 ceph-mon[31894]: osdmap e107: 12 total, 12 up, 12 in 2024-03-09T00:04:06.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:06 smithi062 ceph-mon[36129]: pgmap v301: 129 pgs: 129 active+clean; 91 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 415 op/s 2024-03-09T00:04:06.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:06 smithi062 ceph-mon[36129]: osdmap e107: 12 total, 12 up, 12 in 2024-03-09T00:04:06.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:06 smithi192 ceph-mon[35995]: pgmap v301: 129 pgs: 129 active+clean; 91 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.0 MiB/s wr, 415 op/s 2024-03-09T00:04:06.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:06 smithi192 ceph-mon[35995]: osdmap e107: 12 total, 12 up, 12 in 2024-03-09T00:04:06.726 INFO:tasks.workunit.client.0.smithi022.stdout: 1 988020 10.14 MB/sec execute 374 sec latency 55.425 ms 2024-03-09T00:04:07.726 INFO:tasks.workunit.client.0.smithi022.stdout: 1 988317 10.11 MB/sec execute 375 sec latency 130.688 ms 2024-03-09T00:04:08.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:04:08.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:04:08.604 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:08 smithi022 ceph-mon[31894]: pgmap v303: 129 pgs: 129 active+clean; 91 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 376 op/s 2024-03-09T00:04:08.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:08 smithi062 ceph-mon[36129]: pgmap v303: 129 pgs: 129 active+clean; 91 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 376 op/s 2024-03-09T00:04:08.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:08 smithi192 ceph-mon[35995]: pgmap v303: 129 pgs: 129 active+clean; 91 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 376 op/s 2024-03-09T00:04:08.726 INFO:tasks.workunit.client.0.smithi022.stdout: 1 988564 10.09 MB/sec execute 376 sec latency 83.993 ms 2024-03-09T00:04:08.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.763+0000 7f9bea042640 1 -- 172.21.15.22:0/1882437714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc09d500 msgr2=0x7f9bdc0a0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.763+0000 7f9bea042640 1 --2- 172.21.15.22:0/1882437714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc09d500 0x7f9bdc0a0f30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9be4066fc0 tx=0x7f9be4096f50 comp rx=0 tx=0).stop 2024-03-09T00:04:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.764+0000 7f9bea042640 1 -- 172.21.15.22:0/1882437714 shutdown_connections 2024-03-09T00:04:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.764+0000 7f9bea042640 1 --2- 172.21.15.22:0/1882437714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc0a1610 0x7f9bdc0a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.764+0000 7f9bea042640 1 --2- 172.21.15.22:0/1882437714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc09d500 0x7f9bdc0a0f30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.764+0000 7f9bea042640 1 --2- 172.21.15.22:0/1882437714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc09cb50 0x7f9bdc09cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.765+0000 7f9bea042640 1 -- 172.21.15.22:0/1882437714 >> 172.21.15.22:0/1882437714 conn(0x7f9bdc0092a0 msgr2=0x7f9bdc0096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:08.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.766+0000 7f9bea042640 1 -- 172.21.15.22:0/1882437714 shutdown_connections 2024-03-09T00:04:08.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.767+0000 7f9bea042640 1 -- 172.21.15.22:0/1882437714 wait complete. 2024-03-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.767+0000 7f9bea042640 1 Processor -- start 2024-03-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.767+0000 7f9bea042640 1 -- start start 2024-03-09T00:04:08.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9bea042640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc09cb50 0x7f9bdc1497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9bea042640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc0a1610 0x7f9bdc149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9bea042640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc143990 0x7f9bdc143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9bea042640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc0bd610 con 0x7f9bdc0a1610 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9bea042640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc0bd490 con 0x7f9bdc09cb50 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9bea042640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc0bd790 con 0x7f9bdc143990 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9be883f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc0a1610 0x7f9bdc149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9be883f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc0a1610 0x7f9bdc149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43184/0 (socket says 172.21.15.22:43184) 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9be9040640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc09cb50 0x7f9bdc1497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9be883f640 1 -- 172.21.15.22:0/3777184172 learned_addr learned my addr 172.21.15.22:0/3777184172 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.768+0000 7f9be9841640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc143990 0x7f9bdc143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9be9040640 1 -- 172.21.15.22:0/3777184172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc143990 msgr2=0x7f9bdc143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9be9040640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc143990 0x7f9bdc143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9be9040640 1 -- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc0a1610 msgr2=0x7f9bdc149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9be9040640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc0a1610 0x7f9bdc149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9be9040640 1 -- 172.21.15.22:0/3777184172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc1444b0 con 0x7f9bdc09cb50 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9be9841640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc143990 0x7f9bdc143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9be9040640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc09cb50 0x7f9bdc1497e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9be000d750 tx=0x7f9be000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9bda7fc640 1 -- 172.21.15.22:0/3777184172 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0039070 con 0x7f9bdc09cb50 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.769+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc1447a0 con 0x7f9bdc09cb50 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.770+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc00f6a0 con 0x7f9bdc09cb50 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.770+0000 7f9bda7fc640 1 -- 172.21.15.22:0/3777184172 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9be0032470 con 0x7f9bdc09cb50 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.770+0000 7f9bda7fc640 1 -- 172.21.15.22:0/3777184172 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0031630 con 0x7f9bdc09cb50 2024-03-09T00:04:08.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.773+0000 7f9bedf64740 1 Processor -- start 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.773+0000 7f9bedf64740 1 -- start start 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.774+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589c940 0x56421589cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.774+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589d260 0x5642158a5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.774+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.774+0000 7f9bedf64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642157ffe00 con 0x56421589d260 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.774+0000 7f9bedf64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642157a8b10 con 0x56421589c940 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.774+0000 7f9bedf64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642157fbea0 con 0x5642158a5e40 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59308/0 (socket says 172.21.15.22:59308) 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 -- 172.21.15.22:0/2352313568 learned_addr learned my addr 172.21.15.22:0/2352313568 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:08.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be9040640 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589d260 0x5642158a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9bda7fc640 1 -- 172.21.15.22:0/3777184172 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9be003c070 con 0x7f9bdc09cb50 2024-03-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 -- 172.21.15.22:0/2352313568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589c940 msgr2=0x56421589cd20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589c940 0x56421589cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 -- 172.21.15.22:0/2352313568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589d260 msgr2=0x5642158a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:08.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589d260 0x5642158a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.775+0000 7f9be883f640 1 -- 172.21.15.22:0/2352313568 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642155de7d0 con 0x5642158a5e40 2024-03-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.776+0000 7f9bda7fc640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd00a2ab0 0x7f9bd00a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.776+0000 7f9bda7fc640 1 -- 172.21.15.22:0/3777184172 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f9be00f1250 con 0x7f9bdc09cb50 2024-03-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.777+0000 7f9be9841640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd00a2ab0 0x7f9bd00a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.778+0000 7f9be9841640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd00a2ab0 0x7f9bd00a4f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005fd0 tx=0x7f9bd4009260 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:08.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.782+0000 7f9be883f640 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a8230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9be404f320 tx=0x7f9be4067d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00902bb7f04850d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:08.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.784+0000 7f9bcf7fe640 1 -- 172.21.15.22:0/2352313568 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40762e0 con 0x5642158a5e40 2024-03-09T00:04:08.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.785+0000 7f9bcf7fe640 1 -- 172.21.15.22:0/2352313568 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9be409b040 con 0x5642158a5e40 2024-03-09T00:04:08.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.786+0000 7f9bedf64740 1 -- 172.21.15.22:0/2352313568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 msgr2=0x5642158a8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.786+0000 7f9bedf64740 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a8230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9be404f320 tx=0x7f9be4067d10 comp rx=0 tx=0).stop 2024-03-09T00:04:08.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.786+0000 7f9bedf64740 1 -- 172.21.15.22:0/2352313568 shutdown_connections 2024-03-09T00:04:08.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.786+0000 7f9bedf64740 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a8230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.786+0000 7f9bedf64740 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589d260 0x5642158a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.786+0000 7f9bedf64740 1 --2- 172.21.15.22:0/2352313568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589c940 0x56421589cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.786+0000 7f9bedf64740 1 -- 172.21.15.22:0/2352313568 >> 172.21.15.22:0/2352313568 conn(0x564215897070 msgr2=0x564215897450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:08.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.789+0000 7f9bedf64740 1 -- 172.21.15.22:0/2352313568 shutdown_connections 2024-03-09T00:04:08.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.792+0000 7f9bedf64740 1 -- 172.21.15.22:0/2352313568 wait complete. 2024-03-09T00:04:08.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.792+0000 7f9bedf64740 1 -- start start 2024-03-09T00:04:08.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.792+0000 7f9bedf64740 1 Processor -- start 2024-03-09T00:04:08.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9bedf64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642158a8c20 con 0x56421589c940 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9bedf64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642157bc0a0 con 0x56421589d260 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9bedf64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642158a8da0 con 0x5642158a5e40 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9bedf64740 10 client.0 _ll_register_callbacks cb 0x5642157bba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9be9040640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9be9040640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158a53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43208/0 (socket says 172.21.15.22:43208) 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9be9040640 1 -- 172.21.15.22:0/4168647065 learned_addr learned my addr 172.21.15.22:0/4168647065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.793+0000 7f9be883f640 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.794+0000 7f9be9040640 1 -- 172.21.15.22:0/4168647065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 msgr2=0x5642158a2910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.794+0000 7f9be9040640 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a2910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.794+0000 7f9be9040640 1 -- 172.21.15.22:0/4168647065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 msgr2=0x5642158a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.794+0000 7f9be9040640 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.794+0000 7f9be9040640 1 -- 172.21.15.22:0/4168647065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642155143f0 con 0x56421589c940 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.794+0000 7f9be9040640 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158a53a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9be00bd390 tx=0x7f9be00bcf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f41689ac183cf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.794+0000 7f9bccff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.795+0000 7f9bccff9640 1 -- 172.21.15.22:0/4168647065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0043030 con 0x56421589c940 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.795+0000 7f9bccff9640 1 -- 172.21.15.22:0/4168647065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f9be00ee770 con 0x56421589c940 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.795+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9be409b040 con 0x56421589c940 2024-03-09T00:04:08.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.795+0000 7f9bccff9640 1 -- 172.21.15.22:0/4168647065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be00ee970 con 0x56421589c940 2024-03-09T00:04:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.795+0000 7f9bedf64740 10 client.18549 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.795+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5642154d40f0 con 0x56421589c940 2024-03-09T00:04:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bccff9640 1 -- 172.21.15.22:0/4168647065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9be409b040 con 0x56421589c940 2024-03-09T00:04:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bccff9640 1 -- 172.21.15.22:0/4168647065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9be00c1170 con 0x56421589c940 2024-03-09T00:04:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bedf64740 10 client.18549 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bedf64740 10 client.18549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bedf64740 20 client.18549 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:08.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bedf64740 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642158b5900 0x5642158b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bedf64740 4 client.18549 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:08.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56421580d340 con 0x5642158b5900 2024-03-09T00:04:08.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.796+0000 7f9be883f640 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642158b5900 0x5642158b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.797+0000 7f9bccff9640 10 client.18549 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.797+0000 7f9be883f640 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642158b5900 0x5642158b7dc0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d334af85035e31 server_cookie=39aa4eded197a7e9 in_seq=0 out_seq=0 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.798+0000 7f9bccff9640 1 -- 172.21.15.22:0/4168647065 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56421580d340 con 0x5642158b5900 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.798+0000 7f9bccff9640 10 client.18549 handle_command_reply: tid=0 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 client.18549 shutdown 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 msgr2=0x5642158a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158a53a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9be00bd390 tx=0x7f9be00bcf40 comp rx=0 tx=0).stop 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 shutdown_connections 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642158b5900 0x5642158b7dc0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 --2- 172.21.15.22:0/4168647065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158a53a0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 >> 172.21.15.22:0/4168647065 conn(0x564215897070 msgr2=0x564215898dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bccff9640 0 client.18549 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 shutdown_connections 2024-03-09T00:04:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 1 -- 172.21.15.22:0/4168647065 wait complete. 2024-03-09T00:04:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.799+0000 7f9bedf64740 20 client.18549 trim_cache size 0 max 16384 2024-03-09T00:04:08.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.827+0000 7f9bedf64740 1 Processor -- start 2024-03-09T00:04:08.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.827+0000 7f9bedf64740 1 -- start start 2024-03-09T00:04:08.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.827+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56421589c940 0x5642158b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.827+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9be9040640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56421589c940 0x5642158b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9be883f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9be9040640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56421589c940 0x5642158b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59324/0 (socket says 172.21.15.22:59324) 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9be9040640 1 -- 172.21.15.22:0/3006237999 learned_addr learned my addr 172.21.15.22:0/3006237999 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642158a5e40 0x56421589d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80125a0 con 0x5642158a5e40 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8010140 con 0x56421589d260 2024-03-09T00:04:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.828+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc800e200 con 0x56421589c940 2024-03-09T00:04:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.829+0000 7f9be9841640 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642158a5e40 0x56421589d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.830+0000 7f9be9040640 1 -- 172.21.15.22:0/3006237999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 msgr2=0x5642158bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.830+0000 7f9be9040640 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158bc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.830+0000 7f9be9040640 1 -- 172.21.15.22:0/3006237999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642158a5e40 msgr2=0x56421589d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.830+0000 7f9be9040640 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642158a5e40 0x56421589d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.830+0000 7f9be9040640 1 -- 172.21.15.22:0/3006237999 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642158a2f10 con 0x56421589c940 2024-03-09T00:04:08.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.832+0000 7f9be9040640 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56421589c940 0x5642158b7910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9be00bb6b0 tx=0x7f9be00bd070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd8b4bd43214ca9b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:08.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.834+0000 7f9bceffd640 1 -- 172.21.15.22:0/3006237999 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00bd400 con 0x56421589c940 2024-03-09T00:04:08.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.834+0000 7f9bceffd640 1 -- 172.21.15.22:0/3006237999 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9be0009b20 con 0x56421589c940 2024-03-09T00:04:08.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.835+0000 7f9bceffd640 1 -- 172.21.15.22:0/3006237999 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00ee770 con 0x56421589c940 2024-03-09T00:04:08.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.835+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56421589c940 msgr2=0x5642158b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.835+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56421589c940 0x5642158b7910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9be00bb6b0 tx=0x7f9be00bd070 comp rx=0 tx=0).stop 2024-03-09T00:04:08.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.837+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 shutdown_connections 2024-03-09T00:04:08.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.837+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642158a5e40 0x56421589d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.837+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.837+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3006237999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56421589c940 0x5642158b7910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.837+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 >> 172.21.15.22:0/3006237999 conn(0x564215897070 msgr2=0x5642158a31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:08.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.838+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 shutdown_connections 2024-03-09T00:04:08.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.838+0000 7f9bedf64740 1 -- 172.21.15.22:0/3006237999 wait complete. 2024-03-09T00:04:08.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.838+0000 7f9bedf64740 1 -- start start 2024-03-09T00:04:08.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.838+0000 7f9bedf64740 1 Processor -- start 2024-03-09T00:04:08.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9bedf64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9bedf64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc800e800 con 0x56421589c940 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9bedf64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc800fae0 con 0x56421589d260 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9bedf64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc800ee20 con 0x5642158a5e40 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9be9040640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9be9040640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158ad320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43216/0 (socket says 172.21.15.22:43216) 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9be9040640 1 -- 172.21.15.22:0/3153750764 learned_addr learned my addr 172.21.15.22:0/3153750764 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9be883f640 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9bedf64740 10 client.0 _ll_register_callbacks cb 0x7f9be00b9770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.839+0000 7f9be9841640 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9be883f640 1 -- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 msgr2=0x5642158a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9be883f640 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9be883f640 1 -- 172.21.15.22:0/3153750764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 msgr2=0x5642158ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9be883f640 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9be883f640 1 -- 172.21.15.22:0/3153750764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642158aa1c0 con 0x56421589d260 2024-03-09T00:04:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9be883f640 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a9690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5642158a3d00 tx=0x7f9be4073410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ccc558a23110e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9bccff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:04:08.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9bccff9640 1 -- 172.21.15.22:0/3153750764 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40762e0 con 0x56421589d260 2024-03-09T00:04:08.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564215807690 con 0x56421589d260 2024-03-09T00:04:08.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.840+0000 7f9bccff9640 1 -- 172.21.15.22:0/3153750764 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f9be4067770 con 0x56421589d260 2024-03-09T00:04:08.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.841+0000 7f9bccff9640 1 -- 172.21.15.22:0/3153750764 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be4076480 con 0x56421589d260 2024-03-09T00:04:08.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.842+0000 7f9bedf64740 10 client.27896 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:08.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.842+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56421590cc90 con 0x56421589d260 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bccff9640 1 -- 172.21.15.22:0/3153750764 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564215807690 con 0x56421589d260 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bccff9640 1 -- 172.21.15.22:0/3153750764 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9be40ac520 con 0x56421589d260 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bedf64740 10 client.27896 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bedf64740 10 client.27896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bedf64740 20 client.27896 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56421590d3a0 0x56421590f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bedf64740 4 client.27896 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:04:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.844+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642157854e0 con 0x56421590d3a0 2024-03-09T00:04:08.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.846+0000 7f9be9040640 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56421590d3a0 0x56421590f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:08.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.846+0000 7f9be9040640 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56421590d3a0 0x56421590f7c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238185f88aff8b0d server_cookie=5c0dd660b0b50010 in_seq=0 out_seq=0 2024-03-09T00:04:08.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.846+0000 7f9bccff9640 10 client.27896 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bccff9640 1 -- 172.21.15.22:0/3153750764 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5642157854e0 con 0x56421590d3a0 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bccff9640 10 client.27896 handle_command_reply: tid=0 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 client.27896 shutdown 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 msgr2=0x5642158a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a9690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5642158a3d00 tx=0x7f9be4073410 comp rx=0 tx=0).stop 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 shutdown_connections 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56421590d3a0 0x56421590f7c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642158a5e40 0x5642158a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56421589d260 0x5642158a9690 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5642158a3d00 tx=0x7f9be4073410 comp rx=0 tx=0).stop 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 --2- 172.21.15.22:0/3153750764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56421589c940 0x5642158ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 >> 172.21.15.22:0/3153750764 conn(0x564215897070 msgr2=0x5642158b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.847+0000 7f9bccff9640 0 client.27896 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:08.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.848+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 shutdown_connections 2024-03-09T00:04:08.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.848+0000 7f9bedf64740 1 -- 172.21.15.22:0/3153750764 wait complete. 2024-03-09T00:04:08.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.849+0000 7f9bedf64740 20 client.27896 trim_cache size 0 max 16384 2024-03-09T00:04:08.844 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:04:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:04:08.845 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:08.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.851+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd00a2ab0 msgr2=0x7f9bd00a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.851+0000 7f9bea042640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd00a2ab0 0x7f9bd00a4f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005fd0 tx=0x7f9bd4009260 comp rx=0 tx=0).stop 2024-03-09T00:04:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.851+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc09cb50 msgr2=0x7f9bdc1497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:08.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.851+0000 7f9bea042640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc09cb50 0x7f9bdc1497e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9be000d750 tx=0x7f9be000dc10 comp rx=0 tx=0).stop 2024-03-09T00:04:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.852+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 shutdown_connections 2024-03-09T00:04:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.852+0000 7f9bea042640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd00a2ab0 0x7f9bd00a4f70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.852+0000 7f9bea042640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc143990 0x7f9bdc143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.852+0000 7f9bea042640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bdc0a1610 0x7f9bdc149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.852+0000 7f9bea042640 1 --2- 172.21.15.22:0/3777184172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bdc09cb50 0x7f9bdc1497e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.852+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 >> 172.21.15.22:0/3777184172 conn(0x7f9bdc0092a0 msgr2=0x7f9bdc09b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.853+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 shutdown_connections 2024-03-09T00:04:08.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:08.853+0000 7f9bea042640 1 -- 172.21.15.22:0/3777184172 wait complete. 2024-03-09T00:04:08.867 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:04:08.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:04:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.045+0000 7f6bd770a640 1 -- 172.21.15.22:0/3122012380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd0100460 msgr2=0x7f6bd0100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.046+0000 7f6bd770a640 1 --2- 172.21.15.22:0/3122012380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd0100460 0x7f6bd0100840 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009900 tx=0x7f6bcc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.047+0000 7f6bd770a640 1 -- 172.21.15.22:0/3122012380 shutdown_connections 2024-03-09T00:04:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.047+0000 7f6bd770a640 1 --2- 172.21.15.22:0/3122012380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd0105530 0x7f6bd010d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.047+0000 7f6bd770a640 1 --2- 172.21.15.22:0/3122012380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd0100e10 0x7f6bd0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.047+0000 7f6bd770a640 1 --2- 172.21.15.22:0/3122012380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd0100460 0x7f6bd0100840 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.047+0000 7f6bd770a640 1 -- 172.21.15.22:0/3122012380 >> 172.21.15.22:0/3122012380 conn(0x7f6bd00772f0 msgr2=0x7f6bd0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.048+0000 7f6bd770a640 1 -- 172.21.15.22:0/3122012380 shutdown_connections 2024-03-09T00:04:09.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.048+0000 7f6bd770a640 1 -- 172.21.15.22:0/3122012380 wait complete. 2024-03-09T00:04:09.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.049+0000 7f6bd770a640 1 Processor -- start 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.050+0000 7f6bd770a640 1 -- start start 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd770a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd0100460 0x7f6bd01ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd770a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd0105530 0x7f6bd01acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd770a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01a67b0 0x7f6bd01a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd770a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd010f520 con 0x7f6bd01a67b0 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd770a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd010f3a0 con 0x7f6bd0105530 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd770a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd010f6a0 con 0x7f6bd0100460 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01a67b0 0x7f6bd01a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01a67b0 0x7f6bd01a6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43222/0 (socket says 172.21.15.22:43222) 2024-03-09T00:04:09.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 -- 172.21.15.22:0/567711401 learned_addr learned my addr 172.21.15.22:0/567711401 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:09.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 -- 172.21.15.22:0/567711401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd0100460 msgr2=0x7f6bd01ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd0100460 0x7f6bd01ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 -- 172.21.15.22:0/567711401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd0105530 msgr2=0x7f6bd01acb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:04:09.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd0105530 0x7f6bd01acb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.051+0000 7f6bd5ca7640 1 -- 172.21.15.22:0/567711401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd01a7360 con 0x7f6bd01a67b0 2024-03-09T00:04:09.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.052+0000 7f6bd5ca7640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01a67b0 0x7f6bd01a6c60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800eb10 tx=0x7f6bc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.053+0000 7f6bc67fc640 1 -- 172.21.15.22:0/567711401 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8004b20 con 0x7f6bd01a67b0 2024-03-09T00:04:09.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.053+0000 7f6bc67fc640 1 -- 172.21.15.22:0/567711401 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc8004cc0 con 0x7f6bd01a67b0 2024-03-09T00:04:09.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.053+0000 7f6bc67fc640 1 -- 172.21.15.22:0/567711401 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8010760 con 0x7f6bd01a67b0 2024-03-09T00:04:09.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.054+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd006ad10 con 0x7f6bd01a67b0 2024-03-09T00:04:09.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.054+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd006b1e0 con 0x7f6bd01a67b0 2024-03-09T00:04:09.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.057+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd0101930 con 0x7f6bd01a67b0 2024-03-09T00:04:09.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.059+0000 7f6bc67fc640 1 -- 172.21.15.22:0/567711401 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6bc8010900 con 0x7f6bd01a67b0 2024-03-09T00:04:09.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.060+0000 7f6bc67fc640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bbc0a2b70 0x7f6bbc0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.060+0000 7f6bc67fc640 1 -- 172.21.15.22:0/567711401 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f6bc80cf980 con 0x7f6bd01a67b0 2024-03-09T00:04:09.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.065+0000 7f6bd54a6640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bbc0a2b70 0x7f6bbc0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.067+0000 7f6bd54a6640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bbc0a2b70 0x7f6bbc0a5030 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005c20 tx=0x7f6bcc000bc0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.067+0000 7f6bc67fc640 1 -- 172.21.15.22:0/567711401 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6bc809c230 con 0x7f6bd01a67b0 2024-03-09T00:04:09.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.507+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bd01a7ce0 con 0x7f6bd01a67b0 2024-03-09T00:04:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.509+0000 7f6bc67fc640 1 -- 172.21.15.22:0/567711401 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f6bd01a7ce0 con 0x7f6bd01a67b0 2024-03-09T00:04:09.504 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:04:09.504 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:04:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:04:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bbc0a2b70 msgr2=0x7f6bbc0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bbc0a2b70 0x7f6bbc0a5030 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc005c20 tx=0x7f6bcc000bc0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01a67b0 msgr2=0x7f6bd01a6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01a67b0 0x7f6bd01a6c60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800eb10 tx=0x7f6bc800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 shutdown_connections 2024-03-09T00:04:09.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bbc0a2b70 0x7f6bbc0a5030 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01a67b0 0x7f6bd01a6c60 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd0105530 0x7f6bd01acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 --2- 172.21.15.22:0/567711401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd0100460 0x7f6bd01ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.512+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 >> 172.21.15.22:0/567711401 conn(0x7f6bd00772f0 msgr2=0x7f6bd00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.515+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 shutdown_connections 2024-03-09T00:04:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.515+0000 7f6bd770a640 1 -- 172.21.15.22:0/567711401 wait complete. 2024-03-09T00:04:09.528 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:09.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:04:09.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.711+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1657402490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 msgr2=0x7fe0a81055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.711+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1657402490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 0x7fe0a81055c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0a001a940 tx=0x7fe0a003cc60 comp rx=0 tx=0).stop 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.716+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1657402490 shutdown_connections 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.716+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1657402490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 0x7fe0a810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.716+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1657402490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0a81004d0 0x7fe0a8100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.716+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1657402490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 0x7fe0a81055c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.716+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1657402490 >> 172.21.15.22:0/1657402490 conn(0x7fe0a80771d0 msgr2=0x7fe0a80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.717+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1657402490 shutdown_connections 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.717+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1657402490 wait complete. 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 Processor -- start 2024-03-09T00:04:09.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 -- start start 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0a81004d0 0x7fe0a8072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 0x7fe0a806dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 0x7fe0a806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a806c5e0 con 0x7fe0a8100e70 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a806c460 con 0x7fe0a81004d0 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.718+0000 7fe0ae57e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a806c760 con 0x7fe0a81051e0 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 0x7fe0a806dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0acb1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 0x7fe0a806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 0x7fe0a806dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43230/0 (socket says 172.21.15.22:43230) 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 -- 172.21.15.22:0/1900148997 learned_addr learned my addr 172.21.15.22:0/1900148997 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a7fff640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0a81004d0 0x7fe0a8072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 -- 172.21.15.22:0/1900148997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 msgr2=0x7fe0a806e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 0x7fe0a806e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 -- 172.21.15.22:0/1900148997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0a81004d0 msgr2=0x7fe0a8072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0a81004d0 0x7fe0a8072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.719+0000 7fe0a77fe640 1 -- 172.21.15.22:0/1900148997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a806e670 con 0x7fe0a8100e70 2024-03-09T00:04:09.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.720+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 0x7fe0a806dbf0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00d980 tx=0x7fe09c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.720+0000 7fe0a57fa640 1 -- 172.21.15.22:0/1900148997 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c017030 con 0x7fe0a8100e70 2024-03-09T00:04:09.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.720+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a806e960 con 0x7fe0a8100e70 2024-03-09T00:04:09.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.720+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a8114fe0 con 0x7fe0a8100e70 2024-03-09T00:04:09.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.723+0000 7fe0a57fa640 1 -- 172.21.15.22:0/1900148997 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe09c0099d0 con 0x7fe0a8100e70 2024-03-09T00:04:09.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.724+0000 7fe0a57fa640 1 -- 172.21.15.22:0/1900148997 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0047a0 con 0x7fe0a8100e70 2024-03-09T00:04:09.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.724+0000 7fe0b0dfd740 1 Processor -- start 2024-03-09T00:04:09.726 INFO:tasks.workunit.client.0.smithi022.stdout: 1 989076 10.07 MB/sec execute 377 sec latency 84.994 ms 2024-03-09T00:04:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.733+0000 7fe0a57fa640 1 -- 172.21.15.22:0/1900148997 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe09c009b80 con 0x7fe0a8100e70 2024-03-09T00:04:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.734+0000 7fe0a57fa640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe0900a2a30 0x7fe0900a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.734+0000 7fe0a57fa640 1 -- 172.21.15.22:0/1900148997 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe09c0cec20 con 0x7fe0a8100e70 2024-03-09T00:04:09.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0b0dfd740 1 -- start start 2024-03-09T00:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991760580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991769160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917696a0 0x55999176ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559991727360 con 0x559991760ac0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599916c3ec0 con 0x5599917601a0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999166cd30 con 0x5599917696a0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.735+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe0900a2a30 0x7fe0900a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917696a0 0x55999176ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917696a0 0x55999176ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59372/0 (socket says 172.21.15.22:59372) 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 -- 172.21.15.22:0/351053644 learned_addr learned my addr 172.21.15.22:0/351053644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991769160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0a7fff640 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991760580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe0900a2a30 0x7fe0900a4ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c830 tx=0x7fe09800cc30 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 -- 172.21.15.22:0/351053644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 msgr2=0x559991760580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991760580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 -- 172.21.15.22:0/351053644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 msgr2=0x559991769160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:09.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991769160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.736+0000 7fe0acb1b640 1 -- 172.21.15.22:0/351053644 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999166bbc0 con 0x5599917696a0 2024-03-09T00:04:09.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.739+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917696a0 0x55999176ba90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe098009d60 tx=0x7fe09801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f51fd27c2496a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.742+0000 7fe08e7fc640 1 -- 172.21.15.22:0/351053644 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09801cd80 con 0x5599917696a0 2024-03-09T00:04:09.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.742+0000 7fe08e7fc640 1 -- 172.21.15.22:0/351053644 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe098020070 con 0x5599917696a0 2024-03-09T00:04:09.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.742+0000 7fe08e7fc640 1 -- 172.21.15.22:0/351053644 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980235f0 con 0x5599917696a0 2024-03-09T00:04:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.743+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/351053644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917696a0 msgr2=0x55999176ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.743+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917696a0 0x55999176ba90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe098009d60 tx=0x7fe09801d000 comp rx=0 tx=0).stop 2024-03-09T00:04:09.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.744+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/351053644 shutdown_connections 2024-03-09T00:04:09.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.744+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917696a0 0x55999176ba90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.744+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991769160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.744+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/351053644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991760580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.744+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/351053644 >> 172.21.15.22:0/351053644 conn(0x55999175a8d0 msgr2=0x55999175acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.746+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/351053644 shutdown_connections 2024-03-09T00:04:09.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.747+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/351053644 wait complete. 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.747+0000 7fe0b0dfd740 1 -- start start 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.747+0000 7fe0b0dfd740 1 Processor -- start 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991766610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559991760ac0 0x559991766b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x559991767090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599916624a0 con 0x5599917696a0 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599916bff60 con 0x5599917601a0 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999176c430 con 0x559991760ac0 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0b0dfd740 10 client.0 _ll_register_callbacks cb 0x5599916bdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.748+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559991760ac0 0x559991766b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559991760ac0 0x559991766b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59388/0 (socket says 172.21.15.22:59388) 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991766610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 -- 172.21.15.22:0/57456297 learned_addr learned my addr 172.21.15.22:0/57456297 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x559991767090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 -- 172.21.15.22:0/57456297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 msgr2=0x559991766610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991766610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 -- 172.21.15.22:0/57456297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 msgr2=0x559991767090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x559991767090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 -- 172.21.15.22:0/57456297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599913d83f0 con 0x559991760ac0 2024-03-09T00:04:09.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x559991767090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:04:09.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559991760ac0 0x559991766b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0cbfd0 tx=0x7fe09c0057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0bfd303c07736e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0877fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:09.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.749+0000 7fe0877fe640 1 -- 172.21.15.22:0/57456297 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0a35d0 con 0x559991760ac0 2024-03-09T00:04:09.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.750+0000 7fe0877fe640 1 -- 172.21.15.22:0/57456297 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe09c0974a0 con 0x559991760ac0 2024-03-09T00:04:09.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.751+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe098020070 con 0x559991760ac0 2024-03-09T00:04:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.751+0000 7fe0877fe640 1 -- 172.21.15.22:0/57456297 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe09c09a280 con 0x559991760ac0 2024-03-09T00:04:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.751+0000 7fe0b0dfd740 10 client.28003 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.751+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5599914a27d0 con 0x559991760ac0 2024-03-09T00:04:09.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.751+0000 7fe0877fe640 1 -- 172.21.15.22:0/57456297 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe098020070 con 0x559991760ac0 2024-03-09T00:04:09.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.753+0000 7fe0877fe640 1 -- 172.21.15.22:0/57456297 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe09c099d80 con 0x559991760ac0 2024-03-09T00:04:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.753+0000 7fe0b0dfd740 10 client.28003 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.753+0000 7fe0b0dfd740 10 client.28003 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:04:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.753+0000 7fe0b0dfd740 20 client.28003 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.753+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917797d0 0x55999177bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.753+0000 7fe0b0dfd740 4 client.28003 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.753+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599916d1400 con 0x5599917797d0 2024-03-09T00:04:09.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.758+0000 7fe0a7fff640 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917797d0 0x55999177bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.772+0000 7fe0a7fff640 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917797d0 0x55999177bbc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e808318c43bcb69c server_cookie=f11e3ef82cfae526 in_seq=0 out_seq=0 2024-03-09T00:04:09.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.772+0000 7fe0877fe640 10 client.28003 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:09.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.776+0000 7fe0877fe640 1 -- 172.21.15.22:0/57456297 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599916d1400 con 0x5599917797d0 2024-03-09T00:04:09.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.776+0000 7fe0877fe640 10 client.28003 handle_command_reply: tid=0 2024-03-09T00:04:09.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.776+0000 7fe0b0dfd740 1 client.28003 shutdown 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559991760ac0 msgr2=0x559991766b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559991760ac0 0x559991766b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0cbfd0 tx=0x7fe09c0057d0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 shutdown_connections 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917797d0 0x55999177bbc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x559991767090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559991760ac0 0x559991766b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/57456297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917601a0 0x559991766610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.780+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 >> 172.21.15.22:0/57456297 conn(0x55999175a8d0 msgr2=0x55999175c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.782+0000 7fe0877fe640 0 client.28003 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.784+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 shutdown_connections 2024-03-09T00:04:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.784+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/57456297 wait complete. 2024-03-09T00:04:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.784+0000 7fe0b0dfd740 20 client.28003 trim_cache size 0 max 16384 2024-03-09T00:04:09.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.807+0000 7fe0b0dfd740 1 Processor -- start 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0b0dfd740 1 -- start start 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 0x559991768d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991764ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917696a0 0x559991765420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe094012360 con 0x559991760ac0 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09400ff00 con 0x5599917696a0 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09400e050 con 0x5599917601a0 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991764ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991764ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43272/0 (socket says 172.21.15.22:43272) 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.809+0000 7fe0a77fe640 1 -- 172.21.15.22:0/2063274199 learned_addr learned my addr 172.21.15.22:0/2063274199 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.810+0000 7fe0a77fe640 1 -- 172.21.15.22:0/2063274199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 msgr2=0x559991768d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:09.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.810+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/2063274199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 0x559991768d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.810+0000 7fe0a77fe640 1 -- 172.21.15.22:0/2063274199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917696a0 msgr2=0x559991765420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:09.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.810+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/2063274199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917696a0 0x559991765420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.810+0000 7fe0a77fe640 1 -- 172.21.15.22:0/2063274199 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999176c370 con 0x559991760ac0 2024-03-09T00:04:09.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.811+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/2063274199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991764ee0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00d950 tx=0x7fe09c00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fced5a0a11b33c1d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.812+0000 7fe08dffb640 1 -- 172.21.15.22:0/2063274199 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c081070 con 0x559991760ac0 2024-03-09T00:04:09.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.812+0000 7fe08dffb640 1 -- 172.21.15.22:0/2063274199 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe09c0a35d0 con 0x559991760ac0 2024-03-09T00:04:09.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.812+0000 7fe08dffb640 1 -- 172.21.15.22:0/2063274199 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0974a0 con 0x559991760ac0 2024-03-09T00:04:09.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.814+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/2063274199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 msgr2=0x559991764ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.814+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/2063274199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991764ee0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00d950 tx=0x7fe09c00b510 comp rx=0 tx=0).stop 2024-03-09T00:04:09.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.815+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/2063274199 shutdown_connections 2024-03-09T00:04:09.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.815+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/2063274199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599917696a0 0x559991765420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.815+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/2063274199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559991760ac0 0x559991764ee0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.815+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/2063274199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 0x559991768d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.815+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/2063274199 >> 172.21.15.22:0/2063274199 conn(0x55999175a8d0 msgr2=0x559991780560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.819+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/2063274199 shutdown_connections 2024-03-09T00:04:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.821+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/2063274199 wait complete. 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.821+0000 7fe0b0dfd740 1 -- start start 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.821+0000 7fe0b0dfd740 1 Processor -- start 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 0x55999176f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559991760ac0 0x55999176fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0b0dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x5599917cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09400e5c0 con 0x5599917696a0 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09400f8a0 con 0x559991760ac0 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0b0dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09400ebe0 con 0x5599917601a0 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0b0dfd740 10 client.0 _ll_register_callbacks cb 0x7fe098009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0acb1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x5599917cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0acb1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x5599917cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43288/0 (socket says 172.21.15.22:43288) 2024-03-09T00:04:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0acb1b640 1 -- 172.21.15.22:0/860890687 learned_addr learned my addr 172.21.15.22:0/860890687 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0a7fff640 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 0x55999176f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.822+0000 7fe0a77fe640 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559991760ac0 0x55999176fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.825+0000 7fe0acb1b640 1 -- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 msgr2=0x55999176f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.825+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 0x55999176f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.825+0000 7fe0acb1b640 1 -- 172.21.15.22:0/860890687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559991760ac0 msgr2=0x55999176fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.825+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559991760ac0 0x55999176fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.825+0000 7fe0acb1b640 1 -- 172.21.15.22:0/860890687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559991762210 con 0x5599917696a0 2024-03-09T00:04:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.826+0000 7fe0acb1b640 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x5599917cfcf0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe098004b90 tx=0x7fe098024640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a5b4639935b720 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.826+0000 7fe0877fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.826+0000 7fe0877fe640 1 -- 172.21.15.22:0/860890687 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098052020 con 0x5599917696a0 2024-03-09T00:04:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.826+0000 7fe0877fe640 1 -- 172.21.15.22:0/860890687 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe098024930 con 0x5599917696a0 2024-03-09T00:04:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.826+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe09c0a35d0 con 0x5599917696a0 2024-03-09T00:04:09.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.834+0000 7fe0877fe640 1 -- 172.21.15.22:0/860890687 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe09805d050 con 0x5599917696a0 2024-03-09T00:04:09.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.838+0000 7fe0b0dfd740 10 client.18576 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:09.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.838+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5599917d1290 con 0x5599917696a0 2024-03-09T00:04:09.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.842+0000 7fe0877fe640 1 -- 172.21.15.22:0/860890687 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe09c0a35d0 con 0x5599917696a0 2024-03-09T00:04:09.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.842+0000 7fe0877fe640 1 -- 172.21.15.22:0/860890687 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe09804ea60 con 0x5599917696a0 2024-03-09T00:04:09.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.843+0000 7fe0b0dfd740 10 client.18576 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:09.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.843+0000 7fe0b0dfd740 10 client.18576 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:04:09.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.843+0000 7fe0b0dfd740 20 client.18576 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:09.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.843+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917d1710 0x5599917d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:09.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.843+0000 7fe0b0dfd740 4 client.18576 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:09.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.843+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599916cb750 con 0x5599917d1710 2024-03-09T00:04:09.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.844+0000 7fe0a7fff640 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917d1710 0x5599917d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:09.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.849+0000 7fe0a7fff640 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917d1710 0x5599917d3b30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46d3d87ce10598bc server_cookie=b55e6bef3e8988f6 in_seq=0 out_seq=0 2024-03-09T00:04:09.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.850+0000 7fe0877fe640 10 client.18576 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:09.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.854+0000 7fe0877fe640 1 -- 172.21.15.22:0/860890687 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599916cb750 con 0x5599917d1710 2024-03-09T00:04:09.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.854+0000 7fe0877fe640 10 client.18576 handle_command_reply: tid=0 2024-03-09T00:04:09.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.856+0000 7fe0b0dfd740 1 client.18576 shutdown 2024-03-09T00:04:09.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.856+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 msgr2=0x5599917cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.856+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x5599917cfcf0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe098004b90 tx=0x7fe098024640 comp rx=0 tx=0).stop 2024-03-09T00:04:09.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.857+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 shutdown_connections 2024-03-09T00:04:09.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.857+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5599917d1710 0x5599917d3b30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.857+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5599917696a0 0x5599917cfcf0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.857+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559991760ac0 0x55999176fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.857+0000 7fe0b0dfd740 1 --2- 172.21.15.22:0/860890687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5599917601a0 0x55999176f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.857+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 >> 172.21.15.22:0/860890687 conn(0x55999175a8d0 msgr2=0x559991764e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.857+0000 7fe0877fe640 0 client.18576 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:09.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.862+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 shutdown_connections 2024-03-09T00:04:09.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.862+0000 7fe0b0dfd740 1 -- 172.21.15.22:0/860890687 wait complete. 2024-03-09T00:04:09.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.863+0000 7fe0b0dfd740 20 client.18576 trim_cache size 0 max 16384 2024-03-09T00:04:09.859 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.865+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe0900a2a30 msgr2=0x7fe0900a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.865+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe0900a2a30 0x7fe0900a4ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c830 tx=0x7fe09800cc30 comp rx=0 tx=0).stop 2024-03-09T00:04:09.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.865+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 msgr2=0x7fe0a806dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:09.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.865+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 0x7fe0a806dbf0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00d980 tx=0x7fe09c00de40 comp rx=0 tx=0).stop 2024-03-09T00:04:09.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.868+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 shutdown_connections 2024-03-09T00:04:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.868+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe0900a2a30 0x7fe0900a4ef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.868+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0a81051e0 0x7fe0a806e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.868+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a8100e70 0x7fe0a806dbf0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.868+0000 7fe0ae57e640 1 --2- 172.21.15.22:0/1900148997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0a81004d0 0x7fe0a8072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.868+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 >> 172.21.15.22:0/1900148997 conn(0x7fe0a80771d0 msgr2=0x7fe0a80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.869+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 shutdown_connections 2024-03-09T00:04:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:09.871+0000 7fe0ae57e640 1 -- 172.21.15.22:0/1900148997 wait complete. 2024-03-09T00:04:09.883 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:09.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:04:10.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.065+0000 7ffb258a3640 1 -- 172.21.15.22:0/640369626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb180a1610 msgr2=0x7ffb180a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.065+0000 7ffb258a3640 1 --2- 172.21.15.22:0/640369626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb180a1610 0x7ffb180a91e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ffb20066fc0 tx=0x7ffb20097030 comp rx=0 tx=0).stop 2024-03-09T00:04:10.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.067+0000 7ffb258a3640 1 -- 172.21.15.22:0/640369626 shutdown_connections 2024-03-09T00:04:10.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.067+0000 7ffb258a3640 1 --2- 172.21.15.22:0/640369626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb180a1610 0x7ffb180a91e0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.067+0000 7ffb258a3640 1 --2- 172.21.15.22:0/640369626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb1809d500 0x7ffb180a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.067+0000 7ffb258a3640 1 --2- 172.21.15.22:0/640369626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 0x7ffb1809cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.067+0000 7ffb258a3640 1 -- 172.21.15.22:0/640369626 >> 172.21.15.22:0/640369626 conn(0x7ffb180092a0 msgr2=0x7ffb180096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.069+0000 7ffb258a3640 1 -- 172.21.15.22:0/640369626 shutdown_connections 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.072+0000 7ffb258a3640 1 -- 172.21.15.22:0/640369626 wait complete. 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.072+0000 7ffb258a3640 1 Processor -- start 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.072+0000 7ffb258a3640 1 -- start start 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.073+0000 7ffb258a3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 0x7ffb18014f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.073+0000 7ffb258a3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb180a1610 0x7ffb180154b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.073+0000 7ffb258a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb1800e090 0x7ffb1800e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.073+0000 7ffb258a3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180ac6d0 con 0x7ffb1800e090 2024-03-09T00:04:10.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.073+0000 7ffb258a3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180ac550 con 0x7ffb1809cb50 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.073+0000 7ffb258a3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180ac850 con 0x7ffb180a1610 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.073+0000 7ffb248a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 0x7ffb18014f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 0x7ffb18014f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57486/0 (socket says 172.21.15.22:57486) 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 -- 172.21.15.22:0/2193847192 learned_addr learned my addr 172.21.15.22:0/2193847192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 -- 172.21.15.22:0/2193847192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb180a1610 msgr2=0x7ffb180154b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb180a1610 0x7ffb180154b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 -- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb1800e090 msgr2=0x7ffb1800e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb1800e090 0x7ffb1800e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 -- 172.21.15.22:0/2193847192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1800ee00 con 0x7ffb1809cb50 2024-03-09T00:04:10.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb248a1640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 0x7ffb18014f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400d950 tx=0x7ffb1400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.074+0000 7ffb1dffb640 1 -- 172.21.15.22:0/2193847192 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14017070 con 0x7ffb1809cb50 2024-03-09T00:04:10.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.075+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb18012af0 con 0x7ffb1809cb50 2024-03-09T00:04:10.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.075+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb18013090 con 0x7ffb1809cb50 2024-03-09T00:04:10.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.075+0000 7ffb1dffb640 1 -- 172.21.15.22:0/2193847192 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb14009c30 con 0x7ffb1809cb50 2024-03-09T00:04:10.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.075+0000 7ffb1dffb640 1 -- 172.21.15.22:0/2193847192 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1400f690 con 0x7ffb1809cb50 2024-03-09T00:04:10.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.078+0000 7ffb1dffb640 1 -- 172.21.15.22:0/2193847192 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ffb140109d0 con 0x7ffb1809cb50 2024-03-09T00:04:10.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.079+0000 7ffb1dffb640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffb100a2ab0 0x7ffb100a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.079+0000 7ffb1dffb640 1 -- 172.21.15.22:0/2193847192 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ffb140cec30 con 0x7ffb1809cb50 2024-03-09T00:04:10.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.080+0000 7ffb1ffff640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffb100a2ab0 0x7ffb100a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.083+0000 7ffb297bd740 1 Processor -- start 2024-03-09T00:04:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.083+0000 7ffb297bd740 1 -- start start 2024-03-09T00:04:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8568700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568c40 0x5628b85712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b8573c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb297bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b852f360 con 0x5628b8568320 2024-03-09T00:04:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb297bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b84cbec0 con 0x5628b8568c40 2024-03-09T00:04:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb297bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b8474d30 con 0x5628b8571820 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb250a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b8573c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb250a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b8573c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59434/0 (socket says 172.21.15.22:59434) 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.084+0000 7ffb250a2640 1 -- 172.21.15.22:0/3765578754 learned_addr learned my addr 172.21.15.22:0/3765578754 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.085+0000 7ffb248a1640 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8568700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.085+0000 7ffb250a2640 1 -- 172.21.15.22:0/3765578754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568c40 msgr2=0x5628b85712e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.085+0000 7ffb250a2640 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568c40 0x5628b85712e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.085+0000 7ffb250a2640 1 -- 172.21.15.22:0/3765578754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 msgr2=0x5628b8568700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.085+0000 7ffb250a2640 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8568700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.085+0000 7ffb250a2640 1 -- 172.21.15.22:0/3765578754 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b8473bc0 con 0x5628b8571820 2024-03-09T00:04:10.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.087+0000 7ffb1ffff640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffb100a2ab0 0x7ffb100a4f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800f7b0 tx=0x7ffb0c006bd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.091+0000 7ffb250a2640 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b8573c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb20066bb0 tx=0x7ffb20067f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef42537419e44316 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.093+0000 7ffb0affd640 1 -- 172.21.15.22:0/3765578754 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200764e0 con 0x5628b8571820 2024-03-09T00:04:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.093+0000 7ffb0affd640 1 -- 172.21.15.22:0/3765578754 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2009b070 con 0x5628b8571820 2024-03-09T00:04:10.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.093+0000 7ffb0affd640 1 -- 172.21.15.22:0/3765578754 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200693b0 con 0x5628b8571820 2024-03-09T00:04:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.096+0000 7ffb297bd740 1 -- 172.21.15.22:0/3765578754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 msgr2=0x5628b8573c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.096+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b8573c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb20066bb0 tx=0x7ffb20067f50 comp rx=0 tx=0).stop 2024-03-09T00:04:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.101+0000 7ffb297bd740 1 -- 172.21.15.22:0/3765578754 shutdown_connections 2024-03-09T00:04:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.101+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b8573c10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.101+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568c40 0x5628b85712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.101+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3765578754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8568700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.101+0000 7ffb297bd740 1 -- 172.21.15.22:0/3765578754 >> 172.21.15.22:0/3765578754 conn(0x5628b8562a50 msgr2=0x5628b8562e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.107+0000 7ffb297bd740 1 -- 172.21.15.22:0/3765578754 shutdown_connections 2024-03-09T00:04:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.109+0000 7ffb297bd740 1 -- 172.21.15.22:0/3765578754 wait complete. 2024-03-09T00:04:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.109+0000 7ffb297bd740 1 -- start start 2024-03-09T00:04:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.109+0000 7ffb297bd740 1 Processor -- start 2024-03-09T00:04:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.112+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568320 0x5628b856e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.112+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568c40 0x5628b856ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.112+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b856f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.112+0000 7ffb297bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b846a4a0 con 0x5628b8568c40 2024-03-09T00:04:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.112+0000 7ffb297bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b84c7f60 con 0x5628b8568320 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.112+0000 7ffb297bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b85745b0 con 0x5628b8571820 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.112+0000 7ffb297bd740 10 client.0 _ll_register_callbacks cb 0x5628b84c5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.113+0000 7ffb248a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568320 0x5628b856e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.113+0000 7ffb250a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b856f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.113+0000 7ffb248a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568320 0x5628b856e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57504/0 (socket says 172.21.15.22:57504) 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.113+0000 7ffb248a1640 1 -- 172.21.15.22:0/1287798481 learned_addr learned my addr 172.21.15.22:0/1287798481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.113+0000 7ffb1ffff640 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568c40 0x5628b856ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.114+0000 7ffb1ffff640 1 -- 172.21.15.22:0/1287798481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 msgr2=0x5628b856f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.114+0000 7ffb1ffff640 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b856f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.114+0000 7ffb1ffff640 1 -- 172.21.15.22:0/1287798481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568320 msgr2=0x5628b856e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.114+0000 7ffb1ffff640 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568320 0x5628b856e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.114+0000 7ffb1ffff640 1 -- 172.21.15.22:0/1287798481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b81e03f0 con 0x5628b8568c40 2024-03-09T00:04:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.117+0000 7ffb1ffff640 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568c40 0x5628b856ed40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ffb18005680 tx=0x7ffb0c0046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886d4b1c5d096193 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.117+0000 7ffafffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.117+0000 7ffafffff640 1 -- 172.21.15.22:0/1287798481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c01d5f0 con 0x5628b8568c40 2024-03-09T00:04:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.117+0000 7ffafffff640 1 -- 172.21.15.22:0/1287798481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ffb0c01d790 con 0x5628b8568c40 2024-03-09T00:04:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.118+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb2009b070 con 0x5628b8568c40 2024-03-09T00:04:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.118+0000 7ffafffff640 1 -- 172.21.15.22:0/1287798481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb0c01d990 con 0x5628b8568c40 2024-03-09T00:04:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.118+0000 7ffb297bd740 10 client.18585 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.118+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5628b82aa7d0 con 0x5628b8568c40 2024-03-09T00:04:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffafffff640 1 -- 172.21.15.22:0/1287798481 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2009b070 con 0x5628b8568c40 2024-03-09T00:04:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffafffff640 1 -- 172.21.15.22:0/1287798481 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb0c012070 con 0x5628b8568c40 2024-03-09T00:04:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffb297bd740 10 client.18585 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffb297bd740 10 client.18585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:04:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffb297bd740 20 client.18585 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b857f120 0x5628b8581560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffb297bd740 4 client.18585 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.119+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628b84d9400 con 0x5628b857f120 2024-03-09T00:04:10.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.121+0000 7ffb248a1640 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b857f120 0x5628b8581560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.134+0000 7ffb248a1640 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b857f120 0x5628b8581560 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7cca7490ac2dad2a server_cookie=7d58b9b822b0e1a7 in_seq=0 out_seq=0 2024-03-09T00:04:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.134+0000 7ffafffff640 10 client.18585 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:10.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.141+0000 7ffafffff640 1 -- 172.21.15.22:0/1287798481 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628b84d9400 con 0x5628b857f120 2024-03-09T00:04:10.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.141+0000 7ffafffff640 10 client.18585 handle_command_reply: tid=0 2024-03-09T00:04:10.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.141+0000 7ffb297bd740 1 client.18585 shutdown 2024-03-09T00:04:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568c40 msgr2=0x5628b856ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568c40 0x5628b856ed40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ffb18005680 tx=0x7ffb0c0046e0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 shutdown_connections 2024-03-09T00:04:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b857f120 0x5628b8581560 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b8571820 0x5628b856f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568c40 0x5628b856ed40 secure :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ffb18005680 tx=0x7ffb0c0046e0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1287798481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8568320 0x5628b856e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.145+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 >> 172.21.15.22:0/1287798481 conn(0x5628b8562a50 msgr2=0x5628b85647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.146+0000 7ffafffff640 0 client.18585 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:10.142 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:10 smithi022 ceph-mon[31894]: pgmap v304: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 90 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 349 op/s 2024-03-09T00:04:10.142 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:10 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/567711401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.148+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 shutdown_connections 2024-03-09T00:04:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.150+0000 7ffb297bd740 1 -- 172.21.15.22:0/1287798481 wait complete. 2024-03-09T00:04:10.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.152+0000 7ffb297bd740 20 client.18585 trim_cache size 0 max 16384 2024-03-09T00:04:10.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 Processor -- start 2024-03-09T00:04:10.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 -- start start 2024-03-09T00:04:10.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8570f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b858e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8578840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040124a0 con 0x5628b8568320 2024-03-09T00:04:10.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04010040 con 0x5628b8571820 2024-03-09T00:04:10.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.179+0000 7ffb297bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0400e100 con 0x5628b856f0a0 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.180+0000 7ffb1ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b858e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.180+0000 7ffb250a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8578840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.180+0000 7ffb250a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8578840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59456/0 (socket says 172.21.15.22:59456) 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.180+0000 7ffb250a2640 1 -- 172.21.15.22:0/3292501875 learned_addr learned my addr 172.21.15.22:0/3292501875 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.181+0000 7ffb250a2640 1 -- 172.21.15.22:0/3292501875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 msgr2=0x5628b858e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.181+0000 7ffb250a2640 1 --2- 172.21.15.22:0/3292501875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b858e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.181+0000 7ffb250a2640 1 -- 172.21.15.22:0/3292501875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 msgr2=0x5628b8570f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.181+0000 7ffb250a2640 1 --2- 172.21.15.22:0/3292501875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8570f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.181+0000 7ffb250a2640 1 -- 172.21.15.22:0/3292501875 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b85676a0 con 0x5628b856f0a0 2024-03-09T00:04:10.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.181+0000 7ffb250a2640 1 --2- 172.21.15.22:0/3292501875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8578840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb20050830 tx=0x7ffb20066f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8ac7a1ba4761cc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.184+0000 7ffb0a7fc640 1 -- 172.21.15.22:0/3292501875 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20067770 con 0x5628b856f0a0 2024-03-09T00:04:10.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.184+0000 7ffb0a7fc640 1 -- 172.21.15.22:0/3292501875 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2009b040 con 0x5628b856f0a0 2024-03-09T00:04:10.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.185+0000 7ffb297bd740 1 -- 172.21.15.22:0/3292501875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 msgr2=0x5628b8578840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.185+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3292501875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8578840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb20050830 tx=0x7ffb20066f80 comp rx=0 tx=0).stop 2024-03-09T00:04:10.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.191+0000 7ffb297bd740 1 -- 172.21.15.22:0/3292501875 shutdown_connections 2024-03-09T00:04:10.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.191+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3292501875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8578840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.191+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3292501875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b858e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.191+0000 7ffb297bd740 1 --2- 172.21.15.22:0/3292501875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8570f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.191+0000 7ffb297bd740 1 -- 172.21.15.22:0/3292501875 >> 172.21.15.22:0/3292501875 conn(0x5628b8562a50 msgr2=0x5628b8562e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.198+0000 7ffb297bd740 1 -- 172.21.15.22:0/3292501875 shutdown_connections 2024-03-09T00:04:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.201+0000 7ffb297bd740 1 -- 172.21.15.22:0/3292501875 wait complete. 2024-03-09T00:04:10.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.202+0000 7ffb297bd740 1 -- start start 2024-03-09T00:04:10.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.202+0000 7ffb297bd740 1 Processor -- start 2024-03-09T00:04:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8578ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8576830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb297bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b8576d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb297bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0400e700 con 0x5628b8568320 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb297bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0400f9e0 con 0x5628b8571820 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb297bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0400ed20 con 0x5628b856f0a0 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb297bd740 10 client.0 _ll_register_callbacks cb 0x7ffb20069790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.208+0000 7ffb250a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b8576d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.209+0000 7ffb250a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b8576d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57522/0 (socket says 172.21.15.22:57522) 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.209+0000 7ffb250a2640 1 -- 172.21.15.22:0/1092320800 learned_addr learned my addr 172.21.15.22:0/1092320800 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.209+0000 7ffb1ffff640 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8576830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.210+0000 7ffb250a2640 1 -- 172.21.15.22:0/1092320800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 msgr2=0x5628b8576830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.210+0000 7ffb250a2640 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8576830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.210+0000 7ffb250a2640 1 -- 172.21.15.22:0/1092320800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 msgr2=0x5628b8578ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.210+0000 7ffb250a2640 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8578ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.210+0000 7ffb250a2640 1 -- 172.21.15.22:0/1092320800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b8577410 con 0x5628b8571820 2024-03-09T00:04:10.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.210+0000 7ffb250a2640 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b8576d70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb200647e0 tx=0x7ffb20067980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e437f20599088252 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.212+0000 7ffafffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:04:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.213+0000 7ffafffff640 1 -- 172.21.15.22:0/1092320800 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200cd030 con 0x5628b8571820 2024-03-09T00:04:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.213+0000 7ffafffff640 1 -- 172.21.15.22:0/1092320800 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ffb20067af0 con 0x5628b8571820 2024-03-09T00:04:10.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.216+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb2009b040 con 0x5628b8571820 2024-03-09T00:04:10.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.218+0000 7ffafffff640 1 -- 172.21.15.22:0/1092320800 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb2004f8a0 con 0x5628b8571820 2024-03-09T00:04:10.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.221+0000 7ffb297bd740 10 client.27926 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:10.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.221+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5628b85d96f0 con 0x5628b8571820 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffafffff640 1 -- 172.21.15.22:0/1092320800 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2009b040 con 0x5628b8571820 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffafffff640 1 -- 172.21.15.22:0/1092320800 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb200a6070 con 0x5628b8571820 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffb297bd740 10 client.27926 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffb297bd740 10 client.27926 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffb297bd740 20 client.27926 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b85d9920 0x5628b85dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffb297bd740 4 client.27926 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.222+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5628b84d3750 con 0x5628b85d9920 2024-03-09T00:04:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.225+0000 7ffb248a1640 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b85d9920 0x5628b85dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.231+0000 7ffb248a1640 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b85d9920 0x5628b85dbd10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51ab546a47ba40a8 server_cookie=a0627ce8426584c4 in_seq=0 out_seq=0 2024-03-09T00:04:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.231+0000 7ffafffff640 10 client.27926 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.232+0000 7ffafffff640 1 -- 172.21.15.22:0/1092320800 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5628b84d3750 con 0x5628b85d9920 2024-03-09T00:04:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.232+0000 7ffafffff640 10 client.27926 handle_command_reply: tid=0 2024-03-09T00:04:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.233+0000 7ffb297bd740 1 client.27926 shutdown 2024-03-09T00:04:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.237+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 msgr2=0x5628b8576d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.237+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b8576d70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb200647e0 tx=0x7ffb20067980 comp rx=0 tx=0).stop 2024-03-09T00:04:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.238+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 shutdown_connections 2024-03-09T00:04:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.238+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5628b85d9920 0x5628b85dbd10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.238+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628b8571820 0x5628b8576d70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.238+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628b856f0a0 0x5628b8576830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.238+0000 7ffb297bd740 1 --2- 172.21.15.22:0/1092320800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5628b8568320 0x5628b8578ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.238+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 >> 172.21.15.22:0/1092320800 conn(0x5628b8562a50 msgr2=0x5628b856ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.238+0000 7ffafffff640 0 client.27926 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.240+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 shutdown_connections 2024-03-09T00:04:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.240+0000 7ffb297bd740 1 -- 172.21.15.22:0/1092320800 wait complete. 2024-03-09T00:04:10.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.240+0000 7ffb297bd740 20 client.27926 trim_cache size 0 max 16384 2024-03-09T00:04:10.236 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:10.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffb100a2ab0 msgr2=0x7ffb100a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffb100a2ab0 0x7ffb100a4f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800f7b0 tx=0x7ffb0c006bd0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 msgr2=0x7ffb18014f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 0x7ffb18014f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400d950 tx=0x7ffb1400de10 comp rx=0 tx=0).stop 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 shutdown_connections 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ffb100a2ab0 0x7ffb100a4f70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb1800e090 0x7ffb1800e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb180a1610 0x7ffb180154b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 --2- 172.21.15.22:0/2193847192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb1809cb50 0x7ffb18014f70 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400d950 tx=0x7ffb1400de10 comp rx=0 tx=0).stop 2024-03-09T00:04:10.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.242+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 >> 172.21.15.22:0/2193847192 conn(0x7ffb180092a0 msgr2=0x7ffb1809a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.243+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 shutdown_connections 2024-03-09T00:04:10.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.243+0000 7ffb258a3640 1 -- 172.21.15.22:0/2193847192 wait complete. 2024-03-09T00:04:10.258 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:10.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:04:10.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.440+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2598534453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 msgr2=0x7fd9ec110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.440+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2598534453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 0x7fd9ec110350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009cd0 tx=0x7fd9e402c120 comp rx=0 tx=0).stop 2024-03-09T00:04:10.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.440+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2598534453 shutdown_connections 2024-03-09T00:04:10.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.440+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2598534453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 0x7fd9ec110350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.440+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2598534453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ec1034d0 0x7fd9ec103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.440+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2598534453 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9ec109590 0x7fd9ec109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.440+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2598534453 >> 172.21.15.22:0/2598534453 conn(0x7fd9ec0fef60 msgr2=0x7fd9ec101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.446+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2598534453 shutdown_connections 2024-03-09T00:04:10.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.447+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2598534453 wait complete. 2024-03-09T00:04:10.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.448+0000 7fd9f3cc0640 1 Processor -- start 2024-03-09T00:04:10.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.449+0000 7fd9f3cc0640 1 -- start start 2024-03-09T00:04:10.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.449+0000 7fd9f3cc0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9ec1034d0 0x7fd9ec110210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.449+0000 7fd9f3cc0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 0x7fd9ec10d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.449+0000 7fd9f3cc0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ec109590 0x7fd9ec10d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.449+0000 7fd9f3cc0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec113970 con 0x7fd9ec1034d0 2024-03-09T00:04:10.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.449+0000 7fd9f3cc0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1137f0 con 0x7fd9ec109590 2024-03-09T00:04:10.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.449+0000 7fd9f3cc0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec113af0 con 0x7fd9ec103e90 2024-03-09T00:04:10.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.450+0000 7fd9f225d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ec109590 0x7fd9ec10d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.450+0000 7fd9f225d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ec109590 0x7fd9ec10d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:57536/0 (socket says 172.21.15.22:57536) 2024-03-09T00:04:10.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.450+0000 7fd9f225d640 1 -- 172.21.15.22:0/2753720179 learned_addr learned my addr 172.21.15.22:0/2753720179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.450+0000 7fd9f125b640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 0x7fd9ec10d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.451+0000 7fd9f125b640 1 -- 172.21.15.22:0/2753720179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ec109590 msgr2=0x7fd9ec10d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.451+0000 7fd9f125b640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ec109590 0x7fd9ec10d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.451+0000 7fd9f125b640 1 -- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9ec1034d0 msgr2=0x7fd9ec110210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:04:10.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.451+0000 7fd9f125b640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9ec1034d0 0x7fd9ec110210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.451+0000 7fd9f125b640 1 -- 172.21.15.22:0/2753720179 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec10e120 con 0x7fd9ec103e90 2024-03-09T00:04:10.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.452+0000 7fd9f125b640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 0x7fd9ec10d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009fd0 tx=0x7fd9dc00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.453+0000 7fd9e2ffd640 1 -- 172.21.15.22:0/2753720179 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc015020 con 0x7fd9ec103e90 2024-03-09T00:04:10.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.453+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec1b2fd0 con 0x7fd9ec103e90 2024-03-09T00:04:10.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.453+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec1b3490 con 0x7fd9ec103e90 2024-03-09T00:04:10.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.454+0000 7fd9e2ffd640 1 -- 172.21.15.22:0/2753720179 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9dc011d10 con 0x7fd9ec103e90 2024-03-09T00:04:10.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.455+0000 7fd9e2ffd640 1 -- 172.21.15.22:0/2753720179 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00f2e0 con 0x7fd9ec103e90 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.458+0000 7fd9f697a740 1 Processor -- start 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.459+0000 7fd9f697a740 1 -- start start 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.460+0000 7fd9e2ffd640 1 -- 172.21.15.22:0/2753720179 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd9dc018020 con 0x7fd9ec103e90 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.460+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba72580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.460+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba72ac0 0x56396ba7b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.460+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 0x56396ba7da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.460+0000 7fd9f697a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396ba39360 con 0x56396ba72ac0 2024-03-09T00:04:10.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.460+0000 7fd9f697a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396b9d5ec0 con 0x56396ba7b6a0 2024-03-09T00:04:10.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.460+0000 7fd9f697a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396b97ed30 con 0x56396ba721a0 2024-03-09T00:04:10.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba72580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba72580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50252/0 (socket says 172.21.15.22:50252) 2024-03-09T00:04:10.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 -- 172.21.15.22:0/2855997403 learned_addr learned my addr 172.21.15.22:0/2855997403 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 -- 172.21.15.22:0/2855997403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 msgr2=0x56396ba7da90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:04:10.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 --2- 172.21.15.22:0/2855997403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 0x56396ba7da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 -- 172.21.15.22:0/2855997403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba72ac0 msgr2=0x56396ba7b160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:04:10.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 --2- 172.21.15.22:0/2855997403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba72ac0 0x56396ba7b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.461+0000 7fd9f225d640 1 -- 172.21.15.22:0/2855997403 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396b97dbc0 con 0x56396ba721a0 2024-03-09T00:04:10.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.462+0000 7fd9f225d640 1 --2- 172.21.15.22:0/2855997403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba72580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4032340 tx=0x7fd9e4032370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b6787ffc68768e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.463+0000 7fd9d7fff640 1 -- 172.21.15.22:0/2855997403 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4042030 con 0x56396ba721a0 2024-03-09T00:04:10.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.463+0000 7fd9d7fff640 1 -- 172.21.15.22:0/2855997403 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9e4003e50 con 0x56396ba721a0 2024-03-09T00:04:10.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.463+0000 7fd9d7fff640 1 -- 172.21.15.22:0/2855997403 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4039070 con 0x56396ba721a0 2024-03-09T00:04:10.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.463+0000 7fd9e2ffd640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9d80a2950 0x7fd9d80a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.463+0000 7fd9e2ffd640 1 -- 172.21.15.22:0/2753720179 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fd9dc0cea90 con 0x7fd9ec103e90 2024-03-09T00:04:10.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.466+0000 7fd9f697a740 1 -- 172.21.15.22:0/2855997403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 msgr2=0x56396ba72580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.466+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2855997403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba72580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4032340 tx=0x7fd9e4032370 comp rx=0 tx=0).stop 2024-03-09T00:04:10.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.469+0000 7fd9f697a740 1 -- 172.21.15.22:0/2855997403 shutdown_connections 2024-03-09T00:04:10.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.469+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2855997403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 0x56396ba7da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.469+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2855997403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba72ac0 0x56396ba7b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.469+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2855997403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba72580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.469+0000 7fd9f697a740 1 -- 172.21.15.22:0/2855997403 >> 172.21.15.22:0/2855997403 conn(0x56396ba6c8d0 msgr2=0x56396ba6ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.470+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9d80a2950 0x7fd9d80a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.471+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9d80a2950 0x7fd9d80a4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x56396ba71c70 tx=0x7fd9e8006b70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.474+0000 7fd9f697a740 1 -- 172.21.15.22:0/2855997403 shutdown_connections 2024-03-09T00:04:10.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.475+0000 7fd9f697a740 1 -- 172.21.15.22:0/2855997403 wait complete. 2024-03-09T00:04:10.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 -- start start 2024-03-09T00:04:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 Processor -- start 2024-03-09T00:04:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba72ac0 0x56396ba78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 0x56396ba79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396b9744a0 con 0x56396ba7b6a0 2024-03-09T00:04:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396b9d1f60 con 0x56396ba72ac0 2024-03-09T00:04:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396ba7e430 con 0x56396ba721a0 2024-03-09T00:04:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.476+0000 7fd9f697a740 10 client.0 _ll_register_callbacks cb 0x56396b9cfd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:10.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.477+0000 7fd9f125b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba72ac0 0x56396ba78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.477+0000 7fd9f125b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba72ac0 0x56396ba78bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51736/0 (socket says 172.21.15.22:51736) 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.477+0000 7fd9f125b640 1 -- 172.21.15.22:0/1224784255 learned_addr learned my addr 172.21.15.22:0/1224784255 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.477+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.478+0000 7fd9f125b640 1 -- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 msgr2=0x56396ba786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.478+0000 7fd9f125b640 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba786b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.478+0000 7fd9f125b640 1 -- 172.21.15.22:0/1224784255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 msgr2=0x56396ba79130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.478+0000 7fd9f125b640 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 0x56396ba79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.478+0000 7fd9f125b640 1 -- 172.21.15.22:0/1224784255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56396b6ea3f0 con 0x56396ba72ac0 2024-03-09T00:04:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.478+0000 7fd9f125b640 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba72ac0 0x56396ba78bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00bd70 tx=0x7fd9dc09a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76703bf12b1faa0b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.478+0000 7fd9d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:04:10.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.480+0000 7fd9d57fa640 1 -- 172.21.15.22:0/1224784255 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0174c0 con 0x56396ba72ac0 2024-03-09T00:04:10.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.480+0000 7fd9d57fa640 1 -- 172.21.15.22:0/1224784255 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fd9dc0cbfb0 con 0x56396ba72ac0 2024-03-09T00:04:10.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.480+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9e4003e50 con 0x56396ba72ac0 2024-03-09T00:04:10.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.483+0000 7fd9d57fa640 1 -- 172.21.15.22:0/1224784255 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9dc0cc1b0 con 0x56396ba72ac0 2024-03-09T00:04:10.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.487+0000 7fd9f697a740 10 client.27932 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:10.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.487+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56396b7b47d0 con 0x56396ba72ac0 2024-03-09T00:04:10.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.488+0000 7fd9d57fa640 1 -- 172.21.15.22:0/1224784255 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9e4003e50 con 0x56396ba72ac0 2024-03-09T00:04:10.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.489+0000 7fd9d57fa640 1 -- 172.21.15.22:0/1224784255 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd9dc09a330 con 0x56396ba72ac0 2024-03-09T00:04:10.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.489+0000 7fd9f697a740 10 client.27932 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:10.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.489+0000 7fd9f697a740 10 client.27932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:10.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.489+0000 7fd9f697a740 20 client.27932 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:10.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.489+0000 7fd9f697a740 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396ba8bdf0 0x56396ba8e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.489+0000 7fd9f697a740 4 client.27932 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:10.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.489+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56396b9e3400 con 0x56396ba8bdf0 2024-03-09T00:04:10.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.492+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396ba8bdf0 0x56396ba8e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.492+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396ba8bdf0 0x56396ba8e250 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42b30a2ae630310 server_cookie=e945be691e9c86c4 in_seq=0 out_seq=0 2024-03-09T00:04:10.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.493+0000 7fd9d57fa640 10 client.27932 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:10.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.495+0000 7fd9d57fa640 1 -- 172.21.15.22:0/1224784255 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56396b9e3400 con 0x56396ba8bdf0 2024-03-09T00:04:10.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.495+0000 7fd9d57fa640 10 client.27932 handle_command_reply: tid=0 2024-03-09T00:04:10.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.495+0000 7fd9f697a740 1 client.27932 shutdown 2024-03-09T00:04:10.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.501+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba72ac0 msgr2=0x56396ba78bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.501+0000 7fd9f697a740 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba72ac0 0x56396ba78bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00bd70 tx=0x7fd9dc09a7a0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 shutdown_connections 2024-03-09T00:04:10.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9f697a740 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396ba8bdf0 0x56396ba8e250 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9f697a740 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 0x56396ba79130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9f697a740 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba72ac0 0x56396ba78bf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9f697a740 1 --2- 172.21.15.22:0/1224784255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba721a0 0x56396ba786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 >> 172.21.15.22:0/1224784255 conn(0x56396ba6c8d0 msgr2=0x56396ba6e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9d57fa640 0 client.27932 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.504+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 shutdown_connections 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.505+0000 7fd9f697a740 1 -- 172.21.15.22:0/1224784255 wait complete. 2024-03-09T00:04:10.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.505+0000 7fd9f697a740 20 client.27932 trim_cache size 0 max 16384 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 Processor -- start 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 -- start start 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba721a0 0x56396ba7ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396ba92bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 0x56396ba82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d0012670 con 0x56396ba721a0 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d0010210 con 0x56396ba7b6a0 2024-03-09T00:04:10.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.535+0000 7fd9f697a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d000e2d0 con 0x56396ba72ac0 2024-03-09T00:04:10.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.536+0000 7fd9f225d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 0x56396ba82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.537+0000 7fd9f125b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396ba92bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.537+0000 7fd9f125b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396ba92bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50280/0 (socket says 172.21.15.22:50280) 2024-03-09T00:04:10.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.537+0000 7fd9f125b640 1 -- 172.21.15.22:0/2378269630 learned_addr learned my addr 172.21.15.22:0/2378269630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.539+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba721a0 0x56396ba7ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.539+0000 7fd9f125b640 1 -- 172.21.15.22:0/2378269630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 msgr2=0x56396ba82700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.539+0000 7fd9f125b640 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 0x56396ba82700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.539+0000 7fd9f125b640 1 -- 172.21.15.22:0/2378269630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba721a0 msgr2=0x56396ba7ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.539+0000 7fd9f125b640 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba721a0 0x56396ba7ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.539+0000 7fd9f125b640 1 -- 172.21.15.22:0/2378269630 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396ba7ddc0 con 0x56396ba72ac0 2024-03-09T00:04:10.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.539+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba721a0 0x56396ba7ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:10.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.543+0000 7fd9f125b640 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396ba92bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc010a50 tx=0x7fd9dc09f020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc63836a01069cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.544+0000 7fd9d77fe640 1 -- 172.21.15.22:0/2378269630 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc096ff0 con 0x56396ba72ac0 2024-03-09T00:04:10.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.544+0000 7fd9d77fe640 1 -- 172.21.15.22:0/2378269630 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9dc0a34f0 con 0x56396ba72ac0 2024-03-09T00:04:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.545+0000 7fd9f697a740 1 -- 172.21.15.22:0/2378269630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 msgr2=0x56396ba92bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.545+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396ba92bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc010a50 tx=0x7fd9dc09f020 comp rx=0 tx=0).stop 2024-03-09T00:04:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.545+0000 7fd9f697a740 1 -- 172.21.15.22:0/2378269630 shutdown_connections 2024-03-09T00:04:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.545+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba7b6a0 0x56396ba82700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.545+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396ba92bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.545+0000 7fd9f697a740 1 --2- 172.21.15.22:0/2378269630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba721a0 0x56396ba7ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.545+0000 7fd9f697a740 1 -- 172.21.15.22:0/2378269630 >> 172.21.15.22:0/2378269630 conn(0x56396ba6c8d0 msgr2=0x56396ba72ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.546+0000 7fd9f697a740 1 -- 172.21.15.22:0/2378269630 shutdown_connections 2024-03-09T00:04:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.547+0000 7fd9f697a740 1 -- 172.21.15.22:0/2378269630 wait complete. 2024-03-09T00:04:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.547+0000 7fd9f697a740 1 -- start start 2024-03-09T00:04:10.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.547+0000 7fd9f697a740 1 Processor -- start 2024-03-09T00:04:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba721a0 0x56396badb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396badb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f697a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 0x56396badbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f697a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d000e8d0 con 0x56396ba7b6a0 2024-03-09T00:04:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f697a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d000fbb0 con 0x56396ba721a0 2024-03-09T00:04:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f697a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d000eef0 con 0x56396ba72ac0 2024-03-09T00:04:10.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f697a740 10 client.0 _ll_register_callbacks cb 0x7fd9e4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396badb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396badb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50290/0 (socket says 172.21.15.22:50290) 2024-03-09T00:04:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 -- 172.21.15.22:0/3144641812 learned_addr learned my addr 172.21.15.22:0/3144641812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f225d640 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 0x56396badbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 -- 172.21.15.22:0/3144641812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba721a0 msgr2=0x56396badb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba721a0 0x56396badb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 -- 172.21.15.22:0/3144641812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 msgr2=0x56396badbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 0x56396badbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.548+0000 7fd9f125b640 1 -- 172.21.15.22:0/3144641812 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56396bad6530 con 0x56396ba72ac0 2024-03-09T00:04:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.549+0000 7fd9f125b640 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396badb850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc09aa50 tx=0x7fd9dc0b9de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b166e569ef2651cd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.549+0000 7fd9d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.551+0000 7fd9d57fa640 1 -- 172.21.15.22:0/3144641812 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc081050 con 0x56396ba72ac0 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.551+0000 7fd9d57fa640 1 -- 172.21.15.22:0/3144641812 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fd9dc0ce7c0 con 0x56396ba72ac0 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9dc0a34f0 con 0x56396ba72ac0 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9d57fa640 1 -- 172.21.15.22:0/3144641812 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9dc096ff0 con 0x56396ba72ac0 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 10 client.28033 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56396baf3810 con 0x56396ba72ac0 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9d57fa640 1 -- 172.21.15.22:0/3144641812 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9dc0a34f0 con 0x56396ba72ac0 2024-03-09T00:04:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9d57fa640 1 -- 172.21.15.22:0/3144641812 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd9dc09a330 con 0x56396ba72ac0 2024-03-09T00:04:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 10 client.28033 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 10 client.28033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 20 client.28033 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396baf3c30 0x56396baf6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 4 client.28033 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.552+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56396b9dd750 con 0x56396baf3c30 2024-03-09T00:04:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.554+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396baf3c30 0x56396baf6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.556+0000 7fd9f1a5c640 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396baf3c30 0x56396baf6050 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde3e25780b60a5b server_cookie=995135315fb41d9e in_seq=0 out_seq=0 2024-03-09T00:04:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.556+0000 7fd9d57fa640 10 client.28033 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.558+0000 7fd9d57fa640 1 -- 172.21.15.22:0/3144641812 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56396b9dd750 con 0x56396baf3c30 2024-03-09T00:04:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.558+0000 7fd9d57fa640 10 client.28033 handle_command_reply: tid=0 2024-03-09T00:04:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.558+0000 7fd9f697a740 1 client.28033 shutdown 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 msgr2=0x56396badb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396badb850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc09aa50 tx=0x7fd9dc0b9de0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 shutdown_connections 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56396baf3c30 0x56396baf6050 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56396ba7b6a0 0x56396badbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56396ba72ac0 0x56396badb850 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 --2- 172.21.15.22:0/3144641812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56396ba721a0 0x56396badb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 >> 172.21.15.22:0/3144641812 conn(0x56396ba6c8d0 msgr2=0x56396ba92bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9d57fa640 0 client.28033 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 shutdown_connections 2024-03-09T00:04:10.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.565+0000 7fd9f697a740 1 -- 172.21.15.22:0/3144641812 wait complete. 2024-03-09T00:04:10.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.567+0000 7fd9f697a740 20 client.28033 trim_cache size 0 max 16384 2024-03-09T00:04:10.570 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:10.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.579+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9d80a2950 msgr2=0x7fd9d80a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.579+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9d80a2950 0x7fd9d80a4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x56396ba71c70 tx=0x7fd9e8006b70 comp rx=0 tx=0).stop 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.579+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 msgr2=0x7fd9ec10d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.579+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 0x7fd9ec10d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009fd0 tx=0x7fd9dc00b870 comp rx=0 tx=0).stop 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.580+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 shutdown_connections 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.580+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9d80a2950 0x7fd9d80a4e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.580+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9ec109590 0x7fd9ec10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.580+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9ec103e90 0x7fd9ec10d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.580+0000 7fd9f3cc0640 1 --2- 172.21.15.22:0/2753720179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd9ec1034d0 0x7fd9ec110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.580+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 >> 172.21.15.22:0/2753720179 conn(0x7fd9ec0fef60 msgr2=0x7fd9ec100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.581+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 shutdown_connections 2024-03-09T00:04:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:10.581+0000 7fd9f3cc0640 1 -- 172.21.15.22:0/2753720179 wait complete. 2024-03-09T00:04:10.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:10 smithi062 ceph-mon[36129]: pgmap v304: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 90 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 349 op/s 2024-03-09T00:04:10.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:10 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/567711401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:10.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:10 smithi192 ceph-mon[35995]: pgmap v304: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 90 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 349 op/s 2024-03-09T00:04:10.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:10 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/567711401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:10.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 989784 10.05 MB/sec execute 378 sec latency 78.358 ms 2024-03-09T00:04:11.596 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:04:11.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:04:11.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:04:11.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 991300 10.05 MB/sec execute 379 sec latency 51.030 ms 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 -- 172.21.15.22:0/1676927271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 msgr2=0x7faab810d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1676927271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab810d120 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7faab0009ca0 tx=0x7faab002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 -- 172.21.15.22:0/1676927271 shutdown_connections 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1676927271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab810d120 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1676927271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faab8100e30 0x7faab8104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1676927271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab8100480 0x7faab8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 -- 172.21.15.22:0/1676927271 >> 172.21.15.22:0/1676927271 conn(0x7faab8077350 msgr2=0x7faab8077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 -- 172.21.15.22:0/1676927271 shutdown_connections 2024-03-09T00:04:11.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 -- 172.21.15.22:0/1676927271 wait complete. 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 Processor -- start 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.734+0000 7faabd7f4640 1 -- start start 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faabd7f4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faab8100480 0x7faab819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faabd7f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faabd7f4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab81979d0 0x7faab8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faabd7f4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81105f0 con 0x7faab8105550 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faabd7f4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab8110470 con 0x7faab81979d0 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faabd7f4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab8110770 con 0x7faab8100480 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faab8100480 0x7faab819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36926/0 (socket says 172.21.15.22:36926) 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 -- 172.21.15.22:0/1315023531 learned_addr learned my addr 172.21.15.22:0/1315023531 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faab77fe640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab81979d0 0x7faab8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 -- 172.21.15.22:0/1315023531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faab8100480 msgr2=0x7faab819d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faab8100480 0x7faab819d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 -- 172.21.15.22:0/1315023531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab81979d0 msgr2=0x7faab8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab81979d0 0x7faab8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 -- 172.21.15.22:0/1315023531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab81984f0 con 0x7faab8105550 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faab6ffd640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faab8100480 0x7faab819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.735+0000 7faaaffff640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab819dd60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7faaa000e9c0 tx=0x7faaa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.736+0000 7faab4ff9640 1 -- 172.21.15.22:0/1315023531 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000cd80 con 0x7faab8105550 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.736+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab81987e0 con 0x7faab8105550 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.736+0000 7faab4ff9640 1 -- 172.21.15.22:0/1315023531 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faaa0005500 con 0x7faab8105550 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.736+0000 7faab4ff9640 1 -- 172.21.15.22:0/1315023531 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0010430 con 0x7faab8105550 2024-03-09T00:04:11.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.736+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab81a49d0 con 0x7faab8105550 2024-03-09T00:04:11.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 Processor -- start 2024-03-09T00:04:11.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 -- start start 2024-03-09T00:04:11.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faab4ff9640 1 -- 172.21.15.22:0/1315023531 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7faaa00046c0 con 0x7faab8105550 2024-03-09T00:04:11.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e00ff90bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff91620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ffa29b0 0x55e00ffa4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00fef4b40 con 0x55e00ff911a0 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00fe9dd80 con 0x55e00ff907f0 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.739+0000 7faac03d8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00fef0be0 con 0x55e00ffa29b0 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e00ff90bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e00ff90bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51776/0 (socket says 172.21.15.22:51776) 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 -- 172.21.15.22:0/3909096587 learned_addr learned my addr 172.21.15.22:0/3909096587 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab6ffd640 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff91620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 -- 172.21.15.22:0/3909096587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ffa29b0 msgr2=0x55e00ffa4da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ffa29b0 0x55e00ffa4da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 -- 172.21.15.22:0/3909096587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 msgr2=0x55e00ff91620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff91620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab77fe640 1 -- 172.21.15.22:0/3909096587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e00fcd17e0 con 0x55e00ff907f0 2024-03-09T00:04:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab4ff9640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faaa40a29f0 0x7faaa40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.740+0000 7faab4ff9640 1 -- 172.21.15.22:0/1315023531 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7faaa00cf6f0 con 0x7faab8105550 2024-03-09T00:04:11.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.741+0000 7faab6ffd640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faaa40a29f0 0x7faaa40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.741+0000 7faab77fe640 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e00ff90bd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faab0009ca0 tx=0x7faab0009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20f8d86cdee7665 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.741+0000 7faaad7fa640 1 -- 172.21.15.22:0/3909096587 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0043070 con 0x55e00ff907f0 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.741+0000 7faaad7fa640 1 -- 172.21.15.22:0/3909096587 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faab0032c90 con 0x55e00ff907f0 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.741+0000 7faaad7fa640 1 -- 172.21.15.22:0/3909096587 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0039040 con 0x55e00ff907f0 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 -- 172.21.15.22:0/3909096587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 msgr2=0x55e00ff90bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faab6ffd640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faaa40a29f0 0x7faaa40a4eb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55e00ff91c90 tx=0x7faaa8005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e00ff90bd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faab0009ca0 tx=0x7faab0009900 comp rx=0 tx=0).stop 2024-03-09T00:04:11.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 -- 172.21.15.22:0/3909096587 shutdown_connections 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ffa29b0 0x55e00ffa4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 --2- 172.21.15.22:0/3909096587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e00ff90bd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 -- 172.21.15.22:0/3909096587 >> 172.21.15.22:0/3909096587 conn(0x55e00ff8b4b0 msgr2=0x55e00ff8b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 -- 172.21.15.22:0/3909096587 shutdown_connections 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 -- 172.21.15.22:0/3909096587 wait complete. 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 -- start start 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.742+0000 7faac03d8740 1 Processor -- start 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faac03d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff907f0 0x55e00ff95820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faac03d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff92bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faac03d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ffa29b0 0x55e00ff932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faac03d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ffa5800 con 0x55e00ff911a0 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff907f0 0x55e00ff95820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faaaffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff92bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faaaffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff92bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36956/0 (socket says 172.21.15.22:36956) 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faaaffff640 1 -- 172.21.15.22:0/191595169 learned_addr learned my addr 172.21.15.22:0/191595169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00feb11c0 con 0x55e00ffa29b0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ffa5980 con 0x55e00ff907f0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faab77fe640 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ffa29b0 0x55e00ff932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.743+0000 7faac03d8740 10 client.0 _ll_register_callbacks cb 0x55e00fea07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faaaffff640 1 -- 172.21.15.22:0/191595169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff907f0 msgr2=0x55e00ff95820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faaaffff640 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff907f0 0x55e00ff95820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faaaffff640 1 -- 172.21.15.22:0/191595169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ffa29b0 msgr2=0x55e00ff932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faaaffff640 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ffa29b0 0x55e00ff932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faaaffff640 1 -- 172.21.15.22:0/191595169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e00fd31f70 con 0x55e00ff911a0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab0032c90 con 0x55e00ff911a0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faaaffff640 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff92bb0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7faaa0004fd0 tx=0x7faaa009b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e155dd5a14330e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faa9e7fc640 10 client.18600 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faa9e7fc640 1 -- 172.21.15.22:0/191595169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0086270 con 0x55e00ff911a0 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faa9e7fc640 1 -- 172.21.15.22:0/191595169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7faaa009cbd0 con 0x55e00ff911a0 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faa9e7fc640 1 -- 172.21.15.22:0/191595169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa009cdf0 con 0x55e00ff911a0 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faac03d8740 10 client.18600 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.744+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x55e00fbc9170 con 0x55e00ff911a0 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.745+0000 7faa9e7fc640 1 -- 172.21.15.22:0/191595169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faaa00a3bb0 con 0x55e00ff911a0 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.745+0000 7faac03d8740 10 client.18600 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.745+0000 7faac03d8740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.745+0000 7faac03d8740 20 client.18600 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.745+0000 7faac03d8740 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffb2a10 0x55e00ffb4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.745+0000 7faac03d8740 4 client.18600 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.745+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e00feca960 con 0x55e00ffb2a10 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.746+0000 7faab6ffd640 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffb2a10 0x55e00ffb4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.746+0000 7faab6ffd640 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffb2a10 0x55e00ffb4e00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0240edabe2f851 server_cookie=a2519ef3aff14093 in_seq=0 out_seq=0 2024-03-09T00:04:11.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.746+0000 7faa9e7fc640 10 client.18600 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.747+0000 7faa9e7fc640 1 -- 172.21.15.22:0/191595169 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e00feca960 con 0x55e00ffb2a10 2024-03-09T00:04:11.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.747+0000 7faa9e7fc640 10 client.18600 handle_command_reply: tid=0 2024-03-09T00:04:11.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.747+0000 7faac03d8740 1 client.18600 shutdown 2024-03-09T00:04:11.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 msgr2=0x55e00ff92bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff92bb0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7faaa0004fd0 tx=0x7faaa009b2c0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 shutdown_connections 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffb2a10 0x55e00ffb4e00 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faa9e7fc640 0 client.18600 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ffa29b0 0x55e00ff932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ff92bb0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 --2- 172.21.15.22:0/191595169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff907f0 0x55e00ff95820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 >> 172.21.15.22:0/191595169 conn(0x55e00ff8b4b0 msgr2=0x55e00ff8d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 shutdown_connections 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 1 -- 172.21.15.22:0/191595169 wait complete. 2024-03-09T00:04:11.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.748+0000 7faac03d8740 20 client.18600 trim_cache size 0 max 16384 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.768+0000 7faac03d8740 1 Processor -- start 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faac03d8740 1 -- start start 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faac03d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff8b4b0 0x55e00ff92b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faac03d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff907f0 0x55e00ff952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faac03d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff911a0 0x55e00ff95820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faac03d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81aea60 con 0x55e00ff907f0 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faac03d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81ac600 con 0x55e00ff8b4b0 2024-03-09T00:04:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faac03d8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81aa6c0 con 0x55e00ff911a0 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff907f0 0x55e00ff952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff8b4b0 0x55e00ff92b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff907f0 0x55e00ff952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36962/0 (socket says 172.21.15.22:36962) 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faab77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff911a0 0x55e00ff95820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff8b4b0 0x55e00ff92b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51796/0 (socket says 172.21.15.22:51796) 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 -- 172.21.15.22:0/196705861 learned_addr learned my addr 172.21.15.22:0/196705861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 -- 172.21.15.22:0/196705861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff911a0 msgr2=0x55e00ff95820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 --2- 172.21.15.22:0/196705861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff911a0 0x55e00ff95820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 -- 172.21.15.22:0/196705861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff8b4b0 msgr2=0x55e00ff92b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 --2- 172.21.15.22:0/196705861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff8b4b0 0x55e00ff92b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.769+0000 7faaaffff640 1 -- 172.21.15.22:0/196705861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e00ff93610 con 0x55e00ff907f0 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.770+0000 7faaaffff640 1 --2- 172.21.15.22:0/196705861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff907f0 0x55e00ff952e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7faaa0098270 tx=0x7faaa0086000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d1ce293925ec59 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.770+0000 7faaacff9640 1 -- 172.21.15.22:0/196705861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa007f060 con 0x55e00ff907f0 2024-03-09T00:04:11.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.770+0000 7faaacff9640 1 -- 172.21.15.22:0/196705861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faab0032c90 con 0x55e00ff907f0 2024-03-09T00:04:11.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.770+0000 7faaacff9640 1 -- 172.21.15.22:0/196705861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0086290 con 0x55e00ff907f0 2024-03-09T00:04:11.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 -- 172.21.15.22:0/196705861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff907f0 msgr2=0x55e00ff952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 --2- 172.21.15.22:0/196705861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff907f0 0x55e00ff952e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7faaa0098270 tx=0x7faaa0086000 comp rx=0 tx=0).stop 2024-03-09T00:04:11.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 -- 172.21.15.22:0/196705861 shutdown_connections 2024-03-09T00:04:11.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 --2- 172.21.15.22:0/196705861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff911a0 0x55e00ff95820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 --2- 172.21.15.22:0/196705861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff907f0 0x55e00ff952e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 --2- 172.21.15.22:0/196705861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff8b4b0 0x55e00ff92b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 -- 172.21.15.22:0/196705861 >> 172.21.15.22:0/196705861 conn(0x55e00ff92030 msgr2=0x55e00ff92440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 -- 172.21.15.22:0/196705861 shutdown_connections 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 -- 172.21.15.22:0/196705861 wait complete. 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.771+0000 7faac03d8740 1 -- start start 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 1 Processor -- start 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff8b4b0 0x55e010003440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e010003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ffa9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81aacc0 con 0x55e00ff911a0 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81abfa0 con 0x55e00ff907f0 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab81ab2e0 con 0x55e00ff8b4b0 2024-03-09T00:04:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faac03d8740 10 client.0 _ll_register_callbacks cb 0x7faaa8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff8b4b0 0x55e010003440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff8b4b0 0x55e010003440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50330/0 (socket says 172.21.15.22:50330) 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 -- 172.21.15.22:0/4080394846 learned_addr learned my addr 172.21.15.22:0/4080394846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faaaffff640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e010003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab77fe640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ffa9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 -- 172.21.15.22:0/4080394846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 msgr2=0x55e010003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e010003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 -- 172.21.15.22:0/4080394846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 msgr2=0x55e00ffa9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ffa9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faab6ffd640 1 -- 172.21.15.22:0/4080394846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e00ffa97b0 con 0x55e00ff8b4b0 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.772+0000 7faaaffff640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e010003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faab6ffd640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff8b4b0 0x55e010003440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faaa8004650 tx=0x7faaa80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1137003d1f059bf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faa9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faa9e7fc640 1 -- 172.21.15.22:0/4080394846 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8005020 con 0x55e00ff8b4b0 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faa9e7fc640 1 -- 172.21.15.22:0/4080394846 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7faaa80051c0 con 0x55e00ff8b4b0 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab0032c90 con 0x55e00ff8b4b0 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faa9e7fc640 1 -- 172.21.15.22:0/4080394846 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa80053c0 con 0x55e00ff8b4b0 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faac03d8740 10 client.28048 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.773+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e01000ad00 con 0x55e00ff8b4b0 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faa9e7fc640 1 -- 172.21.15.22:0/4080394846 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faaa8005560 con 0x55e00ff8b4b0 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faa9e7fc640 1 -- 172.21.15.22:0/4080394846 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faaa8031030 con 0x55e00ff8b4b0 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faac03d8740 10 client.28048 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faac03d8740 10 client.28048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faac03d8740 20 client.28048 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faac03d8740 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffa29b0 0x55e01000d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faac03d8740 4 client.28048 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:04:11.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7faab0032c90 con 0x55e00ffa29b0 2024-03-09T00:04:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faaaffff640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffa29b0 0x55e01000d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faa9e7fc640 10 client.28048 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.774+0000 7faaaffff640 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffa29b0 0x55e01000d430 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682de8a79b54aa06 server_cookie=2a83c99b3a4346b6 in_seq=0 out_seq=0 2024-03-09T00:04:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.777+0000 7faa9e7fc640 1 -- 172.21.15.22:0/4080394846 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7faab0032c90 con 0x55e00ffa29b0 2024-03-09T00:04:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.777+0000 7faa9e7fc640 10 client.28048 handle_command_reply: tid=0 2024-03-09T00:04:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.777+0000 7faac03d8740 1 client.28048 shutdown 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.777+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff8b4b0 msgr2=0x55e010003440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.777+0000 7faac03d8740 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff8b4b0 0x55e010003440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faaa8004650 tx=0x7faaa80046c0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 shutdown_connections 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e00ffa29b0 0x55e01000d430 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e00ff911a0 0x55e00ffa9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e00ff907f0 0x55e010003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faa9e7fc640 0 client.28048 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 --2- 172.21.15.22:0/4080394846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ff8b4b0 0x55e010003440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 >> 172.21.15.22:0/4080394846 conn(0x55e00ff92030 msgr2=0x55e00ff8c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 shutdown_connections 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 1 -- 172.21.15.22:0/4080394846 wait complete. 2024-03-09T00:04:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.778+0000 7faac03d8740 20 client.28048 trim_cache size 0 max 16384 2024-03-09T00:04:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:11.777 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:04:11.777 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "105a270a-a29e-4979-8e7b-08d0c94e887e", 2024-03-09T00:04:11.777 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:04:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faaa40a29f0 msgr2=0x7faaa40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faaa40a29f0 0x7faaa40a4eb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55e00ff91c90 tx=0x7faaa8005f90 comp rx=0 tx=0).stop 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 msgr2=0x7faab819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab819dd60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7faaa000e9c0 tx=0x7faaa000ee80 comp rx=0 tx=0).stop 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 shutdown_connections 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faaa40a29f0 0x7faaa40a4eb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab81979d0 0x7faab8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faab8105550 0x7faab819dd60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 --2- 172.21.15.22:0/1315023531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faab8100480 0x7faab819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 >> 172.21.15.22:0/1315023531 conn(0x7faab8077350 msgr2=0x7faab8103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 shutdown_connections 2024-03-09T00:04:11.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.779+0000 7faabd7f4640 1 -- 172.21.15.22:0/1315023531 wait complete. 2024-03-09T00:04:11.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:04:11.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:04:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 -- 172.21.15.22:0/2448399549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 msgr2=0x7f10b810e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 --2- 172.21.15.22:0/2448399549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b810e1b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009900 tx=0x7f10a802bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 -- 172.21.15.22:0/2448399549 shutdown_connections 2024-03-09T00:04:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 --2- 172.21.15.22:0/2448399549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b810e1b0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 --2- 172.21.15.22:0/2448399549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10b8101ec0 0x7f10b8105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 --2- 172.21.15.22:0/2448399549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b8101510 0x7f10b81018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 -- 172.21.15.22:0/2448399549 >> 172.21.15.22:0/2448399549 conn(0x7f10b80fcd30 msgr2=0x7f10b80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 -- 172.21.15.22:0/2448399549 shutdown_connections 2024-03-09T00:04:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.925+0000 7f10bd37e640 1 -- 172.21.15.22:0/2448399549 wait complete. 2024-03-09T00:04:11.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 Processor -- start 2024-03-09T00:04:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 -- start start 2024-03-09T00:04:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10b8101510 0x7f10b819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b8197ab0 0x7f10b8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81116f0 con 0x7f10b81065e0 2024-03-09T00:04:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8111570 con 0x7f10b8197ab0 2024-03-09T00:04:11.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.926+0000 7f10bd37e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8111870 con 0x7f10b8101510 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10b8101510 0x7f10b819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36974/0 (socket says 172.21.15.22:36974) 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10b8101510 0x7f10b819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50354/0 (socket says 172.21.15.22:50354) 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 -- 172.21.15.22:0/36625716 learned_addr learned my addr 172.21.15.22:0/36625716 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b77fe640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b8197ab0 0x7f10b8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 -- 172.21.15.22:0/36625716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10b8101510 msgr2=0x7f10b819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10b8101510 0x7f10b819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 -- 172.21.15.22:0/36625716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b8197ab0 msgr2=0x7f10b8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b8197ab0 0x7f10b8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 -- 172.21.15.22:0/36625716 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b81985d0 con 0x7f10b81065e0 2024-03-09T00:04:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10b67fc640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b819ddf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e9f0 tx=0x7f10ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f109ffff640 1 -- 172.21.15.22:0/36625716 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00ccc0 con 0x7f10b81065e0 2024-03-09T00:04:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f109ffff640 1 -- 172.21.15.22:0/36625716 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10ac005500 con 0x7f10b81065e0 2024-03-09T00:04:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b81988c0 con 0x7f10b81065e0 2024-03-09T00:04:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f109ffff640 1 -- 172.21.15.22:0/36625716 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac010690 con 0x7f10b81065e0 2024-03-09T00:04:11.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.927+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b81a4a70 con 0x7f10b81065e0 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.930+0000 7f109ffff640 1 -- 172.21.15.22:0/36625716 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f10ac010830 con 0x7f10b81065e0 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.930+0000 7f10c0037740 1 Processor -- start 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.930+0000 7f10c0037740 1 -- start start 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.930+0000 7f10c0037740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90719b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.930+0000 7f10c0037740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.930+0000 7f10c0037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90725050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10c0037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9067ce00 con 0x555b90722c60 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10c0037740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b90625b10 con 0x555b90719760 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10c0037740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b90678ea0 con 0x555b9071a080 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50364/0 (socket says 172.21.15.22:50364) 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b6ffd640 1 -- 172.21.15.22:0/201172949 learned_addr learned my addr 172.21.15.22:0/201172949 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b77fe640 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90719b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f109ffff640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10a00a2970 0x7f10a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b6ffd640 1 -- 172.21.15.22:0/201172949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 msgr2=0x555b90719b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90719b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b6ffd640 1 -- 172.21.15.22:0/201172949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 msgr2=0x555b90725050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b77fe640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10a00a2970 0x7f10a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f109ffff640 1 -- 172.21.15.22:0/36625716 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f10ac0cf520 con 0x7f10b81065e0 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.931+0000 7f10b67fc640 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90725050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90725050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f10b6ffd640 1 -- 172.21.15.22:0/201172949 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b9045b7d0 con 0x555b9071a080 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f10b67fc640 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90725050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f10b77fe640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10a00a2970 0x7f10a00a4e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f10a8006010 tx=0x7f10a8009260 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f10b00039f0 tx=0x7f10b0003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f8ade392e0e4f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f109d7fa640 1 -- 172.21.15.22:0/201172949 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0011440 con 0x555b9071a080 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f109d7fa640 1 -- 172.21.15.22:0/201172949 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10b0015030 con 0x555b9071a080 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.932+0000 7f109d7fa640 1 -- 172.21.15.22:0/201172949 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0010a90 con 0x555b9071a080 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 -- 172.21.15.22:0/201172949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 msgr2=0x555b90722720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f10b00039f0 tx=0x7f10b0003eb0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 -- 172.21.15.22:0/201172949 shutdown_connections 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90725050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722720 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 --2- 172.21.15.22:0/201172949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90719b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 -- 172.21.15.22:0/201172949 >> 172.21.15.22:0/201172949 conn(0x555b90713e90 msgr2=0x555b90714270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 -- 172.21.15.22:0/201172949 shutdown_connections 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 -- 172.21.15.22:0/201172949 wait complete. 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.933+0000 7f10c0037740 1 -- start start 2024-03-09T00:04:11.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 1 Processor -- start 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90722350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b9071f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b9071fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b90725a50 con 0x555b90722c60 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b906390a0 con 0x555b90719760 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b90725bd0 con 0x555b9071a080 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10c0037740 10 client.0 _ll_register_callbacks cb 0x555b90638a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b9071fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b9071f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b9071f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50370/0 (socket says 172.21.15.22:50370) 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b67fc640 1 -- 172.21.15.22:0/2508354052 learned_addr learned my addr 172.21.15.22:0/2508354052 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90722350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b77fe640 1 -- 172.21.15.22:0/2508354052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 msgr2=0x555b9071f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b77fe640 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b9071f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b77fe640 1 -- 172.21.15.22:0/2508354052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 msgr2=0x555b90722350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b77fe640 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90722350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b77fe640 1 -- 172.21.15.22:0/2508354052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b903913f0 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.934+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90722350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f10b77fe640 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b9071fc20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f10a8036040 tx=0x7f10a802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e2037c73212fad server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f109aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f109aff5640 1 -- 172.21.15.22:0/2508354052 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a802cbc0 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f109aff5640 1 -- 172.21.15.22:0/2508354052 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f10a8004f80 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10b0015030 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f109aff5640 1 -- 172.21.15.22:0/2508354052 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10a8005180 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f10c0037740 10 client.18615 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555b903510f0 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.935+0000 7f109aff5640 1 -- 172.21.15.22:0/2508354052 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10b0015030 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f109aff5640 1 -- 172.21.15.22:0/2508354052 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f10a8048070 con 0x555b90722c60 2024-03-09T00:04:11.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10c0037740 10 client.18615 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10c0037740 10 client.18615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10c0037740 20 client.18615 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10c0037740 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90732620 0x555b90734a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10c0037740 4 client.18615 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b9068a340 con 0x555b90732620 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90732620 0x555b90734a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90732620 0x555b90734a10 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff349ecb327be13d server_cookie=d3ce8e3dec8241ac in_seq=0 out_seq=0 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.936+0000 7f109aff5640 10 client.18615 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.937+0000 7f109aff5640 1 -- 172.21.15.22:0/2508354052 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b9068a340 con 0x555b90732620 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.937+0000 7f109aff5640 10 client.18615 handle_command_reply: tid=0 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.937+0000 7f10c0037740 1 client.18615 shutdown 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.937+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 msgr2=0x555b9071fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.937+0000 7f10c0037740 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b9071fc20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f10a8036040 tx=0x7f10a802bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 shutdown_connections 2024-03-09T00:04:11.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90732620 0x555b90734a10 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b9071fc20 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f109aff5640 0 client.18615 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b9071f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 --2- 172.21.15.22:0/2508354052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90722350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 >> 172.21.15.22:0/2508354052 conn(0x555b90713e90 msgr2=0x555b90715be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 shutdown_connections 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 1 -- 172.21.15.22:0/2508354052 wait complete. 2024-03-09T00:04:11.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.938+0000 7f10c0037740 20 client.18615 trim_cache size 0 max 16384 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.959+0000 7f10c0037740 1 Processor -- start 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.959+0000 7f10c0037740 1 -- start start 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.959+0000 7f10c0037740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90721e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.959+0000 7f10c0037740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.959+0000 7f10c0037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90729e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10c0037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81b0ce0 con 0x555b90722c60 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10c0037740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81ae880 con 0x555b90719760 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10c0037740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81ac940 con 0x555b9071a080 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90729e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90729e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36998/0 (socket says 172.21.15.22:36998) 2024-03-09T00:04:11.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 -- 172.21.15.22:0/3586568670 learned_addr learned my addr 172.21.15.22:0/3586568670 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90721e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b67fc640 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 -- 172.21.15.22:0/3586568670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 msgr2=0x555b90722350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 -- 172.21.15.22:0/3586568670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 msgr2=0x555b90721e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90721e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 -- 172.21.15.22:0/3586568670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b90719eb0 con 0x555b90722c60 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90721e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.960+0000 7f10b77fe640 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90729e00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f10a80098a0 tx=0x7f10a802c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8e9250e0181b29 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.961+0000 7f109cff9640 1 -- 172.21.15.22:0/3586568670 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004350 con 0x555b90722c60 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.961+0000 7f109cff9640 1 -- 172.21.15.22:0/3586568670 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10a8045070 con 0x555b90722c60 2024-03-09T00:04:11.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.961+0000 7f109cff9640 1 -- 172.21.15.22:0/3586568670 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a80038e0 con 0x555b90722c60 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 -- 172.21.15.22:0/3586568670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 msgr2=0x555b90729e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90729e00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f10a80098a0 tx=0x7f10a802c350 comp rx=0 tx=0).stop 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 -- 172.21.15.22:0/3586568670 shutdown_connections 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b90722c60 0x555b90729e00 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9071a080 0x555b90722350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 --2- 172.21.15.22:0/3586568670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90719760 0x555b90721e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 -- 172.21.15.22:0/3586568670 >> 172.21.15.22:0/3586568670 conn(0x555b90713e90 msgr2=0x555b9071f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 -- 172.21.15.22:0/3586568670 shutdown_connections 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 -- 172.21.15.22:0/3586568670 wait complete. 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 -- start start 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 Processor -- start 2024-03-09T00:04:11.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b90719760 0x555b90729e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.962+0000 7f10c0037740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b9071a080 0x555b9072a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10c0037740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90722c60 0x555b907263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10c0037740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81acf40 con 0x555b9071a080 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10c0037740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81ae220 con 0x555b90722c60 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10c0037740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81ad560 con 0x555b90719760 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10c0037740 10 client.0 _ll_register_callbacks cb 0x7f10a8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b90719760 0x555b90729e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b90719760 0x555b90729e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50384/0 (socket says 172.21.15.22:50384) 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 -- 172.21.15.22:0/2058902436 learned_addr learned my addr 172.21.15.22:0/2058902436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b67fc640 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b9071a080 0x555b9072a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b77fe640 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90722c60 0x555b907263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 -- 172.21.15.22:0/2058902436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90722c60 msgr2=0x555b907263f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90722c60 0x555b907263f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 -- 172.21.15.22:0/2058902436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b9071a080 msgr2=0x555b9072a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b9071a080 0x555b9072a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 -- 172.21.15.22:0/2058902436 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b90726bf0 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f10b6ffd640 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b90719760 0x555b90729e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f10b0011fd0 tx=0x7f10b0010550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17ad268872c556e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.963+0000 7f109aff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f109aff5640 1 -- 172.21.15.22:0/2058902436 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0004740 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10a8045070 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f109aff5640 1 -- 172.21.15.22:0/2058902436 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f10b00048e0 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f109aff5640 1 -- 172.21.15.22:0/2058902436 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b0039050 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 10 client.28066 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555b90789740 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f109aff5640 1 -- 172.21.15.22:0/2058902436 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10a8045070 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f109aff5640 1 -- 172.21.15.22:0/2058902436 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f10b0010660 con 0x555b90719760 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 10 client.28066 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 10 client.28066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:11.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 20 client.28066 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90789b40 0x555b9078bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 4 client.28066 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555b90720620 con 0x555b90789b40 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.964+0000 7f10b67fc640 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90789b40 0x555b9078bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.965+0000 7f109aff5640 10 client.28066 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.965+0000 7f10b67fc640 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90789b40 0x555b9078bf60 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee243afd60ac8c30 server_cookie=c39b74cc17b7801a in_seq=0 out_seq=0 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f109aff5640 1 -- 172.21.15.22:0/2058902436 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555b90720620 con 0x555b90789b40 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f109aff5640 10 client.28066 handle_command_reply: tid=0 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 client.28066 shutdown 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b90719760 msgr2=0x555b90729e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b90719760 0x555b90729e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f10b0011fd0 tx=0x7f10b0010550 comp rx=0 tx=0).stop 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 shutdown_connections 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555b90789b40 0x555b9078bf60 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b90722c60 0x555b907263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555b9071a080 0x555b9072a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 --2- 172.21.15.22:0/2058902436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b90719760 0x555b90729e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 >> 172.21.15.22:0/2058902436 conn(0x555b90713e90 msgr2=0x555b9071def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f109aff5640 0 client.28066 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 shutdown_connections 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 1 -- 172.21.15.22:0/2058902436 wait complete. 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.966+0000 7f10c0037740 20 client.28066 trim_cache size 0 max 16384 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:04:11.967 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:04:11.968 INFO:teuthology.orchestra.run.smithi022.stdout: "105a270a-a29e-4979-8e7b-08d0c94e887e": { 2024-03-09T00:04:11.968 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:04:11.968 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "105a270a-a29e-4979-8e7b-08d0c94e887e", 2024-03-09T00:04:11.968 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:04:11.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:04:11.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:04:11.968 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10a00a2970 msgr2=0x7f10a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10a00a2970 0x7f10a00a4e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f10a8006010 tx=0x7f10a8009260 comp rx=0 tx=0).stop 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 msgr2=0x7f10b819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b819ddf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e9f0 tx=0x7f10ac00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 shutdown_connections 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10a00a2970 0x7f10a00a4e30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b8197ab0 0x7f10b8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10b81065e0 0x7f10b819ddf0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 --2- 172.21.15.22:0/36625716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10b8101510 0x7f10b819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 >> 172.21.15.22:0/36625716 conn(0x7f10b80fcd30 msgr2=0x7f10b80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:11.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.967+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 shutdown_connections 2024-03-09T00:04:11.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:11.968+0000 7f10bd37e640 1 -- 172.21.15.22:0/36625716 wait complete. 2024-03-09T00:04:11.974 INFO:tasks.cephfs.filesystem:scrub status for tag:105a270a-a29e-4979-8e7b-08d0c94e887e - {'path': '/', 'tag': '105a270a-a29e-4979-8e7b-08d0c94e887e', 'options': 'recursive,force'} 2024-03-09T00:04:12.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:12 smithi022 ceph-mon[31894]: pgmap v305: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-09T00:04:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:12 smithi062 ceph-mon[36129]: pgmap v305: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-09T00:04:12.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:12 smithi192 ceph-mon[35995]: pgmap v305: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-09T00:04:12.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 992221 10.03 MB/sec execute 380 sec latency 23.681 ms 2024-03-09T00:04:13.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:13 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:04:13.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:13 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:13.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:13 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:04:13.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:13 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:13.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:13 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:04:13.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:13 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:13.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:13 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:04:13.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:13 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:13.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:13 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:04:13.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:13 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:13.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:13 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:04:13.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:13 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:13.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 994774 10.05 MB/sec execute 381 sec latency 25.254 ms 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f51514be640 1 -- 172.21.15.22:0/1943082109 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c039070 con 0x7f514c104fd0 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 -- 172.21.15.22:0/1943082109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c104fd0 msgr2=0x7f514c1053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 --2- 172.21.15.22:0/1943082109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c104fd0 0x7f514c1053b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f513c009860 tx=0x7f513c02bb90 comp rx=0 tx=0).stop 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 -- 172.21.15.22:0/1943082109 shutdown_connections 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 --2- 172.21.15.22:0/1943082109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c100170 0x7f514c1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 --2- 172.21.15.22:0/1943082109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514c105980 0x7f514c0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 --2- 172.21.15.22:0/1943082109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c104fd0 0x7f514c1053b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 -- 172.21.15.22:0/1943082109 >> 172.21.15.22:0/1943082109 conn(0x7f514c075530 msgr2=0x7f514c075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.019+0000 7f5154724640 1 -- 172.21.15.22:0/1943082109 shutdown_connections 2024-03-09T00:04:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.020+0000 7f5154724640 1 -- 172.21.15.22:0/1943082109 wait complete. 2024-03-09T00:04:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.021+0000 7f5154724640 1 Processor -- start 2024-03-09T00:04:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.021+0000 7f5154724640 1 -- start start 2024-03-09T00:04:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5154724640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c100170 0x7f514c115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5154724640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514c104fd0 0x7f514c10e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5154724640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c105980 0x7f514c10e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5154724640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c117e10 con 0x7f514c105980 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5154724640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c117c90 con 0x7f514c104fd0 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5154724640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c117f90 con 0x7f514c100170 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5152cc1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c105980 0x7f514c10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5152cc1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c105980 0x7f514c10e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37032/0 (socket says 172.21.15.22:37032) 2024-03-09T00:04:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.022+0000 7f5152cc1640 1 -- 172.21.15.22:0/1175899361 learned_addr learned my addr 172.21.15.22:0/1175899361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.023+0000 7f5151cbf640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514c104fd0 0x7f514c10e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.023+0000 7f5152cc1640 1 -- 172.21.15.22:0/1175899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c100170 msgr2=0x7f514c115060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:04:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.023+0000 7f5152cc1640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c100170 0x7f514c115060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.023+0000 7f5152cc1640 1 -- 172.21.15.22:0/1175899361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514c104fd0 msgr2=0x7f514c10e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.023+0000 7f5152cc1640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514c104fd0 0x7f514c10e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.023+0000 7f5152cc1640 1 -- 172.21.15.22:0/1175899361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c10ebe0 con 0x7f514c105980 2024-03-09T00:04:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.025+0000 7f5152cc1640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c105980 0x7f514c10e620 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f514400eaa0 tx=0x7f514400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.026+0000 7f51437fe640 1 -- 172.21.15.22:0/1175899361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400cdb0 con 0x7f514c105980 2024-03-09T00:04:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.026+0000 7f51437fe640 1 -- 172.21.15.22:0/1175899361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5144005020 con 0x7f514c105980 2024-03-09T00:04:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.026+0000 7f51437fe640 1 -- 172.21.15.22:0/1175899361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400cdb0 con 0x7f514c105980 2024-03-09T00:04:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.027+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c10eed0 con 0x7f514c105980 2024-03-09T00:04:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.027+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c112eb0 con 0x7f514c105980 2024-03-09T00:04:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.030+0000 7f51573d6740 1 Processor -- start 2024-03-09T00:04:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.030+0000 7f51437fe640 1 -- 172.21.15.22:0/1175899361 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f51440051d0 con 0x7f514c105980 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51573d6740 1 -- start start 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51573d6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a2097a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51573d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a2098450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51573d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 0x5584a20a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51573d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a1ffc5f0 con 0x5584a2097fd0 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51573d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a1fa5930 con 0x5584a20a1550 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51573d6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a1ff8690 con 0x5584a2098d40 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51437fe640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51380a2c10 0x7f51380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f5151cbf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a2097a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f51524c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a2098450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f5151cbf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a2097a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50402/0 (socket says 172.21.15.22:50402) 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f5151cbf640 1 -- 172.21.15.22:0/2828341239 learned_addr learned my addr 172.21.15.22:0/2828341239 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.031+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 0x5584a20a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f51437fe640 1 -- 172.21.15.22:0/1175899361 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f51440cf170 con 0x7f514c105980 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f5152cc1640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51380a2c10 0x7f51380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f51524c0640 1 -- 172.21.15.22:0/2828341239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 msgr2=0x5584a2097a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f51524c0640 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a2097a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f51524c0640 1 -- 172.21.15.22:0/2828341239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 msgr2=0x5584a20a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f51524c0640 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 0x5584a20a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f51524c0640 1 -- 172.21.15.22:0/2828341239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a1d11210 con 0x5584a2097fd0 2024-03-09T00:04:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f51524c0640 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a2098450 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f513c02c0d0 tx=0x7f513c035e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb055dfbcebd766 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f5140ff9640 1 -- 172.21.15.22:0/2828341239 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c039040 con 0x5584a2097fd0 2024-03-09T00:04:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f5152cc1640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51380a2c10 0x7f51380a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f514400efd0 tx=0x7f5144087040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.032+0000 7f5140ff9640 1 -- 172.21.15.22:0/2828341239 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513c009410 con 0x5584a2097fd0 2024-03-09T00:04:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f5140ff9640 1 -- 172.21.15.22:0/2828341239 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0037f0 con 0x5584a2097fd0 2024-03-09T00:04:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 -- 172.21.15.22:0/2828341239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 msgr2=0x5584a2098450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a2098450 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f513c02c0d0 tx=0x7f513c035e40 comp rx=0 tx=0).stop 2024-03-09T00:04:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 -- 172.21.15.22:0/2828341239 shutdown_connections 2024-03-09T00:04:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 0x5584a20a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a2098450 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 --2- 172.21.15.22:0/2828341239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a2097a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 -- 172.21.15.22:0/2828341239 >> 172.21.15.22:0/2828341239 conn(0x5584a205f5b0 msgr2=0x5584a2092cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.033+0000 7f51573d6740 1 -- 172.21.15.22:0/2828341239 shutdown_connections 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.034+0000 7f51573d6740 1 -- 172.21.15.22:0/2828341239 wait complete. 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.034+0000 7f51573d6740 1 -- start start 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.034+0000 7f51573d6740 1 Processor -- start 2024-03-09T00:04:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.034+0000 7f51573d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f5151cbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f5151cbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37046/0 (socket says 172.21.15.22:37046) 2024-03-09T00:04:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f5151cbf640 1 -- 172.21.15.22:0/273271376 learned_addr learned my addr 172.21.15.22:0/273271376 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a209eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 0x5584a209f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a20a4220 con 0x5584a2097fd0 2024-03-09T00:04:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a1fb8d90 con 0x5584a20a1550 2024-03-09T00:04:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.035+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a20a43a0 con 0x5584a2098d40 2024-03-09T00:04:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f51573d6740 10 client.0 _ll_register_callbacks cb 0x5584a1d10a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f51524c0640 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a209eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5151cbf640 1 -- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 msgr2=0x5584a209eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5151cbf640 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a209eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5151cbf640 1 -- 172.21.15.22:0/273271376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 msgr2=0x5584a209f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5151cbf640 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 0x5584a209f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5151cbf640 1 -- 172.21.15.22:0/273271376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584a1ddd620 con 0x5584a2097fd0 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f51524c0640 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a209eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5151cbf640 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209e610 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5148009d40 tx=0x7f5148003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cb6c4007325f6e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.036+0000 7f5135ffb640 1 -- 172.21.15.22:0/273271376 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148012260 con 0x5584a2097fd0 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.037+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f513c009410 con 0x5584a2097fd0 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.037+0000 7f5135ffb640 1 -- 172.21.15.22:0/273271376 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f514800e800 con 0x5584a2097fd0 2024-03-09T00:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.037+0000 7f5135ffb640 1 -- 172.21.15.22:0/273271376 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f514800ea00 con 0x5584a2097fd0 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.038+0000 7f51573d6740 10 client.18633 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.038+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5584a20b05c0 con 0x5584a2097fd0 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.039+0000 7f5135ffb640 1 -- 172.21.15.22:0/273271376 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513c009410 con 0x5584a2097fd0 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.039+0000 7f5135ffb640 1 -- 172.21.15.22:0/273271376 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5148011440 con 0x5584a2097fd0 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.040+0000 7f51573d6740 10 client.18633 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.040+0000 7f51573d6740 10 client.18633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.040+0000 7f51573d6740 20 client.18633 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.040+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a20b1a30 0x5584a20b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.040+0000 7f51573d6740 4 client.18633 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.040+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584a2009b30 con 0x5584a20b1a30 2024-03-09T00:04:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.040+0000 7f51524c0640 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a20b1a30 0x5584a20b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.042+0000 7f5135ffb640 10 client.18633 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.042+0000 7f51524c0640 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a20b1a30 0x5584a20b3e20 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860b75541c5c3b52 server_cookie=f676b924ff1342a4 in_seq=0 out_seq=0 2024-03-09T00:04:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f5135ffb640 1 -- 172.21.15.22:0/273271376 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584a2009b30 con 0x5584a20b1a30 2024-03-09T00:04:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f5135ffb640 10 client.18633 handle_command_reply: tid=0 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 client.18633 shutdown 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 msgr2=0x5584a209e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209e610 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5148009d40 tx=0x7f5148003d00 comp rx=0 tx=0).stop 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 shutdown_connections 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a20b1a30 0x5584a20b3e20 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a20a1550 0x5584a209f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a2098d40 0x5584a209eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 --2- 172.21.15.22:0/273271376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209e610 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 >> 172.21.15.22:0/273271376 conn(0x5584a205f5b0 msgr2=0x5584a2094a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f5135ffb640 0 client.18633 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 shutdown_connections 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 1 -- 172.21.15.22:0/273271376 wait complete. 2024-03-09T00:04:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.044+0000 7f51573d6740 20 client.18633 trim_cache size 0 max 16384 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.063+0000 7f51573d6740 1 Processor -- start 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51573d6740 1 -- start start 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51573d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2097fd0 0x5584a20a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51573d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2098d40 0x5584a20a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51573d6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a20a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51573d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1c82b0 con 0x5584a2098d40 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51573d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1c5e50 con 0x5584a2097fd0 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51573d6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c04f050 con 0x5584a20a1550 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2098d40 0x5584a20a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2098d40 0x5584a20a8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37050/0 (socket says 172.21.15.22:37050) 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 -- 172.21.15.22:0/2125595828 learned_addr learned my addr 172.21.15.22:0/2125595828 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2097fd0 0x5584a20a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a20a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 -- 172.21.15.22:0/2125595828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 msgr2=0x5584a20a8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a20a8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 -- 172.21.15.22:0/2125595828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2097fd0 msgr2=0x5584a20a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2097fd0 0x5584a20a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 -- 172.21.15.22:0/2125595828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a20972a0 con 0x5584a2098d40 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2097fd0 0x5584a20a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.064+0000 7f51524c0640 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2098d40 0x5584a20a8610 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f513c02c1a0 tx=0x7f513c034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc38e1c2c411084 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f5137fff640 1 -- 172.21.15.22:0/2125595828 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c039040 con 0x5584a2098d40 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f5137fff640 1 -- 172.21.15.22:0/2125595828 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513c0348f0 con 0x5584a2098d40 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f5137fff640 1 -- 172.21.15.22:0/2125595828 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c04d360 con 0x5584a2098d40 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 -- 172.21.15.22:0/2125595828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2098d40 msgr2=0x5584a20a8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2098d40 0x5584a20a8610 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f513c02c1a0 tx=0x7f513c034880 comp rx=0 tx=0).stop 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 -- 172.21.15.22:0/2125595828 shutdown_connections 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a20a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2098d40 0x5584a20a8610 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 --2- 172.21.15.22:0/2125595828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2097fd0 0x5584a20a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 -- 172.21.15.22:0/2125595828 >> 172.21.15.22:0/2125595828 conn(0x5584a205f5b0 msgr2=0x5584a20927f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 -- 172.21.15.22:0/2125595828 shutdown_connections 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.065+0000 7f51573d6740 1 -- 172.21.15.22:0/2125595828 wait complete. 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 -- start start 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 Processor -- start 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2098d40 0x5584a209b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a209bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c072860 con 0x5584a2097fd0 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1c57f0 con 0x5584a2098d40 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c072e60 con 0x5584a20a1550 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51573d6740 10 client.0 _ll_register_callbacks cb 0x7f513c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f5151cbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f5151cbf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37054/0 (socket says 172.21.15.22:37054) 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f5151cbf640 1 -- 172.21.15.22:0/2015899808 learned_addr learned my addr 172.21.15.22:0/2015899808 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a209bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f51524c0640 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2098d40 0x5584a209b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f5151cbf640 1 -- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 msgr2=0x5584a209bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.066+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a209bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5151cbf640 1 -- 172.21.15.22:0/2015899808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2098d40 msgr2=0x5584a209b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2098d40 0x5584a209b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5151cbf640 1 -- 172.21.15.22:0/2015899808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584a2107f40 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5151cbf640 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209b2a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f51480124b0 tx=0x7f51480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ed516eb54a791 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5135ffb640 1 -- 172.21.15.22:0/2015899808 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51480122d0 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5135ffb640 1 -- 172.21.15.22:0/2015899808 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f5148011be0 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f513c0348f0 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5135ffb640 1 -- 172.21.15.22:0/2015899808 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f514803d050 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f51573d6740 10 client.18639 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5584a2109210 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.067+0000 7f5135ffb640 1 -- 172.21.15.22:0/2015899808 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513c0348f0 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f5135ffb640 1 -- 172.21.15.22:0/2015899808 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f514800f420 con 0x5584a2097fd0 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51573d6740 10 client.18639 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51573d6740 10 client.18639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51573d6740 20 client.18639 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51573d6740 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a2109820 0x5584a210bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51573d6740 4 client.18639 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:04:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584a2003e80 con 0x5584a2109820 2024-03-09T00:04:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51524c0640 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a2109820 0x5584a210bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f5135ffb640 10 client.18639 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.068+0000 7f51524c0640 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a2109820 0x5584a210bc10 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70bcd52937dee02 server_cookie=774e9335320cb097 in_seq=0 out_seq=0 2024-03-09T00:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.293+0000 7f5135ffb640 1 -- 172.21.15.22:0/2015899808 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584a2003e80 con 0x5584a2109820 2024-03-09T00:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.293+0000 7f5135ffb640 10 client.18639 handle_command_reply: tid=0 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.293+0000 7f51573d6740 1 client.18639 shutdown 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 msgr2=0x5584a209b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209b2a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f51480124b0 tx=0x7f51480046c0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 shutdown_connections 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5584a2109820 0x5584a210bc10 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5584a20a1550 0x5584a209bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f5135ffb640 0 client.18639 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a2098d40 0x5584a209b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 --2- 172.21.15.22:0/2015899808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5584a2097fd0 0x5584a209b2a0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 >> 172.21.15.22:0/2015899808 conn(0x5584a205f5b0 msgr2=0x5584a209cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 shutdown_connections 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 1 -- 172.21.15.22:0/2015899808 wait complete. 2024-03-09T00:04:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f51573d6740 20 client.18639 trim_cache size 0 max 16384 2024-03-09T00:04:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:04:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:04:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:04:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51380a2c10 msgr2=0x7f51380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f5154724640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51380a2c10 0x7f51380a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f514400efd0 tx=0x7f5144087040 comp rx=0 tx=0).stop 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c105980 msgr2=0x7f514c10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.294+0000 7f5154724640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c105980 0x7f514c10e620 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f514400eaa0 tx=0x7f514400ef60 comp rx=0 tx=0).stop 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 shutdown_connections 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f51380a2c10 0x7f51380a50d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f514c105980 0x7f514c10e620 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514c104fd0 0x7f514c10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 --2- 172.21.15.22:0/1175899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c100170 0x7f514c115060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 >> 172.21.15.22:0/1175899361 conn(0x7f514c075530 msgr2=0x7f514c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 shutdown_connections 2024-03-09T00:04:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:14.295+0000 7f5154724640 1 -- 172.21.15.22:0/1175899361 wait complete. 2024-03-09T00:04:14.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:14 smithi022 ceph-mon[31894]: pgmap v306: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-09T00:04:14.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:14 smithi062 ceph-mon[36129]: pgmap v306: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-09T00:04:14.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:14 smithi192 ceph-mon[35995]: pgmap v306: 129 pgs: 12 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 87 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-09T00:04:14.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 995228 10.03 MB/sec execute 382 sec latency 142.338 ms 2024-03-09T00:04:15.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:15 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:04:15.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:15 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:04:15.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:15 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:04:15.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 996636 10.01 MB/sec execute 383 sec latency 32.506 ms 2024-03-09T00:04:16.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:16 smithi022 ceph-mon[31894]: pgmap v307: 129 pgs: 129 active+clean; 88 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-03-09T00:04:16.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:16 smithi062 ceph-mon[36129]: pgmap v307: 129 pgs: 129 active+clean; 88 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-03-09T00:04:16.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:16 smithi192 ceph-mon[35995]: pgmap v307: 129 pgs: 129 active+clean; 88 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-03-09T00:04:16.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 998645 10.02 MB/sec execute 384 sec latency 16.560 ms 2024-03-09T00:04:17.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1000465 10.01 MB/sec execute 385 sec latency 20.444 ms 2024-03-09T00:04:18.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:18 smithi022 ceph-mon[31894]: pgmap v308: 129 pgs: 129 active+clean; 88 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-03-09T00:04:18.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:18 smithi062 ceph-mon[36129]: pgmap v308: 129 pgs: 129 active+clean; 88 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-03-09T00:04:18.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:18 smithi192 ceph-mon[35995]: pgmap v308: 129 pgs: 129 active+clean; 88 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-03-09T00:04:18.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1002276 10.02 MB/sec execute 386 sec latency 16.874 ms 2024-03-09T00:04:19.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1003725 10.01 MB/sec execute 387 sec latency 84.252 ms 2024-03-09T00:04:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:19 smithi022 ceph-mon[31894]: osdmap e108: 12 total, 12 up, 12 in 2024-03-09T00:04:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:19 smithi022 ceph-mon[31894]: pgmap v310: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 272 op/s 2024-03-09T00:04:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:19 smithi062 ceph-mon[36129]: osdmap e108: 12 total, 12 up, 12 in 2024-03-09T00:04:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:19 smithi062 ceph-mon[36129]: pgmap v310: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 272 op/s 2024-03-09T00:04:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:19 smithi192 ceph-mon[35995]: osdmap e108: 12 total, 12 up, 12 in 2024-03-09T00:04:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:19 smithi192 ceph-mon[35995]: pgmap v310: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 272 op/s 2024-03-09T00:04:20.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1005529 10.02 MB/sec execute 388 sec latency 138.663 ms 2024-03-09T00:04:21.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1006902 10.01 MB/sec execute 389 sec latency 39.237 ms 2024-03-09T00:04:22.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:22 smithi062 ceph-mon[36129]: pgmap v311: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-09T00:04:22.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:22 smithi192 ceph-mon[35995]: pgmap v311: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-09T00:04:22.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:22 smithi022 ceph-mon[31894]: pgmap v311: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-09T00:04:22.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1009030 10.01 MB/sec execute 390 sec latency 36.038 ms 2024-03-09T00:04:23.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1010164 10.00 MB/sec execute 391 sec latency 23.631 ms 2024-03-09T00:04:24.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:24 smithi062 ceph-mon[36129]: pgmap v312: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-09T00:04:24.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:24 smithi192 ceph-mon[35995]: pgmap v312: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-09T00:04:24.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:24 smithi022 ceph-mon[31894]: pgmap v312: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-09T00:04:24.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1012513 10.01 MB/sec execute 392 sec latency 29.687 ms 2024-03-09T00:04:25.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1013104 9.99 MB/sec execute 393 sec latency 144.235 ms 2024-03-09T00:04:26.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:26 smithi062 ceph-mon[36129]: pgmap v313: 129 pgs: 129 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-03-09T00:04:26.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:26 smithi192 ceph-mon[35995]: pgmap v313: 129 pgs: 129 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-03-09T00:04:26.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:26 smithi022 ceph-mon[31894]: pgmap v313: 129 pgs: 129 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-03-09T00:04:26.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1015437 10.00 MB/sec execute 394 sec latency 15.581 ms 2024-03-09T00:04:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.382+0000 7f25c4268640 1 -- 172.21.15.22:0/3872749892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 msgr2=0x7f25bc10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.382+0000 7f25c4268640 1 --2- 172.21.15.22:0/3872749892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 0x7f25bc10e180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009ca0 tx=0x7f25b402c180 comp rx=0 tx=0).stop 2024-03-09T00:04:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.382+0000 7f25c4268640 1 -- 172.21.15.22:0/3872749892 shutdown_connections 2024-03-09T00:04:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.382+0000 7f25c4268640 1 --2- 172.21.15.22:0/3872749892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 0x7f25bc10e180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.382+0000 7f25c4268640 1 --2- 172.21.15.22:0/3872749892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 0x7f25bc068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.382+0000 7f25c4268640 1 --2- 172.21.15.22:0/3872749892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25bc1073e0 0x7f25bc1077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.382+0000 7f25c4268640 1 -- 172.21.15.22:0/3872749892 >> 172.21.15.22:0/3872749892 conn(0x7f25bc077040 msgr2=0x7f25bc077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 -- 172.21.15.22:0/3872749892 shutdown_connections 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 -- 172.21.15.22:0/3872749892 wait complete. 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 Processor -- start 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 -- start start 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 0x7f25bc19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 0x7f25bc19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25bc1073e0 0x7f25bc1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc111670 con 0x7f25bc068970 2024-03-09T00:04:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1114f0 con 0x7f25bc1073e0 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c4268640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1117f0 con 0x7f25bc069310 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c2004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 0x7f25bc19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c2004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 0x7f25bc19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39670/0 (socket says 172.21.15.22:39670) 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c2004640 1 -- 172.21.15.22:0/1794770160 learned_addr learned my addr 172.21.15.22:0/1794770160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c2805640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25bc1073e0 0x7f25bc1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.383+0000 7f25c1803640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 0x7f25bc19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c2004640 1 -- 172.21.15.22:0/1794770160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 msgr2=0x7f25bc19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c2004640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 0x7f25bc19dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c2004640 1 -- 172.21.15.22:0/1794770160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25bc1073e0 msgr2=0x7f25bc1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c2004640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25bc1073e0 0x7f25bc1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c2004640 1 -- 172.21.15.22:0/1794770160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc198050 con 0x7f25bc068970 2024-03-09T00:04:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c2004640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 0x7f25bc19d720 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f25b800e9f0 tx=0x7f25b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25b37fe640 1 -- 172.21.15.22:0/1794770160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800cd80 con 0x7f25bc068970 2024-03-09T00:04:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25b37fe640 1 -- 172.21.15.22:0/1794770160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b8005500 con 0x7f25bc068970 2024-03-09T00:04:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc198340 con 0x7f25bc068970 2024-03-09T00:04:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25b37fe640 1 -- 172.21.15.22:0/1794770160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8010690 con 0x7f25bc068970 2024-03-09T00:04:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.384+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc1a45e0 con 0x7f25bc068970 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.386+0000 7f25b37fe640 1 -- 172.21.15.22:0/1794770160 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f25b80046c0 con 0x7f25bc068970 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.387+0000 7f25c6e47740 1 Processor -- start 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.387+0000 7f25c6e47740 1 -- start start 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.387+0000 7f25b37fe640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f25a80a2a10 0x7f25a80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.387+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe8264a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe8264fd0 0x564fe8265450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe826e550 0x564fe8270940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c6e47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe81c95f0 con 0x564fe826e550 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c6e47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe8172930 con 0x564fe8265d40 2024-03-09T00:04:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c6e47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe81c5690 con 0x564fe8264fd0 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2805640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f25a80a2a10 0x7f25a80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe8264a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c1803640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe8264fd0 0x564fe8265450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25b37fe640 1 -- 172.21.15.22:0/1794770160 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f25b80cf6a0 con 0x7f25bc068970 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c1803640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe8264fd0 0x564fe8265450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51740/0 (socket says 172.21.15.22:51740) 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe8264a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:43602/0 (socket says 172.21.15.22:43602) 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c1803640 1 -- 172.21.15.22:0/812236940 learned_addr learned my addr 172.21.15.22:0/812236940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2805640 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe826e550 0x564fe8270940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 -- 172.21.15.22:0/812236940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe8264fd0 msgr2=0x564fe8265450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe8264fd0 0x564fe8265450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 -- 172.21.15.22:0/812236940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe826e550 msgr2=0x564fe8270940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe826e550 0x564fe8270940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 -- 172.21.15.22:0/812236940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fe7ede210 con 0x564fe8265d40 2024-03-09T00:04:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c1803640 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe8264fd0 0x564fe8265450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2004640 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe8264a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25b80cc9f0 tx=0x7f25b8007850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d0bc39e6a8313f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25c2805640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f25a80a2a10 0x7f25a80a4ed0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25b4004af0 tx=0x7f25b4032340 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.388+0000 7f25b0bfe640 1 -- 172.21.15.22:0/812236940 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b809b440 con 0x564fe8265d40 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25b0bfe640 1 -- 172.21.15.22:0/812236940 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b809c000 con 0x564fe8265d40 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25b0bfe640 1 -- 172.21.15.22:0/812236940 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b49c0 con 0x564fe8265d40 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 -- 172.21.15.22:0/812236940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 msgr2=0x564fe8264a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe8264a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25b80cc9f0 tx=0x7f25b8007850 comp rx=0 tx=0).stop 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 -- 172.21.15.22:0/812236940 shutdown_connections 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe826e550 0x564fe8270940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe8264fd0 0x564fe8265450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 --2- 172.21.15.22:0/812236940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe8264a90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 -- 172.21.15.22:0/812236940 >> 172.21.15.22:0/812236940 conn(0x564fe822c5b0 msgr2=0x564fe825fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 -- 172.21.15.22:0/812236940 shutdown_connections 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.389+0000 7f25c6e47740 1 -- 172.21.15.22:0/812236940 wait complete. 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 -- start start 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 Processor -- start 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8264fd0 0x564fe8278680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8265d40 0x564fe8278bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe8272a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe8271280 con 0x564fe8265d40 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe8185d90 con 0x564fe8264fd0 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fe8271400 con 0x564fe826e550 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c6e47740 10 client.0 _ll_register_callbacks cb 0x564fe7edda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8265d40 0x564fe8278bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8265d40 0x564fe8278bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39696/0 (socket says 172.21.15.22:39696) 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 -- 172.21.15.22:0/459080342 learned_addr learned my addr 172.21.15.22:0/459080342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c2004640 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8264fd0 0x564fe8278680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c2805640 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe8272a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 -- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 msgr2=0x564fe8272a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe8272a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 -- 172.21.15.22:0/459080342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8264fd0 msgr2=0x564fe8278680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8264fd0 0x564fe8278680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.390+0000 7f25c1803640 1 -- 172.21.15.22:0/459080342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fe7faa620 con 0x564fe8265d40 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c1803640 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8265d40 0x564fe8278bc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f25ac012700 tx=0x7f25ac012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0a7168510fb063 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25a5ffb640 1 -- 172.21.15.22:0/459080342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac004260 con 0x564fe8265d40 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25a5ffb640 1 -- 172.21.15.22:0/459080342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f25ac004400 con 0x564fe8265d40 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25b809c000 con 0x564fe8265d40 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25a5ffb640 1 -- 172.21.15.22:0/459080342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ac0119a0 con 0x564fe8265d40 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 10 client.18651 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564fe8281960 con 0x564fe8265d40 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25a5ffb640 1 -- 172.21.15.22:0/459080342 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b809c000 con 0x564fe8265d40 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25a5ffb640 1 -- 172.21.15.22:0/459080342 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f25ac00f460 con 0x564fe8265d40 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 10 client.18651 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 10 client.18651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 20 client.18651 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe8282e20 0x564fe8285210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 4 client.18651 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.391+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fe81d6b30 con 0x564fe8282e20 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.392+0000 7f25c2004640 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe8282e20 0x564fe8285210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.392+0000 7f25a5ffb640 10 client.18651 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.392+0000 7f25c2004640 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe8282e20 0x564fe8285210 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d213b77550979071 server_cookie=35aa245b93ca0f4b in_seq=0 out_seq=0 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25a5ffb640 1 -- 172.21.15.22:0/459080342 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564fe81d6b30 con 0x564fe8282e20 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25a5ffb640 10 client.18651 handle_command_reply: tid=0 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 client.18651 shutdown 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8265d40 msgr2=0x564fe8278bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8265d40 0x564fe8278bc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f25ac012700 tx=0x7f25ac012bc0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 shutdown_connections 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe8282e20 0x564fe8285210 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe8272a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8265d40 0x564fe8278bc0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25a5ffb640 0 client.18651 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 --2- 172.21.15.22:0/459080342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8264fd0 0x564fe8278680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 >> 172.21.15.22:0/459080342 conn(0x564fe822c5b0 msgr2=0x564fe8261a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 shutdown_connections 2024-03-09T00:04:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 1 -- 172.21.15.22:0/459080342 wait complete. 2024-03-09T00:04:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.393+0000 7f25c6e47740 20 client.18651 trim_cache size 0 max 16384 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 Processor -- start 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 -- start start 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe8272a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe826cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe826d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1ae920 con 0x564fe8264fd0 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1ac4c0 con 0x564fe8265d40 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.408+0000 7f25c6e47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1aa580 con 0x564fe826e550 2024-03-09T00:04:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe8272a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe8272a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39706/0 (socket says 172.21.15.22:39706) 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 -- 172.21.15.22:0/374226576 learned_addr learned my addr 172.21.15.22:0/374226576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2805640 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe826d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c1803640 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe826cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 -- 172.21.15.22:0/374226576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 msgr2=0x564fe826d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe826d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 -- 172.21.15.22:0/374226576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 msgr2=0x564fe826cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe826cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 -- 172.21.15.22:0/374226576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fe82642a0 con 0x564fe8264fd0 2024-03-09T00:04:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c1803640 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe826cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25c2004640 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe8272a30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f25b8097d60 tx=0x7f25b801bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8438d1334db90a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25a7fff640 1 -- 172.21.15.22:0/374226576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b4ba0 con 0x564fe8264fd0 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25a7fff640 1 -- 172.21.15.22:0/374226576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b809c000 con 0x564fe8264fd0 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.409+0000 7f25a7fff640 1 -- 172.21.15.22:0/374226576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b809b260 con 0x564fe8264fd0 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 -- 172.21.15.22:0/374226576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 msgr2=0x564fe8272a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe8272a30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f25b8097d60 tx=0x7f25b801bb30 comp rx=0 tx=0).stop 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 -- 172.21.15.22:0/374226576 shutdown_connections 2024-03-09T00:04:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe826d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe826cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 --2- 172.21.15.22:0/374226576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe8272a30 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 -- 172.21.15.22:0/374226576 >> 172.21.15.22:0/374226576 conn(0x564fe822c5b0 msgr2=0x564fe8272320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 -- 172.21.15.22:0/374226576 shutdown_connections 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 -- 172.21.15.22:0/374226576 wait complete. 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 -- start start 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 Processor -- start 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe82d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe82d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.410+0000 7f25c6e47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe82cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c6e47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1aab80 con 0x564fe8264fd0 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c6e47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1abe60 con 0x564fe8265d40 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c6e47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc1ab1a0 con 0x564fe826e550 2024-03-09T00:04:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c6e47740 10 client.0 _ll_register_callbacks cb 0x7f25ac009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe82d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe82d2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39720/0 (socket says 172.21.15.22:39720) 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 -- 172.21.15.22:0/3442403112 learned_addr learned my addr 172.21.15.22:0/3442403112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2805640 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe82cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c1803640 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe82d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 -- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 msgr2=0x564fe82cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe82cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 -- 172.21.15.22:0/3442403112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 msgr2=0x564fe82d34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe82d34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 -- 172.21.15.22:0/3442403112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fe82cdab0 con 0x564fe8264fd0 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c2004640 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe82d2fb0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f25b80982d0 tx=0x7f25b809b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13194b3cf6d04be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25a5ffb640 1 -- 172.21.15.22:0/3442403112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b3020 con 0x564fe8264fd0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25a5ffb640 1 -- 172.21.15.22:0/3442403112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f25b8099c00 con 0x564fe8264fd0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25b809c000 con 0x564fe8264fd0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25a5ffb640 1 -- 172.21.15.22:0/3442403112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b80b49c0 con 0x564fe8264fd0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c6e47740 10 client.18657 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.411+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564fe82da730 con 0x564fe8264fd0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25a5ffb640 1 -- 172.21.15.22:0/3442403112 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b809c000 con 0x564fe8264fd0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25a5ffb640 1 -- 172.21.15.22:0/3442403112 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f25b80a5020 con 0x564fe8264fd0 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c6e47740 10 client.18657 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c6e47740 10 client.18657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c6e47740 20 client.18657 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c6e47740 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe82da9c0 0x564fe82dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c6e47740 4 client.18657 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:04:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564fe81d0e80 con 0x564fe82da9c0 2024-03-09T00:04:27.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c1803640 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe82da9c0 0x564fe82dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:27.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25a5ffb640 10 client.18657 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:27.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.412+0000 7f25c1803640 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe82da9c0 0x564fe82dcdb0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ecee760083a633 server_cookie=2453156483f7451c in_seq=0 out_seq=0 2024-03-09T00:04:27.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1016445 9.99 MB/sec execute 395 sec latency 153.484 ms 2024-03-09T00:04:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.750+0000 7f25a5ffb640 1 -- 172.21.15.22:0/3442403112 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564fe81d0e80 con 0x564fe82da9c0 2024-03-09T00:04:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.750+0000 7f25a5ffb640 10 client.18657 handle_command_reply: tid=0 2024-03-09T00:04:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.750+0000 7f25c6e47740 1 client.18657 shutdown 2024-03-09T00:04:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.750+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 msgr2=0x564fe82d2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.750+0000 7f25c6e47740 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe82d2fb0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f25b80982d0 tx=0x7f25b809b360 comp rx=0 tx=0).stop 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.750+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 shutdown_connections 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.750+0000 7f25c6e47740 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564fe82da9c0 0x564fe82dcdb0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c6e47740 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fe826e550 0x564fe82cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25a5ffb640 0 client.18657 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c6e47740 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564fe8265d40 0x564fe82d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c6e47740 1 --2- 172.21.15.22:0/3442403112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fe8264fd0 0x564fe82d2fb0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 >> 172.21.15.22:0/3442403112 conn(0x564fe822c5b0 msgr2=0x564fe825e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 shutdown_connections 2024-03-09T00:04:27.747 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c6e47740 1 -- 172.21.15.22:0/3442403112 wait complete. 2024-03-09T00:04:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c6e47740 20 client.18657 trim_cache size 0 max 16384 2024-03-09T00:04:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:04:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:04:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:04:27.748 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f25a80a2a10 msgr2=0x7f25a80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c4268640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f25a80a2a10 0x7f25a80a4ed0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25b4004af0 tx=0x7f25b4032340 comp rx=0 tx=0).stop 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 msgr2=0x7f25bc19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.751+0000 7f25c4268640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 0x7f25bc19d720 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f25b800e9f0 tx=0x7f25b800edf0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 shutdown_connections 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f25a80a2a10 0x7f25a80a4ed0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25bc1073e0 0x7f25bc1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc069310 0x7f25bc19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 --2- 172.21.15.22:0/1794770160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25bc068970 0x7f25bc19d720 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 >> 172.21.15.22:0/1794770160 conn(0x7f25bc077040 msgr2=0x7f25bc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 shutdown_connections 2024-03-09T00:04:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:27.752+0000 7f25c4268640 1 -- 172.21.15.22:0/1794770160 wait complete. 2024-03-09T00:04:28.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:28 smithi062 ceph-mon[36129]: pgmap v314: 129 pgs: 129 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-03-09T00:04:28.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:28 smithi192 ceph-mon[35995]: pgmap v314: 129 pgs: 129 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-03-09T00:04:28.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:28 smithi022 ceph-mon[31894]: pgmap v314: 129 pgs: 129 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-03-09T00:04:28.727 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1018426 9.98 MB/sec execute 396 sec latency 20.628 ms 2024-03-09T00:04:29.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1020023 9.98 MB/sec execute 397 sec latency 18.530 ms 2024-03-09T00:04:30.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:30 smithi062 ceph-mon[36129]: pgmap v315: 129 pgs: 129 active+clean; 87 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 363 op/s 2024-03-09T00:04:30.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:30 smithi192 ceph-mon[35995]: pgmap v315: 129 pgs: 129 active+clean; 87 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 363 op/s 2024-03-09T00:04:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:30 smithi022 ceph-mon[31894]: pgmap v315: 129 pgs: 129 active+clean; 87 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 363 op/s 2024-03-09T00:04:30.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1020410 9.96 MB/sec execute 398 sec latency 216.739 ms 2024-03-09T00:04:31.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1022538 9.97 MB/sec execute 399 sec latency 70.090 ms 2024-03-09T00:04:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:32 smithi062 ceph-mon[36129]: pgmap v316: 129 pgs: 129 active+clean; 84 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-03-09T00:04:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:32 smithi192 ceph-mon[35995]: pgmap v316: 129 pgs: 129 active+clean; 84 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-03-09T00:04:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:32 smithi022 ceph-mon[31894]: pgmap v316: 129 pgs: 129 active+clean; 84 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-03-09T00:04:32.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1023862 9.96 MB/sec execute 400 sec latency 22.795 ms 2024-03-09T00:04:33.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1026116 9.96 MB/sec execute 401 sec latency 24.068 ms 2024-03-09T00:04:34.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:34 smithi062 ceph-mon[36129]: pgmap v317: 129 pgs: 129 active+clean; 84 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 289 op/s 2024-03-09T00:04:34.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:34 smithi192 ceph-mon[35995]: pgmap v317: 129 pgs: 129 active+clean; 84 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 289 op/s 2024-03-09T00:04:34.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:34 smithi022 ceph-mon[31894]: pgmap v317: 129 pgs: 129 active+clean; 84 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 289 op/s 2024-03-09T00:04:34.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1027372 9.96 MB/sec execute 402 sec latency 41.209 ms 2024-03-09T00:04:35.640 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:35.641 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:35.641 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100711]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:35.641 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100711]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:35.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1029178 9.95 MB/sec execute 403 sec latency 56.680 ms 2024-03-09T00:04:35.984 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:35.984 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:35.984 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100714]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:35.984 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:04:35 smithi022 sudo[100714]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:36.610 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72063]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:36.610 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72063]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:36.610 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72063]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:36.610 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72063]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:36.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1030611 9.95 MB/sec execute 404 sec latency 49.489 ms 2024-03-09T00:04:36.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:36 smithi022 ceph-mon[31894]: pgmap v318: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 380 op/s 2024-03-09T00:04:36.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:36 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:37.003 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:36 smithi192 ceph-mon[35995]: pgmap v318: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 380 op/s 2024-03-09T00:04:37.003 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:36 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:37.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:36 smithi062 ceph-mon[36129]: pgmap v318: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 380 op/s 2024-03-09T00:04:37.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:36 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:37.110 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:37.111 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:37.111 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72066]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:37.111 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:04:36 smithi062 sudo[72066]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:37.315 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72488]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:37.315 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72488]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:37.315 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72488]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:37.315 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72488]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:37.630 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72491]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:37.630 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72491]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:37.630 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72491]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:37.630 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:04:37 smithi192 sudo[72491]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:37.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1032492 9.94 MB/sec execute 405 sec latency 21.922 ms 2024-03-09T00:04:37.937 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:37.991 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:37.992 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100717]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:37.992 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100717]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:37.992 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:37.992 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:37 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:38.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:38.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:37 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:38.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:38.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:37 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:38.234 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100720]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:38.234 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100720]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:38.234 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100720]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:38.234 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:04:37 smithi022 sudo[100720]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:38.722 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72071]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:38.722 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72071]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:38.723 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72071]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:38.723 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72071]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:38.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1034167 9.95 MB/sec execute 406 sec latency 15.264 ms 2024-03-09T00:04:38.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:38 smithi022 ceph-mon[31894]: pgmap v319: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-03-09T00:04:39.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:38 smithi062 ceph-mon[36129]: pgmap v319: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-03-09T00:04:39.110 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:39.110 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:39.110 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72074]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:39.111 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:04:38 smithi062 sudo[72074]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:39.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:38 smithi192 ceph-mon[35995]: pgmap v319: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-03-09T00:04:39.554 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72494]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:39.554 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72494]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:39.554 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72494]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:39.554 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72494]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:39.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1035869 9.94 MB/sec execute 407 sec latency 35.232 ms 2024-03-09T00:04:39.880 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72497]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:39.880 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72497]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:39.880 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72497]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:39.880 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:04:39 smithi192 sudo[72497]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:40.155 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:39 smithi022 sudo[100723]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:40.155 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:39 smithi022 sudo[100723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:40.155 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:39 smithi022 sudo[100723]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:40.155 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:39 smithi022 sudo[100723]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:40.484 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:40 smithi022 sudo[100726]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:40.484 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:40 smithi022 sudo[100726]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:40.484 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:40 smithi022 sudo[100726]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:40.484 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:04:40 smithi022 sudo[100726]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:40.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1037633 9.95 MB/sec execute 408 sec latency 31.822 ms 2024-03-09T00:04:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/3249804839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94101ee0 msgr2=0x7f6b94105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/3249804839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94101ee0 0x7f6b94105f20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009900 tx=0x7f6b8802bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/3249804839 shutdown_connections 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/3249804839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94106600 0x7f6b9410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/3249804839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94101ee0 0x7f6b94105f20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/3249804839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b94101530 0x7f6b94101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/3249804839 >> 172.21.15.22:0/3249804839 conn(0x7f6b940fcdd0 msgr2=0x7f6b940ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/3249804839 shutdown_connections 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/3249804839 wait complete. 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 Processor -- start 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.841+0000 7f6b9c5d4640 1 -- start start 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b94101530 0x7f6b9419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94106600 0x7f6b9419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94197990 0x7f6b94197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b94111670 con 0x7f6b94197990 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941114f0 con 0x7f6b94106600 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941117f0 con 0x7f6b94101530 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94197990 0x7f6b94197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9a370640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b94101530 0x7f6b9419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94197990 0x7f6b94197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47616/0 (socket says 172.21.15.22:47616) 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 -- 172.21.15.22:0/1905934391 learned_addr learned my addr 172.21.15.22:0/1905934391 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94106600 0x7f6b9419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 -- 172.21.15.22:0/1905934391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b94101530 msgr2=0x7f6b9419d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b94101530 0x7f6b9419d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 -- 172.21.15.22:0/1905934391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94106600 msgr2=0x7f6b9419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94106600 0x7f6b9419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 -- 172.21.15.22:0/1905934391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b941984b0 con 0x7f6b94197990 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9ab71640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94197990 0x7f6b94197e40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400eb10 tx=0x7f6b8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b937fe640 1 -- 172.21.15.22:0/1905934391 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84004b20 con 0x7f6b94197990 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b937fe640 1 -- 172.21.15.22:0/1905934391 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b84004cc0 con 0x7f6b94197990 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b941987a0 con 0x7f6b94197990 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b937fe640 1 -- 172.21.15.22:0/1905934391 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84010760 con 0x7f6b94197990 2024-03-09T00:04:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.842+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b941a49a0 con 0x7f6b94197990 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.845+0000 7f6b937fe640 1 -- 172.21.15.22:0/1905934391 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6b84005020 con 0x7f6b94197990 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.845+0000 7f6b9f1b3740 1 Processor -- start 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9f1b3740 1 -- start start 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a78fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a790450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7a6f45f0 con 0x556e7a790d40 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7a69d930 con 0x556e7a78ffd0 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7a6f0690 con 0x556e7a79d960 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a78fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a790450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a790450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58718/0 (socket says 172.21.15.22:58718) 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 -- 172.21.15.22:0/274684857 learned_addr learned my addr 172.21.15.22:0/274684857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a78fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47624/0 (socket says 172.21.15.22:47624) 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b937fe640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b7c0a2a10 0x7f6b7c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b937fe640 1 -- 172.21.15.22:0/1905934391 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6b840cf770 con 0x7f6b94197990 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 -- 172.21.15.22:0/274684857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 msgr2=0x556e7a79fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 -- 172.21.15.22:0/274684857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 msgr2=0x556e7a78fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9ab71640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b7c0a2a10 0x7f6b7c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a78fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.846+0000 7f6b9a370640 1 -- 172.21.15.22:0/274684857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7a409210 con 0x556e7a78ffd0 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.847+0000 7f6b9a370640 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a790450 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bcb0 tx=0x7f6b8c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb5f044a48bd9a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.847+0000 7f6b9ab71640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b7c0a2a10 0x7f6b7c0a4ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b840ccb70 tx=0x7f6b84005b40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.847+0000 7f6b90bfe640 1 -- 172.21.15.22:0/274684857 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c01c480 con 0x556e7a78ffd0 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.847+0000 7f6b90bfe640 1 -- 172.21.15.22:0/274684857 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c01cb00 con 0x556e7a78ffd0 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.847+0000 7f6b90bfe640 1 -- 172.21.15.22:0/274684857 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c011d20 con 0x556e7a78ffd0 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/274684857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 msgr2=0x556e7a790450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a790450 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bcb0 tx=0x7f6b8c0107a0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/274684857 shutdown_connections 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a790450 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/274684857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a78fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/274684857 >> 172.21.15.22:0/274684857 conn(0x556e7a7575b0 msgr2=0x556e7a78acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/274684857 shutdown_connections 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/274684857 wait complete. 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 -- start start 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.848+0000 7f6b9f1b3740 1 Processor -- start 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a790d40 0x556e7a79afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7a7a0690 con 0x556e7a78ffd0 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7a6b0d90 con 0x556e7a790d40 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7a7a0810 con 0x556e7a79d960 2024-03-09T00:04:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47632/0 (socket says 172.21.15.22:47632) 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3501367694 learned_addr learned my addr 172.21.15.22:0/3501367694 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 10 client.0 _ll_register_callbacks cb 0x556e7a408a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9a370640 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a790d40 0x556e7a79afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 msgr2=0x556e7a79b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3501367694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a790d40 msgr2=0x556e7a79afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a790d40 0x556e7a79afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3501367694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e7a4d5620 con 0x556e7a78ffd0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79aa60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6b8802cfd0 tx=0x7f6b88005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559aa0605ec51bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b79ffb640 1 -- 172.21.15.22:0/3501367694 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88003a00 con 0x556e7a78ffd0 2024-03-09T00:04:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b79ffb640 1 -- 172.21.15.22:0/3501367694 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6b88003ba0 con 0x556e7a78ffd0 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.849+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b8c01cb00 con 0x556e7a78ffd0 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b79ffb640 1 -- 172.21.15.22:0/3501367694 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b88055050 con 0x556e7a78ffd0 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 10 client.18669 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556e7a7aca50 con 0x556e7a78ffd0 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b79ffb640 1 -- 172.21.15.22:0/3501367694 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c01cb00 con 0x556e7a78ffd0 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b79ffb640 1 -- 172.21.15.22:0/3501367694 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b88042440 con 0x556e7a78ffd0 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 10 client.18669 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 10 client.18669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 20 client.18669 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a7b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 4 client.18669 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e7a701b30 con 0x556e7a7adf10 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.850+0000 7f6b9a370640 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a7b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b79ffb640 10 client.18669 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9a370640 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a7b0300 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddad1621a696208 server_cookie=13ce44eac6e482e6 in_seq=0 out_seq=0 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b79ffb640 1 -- 172.21.15.22:0/3501367694 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e7a701b30 con 0x556e7a7adf10 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b79ffb640 10 client.18669 handle_command_reply: tid=0 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 client.18669 shutdown 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 msgr2=0x556e7a79aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79aa60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6b8802cfd0 tx=0x7f6b88005fd0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 shutdown_connections 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a7b0300 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a79b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b79ffb640 0 client.18669 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a790d40 0x556e7a79afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3501367694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79aa60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 >> 172.21.15.22:0/3501367694 conn(0x556e7a7575b0 msgr2=0x556e7a78ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 shutdown_connections 2024-03-09T00:04:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.851+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3501367694 wait complete. 2024-03-09T00:04:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.852+0000 7f6b9f1b3740 20 client.18669 trim_cache size 0 max 16384 2024-03-09T00:04:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 Processor -- start 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 -- start start 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a790d40 0x556e7a7a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a79d960 0x556e7a7a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941ae990 con 0x556e7a78ffd0 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941ac530 con 0x556e7a79d960 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941aa5f0 con 0x556e7a790d40 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47644/0 (socket says 172.21.15.22:47644) 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3276478553 learned_addr learned my addr 172.21.15.22:0/3276478553 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b9a370640 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a790d40 0x556e7a7a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a79d960 0x556e7a7a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3276478553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a790d40 msgr2=0x556e7a7a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a790d40 0x556e7a7a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3276478553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a79d960 msgr2=0x556e7a7a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a79d960 0x556e7a7a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 -- 172.21.15.22:0/3276478553 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7a78f2a0 con 0x556e7a78ffd0 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79d190 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009900 tx=0x7f6b8802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286e969947087fff server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.866+0000 7f6b7bfff640 1 -- 172.21.15.22:0/3276478553 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88042a00 con 0x556e7a78ffd0 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b7bfff640 1 -- 172.21.15.22:0/3276478553 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b88042ba0 con 0x556e7a78ffd0 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b7bfff640 1 -- 172.21.15.22:0/3276478553 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b880406b0 con 0x556e7a78ffd0 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3276478553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 msgr2=0x556e7a79d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79d190 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009900 tx=0x7f6b8802bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3276478553 shutdown_connections 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a79d960 0x556e7a7a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a790d40 0x556e7a7a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/3276478553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a78ffd0 0x556e7a79d190 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3276478553 >> 172.21.15.22:0/3276478553 conn(0x556e7a7575b0 msgr2=0x556e7a799330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3276478553 shutdown_connections 2024-03-09T00:04:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.867+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/3276478553 wait complete. 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 -- start start 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 Processor -- start 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a797860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a797da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a7982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941aabf0 con 0x556e7a790d40 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941abed0 con 0x556e7a78ffd0 2024-03-09T00:04:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941ab210 con 0x556e7a79d960 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9f1b3740 10 client.0 _ll_register_callbacks cb 0x7f6b8c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a797da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a797860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a797da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47656/0 (socket says 172.21.15.22:47656) 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b99b6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a797860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58736/0 (socket says 172.21.15.22:58736) 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 -- 172.21.15.22:0/940308554 learned_addr learned my addr 172.21.15.22:0/940308554 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a7982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 -- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 msgr2=0x556e7a7982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a7982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 -- 172.21.15.22:0/940308554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 msgr2=0x556e7a797860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a797860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 -- 172.21.15.22:0/940308554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e7a804580 con 0x556e7a790d40 2024-03-09T00:04:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a797860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b9a370640 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a797da0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bbd0 tx=0x7f6b8c011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c57d225a1ec8fb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.868+0000 7f6b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b79ffb640 1 -- 172.21.15.22:0/940308554 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c02c070 con 0x556e7a790d40 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b79ffb640 1 -- 172.21.15.22:0/940308554 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6b8c011a10 con 0x556e7a790d40 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b88042ba0 con 0x556e7a790d40 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b79ffb640 1 -- 172.21.15.22:0/940308554 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b8c038050 con 0x556e7a790d40 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 10 client.18675 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556e7a805850 con 0x556e7a790d40 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b79ffb640 1 -- 172.21.15.22:0/940308554 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b88042ba0 con 0x556e7a790d40 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b79ffb640 1 -- 172.21.15.22:0/940308554 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b8c028ce0 con 0x556e7a790d40 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 10 client.18675 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 10 client.18675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 20 client.18675 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a807d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 4 client.18675 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:04:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e7a6fbe80 con 0x556e7a7adf10 2024-03-09T00:04:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.869+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a807d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.870+0000 7f6b79ffb640 10 client.18675 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:40.870+0000 7f6b99b6f640 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a807d00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3cce59fb2d6d56 server_cookie=ba41e0ed8d2aaee in_seq=0 out_seq=0 2024-03-09T00:04:40.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:40 smithi022 ceph-mon[31894]: pgmap v320: 129 pgs: 129 active+clean; 94 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-03-09T00:04:41.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:40 smithi062 ceph-mon[36129]: pgmap v320: 129 pgs: 129 active+clean; 94 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-03-09T00:04:41.110 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:40 smithi062 sudo[72079]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:41.110 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:40 smithi062 sudo[72079]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:41.111 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:40 smithi062 sudo[72079]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:41.111 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:40 smithi062 sudo[72079]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:41.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:40 smithi192 ceph-mon[35995]: pgmap v320: 129 pgs: 129 active+clean; 94 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-03-09T00:04:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b79ffb640 1 -- 172.21.15.22:0/940308554 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e7a6fbe80 con 0x556e7a7adf10 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b79ffb640 10 client.18675 handle_command_reply: tid=0 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 client.18675 shutdown 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 msgr2=0x556e7a797da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a797da0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bbd0 tx=0x7f6b8c011840 comp rx=0 tx=0).stop 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 shutdown_connections 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556e7a7adf10 0x556e7a807d00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e7a79d960 0x556e7a7982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556e7a790d40 0x556e7a797da0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 --2- 172.21.15.22:0/940308554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556e7a78ffd0 0x556e7a797860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 >> 172.21.15.22:0/940308554 conn(0x556e7a7575b0 msgr2=0x556e7a7895b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b79ffb640 0 client.18675 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 shutdown_connections 2024-03-09T00:04:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 1 -- 172.21.15.22:0/940308554 wait complete. 2024-03-09T00:04:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.320+0000 7f6b9f1b3740 20 client.18675 trim_cache size 0 max 16384 2024-03-09T00:04:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:04:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:04:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:04:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b7c0a2a10 msgr2=0x7f6b7c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b7c0a2a10 0x7f6b7c0a4ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b840ccb70 tx=0x7f6b84005b40 comp rx=0 tx=0).stop 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94197990 msgr2=0x7f6b94197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94197990 0x7f6b94197e40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400eb10 tx=0x7f6b8400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 shutdown_connections 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6b7c0a2a10 0x7f6b7c0a4ed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6b94197990 0x7f6b94197e40 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b94106600 0x7f6b9419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 --2- 172.21.15.22:0/1905934391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b94101530 0x7f6b9419d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 >> 172.21.15.22:0/1905934391 conn(0x7f6b940fcdd0 msgr2=0x7f6b940ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 shutdown_connections 2024-03-09T00:04:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:41.321+0000 7f6b9c5d4640 1 -- 172.21.15.22:0/1905934391 wait complete. 2024-03-09T00:04:41.610 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:41 smithi062 sudo[72083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:41.610 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:41 smithi062 sudo[72083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:41.610 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:41 smithi062 sudo[72083]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:41.610 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:04:41 smithi062 sudo[72083]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:41.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1038425 9.93 MB/sec execute 409 sec latency 340.543 ms 2024-03-09T00:04:41.880 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72500]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:41.880 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72500]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:41.880 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72500]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:41.880 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72500]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:41.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:04:41.975 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:04:42.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.097+0000 7f8a976ab640 1 -- 172.21.15.22:0/2017669409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90100480 msgr2=0x7f8a90100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.097+0000 7f8a976ab640 1 --2- 172.21.15.22:0/2017669409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90100480 0x7f8a90100860 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800bd70 tx=0x7f8a8802dc60 comp rx=0 tx=0).stop 2024-03-09T00:04:42.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.098+0000 7f8a976ab640 1 -- 172.21.15.22:0/2017669409 shutdown_connections 2024-03-09T00:04:42.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.098+0000 7f8a976ab640 1 --2- 172.21.15.22:0/2017669409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90105550 0x7f8a9010d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.098+0000 7f8a976ab640 1 --2- 172.21.15.22:0/2017669409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a90100e30 0x7f8a90104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.098+0000 7f8a976ab640 1 --2- 172.21.15.22:0/2017669409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90100480 0x7f8a90100860 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.098+0000 7f8a976ab640 1 -- 172.21.15.22:0/2017669409 >> 172.21.15.22:0/2017669409 conn(0x7f8a90077350 msgr2=0x7f8a90077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.101+0000 7f8a976ab640 1 -- 172.21.15.22:0/2017669409 shutdown_connections 2024-03-09T00:04:42.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.101+0000 7f8a976ab640 1 -- 172.21.15.22:0/2017669409 wait complete. 2024-03-09T00:04:42.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.102+0000 7f8a976ab640 1 Processor -- start 2024-03-09T00:04:42.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.102+0000 7f8a976ab640 1 -- start start 2024-03-09T00:04:42.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a976ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90100480 0x7f8a90115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a976ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90105550 0x7f8a90115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a976ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a90110ab0 0x7f8a90110f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a976ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9010f520 con 0x7f8a90105550 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a976ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9010f3a0 con 0x7f8a90100480 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a976ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9010f6a0 con 0x7f8a90110ab0 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a95447640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90100480 0x7f8a90115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a95447640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90100480 0x7f8a90115950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58754/0 (socket says 172.21.15.22:58754) 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a95447640 1 -- 172.21.15.22:0/1200118512 learned_addr learned my addr 172.21.15.22:0/1200118512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.103+0000 7f8a94c46640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90105550 0x7f8a90115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.104+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a90110ab0 0x7f8a90110f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.104+0000 7f8a95447640 1 -- 172.21.15.22:0/1200118512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a90110ab0 msgr2=0x7f8a90110f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.104+0000 7f8a95447640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a90110ab0 0x7f8a90110f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.104+0000 7f8a95447640 1 -- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90105550 msgr2=0x7f8a90115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.104+0000 7f8a95447640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90105550 0x7f8a90115e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.104+0000 7f8a95447640 1 -- 172.21.15.22:0/1200118512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90111820 con 0x7f8a90100480 2024-03-09T00:04:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.105+0000 7f8a95447640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90100480 0x7f8a90115950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8a88006010 tx=0x7f8a88003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.105+0000 7f8a867fc640 1 -- 172.21.15.22:0/1200118512 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880368c0 con 0x7f8a90100480 2024-03-09T00:04:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.106+0000 7f8a867fc640 1 -- 172.21.15.22:0/1200118512 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a88044070 con 0x7f8a90100480 2024-03-09T00:04:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.106+0000 7f8a867fc640 1 -- 172.21.15.22:0/1200118512 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8803fae0 con 0x7f8a90100480 2024-03-09T00:04:42.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.107+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a90119650 con 0x7f8a90100480 2024-03-09T00:04:42.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.107+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a90119b20 con 0x7f8a90100480 2024-03-09T00:04:42.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.112+0000 7f8a9a36e740 1 Processor -- start 2024-03-09T00:04:42.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.115+0000 7f8a867fc640 1 -- 172.21.15.22:0/1200118512 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8a88004040 con 0x7f8a90100480 2024-03-09T00:04:42.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.115+0000 7f8a867fc640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a7c0a2b80 0x7f8a7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.116+0000 7f8a867fc640 1 -- 172.21.15.22:0/1200118512 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a880f1220 con 0x7f8a90100480 2024-03-09T00:04:42.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.119+0000 7f8a9a36e740 1 -- start start 2024-03-09T00:04:42.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.119+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e3c760 0x557ae6e3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.119+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 0x557ae6e45720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.119+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e45c60 0x557ae6e48050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.119+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6d9fe00 con 0x557ae6e3c760 2024-03-09T00:04:42.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.119+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6d48b10 con 0x557ae6e45c60 2024-03-09T00:04:42.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.119+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6d9bea0 con 0x557ae6e3d080 2024-03-09T00:04:42.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a7c0a2b80 0x7f8a7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e45c60 0x557ae6e48050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e45c60 0x557ae6e48050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58760/0 (socket says 172.21.15.22:58760) 2024-03-09T00:04:42.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 -- 172.21.15.22:0/1236870275 learned_addr learned my addr 172.21.15.22:0/1236870275 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 -- 172.21.15.22:0/1236870275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 msgr2=0x557ae6e45720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:42.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1236870275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 0x557ae6e45720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 -- 172.21.15.22:0/1236870275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e3c760 msgr2=0x557ae6e3cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1236870275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e3c760 0x557ae6e3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 -- 172.21.15.22:0/1236870275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae6b7e7d0 con 0x557ae6e45c60 2024-03-09T00:04:42.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.120+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1236870275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e45c60 0x557ae6e48050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c014550 tx=0x7f8a8c014a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241b35a5dc51daac server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.121+0000 7f8a7b7fe640 1 -- 172.21.15.22:0/1236870275 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c022070 con 0x557ae6e45c60 2024-03-09T00:04:42.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.121+0000 7f8a7b7fe640 1 -- 172.21.15.22:0/1236870275 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c011c80 con 0x557ae6e45c60 2024-03-09T00:04:42.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.121+0000 7f8a7b7fe640 1 -- 172.21.15.22:0/1236870275 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c012b90 con 0x557ae6e45c60 2024-03-09T00:04:42.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.122+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1236870275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e45c60 msgr2=0x557ae6e48050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.122+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1236870275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e45c60 0x557ae6e48050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c014550 tx=0x7f8a8c014a10 comp rx=0 tx=0).stop 2024-03-09T00:04:42.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.122+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a7c0a2b80 0x7f8a7c0a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c01f000 tx=0x7f8a8c01d7e0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.124+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1236870275 shutdown_connections 2024-03-09T00:04:42.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.124+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1236870275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e45c60 0x557ae6e48050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.124+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1236870275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 0x557ae6e45720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.124+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1236870275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e3c760 0x557ae6e3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.124+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1236870275 >> 172.21.15.22:0/1236870275 conn(0x557ae6e36e90 msgr2=0x557ae6e37270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.128+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1236870275 shutdown_connections 2024-03-09T00:04:42.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.130+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1236870275 wait complete. 2024-03-09T00:04:42.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.130+0000 7f8a9a36e740 1 -- start start 2024-03-09T00:04:42.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.130+0000 7f8a9a36e740 1 Processor -- start 2024-03-09T00:04:42.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.131+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.131+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3d080 0x557ae6e42770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.131+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e42cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.131+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6e48a50 con 0x557ae6e45c60 2024-03-09T00:04:42.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.131+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6d5c0a0 con 0x557ae6e3d080 2024-03-09T00:04:42.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.131+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6e48bd0 con 0x557ae6e3c760 2024-03-09T00:04:42.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.131+0000 7f8a9a36e740 10 client.0 _ll_register_callbacks cb 0x557ae6d5ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:42.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.132+0000 7f8a95447640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.132+0000 7f8a95447640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38222/0 (socket says 172.21.15.22:38222) 2024-03-09T00:04:42.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.132+0000 7f8a95447640 1 -- 172.21.15.22:0/308429576 learned_addr learned my addr 172.21.15.22:0/308429576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.132+0000 7f8a95c48640 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e42cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.132+0000 7f8a95c48640 1 -- 172.21.15.22:0/308429576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 msgr2=0x557ae6e453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.132+0000 7f8a95c48640 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.133+0000 7f8a95c48640 1 -- 172.21.15.22:0/308429576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3d080 msgr2=0x557ae6e42770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.133+0000 7f8a95c48640 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3d080 0x557ae6e42770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.133+0000 7f8a95c48640 1 -- 172.21.15.22:0/308429576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ae6ab43f0 con 0x557ae6e45c60 2024-03-09T00:04:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.133+0000 7f8a95c48640 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e42cb0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009230 tx=0x7f8a8c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c095c88453978a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.133+0000 7f8a78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:42.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.135+0000 7f8a78ff9640 1 -- 172.21.15.22:0/308429576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c022050 con 0x557ae6e45c60 2024-03-09T00:04:42.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.135+0000 7f8a78ff9640 1 -- 172.21.15.22:0/308429576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a8c011780 con 0x557ae6e45c60 2024-03-09T00:04:42.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.135+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a8c011c80 con 0x557ae6e45c60 2024-03-09T00:04:42.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.137+0000 7f8a78ff9640 1 -- 172.21.15.22:0/308429576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a8c009d70 con 0x557ae6e45c60 2024-03-09T00:04:42.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.138+0000 7f8a9a36e740 10 client.18684 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:42.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.138+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557ae6a740f0 con 0x557ae6e45c60 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a78ff9640 1 -- 172.21.15.22:0/308429576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c011c80 con 0x557ae6e45c60 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a78ff9640 1 -- 172.21.15.22:0/308429576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a8c005040 con 0x557ae6e45c60 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a9a36e740 10 client.18684 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a9a36e740 10 client.18684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a9a36e740 20 client.18684 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6e55770 0x557ae6e57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a9a36e740 4 client.18684 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:42.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.142+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ae6dad340 con 0x557ae6e55770 2024-03-09T00:04:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.146+0000 7f8a95447640 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6e55770 0x557ae6e57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.152+0000 7f8a78ff9640 10 client.18684 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:42.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.152+0000 7f8a95447640 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6e55770 0x557ae6e57b60 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650b188be2c62ab6 server_cookie=9fff8dd60576332b in_seq=0 out_seq=0 2024-03-09T00:04:42.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.153+0000 7f8a78ff9640 1 -- 172.21.15.22:0/308429576 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ae6dad340 con 0x557ae6e55770 2024-03-09T00:04:42.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.153+0000 7f8a78ff9640 10 client.18684 handle_command_reply: tid=0 2024-03-09T00:04:42.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.154+0000 7f8a9a36e740 1 client.18684 shutdown 2024-03-09T00:04:42.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 msgr2=0x557ae6e42cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e42cb0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009230 tx=0x7f8a8c009260 comp rx=0 tx=0).stop 2024-03-09T00:04:42.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 shutdown_connections 2024-03-09T00:04:42.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6e55770 0x557ae6e57b60 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e42cb0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3d080 0x557ae6e42770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/308429576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.157+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 >> 172.21.15.22:0/308429576 conn(0x557ae6e36e90 msgr2=0x557ae6e38be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.159+0000 7f8a78ff9640 0 client.18684 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:42.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.160+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 shutdown_connections 2024-03-09T00:04:42.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.165+0000 7f8a9a36e740 1 -- 172.21.15.22:0/308429576 wait complete. 2024-03-09T00:04:42.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.166+0000 7f8a9a36e740 20 client.18684 trim_cache size 0 max 16384 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.200+0000 7f8a9a36e740 1 Processor -- start 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.200+0000 7f8a9a36e740 1 -- start start 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.200+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3c760 0x557ae6e44ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.200+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 0x557ae6e453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.200+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e41040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901b7230 con 0x557ae6e45c60 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901b4dd0 con 0x557ae6e3c760 2024-03-09T00:04:42.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90071de0 con 0x557ae6e3d080 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e41040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e41040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47694/0 (socket says 172.21.15.22:47694) 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 -- 172.21.15.22:0/1466699912 learned_addr learned my addr 172.21.15.22:0/1466699912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 -- 172.21.15.22:0/1466699912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 msgr2=0x557ae6e453e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1466699912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 0x557ae6e453e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 -- 172.21.15.22:0/1466699912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3c760 msgr2=0x557ae6e44ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1466699912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3c760 0x557ae6e44ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.201+0000 7f8a95c48640 1 -- 172.21.15.22:0/1466699912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae6e48930 con 0x557ae6e45c60 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.202+0000 7f8a95c48640 1 --2- 172.21.15.22:0/1466699912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e41040 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c005440 tx=0x7f8a8c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a02e94e823d05b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.202+0000 7f8a7affd640 1 -- 172.21.15.22:0/1466699912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c011780 con 0x557ae6e45c60 2024-03-09T00:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.202+0000 7f8a7affd640 1 -- 172.21.15.22:0/1466699912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c009c10 con 0x557ae6e45c60 2024-03-09T00:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.203+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1466699912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 msgr2=0x557ae6e41040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.203+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1466699912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e41040 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c005440 tx=0x7f8a8c005780 comp rx=0 tx=0).stop 2024-03-09T00:04:42.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.204+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1466699912 shutdown_connections 2024-03-09T00:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.204+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1466699912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e41040 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.204+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1466699912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3d080 0x557ae6e453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.204+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/1466699912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6e3c760 0x557ae6e44ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.204+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1466699912 >> 172.21.15.22:0/1466699912 conn(0x557ae6e36e90 msgr2=0x557ae6e57780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.206+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1466699912 shutdown_connections 2024-03-09T00:04:42.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.206+0000 7f8a9a36e740 1 -- 172.21.15.22:0/1466699912 wait complete. 2024-03-09T00:04:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 -- start start 2024-03-09T00:04:42.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 Processor -- start 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e4b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6ea8a10 0x557ae6eaae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900723e0 con 0x557ae6e45c60 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901b4770 con 0x557ae6ea8a10 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90072a00 con 0x557ae6e3c760 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.207+0000 7f8a9a36e740 10 client.0 _ll_register_callbacks cb 0x7f8a8c020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e4af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38244/0 (socket says 172.21.15.22:38244) 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 -- 172.21.15.22:0/603334732 learned_addr learned my addr 172.21.15.22:0/603334732 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95c48640 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6ea8a10 0x557ae6eaae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a94c46640 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e4b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 -- 172.21.15.22:0/603334732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6ea8a10 msgr2=0x557ae6eaae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6ea8a10 0x557ae6eaae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 -- 172.21.15.22:0/603334732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 msgr2=0x557ae6e4b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e4b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.208+0000 7f8a95447640 1 -- 172.21.15.22:0/603334732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ae6eab410 con 0x557ae6e3c760 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.209+0000 7f8a95447640 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e4af70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a880f1010 tx=0x7f8a8803f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9507f184a4e50236 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.209+0000 7f8a78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.209+0000 7f8a78ff9640 1 -- 172.21.15.22:0/603334732 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880da020 con 0x557ae6e3c760 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.209+0000 7f8a78ff9640 1 -- 172.21.15.22:0/603334732 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a880b9c30 con 0x557ae6e3c760 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.209+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a8c009c10 con 0x557ae6e3c760 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.210+0000 7f8a78ff9640 1 -- 172.21.15.22:0/603334732 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a880cd050 con 0x557ae6e3c760 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.210+0000 7f8a9a36e740 10 client.28123 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:42.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.210+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557ae6eac780 con 0x557ae6e3c760 2024-03-09T00:04:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.210+0000 7f8a78ff9640 1 -- 172.21.15.22:0/603334732 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c009c10 con 0x557ae6e3c760 2024-03-09T00:04:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.210+0000 7f8a78ff9640 1 -- 172.21.15.22:0/603334732 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a880ee790 con 0x557ae6e3c760 2024-03-09T00:04:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.211+0000 7f8a9a36e740 10 client.28123 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:42.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.211+0000 7f8a9a36e740 10 client.28123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.211+0000 7f8a9a36e740 20 client.28123 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.211+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6eacea0 0x557ae6eaf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.211+0000 7f8a9a36e740 4 client.28123 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:04:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.211+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ae6da7690 con 0x557ae6eacea0 2024-03-09T00:04:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.212+0000 7f8a94c46640 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6eacea0 0x557ae6eaf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a94c46640 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6eacea0 0x557ae6eaf290 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801f10cdee9652dd server_cookie=72a8f7969b8b52dc in_seq=0 out_seq=0 2024-03-09T00:04:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a78ff9640 10 client.28123 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a78ff9640 1 -- 172.21.15.22:0/603334732 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557ae6da7690 con 0x557ae6eacea0 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a78ff9640 10 client.28123 handle_command_reply: tid=0 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 client.28123 shutdown 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 msgr2=0x557ae6e4af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e4af70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a880f1010 tx=0x7f8a8803f480 comp rx=0 tx=0).stop 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 shutdown_connections 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557ae6eacea0 0x557ae6eaf290 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557ae6ea8a10 0x557ae6eaae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557ae6e45c60 0x557ae6e4b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 --2- 172.21.15.22:0/603334732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ae6e3c760 0x557ae6e4af70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 >> 172.21.15.22:0/603334732 conn(0x557ae6e36e90 msgr2=0x557ae6e40fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.213+0000 7f8a78ff9640 0 client.28123 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.215+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 shutdown_connections 2024-03-09T00:04:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.215+0000 7f8a9a36e740 1 -- 172.21.15.22:0/603334732 wait complete. 2024-03-09T00:04:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.215+0000 7f8a9a36e740 20 client.28123 trim_cache size 0 max 16384 2024-03-09T00:04:42.210 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:42.211 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:04:42.211 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:04:42.211 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:42.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.216+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a7c0a2b80 msgr2=0x7f8a7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.216+0000 7f8a976ab640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a7c0a2b80 0x7f8a7c0a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c01f000 tx=0x7f8a8c01d7e0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.216+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90100480 msgr2=0x7f8a90115950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.216+0000 7f8a976ab640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90100480 0x7f8a90115950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8a88006010 tx=0x7f8a88003f30 comp rx=0 tx=0).stop 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 shutdown_connections 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a7c0a2b80 0x7f8a7c0a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a90110ab0 0x7f8a90110f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90105550 0x7f8a90115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 --2- 172.21.15.22:0/1200118512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a90100480 0x7f8a90115950 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 >> 172.21.15.22:0/1200118512 conn(0x7f8a90077350 msgr2=0x7f8a900fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 shutdown_connections 2024-03-09T00:04:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.217+0000 7f8a976ab640 1 -- 172.21.15.22:0/1200118512 wait complete. 2024-03-09T00:04:42.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:04:42.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:04:42.250 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100841]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:42.250 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100841]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:42.250 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100841]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:42.251 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100841]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.342+0000 7fea94e32640 1 -- 172.21.15.22:0/2231466833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90105530 msgr2=0x7fea9010d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.342+0000 7fea94e32640 1 --2- 172.21.15.22:0/2231466833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90105530 0x7fea9010d100 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fea88009ca0 tx=0x7fea8802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.342+0000 7fea94e32640 1 -- 172.21.15.22:0/2231466833 shutdown_connections 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.342+0000 7fea94e32640 1 --2- 172.21.15.22:0/2231466833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90105530 0x7fea9010d100 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.342+0000 7fea94e32640 1 --2- 172.21.15.22:0/2231466833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea90100e10 0x7fea90104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.342+0000 7fea94e32640 1 --2- 172.21.15.22:0/2231466833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea90100460 0x7fea90100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.342+0000 7fea94e32640 1 -- 172.21.15.22:0/2231466833 >> 172.21.15.22:0/2231466833 conn(0x7fea900772f0 msgr2=0x7fea90077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.343+0000 7fea94e32640 1 -- 172.21.15.22:0/2231466833 shutdown_connections 2024-03-09T00:04:42.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.343+0000 7fea94e32640 1 -- 172.21.15.22:0/2231466833 wait complete. 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.343+0000 7fea94e32640 1 Processor -- start 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.343+0000 7fea94e32640 1 -- start start 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea94e32640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90100460 0x7fea9019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea94e32640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea90105530 0x7fea9019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea94e32640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea901968f0 0x7fea90196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea94e32640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901105d0 con 0x7fea90100460 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea94e32640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90110450 con 0x7fea901968f0 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea94e32640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90110750 con 0x7fea90105530 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90100460 0x7fea9019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90100460 0x7fea9019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47720/0 (socket says 172.21.15.22:47720) 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 -- 172.21.15.22:0/2157870726 learned_addr learned my addr 172.21.15.22:0/2157870726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8dd9b640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea90105530 0x7fea9019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8ed9d640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea901968f0 0x7fea90196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 -- 172.21.15.22:0/2157870726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea90105530 msgr2=0x7fea9019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea90105530 0x7fea9019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 -- 172.21.15.22:0/2157870726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea901968f0 msgr2=0x7fea90196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea901968f0 0x7fea90196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 -- 172.21.15.22:0/2157870726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90197410 con 0x7fea90100460 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.344+0000 7fea8e59c640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90100460 0x7fea9019c740 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fea80009970 tx=0x7fea80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.345+0000 7fea7f7fe640 1 -- 172.21.15.22:0/2157870726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000d020 con 0x7fea90100460 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.345+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea90197700 con 0x7fea90100460 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.345+0000 7fea7f7fe640 1 -- 172.21.15.22:0/2157870726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea80007d40 con 0x7fea90100460 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.345+0000 7fea7f7fe640 1 -- 172.21.15.22:0/2157870726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000ccc0 con 0x7fea90100460 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.345+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea900687a0 con 0x7fea90100460 2024-03-09T00:04:42.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.346+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea900680e0 con 0x7fea90100460 2024-03-09T00:04:42.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.347+0000 7fea7f7fe640 1 -- 172.21.15.22:0/2157870726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fea8000c460 con 0x7fea90100460 2024-03-09T00:04:42.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.348+0000 7fea7f7fe640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fea780a2970 0x7fea780a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.348+0000 7fea8dd9b640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fea780a2970 0x7fea780a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.348+0000 7fea7f7fe640 1 -- 172.21.15.22:0/2157870726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fea800d0aa0 con 0x7fea90100460 2024-03-09T00:04:42.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.348+0000 7fea8dd9b640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fea780a2970 0x7fea780a4e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea900ffcb0 tx=0x7fea84005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.352+0000 7fea7f7fe640 1 -- 172.21.15.22:0/2157870726 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fea8009cdb0 con 0x7fea90100460 2024-03-09T00:04:42.379 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72503]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:42.379 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72503]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:42.380 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72503]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:42.380 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:04:41 smithi192 sudo[72503]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:42.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.598+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea90069220 con 0x7fea90100460 2024-03-09T00:04:42.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.599+0000 7fea7f7fe640 1 -- 172.21.15.22:0/2157870726 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fea800a0e50 con 0x7fea90100460 2024-03-09T00:04:42.595 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:04:42.596 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:04:42.597 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:04:42.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fea780a2970 msgr2=0x7fea780a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fea780a2970 0x7fea780a4e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea900ffcb0 tx=0x7fea84005f90 comp rx=0 tx=0).stop 2024-03-09T00:04:42.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90100460 msgr2=0x7fea9019c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90100460 0x7fea9019c740 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fea80009970 tx=0x7fea80009e30 comp rx=0 tx=0).stop 2024-03-09T00:04:42.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 shutdown_connections 2024-03-09T00:04:42.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fea780a2970 0x7fea780a4e30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea901968f0 0x7fea90196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea90105530 0x7fea9019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 --2- 172.21.15.22:0/2157870726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea90100460 0x7fea9019c740 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.601+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 >> 172.21.15.22:0/2157870726 conn(0x7fea900772f0 msgr2=0x7fea900fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.602+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 shutdown_connections 2024-03-09T00:04:42.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.602+0000 7fea94e32640 1 -- 172.21.15.22:0/2157870726 wait complete. 2024-03-09T00:04:42.610 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:42.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:04:42.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1040683 9.94 MB/sec execute 410 sec latency 36.764 ms 2024-03-09T00:04:42.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 -- 172.21.15.22:0/4061514009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881065a0 msgr2=0x7f248810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 --2- 172.21.15.22:0/4061514009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881065a0 0x7f248810e170 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2470009900 tx=0x7f247002bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:42.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 -- 172.21.15.22:0/4061514009 shutdown_connections 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 --2- 172.21.15.22:0/4061514009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881065a0 0x7f248810e170 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 --2- 172.21.15.22:0/4061514009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2488101e80 0x7f2488105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 --2- 172.21.15.22:0/4061514009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24881014d0 0x7f24881018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 -- 172.21.15.22:0/4061514009 >> 172.21.15.22:0/4061514009 conn(0x7f24880fcd90 msgr2=0x7f24880ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.769+0000 7f248f129640 1 -- 172.21.15.22:0/4061514009 shutdown_connections 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 -- 172.21.15.22:0/4061514009 wait complete. 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 Processor -- start 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 -- start start 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881014d0 0x7f248819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24881065a0 0x7f248819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24881979c0 0x7f2488197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2488111660 con 0x7f24881014d0 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881114e0 con 0x7f24881065a0 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248f129640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881117e0 con 0x7f24881979c0 2024-03-09T00:04:42.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248cec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881014d0 0x7f248819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248cec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881014d0 0x7f248819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47740/0 (socket says 172.21.15.22:47740) 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248cec5640 1 -- 172.21.15.22:0/2749640068 learned_addr learned my addr 172.21.15.22:0/2749640068 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f2487fff640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24881065a0 0x7f248819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.770+0000 7f248d6c6640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24881979c0 0x7f2488197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248cec5640 1 -- 172.21.15.22:0/2749640068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24881979c0 msgr2=0x7f2488197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248cec5640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24881979c0 0x7f2488197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248cec5640 1 -- 172.21.15.22:0/2749640068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24881065a0 msgr2=0x7f248819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248cec5640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24881065a0 0x7f248819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248cec5640 1 -- 172.21.15.22:0/2749640068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24881984e0 con 0x7f24881014d0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248cec5640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881014d0 0x7f248819d7c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f248000ebc0 tx=0x7f248000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f2485ffb640 1 -- 172.21.15.22:0/2749640068 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480010630 con 0x7f24881014d0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f2485ffb640 1 -- 172.21.15.22:0/2749640068 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2480010cb0 con 0x7f24881014d0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24881987d0 con 0x7f24881014d0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f2485ffb640 1 -- 172.21.15.22:0/2749640068 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480018bd0 con 0x7f24881014d0 2024-03-09T00:04:42.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.771+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881a49d0 con 0x7f24881014d0 2024-03-09T00:04:42.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2485ffb640 1 -- 172.21.15.22:0/2749640068 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f24800107d0 con 0x7f24881014d0 2024-03-09T00:04:42.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 Processor -- start 2024-03-09T00:04:42.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 -- start start 2024-03-09T00:04:42.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbe1a0 0x557b75fbe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b75f85360 con 0x557b75fc76a0 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b75f21ec0 con 0x557b75fbe1a0 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2491de2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b75ecad30 con 0x557b75fbeac0 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2487fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2485ffb640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24740a2970 0x7f24740a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f248d6c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbe1a0 0x557b75fbe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f248cec5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2487fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47748/0 (socket says 172.21.15.22:47748) 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.774+0000 7f2487fff640 1 -- 172.21.15.22:0/3925340247 learned_addr learned my addr 172.21.15.22:0/3925340247 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f2487fff640 1 -- 172.21.15.22:0/3925340247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 msgr2=0x557b75fc7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f2487fff640 1 --2- 172.21.15.22:0/3925340247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f2485ffb640 1 -- 172.21.15.22:0/2749640068 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f24800cf850 con 0x7f24881014d0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f2487fff640 1 -- 172.21.15.22:0/3925340247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbe1a0 msgr2=0x557b75fbe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f2487fff640 1 --2- 172.21.15.22:0/3925340247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbe1a0 0x557b75fbe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f248d6c6640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24740a2970 0x7f24740a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f2487fff640 1 -- 172.21.15.22:0/3925340247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b75ec9bc0 con 0x557b75fc76a0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f2487fff640 1 --2- 172.21.15.22:0/3925340247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc9a90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2478009c00 tx=0x7f2478011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138d568f7a26c9b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f247effd640 1 -- 172.21.15.22:0/3925340247 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478004050 con 0x557b75fc76a0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f247effd640 1 -- 172.21.15.22:0/3925340247 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2478005500 con 0x557b75fc76a0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f248d6c6640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24740a2970 0x7f24740a4e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2470005ab0 tx=0x7f247002bc30 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.775+0000 7f247effd640 1 -- 172.21.15.22:0/3925340247 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478004dd0 con 0x557b75fc76a0 2024-03-09T00:04:42.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 -- 172.21.15.22:0/3925340247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 msgr2=0x557b75fc9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 --2- 172.21.15.22:0/3925340247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc9a90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2478009c00 tx=0x7f2478011810 comp rx=0 tx=0).stop 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 -- 172.21.15.22:0/3925340247 shutdown_connections 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 --2- 172.21.15.22:0/3925340247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc9a90 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 --2- 172.21.15.22:0/3925340247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 --2- 172.21.15.22:0/3925340247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbe1a0 0x557b75fbe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 -- 172.21.15.22:0/3925340247 >> 172.21.15.22:0/3925340247 conn(0x557b75fb88d0 msgr2=0x557b75fb8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 -- 172.21.15.22:0/3925340247 shutdown_connections 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 -- 172.21.15.22:0/3925340247 wait complete. 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 -- start start 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.776+0000 7f2491de2740 1 Processor -- start 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbe1a0 0x557b75fc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbeac0 0x557b75fc4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b75ec04a0 con 0x557b75fc76a0 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b75f1df60 con 0x557b75fbeac0 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b75fca490 con 0x557b75fbe1a0 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 10 client.0 _ll_register_callbacks cb 0x557b75f1bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248cec5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbe1a0 0x557b75fc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47762/0 (socket says 172.21.15.22:47762) 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 -- 172.21.15.22:0/1606119712 learned_addr learned my addr 172.21.15.22:0/1606119712 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2487fff640 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbeac0 0x557b75fc4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 -- 172.21.15.22:0/1606119712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbe1a0 msgr2=0x557b75fc4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbe1a0 0x557b75fc4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 -- 172.21.15.22:0/1606119712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbeac0 msgr2=0x557b75fc4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbeac0 0x557b75fc4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 -- 172.21.15.22:0/1606119712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b75c363f0 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2487fff640 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbeac0 0x557b75fc4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f248d6c6640 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc50c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2470035000 tx=0x7f2470009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf607e91b2ac1db3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2467fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2467fff640 1 -- 172.21.15.22:0/1606119712 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470052060 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2467fff640 1 -- 172.21.15.22:0/1606119712 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2470005130 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.777+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2478005500 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2467fff640 1 -- 172.21.15.22:0/1606119712 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2470059050 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 10 client.18708 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557b75d007d0 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2467fff640 1 -- 172.21.15.22:0/1606119712 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2478005500 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2467fff640 1 -- 172.21.15.22:0/1606119712 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2470003d20 con 0x557b75fc76a0 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 10 client.18708 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 10 client.18708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:04:42.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 20 client.18708 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b75fd7800 0x557b75fd9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 4 client.18708 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b75f2f400 con 0x557b75fd7800 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f248cec5640 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b75fd7800 0x557b75fd9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f2467fff640 10 client.18708 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.778+0000 7f248cec5640 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b75fd7800 0x557b75fd9bf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c06581178e71a04 server_cookie=f1acae40b9192a2d in_seq=0 out_seq=0 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2467fff640 1 -- 172.21.15.22:0/1606119712 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b75f2f400 con 0x557b75fd7800 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2467fff640 10 client.18708 handle_command_reply: tid=0 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 client.18708 shutdown 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 msgr2=0x557b75fc50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc50c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2470035000 tx=0x7f2470009260 comp rx=0 tx=0).stop 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 shutdown_connections 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b75fd7800 0x557b75fd9bf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fc76a0 0x557b75fc50c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2467fff640 0 client.18708 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fbeac0 0x557b75fc4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 --2- 172.21.15.22:0/1606119712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbe1a0 0x557b75fc4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 >> 172.21.15.22:0/1606119712 conn(0x557b75fb88d0 msgr2=0x557b75fba620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 shutdown_connections 2024-03-09T00:04:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 1 -- 172.21.15.22:0/1606119712 wait complete. 2024-03-09T00:04:42.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.779+0000 7f2491de2740 20 client.18708 trim_cache size 0 max 16384 2024-03-09T00:04:42.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 Processor -- start 2024-03-09T00:04:42.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 -- start start 2024-03-09T00:04:42.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fc6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fc3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881b0c20 con 0x557b75fbe1a0 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881ae7c0 con 0x557b75fc76a0 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.792+0000 7f2491de2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881ac880 con 0x557b75fbeac0 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fc6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fc6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47770/0 (socket says 172.21.15.22:47770) 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 -- 172.21.15.22:0/2438412167 learned_addr learned my addr 172.21.15.22:0/2438412167 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248d6c6640 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fc3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f2487fff640 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 -- 172.21.15.22:0/2438412167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 msgr2=0x557b75fc2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 -- 172.21.15.22:0/2438412167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 msgr2=0x557b75fc3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fc3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 -- 172.21.15.22:0/2438412167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b75fbe8f0 con 0x557b75fbe1a0 2024-03-09T00:04:42.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f248cec5640 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fc6d70 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f248009baf0 tx=0x7f24800d1d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c074e427729572f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f247e7fc640 1 -- 172.21.15.22:0/2438412167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480010e60 con 0x557b75fbe1a0 2024-03-09T00:04:42.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f247e7fc640 1 -- 172.21.15.22:0/2438412167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2480010ac0 con 0x557b75fbe1a0 2024-03-09T00:04:42.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.793+0000 7f247e7fc640 1 -- 172.21.15.22:0/2438412167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24800981d0 con 0x557b75fbe1a0 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 -- 172.21.15.22:0/2438412167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 msgr2=0x557b75fc6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fc6d70 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f248009baf0 tx=0x7f24800d1d60 comp rx=0 tx=0).stop 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 -- 172.21.15.22:0/2438412167 shutdown_connections 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fc3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fc2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 --2- 172.21.15.22:0/2438412167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fc6d70 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 -- 172.21.15.22:0/2438412167 >> 172.21.15.22:0/2438412167 conn(0x557b75fb88d0 msgr2=0x557b75fde530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 -- 172.21.15.22:0/2438412167 shutdown_connections 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 -- 172.21.15.22:0/2438412167 wait complete. 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 -- start start 2024-03-09T00:04:42.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.794+0000 7f2491de2740 1 Processor -- start 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2491de2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2491de2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fcccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2491de2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fcd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2491de2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881ace80 con 0x557b75fbe1a0 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2491de2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881ae160 con 0x557b75fc76a0 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2491de2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881ad4a0 con 0x557b75fbeac0 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2491de2740 10 client.0 _ll_register_callbacks cb 0x7f2478009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47784/0 (socket says 172.21.15.22:47784) 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 -- 172.21.15.22:0/2148855919 learned_addr learned my addr 172.21.15.22:0/2148855919 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2487fff640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fcccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248d6c6640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fcd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 -- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 msgr2=0x557b75fcccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fcccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 -- 172.21.15.22:0/2148855919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 msgr2=0x557b75fcd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fcd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 -- 172.21.15.22:0/2148855919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b75fcd730 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248d6c6640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fcd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f248cec5640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fce740 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f248009b860 tx=0x7f248009bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f939470a5a0244d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2467fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.795+0000 7f2467fff640 1 -- 172.21.15.22:0/2148855919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480010e60 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2467fff640 1 -- 172.21.15.22:0/2148855919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f24800cce20 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2480010ac0 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2467fff640 1 -- 172.21.15.22:0/2148855919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2480099c00 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 10 client.18714 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557b7602e990 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2467fff640 1 -- 172.21.15.22:0/2148855919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2480010ac0 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2467fff640 1 -- 172.21.15.22:0/2148855919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f24800ce040 con 0x557b75fbe1a0 2024-03-09T00:04:42.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 10 client.18714 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 10 client.18714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 20 client.18714 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b7602f040 0x557b76031460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 4 client.18714 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557b75f29750 con 0x557b7602f040 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.796+0000 7f2487fff640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b7602f040 0x557b76031460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2467fff640 10 client.18714 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2487fff640 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b7602f040 0x557b76031460 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3140a6aa51fa229b server_cookie=6b1d0f435537aa21 in_seq=0 out_seq=0 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2467fff640 1 -- 172.21.15.22:0/2148855919 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557b75f29750 con 0x557b7602f040 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2467fff640 10 client.18714 handle_command_reply: tid=0 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 client.18714 shutdown 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 msgr2=0x557b75fce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fce740 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f248009b860 tx=0x7f248009bce0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 shutdown_connections 2024-03-09T00:04:42.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557b7602f040 0x557b76031460 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b75fc76a0 0x557b75fcd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2467fff640 0 client.18714 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b75fbeac0 0x557b75fcccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 --2- 172.21.15.22:0/2148855919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557b75fbe1a0 0x557b75fce740 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 >> 172.21.15.22:0/2148855919 conn(0x557b75fb88d0 msgr2=0x557b75fc2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 shutdown_connections 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 1 -- 172.21.15.22:0/2148855919 wait complete. 2024-03-09T00:04:42.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.797+0000 7f2491de2740 20 client.18714 trim_cache size 0 max 16384 2024-03-09T00:04:42.799 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24740a2970 msgr2=0x7f24740a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24740a2970 0x7f24740a4e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2470005ab0 tx=0x7f247002bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881014d0 msgr2=0x7f248819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881014d0 0x7f248819d7c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f248000ebc0 tx=0x7f248000c6a0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 shutdown_connections 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24740a2970 0x7f24740a4e30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24881979c0 0x7f2488197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24881065a0 0x7f248819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 --2- 172.21.15.22:0/2749640068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24881014d0 0x7f248819d7c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 >> 172.21.15.22:0/2749640068 conn(0x7f24880fcd90 msgr2=0x7f24880ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 shutdown_connections 2024-03-09T00:04:42.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.798+0000 7f248f129640 1 -- 172.21.15.22:0/2749640068 wait complete. 2024-03-09T00:04:42.802 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:42 smithi022 ceph-mon[31894]: pgmap v321: 129 pgs: 129 active+clean; 96 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-09T00:04:42.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:42 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2157870726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:42.803 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100883]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:42.803 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100883]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:42.803 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100883]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:42.803 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:04:42 smithi022 sudo[100883]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:42.804 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:42.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:04:42.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 -- 172.21.15.22:0/1827950083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81054b0 msgr2=0x7f6ad810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/1827950083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81054b0 0x7f6ad810d080 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ca0 tx=0x7f6ad002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 -- 172.21.15.22:0/1827950083 shutdown_connections 2024-03-09T00:04:42.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/1827950083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81054b0 0x7f6ad810d080 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/1827950083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad8100d90 0x7f6ad8104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/1827950083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ad81003e0 0x7f6ad81007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 -- 172.21.15.22:0/1827950083 >> 172.21.15.22:0/1827950083 conn(0x7f6ad80773a0 msgr2=0x7f6ad80777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 -- 172.21.15.22:0/1827950083 shutdown_connections 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 -- 172.21.15.22:0/1827950083 wait complete. 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 Processor -- start 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.915+0000 7f6ade3f1640 1 -- start start 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ade3f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81003e0 0x7f6ad819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ade3f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad81054b0 0x7f6ad819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ade3f1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ad8196830 0x7f6ad8196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ade3f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8110570 con 0x7f6ad81003e0 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ade3f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81103f0 con 0x7f6ad81054b0 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ade3f1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81106f0 con 0x7f6ad8196830 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81003e0 0x7f6ad819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81003e0 0x7f6ad819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47804/0 (socket says 172.21.15.22:47804) 2024-03-09T00:04:42.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 -- 172.21.15.22:0/3985186073 learned_addr learned my addr 172.21.15.22:0/3985186073 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad81054b0 0x7f6ad819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6adc98e640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ad8196830 0x7f6ad8196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 -- 172.21.15.22:0/3985186073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ad8196830 msgr2=0x7f6ad8196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ad8196830 0x7f6ad8196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 -- 172.21.15.22:0/3985186073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad81054b0 msgr2=0x7f6ad819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad81054b0 0x7f6ad819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 -- 172.21.15.22:0/3985186073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad81973c0 con 0x7f6ad81003e0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad81054b0 0x7f6ad819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81003e0 0x7f6ad819c670 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000eb00 tx=0x7f6ac000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.916+0000 7f6ad57fa640 1 -- 172.21.15.22:0/3985186073 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000bc30 con 0x7f6ad81003e0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.917+0000 7f6ad57fa640 1 -- 172.21.15.22:0/3985186073 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ac000bdd0 con 0x7f6ad81003e0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.917+0000 7f6ad57fa640 1 -- 172.21.15.22:0/3985186073 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0017410 con 0x7f6ad81003e0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.917+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad81976b0 con 0x7f6ad81003e0 2024-03-09T00:04:42.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.917+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad80687d0 con 0x7f6ad81003e0 2024-03-09T00:04:42.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ad57fa640 1 -- 172.21.15.22:0/3985186073 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6ac0005020 con 0x7f6ad81003e0 2024-03-09T00:04:42.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 Processor -- start 2024-03-09T00:04:42.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 -- start start 2024-03-09T00:04:42.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37e1a0 0x55992c37e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37eac0 0x55992c387160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c389a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992c345360 con 0x55992c3876a0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992c2e1ec0 con 0x55992c37eac0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992c28ad30 con 0x55992c37e1a0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ad57fa640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ac40a29f0 0x7f6ac40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6adc98e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37e1a0 0x55992c37e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c389a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37eac0 0x55992c387160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6adc98e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37e1a0 0x55992c37e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38348/0 (socket says 172.21.15.22:38348) 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.919+0000 7f6adc98e640 1 -- 172.21.15.22:0/87097490 learned_addr learned my addr 172.21.15.22:0/87097490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6adc98e640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ac40a29f0 0x7f6ac40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ad57fa640 1 -- 172.21.15.22:0/3985186073 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6ac00d0a80 con 0x7f6ad81003e0 2024-03-09T00:04:42.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ad77fe640 1 -- 172.21.15.22:0/87097490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37e1a0 msgr2=0x55992c37e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/87097490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37e1a0 0x55992c37e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ad77fe640 1 -- 172.21.15.22:0/87097490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37eac0 msgr2=0x55992c387160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/87097490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37eac0 0x55992c387160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ad77fe640 1 -- 172.21.15.22:0/87097490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55992c289bc0 con 0x55992c3876a0 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/87097490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c389a90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8197b30 tx=0x7f6ac8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186eb00c3415a3d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6adc98e640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ac40a29f0 0x7f6ac40a4eb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009430 tx=0x7f6ad002c0f0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ace7fc640 1 -- 172.21.15.22:0/87097490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8004230 con 0x55992c3876a0 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ace7fc640 1 -- 172.21.15.22:0/87097490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ac80043d0 con 0x55992c3876a0 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.920+0000 7f6ace7fc640 1 -- 172.21.15.22:0/87097490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac80035f0 con 0x55992c3876a0 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/87097490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 msgr2=0x55992c389a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/87097490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c389a90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8197b30 tx=0x7f6ac8011810 comp rx=0 tx=0).stop 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/87097490 shutdown_connections 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/87097490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c389a90 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/87097490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37eac0 0x55992c387160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/87097490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37e1a0 0x55992c37e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/87097490 >> 172.21.15.22:0/87097490 conn(0x55992c3788d0 msgr2=0x55992c378cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/87097490 shutdown_connections 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/87097490 wait complete. 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 -- start start 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.921+0000 7f6ae0fd4740 1 Processor -- start 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c384690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c384bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c385110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992c2804a0 con 0x55992c3876a0 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992c2ddf60 con 0x55992c37e1a0 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992c38a4e0 con 0x55992c37eac0 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 10 client.0 _ll_register_callbacks cb 0x55992c2dbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6adc98e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c385110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c384690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c384bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c384690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58866/0 (socket says 172.21.15.22:58866) 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad7fff640 1 -- 172.21.15.22:0/940115657 learned_addr learned my addr 172.21.15.22:0/940115657 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad77fe640 1 -- 172.21.15.22:0/940115657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 msgr2=0x55992c384690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c384690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad77fe640 1 -- 172.21.15.22:0/940115657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 msgr2=0x55992c385110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c385110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad77fe640 1 -- 172.21.15.22:0/940115657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55992bff63f0 con 0x55992c37eac0 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6adc98e640 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c385110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c384bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac800fc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a381d538b9f39 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ab77fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ab77fe640 1 -- 172.21.15.22:0/940115657 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8028020 con 0x55992c37eac0 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ac80043d0 con 0x55992c37eac0 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.922+0000 7f6ab77fe640 1 -- 172.21.15.22:0/940115657 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6ac8004ce0 con 0x55992c37eac0 2024-03-09T00:04:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ab77fe640 1 -- 172.21.15.22:0/940115657 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ac8033050 con 0x55992c37eac0 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 10 client.28159 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55992c0c07d0 con 0x55992c37eac0 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ab77fe640 1 -- 172.21.15.22:0/940115657 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ac80043d0 con 0x55992c37eac0 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ab77fe640 1 -- 172.21.15.22:0/940115657 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ac8019600 con 0x55992c37eac0 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 10 client.28159 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 10 client.28159 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 20 client.28159 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3977e0 0x55992c399bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 4 client.28159 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55992c2ef400 con 0x55992c3977e0 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.923+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3977e0 0x55992c399bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.924+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3977e0 0x55992c399bd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9a748d5a4a3096d server_cookie=654e771d6fa43730 in_seq=0 out_seq=0 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.924+0000 7f6ab77fe640 10 client.28159 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:42.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.924+0000 7f6ab77fe640 1 -- 172.21.15.22:0/940115657 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55992c2ef400 con 0x55992c3977e0 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.924+0000 7f6ab77fe640 10 client.28159 handle_command_reply: tid=0 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.924+0000 7f6ae0fd4740 1 client.28159 shutdown 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 msgr2=0x55992c384bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c384bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac800fc20 comp rx=0 tx=0).stop 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 shutdown_connections 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3977e0 0x55992c399bd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c385110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c384bd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/940115657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c384690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ab77fe640 0 client.28159 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 >> 172.21.15.22:0/940115657 conn(0x55992c3788d0 msgr2=0x55992c37a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 shutdown_connections 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/940115657 wait complete. 2024-03-09T00:04:42.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.925+0000 7f6ae0fd4740 20 client.28159 trim_cache size 0 max 16384 2024-03-09T00:04:42.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 Processor -- start 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 -- start start 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c386dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c37eac0 0x55992c39e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c3876a0 0x55992c3818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81ae940 con 0x55992c37eac0 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81ac4e0 con 0x55992c37e1a0 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.936+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81aa5a0 con 0x55992c3876a0 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c37eac0 0x55992c39e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c37eac0 0x55992c39e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47842/0 (socket says 172.21.15.22:47842) 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 -- 172.21.15.22:0/755972495 learned_addr learned my addr 172.21.15.22:0/755972495 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6adc98e640 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c3876a0 0x55992c3818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c386dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 -- 172.21.15.22:0/755972495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c3876a0 msgr2=0x55992c3818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c3876a0 0x55992c3818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 -- 172.21.15.22:0/755972495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 msgr2=0x55992c386dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c386dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 -- 172.21.15.22:0/755972495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55992c399f40 con 0x55992c37eac0 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c37eac0 0x55992c39e440 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009ef0 tx=0x7f6ac8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7b6c57b8212a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6acdffb640 1 -- 172.21.15.22:0/755972495 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac803c070 con 0x55992c37eac0 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6acdffb640 1 -- 172.21.15.22:0/755972495 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ac801f040 con 0x55992c37eac0 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.937+0000 7f6acdffb640 1 -- 172.21.15.22:0/755972495 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8028040 con 0x55992c37eac0 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/755972495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c37eac0 msgr2=0x55992c39e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c37eac0 0x55992c39e440 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009ef0 tx=0x7f6ac8011810 comp rx=0 tx=0).stop 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/755972495 shutdown_connections 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c3876a0 0x55992c3818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c37eac0 0x55992c39e440 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/755972495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c386dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/755972495 >> 172.21.15.22:0/755972495 conn(0x55992c3788d0 msgr2=0x55992c378cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/755972495 shutdown_connections 2024-03-09T00:04:42.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/755972495 wait complete. 2024-03-09T00:04:42.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 -- start start 2024-03-09T00:04:42.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.938+0000 7f6ae0fd4740 1 Processor -- start 2024-03-09T00:04:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c38ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c38ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c38d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81aaba0 con 0x55992c3876a0 2024-03-09T00:04:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81abe80 con 0x55992c37e1a0 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81ab1c0 con 0x55992c37eac0 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 10 client.0 _ll_register_callbacks cb 0x7f6ac8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c38d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c38d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47850/0 (socket says 172.21.15.22:47850) 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c38ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 -- 172.21.15.22:0/1479740435 learned_addr learned my addr 172.21.15.22:0/1479740435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ad77fe640 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c38ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 -- 172.21.15.22:0/1479740435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 msgr2=0x55992c38ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c38ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 -- 172.21.15.22:0/1479740435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 msgr2=0x55992c38ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c38ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 -- 172.21.15.22:0/1479740435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55992c38dba0 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6adc98e640 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c38d370 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ca0 tx=0x7f6ad00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b92ab40861d28f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ab77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ab77fe640 1 -- 172.21.15.22:0/1479740435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0039040 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ab77fe640 1 -- 172.21.15.22:0/1479740435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6ad0004450 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.939+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ac801f040 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ab77fe640 1 -- 172.21.15.22:0/1479740435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad0060050 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 10 client.18732 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55992c3eebb0 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ab77fe640 1 -- 172.21.15.22:0/1479740435 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ac801f040 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ab77fe640 1 -- 172.21.15.22:0/1479740435 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ad003f070 con 0x55992c3876a0 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 10 client.18732 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 10 client.18732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:04:42.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 20 client.18732 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:42.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3ef310 0x55992c3f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:42.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 4 client.18732 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:42.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55992c2e9750 con 0x55992c3ef310 2024-03-09T00:04:42.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.940+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3ef310 0x55992c3f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:42.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ab77fe640 10 client.18732 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:42.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ad7fff640 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3ef310 0x55992c3f1730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87db3693724d9809 server_cookie=bf8b273b934fdd1 in_seq=0 out_seq=0 2024-03-09T00:04:42.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ab77fe640 1 -- 172.21.15.22:0/1479740435 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55992c2e9750 con 0x55992c3ef310 2024-03-09T00:04:42.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ab77fe640 10 client.18732 handle_command_reply: tid=0 2024-03-09T00:04:42.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 client.18732 shutdown 2024-03-09T00:04:42.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 msgr2=0x55992c38d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c38d370 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ca0 tx=0x7f6ad00095c0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 shutdown_connections 2024-03-09T00:04:42.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55992c3ef310 0x55992c3f1730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55992c3876a0 0x55992c38d370 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55992c37eac0 0x55992c38ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ab77fe640 0 client.18732 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 --2- 172.21.15.22:0/1479740435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55992c37e1a0 0x55992c38ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 >> 172.21.15.22:0/1479740435 conn(0x55992c3788d0 msgr2=0x55992c39e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 shutdown_connections 2024-03-09T00:04:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 1 -- 172.21.15.22:0/1479740435 wait complete. 2024-03-09T00:04:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.941+0000 7f6ae0fd4740 20 client.18732 trim_cache size 0 max 16384 2024-03-09T00:04:42.953 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ac40a29f0 msgr2=0x7f6ac40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ac40a29f0 0x7f6ac40a4eb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009430 tx=0x7f6ad002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81003e0 msgr2=0x7f6ad819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81003e0 0x7f6ad819c670 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000eb00 tx=0x7f6ac000ef00 comp rx=0 tx=0).stop 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 shutdown_connections 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ac40a29f0 0x7f6ac40a4eb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ad8196830 0x7f6ad8196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad81054b0 0x7f6ad819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 --2- 172.21.15.22:0/3985186073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ad81003e0 0x7f6ad819c670 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 >> 172.21.15.22:0/3985186073 conn(0x7f6ad80773a0 msgr2=0x7f6ad80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 shutdown_connections 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:42.942+0000 7f6ade3f1640 1 -- 172.21.15.22:0/3985186073 wait complete. 2024-03-09T00:04:42.955 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:42.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:04:43.032 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:42 smithi062 ceph-mon[36129]: pgmap v321: 129 pgs: 129 active+clean; 96 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-09T00:04:43.032 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:42 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2157870726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:43.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 -- 172.21.15.22:0/707560112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc069920 msgr2=0x7f34bc069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 --2- 172.21.15.22:0/707560112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc069920 0x7f34bc069da0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009930 tx=0x7f34ac02bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 -- 172.21.15.22:0/707560112 shutdown_connections 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 --2- 172.21.15.22:0/707560112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34bc06a2e0 0x7f34bc10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 --2- 172.21.15.22:0/707560112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc069920 0x7f34bc069da0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 --2- 172.21.15.22:0/707560112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc1073b0 0x7f34bc107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 -- 172.21.15.22:0/707560112 >> 172.21.15.22:0/707560112 conn(0x7f34bc076f90 msgr2=0x7f34bc0773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 -- 172.21.15.22:0/707560112 shutdown_connections 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.069+0000 7f34c4280640 1 -- 172.21.15.22:0/707560112 wait complete. 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 Processor -- start 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 -- start start 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34bc069920 0x7f34bc100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc06a2e0 0x7f34bc100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc1073b0 0x7f34bc100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc111640 con 0x7f34bc1073b0 2024-03-09T00:04:43.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1114c0 con 0x7f34bc06a2e0 2024-03-09T00:04:43.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c4280640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1117c0 con 0x7f34bc069920 2024-03-09T00:04:43.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c281d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc1073b0 0x7f34bc100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c281d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc1073b0 0x7f34bc100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47856/0 (socket says 172.21.15.22:47856) 2024-03-09T00:04:43.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c281d640 1 -- 172.21.15.22:0/3483405711 learned_addr learned my addr 172.21.15.22:0/3483405711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:43.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c201c640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34bc069920 0x7f34bc100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.070+0000 7f34c181b640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc06a2e0 0x7f34bc100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c281d640 1 -- 172.21.15.22:0/3483405711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34bc069920 msgr2=0x7f34bc100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c281d640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34bc069920 0x7f34bc100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c281d640 1 -- 172.21.15.22:0/3483405711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc06a2e0 msgr2=0x7f34bc100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c281d640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc06a2e0 0x7f34bc100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c281d640 1 -- 172.21.15.22:0/3483405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc1a5fa0 con 0x7f34bc1073b0 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c281d640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc1073b0 0x7f34bc100f90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f34b400eb10 tx=0x7f34b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34b37fe640 1 -- 172.21.15.22:0/3483405711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004b20 con 0x7f34bc1073b0 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34b37fe640 1 -- 172.21.15.22:0/3483405711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34b4004cc0 con 0x7f34bc1073b0 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc1a6290 con 0x7f34bc1073b0 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34b37fe640 1 -- 172.21.15.22:0/3483405711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4010760 con 0x7f34bc1073b0 2024-03-09T00:04:43.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.071+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc1a67b0 con 0x7f34bc1073b0 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.073+0000 7f34b37fe640 1 -- 172.21.15.22:0/3483405711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f34b4005020 con 0x7f34bc1073b0 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.074+0000 7f34c6e57740 1 Processor -- start 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.074+0000 7f34c6e57740 1 -- start start 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.074+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12c1a0 0x5651bb12c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.074+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb135160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.074+0000 7f34b37fe640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f34a80a2970 0x7f34a80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb1356a0 0x5651bb137a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c6e57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bb0f3360 con 0x5651bb12cac0 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c6e57740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bb08fec0 con 0x5651bb12c1a0 2024-03-09T00:04:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c181b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12c1a0 0x5651bb12c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb135160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47860/0 (socket says 172.21.15.22:47860) 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c181b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12c1a0 0x5651bb12c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58896/0 (socket says 172.21.15.22:58896) 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 -- 172.21.15.22:0/719251447 learned_addr learned my addr 172.21.15.22:0/719251447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c6e57740 1 -- 172.21.15.22:0/719251447 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bb038d30 con 0x5651bb1356a0 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34b37fe640 1 -- 172.21.15.22:0/3483405711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f34b40cf950 con 0x7f34bc1073b0 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c281d640 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb1356a0 0x5651bb137a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c181b640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f34a80a2970 0x7f34a80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 -- 172.21.15.22:0/719251447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb1356a0 msgr2=0x5651bb137a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb1356a0 0x5651bb137a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 -- 172.21.15.22:0/719251447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12c1a0 msgr2=0x5651bb12c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12c1a0 0x5651bb12c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 -- 172.21.15.22:0/719251447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651bb037bc0 con 0x5651bb12cac0 2024-03-09T00:04:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.075+0000 7f34c201c640 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb135160 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009c70 tx=0x7f34b8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927f09cb5abe5923 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.076+0000 7f34c181b640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f34a80a2970 0x7f34a80a4e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34ac005ab0 tx=0x7f34ac02bc30 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.076+0000 7f34b0bfe640 1 -- 172.21.15.22:0/719251447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8004230 con 0x5651bb12cac0 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.076+0000 7f34b0bfe640 1 -- 172.21.15.22:0/719251447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34b8009e20 con 0x5651bb12cac0 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.076+0000 7f34b0bfe640 1 -- 172.21.15.22:0/719251447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8019760 con 0x5651bb12cac0 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 -- 172.21.15.22:0/719251447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 msgr2=0x5651bb135160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb135160 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009c70 tx=0x7f34b8011810 comp rx=0 tx=0).stop 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 -- 172.21.15.22:0/719251447 shutdown_connections 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb1356a0 0x5651bb137a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb135160 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 --2- 172.21.15.22:0/719251447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12c1a0 0x5651bb12c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 -- 172.21.15.22:0/719251447 >> 172.21.15.22:0/719251447 conn(0x5651bb1268d0 msgr2=0x5651bb126cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 -- 172.21.15.22:0/719251447 shutdown_connections 2024-03-09T00:04:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 -- 172.21.15.22:0/719251447 wait complete. 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.077+0000 7f34c6e57740 1 -- start start 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 1 Processor -- start 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb1326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 0x5651bb133130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bb02e4a0 con 0x5651bb12cac0 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bb08bf60 con 0x5651bb1356a0 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bb138490 con 0x5651bb12c1a0 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c6e57740 10 client.0 _ll_register_callbacks cb 0x5651bb089d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c181b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c201c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb1326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c181b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb132bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47868/0 (socket says 172.21.15.22:47868) 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c281d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 0x5651bb133130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c181b640 1 -- 172.21.15.22:0/3948080455 learned_addr learned my addr 172.21.15.22:0/3948080455 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c201c640 1 -- 172.21.15.22:0/3948080455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 msgr2=0x5651bb133130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c201c640 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 0x5651bb133130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c201c640 1 -- 172.21.15.22:0/3948080455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 msgr2=0x5651bb132bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c201c640 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb132bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c201c640 1 -- 172.21.15.22:0/3948080455 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651bada43f0 con 0x5651bb12c1a0 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.078+0000 7f34c181b640 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c201c640 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb1326b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f34b801a000 tx=0x7f34b80043e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a087923247028a4e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3948080455 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8027020 con 0x5651bb12c1a0 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34b8009e20 con 0x5651bb12c1a0 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3948080455 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f34b8005260 con 0x5651bb12c1a0 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3948080455 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b8032050 con 0x5651bb12c1a0 2024-03-09T00:04:43.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 10 client.28177 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5651bae6e7d0 con 0x5651bb12c1a0 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3948080455 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34b8009e20 con 0x5651bb12c1a0 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3948080455 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f34b8018070 con 0x5651bb12c1a0 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 10 client.28177 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 10 client.28177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 20 client.28177 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb1457c0 0x5651bb147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 4 client.28177 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.079+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651bb09d400 con 0x5651bb1457c0 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.080+0000 7f34c181b640 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb1457c0 0x5651bb147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.080+0000 7f34c181b640 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb1457c0 0x5651bb147bb0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf65aaa6f9cdabd server_cookie=6c96eec89fd35716 in_seq=0 out_seq=0 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.080+0000 7f34a5ffb640 10 client.28177 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3948080455 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651bb09d400 con 0x5651bb1457c0 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34a5ffb640 10 client.28177 handle_command_reply: tid=0 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 client.28177 shutdown 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 msgr2=0x5651bb1326b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb1326b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f34b801a000 tx=0x7f34b80043e0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 shutdown_connections 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb1457c0 0x5651bb147bb0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 0x5651bb133130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12cac0 0x5651bb132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3948080455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb1326b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34a5ffb640 0 client.28177 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 >> 172.21.15.22:0/3948080455 conn(0x5651bb1268d0 msgr2=0x5651bb128620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 shutdown_connections 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 1 -- 172.21.15.22:0/3948080455 wait complete. 2024-03-09T00:04:43.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.081+0000 7f34c6e57740 20 client.28177 trim_cache size 0 max 16384 2024-03-09T00:04:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 Processor -- start 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 -- start start 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12c1a0 0x5651bb134de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12cac0 0x5651bb12d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 0x5651bb12df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1b2e40 con 0x5651bb12c1a0 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1b09e0 con 0x5651bb1356a0 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c6e57740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1aeaa0 con 0x5651bb12cac0 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12c1a0 0x5651bb134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12c1a0 0x5651bb134de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47876/0 (socket says 172.21.15.22:47876) 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 -- 172.21.15.22:0/348973033 learned_addr learned my addr 172.21.15.22:0/348973033 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c181b640 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12cac0 0x5651bb12d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 -- 172.21.15.22:0/348973033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12cac0 msgr2=0x5651bb12d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12cac0 0x5651bb12d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 -- 172.21.15.22:0/348973033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 msgr2=0x5651bb12df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 0x5651bb12df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.096+0000 7f34c201c640 1 -- 172.21.15.22:0/348973033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651bb126400 con 0x5651bb12c1a0 2024-03-09T00:04:43.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.097+0000 7f34c201c640 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12c1a0 0x5651bb134de0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009c70 tx=0x7f34b8004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d676e5c4d76e1b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.097+0000 7f34a7fff640 1 -- 172.21.15.22:0/348973033 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8027070 con 0x5651bb12c1a0 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.097+0000 7f34a7fff640 1 -- 172.21.15.22:0/348973033 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34b8004e40 con 0x5651bb12c1a0 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.097+0000 7f34a7fff640 1 -- 172.21.15.22:0/348973033 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80035f0 con 0x5651bb12c1a0 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 -- 172.21.15.22:0/348973033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12c1a0 msgr2=0x5651bb134de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12c1a0 0x5651bb134de0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009c70 tx=0x7f34b8004370 comp rx=0 tx=0).stop 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 -- 172.21.15.22:0/348973033 shutdown_connections 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb1356a0 0x5651bb12df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12cac0 0x5651bb12d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 --2- 172.21.15.22:0/348973033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb12c1a0 0x5651bb134de0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 -- 172.21.15.22:0/348973033 >> 172.21.15.22:0/348973033 conn(0x5651bb1268d0 msgr2=0x5651bb126cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.098+0000 7f34c6e57740 1 -- 172.21.15.22:0/348973033 shutdown_connections 2024-03-09T00:04:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.099+0000 7f34c6e57740 1 -- 172.21.15.22:0/348973033 wait complete. 2024-03-09T00:04:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.099+0000 7f34c6e57740 1 -- start start 2024-03-09T00:04:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.099+0000 7f34c6e57740 1 Processor -- start 2024-03-09T00:04:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb13cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12cac0 0x5651bb13afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c6e57740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb1356a0 0x5651bb13b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c6e57740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1af0a0 con 0x5651bb1356a0 2024-03-09T00:04:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c6e57740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1b0380 con 0x5651bb12cac0 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c6e57740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1af6c0 con 0x5651bb12c1a0 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c281d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb1356a0 0x5651bb13b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c281d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb1356a0 0x5651bb13b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47884/0 (socket says 172.21.15.22:47884) 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c281d640 1 -- 172.21.15.22:0/3181045502 learned_addr learned my addr 172.21.15.22:0/3181045502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c181b640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12cac0 0x5651bb13afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.100+0000 7f34c201c640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb13cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c281d640 1 -- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 msgr2=0x5651bb13cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c281d640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb13cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c281d640 1 -- 172.21.15.22:0/3181045502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12cac0 msgr2=0x5651bb13afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c281d640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12cac0 0x5651bb13afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c6e57740 10 client.0 _ll_register_callbacks cb 0x7f34b80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c281d640 1 -- 172.21.15.22:0/3181045502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651bb19bb60 con 0x5651bb1356a0 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34b8004e40 con 0x5651bb1356a0 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c181b640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12cac0 0x5651bb13afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c281d640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb1356a0 0x5651bb13b6f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f34b40cd2a0 tx=0x7f34b40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7c78afcdc01378 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34a5ffb640 10 client.18750 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3181045502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b409bbb0 con 0x5651bb1356a0 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3181045502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f34b4054750 con 0x5651bb1356a0 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3181045502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b409fd00 con 0x5651bb1356a0 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c6e57740 10 client.18750 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.101+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x5651bb19cbd0 con 0x5651bb1356a0 2024-03-09T00:04:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.104+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3181045502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f34b4005310 con 0x5651bb1356a0 2024-03-09T00:04:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.105+0000 7f34c6e57740 10 client.18750 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.105+0000 7f34c6e57740 10 client.18750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.105+0000 7f34c6e57740 20 client.18750 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.105+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb19d190 0x5651bb19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.105+0000 7f34c6e57740 4 client.18750 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.105+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651bb097750 con 0x5651bb19d190 2024-03-09T00:04:43.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.105+0000 7f34c201c640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb19d190 0x5651bb19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:43.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.106+0000 7f34a5ffb640 10 client.18750 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:43.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.106+0000 7f34c201c640 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb19d190 0x5651bb19f5b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e180860bf3dd8a0 server_cookie=198ad3976c680c25 in_seq=0 out_seq=0 2024-03-09T00:04:43.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.110+0000 7f34a5ffb640 1 -- 172.21.15.22:0/3181045502 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651bb097750 con 0x5651bb19d190 2024-03-09T00:04:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.110+0000 7f34a5ffb640 10 client.18750 handle_command_reply: tid=0 2024-03-09T00:04:43.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.110+0000 7f34c6e57740 1 client.18750 shutdown 2024-03-09T00:04:43.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb1356a0 msgr2=0x5651bb13b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb1356a0 0x5651bb13b6f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f34b40cd2a0 tx=0x7f34b40096c0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 shutdown_connections 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651bb19d190 0x5651bb19f5b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651bb1356a0 0x5651bb13b6f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651bb12cac0 0x5651bb13afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 --2- 172.21.15.22:0/3181045502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651bb12c1a0 0x5651bb13cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 >> 172.21.15.22:0/3181045502 conn(0x5651bb1268d0 msgr2=0x5651bb12d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34a5ffb640 0 client.18750 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 shutdown_connections 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.111+0000 7f34c6e57740 1 -- 172.21.15.22:0/3181045502 wait complete. 2024-03-09T00:04:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.112+0000 7f34c6e57740 20 client.18750 trim_cache size 0 max 16384 2024-03-09T00:04:43.108 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.112+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f34a80a2970 msgr2=0x7f34a80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.112+0000 7f34c4280640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f34a80a2970 0x7f34a80a4e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34ac005ab0 tx=0x7f34ac02bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.112+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc1073b0 msgr2=0x7f34bc100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.112+0000 7f34c4280640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc1073b0 0x7f34bc100f90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f34b400eb10 tx=0x7f34b400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 shutdown_connections 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f34a80a2970 0x7f34a80a4e30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34bc1073b0 0x7f34bc100f90 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc06a2e0 0x7f34bc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 --2- 172.21.15.22:0/3483405711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34bc069920 0x7f34bc100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 >> 172.21.15.22:0/3483405711 conn(0x7f34bc076f90 msgr2=0x7f34bc0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 shutdown_connections 2024-03-09T00:04:43.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:43.113+0000 7f34c4280640 1 -- 172.21.15.22:0/3483405711 wait complete. 2024-03-09T00:04:43.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:42 smithi192 ceph-mon[35995]: pgmap v321: 129 pgs: 129 active+clean; 96 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-09T00:04:43.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:42 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2157870726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:43.360 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72087]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:43.360 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:43.360 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72087]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:43.360 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72087]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:43.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1041615 9.92 MB/sec execute 411 sec latency 40.961 ms 2024-03-09T00:04:43.860 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:43.860 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:43.860 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72091]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:43.860 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:04:43 smithi062 sudo[72091]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:44.124 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:04:44.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:04:44.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:04:44.129 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:43 smithi192 sudo[72506]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:44.130 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:43 smithi192 sudo[72506]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:44.130 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:43 smithi192 sudo[72506]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:44.130 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:43 smithi192 sudo[72506]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:44.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 -- 172.21.15.22:0/2299602349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc100d80 msgr2=0x7f57dc104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 --2- 172.21.15.22:0/2299602349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc100d80 0x7f57dc104dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009860 tx=0x7f57cc02bb70 comp rx=0 tx=0).stop 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 -- 172.21.15.22:0/2299602349 shutdown_connections 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 --2- 172.21.15.22:0/2299602349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57dc1054a0 0x7f57dc10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 --2- 172.21.15.22:0/2299602349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc100d80 0x7f57dc104dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 --2- 172.21.15.22:0/2299602349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 0x7f57dc1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 -- 172.21.15.22:0/2299602349 >> 172.21.15.22:0/2299602349 conn(0x7f57dc0772f0 msgr2=0x7f57dc077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 -- 172.21.15.22:0/2299602349 shutdown_connections 2024-03-09T00:04:44.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.245+0000 7f57e3db2640 1 -- 172.21.15.22:0/2299602349 wait complete. 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 Processor -- start 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 -- start start 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 0x7f57dc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc1054a0 0x7f57dc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57dc1994b0 0x7f57dc1a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc110510 con 0x7f57dc1994b0 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc110390 con 0x7f57dc1054a0 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.246+0000 7f57e3db2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc110690 con 0x7f57dc1003d0 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e134d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc1054a0 0x7f57dc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e234f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57dc1994b0 0x7f57dc1a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 0x7f57dc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e134d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc1054a0 0x7f57dc198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58928/0 (socket says 172.21.15.22:58928) 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 0x7f57dc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38450/0 (socket says 172.21.15.22:38450) 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e134d640 1 -- 172.21.15.22:0/698768861 learned_addr learned my addr 172.21.15.22:0/698768861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 -- 172.21.15.22:0/698768861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc1054a0 msgr2=0x7f57dc198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc1054a0 0x7f57dc198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 -- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57dc1994b0 msgr2=0x7f57dc1a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57dc1994b0 0x7f57dc1a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 -- 172.21.15.22:0/698768861 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc1a4230 con 0x7f57dc1003d0 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e134d640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc1054a0 0x7f57dc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 0x7f57dc198990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57d8005fd0 tx=0x7f57d800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57d2ffd640 1 -- 172.21.15.22:0/698768861 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d800cd00 con 0x7f57dc1003d0 2024-03-09T00:04:44.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc1a4520 con 0x7f57dc1003d0 2024-03-09T00:04:44.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57d2ffd640 1 -- 172.21.15.22:0/698768861 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57d80035f0 con 0x7f57dc1003d0 2024-03-09T00:04:44.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57d2ffd640 1 -- 172.21.15.22:0/698768861 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8018630 con 0x7f57dc1003d0 2024-03-09T00:04:44.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.247+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc1a4ac0 con 0x7f57dc1003d0 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.250+0000 7f57d2ffd640 1 -- 172.21.15.22:0/698768861 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f57d80037a0 con 0x7f57dc1003d0 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.250+0000 7f57e698b740 1 Processor -- start 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e698b740 1 -- start start 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e698b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e698b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e698b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e698b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b370cb40 con 0x5574b37a8d80 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e698b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b36b5d80 con 0x5574b37a96a0 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e698b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b3708be0 con 0x5574b37b2280 2024-03-09T00:04:44.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e234f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38466/0 (socket says 172.21.15.22:38466) 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 -- 172.21.15.22:0/2043429205 learned_addr learned my addr 172.21.15.22:0/2043429205 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57d2ffd640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f57c80a2970 0x7f57c80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 -- 172.21.15.22:0/2043429205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 msgr2=0x5574b37b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 -- 172.21.15.22:0/2043429205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 msgr2=0x5574b37a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e134d640 1 -- 172.21.15.22:0/2043429205 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b34e97e0 con 0x5574b37b2280 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.251+0000 7f57e234f640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f57c80a2970 0x7f57c80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57d2ffd640 1 -- 172.21.15.22:0/698768861 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f57d80cf3b0 con 0x7f57dc1003d0 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57e134d640 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b4670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc035e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db87bc6d6aabf1d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57c7fff640 1 -- 172.21.15.22:0/2043429205 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc039040 con 0x5574b37b2280 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57c7fff640 1 -- 172.21.15.22:0/2043429205 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57cc034640 con 0x5574b37b2280 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57c7fff640 1 -- 172.21.15.22:0/2043429205 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc02ca50 con 0x5574b37b2280 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57e698b740 1 -- 172.21.15.22:0/2043429205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 msgr2=0x5574b37b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57e234f640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f57c80a2970 0x7f57c80a4e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57d4005fd0 tx=0x7f57d4004a40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.252+0000 7f57e698b740 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b4670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc035e70 comp rx=0 tx=0).stop 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 -- 172.21.15.22:0/2043429205 shutdown_connections 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b4670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 --2- 172.21.15.22:0/2043429205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 -- 172.21.15.22:0/2043429205 >> 172.21.15.22:0/2043429205 conn(0x5574b37a34b0 msgr2=0x5574b37a3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 -- 172.21.15.22:0/2043429205 shutdown_connections 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 -- 172.21.15.22:0/2043429205 wait complete. 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 -- start start 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 Processor -- start 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37b2280 0x5574b37af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.253+0000 7f57e698b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b37b5010 con 0x5574b37a8d80 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e698b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b36c91c0 con 0x5574b37b2280 2024-03-09T00:04:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e698b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b37b5190 con 0x5574b37a96a0 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e698b740 10 client.0 _ll_register_callbacks cb 0x5574b36b87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e1b4e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e1b4e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47932/0 (socket says 172.21.15.22:47932) 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e1b4e640 1 -- 172.21.15.22:0/2680684720 learned_addr learned my addr 172.21.15.22:0/2680684720 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e134d640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e234f640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37b2280 0x5574b37af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e134d640 1 -- 172.21.15.22:0/2680684720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37b2280 msgr2=0x5574b37af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e134d640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37b2280 0x5574b37af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e134d640 1 -- 172.21.15.22:0/2680684720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 msgr2=0x5574b37b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e134d640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e134d640 1 -- 172.21.15.22:0/2680684720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574b3549f70 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e134d640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37aefc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57cc037040 tx=0x7f57cc02c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdef4215d5fe32f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57c57fa640 1 -- 172.21.15.22:0/2680684720 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc039040 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57c57fa640 1 -- 172.21.15.22:0/2680684720 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f57cc004050 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.254+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57cc034640 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57c57fa640 1 -- 172.21.15.22:0/2680684720 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57cc059050 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 10 client.28195 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5574b33e1170 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57c57fa640 1 -- 172.21.15.22:0/2680684720 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57cc034640 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57c57fa640 1 -- 172.21.15.22:0/2680684720 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f57cc0035f0 con 0x5574b37a96a0 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 10 client.28195 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 10 client.28195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:44.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 20 client.28195 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b37c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 4 client.28195 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574b36e2960 con 0x5574b37c2140 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b37c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57c57fa640 10 client.28195 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.255+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b37c4530 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb83e66b330d71b4 server_cookie=81cc417e0a90b91b in_seq=0 out_seq=0 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57c57fa640 1 -- 172.21.15.22:0/2680684720 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574b36e2960 con 0x5574b37c2140 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57c57fa640 10 client.28195 handle_command_reply: tid=0 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 client.28195 shutdown 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 msgr2=0x5574b37aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37aefc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57cc037040 tx=0x7f57cc02c5b0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 shutdown_connections 2024-03-09T00:04:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b37c4530 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37b2280 0x5574b37af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57c57fa640 0 client.28195 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37aefc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 --2- 172.21.15.22:0/2680684720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 >> 172.21.15.22:0/2680684720 conn(0x5574b37a34b0 msgr2=0x5574b37a5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 shutdown_connections 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.256+0000 7f57e698b740 1 -- 172.21.15.22:0/2680684720 wait complete. 2024-03-09T00:04:44.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.257+0000 7f57e698b740 20 client.28195 trim_cache size 0 max 16384 2024-03-09T00:04:44.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.272+0000 7f57e698b740 1 Processor -- start 2024-03-09T00:04:44.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.272+0000 7f57e698b740 1 -- start start 2024-03-09T00:04:44.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e698b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a8d80 0x5574b37b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e698b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e698b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37b2280 0x5574b37aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e698b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1b0bd0 con 0x5574b37b2280 2024-03-09T00:04:44.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e698b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1ae770 con 0x5574b37a8d80 2024-03-09T00:04:44.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e698b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1ac830 con 0x5574b37a96a0 2024-03-09T00:04:44.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e234f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37b2280 0x5574b37aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a8d80 0x5574b37b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a8d80 0x5574b37b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58950/0 (socket says 172.21.15.22:58950) 2024-03-09T00:04:44.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e134d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 -- 172.21.15.22:0/2105452939 learned_addr learned my addr 172.21.15.22:0/2105452939 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e234f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37b2280 0x5574b37aa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47940/0 (socket says 172.21.15.22:47940) 2024-03-09T00:04:44.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 -- 172.21.15.22:0/2105452939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 msgr2=0x5574b37b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2105452939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 -- 172.21.15.22:0/2105452939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37b2280 msgr2=0x5574b37aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2105452939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37b2280 0x5574b37aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 -- 172.21.15.22:0/2105452939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b37b5130 con 0x5574b37a8d80 2024-03-09T00:04:44.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.273+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2105452939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a8d80 0x5574b37b16f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f57d8097a70 tx=0x7f57d8086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73cc498c82b1e723 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.274+0000 7f57c77fe640 1 -- 172.21.15.22:0/2105452939 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8009510 con 0x5574b37a8d80 2024-03-09T00:04:44.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.274+0000 7f57c77fe640 1 -- 172.21.15.22:0/2105452939 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57d80cce50 con 0x5574b37a8d80 2024-03-09T00:04:44.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.274+0000 7f57c77fe640 1 -- 172.21.15.22:0/2105452939 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80cf1f0 con 0x5574b37a8d80 2024-03-09T00:04:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.274+0000 7f57e698b740 1 -- 172.21.15.22:0/2105452939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a8d80 msgr2=0x5574b37b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 --2- 172.21.15.22:0/2105452939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a8d80 0x5574b37b16f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f57d8097a70 tx=0x7f57d8086040 comp rx=0 tx=0).stop 2024-03-09T00:04:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 -- 172.21.15.22:0/2105452939 shutdown_connections 2024-03-09T00:04:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 --2- 172.21.15.22:0/2105452939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37b2280 0x5574b37aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 --2- 172.21.15.22:0/2105452939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37a96a0 0x5574b37b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 --2- 172.21.15.22:0/2105452939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a8d80 0x5574b37b16f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 -- 172.21.15.22:0/2105452939 >> 172.21.15.22:0/2105452939 conn(0x5574b37a34b0 msgr2=0x5574b37a3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 -- 172.21.15.22:0/2105452939 shutdown_connections 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 -- 172.21.15.22:0/2105452939 wait complete. 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 -- start start 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 Processor -- start 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.275+0000 7f57e698b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e698b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1ace30 con 0x5574b37a8d80 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e698b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1ae110 con 0x5574b37a96a0 2024-03-09T00:04:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e698b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1ad450 con 0x5574b37b2280 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e698b740 10 client.0 _ll_register_callbacks cb 0x7f57d8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e1b4e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58960/0 (socket says 172.21.15.22:58960) 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 -- 172.21.15.22:0/2207433012 learned_addr learned my addr 172.21.15.22:0/2207433012 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e234f640 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 -- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 msgr2=0x5574b37b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 -- 172.21.15.22:0/2207433012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 msgr2=0x5574b37b93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 -- 172.21.15.22:0/2207433012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574b37b7ca0 con 0x5574b37a96a0 2024-03-09T00:04:44.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e134d640 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b9910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x5574b37b0590 tx=0x7f57cc005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7e5d6abc553032 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57c57fa640 1 -- 172.21.15.22:0/2207433012 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc039040 con 0x5574b37a96a0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57d80cce50 con 0x5574b37a96a0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57c57fa640 1 -- 172.21.15.22:0/2207433012 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f57cc034440 con 0x5574b37a96a0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.276+0000 7f57c57fa640 1 -- 172.21.15.22:0/2207433012 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57cc06b050 con 0x5574b37a96a0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 10 client.28106 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5574b3819ff0 con 0x5574b37a96a0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57c57fa640 1 -- 172.21.15.22:0/2207433012 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57d80cce50 con 0x5574b37a96a0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57c57fa640 1 -- 172.21.15.22:0/2207433012 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f57cc003b60 con 0x5574b37a96a0 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 10 client.28106 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 10 client.28106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:44.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 20 client.28106 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b381c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 4 client.28106 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5574b36ce080 con 0x5574b37c2140 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b381c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57c57fa640 10 client.28106 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.277+0000 7f57e1b4e640 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b381c720 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f2214a9509756b server_cookie=87cc7b2ee426ff6d in_seq=0 out_seq=0 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.279+0000 7f57c57fa640 1 -- 172.21.15.22:0/2207433012 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5574b36ce080 con 0x5574b37c2140 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.279+0000 7f57c57fa640 10 client.28106 handle_command_reply: tid=0 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.279+0000 7f57e698b740 1 client.28106 shutdown 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 msgr2=0x5574b37b9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b9910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x5574b37b0590 tx=0x7f57cc005d20 comp rx=0 tx=0).stop 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 shutdown_connections 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5574b37c2140 0x5574b381c720 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5574b37b2280 0x5574b37b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574b37a96a0 0x5574b37b9910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 --2- 172.21.15.22:0/2207433012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5574b37a8d80 0x5574b37b93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57c57fa640 0 client.28106 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 >> 172.21.15.22:0/2207433012 conn(0x5574b37a34b0 msgr2=0x5574b37ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 shutdown_connections 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 1 -- 172.21.15.22:0/2207433012 wait complete. 2024-03-09T00:04:44.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e698b740 20 client.28106 trim_cache size 0 max 16384 2024-03-09T00:04:44.283 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:44.283 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:04:44.283 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "16311b0d-df09-465b-bfbf-5a6a00d7bee0", 2024-03-09T00:04:44.283 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:04:44.283 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f57c80a2970 msgr2=0x7f57c80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f57c80a2970 0x7f57c80a4e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57d4005fd0 tx=0x7f57d4004a40 comp rx=0 tx=0).stop 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 msgr2=0x7f57dc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 0x7f57dc198990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57d8005fd0 tx=0x7f57d800ee50 comp rx=0 tx=0).stop 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 shutdown_connections 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f57c80a2970 0x7f57c80a4e30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57dc1994b0 0x7f57dc1a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57dc1054a0 0x7f57dc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 --2- 172.21.15.22:0/698768861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57dc1003d0 0x7f57dc198990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.280+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 >> 172.21.15.22:0/698768861 conn(0x7f57dc0772f0 msgr2=0x7f57dc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.281+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 shutdown_connections 2024-03-09T00:04:44.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.281+0000 7f57e3db2640 1 -- 172.21.15.22:0/698768861 wait complete. 2024-03-09T00:04:44.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:04:44.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:04:44.630 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:44 smithi192 sudo[72509]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:44.630 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:44 smithi192 sudo[72509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:44.630 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:44 smithi192 sudo[72509]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:44.630 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:04:44 smithi192 sudo[72509]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:44.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101178]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:44.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:44.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101178]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:44.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101178]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 -- 172.21.15.22:0/4234956305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 msgr2=0x7fe9e41018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 --2- 172.21.15.22:0/4234956305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e41018d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009900 tx=0x7fe9d402bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 -- 172.21.15.22:0/4234956305 shutdown_connections 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 --2- 172.21.15.22:0/4234956305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e41065c0 0x7fe9e410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 --2- 172.21.15.22:0/4234956305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e4101ea0 0x7fe9e4105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 --2- 172.21.15.22:0/4234956305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e41018d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 -- 172.21.15.22:0/4234956305 >> 172.21.15.22:0/4234956305 conn(0x7fe9e40786b0 msgr2=0x7fe9e4078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 -- 172.21.15.22:0/4234956305 shutdown_connections 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.719+0000 7fe9ea560640 1 -- 172.21.15.22:0/4234956305 wait complete. 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 Processor -- start 2024-03-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 -- start start 2024-03-09T00:04:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e41065c0 0x7fe9e4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e4074070 0x7fe9e41a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4111680 con 0x7fe9e41014f0 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4111500 con 0x7fe9e4074070 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.720+0000 7fe9ea560640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4111800 con 0x7fe9e41065c0 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e40735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47978/0 (socket says 172.21.15.22:47978) 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 -- 172.21.15.22:0/2120280556 learned_addr learned my addr 172.21.15.22:0/2120280556 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e4074070 0x7fe9e41a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 -- 172.21.15.22:0/2120280556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e41065c0 msgr2=0x7fe9e4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e41065c0 0x7fe9e4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 -- 172.21.15.22:0/2120280556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e4074070 msgr2=0x7fe9e41a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e4074070 0x7fe9e41a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 -- 172.21.15.22:0/2120280556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e41a64b0 con 0x7fe9e41014f0 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e40735f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006010 tx=0x7fe9d402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e17fa640 1 -- 172.21.15.22:0/2120280556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4041070 con 0x7fe9e41014f0 2024-03-09T00:04:44.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e17fa640 1 -- 172.21.15.22:0/2120280556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9d4004450 con 0x7fe9e41014f0 2024-03-09T00:04:44.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e41a6740 con 0x7fe9e41014f0 2024-03-09T00:04:44.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9e17fa640 1 -- 172.21.15.22:0/2120280556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4003e00 con 0x7fe9e41014f0 2024-03-09T00:04:44.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.721+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e41a6c30 con 0x7fe9e41014f0 2024-03-09T00:04:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.724+0000 7fe9e17fa640 1 -- 172.21.15.22:0/2120280556 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe9d4044070 con 0x7fe9e41014f0 2024-03-09T00:04:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.724+0000 7fe9ed216740 1 Processor -- start 2024-03-09T00:04:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.724+0000 7fe9ed216740 1 -- start start 2024-03-09T00:04:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.724+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ac7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.724+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac8080 0x5595f0ad0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ad0c60 0x5595f0ad3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9ed216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a2ae00 con 0x5595f0ad0c60 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9ed216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f09d3b10 con 0x5595f0ac7760 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9ed216740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a26ea0 con 0x5595f0ac8080 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e17fa640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a29a0 0x7fe9cc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac8080 0x5595f0ad0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ac7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ad0c60 0x5595f0ad3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ac7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:58978/0 (socket says 172.21.15.22:58978) 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 -- 172.21.15.22:0/614171648 learned_addr learned my addr 172.21.15.22:0/614171648 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ad0c60 0x5595f0ad3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47986/0 (socket says 172.21.15.22:47986) 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a29a0 0x7fe9cc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 -- 172.21.15.22:0/614171648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac8080 msgr2=0x5595f0ad0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/614171648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac8080 0x5595f0ad0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e17fa640 1 -- 172.21.15.22:0/2120280556 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe9d40f0c40 con 0x7fe9e41014f0 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 -- 172.21.15.22:0/614171648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ad0c60 msgr2=0x5595f0ad3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/614171648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ad0c60 0x5595f0ad3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 -- 172.21.15.22:0/614171648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f08097d0 con 0x5595f0ac7760 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/614171648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ac7b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc011da0 tx=0x7fe9dc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3ad3bf775ef3c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a29a0 0x7fe9cc0a4e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00fc90 tx=0x7fe9dc01e000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9ca7fc640 1 -- 172.21.15.22:0/614171648 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc022070 con 0x5595f0ac7760 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9ca7fc640 1 -- 172.21.15.22:0/614171648 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc009dc0 con 0x5595f0ac7760 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.725+0000 7fe9ca7fc640 1 -- 172.21.15.22:0/614171648 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc004270 con 0x5595f0ac7760 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 -- 172.21.15.22:0/614171648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 msgr2=0x5595f0ac7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 --2- 172.21.15.22:0/614171648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ac7b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc011da0 tx=0x7fe9dc00f790 comp rx=0 tx=0).stop 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 -- 172.21.15.22:0/614171648 shutdown_connections 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 --2- 172.21.15.22:0/614171648 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ad0c60 0x5595f0ad3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 --2- 172.21.15.22:0/614171648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac8080 0x5595f0ad0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 --2- 172.21.15.22:0/614171648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ac7b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 -- 172.21.15.22:0/614171648 >> 172.21.15.22:0/614171648 conn(0x5595f0ac1e90 msgr2=0x5595f0ac2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 -- 172.21.15.22:0/614171648 shutdown_connections 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.726+0000 7fe9ed216740 1 -- 172.21.15.22:0/614171648 wait complete. 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 -- start start 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 Processor -- start 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ad03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0acd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0acdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0ad39f0 con 0x5595f0ac8080 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f09e70a0 con 0x5595f0ac7760 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0ad3b70 con 0x5595f0ad0c60 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9ed216740 10 client.0 _ll_register_callbacks cb 0x5595f09e6a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0acd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0acd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47996/0 (socket says 172.21.15.22:47996) 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 -- 172.21.15.22:0/985445656 learned_addr learned my addr 172.21.15.22:0/985445656 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0acdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ad03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 -- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 msgr2=0x5595f0acdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0acdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 -- 172.21.15.22:0/985445656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 msgr2=0x5595f0ad03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ad03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.727+0000 7fe9e37fe640 1 -- 172.21.15.22:0/985445656 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595f073f3f0 con 0x5595f0ac8080 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0acd750 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80116f0 tx=0x7fe9d8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ebada60dfff573 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9c37fe640 1 -- 172.21.15.22:0/985445656 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8005020 con 0x5595f0ac8080 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9c37fe640 1 -- 172.21.15.22:0/985445656 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe9d80051c0 con 0x5595f0ac8080 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9dc009dc0 con 0x5595f0ac8080 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9c37fe640 1 -- 172.21.15.22:0/985445656 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d8024050 con 0x5595f0ac8080 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 10 client.18783 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5595f06ff0f0 con 0x5595f0ac8080 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9c37fe640 1 -- 172.21.15.22:0/985445656 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc009dc0 con 0x5595f0ac8080 2024-03-09T00:04:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9c37fe640 1 -- 172.21.15.22:0/985445656 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9d8003680 con 0x5595f0ac8080 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 10 client.18783 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 10 client.18783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 20 client.18783 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0ae0710 0x5595f0ae2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 4 client.18783 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.728+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595f0a38340 con 0x5595f0ae0710 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.729+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0ae0710 0x5595f0ae2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.729+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0ae0710 0x5595f0ae2b00 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a035f893edf63c5 server_cookie=a6d015e9335f48a6 in_seq=0 out_seq=0 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.729+0000 7fe9c37fe640 10 client.18783 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.729+0000 7fe9c37fe640 1 -- 172.21.15.22:0/985445656 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595f0a38340 con 0x5595f0ae0710 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.729+0000 7fe9c37fe640 10 client.18783 handle_command_reply: tid=0 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 client.18783 shutdown 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 msgr2=0x5595f0acd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0acd750 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80116f0 tx=0x7fe9d8011af0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 shutdown_connections 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0ae0710 0x5595f0ae2b00 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0acdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0acd750 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9c37fe640 0 client.18783 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 --2- 172.21.15.22:0/985445656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac7760 0x5595f0ad03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 >> 172.21.15.22:0/985445656 conn(0x5595f0ac1e90 msgr2=0x5595f0ac3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 shutdown_connections 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 1 -- 172.21.15.22:0/985445656 wait complete. 2024-03-09T00:04:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.730+0000 7fe9ed216740 20 client.18783 trim_cache size 0 max 16384 2024-03-09T00:04:44.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1043210 9.92 MB/sec execute 412 sec latency 241.179 ms 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 Processor -- start 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 -- start start 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac7760 0x5595f0acfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0ad03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ad0c60 0x5595f0aca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41b2e10 con 0x5595f0ac8080 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41b09b0 con 0x5595f0ad0c60 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9ed216740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41aea70 con 0x5595f0ac7760 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0ad03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0ad03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47998/0 (socket says 172.21.15.22:47998) 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 -- 172.21.15.22:0/4024263981 learned_addr learned my addr 172.21.15.22:0/4024263981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac7760 0x5595f0acfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ad0c60 0x5595f0aca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 -- 172.21.15.22:0/4024263981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac7760 msgr2=0x5595f0acfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac7760 0x5595f0acfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 -- 172.21.15.22:0/4024263981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ad0c60 msgr2=0x5595f0aca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ad0c60 0x5595f0aca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 -- 172.21.15.22:0/4024263981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f0ae2e70 con 0x5595f0ac8080 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac7760 0x5595f0acfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.746+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0ad03c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8003b20 tx=0x7fe9d80092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4899b751a579bca server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9c9ffb640 1 -- 172.21.15.22:0/4024263981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8000e60 con 0x5595f0ac8080 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9c9ffb640 1 -- 172.21.15.22:0/4024263981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9d8003e90 con 0x5595f0ac8080 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9c9ffb640 1 -- 172.21.15.22:0/4024263981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8004a00 con 0x5595f0ac8080 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 -- 172.21.15.22:0/4024263981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 msgr2=0x5595f0ad03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0ad03c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8003b20 tx=0x7fe9d80092b0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 -- 172.21.15.22:0/4024263981 shutdown_connections 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ad0c60 0x5595f0aca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac8080 0x5595f0ad03c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 --2- 172.21.15.22:0/4024263981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ac7760 0x5595f0acfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 -- 172.21.15.22:0/4024263981 >> 172.21.15.22:0/4024263981 conn(0x5595f0ac1e90 msgr2=0x5595f0acc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:44.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 -- 172.21.15.22:0/4024263981 shutdown_connections 2024-03-09T00:04:44.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.747+0000 7fe9ed216740 1 -- 172.21.15.22:0/4024263981 wait complete. 2024-03-09T00:04:44.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 -- start start 2024-03-09T00:04:44.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 Processor -- start 2024-03-09T00:04:44.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac7760 0x5595f0ad48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac8080 0x5595f0ad4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0ad5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41af070 con 0x5595f0ac7760 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41b0350 con 0x5595f0ac8080 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41af690 con 0x5595f0ad0c60 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9ed216740 10 client.0 _ll_register_callbacks cb 0x7fe9d8009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac7760 0x5595f0ad48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9e8afd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0ad5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac7760 0x5595f0ad48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48004/0 (socket says 172.21.15.22:48004) 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9e3fff640 1 -- 172.21.15.22:0/1491547892 learned_addr learned my addr 172.21.15.22:0/1491547892 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.748+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac8080 0x5595f0ad4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9e8afd640 1 -- 172.21.15.22:0/1491547892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac8080 msgr2=0x5595f0ad4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac8080 0x5595f0ad4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9e8afd640 1 -- 172.21.15.22:0/1491547892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac7760 msgr2=0x5595f0ad48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac7760 0x5595f0ad48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9e8afd640 1 -- 172.21.15.22:0/1491547892 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595f0b36d20 con 0x5595f0ad0c60 2024-03-09T00:04:44.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac7760 0x5595f0ad48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:44.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9e8afd640 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0ad5330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc006010 tx=0x7fe9dc009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c335af54e63f7fc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:44.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:44.749+0000 7fe9c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:04:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101181]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101181]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:44.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:44 smithi022 sudo[101181]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:45.268 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:44 smithi192 sudo[72512]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:45.268 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:44 smithi192 sudo[72512]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:45.268 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:44 smithi192 sudo[72512]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:45.268 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 sudo[72512]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.284+0000 7fe9c37fe640 1 -- 172.21.15.22:0/1491547892 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc022040 con 0x5595f0ad0c60 2024-03-09T00:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.284+0000 7fe9c37fe640 1 -- 172.21.15.22:0/1491547892 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe9dc003d00 con 0x5595f0ad0c60 2024-03-09T00:04:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.284+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595f0a32690 con 0x5595f0ad0c60 2024-03-09T00:04:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.296+0000 7fe9c37fe640 1 -- 172.21.15.22:0/1491547892 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9dc0050c0 con 0x5595f0ad0c60 2024-03-09T00:04:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.296+0000 7fe9ed216740 10 client.28219 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.296+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5595f0b38030 con 0x5595f0ad0c60 2024-03-09T00:04:45.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9c37fe640 1 -- 172.21.15.22:0/1491547892 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5595f0a32690 con 0x5595f0ad0c60 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9c37fe640 1 -- 172.21.15.22:0/1491547892 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9dc036070 con 0x5595f0ad0c60 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9ed216740 10 client.28219 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9ed216740 10 client.28219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9ed216740 20 client.28219 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9ed216740 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0b385c0 0x5595f0b3a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9ed216740 4 client.28219 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.300+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595f09b04e0 con 0x5595f0b385c0 2024-03-09T00:04:45.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.301+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0b385c0 0x5595f0b3a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.306+0000 7fe9c37fe640 10 client.28219 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.306+0000 7fe9e3fff640 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0b385c0 0x5595f0b3a9b0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e2c458c809ca0c server_cookie=ff944111be2add80 in_seq=0 out_seq=0 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.313+0000 7fe9c37fe640 1 -- 172.21.15.22:0/1491547892 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5595f09b04e0 con 0x5595f0b385c0 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.313+0000 7fe9c37fe640 10 client.28219 handle_command_reply: tid=0 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.313+0000 7fe9ed216740 1 client.28219 shutdown 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 msgr2=0x5595f0ad5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0ad5330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc006010 tx=0x7fe9dc009f10 comp rx=0 tx=0).stop 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 shutdown_connections 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5595f0b385c0 0x5595f0b3a9b0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595f0ad0c60 0x5595f0ad5330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595f0ac8080 0x5595f0ad4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9c37fe640 0 client.28219 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 --2- 172.21.15.22:0/1491547892 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595f0ac7760 0x5595f0ad48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 >> 172.21.15.22:0/1491547892 conn(0x5595f0ac1e90 msgr2=0x5595f0ac85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 shutdown_connections 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 1 -- 172.21.15.22:0/1491547892 wait complete. 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.314+0000 7fe9ed216740 20 client.28219 trim_cache size 0 max 16384 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:45.310 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:04:45.311 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:04:45.311 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a29a0 msgr2=0x7fe9cc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a29a0 0x7fe9cc0a4e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00fc90 tx=0x7fe9dc01e000 comp rx=0 tx=0).stop 2024-03-09T00:04:45.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 msgr2=0x7fe9e40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e40735f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006010 tx=0x7fe9d402cb30 comp rx=0 tx=0).stop 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 shutdown_connections 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a29a0 0x7fe9cc0a4e60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e4074070 0x7fe9e41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e41065c0 0x7fe9e4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 --2- 172.21.15.22:0/2120280556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e41014f0 0x7fe9e40735f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 >> 172.21.15.22:0/2120280556 conn(0x7fe9e40786b0 msgr2=0x7fe9e40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 shutdown_connections 2024-03-09T00:04:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.315+0000 7fe9ea560640 1 -- 172.21.15.22:0/2120280556 wait complete. 2024-03-09T00:04:45.324 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:04:45.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.433+0000 7fca3239e640 1 -- 172.21.15.22:0/84831766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102560 msgr2=0x7fca2c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.433+0000 7fca3239e640 1 --2- 172.21.15.22:0/84831766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102560 0x7fca2c102940 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fca24009ca0 tx=0x7fca2402c0f0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 -- 172.21.15.22:0/84831766 shutdown_connections 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 --2- 172.21.15.22:0/84831766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2c10b930 0x7fca2c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 --2- 172.21.15.22:0/84831766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca2c102e80 0x7fca2c10b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 --2- 172.21.15.22:0/84831766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102560 0x7fca2c102940 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 -- 172.21.15.22:0/84831766 >> 172.21.15.22:0/84831766 conn(0x7fca2c0772f0 msgr2=0x7fca2c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 -- 172.21.15.22:0/84831766 shutdown_connections 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 -- 172.21.15.22:0/84831766 wait complete. 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 Processor -- start 2024-03-09T00:04:45.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 -- start start 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca2c102560 0x7fca2c1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102e80 0x7fca2c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.434+0000 7fca3239e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2c10b930 0x7fca2c100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca3239e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1116f0 con 0x7fca2c102e80 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca3239e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c111570 con 0x7fca2c10b930 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca3239e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c111870 con 0x7fca2c102560 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102e80 0x7fca2c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102e80 0x7fca2c1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48034/0 (socket says 172.21.15.22:48034) 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 -- 172.21.15.22:0/124413350 learned_addr learned my addr 172.21.15.22:0/124413350 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca3093b640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2c10b930 0x7fca2c100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2bfff640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca2c102560 0x7fca2c1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 -- 172.21.15.22:0/124413350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca2c102560 msgr2=0x7fca2c1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca2c102560 0x7fca2c1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 -- 172.21.15.22:0/124413350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2c10b930 msgr2=0x7fca2c100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2c10b930 0x7fca2c100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 -- 172.21.15.22:0/124413350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c1a6090 con 0x7fca2c102e80 2024-03-09T00:04:45.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca2b7fe640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102e80 0x7fca2c1006f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d950 tx=0x7fca2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca297fa640 1 -- 172.21.15.22:0/124413350 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20017030 con 0x7fca2c102e80 2024-03-09T00:04:45.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca297fa640 1 -- 172.21.15.22:0/124413350 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca20004020 con 0x7fca2c102e80 2024-03-09T00:04:45.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c1a6380 con 0x7fca2c102e80 2024-03-09T00:04:45.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca297fa640 1 -- 172.21.15.22:0/124413350 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca200047a0 con 0x7fca2c102e80 2024-03-09T00:04:45.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.435+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c1a6920 con 0x7fca2c102e80 2024-03-09T00:04:45.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.436+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c0680e0 con 0x7fca2c102e80 2024-03-09T00:04:45.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.437+0000 7fca297fa640 1 -- 172.21.15.22:0/124413350 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fca200041d0 con 0x7fca2c102e80 2024-03-09T00:04:45.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.438+0000 7fca297fa640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca140a29f0 0x7fca140a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.438+0000 7fca2bfff640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca140a29f0 0x7fca140a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.438+0000 7fca297fa640 1 -- 172.21.15.22:0/124413350 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fca200ceb00 con 0x7fca2c102e80 2024-03-09T00:04:45.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.439+0000 7fca2bfff640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca140a29f0 0x7fca140a4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca24000f80 tx=0x7fca24009360 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.443+0000 7fca297fa640 1 -- 172.21.15.22:0/124413350 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fca2009b460 con 0x7fca2c102e80 2024-03-09T00:04:45.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: pgmap v322: 129 pgs: 129 active+clean; 96 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-03-09T00:04:45.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:45.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:45.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:45.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 sudo[72515]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:45.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 sudo[72515]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:45.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 sudo[72515]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:45.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 sudo[72515]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:45.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: pgmap v322: 129 pgs: 129 active+clean; 96 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-03-09T00:04:45.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:45.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:45 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:45.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.665+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca2c1a6be0 con 0x7fca2c102e80 2024-03-09T00:04:45.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.666+0000 7fca297fa640 1 -- 172.21.15.22:0/124413350 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fca2009ad60 con 0x7fca2c102e80 2024-03-09T00:04:45.661 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:04:45.661 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:04:45.661 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:04:45.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca140a29f0 msgr2=0x7fca140a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca140a29f0 0x7fca140a4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca24000f80 tx=0x7fca24009360 comp rx=0 tx=0).stop 2024-03-09T00:04:45.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102e80 msgr2=0x7fca2c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102e80 0x7fca2c1006f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d950 tx=0x7fca2000de10 comp rx=0 tx=0).stop 2024-03-09T00:04:45.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 shutdown_connections 2024-03-09T00:04:45.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fca140a29f0 0x7fca140a4eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca2c10b930 0x7fca2c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca2c102e80 0x7fca2c1006f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 --2- 172.21.15.22:0/124413350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca2c102560 0x7fca2c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 >> 172.21.15.22:0/124413350 conn(0x7fca2c0772f0 msgr2=0x7fca2c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 shutdown_connections 2024-03-09T00:04:45.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.668+0000 7fca3239e640 1 -- 172.21.15.22:0/124413350 wait complete. 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: pgmap v322: 129 pgs: 129 active+clean; 96 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:04:45.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:45.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:45.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:45.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:45.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:45.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:45.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:45 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:45.675 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:45.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:04:45.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1044656 9.92 MB/sec execute 413 sec latency 102.763 ms 2024-03-09T00:04:45.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.786+0000 7fa960116640 1 -- 172.21.15.22:0/938421477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa958100d90 msgr2=0x7fa958104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.786+0000 7fa960116640 1 --2- 172.21.15.22:0/938421477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa958100d90 0x7fa958104dd0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa948009900 tx=0x7fa94802bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- 172.21.15.22:0/938421477 shutdown_connections 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 --2- 172.21.15.22:0/938421477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9581054b0 0x7fa95810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 --2- 172.21.15.22:0/938421477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa958100d90 0x7fa958104dd0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 --2- 172.21.15.22:0/938421477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9581003e0 0x7fa9581007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- 172.21.15.22:0/938421477 >> 172.21.15.22:0/938421477 conn(0x7fa9580773a0 msgr2=0x7fa9580777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- 172.21.15.22:0/938421477 shutdown_connections 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- 172.21.15.22:0/938421477 wait complete. 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 Processor -- start 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- start start 2024-03-09T00:04:45.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9581003e0 0x7fa95819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9581054b0 0x7fa95819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958197a10 0x7fa958197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958110540 con 0x7fa9581054b0 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581103c0 con 0x7fa958197a10 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa960116640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581106c0 con 0x7fa9581003e0 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa95d6b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9581054b0 0x7fa95819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa95d6b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9581054b0 0x7fa95819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48052/0 (socket says 172.21.15.22:48052) 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.787+0000 7fa95d6b1640 1 -- 172.21.15.22:0/4138782403 learned_addr learned my addr 172.21.15.22:0/4138782403 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95deb2640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9581003e0 0x7fa95819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958197a10 0x7fa958197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95d6b1640 1 -- 172.21.15.22:0/4138782403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9581003e0 msgr2=0x7fa95819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9581003e0 0x7fa95819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95d6b1640 1 -- 172.21.15.22:0/4138782403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958197a10 msgr2=0x7fa958197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958197a10 0x7fa958197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95d6b1640 1 -- 172.21.15.22:0/4138782403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958198530 con 0x7fa9581054b0 2024-03-09T00:04:45.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9581054b0 0x7fa95819dda0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa948036040 tx=0x7fa94802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa94effd640 1 -- 172.21.15.22:0/4138782403 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948041070 con 0x7fa9581054b0 2024-03-09T00:04:45.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa94effd640 1 -- 172.21.15.22:0/4138782403 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9480054e0 con 0x7fa9581054b0 2024-03-09T00:04:45.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9581987c0 con 0x7fa9581054b0 2024-03-09T00:04:45.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa94effd640 1 -- 172.21.15.22:0/4138782403 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948034440 con 0x7fa9581054b0 2024-03-09T00:04:45.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.788+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9581a49b0 con 0x7fa9581054b0 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.790+0000 7fa94effd640 1 -- 172.21.15.22:0/4138782403 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa948044070 con 0x7fa9581054b0 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 Processor -- start 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 -- start start 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107eb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9107a9360 con 0x55f9107eb6a0 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f910745ec0 con 0x55f9107e21a0 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa962dce740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9106eed30 con 0x55f9107e2ac0 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa94effd640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa9440a2970 0x7fa9440a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa95d6b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa95e6b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa95deb2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107eb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa95e6b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:59052/0 (socket says 172.21.15.22:59052) 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa95e6b3640 1 -- 172.21.15.22:0/2369190237 learned_addr learned my addr 172.21.15.22:0/2369190237 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.791+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107eda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48062/0 (socket says 172.21.15.22:48062) 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa9440a2970 0x7fa9440a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa94effd640 1 -- 172.21.15.22:0/4138782403 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa9480f0fa0 con 0x7fa9581054b0 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95d6b1640 1 -- 172.21.15.22:0/2369190237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 msgr2=0x55f9107eb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107eb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95d6b1640 1 -- 172.21.15.22:0/2369190237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 msgr2=0x55f9107e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95d6b1640 1 -- 172.21.15.22:0/2369190237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9106edbc0 con 0x55f9107eb6a0 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:45.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107eda90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ee400 tx=0x7fa948003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1fb52824df7d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa9440a2970 0x7fa9440a4e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa954005f40 tx=0x7fa954004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa943fff640 1 -- 172.21.15.22:0/2369190237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94804b030 con 0x55f9107eb6a0 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa943fff640 1 -- 172.21.15.22:0/2369190237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa948004c80 con 0x55f9107eb6a0 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.792+0000 7fa943fff640 1 -- 172.21.15.22:0/2369190237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9480c5ab0 con 0x55f9107eb6a0 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 -- 172.21.15.22:0/2369190237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 msgr2=0x55f9107eda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107eda90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ee400 tx=0x7fa948003990 comp rx=0 tx=0).stop 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 -- 172.21.15.22:0/2369190237 shutdown_connections 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107eda90 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107eb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 --2- 172.21.15.22:0/2369190237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 -- 172.21.15.22:0/2369190237 >> 172.21.15.22:0/2369190237 conn(0x55f9107dc8d0 msgr2=0x55f9107dccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 -- 172.21.15.22:0/2369190237 shutdown_connections 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 -- 172.21.15.22:0/2369190237 wait complete. 2024-03-09T00:04:45.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 -- start start 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.793+0000 7fa962dce740 1 Processor -- start 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa962dce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa962dce740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa962dce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107e9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa962dce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9106e44a0 con 0x55f9107eb6a0 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa962dce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f910741f60 con 0x55f9107e21a0 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa962dce740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9107a9360 con 0x55f9107e2ac0 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa962dce740 10 client.0 _ll_register_callbacks cb 0x55f91073fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48066/0 (socket says 172.21.15.22:48066) 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 -- 172.21.15.22:0/1843617907 learned_addr learned my addr 172.21.15.22:0/1843617907 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95deb2640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 -- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 msgr2=0x55f9107e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 -- 172.21.15.22:0/1843617907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 msgr2=0x55f9107e8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 -- 172.21.15.22:0/1843617907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91045a3f0 con 0x55f9107eb6a0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95deb2640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107e9180 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa954009820 tx=0x7fa954004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70a601fd8334442 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa9417fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.794+0000 7fa9417fa640 1 -- 172.21.15.22:0/1843617907 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954029070 con 0x55f9107eb6a0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa9417fa640 1 -- 172.21.15.22:0/1843617907 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa954025680 con 0x55f9107eb6a0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa948004c80 con 0x55f9107eb6a0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa9417fa640 1 -- 172.21.15.22:0/1843617907 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa954035050 con 0x55f9107eb6a0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 10 client.18807 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f9105247d0 con 0x55f9107eb6a0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa9417fa640 1 -- 172.21.15.22:0/1843617907 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa948004c80 con 0x55f9107eb6a0 2024-03-09T00:04:45.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa9417fa640 1 -- 172.21.15.22:0/1843617907 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa954022050 con 0x55f9107eb6a0 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 10 client.18807 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 10 client.18807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 20 client.18807 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f9107fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 4 client.18807 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f910753400 con 0x55f9107fb510 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.795+0000 7fa95deb2640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f9107fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa95deb2640 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f9107fd920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67dad8918216c155 server_cookie=8dde79f6dabb0377 in_seq=0 out_seq=0 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa9417fa640 10 client.18807 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa9417fa640 1 -- 172.21.15.22:0/1843617907 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f910753400 con 0x55f9107fb510 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa9417fa640 10 client.18807 handle_command_reply: tid=0 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa962dce740 1 client.18807 shutdown 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 msgr2=0x55f9107e9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa962dce740 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107e9180 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa954009820 tx=0x7fa954004a40 comp rx=0 tx=0).stop 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 shutdown_connections 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa962dce740 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f9107fd920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.796+0000 7fa962dce740 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107eb6a0 0x55f9107e9180 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.797+0000 7fa9417fa640 0 client.18807 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.797+0000 7fa962dce740 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107e2ac0 0x55f9107e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.797+0000 7fa962dce740 1 --2- 172.21.15.22:0/1843617907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.797+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 >> 172.21.15.22:0/1843617907 conn(0x55f9107dc8d0 msgr2=0x55f9107de620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.797+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 shutdown_connections 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.797+0000 7fa962dce740 1 -- 172.21.15.22:0/1843617907 wait complete. 2024-03-09T00:04:45.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.797+0000 7fa962dce740 20 client.18807 trim_cache size 0 max 16384 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 Processor -- start 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 -- start start 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f9107e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581aea50 con 0x55f9107e2ac0 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581ac5f0 con 0x55f9107e21a0 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa962dce740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581aa6b0 con 0x55f9107eb6a0 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.808+0000 7fa95d6b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107e5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48076/0 (socket says 172.21.15.22:48076) 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 -- 172.21.15.22:0/2388124580 learned_addr learned my addr 172.21.15.22:0/2388124580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95deb2640 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f9107e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 -- 172.21.15.22:0/2388124580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 msgr2=0x55f9107e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f9107e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 -- 172.21.15.22:0/2388124580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 msgr2=0x55f9107eae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107eae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 -- 172.21.15.22:0/2388124580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9107e28f0 con 0x55f9107e2ac0 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107e5960 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa9480bd0c0 tx=0x7fa948004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dee12fe7ac076f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa9437fe640 1 -- 172.21.15.22:0/2388124580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94804b040 con 0x55f9107e2ac0 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa9437fe640 1 -- 172.21.15.22:0/2388124580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa948004c80 con 0x55f9107e2ac0 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.809+0000 7fa9437fe640 1 -- 172.21.15.22:0/2388124580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9480c1460 con 0x55f9107e2ac0 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- 172.21.15.22:0/2388124580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 msgr2=0x55f9107e5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107e5960 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa9480bd0c0 tx=0x7fa948004ee0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- 172.21.15.22:0/2388124580 shutdown_connections 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f9107e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107e5960 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 --2- 172.21.15.22:0/2388124580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107eae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- 172.21.15.22:0/2388124580 >> 172.21.15.22:0/2388124580 conn(0x55f9107dc8d0 msgr2=0x55f9107de5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- 172.21.15.22:0/2388124580 shutdown_connections 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- 172.21.15.22:0/2388124580 wait complete. 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- start start 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 Processor -- start 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f91084bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581aacb0 con 0x55f9107e2ac0 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581abf90 con 0x55f9107e21a0 2024-03-09T00:04:45.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581ab2d0 con 0x55f9107eb6a0 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.810+0000 7fa962dce740 10 client.0 _ll_register_callbacks cb 0x7fa9480b8fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48082/0 (socket says 172.21.15.22:48082) 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 -- 172.21.15.22:0/813248990 learned_addr learned my addr 172.21.15.22:0/813248990 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95deb2640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95e6b3640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f91084bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 -- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 msgr2=0x55f91084bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f91084bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 -- 172.21.15.22:0/813248990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 msgr2=0x55f9107f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 -- 172.21.15.22:0/813248990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91084c120 con 0x55f9107e2ac0 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95deb2640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa95d6b1640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107f1c50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ee8f0 tx=0x7fa9480f1350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8247628832c81d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa9417fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa9417fa640 1 -- 172.21.15.22:0/813248990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94804b070 con 0x55f9107e2ac0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa948004c80 con 0x55f9107e2ac0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa9417fa640 1 -- 172.21.15.22:0/813248990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fa9480bd620 con 0x55f9107e2ac0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa9417fa640 1 -- 172.21.15.22:0/813248990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9480bcf00 con 0x55f9107e2ac0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa962dce740 10 client.18813 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.811+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f9108534a0 con 0x55f9107e2ac0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa9417fa640 1 -- 172.21.15.22:0/813248990 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa948004c80 con 0x55f9107e2ac0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa9417fa640 1 -- 172.21.15.22:0/813248990 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa9480c1020 con 0x55f9107e2ac0 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa962dce740 10 client.18813 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa962dce740 10 client.18813 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa962dce740 20 client.18813 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa962dce740 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f910855660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa962dce740 4 client.18813 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f91074d750 con 0x55f9107fb510 2024-03-09T00:04:45.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa95deb2640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f910855660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa9417fa640 10 client.18813 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.812+0000 7fa95deb2640 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f910855660 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3af99857451b8f96 server_cookie=c8cda272c594eb0f in_seq=0 out_seq=0 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa9417fa640 1 -- 172.21.15.22:0/813248990 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f91074d750 con 0x55f9107fb510 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa9417fa640 10 client.18813 handle_command_reply: tid=0 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 client.18813 shutdown 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 msgr2=0x55f9107f1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107f1c50 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ee8f0 tx=0x7fa9480f1350 comp rx=0 tx=0).stop 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 shutdown_connections 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f9107fb510 0x55f910855660 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9107eb6a0 0x55f91084bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f9107e2ac0 0x55f9107f1c50 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 --2- 172.21.15.22:0/813248990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9107e21a0 0x55f9107f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa9417fa640 0 client.18813 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:04:45.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 >> 172.21.15.22:0/813248990 conn(0x55f9107dc8d0 msgr2=0x55f9107db100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 shutdown_connections 2024-03-09T00:04:45.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 1 -- 172.21.15.22:0/813248990 wait complete. 2024-03-09T00:04:45.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa962dce740 20 client.18813 trim_cache size 0 max 16384 2024-03-09T00:04:45.815 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa9440a2970 msgr2=0x7fa9440a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa9440a2970 0x7fa9440a4e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa954005f40 tx=0x7fa954004ab0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9581054b0 msgr2=0x7fa95819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9581054b0 0x7fa95819dda0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa948036040 tx=0x7fa94802cbf0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 shutdown_connections 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa9440a2970 0x7fa9440a4e30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958197a10 0x7fa958197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa9581054b0 0x7fa95819dda0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 --2- 172.21.15.22:0/4138782403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9581003e0 0x7fa95819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 >> 172.21.15.22:0/4138782403 conn(0x7fa9580773a0 msgr2=0x7fa9580687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.813+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 shutdown_connections 2024-03-09T00:04:45.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.814+0000 7fa960116640 1 -- 172.21.15.22:0/4138782403 wait complete. 2024-03-09T00:04:45.820 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:45.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.931+0000 7f7916497640 1 -- 172.21.15.22:0/2335624790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f791010b850 msgr2=0x7f791010dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.931+0000 7f7916497640 1 --2- 172.21.15.22:0/2335624790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f791010b850 0x7f791010dcf0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7908009ca0 tx=0x7f790802c180 comp rx=0 tx=0).stop 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 -- 172.21.15.22:0/2335624790 shutdown_connections 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 --2- 172.21.15.22:0/2335624790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f791010b850 0x7f791010dcf0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 --2- 172.21.15.22:0/2335624790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7910102e30 0x7f791010b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 --2- 172.21.15.22:0/2335624790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7910102510 0x7f79101028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 -- 172.21.15.22:0/2335624790 >> 172.21.15.22:0/2335624790 conn(0x7f79100773a0 msgr2=0x7f79100777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 -- 172.21.15.22:0/2335624790 shutdown_connections 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 -- 172.21.15.22:0/2335624790 wait complete. 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 Processor -- start 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.932+0000 7f7916497640 1 -- start start 2024-03-09T00:04:45.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7910102510 0x7f791019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7910102e30 0x7f791019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f791010b850 0x7f7910197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7910111630 con 0x7f7910102e30 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101114b0 con 0x7f791010b850 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101117b0 con 0x7f7910102510 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7910102e30 0x7f791019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7910102510 0x7f791019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7910102e30 0x7f791019dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48096/0 (socket says 172.21.15.22:48096) 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 -- 172.21.15.22:0/3571261618 learned_addr learned my addr 172.21.15.22:0/3571261618 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7914a34640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f791010b850 0x7f7910197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 -- 172.21.15.22:0/3571261618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7910102510 msgr2=0x7f791019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7910102510 0x7f791019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 -- 172.21.15.22:0/3571261618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f791010b850 msgr2=0x7f7910197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f791010b850 0x7f7910197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 -- 172.21.15.22:0/3571261618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79101980b0 con 0x7f7910102e30 2024-03-09T00:04:45.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790ffff640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7910102510 0x7f791019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:45.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790f7fe640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7910102e30 0x7f791019dc70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9f0 tx=0x7f790400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790d7fa640 1 -- 172.21.15.22:0/3571261618 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790400cd80 con 0x7f7910102e30 2024-03-09T00:04:45.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790d7fa640 1 -- 172.21.15.22:0/3571261618 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7904005500 con 0x7f7910102e30 2024-03-09T00:04:45.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79101983a0 con 0x7f7910102e30 2024-03-09T00:04:45.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f790d7fa640 1 -- 172.21.15.22:0/3571261618 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904010690 con 0x7f7910102e30 2024-03-09T00:04:45.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.933+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79101a45f0 con 0x7f7910102e30 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.936+0000 7f790d7fa640 1 -- 172.21.15.22:0/3571261618 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f79040046c0 con 0x7f7910102e30 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.936+0000 7f791907a740 1 Processor -- start 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.936+0000 7f791907a740 1 -- start start 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.936+0000 7f791907a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae31a0 0x561d8cae3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.936+0000 7f791907a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae3ac0 0x561d8caec160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.936+0000 7f791907a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caeea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f791907a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8caaa360 con 0x561d8cae31a0 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f791907a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8ca46ec0 con 0x561d8cae3ac0 2024-03-09T00:04:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f791907a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8c9efd30 con 0x561d8caec6a0 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae31a0 0x561d8cae3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f790d7fa640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f78f80a2970 0x7f78f80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae31a0 0x561d8cae3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48106/0 (socket says 172.21.15.22:48106) 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 -- 172.21.15.22:0/3306647834 learned_addr learned my addr 172.21.15.22:0/3306647834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f790ffff640 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae3ac0 0x561d8caec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f790f7fe640 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caeea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f78f80a2970 0x7f78f80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f790d7fa640 1 -- 172.21.15.22:0/3571261618 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f79040cf5f0 con 0x7f7910102e30 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 -- 172.21.15.22:0/3306647834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 msgr2=0x561d8caeea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caeea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 -- 172.21.15.22:0/3306647834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae3ac0 msgr2=0x561d8caec160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae3ac0 0x561d8caec160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 -- 172.21.15.22:0/3306647834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d8c9eebc0 con 0x561d8cae31a0 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f790ffff640 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae3ac0 0x561d8caec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f7914a34640 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae31a0 0x561d8cae3580 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7908037040 tx=0x7f7908003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e60423169e81e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f78f67fc640 1 -- 172.21.15.22:0/3306647834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908050070 con 0x561d8cae31a0 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f78f67fc640 1 -- 172.21.15.22:0/3306647834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7908004610 con 0x561d8cae31a0 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.937+0000 7f78f67fc640 1 -- 172.21.15.22:0/3306647834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908039040 con 0x561d8cae31a0 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 -- 172.21.15.22:0/3306647834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae31a0 msgr2=0x561d8cae3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f7914a34640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f78f80a2970 0x7f78f80a4e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f790804d000 tx=0x7f7908048000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae31a0 0x561d8cae3580 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7908037040 tx=0x7f7908003b00 comp rx=0 tx=0).stop 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 -- 172.21.15.22:0/3306647834 shutdown_connections 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caeea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae3ac0 0x561d8caec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 --2- 172.21.15.22:0/3306647834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae31a0 0x561d8cae3580 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 -- 172.21.15.22:0/3306647834 >> 172.21.15.22:0/3306647834 conn(0x561d8cadd8d0 msgr2=0x561d8caddcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 -- 172.21.15.22:0/3306647834 shutdown_connections 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 -- 172.21.15.22:0/3306647834 wait complete. 2024-03-09T00:04:45.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.938+0000 7f791907a740 1 -- start start 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 1 Processor -- start 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8cae96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8c9e54a0 con 0x561d8cae3ac0 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8ca42f60 con 0x561d8cae31a0 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8caef490 con 0x561d8caec6a0 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f791907a740 10 client.0 _ll_register_callbacks cb 0x561d8ca40d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48122/0 (socket says 172.21.15.22:48122) 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 -- 172.21.15.22:0/3288566577 learned_addr learned my addr 172.21.15.22:0/3288566577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f7914a34640 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790ffff640 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8cae96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 -- 172.21.15.22:0/3288566577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 msgr2=0x561d8caea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 -- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 msgr2=0x561d8cae96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8cae96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 -- 172.21.15.22:0/3288566577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d8c75b3f0 con 0x561d8cae3ac0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f790f7fe640 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae9be0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7904097cb0 tx=0x7f790409b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a4d2cc4f84f209 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.939+0000 7f78ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f78ef7fe640 1 -- 172.21.15.22:0/3288566577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040ccdc0 con 0x561d8cae3ac0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f78ef7fe640 1 -- 172.21.15.22:0/3288566577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f79040a3b60 con 0x561d8cae3ac0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7908004610 con 0x561d8cae3ac0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f78ef7fe640 1 -- 172.21.15.22:0/3288566577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f790409f9a0 con 0x561d8cae3ac0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 10 client.18825 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561d8c8257d0 con 0x561d8cae3ac0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f78ef7fe640 1 -- 172.21.15.22:0/3288566577 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7908004610 con 0x561d8cae3ac0 2024-03-09T00:04:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f78ef7fe640 1 -- 172.21.15.22:0/3288566577 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f790409cb80 con 0x561d8cae3ac0 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 10 client.18825 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 10 client.18825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 20 client.18825 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cafc7b0 0x561d8cafeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 4 client.18825 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d8ca54400 con 0x561d8cafc7b0 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.940+0000 7f790ffff640 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cafc7b0 0x561d8cafeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.941+0000 7f78ef7fe640 10 client.18825 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.941+0000 7f790ffff640 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cafc7b0 0x561d8cafeba0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8812b797695def6a server_cookie=17ace7e3d2a50dd8 in_seq=0 out_seq=0 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.941+0000 7f78ef7fe640 1 -- 172.21.15.22:0/3288566577 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d8ca54400 con 0x561d8cafc7b0 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.941+0000 7f78ef7fe640 10 client.18825 handle_command_reply: tid=0 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.941+0000 7f791907a740 1 client.18825 shutdown 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.941+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 msgr2=0x561d8cae9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.941+0000 7f791907a740 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae9be0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7904097cb0 tx=0x7f790409b270 comp rx=0 tx=0).stop 2024-03-09T00:04:45.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 shutdown_connections 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cafc7b0 0x561d8cafeba0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8caea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae9be0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f78ef7fe640 0 client.18825 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 --2- 172.21.15.22:0/3288566577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8cae96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 >> 172.21.15.22:0/3288566577 conn(0x561d8cadd8d0 msgr2=0x561d8cadf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 shutdown_connections 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 1 -- 172.21.15.22:0/3288566577 wait complete. 2024-03-09T00:04:45.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.942+0000 7f791907a740 20 client.18825 trim_cache size 0 max 16384 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 Processor -- start 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 -- start start 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8caebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8cae6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101b0b80 con 0x561d8cae3ac0 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ae720 con 0x561d8cae31a0 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f791907a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ac7e0 con 0x561d8caec6a0 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48134/0 (socket says 172.21.15.22:48134) 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 -- 172.21.15.22:0/1995141092 learned_addr learned my addr 172.21.15.22:0/1995141092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790ffff640 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8caebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f7914a34640 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8cae6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 -- 172.21.15.22:0/1995141092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 msgr2=0x561d8cae6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8cae6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 -- 172.21.15.22:0/1995141092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 msgr2=0x561d8caebdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8caebdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.954+0000 7f790f7fe640 1 -- 172.21.15.22:0/1995141092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d8cadd110 con 0x561d8cae3ac0 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f790f7fe640 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae6920 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f79040049d0 tx=0x7f790400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a820d14b681cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f78f5ffb640 1 -- 172.21.15.22:0/1995141092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040b8020 con 0x561d8cae3ac0 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f78f5ffb640 1 -- 172.21.15.22:0/1995141092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79040a3ff0 con 0x561d8cae3ac0 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f78f5ffb640 1 -- 172.21.15.22:0/1995141092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040cf190 con 0x561d8cae3ac0 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 -- 172.21.15.22:0/1995141092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 msgr2=0x561d8cae6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae6920 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f79040049d0 tx=0x7f790400c7d0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 -- 172.21.15.22:0/1995141092 shutdown_connections 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8caec6a0 0x561d8cae6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8cae6920 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 --2- 172.21.15.22:0/1995141092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8cae31a0 0x561d8caebdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 -- 172.21.15.22:0/1995141092 >> 172.21.15.22:0/1995141092 conn(0x561d8cadd8d0 msgr2=0x561d8cae7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 -- 172.21.15.22:0/1995141092 shutdown_connections 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.955+0000 7f791907a740 1 -- 172.21.15.22:0/1995141092 wait complete. 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 -- start start 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 Processor -- start 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8cae31a0 0x561d8caf2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8caf25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8caec6a0 0x561d8caf2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101acde0 con 0x561d8cae3ac0 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ae0c0 con 0x561d8caec6a0 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ad400 con 0x561d8cae31a0 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f791907a740 10 client.0 _ll_register_callbacks cb 0x7f7900005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8caf25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f790f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8caf25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48148/0 (socket says 172.21.15.22:48148) 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f790f7fe640 1 -- 172.21.15.22:0/2435246010 learned_addr learned my addr 172.21.15.22:0/2435246010 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f790ffff640 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8cae31a0 0x561d8caf2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.956+0000 7f7914a34640 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8caec6a0 0x561d8caf2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f790f7fe640 1 -- 172.21.15.22:0/2435246010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8cae31a0 msgr2=0x561d8caf2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f790f7fe640 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8cae31a0 0x561d8caf2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f790f7fe640 1 -- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8caec6a0 msgr2=0x561d8caf2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f790f7fe640 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8caec6a0 0x561d8caf2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f790f7fe640 1 -- 172.21.15.22:0/2435246010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d8cb52c20 con 0x561d8cae3ac0 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f790f7fe640 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8caf25b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f790409b550 tx=0x7f790400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811f88d0f6228dc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f78ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f78ef7fe640 1 -- 172.21.15.22:0/2435246010 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040b8050 con 0x561d8cae3ac0 2024-03-09T00:04:45.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f78ef7fe640 1 -- 172.21.15.22:0/2435246010 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f79040cf390 con 0x561d8cae3ac0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79040a3ff0 con 0x561d8cae3ac0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f78ef7fe640 1 -- 172.21.15.22:0/2435246010 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f790409b390 con 0x561d8cae3ac0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 10 client.18831 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561d8cb53e80 con 0x561d8cae3ac0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f78ef7fe640 1 -- 172.21.15.22:0/2435246010 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79040a3ff0 con 0x561d8cae3ac0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f78ef7fe640 1 -- 172.21.15.22:0/2435246010 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79040bbad0 con 0x561d8cae3ac0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 10 client.18831 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 10 client.18831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 20 client.18831 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cb54330 0x561d8cb56750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 4 client.18831 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.957+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d8ca4e750 con 0x561d8cb54330 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f790ffff640 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cb54330 0x561d8cb56750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f790ffff640 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cb54330 0x561d8cb56750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=432b9bb23b4862f3 server_cookie=83993dc374e9378b in_seq=0 out_seq=0 2024-03-09T00:04:45.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f78ef7fe640 10 client.18831 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f78ef7fe640 1 -- 172.21.15.22:0/2435246010 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d8ca4e750 con 0x561d8cb54330 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f78ef7fe640 10 client.18831 handle_command_reply: tid=0 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 client.18831 shutdown 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 msgr2=0x561d8caf25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8caf25b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f790409b550 tx=0x7f790400c840 comp rx=0 tx=0).stop 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 shutdown_connections 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x561d8cb54330 0x561d8cb56750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d8caec6a0 0x561d8caf2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d8cae3ac0 0x561d8caf25b0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f78ef7fe640 0 client.18831 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 --2- 172.21.15.22:0/2435246010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561d8cae31a0 0x561d8caf2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 >> 172.21.15.22:0/2435246010 conn(0x561d8cadd8d0 msgr2=0x561d8cadf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 shutdown_connections 2024-03-09T00:04:45.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.958+0000 7f791907a740 1 -- 172.21.15.22:0/2435246010 wait complete. 2024-03-09T00:04:45.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f791907a740 20 client.18831 trim_cache size 0 max 16384 2024-03-09T00:04:45.961 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f78f80a2970 msgr2=0x7f78f80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f78f80a2970 0x7f78f80a4e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f790804d000 tx=0x7f7908048000 comp rx=0 tx=0).stop 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7910102e30 msgr2=0x7f791019dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7910102e30 0x7f791019dc70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9f0 tx=0x7f790400edf0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 shutdown_connections 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f78f80a2970 0x7f78f80a4e30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f791010b850 0x7f7910197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7910102e30 0x7f791019dc70 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 --2- 172.21.15.22:0/3571261618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7910102510 0x7f791019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 >> 172.21.15.22:0/3571261618 conn(0x7f79100773a0 msgr2=0x7f79100fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:45.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 shutdown_connections 2024-03-09T00:04:45.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:45.959+0000 7f7916497640 1 -- 172.21.15.22:0/3571261618 wait complete. 2024-03-09T00:04:45.965 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:04:45.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 -- 172.21.15.22:0/1619555429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c069b90 msgr2=0x7fd27c069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 --2- 172.21.15.22:0/1619555429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c069b90 0x7fd27c069f70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009900 tx=0x7fd26c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 -- 172.21.15.22:0/1619555429 shutdown_connections 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 --2- 172.21.15.22:0/1619555429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd27c1065a0 0x7fd27c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 --2- 172.21.15.22:0/1619555429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06a540 0x7fd27c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 --2- 172.21.15.22:0/1619555429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c069b90 0x7fd27c069f70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 -- 172.21.15.22:0/1619555429 >> 172.21.15.22:0/1619555429 conn(0x7fd27c076f90 msgr2=0x7fd27c0773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 -- 172.21.15.22:0/1619555429 shutdown_connections 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 -- 172.21.15.22:0/1619555429 wait complete. 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 Processor -- start 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.075+0000 7fd281b69640 1 -- start start 2024-03-09T00:04:46.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd281b69640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd27c069b90 0x7fd27c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd281b69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c1065a0 0x7fd27c19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd281b69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c197980 0x7fd27c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd281b69640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c111680 con 0x7fd27c197980 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd281b69640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c111500 con 0x7fd27c1065a0 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd281b69640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c111800 con 0x7fd27c069b90 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c197980 0x7fd27c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c197980 0x7fd27c197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48170/0 (socket says 172.21.15.22:48170) 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd27c069b90 0x7fd27c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27bfff640 1 -- 172.21.15.22:0/3120343143 learned_addr learned my addr 172.21.15.22:0/3120343143 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27affd640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c1065a0 0x7fd27c19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27b7fe640 1 -- 172.21.15.22:0/3120343143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c1065a0 msgr2=0x7fd27c19dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c1065a0 0x7fd27c19dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27b7fe640 1 -- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c197980 msgr2=0x7fd27c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c197980 0x7fd27c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27b7fe640 1 -- 172.21.15.22:0/3120343143 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c1984a0 con 0x7fd27c069b90 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27bfff640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c197980 0x7fd27c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd27c069b90 0x7fd27c19d7d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd26c036040 tx=0x7fd26c02cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd278ff9640 1 -- 172.21.15.22:0/3120343143 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c041070 con 0x7fd27c069b90 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c198730 con 0x7fd27c069b90 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd278ff9640 1 -- 172.21.15.22:0/3120343143 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd26c004450 con 0x7fd27c069b90 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.076+0000 7fd278ff9640 1 -- 172.21.15.22:0/3120343143 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c003d90 con 0x7fd27c069b90 2024-03-09T00:04:46.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.077+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1a4980 con 0x7fd27c069b90 2024-03-09T00:04:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.079+0000 7fd278ff9640 1 -- 172.21.15.22:0/3120343143 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd26c044070 con 0x7fd27c069b90 2024-03-09T00:04:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 Processor -- start 2024-03-09T00:04:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 -- start start 2024-03-09T00:04:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd278ff9640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2640a2970 0x7fd2640a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e431a0 0x563b32e43580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e4c6a0 0x563b32e4ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b32e0a360 con 0x563b32e4c6a0 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b32da6ec0 con 0x563b32e431a0 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd284827740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b32d4fd30 con 0x563b32e43ac0 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd278ff9640 1 -- 172.21.15.22:0/3120343143 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd26c0f0bc0 con 0x7fd27c069b90 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e431a0 0x563b32e43580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e431a0 0x563b32e43580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:59192/0 (socket says 172.21.15.22:59192) 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27b7fe640 1 -- 172.21.15.22:0/265485298 learned_addr learned my addr 172.21.15.22:0/265485298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27bfff640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2640a2970 0x7fd2640a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27affd640 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27bfff640 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e4c6a0 0x563b32e4ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27b7fe640 1 -- 172.21.15.22:0/265485298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 msgr2=0x563b32e4c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e4c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.080+0000 7fd27b7fe640 1 -- 172.21.15.22:0/265485298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e4c6a0 msgr2=0x563b32e4ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.081+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e4c6a0 0x563b32e4ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.081+0000 7fd27b7fe640 1 -- 172.21.15.22:0/265485298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b32d4ebc0 con 0x563b32e431a0 2024-03-09T00:04:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.081+0000 7fd27bfff640 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e4c6a0 0x563b32e4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.081+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e431a0 0x563b32e43580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006010 tx=0x7fd26c0b8b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3b3073d0ead300 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:46.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.081+0000 7fd27bfff640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2640a2970 0x7fd2640a4e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd2740049d0 tx=0x7fd2740115e0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:46.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 sudo[72096]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-09T00:04:46.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 sudo[72096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:46.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 sudo[72096]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:46.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:45 smithi062 sudo[72096]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:46.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.240+0000 7fd261ffb640 1 -- 172.21.15.22:0/265485298 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0d7070 con 0x563b32e431a0 2024-03-09T00:04:46.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.241+0000 7fd261ffb640 1 -- 172.21.15.22:0/265485298 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd26c004c80 con 0x563b32e431a0 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 -- 172.21.15.22:0/265485298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e431a0 msgr2=0x563b32e43580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e431a0 0x563b32e43580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006010 tx=0x7fd26c0b8b80 comp rx=0 tx=0).stop 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 -- 172.21.15.22:0/265485298 shutdown_connections 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e4c6a0 0x563b32e4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 --2- 172.21.15.22:0/265485298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e431a0 0x563b32e43580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 -- 172.21.15.22:0/265485298 >> 172.21.15.22:0/265485298 conn(0x563b32e3d8d0 msgr2=0x563b32e3dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 -- 172.21.15.22:0/265485298 shutdown_connections 2024-03-09T00:04:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 -- 172.21.15.22:0/265485298 wait complete. 2024-03-09T00:04:46.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.242+0000 7fd284827740 1 -- start start 2024-03-09T00:04:46.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 1 Processor -- start 2024-03-09T00:04:46.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e431a0 0x563b32e495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e43ac0 0x563b32e49b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e4a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b32d454a0 con 0x563b32e43ac0 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b32da2f60 con 0x563b32e4c6a0 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b32e53660 con 0x563b32e431a0 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.243+0000 7fd284827740 10 client.0 _ll_register_callbacks cb 0x563b32da0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.244+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e4a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.244+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e4a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:59202/0 (socket says 172.21.15.22:59202) 2024-03-09T00:04:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.244+0000 7fd27bfff640 1 -- 172.21.15.22:0/3339036211 learned_addr learned my addr 172.21.15.22:0/3339036211 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.245+0000 7fd27bfff640 1 -- 172.21.15.22:0/3339036211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e431a0 msgr2=0x563b32e495e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:04:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.245+0000 7fd27bfff640 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e431a0 0x563b32e495e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.245+0000 7fd27bfff640 1 -- 172.21.15.22:0/3339036211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e43ac0 msgr2=0x563b32e49b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.245+0000 7fd27bfff640 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e43ac0 0x563b32e49b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.245+0000 7fd27bfff640 1 -- 172.21.15.22:0/3339036211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b32abb3f0 con 0x563b32e4c6a0 2024-03-09T00:04:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.246+0000 7fd27bfff640 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e4a060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd27402f7c0 tx=0x7fd274009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6438d12f0b7933a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.246+0000 7fd25affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:04:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.251+0000 7fd25affd640 1 -- 172.21.15.22:0/3339036211 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27402d620 con 0x563b32e4c6a0 2024-03-09T00:04:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.251+0000 7fd25affd640 1 -- 172.21.15.22:0/3339036211 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd274009040 con 0x563b32e4c6a0 2024-03-09T00:04:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.251+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd26c004c80 con 0x563b32e4c6a0 2024-03-09T00:04:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd25affd640 1 -- 172.21.15.22:0/3339036211 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd274044050 con 0x563b32e4c6a0 2024-03-09T00:04:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 10 client.28181 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:46.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563b32b857d0 con 0x563b32e4c6a0 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd25affd640 1 -- 172.21.15.22:0/3339036211 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd26c004c80 con 0x563b32e4c6a0 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd25affd640 1 -- 172.21.15.22:0/3339036211 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2740398e0 con 0x563b32e4c6a0 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 10 client.28181 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 10 client.28181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 20 client.28181 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32e60cd0 0x563b32e63190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 4 client.28181 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.252+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b32db4400 con 0x563b32e60cd0 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.253+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32e60cd0 0x563b32e63190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.253+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32e60cd0 0x563b32e63190 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce91afd4431f57f server_cookie=ff7a6227799e6e1c in_seq=0 out_seq=0 2024-03-09T00:04:46.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.253+0000 7fd25affd640 10 client.28181 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.254+0000 7fd25affd640 1 -- 172.21.15.22:0/3339036211 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b32db4400 con 0x563b32e60cd0 2024-03-09T00:04:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.254+0000 7fd25affd640 10 client.28181 handle_command_reply: tid=0 2024-03-09T00:04:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.254+0000 7fd284827740 1 client.28181 shutdown 2024-03-09T00:04:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.254+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 msgr2=0x563b32e4a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e4a060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd27402f7c0 tx=0x7fd274009500 comp rx=0 tx=0).stop 2024-03-09T00:04:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 shutdown_connections 2024-03-09T00:04:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32e60cd0 0x563b32e63190 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e4a060 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e43ac0 0x563b32e49b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 --2- 172.21.15.22:0/3339036211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e431a0 0x563b32e495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 >> 172.21.15.22:0/3339036211 conn(0x563b32e3d8d0 msgr2=0x563b32e3f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd25affd640 0 client.28181 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 shutdown_connections 2024-03-09T00:04:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 1 -- 172.21.15.22:0/3339036211 wait complete. 2024-03-09T00:04:46.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.255+0000 7fd284827740 20 client.28181 trim_cache size 0 max 16384 2024-03-09T00:04:46.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.279+0000 7fd284827740 1 Processor -- start 2024-03-09T00:04:46.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.279+0000 7fd284827740 1 -- start start 2024-03-09T00:04:46.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.279+0000 7fd284827740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd284827740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e46840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd284827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e46d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd284827740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a6210 con 0x563b32e431a0 2024-03-09T00:04:46.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd284827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0726f0 con 0x563b32e4c6a0 2024-03-09T00:04:46.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd27affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e46840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd27affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e46840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38746/0 (socket says 172.21.15.22:38746) 2024-03-09T00:04:46.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd27affd640 1 -- 172.21.15.22:0/1646522928 learned_addr learned my addr 172.21.15.22:0/1646522928 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:46.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd284827740 1 -- 172.21.15.22:0/1646522928 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0707b0 con 0x563b32e43ac0 2024-03-09T00:04:46.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.280+0000 7fd27bfff640 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e46d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.283+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.286+0000 7fd27bfff640 1 -- 172.21.15.22:0/1646522928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 msgr2=0x563b32e46840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:46.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.286+0000 7fd27bfff640 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e46840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.286+0000 7fd27bfff640 1 -- 172.21.15.22:0/1646522928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 msgr2=0x563b32e4bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:46.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.286+0000 7fd27bfff640 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e4bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.286+0000 7fd27bfff640 1 -- 172.21.15.22:0/1646522928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b32e683c0 con 0x563b32e4c6a0 2024-03-09T00:04:46.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.287+0000 7fd27bfff640 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e46d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd274039dd0 tx=0x7fd274013940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760b4ea068829d39 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:46.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.287+0000 7fd2617fa640 1 -- 172.21.15.22:0/1646522928 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27402fc80 con 0x563b32e4c6a0 2024-03-09T00:04:46.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.287+0000 7fd2617fa640 1 -- 172.21.15.22:0/1646522928 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd27404d070 con 0x563b32e4c6a0 2024-03-09T00:04:46.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.287+0000 7fd2617fa640 1 -- 172.21.15.22:0/1646522928 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27402d880 con 0x563b32e4c6a0 2024-03-09T00:04:46.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.289+0000 7fd284827740 1 -- 172.21.15.22:0/1646522928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 msgr2=0x563b32e46d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.289+0000 7fd284827740 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e46d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd274039dd0 tx=0x7fd274013940 comp rx=0 tx=0).stop 2024-03-09T00:04:46.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.290+0000 7fd284827740 1 -- 172.21.15.22:0/1646522928 shutdown_connections 2024-03-09T00:04:46.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.290+0000 7fd284827740 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e4c6a0 0x563b32e46d80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.290+0000 7fd284827740 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e43ac0 0x563b32e46840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.290+0000 7fd284827740 1 --2- 172.21.15.22:0/1646522928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e4bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.290+0000 7fd284827740 1 -- 172.21.15.22:0/1646522928 >> 172.21.15.22:0/1646522928 conn(0x563b32e3d8d0 msgr2=0x563b32e448a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:46.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.291+0000 7fd284827740 1 -- 172.21.15.22:0/1646522928 shutdown_connections 2024-03-09T00:04:46.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.291+0000 7fd284827740 1 -- 172.21.15.22:0/1646522928 wait complete. 2024-03-09T00:04:46.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.291+0000 7fd284827740 1 -- start start 2024-03-09T00:04:46.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.292+0000 7fd284827740 1 Processor -- start 2024-03-09T00:04:46.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.292+0000 7fd284827740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e55960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd284827740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e43ac0 0x563b32e55ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd284827740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e4c6a0 0x563b32eb02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd284827740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c070db0 con 0x563b32e431a0 2024-03-09T00:04:46.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd284827740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c072090 con 0x563b32e43ac0 2024-03-09T00:04:46.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e4c6a0 0x563b32eb02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e4c6a0 0x563b32eb02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38750/0 (socket says 172.21.15.22:38750) 2024-03-09T00:04:46.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd27bfff640 1 -- 172.21.15.22:0/2627902038 learned_addr learned my addr 172.21.15.22:0/2627902038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:46.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0713d0 con 0x563b32e4c6a0 2024-03-09T00:04:46.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd27affd640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e43ac0 0x563b32e55ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.293+0000 7fd284827740 10 client.0 _ll_register_callbacks cb 0x7fd26c003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:46.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.294+0000 7fd27b7fe640 1 -- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e4c6a0 msgr2=0x563b32eb02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:46.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.294+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e4c6a0 0x563b32eb02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.294+0000 7fd27b7fe640 1 -- 172.21.15.22:0/2627902038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e43ac0 msgr2=0x563b32e55ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.294+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e43ac0 0x563b32e55ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.294+0000 7fd27b7fe640 1 -- 172.21.15.22:0/2627902038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b32eb08e0 con 0x563b32e431a0 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.294+0000 7fd27bfff640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e4c6a0 0x563b32eb02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.295+0000 7fd27b7fe640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e55960 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009900 tx=0x7fd26c0b8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e92a3b8602c2470 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.295+0000 7fd25affd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.295+0000 7fd25affd640 1 -- 172.21.15.22:0/2627902038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c004c80 con 0x563b32e431a0 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.295+0000 7fd25affd640 1 -- 172.21.15.22:0/2627902038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd26c0ee130 con 0x563b32e431a0 2024-03-09T00:04:46.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.295+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd27404d070 con 0x563b32e431a0 2024-03-09T00:04:46.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.296+0000 7fd25affd640 1 -- 172.21.15.22:0/2627902038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd26c0ee330 con 0x563b32e431a0 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.296+0000 7fd284827740 10 client.18843 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.296+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563b32eb7c40 con 0x563b32e431a0 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd25affd640 1 -- 172.21.15.22:0/2627902038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd26c0bcf80 con 0x563b32e431a0 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd25affd640 1 -- 172.21.15.22:0/2627902038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd26c0bc3d0 con 0x563b32e431a0 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd284827740 10 client.18843 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd284827740 10 client.18843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd284827740 20 client.18843 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd284827740 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32eb8190 0x563b32eba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd284827740 4 client.18843 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd27404d070 con 0x563b32eb8190 2024-03-09T00:04:46.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.297+0000 7fd27affd640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32eb8190 0x563b32eba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:46.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd27affd640 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32eb8190 0x563b32eba5b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9470600a69daea7 server_cookie=369da07372ac3735 in_seq=0 out_seq=0 2024-03-09T00:04:46.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd25affd640 10 client.18843 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd25affd640 1 -- 172.21.15.22:0/2627902038 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd27404d070 con 0x563b32eb8190 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd25affd640 10 client.18843 handle_command_reply: tid=0 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd284827740 1 client.18843 shutdown 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 msgr2=0x563b32e55960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd284827740 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e55960 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009900 tx=0x7fd26c0b8b80 comp rx=0 tx=0).stop 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 shutdown_connections 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd284827740 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b32eb8190 0x563b32eba5b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.299+0000 7fd284827740 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b32e4c6a0 0x563b32eb02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd25affd640 0 client.18843 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd284827740 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b32e43ac0 0x563b32e55ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd284827740 1 --2- 172.21.15.22:0/2627902038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b32e431a0 0x563b32e55960 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 >> 172.21.15.22:0/2627902038 conn(0x563b32e3d8d0 msgr2=0x563b32e47eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 shutdown_connections 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd284827740 1 -- 172.21.15.22:0/2627902038 wait complete. 2024-03-09T00:04:46.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd284827740 20 client.18843 trim_cache size 0 max 16384 2024-03-09T00:04:46.297 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:04:46.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2640a2970 msgr2=0x7fd2640a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2640a2970 0x7fd2640a4e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd2740049d0 tx=0x7fd2740115e0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd27c069b90 msgr2=0x7fd27c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd27c069b90 0x7fd27c19d7d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd26c036040 tx=0x7fd26c02cb20 comp rx=0 tx=0).stop 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 shutdown_connections 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd2640a2970 0x7fd2640a4e30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd27c197980 0x7fd27c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c1065a0 0x7fd27c19dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 --2- 172.21.15.22:0/3120343143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd27c069b90 0x7fd27c19d7d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.300+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 >> 172.21.15.22:0/3120343143 conn(0x7fd27c076f90 msgr2=0x7fd27c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:46.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.301+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 shutdown_connections 2024-03-09T00:04:46.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:46.301+0000 7fd281b69640 1 -- 172.21.15.22:0/3120343143 wait complete. 2024-03-09T00:04:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 sudo[72099]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 sudo[72099]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 sudo[72099]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 sudo[72099]: pam_unix(sudo:session): session closed for user root 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: pgmap v323: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 388 op/s 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/124413350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:46 smithi062 ceph-mon[36129]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: pgmap v323: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 388 op/s 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/124413350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:46 smithi192 ceph-mon[35995]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:46.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1045945 9.91 MB/sec execute 414 sec latency 29.230 ms 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: pgmap v323: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 388 op/s 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/124413350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:04:46.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:46 smithi022 ceph-mon[31894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-09T00:04:47.308 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:04:47.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:04:47.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:04:47.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 -- 172.21.15.22:0/1063218215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd794100410 msgr2=0x7fd7941007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 --2- 172.21.15.22:0/1063218215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd794100410 0x7fd7941007f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd784009900 tx=0x7fd78402bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:47.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 -- 172.21.15.22:0/1063218215 shutdown_connections 2024-03-09T00:04:47.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 --2- 172.21.15.22:0/1063218215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941054e0 0x7fd79410d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 --2- 172.21.15.22:0/1063218215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd794100dc0 0x7fd794104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 --2- 172.21.15.22:0/1063218215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd794100410 0x7fd7941007f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 -- 172.21.15.22:0/1063218215 >> 172.21.15.22:0/1063218215 conn(0x7fd794076570 msgr2=0x7fd794076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 -- 172.21.15.22:0/1063218215 shutdown_connections 2024-03-09T00:04:47.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 -- 172.21.15.22:0/1063218215 wait complete. 2024-03-09T00:04:47.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.428+0000 7fd798929640 1 Processor -- start 2024-03-09T00:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd798929640 1 -- start start 2024-03-09T00:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd798929640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd794100410 0x7fd79419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd798929640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7941054e0 0x7fd79419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd798929640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd794196910 0x7fd794196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd798929640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794110590 con 0x7fd7941054e0 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd798929640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794110410 con 0x7fd794100410 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd798929640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794110710 con 0x7fd794196910 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7941054e0 0x7fd79419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7941054e0 0x7fd79419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48228/0 (socket says 172.21.15.22:48228) 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 -- 172.21.15.22:0/3001092230 learned_addr learned my addr 172.21.15.22:0/3001092230 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd79259c640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd794100410 0x7fd79419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd792d9d640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd794196910 0x7fd794196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 -- 172.21.15.22:0/3001092230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd794196910 msgr2=0x7fd794196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd794196910 0x7fd794196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 -- 172.21.15.22:0/3001092230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd794100410 msgr2=0x7fd79419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd794100410 0x7fd79419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 -- 172.21.15.22:0/3001092230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd794197430 con 0x7fd7941054e0 2024-03-09T00:04:47.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd792d9d640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd794196910 0x7fd794196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:04:47.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.429+0000 7fd789d9b640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7941054e0 0x7fd79419cc50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00e9f0 tx=0x7fd77c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.430+0000 7fd78bfff640 1 -- 172.21.15.22:0/3001092230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00ccc0 con 0x7fd7941054e0 2024-03-09T00:04:47.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.430+0000 7fd78bfff640 1 -- 172.21.15.22:0/3001092230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd77c005500 con 0x7fd7941054e0 2024-03-09T00:04:47.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.430+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd794197720 con 0x7fd7941054e0 2024-03-09T00:04:47.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.430+0000 7fd78bfff640 1 -- 172.21.15.22:0/3001092230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c010690 con 0x7fd7941054e0 2024-03-09T00:04:47.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.430+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7941a38d0 con 0x7fd7941054e0 2024-03-09T00:04:47.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.432+0000 7fd78bfff640 1 -- 172.21.15.22:0/3001092230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd77c0046c0 con 0x7fd7941054e0 2024-03-09T00:04:47.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 Processor -- start 2024-03-09T00:04:47.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 -- start start 2024-03-09T00:04:47.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a352b40 con 0x55b70a3fa490 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a2fbd80 con 0x55b70a3ef390 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a34ebe0 con 0x55b70a3eea70 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48244/0 (socket says 172.21.15.22:48244) 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd789d9b640 1 -- 172.21.15.22:0/1286073669 learned_addr learned my addr 172.21.15.22:0/1286073669 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd78bfff640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd7800a2970 0x7fd7800a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd79259c640 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd792d9d640 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.433+0000 7fd792d9d640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd7800a2970 0x7fd7800a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd79259c640 1 -- 172.21.15.22:0/1286073669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 msgr2=0x55b70a3eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd79259c640 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3eee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd78bfff640 1 -- 172.21.15.22:0/3001092230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd77c0cf680 con 0x7fd7941054e0 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd79259c640 1 -- 172.21.15.22:0/1286073669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 msgr2=0x55b70a3fc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd79259c640 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd79259c640 1 -- 172.21.15.22:0/1286073669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70a12f7e0 con 0x55b70a3ef390 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd789d9b640 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd79259c640 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd784000c00 tx=0x7fd784009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b2d1253cb96be2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd792d9d640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd7800a2970 0x7fd7800a4e30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005c40 tx=0x7fd78c009200 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd788d99640 1 -- 172.21.15.22:0/1286073669 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784042070 con 0x55b70a3ef390 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd788d99640 1 -- 172.21.15.22:0/1286073669 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd78402ca80 con 0x55b70a3ef390 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.434+0000 7fd788d99640 1 -- 172.21.15.22:0/1286073669 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784004f00 con 0x55b70a3ef390 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1286073669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 msgr2=0x55b70a3f9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd784000c00 tx=0x7fd784009610 comp rx=0 tx=0).stop 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1286073669 shutdown_connections 2024-03-09T00:04:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1286073669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1286073669 >> 172.21.15.22:0/1286073669 conn(0x55b70a3e94b0 msgr2=0x55b70a3e9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1286073669 shutdown_connections 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1286073669 wait complete. 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 -- start start 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 Processor -- start 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.435+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3eea70 0x55b70a3f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a3f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a3fd220 con 0x55b70a3eea70 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a30f1c0 con 0x55b70a3ef390 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a3fd3a0 con 0x55b70a3fa490 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 10 client.0 _ll_register_callbacks cb 0x55b70a2fe7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3eea70 0x55b70a3f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3eea70 0x55b70a3f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48252/0 (socket says 172.21.15.22:48252) 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 -- 172.21.15.22:0/1020570770 learned_addr learned my addr 172.21.15.22:0/1020570770 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd792d9d640 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a3f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd789d9b640 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 -- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 msgr2=0x55b70a3f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a3f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 -- 172.21.15.22:0/1020570770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 msgr2=0x55b70a3f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 -- 172.21.15.22:0/1020570770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b70a18ff70 con 0x55b70a3eea70 2024-03-09T00:04:47.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79259c640 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3eea70 0x55b70a3f9e00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd784009430 tx=0x7fd78402c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a8a69d3004b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd779ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:47.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd779ffb640 1 -- 172.21.15.22:0/1020570770 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78404a070 con 0x55b70a3eea70 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd779ffb640 1 -- 172.21.15.22:0/1020570770 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd784004440 con 0x55b70a3eea70 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd78402ca80 con 0x55b70a3eea70 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd779ffb640 1 -- 172.21.15.22:0/1020570770 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd784056050 con 0x55b70a3eea70 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 10 client.18855 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.436+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b70a027170 con 0x55b70a3eea70 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd779ffb640 1 -- 172.21.15.22:0/1020570770 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd78402ca80 con 0x55b70a3eea70 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd779ffb640 1 -- 172.21.15.22:0/1020570770 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd784003850 con 0x55b70a3eea70 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd79b5e7740 10 client.18855 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd79b5e7740 10 client.18855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd79b5e7740 20 client.18855 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a40a350 0x55b70a40c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd79b5e7740 4 client.18855 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:47.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b70a328960 con 0x55b70a40a350 2024-03-09T00:04:47.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.437+0000 7fd789d9b640 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a40a350 0x55b70a40c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd789d9b640 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a40a350 0x55b70a40c740 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d37d3fc039fa7a7 server_cookie=7ff7603013bc6b07 in_seq=0 out_seq=0 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd779ffb640 10 client.18855 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd779ffb640 1 -- 172.21.15.22:0/1020570770 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b70a328960 con 0x55b70a40a350 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd779ffb640 10 client.18855 handle_command_reply: tid=0 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 client.18855 shutdown 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3eea70 msgr2=0x55b70a3f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3eea70 0x55b70a3f9e00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd784009430 tx=0x7fd78402c970 comp rx=0 tx=0).stop 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 shutdown_connections 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a40a350 0x55b70a40c740 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a3f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd779ffb640 0 client.18855 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/1020570770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3eea70 0x55b70a3f9e00 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 >> 172.21.15.22:0/1020570770 conn(0x55b70a3e94b0 msgr2=0x55b70a3eb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 shutdown_connections 2024-03-09T00:04:47.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 1 -- 172.21.15.22:0/1020570770 wait complete. 2024-03-09T00:04:47.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.438+0000 7fd79b5e7740 20 client.18855 trim_cache size 0 max 16384 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 Processor -- start 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 -- start start 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941ae8d0 con 0x55b70a3fa490 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941ac470 con 0x55b70a3ef390 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941aa530 con 0x55b70a3eea70 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd79259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48266/0 (socket says 172.21.15.22:48266) 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:59248/0 (socket says 172.21.15.22:59248) 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 -- 172.21.15.22:0/2002807536 learned_addr learned my addr 172.21.15.22:0/2002807536 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 -- 172.21.15.22:0/2002807536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 msgr2=0x55b70a3f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 -- 172.21.15.22:0/2002807536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 msgr2=0x55b70a3f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 -- 172.21.15.22:0/2002807536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70a3fcbf0 con 0x55b70a3fa490 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd789d9b640 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:47.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.458+0000 7fd792d9d640 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fb880 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00f7a0 tx=0x7fd78c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5b3eca37859d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd77bfff640 1 -- 172.21.15.22:0/2002807536 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c018040 con 0x55b70a3fa490 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd77bfff640 1 -- 172.21.15.22:0/2002807536 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004d20 con 0x55b70a3fa490 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd77bfff640 1 -- 172.21.15.22:0/2002807536 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0057d0 con 0x55b70a3fa490 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd79b5e7740 1 -- 172.21.15.22:0/2002807536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 msgr2=0x55b70a3fb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fb880 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00f7a0 tx=0x7fd78c009650 comp rx=0 tx=0).stop 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd79b5e7740 1 -- 172.21.15.22:0/2002807536 shutdown_connections 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3fa490 0x55b70a3fb880 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3ef390 0x55b70a3f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/2002807536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3eea70 0x55b70a3f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.459+0000 7fd79b5e7740 1 -- 172.21.15.22:0/2002807536 >> 172.21.15.22:0/2002807536 conn(0x55b70a3e94b0 msgr2=0x55b70a3f2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 -- 172.21.15.22:0/2002807536 shutdown_connections 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 -- 172.21.15.22:0/2002807536 wait complete. 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 -- start start 2024-03-09T00:04:47.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 Processor -- start 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3eea70 0x55b70a3ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3ef390 0x55b70a3ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a400080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941aab30 con 0x55b70a3ef390 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941abe10 con 0x55b70a3eea70 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941ab150 con 0x55b70a3fa490 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79b5e7740 10 client.0 _ll_register_callbacks cb 0x7fd784009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3ef390 0x55b70a3ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd79259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3eea70 0x55b70a3ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd789d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3ef390 0x55b70a3ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48278/0 (socket says 172.21.15.22:48278) 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd789d9b640 1 -- 172.21.15.22:0/67613730 learned_addr learned my addr 172.21.15.22:0/67613730 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.460+0000 7fd792d9d640 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a400080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd789d9b640 1 -- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 msgr2=0x55b70a400080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd789d9b640 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a400080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd789d9b640 1 -- 172.21.15.22:0/67613730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3eea70 msgr2=0x55b70a3ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd789d9b640 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3eea70 0x55b70a3ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd789d9b640 1 -- 172.21.15.22:0/67613730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b70a461490 con 0x55b70a3ef390 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd789d9b640 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3ef390 0x55b70a3ffb40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0ccfd0 tx=0x7fd77c0cf880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60db18d51d0bd82a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd779ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd779ffb640 1 -- 172.21.15.22:0/67613730 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0b3070 con 0x55b70a3ef390 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd779ffb640 1 -- 172.21.15.22:0/67613730 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd77c097f50 con 0x55b70a3ef390 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd78c004d20 con 0x55b70a3ef390 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd779ffb640 1 -- 172.21.15.22:0/67613730 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd77c09b8e0 con 0x55b70a3ef390 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 10 client.18861 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b70a462770 con 0x55b70a3ef390 2024-03-09T00:04:47.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd779ffb640 1 -- 172.21.15.22:0/67613730 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004d20 con 0x55b70a3ef390 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd779ffb640 1 -- 172.21.15.22:0/67613730 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd77c0d2ce0 con 0x55b70a3ef390 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 10 client.18861 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 10 client.18861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 20 client.18861 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a462ed0 0x55b70a4652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 4 client.18861 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.461+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b70a314080 con 0x55b70a462ed0 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.462+0000 7fd79259c640 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a462ed0 0x55b70a4652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.462+0000 7fd779ffb640 10 client.18861 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.462+0000 7fd79259c640 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a462ed0 0x55b70a4652f0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac2306b9aecb41a server_cookie=f278f7d37dcca87d in_seq=0 out_seq=0 2024-03-09T00:04:47.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.464+0000 7fd779ffb640 1 -- 172.21.15.22:0/67613730 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b70a314080 con 0x55b70a462ed0 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.464+0000 7fd779ffb640 10 client.18861 handle_command_reply: tid=0 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.464+0000 7fd79b5e7740 1 client.18861 shutdown 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.464+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3ef390 msgr2=0x55b70a3ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.464+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3ef390 0x55b70a3ffb40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0ccfd0 tx=0x7fd77c0cf880 comp rx=0 tx=0).stop 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.464+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 shutdown_connections 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.464+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b70a462ed0 0x55b70a4652f0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b70a3fa490 0x55b70a400080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b70a3ef390 0x55b70a3ffb40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd79b5e7740 1 --2- 172.21.15.22:0/67613730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b70a3eea70 0x55b70a3ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd779ffb640 0 client.18861 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 >> 172.21.15.22:0/67613730 conn(0x55b70a3e94b0 msgr2=0x55b70a3e7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 shutdown_connections 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd79b5e7740 1 -- 172.21.15.22:0/67613730 wait complete. 2024-03-09T00:04:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd79b5e7740 20 client.18861 trim_cache size 0 max 16384 2024-03-09T00:04:47.466 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:04:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "51af90de-f6df-4c27-8d7e-adb9bbb3ca03", 2024-03-09T00:04:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:04:47.466 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd7800a2970 msgr2=0x7fd7800a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd7800a2970 0x7fd7800a4e30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005c40 tx=0x7fd78c009200 comp rx=0 tx=0).stop 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7941054e0 msgr2=0x7fd79419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7941054e0 0x7fd79419cc50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00e9f0 tx=0x7fd77c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 shutdown_connections 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd7800a2970 0x7fd7800a4e30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd794196910 0x7fd794196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7941054e0 0x7fd79419cc50 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 --2- 172.21.15.22:0/3001092230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd794100410 0x7fd79419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 >> 172.21.15.22:0/3001092230 conn(0x7fd794076570 msgr2=0x7fd7940fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 shutdown_connections 2024-03-09T00:04:47.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.465+0000 7fd798929640 1 -- 172.21.15.22:0/3001092230 wait complete. 2024-03-09T00:04:47.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:04:47.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:04:47.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 -- 172.21.15.22:0/2355378136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c100e10 msgr2=0x7fd01c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 --2- 172.21.15.22:0/2355378136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c100e10 0x7fd01c104e00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd010009900 tx=0x7fd01002bc30 comp rx=0 tx=0).stop 2024-03-09T00:04:47.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 -- 172.21.15.22:0/2355378136 shutdown_connections 2024-03-09T00:04:47.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 --2- 172.21.15.22:0/2355378136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd01c1054e0 0x7fd01c10d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 --2- 172.21.15.22:0/2355378136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c100e10 0x7fd01c104e00 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 --2- 172.21.15.22:0/2355378136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01c100460 0x7fd01c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 -- 172.21.15.22:0/2355378136 >> 172.21.15.22:0/2355378136 conn(0x7fd01c077350 msgr2=0x7fd01c077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 -- 172.21.15.22:0/2355378136 shutdown_connections 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 -- 172.21.15.22:0/2355378136 wait complete. 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 Processor -- start 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.584+0000 7fd023c97640 1 -- start start 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd01c100460 0x7fd01c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c1054e0 0x7fd01c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01c197a00 0x7fd01c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c110570 con 0x7fd01c1054e0 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1103f0 con 0x7fd01c100460 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1106f0 con 0x7fd01c197a00 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c1054e0 0x7fd01c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c1054e0 0x7fd01c19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48296/0 (socket says 172.21.15.22:48296) 2024-03-09T00:04:47.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 -- 172.21.15.22:0/3790184692 learned_addr learned my addr 172.21.15.22:0/3790184692 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021a33640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd01c100460 0x7fd01c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd022234640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01c197a00 0x7fd01c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 -- 172.21.15.22:0/3790184692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01c197a00 msgr2=0x7fd01c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01c197a00 0x7fd01c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 -- 172.21.15.22:0/3790184692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd01c100460 msgr2=0x7fd01c19d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd01c100460 0x7fd01c19d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 -- 172.21.15.22:0/3790184692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c198520 con 0x7fd01c1054e0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd021232640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c1054e0 0x7fd01c19dd90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd010036040 tx=0x7fd01002cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd00effd640 1 -- 172.21.15.22:0/3790184692 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010041070 con 0x7fd01c1054e0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd00effd640 1 -- 172.21.15.22:0/3790184692 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd010004450 con 0x7fd01c1054e0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c1987b0 con 0x7fd01c1054e0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd00effd640 1 -- 172.21.15.22:0/3790184692 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010003dc0 con 0x7fd01c1054e0 2024-03-09T00:04:47.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.585+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c1a49a0 con 0x7fd01c1054e0 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd00effd640 1 -- 172.21.15.22:0/3790184692 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd010044070 con 0x7fd01c1054e0 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 Processor -- start 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 -- start start 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287e7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2873ee00 con 0x564c287db760 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c286e7b10 con 0x564c287dc080 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd02687b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2873aea0 con 0x564c287e4c60 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.588+0000 7fd022234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287dbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd022234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287dbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48308/0 (socket says 172.21.15.22:48308) 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd022234640 1 -- 172.21.15.22:0/1656978890 learned_addr learned my addr 172.21.15.22:0/1656978890 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021232640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021a33640 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd00effd640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0080a2970 0x7fd0080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021a33640 1 -- 172.21.15.22:0/1656978890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 msgr2=0x564c287e7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021a33640 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287e7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021a33640 1 -- 172.21.15.22:0/1656978890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 msgr2=0x564c287dbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021a33640 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287dbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021a33640 1 -- 172.21.15.22:0/1656978890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2851d7d0 con 0x564c287dc080 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021232640 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd022234640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0080a2970 0x7fd0080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd00effd640 1 -- 172.21.15.22:0/3790184692 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd0100f0c00 con 0x7fd01c1054e0 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd021a33640 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd018009bd0 tx=0x7fd018011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab51537a235904d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd007fff640 1 -- 172.21.15.22:0/1656978890 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018004230 con 0x564c287dc080 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd007fff640 1 -- 172.21.15.22:0/1656978890 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd018009e40 con 0x564c287dc080 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.589+0000 7fd007fff640 1 -- 172.21.15.22:0/1656978890 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018004d30 con 0x564c287dc080 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 -- 172.21.15.22:0/1656978890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 msgr2=0x564c287e4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd018009bd0 tx=0x7fd018011810 comp rx=0 tx=0).stop 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd022234640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0080a2970 0x7fd0080a4e30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0140105f0 tx=0x7fd014004a40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 -- 172.21.15.22:0/1656978890 shutdown_connections 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e4720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 --2- 172.21.15.22:0/1656978890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287dbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 -- 172.21.15.22:0/1656978890 >> 172.21.15.22:0/1656978890 conn(0x564c287d5e90 msgr2=0x564c287d6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 -- 172.21.15.22:0/1656978890 shutdown_connections 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 -- 172.21.15.22:0/1656978890 wait complete. 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 -- start start 2024-03-09T00:04:47.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.590+0000 7fd02687b740 1 Processor -- start 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd02687b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287db760 0x564c287e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd02687b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd02687b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd02687b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c287e7a50 con 0x564c287e4c60 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd02687b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c286fb0a0 con 0x564c287dc080 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd02687b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c287e7bd0 con 0x564c287db760 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd02687b740 10 client.0 _ll_register_callbacks cb 0x564c286faa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd021232640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287e1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48312/0 (socket says 172.21.15.22:48312) 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd021232640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:59320/0 (socket says 172.21.15.22:59320) 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 -- 172.21.15.22:0/1058205431 learned_addr learned my addr 172.21.15.22:0/1058205431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd021a33640 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287db760 0x564c287e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 -- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287db760 msgr2=0x564c287e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287db760 0x564c287e4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 -- 172.21.15.22:0/1058205431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 msgr2=0x564c287e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd022234640 1 -- 172.21.15.22:0/1058205431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c284533f0 con 0x564c287e4c60 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.591+0000 7fd021232640 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd022234640 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287e1c40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd01400fac0 tx=0x7fd014004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d510f9c3472c43 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd0057fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd0057fa640 1 -- 172.21.15.22:0/1058205431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01402b060 con 0x564c287e4c60 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd0057fa640 1 -- 172.21.15.22:0/1058205431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd01401d570 con 0x564c287e4c60 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2874c340 con 0x564c287e4c60 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd0057fa640 1 -- 172.21.15.22:0/1058205431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd014032050 con 0x564c287e4c60 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 10 client.18873 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:47.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564c284130f0 con 0x564c287e4c60 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd0057fa640 1 -- 172.21.15.22:0/1058205431 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564c2874c340 con 0x564c287e4c60 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd0057fa640 1 -- 172.21.15.22:0/1058205431 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd014026900 con 0x564c287e4c60 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 10 client.18873 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 20 client.18873 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c287f46d0 0x564c287f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 4 client.18873 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.592+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c28746690 con 0x564c287f46d0 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.593+0000 7fd021a33640 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c287f46d0 0x564c287f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.593+0000 7fd0057fa640 10 client.18873 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.593+0000 7fd021a33640 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c287f46d0 0x564c287f6ae0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c9958d4bcf047d server_cookie=12658d34489d0333 in_seq=0 out_seq=0 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd0057fa640 1 -- 172.21.15.22:0/1058205431 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c28746690 con 0x564c287f46d0 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd0057fa640 10 client.18873 handle_command_reply: tid=0 2024-03-09T00:04:47.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 client.18873 shutdown 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 msgr2=0x564c287e1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287e1c40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd01400fac0 tx=0x7fd014004ab0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 shutdown_connections 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c287f46d0 0x564c287f6ae0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287e1c40 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd0057fa640 0 client.18873 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 --2- 172.21.15.22:0/1058205431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287db760 0x564c287e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 >> 172.21.15.22:0/1058205431 conn(0x564c287d5e90 msgr2=0x564c287d7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 shutdown_connections 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 1 -- 172.21.15.22:0/1058205431 wait complete. 2024-03-09T00:04:47.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.594+0000 7fd02687b740 20 client.18873 trim_cache size 0 max 16384 2024-03-09T00:04:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 Processor -- start 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 -- start start 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287dfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1b0bb0 con 0x564c287db760 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1ae750 con 0x564c287dc080 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd02687b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1ac810 con 0x564c287e4c60 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021232640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287e16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48324/0 (socket says 172.21.15.22:48324) 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 -- 172.21.15.22:0/703581066 learned_addr learned my addr 172.21.15.22:0/703581066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd022234640 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287dfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 -- 172.21.15.22:0/703581066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 msgr2=0x564c287dfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287dfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 -- 172.21.15.22:0/703581066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 msgr2=0x564c287e1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 -- 172.21.15.22:0/703581066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c287dfc70 con 0x564c287db760 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021232640 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.610+0000 7fd021a33640 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287e16c0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd018009990 tx=0x7fd018005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2c3a9fa6c9e556 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd0077fe640 1 -- 172.21.15.22:0/703581066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01801e040 con 0x564c287db760 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd0077fe640 1 -- 172.21.15.22:0/703581066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd018004390 con 0x564c287db760 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd0077fe640 1 -- 172.21.15.22:0/703581066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018024420 con 0x564c287db760 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 -- 172.21.15.22:0/703581066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 msgr2=0x564c287e16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287e16c0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd018009990 tx=0x7fd018005870 comp rx=0 tx=0).stop 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 -- 172.21.15.22:0/703581066 shutdown_connections 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287e4c60 0x564c287dfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287dc080 0x564c287e1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 --2- 172.21.15.22:0/703581066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287db760 0x564c287e16c0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 -- 172.21.15.22:0/703581066 >> 172.21.15.22:0/703581066 conn(0x564c287d5e90 msgr2=0x564c287fb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 -- 172.21.15.22:0/703581066 shutdown_connections 2024-03-09T00:04:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.611+0000 7fd02687b740 1 -- 172.21.15.22:0/703581066 wait complete. 2024-03-09T00:04:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 -- start start 2024-03-09T00:04:47.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 Processor -- start 2024-03-09T00:04:47.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287db760 0x564c287ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287dc080 0x564c287ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287dddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1ace10 con 0x564c287e4c60 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1ae0f0 con 0x564c287db760 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1ad430 con 0x564c287dc080 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd02687b740 10 client.0 _ll_register_callbacks cb 0x7fd014009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287dddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd021a33640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287db760 0x564c287ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287dddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48334/0 (socket says 172.21.15.22:48334) 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 -- 172.21.15.22:0/3717670239 learned_addr learned my addr 172.21.15.22:0/3717670239 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd021232640 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287dc080 0x564c287ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 -- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287dc080 msgr2=0x564c287ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287dc080 0x564c287ec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 -- 172.21.15.22:0/3717670239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287db760 msgr2=0x564c287ebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287db760 0x564c287ebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd022234640 1 -- 172.21.15.22:0/3717670239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c287de5f0 con 0x564c287e4c60 2024-03-09T00:04:47.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.612+0000 7fd021a33640 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287db760 0x564c287ebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd022234640 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287dddc0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd014029040 tx=0x7fd014026640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f838e177dbd56570 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd0057fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd0057fa640 1 -- 172.21.15.22:0/3717670239 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01403a020 con 0x564c287e4c60 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd0057fa640 1 -- 172.21.15.22:0/3717670239 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd0140269c0 con 0x564c287e4c60 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd018004390 con 0x564c287e4c60 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd0057fa640 1 -- 172.21.15.22:0/3717670239 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd014045050 con 0x564c287e4c60 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 10 client.18879 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564c2884b6f0 con 0x564c287e4c60 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd0057fa640 1 -- 172.21.15.22:0/3717670239 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd018004390 con 0x564c287e4c60 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd0057fa640 1 -- 172.21.15.22:0/3717670239 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd014036430 con 0x564c287e4c60 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 10 client.18879 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 10 client.18879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 20 client.18879 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:47.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c2884be10 0x564c2884e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 4 client.18879 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.613+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c286c44e0 con 0x564c2884be10 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.614+0000 7fd021a33640 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c2884be10 0x564c2884e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.614+0000 7fd0057fa640 10 client.18879 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.614+0000 7fd021a33640 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c2884be10 0x564c2884e230 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f21642635f8472 server_cookie=f1bdddd50969e8ac in_seq=0 out_seq=0 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd0057fa640 1 -- 172.21.15.22:0/3717670239 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c286c44e0 con 0x564c2884be10 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd0057fa640 10 client.18879 handle_command_reply: tid=0 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 client.18879 shutdown 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 msgr2=0x564c287dddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287dddc0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd014029040 tx=0x7fd014026640 comp rx=0 tx=0).stop 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 shutdown_connections 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564c2884be10 0x564c2884e230 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564c287e4c60 0x564c287dddc0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564c287dc080 0x564c287ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 --2- 172.21.15.22:0/3717670239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c287db760 0x564c287ebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd0057fa640 0 client.18879 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 >> 172.21.15.22:0/3717670239 conn(0x564c287d5e90 msgr2=0x564c287e16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 shutdown_connections 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 1 -- 172.21.15.22:0/3717670239 wait complete. 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd02687b740 20 client.18879 trim_cache size 0 max 16384 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:04:47.616 INFO:teuthology.orchestra.run.smithi022.stdout: "51af90de-f6df-4c27-8d7e-adb9bbb3ca03": { 2024-03-09T00:04:47.617 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:04:47.617 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "51af90de-f6df-4c27-8d7e-adb9bbb3ca03", 2024-03-09T00:04:47.617 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:04:47.617 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:04:47.617 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:04:47.617 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0080a2970 msgr2=0x7fd0080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0080a2970 0x7fd0080a4e30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0140105f0 tx=0x7fd014004a40 comp rx=0 tx=0).stop 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c1054e0 msgr2=0x7fd01c19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c1054e0 0x7fd01c19dd90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd010036040 tx=0x7fd01002cb20 comp rx=0 tx=0).stop 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 shutdown_connections 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0080a2970 0x7fd0080a4e30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01c197a00 0x7fd01c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd01c1054e0 0x7fd01c19dd90 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 --2- 172.21.15.22:0/3790184692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd01c100460 0x7fd01c19d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.615+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 >> 172.21.15.22:0/3790184692 conn(0x7fd01c077350 msgr2=0x7fd01c069930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:47.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.616+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 shutdown_connections 2024-03-09T00:04:47.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:04:47.616+0000 7fd023c97640 1 -- 172.21.15.22:0/3790184692 wait complete. 2024-03-09T00:04:47.621 INFO:tasks.cephfs.filesystem:scrub status for tag:51af90de-f6df-4c27-8d7e-adb9bbb3ca03 - {'path': '/', 'tag': '51af90de-f6df-4c27-8d7e-adb9bbb3ca03', 'options': 'recursive,force'} 2024-03-09T00:04:47.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1048050 9.92 MB/sec execute 415 sec latency 36.103 ms 2024-03-09T00:04:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:48 smithi062 ceph-mon[36129]: pgmap v324: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-03-09T00:04:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:48 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:04:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:48 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:48 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:04:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:48 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:48 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:04:48.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:48 smithi192 ceph-mon[35995]: pgmap v324: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-03-09T00:04:48.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:48 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:04:48.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:48 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:48.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:48 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:04:48.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:48 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:48.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:48 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:04:48.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1049098 9.90 MB/sec execute 416 sec latency 17.392 ms 2024-03-09T00:04:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:48 smithi022 ceph-mon[31894]: pgmap v324: 129 pgs: 129 active+clean; 90 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-03-09T00:04:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:48 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:04:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:48 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:48 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:04:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:48 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:04:48.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:48 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:04:49.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1050969 9.90 MB/sec execute 417 sec latency 108.996 ms 2024-03-09T00:04:50.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:49 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:50.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:49 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:50.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:49 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:04:50.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:49 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:50.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:49 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:50.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:49 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:04:50.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:49 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:50.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:49 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:50.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:49 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:04:50.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1051755 9.89 MB/sec execute 418 sec latency 201.017 ms 2024-03-09T00:04:51.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:50 smithi192 ceph-mon[35995]: pgmap v325: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-03-09T00:04:51.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:50 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:51.096 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:50 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:51.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:50 smithi062 ceph-mon[36129]: pgmap v325: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-03-09T00:04:51.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:50 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:51.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:50 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:51.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:50 smithi022 ceph-mon[31894]: pgmap v325: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-03-09T00:04:51.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:50 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:51.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:50 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:51.728 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1052772 9.88 MB/sec execute 419 sec latency 57.580 ms 2024-03-09T00:04:52.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:52.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:52.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:51 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:51 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:51 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:52.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1055052 9.89 MB/sec execute 420 sec latency 21.417 ms 2024-03-09T00:04:53.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:52 smithi062 ceph-mon[36129]: pgmap v326: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 334 op/s 2024-03-09T00:04:53.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:52 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi062 to 174.1M 2024-03-09T00:04:53.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:52 smithi062 ceph-mon[36129]: Unable to set osd_memory_target on smithi062 to 182642790: error parsing value: Value '182642790' is below minimum 939524096 2024-03-09T00:04:53.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:52 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi022 to 177.2M 2024-03-09T00:04:53.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:52 smithi062 ceph-mon[36129]: Unable to set osd_memory_target on smithi022 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-09T00:04:53.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:52 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:53.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:52 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:52 smithi192 ceph-mon[35995]: pgmap v326: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 334 op/s 2024-03-09T00:04:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:52 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi062 to 174.1M 2024-03-09T00:04:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:52 smithi192 ceph-mon[35995]: Unable to set osd_memory_target on smithi062 to 182642790: error parsing value: Value '182642790' is below minimum 939524096 2024-03-09T00:04:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:52 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi022 to 177.2M 2024-03-09T00:04:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:52 smithi192 ceph-mon[35995]: Unable to set osd_memory_target on smithi022 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-09T00:04:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:52 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:53.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:52 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:52 smithi022 ceph-mon[31894]: pgmap v326: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 334 op/s 2024-03-09T00:04:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:52 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi062 to 174.1M 2024-03-09T00:04:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:52 smithi022 ceph-mon[31894]: Unable to set osd_memory_target on smithi062 to 182642790: error parsing value: Value '182642790' is below minimum 939524096 2024-03-09T00:04:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:52 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi022 to 177.2M 2024-03-09T00:04:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:52 smithi022 ceph-mon[31894]: Unable to set osd_memory_target on smithi022 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-03-09T00:04:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:52 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:52 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:53.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1055923 9.87 MB/sec execute 421 sec latency 19.554 ms 2024-03-09T00:04:54.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:04:54.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:04:54.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:54.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:04:54.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:04:54.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-09T00:04:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:04:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:04:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:04:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 -- 172.21.15.22:0/684275313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530105260 msgr2=0x7f1530105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 --2- 172.21.15.22:0/684275313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530105260 0x7f1530105640 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1524009900 tx=0x7f152402bc80 comp rx=0 tx=0).stop 2024-03-09T00:04:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 -- 172.21.15.22:0/684275313 shutdown_connections 2024-03-09T00:04:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 --2- 172.21.15.22:0/684275313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1530100e90 0x7f153010d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 --2- 172.21.15.22:0/684275313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15301004d0 0x7f1530100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 --2- 172.21.15.22:0/684275313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530105260 0x7f1530105640 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 -- 172.21.15.22:0/684275313 >> 172.21.15.22:0/684275313 conn(0x7f1530076fd0 msgr2=0x7f15300773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 -- 172.21.15.22:0/684275313 shutdown_connections 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.418+0000 7f153517e640 1 -- 172.21.15.22:0/684275313 wait complete. 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 Processor -- start 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 -- start start 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15301004d0 0x7f153019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530100e90 0x7f153019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1530105260 0x7f15301968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301105e0 con 0x7f1530100e90 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1530110460 con 0x7f15301004d0 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f153517e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1530110760 con 0x7f1530105260 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f152ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15301004d0 0x7f153019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f152ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15301004d0 0x7f153019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41300/0 (socket says 172.21.15.22:41300) 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f152ed9d640 1 -- 172.21.15.22:0/1227170081 learned_addr learned my addr 172.21.15.22:0/1227170081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f152e59c640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530100e90 0x7f153019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.419+0000 7f152f59e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1530105260 0x7f15301968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f152ed9d640 1 -- 172.21.15.22:0/1227170081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1530105260 msgr2=0x7f15301968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f152ed9d640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1530105260 0x7f15301968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f152ed9d640 1 -- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530100e90 msgr2=0x7f153019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f152ed9d640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530100e90 0x7f153019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f152ed9d640 1 -- 172.21.15.22:0/1227170081 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1530197060 con 0x7f15301004d0 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f152e59c640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530100e90 0x7f153019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f152ed9d640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15301004d0 0x7f153019c730 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1524006010 tx=0x7f152402cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f1517fff640 1 -- 172.21.15.22:0/1227170081 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524041070 con 0x7f15301004d0 2024-03-09T00:04:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f1517fff640 1 -- 172.21.15.22:0/1227170081 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1524004450 con 0x7f15301004d0 2024-03-09T00:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15301972f0 con 0x7f15301004d0 2024-03-09T00:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f1517fff640 1 -- 172.21.15.22:0/1227170081 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524003dc0 con 0x7f15301004d0 2024-03-09T00:04:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.420+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15301a35f0 con 0x7f15301004d0 2024-03-09T00:04:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.423+0000 7f1517fff640 1 -- 172.21.15.22:0/1227170081 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1524044070 con 0x7f15301004d0 2024-03-09T00:04:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 Processor -- start 2024-03-09T00:04:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 -- start start 2024-03-09T00:04:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15348a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15349450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15356b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c152ad5f0 con 0x560c15349d40 2024-03-09T00:04:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c15256930 con 0x560c15354760 2024-03-09T00:04:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1537d64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c152a9690 con 0x560c15348fd0 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15348a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15348a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41228/0 (socket says 172.21.15.22:41228) 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 -- 172.21.15.22:0/2355983131 learned_addr learned my addr 172.21.15.22:0/2355983131 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1517fff640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15180a2a10 0x7f15180a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152ed9d640 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15349450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152e59c640 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15356b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15180a2a10 0x7f15180a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f1517fff640 1 -- 172.21.15.22:0/1227170081 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f15240f0c10 con 0x7f15301004d0 2024-03-09T00:04:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 -- 172.21.15.22:0/2355983131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 msgr2=0x560c15349450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15349450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 -- 172.21.15.22:0/2355983131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 msgr2=0x560c15356b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15356b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152f59e640 1 -- 172.21.15.22:0/2355983131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c14fc2210 con 0x560c15349d40 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.424+0000 7f152e59c640 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15356b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.425+0000 7f152f59e640 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15348a90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1528013910 tx=0x7f1528013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef45b88ed79c4de8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.425+0000 7f15157fa640 1 -- 172.21.15.22:0/2355983131 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528021030 con 0x560c15349d40 2024-03-09T00:04:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.425+0000 7f15157fa640 1 -- 172.21.15.22:0/2355983131 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1528011c20 con 0x560c15349d40 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.425+0000 7f15157fa640 1 -- 172.21.15.22:0/2355983131 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152801da60 con 0x560c15349d40 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- 172.21.15.22:0/2355983131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 msgr2=0x560c15348a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15348a90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1528013910 tx=0x7f1528013dd0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f152f59e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15180a2a10 0x7f15180a4ed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1528021000 tx=0x7f1528020000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- 172.21.15.22:0/2355983131 shutdown_connections 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15356b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15349450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 --2- 172.21.15.22:0/2355983131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15348a90 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- 172.21.15.22:0/2355983131 >> 172.21.15.22:0/2355983131 conn(0x560c153105b0 msgr2=0x560c15343cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- 172.21.15.22:0/2355983131 shutdown_connections 2024-03-09T00:04:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- 172.21.15.22:0/2355983131 wait complete. 2024-03-09T00:04:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- start start 2024-03-09T00:04:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 Processor -- start 2024-03-09T00:04:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15351780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15351cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15352200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c15357490 con 0x560c15349d40 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c15269d90 con 0x560c15354760 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c15357610 con 0x560c15348fd0 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f1537d64740 10 client.0 _ll_register_callbacks cb 0x560c14fc1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f152e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15351cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f152e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15351cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41238/0 (socket says 172.21.15.22:41238) 2024-03-09T00:04:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f152e59c640 1 -- 172.21.15.22:0/3116510620 learned_addr learned my addr 172.21.15.22:0/3116510620 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.426+0000 7f152ed9d640 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15351780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f152f59e640 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15352200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f152e59c640 1 -- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 msgr2=0x560c15351780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f152e59c640 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15351780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f152e59c640 1 -- 172.21.15.22:0/3116510620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 msgr2=0x560c15352200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f152e59c640 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15352200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f152e59c640 1 -- 172.21.15.22:0/3116510620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c1508e620 con 0x560c15349d40 2024-03-09T00:04:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f152e59c640 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15351cc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f15200129e0 tx=0x7f1520012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09c40c582539288 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1512ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1512ff5640 1 -- 172.21.15.22:0/3116510620 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152000fa10 con 0x560c15349d40 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1512ff5640 1 -- 172.21.15.22:0/3116510620 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f152000fbb0 con 0x560c15349d40 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1528011c20 con 0x560c15349d40 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1512ff5640 1 -- 172.21.15.22:0/3116510620 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1520025050 con 0x560c15349d40 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1537d64740 10 client.18891 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560c153637e0 con 0x560c15349d40 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.427+0000 7f1512ff5640 1 -- 172.21.15.22:0/3116510620 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1528011c20 con 0x560c15349d40 2024-03-09T00:04:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1512ff5640 1 -- 172.21.15.22:0/3116510620 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1520003b80 con 0x560c15349d40 2024-03-09T00:04:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1537d64740 10 client.18891 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1537d64740 10 client.18891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1537d64740 20 client.18891 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1537d64740 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c15367090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1537d64740 4 client.18891 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c152bab30 con 0x560c15364ca0 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f152ed9d640 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c15367090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f152ed9d640 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c15367090 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857011ba6b79e3d4 server_cookie=78daa67c526799b3 in_seq=0 out_seq=0 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.428+0000 7f1512ff5640 10 client.18891 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1512ff5640 1 -- 172.21.15.22:0/3116510620 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f152409be50 con 0x560c15364ca0 2024-03-09T00:04:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1512ff5640 10 client.18891 handle_command_reply: tid=0 2024-03-09T00:04:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 client.18891 shutdown 2024-03-09T00:04:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 msgr2=0x560c15351cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15351cc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f15200129e0 tx=0x7f1520012ea0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 shutdown_connections 2024-03-09T00:04:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c15367090 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15354760 0x560c15352200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15351cc0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1512ff5640 0 client.18891 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 --2- 172.21.15.22:0/3116510620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15348fd0 0x560c15351780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 >> 172.21.15.22:0/3116510620 conn(0x560c153105b0 msgr2=0x560c15345a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 shutdown_connections 2024-03-09T00:04:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 1 -- 172.21.15.22:0/3116510620 wait complete. 2024-03-09T00:04:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.429+0000 7f1537d64740 20 client.18891 trim_cache size 0 max 16384 2024-03-09T00:04:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.444+0000 7f1537d64740 1 Processor -- start 2024-03-09T00:04:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.444+0000 7f1537d64740 1 -- start start 2024-03-09T00:04:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.444+0000 7f1537d64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c15353eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.444+0000 7f1537d64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15350050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.444+0000 7f1537d64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15354760 0x560c15350590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f1537d64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301ae950 con 0x560c15349d40 2024-03-09T00:04:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f1537d64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301ac4f0 con 0x560c15348fd0 2024-03-09T00:04:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f1537d64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301aa5b0 con 0x560c15354760 2024-03-09T00:04:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15350050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15350050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41240/0 (socket says 172.21.15.22:41240) 2024-03-09T00:04:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152e59c640 1 -- 172.21.15.22:0/4273234441 learned_addr learned my addr 172.21.15.22:0/4273234441 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152f59e640 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15354760 0x560c15350590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152ed9d640 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c15353eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152ed9d640 1 -- 172.21.15.22:0/4273234441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15354760 msgr2=0x560c15350590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152ed9d640 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15354760 0x560c15350590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152ed9d640 1 -- 172.21.15.22:0/4273234441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 msgr2=0x560c15350050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152ed9d640 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15350050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152ed9d640 1 -- 172.21.15.22:0/4273234441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c153482a0 con 0x560c15348fd0 2024-03-09T00:04:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152e59c640 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15350050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.445+0000 7f152ed9d640 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c15353eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f15240ee070 tx=0x7f15240dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5551a42d718a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1514ff9640 1 -- 172.21.15.22:0/4273234441 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152404b040 con 0x560c15348fd0 2024-03-09T00:04:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1514ff9640 1 -- 172.21.15.22:0/4273234441 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560c152bab30 con 0x560c15348fd0 2024-03-09T00:04:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1514ff9640 1 -- 172.21.15.22:0/4273234441 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240ee1e0 con 0x560c15348fd0 2024-03-09T00:04:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 -- 172.21.15.22:0/4273234441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 msgr2=0x560c15353eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c15353eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f15240ee070 tx=0x7f15240dd040 comp rx=0 tx=0).stop 2024-03-09T00:04:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 -- 172.21.15.22:0/4273234441 shutdown_connections 2024-03-09T00:04:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15354760 0x560c15350590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15349d40 0x560c15350050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 --2- 172.21.15.22:0/4273234441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c15353eb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 -- 172.21.15.22:0/4273234441 >> 172.21.15.22:0/4273234441 conn(0x560c153105b0 msgr2=0x560c1534ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 -- 172.21.15.22:0/4273234441 shutdown_connections 2024-03-09T00:04:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.446+0000 7f1537d64740 1 -- 172.21.15.22:0/4273234441 wait complete. 2024-03-09T00:04:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 -- start start 2024-03-09T00:04:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 Processor -- start 2024-03-09T00:04:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c1535a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15349d40 0x560c1535abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15354760 0x560c153bb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301aabb0 con 0x560c15354760 2024-03-09T00:04:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301abe90 con 0x560c15348fd0 2024-03-09T00:04:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301ab1d0 con 0x560c15349d40 2024-03-09T00:04:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f1537d64740 10 client.0 _ll_register_callbacks cb 0x7f152800a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:04:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15354760 0x560c153bb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c1535a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15354760 0x560c153bb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41244/0 (socket says 172.21.15.22:41244) 2024-03-09T00:04:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152f59e640 1 -- 172.21.15.22:0/1110318957 learned_addr learned my addr 172.21.15.22:0/1110318957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:04:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152e59c640 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15349d40 0x560c1535abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152f59e640 1 -- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15349d40 msgr2=0x560c1535abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152f59e640 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15349d40 0x560c1535abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152f59e640 1 -- 172.21.15.22:0/1110318957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 msgr2=0x560c1535a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:04:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.447+0000 7f152f59e640 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c1535a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.459 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f152f59e640 1 -- 172.21.15.22:0/1110318957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c15345fc0 con 0x560c15354760 2024-03-09T00:04:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f152ed9d640 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c1535a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:04:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f152f59e640 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15354760 0x560c153bb1a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f15280117b0 tx=0x7f1528013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571d432fa7d85bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:04:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1512ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:04:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1512ff5640 1 -- 172.21.15.22:0/1110318957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528004a00 con 0x560c15354760 2024-03-09T00:04:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1512ff5640 1 -- 172.21.15.22:0/1110318957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f1528004ba0 con 0x560c15354760 2024-03-09T00:04:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c152bab30 con 0x560c15354760 2024-03-09T00:04:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1512ff5640 1 -- 172.21.15.22:0/1110318957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f152803d050 con 0x560c15354760 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 10 client.18897 fetch_fsmap learned FSMap version 35 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560c153bc720 con 0x560c15354760 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1512ff5640 1 -- 172.21.15.22:0/1110318957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560c152bab30 con 0x560c15354760 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1512ff5640 1 -- 172.21.15.22:0/1110318957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1528038030 con 0x560c15354760 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 10 client.18897 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 10 client.18897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 20 client.18897 populate_metadata read hostname 'smithi022' 2024-03-09T00:04:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c153bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:04:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 4 client.18897 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:04:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.448+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c152b4e80 con 0x560c15364ca0 2024-03-09T00:04:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.449+0000 7f152ed9d640 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c153bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:04:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.449+0000 7f1512ff5640 10 client.18897 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.449+0000 7f152ed9d640 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c153bec80 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb5b8ada00ae7f9 server_cookie=6bf6ee869175148 in_seq=0 out_seq=0 2024-03-09T00:04:54.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1057534 9.86 MB/sec execute 422 sec latency 195.874 ms 2024-03-09T00:04:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1512ff5640 1 -- 172.21.15.22:0/1110318957 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c152b4e80 con 0x560c15364ca0 2024-03-09T00:04:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1512ff5640 10 client.18897 handle_command_reply: tid=0 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 client.18897 shutdown 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15354760 msgr2=0x560c153bb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15354760 0x560c153bb1a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f15280117b0 tx=0x7f1528013f60 comp rx=0 tx=0).stop 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 shutdown_connections 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560c15364ca0 0x560c153bec80 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c15354760 0x560c153bb1a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1512ff5640 0 client.18897 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c15349d40 0x560c1535abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 --2- 172.21.15.22:0/1110318957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c15348fd0 0x560c1535a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 >> 172.21.15.22:0/1110318957 conn(0x560c153105b0 msgr2=0x560c1534ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 shutdown_connections 2024-03-09T00:04:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 1 -- 172.21.15.22:0/1110318957 wait complete. 2024-03-09T00:04:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.800+0000 7f1537d64740 20 client.18897 trim_cache size 0 max 16384 2024-03-09T00:04:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:04:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:04:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:04:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:04:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15180a2a10 msgr2=0x7f15180a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15180a2a10 0x7f15180a4ed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1528021000 tx=0x7f1528020000 comp rx=0 tx=0).stop 2024-03-09T00:04:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15301004d0 msgr2=0x7f153019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:04:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15301004d0 0x7f153019c730 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1524006010 tx=0x7f152402cb00 comp rx=0 tx=0).stop 2024-03-09T00:04:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 shutdown_connections 2024-03-09T00:04:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15180a2a10 0x7f15180a4ed0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1530105260 0x7f15301968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530100e90 0x7f153019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 --2- 172.21.15.22:0/1227170081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15301004d0 0x7f153019c730 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:04:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 >> 172.21.15.22:0/1227170081 conn(0x7f1530076fd0 msgr2=0x7f15300fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:04:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 shutdown_connections 2024-03-09T00:04:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:04:54.801+0000 7f153517e640 1 -- 172.21.15.22:0/1227170081 wait complete. 2024-03-09T00:04:55.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:54 smithi062 ceph-mon[36129]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-09T00:04:55.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:54 smithi062 ceph-mon[36129]: Unable to set osd_memory_target on smithi192 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-09T00:04:55.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:54 smithi062 ceph-mon[36129]: pgmap v327: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-03-09T00:04:55.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:54 smithi192 ceph-mon[35995]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-09T00:04:55.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:54 smithi192 ceph-mon[35995]: Unable to set osd_memory_target on smithi192 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-09T00:04:55.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:54 smithi192 ceph-mon[35995]: pgmap v327: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-03-09T00:04:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:54 smithi022 ceph-mon[31894]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-09T00:04:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:54 smithi022 ceph-mon[31894]: Unable to set osd_memory_target on smithi192 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-09T00:04:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:54 smithi022 ceph-mon[31894]: pgmap v327: 129 pgs: 129 active+clean; 95 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-03-09T00:04:55.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1058902 9.87 MB/sec execute 423 sec latency 71.274 ms 2024-03-09T00:04:56.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1060129 9.86 MB/sec execute 424 sec latency 52.753 ms 2024-03-09T00:04:57.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:56 smithi062 ceph-mon[36129]: pgmap v328: 129 pgs: 129 active+clean; 100 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 380 op/s 2024-03-09T00:04:57.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:56 smithi192 ceph-mon[35995]: pgmap v328: 129 pgs: 129 active+clean; 100 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 380 op/s 2024-03-09T00:04:57.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:56 smithi022 ceph-mon[31894]: pgmap v328: 129 pgs: 129 active+clean; 100 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 380 op/s 2024-03-09T00:04:57.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1062252 9.86 MB/sec execute 425 sec latency 17.450 ms 2024-03-09T00:04:58.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1063599 9.86 MB/sec execute 426 sec latency 20.914 ms 2024-03-09T00:04:59.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:04:58 smithi062 ceph-mon[36129]: pgmap v329: 129 pgs: 129 active+clean; 100 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-03-09T00:04:59.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:04:58 smithi192 ceph-mon[35995]: pgmap v329: 129 pgs: 129 active+clean; 100 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-03-09T00:04:59.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:04:58 smithi022 ceph-mon[31894]: pgmap v329: 129 pgs: 129 active+clean; 100 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-03-09T00:04:59.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1065769 9.86 MB/sec execute 427 sec latency 17.470 ms 2024-03-09T00:05:00.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1066254 9.84 MB/sec execute 428 sec latency 426.438 ms 2024-03-09T00:05:01.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:00 smithi062 ceph-mon[36129]: pgmap v330: 129 pgs: 129 active+clean; 91 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 262 op/s 2024-03-09T00:05:01.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:00 smithi192 ceph-mon[35995]: pgmap v330: 129 pgs: 129 active+clean; 91 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 262 op/s 2024-03-09T00:05:01.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:00 smithi022 ceph-mon[31894]: pgmap v330: 129 pgs: 129 active+clean; 91 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 262 op/s 2024-03-09T00:05:01.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1066254 9.82 MB/sec execute 429 sec latency 1426.543 ms 2024-03-09T00:05:02.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1066254 9.80 MB/sec execute 430 sec latency 2426.660 ms 2024-03-09T00:05:03.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:02 smithi062 ceph-mon[36129]: pgmap v331: 129 pgs: 129 active+clean; 89 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-03-09T00:05:03.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:02 smithi192 ceph-mon[35995]: pgmap v331: 129 pgs: 129 active+clean; 89 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-03-09T00:05:03.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:02 smithi022 ceph-mon[31894]: pgmap v331: 129 pgs: 129 active+clean; 89 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-03-09T00:05:03.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1066254 9.77 MB/sec execute 431 sec latency 3426.717 ms 2024-03-09T00:05:04.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1066367 9.75 MB/sec execute 432 sec latency 4331.794 ms 2024-03-09T00:05:05.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:04 smithi062 ceph-mon[36129]: pgmap v332: 129 pgs: 129 active+clean; 89 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-03-09T00:05:05.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:04 smithi192 ceph-mon[35995]: pgmap v332: 129 pgs: 129 active+clean; 89 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-03-09T00:05:05.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:04 smithi022 ceph-mon[31894]: pgmap v332: 129 pgs: 129 active+clean; 89 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-03-09T00:05:05.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1066717 9.73 MB/sec execute 433 sec latency 173.508 ms 2024-03-09T00:05:06.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1068450 9.73 MB/sec execute 434 sec latency 80.139 ms 2024-03-09T00:05:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:06 smithi062 ceph-mon[36129]: pgmap v333: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-03-09T00:05:07.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:06 smithi192 ceph-mon[35995]: pgmap v333: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-03-09T00:05:07.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:06 smithi022 ceph-mon[31894]: pgmap v333: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-03-09T00:05:07.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1069832 9.73 MB/sec execute 435 sec latency 22.690 ms 2024-03-09T00:05:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 -- 172.21.15.22:0/2078891969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0100230 msgr2=0x7f50c0100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 --2- 172.21.15.22:0/2078891969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0100230 0x7f50c0100610 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009900 tx=0x7f50b002bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 -- 172.21.15.22:0/2078891969 shutdown_connections 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 --2- 172.21.15.22:0/2078891969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c0105300 0x7f50c010ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 --2- 172.21.15.22:0/2078891969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c0100be0 0x7f50c0104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 --2- 172.21.15.22:0/2078891969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0100230 0x7f50c0100610 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 -- 172.21.15.22:0/2078891969 >> 172.21.15.22:0/2078891969 conn(0x7f50c00fbaf0 msgr2=0x7f50c00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 -- 172.21.15.22:0/2078891969 shutdown_connections 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 -- 172.21.15.22:0/2078891969 wait complete. 2024-03-09T00:05:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.896+0000 7f50c4da2640 1 Processor -- start 2024-03-09T00:05:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50c4da2640 1 -- start start 2024-03-09T00:05:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50c4da2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c0100230 0x7f50c019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50c4da2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c0105300 0x7f50c019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50c4da2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0197810 0x7f50c0197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50c4da2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0110400 con 0x7f50c0197810 2024-03-09T00:05:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50c4da2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0110280 con 0x7f50c0105300 2024-03-09T00:05:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50c4da2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0110580 con 0x7f50c0100230 2024-03-09T00:05:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0197810 0x7f50c0197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c0100230 0x7f50c019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0197810 0x7f50c0197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55558/0 (socket says 172.21.15.22:55558) 2024-03-09T00:05:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bffff640 1 -- 172.21.15.22:0/3722133524 learned_addr learned my addr 172.21.15.22:0/3722133524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50beffd640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c0105300 0x7f50c019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bf7fe640 1 -- 172.21.15.22:0/3722133524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c0105300 msgr2=0x7f50c019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c0105300 0x7f50c019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bf7fe640 1 -- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0197810 msgr2=0x7f50c0197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0197810 0x7f50c0197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.897+0000 7f50bf7fe640 1 -- 172.21.15.22:0/3722133524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c01983a0 con 0x7f50c0100230 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.898+0000 7f50bffff640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0197810 0x7f50c0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.898+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c0100230 0x7f50c019d6a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50b002c170 tx=0x7f50b0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.898+0000 7f50bcff9640 1 -- 172.21.15.22:0/3722133524 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0043020 con 0x7f50c0100230 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.898+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c0198630 con 0x7f50c0100230 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.898+0000 7f50bcff9640 1 -- 172.21.15.22:0/3722133524 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50b00054e0 con 0x7f50c0100230 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.898+0000 7f50bcff9640 1 -- 172.21.15.22:0/3722133524 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0003b80 con 0x7f50c0100230 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.898+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c01a4830 con 0x7f50c0100230 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.900+0000 7f50bcff9640 1 -- 172.21.15.22:0/3722133524 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f50b0046020 con 0x7f50c0100230 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.900+0000 7f50c8be6740 1 Processor -- start 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.900+0000 7f50c8be6740 1 -- start start 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d8d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d98b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50c8be6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce89cf15f0 con 0x55ce89d96730 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50c8be6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce89c9a930 con 0x55ce89d8d1b0 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50c8be6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce89ced690 con 0x55ce89d8df20 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50bcff9640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50a80a29e0 0x7f50a80a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d98b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50bffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d98b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55560/0 (socket says 172.21.15.22:55560) 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50bffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d8cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44720/0 (socket says 172.21.15.22:44720) 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 -- 172.21.15.22:0/1450901912 learned_addr learned my addr 172.21.15.22:0/1450901912 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50bffff640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50a80a29e0 0x7f50a80a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d8d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50bcff9640 1 -- 172.21.15.22:0/3722133524 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f50b00f1090 con 0x7f50c0100230 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 -- 172.21.15.22:0/1450901912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 msgr2=0x55ce89d8cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d8cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 -- 172.21.15.22:0/1450901912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 msgr2=0x55ce89d8d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d8d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 -- 172.21.15.22:0/1450901912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce89a06210 con 0x55ce89d96730 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50bffff640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50a80a29e0 0x7f50a80a4ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009aa0 tx=0x7f50b8009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.901+0000 7f50beffd640 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d98b20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009c20 tx=0x7f50b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b837495e0a891952 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50a5ffb640 1 -- 172.21.15.22:0/1450901912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4004230 con 0x55ce89d96730 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50a5ffb640 1 -- 172.21.15.22:0/1450901912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50b40043d0 con 0x55ce89d96730 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50a5ffb640 1 -- 172.21.15.22:0/1450901912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4004d40 con 0x55ce89d96730 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 -- 172.21.15.22:0/1450901912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 msgr2=0x55ce89d98b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d98b20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009c20 tx=0x7f50b4011810 comp rx=0 tx=0).stop 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 -- 172.21.15.22:0/1450901912 shutdown_connections 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d98b20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d8d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1450901912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 -- 172.21.15.22:0/1450901912 >> 172.21.15.22:0/1450901912 conn(0x55ce89d545b0 msgr2=0x55ce89d87ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 -- 172.21.15.22:0/1450901912 shutdown_connections 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.902+0000 7f50c8be6740 1 -- 172.21.15.22:0/1450901912 wait complete. 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 -- start start 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 Processor -- start 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8d1b0 0x55ce89d93710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8df20 0x55ce89d93c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d94190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce89d99460 con 0x55ce89d8d1b0 2024-03-09T00:05:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce89cadd90 con 0x55ce89d8df20 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce89d995e0 con 0x55ce89d96730 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8d1b0 0x55ce89d93710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50c8be6740 10 client.0 _ll_register_callbacks cb 0x55ce89a05a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8d1b0 0x55ce89d93710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55576/0 (socket says 172.21.15.22:55576) 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50bf7fe640 1 -- 172.21.15.22:0/4009954468 learned_addr learned my addr 172.21.15.22:0/4009954468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50bffff640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d94190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.903+0000 7f50beffd640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8df20 0x55ce89d93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50bf7fe640 1 -- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 msgr2=0x55ce89d94190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d94190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50bf7fe640 1 -- 172.21.15.22:0/4009954468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8df20 msgr2=0x55ce89d93c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8df20 0x55ce89d93c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50bf7fe640 1 -- 172.21.15.22:0/4009954468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce89ad2620 con 0x55ce89d8d1b0 2024-03-09T00:05:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50beffd640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8df20 0x55ce89d93c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8d1b0 0x55ce89d93710 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009900 tx=0x7f50b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589e683d2d22fbc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f509effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f509effd640 1 -- 172.21.15.22:0/4009954468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0004c80 con 0x55ce89d8d1b0 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f509effd640 1 -- 172.21.15.22:0/4009954468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f50b00c1120 con 0x55ce89d8d1b0 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50b40043d0 con 0x55ce89d8d1b0 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f509effd640 1 -- 172.21.15.22:0/4009954468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b0005690 con 0x55ce89d8d1b0 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50c8be6740 10 client.18909 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ce89da5790 con 0x55ce89d8d1b0 2024-03-09T00:05:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f509effd640 1 -- 172.21.15.22:0/4009954468 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50b40043d0 con 0x55ce89d8d1b0 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f509effd640 1 -- 172.21.15.22:0/4009954468 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f50b00ee600 con 0x55ce89d8d1b0 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50c8be6740 10 client.18909 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.904+0000 7f50c8be6740 10 client.18909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.905+0000 7f50c8be6740 20 client.18909 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.905+0000 7f50c8be6740 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89da6b30 0x55ce89da8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.905+0000 7f50c8be6740 4 client.18909 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.905+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce89cfeb30 con 0x55ce89da6b30 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.905+0000 7f50beffd640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89da6b30 0x55ce89da8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.905+0000 7f50beffd640 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89da6b30 0x55ce89da8f60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6adae9c4e91262 server_cookie=471d17c871989b3 in_seq=0 out_seq=0 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.905+0000 7f509effd640 10 client.18909 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f509effd640 1 -- 172.21.15.22:0/4009954468 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ce89cfeb30 con 0x55ce89da6b30 2024-03-09T00:05:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f509effd640 10 client.18909 handle_command_reply: tid=0 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 client.18909 shutdown 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8d1b0 msgr2=0x55ce89d93710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8d1b0 0x55ce89d93710 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009900 tx=0x7f50b0005fd0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 shutdown_connections 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89da6b30 0x55ce89da8f60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d94190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8df20 0x55ce89d93c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f509effd640 0 client.18909 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 --2- 172.21.15.22:0/4009954468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8d1b0 0x55ce89d93710 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 >> 172.21.15.22:0/4009954468 conn(0x55ce89d545b0 msgr2=0x55ce89d89bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 shutdown_connections 2024-03-09T00:05:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 1 -- 172.21.15.22:0/4009954468 wait complete. 2024-03-09T00:05:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.906+0000 7f50c8be6740 20 client.18909 trim_cache size 0 max 16384 2024-03-09T00:05:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.919+0000 7f50c8be6740 1 Processor -- start 2024-03-09T00:05:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.919+0000 7f50c8be6740 1 -- start start 2024-03-09T00:05:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.919+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d91fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.919+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8df20 0x55ce89d92520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d90990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50c8be6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01b08e0 con 0x55ce89d8df20 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50c8be6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01ae480 con 0x55ce89d8d1b0 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50c8be6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01ac540 con 0x55ce89d96730 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8df20 0x55ce89d92520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8df20 0x55ce89d92520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55580/0 (socket says 172.21.15.22:55580) 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 -- 172.21.15.22:0/530259080 learned_addr learned my addr 172.21.15.22:0/530259080 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d91fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50bffff640 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d90990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 -- 172.21.15.22:0/530259080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 msgr2=0x55ce89d90990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d90990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 -- 172.21.15.22:0/530259080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 msgr2=0x55ce89d91fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d91fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 -- 172.21.15.22:0/530259080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce89d822f0 con 0x55ce89d8df20 2024-03-09T00:05:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50beffd640 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8df20 0x55ce89d92520 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009850 tx=0x7f50b40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1a54ed4d12c7be server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50a57fa640 1 -- 172.21.15.22:0/530259080 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4009dd0 con 0x55ce89d8df20 2024-03-09T00:05:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50a57fa640 1 -- 172.21.15.22:0/530259080 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50b401f040 con 0x55ce89d8df20 2024-03-09T00:05:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.920+0000 7f50a57fa640 1 -- 172.21.15.22:0/530259080 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b40268a0 con 0x55ce89d8df20 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 -- 172.21.15.22:0/530259080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8df20 msgr2=0x55ce89d92520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8df20 0x55ce89d92520 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009850 tx=0x7f50b40047e0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 -- 172.21.15.22:0/530259080 shutdown_connections 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d96730 0x55ce89d90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d8df20 0x55ce89d92520 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 --2- 172.21.15.22:0/530259080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d91fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 -- 172.21.15.22:0/530259080 >> 172.21.15.22:0/530259080 conn(0x55ce89d545b0 msgr2=0x55ce89d8ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 -- 172.21.15.22:0/530259080 shutdown_connections 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 -- 172.21.15.22:0/530259080 wait complete. 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.921+0000 7f50c8be6740 1 -- start start 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 1 Processor -- start 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d9c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d9c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01acb40 con 0x55ce89d96730 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01ade20 con 0x55ce89d8d1b0 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01ad160 con 0x55ce89d8df20 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50c8be6740 10 client.0 _ll_register_callbacks cb 0x7f50b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d9c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d9cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55590/0 (socket says 172.21.15.22:55590) 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d9c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:41192/0 (socket says 172.21.15.22:41192) 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bffff640 1 -- 172.21.15.22:0/1455185205 learned_addr learned my addr 172.21.15.22:0/1455185205 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50beffd640 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d9c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 -- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 msgr2=0x55ce89d9c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d9c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 -- 172.21.15.22:0/1455185205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 msgr2=0x55ce89d9cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d9cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 -- 172.21.15.22:0/1455185205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce89d8c480 con 0x55ce89d8d1b0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.922+0000 7f50bf7fe640 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d9c170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f50b00b97b0 tx=0x7f50b00f1310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4daa97fbbf16f776 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f509effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f509effd640 1 -- 172.21.15.22:0/1455185205 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00bcee0 con 0x55ce89d8d1b0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50b401f040 con 0x55ce89d8d1b0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f509effd640 1 -- 172.21.15.22:0/1455185205 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f50b00ee570 con 0x55ce89d8d1b0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f509effd640 1 -- 172.21.15.22:0/1455185205 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b0004c80 con 0x55ce89d8d1b0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 10 client.28256 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ce89d8c480 con 0x55ce89d8d1b0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f509effd640 1 -- 172.21.15.22:0/1455185205 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50b401f040 con 0x55ce89d8d1b0 2024-03-09T00:05:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f509effd640 1 -- 172.21.15.22:0/1455185205 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f50b00dd0b0 con 0x55ce89d8d1b0 2024-03-09T00:05:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 10 client.28256 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 10 client.28256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 20 client.28256 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89dfe490 0x55ce89e00880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 4 client.28256 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:05:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ce89cf8e80 con 0x55ce89dfe490 2024-03-09T00:05:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.923+0000 7f50beffd640 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89dfe490 0x55ce89e00880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.924+0000 7f509effd640 10 client.28256 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:07.924+0000 7f50beffd640 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89dfe490 0x55ce89e00880 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce94adea4bfca9ad server_cookie=e0f14cc5800cfa76 in_seq=0 out_seq=0 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f509effd640 1 -- 172.21.15.22:0/1455185205 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ce89cf8e80 con 0x55ce89dfe490 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f509effd640 10 client.28256 handle_command_reply: tid=0 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 client.28256 shutdown 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 msgr2=0x55ce89d9c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d9c170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f50b00b97b0 tx=0x7f50b00f1310 comp rx=0 tx=0).stop 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 shutdown_connections 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ce89dfe490 0x55ce89e00880 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f509effd640 0 client.28256 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce89d96730 0x55ce89d9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce89d8df20 0x55ce89d9c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 --2- 172.21.15.22:0/1455185205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce89d8d1b0 0x55ce89d9c170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 >> 172.21.15.22:0/1455185205 conn(0x55ce89d545b0 msgr2=0x55ce89d91f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 shutdown_connections 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 1 -- 172.21.15.22:0/1455185205 wait complete. 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.187+0000 7f50c8be6740 20 client.28256 trim_cache size 0 max 16384 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:05:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:05:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50a80a29e0 msgr2=0x7f50a80a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50a80a29e0 0x7f50a80a4ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009aa0 tx=0x7f50b8009500 comp rx=0 tx=0).stop 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c0100230 msgr2=0x7f50c019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c0100230 0x7f50c019d6a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50b002c170 tx=0x7f50b0004360 comp rx=0 tx=0).stop 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 shutdown_connections 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50a80a29e0 0x7f50a80a4ea0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50c0197810 0x7f50c0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c0105300 0x7f50c019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 --2- 172.21.15.22:0/3722133524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c0100230 0x7f50c019d6a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 >> 172.21.15.22:0/3722133524 conn(0x7f50c00fbaf0 msgr2=0x7f50c0069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 shutdown_connections 2024-03-09T00:05:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:08.188+0000 7f50c4da2640 1 -- 172.21.15.22:0/3722133524 wait complete. 2024-03-09T00:05:08.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1071153 9.72 MB/sec execute 436 sec latency 176.093 ms 2024-03-09T00:05:09.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:08 smithi062 ceph-mon[36129]: pgmap v334: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-03-09T00:05:09.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:08 smithi192 ceph-mon[35995]: pgmap v334: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-03-09T00:05:09.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:08 smithi022 ceph-mon[31894]: pgmap v334: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-03-09T00:05:09.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1073034 9.73 MB/sec execute 437 sec latency 16.339 ms 2024-03-09T00:05:10.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1074642 9.72 MB/sec execute 438 sec latency 20.398 ms 2024-03-09T00:05:11.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:10 smithi062 ceph-mon[36129]: pgmap v335: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-03-09T00:05:11.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:10 smithi192 ceph-mon[35995]: pgmap v335: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-03-09T00:05:11.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:10 smithi022 ceph-mon[31894]: pgmap v335: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-03-09T00:05:11.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1076404 9.72 MB/sec execute 439 sec latency 44.109 ms 2024-03-09T00:05:12.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1077515 9.71 MB/sec execute 440 sec latency 21.906 ms 2024-03-09T00:05:13.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:12 smithi062 ceph-mon[36129]: pgmap v336: 129 pgs: 129 active+clean; 87 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-03-09T00:05:13.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:12 smithi192 ceph-mon[35995]: pgmap v336: 129 pgs: 129 active+clean; 87 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-03-09T00:05:13.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:12 smithi022 ceph-mon[31894]: pgmap v336: 129 pgs: 129 active+clean; 87 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-03-09T00:05:13.729 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1079946 9.72 MB/sec execute 441 sec latency 15.503 ms 2024-03-09T00:05:14.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1081092 9.71 MB/sec execute 442 sec latency 20.368 ms 2024-03-09T00:05:15.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:14 smithi062 ceph-mon[36129]: pgmap v337: 129 pgs: 129 active+clean; 87 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-03-09T00:05:15.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:14 smithi192 ceph-mon[35995]: pgmap v337: 129 pgs: 129 active+clean; 87 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-03-09T00:05:15.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:14 smithi022 ceph-mon[31894]: pgmap v337: 129 pgs: 129 active+clean; 87 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-03-09T00:05:15.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1083488 9.72 MB/sec execute 443 sec latency 14.395 ms 2024-03-09T00:05:16.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1084233 9.70 MB/sec execute 444 sec latency 73.541 ms 2024-03-09T00:05:17.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:16 smithi062 ceph-mon[36129]: pgmap v338: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-03-09T00:05:17.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:16 smithi192 ceph-mon[35995]: pgmap v338: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-03-09T00:05:17.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:16 smithi022 ceph-mon[31894]: pgmap v338: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 309 op/s 2024-03-09T00:05:17.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:05:17.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:05:17.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1086920 9.72 MB/sec execute 445 sec latency 15.201 ms 2024-03-09T00:05:17.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 -- 172.21.15.22:0/662874372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1003d0 msgr2=0x7fc78c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 --2- 172.21.15.22:0/662874372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1003d0 0x7fc78c1007b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc788009900 tx=0x7fc78802bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 -- 172.21.15.22:0/662874372 shutdown_connections 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 --2- 172.21.15.22:0/662874372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc78c1054a0 0x7fc78c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 --2- 172.21.15.22:0/662874372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc78c100d80 0x7fc78c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 --2- 172.21.15.22:0/662874372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1003d0 0x7fc78c1007b0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 -- 172.21.15.22:0/662874372 >> 172.21.15.22:0/662874372 conn(0x7fc78c0772f0 msgr2=0x7fc78c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 -- 172.21.15.22:0/662874372 shutdown_connections 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.750+0000 7fc79430a640 1 -- 172.21.15.22:0/662874372 wait complete. 2024-03-09T00:05:17.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 Processor -- start 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 -- start start 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc78c1003d0 0x7fc78c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1054a0 0x7fc78c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc78c1994a0 0x7fc78c1a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c110550 con 0x7fc78c1054a0 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1103d0 con 0x7fc78c1003d0 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc79430a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1106d0 con 0x7fc78c1994a0 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1054a0 0x7fc78c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc7928a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc78c1994a0 0x7fc78c1a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1054a0 0x7fc78c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35714/0 (socket says 172.21.15.22:35714) 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc7918a5640 1 -- 172.21.15.22:0/4024034524 learned_addr learned my addr 172.21.15.22:0/4024034524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:17.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.751+0000 7fc7920a6640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc78c1003d0 0x7fc78c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7928a7640 1 -- 172.21.15.22:0/4024034524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc78c1003d0 msgr2=0x7fc78c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7928a7640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc78c1003d0 0x7fc78c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7928a7640 1 -- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1054a0 msgr2=0x7fc78c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7928a7640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1054a0 0x7fc78c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7928a7640 1 -- 172.21.15.22:0/4024034524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c1a4220 con 0x7fc78c1994a0 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7928a7640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc78c1994a0 0x7fc78c1a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc78400ec50 tx=0x7fc78400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7837fe640 1 -- 172.21.15.22:0/4024034524 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784003b70 con 0x7fc78c1994a0 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7837fe640 1 -- 172.21.15.22:0/4024034524 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc784003d10 con 0x7fc78c1994a0 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c1a4510 con 0x7fc78c1994a0 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc7837fe640 1 -- 172.21.15.22:0/4024034524 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784010890 con 0x7fc78c1994a0 2024-03-09T00:05:17.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.752+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c1a4a60 con 0x7fc78c1994a0 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.755+0000 7fc7837fe640 1 -- 172.21.15.22:0/4024034524 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc784003fc0 con 0x7fc78c1994a0 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 Processor -- start 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 -- start start 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d8080 0x55d8b23e0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23e0c60 0x55d8b23e3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b233ae00 con 0x55d8b23d7760 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b22e3b10 con 0x55d8b23e0c60 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc796fbf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b2336ea0 con 0x55d8b23d8080 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.756+0000 7fc7837fe640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7780a29f0 0x7fc7780a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7928a7640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7780a29f0 0x7fc7780a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23d7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35722/0 (socket says 172.21.15.22:35722) 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 -- 172.21.15.22:0/4052579320 learned_addr learned my addr 172.21.15.22:0/4052579320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7920a6640 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d8080 0x55d8b23e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23e0c60 0x55d8b23e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7837fe640 1 -- 172.21.15.22:0/4024034524 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc7840cf710 con 0x7fc78c1994a0 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 -- 172.21.15.22:0/4052579320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d8080 msgr2=0x55d8b23e0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d8080 0x55d8b23e0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 -- 172.21.15.22:0/4052579320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23e0c60 msgr2=0x55d8b23e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23e0c60 0x55d8b23e3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 -- 172.21.15.22:0/4052579320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8b21197d0 con 0x55d8b23d7760 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7920a6640 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d8080 0x55d8b23e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:17.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7928a7640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7780a29f0 0x7fc7780a4eb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc78409b900 tx=0x7fc7840974f0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc7918a5640 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23d7b40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0138c0 tx=0x7fc77c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c32849f6093f4d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc780bfe640 1 -- 172.21.15.22:0/4052579320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c013e40 con 0x55d8b23d7760 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc780bfe640 1 -- 172.21.15.22:0/4052579320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77c013170 con 0x55d8b23d7760 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.757+0000 7fc780bfe640 1 -- 172.21.15.22:0/4052579320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0109e0 con 0x55d8b23d7760 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 -- 172.21.15.22:0/4052579320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 msgr2=0x55d8b23d7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23d7b40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0138c0 tx=0x7fc77c009820 comp rx=0 tx=0).stop 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 -- 172.21.15.22:0/4052579320 shutdown_connections 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23e0c60 0x55d8b23e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d8080 0x55d8b23e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 --2- 172.21.15.22:0/4052579320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23d7b40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 -- 172.21.15.22:0/4052579320 >> 172.21.15.22:0/4052579320 conn(0x55d8b23d1e90 msgr2=0x55d8b23d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:17.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 -- 172.21.15.22:0/4052579320 shutdown_connections 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.758+0000 7fc796fbf740 1 -- 172.21.15.22:0/4052579320 wait complete. 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 -- start start 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 Processor -- start 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b23e3a50 con 0x55d8b23e0c60 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b22f70a0 con 0x55d8b23d8080 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8b23e3bd0 con 0x55d8b23d7760 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc796fbf740 10 client.0 _ll_register_callbacks cb 0x55d8b22f6a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23ddca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35728/0 (socket says 172.21.15.22:35728) 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7918a5640 1 -- 172.21.15.22:0/1690300218 learned_addr learned my addr 172.21.15.22:0/1690300218 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7920a6640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7918a5640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7920a6640 1 -- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 msgr2=0x55d8b23e03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7920a6640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23e03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7920a6640 1 -- 172.21.15.22:0/1690300218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 msgr2=0x55d8b23ddca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7920a6640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23ddca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7920a6640 1 -- 172.21.15.22:0/1690300218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8b204f3f0 con 0x55d8b23d8080 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.759+0000 7fc7918a5640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc7920a6640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23dd760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc7880095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e53a1fc30288e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:05:17.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc775ffb640 1 -- 172.21.15.22:0/1690300218 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7880042c0 con 0x55d8b23d8080 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc77c013170 con 0x55d8b23d8080 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc775ffb640 1 -- 172.21.15.22:0/1690300218 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc788004460 con 0x55d8b23d8080 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc775ffb640 1 -- 172.21.15.22:0/1690300218 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7880046c0 con 0x55d8b23d8080 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 10 client.28268 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d8b200f0f0 con 0x55d8b23d8080 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc775ffb640 1 -- 172.21.15.22:0/1690300218 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77c013170 con 0x55d8b23d8080 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc775ffb640 1 -- 172.21.15.22:0/1690300218 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc7880035f0 con 0x55d8b23d8080 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 10 client.28268 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 10 client.28268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 20 client.28268 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b23f06a0 0x55d8b23f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 4 client.28268 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:17.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.760+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8b2348340 con 0x55d8b23f06a0 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.761+0000 7fc7918a5640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b23f06a0 0x55d8b23f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.761+0000 7fc7918a5640 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b23f06a0 0x55d8b23f2ae0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b072b9a3b6b1b8 server_cookie=fb373e6262e843cb in_seq=0 out_seq=0 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.761+0000 7fc775ffb640 10 client.28268 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc775ffb640 1 -- 172.21.15.22:0/1690300218 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8b2348340 con 0x55d8b23f06a0 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc775ffb640 10 client.28268 handle_command_reply: tid=0 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 client.28268 shutdown 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 msgr2=0x55d8b23dd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23dd760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc7880095d0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 shutdown_connections 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b23f06a0 0x55d8b23f2ae0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc775ffb640 0 client.28268 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:17.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23dd760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1690300218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23e03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 >> 172.21.15.22:0/1690300218 conn(0x55d8b23d1e90 msgr2=0x55d8b23d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:17.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 shutdown_connections 2024-03-09T00:05:17.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 1 -- 172.21.15.22:0/1690300218 wait complete. 2024-03-09T00:05:17.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.762+0000 7fc796fbf740 20 client.28268 trim_cache size 0 max 16384 2024-03-09T00:05:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 Processor -- start 2024-03-09T00:05:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 -- start start 2024-03-09T00:05:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23dfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1ae940 con 0x55d8b23e0c60 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1ac4e0 con 0x55d8b23d8080 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc796fbf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1aa5a0 con 0x55d8b23d7760 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23dfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.778+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7920a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23dfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35740/0 (socket says 172.21.15.22:35740) 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 -- 172.21.15.22:0/1205616524 learned_addr learned my addr 172.21.15.22:0/1205616524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 -- 172.21.15.22:0/1205616524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 msgr2=0x55d8b23f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 -- 172.21.15.22:0/1205616524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 msgr2=0x55d8b23f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 -- 172.21.15.22:0/1205616524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8b23dbcb0 con 0x55d8b23e0c60 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7920a6640 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:17.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc7918a5640 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23dfe90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009250 tx=0x7fc77c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea48617f42d3ee29 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc777fff640 1 -- 172.21.15.22:0/1205616524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c009d30 con 0x55d8b23e0c60 2024-03-09T00:05:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc777fff640 1 -- 172.21.15.22:0/1205616524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77c005500 con 0x55d8b23e0c60 2024-03-09T00:05:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.779+0000 7fc777fff640 1 -- 172.21.15.22:0/1205616524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0306e0 con 0x55d8b23e0c60 2024-03-09T00:05:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- 172.21.15.22:0/1205616524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 msgr2=0x55d8b23dfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23dfe90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009250 tx=0x7fc77c01c040 comp rx=0 tx=0).stop 2024-03-09T00:05:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- 172.21.15.22:0/1205616524 shutdown_connections 2024-03-09T00:05:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23e0c60 0x55d8b23dfe90 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 --2- 172.21.15.22:0/1205616524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23d7760 0x55d8b23f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- 172.21.15.22:0/1205616524 >> 172.21.15.22:0/1205616524 conn(0x55d8b23d1e90 msgr2=0x55d8b23ec8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- 172.21.15.22:0/1205616524 shutdown_connections 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- 172.21.15.22:0/1205616524 wait complete. 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- start start 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 Processor -- start 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23e0c60 0x55d8b23e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1aaba0 con 0x55d8b23d7760 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1abe80 con 0x55d8b23d8080 2024-03-09T00:05:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.780+0000 7fc796fbf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1ab1c0 con 0x55d8b23e0c60 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc796fbf740 10 client.0 _ll_register_callbacks cb 0x7fc788009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23e7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35742/0 (socket says 172.21.15.22:35742) 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 -- 172.21.15.22:0/3707837066 learned_addr learned my addr 172.21.15.22:0/3707837066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7920a6640 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23e0c60 0x55d8b23e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 -- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23e0c60 msgr2=0x55d8b23e5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23e0c60 0x55d8b23e5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 -- 172.21.15.22:0/3707837066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 msgr2=0x55d8b23e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23e7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 -- 172.21.15.22:0/3707837066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8b23e66d0 con 0x55d8b23d7760 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc7918a5640 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23e7a10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00ff40 tx=0x7fc77c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fe65ac29921895 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc775ffb640 1 -- 172.21.15.22:0/3707837066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0131f0 con 0x55d8b23d7760 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc775ffb640 1 -- 172.21.15.22:0/3707837066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc77c00faa0 con 0x55d8b23d7760 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc77c005500 con 0x55d8b23d7760 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc775ffb640 1 -- 172.21.15.22:0/3707837066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc77c030460 con 0x55d8b23d7760 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc796fbf740 10 client.18933 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d8b2447600 con 0x55d8b23d7760 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.781+0000 7fc775ffb640 1 -- 172.21.15.22:0/3707837066 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77c005500 con 0x55d8b23d7760 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc775ffb640 1 -- 172.21.15.22:0/3707837066 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc77c030040 con 0x55d8b23d7760 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc796fbf740 10 client.18933 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc796fbf740 10 client.18933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc796fbf740 20 client.18933 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc796fbf740 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b2447a80 0x55d8b2449e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc796fbf740 4 client.18933 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d8b2342690 con 0x55d8b2447a80 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc7920a6640 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b2447a80 0x55d8b2449e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc7920a6640 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b2447a80 0x55d8b2449e70 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35516a461317e31 server_cookie=69ebb12bb9cc2df7 in_seq=0 out_seq=0 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.782+0000 7fc775ffb640 10 client.18933 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.783+0000 7fc775ffb640 1 -- 172.21.15.22:0/3707837066 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d8b2342690 con 0x55d8b2447a80 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.783+0000 7fc775ffb640 10 client.18933 handle_command_reply: tid=0 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.783+0000 7fc796fbf740 1 client.18933 shutdown 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 msgr2=0x55d8b23e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23e7a10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00ff40 tx=0x7fc77c005850 comp rx=0 tx=0).stop 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 shutdown_connections 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55d8b2447a80 0x55d8b2449e70 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d8b23e0c60 0x55d8b23e5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b23d8080 0x55d8b23e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 --2- 172.21.15.22:0/3707837066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8b23d7760 0x55d8b23e7a10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 >> 172.21.15.22:0/3707837066 conn(0x55d8b23d1e90 msgr2=0x55d8b23f7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc775ffb640 0 client.18933 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 shutdown_connections 2024-03-09T00:05:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 1 -- 172.21.15.22:0/3707837066 wait complete. 2024-03-09T00:05:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc796fbf740 20 client.18933 trim_cache size 0 max 16384 2024-03-09T00:05:17.787 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:05:17.787 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:05:17.787 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:05:17.787 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7780a29f0 msgr2=0x7fc7780a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7780a29f0 0x7fc7780a4eb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc78409b900 tx=0x7fc7840974f0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc78c1994a0 msgr2=0x7fc78c1a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc78c1994a0 0x7fc78c1a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc78400ec50 tx=0x7fc78400c6a0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 shutdown_connections 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7780a29f0 0x7fc7780a4eb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc78c1994a0 0x7fc78c1a3a50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc78c1054a0 0x7fc78c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 --2- 172.21.15.22:0/4024034524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc78c1003d0 0x7fc78c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 >> 172.21.15.22:0/4024034524 conn(0x7fc78c0772f0 msgr2=0x7fc78c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.784+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 shutdown_connections 2024-03-09T00:05:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.785+0000 7fc79430a640 1 -- 172.21.15.22:0/4024034524 wait complete. 2024-03-09T00:05:17.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:05:17.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:05:17.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 -- 172.21.15.22:0/828774604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0018100400 msgr2=0x7f00181007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 --2- 172.21.15.22:0/828774604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0018100400 0x7f00181007e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f0010009900 tx=0x7f001002bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:17.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 -- 172.21.15.22:0/828774604 shutdown_connections 2024-03-09T00:05:17.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 --2- 172.21.15.22:0/828774604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00181054d0 0x7f001810d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 --2- 172.21.15.22:0/828774604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0018100db0 0x7f0018104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 --2- 172.21.15.22:0/828774604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0018100400 0x7f00181007e0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 -- 172.21.15.22:0/828774604 >> 172.21.15.22:0/828774604 conn(0x7f0018076570 msgr2=0x7f0018076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:17.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 -- 172.21.15.22:0/828774604 shutdown_connections 2024-03-09T00:05:17.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.901+0000 7f00203ab640 1 -- 172.21.15.22:0/828774604 wait complete. 2024-03-09T00:05:17.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 Processor -- start 2024-03-09T00:05:17.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 -- start start 2024-03-09T00:05:17.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0018100400 0x7f0018198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181054d0 0x7f0018198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0018199510 0x7f00181a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018110580 con 0x7f00181054d0 2024-03-09T00:05:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018110400 con 0x7f0018100400 2024-03-09T00:05:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f00203ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018110700 con 0x7f0018199510 2024-03-09T00:05:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f001d946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181054d0 0x7f0018198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f001d946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181054d0 0x7f0018198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35760/0 (socket says 172.21.15.22:35760) 2024-03-09T00:05:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f001d946640 1 -- 172.21.15.22:0/319651105 learned_addr learned my addr 172.21.15.22:0/319651105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f001e147640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0018100400 0x7f0018198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.902+0000 7f001e948640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0018199510 0x7f00181a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f001d946640 1 -- 172.21.15.22:0/319651105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0018199510 msgr2=0x7f00181a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f001d946640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0018199510 0x7f00181a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f001d946640 1 -- 172.21.15.22:0/319651105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0018100400 msgr2=0x7f0018198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f001d946640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0018100400 0x7f0018198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f001d946640 1 -- 172.21.15.22:0/319651105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00181a4290 con 0x7f00181054d0 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f001d946640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181054d0 0x7f0018198f80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f000c00e9f0 tx=0x7f000c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f00177fe640 1 -- 172.21.15.22:0/319651105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00ccc0 con 0x7f00181054d0 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00181a4580 con 0x7f00181054d0 2024-03-09T00:05:17.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f00177fe640 1 -- 172.21.15.22:0/319651105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f000c005500 con 0x7f00181054d0 2024-03-09T00:05:17.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f00177fe640 1 -- 172.21.15.22:0/319651105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c010690 con 0x7f00181054d0 2024-03-09T00:05:17.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.903+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00181a4a80 con 0x7f00181054d0 2024-03-09T00:05:17.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.904+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00181018d0 con 0x7f00181054d0 2024-03-09T00:05:17.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.905+0000 7f00177fe640 1 -- 172.21.15.22:0/319651105 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f000c0046c0 con 0x7f00181054d0 2024-03-09T00:05:17.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.906+0000 7f00177fe640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00000a2970 0x7f00000a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:17.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.906+0000 7f001e147640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00000a2970 0x7f00000a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:17.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.906+0000 7f00177fe640 1 -- 172.21.15.22:0/319651105 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f000c0cf630 con 0x7f00181054d0 2024-03-09T00:05:17.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.906+0000 7f001e147640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00000a2970 0x7f00000a4e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0010006010 tx=0x7f0010005dd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:17.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:17.910+0000 7f00177fe640 1 -- 172.21.15.22:0/319651105 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f000c09bf30 con 0x7f00181054d0 2024-03-09T00:05:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.144+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0018108700 con 0x7f00181054d0 2024-03-09T00:05:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.145+0000 7f00177fe640 1 -- 172.21.15.22:0/319651105 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f000c09b830 con 0x7f00181054d0 2024-03-09T00:05:18.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:05:18.140 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:05:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:05:18.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00000a2970 msgr2=0x7f00000a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00000a2970 0x7f00000a4e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0010006010 tx=0x7f0010005dd0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181054d0 msgr2=0x7f0018198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181054d0 0x7f0018198f80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f000c00e9f0 tx=0x7f000c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 shutdown_connections 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00000a2970 0x7f00000a4e30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0018199510 0x7f00181a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00181054d0 0x7f0018198f80 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 --2- 172.21.15.22:0/319651105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0018100400 0x7f0018198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 >> 172.21.15.22:0/319651105 conn(0x7f0018076570 msgr2=0x7f00180fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 shutdown_connections 2024-03-09T00:05:18.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.147+0000 7f00203ab640 1 -- 172.21.15.22:0/319651105 wait complete. 2024-03-09T00:05:18.154 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:05:18.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:05:18.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 -- 172.21.15.22:0/616834341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 msgr2=0x7f18fc10d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 --2- 172.21.15.22:0/616834341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc10d080 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009ca0 tx=0x7f18f402c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 -- 172.21.15.22:0/616834341 shutdown_connections 2024-03-09T00:05:18.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 --2- 172.21.15.22:0/616834341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc10d080 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 --2- 172.21.15.22:0/616834341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc100d90 0x7f18fc104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 --2- 172.21.15.22:0/616834341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc1003e0 0x7f18fc1007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 -- 172.21.15.22:0/616834341 >> 172.21.15.22:0/616834341 conn(0x7f18fc0773a0 msgr2=0x7f18fc0777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 -- 172.21.15.22:0/616834341 shutdown_connections 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.272+0000 7f1903562640 1 -- 172.21.15.22:0/616834341 wait complete. 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 Processor -- start 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 -- start start 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc1003e0 0x7f18fc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc1993c0 0x7f18fc1a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc110570 con 0x7f18fc1054b0 2024-03-09T00:05:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1103f0 con 0x7f18fc1993c0 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1903562640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1106f0 con 0x7f18fc1003e0 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1900afd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1900afd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35776/0 (socket says 172.21.15.22:35776) 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1900afd640 1 -- 172.21.15.22:0/2537543652 learned_addr learned my addr 172.21.15.22:0/2537543652 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f19012fe640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc1003e0 0x7f18fc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.273+0000 7f1901aff640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc1993c0 0x7f18fc1a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1900afd640 1 -- 172.21.15.22:0/2537543652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc1003e0 msgr2=0x7f18fc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1900afd640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc1003e0 0x7f18fc1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1900afd640 1 -- 172.21.15.22:0/2537543652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc1993c0 msgr2=0x7f18fc1a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1900afd640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc1993c0 0x7f18fc1a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1900afd640 1 -- 172.21.15.22:0/2537543652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc1a4170 con 0x7f18fc1054b0 2024-03-09T00:05:18.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1900afd640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc198e30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e9f0 tx=0x7f18ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f18f27fc640 1 -- 172.21.15.22:0/2537543652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00ccc0 con 0x7f18fc1054b0 2024-03-09T00:05:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc1a4460 con 0x7f18fc1054b0 2024-03-09T00:05:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f18f27fc640 1 -- 172.21.15.22:0/2537543652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18ec005500 con 0x7f18fc1054b0 2024-03-09T00:05:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f18f27fc640 1 -- 172.21.15.22:0/2537543652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec010690 con 0x7f18fc1054b0 2024-03-09T00:05:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.274+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc1a49b0 con 0x7f18fc1054b0 2024-03-09T00:05:18.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f18f27fc640 1 -- 172.21.15.22:0/2537543652 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f18ec0046c0 con 0x7f18fc1054b0 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 Processor -- start 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 -- start start 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb410ed0 0x55c0bb4112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb4117f0 0x55c0bb41e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb41eab0 0x55c0bb420ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bb3d8360 con 0x55c0bb410ed0 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bb374ec0 con 0x55c0bb41eab0 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f190621d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bb31dd30 con 0x55c0bb4117f0 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.277+0000 7f18f27fc640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18e80a2970 0x7f18e80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb4117f0 0x55c0bb41e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f1901aff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb410ed0 0x55c0bb4112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb4117f0 0x55c0bb41e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51722/0 (socket says 172.21.15.22:51722) 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f1901aff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb410ed0 0x55c0bb4112b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35780/0 (socket says 172.21.15.22:35780) 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 -- 172.21.15.22:0/2904948177 learned_addr learned my addr 172.21.15.22:0/2904948177 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f18f27fc640 1 -- 172.21.15.22:0/2537543652 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f18ec0cf540 con 0x7f18fc1054b0 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f1901aff640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18e80a2970 0x7f18e80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f1900afd640 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb41eab0 0x55c0bb420ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 -- 172.21.15.22:0/2904948177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb41eab0 msgr2=0x55c0bb420ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb41eab0 0x55c0bb420ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 -- 172.21.15.22:0/2904948177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb410ed0 msgr2=0x55c0bb4112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb410ed0 0x55c0bb4112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 -- 172.21.15.22:0/2904948177 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0bb31cbc0 con 0x55c0bb4117f0 2024-03-09T00:05:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f1900afd640 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb41eab0 0x55c0bb420ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f19012fe640 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb4117f0 0x55c0bb41e570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18f800bd10 tx=0x7f18f80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac0dcf0a4cf2b490 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f1901aff640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18e80a2970 0x7f18e80a4e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18f4004af0 tx=0x7f18f4005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f18e77fe640 1 -- 172.21.15.22:0/2904948177 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8004030 con 0x55c0bb4117f0 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f18e77fe640 1 -- 172.21.15.22:0/2904948177 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18f80041d0 con 0x55c0bb4117f0 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.278+0000 7f18e77fe640 1 -- 172.21.15.22:0/2904948177 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8011840 con 0x55c0bb4117f0 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 -- 172.21.15.22:0/2904948177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb4117f0 msgr2=0x55c0bb41e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb4117f0 0x55c0bb41e570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18f800bd10 tx=0x7f18f80107a0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 -- 172.21.15.22:0/2904948177 shutdown_connections 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb41eab0 0x55c0bb420ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb4117f0 0x55c0bb41e570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 --2- 172.21.15.22:0/2904948177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb410ed0 0x55c0bb4112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 -- 172.21.15.22:0/2904948177 >> 172.21.15.22:0/2904948177 conn(0x55c0bb40b8d0 msgr2=0x55c0bb40bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 -- 172.21.15.22:0/2904948177 shutdown_connections 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.279+0000 7f190621d740 1 -- 172.21.15.22:0/2904948177 wait complete. 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 -- start start 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 Processor -- start 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb410ed0 0x55c0bb41bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb4117f0 0x55c0bb41c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb41eab0 0x55c0bb41c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bb3134a0 con 0x55c0bb4117f0 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bb370f60 con 0x55c0bb410ed0 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bb421870 con 0x55c0bb41eab0 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f190621d740 10 client.0 _ll_register_callbacks cb 0x55c0bb36ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:18.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb4117f0 0x55c0bb41c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb4117f0 0x55c0bb41c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35782/0 (socket says 172.21.15.22:35782) 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 -- 172.21.15.22:0/375155841 learned_addr learned my addr 172.21.15.22:0/375155841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1901aff640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb41eab0 0x55c0bb41c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f19012fe640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb410ed0 0x55c0bb41bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 -- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb41eab0 msgr2=0x55c0bb41c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb41eab0 0x55c0bb41c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 -- 172.21.15.22:0/375155841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb410ed0 msgr2=0x55c0bb41bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb410ed0 0x55c0bb41bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f1900afd640 1 -- 172.21.15.22:0/375155841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0bb0893f0 con 0x55c0bb4117f0 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.280+0000 7f19012fe640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb410ed0 0x55c0bb41bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:05:18.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f1900afd640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb4117f0 0x55c0bb41c020 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0cc940 tx=0x7f18ec09b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ca3e886879be4a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f18e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f18e4ff9640 1 -- 172.21.15.22:0/375155841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec0b4060 con 0x55c0bb4117f0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f18e4ff9640 1 -- 172.21.15.22:0/375155841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f18ec0cf160 con 0x55c0bb4117f0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18f80041d0 con 0x55c0bb4117f0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f18e4ff9640 1 -- 172.21.15.22:0/375155841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18ec0cf360 con 0x55c0bb4117f0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 10 client.18951 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c0bb1537d0 con 0x55c0bb4117f0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f18e4ff9640 1 -- 172.21.15.22:0/375155841 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18f80041d0 con 0x55c0bb4117f0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f18e4ff9640 1 -- 172.21.15.22:0/375155841 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f18ec084ce0 con 0x55c0bb4117f0 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 10 client.18951 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 10 client.18951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 20 client.18951 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb431080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 4 client.18951 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.281+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0bb382400 con 0x55c0bb42ec90 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.282+0000 7f19012fe640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb431080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.282+0000 7f18e4ff9640 10 client.18951 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.282+0000 7f19012fe640 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb431080 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3d52a38f03bd312 server_cookie=2a9de04c1b6a68fb in_seq=0 out_seq=0 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f18e4ff9640 1 -- 172.21.15.22:0/375155841 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0bb382400 con 0x55c0bb42ec90 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f18e4ff9640 10 client.18951 handle_command_reply: tid=0 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 client.18951 shutdown 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb4117f0 msgr2=0x55c0bb41c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb4117f0 0x55c0bb41c020 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0cc940 tx=0x7f18ec09b3a0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 shutdown_connections 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb431080 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb41eab0 0x55c0bb41c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb4117f0 0x55c0bb41c020 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f18e4ff9640 0 client.18951 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 --2- 172.21.15.22:0/375155841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb410ed0 0x55c0bb41bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 >> 172.21.15.22:0/375155841 conn(0x55c0bb40b8d0 msgr2=0x55c0bb40d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 shutdown_connections 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 1 -- 172.21.15.22:0/375155841 wait complete. 2024-03-09T00:05:18.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.283+0000 7f190621d740 20 client.18951 trim_cache size 0 max 16384 2024-03-09T00:05:18.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 Processor -- start 2024-03-09T00:05:18.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 -- start start 2024-03-09T00:05:18.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb41e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb418d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb419280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1ae930 con 0x55c0bb41eab0 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1ac4d0 con 0x55c0bb4117f0 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f190621d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1aa590 con 0x55c0bb410ed0 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f1901aff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb419280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f1901aff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb419280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35792/0 (socket says 172.21.15.22:35792) 2024-03-09T00:05:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f1901aff640 1 -- 172.21.15.22:0/3293550889 learned_addr learned my addr 172.21.15.22:0/3293550889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f1900afd640 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb418d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f19012fe640 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb41e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f1900afd640 1 -- 172.21.15.22:0/3293550889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 msgr2=0x55c0bb41e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.296+0000 7f1900afd640 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb41e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f1900afd640 1 -- 172.21.15.22:0/3293550889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 msgr2=0x55c0bb419280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f1900afd640 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb419280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f1900afd640 1 -- 172.21.15.22:0/3293550889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0bb420ea0 con 0x55c0bb4117f0 2024-03-09T00:05:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f1901aff640 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb419280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f1900afd640 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb418d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18ec098170 tx=0x7f18ec09b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d135b532f563b3e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f18e6ffd640 1 -- 172.21.15.22:0/3293550889 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec09b570 con 0x55c0bb4117f0 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f18e6ffd640 1 -- 172.21.15.22:0/3293550889 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18ec09b7a0 con 0x55c0bb4117f0 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.297+0000 7f18e6ffd640 1 -- 172.21.15.22:0/3293550889 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec0a3d40 con 0x55c0bb4117f0 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- 172.21.15.22:0/3293550889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 msgr2=0x55c0bb418d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb418d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18ec098170 tx=0x7f18ec09b410 comp rx=0 tx=0).stop 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- 172.21.15.22:0/3293550889 shutdown_connections 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb419280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb418d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 --2- 172.21.15.22:0/3293550889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb41e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- 172.21.15.22:0/3293550889 >> 172.21.15.22:0/3293550889 conn(0x55c0bb40b8d0 msgr2=0x55c0bb416df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- 172.21.15.22:0/3293550889 shutdown_connections 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- 172.21.15.22:0/3293550889 wait complete. 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- start start 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 Processor -- start 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb424530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb424a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb424fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1aab90 con 0x55c0bb41eab0 2024-03-09T00:05:18.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1abe70 con 0x55c0bb4117f0 2024-03-09T00:05:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1ab1b0 con 0x55c0bb410ed0 2024-03-09T00:05:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.298+0000 7f190621d740 10 client.0 _ll_register_callbacks cb 0x7f18f800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f1901aff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb424fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb424530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f1901aff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb424fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35798/0 (socket says 172.21.15.22:35798) 2024-03-09T00:05:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb424530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51756/0 (socket says 172.21.15.22:51756) 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f1901aff640 1 -- 172.21.15.22:0/1344672296 learned_addr learned my addr 172.21.15.22:0/1344672296 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f1900afd640 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb424a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 -- 172.21.15.22:0/1344672296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 msgr2=0x55c0bb424a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb424a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 -- 172.21.15.22:0/1344672296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 msgr2=0x55c0bb424fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb424fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 -- 172.21.15.22:0/1344672296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0bb485180 con 0x55c0bb410ed0 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f1901aff640 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb424fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f19012fe640 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb424530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18f8010ca0 tx=0x7f18f8005930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19860936f7766c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f18e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.299+0000 7f18e4ff9640 1 -- 172.21.15.22:0/1344672296 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f802a020 con 0x55c0bb410ed0 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f18e4ff9640 1 -- 172.21.15.22:0/1344672296 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f18f80265a0 con 0x55c0bb410ed0 2024-03-09T00:05:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18ec09b7a0 con 0x55c0bb410ed0 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f18e4ff9640 1 -- 172.21.15.22:0/1344672296 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18f80267c0 con 0x55c0bb410ed0 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 10 client.28384 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c0bb486480 con 0x55c0bb410ed0 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f18e4ff9640 1 -- 172.21.15.22:0/1344672296 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18ec09b7a0 con 0x55c0bb410ed0 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f18e4ff9640 1 -- 172.21.15.22:0/1344672296 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f18f8003ab0 con 0x55c0bb410ed0 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 10 client.28384 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 10 client.28384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 20 client.28384 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb488880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 4 client.28384 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.300+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0bb37c750 con 0x55c0bb42ec90 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f1900afd640 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb488880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f1900afd640 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb488880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a909b520d4a3561 server_cookie=38f45eca8325c3c1 in_seq=0 out_seq=0 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f18e4ff9640 10 client.28384 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f18e4ff9640 1 -- 172.21.15.22:0/1344672296 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0bb37c750 con 0x55c0bb42ec90 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f18e4ff9640 10 client.28384 handle_command_reply: tid=0 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 client.28384 shutdown 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 msgr2=0x55c0bb424530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb424530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18f8010ca0 tx=0x7f18f8005930 comp rx=0 tx=0).stop 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 shutdown_connections 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55c0bb42ec90 0x55c0bb488880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0bb41eab0 0x55c0bb424fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0bb4117f0 0x55c0bb424a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 --2- 172.21.15.22:0/1344672296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0bb410ed0 0x55c0bb424530 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f18e4ff9640 0 client.28384 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 >> 172.21.15.22:0/1344672296 conn(0x55c0bb40b8d0 msgr2=0x55c0bb41baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 shutdown_connections 2024-03-09T00:05:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 1 -- 172.21.15.22:0/1344672296 wait complete. 2024-03-09T00:05:18.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.301+0000 7f190621d740 20 client.28384 trim_cache size 0 max 16384 2024-03-09T00:05:18.307 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18e80a2970 msgr2=0x7f18e80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18e80a2970 0x7f18e80a4e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18f4004af0 tx=0x7f18f4005c50 comp rx=0 tx=0).stop 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 msgr2=0x7f18fc198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc198e30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e9f0 tx=0x7f18ec00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 shutdown_connections 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18e80a2970 0x7f18e80a4e30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc1993c0 0x7f18fc1a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18fc1054b0 0x7f18fc198e30 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 --2- 172.21.15.22:0/2537543652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc1003e0 0x7f18fc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 >> 172.21.15.22:0/2537543652 conn(0x7f18fc0773a0 msgr2=0x7f18fc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 shutdown_connections 2024-03-09T00:05:18.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.302+0000 7f1903562640 1 -- 172.21.15.22:0/2537543652 wait complete. 2024-03-09T00:05:18.310 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:05:18.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:05:18.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 -- 172.21.15.22:0/3764591814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da4100d90 msgr2=0x7f6da4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 --2- 172.21.15.22:0/3764591814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da4100d90 0x7f6da4104dd0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009900 tx=0x7f6d9002bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:18.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 -- 172.21.15.22:0/3764591814 shutdown_connections 2024-03-09T00:05:18.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 --2- 172.21.15.22:0/3764591814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da41054b0 0x7f6da410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 --2- 172.21.15.22:0/3764591814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da4100d90 0x7f6da4104dd0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 --2- 172.21.15.22:0/3764591814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da41003e0 0x7f6da41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 -- 172.21.15.22:0/3764591814 >> 172.21.15.22:0/3764591814 conn(0x7f6da40773a0 msgr2=0x7f6da40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 -- 172.21.15.22:0/3764591814 shutdown_connections 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.456+0000 7f6daad7e640 1 -- 172.21.15.22:0/3764591814 wait complete. 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 Processor -- start 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 -- start start 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da41003e0 0x7f6da4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da41054b0 0x7f6da4198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da4199460 0x7f6da41a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4110540 con 0x7f6da41003e0 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41103c0 con 0x7f6da41054b0 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6daad7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41106c0 con 0x7f6da4199460 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da41003e0 0x7f6da4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da41003e0 0x7f6da4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35818/0 (socket says 172.21.15.22:35818) 2024-03-09T00:05:18.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 -- 172.21.15.22:0/2611715431 learned_addr learned my addr 172.21.15.22:0/2611715431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da3fff640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da41054b0 0x7f6da4198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da931b640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da4199460 0x7f6da41a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 -- 172.21.15.22:0/2611715431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da4199460 msgr2=0x7f6da41a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da4199460 0x7f6da41a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 -- 172.21.15.22:0/2611715431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da41054b0 msgr2=0x7f6da4198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da41054b0 0x7f6da4198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 -- 172.21.15.22:0/2611715431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da41a41e0 con 0x7f6da41003e0 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da41003e0 0x7f6da4198940 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800d950 tx=0x7f6d9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da1ffb640 1 -- 172.21.15.22:0/2611715431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98017030 con 0x7f6da41003e0 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.457+0000 7f6da1ffb640 1 -- 172.21.15.22:0/2611715431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d98004020 con 0x7f6da41003e0 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.458+0000 7f6da1ffb640 1 -- 172.21.15.22:0/2611715431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d980047a0 con 0x7f6da41003e0 2024-03-09T00:05:18.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.458+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da41a44d0 con 0x7f6da41003e0 2024-03-09T00:05:18.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.458+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da41a4a40 con 0x7f6da41003e0 2024-03-09T00:05:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da1ffb640 1 -- 172.21.15.22:0/2611715431 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6d980041d0 con 0x7f6da41003e0 2024-03-09T00:05:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 Processor -- start 2024-03-09T00:05:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 -- start start 2024-03-09T00:05:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da1ffb640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6d940a29f0 0x7f6d940a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9c28e360 con 0x55da9c2c7ac0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9c22aec0 con 0x55da9c2d06a0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9c1d3d30 con 0x55da9c2c71a0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da8b1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da3fff640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6d940a29f0 0x7f6d940a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da931b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da931b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51790/0 (socket says 172.21.15.22:51790) 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da931b640 1 -- 172.21.15.22:0/1385613507 learned_addr learned my addr 172.21.15.22:0/1385613507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.460+0000 7f6da1ffb640 1 -- 172.21.15.22:0/2611715431 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6d980ceb00 con 0x7f6da41003e0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da931b640 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da8b1a640 1 -- 172.21.15.22:0/1385613507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 msgr2=0x55da9c2c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da8b1a640 1 -- 172.21.15.22:0/1385613507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 msgr2=0x55da9c2d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da8b1a640 1 -- 172.21.15.22:0/1385613507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da9c1d2bc0 con 0x55da9c2c7ac0 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da931b640 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da3fff640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6d940a29f0 0x7f6d940a4eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009230 tx=0x7f6d90005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d0160 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6d980cbe50 tx=0x7f6d980970c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58a24c58ac36f2d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6d8effd640 1 -- 172.21.15.22:0/1385613507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98014070 con 0x55da9c2c7ac0 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6d8effd640 1 -- 172.21.15.22:0/1385613507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d9809b460 con 0x55da9c2c7ac0 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.461+0000 7f6d8effd640 1 -- 172.21.15.22:0/1385613507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98020160 con 0x55da9c2c7ac0 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 -- 172.21.15.22:0/1385613507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c7ac0 msgr2=0x55da9c2d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d0160 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6d980cbe50 tx=0x7f6d980970c0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 -- 172.21.15.22:0/1385613507 shutdown_connections 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d0160 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/1385613507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 -- 172.21.15.22:0/1385613507 >> 172.21.15.22:0/1385613507 conn(0x55da9c2c18d0 msgr2=0x55da9c2c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 -- 172.21.15.22:0/1385613507 shutdown_connections 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 -- 172.21.15.22:0/1385613507 wait complete. 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 -- start start 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.462+0000 7f6dad5fd740 1 Processor -- start 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c71a0 0x55da9c2cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c2ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9c1c94a0 con 0x55da9c2d06a0 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9c226f60 con 0x55da9c2c71a0 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9c2d3430 con 0x55da9c2c7ac0 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6dad5fd740 10 client.0 _ll_register_callbacks cb 0x55da9c224d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c2ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c2ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35844/0 (socket says 172.21.15.22:35844) 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 -- 172.21.15.22:0/2373587216 learned_addr learned my addr 172.21.15.22:0/2373587216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c71a0 0x55da9c2cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 -- 172.21.15.22:0/2373587216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 msgr2=0x55da9c2cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 -- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c71a0 msgr2=0x55da9c2cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c71a0 0x55da9c2cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 -- 172.21.15.22:0/2373587216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da9bf3f3f0 con 0x55da9c2d06a0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6da931b640 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c2ce220 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c010ba0 tx=0x7f6d9c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136eab7f3fc026f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.463+0000 7f6d83fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6d83fff640 1 -- 172.21.15.22:0/2373587216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c020060 con 0x55da9c2d06a0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d9809b460 con 0x55da9c2d06a0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6d83fff640 1 -- 172.21.15.22:0/2373587216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6d9c003970 con 0x55da9c2d06a0 2024-03-09T00:05:18.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6d83fff640 1 -- 172.21.15.22:0/2373587216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9c027050 con 0x55da9c2d06a0 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 10 client.18969 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55da9c0097d0 con 0x55da9c2d06a0 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6d83fff640 1 -- 172.21.15.22:0/2373587216 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d9809b460 con 0x55da9c2d06a0 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6d83fff640 1 -- 172.21.15.22:0/2373587216 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6d9c01a030 con 0x55da9c2d06a0 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 10 client.18969 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 10 client.18969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 20 client.18969 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c2e08f0 0x55da9c2e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 4 client.18969 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da9c238400 con 0x55da9c2e08f0 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.464+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c2e08f0 0x55da9c2e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.465+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c2e08f0 0x55da9c2e2ce0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51c630320df4deee server_cookie=2f17d6cecfd0b871 in_seq=0 out_seq=0 2024-03-09T00:05:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.465+0000 7f6d83fff640 10 client.18969 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.465+0000 7f6d83fff640 1 -- 172.21.15.22:0/2373587216 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da9c238400 con 0x55da9c2e08f0 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.465+0000 7f6d83fff640 10 client.18969 handle_command_reply: tid=0 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.465+0000 7f6dad5fd740 1 client.18969 shutdown 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 msgr2=0x55da9c2ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c2ce220 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c010ba0 tx=0x7f6d9c00f790 comp rx=0 tx=0).stop 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 shutdown_connections 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c2e08f0 0x55da9c2e2ce0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c2ce220 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/2373587216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c71a0 0x55da9c2cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 >> 172.21.15.22:0/2373587216 conn(0x55da9c2c18d0 msgr2=0x55da9c2c3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6d83fff640 0 client.18969 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 shutdown_connections 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 1 -- 172.21.15.22:0/2373587216 wait complete. 2024-03-09T00:05:18.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.466+0000 7f6dad5fd740 20 client.18969 trim_cache size 0 max 16384 2024-03-09T00:05:18.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.478+0000 7f6dad5fd740 1 Processor -- start 2024-03-09T00:05:18.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6dad5fd740 1 -- start start 2024-03-09T00:05:18.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c71a0 0x55da9c2cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41b0c60 con 0x55da9c2c71a0 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41ae800 con 0x55da9c2d06a0 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41ac8c0 con 0x55da9c2c7ac0 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c71a0 0x55da9c2cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da931b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c71a0 0x55da9c2cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35848/0 (socket says 172.21.15.22:35848) 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 -- 172.21.15.22:0/3275379467 learned_addr learned my addr 172.21.15.22:0/3275379467 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 -- 172.21.15.22:0/3275379467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 msgr2=0x55da9c2c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/3275379467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 -- 172.21.15.22:0/3275379467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 msgr2=0x55da9c2c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/3275379467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 -- 172.21.15.22:0/3275379467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da9c2d04d0 con 0x55da9c2c71a0 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/3275379467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c71a0 0x55da9c2cfed0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009fd0 tx=0x7f6d98020fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c3c0a2d62c31d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6d8e7fc640 1 -- 172.21.15.22:0/3275379467 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98014040 con 0x55da9c2c71a0 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6d8e7fc640 1 -- 172.21.15.22:0/3275379467 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d9809eeb0 con 0x55da9c2c71a0 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.479+0000 7f6d8e7fc640 1 -- 172.21.15.22:0/3275379467 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d980a32d0 con 0x55da9c2c71a0 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 -- 172.21.15.22:0/3275379467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c71a0 msgr2=0x55da9c2cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/3275379467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c71a0 0x55da9c2cfed0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009fd0 tx=0x7f6d98020fd0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 -- 172.21.15.22:0/3275379467 shutdown_connections 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/3275379467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2d06a0 0x55da9c2c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/3275379467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c7ac0 0x55da9c2c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/3275379467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2c71a0 0x55da9c2cfed0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 -- 172.21.15.22:0/3275379467 >> 172.21.15.22:0/3275379467 conn(0x55da9c2c18d0 msgr2=0x55da9c2e7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 -- 172.21.15.22:0/3275379467 shutdown_connections 2024-03-09T00:05:18.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.480+0000 7f6dad5fd740 1 -- 172.21.15.22:0/3275379467 wait complete. 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 -- start start 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 Processor -- start 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2d5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c335d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41acec0 con 0x55da9c2d06a0 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41ae1a0 con 0x55da9c2c7ac0 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41ad4e0 con 0x55da9c2c71a0 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6dad5fd740 10 client.0 _ll_register_callbacks cb 0x7f6d98004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c335d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c335d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35856/0 (socket says 172.21.15.22:35856) 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 -- 172.21.15.22:0/920601812 learned_addr learned my addr 172.21.15.22:0/920601812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2d5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da3fff640 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 -- 172.21.15.22:0/920601812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 msgr2=0x55da9c2d5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2d5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 -- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c7ac0 msgr2=0x55da9c2d5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.481+0000 7f6da931b640 1 -- 172.21.15.22:0/920601812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da9c3362c0 con 0x55da9c2d06a0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6da931b640 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c335d80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c010ba0 tx=0x7f6d9c00e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd9fd6701b3d40 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6d83fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6d83fff640 1 -- 172.21.15.22:0/920601812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c02f070 con 0x55da9c2d06a0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6d83fff640 1 -- 172.21.15.22:0/920601812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6d9c00e7e0 con 0x55da9c2d06a0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d9809eeb0 con 0x55da9c2d06a0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6d83fff640 1 -- 172.21.15.22:0/920601812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9c03b050 con 0x55da9c2d06a0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 10 client.18975 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55da9c3374c0 con 0x55da9c2d06a0 2024-03-09T00:05:18.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6d83fff640 1 -- 172.21.15.22:0/920601812 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d9809eeb0 con 0x55da9c2d06a0 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6d83fff640 1 -- 172.21.15.22:0/920601812 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6d9c01a040 con 0x55da9c2d06a0 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 10 client.18975 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 10 client.18975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 20 client.18975 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c337b00 0x55da9c339f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 4 client.18975 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55da9c232750 con 0x55da9c337b00 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.482+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c337b00 0x55da9c339f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6d83fff640 10 client.18975 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6da8b1a640 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c337b00 0x55da9c339f20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4378f2f065736a2 server_cookie=61ce4b16560fa2ed in_seq=0 out_seq=0 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6d83fff640 1 -- 172.21.15.22:0/920601812 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55da9c232750 con 0x55da9c337b00 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6d83fff640 10 client.18975 handle_command_reply: tid=0 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 client.18975 shutdown 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 msgr2=0x55da9c335d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c335d80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c010ba0 tx=0x7f6d9c00e610 comp rx=0 tx=0).stop 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 shutdown_connections 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55da9c337b00 0x55da9c339f20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55da9c2d06a0 0x55da9c335d80 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da9c2c7ac0 0x55da9c2d5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6d83fff640 0 client.18975 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 --2- 172.21.15.22:0/920601812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da9c2c71a0 0x55da9c2d5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 >> 172.21.15.22:0/920601812 conn(0x55da9c2c18d0 msgr2=0x55da9c2c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 shutdown_connections 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 1 -- 172.21.15.22:0/920601812 wait complete. 2024-03-09T00:05:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.483+0000 7f6dad5fd740 20 client.18975 trim_cache size 0 max 16384 2024-03-09T00:05:18.486 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6d940a29f0 msgr2=0x7f6d940a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6d940a29f0 0x7f6d940a4eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009230 tx=0x7f6d90005fd0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da41003e0 msgr2=0x7f6da4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da41003e0 0x7f6da4198940 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800d950 tx=0x7f6d9800de10 comp rx=0 tx=0).stop 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 shutdown_connections 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6d940a29f0 0x7f6d940a4eb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da4199460 0x7f6da41a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6da41054b0 0x7f6da4198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 --2- 172.21.15.22:0/2611715431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6da41003e0 0x7f6da4198940 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 >> 172.21.15.22:0/2611715431 conn(0x7f6da40773a0 msgr2=0x7f6da40fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 shutdown_connections 2024-03-09T00:05:18.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.484+0000 7f6daad7e640 1 -- 172.21.15.22:0/2611715431 wait complete. 2024-03-09T00:05:18.490 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:05:18.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:05:18.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 -- 172.21.15.22:0/2340301895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 msgr2=0x7f0254106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 --2- 172.21.15.22:0/2340301895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f0254106760 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f0250009930 tx=0x7f025002bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:18.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 -- 172.21.15.22:0/2340301895 shutdown_connections 2024-03-09T00:05:18.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 --2- 172.21.15.22:0/2340301895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0254100e00 0x7f025410cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 --2- 172.21.15.22:0/2340301895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0254100440 0x7f02541008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 --2- 172.21.15.22:0/2340301895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f0254106760 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 -- 172.21.15.22:0/2340301895 >> 172.21.15.22:0/2340301895 conn(0x7f0254076fb0 msgr2=0x7f02540773c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 -- 172.21.15.22:0/2340301895 shutdown_connections 2024-03-09T00:05:18.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.613+0000 7f025c7cd640 1 -- 172.21.15.22:0/2340301895 wait complete. 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 Processor -- start 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 -- start start 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0254100440 0x7f025410dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0254100e00 0x7f025410e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f025410e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254111740 con 0x7f0254106380 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541115c0 con 0x7f0254100e00 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025c7cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541118c0 con 0x7f0254100440 2024-03-09T00:05:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f025410e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f025410e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35884/0 (socket says 172.21.15.22:35884) 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 -- 172.21.15.22:0/248860216 learned_addr learned my addr 172.21.15.22:0/248860216 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f0259d68640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0254100e00 0x7f025410e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025a569640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0254100440 0x7f025410dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 -- 172.21.15.22:0/248860216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0254100440 msgr2=0x7f025410dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0254100440 0x7f025410dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 -- 172.21.15.22:0/248860216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0254100e00 msgr2=0x7f025410e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0254100e00 0x7f025410e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.614+0000 7f025ad6a640 1 -- 172.21.15.22:0/248860216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02541a6060 con 0x7f0254106380 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.615+0000 7f025ad6a640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f025410e550 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f024c00eb10 tx=0x7f024c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.615+0000 7f024b7fe640 1 -- 172.21.15.22:0/248860216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c004b20 con 0x7f0254106380 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.615+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02541a6350 con 0x7f0254106380 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.615+0000 7f024b7fe640 1 -- 172.21.15.22:0/248860216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f024c004cc0 con 0x7f0254106380 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.615+0000 7f024b7fe640 1 -- 172.21.15.22:0/248860216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c010760 con 0x7f0254106380 2024-03-09T00:05:18.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.615+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02541a68a0 con 0x7f0254106380 2024-03-09T00:05:18.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.617+0000 7f024b7fe640 1 -- 172.21.15.22:0/248860216 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f024c010900 con 0x7f0254106380 2024-03-09T00:05:18.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.617+0000 7f025f3ac740 1 Processor -- start 2024-03-09T00:05:18.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025f3ac740 1 -- start start 2024-03-09T00:05:18.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f23002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025f3ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642f22b3360 con 0x5642f22fdeb0 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025f3ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642f224fec0 con 0x5642f22f52d0 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025f3ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642f21f8d30 con 0x5642f22f49b0 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f0259d68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f23002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:45252/0 (socket says 172.21.15.22:45252) 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f0259d68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f23002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35898/0 (socket says 172.21.15.22:35898) 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 -- 172.21.15.22:0/1810545899 learned_addr learned my addr 172.21.15.22:0/1810545899 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f0259d68640 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f024b7fe640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f02400a2970 0x7f02400a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 -- 172.21.15.22:0/1810545899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 msgr2=0x5642f22f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 -- 172.21.15.22:0/1810545899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 msgr2=0x5642f23002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f23002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025a569640 1 -- 172.21.15.22:0/1810545899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642f21f7bc0 con 0x5642f22f52d0 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.618+0000 7f025ad6a640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f02400a2970 0x7f02400a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.619+0000 7f024b7fe640 1 -- 172.21.15.22:0/248860216 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f024c0cf750 con 0x7f0254106380 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.619+0000 7f025a569640 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22fd970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f025002cfd0 tx=0x7f02500095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2644fdaa5df4cf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.619+0000 7f025ad6a640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f02400a2970 0x7f02400a4e30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f024c09b6b0 tx=0x7f024c005b40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.619+0000 7f0248ff9640 1 -- 172.21.15.22:0/1810545899 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250042070 con 0x5642f22f52d0 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.619+0000 7f0248ff9640 1 -- 172.21.15.22:0/1810545899 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0250004030 con 0x5642f22f52d0 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.619+0000 7f0248ff9640 1 -- 172.21.15.22:0/1810545899 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250004c20 con 0x5642f22f52d0 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 -- 172.21.15.22:0/1810545899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 msgr2=0x5642f22fd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22fd970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f025002cfd0 tx=0x7f02500095f0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 -- 172.21.15.22:0/1810545899 shutdown_connections 2024-03-09T00:05:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f23002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22fd970 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1810545899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 -- 172.21.15.22:0/1810545899 >> 172.21.15.22:0/1810545899 conn(0x5642f22ef0e0 msgr2=0x5642f22ef4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 -- 172.21.15.22:0/1810545899 shutdown_connections 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 -- 172.21.15.22:0/1810545899 wait complete. 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 -- start start 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.620+0000 7f025f3ac740 1 Processor -- start 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f49b0 0x5642f22faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f52d0 0x5642f22fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f22fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642f21ee4a0 con 0x5642f22f49b0 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642f224bf60 con 0x5642f22fdeb0 2024-03-09T00:05:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642f2300c40 con 0x5642f22f52d0 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 10 client.0 _ll_register_callbacks cb 0x5642f2249d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f49b0 0x5642f22faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f49b0 0x5642f22faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35904/0 (socket says 172.21.15.22:35904) 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 -- 172.21.15.22:0/1828755082 learned_addr learned my addr 172.21.15.22:0/1828755082 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025a569640 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f52d0 0x5642f22fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f22fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 -- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f52d0 msgr2=0x5642f22fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f52d0 0x5642f22fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 -- 172.21.15.22:0/1828755082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 msgr2=0x5642f22fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f22fb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 -- 172.21.15.22:0/1828755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642f1f643f0 con 0x5642f22f49b0 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f0259d68640 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f49b0 0x5642f22faf40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0244003660 tx=0x7f0244009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8f3c8aa1650f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f023dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f023dffb640 1 -- 172.21.15.22:0/1828755082 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244012030 con 0x5642f22f49b0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0250004030 con 0x5642f22f49b0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.621+0000 7f023dffb640 1 -- 172.21.15.22:0/1828755082 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f02440121d0 con 0x5642f22f49b0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f023dffb640 1 -- 172.21.15.22:0/1828755082 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0244029050 con 0x5642f22f49b0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 10 client.18987 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5642f202e7d0 con 0x5642f22f49b0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f023dffb640 1 -- 172.21.15.22:0/1828755082 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0250004030 con 0x5642f22f49b0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f023dffb640 1 -- 172.21.15.22:0/1828755082 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f024401d070 con 0x5642f22f49b0 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 10 client.18987 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 10 client.18987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 20 client.18987 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f230e0b0 0x5642f23104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 4 client.18987 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642f225d400 con 0x5642f230e0b0 2024-03-09T00:05:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.622+0000 7f025a569640 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f230e0b0 0x5642f23104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.623+0000 7f023dffb640 10 client.18987 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.623+0000 7f025a569640 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f230e0b0 0x5642f23104a0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b0ef6057a5aa87 server_cookie=dbbedad9af2c42e9 in_seq=0 out_seq=0 2024-03-09T00:05:18.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f023dffb640 1 -- 172.21.15.22:0/1828755082 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642f225d400 con 0x5642f230e0b0 2024-03-09T00:05:18.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f023dffb640 10 client.18987 handle_command_reply: tid=0 2024-03-09T00:05:18.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 client.18987 shutdown 2024-03-09T00:05:18.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f49b0 msgr2=0x5642f22faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f49b0 0x5642f22faf40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0244003660 tx=0x7f0244009ac0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 shutdown_connections 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f230e0b0 0x5642f23104a0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f22fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f52d0 0x5642f22fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f023dffb640 0 client.18987 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 --2- 172.21.15.22:0/1828755082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f49b0 0x5642f22faf40 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.624+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 >> 172.21.15.22:0/1828755082 conn(0x5642f22ef0e0 msgr2=0x5642f22f0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.625+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 shutdown_connections 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.625+0000 7f025f3ac740 1 -- 172.21.15.22:0/1828755082 wait complete. 2024-03-09T00:05:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.625+0000 7f025f3ac740 20 client.18987 trim_cache size 0 max 16384 2024-03-09T00:05:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 Processor -- start 2024-03-09T00:05:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 -- start start 2024-03-09T00:05:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22fd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f22f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541ae920 con 0x5642f22fdeb0 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541ac4c0 con 0x5642f22f52d0 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f025f3ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541aa600 con 0x5642f22f49b0 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f0259d68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f22f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f0259d68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f22f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35912/0 (socket says 172.21.15.22:35912) 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f0259d68640 1 -- 172.21.15.22:0/895033558 learned_addr learned my addr 172.21.15.22:0/895033558 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.637+0000 7f0259d68640 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f025a569640 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f0259d68640 1 -- 172.21.15.22:0/895033558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 msgr2=0x5642f22fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f0259d68640 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22fd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f0259d68640 1 -- 172.21.15.22:0/895033558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 msgr2=0x5642f22f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f0259d68640 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22f9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f0259d68640 1 -- 172.21.15.22:0/895033558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642f2300610 con 0x5642f22fdeb0 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f0259d68640 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f22f81e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x5642f22db110 tx=0x7f0244005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5193fdf1cfbb983b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f023ffff640 1 -- 172.21.15.22:0/895033558 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244012b10 con 0x5642f22fdeb0 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f023ffff640 1 -- 172.21.15.22:0/895033558 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0244012470 con 0x5642f22fdeb0 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.638+0000 7f023ffff640 1 -- 172.21.15.22:0/895033558 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244011a30 con 0x5642f22fdeb0 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 -- 172.21.15.22:0/895033558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 msgr2=0x5642f22f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f22f81e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x5642f22db110 tx=0x7f0244005f90 comp rx=0 tx=0).stop 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 -- 172.21.15.22:0/895033558 shutdown_connections 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22fdeb0 0x5642f22f81e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22f52d0 0x5642f22f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 --2- 172.21.15.22:0/895033558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f22fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 -- 172.21.15.22:0/895033558 >> 172.21.15.22:0/895033558 conn(0x5642f22ef0e0 msgr2=0x5642f22f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 -- 172.21.15.22:0/895033558 shutdown_connections 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 -- 172.21.15.22:0/895033558 wait complete. 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 -- start start 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.639+0000 7f025f3ac740 1 Processor -- start 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f2305280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f52d0 0x5642f2303670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025f3ac740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f2303bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f52d0 0x5642f2303670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f52d0 0x5642f2303670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35924/0 (socket says 172.21.15.22:35924) 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 -- 172.21.15.22:0/932705897 learned_addr learned my addr 172.21.15.22:0/932705897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541aac00 con 0x5642f22f52d0 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541abe60 con 0x5642f22fdeb0 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f0259d68640 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f2305280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541ab200 con 0x5642f22f49b0 2024-03-09T00:05:18.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025f3ac740 10 client.0 _ll_register_callbacks cb 0x7f0244005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f0259d68640 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f2303bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 -- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 msgr2=0x5642f2305280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f2305280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 -- 172.21.15.22:0/932705897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 msgr2=0x5642f2303bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f2303bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 -- 172.21.15.22:0/932705897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642f2304400 con 0x5642f22f52d0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f025a569640 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f52d0 0x5642f2303670 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f025002f020 tx=0x7f0250004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da90d4b6caba9295 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f023dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.640+0000 7f023dffb640 1 -- 172.21.15.22:0/932705897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250042040 con 0x5642f22f52d0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f023dffb640 1 -- 172.21.15.22:0/932705897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f025002c880 con 0x5642f22f52d0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0244012470 con 0x5642f22f52d0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f023dffb640 1 -- 172.21.15.22:0/932705897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0250005d50 con 0x5642f22f52d0 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 10 client.18993 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:18.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5642f23655c0 con 0x5642f22f52d0 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f023dffb640 1 -- 172.21.15.22:0/932705897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0250004cb0 con 0x5642f22f52d0 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f023dffb640 1 -- 172.21.15.22:0/932705897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0250004f50 con 0x5642f22f52d0 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 10 client.18993 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 20 client.18993 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f2365950 0x5642f2367d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 4 client.18993 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0244012470 con 0x5642f2365950 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.641+0000 7f0259d68640 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f2365950 0x5642f2367d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f0259d68640 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f2365950 0x5642f2367d40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7774ec895cd87f06 server_cookie=bb48d2c3e6ea05f4 in_seq=0 out_seq=0 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f023dffb640 10 client.18993 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f023dffb640 1 -- 172.21.15.22:0/932705897 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0244012470 con 0x5642f2365950 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f023dffb640 10 client.18993 handle_command_reply: tid=0 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f025f3ac740 1 client.18993 shutdown 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f52d0 msgr2=0x5642f2303670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f025f3ac740 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f52d0 0x5642f2303670 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f025002f020 tx=0x7f0250004760 comp rx=0 tx=0).stop 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 shutdown_connections 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.642+0000 7f025f3ac740 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5642f2365950 0x5642f2367d40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.643+0000 7f025f3ac740 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642f22fdeb0 0x5642f2303bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.643+0000 7f023dffb640 0 client.18993 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.643+0000 7f025f3ac740 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5642f22f52d0 0x5642f2303670 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.643+0000 7f025f3ac740 1 --2- 172.21.15.22:0/932705897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642f22f49b0 0x5642f2305280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.643+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 >> 172.21.15.22:0/932705897 conn(0x5642f22ef0e0 msgr2=0x5642f22fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.643+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 shutdown_connections 2024-03-09T00:05:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.643+0000 7f025f3ac740 1 -- 172.21.15.22:0/932705897 wait complete. 2024-03-09T00:05:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.644+0000 7f025f3ac740 20 client.18993 trim_cache size 0 max 16384 2024-03-09T00:05:18.646 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.644+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f02400a2970 msgr2=0x7f02400a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.644+0000 7f025c7cd640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f02400a2970 0x7f02400a4e30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f024c09b6b0 tx=0x7f024c005b40 comp rx=0 tx=0).stop 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.644+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 msgr2=0x7f025410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.644+0000 7f025c7cd640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f025410e550 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f024c00eb10 tx=0x7f024c00c2d0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 shutdown_connections 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f02400a2970 0x7f02400a4e30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0254106380 0x7f025410e550 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0254100e00 0x7f025410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 --2- 172.21.15.22:0/248860216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0254100440 0x7f025410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 >> 172.21.15.22:0/248860216 conn(0x7f0254076fb0 msgr2=0x7f02540fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 shutdown_connections 2024-03-09T00:05:18.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:18.645+0000 7f025c7cd640 1 -- 172.21.15.22:0/248860216 wait complete. 2024-03-09T00:05:18.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1087884 9.70 MB/sec execute 446 sec latency 17.117 ms 2024-03-09T00:05:19.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:18 smithi062 ceph-mon[36129]: pgmap v339: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-03-09T00:05:19.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/319651105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:05:19.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:18 smithi192 ceph-mon[35995]: pgmap v339: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-03-09T00:05:19.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/319651105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:05:19.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:18 smithi022 ceph-mon[31894]: pgmap v339: 129 pgs: 129 active+clean; 93 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-03-09T00:05:19.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/319651105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:05:19.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:05:19.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:05:19.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:05:19.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1090472 9.72 MB/sec execute 447 sec latency 21.264 ms 2024-03-09T00:05:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3067736367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 msgr2=0x7fdb181007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3067736367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb181007b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009900 tx=0x7fdb0802bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3067736367 shutdown_connections 2024-03-09T00:05:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3067736367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb181054a0 0x7fdb1810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3067736367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18100d80 0x7fdb18104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3067736367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb181007b0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3067736367 >> 172.21.15.22:0/3067736367 conn(0x7fdb180772f0 msgr2=0x7fdb18077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3067736367 shutdown_connections 2024-03-09T00:05:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.774+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3067736367 wait complete. 2024-03-09T00:05:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 Processor -- start 2024-03-09T00:05:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 -- start start 2024-03-09T00:05:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb1819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb181054a0 0x7fdb1819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18197980 0x7fdb18197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18111660 con 0x7fdb181003d0 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181114e0 con 0x7fdb181054a0 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb1e69a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181117e0 con 0x7fdb18197980 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.775+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb1819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb1819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35936/0 (socket says 172.21.15.22:35936) 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 -- 172.21.15.22:0/3618074343 learned_addr learned my addr 172.21.15.22:0/3618074343 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb177fe640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb181054a0 0x7fdb1819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18197980 0x7fdb18197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 -- 172.21.15.22:0/3618074343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18197980 msgr2=0x7fdb18197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18197980 0x7fdb18197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 -- 172.21.15.22:0/3618074343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb181054a0 msgr2=0x7fdb1819dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb181054a0 0x7fdb1819dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 -- 172.21.15.22:0/3618074343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb181984a0 con 0x7fdb181003d0 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb17fff640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb1819d7d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006010 tx=0x7fdb0802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb157fa640 1 -- 172.21.15.22:0/3618074343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08041070 con 0x7fdb181003d0 2024-03-09T00:05:19.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb157fa640 1 -- 172.21.15.22:0/3618074343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb08004450 con 0x7fdb181003d0 2024-03-09T00:05:19.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb18198730 con 0x7fdb181003d0 2024-03-09T00:05:19.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb157fa640 1 -- 172.21.15.22:0/3618074343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08003e00 con 0x7fdb181003d0 2024-03-09T00:05:19.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.776+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb181a4980 con 0x7fdb181003d0 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb157fa640 1 -- 172.21.15.22:0/3618074343 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdb08044070 con 0x7fdb181003d0 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 Processor -- start 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 -- start start 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb157fa640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb000a29f0 0x7fdb000a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2d9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2e7690 0x55e71b2e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71b23db40 con 0x55e71b2da350 2024-03-09T00:05:19.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71b1e6d80 con 0x55e71b2d9a30 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb21282740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71b239be0 con 0x55e71b2e7690 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb000a29f0 0x7fdb000a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb157fa640 1 -- 172.21.15.22:0/3618074343 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdb080f0c20 con 0x7fdb181003d0 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.779+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2d9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35946/0 (socket says 172.21.15.22:35946) 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb177fe640 1 -- 172.21.15.22:0/3756875825 learned_addr learned my addr 172.21.15.22:0/3756875825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2e7690 0x55e71b2e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb17fff640 1 -- 172.21.15.22:0/3756875825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2e7690 msgr2=0x55e71b2e9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb17fff640 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2e7690 0x55e71b2e9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb17fff640 1 -- 172.21.15.22:0/3756875825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 msgr2=0x55e71b2e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb17fff640 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb17fff640 1 -- 172.21.15.22:0/3756875825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e71b01a7e0 con 0x55e71b2d9a30 2024-03-09T00:05:19.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2e7690 0x55e71b2e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb17fff640 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2d9e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb080ee080 tx=0x7fdb080bced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aaa0622b64c194f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb000a29f0 0x7fdb000a4eb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e71b2d9720 tx=0x7fdb10009a60 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdafe7fc640 1 -- 172.21.15.22:0/3756875825 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0804b040 con 0x55e71b2d9a30 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdafe7fc640 1 -- 172.21.15.22:0/3756875825 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb0802cd20 con 0x55e71b2d9a30 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.780+0000 7fdafe7fc640 1 -- 172.21.15.22:0/3756875825 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb080c10e0 con 0x55e71b2d9a30 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 -- 172.21.15.22:0/3756875825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 msgr2=0x55e71b2d9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2d9e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb080ee080 tx=0x7fdb080bced0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 -- 172.21.15.22:0/3756875825 shutdown_connections 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2e7690 0x55e71b2e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 --2- 172.21.15.22:0/3756875825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2d9e10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 -- 172.21.15.22:0/3756875825 >> 172.21.15.22:0/3756875825 conn(0x55e71b2d44b0 msgr2=0x55e71b2d4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 -- 172.21.15.22:0/3756875825 shutdown_connections 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 -- 172.21.15.22:0/3756875825 wait complete. 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 -- start start 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.781+0000 7fdb21282740 1 Processor -- start 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb21282740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2d9a30 0x55e71b2e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb21282740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb21282740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b2e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb21282740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71b1e6d80 con 0x55e71b2d9a30 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb21282740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71b1fa1c0 con 0x55e71b2e7690 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb21282740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e71b2ea290 con 0x55e71b2da350 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb21282740 10 client.0 _ll_register_callbacks cb 0x55e71b1e97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2d9a30 0x55e71b2e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2d9a30 0x55e71b2e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35956/0 (socket says 172.21.15.22:35956) 2024-03-09T00:05:19.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 -- 172.21.15.22:0/2375484382 learned_addr learned my addr 172.21.15.22:0/2375484382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb177fe640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b2e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 -- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 msgr2=0x55e71b2e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 -- 172.21.15.22:0/2375484382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 msgr2=0x55e71b2e4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b2e4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 -- 172.21.15.22:0/2375484382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e71b07af70 con 0x55e71b2d9a30 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb177fe640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdb17fff640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2d9a30 0x55e71b2e70b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdb0802cfd0 tx=0x7fdb08005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8758f1132314bede server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.782+0000 7fdaf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdaf77fe640 1 -- 172.21.15.22:0/2375484382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0804b070 con 0x55e71b2d9a30 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdaf77fe640 1 -- 172.21.15.22:0/2375484382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdb080b96b0 con 0x55e71b2d9a30 2024-03-09T00:05:19.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb0802cd20 con 0x55e71b2d9a30 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdaf77fe640 1 -- 172.21.15.22:0/2375484382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb08090050 con 0x55e71b2d9a30 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 10 client.19005 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e71af12170 con 0x55e71b2d9a30 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdaf77fe640 1 -- 172.21.15.22:0/2375484382 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb0802cd20 con 0x55e71b2d9a30 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdaf77fe640 1 -- 172.21.15.22:0/2375484382 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdb0808b030 con 0x55e71b2d9a30 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 10 client.19005 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 20 client.19005 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b2f7330 0x55e71b2f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 4 client.19005 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e71b213960 con 0x55e71b2f7330 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.783+0000 7fdb177fe640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b2f7330 0x55e71b2f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.784+0000 7fdaf77fe640 10 client.19005 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.784+0000 7fdb177fe640 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b2f7330 0x55e71b2f9720 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c35e776dd2cad2 server_cookie=a8d4689e7ef213af in_seq=0 out_seq=0 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdaf77fe640 1 -- 172.21.15.22:0/2375484382 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e71b213960 con 0x55e71b2f7330 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdaf77fe640 10 client.19005 handle_command_reply: tid=0 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 client.19005 shutdown 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2d9a30 msgr2=0x55e71b2e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2d9a30 0x55e71b2e70b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdb0802cfd0 tx=0x7fdb08005670 comp rx=0 tx=0).stop 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 shutdown_connections 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b2f7330 0x55e71b2f9720 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b2e4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 --2- 172.21.15.22:0/2375484382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2d9a30 0x55e71b2e70b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdaf77fe640 0 client.19005 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 >> 172.21.15.22:0/2375484382 conn(0x55e71b2d44b0 msgr2=0x55e71b2d6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 shutdown_connections 2024-03-09T00:05:19.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 1 -- 172.21.15.22:0/2375484382 wait complete. 2024-03-09T00:05:19.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.785+0000 7fdb21282740 20 client.19005 trim_cache size 0 max 16384 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 Processor -- start 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 -- start start 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2e7690 0x55e71b2ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181ae920 con 0x55e71b2e7690 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181ac4c0 con 0x55e71b2d9a30 2024-03-09T00:05:19.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb21282740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181aa580 con 0x55e71b2da350 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb1cc37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2e7690 0x55e71b2ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb1cc37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2e7690 0x55e71b2ee540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35970/0 (socket says 172.21.15.22:35970) 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb1cc37640 1 -- 172.21.15.22:0/589110964 learned_addr learned my addr 172.21.15.22:0/589110964 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.800+0000 7fdb177fe640 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdb17fff640 1 -- 172.21.15.22:0/589110964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 msgr2=0x55e71b2e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdb17fff640 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdb17fff640 1 -- 172.21.15.22:0/589110964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2e7690 msgr2=0x55e71b2ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdb17fff640 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2e7690 0x55e71b2ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdb17fff640 1 -- 172.21.15.22:0/589110964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e71b2da180 con 0x55e71b2d9a30 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2e7690 0x55e71b2ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdb17fff640 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2e6b70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb080bcd40 tx=0x7fdb08004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ebb707d0cbf806d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdafdffb640 1 -- 172.21.15.22:0/589110964 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0804b070 con 0x55e71b2d9a30 2024-03-09T00:05:19.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdafdffb640 1 -- 172.21.15.22:0/589110964 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb080c10e0 con 0x55e71b2d9a30 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.801+0000 7fdafdffb640 1 -- 172.21.15.22:0/589110964 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0809be60 con 0x55e71b2d9a30 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- 172.21.15.22:0/589110964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 msgr2=0x55e71b2e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2e6b70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb080bcd40 tx=0x7fdb08004dc0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- 172.21.15.22:0/589110964 shutdown_connections 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2e7690 0x55e71b2ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2da350 0x55e71b2e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 --2- 172.21.15.22:0/589110964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2d9a30 0x55e71b2e6b70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- 172.21.15.22:0/589110964 >> 172.21.15.22:0/589110964 conn(0x55e71b2d44b0 msgr2=0x55e71b2f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- 172.21.15.22:0/589110964 shutdown_connections 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- 172.21.15.22:0/589110964 wait complete. 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- start start 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 Processor -- start 2024-03-09T00:05:19.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2d9a30 0x55e71b2ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b34d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181aab80 con 0x55e71b2da350 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181abe60 con 0x55e71b2e7690 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.802+0000 7fdb21282740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181ab1a0 con 0x55e71b2d9a30 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb21282740 10 client.0 _ll_register_callbacks cb 0x7fdb08005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2ed7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35978/0 (socket says 172.21.15.22:35978) 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 -- 172.21.15.22:0/490087311 learned_addr learned my addr 172.21.15.22:0/490087311 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb17fff640 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2d9a30 0x55e71b2ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb1cc37640 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b34d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 -- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2d9a30 msgr2=0x55e71b2ed2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2d9a30 0x55e71b2ed2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 -- 172.21.15.22:0/490087311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 msgr2=0x55e71b34d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b34d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 -- 172.21.15.22:0/490087311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e71b2ee070 con 0x55e71b2da350 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb177fe640 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2ed7e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c004700 tx=0x7fdb0c012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc69b5210e4e211 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdaf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdaf77fe640 1 -- 172.21.15.22:0/490087311 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c01f810 con 0x55e71b2da350 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb080c10e0 con 0x55e71b2da350 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdaf77fe640 1 -- 172.21.15.22:0/490087311 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdb0c004030 con 0x55e71b2da350 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdaf77fe640 1 -- 172.21.15.22:0/490087311 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb0c004230 con 0x55e71b2da350 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb21282740 10 client.19011 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.803+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e71b34ece0 con 0x55e71b2da350 2024-03-09T00:05:19.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdaf77fe640 1 -- 172.21.15.22:0/490087311 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb080c10e0 con 0x55e71b2da350 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdaf77fe640 1 -- 172.21.15.22:0/490087311 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdb0c0104c0 con 0x55e71b2da350 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb21282740 10 client.19011 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb21282740 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb21282740 20 client.19011 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb21282740 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b34f440 0x55e71b351860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb21282740 4 client.19011 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e71b1ff080 con 0x55e71b34f440 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb17fff640 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b34f440 0x55e71b351860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdaf77fe640 10 client.19011 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.804+0000 7fdb17fff640 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b34f440 0x55e71b351860 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5facbfd20e199316 server_cookie=e14f1e1c9d4e16d in_seq=0 out_seq=0 2024-03-09T00:05:19.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdaf77fe640 1 -- 172.21.15.22:0/490087311 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e71b1ff080 con 0x55e71b34f440 2024-03-09T00:05:19.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdaf77fe640 10 client.19011 handle_command_reply: tid=0 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 client.19011 shutdown 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 msgr2=0x55e71b2ed7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2ed7e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c004700 tx=0x7fdb0c012ae0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 shutdown_connections 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e71b34f440 0x55e71b351860 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e71b2e7690 0x55e71b34d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e71b2da350 0x55e71b2ed7e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdaf77fe640 0 client.19011 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 --2- 172.21.15.22:0/490087311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e71b2d9a30 0x55e71b2ed2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 >> 172.21.15.22:0/490087311 conn(0x55e71b2d44b0 msgr2=0x55e71b2df790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 shutdown_connections 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 1 -- 172.21.15.22:0/490087311 wait complete. 2024-03-09T00:05:19.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.806+0000 7fdb21282740 20 client.19011 trim_cache size 0 max 16384 2024-03-09T00:05:19.808 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:05:19.808 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:05:19.808 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "d5489084-3694-400d-8a5b-d978b0f10c51", 2024-03-09T00:05:19.808 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:05:19.808 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb000a29f0 msgr2=0x7fdb000a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb000a29f0 0x7fdb000a4eb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e71b2d9720 tx=0x7fdb10009a60 comp rx=0 tx=0).stop 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 msgr2=0x7fdb1819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb1819d7d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006010 tx=0x7fdb0802cb30 comp rx=0 tx=0).stop 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 shutdown_connections 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdb000a29f0 0x7fdb000a4eb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18197980 0x7fdb18197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb181054a0 0x7fdb1819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 --2- 172.21.15.22:0/3618074343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb181003d0 0x7fdb1819d7d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 >> 172.21.15.22:0/3618074343 conn(0x7fdb180772f0 msgr2=0x7fdb180fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 shutdown_connections 2024-03-09T00:05:19.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.807+0000 7fdb1e69a640 1 -- 172.21.15.22:0/3618074343 wait complete. 2024-03-09T00:05:19.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:05:19.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:05:19.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 -- 172.21.15.22:0/994392826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac0ff900 msgr2=0x7f79ac0ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 --2- 172.21.15.22:0/994392826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac0ff900 0x7f79ac0ffd80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f79a000aa40 tx=0x7f79a002cf50 comp rx=0 tx=0).stop 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 -- 172.21.15.22:0/994392826 shutdown_connections 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 --2- 172.21.15.22:0/994392826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac1002c0 0x7f79ac10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 --2- 172.21.15.22:0/994392826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac0ff900 0x7f79ac0ffd80 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 --2- 172.21.15.22:0/994392826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ac105910 0x7f79ac105cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 -- 172.21.15.22:0/994392826 >> 172.21.15.22:0/994392826 conn(0x7f79ac076710 msgr2=0x7f79ac0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 -- 172.21.15.22:0/994392826 shutdown_connections 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.926+0000 7f79b299e640 1 -- 172.21.15.22:0/994392826 wait complete. 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 Processor -- start 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 -- start start 2024-03-09T00:05:19.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0ff900 0x7f79ac19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ac1002c0 0x7f79ac19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac105910 0x7f79ac197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac10f400 con 0x7f79ac105910 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac10f280 con 0x7f79ac1002c0 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b299e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac10f580 con 0x7f79ac0ff900 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b0f3b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac105910 0x7f79ac197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b0f3b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac105910 0x7f79ac197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35992/0 (socket says 172.21.15.22:35992) 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79b0f3b640 1 -- 172.21.15.22:0/1933020914 learned_addr learned my addr 172.21.15.22:0/1933020914 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ac1002c0 0x7f79ac19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.927+0000 7f79abfff640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0ff900 0x7f79ac19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b0f3b640 1 -- 172.21.15.22:0/1933020914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0ff900 msgr2=0x7f79ac19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b0f3b640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0ff900 0x7f79ac19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b0f3b640 1 -- 172.21.15.22:0/1933020914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ac1002c0 msgr2=0x7f79ac19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b0f3b640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ac1002c0 0x7f79ac19de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b0f3b640 1 -- 172.21.15.22:0/1933020914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac198260 con 0x7f79ac105910 2024-03-09T00:05:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b0f3b640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac105910 0x7f79ac197ae0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f79a400eb10 tx=0x7f79a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79a97fa640 1 -- 172.21.15.22:0/1933020914 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4004b20 con 0x7f79ac105910 2024-03-09T00:05:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79a97fa640 1 -- 172.21.15.22:0/1933020914 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79a4004cc0 con 0x7f79ac105910 2024-03-09T00:05:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac198550 con 0x7f79ac105910 2024-03-09T00:05:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79a97fa640 1 -- 172.21.15.22:0/1933020914 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4010760 con 0x7f79ac105910 2024-03-09T00:05:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.928+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac1a47f0 con 0x7f79ac105910 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.930+0000 7f79a97fa640 1 -- 172.21.15.22:0/1933020914 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f79a4005020 con 0x7f79ac105910 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.930+0000 7f79b557e740 1 Processor -- start 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b557e740 1 -- start start 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b557e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee5f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b557e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee6cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b557e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b557e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577edc2e00 con 0x55577ee5f490 2024-03-09T00:05:19.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b557e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577ed6bb10 con 0x55577ee6d070 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b557e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577edbeea0 con 0x55577ee5fdb0 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79a97fa640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79940a2970 0x7f79940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee5f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee5f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36004/0 (socket says 172.21.15.22:36004) 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79abfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee6cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79ab7fe640 1 -- 172.21.15.22:0/1687516792 learned_addr learned my addr 172.21.15.22:0/1687516792 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b0f3b640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79940a2970 0x7f79940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79a97fa640 1 -- 172.21.15.22:0/1933020914 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f79a40cf840 con 0x7f79ac105910 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79abfff640 1 -- 172.21.15.22:0/1687516792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 msgr2=0x55577ee6f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79abfff640 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79abfff640 1 -- 172.21.15.22:0/1687516792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 msgr2=0x55577ee5f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79abfff640 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee5f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79abfff640 1 -- 172.21.15.22:0/1687516792 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55577eba17d0 con 0x55577ee5fdb0 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79abfff640 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee6cb30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f799c00bb70 tx=0x7f799c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d11eff254f895856 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.931+0000 7f79b0f3b640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79940a2970 0x7f79940a4e30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79a409b9d0 tx=0x7f79a400efd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79927fc640 1 -- 172.21.15.22:0/1687516792 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c004030 con 0x55577ee5fdb0 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79927fc640 1 -- 172.21.15.22:0/1687516792 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799c0041d0 con 0x55577ee5fdb0 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79927fc640 1 -- 172.21.15.22:0/1687516792 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c0116a0 con 0x55577ee5fdb0 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 -- 172.21.15.22:0/1687516792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 msgr2=0x55577ee6cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee6cb30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f799c00bb70 tx=0x7f799c0107a0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 -- 172.21.15.22:0/1687516792 shutdown_connections 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee6cb30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 --2- 172.21.15.22:0/1687516792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee5f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 -- 172.21.15.22:0/1687516792 >> 172.21.15.22:0/1687516792 conn(0x55577ee59e90 msgr2=0x55577ee5a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 -- 172.21.15.22:0/1687516792 shutdown_connections 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.932+0000 7f79b557e740 1 -- 172.21.15.22:0/1687516792 wait complete. 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 -- start start 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 Processor -- start 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee5fdb0 0x55577ee69af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee6d070 0x55577ee6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577ee6fe30 con 0x55577ee6d070 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577ed7f0a0 con 0x55577ee5fdb0 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577ee6ffb0 con 0x55577ee5f490 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79b557e740 10 client.0 _ll_register_callbacks cb 0x55577ed7ea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee6d070 0x55577ee6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee6d070 0x55577ee6a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36008/0 (socket says 172.21.15.22:36008) 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79ab7fe640 1 -- 172.21.15.22:0/2232205359 learned_addr learned my addr 172.21.15.22:0/2232205359 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.933+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79abfff640 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee5fdb0 0x55577ee69af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79ab7fe640 1 -- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 msgr2=0x55577ee6c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee6c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79ab7fe640 1 -- 172.21.15.22:0/2232205359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee5fdb0 msgr2=0x55577ee69af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee5fdb0 0x55577ee69af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79ab7fe640 1 -- 172.21.15.22:0/2232205359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55577ead73f0 con 0x55577ee6d070 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee6d070 0x55577ee6a030 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f79a000a340 tx=0x7f79a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8f81f2dafe4c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f798b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f798b7fe640 1 -- 172.21.15.22:0/2232205359 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0004460 con 0x55577ee6d070 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f798b7fe640 1 -- 172.21.15.22:0/2232205359 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f79a002db00 con 0x55577ee6d070 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f799c0041d0 con 0x55577ee6d070 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f798b7fe640 1 -- 172.21.15.22:0/2232205359 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a0056050 con 0x55577ee6d070 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79b557e740 10 client.19023 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55577ea970f0 con 0x55577ee6d070 2024-03-09T00:05:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f798b7fe640 1 -- 172.21.15.22:0/2232205359 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799c0041d0 con 0x55577ee6d070 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.934+0000 7f798b7fe640 1 -- 172.21.15.22:0/2232205359 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f79a000b040 con 0x55577ee6d070 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79b557e740 10 client.19023 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79b557e740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79b557e740 20 client.19023 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79b557e740 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577ee7eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79b557e740 4 client.19023 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55577edd0340 con 0x55577ee7cad0 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79abfff640 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577ee7eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f79abfff640 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577ee7eec0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab126497dd0709e server_cookie=a5d2a1593775e602 in_seq=0 out_seq=0 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.935+0000 7f798b7fe640 10 client.19023 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f798b7fe640 1 -- 172.21.15.22:0/2232205359 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55577edd0340 con 0x55577ee7cad0 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f798b7fe640 10 client.19023 handle_command_reply: tid=0 2024-03-09T00:05:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 client.19023 shutdown 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee6d070 msgr2=0x55577ee6a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee6d070 0x55577ee6a030 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f79a000a340 tx=0x7f79a0005fd0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 shutdown_connections 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577ee7eec0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee6d070 0x55577ee6a030 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee5fdb0 0x55577ee69af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f798b7fe640 0 client.19023 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.936+0000 7f79b557e740 1 --2- 172.21.15.22:0/2232205359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee6c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.937+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 >> 172.21.15.22:0/2232205359 conn(0x55577ee59e90 msgr2=0x55577ee5bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.937+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 shutdown_connections 2024-03-09T00:05:19.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.937+0000 7f79b557e740 1 -- 172.21.15.22:0/2232205359 wait complete. 2024-03-09T00:05:19.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.937+0000 7f79b557e740 20 client.19023 trim_cache size 0 max 16384 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 Processor -- start 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 -- start start 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee65360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee68ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1ae970 con 0x55577ee5f490 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1ac510 con 0x55577ee6d070 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79b557e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1aa5d0 con 0x55577ee5fdb0 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee65360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee65360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36012/0 (socket says 172.21.15.22:36012) 2024-03-09T00:05:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 -- 172.21.15.22:0/1547057921 learned_addr learned my addr 172.21.15.22:0/1547057921 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79abfff640 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee68ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 -- 172.21.15.22:0/1547057921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 msgr2=0x55577ee68ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee68ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 -- 172.21.15.22:0/1547057921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 msgr2=0x55577ee6c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.953+0000 7f79ab7fe640 1 -- 172.21.15.22:0/1547057921 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55577ee5f020 con 0x55577ee5f490 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.954+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee65360 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f79a0000c00 tx=0x7f79a000a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59d203b46ed3fb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.954+0000 7f7991ffb640 1 -- 172.21.15.22:0/1547057921 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0004e50 con 0x55577ee5f490 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.954+0000 7f7991ffb640 1 -- 172.21.15.22:0/1547057921 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79a0005540 con 0x55577ee5f490 2024-03-09T00:05:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.954+0000 7f7991ffb640 1 -- 172.21.15.22:0/1547057921 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00046c0 con 0x55577ee5f490 2024-03-09T00:05:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- 172.21.15.22:0/1547057921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 msgr2=0x55577ee65360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee65360 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f79a0000c00 tx=0x7f79a000a370 comp rx=0 tx=0).stop 2024-03-09T00:05:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- 172.21.15.22:0/1547057921 shutdown_connections 2024-03-09T00:05:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee6c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5fdb0 0x55577ee68ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 --2- 172.21.15.22:0/1547057921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5f490 0x55577ee65360 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- 172.21.15.22:0/1547057921 >> 172.21.15.22:0/1547057921 conn(0x55577ee59e90 msgr2=0x55577ee683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- 172.21.15.22:0/1547057921 shutdown_connections 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- 172.21.15.22:0/1547057921 wait complete. 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- start start 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 Processor -- start 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee70f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5fdb0 0x55577ee714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee71a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1aabd0 con 0x55577ee5fdb0 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1abeb0 con 0x55577ee6d070 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1ab1f0 con 0x55577ee5f490 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.955+0000 7f79b557e740 10 client.0 _ll_register_callbacks cb 0x7f79a0035540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5fdb0 0x55577ee714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79ab7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee70f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5fdb0 0x55577ee714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36022/0 (socket says 172.21.15.22:36022) 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 -- 172.21.15.22:0/3202285767 learned_addr learned my addr 172.21.15.22:0/3202285767 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee71a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 -- 172.21.15.22:0/3202285767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 msgr2=0x55577ee70f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee70f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 -- 172.21.15.22:0/3202285767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 msgr2=0x55577ee71a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee71a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 -- 172.21.15.22:0/3202285767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55577eed3250 con 0x55577ee5fdb0 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee71a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79abfff640 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5fdb0 0x55577ee714a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f799c003fd0 tx=0x7f799c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cf706092789af7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f798b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f798b7fe640 1 -- 172.21.15.22:0/3202285767 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c0037f0 con 0x55577ee5fdb0 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f798b7fe640 1 -- 172.21.15.22:0/3202285767 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f799c003990 con 0x55577ee5fdb0 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79a0005540 con 0x55577ee5fdb0 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f798b7fe640 1 -- 172.21.15.22:0/3202285767 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f799c035050 con 0x55577ee5fdb0 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79b557e740 10 client.19029 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55577eed4560 con 0x55577ee5fdb0 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.956+0000 7f798b7fe640 1 -- 172.21.15.22:0/3202285767 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79a0005540 con 0x55577ee5fdb0 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f798b7fe640 1 -- 172.21.15.22:0/3202285767 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f799c011440 con 0x55577ee5fdb0 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79b557e740 10 client.19029 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79b557e740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79b557e740 20 client.19029 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79b557e740 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577eed6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79b557e740 4 client.19029 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55577edca690 con 0x55577ee7cad0 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577eed6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f798b7fe640 10 client.19029 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.957+0000 7f79ab7fe640 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577eed6990 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=789b33304810617a server_cookie=8abc2fe5e6eae456 in_seq=0 out_seq=0 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f798b7fe640 1 -- 172.21.15.22:0/3202285767 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55577edca690 con 0x55577ee7cad0 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f798b7fe640 10 client.19029 handle_command_reply: tid=0 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 client.19029 shutdown 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5fdb0 msgr2=0x55577ee714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5fdb0 0x55577ee714a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f799c003fd0 tx=0x7f799c003c10 comp rx=0 tx=0).stop 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 shutdown_connections 2024-03-09T00:05:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55577ee7cad0 0x55577eed6990 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55577ee6d070 0x55577ee71a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55577ee5fdb0 0x55577ee714a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f798b7fe640 0 client.19029 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 --2- 172.21.15.22:0/3202285767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55577ee5f490 0x55577ee70f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 >> 172.21.15.22:0/3202285767 conn(0x55577ee59e90 msgr2=0x55577ee68c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 shutdown_connections 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 1 -- 172.21.15.22:0/3202285767 wait complete. 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.958+0000 7f79b557e740 20 client.19029 trim_cache size 0 max 16384 2024-03-09T00:05:19.960 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: "d5489084-3694-400d-8a5b-d978b0f10c51": { 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "d5489084-3694-400d-8a5b-d978b0f10c51", 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:05:19.961 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:05:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79940a2970 msgr2=0x7f79940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79940a2970 0x7f79940a4e30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79a409b9d0 tx=0x7f79a400efd0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac105910 msgr2=0x7f79ac197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac105910 0x7f79ac197ae0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f79a400eb10 tx=0x7f79a400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 shutdown_connections 2024-03-09T00:05:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f79940a2970 0x7f79940a4e30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f79ac105910 0x7f79ac197ae0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ac1002c0 0x7f79ac19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 --2- 172.21.15.22:0/1933020914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0ff900 0x7f79ac19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 >> 172.21.15.22:0/1933020914 conn(0x7f79ac076710 msgr2=0x7f79ac0fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 shutdown_connections 2024-03-09T00:05:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:19.959+0000 7f79b299e640 1 -- 172.21.15.22:0/1933020914 wait complete. 2024-03-09T00:05:19.964 INFO:tasks.cephfs.filesystem:scrub status for tag:d5489084-3694-400d-8a5b-d978b0f10c51 - {'path': '/', 'tag': 'd5489084-3694-400d-8a5b-d978b0f10c51', 'options': 'recursive,force'} 2024-03-09T00:05:20.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1091301 9.70 MB/sec execute 448 sec latency 38.300 ms 2024-03-09T00:05:21.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:20 smithi062 ceph-mon[36129]: pgmap v340: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-03-09T00:05:21.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:20 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:05:21.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:20 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:21.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:20 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:05:21.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:20 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:21.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:20 smithi192 ceph-mon[35995]: pgmap v340: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-03-09T00:05:21.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:20 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:05:21.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:20 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:21.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:20 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:05:21.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:20 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:21.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:20 smithi022 ceph-mon[31894]: pgmap v340: 129 pgs: 129 active+clean; 96 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-03-09T00:05:21.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:20 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:05:21.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:20 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:21.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:20 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:05:21.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:20 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 -- 172.21.15.22:0/994828065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 msgr2=0x7f32c8105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 --2- 172.21.15.22:0/994828065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 0x7f32c8105570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009890 tx=0x7f32c002bb70 comp rx=0 tx=0).stop 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 -- 172.21.15.22:0/994828065 shutdown_connections 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 --2- 172.21.15.22:0/994828065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c8100e00 0x7f32c810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 --2- 172.21.15.22:0/994828065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 0x7f32c81008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 --2- 172.21.15.22:0/994828065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 0x7f32c8105570 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 -- 172.21.15.22:0/994828065 >> 172.21.15.22:0/994828065 conn(0x7f32c8077040 msgr2=0x7f32c8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 -- 172.21.15.22:0/994828065 shutdown_connections 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.283+0000 7f32cda4b640 1 -- 172.21.15.22:0/994828065 wait complete. 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 Processor -- start 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 -- start start 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 0x7f32c8101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c8100e00 0x7f32c81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 0x7f32c81028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8110650 con 0x7f32c8100440 2024-03-09T00:05:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81104d0 con 0x7f32c8100e00 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32cda4b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81107d0 con 0x7f32c8105190 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 0x7f32c8101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 0x7f32c8101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32832/0 (socket says 172.21.15.22:32832) 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 -- 172.21.15.22:0/2301621039 learned_addr learned my addr 172.21.15.22:0/2301621039 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c77fe640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 0x7f32c81028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c67fc640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c8100e00 0x7f32c81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 -- 172.21.15.22:0/2301621039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 msgr2=0x7f32c81028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 0x7f32c81028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 -- 172.21.15.22:0/2301621039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c8100e00 msgr2=0x7f32c81023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c8100e00 0x7f32c81023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c6ffd640 1 -- 172.21.15.22:0/2301621039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c81a6100 con 0x7f32c8100440 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.284+0000 7f32c77fe640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 0x7f32c81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.285+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 0x7f32c8101e60 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f32c00042f0 tx=0x7f32c0004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.285+0000 7f32cca49640 1 -- 172.21.15.22:0/2301621039 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0041020 con 0x7f32c8100440 2024-03-09T00:05:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.285+0000 7f32cca49640 1 -- 172.21.15.22:0/2301621039 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32c002c9f0 con 0x7f32c8100440 2024-03-09T00:05:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.285+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c81a6390 con 0x7f32c8100440 2024-03-09T00:05:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.285+0000 7f32cca49640 1 -- 172.21.15.22:0/2301621039 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0039040 con 0x7f32c8100440 2024-03-09T00:05:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.285+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c81a68d0 con 0x7f32c8100440 2024-03-09T00:05:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.287+0000 7f32cca49640 1 -- 172.21.15.22:0/2301621039 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f32c0044020 con 0x7f32c8100440 2024-03-09T00:05:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32cca49640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32b00a2a90 0x7f32b00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 Processor -- start 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 -- start start 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32c67fc640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32b00a2a90 0x7f32b00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32cca49640 1 -- 172.21.15.22:0/2301621039 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f32c00f0ac0 con 0x7f32c8100440 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddeffd0 0x55f14ddf0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf9550 0x55f14ddfb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14dd545f0 con 0x55f14ddeffd0 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14dcfd930 con 0x55f14ddf9550 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32d062f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14dd50690 con 0x55f14ddf0d40 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32c6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32c6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddefa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:44618/0 (socket says 172.21.15.22:44618) 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32c6ffd640 1 -- 172.21.15.22:0/304240662 learned_addr learned my addr 172.21.15.22:0/304240662 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.288+0000 7f32c67fc640 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddeffd0 0x55f14ddf0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c77fe640 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf9550 0x55f14ddfb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c67fc640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32b00a2a90 0x7f32b00a4f50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f32c81038f0 tx=0x7f32bc0095c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c6ffd640 1 -- 172.21.15.22:0/304240662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf9550 msgr2=0x55f14ddfb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf9550 0x55f14ddfb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c6ffd640 1 -- 172.21.15.22:0/304240662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddeffd0 msgr2=0x55f14ddf0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddeffd0 0x55f14ddf0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c6ffd640 1 -- 172.21.15.22:0/304240662 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f14da69210 con 0x55f14ddf0d40 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c77fe640 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf9550 0x55f14ddfb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c67fc640 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddeffd0 0x55f14ddf0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddefa90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32c00c5040 tx=0x7f32c00b91a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da77107693dad655 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32adffb640 1 -- 172.21.15.22:0/304240662 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00bcdd0 con 0x55f14ddf0d40 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32adffb640 1 -- 172.21.15.22:0/304240662 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32c00c0f20 con 0x55f14ddf0d40 2024-03-09T00:05:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.289+0000 7f32adffb640 1 -- 172.21.15.22:0/304240662 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b95b0 con 0x55f14ddf0d40 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 -- 172.21.15.22:0/304240662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 msgr2=0x55f14ddefa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddefa90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32c00c5040 tx=0x7f32c00b91a0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 -- 172.21.15.22:0/304240662 shutdown_connections 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf9550 0x55f14ddfb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddeffd0 0x55f14ddf0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 --2- 172.21.15.22:0/304240662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddefa90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 -- 172.21.15.22:0/304240662 >> 172.21.15.22:0/304240662 conn(0x55f14ddb75b0 msgr2=0x55f14ddeacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 -- 172.21.15.22:0/304240662 shutdown_connections 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 -- 172.21.15.22:0/304240662 wait complete. 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 -- start start 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.290+0000 7f32d062f740 1 Processor -- start 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddf6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14ddfc040 con 0x55f14ddf9550 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14dd10d90 con 0x55f14ddeffd0 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14ddfc1c0 con 0x55f14ddf0d40 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 10 client.0 _ll_register_callbacks cb 0x55f14da68a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32846/0 (socket says 172.21.15.22:32846) 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 -- 172.21.15.22:0/4035738065 learned_addr learned my addr 172.21.15.22:0/4035738065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c67fc640 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddf6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 -- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 msgr2=0x55f14ddf6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddf6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 -- 172.21.15.22:0/4035738065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 msgr2=0x55f14ddf6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 -- 172.21.15.22:0/4035738065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f14db35620 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32c77fe640 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf70c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f32b8013a20 tx=0x7f32b8013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56970e25d3e8e867 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32a6ffd640 1 -- 172.21.15.22:0/4035738065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8004050 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32a6ffd640 1 -- 172.21.15.22:0/4035738065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f32b80041f0 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c00c0f20 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32a6ffd640 1 -- 172.21.15.22:0/4035738065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b8025050 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 10 client.19041 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.291+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f14de082f0 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32a6ffd640 1 -- 172.21.15.22:0/4035738065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32c00c0f20 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32a6ffd640 1 -- 172.21.15.22:0/4035738065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32b801b4e0 con 0x55f14ddf9550 2024-03-09T00:05:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32d062f740 10 client.19041 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32d062f740 10 client.19041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32d062f740 20 client.19041 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32d062f740 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de0bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32d062f740 4 client.19041 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f14dd61b30 con 0x55f14de097b0 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de0bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de0bba0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74642ba6e788930 server_cookie=b56b9b7281a94de1 in_seq=0 out_seq=0 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.292+0000 7f32a6ffd640 10 client.19041 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.293+0000 7f32a6ffd640 1 -- 172.21.15.22:0/4035738065 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f14dd61b30 con 0x55f14de097b0 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.293+0000 7f32a6ffd640 10 client.19041 handle_command_reply: tid=0 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.293+0000 7f32d062f740 1 client.19041 shutdown 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 msgr2=0x55f14ddf70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf70c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f32b8013a20 tx=0x7f32b8013ee0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 shutdown_connections 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de0bba0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32a6ffd640 0 client.19041 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf70c0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf0d40 0x55f14ddf6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 --2- 172.21.15.22:0/4035738065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 >> 172.21.15.22:0/4035738065 conn(0x55f14ddb75b0 msgr2=0x55f14ddeca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 shutdown_connections 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 1 -- 172.21.15.22:0/4035738065 wait complete. 2024-03-09T00:05:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.294+0000 7f32d062f740 20 client.19041 trim_cache size 0 max 16384 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.307+0000 7f32d062f740 1 Processor -- start 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.307+0000 7f32d062f740 1 -- start start 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32d062f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32d062f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf0d40 0x55f14de003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32d062f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf9550 0x55f14de00930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32d062f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81b2d90 con 0x55f14ddf0d40 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32d062f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81b0930 con 0x55f14ddeffd0 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32d062f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81ae9f0 con 0x55f14ddf9550 2024-03-09T00:05:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf0d40 0x55f14de003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf0d40 0x55f14de003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32856/0 (socket says 172.21.15.22:32856) 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 -- 172.21.15.22:0/3457121608 learned_addr learned my addr 172.21.15.22:0/3457121608 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c77fe640 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf9550 0x55f14de00930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 -- 172.21.15.22:0/3457121608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf9550 msgr2=0x55f14de00930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf9550 0x55f14de00930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 -- 172.21.15.22:0/3457121608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 msgr2=0x55f14ddf8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 -- 172.21.15.22:0/3457121608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f14ddef2a0 con 0x55f14ddf0d40 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c77fe640 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf9550 0x55f14de00930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.308+0000 7f32c67fc640 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf0d40 0x55f14de003f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f32bc003d80 tx=0x7f32bc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3a5f28a593479a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32ad7fa640 1 -- 172.21.15.22:0/3457121608 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc004b20 con 0x55f14ddf0d40 2024-03-09T00:05:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32ad7fa640 1 -- 172.21.15.22:0/3457121608 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32bc004cc0 con 0x55f14ddf0d40 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32ad7fa640 1 -- 172.21.15.22:0/3457121608 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00d600 con 0x55f14ddf0d40 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 -- 172.21.15.22:0/3457121608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf0d40 msgr2=0x55f14de003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf0d40 0x55f14de003f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f32bc003d80 tx=0x7f32bc01c000 comp rx=0 tx=0).stop 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 -- 172.21.15.22:0/3457121608 shutdown_connections 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddf9550 0x55f14de00930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf0d40 0x55f14de003f0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 --2- 172.21.15.22:0/3457121608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddeffd0 0x55f14ddf8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 -- 172.21.15.22:0/3457121608 >> 172.21.15.22:0/3457121608 conn(0x55f14ddb75b0 msgr2=0x55f14ddf4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 -- 172.21.15.22:0/3457121608 shutdown_connections 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.309+0000 7f32d062f740 1 -- 172.21.15.22:0/3457121608 wait complete. 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 -- start start 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 Processor -- start 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddeffd0 0x55f14ddf3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf0d40 0x55f14ddf3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81aeff0 con 0x55f14ddf9550 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81b02d0 con 0x55f14ddf0d40 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81af610 con 0x55f14ddeffd0 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32d062f740 10 client.0 _ll_register_callbacks cb 0x7f32c0032640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32868/0 (socket says 172.21.15.22:32868) 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32c77fe640 1 -- 172.21.15.22:0/3034367544 learned_addr learned my addr 172.21.15.22:0/3034367544 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddeffd0 0x55f14ddf3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32c67fc640 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf0d40 0x55f14ddf3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32c77fe640 1 -- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddeffd0 msgr2=0x55f14ddf3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.310+0000 7f32c77fe640 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddeffd0 0x55f14ddf3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32c77fe640 1 -- 172.21.15.22:0/3034367544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf0d40 msgr2=0x55f14ddf3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32c77fe640 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf0d40 0x55f14ddf3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32c77fe640 1 -- 172.21.15.22:0/3034367544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f14de5fe30 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32c77fe640 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf3ea0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f32b8013a20 tx=0x7f32b8013610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4defe232c81890a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32a6ffd640 1 -- 172.21.15.22:0/3034367544 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b801bca0 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32a6ffd640 1 -- 172.21.15.22:0/3034367544 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f32b8005070 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32bc004cc0 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32a6ffd640 1 -- 172.21.15.22:0/3034367544 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b8037050 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 10 client.19047 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f14de61100 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32a6ffd640 1 -- 172.21.15.22:0/3034367544 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32bc004cc0 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32a6ffd640 1 -- 172.21.15.22:0/3034367544 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32b8003c20 con 0x55f14ddf9550 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 10 client.19047 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 10 client.19047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 20 client.19047 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 4 client.19047 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:05:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.311+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f14dd5be80 con 0x55f14de097b0 2024-03-09T00:05:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.312+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.312+0000 7f32a6ffd640 10 client.19047 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.312+0000 7f32c6ffd640 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de637d0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ddb7c33f23a12f server_cookie=af74eedb8e0a1f1d in_seq=0 out_seq=0 2024-03-09T00:05:21.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1092318 9.69 MB/sec execute 449 sec latency 255.348 ms 2024-03-09T00:05:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32a6ffd640 1 -- 172.21.15.22:0/3034367544 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f14dd5be80 con 0x55f14de097b0 2024-03-09T00:05:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32a6ffd640 10 client.19047 handle_command_reply: tid=0 2024-03-09T00:05:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 client.19047 shutdown 2024-03-09T00:05:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 msgr2=0x55f14ddf3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf3ea0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f32b8013a20 tx=0x7f32b8013610 comp rx=0 tx=0).stop 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 shutdown_connections 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f14de097b0 0x55f14de637d0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f14ddf9550 0x55f14ddf3ea0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32a6ffd640 0 client.19047 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14ddf0d40 0x55f14ddf3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 --2- 172.21.15.22:0/3034367544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f14ddeffd0 0x55f14ddf3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 >> 172.21.15.22:0/3034367544 conn(0x55f14ddb75b0 msgr2=0x55f14dde95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.736+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 shutdown_connections 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.737+0000 7f32d062f740 1 -- 172.21.15.22:0/3034367544 wait complete. 2024-03-09T00:05:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.737+0000 7f32d062f740 20 client.19047 trim_cache size 0 max 16384 2024-03-09T00:05:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:05:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:05:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:05:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:05:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.737+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32b00a2a90 msgr2=0x7f32b00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32b00a2a90 0x7f32b00a4f50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f32c81038f0 tx=0x7f32bc0095c0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 msgr2=0x7f32c8101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 0x7f32c8101e60 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f32c00042f0 tx=0x7f32c0004320 comp rx=0 tx=0).stop 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 shutdown_connections 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f32b00a2a90 0x7f32b00a4f50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32c8105190 0x7f32c81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c8100e00 0x7f32c81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 --2- 172.21.15.22:0/2301621039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32c8100440 0x7f32c8101e60 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 >> 172.21.15.22:0/2301621039 conn(0x7f32c8077040 msgr2=0x7f32c8068c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.738+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 shutdown_connections 2024-03-09T00:05:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:21.739+0000 7f32cda4b640 1 -- 172.21.15.22:0/2301621039 wait complete. 2024-03-09T00:05:22.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1094212 9.70 MB/sec execute 450 sec latency 39.470 ms 2024-03-09T00:05:23.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:22 smithi192 ceph-mon[35995]: pgmap v341: 129 pgs: 129 active+clean; 100 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-03-09T00:05:23.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:22 smithi022 ceph-mon[31894]: pgmap v341: 129 pgs: 129 active+clean; 100 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-03-09T00:05:23.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:22 smithi062 ceph-mon[36129]: pgmap v341: 129 pgs: 129 active+clean; 100 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-03-09T00:05:23.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1095663 9.69 MB/sec execute 451 sec latency 37.505 ms 2024-03-09T00:05:24.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1097802 9.70 MB/sec execute 452 sec latency 22.140 ms 2024-03-09T00:05:25.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:24 smithi192 ceph-mon[35995]: pgmap v342: 129 pgs: 129 active+clean; 100 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-03-09T00:05:25.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:24 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:24 smithi022 ceph-mon[31894]: pgmap v342: 129 pgs: 129 active+clean; 100 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-03-09T00:05:25.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:24 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:05:25.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:24 smithi062 ceph-mon[36129]: pgmap v342: 129 pgs: 129 active+clean; 100 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-03-09T00:05:25.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:24 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:05:25.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1099376 9.69 MB/sec execute 453 sec latency 16.845 ms 2024-03-09T00:05:26.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1101223 9.70 MB/sec execute 454 sec latency 59.744 ms 2024-03-09T00:05:27.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:26 smithi192 ceph-mon[35995]: pgmap v343: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 387 op/s 2024-03-09T00:05:27.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:26 smithi022 ceph-mon[31894]: pgmap v343: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 387 op/s 2024-03-09T00:05:27.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:26 smithi062 ceph-mon[36129]: pgmap v343: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 387 op/s 2024-03-09T00:05:27.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1102517 9.69 MB/sec execute 455 sec latency 34.848 ms 2024-03-09T00:05:28.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1104773 9.70 MB/sec execute 456 sec latency 15.833 ms 2024-03-09T00:05:29.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:28 smithi192 ceph-mon[35995]: pgmap v344: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-03-09T00:05:29.161 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:28 smithi062 ceph-mon[36129]: pgmap v344: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-03-09T00:05:29.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:28 smithi022 ceph-mon[31894]: pgmap v344: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-03-09T00:05:29.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1106118 9.69 MB/sec execute 457 sec latency 20.061 ms 2024-03-09T00:05:30.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1108344 9.70 MB/sec execute 458 sec latency 16.546 ms 2024-03-09T00:05:31.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:30 smithi192 ceph-mon[35995]: pgmap v345: 129 pgs: 129 active+clean; 92 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-03-09T00:05:31.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:30 smithi022 ceph-mon[31894]: pgmap v345: 129 pgs: 129 active+clean; 92 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-03-09T00:05:31.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:30 smithi062 ceph-mon[36129]: pgmap v345: 129 pgs: 129 active+clean; 92 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-03-09T00:05:31.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1109338 9.68 MB/sec execute 459 sec latency 44.775 ms 2024-03-09T00:05:32.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1111764 9.69 MB/sec execute 460 sec latency 20.302 ms 2024-03-09T00:05:33.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:32 smithi192 ceph-mon[35995]: pgmap v346: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 353 op/s 2024-03-09T00:05:33.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:32 smithi022 ceph-mon[31894]: pgmap v346: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 353 op/s 2024-03-09T00:05:33.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:32 smithi062 ceph-mon[36129]: pgmap v346: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 353 op/s 2024-03-09T00:05:33.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1112753 9.68 MB/sec execute 461 sec latency 18.727 ms 2024-03-09T00:05:34.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1115319 9.69 MB/sec execute 462 sec latency 17.431 ms 2024-03-09T00:05:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 -- 172.21.15.22:0/1157232235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d701012c0 msgr2=0x7f8d7010d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 --2- 172.21.15.22:0/1157232235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d701012c0 0x7f8d7010d2b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009cd0 tx=0x7f8d6802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 -- 172.21.15.22:0/1157232235 shutdown_connections 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 --2- 172.21.15.22:0/1157232235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d701012c0 0x7f8d7010d2b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 --2- 172.21.15.22:0/1157232235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70100900 0x7f8d70100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 --2- 172.21.15.22:0/1157232235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d70106480 0x7f8d70106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 -- 172.21.15.22:0/1157232235 >> 172.21.15.22:0/1157232235 conn(0x7f8d700767b0 msgr2=0x7f8d700fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 -- 172.21.15.22:0/1157232235 shutdown_connections 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.835+0000 7f8d78129640 1 -- 172.21.15.22:0/1157232235 wait complete. 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 Processor -- start 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 -- start start 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d70100900 0x7f8d700fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d701012c0 0x7f8d700fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70106480 0x7f8d700fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701107b0 con 0x7f8d70100900 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70110630 con 0x7f8d701012c0 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d78129640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70110930 con 0x7f8d70106480 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d70100900 0x7f8d700fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d70100900 0x7f8d700fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41530/0 (socket says 172.21.15.22:41530) 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2107122785 learned_addr learned my addr 172.21.15.22:0/2107122785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d756c4640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d701012c0 0x7f8d700fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d766c6640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70106480 0x7f8d700fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2107122785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70106480 msgr2=0x7f8d700fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70106480 0x7f8d700fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2107122785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d701012c0 msgr2=0x7f8d700fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d701012c0 0x7f8d700fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2107122785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d701a61a0 con 0x7f8d70100900 2024-03-09T00:05:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.836+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d70100900 0x7f8d700fbbc0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00ea60 tx=0x7f8d6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.837+0000 7f8d66ffd640 1 -- 172.21.15.22:0/2107122785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00cde0 con 0x7f8d70100900 2024-03-09T00:05:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.837+0000 7f8d66ffd640 1 -- 172.21.15.22:0/2107122785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c0035f0 con 0x7f8d70100900 2024-03-09T00:05:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.837+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d701a6490 con 0x7f8d70100900 2024-03-09T00:05:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.837+0000 7f8d66ffd640 1 -- 172.21.15.22:0/2107122785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c010670 con 0x7f8d70100900 2024-03-09T00:05:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.837+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d701a69e0 con 0x7f8d70100900 2024-03-09T00:05:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.839+0000 7f8d66ffd640 1 -- 172.21.15.22:0/2107122785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8d6c0037a0 con 0x7f8d70100900 2024-03-09T00:05:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.839+0000 7f8d7ade9740 1 Processor -- start 2024-03-09T00:05:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.839+0000 7f8d66ffd640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8d5c0a2a10 0x7f8d5c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.839+0000 7f8d766c6640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8d5c0a2a10 0x7f8d5c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.839+0000 7f8d66ffd640 1 -- 172.21.15.22:0/2107122785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8d6c0cf500 con 0x7f8d70100900 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.839+0000 7f8d7ade9740 1 -- start start 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893679a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89367a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893683550 0x55e893685940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8935de5f0 con 0x55e89367ad40 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e893587930 con 0x55e893679fd0 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8935da690 con 0x55e893683550 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d766c6640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8d5c0a2a10 0x7f8d5c0a4ed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d6802c630 tx=0x7f8d68005d80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893679a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893679a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41542/0 (socket says 172.21.15.22:41542) 2024-03-09T00:05:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 -- 172.21.15.22:0/1511251017 learned_addr learned my addr 172.21.15.22:0/1511251017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d756c4640 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89367a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d766c6640 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893683550 0x55e893685940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 -- 172.21.15.22:0/1511251017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893683550 msgr2=0x55e893685940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893683550 0x55e893685940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 -- 172.21.15.22:0/1511251017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 msgr2=0x55e89367a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89367a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 -- 172.21.15.22:0/1511251017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8932f3210 con 0x55e89367ad40 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893679a90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0a39d0 tx=0x7f8d6c09b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0bd35b3295ba0e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d5bfff640 1 -- 172.21.15.22:0/1511251017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0b4070 con 0x55e89367ad40 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.840+0000 7f8d5bfff640 1 -- 172.21.15.22:0/1511251017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c09f8b0 con 0x55e89367ad40 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d5bfff640 1 -- 172.21.15.22:0/1511251017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c097dd0 con 0x55e89367ad40 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1511251017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 msgr2=0x55e893679a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893679a90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0a39d0 tx=0x7f8d6c09b0c0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1511251017 shutdown_connections 2024-03-09T00:05:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893683550 0x55e893685940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89367a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1511251017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893679a90 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.841+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1511251017 >> 172.21.15.22:0/1511251017 conn(0x55e8936415b0 msgr2=0x55e893674cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1511251017 shutdown_connections 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1511251017 wait complete. 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 -- start start 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 Processor -- start 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893680610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893680b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e893681090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e893686280 con 0x55e89367ad40 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89359ad90 con 0x55e893683550 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e893686400 con 0x55e893679fd0 2024-03-09T00:05:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d7ade9740 10 client.0 _ll_register_callbacks cb 0x55e8932f2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d756c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893680b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.842+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893680610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893680610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:43586/0 (socket says 172.21.15.22:43586) 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2376504179 learned_addr learned my addr 172.21.15.22:0/2376504179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d766c6640 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e893681090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2376504179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 msgr2=0x55e893681090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e893681090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2376504179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 msgr2=0x55e893680b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893680b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 -- 172.21.15.22:0/2376504179 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8933bf620 con 0x55e893679fd0 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893680610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0cf820 tx=0x7f8d6c0ccad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2de2b80247e7bc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d597fa640 1 -- 172.21.15.22:0/2376504179 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0b8070 con 0x55e893679fd0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d6c09f8b0 con 0x55e893679fd0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d597fa640 1 -- 172.21.15.22:0/2376504179 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8d6c086050 con 0x55e893679fd0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d597fa640 1 -- 172.21.15.22:0/2376504179 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d6c0b4050 con 0x55e893679fd0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d7ade9740 10 client.28486 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.843+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e893686280 con 0x55e893679fd0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d597fa640 1 -- 172.21.15.22:0/2376504179 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c09f8b0 con 0x55e893679fd0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d597fa640 1 -- 172.21.15.22:0/2376504179 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d6c0afb40 con 0x55e893679fd0 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d7ade9740 10 client.28486 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d7ade9740 10 client.28486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d7ade9740 20 client.28486 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e893693770 0x55e893695b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d7ade9740 4 client.28486 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8935ebb30 con 0x55e893693770 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d756c4640 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e893693770 0x55e893695b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d597fa640 10 client.28486 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.844+0000 7f8d756c4640 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e893693770 0x55e893695b60 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5379f9ad8c44179 server_cookie=e0f49eb133ee2b0c in_seq=0 out_seq=0 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d597fa640 1 -- 172.21.15.22:0/2376504179 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8935ebb30 con 0x55e893693770 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d597fa640 10 client.28486 handle_command_reply: tid=0 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 client.28486 shutdown 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 msgr2=0x55e893680610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893680610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0cf820 tx=0x7f8d6c0ccad0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 shutdown_connections 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e893693770 0x55e893695b60 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e893681090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e893680b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d597fa640 0 client.28486 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/2376504179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893680610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 >> 172.21.15.22:0/2376504179 conn(0x55e8936415b0 msgr2=0x55e893676a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 shutdown_connections 2024-03-09T00:05:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.845+0000 7f8d7ade9740 1 -- 172.21.15.22:0/2376504179 wait complete. 2024-03-09T00:05:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.846+0000 7f8d7ade9740 20 client.28486 trim_cache size 0 max 16384 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 Processor -- start 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 -- start start 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893682d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e89367eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e89367f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701b2dc0 con 0x55e89367ad40 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701b0960 con 0x55e893683550 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.859+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701aea20 con 0x55e893679fd0 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e89367eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e89367eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41568/0 (socket says 172.21.15.22:41568) 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893682d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 -- 172.21.15.22:0/3397105818 learned_addr learned my addr 172.21.15.22:0/3397105818 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d766c6640 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e89367f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 -- 172.21.15.22:0/3397105818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 msgr2=0x55e893682d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893682d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 -- 172.21.15.22:0/3397105818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 msgr2=0x55e89367f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e89367f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.860+0000 7f8d756c4640 1 -- 172.21.15.22:0/3397105818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e89367eb60 con 0x55e89367ad40 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.861+0000 7f8d756c4640 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e89367eee0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x55e89367fe70 tx=0x7f8d60005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd7ada17e35493a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.861+0000 7f8d5b7fe640 1 -- 172.21.15.22:0/3397105818 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60022020 con 0x55e89367ad40 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.861+0000 7f8d5b7fe640 1 -- 172.21.15.22:0/3397105818 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d60010430 con 0x55e89367ad40 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.861+0000 7f8d5b7fe640 1 -- 172.21.15.22:0/3397105818 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6001d5c0 con 0x55e89367ad40 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- 172.21.15.22:0/3397105818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 msgr2=0x55e89367eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e89367eee0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x55e89367fe70 tx=0x7f8d60005f90 comp rx=0 tx=0).stop 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- 172.21.15.22:0/3397105818 shutdown_connections 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893683550 0x55e89367f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e89367ad40 0x55e89367eee0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/3397105818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e893679fd0 0x55e893682d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- 172.21.15.22:0/3397105818 >> 172.21.15.22:0/3397105818 conn(0x55e8936415b0 msgr2=0x55e8936747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- 172.21.15.22:0/3397105818 shutdown_connections 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- 172.21.15.22:0/3397105818 wait complete. 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- start start 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 Processor -- start 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89368a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e89367ad40 0x55e893688d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e893683550 0x55e893689240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701af020 con 0x55e893683550 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701b0300 con 0x55e893679fd0 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.862+0000 7f8d7ade9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701af640 con 0x55e89367ad40 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d7ade9740 10 client.0 _ll_register_callbacks cb 0x7f8d6c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d766c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e893683550 0x55e893689240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d75ec5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89368a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d756c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e89367ad40 0x55e893688d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d756c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e89367ad40 0x55e893688d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:43602/0 (socket says 172.21.15.22:43602) 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d766c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e893683550 0x55e893689240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41570/0 (socket says 172.21.15.22:41570) 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d756c4640 1 -- 172.21.15.22:0/1731799706 learned_addr learned my addr 172.21.15.22:0/1731799706 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d75ec5640 1 -- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e89367ad40 msgr2=0x55e893688d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e89367ad40 0x55e893688d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d75ec5640 1 -- 172.21.15.22:0/1731799706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e893683550 msgr2=0x55e893689240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e893683550 0x55e893689240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d75ec5640 1 -- 172.21.15.22:0/1731799706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e893689a40 con 0x55e893679fd0 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d766c6640 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e893683550 0x55e893689240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d756c4640 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e89367ad40 0x55e893688d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d75ec5640 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89368a960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0cce50 tx=0x7f8d6c09b1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10d152e3fa15baf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d597fa640 1 -- 172.21.15.22:0/1731799706 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0b8070 con 0x55e893679fd0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d597fa640 1 -- 172.21.15.22:0/1731799706 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8d6c097dd0 con 0x55e893679fd0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.863+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d60010430 con 0x55e893679fd0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d597fa640 1 -- 172.21.15.22:0/1731799706 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d6c09ac50 con 0x55e893679fd0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 10 client.28391 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e8936ea9e0 con 0x55e893679fd0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d597fa640 1 -- 172.21.15.22:0/1731799706 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d60010430 con 0x55e893679fd0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d597fa640 1 -- 172.21.15.22:0/1731799706 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8d6c0cf170 con 0x55e893679fd0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 10 client.28391 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 10 client.28391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 20 client.28391 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8936ead30 0x55e8936ed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 4 client.28391 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8935e5e80 con 0x55e8936ead30 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.864+0000 7f8d756c4640 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8936ead30 0x55e8936ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.865+0000 7f8d597fa640 10 client.28391 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:34.865+0000 7f8d756c4640 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8936ead30 0x55e8936ed150 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d98259809431d4 server_cookie=986ceb38cbdc915f in_seq=0 out_seq=0 2024-03-09T00:05:35.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:34 smithi192 ceph-mon[35995]: pgmap v347: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-03-09T00:05:35.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:34 smithi022 ceph-mon[31894]: pgmap v347: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-03-09T00:05:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d597fa640 1 -- 172.21.15.22:0/1731799706 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8935e5e80 con 0x55e8936ead30 2024-03-09T00:05:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d597fa640 10 client.28391 handle_command_reply: tid=0 2024-03-09T00:05:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d7ade9740 1 client.28391 shutdown 2024-03-09T00:05:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 msgr2=0x55e89368a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89368a960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0cce50 tx=0x7f8d6c09b1f0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 shutdown_connections 2024-03-09T00:05:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e8936ead30 0x55e8936ed150 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d597fa640 0 client.28391 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e893683550 0x55e893689240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.244+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e89367ad40 0x55e893688d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.245+0000 7f8d7ade9740 1 --2- 172.21.15.22:0/1731799706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e893679fd0 0x55e89368a960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.245+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 >> 172.21.15.22:0/1731799706 conn(0x55e8936415b0 msgr2=0x55e89367eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.245+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 shutdown_connections 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.245+0000 7f8d7ade9740 1 -- 172.21.15.22:0/1731799706 wait complete. 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.245+0000 7f8d7ade9740 20 client.28391 trim_cache size 0 max 16384 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:05:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8d5c0a2a10 msgr2=0x7f8d5c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8d5c0a2a10 0x7f8d5c0a4ed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d6802c630 tx=0x7f8d68005d80 comp rx=0 tx=0).stop 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d70100900 msgr2=0x7f8d700fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d70100900 0x7f8d700fbbc0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00ea60 tx=0x7f8d6c00ef20 comp rx=0 tx=0).stop 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 shutdown_connections 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8d5c0a2a10 0x7f8d5c0a4ed0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70106480 0x7f8d700fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d701012c0 0x7f8d700fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 --2- 172.21.15.22:0/2107122785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d70100900 0x7f8d700fbbc0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 >> 172.21.15.22:0/2107122785 conn(0x7f8d700767b0 msgr2=0x7f8d700fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 shutdown_connections 2024-03-09T00:05:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:35.246+0000 7f8d78129640 1 -- 172.21.15.22:0/2107122785 wait complete. 2024-03-09T00:05:35.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:34 smithi062 ceph-mon[36129]: pgmap v347: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-03-09T00:05:35.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1115797 9.68 MB/sec execute 463 sec latency 118.044 ms 2024-03-09T00:05:36.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1117153 9.67 MB/sec execute 464 sec latency 111.902 ms 2024-03-09T00:05:37.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:36 smithi192 ceph-mon[35995]: pgmap v348: 129 pgs: 129 active+clean; 101 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-03-09T00:05:37.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:36 smithi022 ceph-mon[31894]: pgmap v348: 129 pgs: 129 active+clean; 101 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-03-09T00:05:37.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:36 smithi062 ceph-mon[36129]: pgmap v348: 129 pgs: 129 active+clean; 101 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-03-09T00:05:37.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1119135 9.68 MB/sec execute 465 sec latency 33.589 ms 2024-03-09T00:05:38.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1120523 9.67 MB/sec execute 466 sec latency 52.045 ms 2024-03-09T00:05:39.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:38 smithi192 ceph-mon[35995]: pgmap v349: 129 pgs: 129 active+clean; 101 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-03-09T00:05:39.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:38 smithi022 ceph-mon[31894]: pgmap v349: 129 pgs: 129 active+clean; 101 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-03-09T00:05:39.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:38 smithi062 ceph-mon[36129]: pgmap v349: 129 pgs: 129 active+clean; 101 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-03-09T00:05:39.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1122551 9.67 MB/sec execute 467 sec latency 13.832 ms 2024-03-09T00:05:40.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1123923 9.67 MB/sec execute 468 sec latency 26.288 ms 2024-03-09T00:05:41.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:40 smithi022 ceph-mon[31894]: pgmap v350: 129 pgs: 129 active+clean; 91 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-03-09T00:05:41.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:40 smithi062 ceph-mon[36129]: pgmap v350: 129 pgs: 129 active+clean; 91 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-03-09T00:05:41.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:40 smithi192 ceph-mon[35995]: pgmap v350: 129 pgs: 129 active+clean; 91 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-03-09T00:05:41.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1125772 9.67 MB/sec execute 469 sec latency 107.566 ms 2024-03-09T00:05:42.730 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1126688 9.65 MB/sec execute 470 sec latency 35.637 ms 2024-03-09T00:05:43.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:42 smithi022 ceph-mon[31894]: pgmap v351: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 318 op/s 2024-03-09T00:05:43.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:42 smithi062 ceph-mon[36129]: pgmap v351: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 318 op/s 2024-03-09T00:05:43.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:42 smithi192 ceph-mon[35995]: pgmap v351: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 318 op/s 2024-03-09T00:05:43.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1129139 9.66 MB/sec execute 471 sec latency 14.047 ms 2024-03-09T00:05:44.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1130188 9.65 MB/sec execute 472 sec latency 24.759 ms 2024-03-09T00:05:45.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:44 smithi022 ceph-mon[31894]: pgmap v352: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-03-09T00:05:45.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:44 smithi062 ceph-mon[36129]: pgmap v352: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-03-09T00:05:45.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:44 smithi192 ceph-mon[35995]: pgmap v352: 129 pgs: 129 active+clean; 88 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-03-09T00:05:45.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1132735 9.66 MB/sec execute 473 sec latency 13.113 ms 2024-03-09T00:05:46.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1133778 9.65 MB/sec execute 474 sec latency 49.765 ms 2024-03-09T00:05:47.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:46 smithi022 ceph-mon[31894]: pgmap v353: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-03-09T00:05:47.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:46 smithi062 ceph-mon[36129]: pgmap v353: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-03-09T00:05:47.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:46 smithi192 ceph-mon[35995]: pgmap v353: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-03-09T00:05:47.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1135559 9.65 MB/sec execute 475 sec latency 54.861 ms 2024-03-09T00:05:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 -- 172.21.15.22:0/1014942055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be81003c0 msgr2=0x7f9be810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 --2- 172.21.15.22:0/1014942055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be81003c0 0x7f9be810dfc0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009cd0 tx=0x7f9be002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 -- 172.21.15.22:0/1014942055 shutdown_connections 2024-03-09T00:05:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 --2- 172.21.15.22:0/1014942055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be81003c0 0x7f9be810dfc0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 --2- 172.21.15.22:0/1014942055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9be80ffa00 0x7f9be80ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 --2- 172.21.15.22:0/1014942055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be8107340 0x7f9be8107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 -- 172.21.15.22:0/1014942055 >> 172.21.15.22:0/1014942055 conn(0x7f9be8075320 msgr2=0x7f9be8075730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 -- 172.21.15.22:0/1014942055 shutdown_connections 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.338+0000 7f9bed001640 1 -- 172.21.15.22:0/1014942055 wait complete. 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 Processor -- start 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 -- start start 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be80ffa00 0x7f9be8100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9be81003c0 0x7f9be8101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be8107340 0x7f9be8101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8111540 con 0x7f9be80ffa00 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81113c0 con 0x7f9be8107340 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bed001640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81116c0 con 0x7f9be81003c0 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be80ffa00 0x7f9be8100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be80ffa00 0x7f9be8100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42386/0 (socket says 172.21.15.22:42386) 2024-03-09T00:05:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 -- 172.21.15.22:0/2669349396 learned_addr learned my addr 172.21.15.22:0/2669349396 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9bdffff640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9be81003c0 0x7f9be8101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be759e640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be8107340 0x7f9be8101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 -- 172.21.15.22:0/2669349396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9be81003c0 msgr2=0x7f9be8101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9be81003c0 0x7f9be8101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 -- 172.21.15.22:0/2669349396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be8107340 msgr2=0x7f9be8101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be8107340 0x7f9be8101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.339+0000 7f9be6d9d640 1 -- 172.21.15.22:0/2669349396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be81a5ec0 con 0x7f9be80ffa00 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.340+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be80ffa00 0x7f9be8100be0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9f0 tx=0x7f9bd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.340+0000 7f9be4d99640 1 -- 172.21.15.22:0/2669349396 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800cd80 con 0x7f9be80ffa00 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.340+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be81a61b0 con 0x7f9be80ffa00 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.340+0000 7f9be4d99640 1 -- 172.21.15.22:0/2669349396 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd8010910 con 0x7f9be80ffa00 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.340+0000 7f9be4d99640 1 -- 172.21.15.22:0/2669349396 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80188f0 con 0x7f9be80ffa00 2024-03-09T00:05:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.340+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be81a6750 con 0x7f9be80ffa00 2024-03-09T00:05:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.342+0000 7f9be4d99640 1 -- 172.21.15.22:0/2669349396 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9bd8010ac0 con 0x7f9be80ffa00 2024-03-09T00:05:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.342+0000 7f9befbd9740 1 Processor -- start 2024-03-09T00:05:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.342+0000 7f9befbd9740 1 -- start start 2024-03-09T00:05:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be4d99640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd40a2960 0x7f9bd40a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be759e640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd40a2960 0x7f9bd40a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62ccdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be4d99640 1 -- 172.21.15.22:0/2669349396 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9bd80cf4f0 con 0x7f9be80ffa00 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9befbd9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa62c325f0 con 0x55aa62cd7730 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9befbd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa62bdb930 con 0x55aa62ccef20 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9befbd9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa62c2e690 con 0x55aa62cce1b0 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9bdffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42400/0 (socket says 172.21.15.22:42400) 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be759e640 1 -- 172.21.15.22:0/3892050271 learned_addr learned my addr 172.21.15.22:0/3892050271 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62ccdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be6d9d640 1 -- 172.21.15.22:0/3892050271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 msgr2=0x55aa62cce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be6d9d640 1 -- 172.21.15.22:0/3892050271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 msgr2=0x55aa62cd9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be6d9d640 1 -- 172.21.15.22:0/3892050271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa62947210 con 0x55aa62ccef20 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be759e640 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.343+0000 7f9be759e640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd40a2960 0x7f9bd40a4e20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009a60 tx=0x7f9be002ce80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.344+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62ccdc70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80cc8e0 tx=0x7f9bd809b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33667cb64d722847 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.344+0000 7f9bdd7fa640 1 -- 172.21.15.22:0/3892050271 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80ccb60 con 0x55aa62ccef20 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.344+0000 7f9bdd7fa640 1 -- 172.21.15.22:0/3892050271 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd8097f10 con 0x55aa62ccef20 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.344+0000 7f9bdd7fa640 1 -- 172.21.15.22:0/3892050271 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd809acf0 con 0x55aa62ccef20 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.344+0000 7f9befbd9740 1 -- 172.21.15.22:0/3892050271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 msgr2=0x55aa62ccdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.344+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62ccdc70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80cc8e0 tx=0x7f9bd809b7a0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- 172.21.15.22:0/3892050271 shutdown_connections 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3892050271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62ccdc70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- 172.21.15.22:0/3892050271 >> 172.21.15.22:0/3892050271 conn(0x55aa62c955b0 msgr2=0x55aa62cc8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- 172.21.15.22:0/3892050271 shutdown_connections 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- 172.21.15.22:0/3892050271 wait complete. 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- start start 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 Processor -- start 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62cd4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa62cda3a0 con 0x55aa62cd7730 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa62beed90 con 0x55aa62ccef20 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa62cda520 con 0x55aa62cce1b0 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9befbd9740 10 client.0 _ll_register_callbacks cb 0x55aa62946a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:48.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9be759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9be759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42410/0 (socket says 172.21.15.22:42410) 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9be6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9be759e640 1 -- 172.21.15.22:0/1994016387 learned_addr learned my addr 172.21.15.22:0/1994016387 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.345+0000 7f9bdffff640 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62cd4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9be759e640 1 -- 172.21.15.22:0/1994016387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 msgr2=0x55aa62cd47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9be759e640 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9be759e640 1 -- 172.21.15.22:0/1994016387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 msgr2=0x55aa62cd4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9be759e640 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62cd4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9be759e640 1 -- 172.21.15.22:0/1994016387 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa62a13620 con 0x55aa62cd7730 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9be759e640 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd5270 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009ca0 tx=0x7f9be0045070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a641407af2de4c48 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9bce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1994016387 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0052060 con 0x55aa62cd7730 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1994016387 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9be0034990 con 0x55aa62cd7730 2024-03-09T00:05:48.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd8097f10 con 0x55aa62cd7730 2024-03-09T00:05:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1994016387 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be0059050 con 0x55aa62cd7730 2024-03-09T00:05:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9befbd9740 10 client.19077 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55aa62ce6710 con 0x55aa62cd7730 2024-03-09T00:05:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1994016387 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd8097f10 con 0x55aa62cd7730 2024-03-09T00:05:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1994016387 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9be003f070 con 0x55aa62cd7730 2024-03-09T00:05:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9befbd9740 10 client.19077 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.346+0000 7f9befbd9740 10 client.19077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.347+0000 7f9befbd9740 20 client.19077 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.347+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62ce7bd0 0x55aa62ce9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.347+0000 7f9befbd9740 4 client.19077 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.347+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa62c3fb30 con 0x55aa62ce7bd0 2024-03-09T00:05:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.347+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62ce7bd0 0x55aa62ce9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.347+0000 7f9bce7fc640 10 client.19077 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.347+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62ce7bd0 0x55aa62ce9fc0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d3de498dffa0e7 server_cookie=edcbf63c9bf9e90b in_seq=0 out_seq=0 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1994016387 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa62c3fb30 con 0x55aa62ce7bd0 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9bce7fc640 10 client.19077 handle_command_reply: tid=0 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 client.19077 shutdown 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 msgr2=0x55aa62cd5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd5270 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009ca0 tx=0x7f9be0045070 comp rx=0 tx=0).stop 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 shutdown_connections 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62ce7bd0 0x55aa62ce9fc0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62cd7730 0x55aa62cd5270 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62ccef20 0x55aa62cd4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1994016387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 >> 172.21.15.22:0/1994016387 conn(0x55aa62c955b0 msgr2=0x55aa62ccabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.348+0000 7f9bce7fc640 0 client.19077 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.349+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 shutdown_connections 2024-03-09T00:05:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.349+0000 7f9befbd9740 1 -- 172.21.15.22:0/1994016387 wait complete. 2024-03-09T00:05:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.349+0000 7f9befbd9740 20 client.19077 trim_cache size 0 max 16384 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 Processor -- start 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 -- start start 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cd30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cd3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81b2c40 con 0x55aa62ccef20 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81b07e0 con 0x55aa62cd7730 2024-03-09T00:05:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9befbd9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81ae8a0 con 0x55aa62cce1b0 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9bdffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cd30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9bdffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cd30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42422/0 (socket says 172.21.15.22:42422) 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9bdffff640 1 -- 172.21.15.22:0/3001846223 learned_addr learned my addr 172.21.15.22:0/3001846223 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9be759e640 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cd3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.363+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdffff640 1 -- 172.21.15.22:0/3001846223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 msgr2=0x55aa62cd6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdffff640 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdffff640 1 -- 172.21.15.22:0/3001846223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 msgr2=0x55aa62cd3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdffff640 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cd3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdffff640 1 -- 172.21.15.22:0/3001846223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa62cc32f0 con 0x55aa62ccef20 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9be759e640 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cd3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdffff640 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cd30c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009db0 tx=0x7f9bd0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82e1103b00975ce server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdcff9640 1 -- 172.21.15.22:0/3001846223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0022070 con 0x55aa62ccef20 2024-03-09T00:05:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdcff9640 1 -- 172.21.15.22:0/3001846223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd00035f0 con 0x55aa62ccef20 2024-03-09T00:05:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.364+0000 7f9bdcff9640 1 -- 172.21.15.22:0/3001846223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd001d6c0 con 0x55aa62ccef20 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 -- 172.21.15.22:0/3001846223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 msgr2=0x55aa62cd30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cd30c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009db0 tx=0x7f9bd0005f90 comp rx=0 tx=0).stop 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 -- 172.21.15.22:0/3001846223 shutdown_connections 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cd3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cd30c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 --2- 172.21.15.22:0/3001846223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cd6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 -- 172.21.15.22:0/3001846223 >> 172.21.15.22:0/3001846223 conn(0x55aa62c955b0 msgr2=0x55aa62cc89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 -- 172.21.15.22:0/3001846223 shutdown_connections 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 -- 172.21.15.22:0/3001846223 wait complete. 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.365+0000 7f9befbd9740 1 -- start start 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 1 Processor -- start 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cde9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cdcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cdd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81aeea0 con 0x55aa62ccef20 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81b0180 con 0x55aa62cd7730 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81af4c0 con 0x55aa62cce1b0 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9befbd9740 10 client.0 _ll_register_callbacks cb 0x7f9bd8097500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cdcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cdcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42432/0 (socket says 172.21.15.22:42432) 2024-03-09T00:05:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 -- 172.21.15.22:0/1046763588 learned_addr learned my addr 172.21.15.22:0/1046763588 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cde9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9be759e640 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cdd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 -- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 msgr2=0x55aa62cde9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cde9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 -- 172.21.15.22:0/1046763588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 msgr2=0x55aa62cdd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cdd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 -- 172.21.15.22:0/1046763588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa62cdd9b0 con 0x55aa62ccef20 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bdffff640 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cdcd80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0012cb0 tx=0x7f9bd0012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf99fea9cfdda4e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.366+0000 7f9bce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1046763588 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0022070 con 0x55aa62ccef20 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1046763588 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9bd0004050 con 0x55aa62ccef20 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd00035f0 con 0x55aa62ccef20 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1046763588 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd0037050 con 0x55aa62ccef20 2024-03-09T00:05:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 10 client.19083 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55aa62d3efb0 con 0x55aa62ccef20 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1046763588 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd00035f0 con 0x55aa62ccef20 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1046763588 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9bd001d460 con 0x55aa62ccef20 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 10 client.19083 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 10 client.19083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 20 client.19083 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62d3f700 0x55aa62d41b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 4 client.19083 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aa62c39e80 con 0x55aa62d3f700 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.367+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62d3f700 0x55aa62d41b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.368+0000 7f9be6d9d640 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62d3f700 0x55aa62d41b20 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcd23b516929330 server_cookie=d834495540a7617a in_seq=0 out_seq=0 2024-03-09T00:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.368+0000 7f9bce7fc640 10 client.19083 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9bce7fc640 1 -- 172.21.15.22:0/1046763588 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aa62c39e80 con 0x55aa62d3f700 2024-03-09T00:05:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9bce7fc640 10 client.19083 handle_command_reply: tid=0 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 client.19083 shutdown 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 msgr2=0x55aa62cdcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cdcd80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0012cb0 tx=0x7f9bd0012ea0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 shutdown_connections 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa62d3f700 0x55aa62d41b20 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa62cd7730 0x55aa62cdd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa62ccef20 0x55aa62cdcd80 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9bce7fc640 0 client.19083 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 --2- 172.21.15.22:0/1046763588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa62cce1b0 0x55aa62cde9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 >> 172.21.15.22:0/1046763588 conn(0x55aa62c955b0 msgr2=0x55aa62cd3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 shutdown_connections 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 1 -- 172.21.15.22:0/1046763588 wait complete. 2024-03-09T00:05:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.724+0000 7f9befbd9740 20 client.19083 trim_cache size 0 max 16384 2024-03-09T00:05:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:05:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:05:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:05:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd40a2960 msgr2=0x7f9bd40a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd40a2960 0x7f9bd40a4e20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009a60 tx=0x7f9be002ce80 comp rx=0 tx=0).stop 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be80ffa00 msgr2=0x7f9be8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be80ffa00 0x7f9be8100be0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9f0 tx=0x7f9bd800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 shutdown_connections 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9bd40a2960 0x7f9bd40a4e20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9be8107340 0x7f9be8101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9be81003c0 0x7f9be8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 --2- 172.21.15.22:0/2669349396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9be80ffa00 0x7f9be8100be0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 >> 172.21.15.22:0/2669349396 conn(0x7f9be8075320 msgr2=0x7f9be80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 shutdown_connections 2024-03-09T00:05:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:05:48.725+0000 7f9bed001640 1 -- 172.21.15.22:0/2669349396 wait complete. 2024-03-09T00:05:48.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1136864 9.65 MB/sec execute 476 sec latency 245.929 ms 2024-03-09T00:05:49.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:48 smithi022 ceph-mon[31894]: pgmap v354: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-03-09T00:05:49.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:48 smithi062 ceph-mon[36129]: pgmap v354: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-03-09T00:05:49.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:48 smithi192 ceph-mon[35995]: pgmap v354: 129 pgs: 129 active+clean; 94 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-03-09T00:05:49.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1138262 9.64 MB/sec execute 477 sec latency 23.651 ms 2024-03-09T00:05:49.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:05:49.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:05:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.087+0000 7fe00d745640 1 -- 172.21.15.22:0/2211767473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe008100d80 msgr2=0x7fe008104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.087+0000 7fe00d745640 1 --2- 172.21.15.22:0/2211767473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe008100d80 0x7fe008104dc0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009900 tx=0x7fdff002bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.087+0000 7fe00d745640 1 -- 172.21.15.22:0/2211767473 shutdown_connections 2024-03-09T00:05:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.087+0000 7fe00d745640 1 --2- 172.21.15.22:0/2211767473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0081054a0 0x7fe00810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.087+0000 7fe00d745640 1 --2- 172.21.15.22:0/2211767473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe008100d80 0x7fe008104dc0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.087+0000 7fe00d745640 1 --2- 172.21.15.22:0/2211767473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0081003d0 0x7fe0081007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.087+0000 7fe00d745640 1 -- 172.21.15.22:0/2211767473 >> 172.21.15.22:0/2211767473 conn(0x7fe0080772f0 msgr2=0x7fe008077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 -- 172.21.15.22:0/2211767473 shutdown_connections 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 -- 172.21.15.22:0/2211767473 wait complete. 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 Processor -- start 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 -- start start 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0081003d0 0x7fe00819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0081054a0 0x7fe00819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe008197a60 0x7fe008197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008110530 con 0x7fe0081003d0 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081103b0 con 0x7fe0081054a0 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe00d745640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081106b0 con 0x7fe008197a60 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe006ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0081003d0 0x7fe00819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe006ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0081003d0 0x7fe00819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42452/0 (socket says 172.21.15.22:42452) 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.088+0000 7fe006ffd640 1 -- 172.21.15.22:0/1075043698 learned_addr learned my addr 172.21.15.22:0/1075043698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fdffffff640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0081054a0 0x7fe00819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe0077fe640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe008197a60 0x7fe008197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe006ffd640 1 -- 172.21.15.22:0/1075043698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe008197a60 msgr2=0x7fe008197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe006ffd640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe008197a60 0x7fe008197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe006ffd640 1 -- 172.21.15.22:0/1075043698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0081054a0 msgr2=0x7fe00819ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe006ffd640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0081054a0 0x7fe00819ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe006ffd640 1 -- 172.21.15.22:0/1075043698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008198580 con 0x7fe0081003d0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe006ffd640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0081003d0 0x7fe00819d8b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdff800e9f0 tx=0x7fdff800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe004ff9640 1 -- 172.21.15.22:0/1075043698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800cd80 con 0x7fe0081003d0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe008198870 con 0x7fe0081003d0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe004ff9640 1 -- 172.21.15.22:0/1075043698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdff8010910 con 0x7fe0081003d0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe004ff9640 1 -- 172.21.15.22:0/1075043698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff80188f0 con 0x7fe0081003d0 2024-03-09T00:05:50.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.089+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0081a4b10 con 0x7fe0081003d0 2024-03-09T00:05:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.092+0000 7fe004ff9640 1 -- 172.21.15.22:0/1075043698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdff8010ac0 con 0x7fe0081003d0 2024-03-09T00:05:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 Processor -- start 2024-03-09T00:05:50.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 -- start start 2024-03-09T00:05:50.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7be720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dd718e00 con 0x5629dd7b5760 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dd6c1b10 con 0x5629dd7b6080 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe010404740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dd714ea0 con 0x5629dd7bec60 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe004ff9640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdff40a2970 0x7fdff40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fdffffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42468/0 (socket says 172.21.15.22:42468) 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe0077fe640 1 -- 172.21.15.22:0/3944439202 learned_addr learned my addr 172.21.15.22:0/3944439202 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe006ffd640 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe0077fe640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdff40a2970 0x7fdff40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fdffffff640 1 -- 172.21.15.22:0/3944439202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 msgr2=0x5629dd7be720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fdffffff640 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7be720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fe004ff9640 1 -- 172.21.15.22:0/1075043698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdff80cf600 con 0x7fe0081003d0 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fdffffff640 1 -- 172.21.15.22:0/3944439202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 msgr2=0x5629dd7b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fdffffff640 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.093+0000 7fdffffff640 1 -- 172.21.15.22:0/3944439202 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629dd4f77d0 con 0x5629dd7bec60 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.094+0000 7fe0077fe640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdff40a2970 0x7fdff40a4e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0000099e0 tx=0x7fe000009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.094+0000 7fdffffff640 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7c1050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdff0036040 tx=0x7fdff0005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8492e749fed40cec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.094+0000 7fdffd7fa640 1 -- 172.21.15.22:0/3944439202 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0042070 con 0x5629dd7bec60 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.094+0000 7fdffd7fa640 1 -- 172.21.15.22:0/3944439202 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdff0004030 con 0x5629dd7bec60 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.094+0000 7fdffd7fa640 1 -- 172.21.15.22:0/3944439202 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0004f00 con 0x5629dd7bec60 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 -- 172.21.15.22:0/3944439202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 msgr2=0x5629dd7c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7c1050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdff0036040 tx=0x7fdff0005be0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 -- 172.21.15.22:0/3944439202 shutdown_connections 2024-03-09T00:05:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7c1050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7be720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 --2- 172.21.15.22:0/3944439202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 -- 172.21.15.22:0/3944439202 >> 172.21.15.22:0/3944439202 conn(0x5629dd7afe90 msgr2=0x5629dd7b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 -- 172.21.15.22:0/3944439202 shutdown_connections 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 -- 172.21.15.22:0/3944439202 wait complete. 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 -- start start 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 Processor -- start 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7bb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7bbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.095+0000 7fe010404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dd7c19f0 con 0x5629dd7b5760 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe010404740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dd6d50a0 con 0x5629dd7b6080 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe010404740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629dd7c1b70 con 0x5629dd7bec60 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe010404740 10 client.0 _ll_register_callbacks cb 0x5629dd6d4a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7be490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42474/0 (socket says 172.21.15.22:42474) 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 -- 172.21.15.22:0/3861804277 learned_addr learned my addr 172.21.15.22:0/3861804277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe0077fe640 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7bbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fdffffff640 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7bb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 -- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 msgr2=0x5629dd7bbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7bbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 -- 172.21.15.22:0/3861804277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 msgr2=0x5629dd7bb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7bb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 -- 172.21.15.22:0/3861804277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629dd42d3f0 con 0x5629dd7b5760 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe006ffd640 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7be490 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fdff809b8a0 tx=0x7fdff809b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4822926b0aec6485 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fdfee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fdfee7fc640 1 -- 172.21.15.22:0/3861804277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8026040 con 0x5629dd7b5760 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fdfee7fc640 1 -- 172.21.15.22:0/3861804277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdff80cf3e0 con 0x5629dd7b5760 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.096+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdff0004030 con 0x5629dd7b5760 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fdfee7fc640 1 -- 172.21.15.22:0/3861804277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdff80a4000 con 0x5629dd7b5760 2024-03-09T00:05:50.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 10 client.19095 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5629dd3ed0f0 con 0x5629dd7b5760 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fdfee7fc640 1 -- 172.21.15.22:0/3861804277 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdff0004030 con 0x5629dd7b5760 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fdfee7fc640 1 -- 172.21.15.22:0/3861804277 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdff809cb90 con 0x5629dd7b5760 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 10 client.19095 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 10 client.19095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 20 client.19095 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd7ce6e0 0x5629dd7d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 4 client.19095 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629dd726340 con 0x5629dd7ce6e0 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.097+0000 7fdffffff640 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd7ce6e0 0x5629dd7d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.098+0000 7fdffffff640 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd7ce6e0 0x5629dd7d0af0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bdd3c0243ab0ba5 server_cookie=f27f32612f8ffdea in_seq=0 out_seq=0 2024-03-09T00:05:50.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.098+0000 7fdfee7fc640 10 client.19095 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fdfee7fc640 1 -- 172.21.15.22:0/3861804277 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629dd726340 con 0x5629dd7ce6e0 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fdfee7fc640 10 client.19095 handle_command_reply: tid=0 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 client.19095 shutdown 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 msgr2=0x5629dd7be490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7be490 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fdff809b8a0 tx=0x7fdff809b510 comp rx=0 tx=0).stop 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 shutdown_connections 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd7ce6e0 0x5629dd7d0af0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7bbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b6080 0x5629dd7bb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fdfee7fc640 0 client.19095 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 --2- 172.21.15.22:0/3861804277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b5760 0x5629dd7be490 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 >> 172.21.15.22:0/3861804277 conn(0x5629dd7afe90 msgr2=0x5629dd7b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 shutdown_connections 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 1 -- 172.21.15.22:0/3861804277 wait complete. 2024-03-09T00:05:50.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.099+0000 7fe010404740 20 client.19095 trim_cache size 0 max 16384 2024-03-09T00:05:50.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.115+0000 7fe010404740 1 Processor -- start 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.115+0000 7fe010404740 1 -- start start 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe010404740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b5760 0x5629dd7bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe010404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe010404740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe010404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081b0c10 con 0x5629dd7b6080 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe010404740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081ae7b0 con 0x5629dd7b5760 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe010404740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081ac870 con 0x5629dd7bec60 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7be490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42480/0 (socket says 172.21.15.22:42480) 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 -- 172.21.15.22:0/1201546547 learned_addr learned my addr 172.21.15.22:0/1201546547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe0077fe640 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fe006ffd640 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b5760 0x5629dd7bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 -- 172.21.15.22:0/1201546547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 msgr2=0x5629dd7b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 -- 172.21.15.22:0/1201546547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b5760 msgr2=0x5629dd7bdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b5760 0x5629dd7bdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 -- 172.21.15.22:0/1201546547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629dd7d5cb0 con 0x5629dd7b6080 2024-03-09T00:05:50.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.116+0000 7fdffffff640 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7be490 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdff002eff0 tx=0x7fdff0048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcb4e3e71cfede9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fdffcff9640 1 -- 172.21.15.22:0/1201546547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0053070 con 0x5629dd7b6080 2024-03-09T00:05:50.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fdffcff9640 1 -- 172.21.15.22:0/1201546547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdff004ebe0 con 0x5629dd7b6080 2024-03-09T00:05:50.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fdffcff9640 1 -- 172.21.15.22:0/1201546547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0042050 con 0x5629dd7b6080 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fe010404740 1 -- 172.21.15.22:0/1201546547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 msgr2=0x5629dd7be490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fe010404740 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7be490 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdff002eff0 tx=0x7fdff0048040 comp rx=0 tx=0).stop 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fe010404740 1 -- 172.21.15.22:0/1201546547 shutdown_connections 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fe010404740 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7bec60 0x5629dd7b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fe010404740 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7be490 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fe010404740 1 --2- 172.21.15.22:0/1201546547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7b5760 0x5629dd7bdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.117+0000 7fe010404740 1 -- 172.21.15.22:0/1201546547 >> 172.21.15.22:0/1201546547 conn(0x5629dd7afe90 msgr2=0x5629dd7bb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 -- 172.21.15.22:0/1201546547 shutdown_connections 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 -- 172.21.15.22:0/1201546547 wait complete. 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 -- start start 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 Processor -- start 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7b5760 0x5629dd7c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7bec60 0x5629dd7c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081ace70 con 0x5629dd7b6080 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081ae150 con 0x5629dd7bec60 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081ad490 con 0x5629dd7b5760 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe010404740 10 client.0 _ll_register_callbacks cb 0x7fdff800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fdffffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fdffffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42490/0 (socket says 172.21.15.22:42490) 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fdffffff640 1 -- 172.21.15.22:0/2672828383 learned_addr learned my addr 172.21.15.22:0/2672828383 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe006ffd640 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7b5760 0x5629dd7c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.118+0000 7fe0077fe640 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7bec60 0x5629dd7c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdffffff640 1 -- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7b5760 msgr2=0x5629dd7c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdffffff640 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7b5760 0x5629dd7c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdffffff640 1 -- 172.21.15.22:0/2672828383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7bec60 msgr2=0x5629dd7c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdffffff640 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7bec60 0x5629dd7c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdffffff640 1 -- 172.21.15.22:0/2672828383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629dd7c4660 con 0x5629dd7b6080 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdffffff640 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7c5f80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdff002bc70 tx=0x7fdff0009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b939793df3be0e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdfee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdfee7fc640 1 -- 172.21.15.22:0/2672828383 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0053070 con 0x5629dd7b6080 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdfee7fc640 1 -- 172.21.15.22:0/2672828383 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdff002c880 con 0x5629dd7b6080 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdff004ebe0 con 0x5629dd7b6080 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdfee7fc640 1 -- 172.21.15.22:0/2672828383 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdff0060050 con 0x5629dd7b6080 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fe010404740 10 client.19101 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5629dd825640 con 0x5629dd7b6080 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.119+0000 7fdfee7fc640 1 -- 172.21.15.22:0/2672828383 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdff004ebe0 con 0x5629dd7b6080 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fdfee7fc640 1 -- 172.21.15.22:0/2672828383 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdff004e4f0 con 0x5629dd7b6080 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe010404740 10 client.19101 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe010404740 10 client.19101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe010404740 20 client.19101 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe010404740 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd8258b0 0x5629dd827ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe010404740 4 client.19101 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5629dd720690 con 0x5629dd8258b0 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe006ffd640 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd8258b0 0x5629dd827ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fe006ffd640 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd8258b0 0x5629dd827ca0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f051a16f3dfae3 server_cookie=322f75ff4b1a9122 in_seq=0 out_seq=0 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.120+0000 7fdfee7fc640 10 client.19101 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fdfee7fc640 1 -- 172.21.15.22:0/2672828383 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5629dd720690 con 0x5629dd8258b0 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fdfee7fc640 10 client.19101 handle_command_reply: tid=0 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 client.19101 shutdown 2024-03-09T00:05:50.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 msgr2=0x5629dd7c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7c5f80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdff002bc70 tx=0x7fdff0009290 comp rx=0 tx=0).stop 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 shutdown_connections 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629dd8258b0 0x5629dd827ca0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629dd7bec60 0x5629dd7c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629dd7b6080 0x5629dd7c5f80 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fdfee7fc640 0 client.19101 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 --2- 172.21.15.22:0/2672828383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629dd7b5760 0x5629dd7c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 >> 172.21.15.22:0/2672828383 conn(0x5629dd7afe90 msgr2=0x5629dd7ba0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 shutdown_connections 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 1 -- 172.21.15.22:0/2672828383 wait complete. 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.121+0000 7fe010404740 20 client.19101 trim_cache size 0 max 16384 2024-03-09T00:05:50.121 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:05:50.122 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:05:50.122 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:05:50.122 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdff40a2970 msgr2=0x7fdff40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdff40a2970 0x7fdff40a4e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0000099e0 tx=0x7fe000009500 comp rx=0 tx=0).stop 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0081003d0 msgr2=0x7fe00819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0081003d0 0x7fe00819d8b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdff800e9f0 tx=0x7fdff800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 shutdown_connections 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdff40a2970 0x7fdff40a4e30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe008197a60 0x7fe008197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0081054a0 0x7fe00819ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 --2- 172.21.15.22:0/1075043698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0081003d0 0x7fe00819d8b0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 >> 172.21.15.22:0/1075043698 conn(0x7fe0080772f0 msgr2=0x7fe008068790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 shutdown_connections 2024-03-09T00:05:50.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.122+0000 7fe00d745640 1 -- 172.21.15.22:0/1075043698 wait complete. 2024-03-09T00:05:50.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:05:50.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:05:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.244+0000 7f6ac2d09640 1 -- 172.21.15.22:0/2441903956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 msgr2=0x7f6abc100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.244+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/2441903956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc100840 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009ca0 tx=0x7f6ab802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.244+0000 7f6ac2d09640 1 -- 172.21.15.22:0/2441903956 shutdown_connections 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.244+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/2441903956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6abc105530 0x7f6abc10d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.244+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/2441903956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6abc100e10 0x7f6abc104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.244+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/2441903956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc100840 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.244+0000 7f6ac2d09640 1 -- 172.21.15.22:0/2441903956 >> 172.21.15.22:0/2441903956 conn(0x7f6abc0772f0 msgr2=0x7f6abc077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 -- 172.21.15.22:0/2441903956 shutdown_connections 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 -- 172.21.15.22:0/2441903956 wait complete. 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 Processor -- start 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 -- start start 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6abc105530 0x7f6abc198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6abc199540 0x7f6abc1a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1105f0 con 0x7f6abc100460 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc110470 con 0x7f6abc105530 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac2d09640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc110770 con 0x7f6abc199540 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac0aa5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac0aa5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42506/0 (socket says 172.21.15.22:42506) 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.245+0000 7f6ac0aa5640 1 -- 172.21.15.22:0/1904072868 learned_addr learned my addr 172.21.15.22:0/1904072868 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac12a6640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6abc199540 0x7f6abc1a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ab3fff640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6abc105530 0x7f6abc198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac0aa5640 1 -- 172.21.15.22:0/1904072868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6abc199540 msgr2=0x7f6abc1a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac0aa5640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6abc199540 0x7f6abc1a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac0aa5640 1 -- 172.21.15.22:0/1904072868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6abc105530 msgr2=0x7f6abc198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac0aa5640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6abc105530 0x7f6abc198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac0aa5640 1 -- 172.21.15.22:0/1904072868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc1a42f0 con 0x7f6abc100460 2024-03-09T00:05:50.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac0aa5640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc198a70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005e30 tx=0x7f6ab8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ab1ffb640 1 -- 172.21.15.22:0/1904072868 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8041020 con 0x7f6abc100460 2024-03-09T00:05:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ab1ffb640 1 -- 172.21.15.22:0/1904072868 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ab8034d90 con 0x7f6abc100460 2024-03-09T00:05:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc1a4580 con 0x7f6abc100460 2024-03-09T00:05:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ab1ffb640 1 -- 172.21.15.22:0/1904072868 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8039040 con 0x7f6abc100460 2024-03-09T00:05:50.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.246+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc1a4a70 con 0x7f6abc100460 2024-03-09T00:05:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.248+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc0680e0 con 0x7f6abc100460 2024-03-09T00:05:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.249+0000 7f6ab1ffb640 1 -- 172.21.15.22:0/1904072868 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6ab8044020 con 0x7f6abc100460 2024-03-09T00:05:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.249+0000 7f6ab1ffb640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6aa40a2970 0x7f6aa40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.250+0000 7f6ab3fff640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6aa40a2970 0x7f6aa40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.250+0000 7f6ab1ffb640 1 -- 172.21.15.22:0/1904072868 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6ab80f11e0 con 0x7f6abc100460 2024-03-09T00:05:50.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.250+0000 7f6ab3fff640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6aa40a2970 0x7f6aa40a4e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6abc068fc0 tx=0x7f6aac005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.253+0000 7f6ab1ffb640 1 -- 172.21.15.22:0/1904072868 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6ab804b020 con 0x7f6abc100460 2024-03-09T00:05:50.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.498+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6abc108760 con 0x7f6abc100460 2024-03-09T00:05:50.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.499+0000 7f6ab1ffb640 1 -- 172.21.15.22:0/1904072868 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f6ab80bdc50 con 0x7f6abc100460 2024-03-09T00:05:50.494 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:05:50.494 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:05:50.494 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6aa40a2970 msgr2=0x7f6aa40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6aa40a2970 0x7f6aa40a4e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6abc068fc0 tx=0x7f6aac005f90 comp rx=0 tx=0).stop 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 msgr2=0x7f6abc198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc198a70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005e30 tx=0x7f6ab8032f20 comp rx=0 tx=0).stop 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 shutdown_connections 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6aa40a2970 0x7f6aa40a4e30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6abc199540 0x7f6abc1a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6abc105530 0x7f6abc198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 --2- 172.21.15.22:0/1904072868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6abc100460 0x7f6abc198a70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 >> 172.21.15.22:0/1904072868 conn(0x7f6abc0772f0 msgr2=0x7f6abc0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 shutdown_connections 2024-03-09T00:05:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.501+0000 7f6ac2d09640 1 -- 172.21.15.22:0/1904072868 wait complete. 2024-03-09T00:05:50.506 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:05:50.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:05:50.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 -- 172.21.15.22:0/953867416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 msgr2=0x7f497810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 --2- 172.21.15.22:0/953867416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497810e1a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4970009ca0 tx=0x7f497002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 -- 172.21.15.22:0/953867416 shutdown_connections 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 --2- 172.21.15.22:0/953867416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497810e1a0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 --2- 172.21.15.22:0/953867416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49781003c0 0x7f4978105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 --2- 172.21.15.22:0/953867416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49780ffa10 0x7f49780ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 -- 172.21.15.22:0/953867416 >> 172.21.15.22:0/953867416 conn(0x7f4978076700 msgr2=0x7f49780fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 -- 172.21.15.22:0/953867416 shutdown_connections 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.618+0000 7f497ebd4640 1 -- 172.21.15.22:0/953867416 wait complete. 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 Processor -- start 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 -- start start 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49780ffa10 0x7f497819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4978197930 0x7f4978197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978111670 con 0x7f49781065d0 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781114f0 con 0x7f49780ffa10 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497ebd4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781117f0 con 0x7f4978197930 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f4977fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f4977fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497819dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37134/0 (socket says 172.21.15.22:37134) 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f4977fff640 1 -- 172.21.15.22:0/901937081 learned_addr learned my addr 172.21.15.22:0/901937081 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497d171640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4978197930 0x7f4978197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.619+0000 7f497c970640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49780ffa10 0x7f497819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4977fff640 1 -- 172.21.15.22:0/901937081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4978197930 msgr2=0x7f4978197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4977fff640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4978197930 0x7f4978197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4977fff640 1 -- 172.21.15.22:0/901937081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49780ffa10 msgr2=0x7f497819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4977fff640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49780ffa10 0x7f497819d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4977fff640 1 -- 172.21.15.22:0/901937081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978198450 con 0x7f49781065d0 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4977fff640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497819dcc0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f496400e9c0 tx=0x7f496400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4975ffb640 1 -- 172.21.15.22:0/901937081 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400cd80 con 0x7f49781065d0 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4975ffb640 1 -- 172.21.15.22:0/901937081 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4964005500 con 0x7f49781065d0 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f4975ffb640 1 -- 172.21.15.22:0/901937081 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964010690 con 0x7f49781065d0 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4978198740 con 0x7f49781065d0 2024-03-09T00:05:50.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.620+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49781a4990 con 0x7f49781065d0 2024-03-09T00:05:50.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f4975ffb640 1 -- 172.21.15.22:0/901937081 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f49640046c0 con 0x7f49781065d0 2024-03-09T00:05:50.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 Processor -- start 2024-03-09T00:05:50.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 -- start start 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656581580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556656581ac0 0x55665658a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556656548360 con 0x5566565811a0 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566564e4ec0 con 0x55665658a6a0 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f49817b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665648dd30 con 0x556656581ac0 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f497d171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656581580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f497d171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656581580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37150/0 (socket says 172.21.15.22:37150) 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.623+0000 7f497d171640 1 -- 172.21.15.22:0/1321487057 learned_addr learned my addr 172.21.15.22:0/1321487057 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497c970640 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556656581ac0 0x55665658a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f4975ffb640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49680a2970 0x7f49680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f4977fff640 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497d171640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49680a2970 0x7f49680a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497c970640 1 -- 172.21.15.22:0/1321487057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 msgr2=0x55665658ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497c970640 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497c970640 1 -- 172.21.15.22:0/1321487057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 msgr2=0x556656581580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497c970640 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656581580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f4975ffb640 1 -- 172.21.15.22:0/901937081 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f49640cf4d0 con 0x7f49781065d0 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497c970640 1 -- 172.21.15.22:0/1321487057 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665648cbc0 con 0x556656581ac0 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f4977fff640 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497c970640 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556656581ac0 0x55665658a160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f496c009bf0 tx=0x7f496c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9df44eafad0f8c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f497d171640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49680a2970 0x7f49680a4e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4970009430 tx=0x7f497002c0f0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f4962ffd640 1 -- 172.21.15.22:0/1321487057 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c01e070 con 0x556656581ac0 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f4962ffd640 1 -- 172.21.15.22:0/1321487057 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f496c0035f0 con 0x556656581ac0 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.624+0000 7f4962ffd640 1 -- 172.21.15.22:0/1321487057 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c019690 con 0x556656581ac0 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 -- 172.21.15.22:0/1321487057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556656581ac0 msgr2=0x55665658a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556656581ac0 0x55665658a160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f496c009bf0 tx=0x7f496c011810 comp rx=0 tx=0).stop 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 -- 172.21.15.22:0/1321487057 shutdown_connections 2024-03-09T00:05:50.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556656581ac0 0x55665658a160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 --2- 172.21.15.22:0/1321487057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656581580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 -- 172.21.15.22:0/1321487057 >> 172.21.15.22:0/1321487057 conn(0x55665657b8d0 msgr2=0x55665657bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 -- 172.21.15.22:0/1321487057 shutdown_connections 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 -- 172.21.15.22:0/1321487057 wait complete. 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 -- start start 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.625+0000 7f49817b3740 1 Processor -- start 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f49817b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f49817b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x556656587c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f49817b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55665658a6a0 0x556656588150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f49817b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566564834a0 con 0x55665658a6a0 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f49817b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566564e0f60 con 0x556656581ac0 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f49817b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665658d490 con 0x5566565811a0 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f49817b3740 10 client.0 _ll_register_callbacks cb 0x5566564ded60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55665658a6a0 0x556656588150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55665658a6a0 0x556656588150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37160/0 (socket says 172.21.15.22:37160) 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 -- 172.21.15.22:0/411917404 learned_addr learned my addr 172.21.15.22:0/411917404 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f4977fff640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x556656587c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497c970640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 -- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 msgr2=0x5566565876d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565876d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 -- 172.21.15.22:0/411917404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 msgr2=0x556656587c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x556656587c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 -- 172.21.15.22:0/411917404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566561f93f0 con 0x55665658a6a0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f4977fff640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x556656587c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f497d171640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55665658a6a0 0x556656588150 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f49700096b0 tx=0x7f49700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a28d463d496403 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f4957fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.626+0000 7f4957fff640 1 -- 172.21.15.22:0/411917404 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4970053060 con 0x55665658a6a0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f4957fff640 1 -- 172.21.15.22:0/411917404 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4970003d80 con 0x55665658a6a0 2024-03-09T00:05:50.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f496c0035f0 con 0x55665658a6a0 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f4957fff640 1 -- 172.21.15.22:0/411917404 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f497005a050 con 0x55665658a6a0 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 10 client.19119 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5566562c37d0 con 0x55665658a6a0 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f4957fff640 1 -- 172.21.15.22:0/411917404 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f496c0035f0 con 0x55665658a6a0 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f4957fff640 1 -- 172.21.15.22:0/411917404 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f497003f070 con 0x55665658a6a0 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 10 client.19119 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 10 client.19119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 20 client.19119 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55665659a810 0x55665659cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 4 client.19119 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566564f2400 con 0x55665659a810 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.627+0000 7f497c970640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55665659a810 0x55665659cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f4957fff640 10 client.19119 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f497c970640 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55665659a810 0x55665659cc00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6ec665fd8bc872b server_cookie=2d270d6589258174 in_seq=0 out_seq=0 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f4957fff640 1 -- 172.21.15.22:0/411917404 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566564f2400 con 0x55665659a810 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f4957fff640 10 client.19119 handle_command_reply: tid=0 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 client.19119 shutdown 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55665658a6a0 msgr2=0x556656588150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55665658a6a0 0x556656588150 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f49700096b0 tx=0x7f49700095c0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 shutdown_connections 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55665659a810 0x55665659cc00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55665658a6a0 0x556656588150 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f4957fff640 0 client.19119 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x556656587c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 --2- 172.21.15.22:0/411917404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565876d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.628+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 >> 172.21.15.22:0/411917404 conn(0x55665657b8d0 msgr2=0x55665657d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.629+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 shutdown_connections 2024-03-09T00:05:50.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.629+0000 7f49817b3740 1 -- 172.21.15.22:0/411917404 wait complete. 2024-03-09T00:05:50.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.629+0000 7f49817b3740 20 client.19119 trim_cache size 0 max 16384 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.640+0000 7f49817b3740 1 Processor -- start 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.640+0000 7f49817b3740 1 -- start start 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.640+0000 7f49817b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656589e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49817b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x5566565a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49817b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665658a6a0 0x556656585fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49817b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781b0c70 con 0x5566565811a0 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49817b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781ae810 con 0x556656581ac0 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49817b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781ac8d0 con 0x55665658a6a0 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656589e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656589e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37164/0 (socket says 172.21.15.22:37164) 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 -- 172.21.15.22:0/2600835591 learned_addr learned my addr 172.21.15.22:0/2600835591 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497d171640 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665658a6a0 0x556656585fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f4977fff640 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x5566565a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 -- 172.21.15.22:0/2600835591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665658a6a0 msgr2=0x556656585fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665658a6a0 0x556656585fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 -- 172.21.15.22:0/2600835591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 msgr2=0x5566565a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x5566565a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 -- 172.21.15.22:0/2600835591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566565874a0 con 0x5566565811a0 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497d171640 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665658a6a0 0x556656585fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f4977fff640 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x5566565a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f497c970640 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656589e00 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f496c009b40 tx=0x7f496c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7935533553272d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49627fc640 1 -- 172.21.15.22:0/2600835591 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c030070 con 0x5566565811a0 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49627fc640 1 -- 172.21.15.22:0/2600835591 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f496c01f040 con 0x5566565811a0 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.641+0000 7f49627fc640 1 -- 172.21.15.22:0/2600835591 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c01e040 con 0x5566565811a0 2024-03-09T00:05:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 -- 172.21.15.22:0/2600835591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 msgr2=0x556656589e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656589e00 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f496c009b40 tx=0x7f496c009e70 comp rx=0 tx=0).stop 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 -- 172.21.15.22:0/2600835591 shutdown_connections 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665658a6a0 0x556656585fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556656581ac0 0x5566565a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 --2- 172.21.15.22:0/2600835591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5566565811a0 0x556656589e00 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 -- 172.21.15.22:0/2600835591 >> 172.21.15.22:0/2600835591 conn(0x55665657b8d0 msgr2=0x556656582a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 -- 172.21.15.22:0/2600835591 shutdown_connections 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 -- 172.21.15.22:0/2600835591 wait complete. 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 -- start start 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.642+0000 7f49817b3740 1 Processor -- start 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f49817b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f49817b3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556656581ac0 0x556656591a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f49817b3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f49817b3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781aced0 con 0x556656581ac0 2024-03-09T00:05:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f49817b3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781ae1b0 con 0x55665658a6a0 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f49817b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781ad4f0 con 0x5566565811a0 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f49817b3740 10 client.0 _ll_register_callbacks cb 0x7f496c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f4977fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556656581ac0 0x556656591a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565914c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38818/0 (socket says 172.21.15.22:38818) 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 -- 172.21.15.22:0/911880415 learned_addr learned my addr 172.21.15.22:0/911880415 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f4977fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556656581ac0 0x556656591a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37178/0 (socket says 172.21.15.22:37178) 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497d171640 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 -- 172.21.15.22:0/911880415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 msgr2=0x55665658f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 -- 172.21.15.22:0/911880415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556656581ac0 msgr2=0x556656591a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556656581ac0 0x556656591a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 -- 172.21.15.22:0/911880415 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566565900f0 con 0x5566565811a0 2024-03-09T00:05:50.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f4977fff640 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556656581ac0 0x556656591a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f497c970640 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565914c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f496c009bf0 tx=0x7f496c005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a69cea8c78ba9e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f4957fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.643+0000 7f4957fff640 1 -- 172.21.15.22:0/911880415 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c039060 con 0x5566565811a0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f496c01f040 con 0x5566565811a0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f4957fff640 1 -- 172.21.15.22:0/911880415 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f496c0042d0 con 0x5566565811a0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f4957fff640 1 -- 172.21.15.22:0/911880415 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f496c040050 con 0x5566565811a0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 10 client.28552 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5566565f1750 con 0x5566565811a0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f4957fff640 1 -- 172.21.15.22:0/911880415 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f496c01f040 con 0x5566565811a0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f4957fff640 1 -- 172.21.15.22:0/911880415 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f496c033030 con 0x5566565811a0 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 10 client.28552 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 10 client.28552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 20 client.28552 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566565f1c30 0x5566565f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 4 client.28552 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566564ec750 con 0x5566565f1c30 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.644+0000 7f4977fff640 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566565f1c30 0x5566565f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f4957fff640 10 client.28552 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f4977fff640 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566565f1c30 0x5566565f4020 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=388dfcb2815b2068 server_cookie=3b15dc58c413fca2 in_seq=0 out_seq=0 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f4957fff640 1 -- 172.21.15.22:0/911880415 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566564ec750 con 0x5566565f1c30 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f4957fff640 10 client.28552 handle_command_reply: tid=0 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 client.28552 shutdown 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 msgr2=0x5566565914c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565914c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f496c009bf0 tx=0x7f496c005da0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 shutdown_connections 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5566565f1c30 0x5566565f4020 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55665658a6a0 0x55665658f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556656581ac0 0x556656591a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 --2- 172.21.15.22:0/911880415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566565811a0 0x5566565914c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f4957fff640 0 client.28552 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:05:50.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 >> 172.21.15.22:0/911880415 conn(0x55665657b8d0 msgr2=0x5566565a1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 shutdown_connections 2024-03-09T00:05:50.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 1 -- 172.21.15.22:0/911880415 wait complete. 2024-03-09T00:05:50.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.645+0000 7f49817b3740 20 client.28552 trim_cache size 0 max 16384 2024-03-09T00:05:50.647 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49680a2970 msgr2=0x7f49680a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49680a2970 0x7f49680a4e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4970009430 tx=0x7f497002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 msgr2=0x7f497819dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497819dcc0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f496400e9c0 tx=0x7f496400ee80 comp rx=0 tx=0).stop 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 shutdown_connections 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f49680a2970 0x7f49680a4e30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4978197930 0x7f4978197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49781065d0 0x7f497819dcc0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 --2- 172.21.15.22:0/901937081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49780ffa10 0x7f497819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 >> 172.21.15.22:0/901937081 conn(0x7f4978076700 msgr2=0x7f4978104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 shutdown_connections 2024-03-09T00:05:50.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.646+0000 7f497ebd4640 1 -- 172.21.15.22:0/901937081 wait complete. 2024-03-09T00:05:50.652 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:05:50.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:05:50.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1140337 9.65 MB/sec execute 478 sec latency 35.675 ms 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 -- 172.21.15.22:0/2883655491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 msgr2=0x7f285810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 --2- 172.21.15.22:0/2883655491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285810d080 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2850009ca0 tx=0x7f285002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 -- 172.21.15.22:0/2883655491 shutdown_connections 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 --2- 172.21.15.22:0/2883655491 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285810d080 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 --2- 172.21.15.22:0/2883655491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2858100d90 0x7f2858104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 --2- 172.21.15.22:0/2883655491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28581003e0 0x7f28581007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 -- 172.21.15.22:0/2883655491 >> 172.21.15.22:0/2883655491 conn(0x7f28580773a0 msgr2=0x7f28580777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 -- 172.21.15.22:0/2883655491 shutdown_connections 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.764+0000 7f285d509640 1 -- 172.21.15.22:0/2883655491 wait complete. 2024-03-09T00:05:50.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 Processor -- start 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 -- start start 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28581003e0 0x7f285819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2858199b70 0x7f2858199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285810f460 con 0x7f28581054b0 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285810f2e0 con 0x7f28581003e0 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f285d509640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285810f5e0 con 0x7f2858199b70 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285819ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37192/0 (socket says 172.21.15.22:37192) 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 -- 172.21.15.22:0/187463481 learned_addr learned my addr 172.21.15.22:0/187463481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f2856ffd640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28581003e0 0x7f285819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f28577fe640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2858199b70 0x7f2858199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 -- 172.21.15.22:0/187463481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2858199b70 msgr2=0x7f2858199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2858199b70 0x7f2858199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 -- 172.21.15.22:0/187463481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28581003e0 msgr2=0x7f285819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28581003e0 0x7f285819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.765+0000 7f284ffff640 1 -- 172.21.15.22:0/187463481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285819a770 con 0x7f28581054b0 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.766+0000 7f28577fe640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2858199b70 0x7f2858199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.766+0000 7f284ffff640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285819ffa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f284000e9c0 tx=0x7f284000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.766+0000 7f2854ff9640 1 -- 172.21.15.22:0/187463481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284000cd80 con 0x7f28581054b0 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.766+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28581a6800 con 0x7f28581054b0 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.766+0000 7f2854ff9640 1 -- 172.21.15.22:0/187463481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2840005500 con 0x7f28581054b0 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.766+0000 7f2854ff9640 1 -- 172.21.15.22:0/187463481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840010690 con 0x7f28581054b0 2024-03-09T00:05:50.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.766+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28581a6c60 con 0x7f28581054b0 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.768+0000 7f2854ff9640 1 -- 172.21.15.22:0/187463481 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f28400046c0 con 0x7f28581054b0 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 Processor -- start 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 -- start start 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5962580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda596b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f2854ff9640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28440a2970 0x7f28440a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda596da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda5929360 con 0x55bda5962ac0 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda58c5ec0 con 0x55bda59621a0 2024-03-09T00:05:50.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28601ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda586ed30 con 0x55bda596b6a0 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda596b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f284ffff640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28440a2970 0x7f28440a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5962580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda596b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37208/0 (socket says 172.21.15.22:37208) 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f28577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5962580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:49304/0 (socket says 172.21.15.22:49304) 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.769+0000 7f2856ffd640 1 -- 172.21.15.22:0/1398736770 learned_addr learned my addr 172.21.15.22:0/1398736770 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f2854ff9640 1 -- 172.21.15.22:0/187463481 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f28400cf500 con 0x7f28581054b0 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f28577fe640 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda596da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f28577fe640 1 -- 172.21.15.22:0/1398736770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 msgr2=0x55bda596da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f28577fe640 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda596da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f28577fe640 1 -- 172.21.15.22:0/1398736770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 msgr2=0x55bda596b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f28577fe640 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda596b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f28577fe640 1 -- 172.21.15.22:0/1398736770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bda586dbc0 con 0x55bda59621a0 2024-03-09T00:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f2856ffd640 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda596b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f284ffff640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28440a2970 0x7f28440a4e30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28400cc9a0 tx=0x7f284006b2d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f28577fe640 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5962580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f285002c700 tx=0x7f285002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636391322e163ca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f284d7fa640 1 -- 172.21.15.22:0/1398736770 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850039040 con 0x55bda59621a0 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f284d7fa640 1 -- 172.21.15.22:0/1398736770 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2850003970 con 0x55bda59621a0 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.770+0000 7f284d7fa640 1 -- 172.21.15.22:0/1398736770 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850004250 con 0x55bda59621a0 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 -- 172.21.15.22:0/1398736770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 msgr2=0x55bda5962580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5962580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f285002c700 tx=0x7f285002e040 comp rx=0 tx=0).stop 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 -- 172.21.15.22:0/1398736770 shutdown_connections 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda596da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda596b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 --2- 172.21.15.22:0/1398736770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5962580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 -- 172.21.15.22:0/1398736770 >> 172.21.15.22:0/1398736770 conn(0x55bda595c8d0 msgr2=0x55bda595ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 -- 172.21.15.22:0/1398736770 shutdown_connections 2024-03-09T00:05:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 -- 172.21.15.22:0/1398736770 wait complete. 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.771+0000 7f28601ca740 1 -- start start 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 1 Processor -- start 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda59621a0 0x55bda5968620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda5962ac0 0x55bda5968b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda59690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda58644a0 con 0x55bda596b6a0 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda58c1f60 con 0x55bda5962ac0 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bda596e3d0 con 0x55bda59621a0 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28601ca740 10 client.0 _ll_register_callbacks cb 0x55bda58bfd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda59690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda5962ac0 0x55bda5968b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda59690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37216/0 (socket says 172.21.15.22:37216) 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28577fe640 1 -- 172.21.15.22:0/1338134243 learned_addr learned my addr 172.21.15.22:0/1338134243 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28577fe640 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda59621a0 0x55bda5968620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f2856ffd640 1 -- 172.21.15.22:0/1338134243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda59621a0 msgr2=0x55bda5968620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f2856ffd640 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda59621a0 0x55bda5968620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f2856ffd640 1 -- 172.21.15.22:0/1338134243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 msgr2=0x55bda59690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f2856ffd640 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda59690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f2856ffd640 1 -- 172.21.15.22:0/1338134243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bda55da3f0 con 0x55bda5962ac0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f28577fe640 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda59690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f2856ffd640 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda5962ac0 0x55bda5968b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f284800d480 tx=0x7f2848009a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9b3bae326d1235 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.772+0000 7f283e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f283e7fc640 1 -- 172.21.15.22:0/1338134243 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284801d030 con 0x55bda5962ac0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2850003970 con 0x55bda5962ac0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f283e7fc640 1 -- 172.21.15.22:0/1338134243 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2848004260 con 0x55bda5962ac0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f283e7fc640 1 -- 172.21.15.22:0/1338134243 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2848004460 con 0x55bda5962ac0 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 10 client.28457 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bda56a47d0 con 0x55bda5962ac0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f283e7fc640 1 -- 172.21.15.22:0/1338134243 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2850003970 con 0x55bda5962ac0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f283e7fc640 1 -- 172.21.15.22:0/1338134243 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f28480035f0 con 0x55bda5962ac0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 10 client.28457 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 10 client.28457 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 20 client.28457 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda597b7f0 0x55bda597dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 4 client.28457 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bda58d3400 con 0x55bda597b7f0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.773+0000 7f28577fe640 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda597b7f0 0x55bda597dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.774+0000 7f28577fe640 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda597b7f0 0x55bda597dbe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c610dd7f623b5f5d server_cookie=eabcaf126824d5c4 in_seq=0 out_seq=0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.774+0000 7f283e7fc640 10 client.28457 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.774+0000 7f283e7fc640 1 -- 172.21.15.22:0/1338134243 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bda58d3400 con 0x55bda597b7f0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.774+0000 7f283e7fc640 10 client.28457 handle_command_reply: tid=0 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.774+0000 7f28601ca740 1 client.28457 shutdown 2024-03-09T00:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda5962ac0 msgr2=0x55bda5968b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda5962ac0 0x55bda5968b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f284800d480 tx=0x7f2848009a80 comp rx=0 tx=0).stop 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 shutdown_connections 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda597b7f0 0x55bda597dbe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda59690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f283e7fc640 0 client.28457 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda5962ac0 0x55bda5968b60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 --2- 172.21.15.22:0/1338134243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda59621a0 0x55bda5968620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 >> 172.21.15.22:0/1338134243 conn(0x55bda595c8d0 msgr2=0x55bda595e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 shutdown_connections 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 1 -- 172.21.15.22:0/1338134243 wait complete. 2024-03-09T00:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.775+0000 7f28601ca740 20 client.28457 trim_cache size 0 max 16384 2024-03-09T00:05:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 Processor -- start 2024-03-09T00:05:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 -- start start 2024-03-09T00:05:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda596ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda5962ac0 0x55bda59658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda5965de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581b2e60 con 0x55bda596b6a0 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581b0a00 con 0x55bda59621a0 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28601ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581aeac0 con 0x55bda5962ac0 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda5965de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda5962ac0 0x55bda59658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda5962ac0 0x55bda59658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38852/0 (socket says 172.21.15.22:38852) 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f28577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda5965de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37226/0 (socket says 172.21.15.22:37226) 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.787+0000 7f2856ffd640 1 -- 172.21.15.22:0/571592855 learned_addr learned my addr 172.21.15.22:0/571592855 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f28577fe640 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda596ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f28577fe640 1 -- 172.21.15.22:0/571592855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda5962ac0 msgr2=0x55bda59658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f28577fe640 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda5962ac0 0x55bda59658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f28577fe640 1 -- 172.21.15.22:0/571592855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 msgr2=0x55bda596ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f28577fe640 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda596ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f28577fe640 1 -- 172.21.15.22:0/571592855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bda59628f0 con 0x55bda596b6a0 2024-03-09T00:05:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f28577fe640 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda5965de0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2850004af0 tx=0x7f2850036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ecd78360c0f7bd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f284cff9640 1 -- 172.21.15.22:0/571592855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850039040 con 0x55bda596b6a0 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f284cff9640 1 -- 172.21.15.22:0/571592855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2850003d60 con 0x55bda596b6a0 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.788+0000 7f284cff9640 1 -- 172.21.15.22:0/571592855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850034a50 con 0x55bda596b6a0 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 -- 172.21.15.22:0/571592855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 msgr2=0x55bda5965de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda5965de0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2850004af0 tx=0x7f2850036040 comp rx=0 tx=0).stop 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 -- 172.21.15.22:0/571592855 shutdown_connections 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda596b6a0 0x55bda5965de0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda5962ac0 0x55bda59658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 --2- 172.21.15.22:0/571592855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda596ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 -- 172.21.15.22:0/571592855 >> 172.21.15.22:0/571592855 conn(0x55bda595c8d0 msgr2=0x55bda5966ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 -- 172.21.15.22:0/571592855 shutdown_connections 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 -- 172.21.15.22:0/571592855 wait complete. 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 -- start start 2024-03-09T00:05:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.789+0000 7f28601ca740 1 Processor -- start 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5971390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda59718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda59d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581af0c0 con 0x55bda5962ac0 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581b03a0 con 0x55bda59621a0 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581af6e0 con 0x55bda596b6a0 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 10 client.0 _ll_register_callbacks cb 0x7f2840005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda59718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda59718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37240/0 (socket says 172.21.15.22:37240) 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 -- 172.21.15.22:0/3984534073 learned_addr learned my addr 172.21.15.22:0/3984534073 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28577fe640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5971390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28577fe640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda59d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 -- 172.21.15.22:0/3984534073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 msgr2=0x55bda59d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda59d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 -- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 msgr2=0x55bda5971390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5971390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 -- 172.21.15.22:0/3984534073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bda59721c0 con 0x55bda5962ac0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28577fe640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5971390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f2856ffd640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda59718d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2848003ae0 tx=0x7f284800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486ebfc01c23fe65 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f283e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f283e7fc640 1 -- 172.21.15.22:0/3984534073 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848030070 con 0x55bda5962ac0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f283e7fc640 1 -- 172.21.15.22:0/3984534073 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f2848003e50 con 0x55bda5962ac0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2850003d60 con 0x55bda5962ac0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f283e7fc640 1 -- 172.21.15.22:0/3984534073 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f284803b050 con 0x55bda5962ac0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 10 client.19143 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.790+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55bda59d3340 con 0x55bda5962ac0 2024-03-09T00:05:50.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f283e7fc640 1 -- 172.21.15.22:0/3984534073 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2850003d60 con 0x55bda5962ac0 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f283e7fc640 1 -- 172.21.15.22:0/3984534073 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2848004ac0 con 0x55bda5962ac0 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28601ca740 10 client.19143 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28601ca740 10 client.19143 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28601ca740 20 client.19143 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28601ca740 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda59d3a10 0x55bda59d5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28601ca740 4 client.19143 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bda58cd750 con 0x55bda59d3a10 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28577fe640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda59d3a10 0x55bda59d5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f283e7fc640 10 client.19143 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.791+0000 7f28577fe640 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda59d3a10 0x55bda59d5e30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12ec73ecf7ac1e92 server_cookie=35a27bf7a668365b in_seq=0 out_seq=0 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f283e7fc640 1 -- 172.21.15.22:0/3984534073 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bda58cd750 con 0x55bda59d3a10 2024-03-09T00:05:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f283e7fc640 10 client.19143 handle_command_reply: tid=0 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 client.19143 shutdown 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 msgr2=0x55bda59718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda59718d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2848003ae0 tx=0x7f284800d900 comp rx=0 tx=0).stop 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 shutdown_connections 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55bda59d3a10 0x55bda59d5e30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bda596b6a0 0x55bda59d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f283e7fc640 0 client.19143 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:05:50.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bda5962ac0 0x55bda59718d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 --2- 172.21.15.22:0/3984534073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bda59621a0 0x55bda5971390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 >> 172.21.15.22:0/3984534073 conn(0x55bda595c8d0 msgr2=0x55bda595b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 shutdown_connections 2024-03-09T00:05:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 1 -- 172.21.15.22:0/3984534073 wait complete. 2024-03-09T00:05:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f28601ca740 20 client.19143 trim_cache size 0 max 16384 2024-03-09T00:05:50.795 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:05:50.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28440a2970 msgr2=0x7f28440a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28440a2970 0x7f28440a4e30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28400cc9a0 tx=0x7f284006b2d0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 msgr2=0x7f285819ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285819ffa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f284000e9c0 tx=0x7f284000ee80 comp rx=0 tx=0).stop 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 shutdown_connections 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28440a2970 0x7f28440a4e30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2858199b70 0x7f2858199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28581054b0 0x7f285819ffa0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 --2- 172.21.15.22:0/187463481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28581003e0 0x7f285819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 >> 172.21.15.22:0/187463481 conn(0x7f28580773a0 msgr2=0x7f28580fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.792+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 shutdown_connections 2024-03-09T00:05:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.793+0000 7f285d509640 1 -- 172.21.15.22:0/187463481 wait complete. 2024-03-09T00:05:50.799 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:05:50.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:05:50.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 -- 172.21.15.22:0/521172026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a4100d90 msgr2=0x7f93a4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/521172026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a4100d90 0x7f93a4104dd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f9398009900 tx=0x7f939802bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:50.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 -- 172.21.15.22:0/521172026 shutdown_connections 2024-03-09T00:05:50.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/521172026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a41054b0 0x7f93a410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/521172026 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a4100d90 0x7f93a4104dd0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/521172026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a41003e0 0x7f93a41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 -- 172.21.15.22:0/521172026 >> 172.21.15.22:0/521172026 conn(0x7f93a40773a0 msgr2=0x7f93a40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 -- 172.21.15.22:0/521172026 shutdown_connections 2024-03-09T00:05:50.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.917+0000 7f93aa7b2640 1 -- 172.21.15.22:0/521172026 wait complete. 2024-03-09T00:05:50.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 Processor -- start 2024-03-09T00:05:50.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 -- start start 2024-03-09T00:05:50.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a41003e0 0x7f93a41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a41054b0 0x7f93a4198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a4199440 0x7f93a41a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4110520 con 0x7f93a41003e0 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41103a0 con 0x7f93a41054b0 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.918+0000 7f93aa7b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41106a0 con 0x7f93a4199440 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a41003e0 0x7f93a41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a41003e0 0x7f93a41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37260/0 (socket says 172.21.15.22:37260) 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 -- 172.21.15.22:0/4013996795 learned_addr learned my addr 172.21.15.22:0/4013996795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a41054b0 0x7f93a4198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a4199440 0x7f93a41a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 -- 172.21.15.22:0/4013996795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a4199440 msgr2=0x7f93a41a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a4199440 0x7f93a41a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 -- 172.21.15.22:0/4013996795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a41054b0 msgr2=0x7f93a4198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a41054b0 0x7f93a4198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 -- 172.21.15.22:0/4013996795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a41a41c0 con 0x7f93a41003e0 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a37fe640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a41054b0 0x7f93a4198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a3fff640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a41003e0 0x7f93a41989c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f939400e9f0 tx=0x7f939400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a17fa640 1 -- 172.21.15.22:0/4013996795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400cd80 con 0x7f93a41003e0 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a17fa640 1 -- 172.21.15.22:0/4013996795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9394010910 con 0x7f93a41003e0 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a4196810 con 0x7f93a41003e0 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93a17fa640 1 -- 172.21.15.22:0/4013996795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940188f0 con 0x7f93a41003e0 2024-03-09T00:05:50.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.919+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a4196d30 con 0x7f93a41003e0 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.921+0000 7f93a17fa640 1 -- 172.21.15.22:0/4013996795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9394018ad0 con 0x7f93a41003e0 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 Processor -- start 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 -- start start 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df37508580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df37508ac0 0x55df37511160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df375116a0 0x55df37513a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df374cf360 con 0x55df37508ac0 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3746bec0 con 0x55df375081a0 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93ad391740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37414d30 con 0x55df375116a0 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a17fa640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f938c0a2970 0x7f938c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a8d4f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df37508580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df37508ac0 0x55df37511160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a8d4f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df37508580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:49354/0 (socket says 172.21.15.22:49354) 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a8d4f640 1 -- 172.21.15.22:0/3889383073 learned_addr learned my addr 172.21.15.22:0/3889383073 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a17fa640 1 -- 172.21.15.22:0/4013996795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f93940cf620 con 0x7f93a41003e0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a37fe640 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df375116a0 0x55df37513a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f938c0a2970 0x7f938c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a8d4f640 1 -- 172.21.15.22:0/3889383073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df375116a0 msgr2=0x55df37513a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.922+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df375116a0 0x55df37513a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f93a8d4f640 1 -- 172.21.15.22:0/3889383073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df37508ac0 msgr2=0x55df37511160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df37508ac0 0x55df37511160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f93a8d4f640 1 -- 172.21.15.22:0/3889383073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df37413bc0 con 0x55df375081a0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f938c0a2970 0x7f938c0a4e30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f939c00fc90 tx=0x7f939c018000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df37508580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f939c005f40 tx=0x7f939c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0e94dfbe66679a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f938a7fc640 1 -- 172.21.15.22:0/3889383073 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c022070 con 0x55df375081a0 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f938a7fc640 1 -- 172.21.15.22:0/3889383073 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f939c00ec50 con 0x55df375081a0 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.923+0000 7f938a7fc640 1 -- 172.21.15.22:0/3889383073 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c004050 con 0x55df375081a0 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 -- 172.21.15.22:0/3889383073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 msgr2=0x55df37508580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df37508580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f939c005f40 tx=0x7f939c00f790 comp rx=0 tx=0).stop 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 -- 172.21.15.22:0/3889383073 shutdown_connections 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df375116a0 0x55df37513a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df37508ac0 0x55df37511160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 --2- 172.21.15.22:0/3889383073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df37508580 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 -- 172.21.15.22:0/3889383073 >> 172.21.15.22:0/3889383073 conn(0x55df375028d0 msgr2=0x55df37502cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 -- 172.21.15.22:0/3889383073 shutdown_connections 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 -- 172.21.15.22:0/3889383073 wait complete. 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.924+0000 7f93ad391740 1 -- start start 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 1 Processor -- start 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df3750e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375116a0 0x55df3750f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3740a4a0 con 0x55df375116a0 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37467f60 con 0x55df375081a0 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df37514430 con 0x55df37508ac0 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93ad391740 10 client.0 _ll_register_callbacks cb 0x55df37465d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375116a0 0x55df3750f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375116a0 0x55df3750f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37274/0 (socket says 172.21.15.22:37274) 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 -- 172.21.15.22:0/2251324649 learned_addr learned my addr 172.21.15.22:0/2251324649 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a37fe640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a3fff640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df3750e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 -- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 msgr2=0x55df3750ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 -- 172.21.15.22:0/2251324649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 msgr2=0x55df3750e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df3750e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 -- 172.21.15.22:0/2251324649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df371803f0 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a3fff640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df3750e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375116a0 0x55df3750f160 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f939c006010 tx=0x7f939c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c28364257646751 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.925+0000 7f93837fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93837fe640 1 -- 172.21.15.22:0/2251324649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c033060 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f939c00ec50 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93837fe640 1 -- 172.21.15.22:0/2251324649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f939c009dd0 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93837fe640 1 -- 172.21.15.22:0/2251324649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f939c03a050 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 10 client.19155 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55df3724a7d0 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93837fe640 1 -- 172.21.15.22:0/2251324649 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f939c00ec50 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93837fe640 1 -- 172.21.15.22:0/2251324649 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f939c026a70 con 0x55df375116a0 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 10 client.19155 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 10 client.19155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:50.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 20 client.19155 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df375218b0 0x55df37523ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 4 client.19155 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df37479400 con 0x55df375218b0 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.926+0000 7f93a3fff640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df375218b0 0x55df37523ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.927+0000 7f93837fe640 10 client.19155 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.927+0000 7f93a3fff640 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df375218b0 0x55df37523ca0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33bbf496227e2db9 server_cookie=ddca7885b836dcba in_seq=0 out_seq=0 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.927+0000 7f93837fe640 1 -- 172.21.15.22:0/2251324649 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df37479400 con 0x55df375218b0 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93837fe640 10 client.19155 handle_command_reply: tid=0 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 client.19155 shutdown 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375116a0 msgr2=0x55df3750f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375116a0 0x55df3750f160 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f939c006010 tx=0x7f939c005730 comp rx=0 tx=0).stop 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 shutdown_connections 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df375218b0 0x55df37523ca0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375116a0 0x55df3750f160 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93837fe640 0 client.19155 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 --2- 172.21.15.22:0/2251324649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375081a0 0x55df3750e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 >> 172.21.15.22:0/2251324649 conn(0x55df375028d0 msgr2=0x55df37504620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 shutdown_connections 2024-03-09T00:05:50.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 1 -- 172.21.15.22:0/2251324649 wait complete. 2024-03-09T00:05:50.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.928+0000 7f93ad391740 20 client.19155 trim_cache size 0 max 16384 2024-03-09T00:05:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.940+0000 7f93ad391740 1 Processor -- start 2024-03-09T00:05:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.940+0000 7f93ad391740 1 -- start start 2024-03-09T00:05:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93ad391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37510e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93ad391740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93ad391740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df3750b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93ad391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41b2d70 con 0x55df375081a0 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93ad391740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41b0910 con 0x55df375116a0 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93ad391740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41ae9d0 con 0x55df37508ac0 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37510e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38930/0 (socket says 172.21.15.22:38930) 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 -- 172.21.15.22:0/3138776158 learned_addr learned my addr 172.21.15.22:0/3138776158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37510e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37288/0 (socket says 172.21.15.22:37288) 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df3750b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 -- 172.21.15.22:0/3138776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 msgr2=0x55df3750b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df3750b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 -- 172.21.15.22:0/3138776158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 msgr2=0x55df37510e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37510e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 -- 172.21.15.22:0/3138776158 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df375088f0 con 0x55df37508ac0 2024-03-09T00:05:50.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a3fff640 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37510e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:50.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.941+0000 7f93a37fe640 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750d6c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9398009900 tx=0x7f9398005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b748d1a9420b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f9389ffb640 1 -- 172.21.15.22:0/3138776158 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398004a70 con 0x55df37508ac0 2024-03-09T00:05:50.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f9389ffb640 1 -- 172.21.15.22:0/3138776158 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9398004c10 con 0x55df37508ac0 2024-03-09T00:05:50.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f9389ffb640 1 -- 172.21.15.22:0/3138776158 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93980052f0 con 0x55df37508ac0 2024-03-09T00:05:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 -- 172.21.15.22:0/3138776158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 msgr2=0x55df3750d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750d6c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9398009900 tx=0x7f9398005fd0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 -- 172.21.15.22:0/3138776158 shutdown_connections 2024-03-09T00:05:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df3750b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df3750d6c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 --2- 172.21.15.22:0/3138776158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37510e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 -- 172.21.15.22:0/3138776158 >> 172.21.15.22:0/3138776158 conn(0x55df375028d0 msgr2=0x55df3750cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 -- 172.21.15.22:0/3138776158 shutdown_connections 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.942+0000 7f93ad391740 1 -- 172.21.15.22:0/3138776158 wait complete. 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 -- start start 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 Processor -- start 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37517340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df37517880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df37517dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41aefd0 con 0x55df375081a0 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41b02b0 con 0x55df375116a0 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41af5f0 con 0x55df37508ac0 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93ad391740 10 client.0 _ll_register_callbacks cb 0x7f939c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37517340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37517340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37298/0 (socket says 172.21.15.22:37298) 2024-03-09T00:05:50.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 -- 172.21.15.22:0/1925997672 learned_addr learned my addr 172.21.15.22:0/1925997672 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a8d4f640 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df37517dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a37fe640 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df37517880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 -- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 msgr2=0x55df37517880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df37517880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 -- 172.21.15.22:0/1925997672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 msgr2=0x55df37517dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df37517dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.943+0000 7f93a3fff640 1 -- 172.21.15.22:0/1925997672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df37577ea0 con 0x55df375081a0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93a3fff640 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37517340 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9394097d90 tx=0x7f939401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e1cb845c6c1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93837fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93837fe640 1 -- 172.21.15.22:0/1925997672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940a4020 con 0x55df375081a0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9398004c10 con 0x55df375081a0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93837fe640 1 -- 172.21.15.22:0/1925997672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9394010ac0 con 0x55df375081a0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93837fe640 1 -- 172.21.15.22:0/1925997672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9394097fa0 con 0x55df375081a0 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 10 client.19161 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:50.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55df375791e0 con 0x55df375081a0 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93837fe640 1 -- 172.21.15.22:0/1925997672 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9398004c10 con 0x55df375081a0 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93837fe640 1 -- 172.21.15.22:0/1925997672 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93940a6030 con 0x55df375081a0 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 10 client.19161 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 10 client.19161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 20 client.19161 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df37579860 0x55df3757bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 4 client.19161 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df37473750 con 0x55df37579860 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.944+0000 7f93a37fe640 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df37579860 0x55df3757bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.945+0000 7f93837fe640 10 client.19161 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.945+0000 7f93a37fe640 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df37579860 0x55df3757bc50 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a592f0750d8817d9 server_cookie=d8d8ee5330ad0d8f in_seq=0 out_seq=0 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.945+0000 7f93837fe640 1 -- 172.21.15.22:0/1925997672 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df37473750 con 0x55df37579860 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.945+0000 7f93837fe640 10 client.19161 handle_command_reply: tid=0 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.945+0000 7f93ad391740 1 client.19161 shutdown 2024-03-09T00:05:50.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 msgr2=0x55df37517340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37517340 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9394097d90 tx=0x7f939401a040 comp rx=0 tx=0).stop 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 shutdown_connections 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df37579860 0x55df3757bc50 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93837fe640 0 client.19161 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df375116a0 0x55df37517dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df37508ac0 0x55df37517880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 --2- 172.21.15.22:0/1925997672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df375081a0 0x55df37517340 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 >> 172.21.15.22:0/1925997672 conn(0x55df375028d0 msgr2=0x55df3750d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 shutdown_connections 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 1 -- 172.21.15.22:0/1925997672 wait complete. 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93ad391740 20 client.19161 trim_cache size 0 max 16384 2024-03-09T00:05:50.946 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:05:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f938c0a2970 msgr2=0x7f938c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.946+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f938c0a2970 0x7f938c0a4e30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f939c00fc90 tx=0x7f939c018000 comp rx=0 tx=0).stop 2024-03-09T00:05:50.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a41003e0 msgr2=0x7f93a41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a41003e0 0x7f93a41989c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f939400e9f0 tx=0x7f939400eeb0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 shutdown_connections 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f938c0a2970 0x7f938c0a4e30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a4199440 0x7f93a41a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a41054b0 0x7f93a4198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 --2- 172.21.15.22:0/4013996795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93a41003e0 0x7f93a41989c0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 >> 172.21.15.22:0/4013996795 conn(0x7f93a40773a0 msgr2=0x7f93a40fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 shutdown_connections 2024-03-09T00:05:50.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:50.947+0000 7f93aa7b2640 1 -- 172.21.15.22:0/4013996795 wait complete. 2024-03-09T00:05:51.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:50 smithi022 ceph-mon[31894]: pgmap v355: 129 pgs: 129 active+clean; 91 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2024-03-09T00:05:51.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:50 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1904072868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:05:51.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:50 smithi062 ceph-mon[36129]: pgmap v355: 129 pgs: 129 active+clean; 91 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2024-03-09T00:05:51.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:50 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1904072868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:05:51.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:50 smithi192 ceph-mon[35995]: pgmap v355: 129 pgs: 129 active+clean; 91 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2024-03-09T00:05:51.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:50 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1904072868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:05:51.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1141603 9.64 MB/sec execute 479 sec latency 37.257 ms 2024-03-09T00:05:51.955 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:05:51.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:05:51.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.075+0000 7f7c1c620640 1 -- 172.21.15.22:0/357064708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141065d0 msgr2=0x7f7c1410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.075+0000 7f7c1c620640 1 --2- 172.21.15.22:0/357064708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141065d0 0x7f7c1410e1a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009ca0 tx=0x7f7c1002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- 172.21.15.22:0/357064708 shutdown_connections 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 --2- 172.21.15.22:0/357064708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141065d0 0x7f7c1410e1a0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 --2- 172.21.15.22:0/357064708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14101eb0 0x7f7c14105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 --2- 172.21.15.22:0/357064708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14101500 0x7f7c141018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- 172.21.15.22:0/357064708 >> 172.21.15.22:0/357064708 conn(0x7f7c140fcd20 msgr2=0x7f7c140ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- 172.21.15.22:0/357064708 shutdown_connections 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- 172.21.15.22:0/357064708 wait complete. 2024-03-09T00:05:52.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 Processor -- start 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- start start 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14101500 0x7f7c1419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c141065d0 0x7f7c1419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141979f0 0x7f7c14197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c14111690 con 0x7f7c141979f0 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c14111510 con 0x7f7c14101500 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1c620640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c14111810 con 0x7f7c141065d0 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.076+0000 7f7c1abbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141979f0 0x7f7c14197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141979f0 0x7f7c14197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37314/0 (socket says 172.21.15.22:37314) 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1a3bc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14101500 0x7f7c1419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2002244091 learned_addr learned my addr 172.21.15.22:0/2002244091 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c141065d0 0x7f7c1419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2002244091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c141065d0 msgr2=0x7f7c1419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c141065d0 0x7f7c1419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2002244091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14101500 msgr2=0x7f7c1419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14101500 0x7f7c1419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2002244091 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c14198510 con 0x7f7c141979f0 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141979f0 0x7f7c14197ea0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7c1002c700 tx=0x7f7c10032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c0b7fe640 1 -- 172.21.15.22:0/2002244091 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10041020 con 0x7f7c141979f0 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c141987a0 con 0x7f7c141979f0 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c0b7fe640 1 -- 172.21.15.22:0/2002244091 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c10034cd0 con 0x7f7c141979f0 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c141a49a0 con 0x7f7c141979f0 2024-03-09T00:05:52.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.077+0000 7f7c0b7fe640 1 -- 172.21.15.22:0/2002244091 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10039040 con 0x7f7c141979f0 2024-03-09T00:05:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.080+0000 7f7c0b7fe640 1 -- 172.21.15.22:0/2002244091 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7c10044020 con 0x7f7c141979f0 2024-03-09T00:05:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.080+0000 7f7c1f202740 1 Processor -- start 2024-03-09T00:05:52.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1f202740 1 -- start start 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c0b7fe640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7c000a29f0 0x7f7c000a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bb9d80 0x55fa82bba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7c000a29f0 0x7f7c000a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1f202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa82b1db40 con 0x55fa82bb9d80 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c0b7fe640 1 -- 172.21.15.22:0/2002244091 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7c100f0e30 con 0x7f7c141979f0 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bb9d80 0x55fa82bba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c19bbb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bb9d80 0x55fa82bba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37320/0 (socket says 172.21.15.22:37320) 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 -- 172.21.15.22:0/1511363100 learned_addr learned my addr 172.21.15.22:0/1511363100 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1f202740 1 -- 172.21.15.22:0/1511363100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa82ac6d80 con 0x55fa82bc3280 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1f202740 1 -- 172.21.15.22:0/1511363100 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa82b19be0 con 0x55fa82bba6a0 2024-03-09T00:05:52.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 -- 172.21.15.22:0/1511363100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bba6a0 msgr2=0x55fa82bc2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 -- 172.21.15.22:0/1511363100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 msgr2=0x55fa82bc5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1a3bc640 1 -- 172.21.15.22:0/1511363100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa828fa7e0 con 0x55fa82bb9d80 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.081+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.082+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7c000a29f0 0x7f7c000a4eb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b620 tx=0x7f7c0400ba70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.082+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bb9d80 0x55fa82bba160 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ca0 tx=0x7f7c0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779c4bb10deb3719 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.082+0000 7f7c08bfe640 1 -- 172.21.15.22:0/1511363100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c009e50 con 0x55fa82bb9d80 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.082+0000 7f7c08bfe640 1 -- 172.21.15.22:0/1511363100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c005500 con 0x55fa82bb9d80 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.082+0000 7f7c08bfe640 1 -- 172.21.15.22:0/1511363100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c019670 con 0x55fa82bb9d80 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- 172.21.15.22:0/1511363100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bb9d80 msgr2=0x55fa82bba160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bb9d80 0x55fa82bba160 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ca0 tx=0x7f7c0c011810 comp rx=0 tx=0).stop 2024-03-09T00:05:52.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- 172.21.15.22:0/1511363100 shutdown_connections 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 --2- 172.21.15.22:0/1511363100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bb9d80 0x55fa82bba160 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- 172.21.15.22:0/1511363100 >> 172.21.15.22:0/1511363100 conn(0x55fa82bb44b0 msgr2=0x55fa82bb4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- 172.21.15.22:0/1511363100 shutdown_connections 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- 172.21.15.22:0/1511363100 wait complete. 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- start start 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 Processor -- start 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bc2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa82bc6070 con 0x55fa82bba6a0 2024-03-09T00:05:52.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa82ada1c0 con 0x55fa82bc3280 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa82bc61f0 con 0x55fa82bb9d80 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1f202740 10 client.0 _ll_register_callbacks cb 0x55fa82ac97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c19bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1a3bc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bc2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c19bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37328/0 (socket says 172.21.15.22:37328) 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c1a3bc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bc2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38982/0 (socket says 172.21.15.22:38982) 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.083+0000 7f7c19bbb640 1 -- 172.21.15.22:0/3330123754 learned_addr learned my addr 172.21.15.22:0/3330123754 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c19bbb640 1 -- 172.21.15.22:0/3330123754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 msgr2=0x55fa82bc2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bc2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c19bbb640 1 -- 172.21.15.22:0/3330123754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 msgr2=0x55fa82bc0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c19bbb640 1 -- 172.21.15.22:0/3330123754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa8295af70 con 0x55fa82bba6a0 2024-03-09T00:05:52.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc0030 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7c04010b70 tx=0x7f7c04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f977b6cc5f294ea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7bfdffb640 1 -- 172.21.15.22:0/3330123754 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04027070 con 0x55fa82bba6a0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7bfdffb640 1 -- 172.21.15.22:0/3330123754 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7c0401ed00 con 0x55fa82bba6a0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c0c005500 con 0x55fa82bba6a0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7bfdffb640 1 -- 172.21.15.22:0/3330123754 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c04033050 con 0x55fa82bba6a0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c1f202740 10 client.19173 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fa827f2170 con 0x55fa82bba6a0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.084+0000 7f7bfdffb640 1 -- 172.21.15.22:0/3330123754 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c005500 con 0x55fa82bba6a0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7bfdffb640 1 -- 172.21.15.22:0/3330123754 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c0401f050 con 0x55fa82bba6a0 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1f202740 10 client.19173 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1f202740 10 client.19173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1f202740 20 client.19173 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:52.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82bd3330 0x55fa82bd5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1f202740 4 client.19173 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa82af3960 con 0x55fa82bd3330 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82bd3330 0x55fa82bd5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82bd3330 0x55fa82bd5750 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf7978b72208cf7 server_cookie=c522dcb9d30a14d2 in_seq=0 out_seq=0 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.085+0000 7f7bfdffb640 10 client.19173 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7bfdffb640 1 -- 172.21.15.22:0/3330123754 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa82af3960 con 0x55fa82bd3330 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7bfdffb640 10 client.19173 handle_command_reply: tid=0 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 client.19173 shutdown 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 msgr2=0x55fa82bc0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc0030 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7c04010b70 tx=0x7f7c04004730 comp rx=0 tx=0).stop 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 shutdown_connections 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82bd3330 0x55fa82bd5750 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bc3280 0x55fa82bc0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7bfdffb640 0 client.19173 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc0030 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3330123754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bc2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 >> 172.21.15.22:0/3330123754 conn(0x55fa82bb44b0 msgr2=0x55fa82bb6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 shutdown_connections 2024-03-09T00:05:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.086+0000 7f7c1f202740 1 -- 172.21.15.22:0/3330123754 wait complete. 2024-03-09T00:05:52.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.087+0000 7f7c1f202740 20 client.19173 trim_cache size 0 max 16384 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 Processor -- start 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 -- start start 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bb9d80 0x55fa82bc55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bc3280 0x55fa82bc2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c141b0b60 con 0x55fa82bba6a0 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c141ae700 con 0x55fa82bb9d80 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1f202740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c141ac7c0 con 0x55fa82bc3280 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c19bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1a3bc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bb9d80 0x55fa82bc55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c19bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37334/0 (socket says 172.21.15.22:37334) 2024-03-09T00:05:52.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c19bbb640 1 -- 172.21.15.22:0/3343356397 learned_addr learned my addr 172.21.15.22:0/3343356397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.103+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bc3280 0x55fa82bc2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7c1a3bc640 1 -- 172.21.15.22:0/3343356397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bc3280 msgr2=0x55fa82bc2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bc3280 0x55fa82bc2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7c1a3bc640 1 -- 172.21.15.22:0/3343356397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 msgr2=0x55fa82bc2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7c1a3bc640 1 -- 172.21.15.22:0/3343356397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa82bc09b0 con 0x55fa82bb9d80 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7c19bbb640 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bb9d80 0x55fa82bc55c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c005890 tx=0x7f7c0c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88b5e36817f44bd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7bfffff640 1 -- 172.21.15.22:0/3343356397 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0047c0 con 0x55fa82bb9d80 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7bfffff640 1 -- 172.21.15.22:0/3343356397 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c02c070 con 0x55fa82bb9d80 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.104+0000 7f7bfffff640 1 -- 172.21.15.22:0/3343356397 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0282e0 con 0x55fa82bb9d80 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 -- 172.21.15.22:0/3343356397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bb9d80 msgr2=0x55fa82bc55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bb9d80 0x55fa82bc55c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c005890 tx=0x7f7c0c011810 comp rx=0 tx=0).stop 2024-03-09T00:05:52.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 -- 172.21.15.22:0/3343356397 shutdown_connections 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bc3280 0x55fa82bc2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bba6a0 0x55fa82bc2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 --2- 172.21.15.22:0/3343356397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bb9d80 0x55fa82bc55c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 -- 172.21.15.22:0/3343356397 >> 172.21.15.22:0/3343356397 conn(0x55fa82bb44b0 msgr2=0x55fa82bbb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 -- 172.21.15.22:0/3343356397 shutdown_connections 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 -- 172.21.15.22:0/3343356397 wait complete. 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 -- start start 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 Processor -- start 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.105+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bba6a0 0x55fa82bca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bc3280 0x55fa82bc7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c141acdc0 con 0x55fa82bc3280 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c141ae0a0 con 0x55fa82bba6a0 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c141ad3e0 con 0x55fa82bb9d80 2024-03-09T00:05:52.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 10 client.0 _ll_register_callbacks cb 0x7f7c0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bc3280 0x55fa82bc7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c19bbb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bba6a0 0x55fa82bca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1a3bc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bc3280 0x55fa82bc7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37350/0 (socket says 172.21.15.22:37350) 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2649845771 learned_addr learned my addr 172.21.15.22:0/2649845771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2649845771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 msgr2=0x55fa82bca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2649845771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bba6a0 msgr2=0x55fa82bca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bba6a0 0x55fa82bca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 -- 172.21.15.22:0/2649845771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa82bc8170 con 0x55fa82bc3280 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1abbd640 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bc3280 0x55fa82bc7970 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7c100bd130 tx=0x7f7c100bcb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb447fa20f46c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7bfdffb640 1 -- 172.21.15.22:0/2649845771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c100bd1a0 con 0x55fa82bc3280 2024-03-09T00:05:52.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7bfdffb640 1 -- 172.21.15.22:0/2649845771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7c10003970 con 0x55fa82bc3280 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c0c02c070 con 0x55fa82bc3280 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7bfdffb640 1 -- 172.21.15.22:0/2649845771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c100c12f0 con 0x55fa82bc3280 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 10 client.19179 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.106+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fa82c2b4f0 con 0x55fa82bc3280 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7bfdffb640 1 -- 172.21.15.22:0/2649845771 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c02c070 con 0x55fa82bc3280 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7bfdffb640 1 -- 172.21.15.22:0/2649845771 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c1003c030 con 0x55fa82bc3280 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1f202740 10 client.19179 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1f202740 10 client.19179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1f202740 20 client.19179 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1f202740 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82c2bc10 0x55fa82c2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1f202740 4 client.19179 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fa82adf080 con 0x55fa82c2bc10 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82c2bc10 0x55fa82c2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7bfdffb640 10 client.19179 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.107+0000 7f7c1a3bc640 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82c2bc10 0x55fa82c2e030 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663752eadb6da860 server_cookie=17ffa316405a0246 in_seq=0 out_seq=0 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7bfdffb640 1 -- 172.21.15.22:0/2649845771 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fa82adf080 con 0x55fa82c2bc10 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7bfdffb640 10 client.19179 handle_command_reply: tid=0 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 client.19179 shutdown 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bc3280 msgr2=0x55fa82bc7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bc3280 0x55fa82bc7970 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7c100bd130 tx=0x7f7c100bcb00 comp rx=0 tx=0).stop 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 shutdown_connections 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fa82c2bc10 0x55fa82c2e030 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa82bc3280 0x55fa82bc7970 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa82bba6a0 0x55fa82bca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 --2- 172.21.15.22:0/2649845771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa82bb9d80 0x55fa82bca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7bfdffb640 0 client.19179 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 >> 172.21.15.22:0/2649845771 conn(0x55fa82bb44b0 msgr2=0x55fa82bb2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 shutdown_connections 2024-03-09T00:05:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 1 -- 172.21.15.22:0/2649845771 wait complete. 2024-03-09T00:05:52.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.109+0000 7f7c1f202740 20 client.19179 trim_cache size 0 max 16384 2024-03-09T00:05:52.110 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:05:52.110 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:05:52.110 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "0d053e49-98a3-46c1-ae41-d16c0cd4fc9d", 2024-03-09T00:05:52.110 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:05:52.110 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:05:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7c000a29f0 msgr2=0x7f7c000a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7c000a29f0 0x7f7c000a4eb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b620 tx=0x7f7c0400ba70 comp rx=0 tx=0).stop 2024-03-09T00:05:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141979f0 msgr2=0x7f7c14197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141979f0 0x7f7c14197ea0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7c1002c700 tx=0x7f7c10032f20 comp rx=0 tx=0).stop 2024-03-09T00:05:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 shutdown_connections 2024-03-09T00:05:52.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7c000a29f0 0x7f7c000a4eb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c141979f0 0x7f7c14197ea0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c141065d0 0x7f7c1419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 --2- 172.21.15.22:0/2002244091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c14101500 0x7f7c1419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 >> 172.21.15.22:0/2002244091 conn(0x7f7c140fcd20 msgr2=0x7f7c140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 shutdown_connections 2024-03-09T00:05:52.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.110+0000 7f7c1c620640 1 -- 172.21.15.22:0/2002244091 wait complete. 2024-03-09T00:05:52.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:05:52.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 -- 172.21.15.22:0/702169763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f201022a0 msgr2=0x7f7f200ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 --2- 172.21.15.22:0/702169763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f201022a0 0x7f7f200ff7f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009900 tx=0x7f7f1002bc30 comp rx=0 tx=0).stop 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 -- 172.21.15.22:0/702169763 shutdown_connections 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 --2- 172.21.15.22:0/702169763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f2010df70 0x7f7f20110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 --2- 172.21.15.22:0/702169763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f20102710 0x7f7f200ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 --2- 172.21.15.22:0/702169763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f201022a0 0x7f7f200ff7f0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 -- 172.21.15.22:0/702169763 >> 172.21.15.22:0/702169763 conn(0x7f7f200767b0 msgr2=0x7f7f200fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 -- 172.21.15.22:0/702169763 shutdown_connections 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 -- 172.21.15.22:0/702169763 wait complete. 2024-03-09T00:05:52.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.233+0000 7f7f24aab640 1 Processor -- start 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f24aab640 1 -- start start 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f24aab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f201022a0 0x7f7f20199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f24aab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f20102710 0x7f7f2019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f24aab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f2010df70 0x7f7f2019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f24aab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f20113940 con 0x7f7f2010df70 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f24aab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201137c0 con 0x7f7f201022a0 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f24aab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f20113ac0 con 0x7f7f20102710 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f2010df70 0x7f7f2019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f2010df70 0x7f7f2019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37358/0 (socket says 172.21.15.22:37358) 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 -- 172.21.15.22:0/2591534356 learned_addr learned my addr 172.21.15.22:0/2591534356 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f20102710 0x7f7f2019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f201022a0 0x7f7f20199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 -- 172.21.15.22:0/2591534356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f20102710 msgr2=0x7f7f2019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f20102710 0x7f7f2019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 -- 172.21.15.22:0/2591534356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f201022a0 msgr2=0x7f7f20199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f201022a0 0x7f7f20199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 -- 172.21.15.22:0/2591534356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2019edd0 con 0x7f7f2010df70 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f20102710 0x7f7f2019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.234+0000 7f7f1ed9d640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f2010df70 0x7f7f2019e6a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800eb10 tx=0x7f7f1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.235+0000 7f7f0f7fe640 1 -- 172.21.15.22:0/2591534356 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18004b20 con 0x7f7f2010df70 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.235+0000 7f7f0f7fe640 1 -- 172.21.15.22:0/2591534356 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f18004cc0 con 0x7f7f2010df70 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.235+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2019f0c0 con 0x7f7f2010df70 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.235+0000 7f7f0f7fe640 1 -- 172.21.15.22:0/2591534356 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18010790 con 0x7f7f2010df70 2024-03-09T00:05:52.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.235+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f201a6950 con 0x7f7f2010df70 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.237+0000 7f7f0f7fe640 1 -- 172.21.15.22:0/2591534356 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7f18005020 con 0x7f7f2010df70 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.237+0000 7f7f2776e740 1 Processor -- start 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.237+0000 7f7f2776e740 1 -- start start 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f00ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f013720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f016050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f2776e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55859ef6de00 con 0x55859f00a760 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f2776e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55859ef16b10 con 0x55859f00b080 2024-03-09T00:05:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f2776e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55859ef69ea0 con 0x55859f013c60 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f00ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f00ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37370/0 (socket says 172.21.15.22:37370) 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/1533791451 learned_addr learned my addr 172.21.15.22:0/1533791451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f0f7fe640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7f080a2970 0x7f7f080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f013720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f016050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1ed9d640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7f080a2970 0x7f7f080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/1533791451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 msgr2=0x55859f016050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f016050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/1533791451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 msgr2=0x55859f013720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f013720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/1533791451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55859ed4c7d0 con 0x55859f00a760 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.238+0000 7f7f0f7fe640 1 -- 172.21.15.22:0/2591534356 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7f180cf9e0 con 0x7f7f2010df70 2024-03-09T00:05:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.239+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f00ab40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7f14013b60 tx=0x7f7f14009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d2af4f409cb071 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.239+0000 7f7f1ed9d640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7f080a2970 0x7f7f080a4e30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7f180980a0 tx=0x7f7f180977c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.239+0000 7f7f0cff9640 1 -- 172.21.15.22:0/1533791451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14011d30 con 0x55859f00a760 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.239+0000 7f7f0cff9640 1 -- 172.21.15.22:0/1533791451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f14005500 con 0x55859f00a760 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.239+0000 7f7f0cff9640 1 -- 172.21.15.22:0/1533791451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400f3c0 con 0x55859f00a760 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- 172.21.15.22:0/1533791451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 msgr2=0x55859f00ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f00ab40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7f14013b60 tx=0x7f7f14009d40 comp rx=0 tx=0).stop 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- 172.21.15.22:0/1533791451 shutdown_connections 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f016050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f013720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 --2- 172.21.15.22:0/1533791451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f00ab40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- 172.21.15.22:0/1533791451 >> 172.21.15.22:0/1533791451 conn(0x55859f004e90 msgr2=0x55859f005270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- 172.21.15.22:0/1533791451 shutdown_connections 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- 172.21.15.22:0/1533791451 wait complete. 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- start start 2024-03-09T00:05:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 Processor -- start 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f013440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f0107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f010d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55859f016a50 con 0x55859f00a760 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55859ef2a0a0 con 0x55859f00b080 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55859f016bd0 con 0x55859f013c60 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.240+0000 7f7f2776e740 10 client.0 _ll_register_callbacks cb 0x55859ef29a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f013440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f013440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37378/0 (socket says 172.21.15.22:37378) 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3504436 learned_addr learned my addr 172.21.15.22:0/3504436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f0107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f010d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 msgr2=0x55859f010d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f010d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3504436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 msgr2=0x55859f0107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f0107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3504436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55859ec823f0 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f013440 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7f140057c0 tx=0x7f7f14004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de22ee421613e54 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f05ffb640 1 -- 172.21.15.22:0/3504436 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140051d0 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f05ffb640 1 -- 172.21.15.22:0/3504436 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7f14004030 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f14005500 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f05ffb640 1 -- 172.21.15.22:0/3504436 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f14004230 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f2776e740 10 client.19191 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.241+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55859ec420f0 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f05ffb640 1 -- 172.21.15.22:0/3504436 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f14005500 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f05ffb640 1 -- 172.21.15.22:0/3504436 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7f14020460 con 0x55859f00a760 2024-03-09T00:05:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f2776e740 10 client.19191 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f2776e740 10 client.19191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f2776e740 20 client.19191 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f0236e0 0x55859f025ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f2776e740 4 client.19191 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55859ef7b340 con 0x55859f0236e0 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f0236e0 0x55859f025ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f05ffb640 10 client.19191 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.242+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f0236e0 0x55859f025ad0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9657965c47a498 server_cookie=20d7dcb2f455c815 in_seq=0 out_seq=0 2024-03-09T00:05:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f05ffb640 1 -- 172.21.15.22:0/3504436 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55859ef7b340 con 0x55859f0236e0 2024-03-09T00:05:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f05ffb640 10 client.19191 handle_command_reply: tid=0 2024-03-09T00:05:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 client.19191 shutdown 2024-03-09T00:05:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 msgr2=0x55859f013440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f013440 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7f140057c0 tx=0x7f7f14004c20 comp rx=0 tx=0).stop 2024-03-09T00:05:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 shutdown_connections 2024-03-09T00:05:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f0236e0 0x55859f025ad0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f013c60 0x55859f010d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00b080 0x55859f0107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3504436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00a760 0x55859f013440 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f05ffb640 0 client.19191 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 >> 172.21.15.22:0/3504436 conn(0x55859f004e90 msgr2=0x55859f006be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 shutdown_connections 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.244+0000 7f7f2776e740 1 -- 172.21.15.22:0/3504436 wait complete. 2024-03-09T00:05:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.245+0000 7f7f2776e740 20 client.19191 trim_cache size 0 max 16384 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 Processor -- start 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 -- start start 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00a760 0x55859efffd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00b080 0x55859f00f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f013c60 0x55859f012f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201b2e10 con 0x55859f013c60 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201b09b0 con 0x55859f00a760 2024-03-09T00:05:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f2776e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201aea70 con 0x55859f00b080 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f013c60 0x55859f012f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f013c60 0x55859f012f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37380/0 (socket says 172.21.15.22:37380) 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3780502525 learned_addr learned my addr 172.21.15.22:0/3780502525 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00b080 0x55859f00f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00a760 0x55859efffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3780502525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00b080 msgr2=0x55859f00f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00b080 0x55859f00f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3780502525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00a760 msgr2=0x55859efffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00a760 0x55859efffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 -- 172.21.15.22:0/3780502525 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55859f00aeb0 con 0x55859f013c60 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00b080 0x55859f00f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.261+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f013c60 0x55859f012f00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7f14006010 tx=0x7f7f14005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb0724db380bfc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f07fff640 1 -- 172.21.15.22:0/3780502525 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14020460 con 0x55859f013c60 2024-03-09T00:05:52.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f07fff640 1 -- 172.21.15.22:0/3780502525 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f1402a040 con 0x55859f013c60 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f07fff640 1 -- 172.21.15.22:0/3780502525 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140043d0 con 0x55859f013c60 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 -- 172.21.15.22:0/3780502525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f013c60 msgr2=0x55859f012f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f013c60 0x55859f012f00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7f14006010 tx=0x7f7f14005970 comp rx=0 tx=0).stop 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 -- 172.21.15.22:0/3780502525 shutdown_connections 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f013c60 0x55859f012f00 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00b080 0x55859f00f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 --2- 172.21.15.22:0/3780502525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f00a760 0x55859efffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 -- 172.21.15.22:0/3780502525 >> 172.21.15.22:0/3780502525 conn(0x55859f004e90 msgr2=0x55859f00f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 -- 172.21.15.22:0/3780502525 shutdown_connections 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.262+0000 7f7f2776e740 1 -- 172.21.15.22:0/3780502525 wait complete. 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 -- start start 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 Processor -- start 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00a760 0x55859f0190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00b080 0x55859f019600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f013c60 0x55859f0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201af070 con 0x55859f00b080 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201b0350 con 0x55859f013c60 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f201af690 con 0x55859f00a760 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f2776e740 10 client.0 _ll_register_callbacks cb 0x7f7f14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00b080 0x55859f019600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00b080 0x55859f019600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37384/0 (socket says 172.21.15.22:37384) 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 -- 172.21.15.22:0/274194637 learned_addr learned my addr 172.21.15.22:0/274194637 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f013c60 0x55859f0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00a760 0x55859f0190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 -- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00a760 msgr2=0x55859f0190c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00a760 0x55859f0190c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 -- 172.21.15.22:0/274194637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f013c60 msgr2=0x55859f0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f013c60 0x55859f0729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.263+0000 7f7f1e59c640 1 -- 172.21.15.22:0/274194637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55859f072f80 con 0x55859f00b080 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f1e59c640 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00b080 0x55859f019600 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009230 tx=0x7f7f10004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffd78c85415e2b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f05ffb640 1 -- 172.21.15.22:0/274194637 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1002c8b0 con 0x55859f00b080 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f05ffb640 1 -- 172.21.15.22:0/274194637 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f7f1002ca50 con 0x55859f00b080 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f1402a040 con 0x55859f00b080 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f05ffb640 1 -- 172.21.15.22:0/274194637 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f10058050 con 0x55859f00b080 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f2776e740 10 client.19197 fetch_fsmap learned FSMap version 35 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55859f07a3d0 con 0x55859f00b080 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.264+0000 7f7f05ffb640 1 -- 172.21.15.22:0/274194637 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f1402a040 con 0x55859f00b080 2024-03-09T00:05:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f05ffb640 1 -- 172.21.15.22:0/274194637 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7f10047040 con 0x55859f00b080 2024-03-09T00:05:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f2776e740 10 client.19197 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:05:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f2776e740 10 client.19197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:05:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f2776e740 20 client.19197 populate_metadata read hostname 'smithi022' 2024-03-09T00:05:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f2776e740 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f07ab80 0x55859f07cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:05:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f2776e740 4 client.19197 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:05:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55859ef75690 con 0x55859f07ab80 2024-03-09T00:05:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.265+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f07ab80 0x55859f07cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:05:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f05ffb640 10 client.19197 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f1dd9b640 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f07ab80 0x55859f07cfa0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b0a4b6a1591a9ad server_cookie=4c6434a2dbe7f513 in_seq=0 out_seq=0 2024-03-09T00:05:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f05ffb640 1 -- 172.21.15.22:0/274194637 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55859ef75690 con 0x55859f07ab80 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f05ffb640 10 client.19197 handle_command_reply: tid=0 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 client.19197 shutdown 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00b080 msgr2=0x55859f019600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00b080 0x55859f019600 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009230 tx=0x7f7f10004130 comp rx=0 tx=0).stop 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 shutdown_connections 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55859f07ab80 0x55859f07cfa0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55859f013c60 0x55859f0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55859f00b080 0x55859f019600 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 --2- 172.21.15.22:0/274194637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55859f00a760 0x55859f0190c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f05ffb640 0 client.19197 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 >> 172.21.15.22:0/274194637 conn(0x55859f004e90 msgr2=0x55859f00f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 shutdown_connections 2024-03-09T00:05:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 1 -- 172.21.15.22:0/274194637 wait complete. 2024-03-09T00:05:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.266+0000 7f7f2776e740 20 client.19197 trim_cache size 0 max 16384 2024-03-09T00:05:52.268 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:05:52.268 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:05:52.268 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:05:52.268 INFO:teuthology.orchestra.run.smithi022.stdout: "0d053e49-98a3-46c1-ae41-d16c0cd4fc9d": { 2024-03-09T00:05:52.268 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:05:52.269 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "0d053e49-98a3-46c1-ae41-d16c0cd4fc9d", 2024-03-09T00:05:52.269 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:05:52.269 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:05:52.269 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:05:52.269 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:05:52.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7f080a2970 msgr2=0x7f7f080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7f080a2970 0x7f7f080a4e30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7f180980a0 tx=0x7f7f180977c0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f2010df70 msgr2=0x7f7f2019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f2010df70 0x7f7f2019e6a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800eb10 tx=0x7f7f1800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 shutdown_connections 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7f080a2970 0x7f7f080a4e30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f2010df70 0x7f7f2019e6a0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f20102710 0x7f7f2019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 --2- 172.21.15.22:0/2591534356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f201022a0 0x7f7f20199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 >> 172.21.15.22:0/2591534356 conn(0x7f7f200767b0 msgr2=0x7f7f2010f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 shutdown_connections 2024-03-09T00:05:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:05:52.267+0000 7f7f24aab640 1 -- 172.21.15.22:0/2591534356 wait complete. 2024-03-09T00:05:52.273 INFO:tasks.cephfs.filesystem:scrub status for tag:0d053e49-98a3-46c1-ae41-d16c0cd4fc9d - {'path': '/', 'tag': '0d053e49-98a3-46c1-ae41-d16c0cd4fc9d', 'options': 'recursive,force'} 2024-03-09T00:05:52.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1143214 9.64 MB/sec execute 480 sec latency 135.915 ms 2024-03-09T00:05:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:52 smithi022 ceph-mon[31894]: pgmap v356: 129 pgs: 129 active+clean; 83 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 394 op/s 2024-03-09T00:05:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:52 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:05:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:52 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:52 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:05:53.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:52 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:53.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:52 smithi062 ceph-mon[36129]: pgmap v356: 129 pgs: 129 active+clean; 83 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 394 op/s 2024-03-09T00:05:53.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:52 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:05:53.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:52 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:53.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:52 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:05:53.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:52 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:53.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:52 smithi192 ceph-mon[35995]: pgmap v356: 129 pgs: 129 active+clean; 83 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 394 op/s 2024-03-09T00:05:53.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:52 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:05:53.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:52 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:53.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:52 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:05:53.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:52 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:05:53.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1144363 9.63 MB/sec execute 481 sec latency 17.472 ms 2024-03-09T00:05:54.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:05:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:53 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:05:54.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:53 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:05:54.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:53 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:54.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1146199 9.63 MB/sec execute 482 sec latency 29.463 ms 2024-03-09T00:05:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:54 smithi022 ceph-mon[31894]: pgmap v357: 129 pgs: 129 active+clean; 83 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-03-09T00:05:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:54 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:05:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:54 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:05:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:54 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:55.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:54 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:05:55.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:54 smithi062 ceph-mon[36129]: pgmap v357: 129 pgs: 129 active+clean; 83 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-03-09T00:05:55.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:54 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:05:55.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:54 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:05:55.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:54 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:55.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:54 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:05:55.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:54 smithi192 ceph-mon[35995]: pgmap v357: 129 pgs: 129 active+clean; 83 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-03-09T00:05:55.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:54 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:05:55.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:54 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:05:55.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:54 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:05:55.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:54 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:05:55.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1147738 9.63 MB/sec execute 483 sec latency 20.991 ms 2024-03-09T00:05:56.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1149688 9.63 MB/sec execute 484 sec latency 16.785 ms 2024-03-09T00:05:57.182 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:56 smithi192 ceph-mon[35995]: pgmap v358: 129 pgs: 129 active+clean; 90 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 376 op/s 2024-03-09T00:05:57.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:56 smithi022 ceph-mon[31894]: pgmap v358: 129 pgs: 129 active+clean; 90 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 376 op/s 2024-03-09T00:05:57.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:56 smithi062 ceph-mon[36129]: pgmap v358: 129 pgs: 129 active+clean; 90 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 376 op/s 2024-03-09T00:05:57.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1150947 9.63 MB/sec execute 485 sec latency 47.981 ms 2024-03-09T00:05:58.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1152323 9.62 MB/sec execute 486 sec latency 16.874 ms 2024-03-09T00:05:59.162 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:05:58 smithi062 ceph-mon[36129]: pgmap v359: 129 pgs: 129 active+clean; 90 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-03-09T00:05:59.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:05:58 smithi022 ceph-mon[31894]: pgmap v359: 129 pgs: 129 active+clean; 90 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-03-09T00:05:59.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:05:58 smithi192 ceph-mon[35995]: pgmap v359: 129 pgs: 129 active+clean; 90 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-03-09T00:05:59.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1154489 9.63 MB/sec execute 487 sec latency 14.149 ms 2024-03-09T00:06:00.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1155794 9.62 MB/sec execute 488 sec latency 22.756 ms 2024-03-09T00:06:01.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:00 smithi022 ceph-mon[31894]: pgmap v360: 129 pgs: 129 active+clean; 94 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 338 op/s 2024-03-09T00:06:01.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:00 smithi062 ceph-mon[36129]: pgmap v360: 129 pgs: 129 active+clean; 94 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 338 op/s 2024-03-09T00:06:01.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:00 smithi192 ceph-mon[35995]: pgmap v360: 129 pgs: 129 active+clean; 94 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 338 op/s 2024-03-09T00:06:01.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1156668 9.60 MB/sec execute 489 sec latency 94.885 ms 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 -- 172.21.15.22:0/1930226064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c100410 msgr2=0x7f0a1c105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 --2- 172.21.15.22:0/1930226064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c100410 0x7f0a1c105f40 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009900 tx=0x7f0a1002bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 -- 172.21.15.22:0/1930226064 shutdown_connections 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 --2- 172.21.15.22:0/1930226064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a1c106620 0x7f0a1c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 --2- 172.21.15.22:0/1930226064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c100410 0x7f0a1c105f40 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 --2- 172.21.15.22:0/1930226064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a1c0ffa60 0x7f0a1c0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 -- 172.21.15.22:0/1930226064 >> 172.21.15.22:0/1930226064 conn(0x7f0a1c076740 msgr2=0x7f0a1c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 -- 172.21.15.22:0/1930226064 shutdown_connections 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.814+0000 7f0a225b2640 1 -- 172.21.15.22:0/1930226064 wait complete. 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 Processor -- start 2024-03-09T00:06:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 -- start start 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a1c0ffa60 0x7f0a1c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a1c106620 0x7f0a1c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c197950 0x7f0a1c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c1116b0 con 0x7f0a1c197950 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c111530 con 0x7f0a1c106620 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.815+0000 7f0a225b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c111830 con 0x7f0a1c0ffa60 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c197950 0x7f0a1c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a1c0ffa60 0x7f0a1c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a1b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a1c106620 0x7f0a1c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c197950 0x7f0a1c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36838/0 (socket says 172.21.15.22:36838) 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3717626768 learned_addr learned my addr 172.21.15.22:0/3717626768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3717626768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a1c0ffa60 msgr2=0x7f0a1c19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a1c0ffa60 0x7f0a1c19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3717626768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a1c106620 msgr2=0x7f0a1c19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a1c106620 0x7f0a1c19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3717626768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c198470 con 0x7f0a1c197950 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a1b7fe640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a1c106620 0x7f0a1c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a1c0ffa60 0x7f0a1c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c197950 0x7f0a1c197e00 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400eb10 tx=0x7f0a1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a197fa640 1 -- 172.21.15.22:0/3717626768 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14004b20 con 0x7f0a1c197950 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a197fa640 1 -- 172.21.15.22:0/3717626768 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a14004cc0 con 0x7f0a1c197950 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a197fa640 1 -- 172.21.15.22:0/3717626768 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14010760 con 0x7f0a1c197950 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c198760 con 0x7f0a1c197950 2024-03-09T00:06:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.816+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c1a49b0 con 0x7f0a1c197950 2024-03-09T00:06:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.818+0000 7f0a197fa640 1 -- 172.21.15.22:0/3717626768 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0a14005020 con 0x7f0a1c197950 2024-03-09T00:06:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a197fa640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0a040a2a10 0x7f0a040a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 Processor -- start 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 -- start start 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0a040a2a10 0x7f0a040a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a197fa640 1 -- 172.21.15.22:0/3717626768 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0a140cf8f0 con 0x7f0a1c197950 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458a9da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458a9e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c458a025f0 con 0x55c458aa7550 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4589ab930 con 0x55c458a9ed40 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a25194740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4589fe690 con 0x55c458a9dfd0 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a1b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458a9e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36848/0 (socket says 172.21.15.22:36848) 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a20b4f640 1 -- 172.21.15.22:0/68660795 learned_addr learned my addr 172.21.15.22:0/68660795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.819+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458a9da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0a040a2a10 0x7f0a040a4ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c198a90 tx=0x7f0a0c011810 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a20b4f640 1 -- 172.21.15.22:0/68660795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 msgr2=0x55c458a9e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458a9e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a20b4f640 1 -- 172.21.15.22:0/68660795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 msgr2=0x55c458a9da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458a9da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a20b4f640 1 -- 172.21.15.22:0/68660795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c458717210 con 0x55c458aa7550 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a1b7fe640 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458a9e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa9940 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0a140a3d60 tx=0x7f0a14097ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70917e227d9bd92f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a027fc640 1 -- 172.21.15.22:0/68660795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1409c1f0 con 0x55c458aa7550 2024-03-09T00:06:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a027fc640 1 -- 172.21.15.22:0/68660795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a1409baf0 con 0x55c458aa7550 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.820+0000 7f0a027fc640 1 -- 172.21.15.22:0/68660795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140cf420 con 0x55c458aa7550 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 -- 172.21.15.22:0/68660795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 msgr2=0x55c458aa9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa9940 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0a140a3d60 tx=0x7f0a14097ec0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 -- 172.21.15.22:0/68660795 shutdown_connections 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa9940 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458a9e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 --2- 172.21.15.22:0/68660795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458a9da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 -- 172.21.15.22:0/68660795 >> 172.21.15.22:0/68660795 conn(0x55c458a655b0 msgr2=0x55c458a98cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 -- 172.21.15.22:0/68660795 shutdown_connections 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 -- 172.21.15.22:0/68660795 wait complete. 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 -- start start 2024-03-09T00:06:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 Processor -- start 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.821+0000 7f0a25194740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aa4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c458aaa280 con 0x55c458aa7550 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4589bed90 con 0x55c458a9dfd0 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c458aaa400 con 0x55c458a9ed40 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 10 client.0 _ll_register_callbacks cb 0x55c458716a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36862/0 (socket says 172.21.15.22:36862) 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1228551578 learned_addr learned my addr 172.21.15.22:0/1228551578 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a1b7fe640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aa4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 msgr2=0x55c458aa4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aa4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1228551578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 msgr2=0x55c458aa4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1228551578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4587e3620 con 0x55c458aa7550 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa4f90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0a140049d0 tx=0x7f0a140ccb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b099e9b44f17ae04 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f09fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1228551578 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140b3070 con 0x55c458aa7550 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1228551578 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0a140cf630 con 0x55c458aa7550 2024-03-09T00:06:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a1409baf0 con 0x55c458aa7550 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1228551578 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a140a4290 con 0x55c458aa7550 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 10 client.19209 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.822+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c458ab6500 con 0x55c458aa7550 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1228551578 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a1409baf0 con 0x55c458aa7550 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1228551578 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a1409af40 con 0x55c458aa7550 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a25194740 10 client.19209 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a25194740 10 client.19209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a25194740 20 client.19209 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a25194740 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458ab7850 0x55c458ab9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a25194740 4 client.19209 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c458a0fb30 con 0x55c458ab7850 2024-03-09T00:06:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458ab7850 0x55c458ab9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f09fb7fe640 10 client.19209 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.823+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458ab7850 0x55c458ab9c40 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187aebaa7a20b98c server_cookie=b290f4d44b324748 in_seq=0 out_seq=0 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1228551578 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c458a0fb30 con 0x55c458ab7850 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f09fb7fe640 10 client.19209 handle_command_reply: tid=0 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 client.19209 shutdown 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 msgr2=0x55c458aa4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa4f90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0a140049d0 tx=0x7f0a140ccb50 comp rx=0 tx=0).stop 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 shutdown_connections 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458ab7850 0x55c458ab9c40 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aa4f90 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aa4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f09fb7fe640 0 client.19209 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 --2- 172.21.15.22:0/1228551578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 >> 172.21.15.22:0/1228551578 conn(0x55c458a655b0 msgr2=0x55c458a9aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 shutdown_connections 2024-03-09T00:06:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 1 -- 172.21.15.22:0/1228551578 wait complete. 2024-03-09T00:06:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.824+0000 7f0a25194740 20 client.19209 trim_cache size 0 max 16384 2024-03-09T00:06:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.837+0000 7f0a25194740 1 Processor -- start 2024-03-09T00:06:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.837+0000 7f0a25194740 1 -- start start 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a25194740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a25194740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a25194740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aaebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a25194740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c1b0c30 con 0x55c458aa7550 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a25194740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c1ae7d0 con 0x55c458a9dfd0 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a25194740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c1ac890 con 0x55c458a9ed40 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aaebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aaebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36874/0 (socket says 172.21.15.22:36874) 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3470866051 learned_addr learned my addr 172.21.15.22:0/3470866051 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a1b7fe640 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3470866051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 msgr2=0x55c458aae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3470866051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 msgr2=0x55c458aa6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 -- 172.21.15.22:0/3470866051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c458a9d2a0 con 0x55c458aa7550 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aaebb0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0a1409fe00 tx=0x7f0a14097ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5843d552b9c68421 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a01ffb640 1 -- 172.21.15.22:0/3470866051 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140b3030 con 0x55c458aa7550 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a01ffb640 1 -- 172.21.15.22:0/3470866051 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a1409c1f0 con 0x55c458aa7550 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.838+0000 7f0a01ffb640 1 -- 172.21.15.22:0/3470866051 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14005310 con 0x55c458aa7550 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 -- 172.21.15.22:0/3470866051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 msgr2=0x55c458aaebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aaebb0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0a1409fe00 tx=0x7f0a14097ec0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 -- 172.21.15.22:0/3470866051 shutdown_connections 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aaebb0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9ed40 0x55c458aae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 --2- 172.21.15.22:0/3470866051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9dfd0 0x55c458aa6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 -- 172.21.15.22:0/3470866051 >> 172.21.15.22:0/3470866051 conn(0x55c458a655b0 msgr2=0x55c458a9f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 -- 172.21.15.22:0/3470866051 shutdown_connections 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 -- 172.21.15.22:0/3470866051 wait complete. 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.839+0000 7f0a25194740 1 -- start start 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 1 Processor -- start 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458aae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458aaeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aacc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c1ace90 con 0x55c458aa7550 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c1ae170 con 0x55c458a9ed40 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c1ad4b0 con 0x55c458a9dfd0 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a25194740 10 client.0 _ll_register_callbacks cb 0x7f0a0c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aacc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aacc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36884/0 (socket says 172.21.15.22:36884) 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1014515029 learned_addr learned my addr 172.21.15.22:0/1014515029 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458aae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a1b7fe640 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458aaeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 msgr2=0x55c458aae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458aae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1014515029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 msgr2=0x55c458aaeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458aaeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 -- 172.21.15.22:0/1014515029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c458aad400 con 0x55c458aa7550 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f0a20b4f640 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aacc00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0a1409ba80 tx=0x7f0a140cd170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fa9d172cd7dd53 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f09fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.840+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1014515029 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140b3070 con 0x55c458aa7550 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1014515029 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f0a140ba330 con 0x55c458aa7550 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a1409c1f0 con 0x55c458aa7550 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1014515029 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a1409baf0 con 0x55c458aa7550 2024-03-09T00:06:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 10 client.19215 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c458aad400 con 0x55c458aa7550 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1014515029 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a1409c1f0 con 0x55c458aa7550 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1014515029 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a1409a9b0 con 0x55c458aa7550 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 10 client.19215 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 10 client.19215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 20 client.19215 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458b0e710 0x55c458b10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 4 client.19215 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c458a09e80 con 0x55c458b0e710 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.841+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458b0e710 0x55c458b10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.842+0000 7f0a1bfff640 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458b0e710 0x55c458b10b00 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef622b75eea1b9cb server_cookie=4b35f7005afffb64 in_seq=0 out_seq=0 2024-03-09T00:06:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:01.842+0000 7f09fb7fe640 10 client.19215 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.306+0000 7f09fb7fe640 1 -- 172.21.15.22:0/1014515029 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c458a09e80 con 0x55c458b0e710 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.306+0000 7f09fb7fe640 10 client.19215 handle_command_reply: tid=0 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.306+0000 7f0a25194740 1 client.19215 shutdown 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 msgr2=0x55c458aacc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aacc00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0a1409ba80 tx=0x7f0a140cd170 comp rx=0 tx=0).stop 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 shutdown_connections 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c458b0e710 0x55c458b10b00 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c458aa7550 0x55c458aacc00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c458a9ed40 0x55c458aaeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 --2- 172.21.15.22:0/1014515029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c458a9dfd0 0x55c458aae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 >> 172.21.15.22:0/1014515029 conn(0x55c458a655b0 msgr2=0x55c458aa44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f09fb7fe640 0 client.19215 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 shutdown_connections 2024-03-09T00:06:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 1 -- 172.21.15.22:0/1014515029 wait complete. 2024-03-09T00:06:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.307+0000 7f0a25194740 20 client.19215 trim_cache size 0 max 16384 2024-03-09T00:06:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:06:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:06:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:06:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:06:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0a040a2a10 msgr2=0x7f0a040a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0a040a2a10 0x7f0a040a4ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c198a90 tx=0x7f0a0c011810 comp rx=0 tx=0).stop 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c197950 msgr2=0x7f0a1c197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c197950 0x7f0a1c197e00 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400eb10 tx=0x7f0a1400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 shutdown_connections 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f0a040a2a10 0x7f0a040a4ed0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a1c197950 0x7f0a1c197e00 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a1c106620 0x7f0a1c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 --2- 172.21.15.22:0/3717626768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a1c0ffa60 0x7f0a1c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 >> 172.21.15.22:0/3717626768 conn(0x7f0a1c076740 msgr2=0x7f0a1c1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 shutdown_connections 2024-03-09T00:06:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:02.308+0000 7f0a225b2640 1 -- 172.21.15.22:0/3717626768 wait complete. 2024-03-09T00:06:02.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1156674 9.59 MB/sec execute 490 sec latency 458.447 ms 2024-03-09T00:06:03.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:02 smithi022 ceph-mon[31894]: pgmap v361: 129 pgs: 129 active+clean; 95 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2024-03-09T00:06:03.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:02 smithi062 ceph-mon[36129]: pgmap v361: 129 pgs: 129 active+clean; 95 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2024-03-09T00:06:03.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:02 smithi192 ceph-mon[35995]: pgmap v361: 129 pgs: 129 active+clean; 95 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2024-03-09T00:06:03.731 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1158010 9.59 MB/sec execute 491 sec latency 99.518 ms 2024-03-09T00:06:04.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1159147 9.58 MB/sec execute 492 sec latency 23.347 ms 2024-03-09T00:06:05.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:04 smithi022 ceph-mon[31894]: pgmap v362: 129 pgs: 129 active+clean; 95 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-03-09T00:06:05.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:04 smithi062 ceph-mon[36129]: pgmap v362: 129 pgs: 129 active+clean; 95 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-03-09T00:06:05.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:04 smithi192 ceph-mon[35995]: pgmap v362: 129 pgs: 129 active+clean; 95 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-03-09T00:06:05.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1160019 9.56 MB/sec execute 493 sec latency 61.781 ms 2024-03-09T00:06:06.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:05 smithi022 ceph-mon[31894]: osdmap e109: 12 total, 12 up, 12 in 2024-03-09T00:06:06.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:05 smithi062 ceph-mon[36129]: osdmap e109: 12 total, 12 up, 12 in 2024-03-09T00:06:06.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:05 smithi192 ceph-mon[35995]: osdmap e109: 12 total, 12 up, 12 in 2024-03-09T00:06:06.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1160846 9.56 MB/sec execute 494 sec latency 57.908 ms 2024-03-09T00:06:07.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:06 smithi022 ceph-mon[31894]: pgmap v364: 129 pgs: 129 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-03-09T00:06:07.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:06 smithi062 ceph-mon[36129]: pgmap v364: 129 pgs: 129 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-03-09T00:06:07.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:06 smithi192 ceph-mon[35995]: pgmap v364: 129 pgs: 129 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-03-09T00:06:07.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1161454 9.55 MB/sec execute 495 sec latency 176.538 ms 2024-03-09T00:06:08.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:07 smithi022 ceph-mon[31894]: pgmap v365: 129 pgs: 129 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-03-09T00:06:08.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:07 smithi062 ceph-mon[36129]: pgmap v365: 129 pgs: 129 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-03-09T00:06:08.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:07 smithi192 ceph-mon[35995]: pgmap v365: 129 pgs: 129 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 365 op/s 2024-03-09T00:06:08.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1161589 9.53 MB/sec execute 496 sec latency 124.402 ms 2024-03-09T00:06:09.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1161775 9.51 MB/sec execute 497 sec latency 73.865 ms 2024-03-09T00:06:10.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:10 smithi062 ceph-mon[36129]: pgmap v366: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 320 op/s 2024-03-09T00:06:10.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:10 smithi192 ceph-mon[35995]: pgmap v366: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 320 op/s 2024-03-09T00:06:10.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:10 smithi022 ceph-mon[31894]: pgmap v366: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 87 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 320 op/s 2024-03-09T00:06:10.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1162209 9.50 MB/sec execute 498 sec latency 57.937 ms 2024-03-09T00:06:11.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1163810 9.49 MB/sec execute 499 sec latency 26.795 ms 2024-03-09T00:06:12.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:12 smithi062 ceph-mon[36129]: pgmap v367: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 89 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-03-09T00:06:12.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:12 smithi192 ceph-mon[35995]: pgmap v367: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 89 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-03-09T00:06:12.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:12 smithi022 ceph-mon[31894]: pgmap v367: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 89 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-03-09T00:06:12.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1165159 9.49 MB/sec execute 500 sec latency 103.720 ms 2024-03-09T00:06:13.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1166298 9.48 MB/sec execute 501 sec latency 159.270 ms 2024-03-09T00:06:14.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:14 smithi062 ceph-mon[36129]: pgmap v368: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 89 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-03-09T00:06:14.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:14 smithi192 ceph-mon[35995]: pgmap v368: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 89 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-03-09T00:06:14.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:14 smithi022 ceph-mon[31894]: pgmap v368: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 89 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 328 op/s 2024-03-09T00:06:14.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1168570 9.49 MB/sec execute 502 sec latency 69.739 ms 2024-03-09T00:06:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 -- 172.21.15.22:0/266459438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c105190 msgr2=0x7fdc5c105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 --2- 172.21.15.22:0/266459438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c105190 0x7fdc5c105570 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 -- 172.21.15.22:0/266459438 shutdown_connections 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 --2- 172.21.15.22:0/266459438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c100e00 0x7fdc5c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 --2- 172.21.15.22:0/266459438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc5c100440 0x7fdc5c1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 --2- 172.21.15.22:0/266459438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c105190 0x7fdc5c105570 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 -- 172.21.15.22:0/266459438 >> 172.21.15.22:0/266459438 conn(0x7fdc5c077040 msgr2=0x7fdc5c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 -- 172.21.15.22:0/266459438 shutdown_connections 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.401+0000 7fdc632c0640 1 -- 172.21.15.22:0/266459438 wait complete. 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 Processor -- start 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 -- start start 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc5c100440 0x7fdc5c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c100e00 0x7fdc5c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c105190 0x7fdc5c1a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c110650 con 0x7fdc5c100e00 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1104d0 con 0x7fdc5c105190 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc632c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1107d0 con 0x7fdc5c100440 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc6085b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c100e00 0x7fdc5c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc6085b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c100e00 0x7fdc5c198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43634/0 (socket says 172.21.15.22:43634) 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc6085b640 1 -- 172.21.15.22:0/2079334443 learned_addr learned my addr 172.21.15.22:0/2079334443 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc6105c640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc5c100440 0x7fdc5c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc6185d640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c105190 0x7fdc5c1a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc6085b640 1 -- 172.21.15.22:0/2079334443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc5c100440 msgr2=0x7fdc5c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.402+0000 7fdc6085b640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc5c100440 0x7fdc5c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc6085b640 1 -- 172.21.15.22:0/2079334443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c105190 msgr2=0x7fdc5c1a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc6085b640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c105190 0x7fdc5c1a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc6085b640 1 -- 172.21.15.22:0/2079334443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c1a3ff0 con 0x7fdc5c100e00 2024-03-09T00:06:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc6105c640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc5c100440 0x7fdc5c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc6185d640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c105190 0x7fdc5c1a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc6085b640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c100e00 0x7fdc5c198f70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9f0 tx=0x7fdc5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc5a7fc640 1 -- 172.21.15.22:0/2079334443 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000cd80 con 0x7fdc5c100e00 2024-03-09T00:06:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc5a7fc640 1 -- 172.21.15.22:0/2079334443 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc50005500 con 0x7fdc5c100e00 2024-03-09T00:06:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5c1a41c0 con 0x7fdc5c100e00 2024-03-09T00:06:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc5a7fc640 1 -- 172.21.15.22:0/2079334443 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50010690 con 0x7fdc5c100e00 2024-03-09T00:06:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.403+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c1a4630 con 0x7fdc5c100e00 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.405+0000 7fdc5a7fc640 1 -- 172.21.15.22:0/2079334443 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdc500046c0 con 0x7fdc5c100e00 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 Processor -- start 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 -- start start 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdc5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdc6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efd2a5f0 con 0x5597efdc5fd0 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efcd3930 con 0x5597efdc6d40 2024-03-09T00:06:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efd26690 con 0x5597efdcf550 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc5a7fc640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdc440a2a10 0x7fdc440a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdc6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdc6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43636/0 (socket says 172.21.15.22:43636) 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 -- 172.21.15.22:0/3408874647 learned_addr learned my addr 172.21.15.22:0/3408874647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6185d640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdc440a2a10 0x7fdc440a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6085b640 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6185d640 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdc5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc5a7fc640 1 -- 172.21.15.22:0/2079334443 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fdc500cf7a0 con 0x7fdc5c100e00 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 -- 172.21.15.22:0/3408874647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 msgr2=0x5597efdd1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 -- 172.21.15.22:0/3408874647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 msgr2=0x5597efdc5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdc5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6105c640 1 -- 172.21.15.22:0/3408874647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597efa3f210 con 0x5597efdc5fd0 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6085b640 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.406+0000 7fdc6185d640 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdc5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.407+0000 7fdc6105c640 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdc6450 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c3144709c79b3a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.407+0000 7fdc6185d640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdc440a2a10 0x7fdc440a4ed0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0138a0 tx=0x7fdc4c00a480 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.407+0000 7fdc437fe640 1 -- 172.21.15.22:0/3408874647 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54042070 con 0x5597efdc5fd0 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.407+0000 7fdc437fe640 1 -- 172.21.15.22:0/3408874647 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc54004c70 con 0x5597efdc5fd0 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.407+0000 7fdc437fe640 1 -- 172.21.15.22:0/3408874647 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54005340 con 0x5597efdc5fd0 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- 172.21.15.22:0/3408874647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 msgr2=0x5597efdc6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdc6450 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402c5d0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- 172.21.15.22:0/3408874647 shutdown_connections 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdc6450 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/3408874647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdc5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- 172.21.15.22:0/3408874647 >> 172.21.15.22:0/3408874647 conn(0x5597efd8d5b0 msgr2=0x5597efdc0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- 172.21.15.22:0/3408874647 shutdown_connections 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- 172.21.15.22:0/3408874647 wait complete. 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- start start 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 Processor -- start 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdcd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efdd2220 con 0x5597efdcf550 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efce6d90 con 0x5597efdc6d40 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597efdd23a0 con 0x5597efdc5fd0 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.408+0000 7fdc65f7b740 10 client.0 _ll_register_callbacks cb 0x5597efa3ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdcd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6105c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdcd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43646/0 (socket says 172.21.15.22:43646) 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 -- 172.21.15.22:0/4131914404 learned_addr learned my addr 172.21.15.22:0/4131914404 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6085b640 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 -- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 msgr2=0x5597efdcc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 -- 172.21.15.22:0/4131914404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 msgr2=0x5597efdccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 -- 172.21.15.22:0/4131914404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597efb0b620 con 0x5597efdcf550 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6105c640 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc6185d640 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdcd020 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00ee50 tx=0x7fdc4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dc190e54746e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc40ff9640 1 -- 172.21.15.22:0/4131914404 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c004b20 con 0x5597efdcf550 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc40ff9640 1 -- 172.21.15.22:0/4131914404 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fdc4c004cc0 con 0x5597efdcf550 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597efd37b30 con 0x5597efdcf550 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc40ff9640 1 -- 172.21.15.22:0/4131914404 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc4c011940 con 0x5597efdcf550 2024-03-09T00:06:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc65f7b740 10 client.19227 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.409+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5597efdde580 con 0x5597efdcf550 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc40ff9640 1 -- 172.21.15.22:0/4131914404 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5597efd37b30 con 0x5597efdcf550 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc40ff9640 1 -- 172.21.15.22:0/4131914404 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc4c005760 con 0x5597efdcf550 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc65f7b740 10 client.19227 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc65f7b740 10 client.19227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc65f7b740 20 client.19227 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efddfa40 0x5597efde1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc65f7b740 4 client.19227 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597efd31e80 con 0x5597efddfa40 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc6105c640 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efddfa40 0x5597efde1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc40ff9640 10 client.19227 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.410+0000 7fdc6105c640 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efddfa40 0x5597efde1e30 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7702cc92d5801dd0 server_cookie=4a2218b7e0829fd4 in_seq=0 out_seq=0 2024-03-09T00:06:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc40ff9640 1 -- 172.21.15.22:0/4131914404 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597efd31e80 con 0x5597efddfa40 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc40ff9640 10 client.19227 handle_command_reply: tid=0 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 client.19227 shutdown 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 msgr2=0x5597efdcd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdcd020 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00ee50 tx=0x7fdc4c004ab0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 shutdown_connections 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efddfa40 0x5597efde1e30 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdcd020 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/4131914404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 >> 172.21.15.22:0/4131914404 conn(0x5597efd8d5b0 msgr2=0x5597efdc2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc40ff9640 0 client.19227 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 shutdown_connections 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 1 -- 172.21.15.22:0/4131914404 wait complete. 2024-03-09T00:06:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.412+0000 7fdc65f7b740 20 client.19227 trim_cache size 0 max 16384 2024-03-09T00:06:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 Processor -- start 2024-03-09T00:06:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 -- start start 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdd6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1b0bd0 con 0x5597efdcf550 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1ae770 con 0x5597efdc6d40 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1ac830 con 0x5597efdc5fd0 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc6185d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdd6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc6185d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdd6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43658/0 (socket says 172.21.15.22:43658) 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc6185d640 1 -- 172.21.15.22:0/536662015 learned_addr learned my addr 172.21.15.22:0/536662015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc6105c640 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.426+0000 7fdc6085b640 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6185d640 1 -- 172.21.15.22:0/536662015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 msgr2=0x5597efdcecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6185d640 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6185d640 1 -- 172.21.15.22:0/536662015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 msgr2=0x5597efdd6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6185d640 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6185d640 1 -- 172.21.15.22:0/536662015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597efde7020 con 0x5597efdcf550 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6085b640 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6105c640 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc6185d640 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdd6b50 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005cc0 tx=0x7fdc4c013fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd1bf9547acad12 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc42ffd640 1 -- 172.21.15.22:0/536662015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c00a630 con 0x5597efdcf550 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc42ffd640 1 -- 172.21.15.22:0/536662015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc4c003970 con 0x5597efdcf550 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc42ffd640 1 -- 172.21.15.22:0/536662015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c027ac0 con 0x5597efdcf550 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc65f7b740 1 -- 172.21.15.22:0/536662015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 msgr2=0x5597efdd6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.427+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdd6b50 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005cc0 tx=0x7fdc4c013fd0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- 172.21.15.22:0/536662015 shutdown_connections 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdcf550 0x5597efdd6b50 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/536662015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdc5fd0 0x5597efdcecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- 172.21.15.22:0/536662015 >> 172.21.15.22:0/536662015 conn(0x5597efd8d5b0 msgr2=0x5597efdc07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- 172.21.15.22:0/536662015 shutdown_connections 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- 172.21.15.22:0/536662015 wait complete. 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- start start 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 Processor -- start 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdd6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1ace30 con 0x5597efdc5fd0 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1ae110 con 0x5597efdc6d40 2024-03-09T00:06:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1ad450 con 0x5597efdcf550 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc65f7b740 10 client.0 _ll_register_callbacks cb 0x7fdc54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc6105c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdd6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc6105c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdd6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43674/0 (socket says 172.21.15.22:43674) 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc6105c640 1 -- 172.21.15.22:0/2059425432 learned_addr learned my addr 172.21.15.22:0/2059425432 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.428+0000 7fdc6185d640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6085b640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6105c640 1 -- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 msgr2=0x5597efdd3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6105c640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6105c640 1 -- 172.21.15.22:0/2059425432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 msgr2=0x5597efdd2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6105c640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6105c640 1 -- 172.21.15.22:0/2059425432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597efdd36f0 con 0x5597efdc5fd0 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6085b640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc6105c640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdd6730 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731f516dcdba9221 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc40ff9640 1 -- 172.21.15.22:0/2059425432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54042040 con 0x5597efdc5fd0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc40ff9640 1 -- 172.21.15.22:0/2059425432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fdc5404e2c0 con 0x5597efdc5fd0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc4c003970 con 0x5597efdc5fd0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc40ff9640 1 -- 172.21.15.22:0/2059425432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5405d050 con 0x5597efdc5fd0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc65f7b740 10 client.19233 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5597efe36cd0 con 0x5597efdc5fd0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.429+0000 7fdc40ff9640 1 -- 172.21.15.22:0/2059425432 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc4c003970 con 0x5597efdc5fd0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc40ff9640 1 -- 172.21.15.22:0/2059425432 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc54047460 con 0x5597efdc5fd0 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc65f7b740 10 client.19233 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc65f7b740 10 client.19233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc65f7b740 20 client.19233 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efe37030 0x5597efe39450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc65f7b740 4 client.19233 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:06:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5597efcb0710 con 0x5597efe37030 2024-03-09T00:06:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc6085b640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efe37030 0x5597efe39450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc6085b640 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efe37030 0x5597efe39450 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5d37d186861ba5 server_cookie=647de10d768ea5ba in_seq=0 out_seq=0 2024-03-09T00:06:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.430+0000 7fdc40ff9640 10 client.19233 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:15.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1169256 9.48 MB/sec execute 503 sec latency 87.517 ms 2024-03-09T00:06:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc40ff9640 1 -- 172.21.15.22:0/2059425432 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5597efcb0710 con 0x5597efe37030 2024-03-09T00:06:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc40ff9640 10 client.19233 handle_command_reply: tid=0 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 client.19233 shutdown 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 msgr2=0x5597efdd6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdd6730 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 shutdown_connections 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597efe37030 0x5597efe39450 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597efdcf550 0x5597efdd3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597efdc6d40 0x5597efdd2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc40ff9640 0 client.19233 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 --2- 172.21.15.22:0/2059425432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597efdc5fd0 0x5597efdd6730 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.739+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 >> 172.21.15.22:0/2059425432 conn(0x5597efd8d5b0 msgr2=0x5597efdcadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 shutdown_connections 2024-03-09T00:06:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc65f7b740 1 -- 172.21.15.22:0/2059425432 wait complete. 2024-03-09T00:06:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc65f7b740 20 client.19233 trim_cache size 0 max 16384 2024-03-09T00:06:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:06:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:06:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:06:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:06:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdc440a2a10 msgr2=0x7fdc440a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdc440a2a10 0x7fdc440a4ed0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0138a0 tx=0x7fdc4c00a480 comp rx=0 tx=0).stop 2024-03-09T00:06:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c100e00 msgr2=0x7fdc5c198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c100e00 0x7fdc5c198f70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9f0 tx=0x7fdc5000edf0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 shutdown_connections 2024-03-09T00:06:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdc440a2a10 0x7fdc440a4ed0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5c105190 0x7fdc5c1a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc5c100e00 0x7fdc5c198f70 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 --2- 172.21.15.22:0/2079334443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc5c100440 0x7fdc5c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 >> 172.21.15.22:0/2079334443 conn(0x7fdc5c077040 msgr2=0x7fdc5c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 shutdown_connections 2024-03-09T00:06:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:15.740+0000 7fdc632c0640 1 -- 172.21.15.22:0/2079334443 wait complete. 2024-03-09T00:06:16.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:16 smithi192 ceph-mon[35995]: pgmap v369: 129 pgs: 129 active+clean; 94 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 160 op/s 2024-03-09T00:06:16.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:16 smithi022 ceph-mon[31894]: pgmap v369: 129 pgs: 129 active+clean; 94 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 160 op/s 2024-03-09T00:06:16.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:16 smithi062 ceph-mon[36129]: pgmap v369: 129 pgs: 129 active+clean; 94 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 160 op/s 2024-03-09T00:06:16.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1171324 9.48 MB/sec execute 504 sec latency 23.330 ms 2024-03-09T00:06:17.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1172416 9.48 MB/sec execute 505 sec latency 166.421 ms 2024-03-09T00:06:18.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:18 smithi192 ceph-mon[35995]: pgmap v370: 129 pgs: 129 active+clean; 94 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-03-09T00:06:18.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:18 smithi022 ceph-mon[31894]: pgmap v370: 129 pgs: 129 active+clean; 94 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-03-09T00:06:18.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:18 smithi062 ceph-mon[36129]: pgmap v370: 129 pgs: 129 active+clean; 94 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-03-09T00:06:18.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1173758 9.47 MB/sec execute 506 sec latency 46.271 ms 2024-03-09T00:06:19.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1175800 9.48 MB/sec execute 507 sec latency 26.121 ms 2024-03-09T00:06:20.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:19 smithi022 ceph-mon[31894]: osdmap e110: 12 total, 12 up, 12 in 2024-03-09T00:06:20.246 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:19 smithi022 ceph-mon[31894]: pgmap v372: 129 pgs: 129 active+clean; 97 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-03-09T00:06:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:19 smithi062 ceph-mon[36129]: osdmap e110: 12 total, 12 up, 12 in 2024-03-09T00:06:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:19 smithi062 ceph-mon[36129]: pgmap v372: 129 pgs: 129 active+clean; 97 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-03-09T00:06:20.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:19 smithi192 ceph-mon[35995]: osdmap e110: 12 total, 12 up, 12 in 2024-03-09T00:06:20.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:19 smithi192 ceph-mon[35995]: pgmap v372: 129 pgs: 129 active+clean; 97 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2024-03-09T00:06:20.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1177145 9.47 MB/sec execute 508 sec latency 23.159 ms 2024-03-09T00:06:21.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1179295 9.48 MB/sec execute 509 sec latency 13.823 ms 2024-03-09T00:06:22.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:06:22.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:06:22.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:22 smithi062 ceph-mon[36129]: pgmap v373: 129 pgs: 129 active+clean; 87 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-09T00:06:22.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:22 smithi192 ceph-mon[35995]: pgmap v373: 129 pgs: 129 active+clean; 87 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-09T00:06:22.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 -- 172.21.15.22:0/4216106185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c101ea0 msgr2=0x7f652c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 --2- 172.21.15.22:0/4216106185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c101ea0 0x7f652c105ee0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f651c009900 tx=0x7f651c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:22.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 -- 172.21.15.22:0/4216106185 shutdown_connections 2024-03-09T00:06:22.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 --2- 172.21.15.22:0/4216106185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f652c1065c0 0x7f652c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 --2- 172.21.15.22:0/4216106185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c101ea0 0x7f652c105ee0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 --2- 172.21.15.22:0/4216106185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c1014f0 0x7f652c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 -- 172.21.15.22:0/4216106185 >> 172.21.15.22:0/4216106185 conn(0x7f652c0fcd90 msgr2=0x7f652c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 -- 172.21.15.22:0/4216106185 shutdown_connections 2024-03-09T00:06:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.393+0000 7f6530969640 1 -- 172.21.15.22:0/4216106185 wait complete. 2024-03-09T00:06:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 Processor -- start 2024-03-09T00:06:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 -- start start 2024-03-09T00:06:22.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c1014f0 0x7f652c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f652c1065c0 0x7f652c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c1979d0 0x7f652c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c111630 con 0x7f652c1014f0 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1114b0 con 0x7f652c1979d0 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6530969640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1117b0 con 0x7f652c1065c0 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c1014f0 0x7f652c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c1014f0 0x7f652c19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51116/0 (socket says 172.21.15.22:51116) 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 -- 172.21.15.22:0/125802609 learned_addr learned my addr 172.21.15.22:0/125802609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f6529d9b640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f652c1065c0 0x7f652c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652ad9d640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c1979d0 0x7f652c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 -- 172.21.15.22:0/125802609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f652c1065c0 msgr2=0x7f652c19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f652c1065c0 0x7f652c19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 -- 172.21.15.22:0/125802609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c1979d0 msgr2=0x7f652c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c1979d0 0x7f652c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.394+0000 7f652a59c640 1 -- 172.21.15.22:0/125802609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c1984f0 con 0x7f652c1014f0 2024-03-09T00:06:22.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.395+0000 7f652a59c640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c1014f0 0x7f652c19d820 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f651400e9f0 tx=0x7f651400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.395+0000 7f65237fe640 1 -- 172.21.15.22:0/125802609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400cd80 con 0x7f652c1014f0 2024-03-09T00:06:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.395+0000 7f65237fe640 1 -- 172.21.15.22:0/125802609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6514010910 con 0x7f652c1014f0 2024-03-09T00:06:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.395+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c1987e0 con 0x7f652c1014f0 2024-03-09T00:06:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.395+0000 7f65237fe640 1 -- 172.21.15.22:0/125802609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514018990 con 0x7f652c1014f0 2024-03-09T00:06:22.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.395+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c1a4a30 con 0x7f652c1014f0 2024-03-09T00:06:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.397+0000 7f65237fe640 1 -- 172.21.15.22:0/125802609 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6514010ac0 con 0x7f652c1014f0 2024-03-09T00:06:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.397+0000 7f6533625740 1 Processor -- start 2024-03-09T00:06:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6533625740 1 -- start start 2024-03-09T00:06:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f65237fe640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65180a2970 0x7f65180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6533625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6533625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6533625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6533625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b987e00 con 0x563b4ba24760 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6533625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b930b10 con 0x563b4ba2dc60 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6533625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b983ea0 con 0x563b4ba25080 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652ad9d640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65180a2970 0x7f65180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f65237fe640 1 -- 172.21.15.22:0/125802609 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f65140cf620 con 0x7f652c1014f0 2024-03-09T00:06:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba24b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51118/0 (socket says 172.21.15.22:51118) 2024-03-09T00:06:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 -- 172.21.15.22:0/2295764663 learned_addr learned my addr 172.21.15.22:0/2295764663 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f6529d9b640 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652ad9d640 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 -- 172.21.15.22:0/2295764663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 msgr2=0x563b4ba2d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 -- 172.21.15.22:0/2295764663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 msgr2=0x563b4ba30050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba30050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 -- 172.21.15.22:0/2295764663 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4b7667d0 con 0x563b4ba24760 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652ad9d640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65180a2970 0x7f65180a4e30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6524005fd0 tx=0x7f6524009e60 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.398+0000 7f652a59c640 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba24b40 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f65140a3a90 tx=0x7f651409b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1c4171b5a8afda server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6520ff9640 1 -- 172.21.15.22:0/2295764663 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140679c0 con 0x563b4ba24760 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6520ff9640 1 -- 172.21.15.22:0/2295764663 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f651409b8d0 con 0x563b4ba24760 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6520ff9640 1 -- 172.21.15.22:0/2295764663 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651406f140 con 0x563b4ba24760 2024-03-09T00:06:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6533625740 1 -- 172.21.15.22:0/2295764663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 msgr2=0x563b4ba24b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6533625740 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba24b40 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f65140a3a90 tx=0x7f651409b330 comp rx=0 tx=0).stop 2024-03-09T00:06:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6533625740 1 -- 172.21.15.22:0/2295764663 shutdown_connections 2024-03-09T00:06:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6533625740 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6533625740 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6533625740 1 --2- 172.21.15.22:0/2295764663 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba24b40 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.399+0000 7f6533625740 1 -- 172.21.15.22:0/2295764663 >> 172.21.15.22:0/2295764663 conn(0x563b4ba1ee90 msgr2=0x563b4ba1f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 -- 172.21.15.22:0/2295764663 shutdown_connections 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 -- 172.21.15.22:0/2295764663 wait complete. 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 -- start start 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 Processor -- start 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba2d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4ba309f0 con 0x563b4ba24760 2024-03-09T00:06:22.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4b9440a0 con 0x563b4ba2dc60 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4ba30b70 con 0x563b4ba25080 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6533625740 10 client.0 _ll_register_callbacks cb 0x563b4b943a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba2d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba2d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51132/0 (socket says 172.21.15.22:51132) 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f652a59c640 1 -- 172.21.15.22:0/4138882363 learned_addr learned my addr 172.21.15.22:0/4138882363 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f6529d9b640 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.400+0000 7f652ad9d640 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f652a59c640 1 -- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 msgr2=0x563b4ba2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f652a59c640 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f652a59c640 1 -- 172.21.15.22:0/4138882363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 msgr2=0x563b4ba2aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f652a59c640 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f652a59c640 1 -- 172.21.15.22:0/4138882363 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4b69c3f0 con 0x563b4ba24760 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f652a59c640 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba2d3d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f65140982a0 tx=0x7f651409fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e262d7840503c19 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6511ffb640 1 -- 172.21.15.22:0/4138882363 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140ccb90 con 0x563b4ba24760 2024-03-09T00:06:22.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6511ffb640 1 -- 172.21.15.22:0/4138882363 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f65140ccd30 con 0x563b4ba24760 2024-03-09T00:06:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f651409b8d0 con 0x563b4ba24760 2024-03-09T00:06:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6511ffb640 1 -- 172.21.15.22:0/4138882363 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65140a3b30 con 0x563b4ba24760 2024-03-09T00:06:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 10 client.19245 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563b4b65c0f0 con 0x563b4ba24760 2024-03-09T00:06:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6511ffb640 1 -- 172.21.15.22:0/4138882363 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f651409b8d0 con 0x563b4ba24760 2024-03-09T00:06:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6511ffb640 1 -- 172.21.15.22:0/4138882363 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f651406f0e0 con 0x563b4ba24760 2024-03-09T00:06:22.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 10 client.19245 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 10 client.19245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 20 client.19245 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba3d550 0x563b4ba3f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 4 client.19245 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.401+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b4b995340 con 0x563b4ba3d550 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.402+0000 7f6529d9b640 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba3d550 0x563b4ba3f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.402+0000 7f6529d9b640 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba3d550 0x563b4ba3f940 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9282c63fc5427054 server_cookie=7049230462112839 in_seq=0 out_seq=0 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.402+0000 7f6511ffb640 10 client.19245 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:22.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6511ffb640 1 -- 172.21.15.22:0/4138882363 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b4b995340 con 0x563b4ba3d550 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6511ffb640 10 client.19245 handle_command_reply: tid=0 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 client.19245 shutdown 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 msgr2=0x563b4ba2d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba2d3d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f65140982a0 tx=0x7f651409fa00 comp rx=0 tx=0).stop 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 shutdown_connections 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba3d550 0x563b4ba3f940 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6511ffb640 0 client.19245 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:22.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 --2- 172.21.15.22:0/4138882363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba2d3d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 >> 172.21.15.22:0/4138882363 conn(0x563b4ba1ee90 msgr2=0x563b4ba20be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 shutdown_connections 2024-03-09T00:06:22.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 1 -- 172.21.15.22:0/4138882363 wait complete. 2024-03-09T00:06:22.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.403+0000 7f6533625740 20 client.19245 trim_cache size 0 max 16384 2024-03-09T00:06:22.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.418+0000 7f6533625740 1 Processor -- start 2024-03-09T00:06:22.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.418+0000 7f6533625740 1 -- start start 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6533625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba25cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6533625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6533625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6533625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1b0b80 con 0x563b4ba24760 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6533625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1ae720 con 0x563b4ba2dc60 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6533625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1ac7e0 con 0x563b4ba25080 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba25cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba25cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51142/0 (socket says 172.21.15.22:51142) 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 -- 172.21.15.22:0/2480643201 learned_addr learned my addr 172.21.15.22:0/2480643201 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652ad9d640 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6529d9b640 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 -- 172.21.15.22:0/2480643201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 msgr2=0x563b4ba2ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 -- 172.21.15.22:0/2480643201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 msgr2=0x563b4ba2d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 -- 172.21.15.22:0/2480643201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4ba28cb0 con 0x563b4ba24760 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f652a59c640 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba25cb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6514097d30 tx=0x7f6514000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5417f2eb103bd5c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6513fff640 1 -- 172.21.15.22:0/2480643201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140a0c40 con 0x563b4ba24760 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6513fff640 1 -- 172.21.15.22:0/2480643201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65140b4070 con 0x563b4ba24760 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.419+0000 7f6513fff640 1 -- 172.21.15.22:0/2480643201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514018b30 con 0x563b4ba24760 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 -- 172.21.15.22:0/2480643201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 msgr2=0x563b4ba25cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba25cb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6514097d30 tx=0x7f6514000bc0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 -- 172.21.15.22:0/2480643201 shutdown_connections 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba2dc60 0x563b4ba2d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba25080 0x563b4ba2ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 --2- 172.21.15.22:0/2480643201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba24760 0x563b4ba25cb0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 -- 172.21.15.22:0/2480643201 >> 172.21.15.22:0/2480643201 conn(0x563b4ba1ee90 msgr2=0x563b4ba255a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 -- 172.21.15.22:0/2480643201 shutdown_connections 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.420+0000 7f6533625740 1 -- 172.21.15.22:0/2480643201 wait complete. 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 -- start start 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 Processor -- start 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba24760 0x563b4ba3ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba25080 0x563b4ba40520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba2dc60 0x563b4ba41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1acde0 con 0x563b4ba25080 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1ae0c0 con 0x563b4ba24760 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1ad400 con 0x563b4ba2dc60 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6533625740 10 client.0 _ll_register_callbacks cb 0x7f6524015760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba25080 0x563b4ba40520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f652a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba24760 0x563b4ba3ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba25080 0x563b4ba40520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51146/0 (socket says 172.21.15.22:51146) 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6529d9b640 1 -- 172.21.15.22:0/1832418423 learned_addr learned my addr 172.21.15.22:0/1832418423 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f652ad9d640 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba2dc60 0x563b4ba41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f652a59c640 1 -- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba2dc60 msgr2=0x563b4ba41390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f652a59c640 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba2dc60 0x563b4ba41390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f652a59c640 1 -- 172.21.15.22:0/1832418423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba25080 msgr2=0x563b4ba40520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f652a59c640 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba25080 0x563b4ba40520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f652a59c640 1 -- 172.21.15.22:0/1832418423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4ba41a80 con 0x563b4ba24760 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.421+0000 7f6529d9b640 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba25080 0x563b4ba40520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f652a59c640 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba24760 0x563b4ba3ffe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f651405d040 tx=0x7f65140cca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78b5920fce703b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6511ffb640 1 -- 172.21.15.22:0/1832418423 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514064030 con 0x563b4ba24760 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6511ffb640 1 -- 172.21.15.22:0/1832418423 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6514018d40 con 0x563b4ba24760 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65140b4070 con 0x563b4ba24760 2024-03-09T00:06:22.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6511ffb640 1 -- 172.21.15.22:0/1832418423 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f651409ce60 con 0x563b4ba24760 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 10 client.28568 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563b4ba41a80 con 0x563b4ba24760 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6511ffb640 1 -- 172.21.15.22:0/1832418423 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65140b4070 con 0x563b4ba24760 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6511ffb640 1 -- 172.21.15.22:0/1832418423 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f65140cf0c0 con 0x563b4ba24760 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 10 client.28568 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 10 client.28568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 20 client.28568 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba94500 0x563b4ba96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 4 client.28568 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.422+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563b4b98f690 con 0x563b4ba94500 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6529d9b640 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba94500 0x563b4ba96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6529d9b640 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba94500 0x563b4ba96920 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d38ffb89d58034b server_cookie=50605f630cac94a9 in_seq=0 out_seq=0 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6511ffb640 10 client.28568 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6511ffb640 1 -- 172.21.15.22:0/1832418423 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563b4b98f690 con 0x563b4ba94500 2024-03-09T00:06:22.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6511ffb640 10 client.28568 handle_command_reply: tid=0 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 client.28568 shutdown 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba24760 msgr2=0x563b4ba3ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba24760 0x563b4ba3ffe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f651405d040 tx=0x7f65140cca10 comp rx=0 tx=0).stop 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 shutdown_connections 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563b4ba94500 0x563b4ba96920 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b4ba2dc60 0x563b4ba41390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563b4ba25080 0x563b4ba40520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 --2- 172.21.15.22:0/1832418423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b4ba24760 0x563b4ba3ffe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6511ffb640 0 client.28568 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.423+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 >> 172.21.15.22:0/1832418423 conn(0x563b4ba1ee90 msgr2=0x563b4ba20190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 shutdown_connections 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6533625740 1 -- 172.21.15.22:0/1832418423 wait complete. 2024-03-09T00:06:22.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6533625740 20 client.28568 trim_cache size 0 max 16384 2024-03-09T00:06:22.426 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:22.426 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:06:22.426 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:06:22.426 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65180a2970 msgr2=0x7f65180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65180a2970 0x7f65180a4e30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6524005fd0 tx=0x7f6524009e60 comp rx=0 tx=0).stop 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c1014f0 msgr2=0x7f652c19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c1014f0 0x7f652c19d820 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f651400e9f0 tx=0x7f651400eeb0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 shutdown_connections 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65180a2970 0x7f65180a4e30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c1979d0 0x7f652c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f652c1065c0 0x7f652c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 --2- 172.21.15.22:0/125802609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f652c1014f0 0x7f652c19d820 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 >> 172.21.15.22:0/125802609 conn(0x7f652c0fcd90 msgr2=0x7f652c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 shutdown_connections 2024-03-09T00:06:22.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.424+0000 7f6530969640 1 -- 172.21.15.22:0/125802609 wait complete. 2024-03-09T00:06:22.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:22 smithi022 ceph-mon[31894]: pgmap v373: 129 pgs: 129 active+clean; 87 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-09T00:06:22.430 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:06:22.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:06:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.575+0000 7fc7de059640 1 -- 172.21.15.22:0/3484877311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7d8073fb0 msgr2=0x7fc7d8111d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.575+0000 7fc7de059640 1 --2- 172.21.15.22:0/3484877311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7d8073fb0 0x7fc7d8111d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009ca0 tx=0x7fc7d002c1c0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.578+0000 7fc7de059640 1 -- 172.21.15.22:0/3484877311 shutdown_connections 2024-03-09T00:06:22.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.578+0000 7fc7de059640 1 --2- 172.21.15.22:0/3484877311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 0x7fc7d8114680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.578+0000 7fc7de059640 1 --2- 172.21.15.22:0/3484877311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7d8073fb0 0x7fc7d8111d50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.578+0000 7fc7de059640 1 --2- 172.21.15.22:0/3484877311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7d8073690 0x7fc7d8073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.578+0000 7fc7de059640 1 -- 172.21.15.22:0/3484877311 >> 172.21.15.22:0/3484877311 conn(0x7fc7d80786f0 msgr2=0x7fc7d8078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.583+0000 7fc7de059640 1 -- 172.21.15.22:0/3484877311 shutdown_connections 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.584+0000 7fc7de059640 1 -- 172.21.15.22:0/3484877311 wait complete. 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 Processor -- start 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 -- start start 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7d8073690 0x7fc7d8072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7d8073fb0 0x7fc7d806bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 0x7fc7d806c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8117dc0 con 0x7fc7d8112290 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8117c40 con 0x7fc7d8073690 2024-03-09T00:06:22.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.585+0000 7fc7de059640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8117f40 con 0x7fc7d8073fb0 2024-03-09T00:06:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.586+0000 7fc7d77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7d8073690 0x7fc7d8072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.586+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 0x7fc7d806c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.586+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 0x7fc7d806c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51158/0 (socket says 172.21.15.22:51158) 2024-03-09T00:06:22.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.586+0000 7fc7d7fff640 1 -- 172.21.15.22:0/1787270742 learned_addr learned my addr 172.21.15.22:0/1787270742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.587+0000 7fc7d7fff640 1 -- 172.21.15.22:0/1787270742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7d8073fb0 msgr2=0x7fc7d806bc70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T00:06:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.587+0000 7fc7d7fff640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7d8073fb0 0x7fc7d806bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.587+0000 7fc7d7fff640 1 -- 172.21.15.22:0/1787270742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7d8073690 msgr2=0x7fc7d8072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.587+0000 7fc7d7fff640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7d8073690 0x7fc7d8072bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.587+0000 7fc7d7fff640 1 -- 172.21.15.22:0/1787270742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d806c8e0 con 0x7fc7d8112290 2024-03-09T00:06:22.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.587+0000 7fc7d7fff640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 0x7fc7d806c1b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800b3c0 tx=0x7fc7c800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.587+0000 7fc7d4ff9640 1 -- 172.21.15.22:0/1787270742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8011830 con 0x7fc7d8112290 2024-03-09T00:06:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.588+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d8070710 con 0x7fc7d8112290 2024-03-09T00:06:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.588+0000 7fc7d4ff9640 1 -- 172.21.15.22:0/1787270742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7c8016070 con 0x7fc7d8112290 2024-03-09T00:06:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.589+0000 7fc7d4ff9640 1 -- 172.21.15.22:0/1787270742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800f2e0 con 0x7fc7d8112290 2024-03-09T00:06:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.589+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d8070c60 con 0x7fc7d8112290 2024-03-09T00:06:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.591+0000 7fc7d4ff9640 1 -- 172.21.15.22:0/1787270742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc7c8010380 con 0x7fc7d8112290 2024-03-09T00:06:22.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.592+0000 7fc7d4ff9640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7c00a2a50 0x7fc7c00a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.592+0000 7fc7d4ff9640 1 -- 172.21.15.22:0/1787270742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc7c80ce2e0 con 0x7fc7d8112290 2024-03-09T00:06:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.592+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d8074dd0 con 0x7fc7d8112290 2024-03-09T00:06:22.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.594+0000 7fc7d77fe640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7c00a2a50 0x7fc7c00a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:22.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.595+0000 7fc7d77fe640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7c00a2a50 0x7fc7c00a4f10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc006fd0 tx=0x7fc7cc008040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:22.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.602+0000 7fc7d4ff9640 1 -- 172.21.15.22:0/1787270742 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc7c8020060 con 0x7fc7d8112290 2024-03-09T00:06:22.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1180087 9.46 MB/sec execute 510 sec latency 111.231 ms 2024-03-09T00:06:22.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.886+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7d8067850 con 0x7fc7d8112290 2024-03-09T00:06:22.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.886+0000 7fc7d4ff9640 1 -- 172.21.15.22:0/1787270742 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fc7c809acf0 con 0x7fc7d8112290 2024-03-09T00:06:22.882 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:06:22.882 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:06:22.882 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:06:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7c00a2a50 msgr2=0x7fc7c00a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7c00a2a50 0x7fc7c00a4f10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc006fd0 tx=0x7fc7cc008040 comp rx=0 tx=0).stop 2024-03-09T00:06:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 msgr2=0x7fc7d806c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 0x7fc7d806c1b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800b3c0 tx=0x7fc7c800b880 comp rx=0 tx=0).stop 2024-03-09T00:06:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 shutdown_connections 2024-03-09T00:06:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc7c00a2a50 0x7fc7c00a4f10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7d8112290 0x7fc7d806c1b0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7d8073fb0 0x7fc7d806bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 --2- 172.21.15.22:0/1787270742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7d8073690 0x7fc7d8072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:22.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.888+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 >> 172.21.15.22:0/1787270742 conn(0x7fc7d80786f0 msgr2=0x7fc7d810dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:22.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.889+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 shutdown_connections 2024-03-09T00:06:22.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:22.889+0000 7fc7de059640 1 -- 172.21.15.22:0/1787270742 wait complete. 2024-03-09T00:06:22.894 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:22.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:06:23.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 -- 172.21.15.22:0/102580707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31341054b0 msgr2=0x7f313410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 --2- 172.21.15.22:0/102580707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31341054b0 0x7f313410d080 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f312c009ca0 tx=0x7f312c02c0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 -- 172.21.15.22:0/102580707 shutdown_connections 2024-03-09T00:06:23.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 --2- 172.21.15.22:0/102580707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31341054b0 0x7f313410d080 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 --2- 172.21.15.22:0/102580707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3134100d90 0x7f3134104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 --2- 172.21.15.22:0/102580707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31341003e0 0x7f31341007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 -- 172.21.15.22:0/102580707 >> 172.21.15.22:0/102580707 conn(0x7f31340773a0 msgr2=0x7f31340777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 -- 172.21.15.22:0/102580707 shutdown_connections 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.005+0000 7f3138ad1640 1 -- 172.21.15.22:0/102580707 wait complete. 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 Processor -- start 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 -- start start 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31341003e0 0x7f3134199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31341054b0 0x7f313419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313419e630 0x7f313419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134110570 con 0x7f313419e630 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341103f0 con 0x7f31341054b0 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3138ad1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341106f0 con 0x7f31341003e0 2024-03-09T00:06:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313419e630 0x7f313419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f313259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31341003e0 0x7f3134199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313419e630 0x7f313419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51166/0 (socket says 172.21.15.22:51166) 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3131d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31341054b0 0x7f313419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 -- 172.21.15.22:0/3023923219 learned_addr learned my addr 172.21.15.22:0/3023923219 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 -- 172.21.15.22:0/3023923219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31341003e0 msgr2=0x7f3134199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31341003e0 0x7f3134199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 -- 172.21.15.22:0/3023923219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31341054b0 msgr2=0x7f313419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31341054b0 0x7f313419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 -- 172.21.15.22:0/3023923219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313419f120 con 0x7f313419e630 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3131d9b640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31341054b0 0x7f313419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.006+0000 7f3132d9d640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313419e630 0x7f313419eae0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f312c02c700 tx=0x7f312c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.007+0000 7f31237fe640 1 -- 172.21.15.22:0/3023923219 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c041020 con 0x7f313419e630 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.007+0000 7f31237fe640 1 -- 172.21.15.22:0/3023923219 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f312c034cd0 con 0x7f313419e630 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.007+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313419f3b0 con 0x7f313419e630 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.007+0000 7f31237fe640 1 -- 172.21.15.22:0/3023923219 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c039040 con 0x7f313419e630 2024-03-09T00:06:23.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.007+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31341a6b70 con 0x7f313419e630 2024-03-09T00:06:23.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.009+0000 7f31237fe640 1 -- 172.21.15.22:0/3023923219 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f312c044020 con 0x7f313419e630 2024-03-09T00:06:23.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.009+0000 7f313b78f740 1 Processor -- start 2024-03-09T00:06:23.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313b78f740 1 -- start start 2024-03-09T00:06:23.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313b78f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ec1a0 0x564a477ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313b78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313b78f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313b78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a477b3360 con 0x564a477ecac0 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313b78f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4774fec0 con 0x564a477ec1a0 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313b78f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a476f8d30 con 0x564a477f56a0 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f31237fe640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f311c0a2970 0x7f311c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3131d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ec1a0 0x564a477ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51170/0 (socket says 172.21.15.22:51170) 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313259c640 1 -- 172.21.15.22:0/4252514211 learned_addr learned my addr 172.21.15.22:0/4252514211 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f31237fe640 1 -- 172.21.15.22:0/3023923219 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f312c0f0e40 con 0x7f313419e630 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3131d9b640 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3132d9d640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f311c0a2970 0x7f311c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3131d9b640 1 -- 172.21.15.22:0/4252514211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 msgr2=0x564a477f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3131d9b640 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3131d9b640 1 -- 172.21.15.22:0/4252514211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 msgr2=0x564a477f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3131d9b640 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f3131d9b640 1 -- 172.21.15.22:0/4252514211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a476f7bc0 con 0x564a477ec1a0 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.010+0000 7f313259c640 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.011+0000 7f3131d9b640 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ec1a0 0x564a477ec580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3128009710 tx=0x7f31280096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=753de63f10b88bb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.011+0000 7f3132d9d640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f311c0a2970 0x7f311c0a4e30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f312c037040 tx=0x7f312c0324b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.011+0000 7f3120ff9640 1 -- 172.21.15.22:0/4252514211 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128004050 con 0x564a477ec1a0 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.011+0000 7f3120ff9640 1 -- 172.21.15.22:0/4252514211 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f312800fdf0 con 0x564a477ec1a0 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.011+0000 7f3120ff9640 1 -- 172.21.15.22:0/4252514211 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128005540 con 0x564a477ec1a0 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- 172.21.15.22:0/4252514211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ec1a0 msgr2=0x564a477ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ec1a0 0x564a477ec580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3128009710 tx=0x7f31280096a0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- 172.21.15.22:0/4252514211 shutdown_connections 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 --2- 172.21.15.22:0/4252514211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ec1a0 0x564a477ec580 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- 172.21.15.22:0/4252514211 >> 172.21.15.22:0/4252514211 conn(0x564a477e68d0 msgr2=0x564a477e6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- 172.21.15.22:0/4252514211 shutdown_connections 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- 172.21.15.22:0/4252514211 wait complete. 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- start start 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 Processor -- start 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ec1a0 0x564a477f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 0x564a477f3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a476ee4a0 con 0x564a477ecac0 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4774bf60 con 0x564a477f56a0 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a477f8490 con 0x564a477ec1a0 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313b78f740 10 client.0 _ll_register_callbacks cb 0x564a47749d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51182/0 (socket says 172.21.15.22:51182) 2024-03-09T00:06:23.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f3131d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ec1a0 0x564a477f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.012+0000 7f313259c640 1 -- 172.21.15.22:0/1750123616 learned_addr learned my addr 172.21.15.22:0/1750123616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f3131d9b640 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 0x564a477f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313259c640 1 -- 172.21.15.22:0/1750123616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ec1a0 msgr2=0x564a477f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313259c640 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ec1a0 0x564a477f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313259c640 1 -- 172.21.15.22:0/1750123616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 msgr2=0x564a477f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313259c640 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 0x564a477f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313259c640 1 -- 172.21.15.22:0/1750123616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a474643f0 con 0x564a477ecac0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313259c640 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f2b40 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f3124012770 tx=0x7f3124012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf6264e6ac49fe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f3119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f3119ffb640 1 -- 172.21.15.22:0/1750123616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124004050 con 0x564a477ecac0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f3119ffb640 1 -- 172.21.15.22:0/1750123616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f31240041f0 con 0x564a477ecac0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f312800fdf0 con 0x564a477ecac0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f3119ffb640 1 -- 172.21.15.22:0/1750123616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3124025050 con 0x564a477ecac0 2024-03-09T00:06:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313b78f740 10 client.19266 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564a4752e7d0 con 0x564a477ecac0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.013+0000 7f3119ffb640 1 -- 172.21.15.22:0/1750123616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f312800fdf0 con 0x564a477ecac0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f3119ffb640 1 -- 172.21.15.22:0/1750123616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3124003a60 con 0x564a477ecac0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f313b78f740 10 client.19266 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f313b78f740 10 client.19266 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f313b78f740 20 client.19266 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f313b78f740 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a478058c0 0x564a47807d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f313b78f740 4 client.19266 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a4775d400 con 0x564a478058c0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f3131d9b640 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a478058c0 0x564a47807d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f3131d9b640 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a478058c0 0x564a47807d00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d4a1f25055da173 server_cookie=f68f6e90ab0df7f6 in_seq=0 out_seq=0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.014+0000 7f3119ffb640 10 client.19266 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f3119ffb640 1 -- 172.21.15.22:0/1750123616 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a4775d400 con 0x564a478058c0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f3119ffb640 10 client.19266 handle_command_reply: tid=0 2024-03-09T00:06:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 client.19266 shutdown 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 msgr2=0x564a477f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f2b40 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f3124012770 tx=0x7f3124012c30 comp rx=0 tx=0).stop 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 shutdown_connections 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a478058c0 0x564a47807d00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 0x564a477f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f3119ffb640 0 client.19266 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ecac0 0x564a477f2b40 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 --2- 172.21.15.22:0/1750123616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ec1a0 0x564a477f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 >> 172.21.15.22:0/1750123616 conn(0x564a477e68d0 msgr2=0x564a477e8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 shutdown_connections 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 1 -- 172.21.15.22:0/1750123616 wait complete. 2024-03-09T00:06:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.015+0000 7f313b78f740 20 client.19266 trim_cache size 0 max 16384 2024-03-09T00:06:23.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 Processor -- start 2024-03-09T00:06:23.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 -- start start 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ecac0 0x564a477ef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477efda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341b2d80 con 0x564a477ec1a0 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341b0920 con 0x564a477ecac0 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313b78f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341ae9e0 con 0x564a477f56a0 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51194/0 (socket says 172.21.15.22:51194) 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 -- 172.21.15.22:0/365806880 learned_addr learned my addr 172.21.15.22:0/365806880 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f313259c640 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ecac0 0x564a477ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 -- 172.21.15.22:0/365806880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 msgr2=0x564a477efda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477efda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 -- 172.21.15.22:0/365806880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ecac0 msgr2=0x564a477ef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ecac0 0x564a477ef860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 -- 172.21.15.22:0/365806880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a477e6700 con 0x564a477ec1a0 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.027+0000 7f3131d9b640 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477f4d30 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3128010010 tx=0x7f31280096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3d555d766c446b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f311bfff640 1 -- 172.21.15.22:0/365806880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128011110 con 0x564a477ec1a0 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f311bfff640 1 -- 172.21.15.22:0/365806880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f312800fdf0 con 0x564a477ec1a0 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f311bfff640 1 -- 172.21.15.22:0/365806880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31280035f0 con 0x564a477ec1a0 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 -- 172.21.15.22:0/365806880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 msgr2=0x564a477f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477f4d30 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3128010010 tx=0x7f31280096a0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 -- 172.21.15.22:0/365806880 shutdown_connections 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477f56a0 0x564a477efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477ecac0 0x564a477ef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 --2- 172.21.15.22:0/365806880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477f4d30 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 -- 172.21.15.22:0/365806880 >> 172.21.15.22:0/365806880 conn(0x564a477e68d0 msgr2=0x564a477f0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 -- 172.21.15.22:0/365806880 shutdown_connections 2024-03-09T00:06:23.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.028+0000 7f313b78f740 1 -- 172.21.15.22:0/365806880 wait complete. 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 -- start start 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 Processor -- start 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ecac0 0x564a477fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 0x564a47864640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341aefe0 con 0x564a477ec1a0 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341b02c0 con 0x564a477f56a0 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341af600 con 0x564a477ecac0 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313b78f740 10 client.0 _ll_register_callbacks cb 0x7f3128009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477fb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51208/0 (socket says 172.21.15.22:51208) 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 -- 172.21.15.22:0/1862956326 learned_addr learned my addr 172.21.15.22:0/1862956326 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f313259c640 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ecac0 0x564a477fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 -- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ecac0 msgr2=0x564a477fbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ecac0 0x564a477fbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 -- 172.21.15.22:0/1862956326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 msgr2=0x564a47864640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 0x564a47864640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 -- 172.21.15.22:0/1862956326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a47864bd0 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3131d9b640 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477fb8d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3128009250 tx=0x7f312802f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d76ccdcef656b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.029+0000 7f3119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f3119ffb640 1 -- 172.21.15.22:0/1862956326 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31280116d0 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f3119ffb640 1 -- 172.21.15.22:0/1862956326 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3128034070 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f312800fdf0 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f3119ffb640 1 -- 172.21.15.22:0/1862956326 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3128044050 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 10 client.19272 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564a47865ed0 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f3119ffb640 1 -- 172.21.15.22:0/1862956326 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f312800fdf0 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f3119ffb640 1 -- 172.21.15.22:0/1862956326 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3128036070 con 0x564a477ec1a0 2024-03-09T00:06:23.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 10 client.19272 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 10 client.19272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 20 client.19272 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a47866340 0x564a47868730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 4 client.19272 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a47757750 con 0x564a47866340 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.030+0000 7f313259c640 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a47866340 0x564a47868730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f3119ffb640 10 client.19272 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313259c640 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a47866340 0x564a47868730 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55cd91b2fefc155a server_cookie=55c4bc84f757ec05 in_seq=0 out_seq=0 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f3119ffb640 1 -- 172.21.15.22:0/1862956326 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a47757750 con 0x564a47866340 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f3119ffb640 10 client.19272 handle_command_reply: tid=0 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 client.19272 shutdown 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 msgr2=0x564a477fb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477fb8d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3128009250 tx=0x7f312802f8b0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 shutdown_connections 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x564a47866340 0x564a47868730 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a477f56a0 0x564a47864640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f3119ffb640 0 client.19272 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a477ecac0 0x564a477fbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 --2- 172.21.15.22:0/1862956326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564a477ec1a0 0x564a477fb8d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 >> 172.21.15.22:0/1862956326 conn(0x564a477e68d0 msgr2=0x564a477e7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 shutdown_connections 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 1 -- 172.21.15.22:0/1862956326 wait complete. 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.031+0000 7f313b78f740 20 client.19272 trim_cache size 0 max 16384 2024-03-09T00:06:23.032 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:23.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f311c0a2970 msgr2=0x7f311c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f311c0a2970 0x7f311c0a4e30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f312c037040 tx=0x7f312c0324b0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313419e630 msgr2=0x7f313419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313419e630 0x7f313419eae0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f312c02c700 tx=0x7f312c032f20 comp rx=0 tx=0).stop 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 shutdown_connections 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f311c0a2970 0x7f311c0a4e30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f313419e630 0x7f313419eae0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31341054b0 0x7f313419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 --2- 172.21.15.22:0/3023923219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31341003e0 0x7f3134199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 >> 172.21.15.22:0/3023923219 conn(0x7f31340773a0 msgr2=0x7f31340fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 shutdown_connections 2024-03-09T00:06:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.032+0000 7f3138ad1640 1 -- 172.21.15.22:0/3023923219 wait complete. 2024-03-09T00:06:23.037 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:23.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 -- 172.21.15.22:0/1137178349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b58100d90 msgr2=0x7f8b58104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/1137178349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b58100d90 0x7f8b58104dd0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c009900 tx=0x7f8b4c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 -- 172.21.15.22:0/1137178349 shutdown_connections 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/1137178349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b581054b0 0x7f8b5810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/1137178349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b58100d90 0x7f8b58104dd0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/1137178349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b581003e0 0x7f8b581007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 -- 172.21.15.22:0/1137178349 >> 172.21.15.22:0/1137178349 conn(0x7f8b580773a0 msgr2=0x7f8b580777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 -- 172.21.15.22:0/1137178349 shutdown_connections 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 -- 172.21.15.22:0/1137178349 wait complete. 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.149+0000 7f8b5d84b640 1 Processor -- start 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b5d84b640 1 -- start start 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b5d84b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b581003e0 0x7f8b58198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b5d84b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b581054b0 0x7f8b58198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b5d84b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b58199450 0x7f8b581a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b5d84b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58110590 con 0x7f8b581003e0 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b5d84b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58110410 con 0x7f8b581054b0 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b5d84b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58110710 con 0x7f8b58199450 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b581003e0 0x7f8b58198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b581003e0 0x7f8b58198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51230/0 (socket says 172.21.15.22:51230) 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 -- 172.21.15.22:0/3817851075 learned_addr learned my addr 172.21.15.22:0/3817851075 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b567fc640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b581054b0 0x7f8b58198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b577fe640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b58199450 0x7f8b581a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 -- 172.21.15.22:0/3817851075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b58199450 msgr2=0x7f8b581a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b58199450 0x7f8b581a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 -- 172.21.15.22:0/3817851075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b581054b0 msgr2=0x7f8b58198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b581054b0 0x7f8b58198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 -- 172.21.15.22:0/3817851075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b581a41d0 con 0x7f8b581003e0 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.150+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b581003e0 0x7f8b58198930 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800e9f0 tx=0x7f8b4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.151+0000 7f8b5c849640 1 -- 172.21.15.22:0/3817851075 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800cd80 con 0x7f8b581003e0 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.151+0000 7f8b5c849640 1 -- 172.21.15.22:0/3817851075 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b48010910 con 0x7f8b581003e0 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.151+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b581a44c0 con 0x7f8b581003e0 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.151+0000 7f8b5c849640 1 -- 172.21.15.22:0/3817851075 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48018990 con 0x7f8b581003e0 2024-03-09T00:06:23.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.151+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b581a4a30 con 0x7f8b581003e0 2024-03-09T00:06:23.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b5c849640 1 -- 172.21.15.22:0/3817851075 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8b48010ac0 con 0x7f8b581003e0 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 Processor -- start 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 -- start start 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401980580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b401989160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b40198ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b401947360 con 0x55b4019896a0 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4018e3ec0 con 0x55b4019801a0 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b6042b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40188cd30 con 0x55b401980ac0 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b40198ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b401989160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b40198ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51234/0 (socket says 172.21.15.22:51234) 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 -- 172.21.15.22:0/2785138413 learned_addr learned my addr 172.21.15.22:0/2785138413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b5c849640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8b400a29f0 0x7f8b400a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b577fe640 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401980580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 -- 172.21.15.22:0/2785138413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 msgr2=0x55b401989160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b401989160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 -- 172.21.15.22:0/2785138413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 msgr2=0x55b401980580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401980580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b577fe640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8b400a29f0 0x7f8b400a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b567fc640 1 -- 172.21.15.22:0/2785138413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40188bbc0 con 0x55b4019896a0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.154+0000 7f8b5c849640 1 -- 172.21.15.22:0/3817851075 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8b480cf520 con 0x7f8b581003e0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.155+0000 7f8b567fc640 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b40198ba90 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c000c00 tx=0x7f8b4c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dd5500087ae250 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.155+0000 7f8b3dffb640 1 -- 172.21.15.22:0/2785138413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c045070 con 0x55b4019896a0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.155+0000 7f8b3dffb640 1 -- 172.21.15.22:0/2785138413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c02c650 con 0x55b4019896a0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.155+0000 7f8b3dffb640 1 -- 172.21.15.22:0/2785138413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c004b40 con 0x55b4019896a0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- 172.21.15.22:0/2785138413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 msgr2=0x55b40198ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b577fe640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8b400a29f0 0x7f8b400a4eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009fd0 tx=0x7f8b50004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b40198ba90 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c000c00 tx=0x7f8b4c009560 comp rx=0 tx=0).stop 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- 172.21.15.22:0/2785138413 shutdown_connections 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b40198ba90 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b401989160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2785138413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401980580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- 172.21.15.22:0/2785138413 >> 172.21.15.22:0/2785138413 conn(0x55b40197a8d0 msgr2=0x55b40197acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- 172.21.15.22:0/2785138413 shutdown_connections 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- 172.21.15.22:0/2785138413 wait complete. 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- start start 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 Processor -- start 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019801a0 0x55b401986640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b401986b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019896a0 0x55b4019870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4018824a0 con 0x55b401980ac0 2024-03-09T00:06:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4018dff60 con 0x55b4019896a0 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.156+0000 7f8b6042b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40198c430 con 0x55b4019801a0 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b6042b740 10 client.0 _ll_register_callbacks cb 0x55b4018ddd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b401986b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019801a0 0x55b401986640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b401986b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51246/0 (socket says 172.21.15.22:51246) 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 -- 172.21.15.22:0/3009521975 learned_addr learned my addr 172.21.15.22:0/3009521975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b577fe640 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019896a0 0x55b4019870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 -- 172.21.15.22:0/3009521975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019801a0 msgr2=0x55b401986640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019801a0 0x55b401986640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 -- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019896a0 msgr2=0x55b4019870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019896a0 0x55b4019870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 -- 172.21.15.22:0/3009521975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4015f83f0 con 0x55b401980ac0 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019801a0 0x55b401986640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b567fc640 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b401986b80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c02c820 tx=0x7f8b4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5166156ac4b23a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b36ffd640 1 -- 172.21.15.22:0/3009521975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c045040 con 0x55b401980ac0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b36ffd640 1 -- 172.21.15.22:0/3009521975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8b4c004ee0 con 0x55b401980ac0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b4c02c650 con 0x55b401980ac0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b36ffd640 1 -- 172.21.15.22:0/3009521975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b4c05a050 con 0x55b401980ac0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b6042b740 10 client.19284 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.157+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b4016c27d0 con 0x55b401980ac0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b36ffd640 1 -- 172.21.15.22:0/3009521975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c02c650 con 0x55b401980ac0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b36ffd640 1 -- 172.21.15.22:0/3009521975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b4c04ab70 con 0x55b401980ac0 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b6042b740 10 client.19284 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b6042b740 10 client.19284 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b6042b740 20 client.19284 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b6042b740 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b401999630 0x55b40199ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b6042b740 4 client.19284 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:23.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4018f1400 con 0x55b401999630 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b401999630 0x55b40199ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b401999630 0x55b40199ba20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c5b5c5452aa4971 server_cookie=a33ce2aa579937c5 in_seq=0 out_seq=0 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.158+0000 7f8b36ffd640 10 client.19284 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b36ffd640 1 -- 172.21.15.22:0/3009521975 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4018f1400 con 0x55b401999630 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b36ffd640 10 client.19284 handle_command_reply: tid=0 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 client.19284 shutdown 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 msgr2=0x55b401986b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b401986b80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c02c820 tx=0x7f8b4c0046c0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 shutdown_connections 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b401999630 0x55b40199ba20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019896a0 0x55b4019870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b401986b80 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b36ffd640 0 client.19284 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:23.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 --2- 172.21.15.22:0/3009521975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019801a0 0x55b401986640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 >> 172.21.15.22:0/3009521975 conn(0x55b40197a8d0 msgr2=0x55b40197c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 shutdown_connections 2024-03-09T00:06:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 1 -- 172.21.15.22:0/3009521975 wait complete. 2024-03-09T00:06:23.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.159+0000 7f8b6042b740 20 client.19284 trim_cache size 0 max 16384 2024-03-09T00:06:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 Processor -- start 2024-03-09T00:06:23.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 -- start start 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401988d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b4019a0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b4019838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581b0ad0 con 0x55b4019896a0 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581ae670 con 0x55b4019801a0 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b6042b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581ac730 con 0x55b401980ac0 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b4019838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b4019838c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51258/0 (socket says 172.21.15.22:51258) 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b577fe640 1 -- 172.21.15.22:0/609177713 learned_addr learned my addr 172.21.15.22:0/609177713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401988d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.171+0000 7f8b567fc640 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b4019a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b577fe640 1 -- 172.21.15.22:0/609177713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 msgr2=0x55b4019a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b577fe640 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b4019a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b577fe640 1 -- 172.21.15.22:0/609177713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 msgr2=0x55b401988d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b577fe640 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401988d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b577fe640 1 -- 172.21.15.22:0/609177713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4019808f0 con 0x55b4019896a0 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b577fe640 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b4019838c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8b50013270 tx=0x7f8b50009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f3dd9400e12407 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b3d7fa640 1 -- 172.21.15.22:0/609177713 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500229d0 con 0x55b4019896a0 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b3d7fa640 1 -- 172.21.15.22:0/609177713 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b50022b70 con 0x55b4019896a0 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b3d7fa640 1 -- 172.21.15.22:0/609177713 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5002a640 con 0x55b4019896a0 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b6042b740 1 -- 172.21.15.22:0/609177713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 msgr2=0x55b4019838c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b6042b740 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b4019838c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8b50013270 tx=0x7f8b50009c80 comp rx=0 tx=0).stop 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b6042b740 1 -- 172.21.15.22:0/609177713 shutdown_connections 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b6042b740 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4019896a0 0x55b4019838c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b6042b740 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b401980ac0 0x55b4019a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b6042b740 1 --2- 172.21.15.22:0/609177713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b401988d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.172+0000 7f8b6042b740 1 -- 172.21.15.22:0/609177713 >> 172.21.15.22:0/609177713 conn(0x55b40197a8d0 msgr2=0x55b40197acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 -- 172.21.15.22:0/609177713 shutdown_connections 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 -- 172.21.15.22:0/609177713 wait complete. 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 -- start start 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 Processor -- start 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b40198f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b40198f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019896a0 0x55b40198fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581acd30 con 0x55b401980ac0 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581ae010 con 0x55b4019801a0 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581ad350 con 0x55b4019896a0 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b6042b740 10 client.0 _ll_register_callbacks cb 0x7f8b500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b40198f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b40198f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b567fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b40198f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51270/0 (socket says 172.21.15.22:51270) 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b577fe640 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019896a0 0x55b40198fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.173+0000 7f8b567fc640 1 -- 172.21.15.22:0/2898658747 learned_addr learned my addr 172.21.15.22:0/2898658747 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b567fc640 1 -- 172.21.15.22:0/2898658747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019896a0 msgr2=0x55b40198fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b567fc640 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019896a0 0x55b40198fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b567fc640 1 -- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 msgr2=0x55b40198f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b567fc640 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b40198f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b567fc640 1 -- 172.21.15.22:0/2898658747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4019efb30 con 0x55b401980ac0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b40198f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b567fc640 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b40198f860 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x55b401987c10 tx=0x7f8b4c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc99821f92327ed server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b36ffd640 1 -- 172.21.15.22:0/2898658747 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c063060 con 0x55b401980ac0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b36ffd640 1 -- 172.21.15.22:0/2898658747 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8b4c0050e0 con 0x55b401980ac0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b50022b70 con 0x55b401980ac0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b36ffd640 1 -- 172.21.15.22:0/2898658747 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b4c06a050 con 0x55b401980ac0 2024-03-09T00:06:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b6042b740 10 client.19290 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b4019f0e10 con 0x55b401980ac0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.174+0000 7f8b36ffd640 1 -- 172.21.15.22:0/2898658747 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b50022b70 con 0x55b401980ac0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b36ffd640 1 -- 172.21.15.22:0/2898658747 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b4c04aa90 con 0x55b401980ac0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 10 client.19290 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 10 client.19290 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 20 client.19290 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b4019f14b0 0x55b4019f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 4 client.19290 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4018eb750 con 0x55b4019f14b0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b4019f14b0 0x55b4019f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b56ffd640 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b4019f14b0 0x55b4019f38d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18bd5e578e0466d2 server_cookie=cca2f4f58a77d1dc in_seq=0 out_seq=0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b36ffd640 10 client.19290 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b36ffd640 1 -- 172.21.15.22:0/2898658747 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4018eb750 con 0x55b4019f14b0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b36ffd640 10 client.19290 handle_command_reply: tid=0 2024-03-09T00:06:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 1 client.19290 shutdown 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 msgr2=0x55b40198f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b40198f860 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x55b401987c10 tx=0x7f8b4c005b50 comp rx=0 tx=0).stop 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.175+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 shutdown_connections 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55b4019f14b0 0x55b4019f38d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b4019896a0 0x55b40198fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b401980ac0 0x55b40198f860 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 1 --2- 172.21.15.22:0/2898658747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4019801a0 0x55b40198f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 >> 172.21.15.22:0/2898658747 conn(0x55b40197a8d0 msgr2=0x55b4019a0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b36ffd640 0 client.19290 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 shutdown_connections 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 1 -- 172.21.15.22:0/2898658747 wait complete. 2024-03-09T00:06:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b6042b740 20 client.19290 trim_cache size 0 max 16384 2024-03-09T00:06:23.178 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8b400a29f0 msgr2=0x7f8b400a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8b400a29f0 0x7f8b400a4eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009fd0 tx=0x7f8b50004ab0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b581003e0 msgr2=0x7f8b58198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b581003e0 0x7f8b58198930 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800e9f0 tx=0x7f8b4800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 shutdown_connections 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8b400a29f0 0x7f8b400a4eb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b58199450 0x7f8b581a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b581054b0 0x7f8b58198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 --2- 172.21.15.22:0/3817851075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b581003e0 0x7f8b58198930 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 >> 172.21.15.22:0/3817851075 conn(0x7f8b580773a0 msgr2=0x7f8b580fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 shutdown_connections 2024-03-09T00:06:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.176+0000 7f8b5d84b640 1 -- 172.21.15.22:0/3817851075 wait complete. 2024-03-09T00:06:23.181 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:23 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1787270742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:23.182 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:23.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:06:23.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 -- 172.21.15.22:0/299109302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 msgr2=0x7f29041028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 --2- 172.21.15.22:0/299109302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f29041028f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009900 tx=0x7f28f802bcd0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 -- 172.21.15.22:0/299109302 shutdown_connections 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 --2- 172.21.15.22:0/299109302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f290410b850 0x7f290410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 --2- 172.21.15.22:0/299109302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2904102e30 0x7f290410b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 --2- 172.21.15.22:0/299109302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f29041028f0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 -- 172.21.15.22:0/299109302 >> 172.21.15.22:0/299109302 conn(0x7f29040773a0 msgr2=0x7f29040777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 -- 172.21.15.22:0/299109302 shutdown_connections 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.297+0000 7f290b153640 1 -- 172.21.15.22:0/299109302 wait complete. 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 Processor -- start 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 -- start start 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f2904100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2904102e30 0x7f29041006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f290410b850 0x7f2904100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904111630 con 0x7f2904102510 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041114b0 con 0x7f2904102e30 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f290b153640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041117b0 con 0x7f290410b850 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f2904100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f2904100160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51288/0 (socket says 172.21.15.22:51288) 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 -- 172.21.15.22:0/2571789809 learned_addr learned my addr 172.21.15.22:0/2571789809 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f29096f0640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f290410b850 0x7f2904100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2903fff640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2904102e30 0x7f29041006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 -- 172.21.15.22:0/2571789809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f290410b850 msgr2=0x7f2904100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f290410b850 0x7f2904100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 -- 172.21.15.22:0/2571789809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2904102e30 msgr2=0x7f29041006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2904102e30 0x7f29041006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.298+0000 7f2908eef640 1 -- 172.21.15.22:0/2571789809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29041a5fd0 con 0x7f2904102510 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.299+0000 7f2908eef640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f2904100160 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f28f8005940 tx=0x7f28f802cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.299+0000 7f2901ffb640 1 -- 172.21.15.22:0/2571789809 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8041070 con 0x7f2904102510 2024-03-09T00:06:23.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.299+0000 7f2901ffb640 1 -- 172.21.15.22:0/2571789809 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f8004450 con 0x7f2904102510 2024-03-09T00:06:23.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.299+0000 7f2901ffb640 1 -- 172.21.15.22:0/2571789809 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8003d80 con 0x7f2904102510 2024-03-09T00:06:23.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.299+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29041a6260 con 0x7f2904102510 2024-03-09T00:06:23.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.299+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29041a6700 con 0x7f2904102510 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.301+0000 7f2901ffb640 1 -- 172.21.15.22:0/2571789809 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f28f8044070 con 0x7f2904102510 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 Processor -- start 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 -- start start 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2901ffb640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28f40a29a0 0x7f28f40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d85160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d87a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc5d43360 con 0x564cc5d7cac0 2024-03-09T00:06:23.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc5cdfec0 con 0x564cc5d856a0 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f290de11740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc5c88d30 con 0x564cc5d7c1a0 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f29096f0640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28f40a29a0 0x7f28f40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d85160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d85160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51300/0 (socket says 172.21.15.22:51300) 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2903fff640 1 -- 172.21.15.22:0/2963658486 learned_addr learned my addr 172.21.15.22:0/2963658486 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2901ffb640 1 -- 172.21.15.22:0/2571789809 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f28f80f0c30 con 0x7f2904102510 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2908eef640 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f29096f0640 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d87a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2903fff640 1 -- 172.21.15.22:0/2963658486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 msgr2=0x564cc5d7c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2903fff640 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d7c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2903fff640 1 -- 172.21.15.22:0/2963658486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 msgr2=0x564cc5d87a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.302+0000 7f2903fff640 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d87a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.303+0000 7f2903fff640 1 -- 172.21.15.22:0/2963658486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cc5c87bc0 con 0x564cc5d7cac0 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.303+0000 7f29096f0640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28f40a29a0 0x7f28f40a4e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009fd0 tx=0x7f28fc013b00 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.303+0000 7f2903fff640 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d85160 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009c90 tx=0x7f28f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c90ca1e394a2266 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.303+0000 7f28eeffd640 1 -- 172.21.15.22:0/2963658486 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f001e070 con 0x564cc5d7cac0 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.303+0000 7f28eeffd640 1 -- 172.21.15.22:0/2963658486 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f0019b10 con 0x564cc5d7cac0 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.303+0000 7f28eeffd640 1 -- 172.21.15.22:0/2963658486 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000e9f0 con 0x564cc5d7cac0 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 -- 172.21.15.22:0/2963658486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 msgr2=0x564cc5d85160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d85160 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009c90 tx=0x7f28f0011810 comp rx=0 tx=0).stop 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 -- 172.21.15.22:0/2963658486 shutdown_connections 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d87a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d85160 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 --2- 172.21.15.22:0/2963658486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d7c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 -- 172.21.15.22:0/2963658486 >> 172.21.15.22:0/2963658486 conn(0x564cc5d768d0 msgr2=0x564cc5d76cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 -- 172.21.15.22:0/2963658486 shutdown_connections 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 -- 172.21.15.22:0/2963658486 wait complete. 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 -- start start 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.304+0000 7f290de11740 1 Processor -- start 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f290de11740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d82640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f290de11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d82b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f290de11740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f290de11740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc5c7e4a0 con 0x564cc5d7cac0 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f290de11740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc5cdbf60 con 0x564cc5d856a0 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f290de11740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cc5d88490 con 0x564cc5d7c1a0 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f290de11740 10 client.0 _ll_register_callbacks cb 0x564cc5cd9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d82b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d82b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51308/0 (socket says 172.21.15.22:51308) 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 -- 172.21.15.22:0/4082016629 learned_addr learned my addr 172.21.15.22:0/4082016629 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f29096f0640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2908eef640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d82640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 -- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 msgr2=0x564cc5d82640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d82640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 -- 172.21.15.22:0/4082016629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 msgr2=0x564cc5d830c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d830c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 -- 172.21.15.22:0/4082016629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cc59f43f0 con 0x564cc5d7cac0 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2908eef640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f29096f0640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f2903fff640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d82b80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f28f0005bc0 tx=0x7f28f000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cea46e4de078cc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f28e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.305+0000 7f28e3fff640 1 -- 172.21.15.22:0/4082016629 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0029020 con 0x564cc5d7cac0 2024-03-09T00:06:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f28e3fff640 1 -- 172.21.15.22:0/4082016629 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f28f0005040 con 0x564cc5d7cac0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28f0019b10 con 0x564cc5d7cac0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f28e3fff640 1 -- 172.21.15.22:0/4082016629 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f0034050 con 0x564cc5d7cac0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 10 client.19302 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564cc5abe7d0 con 0x564cc5d7cac0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f28e3fff640 1 -- 172.21.15.22:0/4082016629 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f0019b10 con 0x564cc5d7cac0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f28e3fff640 1 -- 172.21.15.22:0/4082016629 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f28f00046c0 con 0x564cc5d7cac0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 10 client.19302 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 10 client.19302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 20 client.19302 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5d956f0 0x564cc5d97ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 4 client.19302 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cc5ced400 con 0x564cc5d956f0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.306+0000 7f2908eef640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5d956f0 0x564cc5d97ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.307+0000 7f2908eef640 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5d956f0 0x564cc5d97ae0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77763e440a2a0d server_cookie=6fb756781c402186 in_seq=0 out_seq=0 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.307+0000 7f28e3fff640 10 client.19302 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f28e3fff640 1 -- 172.21.15.22:0/4082016629 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cc5ced400 con 0x564cc5d956f0 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f28e3fff640 10 client.19302 handle_command_reply: tid=0 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 client.19302 shutdown 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 msgr2=0x564cc5d82b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d82b80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f28f0005bc0 tx=0x7f28f000e530 comp rx=0 tx=0).stop 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 shutdown_connections 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5d956f0 0x564cc5d97ae0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d82b80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f28e3fff640 0 client.19302 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 --2- 172.21.15.22:0/4082016629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7c1a0 0x564cc5d82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.308+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 >> 172.21.15.22:0/4082016629 conn(0x564cc5d768d0 msgr2=0x564cc5d78620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.309+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 shutdown_connections 2024-03-09T00:06:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.309+0000 7f290de11740 1 -- 172.21.15.22:0/4082016629 wait complete. 2024-03-09T00:06:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.309+0000 7f290de11740 20 client.19302 trim_cache size 0 max 16384 2024-03-09T00:06:23.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 Processor -- start 2024-03-09T00:06:23.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 -- start start 2024-03-09T00:06:23.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7c1a0 0x564cc5d84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7cac0 0x564cc5d7cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d9c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041b2e20 con 0x564cc5d7c1a0 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041b09c0 con 0x564cc5d856a0 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.321+0000 7f290de11740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041aea80 con 0x564cc5d7cac0 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7c1a0 0x564cc5d84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f29096f0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d9c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7c1a0 0x564cc5d84d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51322/0 (socket says 172.21.15.22:51322) 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2903fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7cac0 0x564cc5d7cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 -- 172.21.15.22:0/2664442643 learned_addr learned my addr 172.21.15.22:0/2664442643 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 -- 172.21.15.22:0/2664442643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7cac0 msgr2=0x564cc5d7cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7cac0 0x564cc5d7cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 -- 172.21.15.22:0/2664442643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 msgr2=0x564cc5d9c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d9c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 -- 172.21.15.22:0/2664442643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cc5d7c8f0 con 0x564cc5d7c1a0 2024-03-09T00:06:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f29096f0640 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d9c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f2908eef640 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7c1a0 0x564cc5d84d70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f28f80bccf0 tx=0x7f28f802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b396c2120ee6149b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f28ee7fc640 1 -- 172.21.15.22:0/2664442643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f804b040 con 0x564cc5d7c1a0 2024-03-09T00:06:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f28ee7fc640 1 -- 172.21.15.22:0/2664442643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f8004c80 con 0x564cc5d7c1a0 2024-03-09T00:06:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.322+0000 7f28ee7fc640 1 -- 172.21.15.22:0/2664442643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f80c56e0 con 0x564cc5d7c1a0 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 -- 172.21.15.22:0/2664442643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7c1a0 msgr2=0x564cc5d84d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7c1a0 0x564cc5d84d70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f28f80bccf0 tx=0x7f28f802bcd0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 -- 172.21.15.22:0/2664442643 shutdown_connections 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d856a0 0x564cc5d9c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d7cac0 0x564cc5d7cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 --2- 172.21.15.22:0/2664442643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7c1a0 0x564cc5d84d70 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 -- 172.21.15.22:0/2664442643 >> 172.21.15.22:0/2664442643 conn(0x564cc5d768d0 msgr2=0x564cc5d97ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 -- 172.21.15.22:0/2664442643 shutdown_connections 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 -- 172.21.15.22:0/2664442643 wait complete. 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.323+0000 7f290de11740 1 -- start start 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 1 Processor -- start 2024-03-09T00:06:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d7c1a0 0x564cc5d8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d856a0 0x564cc5d8a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041af080 con 0x564cc5d7cac0 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041b0360 con 0x564cc5d7c1a0 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041af6a0 con 0x564cc5d856a0 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f290de11740 10 client.0 _ll_register_callbacks cb 0x7f28f00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d8c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51328/0 (socket says 172.21.15.22:51328) 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2908eef640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d7c1a0 0x564cc5d8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 -- 172.21.15.22:0/1634410699 learned_addr learned my addr 172.21.15.22:0/1634410699 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f29096f0640 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d856a0 0x564cc5d8a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 -- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d856a0 msgr2=0x564cc5d8a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d856a0 0x564cc5d8a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 -- 172.21.15.22:0/1634410699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d7c1a0 msgr2=0x564cc5d8c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d7c1a0 0x564cc5d8c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2903fff640 1 -- 172.21.15.22:0/1634410699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cc5d8b000 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.324+0000 7f2908eef640 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d7c1a0 0x564cc5d8c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f2903fff640 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d8c8e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f28f000e5a0 tx=0x7f28f0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36c9dcfd22bd03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f28e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f28e3fff640 1 -- 172.21.15.22:0/1634410699 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0029040 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28f8004c80 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f28e3fff640 1 -- 172.21.15.22:0/1634410699 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f28f0019c50 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f28e3fff640 1 -- 172.21.15.22:0/1634410699 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f001e040 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 10 client.19308 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564cc5dec530 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f28e3fff640 1 -- 172.21.15.22:0/1634410699 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f8004c80 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f28e3fff640 1 -- 172.21.15.22:0/1634410699 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f28f0004480 con 0x564cc5d7cac0 2024-03-09T00:06:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 10 client.19308 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 10 client.19308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 20 client.19308 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5decbc0 0x564cc5deefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 4 client.19308 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.325+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564cc5ce7750 con 0x564cc5decbc0 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.326+0000 7f2908eef640 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5decbc0 0x564cc5deefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.326+0000 7f28e3fff640 10 client.19308 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.326+0000 7f2908eef640 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5decbc0 0x564cc5deefe0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae4f5a94fe5d145 server_cookie=1ce2c5fe64ded659 in_seq=0 out_seq=0 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f28e3fff640 1 -- 172.21.15.22:0/1634410699 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cc5ce7750 con 0x564cc5decbc0 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f28e3fff640 10 client.19308 handle_command_reply: tid=0 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 client.19308 shutdown 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 msgr2=0x564cc5d8c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d8c8e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f28f000e5a0 tx=0x7f28f0004c80 comp rx=0 tx=0).stop 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 shutdown_connections 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564cc5decbc0 0x564cc5deefe0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cc5d856a0 0x564cc5d8a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564cc5d7cac0 0x564cc5d8c8e0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f28e3fff640 0 client.19308 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 --2- 172.21.15.22:0/1634410699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cc5d7c1a0 0x564cc5d8c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 >> 172.21.15.22:0/1634410699 conn(0x564cc5d768d0 msgr2=0x564cc5d7cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 shutdown_connections 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 1 -- 172.21.15.22:0/1634410699 wait complete. 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.327+0000 7f290de11740 20 client.19308 trim_cache size 0 max 16384 2024-03-09T00:06:23.328 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28f40a29a0 msgr2=0x7f28f40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28f40a29a0 0x7f28f40a4e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009fd0 tx=0x7f28fc013b00 comp rx=0 tx=0).stop 2024-03-09T00:06:23.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 msgr2=0x7f2904100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f2904100160 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f28f8005940 tx=0x7f28f802cbd0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 shutdown_connections 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f28f40a29a0 0x7f28f40a4e60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f290410b850 0x7f2904100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2904102e30 0x7f29041006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 --2- 172.21.15.22:0/2571789809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2904102510 0x7f2904100160 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 >> 172.21.15.22:0/2571789809 conn(0x7f29040773a0 msgr2=0x7f29040fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 shutdown_connections 2024-03-09T00:06:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:23.328+0000 7f290b153640 1 -- 172.21.15.22:0/2571789809 wait complete. 2024-03-09T00:06:23.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:23 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1787270742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:23.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:23 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1787270742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:23.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1182212 9.47 MB/sec execute 511 sec latency 19.517 ms 2024-03-09T00:06:24.333 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:06:24.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:06:24.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:06:24.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:24 smithi192 ceph-mon[35995]: pgmap v374: 129 pgs: 129 active+clean; 87 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-09T00:06:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 -- 172.21.15.22:0/2915979003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081065c0 msgr2=0x7f470810e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 --2- 172.21.15.22:0/2915979003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081065c0 0x7f470810e190 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4700009ca0 tx=0x7f470002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 -- 172.21.15.22:0/2915979003 shutdown_connections 2024-03-09T00:06:24.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 --2- 172.21.15.22:0/2915979003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081065c0 0x7f470810e190 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 --2- 172.21.15.22:0/2915979003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4708101ea0 0x7f4708105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 --2- 172.21.15.22:0/2915979003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47081014f0 0x7f47081018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 -- 172.21.15.22:0/2915979003 >> 172.21.15.22:0/2915979003 conn(0x7f47080786f0 msgr2=0x7f4708078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 -- 172.21.15.22:0/2915979003 shutdown_connections 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.446+0000 7f470fa59640 1 -- 172.21.15.22:0/2915979003 wait complete. 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 Processor -- start 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 -- start start 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47081014f0 0x7f470819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47081065c0 0x7f470819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081979a0 0x7f4708197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708111680 con 0x7f47081979a0 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708111500 con 0x7f47081014f0 2024-03-09T00:06:24.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470fa59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708111800 con 0x7f47081065c0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081979a0 0x7f4708197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470d7f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47081014f0 0x7f470819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081979a0 0x7f4708197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51342/0 (socket says 172.21.15.22:51342) 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 -- 172.21.15.22:0/3215410639 learned_addr learned my addr 172.21.15.22:0/3215410639 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470cff4640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47081065c0 0x7f470819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 -- 172.21.15.22:0/3215410639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47081065c0 msgr2=0x7f470819dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47081065c0 0x7f470819dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 -- 172.21.15.22:0/3215410639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47081014f0 msgr2=0x7f470819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47081014f0 0x7f470819d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.447+0000 7f470dff6640 1 -- 172.21.15.22:0/3215410639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47081984c0 con 0x7f47081979a0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.448+0000 7f470dff6640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081979a0 0x7f4708197e50 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f470002c700 tx=0x7f4700032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.448+0000 7f46fe7fc640 1 -- 172.21.15.22:0/3215410639 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700041020 con 0x7f47081979a0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.448+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4708198750 con 0x7f47081979a0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.448+0000 7f46fe7fc640 1 -- 172.21.15.22:0/3215410639 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4700034cd0 con 0x7f47081979a0 2024-03-09T00:06:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.448+0000 7f46fe7fc640 1 -- 172.21.15.22:0/3215410639 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700039040 con 0x7f47081979a0 2024-03-09T00:06:24.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.448+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47081a4950 con 0x7f47081979a0 2024-03-09T00:06:24.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.450+0000 7f46fe7fc640 1 -- 172.21.15.22:0/3215410639 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4700044020 con 0x7f47081979a0 2024-03-09T00:06:24.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.450+0000 7f4712637740 1 Processor -- start 2024-03-09T00:06:24.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.450+0000 7f4712637740 1 -- start start 2024-03-09T00:06:24.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f4712637740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f4712637740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b2350 0x557e8e1bf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f4712637740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f4712637740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e8e115b40 con 0x557e8e1b2350 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f4712637740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e8e0bed80 con 0x557e8e1bf690 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f4712637740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e8e111be0 con 0x557e8e1b1a30 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f46fe7fc640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f40a2970 0x7f46f40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b2350 0x557e8e1bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b2350 0x557e8e1bf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51356/0 (socket says 172.21.15.22:51356) 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470dff6640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f40a2970 0x7f46f40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470cff4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 -- 172.21.15.22:0/1250089419 learned_addr learned my addr 172.21.15.22:0/1250089419 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470cff4640 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f46fe7fc640 1 -- 172.21.15.22:0/3215410639 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f47000f0e30 con 0x7f47081979a0 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 -- 172.21.15.22:0/1250089419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 msgr2=0x557e8e1b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 -- 172.21.15.22:0/1250089419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 msgr2=0x557e8e1c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 -- 172.21.15.22:0/1250089419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e8def27e0 con 0x557e8e1b2350 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470cff4640 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470dff6640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f40a2970 0x7f46f40a4e30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47000bce40 tx=0x7f470008ed70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f470d7f5640 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b2350 0x557e8e1bf150 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4704009c90 tx=0x7f4704011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3735659ec87078e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f46f37fe640 1 -- 172.21.15.22:0/1250089419 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704005220 con 0x557e8e1b2350 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f46f37fe640 1 -- 172.21.15.22:0/1250089419 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4704004c80 con 0x557e8e1b2350 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.451+0000 7f46f37fe640 1 -- 172.21.15.22:0/1250089419 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704019710 con 0x557e8e1b2350 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 -- 172.21.15.22:0/1250089419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b2350 msgr2=0x557e8e1bf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b2350 0x557e8e1bf150 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4704009c90 tx=0x7f4704011810 comp rx=0 tx=0).stop 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 -- 172.21.15.22:0/1250089419 shutdown_connections 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b2350 0x557e8e1bf150 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 --2- 172.21.15.22:0/1250089419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 -- 172.21.15.22:0/1250089419 >> 172.21.15.22:0/1250089419 conn(0x557e8e1ac4b0 msgr2=0x557e8e1ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 -- 172.21.15.22:0/1250089419 shutdown_connections 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.452+0000 7f4712637740 1 -- 172.21.15.22:0/1250089419 wait complete. 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 -- start start 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 Processor -- start 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e8e1c24d0 con 0x557e8e1bf690 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e8e0d21c0 con 0x557e8e1b2350 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e8e1c2650 con 0x557e8e1b1a30 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f4712637740 10 client.0 _ll_register_callbacks cb 0x557e8e0c17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:24.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470d7f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1bf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58952/0 (socket says 172.21.15.22:58952) 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 -- 172.21.15.22:0/3661215351 learned_addr learned my addr 172.21.15.22:0/3661215351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 -- 172.21.15.22:0/3661215351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 msgr2=0x557e8e1bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 -- 172.21.15.22:0/3661215351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 msgr2=0x557e8e1bc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.453+0000 7f470cff4640 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1bc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f470cff4640 1 -- 172.21.15.22:0/3661215351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e8df52f70 con 0x557e8e1bf690 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f470d7f5640 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f470cff4640 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1bca40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f470806ab90 tx=0x7f46f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cc84f68d2c5cea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f46f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f46f0ff9640 1 -- 172.21.15.22:0/3661215351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004050 con 0x557e8e1bf690 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f46f0ff9640 1 -- 172.21.15.22:0/3661215351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f46f80041f0 con 0x557e8e1bf690 2024-03-09T00:06:24.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4704004c80 con 0x557e8e1bf690 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f46f0ff9640 1 -- 172.21.15.22:0/3661215351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f802c050 con 0x557e8e1bf690 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 10 client.19320 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557e8ddea170 con 0x557e8e1bf690 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f46f0ff9640 1 -- 172.21.15.22:0/3661215351 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4704004c80 con 0x557e8e1bf690 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f46f0ff9640 1 -- 172.21.15.22:0/3661215351 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f46f8020040 con 0x557e8e1bf690 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 10 client.19320 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 10 client.19320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 20 client.19320 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e1cf8c0 0x557e8e1d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 4 client.19320 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.454+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e8e0eb960 con 0x557e8e1cf8c0 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.455+0000 7f470d7f5640 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e1cf8c0 0x557e8e1d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.455+0000 7f470d7f5640 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e1cf8c0 0x557e8e1d1cb0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8184818c3e9aa89 server_cookie=185db0de7c975676 in_seq=0 out_seq=0 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.455+0000 7f46f0ff9640 10 client.19320 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f46f0ff9640 1 -- 172.21.15.22:0/3661215351 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e8e0eb960 con 0x557e8e1cf8c0 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f46f0ff9640 10 client.19320 handle_command_reply: tid=0 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 client.19320 shutdown 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 msgr2=0x557e8e1bca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1bca40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f470806ab90 tx=0x7f46f80046c0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 shutdown_connections 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e1cf8c0 0x557e8e1d1cb0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1bca40 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f46f0ff9640 0 client.19320 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 --2- 172.21.15.22:0/3661215351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 >> 172.21.15.22:0/3661215351 conn(0x557e8e1ac4b0 msgr2=0x557e8e1ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 shutdown_connections 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 1 -- 172.21.15.22:0/3661215351 wait complete. 2024-03-09T00:06:24.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.457+0000 7f4712637740 20 client.19320 trim_cache size 0 max 16384 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.472+0000 7f4712637740 1 Processor -- start 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.472+0000 7f4712637740 1 -- start start 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f4712637740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b1a30 0x557e8e1bb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f4712637740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b2350 0x557e8e1beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f4712637740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f4712637740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081aea00 con 0x557e8e1b1a30 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f4712637740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081ac5a0 con 0x557e8e1bf690 2024-03-09T00:06:24.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f4712637740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081aa660 con 0x557e8e1b2350 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b1a30 0x557e8e1bb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b1a30 0x557e8e1bb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51376/0 (socket says 172.21.15.22:51376) 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 -- 172.21.15.22:0/2922104351 learned_addr learned my addr 172.21.15.22:0/2922104351 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470d7f5640 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b2350 0x557e8e1beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 -- 172.21.15.22:0/2922104351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b2350 msgr2=0x557e8e1beb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b2350 0x557e8e1beb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 -- 172.21.15.22:0/2922104351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 msgr2=0x557e8e1bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 -- 172.21.15.22:0/2922104351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e8e1b2180 con 0x557e8e1b1a30 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f470cff4640 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b1a30 0x557e8e1bb410 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f46f8011fd0 tx=0x7f46f8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e02cd235bf720c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f46f2ffd640 1 -- 172.21.15.22:0/2922104351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8035020 con 0x557e8e1b1a30 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f46f2ffd640 1 -- 172.21.15.22:0/2922104351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46f8009b70 con 0x557e8e1b1a30 2024-03-09T00:06:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.473+0000 7f46f2ffd640 1 -- 172.21.15.22:0/2922104351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80042a0 con 0x557e8e1b1a30 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 -- 172.21.15.22:0/2922104351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b1a30 msgr2=0x557e8e1bb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b1a30 0x557e8e1bb410 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f46f8011fd0 tx=0x7f46f8011a90 comp rx=0 tx=0).stop 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 -- 172.21.15.22:0/2922104351 shutdown_connections 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1bf690 0x557e8e1bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b2350 0x557e8e1beb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 --2- 172.21.15.22:0/2922104351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1b1a30 0x557e8e1bb410 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 -- 172.21.15.22:0/2922104351 >> 172.21.15.22:0/2922104351 conn(0x557e8e1ac4b0 msgr2=0x557e8e1bad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 -- 172.21.15.22:0/2922104351 shutdown_connections 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 -- 172.21.15.22:0/2922104351 wait complete. 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 -- start start 2024-03-09T00:06:24.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.474+0000 7f4712637740 1 Processor -- start 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f4712637740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f4712637740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f4712637740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f4712637740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081aac60 con 0x557e8e1bf690 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f4712637740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081abf40 con 0x557e8e1b2350 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f4712637740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47081ab280 con 0x557e8e1b1a30 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f4712637740 10 client.0 _ll_register_callbacks cb 0x7f47040097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:24.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470cff4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470d7f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470cff4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58966/0 (socket says 172.21.15.22:58966) 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470cff4640 1 -- 172.21.15.22:0/2295252527 learned_addr learned my addr 172.21.15.22:0/2295252527 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470cff4640 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470d7f5640 1 -- 172.21.15.22:0/2295252527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 msgr2=0x557e8e1c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470d7f5640 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470d7f5640 1 -- 172.21.15.22:0/2295252527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 msgr2=0x557e8e1c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470d7f5640 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470d7f5640 1 -- 172.21.15.22:0/2295252527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e8e225d90 con 0x557e8e1b2350 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f470d7f5640 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1c31a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4704005890 tx=0x7f4704005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77a603ca8a2d89f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.475+0000 7f46f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f46f0ff9640 1 -- 172.21.15.22:0/2295252527 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470402e070 con 0x557e8e1b2350 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f46f0ff9640 1 -- 172.21.15.22:0/2295252527 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f470402a1d0 con 0x557e8e1b2350 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46f8009b70 con 0x557e8e1b2350 2024-03-09T00:06:24.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f46f0ff9640 1 -- 172.21.15.22:0/2295252527 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f470403a050 con 0x557e8e1b2350 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 10 client.28640 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557e8e226f00 con 0x557e8e1b2350 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f46f0ff9640 1 -- 172.21.15.22:0/2295252527 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46f8009b70 con 0x557e8e1b2350 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f46f0ff9640 1 -- 172.21.15.22:0/2295252527 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4704003780 con 0x557e8e1b2350 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 10 client.28640 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 10 client.28640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 20 client.28640 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e227470 0x557e8e229890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 4 client.28640 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e8e0d7080 con 0x557e8e227470 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.476+0000 7f470cff4640 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e227470 0x557e8e229890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.477+0000 7f46f0ff9640 10 client.28640 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.477+0000 7f470cff4640 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e227470 0x557e8e229890 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a48d7775f876ad server_cookie=3cad76ff4e09e5e5 in_seq=0 out_seq=0 2024-03-09T00:06:24.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f46f0ff9640 1 -- 172.21.15.22:0/2295252527 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e8e0d7080 con 0x557e8e227470 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f46f0ff9640 10 client.28640 handle_command_reply: tid=0 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 client.28640 shutdown 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 msgr2=0x557e8e1c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1c31a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4704005890 tx=0x7f4704005650 comp rx=0 tx=0).stop 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 shutdown_connections 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557e8e227470 0x557e8e229890 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f46f0ff9640 0 client.28640 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e8e1bf690 0x557e8e1c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e8e1b2350 0x557e8e1c31a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 --2- 172.21.15.22:0/2295252527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e8e1b1a30 0x557e8e1c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 >> 172.21.15.22:0/2295252527 conn(0x557e8e1ac4b0 msgr2=0x557e8e1bb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 shutdown_connections 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.478+0000 7f4712637740 1 -- 172.21.15.22:0/2295252527 wait complete. 2024-03-09T00:06:24.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f4712637740 20 client.28640 trim_cache size 0 max 16384 2024-03-09T00:06:24.479 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:06:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "a51e0b9c-ab67-4758-bb85-11fc2a7af8d7", 2024-03-09T00:06:24.479 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:06:24.479 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f40a2970 msgr2=0x7f46f40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f40a2970 0x7f46f40a4e30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47000bce40 tx=0x7f470008ed70 comp rx=0 tx=0).stop 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081979a0 msgr2=0x7f4708197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081979a0 0x7f4708197e50 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f470002c700 tx=0x7f4700032f20 comp rx=0 tx=0).stop 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 shutdown_connections 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f40a2970 0x7f46f40a4e30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47081979a0 0x7f4708197e50 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47081065c0 0x7f470819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 --2- 172.21.15.22:0/3215410639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47081014f0 0x7f470819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 >> 172.21.15.22:0/3215410639 conn(0x7f47080786f0 msgr2=0x7f47080ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 shutdown_connections 2024-03-09T00:06:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.479+0000 7f470fa59640 1 -- 172.21.15.22:0/3215410639 wait complete. 2024-03-09T00:06:24.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:24 smithi022 ceph-mon[31894]: pgmap v374: 129 pgs: 129 active+clean; 87 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-09T00:06:24.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:06:24.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:06:24.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:24 smithi062 ceph-mon[36129]: pgmap v374: 129 pgs: 129 active+clean; 87 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-09T00:06:24.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.618+0000 7f3e80929640 1 -- 172.21.15.22:0/1880098369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 msgr2=0x7f3e7c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.618+0000 7f3e80929640 1 --2- 172.21.15.22:0/1880098369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 0x7f3e7c1007b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009900 tx=0x7f3e6c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:24.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.619+0000 7f3e80929640 1 -- 172.21.15.22:0/1880098369 shutdown_connections 2024-03-09T00:06:24.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.619+0000 7f3e80929640 1 --2- 172.21.15.22:0/1880098369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7c1054a0 0x7f3e7c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.619+0000 7f3e80929640 1 --2- 172.21.15.22:0/1880098369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c100d80 0x7f3e7c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.619+0000 7f3e80929640 1 --2- 172.21.15.22:0/1880098369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 0x7f3e7c1007b0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.619+0000 7f3e80929640 1 -- 172.21.15.22:0/1880098369 >> 172.21.15.22:0/1880098369 conn(0x7f3e7c0772f0 msgr2=0x7f3e7c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.619+0000 7f3e80929640 1 -- 172.21.15.22:0/1880098369 shutdown_connections 2024-03-09T00:06:24.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.619+0000 7f3e80929640 1 -- 172.21.15.22:0/1880098369 wait complete. 2024-03-09T00:06:24.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.621+0000 7f3e80929640 1 Processor -- start 2024-03-09T00:06:24.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.621+0000 7f3e80929640 1 -- start start 2024-03-09T00:06:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.621+0000 7f3e80929640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 0x7f3e7c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e80929640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7c1054a0 0x7f3e7c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e80929640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c196660 0x7f3e7c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e80929640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1105a0 con 0x7f3e7c1003d0 2024-03-09T00:06:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e80929640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c110420 con 0x7f3e7c1054a0 2024-03-09T00:06:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e80929640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c110720 con 0x7f3e7c196660 2024-03-09T00:06:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7c1054a0 0x7f3e7c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7c1054a0 0x7f3e7c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33948/0 (socket says 172.21.15.22:33948) 2024-03-09T00:06:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e71d9b640 1 -- 172.21.15.22:0/763195997 learned_addr learned my addr 172.21.15.22:0/763195997 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.622+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c196660 0x7f3e7c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 0x7f3e7c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7ad9d640 1 -- 172.21.15.22:0/763195997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7c1054a0 msgr2=0x7f3e7c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7c1054a0 0x7f3e7c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7ad9d640 1 -- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 msgr2=0x7f3e7c19c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 0x7f3e7c19c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7ad9d640 1 -- 172.21.15.22:0/763195997 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c197260 con 0x7f3e7c196660 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 0x7f3e7c19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c196660 0x7f3e7c196ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e74007bf0 tx=0x7f3e7400e420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e73fff640 1 -- 172.21.15.22:0/763195997 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74007e00 con 0x7f3e7c196660 2024-03-09T00:06:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c068450 con 0x7f3e7c196660 2024-03-09T00:06:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.623+0000 7f3e73fff640 1 -- 172.21.15.22:0/763195997 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e7400eb30 con 0x7f3e7c196660 2024-03-09T00:06:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.624+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c068950 con 0x7f3e7c196660 2024-03-09T00:06:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.624+0000 7f3e73fff640 1 -- 172.21.15.22:0/763195997 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74007e00 con 0x7f3e7c196660 2024-03-09T00:06:24.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.627+0000 7f3e835e5740 1 Processor -- start 2024-03-09T00:06:24.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.627+0000 7f3e73fff640 1 -- 172.21.15.22:0/763195997 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3e7400a440 con 0x7f3e7c196660 2024-03-09T00:06:24.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e835e5740 1 -- start start 2024-03-09T00:06:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b813520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b825320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b8186e0 0x564e7b827870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e835e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7b776e00 con 0x564e7b8186e0 2024-03-09T00:06:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e835e5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7b71fb10 con 0x564e7b813af0 2024-03-09T00:06:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e835e5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7b772ea0 con 0x564e7b813140 2024-03-09T00:06:24.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b8186e0 0x564e7b827870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e7a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b825320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e7a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b825320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33964/0 (socket says 172.21.15.22:33964) 2024-03-09T00:06:24.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b8186e0 0x564e7b827870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51400/0 (socket says 172.21.15.22:51400) 2024-03-09T00:06:24.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e7a59c640 1 -- 172.21.15.22:0/2286883141 learned_addr learned my addr 172.21.15.22:0/2286883141 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.628+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b813520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e7a59c640 1 -- 172.21.15.22:0/2286883141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 msgr2=0x564e7b813520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b813520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e7a59c640 1 -- 172.21.15.22:0/2286883141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b8186e0 msgr2=0x564e7b827870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b8186e0 0x564e7b827870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e7a59c640 1 -- 172.21.15.22:0/2286883141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e7b5557d0 con 0x564e7b813af0 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e73fff640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e680a2bf0 0x7f3e680a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e73fff640 1 -- 172.21.15.22:0/763195997 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3e740d0a00 con 0x7f3e7c196660 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b825320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009690 tx=0x7f3e6c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75c9fc14689b03e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e680a2bf0 0x7f3e680a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.629+0000 7f3e70d99640 1 -- 172.21.15.22:0/2286883141 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c054020 con 0x564e7b813af0 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e70d99640 1 -- 172.21.15.22:0/2286883141 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e6c004250 con 0x564e7b813af0 2024-03-09T00:06:24.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e70d99640 1 -- 172.21.15.22:0/2286883141 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c003bf0 con 0x564e7b813af0 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e835e5740 1 -- 172.21.15.22:0/2286883141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 msgr2=0x564e7b825320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e680a2bf0 0x7f3e680a50b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x564e7b80f2b0 tx=0x7f3e64009530 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b825320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009690 tx=0x7f3e6c000bc0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e835e5740 1 -- 172.21.15.22:0/2286883141 shutdown_connections 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b8186e0 0x564e7b827870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b825320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2286883141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b813520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.630+0000 7f3e835e5740 1 -- 172.21.15.22:0/2286883141 >> 172.21.15.22:0/2286883141 conn(0x564e7b80de90 msgr2=0x564e7b80e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.631+0000 7f3e835e5740 1 -- 172.21.15.22:0/2286883141 shutdown_connections 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.631+0000 7f3e835e5740 1 -- 172.21.15.22:0/2286883141 wait complete. 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.631+0000 7f3e835e5740 1 -- start start 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.631+0000 7f3e835e5740 1 Processor -- start 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.631+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813140 0x564e7b81c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b8195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b819ae0 0x564e7b819f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e835e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7b8281e0 con 0x564e7b813af0 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e835e5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7b7330a0 con 0x564e7b813140 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e835e5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7b828360 con 0x564e7b819ae0 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e835e5740 10 client.0 _ll_register_callbacks cb 0x564e7b732a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e7a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813140 0x564e7b81c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b8195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e7ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b819ae0 0x564e7b819f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b8195a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51406/0 (socket says 172.21.15.22:51406) 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 -- 172.21.15.22:0/667606797 learned_addr learned my addr 172.21.15.22:0/667606797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 -- 172.21.15.22:0/667606797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b819ae0 msgr2=0x564e7b819f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b819ae0 0x564e7b819f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 -- 172.21.15.22:0/667606797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813140 msgr2=0x564e7b81c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813140 0x564e7b81c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 -- 172.21.15.22:0/667606797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e7b48b3f0 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b8195a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400ec70 tx=0x7f3e64005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e602c991ea0e8b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e61ffb640 1 -- 172.21.15.22:0/667606797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6402e060 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.632+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e6c004250 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e61ffb640 1 -- 172.21.15.22:0/667606797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3e640293f0 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e61ffb640 1 -- 172.21.15.22:0/667606797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e64035050 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 10 client.19335 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564e7b44b0f0 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e61ffb640 1 -- 172.21.15.22:0/667606797 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e6c004250 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e61ffb640 1 -- 172.21.15.22:0/667606797 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e64028070 con 0x564e7b813af0 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 10 client.19335 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 10 client.19335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 20 client.19335 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:24.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b834fe0 0x564e7b8373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 4 client.19335 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:24.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e7b784340 con 0x564e7b834fe0 2024-03-09T00:06:24.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.633+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b834fe0 0x564e7b8373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.634+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b834fe0 0x564e7b8373d0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8253a8c67bc2fe7 server_cookie=d9940257a1ff251c in_seq=0 out_seq=0 2024-03-09T00:06:24.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.634+0000 7f3e61ffb640 10 client.19335 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.637+0000 7f3e61ffb640 1 -- 172.21.15.22:0/667606797 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e7b784340 con 0x564e7b834fe0 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.637+0000 7f3e61ffb640 10 client.19335 handle_command_reply: tid=0 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.637+0000 7f3e835e5740 1 client.19335 shutdown 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.637+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 msgr2=0x564e7b8195a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.637+0000 7f3e835e5740 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b8195a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400ec70 tx=0x7f3e64005f90 comp rx=0 tx=0).stop 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 shutdown_connections 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b834fe0 0x564e7b8373d0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e61ffb640 0 client.19335 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b819ae0 0x564e7b819f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b8195a0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 --2- 172.21.15.22:0/667606797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813140 0x564e7b81c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 >> 172.21.15.22:0/667606797 conn(0x564e7b80de90 msgr2=0x564e7b80fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 shutdown_connections 2024-03-09T00:06:24.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 1 -- 172.21.15.22:0/667606797 wait complete. 2024-03-09T00:06:24.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.638+0000 7f3e835e5740 20 client.19335 trim_cache size 0 max 16384 2024-03-09T00:06:24.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 Processor -- start 2024-03-09T00:06:24.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 -- start start 2024-03-09T00:06:24.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b81bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b81c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b819ae0 0x564e7b831090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1a5fe0 con 0x564e7b819ae0 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1a3b80 con 0x564e7b813af0 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e835e5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c071e10 con 0x564e7b813140 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b819ae0 0x564e7b831090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b819ae0 0x564e7b831090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51414/0 (socket says 172.21.15.22:51414) 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 -- 172.21.15.22:0/2413578252 learned_addr learned my addr 172.21.15.22:0/2413578252 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b81c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b81bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 -- 172.21.15.22:0/2413578252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 msgr2=0x564e7b81bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b81bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 -- 172.21.15.22:0/2413578252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 msgr2=0x564e7b81c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b81c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.653+0000 7f3e7ad9d640 1 -- 172.21.15.22:0/2413578252 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e7b83c5c0 con 0x564e7b819ae0 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b819ae0 0x564e7b831090 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3e740cdd50 tx=0x7f3e7409caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8920585730906659 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e63fff640 1 -- 172.21.15.22:0/2413578252 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740b5070 con 0x564e7b819ae0 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e63fff640 1 -- 172.21.15.22:0/2413578252 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e7400ae20 con 0x564e7b819ae0 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e63fff640 1 -- 172.21.15.22:0/2413578252 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740cdf70 con 0x564e7b819ae0 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e835e5740 1 -- 172.21.15.22:0/2413578252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b819ae0 msgr2=0x564e7b831090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b819ae0 0x564e7b831090 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3e740cdd50 tx=0x7f3e7409caa0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e835e5740 1 -- 172.21.15.22:0/2413578252 shutdown_connections 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b819ae0 0x564e7b831090 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b813af0 0x564e7b81c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2413578252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b81bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.654+0000 7f3e835e5740 1 -- 172.21.15.22:0/2413578252 >> 172.21.15.22:0/2413578252 conn(0x564e7b80de90 msgr2=0x564e7b827320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 -- 172.21.15.22:0/2413578252 shutdown_connections 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 -- 172.21.15.22:0/2413578252 wait complete. 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 -- start start 2024-03-09T00:06:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 Processor -- start 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b82c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b828b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b819ae0 0x564e7b829070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c072410 con 0x564e7b813af0 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1a3520 con 0x564e7b819ae0 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c072a30 con 0x564e7b813140 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e835e5740 10 client.0 _ll_register_callbacks cb 0x7f3e6400eb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b828b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b828b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51426/0 (socket says 172.21.15.22:51426) 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 -- 172.21.15.22:0/2602220111 learned_addr learned my addr 172.21.15.22:0/2602220111 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b82c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b819ae0 0x564e7b829070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 -- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 msgr2=0x564e7b82c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b82c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 -- 172.21.15.22:0/2602220111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b819ae0 msgr2=0x564e7b829070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b819ae0 0x564e7b829070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.655+0000 7f3e71d9b640 1 -- 172.21.15.22:0/2602220111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e7b8296b0 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e7ad9d640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b819ae0 0x564e7b829070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e71d9b640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b828b30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x564e7b81ab70 tx=0x7f3e6400e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af10a7e420d385c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e61ffb640 1 -- 172.21.15.22:0/2602220111 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6402e040 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e61ffb640 1 -- 172.21.15.22:0/2602220111 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3e64028df0 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e7400ae20 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e61ffb640 1 -- 172.21.15.22:0/2602220111 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e64047050 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 10 client.19341 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564e7b88c310 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e61ffb640 1 -- 172.21.15.22:0/2602220111 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e7400ae20 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e61ffb640 1 -- 172.21.15.22:0/2602220111 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e64028040 con 0x564e7b813af0 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 10 client.19341 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:24.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 10 client.19341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 20 client.19341 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b88ca00 0x564e7b88ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 4 client.19341 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.656+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e7b77e690 con 0x564e7b88ca00 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b88ca00 0x564e7b88ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e7a59c640 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b88ca00 0x564e7b88ee20 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f890ce7b7388b34d server_cookie=14eeb60634264881 in_seq=0 out_seq=0 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e61ffb640 10 client.19341 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e61ffb640 1 -- 172.21.15.22:0/2602220111 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564e7b77e690 con 0x564e7b88ca00 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e61ffb640 10 client.19341 handle_command_reply: tid=0 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 client.19341 shutdown 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 msgr2=0x564e7b828b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b828b30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x564e7b81ab70 tx=0x7f3e6400e4d0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 shutdown_connections 2024-03-09T00:06:24.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564e7b88ca00 0x564e7b88ee20 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e7b819ae0 0x564e7b829070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e61ffb640 0 client.19341 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e7b813af0 0x564e7b828b30 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 --2- 172.21.15.22:0/2602220111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e7b813140 0x564e7b82c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 >> 172.21.15.22:0/2602220111 conn(0x564e7b80de90 msgr2=0x564e7b831090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 shutdown_connections 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 1 -- 172.21.15.22:0/2602220111 wait complete. 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.657+0000 7f3e835e5740 20 client.19341 trim_cache size 0 max 16384 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:24.659 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:06:24.660 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:06:24.660 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:24.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e680a2bf0 msgr2=0x7f3e680a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e680a2bf0 0x7f3e680a50b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x564e7b80f2b0 tx=0x7f3e64009530 comp rx=0 tx=0).stop 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c196660 msgr2=0x7f3e7c196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c196660 0x7f3e7c196ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e74007bf0 tx=0x7f3e7400e420 comp rx=0 tx=0).stop 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 shutdown_connections 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3e680a2bf0 0x7f3e680a50b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c196660 0x7f3e7c196ae0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7c1054a0 0x7f3e7c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 --2- 172.21.15.22:0/763195997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e7c1003d0 0x7f3e7c19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 >> 172.21.15.22:0/763195997 conn(0x7f3e7c0772f0 msgr2=0x7f3e7c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 shutdown_connections 2024-03-09T00:06:24.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.658+0000 7f3e80929640 1 -- 172.21.15.22:0/763195997 wait complete. 2024-03-09T00:06:24.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:06:24.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:06:24.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1183329 9.46 MB/sec execute 512 sec latency 18.744 ms 2024-03-09T00:06:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.771+0000 7f3df0281640 1 -- 172.21.15.22:0/2709470413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 msgr2=0x7f3de8069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.771+0000 7f3df0281640 1 --2- 172.21.15.22:0/2709470413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de8069ff0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009900 tx=0x7f3de402bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- 172.21.15.22:0/2709470413 shutdown_connections 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 --2- 172.21.15.22:0/2709470413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de8106610 0x7f3de810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 --2- 172.21.15.22:0/2709470413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3de806a5c0 0x7f3de8105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 --2- 172.21.15.22:0/2709470413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de8069ff0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- 172.21.15.22:0/2709470413 >> 172.21.15.22:0/2709470413 conn(0x7f3de8077290 msgr2=0x7f3de80776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- 172.21.15.22:0/2709470413 shutdown_connections 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- 172.21.15.22:0/2709470413 wait complete. 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 Processor -- start 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- start start 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3de8106610 0x7f3de819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de8197a40 0x7f3de8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de81116e0 con 0x7f3de8069c10 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8111560 con 0x7f3de8197a40 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.772+0000 7f3df0281640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8111860 con 0x7f3de8106610 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de819d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51456/0 (socket says 172.21.15.22:51456) 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 -- 172.21.15.22:0/1844995148 learned_addr learned my addr 172.21.15.22:0/1844995148 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3ded81c640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3de8106610 0x7f3de819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee81e640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de8197a40 0x7f3de8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 -- 172.21.15.22:0/1844995148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3de8106610 msgr2=0x7f3de819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3de8106610 0x7f3de819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 -- 172.21.15.22:0/1844995148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de8197a40 msgr2=0x7f3de8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de8197a40 0x7f3de8197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:24.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 -- 172.21.15.22:0/1844995148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8198560 con 0x7f3de8069c10 2024-03-09T00:06:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3dee01d640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de819d890 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3de4006010 tx=0x7f3de402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3ddf7fe640 1 -- 172.21.15.22:0/1844995148 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4041070 con 0x7f3de8069c10 2024-03-09T00:06:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3ddf7fe640 1 -- 172.21.15.22:0/1844995148 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de4004450 con 0x7f3de8069c10 2024-03-09T00:06:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de81987f0 con 0x7f3de8069c10 2024-03-09T00:06:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3ddf7fe640 1 -- 172.21.15.22:0/1844995148 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4003e00 con 0x7f3de8069c10 2024-03-09T00:06:24.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.773+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de81a49e0 con 0x7f3de8069c10 2024-03-09T00:06:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.774+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de8102a10 con 0x7f3de8069c10 2024-03-09T00:06:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.775+0000 7f3ddf7fe640 1 -- 172.21.15.22:0/1844995148 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3de4044070 con 0x7f3de8069c10 2024-03-09T00:06:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.776+0000 7f3ddf7fe640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3dd40a2970 0x7f3dd40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.776+0000 7f3ded81c640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3dd40a2970 0x7f3dd40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.776+0000 7f3ddf7fe640 1 -- 172.21.15.22:0/1844995148 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3de40f0c60 con 0x7f3de8069c10 2024-03-09T00:06:24.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.776+0000 7f3ded81c640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3dd40a2970 0x7f3dd40a4e30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3de80ffd60 tx=0x7f3dd8005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:24.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:24.780+0000 7f3ddf7fe640 1 -- 172.21.15.22:0/1844995148 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3de40bd6d0 con 0x7f3de8069c10 2024-03-09T00:06:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.027+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3de8068530 con 0x7f3de8069c10 2024-03-09T00:06:25.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.027+0000 7f3ddf7fe640 1 -- 172.21.15.22:0/1844995148 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f3de40bcfd0 con 0x7f3de8069c10 2024-03-09T00:06:25.022 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:06:25.022 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:06:25.023 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:06:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3dd40a2970 msgr2=0x7f3dd40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3dd40a2970 0x7f3dd40a4e30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3de80ffd60 tx=0x7f3dd8005f90 comp rx=0 tx=0).stop 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 msgr2=0x7f3de819d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de819d890 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3de4006010 tx=0x7f3de402cb30 comp rx=0 tx=0).stop 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 shutdown_connections 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3dd40a2970 0x7f3dd40a4e30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de8197a40 0x7f3de8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3de8106610 0x7f3de819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 --2- 172.21.15.22:0/1844995148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de8069c10 0x7f3de819d890 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 >> 172.21.15.22:0/1844995148 conn(0x7f3de8077290 msgr2=0x7f3de80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 shutdown_connections 2024-03-09T00:06:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.029+0000 7f3df0281640 1 -- 172.21.15.22:0/1844995148 wait complete. 2024-03-09T00:06:25.035 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:25.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:06:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 -- 172.21.15.22:0/253154803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3354100d90 msgr2=0x7f3354104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 --2- 172.21.15.22:0/253154803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3354100d90 0x7f3354104dd0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3348009900 tx=0x7f334802bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 -- 172.21.15.22:0/253154803 shutdown_connections 2024-03-09T00:06:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 --2- 172.21.15.22:0/253154803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33541054b0 0x7f335410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 --2- 172.21.15.22:0/253154803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3354100d90 0x7f3354104dd0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 --2- 172.21.15.22:0/253154803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33541003e0 0x7f33541007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 -- 172.21.15.22:0/253154803 >> 172.21.15.22:0/253154803 conn(0x7f33540773a0 msgr2=0x7f33540777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 -- 172.21.15.22:0/253154803 shutdown_connections 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.147+0000 7f3358f59640 1 -- 172.21.15.22:0/253154803 wait complete. 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 Processor -- start 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 -- start start 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33541003e0 0x7f335410f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33541054b0 0x7f335410fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3354110410 0x7f33541a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3354069100 con 0x7f33541054b0 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3354068f80 con 0x7f33541003e0 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3358f59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3354069280 con 0x7f3354110410 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33541054b0 0x7f335410fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33541054b0 0x7f335410fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51476/0 (socket says 172.21.15.22:51476) 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 -- 172.21.15.22:0/3571621751 learned_addr learned my addr 172.21.15.22:0/3571621751 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f335259c640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33541003e0 0x7f335410f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3354110410 0x7f33541a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 -- 172.21.15.22:0/3571621751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3354110410 msgr2=0x7f33541a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3354110410 0x7f33541a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 -- 172.21.15.22:0/3571621751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33541003e0 msgr2=0x7f335410f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33541003e0 0x7f335410f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.148+0000 7f3351d9b640 1 -- 172.21.15.22:0/3571621751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33541a63e0 con 0x7f33541054b0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.149+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33541054b0 0x7f335410fed0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f334802c240 tx=0x7f3348004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.149+0000 7f33437fe640 1 -- 172.21.15.22:0/3571621751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348043020 con 0x7f33541054b0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.149+0000 7f33437fe640 1 -- 172.21.15.22:0/3571621751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33480054e0 con 0x7f33541054b0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.149+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33541a6670 con 0x7f33541054b0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.149+0000 7f33437fe640 1 -- 172.21.15.22:0/3571621751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348003b80 con 0x7f33541054b0 2024-03-09T00:06:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.149+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33541a6bc0 con 0x7f33541054b0 2024-03-09T00:06:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.151+0000 7f33437fe640 1 -- 172.21.15.22:0/3571621751 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3348046020 con 0x7f33541054b0 2024-03-09T00:06:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 Processor -- start 2024-03-09T00:06:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f33437fe640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f333c0a29a0 0x7f333c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 -- start start 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f333c0a29a0 0x7f333c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f33437fe640 1 -- 172.21.15.22:0/3571621751 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f33480f1260 con 0x7f33541054b0 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88e1a0 0x5608da88e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88eac0 0x5608da897160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da8976a0 0x5608da899a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da855360 con 0x5608da88eac0 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da7f1ec0 con 0x5608da88e1a0 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335bc0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da79ad30 con 0x5608da8976a0 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f3351d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88eac0 0x5608da897160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f3351d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88eac0 0x5608da897160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51492/0 (socket says 172.21.15.22:51492) 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f3351d9b640 1 -- 172.21.15.22:0/3662842739 learned_addr learned my addr 172.21.15.22:0/3662842739 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.152+0000 7f335259c640 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88e1a0 0x5608da88e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da8976a0 0x5608da899a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3351d9b640 1 -- 172.21.15.22:0/3662842739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da8976a0 msgr2=0x5608da899a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da8976a0 0x5608da899a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3351d9b640 1 -- 172.21.15.22:0/3662842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88e1a0 msgr2=0x5608da88e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88e1a0 0x5608da88e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3351d9b640 1 -- 172.21.15.22:0/3662842739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608da799bc0 con 0x5608da88eac0 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f335259c640 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88e1a0 0x5608da88e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da8976a0 0x5608da899a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f333c0a29a0 0x7f333c0a4e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f334c005fd0 tx=0x7f334c011810 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88eac0 0x5608da897160 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f33480b9f40 tx=0x7f33480035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819b5c4a7724dbe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3340ff9640 1 -- 172.21.15.22:0/3662842739 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348005690 con 0x5608da88eac0 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3340ff9640 1 -- 172.21.15.22:0/3662842739 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33480bdc70 con 0x5608da88eac0 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.153+0000 7f3340ff9640 1 -- 172.21.15.22:0/3662842739 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33480b9be0 con 0x5608da88eac0 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- 172.21.15.22:0/3662842739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88eac0 msgr2=0x5608da897160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88eac0 0x5608da897160 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f33480b9f40 tx=0x7f33480035f0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- 172.21.15.22:0/3662842739 shutdown_connections 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da8976a0 0x5608da899a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88eac0 0x5608da897160 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3662842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88e1a0 0x5608da88e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- 172.21.15.22:0/3662842739 >> 172.21.15.22:0/3662842739 conn(0x5608da8888d0 msgr2=0x5608da888cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- 172.21.15.22:0/3662842739 shutdown_connections 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- 172.21.15.22:0/3662842739 wait complete. 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- start start 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 Processor -- start 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88e1a0 0x5608da8946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88eac0 0x5608da894be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da8976a0 0x5608da895120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da7904a0 con 0x5608da88e1a0 2024-03-09T00:06:25.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da7edf60 con 0x5608da8976a0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.154+0000 7f335bc0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da89a250 con 0x5608da88eac0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335bc0a740 10 client.0 _ll_register_callbacks cb 0x5608da7ebd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88e1a0 0x5608da8946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88e1a0 0x5608da8946a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51496/0 (socket says 172.21.15.22:51496) 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 -- 172.21.15.22:0/801781253 learned_addr learned my addr 172.21.15.22:0/801781253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f3352d9d640 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da8976a0 0x5608da895120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f3351d9b640 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88eac0 0x5608da894be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 -- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88eac0 msgr2=0x5608da894be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88eac0 0x5608da894be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 -- 172.21.15.22:0/801781253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da8976a0 msgr2=0x5608da895120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da8976a0 0x5608da895120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 -- 172.21.15.22:0/801781253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608da5063f0 con 0x5608da88e1a0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335259c640 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88e1a0 0x5608da8946a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f334400b940 tx=0x7f334400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2835c8cf254ea561 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f3339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f3339ffb640 1 -- 172.21.15.22:0/801781253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334401b670 con 0x5608da88e1a0 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33480bdc70 con 0x5608da88e1a0 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f3339ffb640 1 -- 172.21.15.22:0/801781253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f334401b810 con 0x5608da88e1a0 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f3339ffb640 1 -- 172.21.15.22:0/801781253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3344024050 con 0x5608da88e1a0 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335bc0a740 10 client.19359 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.155+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5608da5d07d0 con 0x5608da88e1a0 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f3339ffb640 1 -- 172.21.15.22:0/801781253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33480bdc70 con 0x5608da88e1a0 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f3339ffb640 1 -- 172.21.15.22:0/801781253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33440115f0 con 0x5608da88e1a0 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f335bc0a740 10 client.19359 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f335bc0a740 10 client.19359 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f335bc0a740 20 client.19359 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f335bc0a740 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8a7520 0x5608da8a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f335bc0a740 4 client.19359 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608da7ff400 con 0x5608da8a7520 2024-03-09T00:06:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f3351d9b640 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8a7520 0x5608da8a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f3351d9b640 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8a7520 0x5608da8a9910 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e2c0888211a6cb0 server_cookie=87be22e788d4e433 in_seq=0 out_seq=0 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.156+0000 7f3339ffb640 10 client.19359 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.157+0000 7f3339ffb640 1 -- 172.21.15.22:0/801781253 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608da7ff400 con 0x5608da8a7520 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.157+0000 7f3339ffb640 10 client.19359 handle_command_reply: tid=0 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.157+0000 7f335bc0a740 1 client.19359 shutdown 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.157+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88e1a0 msgr2=0x5608da8946a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.157+0000 7f335bc0a740 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88e1a0 0x5608da8946a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f334400b940 tx=0x7f334400be00 comp rx=0 tx=0).stop 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 shutdown_connections 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8a7520 0x5608da8a9910 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da8976a0 0x5608da895120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88eac0 0x5608da894be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f3339ffb640 0 client.19359 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 --2- 172.21.15.22:0/801781253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da88e1a0 0x5608da8946a0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 >> 172.21.15.22:0/801781253 conn(0x5608da8888d0 msgr2=0x5608da88a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 shutdown_connections 2024-03-09T00:06:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 1 -- 172.21.15.22:0/801781253 wait complete. 2024-03-09T00:06:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.158+0000 7f335bc0a740 20 client.19359 trim_cache size 0 max 16384 2024-03-09T00:06:25.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 Processor -- start 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 -- start start 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da893680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da891900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da891e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541b2d70 con 0x5608da8976a0 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541b0910 con 0x5608da88eac0 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335bc0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541ae9d0 con 0x5608da88e1a0 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da891e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da891e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51500/0 (socket says 172.21.15.22:51500) 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f335259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da893680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 -- 172.21.15.22:0/3680282745 learned_addr learned my addr 172.21.15.22:0/3680282745 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da891900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 -- 172.21.15.22:0/3680282745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 msgr2=0x5608da893680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da893680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 -- 172.21.15.22:0/3680282745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 msgr2=0x5608da891900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da891900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3352d9d640 1 -- 172.21.15.22:0/3680282745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608da88d290 con 0x5608da8976a0 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.170+0000 7f3351d9b640 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da891900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f3352d9d640 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da891e40 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f334c003e20 tx=0x7f334c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64936a528234483 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f333bfff640 1 -- 172.21.15.22:0/3680282745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c017040 con 0x5608da8976a0 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f333bfff640 1 -- 172.21.15.22:0/3680282745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f334c025070 con 0x5608da8976a0 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f333bfff640 1 -- 172.21.15.22:0/3680282745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c005280 con 0x5608da8976a0 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f335bc0a740 1 -- 172.21.15.22:0/3680282745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 msgr2=0x5608da891e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da891e40 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f334c003e20 tx=0x7f334c009500 comp rx=0 tx=0).stop 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f335bc0a740 1 -- 172.21.15.22:0/3680282745 shutdown_connections 2024-03-09T00:06:25.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da891e40 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da891900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f335bc0a740 1 --2- 172.21.15.22:0/3680282745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da893680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.171+0000 7f335bc0a740 1 -- 172.21.15.22:0/3680282745 >> 172.21.15.22:0/3680282745 conn(0x5608da8888d0 msgr2=0x5608da892f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 -- 172.21.15.22:0/3680282745 shutdown_connections 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 -- 172.21.15.22:0/3680282745 wait complete. 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 -- start start 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 Processor -- start 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da89d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da89dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da8f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541aefd0 con 0x5608da8976a0 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541b02b0 con 0x5608da88eac0 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541af5f0 con 0x5608da88e1a0 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335bc0a740 10 client.0 _ll_register_callbacks cb 0x7f33480b9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:25.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da8f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da8f7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51506/0 (socket says 172.21.15.22:51506) 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 -- 172.21.15.22:0/2657043822 learned_addr learned my addr 172.21.15.22:0/2657043822 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f335259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da89d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3351d9b640 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da89dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 -- 172.21.15.22:0/2657043822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 msgr2=0x5608da89d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da89d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 -- 172.21.15.22:0/2657043822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 msgr2=0x5608da89dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da89dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.172+0000 7f3352d9d640 1 -- 172.21.15.22:0/2657043822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608da8f8070 con 0x5608da8976a0 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f3352d9d640 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da8f7a00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f334c021c90 tx=0x7f334c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba1d2600b6eb037 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f3339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f3339ffb640 1 -- 172.21.15.22:0/2657043822 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c017050 con 0x5608da8976a0 2024-03-09T00:06:25.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f3339ffb640 1 -- 172.21.15.22:0/2657043822 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f334c009030 con 0x5608da8976a0 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f334c025070 con 0x5608da8976a0 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f3339ffb640 1 -- 172.21.15.22:0/2657043822 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f334c003b00 con 0x5608da8976a0 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f335bc0a740 10 client.19365 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5608da8ff380 con 0x5608da8976a0 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f3339ffb640 1 -- 172.21.15.22:0/2657043822 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f334c025070 con 0x5608da8976a0 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f3339ffb640 1 -- 172.21.15.22:0/2657043822 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f334c020420 con 0x5608da8976a0 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f335bc0a740 10 client.19365 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f335bc0a740 10 client.19365 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f335bc0a740 20 client.19365 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.173+0000 7f335bc0a740 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8ffa70 0x5608da901e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 4 client.19365 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5608da7f9750 con 0x5608da8ffa70 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335259c640 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8ffa70 0x5608da901e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335259c640 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8ffa70 0x5608da901e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=feb24648598341d9 server_cookie=4c2fe08505bc4a31 in_seq=0 out_seq=0 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f3339ffb640 10 client.19365 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f3339ffb640 1 -- 172.21.15.22:0/2657043822 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5608da7f9750 con 0x5608da8ffa70 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f3339ffb640 10 client.19365 handle_command_reply: tid=0 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 client.19365 shutdown 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 msgr2=0x5608da8f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da8f7a00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f334c021c90 tx=0x7f334c005ae0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 shutdown_connections 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5608da8ffa70 0x5608da901e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608da8976a0 0x5608da8f7a00 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608da88eac0 0x5608da89dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f3339ffb640 0 client.19365 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 --2- 172.21.15.22:0/2657043822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608da88e1a0 0x5608da89d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 >> 172.21.15.22:0/2657043822 conn(0x5608da8888d0 msgr2=0x5608da893800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 shutdown_connections 2024-03-09T00:06:25.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.174+0000 7f335bc0a740 1 -- 172.21.15.22:0/2657043822 wait complete. 2024-03-09T00:06:25.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f335bc0a740 20 client.19365 trim_cache size 0 max 16384 2024-03-09T00:06:25.177 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f333c0a29a0 msgr2=0x7f333c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f333c0a29a0 0x7f333c0a4e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f334c005fd0 tx=0x7f334c011810 comp rx=0 tx=0).stop 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33541054b0 msgr2=0x7f335410fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33541054b0 0x7f335410fed0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f334802c240 tx=0x7f3348004360 comp rx=0 tx=0).stop 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 shutdown_connections 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f333c0a29a0 0x7f333c0a4e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3354110410 0x7f33541a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33541054b0 0x7f335410fed0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 --2- 172.21.15.22:0/3571621751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33541003e0 0x7f335410f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 >> 172.21.15.22:0/3571621751 conn(0x7f33540773a0 msgr2=0x7f33540fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 shutdown_connections 2024-03-09T00:06:25.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.175+0000 7f3358f59640 1 -- 172.21.15.22:0/3571621751 wait complete. 2024-03-09T00:06:25.180 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:25.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 -- 172.21.15.22:0/372616468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5548100440 msgr2=0x7f55481008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 --2- 172.21.15.22:0/372616468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5548100440 0x7f55481008c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f553c009900 tx=0x7f553c02bc80 comp rx=0 tx=0).stop 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 -- 172.21.15.22:0/372616468 shutdown_connections 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 --2- 172.21.15.22:0/372616468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100e00 0x7f554810d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 --2- 172.21.15.22:0/372616468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5548100440 0x7f55481008c0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 --2- 172.21.15.22:0/372616468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55481062d0 0x7f55481066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 -- 172.21.15.22:0/372616468 >> 172.21.15.22:0/372616468 conn(0x7f5548076fd0 msgr2=0x7f55480773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 -- 172.21.15.22:0/372616468 shutdown_connections 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 -- 172.21.15.22:0/372616468 wait complete. 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.290+0000 7f554e5b3640 1 Processor -- start 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554e5b3640 1 -- start start 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554e5b3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100440 0x7f55481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554e5b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5548100e00 0x7f5548198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554e5b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55481062d0 0x7f55481a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554e5b3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481105d0 con 0x7f55481062d0 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554e5b3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548110450 con 0x7f5548100e00 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554e5b3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548110750 con 0x7f5548100440 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554cb50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55481062d0 0x7f55481a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100440 0x7f55481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f55477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5548100e00 0x7f5548198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100440 0x7f55481989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59140/0 (socket says 172.21.15.22:59140) 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554cb50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55481062d0 0x7f55481a3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51520/0 (socket says 172.21.15.22:51520) 2024-03-09T00:06:25.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 -- 172.21.15.22:0/3966627032 learned_addr learned my addr 172.21.15.22:0/3966627032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 -- 172.21.15.22:0/3966627032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5548100e00 msgr2=0x7f5548198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5548100e00 0x7f5548198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 -- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55481062d0 msgr2=0x7f55481a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55481062d0 0x7f55481a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f5547fff640 1 -- 172.21.15.22:0/3966627032 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55481a3f70 con 0x7f5548100440 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f554cb50640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55481062d0 0x7f55481a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.291+0000 7f55477fe640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5548100e00 0x7f5548198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.292+0000 7f5547fff640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100440 0x7f55481989b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f553800e9f0 tx=0x7f553800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.292+0000 7f55457fa640 1 -- 172.21.15.22:0/3966627032 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553800cd80 con 0x7f5548100440 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.292+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55481a4140 con 0x7f5548100440 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.292+0000 7f55457fa640 1 -- 172.21.15.22:0/3966627032 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5538010910 con 0x7f5548100440 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.292+0000 7f55457fa640 1 -- 172.21.15.22:0/3966627032 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55380188f0 con 0x7f5548100440 2024-03-09T00:06:25.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.292+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55481a4660 con 0x7f5548100440 2024-03-09T00:06:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f55457fa640 1 -- 172.21.15.22:0/3966627032 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5538010ac0 con 0x7f5548100440 2024-03-09T00:06:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 Processor -- start 2024-03-09T00:06:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 -- start start 2024-03-09T00:06:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8571580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb857a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb857ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbb8538360 con 0x55dbb857a6a0 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbb84d4ec0 con 0x55dbb85711a0 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.294+0000 7f555118f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbb847dd30 con 0x55dbb8571ac0 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55457fa640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f55300a2970 0x7f55300a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb857ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f554cb50640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8571580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb857ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51536/0 (socket says 172.21.15.22:51536) 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 -- 172.21.15.22:0/1558402386 learned_addr learned my addr 172.21.15.22:0/1558402386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f5547fff640 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb857a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f554cb50640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f55300a2970 0x7f55300a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55457fa640 1 -- 172.21.15.22:0/3966627032 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f55380cf5c0 con 0x7f5548100440 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 -- 172.21.15.22:0/1558402386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 msgr2=0x55dbb857a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb857a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 -- 172.21.15.22:0/1558402386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 msgr2=0x55dbb8571580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8571580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 -- 172.21.15.22:0/1558402386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbb847cbc0 con 0x55dbb857a6a0 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f55477fe640 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb857ca90 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f553c02c1c0 tx=0x7f553c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693ce2d093abf5f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f554cb50640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f55300a2970 0x7f55300a4e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55400049d0 tx=0x7f5540004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f552e7fc640 1 -- 172.21.15.22:0/1558402386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c004a70 con 0x55dbb857a6a0 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f552e7fc640 1 -- 172.21.15.22:0/1558402386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f553c004c10 con 0x55dbb857a6a0 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.295+0000 7f552e7fc640 1 -- 172.21.15.22:0/1558402386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c005350 con 0x55dbb857a6a0 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 -- 172.21.15.22:0/1558402386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 msgr2=0x55dbb857ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb857ca90 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f553c02c1c0 tx=0x7f553c009630 comp rx=0 tx=0).stop 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 -- 172.21.15.22:0/1558402386 shutdown_connections 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb857ca90 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb857a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 --2- 172.21.15.22:0/1558402386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8571580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 -- 172.21.15.22:0/1558402386 >> 172.21.15.22:0/1558402386 conn(0x55dbb856b8d0 msgr2=0x55dbb856bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 -- 172.21.15.22:0/1558402386 shutdown_connections 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 -- 172.21.15.22:0/1558402386 wait complete. 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.296+0000 7f555118f740 1 -- start start 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 1 Processor -- start 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8577620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb8571ac0 0x55dbb8577b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb85780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbb84734a0 con 0x55dbb857a6a0 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbb84d0f60 con 0x55dbb8571ac0 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbb857d430 con 0x55dbb85711a0 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f555118f740 10 client.0 _ll_register_callbacks cb 0x55dbb84ced60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f55477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb8571ac0 0x55dbb8577b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f554cb50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb85780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8577620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8577620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59162/0 (socket says 172.21.15.22:59162) 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f55477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb8571ac0 0x55dbb8577b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34122/0 (socket says 172.21.15.22:34122) 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 -- 172.21.15.22:0/1393628214 learned_addr learned my addr 172.21.15.22:0/1393628214 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 -- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb8571ac0 msgr2=0x55dbb8577b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb8571ac0 0x55dbb8577b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 -- 172.21.15.22:0/1393628214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 msgr2=0x55dbb85780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb85780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f5547fff640 1 -- 172.21.15.22:0/1393628214 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbb81e93f0 con 0x55dbb85711a0 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.297+0000 7f55477fe640 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb8571ac0 0x55dbb8577b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f5547fff640 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8577620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55380982a0 tx=0x7f553809f970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83829bb9597300ea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f55277fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f55277fe640 1 -- 172.21.15.22:0/1393628214 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55380862d0 con 0x55dbb85711a0 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f55277fe640 1 -- 172.21.15.22:0/1393628214 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f55380cf160 con 0x55dbb85711a0 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f553c004c10 con 0x55dbb85711a0 2024-03-09T00:06:25.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f55277fe640 1 -- 172.21.15.22:0/1393628214 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55380cf360 con 0x55dbb85711a0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 10 client.28804 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55dbb82b37d0 con 0x55dbb85711a0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f55277fe640 1 -- 172.21.15.22:0/1393628214 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f553c004c10 con 0x55dbb85711a0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f55277fe640 1 -- 172.21.15.22:0/1393628214 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5538097620 con 0x55dbb85711a0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 10 client.28804 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 10 client.28804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 20 client.28804 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb858a7d0 0x55dbb858cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 4 client.28804 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.298+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbb84e2400 con 0x55dbb858a7d0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.299+0000 7f55477fe640 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb858a7d0 0x55dbb858cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.299+0000 7f55477fe640 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb858a7d0 0x55dbb858cbc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e164f9dcd30fc892 server_cookie=9c6b535ea2b22274 in_seq=0 out_seq=0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.299+0000 7f55277fe640 10 client.28804 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.299+0000 7f55277fe640 1 -- 172.21.15.22:0/1393628214 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbb84e2400 con 0x55dbb858a7d0 2024-03-09T00:06:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.299+0000 7f55277fe640 10 client.28804 handle_command_reply: tid=0 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.299+0000 7f555118f740 1 client.28804 shutdown 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 msgr2=0x55dbb8577620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8577620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55380982a0 tx=0x7f553809f970 comp rx=0 tx=0).stop 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 shutdown_connections 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb858a7d0 0x55dbb858cbc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb85780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb8571ac0 0x55dbb8577b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f55277fe640 0 client.28804 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 --2- 172.21.15.22:0/1393628214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8577620 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 >> 172.21.15.22:0/1393628214 conn(0x55dbb856b8d0 msgr2=0x55dbb856d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 shutdown_connections 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 1 -- 172.21.15.22:0/1393628214 wait complete. 2024-03-09T00:06:25.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.300+0000 7f555118f740 20 client.28804 trim_cache size 0 max 16384 2024-03-09T00:06:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.312+0000 7f555118f740 1 Processor -- start 2024-03-09T00:06:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.312+0000 7f555118f740 1 -- start start 2024-03-09T00:06:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f555118f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8579d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f555118f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb8574880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f555118f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb8574dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f555118f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481b0b30 con 0x55dbb857a6a0 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f555118f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481ae6d0 con 0x55dbb85711a0 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f555118f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481ac790 con 0x55dbb8571ac0 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f554cb50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb8574dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f55477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb8574880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f554cb50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb8574dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51556/0 (socket says 172.21.15.22:51556) 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f554cb50640 1 -- 172.21.15.22:0/270425710 learned_addr learned my addr 172.21.15.22:0/270425710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f5547fff640 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8579d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f55477fe640 1 -- 172.21.15.22:0/270425710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 msgr2=0x55dbb8579d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f55477fe640 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8579d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f55477fe640 1 -- 172.21.15.22:0/270425710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 msgr2=0x55dbb8574dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f55477fe640 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb8574dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f55477fe640 1 -- 172.21.15.22:0/270425710 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbb85718f0 con 0x55dbb8571ac0 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f554cb50640 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb8574dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f55477fe640 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb8574880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f553c004ee0 tx=0x7f553c004f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2608590f47c87a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f552dffb640 1 -- 172.21.15.22:0/270425710 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c02cb80 con 0x55dbb8571ac0 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f552dffb640 1 -- 172.21.15.22:0/270425710 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f553c02cd20 con 0x55dbb8571ac0 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.313+0000 7f552dffb640 1 -- 172.21.15.22:0/270425710 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c034730 con 0x55dbb8571ac0 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 -- 172.21.15.22:0/270425710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 msgr2=0x55dbb8574880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb8574880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f553c004ee0 tx=0x7f553c004f30 comp rx=0 tx=0).stop 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 -- 172.21.15.22:0/270425710 shutdown_connections 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb857a6a0 0x55dbb8574dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb8571ac0 0x55dbb8574880 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 --2- 172.21.15.22:0/270425710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb85711a0 0x55dbb8579d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 -- 172.21.15.22:0/270425710 >> 172.21.15.22:0/270425710 conn(0x55dbb856b8d0 msgr2=0x55dbb856bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 -- 172.21.15.22:0/270425710 shutdown_connections 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 -- 172.21.15.22:0/270425710 wait complete. 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.314+0000 7f555118f740 1 -- start start 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 Processor -- start 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8581880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb8571ac0 0x55dbb857fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb857a6a0 0x55dbb8580160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481acd90 con 0x55dbb8571ac0 2024-03-09T00:06:25.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481ae070 con 0x55dbb857a6a0 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481ad3b0 con 0x55dbb85711a0 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 10 client.0 _ll_register_callbacks cb 0x7f55400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb8571ac0 0x55dbb857fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f5547fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8581880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb8571ac0 0x55dbb857fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51572/0 (socket says 172.21.15.22:51572) 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 -- 172.21.15.22:0/3151549632 learned_addr learned my addr 172.21.15.22:0/3151549632 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f554cb50640 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb857a6a0 0x55dbb8580160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 -- 172.21.15.22:0/3151549632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 msgr2=0x55dbb8581880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8581880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 -- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb857a6a0 msgr2=0x55dbb8580160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb857a6a0 0x55dbb8580160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 -- 172.21.15.22:0/3151549632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbb85807a0 con 0x55dbb8571ac0 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f5547fff640 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8581880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55477fe640 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb8571ac0 0x55dbb857fc20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f553c048040 tx=0x7f553c02c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80bc7affa9a36f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55277fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55277fe640 1 -- 172.21.15.22:0/3151549632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c004240 con 0x55dbb8571ac0 2024-03-09T00:06:25.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f55277fe640 1 -- 172.21.15.22:0/3151549632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f553c0043e0 con 0x55dbb8571ac0 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.315+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbb84dc750 con 0x55dbb8571ac0 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f55277fe640 1 -- 172.21.15.22:0/3151549632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553c004fa0 con 0x55dbb8571ac0 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 10 client.19383 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55dbb85e1ad0 con 0x55dbb8571ac0 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f55277fe640 1 -- 172.21.15.22:0/3151549632 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dbb84dc750 con 0x55dbb8571ac0 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f55277fe640 1 -- 172.21.15.22:0/3151549632 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f553c050030 con 0x55dbb8571ac0 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 10 client.19383 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 10 client.19383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:06:25.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 20 client.19383 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb85e1db0 0x55dbb85e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 4 client.19383 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dbb845a640 con 0x55dbb85e1db0 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.316+0000 7f5547fff640 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb85e1db0 0x55dbb85e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f5547fff640 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb85e1db0 0x55dbb85e41a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=660978f857a59098 server_cookie=349a4fb2ab641a5a in_seq=0 out_seq=0 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f55277fe640 10 client.19383 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f55277fe640 1 -- 172.21.15.22:0/3151549632 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dbb845a640 con 0x55dbb85e1db0 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f55277fe640 10 client.19383 handle_command_reply: tid=0 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 client.19383 shutdown 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb8571ac0 msgr2=0x55dbb857fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb8571ac0 0x55dbb857fc20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f553c048040 tx=0x7f553c02c380 comp rx=0 tx=0).stop 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 shutdown_connections 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55dbb85e1db0 0x55dbb85e41a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbb857a6a0 0x55dbb8580160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbb8571ac0 0x55dbb857fc20 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f55277fe640 0 client.19383 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 --2- 172.21.15.22:0/3151549632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbb85711a0 0x55dbb8581880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 >> 172.21.15.22:0/3151549632 conn(0x55dbb856b8d0 msgr2=0x55dbb85775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 shutdown_connections 2024-03-09T00:06:25.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 1 -- 172.21.15.22:0/3151549632 wait complete. 2024-03-09T00:06:25.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f555118f740 20 client.19383 trim_cache size 0 max 16384 2024-03-09T00:06:25.326 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:25.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f55300a2970 msgr2=0x7f55300a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f554e5b3640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f55300a2970 0x7f55300a4e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55400049d0 tx=0x7f5540004ab0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100440 msgr2=0x7f55481989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.317+0000 7f554e5b3640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100440 0x7f55481989b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f553800e9f0 tx=0x7f553800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 shutdown_connections 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f55300a2970 0x7f55300a4e30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55481062d0 0x7f55481a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5548100e00 0x7f5548198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 --2- 172.21.15.22:0/3966627032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5548100440 0x7f55481989b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 >> 172.21.15.22:0/3966627032 conn(0x7f5548076fd0 msgr2=0x7f55480ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 shutdown_connections 2024-03-09T00:06:25.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.318+0000 7f554e5b3640 1 -- 172.21.15.22:0/3966627032 wait complete. 2024-03-09T00:06:25.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:25 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:06:25.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:25 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:25.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:25 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:06:25.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:25 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:25.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:25 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:06:25.330 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:25 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1844995148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:25.331 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:25.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:06:25.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:25 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:06:25.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:25 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:25.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:25 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:06:25.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:25 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:25.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:25 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:06:25.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:25 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1844995148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:25.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1847159643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 msgr2=0x7f3cd810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1847159643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd810d090 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009ca0 tx=0x7f3cd002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1847159643 shutdown_connections 2024-03-09T00:06:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1847159643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd810d090 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1847159643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cd8100da0 0x7f3cd8104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1847159643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd81003f0 0x7f3cd81007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1847159643 >> 172.21.15.22:0/1847159643 conn(0x7f3cd8076570 msgr2=0x7f3cd8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1847159643 shutdown_connections 2024-03-09T00:06:25.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.445+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1847159643 wait complete. 2024-03-09T00:06:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 Processor -- start 2024-03-09T00:06:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 -- start start 2024-03-09T00:06:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd81003f0 0x7f3cd819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cd8197940 0x7f3cd8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81116a0 con 0x7f3cd81054c0 2024-03-09T00:06:25.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8111520 con 0x7f3cd8197940 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cdcc27640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8111820 con 0x7f3cd81003f0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cd5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cd5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51600/0 (socket says 172.21.15.22:51600) 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/1944418245 learned_addr learned my addr 172.21.15.22:0/1944418245 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cd659c640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd81003f0 0x7f3cd819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.446+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cd8197940 0x7f3cd8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/1944418245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd81003f0 msgr2=0x7f3cd819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd81003f0 0x7f3cd819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/1944418245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cd8197940 msgr2=0x7f3cd8197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cd8197940 0x7f3cd8197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/1944418245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd8198460 con 0x7f3cd81054c0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd819dcd0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00e9c0 tx=0x7f3ccc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cc77fe640 1 -- 172.21.15.22:0/1944418245 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00cd80 con 0x7f3cd81054c0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cc77fe640 1 -- 172.21.15.22:0/1944418245 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ccc005500 con 0x7f3cd81054c0 2024-03-09T00:06:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd8198750 con 0x7f3cd81054c0 2024-03-09T00:06:25.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cc77fe640 1 -- 172.21.15.22:0/1944418245 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc010690 con 0x7f3cd81054c0 2024-03-09T00:06:25.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.447+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd81a49a0 con 0x7f3cd81054c0 2024-03-09T00:06:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.449+0000 7f3cc77fe640 1 -- 172.21.15.22:0/1944418245 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3ccc0046c0 con 0x7f3cd81054c0 2024-03-09T00:06:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 Processor -- start 2024-03-09T00:06:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 -- start start 2024-03-09T00:06:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939a1a0 0x56524939a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x5652493a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cc77fe640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cc00a2970 0x7f3cc00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565249361360 con 0x56524939aac0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652492fdec0 con 0x56524939a1a0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cdf808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652492a6d30 con 0x5652493a36a0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cd6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939a1a0 0x56524939a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cd659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cd6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939a1a0 0x56524939a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34168/0 (socket says 172.21.15.22:34168) 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.450+0000 7f3cd6d9d640 1 -- 172.21.15.22:0/2565430160 learned_addr learned my addr 172.21.15.22:0/2565430160 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x5652493a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cc00a2970 0x7f3cc00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cc77fe640 1 -- 172.21.15.22:0/1944418245 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3ccc0cf5a0 con 0x7f3cd81054c0 2024-03-09T00:06:25.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd659c640 1 -- 172.21.15.22:0/2565430160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 msgr2=0x5652493a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd659c640 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x5652493a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd659c640 1 -- 172.21.15.22:0/2565430160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939a1a0 msgr2=0x56524939a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd659c640 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939a1a0 0x56524939a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd659c640 1 -- 172.21.15.22:0/2565430160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652492a5bc0 con 0x56524939aac0 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd659c640 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a3160 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009bd0 tx=0x7f3cc8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d442dcf112669ae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cc00a2970 0x7f3cc00a4e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0032530 tx=0x7f3cd0005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cc4ff9640 1 -- 172.21.15.22:0/2565430160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8004310 con 0x56524939aac0 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cc4ff9640 1 -- 172.21.15.22:0/2565430160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cc80044b0 con 0x56524939aac0 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.451+0000 7f3cc4ff9640 1 -- 172.21.15.22:0/2565430160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80035f0 con 0x56524939aac0 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 -- 172.21.15.22:0/2565430160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 msgr2=0x5652493a3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a3160 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009bd0 tx=0x7f3cc8011440 comp rx=0 tx=0).stop 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 -- 172.21.15.22:0/2565430160 shutdown_connections 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x5652493a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a3160 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 --2- 172.21.15.22:0/2565430160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939a1a0 0x56524939a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 -- 172.21.15.22:0/2565430160 >> 172.21.15.22:0/2565430160 conn(0x5652493948d0 msgr2=0x565249394cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 -- 172.21.15.22:0/2565430160 shutdown_connections 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 -- 172.21.15.22:0/2565430160 wait complete. 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.452+0000 7f3cdf808740 1 -- start start 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 Processor -- start 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652493a36a0 0x5652493a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56524929c4a0 con 0x56524939aac0 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652492f9f60 con 0x5652493a36a0 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652493a6490 con 0x56524939a1a0 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 10 client.0 _ll_register_callbacks cb 0x5652492f7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51620/0 (socket says 172.21.15.22:51620) 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/749127511 learned_addr learned my addr 172.21.15.22:0/749127511 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652493a36a0 0x5652493a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/749127511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 msgr2=0x5652493a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/749127511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652493a36a0 msgr2=0x5652493a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652493a36a0 0x5652493a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 -- 172.21.15.22:0/749127511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652490123f0 con 0x56524939aac0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a0bf0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc000c00 tx=0x7f3ccc00aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22819a204b5593f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cbdffb640 1 -- 172.21.15.22:0/749127511 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc09b160 con 0x56524939aac0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cbdffb640 1 -- 172.21.15.22:0/749127511 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3ccc0b0310 con 0x56524939aac0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.453+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cc80044b0 con 0x56524939aac0 2024-03-09T00:06:25.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cbdffb640 1 -- 172.21.15.22:0/749127511 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ccc0af980 con 0x56524939aac0 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 10 client.19395 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5652490dc7d0 con 0x56524939aac0 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cbdffb640 1 -- 172.21.15.22:0/749127511 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cc80044b0 con 0x56524939aac0 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cbdffb640 1 -- 172.21.15.22:0/749127511 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3ccc0a3b10 con 0x56524939aac0 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 10 client.19395 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 10 client.19395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 20 client.19395 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652493b3830 0x5652493b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 4 client.19395 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56524930b400 con 0x5652493b3830 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.454+0000 7f3cd659c640 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652493b3830 0x5652493b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.455+0000 7f3cd659c640 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652493b3830 0x5652493b5ca0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a41a6f78fa1107 server_cookie=ef7711f9a6a1a663 in_seq=0 out_seq=0 2024-03-09T00:06:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.455+0000 7f3cbdffb640 10 client.19395 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:25.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cbdffb640 1 -- 172.21.15.22:0/749127511 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56524930b400 con 0x5652493b3830 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cbdffb640 10 client.19395 handle_command_reply: tid=0 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 client.19395 shutdown 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 msgr2=0x5652493a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a0bf0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc000c00 tx=0x7f3ccc00aea0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 shutdown_connections 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5652493b3830 0x5652493b5ca0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652493a36a0 0x5652493a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cbdffb640 0 client.19395 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939aac0 0x5652493a0bf0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 --2- 172.21.15.22:0/749127511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 >> 172.21.15.22:0/749127511 conn(0x5652493948d0 msgr2=0x565249396620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 shutdown_connections 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 1 -- 172.21.15.22:0/749127511 wait complete. 2024-03-09T00:06:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.456+0000 7f3cdf808740 20 client.19395 trim_cache size 0 max 16384 2024-03-09T00:06:25.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 Processor -- start 2024-03-09T00:06:25.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 -- start start 2024-03-09T00:06:25.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x5652493aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652493a36a0 0x56524939f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81b0ac0 con 0x5652493a36a0 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81ae660 con 0x56524939aac0 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cdf808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81ac720 con 0x56524939a1a0 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cd6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652493a36a0 0x56524939f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cd6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652493a36a0 0x56524939f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51634/0 (socket says 172.21.15.22:51634) 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.468+0000 7f3cd6d9d640 1 -- 172.21.15.22:0/380645861 learned_addr learned my addr 172.21.15.22:0/380645861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd659c640 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x5652493aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd6d9d640 1 -- 172.21.15.22:0/380645861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 msgr2=0x5652493a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd6d9d640 1 -- 172.21.15.22:0/380645861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 msgr2=0x5652493aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x5652493aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd6d9d640 1 -- 172.21.15.22:0/380645861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56524939a8f0 con 0x5652493a36a0 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x5652493aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:25.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652493a36a0 0x56524939f690 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3cd002c700 tx=0x7f3cd002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5bd5a6a4e14c9a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cbffff640 1 -- 172.21.15.22:0/380645861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0039040 con 0x5652493a36a0 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cbffff640 1 -- 172.21.15.22:0/380645861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0054070 con 0x5652493a36a0 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.469+0000 7f3cbffff640 1 -- 172.21.15.22:0/380645861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0034440 con 0x5652493a36a0 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- 172.21.15.22:0/380645861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652493a36a0 msgr2=0x56524939f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652493a36a0 0x56524939f690 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3cd002c700 tx=0x7f3cd002e040 comp rx=0 tx=0).stop 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- 172.21.15.22:0/380645861 shutdown_connections 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5652493a36a0 0x56524939f690 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x5652493aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 --2- 172.21.15.22:0/380645861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56524939a1a0 0x5652493a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- 172.21.15.22:0/380645861 >> 172.21.15.22:0/380645861 conn(0x5652493948d0 msgr2=0x56524939ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- 172.21.15.22:0/380645861 shutdown_connections 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- 172.21.15.22:0/380645861 wait complete. 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- start start 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 Processor -- start 2024-03-09T00:06:25.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939a1a0 0x565249407c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x565249408190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x565249402060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81acd20 con 0x56524939a1a0 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81ae000 con 0x56524939aac0 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81ad340 con 0x5652493a36a0 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cdf808740 10 client.0 _ll_register_callbacks cb 0x7f3cc8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cd659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939a1a0 0x565249407c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cd659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939a1a0 0x565249407c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51638/0 (socket says 172.21.15.22:51638) 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cd659c640 1 -- 172.21.15.22:0/1856792368 learned_addr learned my addr 172.21.15.22:0/1856792368 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x565249408190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.470+0000 7f3cd6d9d640 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x565249402060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cd659c640 1 -- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 msgr2=0x565249402060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cd659c640 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x565249402060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cd659c640 1 -- 172.21.15.22:0/1856792368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 msgr2=0x565249408190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cd659c640 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x565249408190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cd659c640 1 -- 172.21.15.22:0/1856792368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565249402750 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cd659c640 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939a1a0 0x565249407c50 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009250 tx=0x7f3cc80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaf435d5025d3c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cbdffb640 1 -- 172.21.15.22:0/1856792368 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8032060 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cbdffb640 1 -- 172.21.15.22:0/1856792368 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3cc8004030 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cd0054070 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cbdffb640 1 -- 172.21.15.22:0/1856792368 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc8039050 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cdf808740 10 client.19401 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56524940f3c0 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cbdffb640 1 -- 172.21.15.22:0/1856792368 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0054070 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.471+0000 7f3cbdffb640 1 -- 172.21.15.22:0/1856792368 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3cc8024040 con 0x56524939a1a0 2024-03-09T00:06:25.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cdf808740 10 client.19401 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cdf808740 10 client.19401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cdf808740 20 client.19401 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cdf808740 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56524940f930 0x565249411d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cdf808740 4 client.19401 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565249305750 con 0x56524940f930 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56524940f930 0x565249411d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cbdffb640 10 client.19401 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cd5d9b640 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56524940f930 0x565249411d50 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d4b68c4ae6ff15 server_cookie=2e3f823fde03193d in_seq=0 out_seq=0 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cbdffb640 1 -- 172.21.15.22:0/1856792368 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565249305750 con 0x56524940f930 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cbdffb640 10 client.19401 handle_command_reply: tid=0 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.472+0000 7f3cdf808740 1 client.19401 shutdown 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939a1a0 msgr2=0x565249407c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939a1a0 0x565249407c50 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009250 tx=0x7f3cc80047c0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 shutdown_connections 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56524940f930 0x565249411d50 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652493a36a0 0x565249402060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56524939aac0 0x565249408190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cbdffb640 0 client.19401 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 --2- 172.21.15.22:0/1856792368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56524939a1a0 0x565249407c50 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 >> 172.21.15.22:0/1856792368 conn(0x5652493948d0 msgr2=0x5652493a0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 shutdown_connections 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 1 -- 172.21.15.22:0/1856792368 wait complete. 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdf808740 20 client.19401 trim_cache size 0 max 16384 2024-03-09T00:06:25.474 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:25.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cc00a2970 msgr2=0x7f3cc00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cc00a2970 0x7f3cc00a4e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0032530 tx=0x7f3cd0005c50 comp rx=0 tx=0).stop 2024-03-09T00:06:25.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 msgr2=0x7f3cd819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:25.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd819dcd0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00e9c0 tx=0x7f3ccc00ee80 comp rx=0 tx=0).stop 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 shutdown_connections 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3cc00a2970 0x7f3cc00a4e30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cd8197940 0x7f3cd8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3cd81054c0 0x7f3cd819dcd0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 --2- 172.21.15.22:0/1944418245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd81003f0 0x7f3cd819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 >> 172.21.15.22:0/1944418245 conn(0x7f3cd8076570 msgr2=0x7f3cd80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 shutdown_connections 2024-03-09T00:06:25.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:25.473+0000 7f3cdcc27640 1 -- 172.21.15.22:0/1944418245 wait complete. 2024-03-09T00:06:25.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:25 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:06:25.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:25 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:25.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:25 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:06:25.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:25 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:25.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:25 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:06:25.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:25 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1844995148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:25.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1185319 9.46 MB/sec execute 513 sec latency 16.803 ms 2024-03-09T00:06:26.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:26 smithi192 ceph-mon[35995]: pgmap v375: 129 pgs: 129 active+clean; 93 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-09T00:06:26.479 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:06:26.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:06:26.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:06:26.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:26 smithi022 ceph-mon[31894]: pgmap v375: 129 pgs: 129 active+clean; 93 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-09T00:06:26.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:26 smithi062 ceph-mon[36129]: pgmap v375: 129 pgs: 129 active+clean; 93 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-09T00:06:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 -- 172.21.15.22:0/3585168873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed810db40 msgr2=0x7f6ed810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 --2- 172.21.15.22:0/3585168873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed810db40 0x7f6ed810dfc0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009900 tx=0x7f6ec802bc40 comp rx=0 tx=0).stop 2024-03-09T00:06:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 -- 172.21.15.22:0/3585168873 shutdown_connections 2024-03-09T00:06:26.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 --2- 172.21.15.22:0/3585168873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed811aad0 0x7f6ed811cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 --2- 172.21.15.22:0/3585168873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed810db40 0x7f6ed810dfc0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 --2- 172.21.15.22:0/3585168873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 0x7f6ed810d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 -- 172.21.15.22:0/3585168873 >> 172.21.15.22:0/3585168873 conn(0x7f6ed806c3d0 msgr2=0x7f6ed806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 -- 172.21.15.22:0/3585168873 shutdown_connections 2024-03-09T00:06:26.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.622+0000 7f6edec73640 1 -- 172.21.15.22:0/3585168873 wait complete. 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 Processor -- start 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 -- start start 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed810db40 0x7f6ed81ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 0x7f6ed81acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed811aad0 0x7f6ed81a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8120450 con 0x7f6ed811aad0 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81202d0 con 0x7f6ed810fb10 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edec73640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81205d0 con 0x7f6ed810db40 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 0x7f6ed81acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6edca0f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed810db40 0x7f6ed81ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.623+0000 7f6edd210640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed811aad0 0x7f6ed81a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 0x7f6ed81acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34230/0 (socket says 172.21.15.22:34230) 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6edca0f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed810db40 0x7f6ed81ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59248/0 (socket says 172.21.15.22:59248) 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6edd210640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed811aad0 0x7f6ed81a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51656/0 (socket says 172.21.15.22:51656) 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 -- 172.21.15.22:0/2399881497 learned_addr learned my addr 172.21.15.22:0/2399881497 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 -- 172.21.15.22:0/2399881497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed810db40 msgr2=0x7f6ed81ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed810db40 0x7f6ed81ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 -- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed811aad0 msgr2=0x7f6ed81a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed811aad0 0x7f6ed81a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 -- 172.21.15.22:0/2399881497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed81a6f80 con 0x7f6ed810fb10 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6edd210640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed811aad0 0x7f6ed81a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed7fff640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 0x7f6ed81acb80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802efd0 tx=0x7f6ec80043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed5ffb640 1 -- 172.21.15.22:0/2399881497 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8041070 con 0x7f6ed810fb10 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed5ffb640 1 -- 172.21.15.22:0/2399881497 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ec802cca0 con 0x7f6ed810fb10 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.624+0000 7f6ed5ffb640 1 -- 172.21.15.22:0/2399881497 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8003d90 con 0x7f6ed810fb10 2024-03-09T00:06:26.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.625+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed81a7210 con 0x7f6ed810fb10 2024-03-09T00:06:26.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.625+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed81b3490 con 0x7f6ed810fb10 2024-03-09T00:06:26.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ed5ffb640 1 -- 172.21.15.22:0/2399881497 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6ec8044070 con 0x7f6ed810fb10 2024-03-09T00:06:26.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 Processor -- start 2024-03-09T00:06:26.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 -- start start 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e16a0 0x55fc974e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ed5ffb640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ec00a2a30 0x7f6ec00a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974ec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc97444b40 con 0x55fc974e0d80 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc973edd80 con 0x55fc974e16a0 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ee1852740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc97440be0 con 0x55fc974ea280 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6edd210640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6ed7fff640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ec00a2a30 0x7f6ec00a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6edd210640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974e1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51666/0 (socket says 172.21.15.22:51666) 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6edca0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e16a0 0x55fc974e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.627+0000 7f6edd210640 1 -- 172.21.15.22:0/1066035153 learned_addr learned my addr 172.21.15.22:0/1066035153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6edd210640 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6ed5ffb640 1 -- 172.21.15.22:0/2399881497 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6ec80f0a90 con 0x7f6ed810fb10 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6edca0f640 1 -- 172.21.15.22:0/1066035153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 msgr2=0x55fc974ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6edca0f640 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6edca0f640 1 -- 172.21.15.22:0/1066035153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 msgr2=0x55fc974e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6edca0f640 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974e1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6edca0f640 1 -- 172.21.15.22:0/1066035153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc972217e0 con 0x55fc974e16a0 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6edca0f640 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e16a0 0x55fc974e9d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009fa0 tx=0x7f6ed00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b8e39c2beacdfe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6ed7fff640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ec00a2a30 0x7f6ec00a4ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80bcbb0 tx=0x7f6ec804b590 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6ebeffd640 1 -- 172.21.15.22:0/1066035153 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0003c30 con 0x55fc974e16a0 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6ebeffd640 1 -- 172.21.15.22:0/1066035153 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ed0003dd0 con 0x55fc974e16a0 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.628+0000 7f6ebeffd640 1 -- 172.21.15.22:0/1066035153 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000fb20 con 0x55fc974e16a0 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 -- 172.21.15.22:0/1066035153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e16a0 msgr2=0x55fc974e9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e16a0 0x55fc974e9d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009fa0 tx=0x7f6ed00118d0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 -- 172.21.15.22:0/1066035153 shutdown_connections 2024-03-09T00:06:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e16a0 0x55fc974e9d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1066035153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 -- 172.21.15.22:0/1066035153 >> 172.21.15.22:0/1066035153 conn(0x55fc974db4b0 msgr2=0x55fc974db890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 -- 172.21.15.22:0/1066035153 shutdown_connections 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 -- 172.21.15.22:0/1066035153 wait complete. 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 -- start start 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.629+0000 7f6ee1852740 1 Processor -- start 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974e7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6ee1852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc974ed070 con 0x55fc974e16a0 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6ee1852740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc974011c0 con 0x55fc974e0d80 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6ee1852740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc974ed1f0 con 0x55fc974ea280 2024-03-09T00:06:26.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6ee1852740 10 client.0 _ll_register_callbacks cb 0x55fc973f07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edca0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edca0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51674/0 (socket says 172.21.15.22:51674) 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edca0f640 1 -- 172.21.15.22:0/4272048955 learned_addr learned my addr 172.21.15.22:0/4272048955 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 -- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 msgr2=0x55fc974e7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974e7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 -- 172.21.15.22:0/4272048955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 msgr2=0x55fc974e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974e70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 -- 172.21.15.22:0/4272048955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc97281f70 con 0x55fc974e0d80 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edca0f640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6edd210640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e9d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005f60 tx=0x7f6ecc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854e0c8678fac6a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.630+0000 7f6eb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:06:26.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6eb7fff640 1 -- 172.21.15.22:0/4272048955 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00fc20 con 0x55fc974e0d80 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ed0003dd0 con 0x55fc974e0d80 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6eb7fff640 1 -- 172.21.15.22:0/4272048955 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6ecc00fdc0 con 0x55fc974e0d80 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6eb7fff640 1 -- 172.21.15.22:0/4272048955 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ecc005020 con 0x55fc974e0d80 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 10 client.28712 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fc97119170 con 0x55fc974e0d80 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6eb7fff640 1 -- 172.21.15.22:0/4272048955 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ed0003dd0 con 0x55fc974e0d80 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6eb7fff640 1 -- 172.21.15.22:0/4272048955 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ecc023450 con 0x55fc974e0d80 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 10 client.28712 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 10 client.28712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 20 client.28712 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc974fa3f0 0x55fc974fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 4 client.28712 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc9741a960 con 0x55fc974fa3f0 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.631+0000 7f6edca0f640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc974fa3f0 0x55fc974fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.632+0000 7f6edca0f640 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc974fa3f0 0x55fc974fc7e0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f085a0d867f310ef server_cookie=bec21404745ae7a9 in_seq=0 out_seq=0 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.632+0000 7f6eb7fff640 10 client.28712 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6eb7fff640 1 -- 172.21.15.22:0/4272048955 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc9741a960 con 0x55fc974fa3f0 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6eb7fff640 10 client.28712 handle_command_reply: tid=0 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 client.28712 shutdown 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 msgr2=0x55fc974e9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e9d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005f60 tx=0x7f6ecc005f90 comp rx=0 tx=0).stop 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 shutdown_connections 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc974fa3f0 0x55fc974fc7e0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974e7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6eb7fff640 0 client.28712 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 --2- 172.21.15.22:0/4272048955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e9d10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 >> 172.21.15.22:0/4272048955 conn(0x55fc974db4b0 msgr2=0x55fc974dd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 shutdown_connections 2024-03-09T00:06:26.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 1 -- 172.21.15.22:0/4272048955 wait complete. 2024-03-09T00:06:26.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.633+0000 7f6ee1852740 20 client.28712 trim_cache size 0 max 16384 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 Processor -- start 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 -- start start 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81bfa30 con 0x55fc974e16a0 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81bd5d0 con 0x55fc974e0d80 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6ee1852740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81bb690 con 0x55fc974ea280 2024-03-09T00:06:26.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6edca0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6edca0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51688/0 (socket says 172.21.15.22:51688) 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6edd210640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6edca0f640 1 -- 172.21.15.22:0/1737129677 learned_addr learned my addr 172.21.15.22:0/1737129677 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.649+0000 7f6edd210640 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6edd210640 1 -- 172.21.15.22:0/1737129677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 msgr2=0x55fc974f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6edd210640 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6edd210640 1 -- 172.21.15.22:0/1737129677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 msgr2=0x55fc974f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6edd210640 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6edd210640 1 -- 172.21.15.22:0/1737129677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc974e7b00 con 0x55fc974e0d80 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6edd210640 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e1a80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005910 tx=0x7f6ecc009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d54651e92241539b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6ebe7fc640 1 -- 172.21.15.22:0/1737129677 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc0139d0 con 0x55fc974e0d80 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6ebe7fc640 1 -- 172.21.15.22:0/1737129677 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc023450 con 0x55fc974e0d80 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.650+0000 7f6ebe7fc640 1 -- 172.21.15.22:0/1737129677 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc0046c0 con 0x55fc974e0d80 2024-03-09T00:06:26.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 -- 172.21.15.22:0/1737129677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 msgr2=0x55fc974e1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e1a80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005910 tx=0x7f6ecc009f70 comp rx=0 tx=0).stop 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 -- 172.21.15.22:0/1737129677 shutdown_connections 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974ea280 0x55fc974f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e16a0 0x55fc974f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1737129677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974e0d80 0x55fc974e1a80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 -- 172.21.15.22:0/1737129677 >> 172.21.15.22:0/1737129677 conn(0x55fc974db4b0 msgr2=0x55fc975010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 -- 172.21.15.22:0/1737129677 shutdown_connections 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 -- 172.21.15.22:0/1737129677 wait complete. 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 -- start start 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.651+0000 7f6ee1852740 1 Processor -- start 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974e16a0 0x55fc974f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974ea280 0x55fc97550860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81bbc90 con 0x55fc974e0d80 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81bcf70 con 0x55fc974ea280 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81bc2b0 con 0x55fc974e16a0 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 10 client.0 _ll_register_callbacks cb 0x7f6ed0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974f05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51694/0 (socket says 172.21.15.22:51694) 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 -- 172.21.15.22:0/1038718732 learned_addr learned my addr 172.21.15.22:0/1038718732 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edca0f640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974e16a0 0x55fc974f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974ea280 0x55fc97550860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 -- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974e16a0 msgr2=0x55fc974f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974e16a0 0x55fc974f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 -- 172.21.15.22:0/1038718732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974ea280 msgr2=0x55fc97550860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974ea280 0x55fc97550860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 -- 172.21.15.22:0/1038718732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc97550dd0 con 0x55fc974e0d80 2024-03-09T00:06:26.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edca0f640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974e16a0 0x55fc974f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6edd210640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974f05e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc006010 tx=0x7f6ecc00fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fb7baa4cfd781a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6eb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6eb7fff640 1 -- 172.21.15.22:0/1038718732 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc0139d0 con 0x55fc974e0d80 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6eb7fff640 1 -- 172.21.15.22:0/1038718732 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6ecc0239c0 con 0x55fc974e0d80 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.652+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ecc023450 con 0x55fc974e0d80 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6eb7fff640 1 -- 172.21.15.22:0/1038718732 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ecc004680 con 0x55fc974e0d80 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 10 client.19419 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fc975520f0 con 0x55fc974e0d80 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6eb7fff640 1 -- 172.21.15.22:0/1038718732 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc023450 con 0x55fc974e0d80 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6eb7fff640 1 -- 172.21.15.22:0/1038718732 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ecc010840 con 0x55fc974e0d80 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 10 client.19419 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 10 client.19419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 20 client.19419 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc975526b0 0x55fc97554aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 4 client.19419 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:06:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fc97406080 con 0x55fc975526b0 2024-03-09T00:06:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.653+0000 7f6edca0f640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc975526b0 0x55fc97554aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.654+0000 7f6edca0f640 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc975526b0 0x55fc97554aa0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542cf116d45dd297 server_cookie=ea2f7b457dd1f4f1 in_seq=0 out_seq=0 2024-03-09T00:06:26.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.654+0000 7f6eb7fff640 10 client.19419 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6eb7fff640 1 -- 172.21.15.22:0/1038718732 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fc97406080 con 0x55fc975526b0 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6eb7fff640 10 client.19419 handle_command_reply: tid=0 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 client.19419 shutdown 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 msgr2=0x55fc974f05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974f05e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc006010 tx=0x7f6ecc00fc20 comp rx=0 tx=0).stop 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 shutdown_connections 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55fc975526b0 0x55fc97554aa0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc974ea280 0x55fc97550860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc974e16a0 0x55fc974f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6eb7fff640 0 client.19419 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 --2- 172.21.15.22:0/1038718732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fc974e0d80 0x55fc974f05e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 >> 172.21.15.22:0/1038718732 conn(0x55fc974db4b0 msgr2=0x55fc974dc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 shutdown_connections 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 1 -- 172.21.15.22:0/1038718732 wait complete. 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.655+0000 7f6ee1852740 20 client.19419 trim_cache size 0 max 16384 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:06:26.656 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "edf2dea2-c554-48c7-a276-e41a06d9023d", 2024-03-09T00:06:26.657 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:06:26.657 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:26.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ec00a2a30 msgr2=0x7f6ec00a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ec00a2a30 0x7f6ec00a4ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80bcbb0 tx=0x7f6ec804b590 comp rx=0 tx=0).stop 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 msgr2=0x7f6ed81acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 0x7f6ed81acb80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802efd0 tx=0x7f6ec80043b0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 shutdown_connections 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6ec00a2a30 0x7f6ec00a4ef0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ed811aad0 0x7f6ed81a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed810fb10 0x7f6ed81acb80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 --2- 172.21.15.22:0/2399881497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed810db40 0x7f6ed81ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 >> 172.21.15.22:0/2399881497 conn(0x7f6ed806c3d0 msgr2=0x7f6ed8071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 shutdown_connections 2024-03-09T00:06:26.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.656+0000 7f6edec73640 1 -- 172.21.15.22:0/2399881497 wait complete. 2024-03-09T00:06:26.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:06:26.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:06:26.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1186717 9.46 MB/sec execute 514 sec latency 17.802 ms 2024-03-09T00:06:26.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.812+0000 7f94bed9f640 1 -- 172.21.15.22:0/286415118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8100c70 msgr2=0x7f94b8104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.812+0000 7f94bed9f640 1 --2- 172.21.15.22:0/286415118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8100c70 0x7f94b8104cb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009900 tx=0x7f94a802bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.812+0000 7f94bed9f640 1 -- 172.21.15.22:0/286415118 shutdown_connections 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.812+0000 7f94bed9f640 1 --2- 172.21.15.22:0/286415118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94b8105390 0x7f94b810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.812+0000 7f94bed9f640 1 --2- 172.21.15.22:0/286415118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8100c70 0x7f94b8104cb0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.812+0000 7f94bed9f640 1 --2- 172.21.15.22:0/286415118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b81002c0 0x7f94b81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.812+0000 7f94bed9f640 1 -- 172.21.15.22:0/286415118 >> 172.21.15.22:0/286415118 conn(0x7f94b80fbb40 msgr2=0x7f94b80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 -- 172.21.15.22:0/286415118 shutdown_connections 2024-03-09T00:06:26.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 -- 172.21.15.22:0/286415118 wait complete. 2024-03-09T00:06:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 Processor -- start 2024-03-09T00:06:26.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 -- start start 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94b81002c0 0x7f94b8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8105390 0x7f94b8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8199390 0x7f94b81a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8110470 con 0x7f94b8105390 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81102f0 con 0x7f94b8199390 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.813+0000 7f94bed9f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81105f0 con 0x7f94b81002c0 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8105390 0x7f94b8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bdd9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94b81002c0 0x7f94b8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8105390 0x7f94b8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51724/0 (socket says 172.21.15.22:51724) 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 -- 172.21.15.22:0/3736389301 learned_addr learned my addr 172.21.15.22:0/3736389301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94be59e640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8199390 0x7f94b81a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 -- 172.21.15.22:0/3736389301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94b81002c0 msgr2=0x7f94b8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94b81002c0 0x7f94b8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 -- 172.21.15.22:0/3736389301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8199390 msgr2=0x7f94b81a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8199390 0x7f94b81a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 -- 172.21.15.22:0/3736389301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b81a4110 con 0x7f94b8105390 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94b81002c0 0x7f94b8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bd59c640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8105390 0x7f94b8198db0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f94a802c240 tx=0x7f94a8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94aeffd640 1 -- 172.21.15.22:0/3736389301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8043020 con 0x7f94b8105390 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94aeffd640 1 -- 172.21.15.22:0/3736389301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94a80054e0 con 0x7f94b8105390 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b81a43a0 con 0x7f94b8105390 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94aeffd640 1 -- 172.21.15.22:0/3736389301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8003b80 con 0x7f94b8105390 2024-03-09T00:06:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.814+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b81a48f0 con 0x7f94b8105390 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.816+0000 7f94aeffd640 1 -- 172.21.15.22:0/3736389301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f94a8046020 con 0x7f94b8105390 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 Processor -- start 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 -- start start 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf32cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf3358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf3381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcf28fe00 con 0x557fcf32d220 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcf238b10 con 0x557fcf32c900 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcf28bea0 con 0x557fcf335e00 2024-03-09T00:06:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94aeffd640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f94a40a2970 0x7f94a40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94be59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf32cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94bd59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf3381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94be59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf32cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34268/0 (socket says 172.21.15.22:34268) 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94be59e640 1 -- 172.21.15.22:0/1232339083 learned_addr learned my addr 172.21.15.22:0/1232339083 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf3358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.817+0000 7f94be59e640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f94a40a2970 0x7f94a40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94aeffd640 1 -- 172.21.15.22:0/3736389301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f94a80f11f0 con 0x7f94b8105390 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94bd59c640 1 -- 172.21.15.22:0/1232339083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 msgr2=0x557fcf32cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94bd59c640 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf32cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94bd59c640 1 -- 172.21.15.22:0/1232339083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 msgr2=0x557fcf3358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94bd59c640 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf3358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94bd59c640 1 -- 172.21.15.22:0/1232339083 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fcf06e7d0 con 0x557fcf335e00 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94bd59c640 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf3381f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94a80ee650 tx=0x7f94a8004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810ea905cebcfc03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94be59e640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f94a40a2970 0x7f94a40a4e30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94b4005c40 tx=0x7f94b4009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94a3fff640 1 -- 172.21.15.22:0/1232339083 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80bcec0 con 0x557fcf335e00 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94a3fff640 1 -- 172.21.15.22:0/1232339083 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94a80bdc60 con 0x557fcf335e00 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.818+0000 7f94a3fff640 1 -- 172.21.15.22:0/1232339083 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80ee7c0 con 0x557fcf335e00 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1232339083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 msgr2=0x557fcf3381f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf3381f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94a80ee650 tx=0x7f94a8004ee0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1232339083 shutdown_connections 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf3381f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf3358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1232339083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf32cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1232339083 >> 172.21.15.22:0/1232339083 conn(0x557fcf327030 msgr2=0x557fcf327410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1232339083 shutdown_connections 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1232339083 wait complete. 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 -- start start 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.819+0000 7f94c2bdc740 1 Processor -- start 2024-03-09T00:06:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf3355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf332940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf332e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcf3389b0 con 0x557fcf32d220 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcf24c0a0 con 0x557fcf32c900 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcf338b30 con 0x557fcf335e00 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 10 client.0 _ll_register_callbacks cb 0x557fcf24ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf332940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf332940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51744/0 (socket says 172.21.15.22:51744) 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 -- 172.21.15.22:0/1233749855 learned_addr learned my addr 172.21.15.22:0/1233749855 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94be59e640 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf332e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf3355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 -- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 msgr2=0x557fcf332e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf332e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 -- 172.21.15.22:0/1233749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 msgr2=0x557fcf3355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf3355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 -- 172.21.15.22:0/1233749855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fcefa43f0 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94bd59c640 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf332940 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f94a80eeb40 tx=0x7f94a8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6326cfcb058e9fa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94a17fa640 1 -- 172.21.15.22:0/1233749855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80bcec0 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94a17fa640 1 -- 172.21.15.22:0/1233749855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f94a802cb90 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.820+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94a80bdc60 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94a17fa640 1 -- 172.21.15.22:0/1233749855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94a80bd560 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 10 client.19431 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557fcef640f0 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94a17fa640 1 -- 172.21.15.22:0/1233749855 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94a80bdc60 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94a17fa640 1 -- 172.21.15.22:0/1233749855 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f94a80d6110 con 0x557fcf32d220 2024-03-09T00:06:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 10 client.19431 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 10 client.19431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 20 client.19431 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf3478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 4 client.19431 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fcf29d340 con 0x557fcf3454b0 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.821+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf3478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.822+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf3478a0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a426ec4d5e39301 server_cookie=5ca68f290368c27c in_seq=0 out_seq=0 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.822+0000 7f94a17fa640 10 client.19431 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94a17fa640 1 -- 172.21.15.22:0/1233749855 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fcf29d340 con 0x557fcf3454b0 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94a17fa640 10 client.19431 handle_command_reply: tid=0 2024-03-09T00:06:26.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 client.19431 shutdown 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 msgr2=0x557fcf332940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf332940 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f94a80eeb40 tx=0x7f94a8003680 comp rx=0 tx=0).stop 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 shutdown_connections 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf3478a0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf335e00 0x557fcf332e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94a17fa640 0 client.19431 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32d220 0x557fcf332940 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/1233749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32c900 0x557fcf3355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 >> 172.21.15.22:0/1233749855 conn(0x557fcf327030 msgr2=0x557fcf328d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 shutdown_connections 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 1 -- 172.21.15.22:0/1233749855 wait complete. 2024-03-09T00:06:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.823+0000 7f94c2bdc740 20 client.19431 trim_cache size 0 max 16384 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 Processor -- start 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 -- start start 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32c900 0x557fcf335070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32d220 0x557fcf3355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf335e00 0x557fcf33cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81b09c0 con 0x557fcf32c900 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81ae560 con 0x557fcf335e00 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81ac620 con 0x557fcf32d220 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32c900 0x557fcf335070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32c900 0x557fcf335070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51750/0 (socket says 172.21.15.22:51750) 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 -- 172.21.15.22:0/775822906 learned_addr learned my addr 172.21.15.22:0/775822906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bd59c640 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32d220 0x557fcf3355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 -- 172.21.15.22:0/775822906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32d220 msgr2=0x557fcf3355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32d220 0x557fcf3355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 -- 172.21.15.22:0/775822906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf335e00 msgr2=0x557fcf33cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf335e00 0x557fcf33cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.839+0000 7f94bdd9d640 1 -- 172.21.15.22:0/775822906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fcf326890 con 0x557fcf32c900 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32c900 0x557fcf335070 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f94b0011ab0 tx=0x7f94b000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6679c0446c9652a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94a37fe640 1 -- 172.21.15.22:0/775822906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0010b90 con 0x557fcf32c900 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94a37fe640 1 -- 172.21.15.22:0/775822906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94b0005500 con 0x557fcf32c900 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94a37fe640 1 -- 172.21.15.22:0/775822906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b001f0e0 con 0x557fcf32c900 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94c2bdc740 1 -- 172.21.15.22:0/775822906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32c900 msgr2=0x557fcf335070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32c900 0x557fcf335070 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f94b0011ab0 tx=0x7f94b000b390 comp rx=0 tx=0).stop 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94c2bdc740 1 -- 172.21.15.22:0/775822906 shutdown_connections 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf335e00 0x557fcf33cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32d220 0x557fcf3355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.840+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/775822906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf32c900 0x557fcf335070 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 -- 172.21.15.22:0/775822906 >> 172.21.15.22:0/775822906 conn(0x557fcf327030 msgr2=0x557fcf331210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 -- 172.21.15.22:0/775822906 shutdown_connections 2024-03-09T00:06:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 -- 172.21.15.22:0/775822906 wait complete. 2024-03-09T00:06:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 -- start start 2024-03-09T00:06:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 Processor -- start 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32c900 0x557fcf33d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32d220 0x557fcf33b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf335e00 0x557fcf33bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81acc20 con 0x557fcf335e00 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81adf00 con 0x557fcf32d220 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.841+0000 7f94c2bdc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81ad240 con 0x557fcf32c900 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94c2bdc740 10 client.0 _ll_register_callbacks cb 0x7f94b000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94be59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf335e00 0x557fcf33bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bd59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32d220 0x557fcf33b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32c900 0x557fcf33d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bd59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32d220 0x557fcf33b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34286/0 (socket says 172.21.15.22:34286) 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32c900 0x557fcf33d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59332/0 (socket says 172.21.15.22:59332) 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bd59c640 1 -- 172.21.15.22:0/2359680066 learned_addr learned my addr 172.21.15.22:0/2359680066 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94be59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf335e00 0x557fcf33bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51756/0 (socket says 172.21.15.22:51756) 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 -- 172.21.15.22:0/2359680066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32d220 msgr2=0x557fcf33b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32d220 0x557fcf33b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 -- 172.21.15.22:0/2359680066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf335e00 msgr2=0x557fcf33bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf335e00 0x557fcf33bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 -- 172.21.15.22:0/2359680066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fcf39b770 con 0x557fcf32c900 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94be59e640 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf335e00 0x557fcf33bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94bdd9d640 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32c900 0x557fcf33d010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94b0004a60 tx=0x7f94b0011760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8f4cc5d7b5a5a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94a17fa640 1 -- 172.21.15.22:0/2359680066 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0004020 con 0x557fcf32c900 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94b0005500 con 0x557fcf32c900 2024-03-09T00:06:26.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94a17fa640 1 -- 172.21.15.22:0/2359680066 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f94b00041c0 con 0x557fcf32c900 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94a17fa640 1 -- 172.21.15.22:0/2359680066 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94b0038050 con 0x557fcf32c900 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.842+0000 7f94c2bdc740 10 client.28864 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557fcf39c8e0 con 0x557fcf32c900 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94a17fa640 1 -- 172.21.15.22:0/2359680066 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94b0005500 con 0x557fcf32c900 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94a17fa640 1 -- 172.21.15.22:0/2359680066 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f94b0005120 con 0x557fcf32c900 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94c2bdc740 10 client.28864 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94c2bdc740 10 client.28864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94c2bdc740 20 client.28864 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf39ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94c2bdc740 4 client.28864 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557fcf297690 con 0x557fcf3454b0 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94bd59c640 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf39ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94bd59c640 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf39ef30 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae1d404d4c842dc server_cookie=af1f1e0c0b61944a in_seq=0 out_seq=0 2024-03-09T00:06:26.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.843+0000 7f94a17fa640 10 client.28864 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94a17fa640 1 -- 172.21.15.22:0/2359680066 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557fcf297690 con 0x557fcf3454b0 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94a17fa640 10 client.28864 handle_command_reply: tid=0 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 client.28864 shutdown 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32c900 msgr2=0x557fcf33d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32c900 0x557fcf33d010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94b0004a60 tx=0x7f94b0011760 comp rx=0 tx=0).stop 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 shutdown_connections 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557fcf3454b0 0x557fcf39ef30 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94a17fa640 0 client.28864 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557fcf335e00 0x557fcf33bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fcf32d220 0x557fcf33b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 --2- 172.21.15.22:0/2359680066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fcf32c900 0x557fcf33d010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 >> 172.21.15.22:0/2359680066 conn(0x557fcf327030 msgr2=0x557fcf32d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 shutdown_connections 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 1 -- 172.21.15.22:0/2359680066 wait complete. 2024-03-09T00:06:26.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.845+0000 7f94c2bdc740 20 client.28864 trim_cache size 0 max 16384 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: "edf2dea2-c554-48c7-a276-e41a06d9023d": { 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "edf2dea2-c554-48c7-a276-e41a06d9023d", 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:06:26.846 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f94a40a2970 msgr2=0x7f94a40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f94a40a2970 0x7f94a40a4e30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94b4005c40 tx=0x7f94b4009500 comp rx=0 tx=0).stop 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8105390 msgr2=0x7f94b8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8105390 0x7f94b8198db0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f94a802c240 tx=0x7f94a8004360 comp rx=0 tx=0).stop 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 shutdown_connections 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f94a40a2970 0x7f94a40a4e30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94b8199390 0x7f94b81a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94b8105390 0x7f94b8198db0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 --2- 172.21.15.22:0/3736389301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94b81002c0 0x7f94b8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 >> 172.21.15.22:0/3736389301 conn(0x7f94b80fbb40 msgr2=0x7f94b80fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 shutdown_connections 2024-03-09T00:06:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:26.846+0000 7f94bed9f640 1 -- 172.21.15.22:0/3736389301 wait complete. 2024-03-09T00:06:26.851 INFO:tasks.cephfs.filesystem:scrub status for tag:edf2dea2-c554-48c7-a276-e41a06d9023d - {'path': '/', 'tag': 'edf2dea2-c554-48c7-a276-e41a06d9023d', 'options': 'recursive,force'} 2024-03-09T00:06:27.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:27 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:06:27.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:27 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:27.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:27 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:06:27.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:27 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:27.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:27 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:06:27.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:27 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:27.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:27 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:06:27.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:27 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:27.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:27 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:06:27.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:27 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:27.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:27 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:06:27.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:27 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:06:27.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1188185 9.45 MB/sec execute 515 sec latency 93.052 ms 2024-03-09T00:06:28.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:28 smithi192 ceph-mon[35995]: pgmap v376: 129 pgs: 129 active+clean; 93 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-09T00:06:28.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:28 smithi022 ceph-mon[31894]: pgmap v376: 129 pgs: 129 active+clean; 93 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-09T00:06:28.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:28 smithi062 ceph-mon[36129]: pgmap v376: 129 pgs: 129 active+clean; 93 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-09T00:06:28.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1189954 9.46 MB/sec execute 516 sec latency 106.067 ms 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 -- 172.21.15.22:0/1691433093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0105190 msgr2=0x7f37f0105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 --2- 172.21.15.22:0/1691433093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0105190 0x7f37f0105570 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009900 tx=0x7f37e002bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 -- 172.21.15.22:0/1691433093 shutdown_connections 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 --2- 172.21.15.22:0/1691433093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f0100e00 0x7f37f010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 --2- 172.21.15.22:0/1691433093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37f0100440 0x7f37f01008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 --2- 172.21.15.22:0/1691433093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0105190 0x7f37f0105570 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 -- 172.21.15.22:0/1691433093 >> 172.21.15.22:0/1691433093 conn(0x7f37f0077040 msgr2=0x7f37f0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 -- 172.21.15.22:0/1691433093 shutdown_connections 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 -- 172.21.15.22:0/1691433093 wait complete. 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 Processor -- start 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.856+0000 7f37f5789640 1 -- start start 2024-03-09T00:06:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0100440 0x7f37f019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f0100e00 0x7f37f019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37f0105190 0x7f37f0196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f0110630 con 0x7f37f0100440 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01104b0 con 0x7f37f0105190 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01107b0 con 0x7f37f0100e00 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0100440 0x7f37f019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0100440 0x7f37f019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51776/0 (socket says 172.21.15.22:51776) 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 -- 172.21.15.22:0/3493588661 learned_addr learned my addr 172.21.15.22:0/3493588661 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f0100e00 0x7f37f019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37f0105190 0x7f37f0196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 -- 172.21.15.22:0/3493588661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f0100e00 msgr2=0x7f37f019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f0100e00 0x7f37f019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 -- 172.21.15.22:0/3493588661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37f0105190 msgr2=0x7f37f0196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37f0105190 0x7f37f0196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 -- 172.21.15.22:0/3493588661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f0197010 con 0x7f37f0100440 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37f0105190 0x7f37f0196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0100440 0x7f37f019c6e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f37e0005940 tx=0x7f37e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37e7fff640 1 -- 172.21.15.22:0/3493588661 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0041070 con 0x7f37f0100440 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37e7fff640 1 -- 172.21.15.22:0/3493588661 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e002ccc0 con 0x7f37f0100440 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37e7fff640 1 -- 172.21.15.22:0/3493588661 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0003d80 con 0x7f37f0100440 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f01972a0 con 0x7f37f0100440 2024-03-09T00:06:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.857+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f0068970 con 0x7f37f0100440 2024-03-09T00:06:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.859+0000 7f37e7fff640 1 -- 172.21.15.22:0/3493588661 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f37e0044070 con 0x7f37f0100440 2024-03-09T00:06:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 Processor -- start 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37e7fff640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f37dc0a2a10 0x7f37dc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 -- start start 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37e7fff640 1 -- 172.21.15.22:0/3493588661 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f37e00f1030 con 0x7f37f0100440 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f37dc0a2a10 0x7f37dc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651909365f0 con 0x5651909d63e0 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651908df930 con 0x5651909d7150 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.860+0000 7f37f836a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565190932690 con 0x5651909df960 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51784/0 (socket says 172.21.15.22:51784) 2024-03-09T00:06:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37ee7fc640 1 -- 172.21.15.22:0/4034770610 learned_addr learned my addr 172.21.15.22:0/4034770610 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37eeffd640 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37eeffd640 1 -- 172.21.15.22:0/4034770610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 msgr2=0x5651909e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37eeffd640 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37eeffd640 1 -- 172.21.15.22:0/4034770610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 msgr2=0x5651909d6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37eeffd640 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37eeffd640 1 -- 172.21.15.22:0/4034770610 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56519064b210 con 0x5651909d7150 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f37dc0a2a10 0x7f37dc0a4ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005fd0 tx=0x7f37e800e6c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37eeffd640 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909d5ea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ee490 tx=0x7f37e00bcfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3396ac5583be496e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37e57fa640 1 -- 172.21.15.22:0/4034770610 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e004b040 con 0x5651909d7150 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37e57fa640 1 -- 172.21.15.22:0/4034770610 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e00bd3a0 con 0x5651909d7150 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.861+0000 7f37e57fa640 1 -- 172.21.15.22:0/4034770610 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00b9a10 con 0x5651909d7150 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 -- 172.21.15.22:0/4034770610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 msgr2=0x5651909d5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909d5ea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ee490 tx=0x7f37e00bcfa0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 -- 172.21.15.22:0/4034770610 shutdown_connections 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 --2- 172.21.15.22:0/4034770610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909d5ea0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 -- 172.21.15.22:0/4034770610 >> 172.21.15.22:0/4034770610 conn(0x5651909995b0 msgr2=0x5651909d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 -- 172.21.15.22:0/4034770610 shutdown_connections 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 -- 172.21.15.22:0/4034770610 wait complete. 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 -- start start 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.862+0000 7f37f836a740 1 Processor -- start 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37f836a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d63e0 0x5651909dca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37f836a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d7150 0x5651909dcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37f836a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37f836a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651909e2450 con 0x5651909d7150 2024-03-09T00:06:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37f836a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651908f2d90 con 0x5651909d63e0 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37f836a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651909e25d0 con 0x5651909df960 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37f836a740 10 client.0 _ll_register_callbacks cb 0x56519064aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d7150 0x5651909dcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37eeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d63e0 0x5651909dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37eeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d63e0 0x5651909dca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34326/0 (socket says 172.21.15.22:34326) 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37eeffd640 1 -- 172.21.15.22:0/3045176744 learned_addr learned my addr 172.21.15.22:0/3045176744 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d7150 0x5651909dcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51796/0 (socket says 172.21.15.22:51796) 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 -- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 msgr2=0x5651909dd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909dd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 -- 172.21.15.22:0/3045176744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d63e0 msgr2=0x5651909dca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d63e0 0x5651909dca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 -- 172.21.15.22:0/3045176744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565190717620 con 0x5651909d7150 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d63e0 0x5651909dca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d7150 0x5651909dcf80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f37d80126d0 tx=0x7f37d8012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9038e5a94dfa7efa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.863+0000 7f37d67fc640 1 -- 172.21.15.22:0/3045176744 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004050 con 0x5651909d7150 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37d67fc640 1 -- 172.21.15.22:0/3045176744 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f37d80041f0 con 0x5651909d7150 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37e00bd3a0 con 0x5651909d7150 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37d67fc640 1 -- 172.21.15.22:0/3045176744 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37d80043f0 con 0x5651909d7150 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 10 client.19449 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5651909ee740 con 0x5651909d7150 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37d67fc640 1 -- 172.21.15.22:0/3045176744 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e00bd3a0 con 0x5651909d7150 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37d67fc640 1 -- 172.21.15.22:0/3045176744 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37d8003cd0 con 0x5651909d7150 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 10 client.19449 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 10 client.19449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 20 client.19449 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651909efc00 0x5651909f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 4 client.19449 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565190943b30 con 0x5651909efc00 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.864+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651909efc00 0x5651909f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.865+0000 7f37d67fc640 10 client.19449 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.865+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651909efc00 0x5651909f1ff0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f413b408a3273a server_cookie=9b55dd0fd6f38181 in_seq=0 out_seq=0 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37d67fc640 1 -- 172.21.15.22:0/3045176744 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565190943b30 con 0x5651909efc00 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37d67fc640 10 client.19449 handle_command_reply: tid=0 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 client.19449 shutdown 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d7150 msgr2=0x5651909dcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d7150 0x5651909dcf80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f37d80126d0 tx=0x7f37d8012b90 comp rx=0 tx=0).stop 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 shutdown_connections 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5651909efc00 0x5651909f1ff0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x5651909dd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d7150 0x5651909dcf80 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37d67fc640 0 client.19449 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 --2- 172.21.15.22:0/3045176744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d63e0 0x5651909dca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 >> 172.21.15.22:0/3045176744 conn(0x5651909995b0 msgr2=0x5651909cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 shutdown_connections 2024-03-09T00:06:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 1 -- 172.21.15.22:0/3045176744 wait complete. 2024-03-09T00:06:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.866+0000 7f37f836a740 20 client.19449 trim_cache size 0 max 16384 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 Processor -- start 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 -- start start 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909d7150 0x5651909db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909df960 0x5651909db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01ae890 con 0x5651909d63e0 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01ac430 con 0x5651909df960 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37f836a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01aa4f0 con 0x5651909d7150 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37ef7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909df960 0x5651909db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909df170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51802/0 (socket says 172.21.15.22:51802) 2024-03-09T00:06:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37eeffd640 1 -- 172.21.15.22:0/1077403204 learned_addr learned my addr 172.21.15.22:0/1077403204 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.881+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909d7150 0x5651909db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37ef7fe640 1 -- 172.21.15.22:0/1077403204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909d7150 msgr2=0x5651909db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909d7150 0x5651909db390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37ef7fe640 1 -- 172.21.15.22:0/1077403204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 msgr2=0x5651909df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37ef7fe640 1 -- 172.21.15.22:0/1077403204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651909d56b0 con 0x5651909df960 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909df960 0x5651909db8d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5651909d1ec0 tx=0x7f37e8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8f7c94583d63c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37e4ff9640 1 -- 172.21.15.22:0/1077403204 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8003b70 con 0x5651909df960 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37e4ff9640 1 -- 172.21.15.22:0/1077403204 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e8023070 con 0x5651909df960 2024-03-09T00:06:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.882+0000 7f37e4ff9640 1 -- 172.21.15.22:0/1077403204 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8011960 con 0x5651909df960 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- 172.21.15.22:0/1077403204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909df960 msgr2=0x5651909db8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909df960 0x5651909db8d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5651909d1ec0 tx=0x7f37e8019000 comp rx=0 tx=0).stop 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- 172.21.15.22:0/1077403204 shutdown_connections 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909df960 0x5651909db8d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909d7150 0x5651909db390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 --2- 172.21.15.22:0/1077403204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- 172.21.15.22:0/1077403204 >> 172.21.15.22:0/1077403204 conn(0x5651909995b0 msgr2=0x5651909d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- 172.21.15.22:0/1077403204 shutdown_connections 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- 172.21.15.22:0/1077403204 wait complete. 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- start start 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 Processor -- start 2024-03-09T00:06:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x565190a46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01aaaf0 con 0x5651909d63e0 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01abdd0 con 0x5651909d7150 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01ab110 con 0x5651909df960 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37f836a740 10 client.0 _ll_register_callbacks cb 0x7f37e0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.883+0000 7f37ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909da2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:34336/0 (socket says 172.21.15.22:34336) 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 -- 172.21.15.22:0/3347388377 learned_addr learned my addr 172.21.15.22:0/3347388377 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ef7fe640 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x565190a46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 -- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 msgr2=0x565190a46080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x565190a46080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 -- 172.21.15.22:0/3347388377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 msgr2=0x5651909d9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 -- 172.21.15.22:0/3347388377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565190a465c0 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37ee7fc640 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909da2c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5651909dc2e0 tx=0x7f37d8010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=343e1adae1b28d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37d67fc640 1 -- 172.21.15.22:0/3347388377 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8000e60 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37d67fc640 1 -- 172.21.15.22:0/3347388377 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f37d8005250 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37e8023070 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37d67fc640 1 -- 172.21.15.22:0/3347388377 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37d8039050 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37f836a740 10 client.28748 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.884+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x565190a47800 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37d67fc640 1 -- 172.21.15.22:0/3347388377 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e8023070 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37d67fc640 1 -- 172.21.15.22:0/3347388377 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37d8004590 con 0x5651909d7150 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37f836a740 10 client.28748 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37f836a740 10 client.28748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37f836a740 20 client.28748 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37f836a740 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x565190a47f00 0x565190a4a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37f836a740 4 client.28748 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:06:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56519093de80 con 0x565190a47f00 2024-03-09T00:06:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x565190a47f00 0x565190a4a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37d67fc640 10 client.28748 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:28.885+0000 7f37eeffd640 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x565190a47f00 0x565190a4a320 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86c64ca46ec92fe server_cookie=2aa14bda5c716bb0 in_seq=0 out_seq=0 2024-03-09T00:06:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37d67fc640 1 -- 172.21.15.22:0/3347388377 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56519093de80 con 0x565190a47f00 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37d67fc640 10 client.28748 handle_command_reply: tid=0 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 client.28748 shutdown 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 msgr2=0x5651909da2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909da2c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5651909dc2e0 tx=0x7f37d8010ca0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 shutdown_connections 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x565190a47f00 0x565190a4a320 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651909df960 0x565190a46080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651909d7150 0x5651909da2c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 --2- 172.21.15.22:0/3347388377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651909d63e0 0x5651909d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37d67fc640 0 client.28748 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 >> 172.21.15.22:0/3347388377 conn(0x5651909995b0 msgr2=0x5651909db310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 shutdown_connections 2024-03-09T00:06:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 1 -- 172.21.15.22:0/3347388377 wait complete. 2024-03-09T00:06:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.259+0000 7f37f836a740 20 client.28748 trim_cache size 0 max 16384 2024-03-09T00:06:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:06:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:06:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:06:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f37dc0a2a10 msgr2=0x7f37dc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f37dc0a2a10 0x7f37dc0a4ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005fd0 tx=0x7f37e800e6c0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0100440 msgr2=0x7f37f019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0100440 0x7f37f019c6e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f37e0005940 tx=0x7f37e00042f0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 shutdown_connections 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f37dc0a2a10 0x7f37dc0a4ed0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37f0105190 0x7f37f0196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f0100e00 0x7f37f019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 --2- 172.21.15.22:0/3493588661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0100440 0x7f37f019c6e0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 >> 172.21.15.22:0/3493588661 conn(0x7f37f0077040 msgr2=0x7f37f00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 shutdown_connections 2024-03-09T00:06:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:29.260+0000 7f37f5789640 1 -- 172.21.15.22:0/3493588661 wait complete. 2024-03-09T00:06:29.732 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1190525 9.44 MB/sec execute 517 sec latency 199.491 ms 2024-03-09T00:06:30.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:30 smithi192 ceph-mon[35995]: pgmap v377: 129 pgs: 129 active+clean; 95 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-03-09T00:06:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:30 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:06:30.412 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:30 smithi062 ceph-mon[36129]: pgmap v377: 129 pgs: 129 active+clean; 95 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-03-09T00:06:30.412 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:30 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:06:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:30 smithi022 ceph-mon[31894]: pgmap v377: 129 pgs: 129 active+clean; 95 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-03-09T00:06:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:30 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:06:30.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1192738 9.45 MB/sec execute 518 sec latency 16.593 ms 2024-03-09T00:06:31.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1193955 9.44 MB/sec execute 519 sec latency 16.232 ms 2024-03-09T00:06:32.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:32 smithi022 ceph-mon[31894]: pgmap v378: 129 pgs: 129 active+clean; 99 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 349 op/s 2024-03-09T00:06:32.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:32 smithi062 ceph-mon[36129]: pgmap v378: 129 pgs: 129 active+clean; 99 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 349 op/s 2024-03-09T00:06:32.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:32 smithi192 ceph-mon[35995]: pgmap v378: 129 pgs: 129 active+clean; 99 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 349 op/s 2024-03-09T00:06:32.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1195985 9.44 MB/sec execute 520 sec latency 17.099 ms 2024-03-09T00:06:33.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1197298 9.44 MB/sec execute 521 sec latency 47.243 ms 2024-03-09T00:06:34.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:34 smithi022 ceph-mon[31894]: pgmap v379: 129 pgs: 129 active+clean; 99 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-09T00:06:34.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:34 smithi062 ceph-mon[36129]: pgmap v379: 129 pgs: 129 active+clean; 99 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-09T00:06:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:34 smithi192 ceph-mon[35995]: pgmap v379: 129 pgs: 129 active+clean; 99 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-09T00:06:34.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1199231 9.44 MB/sec execute 522 sec latency 37.638 ms 2024-03-09T00:06:35.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1200874 9.44 MB/sec execute 523 sec latency 18.590 ms 2024-03-09T00:06:36.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:36 smithi022 ceph-mon[31894]: pgmap v380: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 363 op/s 2024-03-09T00:06:36.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:36 smithi062 ceph-mon[36129]: pgmap v380: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 363 op/s 2024-03-09T00:06:36.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:36 smithi192 ceph-mon[35995]: pgmap v380: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 363 op/s 2024-03-09T00:06:36.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1202875 9.44 MB/sec execute 524 sec latency 19.416 ms 2024-03-09T00:06:37.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1204436 9.44 MB/sec execute 525 sec latency 22.805 ms 2024-03-09T00:06:38.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:38 smithi022 ceph-mon[31894]: pgmap v381: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-03-09T00:06:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:38 smithi062 ceph-mon[36129]: pgmap v381: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-03-09T00:06:38.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:38 smithi192 ceph-mon[35995]: pgmap v381: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-03-09T00:06:38.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1205724 9.44 MB/sec execute 526 sec latency 43.366 ms 2024-03-09T00:06:39.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1207786 9.44 MB/sec execute 527 sec latency 25.452 ms 2024-03-09T00:06:40.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:40 smithi022 ceph-mon[31894]: pgmap v382: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-03-09T00:06:40.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:40 smithi062 ceph-mon[36129]: pgmap v382: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-03-09T00:06:40.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:40 smithi192 ceph-mon[35995]: pgmap v382: 129 pgs: 129 active+clean; 90 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-03-09T00:06:40.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1209056 9.44 MB/sec execute 528 sec latency 69.979 ms 2024-03-09T00:06:41.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1211229 9.44 MB/sec execute 529 sec latency 13.441 ms 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 -- 172.21.15.22:0/1929352308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0100420 msgr2=0x7fade0100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 --2- 172.21.15.22:0/1929352308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0100420 0x7fade0100800 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009900 tx=0x7fadd002bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 -- 172.21.15.22:0/1929352308 shutdown_connections 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 --2- 172.21.15.22:0/1929352308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade01054f0 0x7fade010d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 --2- 172.21.15.22:0/1929352308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade0100dd0 0x7fade0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 --2- 172.21.15.22:0/1929352308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0100420 0x7fade0100800 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 -- 172.21.15.22:0/1929352308 >> 172.21.15.22:0/1929352308 conn(0x7fade00772d0 msgr2=0x7fade00776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 -- 172.21.15.22:0/1929352308 shutdown_connections 2024-03-09T00:06:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 -- 172.21.15.22:0/1929352308 wait complete. 2024-03-09T00:06:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.350+0000 7fade6b9d640 1 Processor -- start 2024-03-09T00:06:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade6b9d640 1 -- start start 2024-03-09T00:06:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade6b9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0100420 0x7fade019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade6b9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade01054f0 0x7fade019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade6b9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0197980 0x7fade0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade6b9d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01105b0 con 0x7fade0197980 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade6b9d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0110430 con 0x7fade01054f0 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade6b9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0110730 con 0x7fade0100420 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade01054f0 0x7fade019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade4939640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0100420 0x7fade019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade513a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0197980 0x7fade0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade01054f0 0x7fade019dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35082/0 (socket says 172.21.15.22:35082) 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 -- 172.21.15.22:0/2200813674 learned_addr learned my addr 172.21.15.22:0/2200813674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 -- 172.21.15.22:0/2200813674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0100420 msgr2=0x7fade019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0100420 0x7fade019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 -- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0197980 msgr2=0x7fade0197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0197980 0x7fade0197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7faddffff640 1 -- 172.21.15.22:0/2200813674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade01984a0 con 0x7fade01054f0 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade4939640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0100420 0x7fade019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.351+0000 7fade513a640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0197980 0x7fade0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.352+0000 7faddffff640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade01054f0 0x7fade019dcc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fadd400e9c0 tx=0x7fadd400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.352+0000 7fadddffb640 1 -- 172.21.15.22:0/2200813674 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400cd80 con 0x7fade01054f0 2024-03-09T00:06:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.352+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade0198790 con 0x7fade01054f0 2024-03-09T00:06:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.352+0000 7fadddffb640 1 -- 172.21.15.22:0/2200813674 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd4005500 con 0x7fade01054f0 2024-03-09T00:06:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.352+0000 7fadddffb640 1 -- 172.21.15.22:0/2200813674 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4010690 con 0x7fade01054f0 2024-03-09T00:06:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.352+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade01a49e0 con 0x7fade01054f0 2024-03-09T00:06:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.354+0000 7fadddffb640 1 -- 172.21.15.22:0/2200813674 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fadd40046c0 con 0x7fade01054f0 2024-03-09T00:06:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 Processor -- start 2024-03-09T00:06:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fadddffb640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadc80a2a10 0x7fadc80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 -- start start 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fadddffb640 1 -- 172.21.15.22:0/2200813674 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fadd40cf5d0 con 0x7fade01054f0 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686311d40 0x557686310a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686311450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x55768631c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576862755f0 con 0x55768631a550 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768621e930 con 0x557686310fd0 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.355+0000 7fade9777740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557686271690 con 0x557686311d40 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686311d40 0x557686310a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686311d40 0x557686310a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41492/0 (socket says 172.21.15.22:41492) 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 -- 172.21.15.22:0/3804749114 learned_addr learned my addr 172.21.15.22:0/3804749114 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7faddffff640 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686311450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade513a640 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x55768631c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 -- 172.21.15.22:0/3804749114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 msgr2=0x557686311450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686311450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 -- 172.21.15.22:0/3804749114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 msgr2=0x55768631c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x55768631c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 -- 172.21.15.22:0/3804749114 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557685f8a210 con 0x557686311d40 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade513a640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadc80a2a10 0x7fadc80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7faddffff640 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686311450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fade4939640 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686311d40 0x557686310a90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fadd002c170 tx=0x7fadd0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87178809176c43b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.356+0000 7fadc6ffd640 1 -- 172.21.15.22:0/3804749114 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0004a70 con 0x557686311d40 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.357+0000 7fadc6ffd640 1 -- 172.21.15.22:0/3804749114 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd0004c10 con 0x557686311d40 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.357+0000 7fadc6ffd640 1 -- 172.21.15.22:0/3804749114 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0041d00 con 0x557686311d40 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.357+0000 7fade513a640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadc80a2a10 0x7fadc80a4ed0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55768630ce50 tx=0x7fadd8009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.357+0000 7fade9777740 1 -- 172.21.15.22:0/3804749114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686311d40 msgr2=0x557686310a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.357+0000 7fade9777740 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686311d40 0x557686310a90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fadd002c170 tx=0x7fadd0005e80 comp rx=0 tx=0).stop 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 -- 172.21.15.22:0/3804749114 shutdown_connections 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x55768631c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686311450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 --2- 172.21.15.22:0/3804749114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686311d40 0x557686310a90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 -- 172.21.15.22:0/3804749114 >> 172.21.15.22:0/3804749114 conn(0x5576862d85b0 msgr2=0x55768630bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 -- 172.21.15.22:0/3804749114 shutdown_connections 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 -- 172.21.15.22:0/3804749114 wait complete. 2024-03-09T00:06:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 -- start start 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.358+0000 7fade9777740 1 Processor -- start 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade9777740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686317570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade9777740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686311d40 0x557686317ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade9777740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x557686317ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade9777740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768631d2c0 con 0x557686311d40 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade9777740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557686231d90 con 0x557686310fd0 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade9777740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768631d440 con 0x55768631a550 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade9777740 10 client.0 _ll_register_callbacks cb 0x557685f89a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade513a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x557686317ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7faddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686311d40 0x557686317ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7faddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686311d40 0x557686317ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49746/0 (socket says 172.21.15.22:49746) 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7faddffff640 1 -- 172.21.15.22:0/1859720734 learned_addr learned my addr 172.21.15.22:0/1859720734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade4939640 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686317570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade4939640 1 -- 172.21.15.22:0/1859720734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 msgr2=0x557686317ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade4939640 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x557686317ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade4939640 1 -- 172.21.15.22:0/1859720734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686311d40 msgr2=0x557686317ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade4939640 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686311d40 0x557686317ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade4939640 1 -- 172.21.15.22:0/1859720734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557686056620 con 0x557686310fd0 2024-03-09T00:06:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fade4939640 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686317570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fadd002cfd0 tx=0x7fadd00417a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=401c508605652118 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.359+0000 7fadbffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fadbffff640 1 -- 172.21.15.22:0/1859720734 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0004a70 con 0x557686310fd0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadd0004c10 con 0x557686310fd0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fadbffff640 1 -- 172.21.15.22:0/1859720734 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fadd002ca90 con 0x557686310fd0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fadbffff640 1 -- 172.21.15.22:0/1859720734 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd005c050 con 0x557686310fd0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 10 client.28760 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5576863295d0 con 0x557686310fd0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fadbffff640 1 -- 172.21.15.22:0/1859720734 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd0004c10 con 0x557686310fd0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fadbffff640 1 -- 172.21.15.22:0/1859720734 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fadd004a600 con 0x557686310fd0 2024-03-09T00:06:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 10 client.28760 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 10 client.28760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 20 client.28760 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55768632a920 0x55768632cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 4 client.28760 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557686282b30 con 0x55768632a920 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.360+0000 7faddffff640 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55768632a920 0x55768632cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.361+0000 7faddffff640 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55768632a920 0x55768632cd10 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9322afb047f07cfd server_cookie=6bcf6c1cce498c91 in_seq=0 out_seq=0 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.361+0000 7fadbffff640 10 client.28760 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.361+0000 7fadbffff640 1 -- 172.21.15.22:0/1859720734 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557686282b30 con 0x55768632a920 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.361+0000 7fadbffff640 10 client.28760 handle_command_reply: tid=0 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.361+0000 7fade9777740 1 client.28760 shutdown 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 msgr2=0x557686317570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686317570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fadd002cfd0 tx=0x7fadd00417a0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 shutdown_connections 2024-03-09T00:06:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55768632a920 0x55768632cd10 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x557686317ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686311d40 0x557686317ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 --2- 172.21.15.22:0/1859720734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686310fd0 0x557686317570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 >> 172.21.15.22:0/1859720734 conn(0x5576862d85b0 msgr2=0x55768630da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fadbffff640 0 client.28760 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 shutdown_connections 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 1 -- 172.21.15.22:0/1859720734 wait complete. 2024-03-09T00:06:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.362+0000 7fade9777740 20 client.28760 trim_cache size 0 max 16384 2024-03-09T00:06:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 Processor -- start 2024-03-09T00:06:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 -- start start 2024-03-09T00:06:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x5576863312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x557686315e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686331810 0x557686316380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01ae8a0 con 0x55768631a550 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01ac440 con 0x557686311d40 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7fade9777740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01aa500 con 0x557686331810 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.382+0000 7faddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x557686315e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7fade513a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686331810 0x557686316380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x557686315e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49750/0 (socket says 172.21.15.22:49750) 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 -- 172.21.15.22:0/598169481 learned_addr learned my addr 172.21.15.22:0/598169481 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7fade4939640 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x5576863312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 -- 172.21.15.22:0/598169481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686331810 msgr2=0x557686316380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686331810 0x557686316380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 -- 172.21.15.22:0/598169481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 msgr2=0x5576863312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x5576863312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 -- 172.21.15.22:0/598169481 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557686307590 con 0x55768631a550 2024-03-09T00:06:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7fade513a640 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686331810 0x557686316380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7faddffff640 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x557686315e40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c6a0 tx=0x7fadd4009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b28042b73b426a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7fadc67fc640 1 -- 172.21.15.22:0/598169481 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b8020 con 0x55768631a550 2024-03-09T00:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7fadc67fc640 1 -- 172.21.15.22:0/598169481 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd40b3070 con 0x55768631a550 2024-03-09T00:06:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.383+0000 7fadc67fc640 1 -- 172.21.15.22:0/598169481 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40ccba0 con 0x55768631a550 2024-03-09T00:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.384+0000 7fade9777740 1 -- 172.21.15.22:0/598169481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 msgr2=0x557686315e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x557686315e40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c6a0 tx=0x7fadd4009700 comp rx=0 tx=0).stop 2024-03-09T00:06:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 -- 172.21.15.22:0/598169481 shutdown_connections 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557686331810 0x557686316380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55768631a550 0x557686315e40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 --2- 172.21.15.22:0/598169481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x5576863312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 -- 172.21.15.22:0/598169481 >> 172.21.15.22:0/598169481 conn(0x5576862d85b0 msgr2=0x557686312910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 -- 172.21.15.22:0/598169481 shutdown_connections 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 -- 172.21.15.22:0/598169481 wait complete. 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 -- start start 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.385+0000 7fade9777740 1 Processor -- start 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade9777740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x557686321a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade9777740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x55768631fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade9777740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686331810 0x557686320330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade9777740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01aab00 con 0x557686331810 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade9777740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01abde0 con 0x557686311d40 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade9777740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01ab120 con 0x55768631a550 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade9777740 10 client.0 _ll_register_callbacks cb 0x7fadd80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x55768631fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x55768631fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41500/0 (socket says 172.21.15.22:41500) 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade4939640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x557686321a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 -- 172.21.15.22:0/1851533753 learned_addr learned my addr 172.21.15.22:0/1851533753 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade513a640 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686331810 0x557686320330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 -- 172.21.15.22:0/1851533753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 msgr2=0x557686321a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x557686321a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 -- 172.21.15.22:0/1851533753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686331810 msgr2=0x557686320330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686331810 0x557686320330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7faddffff640 1 -- 172.21.15.22:0/1851533753 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557686320b60 con 0x55768631a550 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.386+0000 7fade4939640 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x557686321a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7faddffff640 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x55768631fdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadd40058c0 tx=0x7fadd4004aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7273f5a86fdccd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fadbffff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fadbffff640 1 -- 172.21.15.22:0/1851533753 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b8070 con 0x55768631a550 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fadbffff640 1 -- 172.21.15.22:0/1851533753 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fadd40849a0 con 0x55768631a550 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadd40b3070 con 0x55768631a550 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fadbffff640 1 -- 172.21.15.22:0/1851533753 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd40a3fd0 con 0x55768631a550 2024-03-09T00:06:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fade9777740 10 client.28900 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557686381a50 con 0x55768631a550 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.387+0000 7fadbffff640 1 -- 172.21.15.22:0/1851533753 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd40b3070 con 0x55768631a550 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fadbffff640 1 -- 172.21.15.22:0/1851533753 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fadd40a3b40 con 0x55768631a550 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade9777740 10 client.28900 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade9777740 10 client.28900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade9777740 20 client.28900 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade9777740 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557686381ff0 0x5576863843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade9777740 4 client.28900 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55768627ce80 con 0x557686381ff0 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade4939640 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557686381ff0 0x5576863843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fadbffff640 10 client.28900 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.388+0000 7fade4939640 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557686381ff0 0x5576863843e0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce6b8de8b4bfd2c server_cookie=bb16bf7b3892594c in_seq=0 out_seq=0 2024-03-09T00:06:42.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:42 smithi022 ceph-mon[31894]: pgmap v383: 129 pgs: 129 active+clean; 96 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-03-09T00:06:42.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:42 smithi062 ceph-mon[36129]: pgmap v383: 129 pgs: 129 active+clean; 96 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-03-09T00:06:42.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:42 smithi192 ceph-mon[35995]: pgmap v383: 129 pgs: 129 active+clean; 96 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-03-09T00:06:42.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1211986 9.43 MB/sec execute 530 sec latency 274.608 ms 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.850+0000 7fadbffff640 1 -- 172.21.15.22:0/1851533753 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55768627ce80 con 0x557686381ff0 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.850+0000 7fadbffff640 10 client.28900 handle_command_reply: tid=0 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.850+0000 7fade9777740 1 client.28900 shutdown 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.850+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 msgr2=0x55768631fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.850+0000 7fade9777740 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x55768631fdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadd40058c0 tx=0x7fadd4004aa0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 shutdown_connections 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557686381ff0 0x5576863843e0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557686331810 0x557686320330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55768631a550 0x55768631fdf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 --2- 172.21.15.22:0/1851533753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557686311d40 0x557686321a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fadbffff640 0 client.28900 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 >> 172.21.15.22:0/1851533753 conn(0x5576862d85b0 msgr2=0x557686315dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 shutdown_connections 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 1 -- 172.21.15.22:0/1851533753 wait complete. 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade9777740 20 client.28900 trim_cache size 0 max 16384 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:06:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:06:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadc80a2a10 msgr2=0x7fadc80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade6b9d640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadc80a2a10 0x7fadc80a4ed0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55768630ce50 tx=0x7fadd8009500 comp rx=0 tx=0).stop 2024-03-09T00:06:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade01054f0 msgr2=0x7fade019dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.851+0000 7fade6b9d640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade01054f0 0x7fade019dcc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fadd400e9c0 tx=0x7fadd400ee80 comp rx=0 tx=0).stop 2024-03-09T00:06:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 shutdown_connections 2024-03-09T00:06:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadc80a2a10 0x7fadc80a4ed0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade0197980 0x7fade0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade01054f0 0x7fade019dcc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 --2- 172.21.15.22:0/2200813674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0100420 0x7fade019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 >> 172.21.15.22:0/2200813674 conn(0x7fade00772d0 msgr2=0x7fade0104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 shutdown_connections 2024-03-09T00:06:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:42.852+0000 7fade6b9d640 1 -- 172.21.15.22:0/2200813674 wait complete. 2024-03-09T00:06:43.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1213498 9.42 MB/sec execute 531 sec latency 331.987 ms 2024-03-09T00:06:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:44 smithi022 ceph-mon[31894]: pgmap v384: 129 pgs: 129 active+clean; 96 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-03-09T00:06:44.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:44 smithi062 ceph-mon[36129]: pgmap v384: 129 pgs: 129 active+clean; 96 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-03-09T00:06:44.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:44 smithi192 ceph-mon[35995]: pgmap v384: 129 pgs: 129 active+clean; 96 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-03-09T00:06:44.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1214957 9.43 MB/sec execute 532 sec latency 22.075 ms 2024-03-09T00:06:45.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1216522 9.42 MB/sec execute 533 sec latency 28.514 ms 2024-03-09T00:06:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:46 smithi022 ceph-mon[31894]: pgmap v385: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 396 op/s 2024-03-09T00:06:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:46 smithi062 ceph-mon[36129]: pgmap v385: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 396 op/s 2024-03-09T00:06:46.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:46 smithi192 ceph-mon[35995]: pgmap v385: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 396 op/s 2024-03-09T00:06:46.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1218456 9.43 MB/sec execute 534 sec latency 30.119 ms 2024-03-09T00:06:47.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1219905 9.42 MB/sec execute 535 sec latency 39.063 ms 2024-03-09T00:06:48.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:48 smithi022 ceph-mon[31894]: pgmap v386: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 285 op/s 2024-03-09T00:06:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:48 smithi062 ceph-mon[36129]: pgmap v386: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 285 op/s 2024-03-09T00:06:48.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:48 smithi192 ceph-mon[35995]: pgmap v386: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 285 op/s 2024-03-09T00:06:48.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1221826 9.43 MB/sec execute 536 sec latency 56.637 ms 2024-03-09T00:06:49.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1222821 9.41 MB/sec execute 537 sec latency 25.615 ms 2024-03-09T00:06:50.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:50 smithi022 ceph-mon[31894]: pgmap v387: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2024-03-09T00:06:50.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:50 smithi062 ceph-mon[36129]: pgmap v387: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2024-03-09T00:06:50.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:50 smithi192 ceph-mon[35995]: pgmap v387: 129 pgs: 129 active+clean; 86 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2024-03-09T00:06:50.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1225297 9.43 MB/sec execute 538 sec latency 16.562 ms 2024-03-09T00:06:51.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1226224 9.41 MB/sec execute 539 sec latency 18.682 ms 2024-03-09T00:06:52.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:52 smithi022 ceph-mon[31894]: pgmap v388: 129 pgs: 129 active+clean; 92 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-03-09T00:06:52.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:52 smithi062 ceph-mon[36129]: pgmap v388: 129 pgs: 129 active+clean; 92 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-03-09T00:06:52.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:52 smithi192 ceph-mon[35995]: pgmap v388: 129 pgs: 129 active+clean; 92 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-03-09T00:06:52.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1228663 9.42 MB/sec execute 540 sec latency 16.160 ms 2024-03-09T00:06:53.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1229321 9.41 MB/sec execute 541 sec latency 83.555 ms 2024-03-09T00:06:54.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:54 smithi022 ceph-mon[31894]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 287 op/s 2024-03-09T00:06:54.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:54 smithi062 ceph-mon[36129]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 287 op/s 2024-03-09T00:06:54.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:54 smithi192 ceph-mon[35995]: pgmap v389: 129 pgs: 129 active+clean; 92 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 287 op/s 2024-03-09T00:06:54.733 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1231178 9.41 MB/sec execute 542 sec latency 30.297 ms 2024-03-09T00:06:55.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:55 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:06:55.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:55 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:06:55.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:55 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:06:55.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1232617 9.41 MB/sec execute 543 sec latency 32.199 ms 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.941+0000 7f3b5247e640 1 -- 172.21.15.22:0/1318794132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c107400 msgr2=0x7f3b4c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.941+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1318794132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c107400 0x7f3b4c1077e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 -- 172.21.15.22:0/1318794132 shutdown_connections 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1318794132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4c069330 0x7f3b4c10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1318794132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c068970 0x7f3b4c068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1318794132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c107400 0x7f3b4c1077e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 -- 172.21.15.22:0/1318794132 >> 172.21.15.22:0/1318794132 conn(0x7f3b4c077040 msgr2=0x7f3b4c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 -- 172.21.15.22:0/1318794132 shutdown_connections 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 -- 172.21.15.22:0/1318794132 wait complete. 2024-03-09T00:06:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 Processor -- start 2024-03-09T00:06:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.942+0000 7f3b5247e640 1 -- start start 2024-03-09T00:06:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c068970 0x7f3b4c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c069330 0x7f3b4c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4c107400 0x7f3b4c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1116b0 con 0x7f3b4c069330 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c111530 con 0x7f3b4c107400 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c111830 con 0x7f3b4c068970 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c069330 0x7f3b4c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c068970 0x7f3b4c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c069330 0x7f3b4c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47910/0 (socket says 172.21.15.22:47910) 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b50a1b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4c107400 0x7f3b4c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1518530012 learned_addr learned my addr 172.21.15.22:0/1518530012 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1518530012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c068970 msgr2=0x7f3b4c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c068970 0x7f3b4c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1518530012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4c107400 msgr2=0x7f3b4c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4c107400 0x7f3b4c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1518530012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c197ff0 con 0x7f3b4c069330 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4c107400 0x7f3b4c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c069330 0x7f3b4c19dc00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9f0 tx=0x7f3b4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b497fa640 1 -- 172.21.15.22:0/1518530012 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4000cd80 con 0x7f3b4c069330 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b497fa640 1 -- 172.21.15.22:0/1518530012 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b40005500 con 0x7f3b4c069330 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c1982e0 con 0x7f3b4c069330 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b497fa640 1 -- 172.21.15.22:0/1518530012 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40010690 con 0x7f3b4c069330 2024-03-09T00:06:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.943+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c1a4580 con 0x7f3b4c069330 2024-03-09T00:06:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.946+0000 7f3b497fa640 1 -- 172.21.15.22:0/1518530012 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3b400046c0 con 0x7f3b4c069330 2024-03-09T00:06:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.946+0000 7f3b55058740 1 Processor -- start 2024-03-09T00:06:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.946+0000 7f3b55058740 1 -- start start 2024-03-09T00:06:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.946+0000 7f3b55058740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f403ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b55058740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f403f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b55058740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f404a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b55058740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f3fa35f0 con 0x5592f4048550 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b55058740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f3f4c930 con 0x5592f403efd0 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b55058740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f3f9f690 con 0x5592f403fd40 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f404a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f403ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f404a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47920/0 (socket says 172.21.15.22:47920) 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/2322772424 learned_addr learned my addr 172.21.15.22:0/2322772424 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b497fa640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3b340a2a10 0x7f3b340a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f403f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3b340a2a10 0x7f3b340a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 -- 172.21.15.22:0/2322772424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 msgr2=0x5592f403f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f403f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 -- 172.21.15.22:0/2322772424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 msgr2=0x5592f404a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f404a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b497fa640 1 -- 172.21.15.22:0/1518530012 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3b400cf650 con 0x7f3b4c069330 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 -- 172.21.15.22:0/2322772424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592f3cb8210 con 0x5592f403fd40 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f404a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f403ea90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3b44005fd0 tx=0x7f3b4400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4c4a37f808dd56 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3b340a2a10 0x7f3b340a4ed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b4401e000 tx=0x7f3b44020000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.947+0000 7f3b327fc640 1 -- 172.21.15.22:0/2322772424 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b440120d0 con 0x5592f403fd40 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b327fc640 1 -- 172.21.15.22:0/2322772424 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b44023030 con 0x5592f403fd40 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b327fc640 1 -- 172.21.15.22:0/2322772424 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44010430 con 0x5592f403fd40 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 -- 172.21.15.22:0/2322772424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 msgr2=0x5592f403ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f403ea90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3b44005fd0 tx=0x7f3b4400f790 comp rx=0 tx=0).stop 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 -- 172.21.15.22:0/2322772424 shutdown_connections 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f404a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f403f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 --2- 172.21.15.22:0/2322772424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f403ea90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 -- 172.21.15.22:0/2322772424 >> 172.21.15.22:0/2322772424 conn(0x5592f40065b0 msgr2=0x5592f4039cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 -- 172.21.15.22:0/2322772424 shutdown_connections 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.948+0000 7f3b55058740 1 -- 172.21.15.22:0/2322772424 wait complete. 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 -- start start 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 Processor -- start 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f40455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f4045b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f4046070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f404b220 con 0x5592f4048550 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f3f5fd90 con 0x5592f403efd0 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f404b3a0 con 0x5592f403fd40 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b55058740 10 client.0 _ll_register_callbacks cb 0x5592f3cb7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b50a1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f4046070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b50a1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f4046070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47922/0 (socket says 172.21.15.22:47922) 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b50a1b640 1 -- 172.21.15.22:0/2815453479 learned_addr learned my addr 172.21.15.22:0/2815453479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f4045b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.949+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f40455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/2815453479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 msgr2=0x5592f40455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f40455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/2815453479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 msgr2=0x5592f4046070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f4046070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/2815453479 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592f3d84620 con 0x5592f403fd40 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f40455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f4046070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f4045b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b40098280 tx=0x7f3b40009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde718afc67254ec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/2815453479 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4009b210 con 0x5592f403fd40 2024-03-09T00:06:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b44023030 con 0x5592f403fd40 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/2815453479 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3b40097f80 con 0x5592f403fd40 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/2815453479 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b4009bfb0 con 0x5592f403fd40 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b55058740 10 client.28912 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5592f4057610 con 0x5592f403fd40 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.950+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/2815453479 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b44023030 con 0x5592f403fd40 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/2815453479 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3b40097600 con 0x5592f403fd40 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b55058740 10 client.28912 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b55058740 10 client.28912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b55058740 20 client.28912 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b55058740 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f4058a80 0x5592f405ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b55058740 4 client.28912 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592f3fb0b30 con 0x5592f4058a80 2024-03-09T00:06:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f4058a80 0x5592f405ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f4058a80 0x5592f405ae70 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e2825ad432098e server_cookie=e07403d6bf8397df in_seq=0 out_seq=0 2024-03-09T00:06:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.951+0000 7f3b2b7fe640 10 client.28912 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/2815453479 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592f3fb0b30 con 0x5592f4058a80 2024-03-09T00:06:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b2b7fe640 10 client.28912 handle_command_reply: tid=0 2024-03-09T00:06:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 client.28912 shutdown 2024-03-09T00:06:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 msgr2=0x5592f4045b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f4045b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b40098280 tx=0x7f3b40009310 comp rx=0 tx=0).stop 2024-03-09T00:06:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 shutdown_connections 2024-03-09T00:06:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f4058a80 0x5592f405ae70 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f4048550 0x5592f4046070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403fd40 0x5592f4045b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 --2- 172.21.15.22:0/2815453479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f40455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b2b7fe640 0 client.28912 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 >> 172.21.15.22:0/2815453479 conn(0x5592f40065b0 msgr2=0x5592f403ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 shutdown_connections 2024-03-09T00:06:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 1 -- 172.21.15.22:0/2815453479 wait complete. 2024-03-09T00:06:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.953+0000 7f3b55058740 20 client.28912 trim_cache size 0 max 16384 2024-03-09T00:06:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 Processor -- start 2024-03-09T00:06:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 -- start start 2024-03-09T00:06:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403efd0 0x5592f4047d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f404f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f4048550 0x5592f404fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1b0a20 con 0x5592f403fd40 2024-03-09T00:06:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1ae5c0 con 0x5592f4048550 2024-03-09T00:06:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b55058740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1ac680 con 0x5592f403efd0 2024-03-09T00:06:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f404f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f404f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47930/0 (socket says 172.21.15.22:47930) 2024-03-09T00:06:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1173487513 learned_addr learned my addr 172.21.15.22:0/1173487513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403efd0 0x5592f4047d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1173487513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403efd0 msgr2=0x5592f4047d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403efd0 0x5592f4047d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1173487513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f4048550 msgr2=0x5592f404fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f4048550 0x5592f404fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f4048550 0x5592f404fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/1173487513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592f403e5e0 con 0x5592f403fd40 2024-03-09T00:06:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.968+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f404f610 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009380 tx=0x7f3b4000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3670faf18d4e164 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b31ffb640 1 -- 172.21.15.22:0/1173487513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b400b8020 con 0x5592f403fd40 2024-03-09T00:06:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b31ffb640 1 -- 172.21.15.22:0/1173487513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b400862a0 con 0x5592f403fd40 2024-03-09T00:06:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b31ffb640 1 -- 172.21.15.22:0/1173487513 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40097250 con 0x5592f403fd40 2024-03-09T00:06:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 -- 172.21.15.22:0/1173487513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 msgr2=0x5592f404f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f404f610 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009380 tx=0x7f3b4000cf20 comp rx=0 tx=0).stop 2024-03-09T00:06:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 -- 172.21.15.22:0/1173487513 shutdown_connections 2024-03-09T00:06:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f4048550 0x5592f404fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f404f610 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 --2- 172.21.15.22:0/1173487513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f403efd0 0x5592f4047d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 -- 172.21.15.22:0/1173487513 >> 172.21.15.22:0/1173487513 conn(0x5592f40065b0 msgr2=0x5592f40397f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 -- 172.21.15.22:0/1173487513 shutdown_connections 2024-03-09T00:06:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.969+0000 7f3b55058740 1 -- 172.21.15.22:0/1173487513 wait complete. 2024-03-09T00:06:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 -- start start 2024-03-09T00:06:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 Processor -- start 2024-03-09T00:06:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f404f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f4041ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f4048550 0x5592f4042400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1acc80 con 0x5592f403fd40 2024-03-09T00:06:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1adf60 con 0x5592f403efd0 2024-03-09T00:06:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1ad2a0 con 0x5592f4048550 2024-03-09T00:06:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b55058740 10 client.0 _ll_register_callbacks cb 0x7f3b44009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f4041ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b4b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f4041ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47946/0 (socket says 172.21.15.22:47946) 2024-03-09T00:06:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/3953522978 learned_addr learned my addr 172.21.15.22:0/3953522978 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f4048550 0x5592f4042400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.970+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f404f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f4048550 msgr2=0x5592f4042400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f4048550 0x5592f4042400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/3953522978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 msgr2=0x5592f404f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f404f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b4b7fe640 1 -- 172.21.15.22:0/3953522978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592f4042aa0 con 0x5592f403fd40 2024-03-09T00:06:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b50a1b640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f4048550 0x5592f4042400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b4b7fe640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f4041ec0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9c0 tx=0x7f3b4009b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48235824f080592 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/3953522978 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4009b210 con 0x5592f403fd40 2024-03-09T00:06:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b400862a0 con 0x5592f403fd40 2024-03-09T00:06:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/3953522978 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3b400a3dc0 con 0x5592f403fd40 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/3953522978 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b400cf1f0 con 0x5592f403fd40 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 10 client.19491 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5592f40b4210 con 0x5592f403fd40 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/3953522978 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b400862a0 con 0x5592f403fd40 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/3953522978 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3b400a6080 con 0x5592f403fd40 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 10 client.19491 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 10 client.19491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 20 client.19491 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f40b47b0 0x5592f40b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 4 client.19491 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.971+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5592f404ad50 con 0x5592f40b47b0 2024-03-09T00:06:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.972+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f40b47b0 0x5592f40b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.972+0000 7f3b2b7fe640 10 client.19491 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:55.972+0000 7f3b4bfff640 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f40b47b0 0x5592f40b6bd0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52113a948a2327b server_cookie=e0cc2d5176394522 in_seq=0 out_seq=0 2024-03-09T00:06:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b2b7fe640 1 -- 172.21.15.22:0/3953522978 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5592f404ad50 con 0x5592f40b47b0 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b2b7fe640 10 client.19491 handle_command_reply: tid=0 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 client.19491 shutdown 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 msgr2=0x5592f4041ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f4041ec0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9c0 tx=0x7f3b4009b8b0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 shutdown_connections 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5592f40b47b0 0x5592f40b6bd0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5592f4048550 0x5592f4042400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592f403fd40 0x5592f4041ec0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 --2- 172.21.15.22:0/3953522978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592f403efd0 0x5592f404f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b2b7fe640 0 client.19491 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 >> 172.21.15.22:0/3953522978 conn(0x5592f40065b0 msgr2=0x5592f40455b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 shutdown_connections 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 1 -- 172.21.15.22:0/3953522978 wait complete. 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.346+0000 7f3b55058740 20 client.19491 trim_cache size 0 max 16384 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:06:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:06:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3b340a2a10 msgr2=0x7f3b340a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3b340a2a10 0x7f3b340a4ed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b4401e000 tx=0x7f3b44020000 comp rx=0 tx=0).stop 2024-03-09T00:06:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c069330 msgr2=0x7f3b4c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c069330 0x7f3b4c19dc00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9f0 tx=0x7f3b4000edf0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 shutdown_connections 2024-03-09T00:06:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f3b340a2a10 0x7f3b340a4ed0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4c107400 0x7f3b4c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b4c069330 0x7f3b4c19dc00 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 --2- 172.21.15.22:0/1518530012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c068970 0x7f3b4c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 >> 172.21.15.22:0/1518530012 conn(0x7f3b4c077040 msgr2=0x7f3b4c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 shutdown_connections 2024-03-09T00:06:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:06:56.347+0000 7f3b5247e640 1 -- 172.21.15.22:0/1518530012 wait complete. 2024-03-09T00:06:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:56 smithi022 ceph-mon[31894]: pgmap v390: 129 pgs: 129 active+clean; 98 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 379 op/s 2024-03-09T00:06:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:56 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:06:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:56 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:06:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:56 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:06:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:56 smithi062 ceph-mon[36129]: pgmap v390: 129 pgs: 129 active+clean; 98 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 379 op/s 2024-03-09T00:06:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:56 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:06:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:56 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:06:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:56 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:06:56.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:56 smithi192 ceph-mon[35995]: pgmap v390: 129 pgs: 129 active+clean; 98 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 379 op/s 2024-03-09T00:06:56.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:56 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:06:56.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:56 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:06:56.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:56 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:06:56.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1233336 9.40 MB/sec execute 544 sec latency 324.906 ms 2024-03-09T00:06:56.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:06:56.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:06:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.964+0000 7f443b43d640 1 -- 172.21.15.22:0/1708438192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 msgr2=0x7f443410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.964+0000 7f443b43d640 1 --2- 172.21.15.22:0/1708438192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443410e1e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f4430009ca0 tx=0x7f443002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- 172.21.15.22:0/1708438192 shutdown_connections 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 --2- 172.21.15.22:0/1708438192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443410e1e0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 --2- 172.21.15.22:0/1708438192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4434069110 0x7f4434105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 --2- 172.21.15.22:0/1708438192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4434068760 0x7f4434068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- 172.21.15.22:0/1708438192 >> 172.21.15.22:0/1708438192 conn(0x7f44340772f0 msgr2=0x7f4434077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- 172.21.15.22:0/1708438192 shutdown_connections 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- 172.21.15.22:0/1708438192 wait complete. 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 Processor -- start 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- start start 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4434068760 0x7f4434199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443419e5e0 0x7f443419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4434111680 con 0x7f4434106630 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4434111500 con 0x7f443419e5e0 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f443b43d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4434111800 con 0x7f4434068760 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.965+0000 7f44389d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47974/0 (socket says 172.21.15.22:47974) 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 -- 172.21.15.22:0/439807864 learned_addr learned my addr 172.21.15.22:0/439807864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44391d9640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4434068760 0x7f4434199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44399da640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443419e5e0 0x7f443419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 -- 172.21.15.22:0/439807864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4434068760 msgr2=0x7f4434199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4434068760 0x7f4434199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 -- 172.21.15.22:0/439807864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443419e5e0 msgr2=0x7f443419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443419e5e0 0x7f443419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 -- 172.21.15.22:0/439807864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443419f0d0 con 0x7f4434106630 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f44389d8640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443419a260 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f442400e9c0 tx=0x7f442400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:56.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f442a7fc640 1 -- 172.21.15.22:0/439807864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400ccc0 con 0x7f4434106630 2024-03-09T00:06:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443419f3c0 con 0x7f4434106630 2024-03-09T00:06:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f442a7fc640 1 -- 172.21.15.22:0/439807864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4424005500 con 0x7f4434106630 2024-03-09T00:06:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f442a7fc640 1 -- 172.21.15.22:0/439807864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424010690 con 0x7f4434106630 2024-03-09T00:06:56.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.966+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44341a6b60 con 0x7f4434106630 2024-03-09T00:06:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.968+0000 7f442a7fc640 1 -- 172.21.15.22:0/439807864 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4424004e80 con 0x7f4434106630 2024-03-09T00:06:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f442a7fc640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44200a2970 0x7f44200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 Processor -- start 2024-03-09T00:06:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 -- start start 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f44391d9640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44200a2970 0x7f44200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f442a7fc640 1 -- 172.21.15.22:0/439807864 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f44240cf6c0 con 0x7f4434106630 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755a760 0x55b2a755ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755b080 0x55b2a7563720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7566050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a74bde00 con 0x55b2a7563c60 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a7466b10 con 0x55b2a755b080 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.969+0000 7f443e01d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a74b9ea0 con 0x55b2a755a760 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755a760 0x55b2a755ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44399da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7566050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755a760 0x55b2a755ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:53686/0 (socket says 172.21.15.22:53686) 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 -- 172.21.15.22:0/3836596940 learned_addr learned my addr 172.21.15.22:0/3836596940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44389d8640 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755b080 0x55b2a7563720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44200a2970 0x7f44200a4e30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f442c009c20 tx=0x7f442c011810 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 -- 172.21.15.22:0/3836596940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755b080 msgr2=0x55b2a7563720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755b080 0x55b2a7563720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 -- 172.21.15.22:0/3836596940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 msgr2=0x55b2a7566050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7566050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 -- 172.21.15.22:0/3836596940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a729c7d0 con 0x55b2a755a760 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44389d8640 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755b080 0x55b2a7563720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:56.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f44391d9640 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755a760 0x55b2a755ab40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f442c01d4f0 tx=0x7f442c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a56c1df424b2ed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f441f7fe640 1 -- 172.21.15.22:0/3836596940 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c017040 con 0x55b2a755a760 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f441f7fe640 1 -- 172.21.15.22:0/3836596940 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f442c019050 con 0x55b2a755a760 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.970+0000 7f441f7fe640 1 -- 172.21.15.22:0/3836596940 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c026b30 con 0x55b2a755a760 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 -- 172.21.15.22:0/3836596940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755a760 msgr2=0x55b2a755ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755a760 0x55b2a755ab40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f442c01d4f0 tx=0x7f442c01c000 comp rx=0 tx=0).stop 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 -- 172.21.15.22:0/3836596940 shutdown_connections 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7566050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755b080 0x55b2a7563720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 --2- 172.21.15.22:0/3836596940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755a760 0x55b2a755ab40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 -- 172.21.15.22:0/3836596940 >> 172.21.15.22:0/3836596940 conn(0x55b2a7554e90 msgr2=0x55b2a7555270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 -- 172.21.15.22:0/3836596940 shutdown_connections 2024-03-09T00:06:56.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.971+0000 7f443e01d740 1 -- 172.21.15.22:0/3836596940 wait complete. 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 -- start start 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 Processor -- start 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7563350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a75606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7560c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a7566a50 con 0x55b2a7563c60 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a747a0a0 con 0x55b2a755a760 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a7566bd0 con 0x55b2a755b080 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.972+0000 7f443e01d740 10 client.0 _ll_register_callbacks cb 0x55b2a7479a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44391d9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7563350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44389d8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a75606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44391d9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7563350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:39122/0 (socket says 172.21.15.22:39122) 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44391d9640 1 -- 172.21.15.22:0/1437783305 learned_addr learned my addr 172.21.15.22:0/1437783305 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:56.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44399da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7560c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44389d8640 1 -- 172.21.15.22:0/1437783305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 msgr2=0x55b2a7563350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44389d8640 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7563350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44389d8640 1 -- 172.21.15.22:0/1437783305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 msgr2=0x55b2a7560c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44389d8640 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7560c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44389d8640 1 -- 172.21.15.22:0/1437783305 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2a71d23f0 con 0x55b2a755b080 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44391d9640 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7563350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f44389d8640 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a75606e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f44240cc980 tx=0x7f442409bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c932e839371b9c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f441cff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f441cff9640 1 -- 172.21.15.22:0/1437783305 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240b3020 con 0x55b2a755b080 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f442c019050 con 0x55b2a755b080 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.973+0000 7f441cff9640 1 -- 172.21.15.22:0/1437783305 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4424097f30 con 0x55b2a755b080 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f441cff9640 1 -- 172.21.15.22:0/1437783305 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44240cf040 con 0x55b2a755b080 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 10 client.28930 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b2a71920f0 con 0x55b2a755b080 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f441cff9640 1 -- 172.21.15.22:0/1437783305 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f442c019050 con 0x55b2a755b080 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f441cff9640 1 -- 172.21.15.22:0/1437783305 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f44240a3bd0 con 0x55b2a755b080 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 10 client.28930 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 10 client.28930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 20 client.28930 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a7575a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 4 client.28930 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2a74cb340 con 0x55b2a7573650 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.974+0000 7f44391d9640 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a7575a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.975+0000 7f44391d9640 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a7575a40 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f774251bf6d6cd server_cookie=7e39c1c90addea78 in_seq=0 out_seq=0 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.975+0000 7f441cff9640 10 client.28930 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.975+0000 7f441cff9640 1 -- 172.21.15.22:0/1437783305 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2a74cb340 con 0x55b2a7573650 2024-03-09T00:06:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.975+0000 7f441cff9640 10 client.28930 handle_command_reply: tid=0 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 client.28930 shutdown 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 msgr2=0x55b2a75606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a75606e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f44240cc980 tx=0x7f442409bfc0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 shutdown_connections 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a7575a40 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f441cff9640 0 client.28930 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a7560c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a75606e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 --2- 172.21.15.22:0/1437783305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7563350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 >> 172.21.15.22:0/1437783305 conn(0x55b2a7554e90 msgr2=0x55b2a7556be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 shutdown_connections 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 1 -- 172.21.15.22:0/1437783305 wait complete. 2024-03-09T00:06:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.976+0000 7f443e01d740 20 client.28930 trim_cache size 0 max 16384 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.991+0000 7f443e01d740 1 Processor -- start 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.991+0000 7f443e01d740 1 -- start start 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f443e01d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7562e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f443e01d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a7563350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f443e01d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a75606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f443e01d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341b2da0 con 0x55b2a7563c60 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f443e01d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341b0940 con 0x55b2a755a760 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f443e01d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341aea00 con 0x55b2a755b080 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a75606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44391d9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7562e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a75606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48010/0 (socket says 172.21.15.22:48010) 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44391d9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7562e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:39136/0 (socket says 172.21.15.22:39136) 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 -- 172.21.15.22:0/1809663468 learned_addr learned my addr 172.21.15.22:0/1809663468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44389d8640 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a7563350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 -- 172.21.15.22:0/1809663468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 msgr2=0x55b2a7563350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a7563350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 -- 172.21.15.22:0/1809663468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 msgr2=0x55b2a7562e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7562e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 -- 172.21.15.22:0/1809663468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a755ec30 con 0x55b2a7563c60 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44391d9640 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7562e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f44399da640 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a75606a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f4430009ca0 tx=0x7f4430005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5e4e536b59256d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.992+0000 7f441effd640 1 -- 172.21.15.22:0/1809663468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430039040 con 0x55b2a7563c60 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f441effd640 1 -- 172.21.15.22:0/1809663468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4430045070 con 0x55b2a7563c60 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f441effd640 1 -- 172.21.15.22:0/1809663468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430004040 con 0x55b2a7563c60 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 -- 172.21.15.22:0/1809663468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 msgr2=0x55b2a75606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:56.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a75606a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f4430009ca0 tx=0x7f4430005c50 comp rx=0 tx=0).stop 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 -- 172.21.15.22:0/1809663468 shutdown_connections 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a7563c60 0x55b2a75606a0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a755b080 0x55b2a7563350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 --2- 172.21.15.22:0/1809663468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a7562e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 -- 172.21.15.22:0/1809663468 >> 172.21.15.22:0/1809663468 conn(0x55b2a7554e90 msgr2=0x55b2a7575a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 -- 172.21.15.22:0/1809663468 shutdown_connections 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.993+0000 7f443e01d740 1 -- 172.21.15.22:0/1809663468 wait complete. 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 -- start start 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 Processor -- start 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a756b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a755b080 0x55b2a7567610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a7563c60 0x55b2a7567c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341af000 con 0x55b2a755b080 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341b02e0 con 0x55b2a755a760 2024-03-09T00:06:56.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341af620 con 0x55b2a7563c60 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f443e01d740 10 client.0 _ll_register_callbacks cb 0x7f4424004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f44389d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a755b080 0x55b2a7567610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f44389d8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a755b080 0x55b2a7567610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48014/0 (socket says 172.21.15.22:48014) 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f44389d8640 1 -- 172.21.15.22:0/3250182028 learned_addr learned my addr 172.21.15.22:0/3250182028 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f44399da640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a7563c60 0x55b2a7567c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.994+0000 7f44391d9640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a756b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f44389d8640 1 -- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a7563c60 msgr2=0x55b2a7567c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f44389d8640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a7563c60 0x55b2a7567c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f44389d8640 1 -- 172.21.15.22:0/3250182028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 msgr2=0x55b2a756b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f44389d8640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a756b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f44389d8640 1 -- 172.21.15.22:0/3250182028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2a7568420 con 0x55b2a755b080 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f44391d9640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a756b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f44389d8640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a755b080 0x55b2a7567610 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f442409b5c0 tx=0x7f4424004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd8bfe0aad80ab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:56.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f441cff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f441cff9640 1 -- 172.21.15.22:0/3250182028 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240b3040 con 0x55b2a755b080 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f441cff9640 1 -- 172.21.15.22:0/3250182028 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4424097f30 con 0x55b2a755b080 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4430045070 con 0x55b2a755b080 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f441cff9640 1 -- 172.21.15.22:0/3250182028 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44240a4060 con 0x55b2a755b080 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f443e01d740 10 client.19509 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b2a75caba0 con 0x55b2a755b080 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f441cff9640 1 -- 172.21.15.22:0/3250182028 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4430045070 con 0x55b2a755b080 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f441cff9640 1 -- 172.21.15.22:0/3250182028 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f44240051e0 con 0x55b2a755b080 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f443e01d740 10 client.19509 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.995+0000 7f443e01d740 10 client.19509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.996+0000 7f443e01d740 20 client.19509 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.996+0000 7f443e01d740 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a75cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.996+0000 7f443e01d740 4 client.19509 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:06:56.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.996+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2a7561620 con 0x55b2a7573650 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.996+0000 7f44391d9640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a75cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.996+0000 7f44391d9640 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a75cd120 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2142d1b7cea22af5 server_cookie=ae28f39a4c8139ff in_seq=0 out_seq=0 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.996+0000 7f441cff9640 10 client.19509 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f441cff9640 1 -- 172.21.15.22:0/3250182028 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b2a7561620 con 0x55b2a7573650 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f441cff9640 10 client.19509 handle_command_reply: tid=0 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 client.19509 shutdown 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a755b080 msgr2=0x55b2a7567610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a755b080 0x55b2a7567610 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f442409b5c0 tx=0x7f4424004960 comp rx=0 tx=0).stop 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 shutdown_connections 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b2a7573650 0x55b2a75cd120 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2a7563c60 0x55b2a7567c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2a755b080 0x55b2a7567610 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f441cff9640 0 client.19509 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 --2- 172.21.15.22:0/3250182028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b2a755a760 0x55b2a756b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 >> 172.21.15.22:0/3250182028 conn(0x55b2a7554e90 msgr2=0x55b2a75606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 shutdown_connections 2024-03-09T00:06:57.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 1 -- 172.21.15.22:0/3250182028 wait complete. 2024-03-09T00:06:57.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.997+0000 7f443e01d740 20 client.19509 trim_cache size 0 max 16384 2024-03-09T00:06:57.001 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:57.001 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:06:57.001 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:06:57.001 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:57.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44200a2970 msgr2=0x7f44200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44200a2970 0x7f44200a4e30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f442c009c20 tx=0x7f442c011810 comp rx=0 tx=0).stop 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 msgr2=0x7f443419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443419a260 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f442400e9c0 tx=0x7f442400ee80 comp rx=0 tx=0).stop 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 shutdown_connections 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f44200a2970 0x7f44200a4e30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f443419e5e0 0x7f443419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434106630 0x7f443419a260 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 --2- 172.21.15.22:0/439807864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4434068760 0x7f4434199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 >> 172.21.15.22:0/439807864 conn(0x7f44340772f0 msgr2=0x7f443410c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 shutdown_connections 2024-03-09T00:06:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:56.998+0000 7f443b43d640 1 -- 172.21.15.22:0/439807864 wait complete. 2024-03-09T00:06:57.004 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:06:57.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:06:57.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 -- 172.21.15.22:0/1198624399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0100460 msgr2=0x7f5bf0100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/1198624399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0100460 0x7f5bf0100840 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009900 tx=0x7f5be002bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:57.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 -- 172.21.15.22:0/1198624399 shutdown_connections 2024-03-09T00:06:57.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/1198624399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf0105530 0x7f5bf010d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/1198624399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0100e10 0x7f5bf0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/1198624399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0100460 0x7f5bf0100840 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 -- 172.21.15.22:0/1198624399 >> 172.21.15.22:0/1198624399 conn(0x7f5bf00772f0 msgr2=0x7f5bf0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 -- 172.21.15.22:0/1198624399 shutdown_connections 2024-03-09T00:06:57.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.121+0000 7f5bf4f45640 1 -- 172.21.15.22:0/1198624399 wait complete. 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 Processor -- start 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 -- start start 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0100460 0x7f5bf01967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf0105530 0x7f5bf0196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0197250 0x7f5bf01a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf0110640 con 0x7f5bf0197250 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf01104c0 con 0x7f5bf0100460 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bf4f45640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf01107c0 con 0x7f5bf0105530 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5beed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0197250 0x7f5bf01a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5beed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0197250 0x7f5bf01a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48038/0 (socket says 172.21.15.22:48038) 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5beed9d640 1 -- 172.21.15.22:0/900217535 learned_addr learned my addr 172.21.15.22:0/900217535 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5be7fff640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf0105530 0x7f5bf0196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.122+0000 7f5bee59c640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0100460 0x7f5bf01967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5beed9d640 1 -- 172.21.15.22:0/900217535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf0105530 msgr2=0x7f5bf0196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5beed9d640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf0105530 0x7f5bf0196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5beed9d640 1 -- 172.21.15.22:0/900217535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0100460 msgr2=0x7f5bf01967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5beed9d640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0100460 0x7f5bf01967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5beed9d640 1 -- 172.21.15.22:0/900217535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf01a6460 con 0x7f5bf0197250 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5beed9d640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0197250 0x7f5bf01a5e50 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5be800eb40 tx=0x7f5be800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5be77fe640 1 -- 172.21.15.22:0/900217535 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8004b20 con 0x7f5bf0197250 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5be77fe640 1 -- 172.21.15.22:0/900217535 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5be8004cc0 con 0x7f5bf0197250 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf01a6750 con 0x7f5bf0197250 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5be77fe640 1 -- 172.21.15.22:0/900217535 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8010760 con 0x7f5bf0197250 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.123+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf01a6cf0 con 0x7f5bf0197250 2024-03-09T00:06:57.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.124+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf00680e0 con 0x7f5bf0197250 2024-03-09T00:06:57.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.125+0000 7f5be77fe640 1 -- 172.21.15.22:0/900217535 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5be8005020 con 0x7f5bf0197250 2024-03-09T00:06:57.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.126+0000 7f5be77fe640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bdc0a29f0 0x7f5bdc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.126+0000 7f5bee59c640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bdc0a29f0 0x7f5bdc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.126+0000 7f5be77fe640 1 -- 172.21.15.22:0/900217535 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5be80cf8b0 con 0x7f5bf0197250 2024-03-09T00:06:57.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.127+0000 7f5bee59c640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bdc0a29f0 0x7f5bdc0a4eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5be0000c00 tx=0x7f5be002df90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.131+0000 7f5be77fe640 1 -- 172.21.15.22:0/900217535 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5be809c1b0 con 0x7f5bf0197250 2024-03-09T00:06:57.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.366+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bf0100e10 con 0x7f5bf0197250 2024-03-09T00:06:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.367+0000 7f5be77fe640 1 -- 172.21.15.22:0/900217535 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f5be809bab0 con 0x7f5bf0197250 2024-03-09T00:06:57.362 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:06:57.362 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:06:57.362 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:06:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bdc0a29f0 msgr2=0x7f5bdc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bdc0a29f0 0x7f5bdc0a4eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5be0000c00 tx=0x7f5be002df90 comp rx=0 tx=0).stop 2024-03-09T00:06:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0197250 msgr2=0x7f5bf01a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0197250 0x7f5bf01a5e50 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5be800eb40 tx=0x7f5be800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 shutdown_connections 2024-03-09T00:06:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5bdc0a29f0 0x7f5bdc0a4eb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bf0197250 0x7f5bf01a5e50 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf0105530 0x7f5bf0196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 --2- 172.21.15.22:0/900217535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bf0100460 0x7f5bf01967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 >> 172.21.15.22:0/900217535 conn(0x7f5bf00772f0 msgr2=0x7f5bf00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 shutdown_connections 2024-03-09T00:06:57.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.369+0000 7f5bf4f45640 1 -- 172.21.15.22:0/900217535 wait complete. 2024-03-09T00:06:57.375 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:57.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:06:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 -- 172.21.15.22:0/3998185345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 msgr2=0x7f715810dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3998185345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f715810dcf0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f7148009900 tx=0x7f714802bcd0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 -- 172.21.15.22:0/3998185345 shutdown_connections 2024-03-09T00:06:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3998185345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f715810dcf0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3998185345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7158102e30 0x7f715810b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3998185345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158102510 0x7f71581028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 -- 172.21.15.22:0/3998185345 >> 172.21.15.22:0/3998185345 conn(0x7f71580773a0 msgr2=0x7f71580777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 -- 172.21.15.22:0/3998185345 shutdown_connections 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 -- 172.21.15.22:0/3998185345 wait complete. 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 Processor -- start 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.489+0000 7f715f1b9640 1 -- start start 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715f1b9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7158102510 0x7f7158100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715f1b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158102e30 0x7f71581006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715f1b9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f7158100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715f1b9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7158111610 con 0x7f715810b850 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715f1b9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7158111490 con 0x7f7158102510 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715f1b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7158111790 con 0x7f7158102e30 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f7158100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f7158100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48058/0 (socket says 172.21.15.22:48058) 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 -- 172.21.15.22:0/3253325696 learned_addr learned my addr 172.21.15.22:0/3253325696 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715cf55640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7158102510 0x7f7158100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f7157fff640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158102e30 0x7f71581006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 -- 172.21.15.22:0/3253325696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158102e30 msgr2=0x7f71581006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158102e30 0x7f71581006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 -- 172.21.15.22:0/3253325696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7158102510 msgr2=0x7f7158100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7158102510 0x7f7158100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 -- 172.21.15.22:0/3253325696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71581a5fd0 con 0x7f715810b850 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f715d756640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f7158100be0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7148005940 tx=0x7f714802cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.490+0000 7f7155ffb640 1 -- 172.21.15.22:0/3253325696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148041070 con 0x7f715810b850 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.491+0000 7f7155ffb640 1 -- 172.21.15.22:0/3253325696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7148004450 con 0x7f715810b850 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.491+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71581a6260 con 0x7f715810b850 2024-03-09T00:06:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.491+0000 7f7155ffb640 1 -- 172.21.15.22:0/3253325696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148003de0 con 0x7f715810b850 2024-03-09T00:06:57.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.491+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71581a67a0 con 0x7f715810b850 2024-03-09T00:06:57.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 Processor -- start 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7155ffb640 1 -- 172.21.15.22:0/3253325696 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7148044070 con 0x7f715810b850 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 -- start start 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f04b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f054160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f056a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4f012360 con 0x557d4f0546a0 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4efaeec0 con 0x557d4f04b1a0 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7161d96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4ef57d30 con 0x557d4f04bac0 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f056a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f715cf55640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f054160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f056a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48072/0 (socket says 172.21.15.22:48072) 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f715cf55640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f054160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:53756/0 (socket says 172.21.15.22:53756) 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 -- 172.21.15.22:0/1271415192 learned_addr learned my addr 172.21.15.22:0/1271415192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f04b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7155ffb640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71400a29f0 0x7f71400a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f715d756640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71400a29f0 0x7f71400a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 -- 172.21.15.22:0/1271415192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 msgr2=0x557d4f054160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f054160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 -- 172.21.15.22:0/1271415192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 msgr2=0x557d4f04b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f04b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.494+0000 7f7157fff640 1 -- 172.21.15.22:0/1271415192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4ef56bc0 con 0x557d4f0546a0 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.495+0000 7f715cf55640 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f054160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.495+0000 7f7155ffb640 1 -- 172.21.15.22:0/3253325696 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f71480f0c20 con 0x7f715810b850 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.495+0000 7f7157fff640 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f056a90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f714c011780 tx=0x7f714c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a023637a67d9da server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.495+0000 7f713effd640 1 -- 172.21.15.22:0/1271415192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c004050 con 0x557d4f0546a0 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.495+0000 7f715d756640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71400a29f0 0x7f71400a4eb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f71480bcfd0 tx=0x7f7148088250 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.495+0000 7f713effd640 1 -- 172.21.15.22:0/1271415192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f714c00fca0 con 0x557d4f0546a0 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.495+0000 7f713effd640 1 -- 172.21.15.22:0/1271415192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c003af0 con 0x557d4f0546a0 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 -- 172.21.15.22:0/1271415192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 msgr2=0x557d4f056a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f056a90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f714c011780 tx=0x7f714c004730 comp rx=0 tx=0).stop 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 -- 172.21.15.22:0/1271415192 shutdown_connections 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f056a90 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f054160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 --2- 172.21.15.22:0/1271415192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f04b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 -- 172.21.15.22:0/1271415192 >> 172.21.15.22:0/1271415192 conn(0x557d4f0458d0 msgr2=0x557d4f045cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 -- 172.21.15.22:0/1271415192 shutdown_connections 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 -- 172.21.15.22:0/1271415192 wait complete. 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 -- start start 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.496+0000 7f7161d96740 1 Processor -- start 2024-03-09T00:06:57.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f0516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f051be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f052120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4ef4d4a0 con 0x557d4f0546a0 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4efaaf60 con 0x557d4f04b1a0 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4f057490 con 0x557d4f04bac0 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 10 client.0 _ll_register_callbacks cb 0x557d4efa8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f052120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f052120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48088/0 (socket says 172.21.15.22:48088) 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f715cf55640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f051be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 -- 172.21.15.22:0/3136915693 learned_addr learned my addr 172.21.15.22:0/3136915693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f0516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 -- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 msgr2=0x557d4f051be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f051be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 -- 172.21.15.22:0/3136915693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 msgr2=0x557d4f0516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f0516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 -- 172.21.15.22:0/3136915693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d4ecc33f0 con 0x557d4f0546a0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f715cf55640 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f051be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7157fff640 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f052120 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f714c010010 tx=0x7f714c003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e3f2bf391ce46f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7137fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7137fff640 1 -- 172.21.15.22:0/3136915693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c005020 con 0x557d4f0546a0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7137fff640 1 -- 172.21.15.22:0/3136915693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f714c0051c0 con 0x557d4f0546a0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.497+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f714c00fca0 con 0x557d4f0546a0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7137fff640 1 -- 172.21.15.22:0/3136915693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f714c0053c0 con 0x557d4f0546a0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 10 client.19527 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557d4ed8d7d0 con 0x557d4f0546a0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7137fff640 1 -- 172.21.15.22:0/3136915693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f714c00fca0 con 0x557d4f0546a0 2024-03-09T00:06:57.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7137fff640 1 -- 172.21.15.22:0/3136915693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f714c02c030 con 0x557d4f0546a0 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 10 client.19527 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 10 client.19527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 20 client.19527 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f064720 0x557d4f066b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 4 client.19527 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d4efbc400 con 0x557d4f064720 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.498+0000 7f715cf55640 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f064720 0x557d4f066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f715cf55640 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f064720 0x557d4f066b10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cb31d9ff3f0d5ef server_cookie=a728e14dbd1e818e in_seq=0 out_seq=0 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7137fff640 10 client.19527 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7137fff640 1 -- 172.21.15.22:0/3136915693 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d4efbc400 con 0x557d4f064720 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7137fff640 10 client.19527 handle_command_reply: tid=0 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 client.19527 shutdown 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 msgr2=0x557d4f052120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f052120 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f714c010010 tx=0x7f714c003690 comp rx=0 tx=0).stop 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 shutdown_connections 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f064720 0x557d4f066b10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f0546a0 0x557d4f052120 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04bac0 0x557d4f051be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 --2- 172.21.15.22:0/3136915693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04b1a0 0x557d4f0516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 >> 172.21.15.22:0/3136915693 conn(0x557d4f0458d0 msgr2=0x557d4f047620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7137fff640 0 client.19527 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 shutdown_connections 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.499+0000 7f7161d96740 1 -- 172.21.15.22:0/3136915693 wait complete. 2024-03-09T00:06:57.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.500+0000 7f7161d96740 20 client.19527 trim_cache size 0 max 16384 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 Processor -- start 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 -- start start 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04b1a0 0x557d4f053dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04bac0 0x557d4f066730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f0546a0 0x557d4f066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581b2d90 con 0x557d4f04bac0 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581b0930 con 0x557d4f0546a0 2024-03-09T00:06:57.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7161d96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581ae9f0 con 0x557d4f04b1a0 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f715cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04bac0 0x557d4f066730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f715cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04bac0 0x557d4f066730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48100/0 (socket says 172.21.15.22:48100) 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f715cf55640 1 -- 172.21.15.22:0/1376762443 learned_addr learned my addr 172.21.15.22:0/1376762443 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7157fff640 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04b1a0 0x557d4f053dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.512+0000 7f7157fff640 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f0546a0 0x557d4f066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f715cf55640 1 -- 172.21.15.22:0/1376762443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04b1a0 msgr2=0x557d4f053dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f715cf55640 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04b1a0 0x557d4f053dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f715cf55640 1 -- 172.21.15.22:0/1376762443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f0546a0 msgr2=0x557d4f066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f715cf55640 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f0546a0 0x557d4f066c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f715cf55640 1 -- 172.21.15.22:0/1376762443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4f04d270 con 0x557d4f04bac0 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f715cf55640 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04bac0 0x557d4f066730 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7150009370 tx=0x7f7150009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31433fe1a7219183 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f713e7fc640 1 -- 172.21.15.22:0/1376762443 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150004770 con 0x557d4f04bac0 2024-03-09T00:06:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f713e7fc640 1 -- 172.21.15.22:0/1376762443 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7150023070 con 0x557d4f04bac0 2024-03-09T00:06:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.513+0000 7f713e7fc640 1 -- 172.21.15.22:0/1376762443 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715001d950 con 0x557d4f04bac0 2024-03-09T00:06:57.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 -- 172.21.15.22:0/1376762443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04bac0 msgr2=0x557d4f066730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04bac0 0x557d4f066730 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7150009370 tx=0x7f7150009500 comp rx=0 tx=0).stop 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 -- 172.21.15.22:0/1376762443 shutdown_connections 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f0546a0 0x557d4f066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04bac0 0x557d4f066730 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 --2- 172.21.15.22:0/1376762443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f04b1a0 0x557d4f053dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 -- 172.21.15.22:0/1376762443 >> 172.21.15.22:0/1376762443 conn(0x557d4f0458d0 msgr2=0x557d4f04bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 -- 172.21.15.22:0/1376762443 shutdown_connections 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 -- 172.21.15.22:0/1376762443 wait complete. 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 -- start start 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 Processor -- start 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.514+0000 7f7161d96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04b1a0 0x557d4f059170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04bac0 0x557d4f0596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f0546a0 0x557d4f059bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581aeff0 con 0x557d4f04b1a0 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581b02d0 con 0x557d4f04bac0 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581af610 con 0x557d4f0546a0 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 10 client.0 _ll_register_callbacks cb 0x7f714c009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04b1a0 0x557d4f059170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04b1a0 0x557d4f059170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48104/0 (socket says 172.21.15.22:48104) 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 -- 172.21.15.22:0/133616657 learned_addr learned my addr 172.21.15.22:0/133616657 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f715cf55640 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04bac0 0x557d4f0596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f0546a0 0x557d4f059bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 -- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f0546a0 msgr2=0x557d4f059bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f0546a0 0x557d4f059bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 -- 172.21.15.22:0/133616657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04bac0 msgr2=0x557d4f0596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04bac0 0x557d4f0596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 -- 172.21.15.22:0/133616657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d4f0b9b30 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7157fff640 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04b1a0 0x557d4f059170 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f714c0206b0 tx=0x7f714c003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08ee998aadc5878 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7137fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7137fff640 1 -- 172.21.15.22:0/133616657 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0050e0 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7137fff640 1 -- 172.21.15.22:0/133616657 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f714c005560 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7150023070 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7137fff640 1 -- 172.21.15.22:0/133616657 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f714c011070 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 10 client.19533 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.515+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557d4f0bae10 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7137fff640 1 -- 172.21.15.22:0/133616657 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7150023070 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7137fff640 1 -- 172.21.15.22:0/133616657 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f714c02c030 con 0x557d4f04b1a0 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7161d96740 10 client.19533 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7161d96740 10 client.19533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:06:57.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7161d96740 20 client.19533 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7161d96740 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f0bb450 0x557d4f0bd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7161d96740 4 client.19533 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d4efb6750 con 0x557d4f0bb450 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f715cf55640 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f0bb450 0x557d4f0bd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f715cf55640 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f0bb450 0x557d4f0bd8f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5109234599b21c4f server_cookie=c89ac739b0b36161 in_seq=0 out_seq=0 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.516+0000 7f7137fff640 10 client.19533 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7137fff640 1 -- 172.21.15.22:0/133616657 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d4efb6750 con 0x557d4f0bb450 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7137fff640 10 client.19533 handle_command_reply: tid=0 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 client.19533 shutdown 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04b1a0 msgr2=0x557d4f059170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04b1a0 0x557d4f059170 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f714c0206b0 tx=0x7f714c003690 comp rx=0 tx=0).stop 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 shutdown_connections 2024-03-09T00:06:57.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557d4f0bb450 0x557d4f0bd8f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557d4f0546a0 0x557d4f059bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7137fff640 0 client.19533 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d4f04bac0 0x557d4f0596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 --2- 172.21.15.22:0/133616657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d4f04b1a0 0x557d4f059170 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 >> 172.21.15.22:0/133616657 conn(0x557d4f0458d0 msgr2=0x557d4f066730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 shutdown_connections 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 1 -- 172.21.15.22:0/133616657 wait complete. 2024-03-09T00:06:57.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f7161d96740 20 client.19533 trim_cache size 0 max 16384 2024-03-09T00:06:57.520 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71400a29f0 msgr2=0x7f71400a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71400a29f0 0x7f71400a4eb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f71480bcfd0 tx=0x7f7148088250 comp rx=0 tx=0).stop 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 msgr2=0x7f7158100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.517+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f7158100be0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7148005940 tx=0x7f714802cc40 comp rx=0 tx=0).stop 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 shutdown_connections 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f71400a29f0 0x7f71400a4eb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f715810b850 0x7f7158100be0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158102e30 0x7f71581006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 --2- 172.21.15.22:0/3253325696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7158102510 0x7f7158100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 >> 172.21.15.22:0/3253325696 conn(0x7f71580773a0 msgr2=0x7f71580fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 shutdown_connections 2024-03-09T00:06:57.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.518+0000 7f715f1b9640 1 -- 172.21.15.22:0/3253325696 wait complete. 2024-03-09T00:06:57.523 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:57.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:06:57.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.638+0000 7f3338a6c640 1 -- 172.21.15.22:0/4114813675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341003e0 msgr2=0x7f33341007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.638+0000 7f3338a6c640 1 --2- 172.21.15.22:0/4114813675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341003e0 0x7f33341007c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3324009ca0 tx=0x7f332402c0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 -- 172.21.15.22:0/4114813675 shutdown_connections 2024-03-09T00:06:57.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 --2- 172.21.15.22:0/4114813675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33341054b0 0x7f333410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 --2- 172.21.15.22:0/4114813675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3334100d90 0x7f3334104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 --2- 172.21.15.22:0/4114813675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341003e0 0x7f33341007c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 -- 172.21.15.22:0/4114813675 >> 172.21.15.22:0/4114813675 conn(0x7f33340773a0 msgr2=0x7f33340777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 -- 172.21.15.22:0/4114813675 shutdown_connections 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 -- 172.21.15.22:0/4114813675 wait complete. 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.639+0000 7f3338a6c640 1 Processor -- start 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3338a6c640 1 -- start start 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3338a6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33341003e0 0x7f333419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3338a6c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341054b0 0x7f333419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3338a6c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3334197a40 0x7f3334197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3338a6c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33341105e0 con 0x7f33341054b0 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3338a6c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334110460 con 0x7f33341003e0 2024-03-09T00:06:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3338a6c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334110760 con 0x7f3334197a40 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341054b0 0x7f333419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341054b0 0x7f333419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48130/0 (socket says 172.21.15.22:48130) 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 -- 172.21.15.22:0/2918956833 learned_addr learned my addr 172.21.15.22:0/2918956833 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f333259c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33341003e0 0x7f333419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3332d9d640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3334197a40 0x7f3334197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 -- 172.21.15.22:0/2918956833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3334197a40 msgr2=0x7f3334197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3334197a40 0x7f3334197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 -- 172.21.15.22:0/2918956833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33341003e0 msgr2=0x7f333419d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33341003e0 0x7f333419d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 -- 172.21.15.22:0/2918956833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334198560 con 0x7f33341054b0 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f333259c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33341003e0 0x7f333419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.640+0000 7f3331d9b640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341054b0 0x7f333419ddd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f331c00d740 tx=0x7f331c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.641+0000 7f332b7fe640 1 -- 172.21.15.22:0/2918956833 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c017070 con 0x7f33341054b0 2024-03-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.641+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3334198850 con 0x7f33341054b0 2024-03-09T00:06:57.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.641+0000 7f332b7fe640 1 -- 172.21.15.22:0/2918956833 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f331c00de50 con 0x7f33341054b0 2024-03-09T00:06:57.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.641+0000 7f332b7fe640 1 -- 172.21.15.22:0/2918956833 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c004e00 con 0x7f33341054b0 2024-03-09T00:06:57.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.641+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33341a4a50 con 0x7f33341054b0 2024-03-09T00:06:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.643+0000 7f332b7fe640 1 -- 172.21.15.22:0/2918956833 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f331c004fa0 con 0x7f33341054b0 2024-03-09T00:06:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.643+0000 7f333b650740 1 Processor -- start 2024-03-09T00:06:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333b650740 1 -- start start 2024-03-09T00:06:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333b650740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559f9b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333b650740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fa4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f332b7fe640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33200a2970 0x7f33200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333b650740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fa6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333b650740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560559f62360 con 0x560559f9b1a0 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333b650740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560559efeec0 con 0x560559f9bac0 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333b650740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560559ea7d30 con 0x560559fa46a0 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559f9b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f333259c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33200a2970 0x7f33200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559f9b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48140/0 (socket says 172.21.15.22:48140) 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 -- 172.21.15.22:0/1174138710 learned_addr learned my addr 172.21.15.22:0/1174138710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f332b7fe640 1 -- 172.21.15.22:0/2918956833 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f331c0ceb60 con 0x7f33341054b0 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fa6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3331d9b640 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fa4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 -- 172.21.15.22:0/1174138710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 msgr2=0x560559fa6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fa6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 -- 172.21.15.22:0/1174138710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 msgr2=0x560559fa4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fa4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.644+0000 7f3332d9d640 1 -- 172.21.15.22:0/1174138710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560559ea6bc0 con 0x560559f9b1a0 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.645+0000 7f3332d9d640 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559f9b580 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f332c0176a0 tx=0x7f332c009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83cc4b33a3b2e80 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.645+0000 7f333259c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33200a2970 0x7f33200a4e30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3324004af0 tx=0x7f3324005c50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.645+0000 7f3328ff9640 1 -- 172.21.15.22:0/1174138710 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0158c0 con 0x560559f9b1a0 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.645+0000 7f3328ff9640 1 -- 172.21.15.22:0/1174138710 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f332c01d070 con 0x560559f9b1a0 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.645+0000 7f3328ff9640 1 -- 172.21.15.22:0/1174138710 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c018c10 con 0x560559f9b1a0 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.645+0000 7f333b650740 1 -- 172.21.15.22:0/1174138710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 msgr2=0x560559f9b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.645+0000 7f333b650740 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559f9b580 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f332c0176a0 tx=0x7f332c009450 comp rx=0 tx=0).stop 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- 172.21.15.22:0/1174138710 shutdown_connections 2024-03-09T00:06:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fa6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fa4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 --2- 172.21.15.22:0/1174138710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559f9b580 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- 172.21.15.22:0/1174138710 >> 172.21.15.22:0/1174138710 conn(0x560559f958d0 msgr2=0x560559f95cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- 172.21.15.22:0/1174138710 shutdown_connections 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- 172.21.15.22:0/1174138710 wait complete. 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- start start 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 Processor -- start 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fa1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559fa46a0 0x560559fa20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560559e9d4a0 con 0x560559f9b1a0 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560559efaf60 con 0x560559fa46a0 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560559fa7430 con 0x560559f9bac0 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333b650740 10 client.0 _ll_register_callbacks cb 0x560559ef8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.646+0000 7f333259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48156/0 (socket says 172.21.15.22:48156) 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333259c640 1 -- 172.21.15.22:0/3206588390 learned_addr learned my addr 172.21.15.22:0/3206588390 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f3331d9b640 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fa1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f3332d9d640 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559fa46a0 0x560559fa20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333259c640 1 -- 172.21.15.22:0/3206588390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 msgr2=0x560559fa1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333259c640 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fa1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333259c640 1 -- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559fa46a0 msgr2=0x560559fa20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333259c640 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559fa46a0 0x560559fa20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333259c640 1 -- 172.21.15.22:0/3206588390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560559c133f0 con 0x560559f9b1a0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333259c640 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa1640 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3324009ca0 tx=0x7f332402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2aca1a4bca19149 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f3319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f3319ffb640 1 -- 172.21.15.22:0/3206588390 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324044070 con 0x560559f9b1a0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f3319ffb640 1 -- 172.21.15.22:0/3206588390 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f33240349f0 con 0x560559f9b1a0 2024-03-09T00:06:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f332c01d070 con 0x560559f9b1a0 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f3319ffb640 1 -- 172.21.15.22:0/3206588390 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3324059050 con 0x560559f9b1a0 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333b650740 10 client.19545 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.647+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560559cdd7d0 con 0x560559f9b1a0 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f3319ffb640 1 -- 172.21.15.22:0/3206588390 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f332c01d070 con 0x560559f9b1a0 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f3319ffb640 1 -- 172.21.15.22:0/3206588390 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f332403f440 con 0x560559f9b1a0 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f333b650740 10 client.19545 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f333b650740 10 client.19545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f333b650740 20 client.19545 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f333b650740 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560559fb4840 0x560559fb6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f333b650740 4 client.19545 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560559f0c400 con 0x560559fb4840 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f3331d9b640 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560559fb4840 0x560559fb6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f3331d9b640 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560559fb4840 0x560559fb6c70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=828d69d71a0bee9b server_cookie=b569e08f9e871f41 in_seq=0 out_seq=0 2024-03-09T00:06:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.648+0000 7f3319ffb640 10 client.19545 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f3319ffb640 1 -- 172.21.15.22:0/3206588390 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560559f0c400 con 0x560559fb4840 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f3319ffb640 10 client.19545 handle_command_reply: tid=0 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 client.19545 shutdown 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 msgr2=0x560559fa1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa1640 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3324009ca0 tx=0x7f332402c0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 shutdown_connections 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560559fb4840 0x560559fb6c70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559fa46a0 0x560559fa20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fa1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 --2- 172.21.15.22:0/3206588390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa1640 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 >> 172.21.15.22:0/3206588390 conn(0x560559f958d0 msgr2=0x560559f97620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f3319ffb640 0 client.19545 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 shutdown_connections 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 1 -- 172.21.15.22:0/3206588390 wait complete. 2024-03-09T00:06:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.649+0000 7f333b650740 20 client.19545 trim_cache size 0 max 16384 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 Processor -- start 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 -- start start 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fabba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33341b0a40 con 0x560559f9b1a0 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33341ae5e0 con 0x560559f9bac0 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333b650740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33341ac6a0 con 0x560559fa46a0 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48172/0 (socket says 172.21.15.22:48172) 2024-03-09T00:06:57.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f333259c640 1 -- 172.21.15.22:0/1037487377 learned_addr learned my addr 172.21.15.22:0/1037487377 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.661+0000 7f3332d9d640 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fabba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f3331d9b640 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f333259c640 1 -- 172.21.15.22:0/1037487377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 msgr2=0x560559fabba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f333259c640 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fabba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f333259c640 1 -- 172.21.15.22:0/1037487377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 msgr2=0x560559fab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f333259c640 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f333259c640 1 -- 172.21.15.22:0/1037487377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560559fbbe60 con 0x560559f9b1a0 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f333259c640 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa3d70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f332402c630 tx=0x7f3324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a61c5540877058 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f331bfff640 1 -- 172.21.15.22:0/1037487377 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324044070 con 0x560559f9b1a0 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f331bfff640 1 -- 172.21.15.22:0/1037487377 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3324062070 con 0x560559f9b1a0 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.662+0000 7f331bfff640 1 -- 172.21.15.22:0/1037487377 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324039040 con 0x560559f9b1a0 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- 172.21.15.22:0/1037487377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 msgr2=0x560559fa3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa3d70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f332402c630 tx=0x7f3324004a40 comp rx=0 tx=0).stop 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- 172.21.15.22:0/1037487377 shutdown_connections 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559fa46a0 0x560559fabba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9bac0 0x560559fab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 --2- 172.21.15.22:0/1037487377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559f9b1a0 0x560559fa3d70 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- 172.21.15.22:0/1037487377 >> 172.21.15.22:0/1037487377 conn(0x560559f958d0 msgr2=0x560559f9c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- 172.21.15.22:0/1037487377 shutdown_connections 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- 172.21.15.22:0/1037487377 wait complete. 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- start start 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 Processor -- start 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9b1a0 0x560559fab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559fa46a0 0x560559fa9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33341acca0 con 0x560559fa46a0 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33341adf80 con 0x560559f9b1a0 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.663+0000 7f333b650740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33341ad2c0 con 0x560559f9bac0 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f333b650740 10 client.0 _ll_register_callbacks cb 0x7f332c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559fa46a0 0x560559fa9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559fa46a0 0x560559fa9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48186/0 (socket says 172.21.15.22:48186) 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 -- 172.21.15.22:0/2706160671 learned_addr learned my addr 172.21.15.22:0/2706160671 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f333259c640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9b1a0 0x560559fab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3331d9b640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 -- 172.21.15.22:0/2706160671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 msgr2=0x560559fab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 -- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9b1a0 msgr2=0x560559fab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9b1a0 0x560559fab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 -- 172.21.15.22:0/2706160671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560559fa9f30 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f333259c640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9b1a0 0x560559fab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3332d9d640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559fa46a0 0x560559fa9730 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f332c017670 tx=0x7f332c018620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f330f7f85c79bb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3319ffb640 1 -- 172.21.15.22:0/2706160671 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0158c0 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3319ffb640 1 -- 172.21.15.22:0/2706160671 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f332c015a60 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3324062070 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f3319ffb640 1 -- 172.21.15.22:0/2706160671 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f332c034050 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f333b650740 10 client.19551 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.664+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56055a00b780 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f3319ffb640 1 -- 172.21.15.22:0/2706160671 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3324062070 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f3319ffb640 1 -- 172.21.15.22:0/2706160671 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f332c029a90 con 0x560559fa46a0 2024-03-09T00:06:57.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333b650740 10 client.19551 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333b650740 10 client.19551 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333b650740 20 client.19551 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333b650740 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x56055a00bb70 0x56055a00df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333b650740 4 client.19551 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560559f06750 con 0x56055a00bb70 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333259c640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x56055a00bb70 0x56055a00df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f333259c640 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x56055a00bb70 0x56055a00df90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78fb03546b44f844 server_cookie=4c26b1cd502ce29d in_seq=0 out_seq=0 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.665+0000 7f3319ffb640 10 client.19551 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3319ffb640 1 -- 172.21.15.22:0/2706160671 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560559f06750 con 0x56055a00bb70 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3319ffb640 10 client.19551 handle_command_reply: tid=0 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 client.19551 shutdown 2024-03-09T00:06:57.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559fa46a0 msgr2=0x560559fa9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559fa46a0 0x560559fa9730 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f332c017670 tx=0x7f332c018620 comp rx=0 tx=0).stop 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 shutdown_connections 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x56055a00bb70 0x56055a00df90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560559fa46a0 0x560559fa9730 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560559f9bac0 0x560559fab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3319ffb640 0 client.19551 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 --2- 172.21.15.22:0/2706160671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560559f9b1a0 0x560559fab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 >> 172.21.15.22:0/2706160671 conn(0x560559f958d0 msgr2=0x560559f96c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 shutdown_connections 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 1 -- 172.21.15.22:0/2706160671 wait complete. 2024-03-09T00:06:57.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f333b650740 20 client.19551 trim_cache size 0 max 16384 2024-03-09T00:06:57.669 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33200a2970 msgr2=0x7f33200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33200a2970 0x7f33200a4e30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3324004af0 tx=0x7f3324005c50 comp rx=0 tx=0).stop 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341054b0 msgr2=0x7f333419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341054b0 0x7f333419ddd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f331c00d740 tx=0x7f331c00dc00 comp rx=0 tx=0).stop 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 shutdown_connections 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f33200a2970 0x7f33200a4e30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3334197a40 0x7f3334197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f33341054b0 0x7f333419ddd0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 --2- 172.21.15.22:0/2918956833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33341003e0 0x7f333419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.666+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 >> 172.21.15.22:0/2918956833 conn(0x7f33340773a0 msgr2=0x7f33340687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.667+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 shutdown_connections 2024-03-09T00:06:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.667+0000 7f3338a6c640 1 -- 172.21.15.22:0/2918956833 wait complete. 2024-03-09T00:06:57.673 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:06:57.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:06:57.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1235754 9.40 MB/sec execute 545 sec latency 18.437 ms 2024-03-09T00:06:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.793+0000 7f95dbb59640 1 -- 172.21.15.22:0/2967385528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 msgr2=0x7f95d410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.793+0000 7f95dbb59640 1 --2- 172.21.15.22:0/2967385528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d410cfb0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009ca0 tx=0x7f95c402c0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.793+0000 7f95dbb59640 1 -- 172.21.15.22:0/2967385528 shutdown_connections 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.793+0000 7f95dbb59640 1 --2- 172.21.15.22:0/2967385528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d410cfb0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.793+0000 7f95dbb59640 1 --2- 172.21.15.22:0/2967385528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95d4100440 0x7f95d41008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.793+0000 7f95dbb59640 1 --2- 172.21.15.22:0/2967385528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d4106280 0x7f95d4106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.793+0000 7f95dbb59640 1 -- 172.21.15.22:0/2967385528 >> 172.21.15.22:0/2967385528 conn(0x7f95d4076f90 msgr2=0x7f95d40773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 -- 172.21.15.22:0/2967385528 shutdown_connections 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 -- 172.21.15.22:0/2967385528 wait complete. 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 Processor -- start 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 -- start start 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d4100440 0x7f95d419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95d4106280 0x7f95d41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41105a0 con 0x7f95d4100e00 2024-03-09T00:06:57.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4110420 con 0x7f95d4106280 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95dbb59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4110720 con 0x7f95d4100440 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95d90f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95d98f5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d4100440 0x7f95d419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95da0f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95d4106280 0x7f95d41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95d90f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d419dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48198/0 (socket says 172.21.15.22:48198) 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.794+0000 7f95d90f4640 1 -- 172.21.15.22:0/3542618409 learned_addr learned my addr 172.21.15.22:0/3542618409 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d90f4640 1 -- 172.21.15.22:0/3542618409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d4100440 msgr2=0x7f95d419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d90f4640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d4100440 0x7f95d419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d90f4640 1 -- 172.21.15.22:0/3542618409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95d4106280 msgr2=0x7f95d41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d90f4640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95d4106280 0x7f95d41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d90f4640 1 -- 172.21.15.22:0/3542618409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d4198070 con 0x7f95d4100e00 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95d4106280 0x7f95d41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d90f4640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d419dc80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f95c800e9f0 tx=0x7f95c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d2ffd640 1 -- 172.21.15.22:0/3542618409 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800cd80 con 0x7f95d4100e00 2024-03-09T00:06:57.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d2ffd640 1 -- 172.21.15.22:0/3542618409 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c8010910 con 0x7f95d4100e00 2024-03-09T00:06:57.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d4198360 con 0x7f95d4100e00 2024-03-09T00:06:57.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95d2ffd640 1 -- 172.21.15.22:0/3542618409 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8018990 con 0x7f95d4100e00 2024-03-09T00:06:57.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.795+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d41a4600 con 0x7f95d4100e00 2024-03-09T00:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.797+0000 7f95d2ffd640 1 -- 172.21.15.22:0/3542618409 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f95c8010ac0 con 0x7f95d4100e00 2024-03-09T00:06:57.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 Processor -- start 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 -- start start 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb0126580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb0126ac0 0x55abb012f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb012f6a0 0x55abb0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95d2ffd640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95bc0a2970 0x7f95bc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb00ed360 con 0x55abb012f6a0 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb0089ec0 con 0x55abb0126ac0 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95de807740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb0032d30 con 0x55abb01261a0 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95da0f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb0126580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95d98f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb0126ac0 0x55abb012f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95d90f4640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95bc0a2970 0x7f95bc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95da0f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb0126580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:53870/0 (socket says 172.21.15.22:53870) 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95da0f6640 1 -- 172.21.15.22:0/988585009 learned_addr learned my addr 172.21.15.22:0/988585009 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95d98f5640 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb0126ac0 0x55abb012f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:39306/0 (socket says 172.21.15.22:39306) 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95da0f6640 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb012f6a0 0x55abb0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.798+0000 7f95d2ffd640 1 -- 172.21.15.22:0/3542618409 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f95c80cf600 con 0x7f95d4100e00 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95da0f6640 1 -- 172.21.15.22:0/988585009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb0126ac0 msgr2=0x55abb012f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95da0f6640 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb0126ac0 0x55abb012f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95da0f6640 1 -- 172.21.15.22:0/988585009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb012f6a0 msgr2=0x55abb0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95da0f6640 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb012f6a0 0x55abb0131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95da0f6640 1 -- 172.21.15.22:0/988585009 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb0031bc0 con 0x55abb01261a0 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95d90f4640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95bc0a2970 0x7f95bc0a4e30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95c809b5b0 tx=0x7f95c8018420 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95da0f6640 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb0126580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006010 tx=0x7f95c402e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=280a87a376748c22 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95bbfff640 1 -- 172.21.15.22:0/988585009 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4039040 con 0x55abb01261a0 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95bbfff640 1 -- 172.21.15.22:0/988585009 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c4003970 con 0x55abb01261a0 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.799+0000 7f95bbfff640 1 -- 172.21.15.22:0/988585009 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4004230 con 0x55abb01261a0 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 -- 172.21.15.22:0/988585009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 msgr2=0x55abb0126580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb0126580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f95c4006010 tx=0x7f95c402e040 comp rx=0 tx=0).stop 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 -- 172.21.15.22:0/988585009 shutdown_connections 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb012f6a0 0x55abb0131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb0126ac0 0x55abb012f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 --2- 172.21.15.22:0/988585009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb0126580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 -- 172.21.15.22:0/988585009 >> 172.21.15.22:0/988585009 conn(0x55abb01208d0 msgr2=0x55abb0120cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 -- 172.21.15.22:0/988585009 shutdown_connections 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 -- 172.21.15.22:0/988585009 wait complete. 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 -- start start 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.800+0000 7f95de807740 1 Processor -- start 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb00284a0 con 0x55abb0126ac0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb0085f60 con 0x55abb012f6a0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abb01323d0 con 0x55abb01261a0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 10 client.0 _ll_register_callbacks cb 0x55abb0083d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95da0f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48210/0 (socket says 172.21.15.22:48210) 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 -- 172.21.15.22:0/3670034298 learned_addr learned my addr 172.21.15.22:0/3670034298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 -- 172.21.15.22:0/3670034298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 msgr2=0x55abb012c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 -- 172.21.15.22:0/3670034298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 msgr2=0x55abb012d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 -- 172.21.15.22:0/3670034298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abafd9e3f0 con 0x55abb0126ac0 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:57.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95d98f5640 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012cb60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f95cc012720 tx=0x7f95cc012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dc26a4cd6c647b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95b97fa640 1 -- 172.21.15.22:0/3670034298 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004030 con 0x55abb0126ac0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95b97fa640 1 -- 172.21.15.22:0/3670034298 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f95cc0041d0 con 0x55abb0126ac0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.801+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95c4003970 con 0x55abb0126ac0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95b97fa640 1 -- 172.21.15.22:0/3670034298 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95cc025050 con 0x55abb0126ac0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 10 client.19563 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55abafe687d0 con 0x55abb0126ac0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95b97fa640 1 -- 172.21.15.22:0/3670034298 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c4003970 con 0x55abb0126ac0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95b97fa640 1 -- 172.21.15.22:0/3670034298 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95cc003a60 con 0x55abb0126ac0 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 10 client.19563 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 10 client.19563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 20 client.19563 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb013f850 0x55abb0141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 4 client.19563 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:57.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abb0097400 con 0x55abb013f850 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.802+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb013f850 0x55abb0141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.803+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb013f850 0x55abb0141cb0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490159744b2b8903 server_cookie=be4b1fc772fd168d in_seq=0 out_seq=0 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.803+0000 7f95b97fa640 10 client.19563 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95b97fa640 1 -- 172.21.15.22:0/3670034298 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abb0097400 con 0x55abb013f850 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95b97fa640 10 client.19563 handle_command_reply: tid=0 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 client.19563 shutdown 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 msgr2=0x55abb012cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012cb60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f95cc012720 tx=0x7f95cc012be0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 shutdown_connections 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb013f850 0x55abb0141cb0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95b97fa640 0 client.19563 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012cb60 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 --2- 172.21.15.22:0/3670034298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 >> 172.21.15.22:0/3670034298 conn(0x55abb01208d0 msgr2=0x55abb0122620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 shutdown_connections 2024-03-09T00:06:57.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 1 -- 172.21.15.22:0/3670034298 wait complete. 2024-03-09T00:06:57.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.804+0000 7f95de807740 20 client.19563 trim_cache size 0 max 16384 2024-03-09T00:06:57.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 Processor -- start 2024-03-09T00:06:57.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 -- start start 2024-03-09T00:06:57.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41b0ac0 con 0x55abb0126ac0 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41ae660 con 0x55abb012f6a0 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95de807740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41ac720 con 0x55abb01261a0 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48222/0 (socket says 172.21.15.22:48222) 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 -- 172.21.15.22:0/4022307217 learned_addr learned my addr 172.21.15.22:0/4022307217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95da0f6640 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95da0f6640 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 -- 172.21.15.22:0/4022307217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 msgr2=0x55abb012ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 -- 172.21.15.22:0/4022307217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 msgr2=0x55abb012b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 -- 172.21.15.22:0/4022307217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abb012ab90 con 0x55abb0126ac0 2024-03-09T00:06:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.818+0000 7f95d98f5640 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012af10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f95cc000c00 tx=0x7f95cc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9221f0a59c2d829 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95bb7fe640 1 -- 172.21.15.22:0/4022307217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc022040 con 0x55abb0126ac0 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95bb7fe640 1 -- 172.21.15.22:0/4022307217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95cc0055b0 con 0x55abb0126ac0 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95bb7fe640 1 -- 172.21.15.22:0/4022307217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc003850 con 0x55abb0126ac0 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 -- 172.21.15.22:0/4022307217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 msgr2=0x55abb012af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012af10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f95cc000c00 tx=0x7f95cc005850 comp rx=0 tx=0).stop 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 -- 172.21.15.22:0/4022307217 shutdown_connections 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb012f6a0 0x55abb012b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb012af10 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 --2- 172.21.15.22:0/4022307217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb01261a0 0x55abb012ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 -- 172.21.15.22:0/4022307217 >> 172.21.15.22:0/4022307217 conn(0x55abb01208d0 msgr2=0x55abb013bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 -- 172.21.15.22:0/4022307217 shutdown_connections 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.819+0000 7f95de807740 1 -- 172.21.15.22:0/4022307217 wait complete. 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 -- start start 2024-03-09T00:06:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 Processor -- start 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb01261a0 0x55abb0136b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb0134f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb012f6a0 0x55abb0135600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41acd20 con 0x55abb0126ac0 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41ae000 con 0x55abb01261a0 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41ad340 con 0x55abb012f6a0 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95de807740 10 client.0 _ll_register_callbacks cb 0x7f95c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:57.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95da0f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb01261a0 0x55abb0136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95d98f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb0134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95da0f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb01261a0 0x55abb0136b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:39328/0 (socket says 172.21.15.22:39328) 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95da0f6640 1 -- 172.21.15.22:0/3010506798 learned_addr learned my addr 172.21.15.22:0/3010506798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb012f6a0 0x55abb0135600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95da0f6640 1 -- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb012f6a0 msgr2=0x55abb0135600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.820+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb012f6a0 0x55abb0135600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95da0f6640 1 -- 172.21.15.22:0/3010506798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 msgr2=0x55abb0134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb0134f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95da0f6640 1 -- 172.21.15.22:0/3010506798 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abb0195de0 con 0x55abb01261a0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95d98f5640 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb0134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95da0f6640 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb01261a0 0x55abb0136b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95c4032bc0 tx=0x7f95c4036040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b1ca0fffcb5300 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95b97fa640 1 -- 172.21.15.22:0/3010506798 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4039040 con 0x55abb01261a0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95b97fa640 1 -- 172.21.15.22:0/3010506798 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f95c4032590 con 0x55abb01261a0 2024-03-09T00:06:57.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95cc0055b0 con 0x55abb01261a0 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95b97fa640 1 -- 172.21.15.22:0/3010506798 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c4032790 con 0x55abb01261a0 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95de807740 10 client.28853 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55abb0196ff0 con 0x55abb01261a0 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95b97fa640 1 -- 172.21.15.22:0/3010506798 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95cc0055b0 con 0x55abb01261a0 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95b97fa640 1 -- 172.21.15.22:0/3010506798 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95c403f070 con 0x55abb01261a0 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95de807740 10 client.28853 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95de807740 10 client.28853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.821+0000 7f95de807740 20 client.28853 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.822+0000 7f95de807740 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb01972d0 0x55abb01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.822+0000 7f95de807740 4 client.28853 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.822+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55abb0091750 con 0x55abb01972d0 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.822+0000 7f95d98f5640 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb01972d0 0x55abb01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.822+0000 7f95b97fa640 10 client.28853 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.822+0000 7f95d98f5640 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb01972d0 0x55abb01996c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508469f90882662a server_cookie=614c95b550c0915e in_seq=0 out_seq=0 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95b97fa640 1 -- 172.21.15.22:0/3010506798 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55abb0091750 con 0x55abb01972d0 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95b97fa640 10 client.28853 handle_command_reply: tid=0 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 client.28853 shutdown 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb01261a0 msgr2=0x55abb0136b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb01261a0 0x55abb0136b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95c4032bc0 tx=0x7f95c4036040 comp rx=0 tx=0).stop 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 shutdown_connections 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55abb01972d0 0x55abb01996c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55abb012f6a0 0x55abb0135600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abb0126ac0 0x55abb0134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 --2- 172.21.15.22:0/3010506798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55abb01261a0 0x55abb0136b70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95b97fa640 0 client.28853 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.823+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 >> 172.21.15.22:0/3010506798 conn(0x55abb01208d0 msgr2=0x55abb013bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 shutdown_connections 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95de807740 1 -- 172.21.15.22:0/3010506798 wait complete. 2024-03-09T00:06:57.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95de807740 20 client.28853 trim_cache size 0 max 16384 2024-03-09T00:06:57.824 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:06:57.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95bc0a2970 msgr2=0x7f95bc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95bc0a2970 0x7f95bc0a4e30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95c809b5b0 tx=0x7f95c8018420 comp rx=0 tx=0).stop 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 msgr2=0x7f95d419dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d419dc80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f95c800e9f0 tx=0x7f95c800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 shutdown_connections 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f95bc0a2970 0x7f95bc0a4e30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95d4106280 0x7f95d41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95d4100e00 0x7f95d419dc80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 --2- 172.21.15.22:0/3542618409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d4100440 0x7f95d419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 >> 172.21.15.22:0/3542618409 conn(0x7f95d4076f90 msgr2=0x7f95d40ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 shutdown_connections 2024-03-09T00:06:57.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:57.824+0000 7f95dbb59640 1 -- 172.21.15.22:0/3542618409 wait complete. 2024-03-09T00:06:58.431 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:58 smithi192 ceph-mon[35995]: pgmap v391: 129 pgs: 129 active+clean; 98 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-03-09T00:06:58.431 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:06:58 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/900217535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:58 smithi022 ceph-mon[31894]: pgmap v391: 129 pgs: 129 active+clean; 98 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-03-09T00:06:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:06:58 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/900217535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:58 smithi062 ceph-mon[36129]: pgmap v391: 129 pgs: 129 active+clean; 98 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-03-09T00:06:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:06:58 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/900217535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:06:58.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1236602 9.40 MB/sec execute 546 sec latency 72.294 ms 2024-03-09T00:06:58.831 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:06:58.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:06:58.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:06:59.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c8ffff640 1 -- 172.21.15.22:0/1028806199 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8802c860 con 0x7f9c98106240 2024-03-09T00:06:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c9f324640 1 -- 172.21.15.22:0/1028806199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98106240 msgr2=0x7f9c980684f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c9f324640 1 --2- 172.21.15.22:0/1028806199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98106240 0x7f9c980684f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009930 tx=0x7f9c8802bc60 comp rx=0 tx=0).stop 2024-03-09T00:06:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c9f324640 1 -- 172.21.15.22:0/1028806199 shutdown_connections 2024-03-09T00:06:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c9f324640 1 --2- 172.21.15.22:0/1028806199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c98103db0 0x7f9c98104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c9f324640 1 --2- 172.21.15.22:0/1028806199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c98068a30 0x7f9c98068e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c9f324640 1 --2- 172.21.15.22:0/1028806199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98106240 0x7f9c980684f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.007+0000 7f9c9f324640 1 -- 172.21.15.22:0/1028806199 >> 172.21.15.22:0/1028806199 conn(0x7f9c980772f0 msgr2=0x7f9c98077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.012+0000 7f9c9f324640 1 -- 172.21.15.22:0/1028806199 shutdown_connections 2024-03-09T00:06:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.014+0000 7f9c9f324640 1 -- 172.21.15.22:0/1028806199 wait complete. 2024-03-09T00:06:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.014+0000 7f9c9f324640 1 Processor -- start 2024-03-09T00:06:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.014+0000 7f9c9f324640 1 -- start start 2024-03-09T00:06:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9f324640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c98068a30 0x7f9c98072930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9f324640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98103db0 0x7f9c98072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9f324640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9806c9f0 0x7f9c9806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9f324640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9806b1c0 con 0x7f9c9806c9f0 2024-03-09T00:06:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9f324640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9806b040 con 0x7f9c98103db0 2024-03-09T00:06:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9f324640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9806b340 con 0x7f9c98068a30 2024-03-09T00:06:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9d8c1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9806c9f0 0x7f9c9806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9d8c1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9806c9f0 0x7f9c9806cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48232/0 (socket says 172.21.15.22:48232) 2024-03-09T00:06:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.015+0000 7f9c9d8c1640 1 -- 172.21.15.22:0/876624174 learned_addr learned my addr 172.21.15.22:0/876624174 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98103db0 0x7f9c98072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c98068a30 0x7f9c98072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9d8c1640 1 -- 172.21.15.22:0/876624174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c98068a30 msgr2=0x7f9c98072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c98068a30 0x7f9c98072930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9d8c1640 1 -- 172.21.15.22:0/876624174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98103db0 msgr2=0x7f9c98072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98103db0 0x7f9c98072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9d8c1640 1 -- 172.21.15.22:0/876624174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9806d7b0 con 0x7f9c9806c9f0 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.016+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98103db0 0x7f9c98072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.017+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9806c9f0 0x7f9c9806cea0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000eaa0 tx=0x7f9c9000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.018+0000 7f9c8e7fc640 1 -- 172.21.15.22:0/876624174 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90004d20 con 0x7f9c9806c9f0 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.018+0000 7f9c8e7fc640 1 -- 172.21.15.22:0/876624174 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c90005020 con 0x7f9c9806c9f0 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.018+0000 7f9c8e7fc640 1 -- 172.21.15.22:0/876624174 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900116d0 con 0x7f9c9806c9f0 2024-03-09T00:06:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.018+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c98106700 con 0x7f9c9806c9f0 2024-03-09T00:06:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.018+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c98106c00 con 0x7f9c9806c9f0 2024-03-09T00:06:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.023+0000 7f9c8e7fc640 1 -- 172.21.15.22:0/876624174 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9c90011870 con 0x7f9c9806c9f0 2024-03-09T00:06:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.023+0000 7f9ca1fd7740 1 Processor -- start 2024-03-09T00:06:59.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.024+0000 7f9c8e7fc640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c840a2b70 0x7f9c840a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9c8e7fc640 1 -- 172.21.15.22:0/876624174 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9c900cf2b0 con 0x7f9c9806c9f0 2024-03-09T00:06:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9ca1fd7740 1 -- start start 2024-03-09T00:06:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c840a2b70 0x7f9c840a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627a9e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627aa6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627aa7280 0x562627aa9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562627a01b40 con 0x562627a9e6a0 2024-03-09T00:06:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626279aad80 con 0x562627aa7280 2024-03-09T00:06:59.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.025+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626279fdbe0 con 0x562627a9dd80 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.026+0000 7f9c9c8bf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627aa6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.026+0000 7f9c9c8bf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627aa6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48236/0 (socket says 172.21.15.22:48236) 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.026+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/3627475413 learned_addr learned my addr 172.21.15.22:0/3627475413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.026+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627a9e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.026+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627aa7280 0x562627aa9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.027+0000 7f9c9d8c1640 1 -- 172.21.15.22:0/3627475413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 msgr2=0x562627a9e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.027+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627a9e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.027+0000 7f9c9d8c1640 1 -- 172.21.15.22:0/3627475413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 msgr2=0x562627aa6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.027+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627aa6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.027+0000 7f9c9d8c1640 1 -- 172.21.15.22:0/3627475413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626277de7e0 con 0x562627aa7280 2024-03-09T00:06:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.027+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627aa6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:59.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.028+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c840a2b70 0x7f9c840a5030 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9c94015a40 tx=0x7f9c94015530 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.028+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627aa7280 0x562627aa9670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c900cc640 tx=0x7f9c9009cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5689c85285577eb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.036+0000 7f9c837fe640 1 -- 172.21.15.22:0/3627475413 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9001f040 con 0x562627aa7280 2024-03-09T00:06:59.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.036+0000 7f9c837fe640 1 -- 172.21.15.22:0/3627475413 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c900a4d30 con 0x562627aa7280 2024-03-09T00:06:59.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.036+0000 7f9c837fe640 1 -- 172.21.15.22:0/3627475413 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900a48a0 con 0x562627aa7280 2024-03-09T00:06:59.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.037+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3627475413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627aa7280 msgr2=0x562627aa9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.037+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627aa7280 0x562627aa9670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c900cc640 tx=0x7f9c9009cdd0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.038+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3627475413 shutdown_connections 2024-03-09T00:06:59.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.038+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627aa7280 0x562627aa9670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.038+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627aa6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.038+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3627475413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627a9e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.039+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3627475413 >> 172.21.15.22:0/3627475413 conn(0x562627a984b0 msgr2=0x562627a98890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.039+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3627475413 shutdown_connections 2024-03-09T00:06:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.039+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3627475413 wait complete. 2024-03-09T00:06:59.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.040+0000 7f9ca1fd7740 1 -- start start 2024-03-09T00:06:59.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.040+0000 7f9ca1fd7740 1 Processor -- start 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627aa6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9e6a0 0x562627aa4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aa47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562627aaa010 con 0x562627aa7280 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626279be1c0 con 0x562627a9dd80 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562627aaa190 con 0x562627a9e6a0 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9c9c8bf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9e6a0 0x562627aa4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9c9c8bf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9e6a0 0x562627aa4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:53926/0 (socket says 172.21.15.22:53926) 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/4091362167 learned_addr learned my addr 172.21.15.22:0/4091362167 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9ca1fd7740 10 client.0 _ll_register_callbacks cb 0x5626279ad7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627aa6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.041+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aa47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/4091362167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 msgr2=0x562627aa6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627aa6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/4091362167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 msgr2=0x562627aa47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aa47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/4091362167 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56262783ef70 con 0x562627a9e6a0 2024-03-09T00:06:59.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aa47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9e6a0 0x562627aa4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c94017ec0 tx=0x7f9c940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d8ec8279bd8e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.042+0000 7f9c80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:06:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.043+0000 7f9c80ff9640 1 -- 172.21.15.22:0/4091362167 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9401e070 con 0x562627a9e6a0 2024-03-09T00:06:59.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.043+0000 7f9c80ff9640 1 -- 172.21.15.22:0/4091362167 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9c94024470 con 0x562627a9e6a0 2024-03-09T00:06:59.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.045+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c900a4d30 con 0x562627a9e6a0 2024-03-09T00:06:59.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.047+0000 7f9c80ff9640 1 -- 172.21.15.22:0/4091362167 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c94024670 con 0x562627a9e6a0 2024-03-09T00:06:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.047+0000 7f9ca1fd7740 10 client.29008 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.047+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5626276d6170 con 0x562627a9e6a0 2024-03-09T00:06:59.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.048+0000 7f9c80ff9640 1 -- 172.21.15.22:0/4091362167 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c94029070 con 0x562627a9e6a0 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.048+0000 7f9c80ff9640 1 -- 172.21.15.22:0/4091362167 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c9402e7e0 con 0x562627a9e6a0 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.049+0000 7f9ca1fd7740 10 client.29008 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.049+0000 7f9ca1fd7740 10 client.29008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.049+0000 7f9ca1fd7740 20 client.29008 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.049+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627ab73b0 0x562627ab97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.049+0000 7f9ca1fd7740 4 client.29008 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.049+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c900a4d30 con 0x562627ab73b0 2024-03-09T00:06:59.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.049+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627ab73b0 0x562627ab97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.050+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627ab73b0 0x562627ab97a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12429209d7f9a946 server_cookie=8e33c1b31716f426 in_seq=0 out_seq=0 2024-03-09T00:06:59.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.050+0000 7f9c80ff9640 10 client.29008 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9c80ff9640 1 -- 172.21.15.22:0/4091362167 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f9c900a4d30 con 0x562627ab73b0 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9c80ff9640 10 client.29008 handle_command_reply: tid=0 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 client.29008 shutdown 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9e6a0 msgr2=0x562627aa4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9e6a0 0x562627aa4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c94017ec0 tx=0x7f9c940046c0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 shutdown_connections 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627ab73b0 0x562627ab97a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9c80ff9640 0 client.29008 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aa47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9e6a0 0x562627aa4060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/4091362167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627aa6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.054+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 >> 172.21.15.22:0/4091362167 conn(0x562627a984b0 msgr2=0x562627a9a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.056+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 shutdown_connections 2024-03-09T00:06:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.057+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/4091362167 wait complete. 2024-03-09T00:06:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.057+0000 7f9ca1fd7740 20 client.29008 trim_cache size 0 max 16384 2024-03-09T00:06:59.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.081+0000 7f9ca1fd7740 1 Processor -- start 2024-03-09T00:06:59.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9ca1fd7740 1 -- start start 2024-03-09T00:06:59.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627aa4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9e6a0 0x562627aae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c0123f0 con 0x562627aa7280 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c00ff90 con 0x562627a9e6a0 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c00e050 con 0x562627a9dd80 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9e6a0 0x562627aae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9d0c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627aa4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9e6a0 0x562627aae400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:39366/0 (socket says 172.21.15.22:39366) 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/693463682 learned_addr learned my addr 172.21.15.22:0/693463682 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/693463682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 msgr2=0x562627aa4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627aa4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/693463682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 msgr2=0x562627aae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.082+0000 7f9c9c8bf640 1 -- 172.21.15.22:0/693463682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562627a9e4d0 con 0x562627a9e6a0 2024-03-09T00:06:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.083+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9e6a0 0x562627aae400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c940225b0 tx=0x7f9c94006e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb69588d94f2671 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.083+0000 7f9c82ffd640 1 -- 172.21.15.22:0/693463682 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9403c070 con 0x562627a9e6a0 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.083+0000 7f9c82ffd640 1 -- 172.21.15.22:0/693463682 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c94029040 con 0x562627a9e6a0 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.083+0000 7f9c82ffd640 1 -- 172.21.15.22:0/693463682 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94024a70 con 0x562627a9e6a0 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/693463682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9e6a0 msgr2=0x562627aae400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9e6a0 0x562627aae400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c940225b0 tx=0x7f9c94006e40 comp rx=0 tx=0).stop 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/693463682 shutdown_connections 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627aa7280 0x562627aae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9e6a0 0x562627aae400 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/693463682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627a9dd80 0x562627aa4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/693463682 >> 172.21.15.22:0/693463682 conn(0x562627a984b0 msgr2=0x562627abe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/693463682 shutdown_connections 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.084+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/693463682 wait complete. 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.085+0000 7f9ca1fd7740 1 -- start start 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.085+0000 7f9ca1fd7740 1 Processor -- start 2024-03-09T00:06:59.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627b06df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627b07330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9ca1fd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627aa7280 0x562627aad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c00e650 con 0x562627a9e6a0 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c00f930 con 0x562627a9dd80 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9ca1fd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c00ec70 con 0x562627aa7280 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9ca1fd7740 10 client.0 _ll_register_callbacks cb 0x7f9c90098170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627b06df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627b06df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:39374/0 (socket says 172.21.15.22:39374) 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 -- 172.21.15.22:0/3342570165 learned_addr learned my addr 172.21.15.22:0/3342570165 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627b07330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d8c1640 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627aa7280 0x562627aad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 -- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627aa7280 msgr2=0x562627aad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627aa7280 0x562627aad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 -- 172.21.15.22:0/3342570165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 msgr2=0x562627b07330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627b07330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 -- 172.21.15.22:0/3342570165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562627ab9b10 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c9d0c0640 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627b06df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9c8802c270 tx=0x7f9c8802c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39b117644c67bc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.086+0000 7f9c80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9c80ff9640 1 -- 172.21.15.22:0/3342570165 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88049040 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c94029040 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9c80ff9640 1 -- 172.21.15.22:0/3342570165 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f9c8802c900 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9c80ff9640 1 -- 172.21.15.22:0/3342570165 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c88064050 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 10 client.28871 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x562627b0f590 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9c80ff9640 1 -- 172.21.15.22:0/3342570165 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c94029040 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9c80ff9640 1 -- 172.21.15.22:0/3342570165 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9c88051440 con 0x562627a9dd80 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 10 client.28871 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 10 client.28871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:59.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 20 client.28871 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:59.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627b0fce0 0x562627b12100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 4 client.28871 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:06:59.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5626279d7960 con 0x562627b0fce0 2024-03-09T00:06:59.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.087+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627b0fce0 0x562627b12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.088+0000 7f9c80ff9640 10 client.28871 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.088+0000 7f9c9c8bf640 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627b0fce0 0x562627b12100 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99471d17b27419b5 server_cookie=24ee31008fee14d3 in_seq=0 out_seq=0 2024-03-09T00:06:59.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9c80ff9640 1 -- 172.21.15.22:0/3342570165 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5626279d7960 con 0x562627b0fce0 2024-03-09T00:06:59.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9c80ff9640 10 client.28871 handle_command_reply: tid=0 2024-03-09T00:06:59.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 client.28871 shutdown 2024-03-09T00:06:59.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 msgr2=0x562627b06df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627b06df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9c8802c270 tx=0x7f9c8802c810 comp rx=0 tx=0).stop 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 shutdown_connections 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562627b0fce0 0x562627b12100 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562627aa7280 0x562627aad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562627a9e6a0 0x562627b07330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 --2- 172.21.15.22:0/3342570165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562627a9dd80 0x562627b06df0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 >> 172.21.15.22:0/3342570165 conn(0x562627a984b0 msgr2=0x562627a998e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9c80ff9640 0 client.28871 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 shutdown_connections 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.089+0000 7f9ca1fd7740 1 -- 172.21.15.22:0/3342570165 wait complete. 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9ca1fd7740 20 client.28871 trim_cache size 0 max 16384 2024-03-09T00:06:59.088 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:59.089 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:06:59.089 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "ba65b749-b15b-48ac-a02c-b6b45ecf59a5", 2024-03-09T00:06:59.089 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:06:59.089 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c840a2b70 msgr2=0x7f9c840a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c840a2b70 0x7f9c840a5030 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9c94015a40 tx=0x7f9c94015530 comp rx=0 tx=0).stop 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9806c9f0 msgr2=0x7f9c9806cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9806c9f0 0x7f9c9806cea0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000eaa0 tx=0x7f9c9000eea0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 shutdown_connections 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9c840a2b70 0x7f9c840a5030 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c9806c9f0 0x7f9c9806cea0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c98103db0 0x7f9c98072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 --2- 172.21.15.22:0/876624174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c98068a30 0x7f9c98072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 >> 172.21.15.22:0/876624174 conn(0x7f9c980772f0 msgr2=0x7f9c98102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 shutdown_connections 2024-03-09T00:06:59.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.090+0000 7f9c9f324640 1 -- 172.21.15.22:0/876624174 wait complete. 2024-03-09T00:06:59.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:06:59.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 -- 172.21.15.22:0/1690399312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0101eb0 msgr2=0x7f87e0105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/1690399312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0101eb0 0x7f87e0105ef0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009900 tx=0x7f87cc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 -- 172.21.15.22:0/1690399312 shutdown_connections 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/1690399312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87e01065d0 0x7f87e010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/1690399312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0101eb0 0x7f87e0105ef0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/1690399312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87e0101500 0x7f87e01018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 -- 172.21.15.22:0/1690399312 >> 172.21.15.22:0/1690399312 conn(0x7f87e00fcd20 msgr2=0x7f87e00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 -- 172.21.15.22:0/1690399312 shutdown_connections 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 -- 172.21.15.22:0/1690399312 wait complete. 2024-03-09T00:06:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 Processor -- start 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.218+0000 7f87e6f7e640 1 -- start start 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e6f7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87e0101500 0x7f87e00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e6f7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87e01065d0 0x7f87e0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e6f7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0074070 0x7f87e01a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e6f7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0111660 con 0x7f87e0074070 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e6f7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01114e0 con 0x7f87e01065d0 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e6f7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01117e0 con 0x7f87e0101500 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0074070 0x7f87e01a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0074070 0x7f87e01a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48268/0 (socket says 172.21.15.22:48268) 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 -- 172.21.15.22:0/533414680 learned_addr learned my addr 172.21.15.22:0/533414680 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87dffff640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87e01065d0 0x7f87e0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87e0101500 0x7f87e00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 -- 172.21.15.22:0/533414680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87e0101500 msgr2=0x7f87e00735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87e0101500 0x7f87e00735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 -- 172.21.15.22:0/533414680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87e01065d0 msgr2=0x7f87e0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87e01065d0 0x7f87e0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 -- 172.21.15.22:0/533414680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e01a6550 con 0x7f87e0074070 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87dffff640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87e01065d0 0x7f87e0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e551b640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0074070 0x7f87e01a5eb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009fd0 tx=0x7f87d800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87ddffb640 1 -- 172.21.15.22:0/533414680 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8017070 con 0x7f87e0074070 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e01a6840 con 0x7f87e0074070 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.219+0000 7f87ddffb640 1 -- 172.21.15.22:0/533414680 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d8003e50 con 0x7f87e0074070 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.220+0000 7f87ddffb640 1 -- 172.21.15.22:0/533414680 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80106c0 con 0x7f87e0074070 2024-03-09T00:06:59.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.220+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01a6d60 con 0x7f87e0074070 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87ddffb640 1 -- 172.21.15.22:0/533414680 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f87d8003970 con 0x7f87e0074070 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 Processor -- start 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 -- start start 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87ddffb640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87d00a2970 0x7f87d00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d6748b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6751720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87ddffb640 1 -- 172.21.15.22:0/533414680 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f87d80cef00 con 0x7f87e0074070 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6751c60 0x55b7d6754050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d66abe00 con 0x55b7d6749080 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d6654b10 con 0x55b7d6748760 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87e9b66740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d66a7ea0 con 0x55b7d6751c60 2024-03-09T00:06:59.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.222+0000 7f87dffff640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87d00a2970 0x7f87d00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6751720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6751720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48272/0 (socket says 172.21.15.22:48272) 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87dffff640 1 -- 172.21.15.22:0/2580122978 learned_addr learned my addr 172.21.15.22:0/2580122978 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e551b640 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6751c60 0x55b7d6754050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d6748b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e551b640 1 -- 172.21.15.22:0/2580122978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 msgr2=0x55b7d6748b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e551b640 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d6748b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e551b640 1 -- 172.21.15.22:0/2580122978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 msgr2=0x55b7d6751720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e551b640 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6751720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e551b640 1 -- 172.21.15.22:0/2580122978 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7d648a7d0 con 0x55b7d6751c60 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87dffff640 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6751720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d6748b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87dffff640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87d00a2970 0x7f87d00a4e30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87cc02c820 tx=0x7f87cc009680 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87e551b640 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6751c60 0x55b7d6754050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cc250 tx=0x7f87d809b0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542af5cd503492d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87caffd640 1 -- 172.21.15.22:0/2580122978 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8020040 con 0x55b7d6751c60 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87caffd640 1 -- 172.21.15.22:0/2580122978 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d80a3900 con 0x55b7d6751c60 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.223+0000 7f87caffd640 1 -- 172.21.15.22:0/2580122978 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8097880 con 0x55b7d6751c60 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 -- 172.21.15.22:0/2580122978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6751c60 msgr2=0x55b7d6754050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6751c60 0x55b7d6754050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cc250 tx=0x7f87d809b0f0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 -- 172.21.15.22:0/2580122978 shutdown_connections 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6751c60 0x55b7d6754050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6751720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2580122978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d6748b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 -- 172.21.15.22:0/2580122978 >> 172.21.15.22:0/2580122978 conn(0x55b7d6742e90 msgr2=0x55b7d6743270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 -- 172.21.15.22:0/2580122978 shutdown_connections 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 -- 172.21.15.22:0/2580122978 wait complete. 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 -- start start 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.224+0000 7f87e9b66740 1 Processor -- start 2024-03-09T00:06:59.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d67513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d674e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6751c60 0x55b7d674ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e9b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d67549f0 con 0x55b7d6751c60 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e9b66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d66680a0 con 0x55b7d6748760 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e9b66740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d6754b70 con 0x55b7d6749080 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e9b66740 10 client.0 _ll_register_callbacks cb 0x55b7d6667a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6751c60 0x55b7d674ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87dffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d674e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e4d1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d67513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6751c60 0x55b7d674ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48286/0 (socket says 172.21.15.22:48286) 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 -- 172.21.15.22:0/154784906 learned_addr learned my addr 172.21.15.22:0/154784906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 -- 172.21.15.22:0/154784906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 msgr2=0x55b7d674e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d674e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 -- 172.21.15.22:0/154784906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 msgr2=0x55b7d67513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d67513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.225+0000 7f87e551b640 1 -- 172.21.15.22:0/154784906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7d63c03f0 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e551b640 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6751c60 0x55b7d674ec90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f87d8004350 tx=0x7f87d809b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160527f600f0c759 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87bffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87bffff640 1 -- 172.21.15.22:0/154784906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80b3070 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87bffff640 1 -- 172.21.15.22:0/154784906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f87d80cc4d0 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87d80a3900 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87bffff640 1 -- 172.21.15.22:0/154784906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d8081050 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 10 client.19593 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b7d63800f0 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87bffff640 1 -- 172.21.15.22:0/154784906 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d80a3900 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87bffff640 1 -- 172.21.15.22:0/154784906 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87d809a6f0 con 0x55b7d6751c60 2024-03-09T00:06:59.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 10 client.19593 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 10 client.19593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 20 client.19593 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67614b0 0x55b7d67638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 4 client.19593 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.226+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7d66b9340 con 0x55b7d67614b0 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.227+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67614b0 0x55b7d67638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.227+0000 7f87bffff640 10 client.19593 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.227+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67614b0 0x55b7d67638a0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274d05e446480b84 server_cookie=4aceaf6ce384cfd7 in_seq=0 out_seq=0 2024-03-09T00:06:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87bffff640 1 -- 172.21.15.22:0/154784906 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7d66b9340 con 0x55b7d67614b0 2024-03-09T00:06:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87bffff640 10 client.19593 handle_command_reply: tid=0 2024-03-09T00:06:59.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 client.19593 shutdown 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6751c60 msgr2=0x55b7d674ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6751c60 0x55b7d674ec90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f87d8004350 tx=0x7f87d809b310 comp rx=0 tx=0).stop 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 shutdown_connections 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67614b0 0x55b7d67638a0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6751c60 0x55b7d674ec90 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d674e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87bffff640 0 client.19593 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 --2- 172.21.15.22:0/154784906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6748760 0x55b7d67513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 >> 172.21.15.22:0/154784906 conn(0x55b7d6742e90 msgr2=0x55b7d6744be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 shutdown_connections 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 1 -- 172.21.15.22:0/154784906 wait complete. 2024-03-09T00:06:59.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.229+0000 7f87e9b66740 20 client.19593 trim_cache size 0 max 16384 2024-03-09T00:06:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.244+0000 7f87e9b66740 1 Processor -- start 2024-03-09T00:06:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.244+0000 7f87e9b66740 1 -- start start 2024-03-09T00:06:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6748760 0x55b7d6749cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d6750e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d67513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e9b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01b2d60 con 0x55b7d6748760 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e9b66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01b0900 con 0x55b7d6751c60 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e9b66740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01ae9c0 con 0x55b7d6749080 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6748760 0x55b7d6749cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6748760 0x55b7d6749cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48294/0 (socket says 172.21.15.22:48294) 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 -- 172.21.15.22:0/2176372920 learned_addr learned my addr 172.21.15.22:0/2176372920 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e551b640 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d67513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87dffff640 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d6750e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 -- 172.21.15.22:0/2176372920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 msgr2=0x55b7d6750e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d6750e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 -- 172.21.15.22:0/2176372920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 msgr2=0x55b7d67513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d67513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 -- 172.21.15.22:0/2176372920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7d674cca0 con 0x55b7d6748760 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6748760 0x55b7d6749cb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009d80 tx=0x7f87d4014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8227f90593fe1d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87ca7fc640 1 -- 172.21.15.22:0/2176372920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4004050 con 0x55b7d6748760 2024-03-09T00:06:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87ca7fc640 1 -- 172.21.15.22:0/2176372920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d401d8b0 con 0x55b7d6748760 2024-03-09T00:06:59.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.245+0000 7f87ca7fc640 1 -- 172.21.15.22:0/2176372920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4003bc0 con 0x55b7d6748760 2024-03-09T00:06:59.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 -- 172.21.15.22:0/2176372920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6748760 msgr2=0x55b7d6749cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6748760 0x55b7d6749cb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009d80 tx=0x7f87d4014840 comp rx=0 tx=0).stop 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 -- 172.21.15.22:0/2176372920 shutdown_connections 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d67513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6749080 0x55b7d6750e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 --2- 172.21.15.22:0/2176372920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6748760 0x55b7d6749cb0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 -- 172.21.15.22:0/2176372920 >> 172.21.15.22:0/2176372920 conn(0x55b7d6742e90 msgr2=0x55b7d67495a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 -- 172.21.15.22:0/2176372920 shutdown_connections 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 -- 172.21.15.22:0/2176372920 wait complete. 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 -- start start 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.246+0000 7f87e9b66740 1 Processor -- start 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6748760 0x55b7d6759080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6755400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e9b66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d6755940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e9b66740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01aefc0 con 0x55b7d6749080 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e9b66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01b02a0 con 0x55b7d6751c60 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e9b66740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e01af5e0 con 0x55b7d6748760 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e9b66740 10 client.0 _ll_register_callbacks cb 0x7f87cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6755400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6755400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48302/0 (socket says 172.21.15.22:48302) 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 -- 172.21.15.22:0/1835391490 learned_addr learned my addr 172.21.15.22:0/1835391490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6748760 0x55b7d6759080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e551b640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d6755940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 -- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6748760 msgr2=0x55b7d6759080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6748760 0x55b7d6759080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 -- 172.21.15.22:0/1835391490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 msgr2=0x55b7d6755940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d6755940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 -- 172.21.15.22:0/1835391490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7d6756140 con 0x55b7d6749080 2024-03-09T00:06:59.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87e551b640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d6755940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87dffff640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6755400 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f87e006a980 tx=0x7f87cc048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6b194c6eea34b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.247+0000 7f87bffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87bffff640 1 -- 172.21.15.22:0/1835391490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc005410 con 0x55b7d6749080 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87bffff640 1 -- 172.21.15.22:0/1835391490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f87cc0055b0 con 0x55b7d6749080 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87d401d8b0 con 0x55b7d6749080 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87bffff640 1 -- 172.21.15.22:0/1835391490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87cc060050 con 0x55b7d6749080 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 10 client.19599 fetch_fsmap learned FSMap version 35 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b7d67b8600 con 0x55b7d6749080 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87bffff640 1 -- 172.21.15.22:0/1835391490 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d401d8b0 con 0x55b7d6749080 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87bffff640 1 -- 172.21.15.22:0/1835391490 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87cc034740 con 0x55b7d6749080 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 10 client.19599 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 10 client.19599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:06:59.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 20 client.19599 populate_metadata read hostname 'smithi022' 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67b8d60 0x55b7d67bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 4 client.19599 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b7d66b3690 con 0x55b7d67b8d60 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.248+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67b8d60 0x55b7d67bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e4d1a640 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67b8d60 0x55b7d67bb180 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed328ea16fb4b90 server_cookie=87f0a5297016517b in_seq=0 out_seq=0 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87bffff640 10 client.19599 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87bffff640 1 -- 172.21.15.22:0/1835391490 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b7d66b3690 con 0x55b7d67b8d60 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87bffff640 10 client.19599 handle_command_reply: tid=0 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 client.19599 shutdown 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 msgr2=0x55b7d6755400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6755400 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f87e006a980 tx=0x7f87cc048000 comp rx=0 tx=0).stop 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 shutdown_connections 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b7d67b8d60 0x55b7d67bb180 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7d6751c60 0x55b7d6755940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87bffff640 0 client.19599 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b7d6749080 0x55b7d6755400 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 --2- 172.21.15.22:0/1835391490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7d6748760 0x55b7d6759080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 >> 172.21.15.22:0/1835391490 conn(0x55b7d6742e90 msgr2=0x55b7d674cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 shutdown_connections 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.249+0000 7f87e9b66740 1 -- 172.21.15.22:0/1835391490 wait complete. 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e9b66740 20 client.19599 trim_cache size 0 max 16384 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:06:59.251 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:06:59.252 INFO:teuthology.orchestra.run.smithi022.stdout: "ba65b749-b15b-48ac-a02c-b6b45ecf59a5": { 2024-03-09T00:06:59.252 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:06:59.252 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "ba65b749-b15b-48ac-a02c-b6b45ecf59a5", 2024-03-09T00:06:59.252 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:06:59.252 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:06:59.252 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:06:59.252 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87d00a2970 msgr2=0x7f87d00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87d00a2970 0x7f87d00a4e30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87cc02c820 tx=0x7f87cc009680 comp rx=0 tx=0).stop 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0074070 msgr2=0x7f87e01a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0074070 0x7f87e01a5eb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009fd0 tx=0x7f87d800b260 comp rx=0 tx=0).stop 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 shutdown_connections 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87d00a2970 0x7f87d00a4e30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87e0074070 0x7f87e01a5eb0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87e01065d0 0x7f87e0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 --2- 172.21.15.22:0/533414680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87e0101500 0x7f87e00735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:06:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 >> 172.21.15.22:0/533414680 conn(0x7f87e00fcd20 msgr2=0x7f87e00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:06:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 shutdown_connections 2024-03-09T00:06:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:06:59.250+0000 7f87e6f7e640 1 -- 172.21.15.22:0/533414680 wait complete. 2024-03-09T00:06:59.255 INFO:tasks.cephfs.filesystem:scrub status for tag:ba65b749-b15b-48ac-a02c-b6b45ecf59a5 - {'path': '/', 'tag': 'ba65b749-b15b-48ac-a02c-b6b45ecf59a5', 'options': 'recursive,force'} 2024-03-09T00:06:59.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1238281 9.39 MB/sec execute 547 sec latency 36.370 ms 2024-03-09T00:07:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:00 smithi022 ceph-mon[31894]: pgmap v392: 129 pgs: 129 active+clean; 99 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2024-03-09T00:07:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:00 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:07:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:00 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:00 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:07:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:00 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:00 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:07:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:00 smithi062 ceph-mon[36129]: pgmap v392: 129 pgs: 129 active+clean; 99 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2024-03-09T00:07:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:00 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:07:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:00 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:00 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:07:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:00 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:00 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:07:00.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:00 smithi192 ceph-mon[35995]: pgmap v392: 129 pgs: 129 active+clean; 99 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2024-03-09T00:07:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:00 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:07:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:00 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:00 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:07:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:00 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:00 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:07:00.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1239890 9.40 MB/sec execute 548 sec latency 12.041 ms 2024-03-09T00:07:01.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240050 9.38 MB/sec execute 549 sec latency 835.003 ms 2024-03-09T00:07:02.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:02 smithi022 ceph-mon[31894]: pgmap v393: 129 pgs: 129 active+clean; 88 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-03-09T00:07:02.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:02 smithi062 ceph-mon[36129]: pgmap v393: 129 pgs: 129 active+clean; 88 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-03-09T00:07:02.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:02 smithi192 ceph-mon[35995]: pgmap v393: 129 pgs: 129 active+clean; 88 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-03-09T00:07:02.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240050 9.36 MB/sec execute 550 sec latency 1835.119 ms 2024-03-09T00:07:03.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240149 9.35 MB/sec execute 551 sec latency 2739.027 ms 2024-03-09T00:07:04.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:04 smithi022 ceph-mon[31894]: pgmap v394: 129 pgs: 129 active+clean; 88 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-03-09T00:07:04.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:04 smithi062 ceph-mon[36129]: pgmap v394: 129 pgs: 129 active+clean; 88 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-03-09T00:07:04.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:04 smithi192 ceph-mon[35995]: pgmap v394: 129 pgs: 129 active+clean; 88 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-03-09T00:07:04.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240510 9.33 MB/sec execute 552 sec latency 140.967 ms 2024-03-09T00:07:05.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240515 9.31 MB/sec execute 553 sec latency 405.335 ms 2024-03-09T00:07:06.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:06 smithi022 ceph-mon[31894]: pgmap v395: 129 pgs: 129 active+clean; 91 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-09T00:07:06.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:06 smithi022 ceph-mon[31894]: osdmap e111: 12 total, 12 up, 12 in 2024-03-09T00:07:06.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:06 smithi062 ceph-mon[36129]: pgmap v395: 129 pgs: 129 active+clean; 91 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-09T00:07:06.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:06 smithi062 ceph-mon[36129]: osdmap e111: 12 total, 12 up, 12 in 2024-03-09T00:07:06.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:06 smithi192 ceph-mon[35995]: pgmap v395: 129 pgs: 129 active+clean; 91 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-03-09T00:07:06.633 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:06 smithi192 ceph-mon[35995]: osdmap e111: 12 total, 12 up, 12 in 2024-03-09T00:07:06.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240516 9.30 MB/sec execute 554 sec latency 809.417 ms 2024-03-09T00:07:07.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240517 9.28 MB/sec execute 555 sec latency 783.990 ms 2024-03-09T00:07:08.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:08 smithi022 ceph-mon[31894]: pgmap v397: 129 pgs: 129 active+clean; 91 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-09T00:07:08.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:08 smithi062 ceph-mon[36129]: pgmap v397: 129 pgs: 129 active+clean; 91 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-09T00:07:08.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:08 smithi192 ceph-mon[35995]: pgmap v397: 129 pgs: 129 active+clean; 91 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-09T00:07:08.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1240971 9.27 MB/sec execute 556 sec latency 798.521 ms 2024-03-09T00:07:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.496+0000 7f249364a640 1 -- 172.21.15.22:0/202318666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 msgr2=0x7f248c10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.496+0000 7f249364a640 1 --2- 172.21.15.22:0/202318666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c10dcb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2488009930 tx=0x7f248802bc60 comp rx=0 tx=0).stop 2024-03-09T00:07:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.496+0000 7f249364a640 1 -- 172.21.15.22:0/202318666 shutdown_connections 2024-03-09T00:07:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.496+0000 7f249364a640 1 --2- 172.21.15.22:0/202318666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c108280 0x7f248c114790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.496+0000 7f249364a640 1 --2- 172.21.15.22:0/202318666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f248c1078c0 0x7f248c107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.496+0000 7f249364a640 1 --2- 172.21.15.22:0/202318666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c10dcb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.496+0000 7f249364a640 1 -- 172.21.15.22:0/202318666 >> 172.21.15.22:0/202318666 conn(0x7f248c075860 msgr2=0x7f248c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.503+0000 7f249364a640 1 -- 172.21.15.22:0/202318666 shutdown_connections 2024-03-09T00:07:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.504+0000 7f249364a640 1 -- 172.21.15.22:0/202318666 wait complete. 2024-03-09T00:07:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.505+0000 7f249364a640 1 Processor -- start 2024-03-09T00:07:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.505+0000 7f249364a640 1 -- start start 2024-03-09T00:07:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.505+0000 7f249364a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c1078c0 0x7f248c19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.505+0000 7f249364a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f248c108280 0x7f248c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.506+0000 7f249364a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c1a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.506+0000 7f249364a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c117dd0 con 0x7f248c1078c0 2024-03-09T00:07:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.506+0000 7f249364a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c117c50 con 0x7f248c108280 2024-03-09T00:07:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.506+0000 7f249364a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c117f50 con 0x7f248c10d8d0 2024-03-09T00:07:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.506+0000 7f2491be7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c1a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.506+0000 7f2491be7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c1a2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58934/0 (socket says 172.21.15.22:58934) 2024-03-09T00:07:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.506+0000 7f2491be7640 1 -- 172.21.15.22:0/639496312 learned_addr learned my addr 172.21.15.22:0/639496312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.507+0000 7f24913e6640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c1078c0 0x7f248c19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.508+0000 7f2490be5640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f248c108280 0x7f248c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.508+0000 7f2491be7640 1 -- 172.21.15.22:0/639496312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f248c108280 msgr2=0x7f248c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.508+0000 7f2491be7640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f248c108280 0x7f248c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.508+0000 7f2491be7640 1 -- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c1078c0 msgr2=0x7f248c19e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.508+0000 7f2491be7640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c1078c0 0x7f248c19e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.508+0000 7f2491be7640 1 -- 172.21.15.22:0/639496312 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c1a31b0 con 0x7f248c10d8d0 2024-03-09T00:07:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.509+0000 7f2491be7640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c1a2a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f247c009fd0 tx=0x7f247c00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.509+0000 7f24827fc640 1 -- 172.21.15.22:0/639496312 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c011830 con 0x7f248c10d8d0 2024-03-09T00:07:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.509+0000 7f24827fc640 1 -- 172.21.15.22:0/639496312 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f247c016070 con 0x7f248c10d8d0 2024-03-09T00:07:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.509+0000 7f24827fc640 1 -- 172.21.15.22:0/639496312 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00f2e0 con 0x7f248c10d8d0 2024-03-09T00:07:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.509+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c1a34a0 con 0x7f248c10d8d0 2024-03-09T00:07:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.509+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c1aad30 con 0x7f248c10d8d0 2024-03-09T00:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.515+0000 7f2496220740 1 Processor -- start 2024-03-09T00:07:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.515+0000 7f24827fc640 1 -- 172.21.15.22:0/639496312 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f247c00f480 con 0x7f248c10d8d0 2024-03-09T00:07:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.515+0000 7f2496220740 1 -- start start 2024-03-09T00:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.515+0000 7f2496220740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e4d40 0x56338b1e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2496220740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f24827fc640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24780a2ae0 0x7f24780a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2490be5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e4d40 0x56338b1e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2490be5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e4d40 0x56338b1e3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58946/0 (socket says 172.21.15.22:58946) 2024-03-09T00:07:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2490be5640 1 -- 172.21.15.22:0/1538324002 learned_addr learned my addr 172.21.15.22:0/1538324002 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2496220740 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f24827fc640 1 -- 172.21.15.22:0/639496312 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f247c0ce810 con 0x7f248c10d8d0 2024-03-09T00:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338b1485f0 con 0x56338b1e3fd0 2024-03-09T00:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338b0f1930 con 0x56338b1ed550 2024-03-09T00:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f2491be7640 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.516+0000 7f24913e6640 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.517+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338b144690 con 0x56338b1e4d40 2024-03-09T00:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.520+0000 7f2491be7640 1 -- 172.21.15.22:0/1538324002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e4d40 msgr2=0x56338b1e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.520+0000 7f2491be7640 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e4d40 0x56338b1e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.520+0000 7f2491be7640 1 -- 172.21.15.22:0/1538324002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 msgr2=0x56338b1e4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.520+0000 7f2491be7640 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1e4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.520+0000 7f2491be7640 1 -- 172.21.15.22:0/1538324002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338ae5d210 con 0x56338b1ed550 2024-03-09T00:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.520+0000 7f2491be7640 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1ef940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f247c0cbc20 tx=0x7f247c0119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e551d0a5277254d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.521+0000 7f2490be5640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24780a2ae0 0x7f24780a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.524+0000 7f2490be5640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24780a2ae0 0x7f24780a4fa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2484009fd0 tx=0x7f2484004a80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.529+0000 7f24777fe640 1 -- 172.21.15.22:0/1538324002 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0a0070 con 0x56338b1ed550 2024-03-09T00:07:09.524 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.529+0000 7f24777fe640 1 -- 172.21.15.22:0/1538324002 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f247c0cbd90 con 0x56338b1ed550 2024-03-09T00:07:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.530+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 msgr2=0x56338b1ef940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.530+0000 7f2496220740 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1ef940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f247c0cbc20 tx=0x7f247c0119d0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.532+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 shutdown_connections 2024-03-09T00:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.532+0000 7f2496220740 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1ef940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.532+0000 7f2496220740 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.532+0000 7f2496220740 1 --2- 172.21.15.22:0/1538324002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e4d40 0x56338b1e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.532+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 >> 172.21.15.22:0/1538324002 conn(0x56338b1ab5b0 msgr2=0x56338b1decc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.534+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 shutdown_connections 2024-03-09T00:07:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.534+0000 7f2496220740 1 -- 172.21.15.22:0/1538324002 wait complete. 2024-03-09T00:07:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.535+0000 7f2496220740 1 -- start start 2024-03-09T00:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.535+0000 7f2496220740 1 Processor -- start 2024-03-09T00:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2496220740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2496220740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e4d40 0x56338b1eaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2496220740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2490be5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e4d40 0x56338b1eaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2490be5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e4d40 0x56338b1eaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:48574/0 (socket says 172.21.15.22:48574) 2024-03-09T00:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2490be5640 1 -- 172.21.15.22:0/4090412990 learned_addr learned my addr 172.21.15.22:0/4090412990 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f24913e6640 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338b1f02d0 con 0x56338b1e3fd0 2024-03-09T00:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338b104d90 con 0x56338b1e4d40 2024-03-09T00:07:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338b1f0450 con 0x56338b1ed550 2024-03-09T00:07:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.536+0000 7f2491be7640 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2496220740 10 client.0 _ll_register_callbacks cb 0x56338ae5ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2490be5640 1 -- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 msgr2=0x56338b1eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2490be5640 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1eaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2490be5640 1 -- 172.21.15.22:0/4090412990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 msgr2=0x56338b1ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2490be5640 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2490be5640 1 -- 172.21.15.22:0/4090412990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56338af29620 con 0x56338b1e4d40 2024-03-09T00:07:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2490be5640 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e4d40 0x56338b1eaab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2484007050 tx=0x7f2484004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=234bc17277e42a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.537+0000 7f2474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.538+0000 7f2474ff9640 1 -- 172.21.15.22:0/4090412990 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248402a060 con 0x56338b1e4d40 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.538+0000 7f2474ff9640 1 -- 172.21.15.22:0/4090412990 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f248401a610 con 0x56338b1e4d40 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.538+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56338b155b30 con 0x56338b1e4d40 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.538+0000 7f2474ff9640 1 -- 172.21.15.22:0/4090412990 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2484031050 con 0x56338b1e4d40 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.538+0000 7f2496220740 10 client.28892 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.538+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56338b1fc6c0 con 0x56338b1e4d40 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.539+0000 7f2474ff9640 1 -- 172.21.15.22:0/4090412990 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2484016040 con 0x56338b1e4d40 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.539+0000 7f2474ff9640 1 -- 172.21.15.22:0/4090412990 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2484025930 con 0x56338b1e4d40 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.539+0000 7f2496220740 10 client.28892 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.540+0000 7f2496220740 10 client.28892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.540+0000 7f2496220740 20 client.28892 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.540+0000 7f2496220740 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b1fdb80 0x56338b1fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.540+0000 7f2496220740 4 client.28892 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.540+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56338b155b30 con 0x56338b1fdb80 2024-03-09T00:07:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.540+0000 7f24913e6640 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b1fdb80 0x56338b1fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.541+0000 7f24913e6640 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b1fdb80 0x56338b1fff70 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31b5b8c25c4d58f server_cookie=6850b09a71b3cf68 in_seq=0 out_seq=0 2024-03-09T00:07:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.541+0000 7f2474ff9640 10 client.28892 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.542+0000 7f2474ff9640 1 -- 172.21.15.22:0/4090412990 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56338b155b30 con 0x56338b1fdb80 2024-03-09T00:07:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.542+0000 7f2474ff9640 10 client.28892 handle_command_reply: tid=0 2024-03-09T00:07:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.542+0000 7f2496220740 1 client.28892 shutdown 2024-03-09T00:07:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e4d40 msgr2=0x56338b1eaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e4d40 0x56338b1eaab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2484007050 tx=0x7f2484004af0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 shutdown_connections 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b1fdb80 0x56338b1fff70 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e4d40 0x56338b1eaab0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 --2- 172.21.15.22:0/4090412990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e3fd0 0x56338b1ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.543+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 >> 172.21.15.22:0/4090412990 conn(0x56338b1ab5b0 msgr2=0x56338b1e0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.544+0000 7f2474ff9640 0 client.28892 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.544+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 shutdown_connections 2024-03-09T00:07:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.544+0000 7f2496220740 1 -- 172.21.15.22:0/4090412990 wait complete. 2024-03-09T00:07:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.544+0000 7f2496220740 20 client.28892 trim_cache size 0 max 16384 2024-03-09T00:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.569+0000 7f2496220740 1 Processor -- start 2024-03-09T00:07:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.570+0000 7f2496220740 1 -- start start 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.570+0000 7f2496220740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e3fd0 0x56338b1e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.570+0000 7f2496220740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1e9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.570+0000 7f2496220740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f24913e6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e3fd0 0x56338b1e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f24913e6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e3fd0 0x56338b1e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:58970/0 (socket says 172.21.15.22:58970) 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f24913e6640 1 -- 172.21.15.22:0/2426686752 learned_addr learned my addr 172.21.15.22:0/2426686752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f2491be7640 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24700124b0 con 0x56338b1e4d40 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2470010050 con 0x56338b1ed550 2024-03-09T00:07:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247000e110 con 0x56338b1e3fd0 2024-03-09T00:07:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.571+0000 7f2490be5640 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.572+0000 7f2491be7640 1 -- 172.21.15.22:0/2426686752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e3fd0 msgr2=0x56338b1e8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.572+0000 7f2491be7640 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e3fd0 0x56338b1e8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.572+0000 7f2491be7640 1 -- 172.21.15.22:0/2426686752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 msgr2=0x56338b1e9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.572+0000 7f2491be7640 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1e9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.572+0000 7f2491be7640 1 -- 172.21.15.22:0/2426686752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338b1fa050 con 0x56338b1ed550 2024-03-09T00:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.572+0000 7f2491be7640 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1e77f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f247c0cc110 tx=0x7f247c00b3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a76f296e4a1217 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.573+0000 7f2476ffd640 1 -- 172.21.15.22:0/2426686752 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0ce360 con 0x56338b1ed550 2024-03-09T00:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.573+0000 7f2476ffd640 1 -- 172.21.15.22:0/2426686752 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f247c099ee0 con 0x56338b1ed550 2024-03-09T00:07:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.573+0000 7f2476ffd640 1 -- 172.21.15.22:0/2426686752 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c086180 con 0x56338b1ed550 2024-03-09T00:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.574+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 msgr2=0x56338b1e77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.574+0000 7f2496220740 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1e77f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f247c0cc110 tx=0x7f247c00b3d0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.574+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 shutdown_connections 2024-03-09T00:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.574+0000 7f2496220740 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1ed550 0x56338b1e77f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.574+0000 7f2496220740 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1e9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.574+0000 7f2496220740 1 --2- 172.21.15.22:0/2426686752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1e3fd0 0x56338b1e8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.574+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 >> 172.21.15.22:0/2426686752 conn(0x56338b1ab5b0 msgr2=0x56338b1de7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.577+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 shutdown_connections 2024-03-09T00:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.577+0000 7f2496220740 1 -- 172.21.15.22:0/2426686752 wait complete. 2024-03-09T00:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.578+0000 7f2496220740 1 -- start start 2024-03-09T00:07:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.578+0000 7f2496220740 1 Processor -- start 2024-03-09T00:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.578+0000 7f2496220740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e3fd0 0x56338b1f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.578+0000 7f2496220740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.578+0000 7f2496220740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.578+0000 7f2496220740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247000e710 con 0x56338b1e4d40 2024-03-09T00:07:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.578+0000 7f2496220740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247000f9f0 con 0x56338b1e3fd0 2024-03-09T00:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.579+0000 7f2496220740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247000ed30 con 0x56338b1ed550 2024-03-09T00:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.579+0000 7f2496220740 10 client.0 _ll_register_callbacks cb 0x7f247c097820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.579+0000 7f2490be5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.579+0000 7f2490be5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57662/0 (socket says 172.21.15.22:57662) 2024-03-09T00:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.579+0000 7f2490be5640 1 -- 172.21.15.22:0/3291886665 learned_addr learned my addr 172.21.15.22:0/3291886665 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.579+0000 7f24913e6640 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e3fd0 0x56338b1f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.579+0000 7f2491be7640 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f24913e6640 1 -- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 msgr2=0x56338b1f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f24913e6640 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f24913e6640 1 -- 172.21.15.22:0/3291886665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 msgr2=0x56338b1f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f24913e6640 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f24913e6640 1 -- 172.21.15.22:0/3291886665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56338b265300 con 0x56338b1e3fd0 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f24913e6640 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e3fd0 0x56338b1f32e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f24880096c0 tx=0x7f248802c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7c962828dae080 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f2474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f2474ff9640 1 -- 172.21.15.22:0/3291886665 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248802ccf0 con 0x56338b1e3fd0 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f2474ff9640 1 -- 172.21.15.22:0/3291886665 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f2488048360 con 0x56338b1e3fd0 2024-03-09T00:07:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.580+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f247c099ee0 con 0x56338b1e3fd0 2024-03-09T00:07:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.583+0000 7f2474ff9640 1 -- 172.21.15.22:0/3291886665 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2488048560 con 0x56338b1e3fd0 2024-03-09T00:07:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.584+0000 7f2496220740 10 client.28898 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.584+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56338b2665d0 con 0x56338b1e3fd0 2024-03-09T00:07:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.585+0000 7f2474ff9640 1 -- 172.21.15.22:0/3291886665 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f247c099ee0 con 0x56338b1e3fd0 2024-03-09T00:07:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.585+0000 7f2474ff9640 1 -- 172.21.15.22:0/3291886665 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2488048040 con 0x56338b1e3fd0 2024-03-09T00:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.586+0000 7f2496220740 10 client.28898 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.586+0000 7f2496220740 10 client.28898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.586+0000 7f2496220740 20 client.28898 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.586+0000 7f2496220740 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b266cf0 0x56338b269110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.586+0000 7f2496220740 4 client.28898 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.586+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56338b14fe80 con 0x56338b266cf0 2024-03-09T00:07:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.587+0000 7f2490be5640 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b266cf0 0x56338b269110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.588+0000 7f2490be5640 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b266cf0 0x56338b269110 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaf44a886cc42181 server_cookie=20da4cfdb4809ec6 in_seq=0 out_seq=0 2024-03-09T00:07:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.589+0000 7f2474ff9640 10 client.28898 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:09.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1241377 9.25 MB/sec execute 557 sec latency 222.083 ms 2024-03-09T00:07:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2474ff9640 1 -- 172.21.15.22:0/3291886665 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56338b14fe80 con 0x56338b266cf0 2024-03-09T00:07:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2474ff9640 10 client.28898 handle_command_reply: tid=0 2024-03-09T00:07:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 client.28898 shutdown 2024-03-09T00:07:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e3fd0 msgr2=0x56338b1f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e3fd0 0x56338b1f32e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f24880096c0 tx=0x7f248802c6b0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 shutdown_connections 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56338b266cf0 0x56338b269110 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56338b1ed550 0x56338b1f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56338b1e4d40 0x56338b1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.954+0000 7f2496220740 1 --2- 172.21.15.22:0/3291886665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56338b1e3fd0 0x56338b1f32e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.955+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 >> 172.21.15.22:0/3291886665 conn(0x56338b1ab5b0 msgr2=0x56338b1e8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.955+0000 7f2474ff9640 0 client.28898 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.955+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 shutdown_connections 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.955+0000 7f2496220740 1 -- 172.21.15.22:0/3291886665 wait complete. 2024-03-09T00:07:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.955+0000 7f2496220740 20 client.28898 trim_cache size 0 max 16384 2024-03-09T00:07:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:07:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:07:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:07:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:07:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24780a2ae0 msgr2=0x7f24780a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24780a2ae0 0x7f24780a4fa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2484009fd0 tx=0x7f2484004a80 comp rx=0 tx=0).stop 2024-03-09T00:07:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 msgr2=0x7f248c1a2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c1a2a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f247c009fd0 tx=0x7f247c00b8e0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 shutdown_connections 2024-03-09T00:07:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f24780a2ae0 0x7f24780a4fa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f248c10d8d0 0x7f248c1a2a80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f248c108280 0x7f248c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 --2- 172.21.15.22:0/639496312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c1078c0 0x7f248c19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 >> 172.21.15.22:0/639496312 conn(0x7f248c075860 msgr2=0x7f248c075d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 shutdown_connections 2024-03-09T00:07:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:09.956+0000 7f249364a640 1 -- 172.21.15.22:0/639496312 wait complete. 2024-03-09T00:07:10.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:10 smithi022 ceph-mon[31894]: pgmap v398: 129 pgs: 129 active+clean; 90 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-03-09T00:07:10.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:10 smithi062 ceph-mon[36129]: pgmap v398: 129 pgs: 129 active+clean; 90 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-03-09T00:07:10.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:10 smithi192 ceph-mon[35995]: pgmap v398: 129 pgs: 129 active+clean; 90 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-03-09T00:07:10.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1242199 9.25 MB/sec execute 558 sec latency 122.474 ms 2024-03-09T00:07:11.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1243147 9.24 MB/sec execute 559 sec latency 51.875 ms 2024-03-09T00:07:12.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:12 smithi022 ceph-mon[31894]: pgmap v399: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 96 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-03-09T00:07:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:12 smithi062 ceph-mon[36129]: pgmap v399: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 96 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-03-09T00:07:12.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:12 smithi192 ceph-mon[35995]: pgmap v399: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 96 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-03-09T00:07:12.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1244277 9.23 MB/sec execute 560 sec latency 21.116 ms 2024-03-09T00:07:13.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1246579 9.24 MB/sec execute 561 sec latency 41.518 ms 2024-03-09T00:07:14.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:14 smithi022 ceph-mon[31894]: pgmap v400: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 96 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-03-09T00:07:14.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:14 smithi062 ceph-mon[36129]: pgmap v400: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 96 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-03-09T00:07:14.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:14 smithi192 ceph-mon[35995]: pgmap v400: 129 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 94 active+clean; 96 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-03-09T00:07:14.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1247447 9.23 MB/sec execute 562 sec latency 111.563 ms 2024-03-09T00:07:15.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1249467 9.24 MB/sec execute 563 sec latency 32.133 ms 2024-03-09T00:07:16.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:16 smithi022 ceph-mon[31894]: pgmap v401: 129 pgs: 129 active+clean; 90 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-03-09T00:07:16.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:16 smithi062 ceph-mon[36129]: pgmap v401: 129 pgs: 129 active+clean; 90 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-03-09T00:07:16.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:16 smithi192 ceph-mon[35995]: pgmap v401: 129 pgs: 129 active+clean; 90 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-03-09T00:07:16.734 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1250591 9.23 MB/sec execute 564 sec latency 16.912 ms 2024-03-09T00:07:17.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1252583 9.23 MB/sec execute 565 sec latency 17.763 ms 2024-03-09T00:07:18.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:18 smithi022 ceph-mon[31894]: pgmap v402: 129 pgs: 129 active+clean; 90 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-03-09T00:07:18.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:18 smithi062 ceph-mon[36129]: pgmap v402: 129 pgs: 129 active+clean; 90 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-03-09T00:07:18.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:18 smithi192 ceph-mon[35995]: pgmap v402: 129 pgs: 129 active+clean; 90 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 159 op/s 2024-03-09T00:07:18.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1254102 9.23 MB/sec execute 566 sec latency 57.321 ms 2024-03-09T00:07:19.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1255334 9.23 MB/sec execute 567 sec latency 152.802 ms 2024-03-09T00:07:20.095 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:19 smithi062 ceph-mon[36129]: osdmap e112: 12 total, 12 up, 12 in 2024-03-09T00:07:20.095 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:19 smithi062 ceph-mon[36129]: pgmap v404: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-03-09T00:07:20.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:19 smithi022 ceph-mon[31894]: osdmap e112: 12 total, 12 up, 12 in 2024-03-09T00:07:20.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:19 smithi022 ceph-mon[31894]: pgmap v404: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-03-09T00:07:20.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:19 smithi192 ceph-mon[35995]: osdmap e112: 12 total, 12 up, 12 in 2024-03-09T00:07:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:19 smithi192 ceph-mon[35995]: pgmap v404: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-03-09T00:07:20.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1257315 9.23 MB/sec execute 568 sec latency 37.607 ms 2024-03-09T00:07:21.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1258641 9.23 MB/sec execute 569 sec latency 24.957 ms 2024-03-09T00:07:22.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:22 smithi192 ceph-mon[35995]: pgmap v405: 129 pgs: 129 active+clean; 101 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-09T00:07:22.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:22 smithi022 ceph-mon[31894]: pgmap v405: 129 pgs: 129 active+clean; 101 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-09T00:07:22.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:22 smithi062 ceph-mon[36129]: pgmap v405: 129 pgs: 129 active+clean; 101 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-09T00:07:22.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1260870 9.23 MB/sec execute 570 sec latency 21.740 ms 2024-03-09T00:07:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.053+0000 7fdedb19e640 1 -- 172.21.15.22:0/365432208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4100440 msgr2=0x7fded41008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.053+0000 7fdedb19e640 1 --2- 172.21.15.22:0/365432208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4100440 0x7fded41008c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009900 tx=0x7fdec402bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.053+0000 7fdedb19e640 1 -- 172.21.15.22:0/365432208 shutdown_connections 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.053+0000 7fdedb19e640 1 --2- 172.21.15.22:0/365432208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded4100e00 0x7fded410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.053+0000 7fdedb19e640 1 --2- 172.21.15.22:0/365432208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4100440 0x7fded41008c0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.053+0000 7fdedb19e640 1 --2- 172.21.15.22:0/365432208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4105190 0x7fded4105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.053+0000 7fdedb19e640 1 -- 172.21.15.22:0/365432208 >> 172.21.15.22:0/365432208 conn(0x7fded4077040 msgr2=0x7fded4077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 -- 172.21.15.22:0/365432208 shutdown_connections 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 -- 172.21.15.22:0/365432208 wait complete. 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 Processor -- start 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 -- start start 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded4100440 0x7fded419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4100e00 0x7fded419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4105190 0x7fded4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded4110650 con 0x7fded4105190 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41104d0 con 0x7fded4100e00 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fdedb19e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41107d0 con 0x7fded4100440 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fded973b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4105190 0x7fded4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fded973b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4105190 0x7fded4197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48434/0 (socket says 172.21.15.22:48434) 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fded973b640 1 -- 172.21.15.22:0/2129477261 learned_addr learned my addr 172.21.15.22:0/2129477261 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.054+0000 7fded8f3a640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded4100440 0x7fded419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fdecbfff640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4100e00 0x7fded419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fded973b640 1 -- 172.21.15.22:0/2129477261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded4100440 msgr2=0x7fded419d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fded973b640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded4100440 0x7fded419d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fded973b640 1 -- 172.21.15.22:0/2129477261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4100e00 msgr2=0x7fded419dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fded973b640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4100e00 0x7fded419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fded973b640 1 -- 172.21.15.22:0/2129477261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded41981d0 con 0x7fded4105190 2024-03-09T00:07:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fded973b640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4105190 0x7fded4197a50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00ea60 tx=0x7fdecc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fdec9ffb640 1 -- 172.21.15.22:0/2129477261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00ce40 con 0x7fded4105190 2024-03-09T00:07:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fdec9ffb640 1 -- 172.21.15.22:0/2129477261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdecc005020 con 0x7fded4105190 2024-03-09T00:07:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded41984c0 con 0x7fded4105190 2024-03-09T00:07:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fdec9ffb640 1 -- 172.21.15.22:0/2129477261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc010690 con 0x7fded4105190 2024-03-09T00:07:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.055+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded41a4710 con 0x7fded4105190 2024-03-09T00:07:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.058+0000 7fdec9ffb640 1 -- 172.21.15.22:0/2129477261 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdecc005520 con 0x7fded4105190 2024-03-09T00:07:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.058+0000 7fdeddd7f740 1 Processor -- start 2024-03-09T00:07:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.058+0000 7fdeddd7f740 1 -- start start 2024-03-09T00:07:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.058+0000 7fdec9ffb640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdebc0a2a90 0x7fdebc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.058+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a364a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a365450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a370940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668a2c95f0 con 0x55668a364fd0 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668a272930 con 0x55668a36e550 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668a2c5690 con 0x55668a365d40 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdecbfff640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdebc0a2a90 0x7fdebc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdec9ffb640 1 -- 172.21.15.22:0/2129477261 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdecc0cf710 con 0x7fded4105190 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded973b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a370940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded8f3a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a364a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdecbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a365450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded8f3a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a364a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:42604/0 (socket says 172.21.15.22:42604) 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded8f3a640 1 -- 172.21.15.22:0/791032345 learned_addr learned my addr 172.21.15.22:0/791032345 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded973b640 1 -- 172.21.15.22:0/791032345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 msgr2=0x55668a364a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded973b640 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a364a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded973b640 1 -- 172.21.15.22:0/791032345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 msgr2=0x55668a365450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded973b640 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a365450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded973b640 1 -- 172.21.15.22:0/791032345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556689fde210 con 0x55668a36e550 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded8f3a640 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a364a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdecbfff640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdebc0a2a90 0x7fdebc0a4f50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdec4036040 tx=0x7fdec402c650 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fded973b640 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a370940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cca60 tx=0x7fdecc09b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4394eabcbd63c4f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdebaffd640 1 -- 172.21.15.22:0/791032345 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0a4110 con 0x55668a36e550 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdebaffd640 1 -- 172.21.15.22:0/791032345 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdecc099c70 con 0x55668a36e550 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.059+0000 7fdebaffd640 1 -- 172.21.15.22:0/791032345 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc08aff0 con 0x55668a36e550 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 -- 172.21.15.22:0/791032345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 msgr2=0x55668a370940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a370940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cca60 tx=0x7fdecc09b870 comp rx=0 tx=0).stop 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 -- 172.21.15.22:0/791032345 shutdown_connections 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a370940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a365450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/791032345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a364a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 -- 172.21.15.22:0/791032345 >> 172.21.15.22:0/791032345 conn(0x55668a32c5b0 msgr2=0x55668a35fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 -- 172.21.15.22:0/791032345 shutdown_connections 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.060+0000 7fdeddd7f740 1 -- 172.21.15.22:0/791032345 wait complete. 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 -- start start 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 Processor -- start 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a364fd0 0x55668a36b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a36bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668a371280 con 0x55668a36e550 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668a285d90 con 0x55668a364fd0 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668a371400 con 0x55668a365d40 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdeddd7f740 10 client.0 _ll_register_callbacks cb 0x556689fdda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fdecbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a36bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48460/0 (socket says 172.21.15.22:48460) 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 -- 172.21.15.22:0/2327411596 learned_addr learned my addr 172.21.15.22:0/2327411596 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded8f3a640 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a364fd0 0x55668a36b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 -- 172.21.15.22:0/2327411596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 msgr2=0x55668a36bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a36bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 -- 172.21.15.22:0/2327411596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a364fd0 msgr2=0x55668a36b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a364fd0 0x55668a36b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.061+0000 7fded973b640 1 -- 172.21.15.22:0/2327411596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55668a0aa620 con 0x55668a36e550 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fded973b640 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36c000 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cd060 tx=0x7fdecc005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe6f63d28c87ee8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeb3fff640 1 -- 172.21.15.22:0/2327411596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc06c3f0 con 0x55668a36e550 2024-03-09T00:07:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeb3fff640 1 -- 172.21.15.22:0/2327411596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdecc06c590 con 0x55668a36e550 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdecc099c70 con 0x55668a36e550 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeb3fff640 1 -- 172.21.15.22:0/2327411596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdecc086110 con 0x55668a36e550 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 10 client.19626 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55668a37d560 con 0x55668a36e550 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeb3fff640 1 -- 172.21.15.22:0/2327411596 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdecc099c70 con 0x55668a36e550 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeb3fff640 1 -- 172.21.15.22:0/2327411596 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdecc06b070 con 0x55668a36e550 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 10 client.19626 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 10 client.19626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 20 client.19626 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a37e8b0 0x55668a380ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 4 client.19626 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.062+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55668a2d6b30 con 0x55668a37e8b0 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.063+0000 7fded8f3a640 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a37e8b0 0x55668a380ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.063+0000 7fded8f3a640 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a37e8b0 0x55668a380ca0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc445682a43077e server_cookie=d7869fe9b0c26535 in_seq=0 out_seq=0 2024-03-09T00:07:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.063+0000 7fdeb3fff640 10 client.19626 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.063+0000 7fdeb3fff640 1 -- 172.21.15.22:0/2327411596 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55668a2d6b30 con 0x55668a37e8b0 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeb3fff640 10 client.19626 handle_command_reply: tid=0 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 client.19626 shutdown 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 msgr2=0x55668a36c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36c000 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cd060 tx=0x7fdecc005400 comp rx=0 tx=0).stop 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 shutdown_connections 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a37e8b0 0x55668a380ca0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36c000 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a36bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeb3fff640 0 client.19626 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/2327411596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a364fd0 0x55668a36b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 >> 172.21.15.22:0/2327411596 conn(0x55668a32c5b0 msgr2=0x55668a361a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 shutdown_connections 2024-03-09T00:07:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 1 -- 172.21.15.22:0/2327411596 wait complete. 2024-03-09T00:07:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.064+0000 7fdeddd7f740 20 client.19626 trim_cache size 0 max 16384 2024-03-09T00:07:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 Processor -- start 2024-03-09T00:07:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 -- start start 2024-03-09T00:07:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a364fd0 0x55668a36dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a365d40 0x55668a36a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41b0c60 con 0x55668a36e550 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41ae800 con 0x55668a365d40 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41ac8c0 con 0x55668a364fd0 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fded973b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fded973b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48466/0 (socket says 172.21.15.22:48466) 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fded973b640 1 -- 172.21.15.22:0/1211992459 learned_addr learned my addr 172.21.15.22:0/1211992459 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fded8f3a640 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a364fd0 0x55668a36dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.079+0000 7fdecbfff640 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a365d40 0x55668a36a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fded973b640 1 -- 172.21.15.22:0/1211992459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a364fd0 msgr2=0x55668a36dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fded973b640 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a364fd0 0x55668a36dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fded973b640 1 -- 172.21.15.22:0/1211992459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a365d40 msgr2=0x55668a36a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fded973b640 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a365d40 0x55668a36a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fded973b640 1 -- 172.21.15.22:0/1211992459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55668a364e00 con 0x55668a36e550 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fdecbfff640 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a365d40 0x55668a36a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fded973b640 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36bc50 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0a3be0 tx=0x7fdecc09b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbfac6ebaa5139d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fdeba7fc640 1 -- 172.21.15.22:0/1211992459 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc08aeb0 con 0x55668a36e550 2024-03-09T00:07:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fdeba7fc640 1 -- 172.21.15.22:0/1211992459 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdecc09fac0 con 0x55668a36e550 2024-03-09T00:07:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.080+0000 7fdeba7fc640 1 -- 172.21.15.22:0/1211992459 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc06c3f0 con 0x55668a36e550 2024-03-09T00:07:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 -- 172.21.15.22:0/1211992459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 msgr2=0x55668a36bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36bc50 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0a3be0 tx=0x7fdecc09b870 comp rx=0 tx=0).stop 2024-03-09T00:07:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 -- 172.21.15.22:0/1211992459 shutdown_connections 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a36e550 0x55668a36bc50 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a365d40 0x55668a36a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/1211992459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a364fd0 0x55668a36dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 -- 172.21.15.22:0/1211992459 >> 172.21.15.22:0/1211992459 conn(0x55668a32c5b0 msgr2=0x55668a36b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 -- 172.21.15.22:0/1211992459 shutdown_connections 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 -- 172.21.15.22:0/1211992459 wait complete. 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.081+0000 7fdeddd7f740 1 -- start start 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 1 Processor -- start 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a3680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a368620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a368b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41acec0 con 0x55668a364fd0 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41ae1a0 con 0x55668a36e550 2024-03-09T00:07:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41ad4e0 con 0x55668a365d40 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdeddd7f740 10 client.0 _ll_register_callbacks cb 0x7fdecc097c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a3680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a3680e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48482/0 (socket says 172.21.15.22:48482) 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 -- 172.21.15.22:0/4084881316 learned_addr learned my addr 172.21.15.22:0/4084881316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded973b640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a368b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fdecbfff640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a368620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 -- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 msgr2=0x55668a368620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a368620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 -- 172.21.15.22:0/4084881316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 msgr2=0x55668a368b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a368b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded8f3a640 1 -- 172.21.15.22:0/4084881316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55668a3d4980 con 0x55668a364fd0 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.082+0000 7fded973b640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a368b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fded8f3a640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a3680e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x55668a371490 tx=0x7fded001f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ab00a72ec76404 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeb3fff640 1 -- 172.21.15.22:0/4084881316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0010d10 con 0x55668a364fd0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeb3fff640 1 -- 172.21.15.22:0/4084881316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fded0004030 con 0x55668a364fd0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdecc09fac0 con 0x55668a364fd0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeb3fff640 1 -- 172.21.15.22:0/4084881316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fded002e050 con 0x55668a364fd0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 10 client.19632 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55668a3d5ba0 con 0x55668a364fd0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeb3fff640 1 -- 172.21.15.22:0/4084881316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdecc09fac0 con 0x55668a364fd0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeb3fff640 1 -- 172.21.15.22:0/4084881316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fded0003c90 con 0x55668a364fd0 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 10 client.19632 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 10 client.19632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 20 client.19632 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a3d62d0 0x55668a3d86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 4 client.19632 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:07:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.083+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55668a2d0e80 con 0x55668a3d62d0 2024-03-09T00:07:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.084+0000 7fdecbfff640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a3d62d0 0x55668a3d86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.084+0000 7fdeb3fff640 10 client.19632 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.084+0000 7fdecbfff640 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a3d62d0 0x55668a3d86f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fc6d46d2e3f636 server_cookie=bb456308a3bf08e3 in_seq=0 out_seq=0 2024-03-09T00:07:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeb3fff640 1 -- 172.21.15.22:0/4084881316 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55668a2d0e80 con 0x55668a3d62d0 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeb3fff640 10 client.19632 handle_command_reply: tid=0 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 client.19632 shutdown 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 msgr2=0x55668a3680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a3680e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x55668a371490 tx=0x7fded001f770 comp rx=0 tx=0).stop 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 shutdown_connections 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55668a3d62d0 0x55668a3d86f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeb3fff640 0 client.19632 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55668a36e550 0x55668a368b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55668a365d40 0x55668a368620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 --2- 172.21.15.22:0/4084881316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55668a364fd0 0x55668a3680e0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 >> 172.21.15.22:0/4084881316 conn(0x55668a32c5b0 msgr2=0x55668a36a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 shutdown_connections 2024-03-09T00:07:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.521+0000 7fdeddd7f740 1 -- 172.21.15.22:0/4084881316 wait complete. 2024-03-09T00:07:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdeddd7f740 20 client.19632 trim_cache size 0 max 16384 2024-03-09T00:07:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:07:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:07:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:07:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdebc0a2a90 msgr2=0x7fdebc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdebc0a2a90 0x7fdebc0a4f50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdec4036040 tx=0x7fdec402c650 comp rx=0 tx=0).stop 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4105190 msgr2=0x7fded4197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4105190 0x7fded4197a50 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00ea60 tx=0x7fdecc00ef20 comp rx=0 tx=0).stop 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 shutdown_connections 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdebc0a2a90 0x7fdebc0a4f50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fded4105190 0x7fded4197a50 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4100e00 0x7fded419dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 --2- 172.21.15.22:0/2129477261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded4100440 0x7fded419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.522+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 >> 172.21.15.22:0/2129477261 conn(0x7fded4077040 msgr2=0x7fded4068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.523+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 shutdown_connections 2024-03-09T00:07:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:23.523+0000 7fdedb19e640 1 -- 172.21.15.22:0/2129477261 wait complete. 2024-03-09T00:07:23.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1261454 9.22 MB/sec execute 571 sec latency 198.777 ms 2024-03-09T00:07:24.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:24 smithi192 ceph-mon[35995]: pgmap v406: 129 pgs: 129 active+clean; 101 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-09T00:07:24.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:24 smithi022 ceph-mon[31894]: pgmap v406: 129 pgs: 129 active+clean; 101 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-09T00:07:24.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:24 smithi062 ceph-mon[36129]: pgmap v406: 129 pgs: 129 active+clean; 101 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-09T00:07:24.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1263172 9.22 MB/sec execute 572 sec latency 132.123 ms 2024-03-09T00:07:25.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1264577 9.22 MB/sec execute 573 sec latency 21.029 ms 2024-03-09T00:07:26.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:26 smithi192 ceph-mon[35995]: pgmap v407: 129 pgs: 129 active+clean; 88 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-03-09T00:07:26.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:26 smithi022 ceph-mon[31894]: pgmap v407: 129 pgs: 129 active+clean; 88 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-03-09T00:07:26.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:26 smithi062 ceph-mon[36129]: pgmap v407: 129 pgs: 129 active+clean; 88 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-03-09T00:07:26.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1266191 9.21 MB/sec execute 574 sec latency 14.764 ms 2024-03-09T00:07:27.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1268078 9.22 MB/sec execute 575 sec latency 29.205 ms 2024-03-09T00:07:28.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:28 smithi022 ceph-mon[31894]: pgmap v408: 129 pgs: 129 active+clean; 88 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-03-09T00:07:28.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:28 smithi062 ceph-mon[36129]: pgmap v408: 129 pgs: 129 active+clean; 88 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-03-09T00:07:28.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:28 smithi192 ceph-mon[35995]: pgmap v408: 129 pgs: 129 active+clean; 88 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-03-09T00:07:28.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1269532 9.21 MB/sec execute 576 sec latency 29.395 ms 2024-03-09T00:07:29.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:07:29.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:07:29.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 -- 172.21.15.22:0/1969698081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 msgr2=0x7f637c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 --2- 172.21.15.22:0/1969698081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1007b0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f6378009900 tx=0x7f637802bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:29.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 -- 172.21.15.22:0/1969698081 shutdown_connections 2024-03-09T00:07:29.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 --2- 172.21.15.22:0/1969698081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f637c1054a0 0x7f637c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 --2- 172.21.15.22:0/1969698081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c100d80 0x7f637c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 --2- 172.21.15.22:0/1969698081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1007b0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 -- 172.21.15.22:0/1969698081 >> 172.21.15.22:0/1969698081 conn(0x7f637c0772f0 msgr2=0x7f637c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 -- 172.21.15.22:0/1969698081 shutdown_connections 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.377+0000 7f638439e640 1 -- 172.21.15.22:0/1969698081 wait complete. 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 Processor -- start 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 -- start start 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c1054a0 0x7f637c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f637c199490 0x7f637c1a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c110560 con 0x7f637c1003d0 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1103e0 con 0x7f637c1054a0 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638439e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1106e0 con 0x7f637c199490 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48508/0 (socket says 172.21.15.22:48508) 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 -- 172.21.15.22:0/1689777205 learned_addr learned my addr 172.21.15.22:0/1689777205 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f6381939640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c1054a0 0x7f637c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 -- 172.21.15.22:0/1689777205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f637c199490 msgr2=0x7f637c1a3a40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f637c199490 0x7f637c1a3a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 -- 172.21.15.22:0/1689777205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c1054a0 msgr2=0x7f637c198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c1054a0 0x7f637c198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 -- 172.21.15.22:0/1689777205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c1a4210 con 0x7f637c1003d0 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.378+0000 7f638213a640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1989c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6378006010 tx=0x7f637802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.379+0000 7f63737fe640 1 -- 172.21.15.22:0/1689777205 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378041070 con 0x7f637c1003d0 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.379+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c1a44a0 con 0x7f637c1003d0 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.379+0000 7f63737fe640 1 -- 172.21.15.22:0/1689777205 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6378004450 con 0x7f637c1003d0 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.379+0000 7f63737fe640 1 -- 172.21.15.22:0/1689777205 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378003e00 con 0x7f637c1003d0 2024-03-09T00:07:29.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.379+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c1a49e0 con 0x7f637c1003d0 2024-03-09T00:07:29.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.381+0000 7f63737fe640 1 -- 172.21.15.22:0/1689777205 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6378044070 con 0x7f637c1003d0 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.381+0000 7f6386f80740 1 Processor -- start 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6386f80740 1 -- start start 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f63737fe640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63680a2970 0x7f63680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6386f80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee51760 0x560d2ee51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6386f80740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee52080 0x560d2ee5a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6386f80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee5d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6386f80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2edb4e00 con 0x560d2ee51760 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f63737fe640 1 -- 172.21.15.22:0/1689777205 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f63780f0c90 con 0x7f637c1003d0 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6386f80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2ed5db10 con 0x560d2ee5ac60 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6386f80740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2edb0ea0 con 0x560d2ee52080 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638293b640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63680a2970 0x7f63680a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee51760 0x560d2ee51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6381939640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee52080 0x560d2ee5a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee51760 0x560d2ee51b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48520/0 (socket says 172.21.15.22:48520) 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 -- 172.21.15.22:0/137588808 learned_addr learned my addr 172.21.15.22:0/137588808 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f6381939640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee52080 0x560d2ee5a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:42646/0 (socket says 172.21.15.22:42646) 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638293b640 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee5d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 -- 172.21.15.22:0/137588808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee52080 msgr2=0x560d2ee5a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee52080 0x560d2ee5a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 -- 172.21.15.22:0/137588808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 msgr2=0x560d2ee5d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee5d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 -- 172.21.15.22:0/137588808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d2eb937d0 con 0x560d2ee51760 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638293b640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63680a2970 0x7f63680a4e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6374005fd0 tx=0x7f63740129b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.382+0000 7f638213a640 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee51760 0x560d2ee51b40 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f63780bcf90 tx=0x7f63780bcf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c1eb26b166ba72 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6370bfe640 1 -- 172.21.15.22:0/137588808 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637804b040 con 0x560d2ee51760 2024-03-09T00:07:29.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6370bfe640 1 -- 172.21.15.22:0/137588808 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63780bd000 con 0x560d2ee51760 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6370bfe640 1 -- 172.21.15.22:0/137588808 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780ee260 con 0x560d2ee51760 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6386f80740 1 -- 172.21.15.22:0/137588808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee51760 msgr2=0x560d2ee51b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6386f80740 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee51760 0x560d2ee51b40 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f63780bcf90 tx=0x7f63780bcf00 comp rx=0 tx=0).stop 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6386f80740 1 -- 172.21.15.22:0/137588808 shutdown_connections 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6386f80740 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee5d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6386f80740 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee52080 0x560d2ee5a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6386f80740 1 --2- 172.21.15.22:0/137588808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee51760 0x560d2ee51b40 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.383+0000 7f6386f80740 1 -- 172.21.15.22:0/137588808 >> 172.21.15.22:0/137588808 conn(0x560d2ee4be90 msgr2=0x560d2ee4c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 -- 172.21.15.22:0/137588808 shutdown_connections 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 -- 172.21.15.22:0/137588808 wait complete. 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 -- start start 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 Processor -- start 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee52080 0x560d2ee57730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee5ac60 0x560d2ee57c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2ee61c20 con 0x560d2ee5ac60 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2ed710a0 con 0x560d2ee52080 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2ee61da0 con 0x560d2ee51760 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f638293b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee5ac60 0x560d2ee57c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6381939640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee52080 0x560d2ee57730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f638213a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f638293b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee5ac60 0x560d2ee57c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48530/0 (socket says 172.21.15.22:48530) 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f6386f80740 10 client.0 _ll_register_callbacks cb 0x560d2ed70a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f638293b640 1 -- 172.21.15.22:0/3911091621 learned_addr learned my addr 172.21.15.22:0/3911091621 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.384+0000 7f638213a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee5a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:42652/0 (socket says 172.21.15.22:42652) 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f638293b640 1 -- 172.21.15.22:0/3911091621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 msgr2=0x560d2ee5a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f638293b640 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee5a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f638293b640 1 -- 172.21.15.22:0/3911091621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee52080 msgr2=0x560d2ee57730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f638293b640 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee52080 0x560d2ee57730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f638293b640 1 -- 172.21.15.22:0/3911091621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d2eac93f0 con 0x560d2ee5ac60 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6381939640 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee52080 0x560d2ee57730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f638293b640 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee5ac60 0x560d2ee57c70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6374010a90 tx=0x7f6374004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d380922bedffd27 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6365ffb640 1 -- 172.21.15.22:0/3911091621 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637400db90 con 0x560d2ee5ac60 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6365ffb640 1 -- 172.21.15.22:0/3911091621 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f637400dd30 con 0x560d2ee5ac60 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63780bd000 con 0x560d2ee5ac60 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6365ffb640 1 -- 172.21.15.22:0/3911091621 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6374027400 con 0x560d2ee5ac60 2024-03-09T00:07:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6386f80740 10 client.19644 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560d2ea890f0 con 0x560d2ee5ac60 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6365ffb640 1 -- 172.21.15.22:0/3911091621 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63780bd000 con 0x560d2ee5ac60 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.385+0000 7f6365ffb640 1 -- 172.21.15.22:0/3911091621 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f637401f640 con 0x560d2ee5ac60 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.386+0000 7f6386f80740 10 client.19644 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.386+0000 7f6386f80740 10 client.19644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.386+0000 7f6386f80740 20 client.19644 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.386+0000 7f6386f80740 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2ee6e7b0 0x560d2ee70ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.386+0000 7f6386f80740 4 client.19644 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.386+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d2edc2340 con 0x560d2ee6e7b0 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.386+0000 7f638213a640 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2ee6e7b0 0x560d2ee70ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.387+0000 7f638213a640 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2ee6e7b0 0x560d2ee70ba0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f4dc9967adca29 server_cookie=4d314496868d43 in_seq=0 out_seq=0 2024-03-09T00:07:29.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.387+0000 7f6365ffb640 10 client.19644 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6365ffb640 1 -- 172.21.15.22:0/3911091621 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d2edc2340 con 0x560d2ee6e7b0 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6365ffb640 10 client.19644 handle_command_reply: tid=0 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 client.19644 shutdown 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee5ac60 msgr2=0x560d2ee57c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee5ac60 0x560d2ee57c70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6374010a90 tx=0x7f6374004a40 comp rx=0 tx=0).stop 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 shutdown_connections 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2ee6e7b0 0x560d2ee70ba0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee5ac60 0x560d2ee57c70 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee52080 0x560d2ee57730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6365ffb640 0 client.19644 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 --2- 172.21.15.22:0/3911091621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee5a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 >> 172.21.15.22:0/3911091621 conn(0x560d2ee4be90 msgr2=0x560d2ee4dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 shutdown_connections 2024-03-09T00:07:29.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 1 -- 172.21.15.22:0/3911091621 wait complete. 2024-03-09T00:07:29.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.388+0000 7f6386f80740 20 client.19644 trim_cache size 0 max 16384 2024-03-09T00:07:29.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.403+0000 7f6386f80740 1 Processor -- start 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.403+0000 7f6386f80740 1 -- start start 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.403+0000 7f6386f80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee51760 0x560d2ee59e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.403+0000 7f6386f80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6386f80740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee5ac60 0x560d2ee56000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6386f80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1aea40 con 0x560d2ee52080 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6386f80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1ac5e0 con 0x560d2ee51760 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6386f80740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1aa6a0 con 0x560d2ee5ac60 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f638213a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee51760 0x560d2ee59e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee5a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48544/0 (socket says 172.21.15.22:48544) 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 -- 172.21.15.22:0/757494614 learned_addr learned my addr 172.21.15.22:0/757494614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f638293b640 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee5ac60 0x560d2ee56000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 -- 172.21.15.22:0/757494614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee5ac60 msgr2=0x560d2ee56000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee5ac60 0x560d2ee56000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 -- 172.21.15.22:0/757494614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee51760 msgr2=0x560d2ee59e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee51760 0x560d2ee59e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 -- 172.21.15.22:0/757494614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d2ee51eb0 con 0x560d2ee52080 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f638213a640 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee51760 0x560d2ee59e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6381939640 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee5a3a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f636c012770 tx=0x7f636c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0369a22c0e872a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6367fff640 1 -- 172.21.15.22:0/757494614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c012db0 con 0x560d2ee52080 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6367fff640 1 -- 172.21.15.22:0/757494614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f636c00fda0 con 0x560d2ee52080 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.404+0000 7f6367fff640 1 -- 172.21.15.22:0/757494614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0047a0 con 0x560d2ee52080 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 -- 172.21.15.22:0/757494614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 msgr2=0x560d2ee5a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee5a3a0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f636c012770 tx=0x7f636c005f90 comp rx=0 tx=0).stop 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 -- 172.21.15.22:0/757494614 shutdown_connections 2024-03-09T00:07:29.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee5ac60 0x560d2ee56000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee5a3a0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 --2- 172.21.15.22:0/757494614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee51760 0x560d2ee59e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 -- 172.21.15.22:0/757494614 >> 172.21.15.22:0/757494614 conn(0x560d2ee4be90 msgr2=0x560d2ee75520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 -- 172.21.15.22:0/757494614 shutdown_connections 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 -- 172.21.15.22:0/757494614 wait complete. 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 -- start start 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.405+0000 7f6386f80740 1 Processor -- start 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee62d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1aaca0 con 0x560d2ee52080 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1abf80 con 0x560d2ee5ac60 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1ab2c0 con 0x560d2ee51760 2024-03-09T00:07:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 10 client.0 _ll_register_callbacks cb 0x7f6374010950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee62720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48546/0 (socket says 172.21.15.22:48546) 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 -- 172.21.15.22:0/775803569 learned_addr learned my addr 172.21.15.22:0/775803569 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f638213a640 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f638293b640 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee62d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 -- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 msgr2=0x560d2ee663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 -- 172.21.15.22:0/775803569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 msgr2=0x560d2ee62d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee62d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 -- 172.21.15.22:0/775803569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d2ee63560 con 0x560d2ee52080 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6381939640 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee62720 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f636c00ff90 tx=0x7f636c0096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c31f4ad56a739e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6365ffb640 1 -- 172.21.15.22:0/775803569 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00f8c0 con 0x560d2ee52080 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6365ffb640 1 -- 172.21.15.22:0/775803569 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f636c00fa60 con 0x560d2ee52080 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.406+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f636c00fda0 con 0x560d2ee52080 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6365ffb640 1 -- 172.21.15.22:0/775803569 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f636c004050 con 0x560d2ee52080 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 10 client.19650 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560d2eec5db0 con 0x560d2ee52080 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6365ffb640 1 -- 172.21.15.22:0/775803569 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f636c00fda0 con 0x560d2ee52080 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6365ffb640 1 -- 172.21.15.22:0/775803569 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f636c0035f0 con 0x560d2ee52080 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 10 client.19650 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 10 client.19650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 20 client.19650 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2eec6460 0x560d2eec8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 4 client.19650 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d2edbc690 con 0x560d2eec6460 2024-03-09T00:07:29.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.407+0000 7f638213a640 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2eec6460 0x560d2eec8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6365ffb640 10 client.19650 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f638213a640 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2eec6460 0x560d2eec8880 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c789e7313f2e4454 server_cookie=37167333e971a4d6 in_seq=0 out_seq=0 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6365ffb640 1 -- 172.21.15.22:0/775803569 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560d2edbc690 con 0x560d2eec6460 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6365ffb640 10 client.19650 handle_command_reply: tid=0 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 client.19650 shutdown 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 msgr2=0x560d2ee62720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee62720 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f636c00ff90 tx=0x7f636c0096b0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 shutdown_connections 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x560d2eec6460 0x560d2eec8880 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d2ee5ac60 0x560d2ee62d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d2ee52080 0x560d2ee62720 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 --2- 172.21.15.22:0/775803569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d2ee51760 0x560d2ee663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6365ffb640 0 client.19650 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 >> 172.21.15.22:0/775803569 conn(0x560d2ee4be90 msgr2=0x560d2ee55f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 shutdown_connections 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 1 -- 172.21.15.22:0/775803569 wait complete. 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.408+0000 7f6386f80740 20 client.19650 trim_cache size 0 max 16384 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:07:29.410 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:07:29.411 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:07:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63680a2970 msgr2=0x7f63680a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63680a2970 0x7f63680a4e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6374005fd0 tx=0x7f63740129b0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 msgr2=0x7f637c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1989c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6378006010 tx=0x7f637802cb30 comp rx=0 tx=0).stop 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 shutdown_connections 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63680a2970 0x7f63680a4e30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f637c199490 0x7f637c1a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f637c1054a0 0x7f637c198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 --2- 172.21.15.22:0/1689777205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637c1003d0 0x7f637c1989c0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 >> 172.21.15.22:0/1689777205 conn(0x7f637c0772f0 msgr2=0x7f637c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 shutdown_connections 2024-03-09T00:07:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.409+0000 7f638439e640 1 -- 172.21.15.22:0/1689777205 wait complete. 2024-03-09T00:07:29.414 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:07:29.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:07:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 -- 172.21.15.22:0/592358192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102e80 msgr2=0x7f11b010b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 --2- 172.21.15.22:0/592358192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102e80 0x7f11b010b360 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009900 tx=0x7f11a002bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 -- 172.21.15.22:0/592358192 shutdown_connections 2024-03-09T00:07:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 --2- 172.21.15.22:0/592358192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11b010b930 0x7f11b010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 --2- 172.21.15.22:0/592358192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102e80 0x7f11b010b360 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 --2- 172.21.15.22:0/592358192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11b0102560 0x7f11b0102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 -- 172.21.15.22:0/592358192 >> 172.21.15.22:0/592358192 conn(0x7f11b00772f0 msgr2=0x7f11b0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 -- 172.21.15.22:0/592358192 shutdown_connections 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 -- 172.21.15.22:0/592358192 wait complete. 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.533+0000 7f11b7b09640 1 Processor -- start 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b7b09640 1 -- start start 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b7b09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102560 0x7f11b019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b7b09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11b0102e80 0x7f11b019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b7b09640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11b010b930 0x7f11b0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b7b09640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0111710 con 0x7f11b0102560 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b7b09640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0111590 con 0x7f11b0102e80 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b7b09640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0111890 con 0x7f11b010b930 2024-03-09T00:07:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102560 0x7f11b019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102560 0x7f11b019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48560/0 (socket says 172.21.15.22:48560) 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 -- 172.21.15.22:0/2072266178 learned_addr learned my addr 172.21.15.22:0/2072266178 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b60a6640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11b010b930 0x7f11b0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b50a4640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11b0102e80 0x7f11b019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 -- 172.21.15.22:0/2072266178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11b010b930 msgr2=0x7f11b0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11b010b930 0x7f11b0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 -- 172.21.15.22:0/2072266178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11b0102e80 msgr2=0x7f11b019dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11b0102e80 0x7f11b019dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 -- 172.21.15.22:0/2072266178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b0198190 con 0x7f11b0102560 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11b58a5640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102560 0x7f11b019d810 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00e9f0 tx=0x7f11ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.534+0000 7f11a6ffd640 1 -- 172.21.15.22:0/2072266178 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00cd80 con 0x7f11b0102560 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.535+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b0198480 con 0x7f11b0102560 2024-03-09T00:07:29.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.535+0000 7f11a6ffd640 1 -- 172.21.15.22:0/2072266178 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11ac005500 con 0x7f11b0102560 2024-03-09T00:07:29.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.535+0000 7f11a6ffd640 1 -- 172.21.15.22:0/2072266178 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac010690 con 0x7f11b0102560 2024-03-09T00:07:29.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.535+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b01a46d0 con 0x7f11b0102560 2024-03-09T00:07:29.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.536+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b00680e0 con 0x7f11b0102560 2024-03-09T00:07:29.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.537+0000 7f11a6ffd640 1 -- 172.21.15.22:0/2072266178 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f11ac0046c0 con 0x7f11b0102560 2024-03-09T00:07:29.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.538+0000 7f11a6ffd640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f119c0a2970 0x7f119c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.538+0000 7f11a6ffd640 1 -- 172.21.15.22:0/2072266178 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f11ac0cf8a0 con 0x7f11b0102560 2024-03-09T00:07:29.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.538+0000 7f11b50a4640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f119c0a2970 0x7f119c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.539+0000 7f11b50a4640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f119c0a2970 0x7f119c0a4e30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11a0006010 tx=0x7f11a0005dd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.542+0000 7f11a6ffd640 1 -- 172.21.15.22:0/2072266178 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f11ac053260 con 0x7f11b0102560 2024-03-09T00:07:29.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1271462 9.22 MB/sec execute 577 sec latency 46.681 ms 2024-03-09T00:07:29.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.813+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11b0198ef0 con 0x7f11b0102560 2024-03-09T00:07:29.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.814+0000 7f11a6ffd640 1 -- 172.21.15.22:0/2072266178 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f11ac09c1a0 con 0x7f11b0102560 2024-03-09T00:07:29.810 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:07:29.811 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:02:00.108611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:07:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-03-09T00:07:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f119c0a2970 msgr2=0x7f119c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f119c0a2970 0x7f119c0a4e30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11a0006010 tx=0x7f11a0005dd0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102560 msgr2=0x7f11b019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102560 0x7f11b019d810 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00e9f0 tx=0x7f11ac00edf0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 shutdown_connections 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f119c0a2970 0x7f119c0a4e30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11b010b930 0x7f11b0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11b0102e80 0x7f11b019dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 --2- 172.21.15.22:0/2072266178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b0102560 0x7f11b019d810 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 >> 172.21.15.22:0/2072266178 conn(0x7f11b00772f0 msgr2=0x7f11b00fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 shutdown_connections 2024-03-09T00:07:29.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.818+0000 7f11b7b09640 1 -- 172.21.15.22:0/2072266178 wait complete. 2024-03-09T00:07:29.832 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:07:29.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.983+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/862637500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc100d90 msgr2=0x7ff1cc104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.983+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/862637500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc100d90 0x7ff1cc104dd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009900 tx=0x7ff1bc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.983+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/862637500 shutdown_connections 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.983+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/862637500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc1054b0 0x7ff1cc10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.983+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/862637500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc100d90 0x7ff1cc104dd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.983+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/862637500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1cc1003e0 0x7ff1cc1007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.983+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/862637500 >> 172.21.15.22:0/862637500 conn(0x7ff1cc0773a0 msgr2=0x7ff1cc0777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/862637500 shutdown_connections 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/862637500 wait complete. 2024-03-09T00:07:29.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 Processor -- start 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 -- start start 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc1003e0 0x7ff1cc19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1cc1054b0 0x7ff1cc19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc1978e0 0x7ff1cc197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc110540 con 0x7ff1cc1978e0 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1103c0 con 0x7ff1cc1054b0 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.984+0000 7ff1d1cd1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1106c0 con 0x7ff1cc1003e0 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc1978e0 0x7ff1cc197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc1978e0 0x7ff1cc197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48588/0 (socket says 172.21.15.22:48588) 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 -- 172.21.15.22:0/1318141221 learned_addr learned my addr 172.21.15.22:0/1318141221 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1caffd640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1cc1054b0 0x7ff1cc19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc1003e0 0x7ff1cc19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 -- 172.21.15.22:0/1318141221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc1003e0 msgr2=0x7ff1cc19d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc1003e0 0x7ff1cc19d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 -- 172.21.15.22:0/1318141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1cc1054b0 msgr2=0x7ff1cc19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1cc1054b0 0x7ff1cc19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 -- 172.21.15.22:0/1318141221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc198400 con 0x7ff1cc1978e0 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc1978e0 0x7ff1cc197d90 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400eb10 tx=0x7ff1c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1c8ff9640 1 -- 172.21.15.22:0/1318141221 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004b20 con 0x7ff1cc1978e0 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1c8ff9640 1 -- 172.21.15.22:0/1318141221 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1c4004cc0 con 0x7ff1cc1978e0 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.985+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc1986f0 con 0x7ff1cc1978e0 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.986+0000 7ff1c8ff9640 1 -- 172.21.15.22:0/1318141221 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4010760 con 0x7ff1cc1978e0 2024-03-09T00:07:29.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.986+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc1a4940 con 0x7ff1cc1978e0 2024-03-09T00:07:29.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.987+0000 7ff1c8ff9640 1 -- 172.21.15.22:0/1318141221 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff1c4005000 con 0x7ff1cc1978e0 2024-03-09T00:07:29.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1c8ff9640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1b40a2850 0x7ff1b40a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1c8ff9640 1 -- 172.21.15.22:0/1318141221 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff1c40cf720 con 0x7ff1cc1978e0 2024-03-09T00:07:29.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1b40a2850 0x7ff1b40a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 Processor -- start 2024-03-09T00:07:29.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 -- start start 2024-03-09T00:07:29.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a771580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a77a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a77a6a0 0x557f8a77ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8a738360 con 0x557f8a771ac0 2024-03-09T00:07:29.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8a6d4ec0 con 0x557f8a7711a0 2024-03-09T00:07:29.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1d498c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8a67dd30 con 0x557f8a77a6a0 2024-03-09T00:07:29.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.988+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1b40a2850 0x7ff1b40a4d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc198a20 tx=0x7ff1c0005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a77a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a771580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a77a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48590/0 (socket says 172.21.15.22:48590) 2024-03-09T00:07:29.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 -- 172.21.15.22:0/563295946 learned_addr learned my addr 172.21.15.22:0/563295946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a77a6a0 0x557f8a77ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 -- 172.21.15.22:0/563295946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a77a6a0 msgr2=0x557f8a77ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a77a6a0 0x557f8a77ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 -- 172.21.15.22:0/563295946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 msgr2=0x557f8a771580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a771580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 -- 172.21.15.22:0/563295946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f8a67cbc0 con 0x557f8a771ac0 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1caffd640 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a77a160 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc06aa60 tx=0x7ff1bc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c752bc9ef83048 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:29.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1b1ffb640 1 -- 172.21.15.22:0/563295946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc042070 con 0x557f8a771ac0 2024-03-09T00:07:29.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1b1ffb640 1 -- 172.21.15.22:0/563295946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1bc004030 con 0x557f8a771ac0 2024-03-09T00:07:29.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.989+0000 7ff1b1ffb640 1 -- 172.21.15.22:0/563295946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc004f20 con 0x557f8a771ac0 2024-03-09T00:07:29.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 -- 172.21.15.22:0/563295946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 msgr2=0x557f8a77a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:29.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a77a160 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc06aa60 tx=0x7ff1bc005be0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 -- 172.21.15.22:0/563295946 shutdown_connections 2024-03-09T00:07:29.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a77a6a0 0x557f8a77ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a77a160 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 --2- 172.21.15.22:0/563295946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a771580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 -- 172.21.15.22:0/563295946 >> 172.21.15.22:0/563295946 conn(0x557f8a76b8d0 msgr2=0x557f8a76bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 -- 172.21.15.22:0/563295946 shutdown_connections 2024-03-09T00:07:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 -- 172.21.15.22:0/563295946 wait complete. 2024-03-09T00:07:29.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.990+0000 7ff1d498c740 1 -- start start 2024-03-09T00:07:29.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 1 Processor -- start 2024-03-09T00:07:29.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a7776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a771ac0 0x557f8a777bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a778130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:29.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8a6734a0 con 0x557f8a77a6a0 2024-03-09T00:07:29.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8a6d0f60 con 0x557f8a771ac0 2024-03-09T00:07:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8a77d430 con 0x557f8a7711a0 2024-03-09T00:07:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1d498c740 10 client.0 _ll_register_callbacks cb 0x557f8a6ced60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a771ac0 0x557f8a777bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a7776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a778130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a771ac0 0x557f8a777bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55468/0 (socket says 172.21.15.22:55468) 2024-03-09T00:07:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 -- 172.21.15.22:0/4178180380 learned_addr learned my addr 172.21.15.22:0/4178180380 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 -- 172.21.15.22:0/4178180380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 msgr2=0x557f8a7776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a7776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 -- 172.21.15.22:0/4178180380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 msgr2=0x557f8a778130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a778130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 -- 172.21.15.22:0/4178180380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f8a3e93f0 con 0x557f8a771ac0 2024-03-09T00:07:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1caffd640 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a771ac0 0x557f8a777bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc004490 tx=0x7ff1bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefeba7def1429b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.991+0000 7ff1aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:07:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4178180380 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc052060 con 0x557f8a771ac0 2024-03-09T00:07:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4178180380 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff1bc02ca90 con 0x557f8a771ac0 2024-03-09T00:07:30.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1bc004030 con 0x557f8a771ac0 2024-03-09T00:07:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4178180380 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1bc059050 con 0x557f8a771ac0 2024-03-09T00:07:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 10 client.28946 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557f8a4b37d0 con 0x557f8a771ac0 2024-03-09T00:07:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4178180380 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1bc004030 con 0x557f8a771ac0 2024-03-09T00:07:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4178180380 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff1bc0052a0 con 0x557f8a771ac0 2024-03-09T00:07:30.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 10 client.28946 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 10 client.28946 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:07:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 20 client.28946 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a78caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 4 client.28946 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f8a6e2400 con 0x557f8a78a6b0 2024-03-09T00:07:30.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.992+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a78caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.993+0000 7ff1aaffd640 10 client.28946 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:07:30.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.993+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a78caa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eb799a934b643c9 server_cookie=6add7fed3c7ef75e in_seq=0 out_seq=0 2024-03-09T00:07:30.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4178180380 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f8a6e2400 con 0x557f8a78a6b0 2024-03-09T00:07:30.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1aaffd640 10 client.28946 handle_command_reply: tid=0 2024-03-09T00:07:30.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 client.28946 shutdown 2024-03-09T00:07:30.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a771ac0 msgr2=0x557f8a777bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a771ac0 0x557f8a777bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc004490 tx=0x7ff1bc0046c0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 shutdown_connections 2024-03-09T00:07:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a78caa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a778130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1aaffd640 0 client.28946 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:07:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a771ac0 0x557f8a777bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4178180380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a7776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 >> 172.21.15.22:0/4178180380 conn(0x557f8a76b8d0 msgr2=0x557f8a76d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 shutdown_connections 2024-03-09T00:07:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 1 -- 172.21.15.22:0/4178180380 wait complete. 2024-03-09T00:07:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:29.994+0000 7ff1d498c740 20 client.28946 trim_cache size 0 max 16384 2024-03-09T00:07:30.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 Processor -- start 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 -- start start 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a779de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a772940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a77a6a0 0x557f8a772e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1ae5a0 con 0x557f8a771ac0 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1ac140 con 0x557f8a77a6a0 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1d498c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1aa200 con 0x557f8a7711a0 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a772940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a772940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48612/0 (socket says 172.21.15.22:48612) 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 -- 172.21.15.22:0/148435287 learned_addr learned my addr 172.21.15.22:0/148435287 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a77a6a0 0x557f8a772e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a779de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 -- 172.21.15.22:0/148435287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 msgr2=0x557f8a779de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a779de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 -- 172.21.15.22:0/148435287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a77a6a0 msgr2=0x557f8a772e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a77a6a0 0x557f8a772e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.006+0000 7ff1caffd640 1 -- 172.21.15.22:0/148435287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f8a787640 con 0x557f8a771ac0 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1caffd640 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a772940 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a93755823e2ef8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1b17fa640 1 -- 172.21.15.22:0/148435287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc052070 con 0x557f8a771ac0 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1b17fa640 1 -- 172.21.15.22:0/148435287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1bc0044c0 con 0x557f8a771ac0 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1b17fa640 1 -- 172.21.15.22:0/148435287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc042070 con 0x557f8a771ac0 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 -- 172.21.15.22:0/148435287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 msgr2=0x557f8a772940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a772940 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc005dd0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 -- 172.21.15.22:0/148435287 shutdown_connections 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a77a6a0 0x557f8a772e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a771ac0 0x557f8a772940 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 --2- 172.21.15.22:0/148435287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a7711a0 0x557f8a779de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 -- 172.21.15.22:0/148435287 >> 172.21.15.22:0/148435287 conn(0x557f8a76b8d0 msgr2=0x557f8a76bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 -- 172.21.15.22:0/148435287 shutdown_connections 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.007+0000 7ff1d498c740 1 -- 172.21.15.22:0/148435287 wait complete. 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 -- start start 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 Processor -- start 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a780140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a771ac0 0x557f8a780680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a780bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1aa800 con 0x557f8a77a6a0 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1abae0 con 0x557f8a7711a0 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1aae20 con 0x557f8a771ac0 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1d498c740 10 client.0 _ll_register_callbacks cb 0x7ff1c40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a780bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a780bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48618/0 (socket says 172.21.15.22:48618) 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 -- 172.21.15.22:0/4142566540 learned_addr learned my addr 172.21.15.22:0/4142566540 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1caffd640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a771ac0 0x557f8a780680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a780140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 -- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a771ac0 msgr2=0x557f8a780680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a771ac0 0x557f8a780680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 -- 172.21.15.22:0/4142566540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 msgr2=0x557f8a780140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a780140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cbfff640 1 -- 172.21.15.22:0/4142566540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f8a7e0920 con 0x557f8a77a6a0 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a780140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.008+0000 7ff1caffd640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a771ac0 0x557f8a780680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1cbfff640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a780bc0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4004f90 tx=0x7ff1c409b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90828815116074d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4142566540 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40b8070 con 0x557f8a77a6a0 2024-03-09T00:07:30.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4142566540 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7ff1c409bd80 con 0x557f8a77a6a0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f8a6dc750 con 0x557f8a77a6a0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4142566540 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1c4081050 con 0x557f8a77a6a0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 10 client.19674 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557f8a7e1c80 con 0x557f8a77a6a0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4142566540 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557f8a6dc750 con 0x557f8a77a6a0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4142566540 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff1c409ae00 con 0x557f8a77a6a0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 10 client.19674 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 10 client.19674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 20 client.19674 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a7e4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 4 client.19674 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f8a65a640 con 0x557f8a78a6b0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.009+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a7e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1cb7fe640 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a7e4390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f97c1eb6a1a44b1 server_cookie=3caa54f1b2b5257b in_seq=0 out_seq=0 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1aaffd640 10 client.19674 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1aaffd640 1 -- 172.21.15.22:0/4142566540 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f8a65a640 con 0x557f8a78a6b0 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1aaffd640 10 client.19674 handle_command_reply: tid=0 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 client.19674 shutdown 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 msgr2=0x557f8a780bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a780bc0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4004f90 tx=0x7ff1c409b350 comp rx=0 tx=0).stop 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 shutdown_connections 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x557f8a78a6b0 0x557f8a7e4390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557f8a77a6a0 0x557f8a780bc0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f8a771ac0 0x557f8a780680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1aaffd640 0 client.19674 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 --2- 172.21.15.22:0/4142566540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f8a7711a0 0x557f8a780140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 >> 172.21.15.22:0/4142566540 conn(0x557f8a76b8d0 msgr2=0x557f8a772940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 shutdown_connections 2024-03-09T00:07:30.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 1 -- 172.21.15.22:0/4142566540 wait complete. 2024-03-09T00:07:30.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.010+0000 7ff1d498c740 20 client.19674 trim_cache size 0 max 16384 2024-03-09T00:07:30.022 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:07:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1b40a2850 msgr2=0x7ff1b40a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1b40a2850 0x7ff1b40a4d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc198a20 tx=0x7ff1c0005f90 comp rx=0 tx=0).stop 2024-03-09T00:07:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc1978e0 msgr2=0x7ff1cc197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc1978e0 0x7ff1cc197d90 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400eb10 tx=0x7ff1c400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 shutdown_connections 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1b40a2850 0x7ff1b40a4d10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1cc1978e0 0x7ff1cc197d90 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1cc1054b0 0x7ff1cc19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 --2- 172.21.15.22:0/1318141221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc1003e0 0x7ff1cc19d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 >> 172.21.15.22:0/1318141221 conn(0x7ff1cc0773a0 msgr2=0x7ff1cc068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 shutdown_connections 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.011+0000 7ff1d1cd1640 1 -- 172.21.15.22:0/1318141221 wait complete. 2024-03-09T00:07:30.024 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:07:30.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 -- 172.21.15.22:0/4281052592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 msgr2=0x7fdf74069920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 --2- 172.21.15.22:0/4281052592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 0x7fdf74069920 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009900 tx=0x7fdf6c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 -- 172.21.15.22:0/4281052592 shutdown_connections 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 --2- 172.21.15.22:0/4281052592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf74103ce0 0x7fdf74104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 --2- 172.21.15.22:0/4281052592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 0x7fdf7406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 --2- 172.21.15.22:0/4281052592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 0x7fdf74069920 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 -- 172.21.15.22:0/4281052592 >> 172.21.15.22:0/4281052592 conn(0x7fdf74076f80 msgr2=0x7fdf74077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 -- 172.21.15.22:0/4281052592 shutdown_connections 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.139+0000 7fdf79516640 1 -- 172.21.15.22:0/4281052592 wait complete. 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 Processor -- start 2024-03-09T00:07:30.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 -- start start 2024-03-09T00:07:30.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 0x7fdf7419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf74103ce0 0x7fdf7419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 0x7fdf741a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74117c40 con 0x7fdf74106280 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74117ac0 con 0x7fdf74103ce0 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf79516640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74117dc0 con 0x7fdf74069e60 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 0x7fdf7419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 0x7fdf741a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 0x7fdf7419e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:42760/0 (socket says 172.21.15.22:42760) 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf74103ce0 0x7fdf7419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.140+0000 7fdf72ffd640 1 -- 172.21.15.22:0/144309946 learned_addr learned my addr 172.21.15.22:0/144309946 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.141+0000 7fdf737fe640 1 -- 172.21.15.22:0/144309946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 msgr2=0x7fdf7419e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.141+0000 7fdf737fe640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 0x7fdf7419e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.141+0000 7fdf737fe640 1 -- 172.21.15.22:0/144309946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf74103ce0 msgr2=0x7fdf7419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.141+0000 7fdf737fe640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf74103ce0 0x7fdf7419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.141+0000 7fdf737fe640 1 -- 172.21.15.22:0/144309946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf741a3170 con 0x7fdf74106280 2024-03-09T00:07:30.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.141+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 0x7fdf7419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:30.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.141+0000 7fdf737fe640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 0x7fdf741a2a40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009fd0 tx=0x7fdf6400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.143+0000 7fdf6bfff640 1 -- 172.21.15.22:0/144309946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400ce20 con 0x7fdf74106280 2024-03-09T00:07:30.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.143+0000 7fdf6bfff640 1 -- 172.21.15.22:0/144309946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf64005020 con 0x7fdf74106280 2024-03-09T00:07:30.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.143+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf741a3460 con 0x7fdf74106280 2024-03-09T00:07:30.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.143+0000 7fdf6bfff640 1 -- 172.21.15.22:0/144309946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64010720 con 0x7fdf74106280 2024-03-09T00:07:30.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.143+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf741aacf0 con 0x7fdf74106280 2024-03-09T00:07:30.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf6bfff640 1 -- 172.21.15.22:0/144309946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdf64005540 con 0x7fdf74106280 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 Processor -- start 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 -- start start 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438125a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381263160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x564381265a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564381221360 con 0x56438125a1a0 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643811bdec0 con 0x56438125aac0 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.146+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564381166d30 con 0x5643812636a0 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438125a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381263160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438125a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48636/0 (socket says 172.21.15.22:48636) 2024-03-09T00:07:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381263160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55526/0 (socket says 172.21.15.22:55526) 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf727fc640 1 -- 172.21.15.22:0/2560226951 learned_addr learned my addr 172.21.15.22:0/2560226951 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf727fc640 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x564381265a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf6bfff640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf600a29f0 0x7fdf600a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 -- 172.21.15.22:0/2560226951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 msgr2=0x564381265a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x564381265a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf737fe640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf600a29f0 0x7fdf600a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 -- 172.21.15.22:0/2560226951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 msgr2=0x56438125a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438125a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 -- 172.21.15.22:0/2560226951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564381165bc0 con 0x56438125aac0 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf727fc640 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x564381265a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:07:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf6bfff640 1 -- 172.21.15.22:0/144309946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdf640cf820 con 0x7fdf74106280 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381263160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005ab0 tx=0x7fdf6c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42930ea47f01b3f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf737fe640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf600a29f0 0x7fdf600a4eb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdf640ccc20 tx=0x7fdf64086040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf697fa640 1 -- 172.21.15.22:0/2560226951 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c042070 con 0x56438125aac0 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf697fa640 1 -- 172.21.15.22:0/2560226951 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf6c004a70 con 0x56438125aac0 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.147+0000 7fdf697fa640 1 -- 172.21.15.22:0/2560226951 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0051a0 con 0x56438125aac0 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/2560226951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 msgr2=0x564381263160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381263160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005ab0 tx=0x7fdf6c009560 comp rx=0 tx=0).stop 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/2560226951 shutdown_connections 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x564381265a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381263160 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/2560226951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438125a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/2560226951 >> 172.21.15.22:0/2560226951 conn(0x5643812548d0 msgr2=0x564381254cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/2560226951 shutdown_connections 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.148+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/2560226951 wait complete. 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 -- start start 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 Processor -- start 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x564381260670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381260bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x5643812610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438115c4a0 con 0x56438125a1a0 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643811b9f60 con 0x56438125aac0 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564381266490 con 0x5643812636a0 2024-03-09T00:07:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf7c1cb740 10 client.0 _ll_register_callbacks cb 0x5643811b7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x564381260670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381260bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x564381260670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48650/0 (socket says 172.21.15.22:48650) 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf727fc640 1 -- 172.21.15.22:0/4012965573 learned_addr learned my addr 172.21.15.22:0/4012965573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf727fc640 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x5643812610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf72ffd640 1 -- 172.21.15.22:0/4012965573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 msgr2=0x5643812610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x5643812610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf72ffd640 1 -- 172.21.15.22:0/4012965573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 msgr2=0x564381260670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x564381260670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf72ffd640 1 -- 172.21.15.22:0/4012965573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564380ed23f0 con 0x56438125aac0 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.149+0000 7fdf727fc640 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x564381260670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:30.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381260bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c000c00 tx=0x7fdf6c02cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d7f6f4fbce2493 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/4012965573 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c042040 con 0x56438125aac0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/4012965573 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdf6c005170 con 0x56438125aac0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf6c004a70 con 0x56438125aac0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/4012965573 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf6c005370 con 0x56438125aac0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf7c1cb740 10 client.28964 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564380f9c7d0 con 0x56438125aac0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/4012965573 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf6c004a70 con 0x56438125aac0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.150+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/4012965573 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf6c034440 con 0x56438125aac0 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf7c1cb740 10 client.28964 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf7c1cb740 10 client.28964 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf7c1cb740 20 client.28964 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x564381273700 0x564381275af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf7c1cb740 4 client.28964 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643811cb400 con 0x564381273700 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf727fc640 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x564381273700 0x564381275af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf727fc640 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x564381273700 0x564381275af0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac1fe55d9399e346 server_cookie=b122ee6194178f92 in_seq=0 out_seq=0 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.151+0000 7fdf5a7fc640 10 client.28964 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/4012965573 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643811cb400 con 0x564381273700 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf5a7fc640 10 client.28964 handle_command_reply: tid=0 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 client.28964 shutdown 2024-03-09T00:07:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 msgr2=0x564381260bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381260bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c000c00 tx=0x7fdf6c02cd30 comp rx=0 tx=0).stop 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 shutdown_connections 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x564381273700 0x564381275af0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643812636a0 0x5643812610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x564381260bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/4012965573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x564381260670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf5a7fc640 0 client.28964 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 >> 172.21.15.22:0/4012965573 conn(0x5643812548d0 msgr2=0x564381256620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 shutdown_connections 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/4012965573 wait complete. 2024-03-09T00:07:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.152+0000 7fdf7c1cb740 20 client.28964 trim_cache size 0 max 16384 2024-03-09T00:07:30.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.164+0000 7fdf7c1cb740 1 Processor -- start 2024-03-09T00:07:30.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.164+0000 7fdf7c1cb740 1 -- start start 2024-03-09T00:07:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125a1a0 0x564381262da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x56438125afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643812636a0 0x564381254cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741b70c0 con 0x5643812636a0 2024-03-09T00:07:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741b4c60 con 0x56438125aac0 2024-03-09T00:07:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741b2d20 con 0x56438125a1a0 2024-03-09T00:07:30.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125a1a0 0x564381262da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125a1a0 0x564381262da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:42794/0 (socket says 172.21.15.22:42794) 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 -- 172.21.15.22:0/1550096316 learned_addr learned my addr 172.21.15.22:0/1550096316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643812636a0 0x564381254cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x56438125afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 -- 172.21.15.22:0/1550096316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 msgr2=0x56438125afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x56438125afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 -- 172.21.15.22:0/1550096316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643812636a0 msgr2=0x564381254cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643812636a0 0x564381254cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 -- 172.21.15.22:0/1550096316 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564381254110 con 0x56438125a1a0 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.165+0000 7fdf727fc640 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125a1a0 0x564381262da0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0098f0 tx=0x7fdf5c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18eee13d724fef81 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.166+0000 7fdf68ff9640 1 -- 172.21.15.22:0/1550096316 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c012230 con 0x56438125a1a0 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.166+0000 7fdf68ff9640 1 -- 172.21.15.22:0/1550096316 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c005500 con 0x56438125a1a0 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.166+0000 7fdf68ff9640 1 -- 172.21.15.22:0/1550096316 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0047a0 con 0x56438125a1a0 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.166+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/1550096316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125a1a0 msgr2=0x564381262da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.166+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125a1a0 0x564381262da0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0098f0 tx=0x7fdf5c009780 comp rx=0 tx=0).stop 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/1550096316 shutdown_connections 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5643812636a0 0x564381254cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56438125aac0 0x56438125afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/1550096316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125a1a0 0x564381262da0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/1550096316 >> 172.21.15.22:0/1550096316 conn(0x5643812548d0 msgr2=0x56438125ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/1550096316 shutdown_connections 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/1550096316 wait complete. 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- start start 2024-03-09T00:07:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 Processor -- start 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438126a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125aac0 0x564381268d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643812636a0 0x5643812692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741b3320 con 0x56438125a1a0 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741b4600 con 0x5643812636a0 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741b3940 con 0x56438125aac0 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf7c1cb740 10 client.0 _ll_register_callbacks cb 0x7fdf5c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438126a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438126a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48668/0 (socket says 172.21.15.22:48668) 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf727fc640 1 -- 172.21.15.22:0/3759990702 learned_addr learned my addr 172.21.15.22:0/3759990702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125aac0 0x564381268d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.167+0000 7fdf727fc640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643812636a0 0x5643812692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf727fc640 1 -- 172.21.15.22:0/3759990702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125aac0 msgr2=0x564381268d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf727fc640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125aac0 0x564381268d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf727fc640 1 -- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643812636a0 msgr2=0x5643812692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf727fc640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643812636a0 0x5643812692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf727fc640 1 -- 172.21.15.22:0/3759990702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564381269af0 con 0x56438125a1a0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125aac0 0x564381268d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf727fc640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438126a9e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c006010 tx=0x7fdf5c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b585b825e6001fd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/3759990702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c011810 con 0x56438125a1a0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/3759990702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fdf5c0119b0 con 0x56438125a1a0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf5c005500 con 0x56438125a1a0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/3759990702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf5c011bb0 con 0x56438125a1a0 2024-03-09T00:07:30.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf7c1cb740 10 client.19692 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5643812ca780 con 0x56438125a1a0 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/3759990702 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c005500 con 0x56438125a1a0 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.168+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/3759990702 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf5c003c50 con 0x56438125a1a0 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 10 client.19692 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 10 client.19692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 20 client.19692 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5643812cadc0 0x5643812cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 4 client.19692 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643811c5750 con 0x5643812cadc0 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5643812cadc0 0x5643812cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf72ffd640 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5643812cadc0 0x5643812cd1e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a7d54aa8a8f7eac server_cookie=9f2f27f5a516ddf0 in_seq=0 out_seq=0 2024-03-09T00:07:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf5a7fc640 10 client.19692 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf5a7fc640 1 -- 172.21.15.22:0/3759990702 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643811c5750 con 0x5643812cadc0 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf5a7fc640 10 client.19692 handle_command_reply: tid=0 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 client.19692 shutdown 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 msgr2=0x56438126a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438126a9e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c006010 tx=0x7fdf5c001e70 comp rx=0 tx=0).stop 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 shutdown_connections 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5643812cadc0 0x5643812cd1e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643812636a0 0x5643812692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56438125aac0 0x564381268d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf5a7fc640 0 client.19692 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.169+0000 7fdf7c1cb740 1 --2- 172.21.15.22:0/3759990702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56438125a1a0 0x56438126a9e0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 >> 172.21.15.22:0/3759990702 conn(0x5643812548d0 msgr2=0x56438125eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 shutdown_connections 2024-03-09T00:07:30.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf7c1cb740 1 -- 172.21.15.22:0/3759990702 wait complete. 2024-03-09T00:07:30.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf7c1cb740 20 client.19692 trim_cache size 0 max 16384 2024-03-09T00:07:30.172 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:07:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf600a29f0 msgr2=0x7fdf600a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf600a29f0 0x7fdf600a4eb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdf640ccc20 tx=0x7fdf64086040 comp rx=0 tx=0).stop 2024-03-09T00:07:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 msgr2=0x7fdf741a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 0x7fdf741a2a40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009fd0 tx=0x7fdf6400ef10 comp rx=0 tx=0).stop 2024-03-09T00:07:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 shutdown_connections 2024-03-09T00:07:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf600a29f0 0x7fdf600a4eb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf74106280 0x7fdf741a2a40 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf74103ce0 0x7fdf7419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 --2- 172.21.15.22:0/144309946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf74069e60 0x7fdf7419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 >> 172.21.15.22:0/144309946 conn(0x7fdf74076f80 msgr2=0x7fdf74101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 shutdown_connections 2024-03-09T00:07:30.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.170+0000 7fdf79516640 1 -- 172.21.15.22:0/144309946 wait complete. 2024-03-09T00:07:30.176 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:07:30.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:07:30.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 -- 172.21.15.22:0/2237561420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c101e80 msgr2=0x7f8a9c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2237561420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c101e80 0x7f8a9c105ec0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009900 tx=0x7f8a9002bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:30.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 -- 172.21.15.22:0/2237561420 shutdown_connections 2024-03-09T00:07:30.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2237561420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a9c1065a0 0x7f8a9c10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2237561420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c101e80 0x7f8a9c105ec0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2237561420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a9c1014d0 0x7f8a9c1018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 -- 172.21.15.22:0/2237561420 >> 172.21.15.22:0/2237561420 conn(0x7f8a9c0fcd90 msgr2=0x7f8a9c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 -- 172.21.15.22:0/2237561420 shutdown_connections 2024-03-09T00:07:30.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.289+0000 7f8aa4620640 1 -- 172.21.15.22:0/2237561420 wait complete. 2024-03-09T00:07:30.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 Processor -- start 2024-03-09T00:07:30.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 -- start start 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a9c1014d0 0x7f8a9c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a9c1065a0 0x7f8a9c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c074070 0x7f8a9c1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c111630 con 0x7f8a9c074070 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1114b0 con 0x7f8a9c1014d0 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa4620640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1117b0 con 0x7f8a9c1065a0 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa2bbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c074070 0x7f8a9c1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa23bc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a9c1014d0 0x7f8a9c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa2bbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c074070 0x7f8a9c1a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48692/0 (socket says 172.21.15.22:48692) 2024-03-09T00:07:30.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa2bbd640 1 -- 172.21.15.22:0/2435725022 learned_addr learned my addr 172.21.15.22:0/2435725022 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a9c1065a0 0x7f8a9c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.290+0000 7f8aa2bbd640 1 -- 172.21.15.22:0/2435725022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a9c1065a0 msgr2=0x7f8a9c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8aa2bbd640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a9c1065a0 0x7f8a9c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8aa2bbd640 1 -- 172.21.15.22:0/2435725022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a9c1014d0 msgr2=0x7f8a9c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8aa2bbd640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a9c1014d0 0x7f8a9c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8aa2bbd640 1 -- 172.21.15.22:0/2435725022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c1a6480 con 0x7f8a9c074070 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8aa2bbd640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c074070 0x7f8a9c1a5d80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00eb10 tx=0x7f8a8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8a9b7fe640 1 -- 172.21.15.22:0/2435725022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c004b20 con 0x7f8a9c074070 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c1a6770 con 0x7f8a9c074070 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8a9b7fe640 1 -- 172.21.15.22:0/2435725022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c004cc0 con 0x7f8a9c074070 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8a9b7fe640 1 -- 172.21.15.22:0/2435725022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c010760 con 0x7f8a9c074070 2024-03-09T00:07:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.291+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c1a6c40 con 0x7f8a9c074070 2024-03-09T00:07:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.293+0000 7f8a9b7fe640 1 -- 172.21.15.22:0/2435725022 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8a8c005020 con 0x7f8a9c074070 2024-03-09T00:07:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 Processor -- start 2024-03-09T00:07:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 -- start start 2024-03-09T00:07:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a5ac0 0x5581cb1ae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb16c360 con 0x5581cb1a51a0 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb108ec0 con 0x5581cb1ae6a0 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa7201740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb0b1d30 con 0x5581cb1a5ac0 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa1bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8a9b7fe640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a840a29a0 0x7f8a840a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa1bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48702/0 (socket says 172.21.15.22:48702) 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa23bc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a5ac0 0x5581cb1ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/2095557416 learned_addr learned my addr 172.21.15.22:0/2095557416 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8aa2bbd640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a840a29a0 0x7f8a840a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.294+0000 7f8a9b7fe640 1 -- 172.21.15.22:0/2435725022 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a8c0cf750 con 0x7f8a9c074070 2024-03-09T00:07:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8aa23bc640 1 -- 172.21.15.22:0/2095557416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1ae6a0 msgr2=0x5581cb1b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8aa23bc640 1 -- 172.21.15.22:0/2095557416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 msgr2=0x5581cb1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8aa23bc640 1 -- 172.21.15.22:0/2095557416 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581cb0b0bc0 con 0x5581cb1a5ac0 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a5ac0 0x5581cb1ae160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009c00 tx=0x7f8a94011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4593024c56acdbee server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8aa2bbd640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a840a29a0 0x7f8a840a4e60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c09b6b0 tx=0x7f8a8c090e40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8a98bfe640 1 -- 172.21.15.22:0/2095557416 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94004050 con 0x5581cb1a5ac0 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8a98bfe640 1 -- 172.21.15.22:0/2095557416 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a94005500 con 0x5581cb1a5ac0 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.295+0000 7f8a98bfe640 1 -- 172.21.15.22:0/2095557416 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94004dd0 con 0x5581cb1a5ac0 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 -- 172.21.15.22:0/2095557416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a5ac0 msgr2=0x5581cb1ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a5ac0 0x5581cb1ae160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009c00 tx=0x7f8a94011810 comp rx=0 tx=0).stop 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 -- 172.21.15.22:0/2095557416 shutdown_connections 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a5ac0 0x5581cb1ae160 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 --2- 172.21.15.22:0/2095557416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 -- 172.21.15.22:0/2095557416 >> 172.21.15.22:0/2095557416 conn(0x5581cb19f8d0 msgr2=0x5581cb19fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 -- 172.21.15.22:0/2095557416 shutdown_connections 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 -- 172.21.15.22:0/2095557416 wait complete. 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 -- start start 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.296+0000 7f8aa7201740 1 Processor -- start 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a51a0 0x5581cb1ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1ae6a0 0x5581cb1ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa7201740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb0a74a0 con 0x5581cb1ae6a0 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa7201740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb104f60 con 0x5581cb1a5ac0 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa7201740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb1b1490 con 0x5581cb1a51a0 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa7201740 10 client.0 _ll_register_callbacks cb 0x5581cb102d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a51a0 0x5581cb1ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a51a0 0x5581cb1ab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:42852/0 (socket says 172.21.15.22:42852) 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/393446350 learned_addr learned my addr 172.21.15.22:0/393446350 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1ae6a0 0x5581cb1ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/393446350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 msgr2=0x5581cb1abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/393446350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1ae6a0 msgr2=0x5581cb1ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1ae6a0 0x5581cb1ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/393446350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581cae1d3f0 con 0x5581cb1a51a0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a51a0 0x5581cb1ab5c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a9002c170 tx=0x7f8a90005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9471b52698937470 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.297+0000 7f8a81ffb640 1 -- 172.21.15.22:0/393446350 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90004230 con 0x5581cb1a51a0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8a81ffb640 1 -- 172.21.15.22:0/393446350 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a900043d0 con 0x5581cb1a51a0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a94005500 con 0x5581cb1a51a0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8a81ffb640 1 -- 172.21.15.22:0/393446350 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a90055050 con 0x5581cb1a51a0 2024-03-09T00:07:30.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 10 client.29137 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5581caee77d0 con 0x5581cb1a51a0 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8a81ffb640 1 -- 172.21.15.22:0/393446350 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a94005500 con 0x5581cb1a51a0 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8a81ffb640 1 -- 172.21.15.22:0/393446350 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a90003c30 con 0x5581cb1a51a0 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 10 client.29137 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 10 client.29137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 20 client.29137 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb1be790 0x5581cb1c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 4 client.29137 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581cb116400 con 0x5581cb1be790 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.298+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb1be790 0x5581cb1c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb1be790 0x5581cb1c0b80 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f24b5bf49c9401 server_cookie=6bf226ce1ddc5884 in_seq=0 out_seq=0 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8a81ffb640 10 client.29137 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8a81ffb640 1 -- 172.21.15.22:0/393446350 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581cb116400 con 0x5581cb1be790 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8a81ffb640 10 client.29137 handle_command_reply: tid=0 2024-03-09T00:07:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 client.29137 shutdown 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a51a0 msgr2=0x5581cb1ab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a51a0 0x5581cb1ab5c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a9002c170 tx=0x7f8a90005fd0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 shutdown_connections 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb1be790 0x5581cb1c0b80 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1ae6a0 0x5581cb1ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8a81ffb640 0 client.29137 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 --2- 172.21.15.22:0/393446350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1a51a0 0x5581cb1ab5c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 >> 172.21.15.22:0/393446350 conn(0x5581cb19f8d0 msgr2=0x5581cb1a1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 shutdown_connections 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.299+0000 7f8aa7201740 1 -- 172.21.15.22:0/393446350 wait complete. 2024-03-09T00:07:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.300+0000 7f8aa7201740 20 client.29137 trim_cache size 0 max 16384 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 Processor -- start 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 -- start start 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a51a0 0x5581cb1adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1b2dc0 con 0x5581cb1a5ac0 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1b0960 con 0x5581cb1a51a0 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.312+0000 7f8aa7201740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1aea20 con 0x5581cb1ae6a0 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48722/0 (socket says 172.21.15.22:48722) 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 -- 172.21.15.22:0/649838206 learned_addr learned my addr 172.21.15.22:0/649838206 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a51a0 0x5581cb1adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 -- 172.21.15.22:0/649838206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 msgr2=0x5581cb1b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 -- 172.21.15.22:0/649838206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a51a0 msgr2=0x5581cb1adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a51a0 0x5581cb1adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 -- 172.21.15.22:0/649838206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581cb19f700 con 0x5581cb1a5ac0 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b56c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009230 tx=0x7f8a94004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c894e44e2be146c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8a83fff640 1 -- 172.21.15.22:0/649838206 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9402c070 con 0x5581cb1a5ac0 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8a83fff640 1 -- 172.21.15.22:0/649838206 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a94004520 con 0x5581cb1a5ac0 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.313+0000 7f8a83fff640 1 -- 172.21.15.22:0/649838206 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9402d4d0 con 0x5581cb1a5ac0 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 -- 172.21.15.22:0/649838206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a5ac0 msgr2=0x5581cb1b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b56c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009230 tx=0x7f8a94004970 comp rx=0 tx=0).stop 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 -- 172.21.15.22:0/649838206 shutdown_connections 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b56c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 --2- 172.21.15.22:0/649838206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a51a0 0x5581cb1adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 -- 172.21.15.22:0/649838206 >> 172.21.15.22:0/649838206 conn(0x5581cb19f8d0 msgr2=0x5581cb1c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 -- 172.21.15.22:0/649838206 shutdown_connections 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 -- 172.21.15.22:0/649838206 wait complete. 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 -- start start 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.314+0000 7f8aa7201740 1 Processor -- start 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1af020 con 0x5581cb1a51a0 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1b0300 con 0x5581cb1a5ac0 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1af640 con 0x5581cb1ae6a0 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 10 client.0 _ll_register_callbacks cb 0x7f8a94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48734/0 (socket says 172.21.15.22:48734) 2024-03-09T00:07:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/3094089862 learned_addr learned my addr 172.21.15.22:0/3094089862 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 msgr2=0x5581cb1b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/3094089862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 msgr2=0x5581cb1b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 -- 172.21.15.22:0/3094089862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581cb1b4080 con 0x5581cb1a51a0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa1bbb640 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1b5450 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009900 tx=0x7f8a90009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b98c60d279f3f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8a81ffb640 1 -- 172.21.15.22:0/3094089862 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90005560 con 0x5581cb1a51a0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8a81ffb640 1 -- 172.21.15.22:0/3094089862 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f8a90004000 con 0x5581cb1a51a0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a94004520 con 0x5581cb1a51a0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8a81ffb640 1 -- 172.21.15.22:0/3094089862 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a9006e050 con 0x5581cb1a51a0 2024-03-09T00:07:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 10 client.19710 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.315+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5581cb2156f0 con 0x5581cb1a51a0 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8a81ffb640 1 -- 172.21.15.22:0/3094089862 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a94004520 con 0x5581cb1a51a0 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8a81ffb640 1 -- 172.21.15.22:0/3094089862 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a90066020 con 0x5581cb1a51a0 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa7201740 10 client.19710 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa7201740 10 client.19710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa7201740 20 client.19710 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa7201740 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb215d00 0x5581cb2180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa7201740 4 client.19710 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581cb110750 con 0x5581cb215d00 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb215d00 0x5581cb2180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8aa23bc640 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb215d00 0x5581cb2180f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6556b80fab42d32c server_cookie=32f2ffd0132619af in_seq=0 out_seq=0 2024-03-09T00:07:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.316+0000 7f8a81ffb640 10 client.19710 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:30.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8a81ffb640 1 -- 172.21.15.22:0/3094089862 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581cb110750 con 0x5581cb215d00 2024-03-09T00:07:30.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8a81ffb640 10 client.19710 handle_command_reply: tid=0 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 client.19710 shutdown 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 msgr2=0x5581cb1b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1b5450 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009900 tx=0x7f8a90009620 comp rx=0 tx=0).stop 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 shutdown_connections 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5581cb215d00 0x5581cb2180f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5581cb1ae6a0 0x5581cb1b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581cb1a5ac0 0x5581cb1b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 --2- 172.21.15.22:0/3094089862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581cb1a51a0 0x5581cb1b5450 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8a81ffb640 0 client.19710 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 >> 172.21.15.22:0/3094089862 conn(0x5581cb19f8d0 msgr2=0x5581cb1a9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 shutdown_connections 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 1 -- 172.21.15.22:0/3094089862 wait complete. 2024-03-09T00:07:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.321+0000 7f8aa7201740 20 client.19710 trim_cache size 0 max 16384 2024-03-09T00:07:30.319 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a840a29a0 msgr2=0x7f8a840a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a840a29a0 0x7f8a840a4e60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c09b6b0 tx=0x7f8a8c090e40 comp rx=0 tx=0).stop 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c074070 msgr2=0x7f8a9c1a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c074070 0x7f8a9c1a5d80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00eb10 tx=0x7f8a8c00c2d0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 shutdown_connections 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a840a29a0 0x7f8a840a4e60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9c074070 0x7f8a9c1a5d80 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a9c1065a0 0x7f8a9c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 --2- 172.21.15.22:0/2435725022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a9c1014d0 0x7f8a9c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 >> 172.21.15.22:0/2435725022 conn(0x7f8a9c0fcd90 msgr2=0x7f8a9c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 shutdown_connections 2024-03-09T00:07:30.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:30.322+0000 7f8aa4620640 1 -- 172.21.15.22:0/2435725022 wait complete. 2024-03-09T00:07:30.411 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:30 smithi062 ceph-mon[36129]: pgmap v409: 129 pgs: 129 active+clean; 87 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-03-09T00:07:30.411 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:30 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2072266178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:07:30.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:30 smithi022 ceph-mon[31894]: pgmap v409: 129 pgs: 129 active+clean; 87 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-03-09T00:07:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:30 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2072266178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:07:30.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:30 smithi192 ceph-mon[35995]: pgmap v409: 129 pgs: 129 active+clean; 87 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-03-09T00:07:30.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:30 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2072266178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:07:30.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1272302 9.21 MB/sec execute 578 sec latency 29.597 ms 2024-03-09T00:07:31.327 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:07:31.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:07:31.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.445+0000 7f13da71b640 1 -- 172.21.15.22:0/1888048003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 msgr2=0x7f13d410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 --2- 172.21.15.22:0/1888048003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d410d070 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009ca0 tx=0x7f13cc02c0f0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 -- 172.21.15.22:0/1888048003 shutdown_connections 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 --2- 172.21.15.22:0/1888048003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d410d070 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 --2- 172.21.15.22:0/1888048003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d4100d80 0x7f13d4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 --2- 172.21.15.22:0/1888048003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d41003d0 0x7f13d41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 -- 172.21.15.22:0/1888048003 >> 172.21.15.22:0/1888048003 conn(0x7f13d40772f0 msgr2=0x7f13d4077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 -- 172.21.15.22:0/1888048003 shutdown_connections 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 -- 172.21.15.22:0/1888048003 wait complete. 2024-03-09T00:07:31.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 Processor -- start 2024-03-09T00:07:31.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.446+0000 7f13da71b640 1 -- start start 2024-03-09T00:07:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13da71b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d41003d0 0x7f13d419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13da71b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13da71b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d41968a0 0x7f13d4196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13da71b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4110560 con 0x7f13d41054a0 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13da71b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41103e0 con 0x7f13d41968a0 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13da71b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41106e0 con 0x7f13d41003d0 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46360/0 (socket says 172.21.15.22:46360) 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 -- 172.21.15.22:0/2346925038 learned_addr learned my addr 172.21.15.22:0/2346925038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d3fff640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d41003d0 0x7f13d419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d8cb8640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d41968a0 0x7f13d4196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 -- 172.21.15.22:0/2346925038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d41003d0 msgr2=0x7f13d419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d41003d0 0x7f13d419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 -- 172.21.15.22:0/2346925038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d41968a0 msgr2=0x7f13d4196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d41968a0 0x7f13d4196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 -- 172.21.15.22:0/2346925038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4197430 con 0x7f13d41054a0 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d419cc20 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e9f0 tx=0x7f13c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.447+0000 7f13d17fa640 1 -- 172.21.15.22:0/2346925038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c800ccc0 con 0x7f13d41054a0 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.448+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d4197720 con 0x7f13d41054a0 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.448+0000 7f13d17fa640 1 -- 172.21.15.22:0/2346925038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c8005500 con 0x7f13d41054a0 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.448+0000 7f13d17fa640 1 -- 172.21.15.22:0/2346925038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8010690 con 0x7f13d41054a0 2024-03-09T00:07:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.448+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d41a3860 con 0x7f13d41054a0 2024-03-09T00:07:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13d17fa640 1 -- 172.21.15.22:0/2346925038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f13c80046c0 con 0x7f13d41054a0 2024-03-09T00:07:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13d17fa640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f13bc0a2970 0x7f13bc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 Processor -- start 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 -- start start 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13d17fa640 1 -- 172.21.15.22:0/2346925038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f13c80cf5a0 con 0x7f13d41054a0 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13d3fff640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f13bc0a2970 0x7f13bc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c4a40 0x55df355c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c5360 0x55df355cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df35528b40 con 0x55df355c5360 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df354d1d80 con 0x55df355c4a40 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.450+0000 7f13dd3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df35524be0 con 0x55df355d0490 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c5360 0x55df355cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c5360 0x55df355cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46376/0 (socket says 172.21.15.22:46376) 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 -- 172.21.15.22:0/2110758537 learned_addr learned my addr 172.21.15.22:0/2110758537 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d8cb8640 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d3fff640 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c4a40 0x55df355c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d3fff640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f13bc0a2970 0x7f13bc0a4e30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009c20 tx=0x7f13c4011810 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 -- 172.21.15.22:0/2110758537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 msgr2=0x55df355d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 -- 172.21.15.22:0/2110758537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c4a40 msgr2=0x55df355c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c4a40 0x55df355c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 -- 172.21.15.22:0/2110758537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df353057e0 con 0x55df355c5360 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c5360 0x55df355cff50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f13c8097c00 tx=0x7f13c809b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec07729009ca91de server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13ba7fc640 1 -- 172.21.15.22:0/2110758537 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c80b4070 con 0x55df355c5360 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13ba7fc640 1 -- 172.21.15.22:0/2110758537 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c809b7a0 con 0x55df355c5360 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.451+0000 7f13ba7fc640 1 -- 172.21.15.22:0/2110758537 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8097e10 con 0x55df355c5360 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 -- 172.21.15.22:0/2110758537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c5360 msgr2=0x55df355cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c5360 0x55df355cff50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f13c8097c00 tx=0x7f13c809b630 comp rx=0 tx=0).stop 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 -- 172.21.15.22:0/2110758537 shutdown_connections 2024-03-09T00:07:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c5360 0x55df355cff50 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2110758537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c4a40 0x55df355c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 -- 172.21.15.22:0/2110758537 >> 172.21.15.22:0/2110758537 conn(0x55df355bf4b0 msgr2=0x55df355bf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 -- 172.21.15.22:0/2110758537 shutdown_connections 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 -- 172.21.15.22:0/2110758537 wait complete. 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.452+0000 7f13dd3db740 1 -- start start 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 1 Processor -- start 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df355d3280 con 0x55df355c4a40 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df354e51c0 con 0x55df355c5360 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df355d3400 con 0x55df355d0490 2024-03-09T00:07:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13dd3db740 10 client.0 _ll_register_callbacks cb 0x55df354d47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46388/0 (socket says 172.21.15.22:46388) 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 -- 172.21.15.22:0/3298765730 learned_addr learned my addr 172.21.15.22:0/3298765730 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d37fe640 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d8cb8640 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 -- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 msgr2=0x55df355cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 -- 172.21.15.22:0/3298765730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 msgr2=0x55df355cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 -- 172.21.15.22:0/3298765730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df35365f70 con 0x55df355c4a40 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13d3fff640 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355cfeb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f13c401e970 tx=0x7f13c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1852ce7168ae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.453+0000 7f13b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13b37fe640 1 -- 172.21.15.22:0/3298765730 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4017040 con 0x55df355c4a40 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13b37fe640 1 -- 172.21.15.22:0/3298765730 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f13c400f750 con 0x55df355c4a40 2024-03-09T00:07:31.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13c809b7a0 con 0x55df355c4a40 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13b37fe640 1 -- 172.21.15.22:0/3298765730 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c400f950 con 0x55df355c4a40 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 10 client.19722 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55df351fd170 con 0x55df355c4a40 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13b37fe640 1 -- 172.21.15.22:0/3298765730 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c809b7a0 con 0x55df355c4a40 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13b37fe640 1 -- 172.21.15.22:0/3298765730 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f13c4025070 con 0x55df355c4a40 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 10 client.19722 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 10 client.19722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 20 client.19722 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df355e0640 0x55df355e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 4 client.19722 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df354fe960 con 0x55df355e0640 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.454+0000 7f13d37fe640 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df355e0640 0x55df355e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.455+0000 7f13b37fe640 10 client.19722 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.455+0000 7f13d37fe640 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df355e0640 0x55df355e2a50 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9418a0376b05e0 server_cookie=6a6a311719d2ab0c in_seq=0 out_seq=0 2024-03-09T00:07:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13b37fe640 1 -- 172.21.15.22:0/3298765730 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df354fe960 con 0x55df355e0640 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13b37fe640 10 client.19722 handle_command_reply: tid=0 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 client.19722 shutdown 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 msgr2=0x55df355cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355cfeb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f13c401e970 tx=0x7f13c4004730 comp rx=0 tx=0).stop 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 shutdown_connections 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df355e0640 0x55df355e2a50 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 --2- 172.21.15.22:0/3298765730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355cfeb0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13b37fe640 0 client.19722 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 >> 172.21.15.22:0/3298765730 conn(0x55df355bf4b0 msgr2=0x55df355c1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 shutdown_connections 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 1 -- 172.21.15.22:0/3298765730 wait complete. 2024-03-09T00:07:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.456+0000 7f13dd3db740 20 client.19722 trim_cache size 0 max 16384 2024-03-09T00:07:31.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 Processor -- start 2024-03-09T00:07:31.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 -- start start 2024-03-09T00:07:31.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355dc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41ae9a0 con 0x55df355c4a40 2024-03-09T00:07:31.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41ac540 con 0x55df355c5360 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13dd3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41aa600 con 0x55df355d0490 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.471+0000 7f13d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355dc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355dc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46396/0 (socket says 172.21.15.22:46396) 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 -- 172.21.15.22:0/515745479 learned_addr learned my addr 172.21.15.22:0/515745479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d8cb8640 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 -- 172.21.15.22:0/515745479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 msgr2=0x55df355cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 -- 172.21.15.22:0/515745479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 msgr2=0x55df355cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 -- 172.21.15.22:0/515745479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df355c5190 con 0x55df355c4a40 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d37fe640 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13d3fff640 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355dc130 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f13c401ef10 tx=0x7f13c4009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748fc470c3165b79 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13b9ffb640 1 -- 172.21.15.22:0/515745479 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4015020 con 0x55df355c4a40 2024-03-09T00:07:31.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13b9ffb640 1 -- 172.21.15.22:0/515745479 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c4019050 con 0x55df355c4a40 2024-03-09T00:07:31.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.472+0000 7f13b9ffb640 1 -- 172.21.15.22:0/515745479 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4017070 con 0x55df355c4a40 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 -- 172.21.15.22:0/515745479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 msgr2=0x55df355dc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355dc130 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f13c401ef10 tx=0x7f13c4009dd0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 -- 172.21.15.22:0/515745479 shutdown_connections 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df355cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 --2- 172.21.15.22:0/515745479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df355dc130 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 -- 172.21.15.22:0/515745479 >> 172.21.15.22:0/515745479 conn(0x55df355bf4b0 msgr2=0x55df355cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 -- 172.21.15.22:0/515745479 shutdown_connections 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 -- 172.21.15.22:0/515745479 wait complete. 2024-03-09T00:07:31.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.473+0000 7f13dd3db740 1 -- start start 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 1 Processor -- start 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df3562fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df35630400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41aac00 con 0x55df355c4a40 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41abee0 con 0x55df355c5360 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41ab220 con 0x55df355d0490 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13dd3db740 10 client.0 _ll_register_callbacks cb 0x7f13c4009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df3562fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df3562fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46402/0 (socket says 172.21.15.22:46402) 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 -- 172.21.15.22:0/2688546957 learned_addr learned my addr 172.21.15.22:0/2688546957 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d8cb8640 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df35630400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 -- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 msgr2=0x55df355d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 -- 172.21.15.22:0/2688546957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 msgr2=0x55df35630400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df35630400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.474+0000 7f13d3fff640 1 -- 172.21.15.22:0/2688546957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df355d5080 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13d3fff640 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df3562fec0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f13c4027df0 tx=0x7f13c4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbffa45fc905c3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13b37fe640 1 -- 172.21.15.22:0/2688546957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4015030 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13b37fe640 1 -- 172.21.15.22:0/2688546957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f13c4004870 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13c4019050 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13b37fe640 1 -- 172.21.15.22:0/2688546957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c4017040 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 10 client.19728 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55df35638690 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13b37fe640 1 -- 172.21.15.22:0/2688546957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c4019050 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13b37fe640 1 -- 172.21.15.22:0/2688546957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f13c4025040 con 0x55df355c4a40 2024-03-09T00:07:31.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 10 client.19728 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 10 client.19728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 20 client.19728 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df35638ac0 0x55df3563aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 4 client.19728 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.475+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55df354ea080 con 0x55df35638ac0 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.476+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df35638ac0 0x55df3563aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.476+0000 7f13b37fe640 10 client.19728 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.476+0000 7f13d37fe640 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df35638ac0 0x55df3563aeb0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd45c298e8c26128 server_cookie=c6a398f02290619e in_seq=0 out_seq=0 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13b37fe640 1 -- 172.21.15.22:0/2688546957 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55df354ea080 con 0x55df35638ac0 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13b37fe640 10 client.19728 handle_command_reply: tid=0 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 client.19728 shutdown 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 msgr2=0x55df3562fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df3562fec0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f13c4027df0 tx=0x7f13c4009660 comp rx=0 tx=0).stop 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 shutdown_connections 2024-03-09T00:07:31.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55df35638ac0 0x55df3563aeb0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df355d0490 0x55df355d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13b37fe640 0 client.19728 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df355c5360 0x55df35630400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 --2- 172.21.15.22:0/2688546957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55df355c4a40 0x55df3562fec0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 >> 172.21.15.22:0/2688546957 conn(0x55df355bf4b0 msgr2=0x55df355c1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 shutdown_connections 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 1 -- 172.21.15.22:0/2688546957 wait complete. 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.477+0000 7f13dd3db740 20 client.19728 trim_cache size 0 max 16384 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:07:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "257430e8-2954-4382-ac05-beb17bc768c1", 2024-03-09T00:07:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:07:31.478 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:07:31.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f13bc0a2970 msgr2=0x7f13bc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f13bc0a2970 0x7f13bc0a4e30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009c20 tx=0x7f13c4011810 comp rx=0 tx=0).stop 2024-03-09T00:07:31.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 msgr2=0x7f13d419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d419cc20 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e9f0 tx=0x7f13c800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 shutdown_connections 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f13bc0a2970 0x7f13bc0a4e30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13d41968a0 0x7f13d4196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13d41054a0 0x7f13d419cc20 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 --2- 172.21.15.22:0/2346925038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d41003d0 0x7f13d419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 >> 172.21.15.22:0/2346925038 conn(0x7f13d40772f0 msgr2=0x7f13d40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 shutdown_connections 2024-03-09T00:07:31.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.478+0000 7f13da71b640 1 -- 172.21.15.22:0/2346925038 wait complete. 2024-03-09T00:07:31.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:07:31.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:07:31.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 -- 172.21.15.22:0/517591637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 msgr2=0x7fc0901007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 --2- 172.21.15.22:0/517591637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc0901007d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc080009900 tx=0x7fc08002bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 -- 172.21.15.22:0/517591637 shutdown_connections 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 --2- 172.21.15.22:0/517591637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0901054c0 0x7fc09010d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 --2- 172.21.15.22:0/517591637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc090100da0 0x7fc090104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 --2- 172.21.15.22:0/517591637 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc0901007d0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 -- 172.21.15.22:0/517591637 >> 172.21.15.22:0/517591637 conn(0x7fc090076fa0 msgr2=0x7fc0900fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 -- 172.21.15.22:0/517591637 shutdown_connections 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.599+0000 7fc0961f1640 1 -- 172.21.15.22:0/517591637 wait complete. 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 Processor -- start 2024-03-09T00:07:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 -- start start 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc090199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0901054c0 0x7fc09019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc09019e670 0x7fc09019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc090110570 con 0x7fc0901003f0 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901103f0 con 0x7fc0901054c0 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc0961f1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901106f0 con 0x7fc09019e670 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc090199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc090199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46424/0 (socket says 172.21.15.22:46424) 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 -- 172.21.15.22:0/2329561196 learned_addr learned my addr 172.21.15.22:0/2329561196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08effd640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0901054c0 0x7fc09019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08ffff640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc09019e670 0x7fc09019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 -- 172.21.15.22:0/2329561196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc09019e670 msgr2=0x7fc09019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc09019e670 0x7fc09019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 -- 172.21.15.22:0/2329561196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0901054c0 msgr2=0x7fc09019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0901054c0 0x7fc09019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.600+0000 7fc08f7fe640 1 -- 172.21.15.22:0/2329561196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09019f160 con 0x7fc0901003f0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.601+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc090199db0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc080006010 tx=0x7fc08002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.601+0000 7fc08cff9640 1 -- 172.21.15.22:0/2329561196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080041070 con 0x7fc0901003f0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.601+0000 7fc08cff9640 1 -- 172.21.15.22:0/2329561196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc080004450 con 0x7fc0901003f0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.601+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09019f3f0 con 0x7fc0901003f0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.601+0000 7fc08cff9640 1 -- 172.21.15.22:0/2329561196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080003e00 con 0x7fc0901003f0 2024-03-09T00:07:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.601+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0901a6bb0 con 0x7fc0901003f0 2024-03-09T00:07:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.603+0000 7fc08cff9640 1 -- 172.21.15.22:0/2329561196 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc080044070 con 0x7fc0901003f0 2024-03-09T00:07:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 Processor -- start 2024-03-09T00:07:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 -- start start 2024-03-09T00:07:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725f0c60 0x55c4725f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47254ae00 con 0x55c4725e7760 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4724f3b10 con 0x55c4725f0c60 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc098dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c472546ea0 con 0x55c4725e8080 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725e7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46432/0 (socket says 172.21.15.22:46432) 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc08cff9640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc0780a2970 0x7fc0780a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc08ffff640 1 -- 172.21.15.22:0/1626472890 learned_addr learned my addr 172.21.15.22:0/1626472890 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc08f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc08effd640 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725f0c60 0x55c4725f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.604+0000 7fc08ffff640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc0780a2970 0x7fc0780a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08effd640 1 -- 172.21.15.22:0/1626472890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 msgr2=0x55c4725f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08effd640 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08effd640 1 -- 172.21.15.22:0/1626472890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 msgr2=0x55c4725e7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08effd640 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725e7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08cff9640 1 -- 172.21.15.22:0/2329561196 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc0800f0bb0 con 0x7fc0901003f0 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08effd640 1 -- 172.21.15.22:0/1626472890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4723297d0 con 0x55c4725f0c60 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08ffff640 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725e7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08effd640 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725f0c60 0x55c4725f3050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc0900683d0 tx=0x7fc084011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43911526c5f0fe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc08ffff640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc0780a2970 0x7fc0780a4e30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0880049d0 tx=0x7fc088004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc075ffb640 1 -- 172.21.15.22:0/1626472890 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084004230 con 0x55c4725f0c60 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc075ffb640 1 -- 172.21.15.22:0/1626472890 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc084009e10 con 0x55c4725f0c60 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.605+0000 7fc075ffb640 1 -- 172.21.15.22:0/1626472890 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084004d90 con 0x55c4725f0c60 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 -- 172.21.15.22:0/1626472890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725f0c60 msgr2=0x55c4725f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725f0c60 0x55c4725f3050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc0900683d0 tx=0x7fc084011810 comp rx=0 tx=0).stop 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 -- 172.21.15.22:0/1626472890 shutdown_connections 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725f0c60 0x55c4725f3050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1626472890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725e7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 -- 172.21.15.22:0/1626472890 >> 172.21.15.22:0/1626472890 conn(0x55c4725e1e90 msgr2=0x55c4725e2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 -- 172.21.15.22:0/1626472890 shutdown_connections 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 -- 172.21.15.22:0/1626472890 wait complete. 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 -- start start 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.606+0000 7fc098dd5740 1 Processor -- start 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e8080 0x55c4725ed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc098dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4725f3a50 con 0x55c4725e8080 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc098dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4725070a0 con 0x55c4725e7760 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc098dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4725f3bd0 con 0x55c4725f0c60 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc098dd5740 10 client.0 _ll_register_callbacks cb 0x55c472506a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e8080 0x55c4725ed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e8080 0x55c4725ed6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46446/0 (socket says 172.21.15.22:46446) 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08effd640 1 -- 172.21.15.22:0/1623190565 learned_addr learned my addr 172.21.15.22:0/1623190565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08f7fe640 1 -- 172.21.15.22:0/1623190565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 msgr2=0x55c4725edc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725edc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08f7fe640 1 -- 172.21.15.22:0/1623190565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e8080 msgr2=0x55c4725ed6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e8080 0x55c4725ed6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08f7fe640 1 -- 172.21.15.22:0/1623190565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c47225f3f0 con 0x55c4725e7760 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725f0340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc0800ee010 tx=0x7fc080005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0aaf9208f2084 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc06effd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.607+0000 7fc06effd640 1 -- 172.21.15.22:0/1623190565 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08004b070 con 0x55c4725e7760 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc06effd640 1 -- 172.21.15.22:0/1623190565 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc0800b9640 con 0x55c4725e7760 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc084009e10 con 0x55c4725e7760 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc06effd640 1 -- 172.21.15.22:0/1623190565 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0800bcfd0 con 0x55c4725e7760 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 10 client.29018 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c47221f0f0 con 0x55c4725e7760 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc06effd640 1 -- 172.21.15.22:0/1623190565 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc084009e10 con 0x55c4725e7760 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc06effd640 1 -- 172.21.15.22:0/1623190565 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0800bc420 con 0x55c4725e7760 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 10 client.29018 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 10 client.29018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 20 client.29018 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472600520 0x55c472602910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 4 client.29018 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c472558340 con 0x55c472600520 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.608+0000 7fc08effd640 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472600520 0x55c472602910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.609+0000 7fc06effd640 10 client.29018 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.609+0000 7fc08effd640 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472600520 0x55c472602910 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee5731f18403d34 server_cookie=58ac008425fd555b in_seq=0 out_seq=0 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.609+0000 7fc06effd640 1 -- 172.21.15.22:0/1623190565 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c472558340 con 0x55c472600520 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.609+0000 7fc06effd640 10 client.29018 handle_command_reply: tid=0 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.609+0000 7fc098dd5740 1 client.29018 shutdown 2024-03-09T00:07:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 msgr2=0x55c4725f0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725f0340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc0800ee010 tx=0x7fc080005b50 comp rx=0 tx=0).stop 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 shutdown_connections 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472600520 0x55c472602910 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725edc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e8080 0x55c4725ed6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc06effd640 0 client.29018 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 --2- 172.21.15.22:0/1623190565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725f0340 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 >> 172.21.15.22:0/1623190565 conn(0x55c4725e1e90 msgr2=0x55c4725e3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 shutdown_connections 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 1 -- 172.21.15.22:0/1623190565 wait complete. 2024-03-09T00:07:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.610+0000 7fc098dd5740 20 client.29018 trim_cache size 0 max 16384 2024-03-09T00:07:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 Processor -- start 2024-03-09T00:07:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 -- start start 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725f0c60 0x55c4725e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901b2cb0 con 0x55c4725f0c60 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901b0850 con 0x55c4725e7760 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc098dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901ae910 con 0x55c4725e8080 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725f0c60 0x55c4725e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725f0c60 0x55c4725e85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46460/0 (socket says 172.21.15.22:46460) 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 -- 172.21.15.22:0/632884490 learned_addr learned my addr 172.21.15.22:0/632884490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08effd640 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 -- 172.21.15.22:0/632884490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 msgr2=0x55c4725f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 -- 172.21.15.22:0/632884490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 msgr2=0x55c4725efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 -- 172.21.15.22:0/632884490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4726079f0 con 0x55c4725f0c60 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.627+0000 7fc08ffff640 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725f0c60 0x55c4725e85a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc088013430 tx=0x7fc088005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd31bb9a6363bd8a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc0757fa640 1 -- 172.21.15.22:0/632884490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088023b60 con 0x55c4725f0c60 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc0757fa640 1 -- 172.21.15.22:0/632884490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc088023d00 con 0x55c4725f0c60 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc0757fa640 1 -- 172.21.15.22:0/632884490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08802b640 con 0x55c4725f0c60 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 -- 172.21.15.22:0/632884490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725f0c60 msgr2=0x55c4725e85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725f0c60 0x55c4725e85a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc088013430 tx=0x7fc088005bd0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 -- 172.21.15.22:0/632884490 shutdown_connections 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725f0c60 0x55c4725e85a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725e8080 0x55c4725f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 --2- 172.21.15.22:0/632884490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e7760 0x55c4725efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 -- 172.21.15.22:0/632884490 >> 172.21.15.22:0/632884490 conn(0x55c4725e1e90 msgr2=0x55c4725ed690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 -- 172.21.15.22:0/632884490 shutdown_connections 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.628+0000 7fc098dd5740 1 -- 172.21.15.22:0/632884490 wait complete. 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 -- start start 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 Processor -- start 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725f8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e8080 0x55c4725f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901aef10 con 0x55c4725e7760 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901b01f0 con 0x55c4725e8080 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901af530 con 0x55c4725f0c60 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc098dd5740 10 client.0 _ll_register_callbacks cb 0x7fc0800bcc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc08f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc08f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725f8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46468/0 (socket says 172.21.15.22:46468) 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc08f7fe640 1 -- 172.21.15.22:0/382770472 learned_addr learned my addr 172.21.15.22:0/382770472 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc08effd640 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e8080 0x55c4725f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.629+0000 7fc08ffff640 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc08f7fe640 1 -- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 msgr2=0x55c4725f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc08f7fe640 1 -- 172.21.15.22:0/382770472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e8080 msgr2=0x55c4725f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e8080 0x55c4725f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc08f7fe640 1 -- 172.21.15.22:0/382770472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4725f5210 con 0x55c4725e7760 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc08f7fe640 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725f8130 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc0800b9430 tx=0x7fc0800056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb45e912c8b3f7d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc06effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc06effd640 1 -- 172.21.15.22:0/382770472 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08004b040 con 0x55c4725e7760 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc06effd640 1 -- 172.21.15.22:0/382770472 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fc0800bd430 con 0x55c4725e7760 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc088023d00 con 0x55c4725e7760 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc06effd640 1 -- 172.21.15.22:0/382770472 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0800c56c0 con 0x55c4725e7760 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc098dd5740 10 client.19746 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c4725f5210 con 0x55c4725e7760 2024-03-09T00:07:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc06effd640 1 -- 172.21.15.22:0/382770472 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc088023d00 con 0x55c4725e7760 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.630+0000 7fc06effd640 1 -- 172.21.15.22:0/382770472 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0800bc020 con 0x55c4725e7760 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc098dd5740 10 client.19746 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc098dd5740 10 client.19746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc098dd5740 20 client.19746 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc098dd5740 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472657b60 0x55c472659f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc098dd5740 4 client.19746 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c472552690 con 0x55c472657b60 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc08effd640 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472657b60 0x55c472659f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc08effd640 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472657b60 0x55c472659f50 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d765cbfd5aed870 server_cookie=4d4c8e0b84d247b6 in_seq=0 out_seq=0 2024-03-09T00:07:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.631+0000 7fc06effd640 10 client.19746 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc06effd640 1 -- 172.21.15.22:0/382770472 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c472552690 con 0x55c472657b60 2024-03-09T00:07:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc06effd640 10 client.19746 handle_command_reply: tid=0 2024-03-09T00:07:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 client.19746 shutdown 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 msgr2=0x55c4725f8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725f8130 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc0800b9430 tx=0x7fc0800056e0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 shutdown_connections 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c472657b60 0x55c472659f50 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4725f0c60 0x55c4725f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4725e8080 0x55c4725f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 --2- 172.21.15.22:0/382770472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4725e7760 0x55c4725f8130 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc06effd640 0 client.19746 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 >> 172.21.15.22:0/382770472 conn(0x55c4725e1e90 msgr2=0x55c4725e85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 shutdown_connections 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 1 -- 172.21.15.22:0/382770472 wait complete. 2024-03-09T00:07:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.633+0000 7fc098dd5740 20 client.19746 trim_cache size 0 max 16384 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "257430e8-2954-4382-ac05-beb17bc768c1": { 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "257430e8-2954-4382-ac05-beb17bc768c1", 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:07:31.634 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc0780a2970 msgr2=0x7fc0780a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc0780a2970 0x7fc0780a4e30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0880049d0 tx=0x7fc088004ab0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 msgr2=0x7fc090199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc090199db0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc080006010 tx=0x7fc08002cb30 comp rx=0 tx=0).stop 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 shutdown_connections 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc0780a2970 0x7fc0780a4e30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc09019e670 0x7fc09019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0901054c0 0x7fc09019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 --2- 172.21.15.22:0/2329561196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0901003f0 0x7fc090199db0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 >> 172.21.15.22:0/2329561196 conn(0x7fc090076fa0 msgr2=0x7fc090069f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 shutdown_connections 2024-03-09T00:07:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:07:31.634+0000 7fc0961f1640 1 -- 172.21.15.22:0/2329561196 wait complete. 2024-03-09T00:07:31.639 INFO:tasks.cephfs.filesystem:scrub status for tag:257430e8-2954-4382-ac05-beb17bc768c1 - {'path': '/', 'tag': '257430e8-2954-4382-ac05-beb17bc768c1', 'options': 'recursive,force'} 2024-03-09T00:07:31.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1274751 9.22 MB/sec execute 579 sec latency 19.698 ms 2024-03-09T00:07:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:31 smithi062 ceph-mon[36129]: pgmap v410: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-03-09T00:07:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:31 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:07:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:31 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:31 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:07:32.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:31 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:32.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:31 smithi192 ceph-mon[35995]: pgmap v410: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-03-09T00:07:32.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:31 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:07:32.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:31 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:32.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:31 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:07:32.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:31 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:31 smithi022 ceph-mon[31894]: pgmap v410: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-03-09T00:07:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:31 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:07:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:31 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:31 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:07:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:31 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:07:32.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1275728 9.21 MB/sec execute 580 sec latency 19.264 ms 2024-03-09T00:07:33.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1277946 9.21 MB/sec execute 581 sec latency 16.282 ms 2024-03-09T00:07:34.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:34 smithi192 ceph-mon[35995]: pgmap v411: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-03-09T00:07:34.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:34 smithi022 ceph-mon[31894]: pgmap v411: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-03-09T00:07:34.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:34 smithi062 ceph-mon[36129]: pgmap v411: 129 pgs: 129 active+clean; 94 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-03-09T00:07:34.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1279141 9.21 MB/sec execute 582 sec latency 35.434 ms 2024-03-09T00:07:35.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:35 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:07:35.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:35 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:07:35.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:35 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:07:35.735 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1280947 9.21 MB/sec execute 583 sec latency 25.692 ms 2024-03-09T00:07:36.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:36 smithi192 ceph-mon[35995]: pgmap v412: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-03-09T00:07:36.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:36 smithi022 ceph-mon[31894]: pgmap v412: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.611+0000 7f176326f640 1 -- 172.21.15.22:0/492930820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1095b0 msgr2=0x7f175c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.611+0000 7f176326f640 1 --2- 172.21.15.22:0/492930820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1095b0 0x7f175c109990 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1758009900 tx=0x7f175802bc30 comp rx=0 tx=0).stop 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- 172.21.15.22:0/492930820 shutdown_connections 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 --2- 172.21.15.22:0/492930820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f175c103eb0 0x7f175c110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 --2- 172.21.15.22:0/492930820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c1034f0 0x7f175c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 --2- 172.21.15.22:0/492930820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1095b0 0x7f175c109990 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- 172.21.15.22:0/492930820 >> 172.21.15.22:0/492930820 conn(0x7f175c0fefa0 msgr2=0x7f175c1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- 172.21.15.22:0/492930820 shutdown_connections 2024-03-09T00:07:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- 172.21.15.22:0/492930820 wait complete. 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 Processor -- start 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- start start 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1034f0 0x7f175c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f175c103eb0 0x7f175c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c1095b0 0x7f175c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c113870 con 0x7f175c1034f0 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1136f0 con 0x7f175c103eb0 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.612+0000 7f176326f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1139f0 con 0x7f175c1095b0 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1034f0 0x7f175c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1034f0 0x7f175c19f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46492/0 (socket says 172.21.15.22:46492) 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 -- 172.21.15.22:0/1668003540 learned_addr learned my addr 172.21.15.22:0/1668003540 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176180c640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c1095b0 0x7f175c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176080a640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f175c103eb0 0x7f175c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 -- 172.21.15.22:0/1668003540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c1095b0 msgr2=0x7f175c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c1095b0 0x7f175c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 -- 172.21.15.22:0/1668003540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f175c103eb0 msgr2=0x7f175c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f175c103eb0 0x7f175c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 -- 172.21.15.22:0/1668003540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c19a230 con 0x7f175c1034f0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176100b640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1034f0 0x7f175c19f900 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1758005940 tx=0x7f17580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f17527fc640 1 -- 172.21.15.22:0/1668003540 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758041070 con 0x7f175c1034f0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c19a4c0 con 0x7f175c1034f0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f17527fc640 1 -- 172.21.15.22:0/1668003540 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f175802ccc0 con 0x7f175c1034f0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f17527fc640 1 -- 172.21.15.22:0/1668003540 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758003d80 con 0x7f175c1034f0 2024-03-09T00:07:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.613+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c1a67c0 con 0x7f175c1034f0 2024-03-09T00:07:36.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:36 smithi062 ceph-mon[36129]: pgmap v412: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 378 op/s 2024-03-09T00:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.616+0000 7f17527fc640 1 -- 172.21.15.22:0/1668003540 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1758044070 con 0x7f175c1034f0 2024-03-09T00:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.616+0000 7f1765e4d740 1 Processor -- start 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.616+0000 7f1765e4d740 1 -- start start 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c80d40 0x5636e7c7fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f17527fc640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17480a2a10 0x7f17480a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c7ffd0 0x5636e7c80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c89550 0x5636e7c8b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f1765e4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e7be45f0 con 0x5636e7c7ffd0 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f1765e4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e7b8d930 con 0x5636e7c89550 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f1765e4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e7be0690 con 0x5636e7c80d40 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176080a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c7ffd0 0x5636e7c80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c80d40 0x5636e7c7fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176100b640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17480a2a10 0x7f17480a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c80d40 0x5636e7c7fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:46008/0 (socket says 172.21.15.22:46008) 2024-03-09T00:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176080a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c7ffd0 0x5636e7c80450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46508/0 (socket says 172.21.15.22:46508) 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 -- 172.21.15.22:0/1579679458 learned_addr learned my addr 172.21.15.22:0/1579679458 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f17527fc640 1 -- 172.21.15.22:0/1668003540 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f17580f0d00 con 0x7f175c1034f0 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c89550 0x5636e7c8b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 -- 172.21.15.22:0/1579679458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c89550 msgr2=0x5636e7c8b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c89550 0x5636e7c8b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 -- 172.21.15.22:0/1579679458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c7ffd0 msgr2=0x5636e7c80450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c7ffd0 0x5636e7c80450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 -- 172.21.15.22:0/1579679458 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e78f9210 con 0x5636e7c80d40 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176080a640 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c7ffd0 0x5636e7c80450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176100b640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17480a2a10 0x7f17480a4ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17580bcd70 tx=0x7f175804b770 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.617+0000 7f176180c640 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c80d40 0x5636e7c7fa90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1754013b90 tx=0x7f175400a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8351acc941091f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f17477fe640 1 -- 172.21.15.22:0/1579679458 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754013020 con 0x5636e7c80d40 2024-03-09T00:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f17477fe640 1 -- 172.21.15.22:0/1579679458 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1754003e50 con 0x5636e7c80d40 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f17477fe640 1 -- 172.21.15.22:0/1579679458 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540046a0 con 0x5636e7c80d40 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 -- 172.21.15.22:0/1579679458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c80d40 msgr2=0x5636e7c7fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c80d40 0x5636e7c7fa90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1754013b90 tx=0x7f175400a910 comp rx=0 tx=0).stop 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 -- 172.21.15.22:0/1579679458 shutdown_connections 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c89550 0x5636e7c8b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c7ffd0 0x5636e7c80450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1579679458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c80d40 0x5636e7c7fa90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 -- 172.21.15.22:0/1579679458 >> 172.21.15.22:0/1579679458 conn(0x5636e7c475b0 msgr2=0x5636e7c7acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 -- 172.21.15.22:0/1579679458 shutdown_connections 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.618+0000 7f1765e4d740 1 -- 172.21.15.22:0/1579679458 wait complete. 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 -- start start 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 Processor -- start 2024-03-09T00:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c86540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c86a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c86fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e7c8c220 con 0x5636e7c89550 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e7ba0d90 con 0x5636e7c80d40 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e7c8c3a0 con 0x5636e7c7ffd0 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f1765e4d740 10 client.0 _ll_register_callbacks cb 0x5636e78f8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f176180c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c86540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f176180c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c86540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:46012/0 (socket says 172.21.15.22:46012) 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f176180c640 1 -- 172.21.15.22:0/1202911879 learned_addr learned my addr 172.21.15.22:0/1202911879 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f176180c640 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c86fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.619+0000 7f176080a640 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c86a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f176180c640 1 -- 172.21.15.22:0/1202911879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 msgr2=0x5636e7c86a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f176180c640 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c86a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f176180c640 1 -- 172.21.15.22:0/1202911879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 msgr2=0x5636e7c86fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f176180c640 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c86fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f176180c640 1 -- 172.21.15.22:0/1202911879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e79c5620 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f176180c640 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c86540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f175400ad80 tx=0x7f1754004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278778283fa5df68 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1744ff9640 1 -- 172.21.15.22:0/1202911879 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754004420 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1754003e50 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1744ff9640 1 -- 172.21.15.22:0/1202911879 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f17540045c0 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1744ff9640 1 -- 172.21.15.22:0/1202911879 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17540047c0 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1765e4d740 10 client.29191 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5636e7c984e0 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1744ff9640 1 -- 172.21.15.22:0/1202911879 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1754003e50 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.620+0000 7f1744ff9640 1 -- 172.21.15.22:0/1202911879 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f175402d030 con 0x5636e7c7ffd0 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f1765e4d740 10 client.29191 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f1765e4d740 10 client.29191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f1765e4d740 20 client.29191 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7c99890 0x5636e7c9bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f1765e4d740 4 client.29191 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636e7bf1b30 con 0x5636e7c99890 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f176080a640 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7c99890 0x5636e7c9bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f176080a640 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7c99890 0x5636e7c9bc80 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a926fd07db202a server_cookie=ea2ffaebbf48cced in_seq=0 out_seq=0 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.621+0000 7f1744ff9640 10 client.29191 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1744ff9640 1 -- 172.21.15.22:0/1202911879 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636e7bf1b30 con 0x5636e7c99890 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1744ff9640 10 client.29191 handle_command_reply: tid=0 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 client.29191 shutdown 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 msgr2=0x5636e7c86540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c86540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f175400ad80 tx=0x7f1754004320 comp rx=0 tx=0).stop 2024-03-09T00:07:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 shutdown_connections 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7c99890 0x5636e7c9bc80 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c86fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1744ff9640 0 client.29191 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c86a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 --2- 172.21.15.22:0/1202911879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c86540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 >> 172.21.15.22:0/1202911879 conn(0x5636e7c475b0 msgr2=0x5636e7c7ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 shutdown_connections 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 1 -- 172.21.15.22:0/1202911879 wait complete. 2024-03-09T00:07:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.622+0000 7f1765e4d740 20 client.29191 trim_cache size 0 max 16384 2024-03-09T00:07:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.635+0000 7f1765e4d740 1 Processor -- start 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.635+0000 7f1765e4d740 1 -- start start 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c88c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c7c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c837c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f1765e4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1b2dd0 con 0x5636e7c89550 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f1765e4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1b0970 con 0x5636e7c80d40 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f1765e4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1aea30 con 0x5636e7c7ffd0 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c837c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46516/0 (socket says 172.21.15.22:46516) 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 -- 172.21.15.22:0/270585558 learned_addr learned my addr 172.21.15.22:0/270585558 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176080a640 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c7c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c88c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 -- 172.21.15.22:0/270585558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 msgr2=0x5636e7c88c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c88c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 -- 172.21.15.22:0/270585558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 msgr2=0x5636e7c7c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c7c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 -- 172.21.15.22:0/270585558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e7c7f2a0 con 0x5636e7c89550 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f176180c640 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c837c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f17540042f0 tx=0x7f17540021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6749183a428860f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.636+0000 7f1746ffd640 1 -- 172.21.15.22:0/270585558 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754004390 con 0x5636e7c89550 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1746ffd640 1 -- 172.21.15.22:0/270585558 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1754004530 con 0x5636e7c89550 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1746ffd640 1 -- 172.21.15.22:0/270585558 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754003de0 con 0x5636e7c89550 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 -- 172.21.15.22:0/270585558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 msgr2=0x5636e7c837c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c837c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f17540042f0 tx=0x7f17540021f0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 -- 172.21.15.22:0/270585558 shutdown_connections 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c837c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c7c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 --2- 172.21.15.22:0/270585558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c88c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 -- 172.21.15.22:0/270585558 >> 172.21.15.22:0/270585558 conn(0x5636e7c475b0 msgr2=0x5636e7c7c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 -- 172.21.15.22:0/270585558 shutdown_connections 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 -- 172.21.15.22:0/270585558 wait complete. 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.637+0000 7f1765e4d740 1 -- start start 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 1 Processor -- start 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c90990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c8ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c8f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1af030 con 0x5636e7c89550 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1b0310 con 0x5636e7c80d40 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1af650 con 0x5636e7c7ffd0 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f1765e4d740 10 client.0 _ll_register_callbacks cb 0x7f175400aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176180c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c8f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176080a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c8ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176180c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c8f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46528/0 (socket says 172.21.15.22:46528) 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176180c640 1 -- 172.21.15.22:0/2481284942 learned_addr learned my addr 172.21.15.22:0/2481284942 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176180c640 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c90990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176080a640 1 -- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 msgr2=0x5636e7c90990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176080a640 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c90990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176080a640 1 -- 172.21.15.22:0/2481284942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 msgr2=0x5636e7c8f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176080a640 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c8f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176080a640 1 -- 172.21.15.22:0/2481284942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e7c8fa70 con 0x5636e7c80d40 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.638+0000 7f176180c640 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c8f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f176080a640 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c8ed30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5636e7c9c3e0 tx=0x7f174c01f770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f257381aaf1585e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:07:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1744ff9640 1 -- 172.21.15.22:0/2481284942 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c004460 con 0x5636e7c80d40 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1754004530 con 0x5636e7c80d40 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1744ff9640 1 -- 172.21.15.22:0/2481284942 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f174c012880 con 0x5636e7c80d40 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1744ff9640 1 -- 172.21.15.22:0/2481284942 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f174c030050 con 0x5636e7c80d40 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 10 client.29036 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5636e7cf0c30 con 0x5636e7c80d40 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1744ff9640 1 -- 172.21.15.22:0/2481284942 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1754004530 con 0x5636e7c80d40 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1744ff9640 1 -- 172.21.15.22:0/2481284942 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f174c028420 con 0x5636e7c80d40 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 10 client.29036 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 10 client.29036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 20 client.29036 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7cf1360 0x5636e7cf3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 4 client.29036 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.639+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5636e7bebe80 con 0x5636e7cf1360 2024-03-09T00:07:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.640+0000 7f176180c640 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7cf1360 0x5636e7cf3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.640+0000 7f1744ff9640 10 client.29036 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:36.640+0000 7f176180c640 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7cf1360 0x5636e7cf3780 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2b4a833fa5e9a5 server_cookie=3280ab53f0de3962 in_seq=0 out_seq=0 2024-03-09T00:07:36.739 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1282395 9.21 MB/sec execute 584 sec latency 60.649 ms 2024-03-09T00:07:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.090+0000 7f1744ff9640 1 -- 172.21.15.22:0/2481284942 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5636e7bebe80 con 0x5636e7cf1360 2024-03-09T00:07:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.090+0000 7f1744ff9640 10 client.29036 handle_command_reply: tid=0 2024-03-09T00:07:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.090+0000 7f1765e4d740 1 client.29036 shutdown 2024-03-09T00:07:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.090+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 msgr2=0x5636e7c8ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.090+0000 7f1765e4d740 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c8ed30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5636e7c9c3e0 tx=0x7f174c01f770 comp rx=0 tx=0).stop 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 shutdown_connections 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5636e7cf1360 0x5636e7cf3780 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636e7c89550 0x5636e7c8f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636e7c80d40 0x5636e7c8ed30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1744ff9640 0 client.29036 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 --2- 172.21.15.22:0/2481284942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636e7c7ffd0 0x5636e7c90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 >> 172.21.15.22:0/2481284942 conn(0x5636e7c475b0 msgr2=0x5636e7c86d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 shutdown_connections 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 1 -- 172.21.15.22:0/2481284942 wait complete. 2024-03-09T00:07:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f1765e4d740 20 client.29036 trim_cache size 0 max 16384 2024-03-09T00:07:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:07:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:07:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:07:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:07:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17480a2a10 msgr2=0x7f17480a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f176326f640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17480a2a10 0x7f17480a4ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17580bcd70 tx=0x7f175804b770 comp rx=0 tx=0).stop 2024-03-09T00:07:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1034f0 msgr2=0x7f175c19f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f176326f640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1034f0 0x7f175c19f900 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1758005940 tx=0x7f17580042f0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 shutdown_connections 2024-03-09T00:07:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.091+0000 7f176326f640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17480a2a10 0x7f17480a4ed0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.092+0000 7f176326f640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c1095b0 0x7f175c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.092+0000 7f176326f640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f175c103eb0 0x7f175c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.092+0000 7f176326f640 1 --2- 172.21.15.22:0/1668003540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f175c1034f0 0x7f175c19f900 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.092+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 >> 172.21.15.22:0/1668003540 conn(0x7f175c0fefa0 msgr2=0x7f175c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.092+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 shutdown_connections 2024-03-09T00:07:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:37.092+0000 7f176326f640 1 -- 172.21.15.22:0/1668003540 wait complete. 2024-03-09T00:07:37.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1283463 9.20 MB/sec execute 585 sec latency 304.150 ms 2024-03-09T00:07:38.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:38 smithi022 ceph-mon[31894]: pgmap v413: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-03-09T00:07:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:38 smithi062 ceph-mon[36129]: pgmap v413: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-03-09T00:07:38.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:38 smithi192 ceph-mon[35995]: pgmap v413: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-03-09T00:07:38.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1285569 9.21 MB/sec execute 586 sec latency 53.445 ms 2024-03-09T00:07:39.738 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1286445 9.20 MB/sec execute 587 sec latency 18.868 ms 2024-03-09T00:07:40.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:40 smithi022 ceph-mon[31894]: pgmap v414: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-09T00:07:40.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:40 smithi192 ceph-mon[35995]: pgmap v414: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-09T00:07:40.917 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:40 smithi062 ceph-mon[36129]: pgmap v414: 129 pgs: 129 active+clean; 95 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-09T00:07:40.917 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1288758 9.20 MB/sec execute 588 sec latency 38.154 ms 2024-03-09T00:07:41.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1289841 9.20 MB/sec execute 589 sec latency 25.628 ms 2024-03-09T00:07:42.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:42 smithi022 ceph-mon[31894]: pgmap v415: 129 pgs: 129 active+clean; 85 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-03-09T00:07:42.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:42 smithi062 ceph-mon[36129]: pgmap v415: 129 pgs: 129 active+clean; 85 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-03-09T00:07:42.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:42 smithi192 ceph-mon[35995]: pgmap v415: 129 pgs: 129 active+clean; 85 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-03-09T00:07:42.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1292019 9.20 MB/sec execute 590 sec latency 23.929 ms 2024-03-09T00:07:43.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1293248 9.20 MB/sec execute 591 sec latency 17.123 ms 2024-03-09T00:07:44.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:44 smithi022 ceph-mon[31894]: pgmap v416: 129 pgs: 129 active+clean; 85 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-03-09T00:07:44.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:44 smithi062 ceph-mon[36129]: pgmap v416: 129 pgs: 129 active+clean; 85 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-03-09T00:07:44.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:44 smithi192 ceph-mon[35995]: pgmap v416: 129 pgs: 129 active+clean; 85 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-03-09T00:07:44.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1294681 9.19 MB/sec execute 592 sec latency 93.087 ms 2024-03-09T00:07:45.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1296325 9.20 MB/sec execute 593 sec latency 115.810 ms 2024-03-09T00:07:46.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:46 smithi022 ceph-mon[31894]: pgmap v417: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 343 op/s 2024-03-09T00:07:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:46 smithi062 ceph-mon[36129]: pgmap v417: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 343 op/s 2024-03-09T00:07:46.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:46 smithi192 ceph-mon[35995]: pgmap v417: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 343 op/s 2024-03-09T00:07:46.736 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1297483 9.19 MB/sec execute 594 sec latency 17.611 ms 2024-03-09T00:07:47.737 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1299803 9.20 MB/sec execute 595 sec latency 27.489 ms 2024-03-09T00:07:48.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:48 smithi022 ceph-mon[31894]: pgmap v418: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-03-09T00:07:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:48 smithi062 ceph-mon[36129]: pgmap v418: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-03-09T00:07:48.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:48 smithi192 ceph-mon[35995]: pgmap v418: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-03-09T00:07:48.737 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1300831 9.19 MB/sec execute 596 sec latency 23.297 ms 2024-03-09T00:07:49.737 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1301653 9.18 MB/sec execute 597 sec latency 97.218 ms 2024-03-09T00:07:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 -- 172.21.15.22:0/1621017866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f88068970 msgr2=0x7f4f88068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/1621017866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f88068970 0x7f4f88068dd0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f7402bcd0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 -- 172.21.15.22:0/1621017866 shutdown_connections 2024-03-09T00:07:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/1621017866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f88069310 0x7f4f8810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/1621017866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f88068970 0x7f4f88068dd0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/1621017866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f881073e0 0x7f4f881077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 -- 172.21.15.22:0/1621017866 >> 172.21.15.22:0/1621017866 conn(0x7f4f88077040 msgr2=0x7f4f88077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 -- 172.21.15.22:0/1621017866 shutdown_connections 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.185+0000 7f4f8ea81640 1 -- 172.21.15.22:0/1621017866 wait complete. 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 Processor -- start 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 -- start start 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f88068970 0x7f4f8819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f88069310 0x7f4f8819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f881073e0 0x7f4f881979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f881116b0 con 0x7f4f881073e0 2024-03-09T00:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f88111530 con 0x7f4f88069310 2024-03-09T00:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8ea81640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f88111830 con 0x7f4f88068970 2024-03-09T00:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8d01e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f881073e0 0x7f4f881979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f88068970 0x7f4f8819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8d01e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f881073e0 0x7f4f881979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42900/0 (socket says 172.21.15.22:42900) 2024-03-09T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f8d01e640 1 -- 172.21.15.22:0/3411826177 learned_addr learned my addr 172.21.15.22:0/3411826177 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.186+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f88069310 0x7f4f8819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3411826177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f88069310 msgr2=0x7f4f8819dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f88069310 0x7f4f8819dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f881073e0 msgr2=0x7f4f881979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f881073e0 0x7f4f881979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3411826177 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f88198170 con 0x7f4f88068970 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f88069310 0x7f4f8819dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f88068970 0x7f4f8819d840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00e9f0 tx=0x7f4f7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f85ffb640 1 -- 172.21.15.22:0/3411826177 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00ccc0 con 0x7f4f88068970 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f88198460 con 0x7f4f88068970 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f85ffb640 1 -- 172.21.15.22:0/3411826177 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f7c005020 con 0x7f4f88068970 2024-03-09T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f85ffb640 1 -- 172.21.15.22:0/3411826177 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c0049e0 con 0x7f4f88068970 2024-03-09T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.187+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f881a4700 con 0x7f4f88068970 2024-03-09T00:07:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.190+0000 7f4f85ffb640 1 -- 172.21.15.22:0/3411826177 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4f7c005560 con 0x7f4f88068970 2024-03-09T00:07:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.190+0000 7f4f9165e740 1 Processor -- start 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.190+0000 7f4f9165e740 1 -- start start 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020337d40 0x55a020336a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a020337450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a020342940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f9165e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a02029b5f0 con 0x55a020337d40 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f9165e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a020244930 con 0x55a020340550 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f9165e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a020297690 con 0x55a020336fd0 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f85ffb640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4f780a2a10 0x7f4f780a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a020337450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a020337450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:52374/0 (socket says 172.21.15.22:52374) 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3373868511 learned_addr learned my addr 172.21.15.22:0/3373868511 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f87fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a020342940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020337d40 0x55a020336a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f85ffb640 1 -- 172.21.15.22:0/3411826177 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4f7c0cf1e0 con 0x7f4f88068970 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4f780a2a10 0x7f4f780a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f87fff640 1 -- 172.21.15.22:0/3373868511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 msgr2=0x55a020337450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a020337450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f87fff640 1 -- 172.21.15.22:0/3373868511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020337d40 msgr2=0x55a020336a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020337d40 0x55a020336a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f87fff640 1 -- 172.21.15.22:0/3373868511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01ffb0210 con 0x55a020340550 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.191+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020337d40 0x55a020336a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.192+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a020342940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f7402c210 tx=0x7f4f7402c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d2738249f28ae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.192+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4f780a2a10 0x7f4f780a4ed0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007fd0 tx=0x7f4f800074b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.192+0000 7f4f72ffd640 1 -- 172.21.15.22:0/3373868511 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740042c0 con 0x55a020340550 2024-03-09T00:07:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.192+0000 7f4f72ffd640 1 -- 172.21.15.22:0/3373868511 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f7402ce00 con 0x55a020340550 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.192+0000 7f4f72ffd640 1 -- 172.21.15.22:0/3373868511 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74004900 con 0x55a020340550 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 -- 172.21.15.22:0/3373868511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 msgr2=0x55a020342940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a020342940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f7402c210 tx=0x7f4f7402c6f0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 -- 172.21.15.22:0/3373868511 shutdown_connections 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a020342940 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a020337450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3373868511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020337d40 0x55a020336a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 -- 172.21.15.22:0/3373868511 >> 172.21.15.22:0/3373868511 conn(0x55a0202fe5b0 msgr2=0x55a020331cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 -- 172.21.15.22:0/3373868511 shutdown_connections 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 -- 172.21.15.22:0/3373868511 wait complete. 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.193+0000 7f4f9165e740 1 -- start start 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 1 Processor -- start 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a02033d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a02033dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020340550 0x55a02033e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a020343220 con 0x55a020340550 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a020257d90 con 0x55a020337d40 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0203433a0 con 0x55a020336fd0 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f9165e740 10 client.0 _ll_register_callbacks cb 0x55a01ffafa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f8d01e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020340550 0x55a02033e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a02033d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f87fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a02033dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f8d01e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020340550 0x55a02033e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42918/0 (socket says 172.21.15.22:42918) 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f8d01e640 1 -- 172.21.15.22:0/2117791384 learned_addr learned my addr 172.21.15.22:0/2117791384 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f8c81d640 1 -- 172.21.15.22:0/2117791384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 msgr2=0x55a02033dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.194+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a02033dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f8c81d640 1 -- 172.21.15.22:0/2117791384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020340550 msgr2=0x55a02033e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020340550 0x55a02033e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f8c81d640 1 -- 172.21.15.22:0/2117791384 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a02007c620 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020340550 0x55a02033e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a02033d590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0cc590 tx=0x7f4f7c06af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfed33bc7c22955 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f67fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f67fff640 1 -- 172.21.15.22:0/2117791384 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c01c040 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f67fff640 1 -- 172.21.15.22:0/2117791384 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4f7c0a3750 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f7402ce00 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f67fff640 1 -- 172.21.15.22:0/2117791384 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f7c09f360 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f9165e740 10 client.29209 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a02034f520 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f67fff640 1 -- 172.21.15.22:0/2117791384 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f7402ce00 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.195+0000 7f4f67fff640 1 -- 172.21.15.22:0/2117791384 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f7c0b4090 con 0x55a020336fd0 2024-03-09T00:07:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f9165e740 10 client.29209 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f9165e740 10 client.29209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f9165e740 20 client.29209 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f9165e740 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203509e0 0x55a020352dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f9165e740 4 client.29209 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0202a8b30 con 0x55a0203509e0 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f87fff640 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203509e0 0x55a020352dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f87fff640 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203509e0 0x55a020352dd0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e6c40bf9072275 server_cookie=21a4fb4126469d7 in_seq=0 out_seq=0 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.196+0000 7f4f67fff640 10 client.29209 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.197+0000 7f4f67fff640 1 -- 172.21.15.22:0/2117791384 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a0202a8b30 con 0x55a0203509e0 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.197+0000 7f4f67fff640 10 client.29209 handle_command_reply: tid=0 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.197+0000 7f4f9165e740 1 client.29209 shutdown 2024-03-09T00:07:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 msgr2=0x55a02033d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a02033d590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0cc590 tx=0x7f4f7c06af10 comp rx=0 tx=0).stop 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 shutdown_connections 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203509e0 0x55a020352dd0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020340550 0x55a02033e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a02033dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 --2- 172.21.15.22:0/2117791384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020336fd0 0x55a02033d590 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f67fff640 0 client.29209 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 >> 172.21.15.22:0/2117791384 conn(0x55a0202fe5b0 msgr2=0x55a020333a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 shutdown_connections 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 1 -- 172.21.15.22:0/2117791384 wait complete. 2024-03-09T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.198+0000 7f4f9165e740 20 client.29209 trim_cache size 0 max 16384 2024-03-09T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 Processor -- start 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 -- start start 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a02033fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a020347610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020340550 0x55a020347b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f881ae9c0 con 0x55a020336fd0 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f881ac560 con 0x55a020337d40 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f9165e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f881aa620 con 0x55a020340550 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a02033fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f87fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a020347610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a02033fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42932/0 (socket says 172.21.15.22:42932) 2024-03-09T00:07:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3302775515 learned_addr learned my addr 172.21.15.22:0/3302775515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020340550 0x55a020347b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f87fff640 1 -- 172.21.15.22:0/3302775515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020340550 msgr2=0x55a020347b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020340550 0x55a020347b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f87fff640 1 -- 172.21.15.22:0/3302775515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 msgr2=0x55a02033fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a02033fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f87fff640 1 -- 172.21.15.22:0/3302775515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a020332890 con 0x55a020337d40 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a02033fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.214+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a020347610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f74009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13a3154021407db server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f727fc640 1 -- 172.21.15.22:0/3302775515 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74004240 con 0x55a020337d40 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f727fc640 1 -- 172.21.15.22:0/3302775515 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f740528e0 con 0x55a020337d40 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f727fc640 1 -- 172.21.15.22:0/3302775515 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740035f0 con 0x55a020337d40 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 -- 172.21.15.22:0/3302775515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 msgr2=0x55a020347610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a020347610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f74009250 comp rx=0 tx=0).stop 2024-03-09T00:07:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 -- 172.21.15.22:0/3302775515 shutdown_connections 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020340550 0x55a020347b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020337d40 0x55a020347610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3302775515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a02033fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 -- 172.21.15.22:0/3302775515 >> 172.21.15.22:0/3302775515 conn(0x55a0202fe5b0 msgr2=0x55a020357700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.215+0000 7f4f9165e740 1 -- 172.21.15.22:0/3302775515 shutdown_connections 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 -- 172.21.15.22:0/3302775515 wait complete. 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 -- start start 2024-03-09T00:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 Processor -- start 2024-03-09T00:07:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a020347880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020337d40 0x55a020339e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a02033a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f881aac20 con 0x55a020336fd0 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f881abf00 con 0x55a020340550 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f881ab240 con 0x55a020337d40 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f9165e740 10 client.0 _ll_register_callbacks cb 0x7f4f7c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a020347880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.216+0000 7f4f8c81d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a020347880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42938/0 (socket says 172.21.15.22:42938) 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3898460144 learned_addr learned my addr 172.21.15.22:0/3898460144 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020337d40 0x55a020339e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a02033a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020337d40 msgr2=0x55a020339e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020337d40 0x55a020339e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3898460144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 msgr2=0x55a02033a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a02033a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8c81d640 1 -- 172.21.15.22:0/3898460144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a02033aa50 con 0x55a020336fd0 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8d01e640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a02033a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f8c81d640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a020347880 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c09b140 tx=0x7f4f7c0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb8344ffa07d51a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f67fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f67fff640 1 -- 172.21.15.22:0/3898460144 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c01c070 con 0x55a020336fd0 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f67fff640 1 -- 172.21.15.22:0/3898460144 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f4f7c09f160 con 0x55a020336fd0 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f740528e0 con 0x55a020336fd0 2024-03-09T00:07:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f67fff640 1 -- 172.21.15.22:0/3898460144 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f7c09ada0 con 0x55a020336fd0 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f9165e740 10 client.19782 fetch_fsmap learned FSMap version 35 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.217+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55a0203a7d50 con 0x55a020336fd0 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f67fff640 1 -- 172.21.15.22:0/3898460144 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f740528e0 con 0x55a020336fd0 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f67fff640 1 -- 172.21.15.22:0/3898460144 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f7c097190 con 0x55a020336fd0 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f9165e740 10 client.19782 fetch_fsmap finished waiting for FSMap version 35 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f9165e740 10 client.19782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f9165e740 20 client.19782 populate_metadata read hostname 'smithi022' 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203a83c0 0x55a0203aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f9165e740 4 client.19782 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a0202a2e80 con 0x55a0203a83c0 2024-03-09T00:07:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203a83c0 0x55a0203aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:07:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f67fff640 10 client.19782 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.218+0000 7f4f87fff640 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203a83c0 0x55a0203aa7b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4b5a5fc70b15b4 server_cookie=a1791637a361855d in_seq=0 out_seq=0 2024-03-09T00:07:50.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:50 smithi022 ceph-mon[31894]: pgmap v419: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-03-09T00:07:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f67fff640 1 -- 172.21.15.22:0/3898460144 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a0202a2e80 con 0x55a0203a83c0 2024-03-09T00:07:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f67fff640 10 client.19782 handle_command_reply: tid=0 2024-03-09T00:07:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 client.19782 shutdown 2024-03-09T00:07:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 msgr2=0x55a020347880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a020347880 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c09b140 tx=0x7f4f7c0054d0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 shutdown_connections 2024-03-09T00:07:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a0203a83c0 0x55a0203aa7b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a020340550 0x55a02033a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a020337d40 0x55a020339e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f67fff640 0 client.19782 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 --2- 172.21.15.22:0/3898460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a020336fd0 0x55a020347880 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.608+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 >> 172.21.15.22:0/3898460144 conn(0x55a0202fe5b0 msgr2=0x55a0203317f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.609+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 shutdown_connections 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.609+0000 7f4f9165e740 1 -- 172.21.15.22:0/3898460144 wait complete. 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.609+0000 7f4f9165e740 20 client.19782 trim_cache size 0 max 16384 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:07:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:07:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:07:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:07:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4f780a2a10 msgr2=0x7f4f780a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4f780a2a10 0x7f4f780a4ed0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007fd0 tx=0x7f4f800074b0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f88068970 msgr2=0x7f4f8819d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:07:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f88068970 0x7f4f8819d840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00e9f0 tx=0x7f4f7c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 shutdown_connections 2024-03-09T00:07:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4f780a2a10 0x7f4f780a4ed0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f881073e0 0x7f4f881979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f88069310 0x7f4f8819dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 --2- 172.21.15.22:0/3411826177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f88068970 0x7f4f8819d840 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:07:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 >> 172.21.15.22:0/3411826177 conn(0x7f4f88077040 msgr2=0x7f4f880ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:07:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 shutdown_connections 2024-03-09T00:07:50.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:07:50.610+0000 7f4f8ea81640 1 -- 172.21.15.22:0/3411826177 wait complete. 2024-03-09T00:07:50.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:50 smithi062 ceph-mon[36129]: pgmap v419: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-03-09T00:07:50.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:50 smithi192 ceph-mon[35995]: pgmap v419: 129 pgs: 129 active+clean; 93 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-03-09T00:07:50.737 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1303179 9.18 MB/sec execute 598 sec latency 153.657 ms 2024-03-09T00:07:51.737 INFO:tasks.workunit.client.0.smithi022.stdout: 1 1304155 9.17 MB/sec execute 599 sec latency 18.380 ms 2024-03-09T00:07:52.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:52 smithi022 ceph-mon[31894]: pgmap v420: 129 pgs: 129 active+clean; 98 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-03-09T00:07:52.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:52 smithi062 ceph-mon[36129]: pgmap v420: 129 pgs: 129 active+clean; 98 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-03-09T00:07:52.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:52 smithi192 ceph-mon[35995]: pgmap v420: 129 pgs: 129 active+clean; 98 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-03-09T00:07:52.737 INFO:tasks.workunit.client.0.smithi022.stdout: 1 cleanup 600 sec 2024-03-09T00:07:52.958 INFO:tasks.workunit.client.0.smithi022.stdout: 0 cleanup 600 sec 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: Operation Count AvgLat MaxLat 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: ---------------------------------------- 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: NTCreateX 175595 1.182 340.536 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: Close 128976 0.003 0.090 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: Rename 7435 5.326 195.169 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: Unlink 35463 1.631 70.340 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: Deltree 4 87.053 99.975 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: Mkdir 2 0.007 0.008 2024-03-09T00:07:52.959 INFO:tasks.workunit.client.0.smithi022.stdout: Qpathinfo 159163 1.148 4331.782 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: Qfileinfo 27880 0.002 0.043 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: Qfsinfo 29162 0.145 241.170 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: Sfileinfo 14308 0.021 5.086 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: Find 61496 0.216 152.790 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: WriteX 87458 0.169 809.409 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: ReadX 275205 0.031 118.031 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: LockX 572 1.011 31.327 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: UnlockX 572 0.970 42.089 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: Flush 12306 5.442 248.595 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-03-09T00:07:52.960 INFO:tasks.workunit.client.0.smithi022.stdout:Throughput 9.17207 MB/sec 1 clients 1 procs max_latency=4331.794 ms 2024-03-09T00:07:52.961 INFO:tasks.workunit.client.0.smithi022.stderr:+ dbench 10 2024-03-09T00:07:52.961 INFO:tasks.workunit.client.0.smithi022.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-09T00:07:52.961 INFO:tasks.workunit.client.0.smithi022.stdout: 2024-03-09T00:07:52.961 INFO:tasks.workunit.client.0.smithi022.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-09T00:07:52.962 INFO:tasks.workunit.client.0.smithi022.stdout:8 of 10 processes prepared for launch 0 sec 2024-03-09T00:07:53.060 INFO:tasks.workunit.client.0.smithi022.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-09T00:07:53.060 INFO:tasks.workunit.client.0.smithi022.stdout:releasing clients 2024-03-09T00:07:54.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 334 120.82 MB/sec warmup 1 sec latency 198.187 ms 2024-03-09T00:07:54.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:54 smithi022 ceph-mon[31894]: pgmap v421: 129 pgs: 129 active+clean; 98 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 143 op/s 2024-03-09T00:07:54.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:54 smithi062 ceph-mon[36129]: pgmap v421: 129 pgs: 129 active+clean; 98 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 143 op/s 2024-03-09T00:07:54.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:54 smithi192 ceph-mon[35995]: pgmap v421: 129 pgs: 129 active+clean; 98 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 143 op/s 2024-03-09T00:07:55.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 512 88.77 MB/sec warmup 2 sec latency 387.895 ms 2024-03-09T00:07:56.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 534 61.30 MB/sec warmup 3 sec latency 1387.952 ms 2024-03-09T00:07:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:56 smithi022 ceph-mon[31894]: pgmap v422: 129 pgs: 129 active+clean; 91 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-09T00:07:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:56 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:07:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:56 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:07:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:56 smithi062 ceph-mon[36129]: pgmap v422: 129 pgs: 129 active+clean; 91 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-09T00:07:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:56 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:07:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:56 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:07:56.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:56 smithi192 ceph-mon[35995]: pgmap v422: 129 pgs: 129 active+clean; 91 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-09T00:07:56.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:56 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:07:56.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:56 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:07:57.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 712 50.64 MB/sec warmup 4 sec latency 1574.081 ms 2024-03-09T00:07:57.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:57 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:07:57.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:57 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:07:57.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:57 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:07:57.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:57 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:07:57.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:57 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:07:57.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:57 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:07:57.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:57 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:07:57.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:57 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:07:57.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:57 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:07:58.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 925 42.38 MB/sec warmup 5 sec latency 306.086 ms 2024-03-09T00:07:58.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:07:58 smithi062 ceph-mon[36129]: pgmap v423: 129 pgs: 129 active+clean; 91 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-03-09T00:07:58.879 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:07:58 smithi192 ceph-mon[35995]: pgmap v423: 129 pgs: 129 active+clean; 91 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-03-09T00:07:58.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:07:58 smithi022 ceph-mon[31894]: pgmap v423: 129 pgs: 129 active+clean; 91 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-03-09T00:07:59.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 1359 41.48 MB/sec warmup 6 sec latency 196.269 ms 2024-03-09T00:08:00.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 2038 37.21 MB/sec warmup 7 sec latency 234.712 ms 2024-03-09T00:08:00.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:00 smithi062 ceph-mon[36129]: pgmap v424: 129 pgs: 129 active+clean; 100 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-03-09T00:08:00.879 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:00 smithi192 ceph-mon[35995]: pgmap v424: 129 pgs: 129 active+clean; 100 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-03-09T00:08:00.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:00 smithi022 ceph-mon[31894]: pgmap v424: 129 pgs: 129 active+clean; 100 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-03-09T00:08:01.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 2409 35.85 MB/sec warmup 8 sec latency 249.895 ms 2024-03-09T00:08:01.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:08:01.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:08:01.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 -- 172.21.15.22:0/1758475806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a8101ea0 msgr2=0x7f46a8105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 --2- 172.21.15.22:0/1758475806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a8101ea0 0x7f46a8105ee0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f469002bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 -- 172.21.15.22:0/1758475806 shutdown_connections 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 --2- 172.21.15.22:0/1758475806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a81065c0 0x7f46a810e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 --2- 172.21.15.22:0/1758475806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a8101ea0 0x7f46a8105ee0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 --2- 172.21.15.22:0/1758475806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81014f0 0x7f46a81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 -- 172.21.15.22:0/1758475806 >> 172.21.15.22:0/1758475806 conn(0x7f46a80786b0 msgr2=0x7f46a8078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 -- 172.21.15.22:0/1758475806 shutdown_connections 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 -- 172.21.15.22:0/1758475806 wait complete. 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.758+0000 7f46acba9640 1 Processor -- start 2024-03-09T00:08:01.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46acba9640 1 -- start start 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46acba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a81014f0 0x7f46a819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46acba9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a81065c0 0x7f46a819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46acba9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81979d0 0x7f46a8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46acba9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8111630 con 0x7f46a81065c0 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46acba9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81114b0 con 0x7f46a81979d0 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46acba9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81117b0 con 0x7f46a81014f0 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a81065c0 0x7f46a819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a81065c0 0x7f46a819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39574/0 (socket says 172.21.15.22:39574) 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a81014f0 0x7f46a819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 -- 172.21.15.22:0/3025262276 learned_addr learned my addr 172.21.15.22:0/3025262276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81979d0 0x7f46a8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 -- 172.21.15.22:0/3025262276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a81014f0 msgr2=0x7f46a819d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a81014f0 0x7f46a819d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 -- 172.21.15.22:0/3025262276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81979d0 msgr2=0x7f46a8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81979d0 0x7f46a8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 -- 172.21.15.22:0/3025262276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a81984f0 con 0x7f46a81065c0 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81979d0 0x7f46a8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a81065c0 0x7f46a819dd60 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4690036040 tx=0x7f469002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.759+0000 7f469f7fe640 1 -- 172.21.15.22:0/3025262276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690041070 con 0x7f46a81065c0 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.760+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a8198780 con 0x7f46a81065c0 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.760+0000 7f469f7fe640 1 -- 172.21.15.22:0/3025262276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46900054e0 con 0x7f46a81065c0 2024-03-09T00:08:01.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.760+0000 7f469f7fe640 1 -- 172.21.15.22:0/3025262276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690034440 con 0x7f46a81065c0 2024-03-09T00:08:01.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.760+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a81a4970 con 0x7f46a81065c0 2024-03-09T00:08:01.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.762+0000 7f469f7fe640 1 -- 172.21.15.22:0/3025262276 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4690044070 con 0x7f46a81065c0 2024-03-09T00:08:01.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.762+0000 7f46af784740 1 Processor -- start 2024-03-09T00:08:01.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.762+0000 7f469f7fe640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46940a2970 0x7f46940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.762+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46940a2970 0x7f46940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.762+0000 7f46af784740 1 -- start start 2024-03-09T00:08:01.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.762+0000 7f469f7fe640 1 -- 172.21.15.22:0/3025262276 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f46900f0f70 con 0x7f46a81065c0 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46af784740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb72760 0x5612beb72b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46af784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb7b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46af784740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb7bc60 0x5612beb7e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46af784740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bead5e00 con 0x5612beb7bc60 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46af784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bea7eb10 con 0x5612beb73080 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46af784740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bead1ea0 con 0x5612beb72760 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb72760 0x5612beb72b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb7b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35586/0 (socket says 172.21.15.22:35586) 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1336764298 learned_addr learned my addr 172.21.15.22:0/1336764298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb7bc60 0x5612beb7e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46940a2970 0x7f46940a4e30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4698005fd0 tx=0x7f46980107a0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1336764298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb72760 msgr2=0x5612beb72b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb72760 0x5612beb72b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1336764298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb7bc60 msgr2=0x5612beb7e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb7bc60 0x5612beb7e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1336764298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612be8b47d0 con 0x5612beb73080 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb7b720 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f46900b97f0 tx=0x7f46900bd160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7648caf1711949a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.763+0000 7f469cff9640 1 -- 172.21.15.22:0/1336764298 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469004b070 con 0x5612beb73080 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f469cff9640 1 -- 172.21.15.22:0/1336764298 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4690004c80 con 0x5612beb73080 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f469cff9640 1 -- 172.21.15.22:0/1336764298 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900b9a00 con 0x5612beb73080 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 -- 172.21.15.22:0/1336764298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 msgr2=0x5612beb7b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb7b720 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f46900b97f0 tx=0x7f46900bd160 comp rx=0 tx=0).stop 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 -- 172.21.15.22:0/1336764298 shutdown_connections 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb7bc60 0x5612beb7e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb7b720 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 --2- 172.21.15.22:0/1336764298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb72760 0x5612beb72b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 -- 172.21.15.22:0/1336764298 >> 172.21.15.22:0/1336764298 conn(0x5612beb6ce90 msgr2=0x5612beb6d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 -- 172.21.15.22:0/1336764298 shutdown_connections 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.764+0000 7f46af784740 1 -- 172.21.15.22:0/1336764298 wait complete. 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 -- start start 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 Processor -- start 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb7b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb78800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb78d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb7e810 con 0x5612beb73080 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bea920a0 con 0x5612beb72760 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb7e990 con 0x5612beb7bc60 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46af784740 10 client.0 _ll_register_callbacks cb 0x5612bea91a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb78800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb78800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39592/0 (socket says 172.21.15.22:39592) 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb7b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1144725157 learned_addr learned my addr 172.21.15.22:0/1144725157 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:01.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb78d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 msgr2=0x5612beb78d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb78d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1144725157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 msgr2=0x5612beb7b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb7b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1144725157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612be7ea3f0 con 0x5612beb73080 2024-03-09T00:08:01.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a659c640 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb7b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb78800 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f46900bd090 tx=0x7f46900bb880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b164694fa8f65fa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.765+0000 7f468dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f468dffb640 1 -- 172.21.15.22:0/1144725157 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900d5070 con 0x5612beb73080 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f468dffb640 1 -- 172.21.15.22:0/1144725157 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f46900dd050 con 0x5612beb73080 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4690004c80 con 0x5612beb73080 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f468dffb640 1 -- 172.21.15.22:0/1144725157 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469004b050 con 0x5612beb73080 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 10 client.19794 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:01.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5612be7aa0f0 con 0x5612beb73080 2024-03-09T00:08:01.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f468dffb640 1 -- 172.21.15.22:0/1144725157 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4690004c80 con 0x5612beb73080 2024-03-09T00:08:01.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f468dffb640 1 -- 172.21.15.22:0/1144725157 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f46900bd5f0 con 0x5612beb73080 2024-03-09T00:08:01.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 10 client.19794 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:01.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 10 client.19794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:01.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 20 client.19794 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:01.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612beb8b300 0x5612beb8d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 4 client.19794 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612beae3340 con 0x5612beb8b300 2024-03-09T00:08:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.766+0000 7f46a659c640 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612beb8b300 0x5612beb8d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.767+0000 7f468dffb640 10 client.19794 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.767+0000 7f46a659c640 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612beb8b300 0x5612beb8d6f0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67509c6a1d82f39 server_cookie=691ed8442b9703ba in_seq=0 out_seq=0 2024-03-09T00:08:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f468dffb640 1 -- 172.21.15.22:0/1144725157 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612beae3340 con 0x5612beb8b300 2024-03-09T00:08:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f468dffb640 10 client.19794 handle_command_reply: tid=0 2024-03-09T00:08:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 client.19794 shutdown 2024-03-09T00:08:01.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 msgr2=0x5612beb78800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb78800 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f46900bd090 tx=0x7f46900bb880 comp rx=0 tx=0).stop 2024-03-09T00:08:01.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 shutdown_connections 2024-03-09T00:08:01.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612beb8b300 0x5612beb8d6f0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb78d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb78800 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 --2- 172.21.15.22:0/1144725157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb7b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f468dffb640 0 client.19794 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:01.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 >> 172.21.15.22:0/1144725157 conn(0x5612beb6ce90 msgr2=0x5612beb6ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:01.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 shutdown_connections 2024-03-09T00:08:01.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.769+0000 7f46af784740 1 -- 172.21.15.22:0/1144725157 wait complete. 2024-03-09T00:08:01.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.770+0000 7f46af784740 20 client.19794 trim_cache size 0 max 16384 2024-03-09T00:08:01.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 Processor -- start 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 -- start start 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb77610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81aea70 con 0x5612beb73080 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81ac5b0 con 0x5612beb72760 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46af784740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81aa670 con 0x5612beb7bc60 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46a5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb770d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39608/0 (socket says 172.21.15.22:39608) 2024-03-09T00:08:01.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.787+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1917273355 learned_addr learned my addr 172.21.15.22:0/1917273355 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a659c640 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb77610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1917273355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 msgr2=0x5612beb77610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb77610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1917273355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 msgr2=0x5612beb877e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb877e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a5d9b640 1 -- 172.21.15.22:0/1917273355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612beb72eb0 con 0x5612beb73080 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb770d0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f46900b9d60 tx=0x7f46900bd160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf6eaa2c37512c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f468ffff640 1 -- 172.21.15.22:0/1917273355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900d5070 con 0x5612beb73080 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f468ffff640 1 -- 172.21.15.22:0/1917273355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4690004c80 con 0x5612beb73080 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.788+0000 7f468ffff640 1 -- 172.21.15.22:0/1917273355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900bced0 con 0x5612beb73080 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 -- 172.21.15.22:0/1917273355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 msgr2=0x5612beb770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb770d0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f46900b9d60 tx=0x7f46900bd160 comp rx=0 tx=0).stop 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 -- 172.21.15.22:0/1917273355 shutdown_connections 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb77610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb73080 0x5612beb770d0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 --2- 172.21.15.22:0/1917273355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb72760 0x5612beb877e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 -- 172.21.15.22:0/1917273355 >> 172.21.15.22:0/1917273355 conn(0x5612beb6ce90 msgr2=0x5612beb787c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 -- 172.21.15.22:0/1917273355 shutdown_connections 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 -- 172.21.15.22:0/1917273355 wait complete. 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 -- start start 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.789+0000 7f46af784740 1 Processor -- start 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb72760 0x5612beb82790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb80d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81aac70 con 0x5612beb72760 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81abf50 con 0x5612beb73080 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81ab290 con 0x5612beb7bc60 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 10 client.0 _ll_register_callbacks cb 0x7f469800bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:01.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb72760 0x5612beb82790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb72760 0x5612beb82790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39612/0 (socket says 172.21.15.22:39612) 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 -- 172.21.15.22:0/1207428726 learned_addr learned my addr 172.21.15.22:0/1207428726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a6d9d640 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb80d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 -- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 msgr2=0x5612beb80d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb80d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 -- 172.21.15.22:0/1207428726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 msgr2=0x5612beb82cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb82cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 -- 172.21.15.22:0/1207428726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612beb81450 con 0x5612beb72760 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46a659c640 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb72760 0x5612beb82790 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f46a0003c80 tx=0x7f46a0012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332e0f9bdd4e0ee4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f468dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:01.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f468dffb640 1 -- 172.21.15.22:0/1207428726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0004d20 con 0x5612beb72760 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f468dffb640 1 -- 172.21.15.22:0/1207428726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f46a001d940 con 0x5612beb72760 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4690004c80 con 0x5612beb72760 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f468dffb640 1 -- 172.21.15.22:0/1207428726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46a0031050 con 0x5612beb72760 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 10 client.19800 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.790+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5612bebe21e0 con 0x5612beb72760 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f468dffb640 1 -- 172.21.15.22:0/1207428726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4690004c80 con 0x5612beb72760 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f468dffb640 1 -- 172.21.15.22:0/1207428726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f46a0005020 con 0x5612beb72760 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46af784740 10 client.19800 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46af784740 10 client.19800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46af784740 20 client.19800 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46af784740 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612bebe2920 0x5612bebe4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46af784740 4 client.19800 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5612beadd690 con 0x5612bebe2920 2024-03-09T00:08:01.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612bebe2920 0x5612bebe4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f468dffb640 10 client.19800 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:01.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.791+0000 7f46a5d9b640 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612bebe2920 0x5612bebe4d40 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2496d9adc5367ff server_cookie=5c3f0b5eed0fe2d in_seq=0 out_seq=0 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f468dffb640 1 -- 172.21.15.22:0/1207428726 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5612beadd690 con 0x5612bebe2920 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f468dffb640 10 client.19800 handle_command_reply: tid=0 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 client.19800 shutdown 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb72760 msgr2=0x5612beb82790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb72760 0x5612beb82790 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f46a0003c80 tx=0x7f46a0012e20 comp rx=0 tx=0).stop 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 shutdown_connections 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5612bebe2920 0x5612bebe4d40 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5612beb7bc60 0x5612beb80d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612beb73080 0x5612beb82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f468dffb640 0 client.19800 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:01.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 --2- 172.21.15.22:0/1207428726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5612beb72760 0x5612beb82790 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 >> 172.21.15.22:0/1207428726 conn(0x5612beb6ce90 msgr2=0x5612beb787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 shutdown_connections 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 1 -- 172.21.15.22:0/1207428726 wait complete. 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.795+0000 7f46af784740 20 client.19800 trim_cache size 0 max 16384 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:08:01.796 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:08:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46940a2970 msgr2=0x7f46940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46940a2970 0x7f46940a4e30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4698005fd0 tx=0x7f46980107a0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a81065c0 msgr2=0x7f46a819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a81065c0 0x7f46a819dd60 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4690036040 tx=0x7f469002cbf0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 shutdown_connections 2024-03-09T00:08:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46940a2970 0x7f46940a4e30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81979d0 0x7f46a8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46a81065c0 0x7f46a819dd60 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 --2- 172.21.15.22:0/3025262276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a81014f0 0x7f46a819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 >> 172.21.15.22:0/3025262276 conn(0x7f46a80786b0 msgr2=0x7f46a80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 shutdown_connections 2024-03-09T00:08:01.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.796+0000 7f46acba9640 1 -- 172.21.15.22:0/3025262276 wait complete. 2024-03-09T00:08:01.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:08:01.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:08:01.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 -- 172.21.15.22:0/4232922640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170006a5c0 msgr2=0x7f1700105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 --2- 172.21.15.22:0/4232922640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170006a5c0 0x7f1700105f30 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009900 tx=0x7f16f402bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:01.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 -- 172.21.15.22:0/4232922640 shutdown_connections 2024-03-09T00:08:01.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 --2- 172.21.15.22:0/4232922640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1700106610 0x7f170010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 --2- 172.21.15.22:0/4232922640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f170006a5c0 0x7f1700105f30 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 --2- 172.21.15.22:0/4232922640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1700069c10 0x7f1700069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 -- 172.21.15.22:0/4232922640 >> 172.21.15.22:0/4232922640 conn(0x7f1700077290 msgr2=0x7f17000776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 -- 172.21.15.22:0/4232922640 shutdown_connections 2024-03-09T00:08:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.912+0000 7f1706777640 1 -- 172.21.15.22:0/4232922640 wait complete. 2024-03-09T00:08:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 Processor -- start 2024-03-09T00:08:01.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 -- start start 2024-03-09T00:08:01.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1700069c10 0x7f1700100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1700106610 0x7f1700100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1700100ca0 0x7f17001a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700111680 con 0x7f1700106610 2024-03-09T00:08:01.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700111500 con 0x7f1700069c10 2024-03-09T00:08:01.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1706777640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700111800 con 0x7f1700100ca0 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1700106610 0x7f1700100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1700106610 0x7f1700100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39644/0 (socket says 172.21.15.22:39644) 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 -- 172.21.15.22:0/2055216332 learned_addr learned my addr 172.21.15.22:0/2055216332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f1704d14640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1700100ca0 0x7f17001a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16fffff640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1700069c10 0x7f1700100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 -- 172.21.15.22:0/2055216332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1700100ca0 msgr2=0x7f17001a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1700100ca0 0x7f17001a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 -- 172.21.15.22:0/2055216332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1700069c10 msgr2=0x7f1700100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1700069c10 0x7f1700100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.913+0000 7f16ff7fe640 1 -- 172.21.15.22:0/2055216332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17001a6470 con 0x7f1700106610 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.914+0000 7f1704d14640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1700100ca0 0x7f17001a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.914+0000 7f16ff7fe640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1700106610 0x7f1700100760 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f16f4036040 tx=0x7f16f402cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.914+0000 7f16fd7fa640 1 -- 172.21.15.22:0/2055216332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4041070 con 0x7f1700106610 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.914+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17001a6700 con 0x7f1700106610 2024-03-09T00:08:01.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.914+0000 7f16fd7fa640 1 -- 172.21.15.22:0/2055216332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16f4004450 con 0x7f1700106610 2024-03-09T00:08:01.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.914+0000 7f16fd7fa640 1 -- 172.21.15.22:0/2055216332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4003e50 con 0x7f1700106610 2024-03-09T00:08:01.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.914+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17001a6bf0 con 0x7f1700106610 2024-03-09T00:08:01.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.915+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1700102a10 con 0x7f1700106610 2024-03-09T00:08:01.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.916+0000 7f16fd7fa640 1 -- 172.21.15.22:0/2055216332 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f16f4044070 con 0x7f1700106610 2024-03-09T00:08:01.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.917+0000 7f16fd7fa640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16e80a2b40 0x7f16e80a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:01.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.917+0000 7f16fffff640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16e80a2b40 0x7f16e80a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:01.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.917+0000 7f16fd7fa640 1 -- 172.21.15.22:0/2055216332 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f16f40f0ec0 con 0x7f1700106610 2024-03-09T00:08:01.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.917+0000 7f16fffff640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16e80a2b40 0x7f16e80a5000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005d00 tx=0x7f16f000b410 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:01.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:01.921+0000 7f16fd7fa640 1 -- 172.21.15.22:0/2055216332 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f16f40bd2e0 con 0x7f1700106610 2024-03-09T00:08:02.060 INFO:tasks.workunit.client.0.smithi022.stdout: 10 2951 36.56 MB/sec warmup 9 sec latency 68.235 ms 2024-03-09T00:08:02.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.185+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1700109830 con 0x7f1700106610 2024-03-09T00:08:02.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.186+0000 7f16fd7fa640 1 -- 172.21.15.22:0/2055216332 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f16f40c1380 con 0x7f1700106610 2024-03-09T00:08:02.181 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:08:02.181 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:07:54.629831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:08:02.181 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16e80a2b40 msgr2=0x7f16e80a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16e80a2b40 0x7f16e80a5000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005d00 tx=0x7f16f000b410 comp rx=0 tx=0).stop 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1700106610 msgr2=0x7f1700100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1700106610 0x7f1700100760 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f16f4036040 tx=0x7f16f402cbf0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 shutdown_connections 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f16e80a2b40 0x7f16e80a5000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1700100ca0 0x7f17001a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1700106610 0x7f1700100760 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 --2- 172.21.15.22:0/2055216332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1700069c10 0x7f1700100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 >> 172.21.15.22:0/2055216332 conn(0x7f1700077290 msgr2=0x7f1700107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 shutdown_connections 2024-03-09T00:08:02.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.189+0000 7f1706777640 1 -- 172.21.15.22:0/2055216332 wait complete. 2024-03-09T00:08:02.198 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:08:02.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:08:02.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 -- 172.21.15.22:0/3855194907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa40ff940 msgr2=0x7f5aa40ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3855194907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa40ff940 0x7f5aa40ffdc0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009930 tx=0x7f5a9402bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 -- 172.21.15.22:0/3855194907 shutdown_connections 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3855194907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa4100300 0x7f5aa410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3855194907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa40ff940 0x7f5aa40ffdc0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3855194907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa4105950 0x7f5aa4105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 -- 172.21.15.22:0/3855194907 >> 172.21.15.22:0/3855194907 conn(0x7f5aa4076710 msgr2=0x7f5aa40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 -- 172.21.15.22:0/3855194907 shutdown_connections 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.339+0000 7f5aac06f640 1 -- 172.21.15.22:0/3855194907 wait complete. 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 Processor -- start 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 -- start start 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40ff940 0x7f5aa419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa4100300 0x7f5aa419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa4105950 0x7f5aa41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa410f4e0 con 0x7f5aa4100300 2024-03-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa410f360 con 0x7f5aa4105950 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aac06f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa410f660 con 0x7f5aa40ff940 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa960a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa4100300 0x7f5aa419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa960a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa4100300 0x7f5aa419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39660/0 (socket says 172.21.15.22:39660) 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa960a640 1 -- 172.21.15.22:0/3346818861 learned_addr learned my addr 172.21.15.22:0/3346818861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aaa60c640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa4105950 0x7f5aa41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40ff940 0x7f5aa419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa960a640 1 -- 172.21.15.22:0/3346818861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40ff940 msgr2=0x7f5aa419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa960a640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40ff940 0x7f5aa419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa960a640 1 -- 172.21.15.22:0/3346818861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa4105950 msgr2=0x7f5aa41979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.340+0000 7f5aa960a640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa4105950 0x7f5aa41979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.341+0000 7f5aa960a640 1 -- 172.21.15.22:0/3346818861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4198150 con 0x7f5aa4100300 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.341+0000 7f5aa960a640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa4100300 0x7f5aa419dd10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009230 tx=0x7f5a9402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.341+0000 7f5a9affd640 1 -- 172.21.15.22:0/3346818861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94041070 con 0x7f5aa4100300 2024-03-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.341+0000 7f5a9affd640 1 -- 172.21.15.22:0/3346818861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a94004450 con 0x7f5aa4100300 2024-03-09T00:08:02.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.341+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa41983e0 con 0x7f5aa4100300 2024-03-09T00:08:02.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.341+0000 7f5a9affd640 1 -- 172.21.15.22:0/3346818861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94034440 con 0x7f5aa4100300 2024-03-09T00:08:02.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.341+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa41a4690 con 0x7f5aa4100300 2024-03-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.343+0000 7f5a9affd640 1 -- 172.21.15.22:0/3346818861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5a94044070 con 0x7f5aa4100300 2024-03-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.343+0000 7f5aaec4d740 1 Processor -- start 2024-03-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaec4d740 1 -- start start 2024-03-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f39106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f3910f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3910f6a0 0x555f39111a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f390cd360 con 0x555f3910f6a0 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f39069ec0 con 0x555f39106ac0 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f39012d30 con 0x555f391061a0 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f3910f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa960a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3910f6a0 0x555f39111a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f3910f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35666/0 (socket says 172.21.15.22:35666) 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/719876307 learned_addr learned my addr 172.21.15.22:0/719876307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaa60c640 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f39106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5a9affd640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a900a2970 0x7f5a900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/719876307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 msgr2=0x555f39106580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f39106580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aaa60c640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a900a2970 0x7f5a900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/719876307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3910f6a0 msgr2=0x555f39111a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3910f6a0 0x555f39111a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/719876307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f39011bc0 con 0x555f39106ac0 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.344+0000 7f5a9affd640 1 -- 172.21.15.22:0/3346818861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5a940f0b90 con 0x7f5aa4100300 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f3910f160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000bc10 tx=0x7f5aa00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b56c05aab833d89 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaa60c640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a900a2970 0x7f5a900a4e30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00a920 tx=0x7f5a9c004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5a8ffff640 1 -- 172.21.15.22:0/719876307 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0005020 con 0x555f39106ac0 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5a8ffff640 1 -- 172.21.15.22:0/719876307 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aa00047a0 con 0x555f39106ac0 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5a8ffff640 1 -- 172.21.15.22:0/719876307 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0011680 con 0x555f39106ac0 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaec4d740 1 -- 172.21.15.22:0/719876307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 msgr2=0x555f3910f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f3910f160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000bc10 tx=0x7f5aa00107a0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaec4d740 1 -- 172.21.15.22:0/719876307 shutdown_connections 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3910f6a0 0x555f39111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f3910f160 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/719876307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f39106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.345+0000 7f5aaec4d740 1 -- 172.21.15.22:0/719876307 >> 172.21.15.22:0/719876307 conn(0x555f391008d0 msgr2=0x555f39100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 -- 172.21.15.22:0/719876307 shutdown_connections 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 -- 172.21.15.22:0/719876307 wait complete. 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 -- start start 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 Processor -- start 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f3910c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f39106ac0 0x555f3910cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f3910d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f390084a0 con 0x555f39106ac0 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f39065f60 con 0x555f3910f6a0 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f39112490 con 0x555f391061a0 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aaec4d740 10 client.0 _ll_register_callbacks cb 0x555f39063d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aa960a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f39106ac0 0x555f3910cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aa960a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f39106ac0 0x555f3910cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39664/0 (socket says 172.21.15.22:39664) 2024-03-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aa9e0b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f3910c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.346+0000 7f5aa960a640 1 -- 172.21.15.22:0/2963731475 learned_addr learned my addr 172.21.15.22:0/2963731475 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aaa60c640 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f3910d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aa960a640 1 -- 172.21.15.22:0/2963731475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 msgr2=0x555f3910c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aa960a640 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f3910c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aa960a640 1 -- 172.21.15.22:0/2963731475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 msgr2=0x555f3910d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aa960a640 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f3910d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aa960a640 1 -- 172.21.15.22:0/2963731475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f38d7e3f0 con 0x555f39106ac0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aa960a640 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f39106ac0 0x555f3910cbd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5a940b9410 tx=0x7f5a9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd455691316bfbb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5a8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/2963731475 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9404b040 con 0x555f39106ac0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/2963731475 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5a940c5210 con 0x555f39106ac0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aa00047a0 con 0x555f39106ac0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/2963731475 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a940bcd80 con 0x555f39106ac0 2024-03-09T00:08:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aaec4d740 10 client.19818 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x555f38e487d0 con 0x555f39106ac0 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.347+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/2963731475 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aa00047a0 con 0x555f39106ac0 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/2963731475 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a940bc400 con 0x555f39106ac0 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aaec4d740 10 client.19818 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aaec4d740 10 client.19818 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aaec4d740 20 client.19818 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f3911f890 0x555f39121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aaec4d740 4 client.19818 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f39077400 con 0x555f3911f890 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f3911f890 0x555f39121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f3911f890 0x555f39121ca0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5b9b495ef19ac8b server_cookie=3344c90204bed877 in_seq=0 out_seq=0 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.348+0000 7f5a8d7fa640 10 client.19818 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/2963731475 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f39077400 con 0x555f3911f890 2024-03-09T00:08:02.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5a8d7fa640 10 client.19818 handle_command_reply: tid=0 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 client.19818 shutdown 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f39106ac0 msgr2=0x555f3910cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f39106ac0 0x555f3910cbd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5a940b9410 tx=0x7f5a9402bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 shutdown_connections 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f3911f890 0x555f39121ca0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f3910d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f39106ac0 0x555f3910cbd0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/2963731475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f391061a0 0x555f3910c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5a8d7fa640 0 client.19818 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 >> 172.21.15.22:0/2963731475 conn(0x555f391008d0 msgr2=0x555f39102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 shutdown_connections 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 1 -- 172.21.15.22:0/2963731475 wait complete. 2024-03-09T00:08:02.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.349+0000 7f5aaec4d740 20 client.19818 trim_cache size 0 max 16384 2024-03-09T00:08:02.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.369+0000 7f5aaec4d740 1 Processor -- start 2024-03-09T00:08:02.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.370+0000 7f5aaec4d740 1 -- start start 2024-03-09T00:08:02.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.371+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f3910edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.371+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f39106ac0 0x555f39107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.371+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f391098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.371+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41b0b80 con 0x555f391061a0 2024-03-09T00:08:02.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.371+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41ae6c0 con 0x555f3910f6a0 2024-03-09T00:08:02.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.371+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41ac780 con 0x555f39106ac0 2024-03-09T00:08:02.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.372+0000 7f5aa9e0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f3910edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.372+0000 7f5aa9e0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f3910edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39672/0 (socket says 172.21.15.22:39672) 2024-03-09T00:08:02.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.372+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/4294263355 learned_addr learned my addr 172.21.15.22:0/4294263355 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.373+0000 7f5aa960a640 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f39106ac0 0x555f39107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.373+0000 7f5aaa60c640 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f391098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.373+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/4294263355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f39106ac0 msgr2=0x555f39107a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.373+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f39106ac0 0x555f39107a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.373+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/4294263355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 msgr2=0x555f391098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.373+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f391098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.373+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/4294263355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3910abe0 con 0x555f391061a0 2024-03-09T00:08:02.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.374+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f3910edc0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0006010 tx=0x7f5aa000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fddeb4fe590a986 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.375+0000 7f5a8f7fe640 1 -- 172.21.15.22:0/4294263355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000be80 con 0x555f391061a0 2024-03-09T00:08:02.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.375+0000 7f5a8f7fe640 1 -- 172.21.15.22:0/4294263355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aa0005460 con 0x555f391061a0 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.375+0000 7f5a8f7fe640 1 -- 172.21.15.22:0/4294263355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0011440 con 0x555f391061a0 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 -- 172.21.15.22:0/4294263355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 msgr2=0x555f3910edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f3910edc0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0006010 tx=0x7f5aa000b410 comp rx=0 tx=0).stop 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 -- 172.21.15.22:0/4294263355 shutdown_connections 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3910f6a0 0x555f391098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f39106ac0 0x555f39107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/4294263355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f3910edc0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 -- 172.21.15.22:0/4294263355 >> 172.21.15.22:0/4294263355 conn(0x555f391008d0 msgr2=0x555f39106ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 -- 172.21.15.22:0/4294263355 shutdown_connections 2024-03-09T00:08:02.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.376+0000 7f5aaec4d740 1 -- 172.21.15.22:0/4294263355 wait complete. 2024-03-09T00:08:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 -- start start 2024-03-09T00:08:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 Processor -- start 2024-03-09T00:08:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f39115a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f39115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3910f6a0 0x555f39180ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41acd80 con 0x555f391061a0 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41ae060 con 0x555f39106ac0 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41ad3a0 con 0x555f3910f6a0 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.377+0000 7f5aaec4d740 10 client.0 _ll_register_callbacks cb 0x7f5aa000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.378+0000 7f5aaa60c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3910f6a0 0x555f39180ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.378+0000 7f5aaa60c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3910f6a0 0x555f39180ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39614/0 (socket says 172.21.15.22:39614) 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.378+0000 7f5aaa60c640 1 -- 172.21.15.22:0/3507787422 learned_addr learned my addr 172.21.15.22:0/3507787422 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.378+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f39115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.378+0000 7f5aa960a640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f39115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.378+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3910f6a0 msgr2=0x555f39180ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.379+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3910f6a0 0x555f39180ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.379+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/3507787422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 msgr2=0x555f39115fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.379+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f39115fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.379+0000 7f5aa9e0b640 1 -- 172.21.15.22:0/3507787422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f39181490 con 0x555f391061a0 2024-03-09T00:08:02.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.379+0000 7f5aaa60c640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3910f6a0 0x555f39180ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:02.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.379+0000 7f5aa960a640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f39115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:02.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.379+0000 7f5aa9e0b640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f39115a70 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0010ca0 tx=0x7f5aa000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a23ecf550a6370f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.382+0000 7f5a8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.385+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/3507787422 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0005610 con 0x555f391061a0 2024-03-09T00:08:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.385+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/3507787422 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5aa0004b60 con 0x555f391061a0 2024-03-09T00:08:02.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.385+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aa0005460 con 0x555f391061a0 2024-03-09T00:08:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.388+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/3507787422 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa0042050 con 0x555f391061a0 2024-03-09T00:08:02.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.389+0000 7f5aaec4d740 10 client.19824 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:02.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.389+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x555f39191050 con 0x555f391061a0 2024-03-09T00:08:02.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.392+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/3507787422 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aa0005460 con 0x555f391061a0 2024-03-09T00:08:02.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.392+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/3507787422 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5aa0030070 con 0x555f391061a0 2024-03-09T00:08:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.396+0000 7f5aaec4d740 10 client.19824 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.396+0000 7f5aaec4d740 10 client.19824 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:08:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.396+0000 7f5aaec4d740 20 client.19824 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.396+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f39191480 0x555f39193870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.396+0000 7f5aaec4d740 4 client.19824 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:08:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.396+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555f39071750 con 0x555f39191480 2024-03-09T00:08:02.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.399+0000 7f5aa960a640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f39191480 0x555f39193870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.402+0000 7f5aa960a640 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f39191480 0x555f39193870 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8810eb1b7d355f90 server_cookie=643e8fdcfdb55bf4 in_seq=0 out_seq=0 2024-03-09T00:08:02.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.402+0000 7f5a8d7fa640 10 client.19824 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:02.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.402+0000 7f5a8d7fa640 1 -- 172.21.15.22:0/3507787422 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555f39071750 con 0x555f39191480 2024-03-09T00:08:02.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.402+0000 7f5a8d7fa640 10 client.19824 handle_command_reply: tid=0 2024-03-09T00:08:02.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.402+0000 7f5aaec4d740 1 client.19824 shutdown 2024-03-09T00:08:02.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.407+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 msgr2=0x555f39115a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.407+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f39115a70 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0010ca0 tx=0x7f5aa000be80 comp rx=0 tx=0).stop 2024-03-09T00:08:02.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.409+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 shutdown_connections 2024-03-09T00:08:02.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.409+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555f39191480 0x555f39193870 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.409+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3910f6a0 0x555f39180ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.409+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f39106ac0 0x555f39115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.409+0000 7f5aaec4d740 1 --2- 172.21.15.22:0/3507787422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f391061a0 0x555f39115a70 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.409+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 >> 172.21.15.22:0/3507787422 conn(0x555f391008d0 msgr2=0x555f39107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.409+0000 7f5a8d7fa640 0 client.19824 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:02.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.411+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 shutdown_connections 2024-03-09T00:08:02.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.414+0000 7f5aaec4d740 1 -- 172.21.15.22:0/3507787422 wait complete. 2024-03-09T00:08:02.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.415+0000 7f5aaec4d740 20 client.19824 trim_cache size 0 max 16384 2024-03-09T00:08:02.412 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:08:02.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.418+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a900a2970 msgr2=0x7f5a900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.419+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a900a2970 0x7f5a900a4e30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00a920 tx=0x7f5a9c004ab0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.419+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa4100300 msgr2=0x7f5aa419dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.419+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa4100300 0x7f5aa419dd10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009230 tx=0x7f5a9402cc70 comp rx=0 tx=0).stop 2024-03-09T00:08:02.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.421+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 shutdown_connections 2024-03-09T00:08:02.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.421+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5a900a2970 0x7f5a900a4e30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.421+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa4105950 0x7f5aa41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.421+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa4100300 0x7f5aa419dd10 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.421+0000 7f5aac06f640 1 --2- 172.21.15.22:0/3346818861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40ff940 0x7f5aa419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.421+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 >> 172.21.15.22:0/3346818861 conn(0x7f5aa4076710 msgr2=0x7f5aa4102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.422+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 shutdown_connections 2024-03-09T00:08:02.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.422+0000 7f5aac06f640 1 -- 172.21.15.22:0/3346818861 wait complete. 2024-03-09T00:08:02.437 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:08:02.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:08:02.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.625+0000 7fe30f116640 1 -- 172.21.15.22:0/1346342737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 msgr2=0x7fe3081007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.625+0000 7fe30f116640 1 --2- 172.21.15.22:0/1346342737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 0x7fe3081007f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009900 tx=0x7fe2fc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.629+0000 7fe30f116640 1 -- 172.21.15.22:0/1346342737 shutdown_connections 2024-03-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.629+0000 7fe30f116640 1 --2- 172.21.15.22:0/1346342737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3081054e0 0x7fe30810d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.629+0000 7fe30f116640 1 --2- 172.21.15.22:0/1346342737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe308100dc0 0x7fe308104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.629+0000 7fe30f116640 1 --2- 172.21.15.22:0/1346342737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 0x7fe3081007f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.630+0000 7fe30f116640 1 -- 172.21.15.22:0/1346342737 >> 172.21.15.22:0/1346342737 conn(0x7fe308077380 msgr2=0x7fe308077790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.633+0000 7fe30f116640 1 -- 172.21.15.22:0/1346342737 shutdown_connections 2024-03-09T00:08:02.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.633+0000 7fe30f116640 1 -- 172.21.15.22:0/1346342737 wait complete. 2024-03-09T00:08:02.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.634+0000 7fe30f116640 1 Processor -- start 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30f116640 1 -- start start 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30f116640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 0x7fe30819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30f116640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3081054e0 0x7fe30819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30f116640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3081968e0 0x7fe308196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30f116640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081105c0 con 0x7fe3081054e0 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30f116640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe308110440 con 0x7fe3081968e0 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30f116640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe308110740 con 0x7fe308100410 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe307fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3081054e0 0x7fe30819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe307fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3081054e0 0x7fe30819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39702/0 (socket says 172.21.15.22:39702) 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe307fff640 1 -- 172.21.15.22:0/3298307747 learned_addr learned my addr 172.21.15.22:0/3298307747 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.636+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 0x7fe30819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.637+0000 7fe307fff640 1 -- 172.21.15.22:0/3298307747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 msgr2=0x7fe30819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.637+0000 7fe307fff640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 0x7fe30819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.637+0000 7fe307fff640 1 -- 172.21.15.22:0/3298307747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3081968e0 msgr2=0x7fe308196d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.637+0000 7fe307fff640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3081968e0 0x7fe308196d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.637+0000 7fe307fff640 1 -- 172.21.15.22:0/3298307747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308197400 con 0x7fe3081054e0 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.637+0000 7fe307fff640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3081054e0 0x7fe30819cc70 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400d770 tx=0x7fe2f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.638+0000 7fe305ffb640 1 -- 172.21.15.22:0/3298307747 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4017070 con 0x7fe3081054e0 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.638+0000 7fe305ffb640 1 -- 172.21.15.22:0/3298307747 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2f4009c30 con 0x7fe3081054e0 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.638+0000 7fe305ffb640 1 -- 172.21.15.22:0/3298307747 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400f620 con 0x7fe3081054e0 2024-03-09T00:08:02.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.639+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3081976f0 con 0x7fe3081054e0 2024-03-09T00:08:02.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.639+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe308069930 con 0x7fe3081054e0 2024-03-09T00:08:02.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.643+0000 7fe311dcf740 1 Processor -- start 2024-03-09T00:08:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.644+0000 7fe311dcf740 1 -- start start 2024-03-09T00:08:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.644+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9665580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.644+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.644+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9670a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.644+0000 7fe311dcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d962c360 con 0x5651d96651a0 2024-03-09T00:08:02.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.644+0000 7fe311dcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d95c8ec0 con 0x5651d966e6a0 2024-03-09T00:08:02.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.644+0000 7fe311dcf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d9571d30 con 0x5651d9665ac0 2024-03-09T00:08:02.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39640/0 (socket says 172.21.15.22:39640) 2024-03-09T00:08:02.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 -- 172.21.15.22:0/3827803043 learned_addr learned my addr 172.21.15.22:0/3827803043 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe305ffb640 1 -- 172.21.15.22:0/3298307747 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe2f401a070 con 0x7fe3081054e0 2024-03-09T00:08:02.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30d6b3640 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9665580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe307fff640 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9670a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 -- 172.21.15.22:0/3827803043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 msgr2=0x5651d9670a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9670a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 -- 172.21.15.22:0/3827803043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 msgr2=0x5651d9665580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9665580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe30ceb2640 1 -- 172.21.15.22:0/3827803043 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651d9570bc0 con 0x5651d9665ac0 2024-03-09T00:08:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.645+0000 7fe307fff640 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9670a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:08:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.647+0000 7fe305ffb640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2f80a2b70 0x7fe2f80a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.647+0000 7fe305ffb640 1 -- 172.21.15.22:0/3298307747 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe2f40ced50 con 0x7fe3081054e0 2024-03-09T00:08:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.647+0000 7fe30d6b3640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2f80a2b70 0x7fe2f80a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.648+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966e160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009900 tx=0x7fe2fc02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34815e2560c5220 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.648+0000 7fe2f2ffd640 1 -- 172.21.15.22:0/3827803043 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc044020 con 0x5651d9665ac0 2024-03-09T00:08:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.648+0000 7fe2f2ffd640 1 -- 172.21.15.22:0/3827803043 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2fc004240 con 0x5651d9665ac0 2024-03-09T00:08:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.649+0000 7fe311dcf740 1 -- 172.21.15.22:0/3827803043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 msgr2=0x5651d966e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.649+0000 7fe311dcf740 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966e160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009900 tx=0x7fe2fc02c5d0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.651+0000 7fe311dcf740 1 -- 172.21.15.22:0/3827803043 shutdown_connections 2024-03-09T00:08:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.651+0000 7fe311dcf740 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9670a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.651+0000 7fe311dcf740 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966e160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.651+0000 7fe311dcf740 1 --2- 172.21.15.22:0/3827803043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9665580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.651+0000 7fe311dcf740 1 -- 172.21.15.22:0/3827803043 >> 172.21.15.22:0/3827803043 conn(0x5651d965f8d0 msgr2=0x5651d965fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.652+0000 7fe311dcf740 1 -- 172.21.15.22:0/3827803043 shutdown_connections 2024-03-09T00:08:02.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.653+0000 7fe311dcf740 1 -- 172.21.15.22:0/3827803043 wait complete. 2024-03-09T00:08:02.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.653+0000 7fe311dcf740 1 -- start start 2024-03-09T00:08:02.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.653+0000 7fe311dcf740 1 Processor -- start 2024-03-09T00:08:02.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.656+0000 7fe30d6b3640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2f80a2b70 0x7fe2f80a5030 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe30000aca0 tx=0x7fe30000a2b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.657+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d966b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.657+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.657+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d966c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.657+0000 7fe311dcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d95674a0 con 0x5651d96651a0 2024-03-09T00:08:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.657+0000 7fe311dcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d95c4f60 con 0x5651d966e6a0 2024-03-09T00:08:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.657+0000 7fe311dcf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d96714d0 con 0x5651d9665ac0 2024-03-09T00:08:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.657+0000 7fe311dcf740 10 client.0 _ll_register_callbacks cb 0x5651d95c2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.658+0000 7fe307fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.659+0000 7fe307fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39656/0 (socket says 172.21.15.22:39656) 2024-03-09T00:08:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.659+0000 7fe307fff640 1 -- 172.21.15.22:0/2777738070 learned_addr learned my addr 172.21.15.22:0/2777738070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.661+0000 7fe307fff640 1 -- 172.21.15.22:0/2777738070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 msgr2=0x5651d966c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.661+0000 7fe307fff640 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d966c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.661+0000 7fe307fff640 1 -- 172.21.15.22:0/2777738070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 msgr2=0x5651d966b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.661+0000 7fe307fff640 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d966b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.661+0000 7fe307fff640 1 -- 172.21.15.22:0/2777738070 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651d92dd3f0 con 0x5651d9665ac0 2024-03-09T00:08:02.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.661+0000 7fe307fff640 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966bc90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400d740 tx=0x7fe2f4009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3618337b05578d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.661+0000 7fe2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.665+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2777738070 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4021070 con 0x5651d9665ac0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.665+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651d95d6400 con 0x5651d9665ac0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.665+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2777738070 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe2f409a3a0 con 0x5651d9665ac0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.665+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2777738070 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2f40a36f0 con 0x5651d9665ac0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.665+0000 7fe311dcf740 10 client.29269 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.665+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5651d93a77d0 con 0x5651d9665ac0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2777738070 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5651d95d6400 con 0x5651d9665ac0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2777738070 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe2f4099e10 con 0x5651d9665ac0 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe311dcf740 10 client.29269 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe311dcf740 10 client.29269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe311dcf740 20 client.29269 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d967edd0 0x5651d9681290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe311dcf740 4 client.29269 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:02.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.666+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651d95d0750 con 0x5651d967edd0 2024-03-09T00:08:02.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.671+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d967edd0 0x5651d9681290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.673+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d967edd0 0x5651d9681290 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12810dffe37860a2 server_cookie=f32db2e4adc46f20 in_seq=0 out_seq=0 2024-03-09T00:08:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.674+0000 7fe2e7fff640 10 client.29269 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.677+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2777738070 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651d95d0750 con 0x5651d967edd0 2024-03-09T00:08:02.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.678+0000 7fe2e7fff640 10 client.29269 handle_command_reply: tid=0 2024-03-09T00:08:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.678+0000 7fe311dcf740 1 client.29269 shutdown 2024-03-09T00:08:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.679+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 msgr2=0x5651d966bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.679+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966bc90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400d740 tx=0x7fe2f4009f20 comp rx=0 tx=0).stop 2024-03-09T00:08:02.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 shutdown_connections 2024-03-09T00:08:02.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d967edd0 0x5651d9681290 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d966c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d966bc90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2777738070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d966b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 >> 172.21.15.22:0/2777738070 conn(0x5651d965f8d0 msgr2=0x5651d9661620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe2e7fff640 0 client.29269 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:02.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.683+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 shutdown_connections 2024-03-09T00:08:02.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.684+0000 7fe311dcf740 1 -- 172.21.15.22:0/2777738070 wait complete. 2024-03-09T00:08:02.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.684+0000 7fe311dcf740 20 client.29269 trim_cache size 0 max 16384 2024-03-09T00:08:02.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 Processor -- start 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 -- start start 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d966de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d9666a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9666fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081bfc60 con 0x5651d96651a0 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081bd7a0 con 0x5651d966e6a0 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.711+0000 7fe311dcf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081bb860 con 0x5651d9665ac0 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.712+0000 7fe30d6b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9666fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.712+0000 7fe30d6b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9666fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35722/0 (socket says 172.21.15.22:35722) 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.712+0000 7fe30d6b3640 1 -- 172.21.15.22:0/1173326711 learned_addr learned my addr 172.21.15.22:0/1173326711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.712+0000 7fe307fff640 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d9666a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.715+0000 7fe30d6b3640 1 -- 172.21.15.22:0/1173326711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 msgr2=0x5651d9666a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.715+0000 7fe30d6b3640 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d9666a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.715+0000 7fe30d6b3640 1 -- 172.21.15.22:0/1173326711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 msgr2=0x5651d966de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.715+0000 7fe30d6b3640 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d966de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.715+0000 7fe30d6b3640 1 -- 172.21.15.22:0/1173326711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651d96864c0 con 0x5651d966e6a0 2024-03-09T00:08:02.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.715+0000 7fe30d6b3640 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9666fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe30000a980 tx=0x7fe300004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0a24fa41deb38d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.716+0000 7fe2f27fc640 1 -- 172.21.15.22:0/1173326711 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300021070 con 0x5651d966e6a0 2024-03-09T00:08:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.716+0000 7fe2f27fc640 1 -- 172.21.15.22:0/1173326711 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe30001db30 con 0x5651d966e6a0 2024-03-09T00:08:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.716+0000 7fe2f27fc640 1 -- 172.21.15.22:0/1173326711 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30001ccb0 con 0x5651d966e6a0 2024-03-09T00:08:02.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.717+0000 7fe311dcf740 1 -- 172.21.15.22:0/1173326711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 msgr2=0x5651d9666fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.717+0000 7fe311dcf740 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9666fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe30000a980 tx=0x7fe300004ab0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.720+0000 7fe311dcf740 1 -- 172.21.15.22:0/1173326711 shutdown_connections 2024-03-09T00:08:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.720+0000 7fe311dcf740 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d966e6a0 0x5651d9666fc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.720+0000 7fe311dcf740 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d9665ac0 0x5651d9666a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.720+0000 7fe311dcf740 1 --2- 172.21.15.22:0/1173326711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d966de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.720+0000 7fe311dcf740 1 -- 172.21.15.22:0/1173326711 >> 172.21.15.22:0/1173326711 conn(0x5651d965f8d0 msgr2=0x5651d965fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.723+0000 7fe311dcf740 1 -- 172.21.15.22:0/1173326711 shutdown_connections 2024-03-09T00:08:02.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.725+0000 7fe311dcf740 1 -- 172.21.15.22:0/1173326711 wait complete. 2024-03-09T00:08:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.725+0000 7fe311dcf740 1 -- start start 2024-03-09T00:08:02.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.725+0000 7fe311dcf740 1 Processor -- start 2024-03-09T00:08:02.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.726+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9673740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.726+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d9665ac0 0x5651d9673c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe311dcf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d966e6a0 0x5651d96e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe311dcf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081bbe60 con 0x5651d96651a0 2024-03-09T00:08:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe311dcf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081bd140 con 0x5651d9665ac0 2024-03-09T00:08:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe311dcf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3081bc480 con 0x5651d966e6a0 2024-03-09T00:08:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe311dcf740 10 client.0 _ll_register_callbacks cb 0x7fe2fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe30ceb2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9673740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe30ceb2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9673740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39730/0 (socket says 172.21.15.22:39730) 2024-03-09T00:08:02.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.727+0000 7fe30ceb2640 1 -- 172.21.15.22:0/2134562229 learned_addr learned my addr 172.21.15.22:0/2134562229 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.728+0000 7fe30ceb2640 1 -- 172.21.15.22:0/2134562229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d966e6a0 msgr2=0x5651d96e53c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:08:02.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.728+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d966e6a0 0x5651d96e53c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.728+0000 7fe30ceb2640 1 -- 172.21.15.22:0/2134562229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d9665ac0 msgr2=0x5651d9673c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:08:02.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.728+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d9665ac0 0x5651d9673c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.728+0000 7fe30ceb2640 1 -- 172.21.15.22:0/2134562229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651d96e5900 con 0x5651d96651a0 2024-03-09T00:08:02.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.728+0000 7fe30ceb2640 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9673740 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009900 tx=0x7fe2fc02c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf962175d48b05 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.728+0000 7fe2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:02.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.729+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2134562229 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc04d070 con 0x5651d96651a0 2024-03-09T00:08:02.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.729+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2134562229 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe2fc02c880 con 0x5651d96651a0 2024-03-09T00:08:02.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.729+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe30001db30 con 0x5651d96651a0 2024-03-09T00:08:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.729+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2134562229 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2fc044040 con 0x5651d96651a0 2024-03-09T00:08:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.730+0000 7fe311dcf740 10 client.19836 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:02.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.730+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5651d96e6a60 con 0x5651d96651a0 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.731+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2134562229 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2fc04d6f0 con 0x5651d96651a0 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.731+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2134562229 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe2fc003720 con 0x5651d96651a0 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.735+0000 7fe311dcf740 10 client.19836 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.735+0000 7fe311dcf740 10 client.19836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.735+0000 7fe311dcf740 20 client.19836 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.735+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d96e6d50 0x5651d96e9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.735+0000 7fe311dcf740 4 client.19836 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:08:02.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.735+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe30001db30 con 0x5651d96e6d50 2024-03-09T00:08:02.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.737+0000 7fe307fff640 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d96e6d50 0x5651d96e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:02 smithi022 ceph-mon[31894]: pgmap v425: 129 pgs: 129 active+clean; 227 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 18 MiB/s wr, 539 op/s 2024-03-09T00:08:02.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:02 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2055216332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:08:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.742+0000 7fe307fff640 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d96e6d50 0x5651d96e9140 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38153cfb94591006 server_cookie=72f71a3f2ddd9161 in_seq=0 out_seq=0 2024-03-09T00:08:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.743+0000 7fe2e7fff640 10 client.19836 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.743+0000 7fe2e7fff640 1 -- 172.21.15.22:0/2134562229 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe30001db30 con 0x5651d96e6d50 2024-03-09T00:08:02.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.743+0000 7fe2e7fff640 10 client.19836 handle_command_reply: tid=0 2024-03-09T00:08:02.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.743+0000 7fe311dcf740 1 client.19836 shutdown 2024-03-09T00:08:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.743+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 msgr2=0x5651d9673740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.743+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9673740 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009900 tx=0x7fe2fc02c340 comp rx=0 tx=0).stop 2024-03-09T00:08:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 shutdown_connections 2024-03-09T00:08:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5651d96e6d50 0x5651d96e9140 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe2e7fff640 0 client.19836 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651d966e6a0 0x5651d96e53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651d9665ac0 0x5651d9673c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe311dcf740 1 --2- 172.21.15.22:0/2134562229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651d96651a0 0x5651d9673740 secure :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009900 tx=0x7fe2fc02c340 comp rx=0 tx=0).stop 2024-03-09T00:08:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 >> 172.21.15.22:0/2134562229 conn(0x5651d965f8d0 msgr2=0x5651d9666a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.744+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 shutdown_connections 2024-03-09T00:08:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.745+0000 7fe311dcf740 1 -- 172.21.15.22:0/2134562229 wait complete. 2024-03-09T00:08:02.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.747+0000 7fe311dcf740 20 client.19836 trim_cache size 0 max 16384 2024-03-09T00:08:02.745 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:08:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.751+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2f80a2b70 msgr2=0x7fe2f80a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.751+0000 7fe30f116640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2f80a2b70 0x7fe2f80a5030 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe30000aca0 tx=0x7fe30000a2b0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.751+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3081054e0 msgr2=0x7fe30819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.751+0000 7fe30f116640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3081054e0 0x7fe30819cc70 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400d770 tx=0x7fe2f400dc30 comp rx=0 tx=0).stop 2024-03-09T00:08:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.752+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 shutdown_connections 2024-03-09T00:08:02.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.752+0000 7fe30f116640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe2f80a2b70 0x7fe2f80a5030 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.752+0000 7fe30f116640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3081968e0 0x7fe308196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.752+0000 7fe30f116640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3081054e0 0x7fe30819cc70 secure :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400d770 tx=0x7fe2f400dc30 comp rx=0 tx=0).stop 2024-03-09T00:08:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.752+0000 7fe30f116640 1 --2- 172.21.15.22:0/3298307747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe308100410 0x7fe30819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.752+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 >> 172.21.15.22:0/3298307747 conn(0x7fe308077380 msgr2=0x7fe3080fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.753+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 shutdown_connections 2024-03-09T00:08:02.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.753+0000 7fe30f116640 1 -- 172.21.15.22:0/3298307747 wait complete. 2024-03-09T00:08:02.768 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:08:02.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:08:02.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:02 smithi062 ceph-mon[36129]: pgmap v425: 129 pgs: 129 active+clean; 227 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 18 MiB/s wr, 539 op/s 2024-03-09T00:08:02.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:02 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2055216332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:08:02.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:02 smithi192 ceph-mon[35995]: pgmap v425: 129 pgs: 129 active+clean; 227 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 18 MiB/s wr, 539 op/s 2024-03-09T00:08:02.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:02 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2055216332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:08:02.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.962+0000 7f395de44640 1 -- 172.21.15.22:0/4147964532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395810fa10 msgr2=0x7f395810d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.962+0000 7f395de44640 1 --2- 172.21.15.22:0/4147964532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395810fa10 0x7f395810d630 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f3950009ca0 tx=0x7f395002c140 comp rx=0 tx=0).stop 2024-03-09T00:08:02.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.965+0000 7f395de44640 1 -- 172.21.15.22:0/4147964532 shutdown_connections 2024-03-09T00:08:02.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.965+0000 7f395de44640 1 --2- 172.21.15.22:0/4147964532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395811aad0 0x7f395811cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.966+0000 7f395de44640 1 --2- 172.21.15.22:0/4147964532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f395810fe80 0x7f395810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.966+0000 7f395de44640 1 --2- 172.21.15.22:0/4147964532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395810fa10 0x7f395810d630 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.966+0000 7f395de44640 1 -- 172.21.15.22:0/4147964532 >> 172.21.15.22:0/4147964532 conn(0x7f395806c3e0 msgr2=0x7f395806c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.969+0000 7f395de44640 1 -- 172.21.15.22:0/4147964532 shutdown_connections 2024-03-09T00:08:02.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.970+0000 7f395de44640 1 -- 172.21.15.22:0/4147964532 wait complete. 2024-03-09T00:08:02.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.970+0000 7f395de44640 1 Processor -- start 2024-03-09T00:08:02.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.971+0000 7f395de44640 1 -- start start 2024-03-09T00:08:02.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.971+0000 7f395de44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f395810fa10 0x7f39581180b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.971+0000 7f395de44640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395810fe80 0x7f3958111180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.971+0000 7f395de44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395811aad0 0x7f39581116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.971+0000 7f395de44640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39581204d0 con 0x7f395811aad0 2024-03-09T00:08:02.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.971+0000 7f395de44640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3958120350 con 0x7f395810fe80 2024-03-09T00:08:02.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.971+0000 7f395de44640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3958120650 con 0x7f395810fa10 2024-03-09T00:08:02.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.972+0000 7f3957fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395811aad0 0x7f39581116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.972+0000 7f3957fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395811aad0 0x7f39581116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39760/0 (socket says 172.21.15.22:39760) 2024-03-09T00:08:02.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.972+0000 7f3957fff640 1 -- 172.21.15.22:0/1359376323 learned_addr learned my addr 172.21.15.22:0/1359376323 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.973+0000 7f3957fff640 1 -- 172.21.15.22:0/1359376323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f395810fa10 msgr2=0x7f39581180b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.974+0000 7f3956ffd640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395810fe80 0x7f3958111180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.974+0000 7f3957fff640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f395810fa10 0x7f39581180b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.974+0000 7f3957fff640 1 -- 172.21.15.22:0/1359376323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395810fe80 msgr2=0x7f3958111180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.974+0000 7f3957fff640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395810fe80 0x7f3958111180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.974+0000 7f3957fff640 1 -- 172.21.15.22:0/1359376323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3958111cb0 con 0x7f395811aad0 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.974+0000 7f3956ffd640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395810fe80 0x7f3958111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:08:02.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.974+0000 7f3957fff640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395811aad0 0x7f39581116c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c7a0 tx=0x7f394c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.977+0000 7f3954ff9640 1 -- 172.21.15.22:0/1359376323 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c007bb0 con 0x7f395811aad0 2024-03-09T00:08:02.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.977+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3958111fa0 con 0x7f395811aad0 2024-03-09T00:08:02.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.977+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3958115f50 con 0x7f395811aad0 2024-03-09T00:08:02.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.977+0000 7f3954ff9640 1 -- 172.21.15.22:0/1359376323 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f394c007d50 con 0x7f395811aad0 2024-03-09T00:08:02.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.977+0000 7f3954ff9640 1 -- 172.21.15.22:0/1359376323 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00f450 con 0x7f395811aad0 2024-03-09T00:08:02.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.982+0000 7f3954ff9640 1 -- 172.21.15.22:0/1359376323 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f394c00f5f0 con 0x7f395811aad0 2024-03-09T00:08:02.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.982+0000 7f3960a22740 1 Processor -- start 2024-03-09T00:08:02.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.983+0000 7f3954ff9640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39400a2b70 0x7f39400a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.983+0000 7f3956ffd640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39400a2b70 0x7f39400a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.983+0000 7f3954ff9640 1 -- 172.21.15.22:0/1359376323 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f394c0cff20 con 0x7f395811aad0 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.983+0000 7f3960a22740 1 -- start start 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.984+0000 7f3960a22740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.984+0000 7f3960a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3eaa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.984+0000 7f3960a22740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 0x563fd3eaca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.984+0000 7f3960a22740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fd3e68360 con 0x563fd3ea11a0 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.984+0000 7f3960a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fd3e04ec0 con 0x563fd3ea1ac0 2024-03-09T00:08:02.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.984+0000 7f3960a22740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fd3dadd30 con 0x563fd3eaa6a0 2024-03-09T00:08:02.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.985+0000 7f39577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.985+0000 7f39577fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39764/0 (socket says 172.21.15.22:39764) 2024-03-09T00:08:02.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.985+0000 7f39577fe640 1 -- 172.21.15.22:0/283634442 learned_addr learned my addr 172.21.15.22:0/283634442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.986+0000 7f3957fff640 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 0x563fd3eaca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.987+0000 7f3956ffd640 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3eaa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.988+0000 7f39577fe640 1 -- 172.21.15.22:0/283634442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 msgr2=0x563fd3eaca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.988+0000 7f39577fe640 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 0x563fd3eaca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.988+0000 7f39577fe640 1 -- 172.21.15.22:0/283634442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 msgr2=0x563fd3eaa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.988+0000 7f39577fe640 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3eaa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.988+0000 7f39577fe640 1 -- 172.21.15.22:0/283634442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fd3dacbc0 con 0x563fd3ea11a0 2024-03-09T00:08:02.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.988+0000 7f3956ffd640 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3eaa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:02.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.989+0000 7f3956ffd640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39400a2b70 0x7f39400a5030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f39581128f0 tx=0x7f394800b560 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.989+0000 7f39577fe640 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea1580 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f3950037040 tx=0x7f3950032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be7079dc9b07922 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.991+0000 7f393dffb640 1 -- 172.21.15.22:0/283634442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950041020 con 0x563fd3ea11a0 2024-03-09T00:08:02.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.991+0000 7f393dffb640 1 -- 172.21.15.22:0/283634442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3950003970 con 0x563fd3ea11a0 2024-03-09T00:08:02.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.991+0000 7f393dffb640 1 -- 172.21.15.22:0/283634442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950039040 con 0x563fd3ea11a0 2024-03-09T00:08:02.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.992+0000 7f3960a22740 1 -- 172.21.15.22:0/283634442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 msgr2=0x563fd3ea1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:02.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.992+0000 7f3960a22740 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea1580 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f3950037040 tx=0x7f3950032450 comp rx=0 tx=0).stop 2024-03-09T00:08:02.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.992+0000 7f3960a22740 1 -- 172.21.15.22:0/283634442 shutdown_connections 2024-03-09T00:08:02.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.992+0000 7f3960a22740 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 0x563fd3eaca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.992+0000 7f3960a22740 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3eaa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.992+0000 7f3960a22740 1 --2- 172.21.15.22:0/283634442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea1580 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.992+0000 7f3960a22740 1 -- 172.21.15.22:0/283634442 >> 172.21.15.22:0/283634442 conn(0x563fd3e9b8d0 msgr2=0x563fd3e9bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:02.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.996+0000 7f3960a22740 1 -- 172.21.15.22:0/283634442 shutdown_connections 2024-03-09T00:08:02.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.997+0000 7f3960a22740 1 -- 172.21.15.22:0/283634442 wait complete. 2024-03-09T00:08:02.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 -- start start 2024-03-09T00:08:02.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 Processor -- start 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea11a0 0x563fd3ea7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3ea7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fd3da34a0 con 0x563fd3eaa6a0 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fd3e00f60 con 0x563fd3ea1ac0 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fd3ead430 con 0x563fd3ea11a0 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.998+0000 7f3960a22740 10 client.0 _ll_register_callbacks cb 0x563fd3dfed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.999+0000 7f3957fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.999+0000 7f3957fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39780/0 (socket says 172.21.15.22:39780) 2024-03-09T00:08:02.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.999+0000 7f3957fff640 1 -- 172.21.15.22:0/1375550936 learned_addr learned my addr 172.21.15.22:0/1375550936 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:02.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:02.999+0000 7f3956ffd640 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3ea7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.000+0000 7f39577fe640 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea11a0 0x563fd3ea7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.000+0000 7f3957fff640 1 -- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea11a0 msgr2=0x563fd3ea7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.000+0000 7f3957fff640 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea11a0 0x563fd3ea7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.000+0000 7f3957fff640 1 -- 172.21.15.22:0/1375550936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 msgr2=0x563fd3ea7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.000+0000 7f3957fff640 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3ea7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.000+0000 7f3957fff640 1 -- 172.21.15.22:0/1375550936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fd3b193f0 con 0x563fd3eaa6a0 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.000+0000 7f3957fff640 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea80a0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f394c005de0 tx=0x7f394c0a0270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340a3965c654822a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.003+0000 7f3936ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.003+0000 7f3936ffd640 1 -- 172.21.15.22:0/1375550936 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c01d040 con 0x563fd3eaa6a0 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.003+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3950003970 con 0x563fd3eaa6a0 2024-03-09T00:08:02.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.003+0000 7f3936ffd640 1 -- 172.21.15.22:0/1375550936 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f394c0ba8c0 con 0x563fd3eaa6a0 2024-03-09T00:08:03.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.010+0000 7f3936ffd640 1 -- 172.21.15.22:0/1375550936 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f394c0a9050 con 0x563fd3eaa6a0 2024-03-09T00:08:03.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.013+0000 7f3960a22740 10 client.19848 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:03.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.013+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563fd3be37d0 con 0x563fd3eaa6a0 2024-03-09T00:08:03.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3936ffd640 1 -- 172.21.15.22:0/1375550936 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3950003970 con 0x563fd3eaa6a0 2024-03-09T00:08:03.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3936ffd640 1 -- 172.21.15.22:0/1375550936 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f394c09b570 con 0x563fd3eaa6a0 2024-03-09T00:08:03.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3960a22740 10 client.19848 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:03.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3960a22740 10 client.19848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:03.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3960a22740 20 client.19848 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:03.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3960a22740 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3ebad50 0x563fd3ebd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3960a22740 4 client.19848 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:03.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.017+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fd3e12400 con 0x563fd3ebad50 2024-03-09T00:08:03.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.018+0000 7f39577fe640 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3ebad50 0x563fd3ebd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.022+0000 7f39577fe640 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3ebad50 0x563fd3ebd1b0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da35766c6f11312e server_cookie=ce93874db4e52949 in_seq=0 out_seq=0 2024-03-09T00:08:03.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.022+0000 7f3936ffd640 10 client.19848 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:03.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.023+0000 7f3936ffd640 1 -- 172.21.15.22:0/1375550936 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fd3e12400 con 0x563fd3ebad50 2024-03-09T00:08:03.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.023+0000 7f3936ffd640 10 client.19848 handle_command_reply: tid=0 2024-03-09T00:08:03.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.023+0000 7f3960a22740 1 client.19848 shutdown 2024-03-09T00:08:03.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.028+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3eaa6a0 msgr2=0x563fd3ea80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.028+0000 7f3960a22740 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea80a0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f394c005de0 tx=0x7f394c0a0270 comp rx=0 tx=0).stop 2024-03-09T00:08:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.032+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 shutdown_connections 2024-03-09T00:08:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.032+0000 7f3960a22740 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3ebad50 0x563fd3ebd1b0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.032+0000 7f3960a22740 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea80a0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.032+0000 7f3960a22740 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea1ac0 0x563fd3ea7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.032+0000 7f3960a22740 1 --2- 172.21.15.22:0/1375550936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea11a0 0x563fd3ea7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.032+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 >> 172.21.15.22:0/1375550936 conn(0x563fd3e9b8d0 msgr2=0x563fd3e9d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.032+0000 7f3936ffd640 0 client.19848 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.037+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 shutdown_connections 2024-03-09T00:08:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.039+0000 7f3960a22740 1 -- 172.21.15.22:0/1375550936 wait complete. 2024-03-09T00:08:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.040+0000 7f3960a22740 20 client.19848 trim_cache size 0 max 16384 2024-03-09T00:08:03.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3017 33.27 MB/sec warmup 10 sec latency 495.630 ms 2024-03-09T00:08:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 Processor -- start 2024-03-09T00:08:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 -- start start 2024-03-09T00:08:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea1ac0 0x563fd3ec1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440124b0 con 0x563fd3ea11a0 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394400fff0 con 0x563fd3eaa6a0 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.068+0000 7f3960a22740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394400e0b0 con 0x563fd3ea1ac0 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea1ac0 0x563fd3ec1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35760/0 (socket says 172.21.15.22:35760) 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 -- 172.21.15.22:0/3404469120 learned_addr learned my addr 172.21.15.22:0/3404469120 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 -- 172.21.15.22:0/3404469120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea1ac0 msgr2=0x563fd3ec1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 --2- 172.21.15.22:0/3404469120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea1ac0 0x563fd3ec1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 -- 172.21.15.22:0/3404469120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 msgr2=0x563fd3ea9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 --2- 172.21.15.22:0/3404469120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 -- 172.21.15.22:0/3404469120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fd3ea18f0 con 0x563fd3eaa6a0 2024-03-09T00:08:03.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.069+0000 7f3957fff640 1 --2- 172.21.15.22:0/3404469120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea5ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f394c09a3d0 tx=0x7f394c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ad946dad244a62 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.071+0000 7f393d7fa640 1 -- 172.21.15.22:0/3404469120 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c01d040 con 0x563fd3eaa6a0 2024-03-09T00:08:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.072+0000 7f393d7fa640 1 -- 172.21.15.22:0/3404469120 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f394c09c820 con 0x563fd3eaa6a0 2024-03-09T00:08:03.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.072+0000 7f393d7fa640 1 -- 172.21.15.22:0/3404469120 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0b0280 con 0x563fd3eaa6a0 2024-03-09T00:08:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.073+0000 7f3960a22740 1 -- 172.21.15.22:0/3404469120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3eaa6a0 msgr2=0x563fd3ea5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.073+0000 7f3960a22740 1 --2- 172.21.15.22:0/3404469120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea5ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f394c09a3d0 tx=0x7f394c01b000 comp rx=0 tx=0).stop 2024-03-09T00:08:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.074+0000 7f3960a22740 1 -- 172.21.15.22:0/3404469120 shutdown_connections 2024-03-09T00:08:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.074+0000 7f3960a22740 1 --2- 172.21.15.22:0/3404469120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3eaa6a0 0x563fd3ea5ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.074+0000 7f3960a22740 1 --2- 172.21.15.22:0/3404469120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3ea1ac0 0x563fd3ec1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.074+0000 7f3960a22740 1 --2- 172.21.15.22:0/3404469120 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea11a0 0x563fd3ea9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.074+0000 7f3960a22740 1 -- 172.21.15.22:0/3404469120 >> 172.21.15.22:0/3404469120 conn(0x563fd3e9b8d0 msgr2=0x563fd3ea1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.076+0000 7f3960a22740 1 -- 172.21.15.22:0/3404469120 shutdown_connections 2024-03-09T00:08:03.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.076+0000 7f3960a22740 1 -- 172.21.15.22:0/3404469120 wait complete. 2024-03-09T00:08:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.076+0000 7f3960a22740 1 -- start start 2024-03-09T00:08:03.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.077+0000 7f3960a22740 1 Processor -- start 2024-03-09T00:08:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.077+0000 7f3960a22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea11a0 0x563fd3eaf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.077+0000 7f3960a22740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea1ac0 0x563fd3eafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.078+0000 7f3960a22740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 0x563fd3f102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.078+0000 7f3960a22740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394400e6b0 con 0x563fd3ea1ac0 2024-03-09T00:08:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.078+0000 7f3960a22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394400f990 con 0x563fd3ea11a0 2024-03-09T00:08:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.078+0000 7f3960a22740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394400ecd0 con 0x563fd3eaa6a0 2024-03-09T00:08:03.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.078+0000 7f3960a22740 10 client.0 _ll_register_callbacks cb 0x7f3950037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:03.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.080+0000 7f39577fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea11a0 0x563fd3eaf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.080+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea1ac0 0x563fd3eafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.080+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea1ac0 0x563fd3eafd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39798/0 (socket says 172.21.15.22:39798) 2024-03-09T00:08:03.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.080+0000 7f3956ffd640 1 -- 172.21.15.22:0/2739756770 learned_addr learned my addr 172.21.15.22:0/2739756770 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.082+0000 7f3956ffd640 1 -- 172.21.15.22:0/2739756770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 msgr2=0x563fd3f102d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:08:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.082+0000 7f3956ffd640 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 0x563fd3f102d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.082+0000 7f3956ffd640 1 -- 172.21.15.22:0/2739756770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea11a0 msgr2=0x563fd3eaf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.082+0000 7f3956ffd640 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea11a0 0x563fd3eaf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.082+0000 7f3956ffd640 1 -- 172.21.15.22:0/2739756770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fd3f10810 con 0x563fd3ea1ac0 2024-03-09T00:08:03.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.089+0000 7f3956ffd640 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea1ac0 0x563fd3eafd00 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f39480058c0 tx=0x7f3948004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95d1cd5baafa9c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.089+0000 7f3936ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:03.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.090+0000 7f3936ffd640 1 -- 172.21.15.22:0/2739756770 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39480119d0 con 0x563fd3ea1ac0 2024-03-09T00:08:03.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.090+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f394c09c820 con 0x563fd3ea1ac0 2024-03-09T00:08:03.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.090+0000 7f3936ffd640 1 -- 172.21.15.22:0/2739756770 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f3948011b70 con 0x563fd3ea1ac0 2024-03-09T00:08:03.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.090+0000 7f3936ffd640 1 -- 172.21.15.22:0/2739756770 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3948011d70 con 0x563fd3ea1ac0 2024-03-09T00:08:03.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.091+0000 7f3960a22740 10 client.19851 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:03.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.091+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563fd3f11af0 con 0x563fd3ea1ac0 2024-03-09T00:08:03.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.092+0000 7f3936ffd640 1 -- 172.21.15.22:0/2739756770 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f394c09c820 con 0x563fd3ea1ac0 2024-03-09T00:08:03.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.093+0000 7f3936ffd640 1 -- 172.21.15.22:0/2739756770 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3948004cb0 con 0x563fd3ea1ac0 2024-03-09T00:08:03.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.093+0000 7f3960a22740 10 client.19851 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:03.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.093+0000 7f3960a22740 10 client.19851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:03.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.093+0000 7f3960a22740 20 client.19851 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:03.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.093+0000 7f3960a22740 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3f122b0 0x563fd3f146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.093+0000 7f3960a22740 4 client.19851 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:08:03.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.093+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563fd3e0c750 con 0x563fd3f122b0 2024-03-09T00:08:03.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.097+0000 7f39577fe640 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3f122b0 0x563fd3f146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.112+0000 7f39577fe640 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3f122b0 0x563fd3f146d0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e96024c3a8bcb0 server_cookie=209b64d0f185d00 in_seq=0 out_seq=0 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.112+0000 7f3936ffd640 10 client.19851 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.113+0000 7f3936ffd640 1 -- 172.21.15.22:0/2739756770 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563fd3e0c750 con 0x563fd3f122b0 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.113+0000 7f3936ffd640 10 client.19851 handle_command_reply: tid=0 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.113+0000 7f3960a22740 1 client.19851 shutdown 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.113+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea1ac0 msgr2=0x563fd3eafd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.113+0000 7f3960a22740 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea1ac0 0x563fd3eafd00 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f39480058c0 tx=0x7f3948004730 comp rx=0 tx=0).stop 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.114+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 shutdown_connections 2024-03-09T00:08:03.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.114+0000 7f3960a22740 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x563fd3f122b0 0x563fd3f146d0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.114+0000 7f3960a22740 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fd3eaa6a0 0x563fd3f102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.114+0000 7f3960a22740 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fd3ea1ac0 0x563fd3eafd00 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.114+0000 7f3960a22740 1 --2- 172.21.15.22:0/2739756770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fd3ea11a0 0x563fd3eaf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.114+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 >> 172.21.15.22:0/2739756770 conn(0x563fd3e9b8d0 msgr2=0x563fd3ec1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.114+0000 7f3936ffd640 0 client.19851 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:03.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.116+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 shutdown_connections 2024-03-09T00:08:03.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.116+0000 7f3960a22740 1 -- 172.21.15.22:0/2739756770 wait complete. 2024-03-09T00:08:03.113 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:08:03.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.118+0000 7f3960a22740 20 client.19851 trim_cache size 0 max 16384 2024-03-09T00:08:03.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.121+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39400a2b70 msgr2=0x7f39400a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.121+0000 7f395de44640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39400a2b70 0x7f39400a5030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f39581128f0 tx=0x7f394800b560 comp rx=0 tx=0).stop 2024-03-09T00:08:03.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.121+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395811aad0 msgr2=0x7f39581116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.121+0000 7f395de44640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395811aad0 0x7f39581116c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c7a0 tx=0x7f394c00cc60 comp rx=0 tx=0).stop 2024-03-09T00:08:03.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.122+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 shutdown_connections 2024-03-09T00:08:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.122+0000 7f395de44640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f39400a2b70 0x7f39400a5030 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.122+0000 7f395de44640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f395811aad0 0x7f39581116c0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.122+0000 7f395de44640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395810fe80 0x7f3958111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.122+0000 7f395de44640 1 --2- 172.21.15.22:0/1359376323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f395810fa10 0x7f39581180b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.122+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 >> 172.21.15.22:0/1359376323 conn(0x7f395806c3e0 msgr2=0x7f3958119530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.123+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 shutdown_connections 2024-03-09T00:08:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:03.124+0000 7f395de44640 1 -- 172.21.15.22:0/1359376323 wait complete. 2024-03-09T00:08:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.764+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2317755176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b30100440 msgr2=0x7f5b30100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.764+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2317755176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b30100440 0x7f5b30100820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009be0 tx=0x7f5b2802bf10 comp rx=0 tx=0).stop 2024-03-09T00:08:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.764+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2317755176 shutdown_connections 2024-03-09T00:08:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.764+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2317755176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b30105510 0x7f5b3010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.764+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2317755176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b30100df0 0x7f5b30104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.764+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2317755176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b30100440 0x7f5b30100820 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009be0 tx=0x7f5b2802bf10 comp rx=0 tx=0).stop 2024-03-09T00:08:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.764+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2317755176 >> 172.21.15.22:0/2317755176 conn(0x7f5b30077310 msgr2=0x7f5b30077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.767+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2317755176 shutdown_connections 2024-03-09T00:08:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.769+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2317755176 wait complete. 2024-03-09T00:08:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.769+0000 7f5b37fe1640 1 Processor -- start 2024-03-09T00:08:03.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b37fe1640 1 -- start start 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b37fe1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b30105510 0x7f5b3010ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b37fe1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3010d200 0x7f5b30107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b37fe1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b30108370 0x7f5b30114c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b37fe1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300709f0 con 0x7f5b30108370 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b37fe1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b30070870 con 0x7f5b30105510 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b37fe1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b30070b70 con 0x7f5b3010d200 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b3557c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3010d200 0x7f5b30107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.767 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b3557c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3010d200 0x7f5b30107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39742/0 (socket says 172.21.15.22:39742) 2024-03-09T00:08:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.772+0000 7f5b3557c640 1 -- 172.21.15.22:0/2665990764 learned_addr learned my addr 172.21.15.22:0/2665990764 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b30105510 0x7f5b3010ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.768 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b3557c640 1 -- 172.21.15.22:0/2665990764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b30105510 msgr2=0x7f5b3010ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b3557c640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b30105510 0x7f5b3010ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b3557c640 1 -- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b30108370 msgr2=0x7f5b30114c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b3557c640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b30108370 0x7f5b30114c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b3557c640 1 -- 172.21.15.22:0/2665990764 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b301151b0 con 0x7f5b3010d200 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b3557c640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3010d200 0x7f5b30107d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c01fd70 tx=0x7f5b2c01d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.773+0000 7f5b26ffd640 1 -- 172.21.15.22:0/2665990764 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c01fe60 con 0x7f5b3010d200 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.774+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b301154a0 con 0x7f5b3010d200 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.774+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b30115a40 con 0x7f5b3010d200 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.774+0000 7f5b26ffd640 1 -- 172.21.15.22:0/2665990764 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c0035f0 con 0x7f5b3010d200 2024-03-09T00:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.774+0000 7f5b26ffd640 1 -- 172.21.15.22:0/2665990764 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c021680 con 0x7f5b3010d200 2024-03-09T00:08:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.779+0000 7f5b26ffd640 1 -- 172.21.15.22:0/2665990764 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5b2c004000 con 0x7f5b3010d200 2024-03-09T00:08:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.779+0000 7f5b3abbd740 1 Processor -- start 2024-03-09T00:08:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.780+0000 7f5b3abbd740 1 -- start start 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f03a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f03fd0 0x55c573f04450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f0d550 0x55c573f0f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c573e685f0 con 0x55c573f04d40 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c573e11930 con 0x55c573f0d550 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c573e64690 con 0x55c573f03fd0 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3657e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f03a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f03fd0 0x55c573f04450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f03fd0 0x55c573f04450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39752/0 (socket says 172.21.15.22:39752) 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 -- 172.21.15.22:0/85684768 learned_addr learned my addr 172.21.15.22:0/85684768 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3657e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f03a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39826/0 (socket says 172.21.15.22:39826) 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b3557c640 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f0d550 0x55c573f0f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 -- 172.21.15.22:0/85684768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f0d550 msgr2=0x55c573f0f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f0d550 0x55c573f0f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 -- 172.21.15.22:0/85684768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 msgr2=0x55c573f03a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f03a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.781+0000 7f5b35d7d640 1 -- 172.21.15.22:0/85684768 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c573b7d210 con 0x55c573f03fd0 2024-03-09T00:08:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.782+0000 7f5b26ffd640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5b1c0a2b60 0x7f5b1c0a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.782+0000 7f5b26ffd640 1 -- 172.21.15.22:0/2665990764 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5b2c0e0760 con 0x7f5b3010d200 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.782+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f03fd0 0x55c573f04450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b2802e010 tx=0x7f5b280098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bdd9ad8344d5362 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.782+0000 7f5b1bfff640 1 -- 172.21.15.22:0/85684768 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28044020 con 0x55c573f03fd0 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.782+0000 7f5b1bfff640 1 -- 172.21.15.22:0/85684768 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b2802ca30 con 0x55c573f03fd0 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.782+0000 7f5b1bfff640 1 -- 172.21.15.22:0/85684768 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2803d680 con 0x55c573f03fd0 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.783+0000 7f5b3657e640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5b1c0a2b60 0x7f5b1c0a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.783+0000 7f5b3abbd740 1 -- 172.21.15.22:0/85684768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f03fd0 msgr2=0x55c573f04450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.783+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f03fd0 0x55c573f04450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b2802e010 tx=0x7f5b280098f0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.787+0000 7f5b3abbd740 1 -- 172.21.15.22:0/85684768 shutdown_connections 2024-03-09T00:08:03.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.787+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f0d550 0x55c573f0f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.787+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f03fd0 0x55c573f04450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.784 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.787+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/85684768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f03a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.788+0000 7f5b3abbd740 1 -- 172.21.15.22:0/85684768 >> 172.21.15.22:0/85684768 conn(0x55c573ecb5b0 msgr2=0x55c573efecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.790+0000 7f5b3abbd740 1 -- 172.21.15.22:0/85684768 shutdown_connections 2024-03-09T00:08:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.790+0000 7f5b3abbd740 1 -- 172.21.15.22:0/85684768 wait complete. 2024-03-09T00:08:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.791+0000 7f5b3abbd740 1 -- start start 2024-03-09T00:08:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.791+0000 7f5b3abbd740 1 Processor -- start 2024-03-09T00:08:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.791+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.792+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.792+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f0d550 0x55c573f0af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.792+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c573f10220 con 0x55c573f04d40 2024-03-09T00:08:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.792+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c573e24d90 con 0x55c573f03fd0 2024-03-09T00:08:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.792+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c573f103a0 con 0x55c573f0d550 2024-03-09T00:08:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.792+0000 7f5b3abbd740 10 client.0 _ll_register_callbacks cb 0x55c573b7ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.794+0000 7f5b3557c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f0aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.794+0000 7f5b3557c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f0aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39838/0 (socket says 172.21.15.22:39838) 2024-03-09T00:08:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.794+0000 7f5b3557c640 1 -- 172.21.15.22:0/377513575 learned_addr learned my addr 172.21.15.22:0/377513575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.794+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.794+0000 7f5b3657e640 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f0d550 0x55c573f0af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.794+0000 7f5b3657e640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5b1c0a2b60 0x7f5b1c0a5020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005fd0 tx=0x7f5b20005dc0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.795+0000 7f5b3557c640 1 -- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f0d550 msgr2=0x55c573f0af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.795+0000 7f5b3557c640 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f0d550 0x55c573f0af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.795+0000 7f5b3557c640 1 -- 172.21.15.22:0/377513575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 msgr2=0x55c573f0a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.795+0000 7f5b3557c640 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.795+0000 7f5b3557c640 1 -- 172.21.15.22:0/377513575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c573c49620 con 0x55c573f04d40 2024-03-09T00:08:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.795+0000 7f5b3557c640 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f0aa20 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0ac660 tx=0x7f5b2c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205b98c839acbd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.795+0000 7f5b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.796+0000 7f5b197fa640 1 -- 172.21.15.22:0/377513575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0ac960 con 0x55c573f04d40 2024-03-09T00:08:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.796+0000 7f5b197fa640 1 -- 172.21.15.22:0/377513575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5b2c0ddcd0 con 0x55c573f04d40 2024-03-09T00:08:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.797+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b2802ca30 con 0x55c573f04d40 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.798+0000 7f5b197fa640 1 -- 172.21.15.22:0/377513575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b2c0c5050 con 0x55c573f04d40 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.799+0000 7f5b3abbd740 10 client.19857 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.799+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c573f1c4f0 con 0x55c573f04d40 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.799+0000 7f5b197fa640 1 -- 172.21.15.22:0/377513575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b2802ca30 con 0x55c573f04d40 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.799+0000 7f5b197fa640 1 -- 172.21.15.22:0/377513575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5b2c0e02a0 con 0x55c573f04d40 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.800+0000 7f5b3abbd740 10 client.19857 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.800+0000 7f5b3abbd740 10 client.19857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.800+0000 7f5b3abbd740 20 client.19857 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.800+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f1df50 0x55c573f203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.800+0000 7f5b3abbd740 4 client.19857 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.800+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c573e75b30 con 0x55c573f1df50 2024-03-09T00:08:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.800+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f1df50 0x55c573f203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.801+0000 7f5b197fa640 10 client.19857 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.801+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f1df50 0x55c573f203b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130859de46672afe server_cookie=a3a84f09ef1b6a4e in_seq=0 out_seq=0 2024-03-09T00:08:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.804+0000 7f5b197fa640 1 -- 172.21.15.22:0/377513575 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c573e75b30 con 0x55c573f1df50 2024-03-09T00:08:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.804+0000 7f5b197fa640 10 client.19857 handle_command_reply: tid=0 2024-03-09T00:08:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.804+0000 7f5b3abbd740 1 client.19857 shutdown 2024-03-09T00:08:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.804+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 msgr2=0x55c573f0aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.804+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f0aa20 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0ac660 tx=0x7f5b2c0056d0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.807+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 shutdown_connections 2024-03-09T00:08:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.807+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f1df50 0x55c573f203b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.807+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f0d550 0x55c573f0af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.807+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f04d40 0x55c573f0aa20 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.807+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/377513575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.807+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 >> 172.21.15.22:0/377513575 conn(0x55c573ecb5b0 msgr2=0x55c573f00a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.807+0000 7f5b197fa640 0 client.19857 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.809+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 shutdown_connections 2024-03-09T00:08:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.812+0000 7f5b3abbd740 1 -- 172.21.15.22:0/377513575 wait complete. 2024-03-09T00:08:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.813+0000 7f5b3abbd740 20 client.19857 trim_cache size 0 max 16384 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.840+0000 7f5b3abbd740 1 Processor -- start 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3abbd740 1 -- start start 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 0x55c573f0ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f07780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140123e0 con 0x55c573f0d550 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1400ff20 con 0x55c573f03fd0 2024-03-09T00:08:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3657e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f07780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3657e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f07780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39850/0 (socket says 172.21.15.22:39850) 2024-03-09T00:08:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3657e640 1 -- 172.21.15.22:0/1916412971 learned_addr learned my addr 172.21.15.22:0/1916412971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b3abbd740 1 -- 172.21.15.22:0/1916412971 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1400dfe0 con 0x55c573f04d40 2024-03-09T00:08:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.841+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.842+0000 7f5b3657e640 1 -- 172.21.15.22:0/1916412971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 msgr2=0x55c573f0ad80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.842+0000 7f5b3657e640 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 0x55c573f0ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.842+0000 7f5b3657e640 1 -- 172.21.15.22:0/1916412971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 msgr2=0x55c573f0cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.842+0000 7f5b3657e640 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.842+0000 7f5b3657e640 1 -- 172.21.15.22:0/1916412971 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c573f032a0 con 0x55c573f0d550 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.842+0000 7f5b3657e640 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f07780 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000ef40 tx=0x7f5b20011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3e1a91878d7656 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.843+0000 7f5b1b7fe640 1 -- 172.21.15.22:0/1916412971 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2001e840 con 0x55c573f0d550 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.843+0000 7f5b1b7fe640 1 -- 172.21.15.22:0/1916412971 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b20022070 con 0x55c573f0d550 2024-03-09T00:08:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.843+0000 7f5b1b7fe640 1 -- 172.21.15.22:0/1916412971 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2001d430 con 0x55c573f0d550 2024-03-09T00:08:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.844+0000 7f5b3abbd740 1 -- 172.21.15.22:0/1916412971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 msgr2=0x55c573f07780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.844+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f07780 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000ef40 tx=0x7f5b20011000 comp rx=0 tx=0).stop 2024-03-09T00:08:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.845+0000 7f5b3abbd740 1 -- 172.21.15.22:0/1916412971 shutdown_connections 2024-03-09T00:08:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.845+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f07780 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.845+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 0x55c573f0ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.845+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/1916412971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f0cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.845+0000 7f5b3abbd740 1 -- 172.21.15.22:0/1916412971 >> 172.21.15.22:0/1916412971 conn(0x55c573ecb5b0 msgr2=0x55c573f08dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.847+0000 7f5b3abbd740 1 -- 172.21.15.22:0/1916412971 shutdown_connections 2024-03-09T00:08:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.848+0000 7f5b3abbd740 1 -- 172.21.15.22:0/1916412971 wait complete. 2024-03-09T00:08:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.848+0000 7f5b3abbd740 1 -- start start 2024-03-09T00:08:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 1 Processor -- start 2024-03-09T00:08:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f13200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 0x55c573f13740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f13c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1400e5e0 con 0x55c573f0d550 2024-03-09T00:08:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1400f8c0 con 0x55c573f03fd0 2024-03-09T00:08:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1400ec00 con 0x55c573f04d40 2024-03-09T00:08:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.849+0000 7f5b3abbd740 10 client.0 _ll_register_callbacks cb 0x7f5b2c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f13c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f13c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39862/0 (socket says 172.21.15.22:39862) 2024-03-09T00:08:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 -- 172.21.15.22:0/3410256183 learned_addr learned my addr 172.21.15.22:0/3410256183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 -- 172.21.15.22:0/3410256183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 msgr2=0x55c573f13740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:08:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 0x55c573f13740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f13200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 -- 172.21.15.22:0/3410256183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 msgr2=0x55c573f13200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f13200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.850+0000 7f5b3657e640 1 -- 172.21.15.22:0/3410256183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c573f74680 con 0x55c573f0d550 2024-03-09T00:08:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.851+0000 7f5b3657e640 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f13c80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5b20004700 tx=0x7f5b20010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d14fb8f64cbe7ef server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.851+0000 7f5b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.851+0000 7f5b197fa640 1 -- 172.21.15.22:0/3410256183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20037060 con 0x55c573f0d550 2024-03-09T00:08:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.851+0000 7f5b197fa640 1 -- 172.21.15.22:0/3410256183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f5b20027b80 con 0x55c573f0d550 2024-03-09T00:08:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.852+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b20022070 con 0x55c573f0d550 2024-03-09T00:08:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.852+0000 7f5b197fa640 1 -- 172.21.15.22:0/3410256183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b2000ebd0 con 0x55c573f0d550 2024-03-09T00:08:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.852+0000 7f5b3abbd740 10 client.19863 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.852+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c573f75970 con 0x55c573f0d550 2024-03-09T00:08:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b197fa640 1 -- 172.21.15.22:0/3410256183 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b20022070 con 0x55c573f0d550 2024-03-09T00:08:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b197fa640 1 -- 172.21.15.22:0/3410256183 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5b200292b0 con 0x55c573f0d550 2024-03-09T00:08:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b3abbd740 10 client.19863 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b3abbd740 10 client.19863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b3abbd740 20 client.19863 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f75ed0 0x55c573f782f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b3abbd740 4 client.19863 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:08:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c573e6fe80 con 0x55c573f75ed0 2024-03-09T00:08:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.853+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f75ed0 0x55c573f782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.854+0000 7f5b35d7d640 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f75ed0 0x55c573f782f0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be72f4c7607fa8b6 server_cookie=d5d2af481e9beea1 in_seq=0 out_seq=0 2024-03-09T00:08:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:03.854+0000 7f5b197fa640 10 client.19863 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:04.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3035 30.49 MB/sec warmup 11 sec latency 520.544 ms 2024-03-09T00:08:04.140 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:08:04.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:08:04.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:08:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.365+0000 7f71976b2640 1 -- 172.21.15.22:0/1155915290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 msgr2=0x7f719010d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.365+0000 7f71976b2640 1 --2- 172.21.15.22:0/1155915290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 0x7f719010d600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7180009900 tx=0x7f718002bc60 comp rx=0 tx=0).stop 2024-03-09T00:08:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.372+0000 7f71976b2640 1 -- 172.21.15.22:0/1155915290 shutdown_connections 2024-03-09T00:08:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.372+0000 7f71976b2640 1 --2- 172.21.15.22:0/1155915290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f719011aad0 0x7f719011cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.372+0000 7f71976b2640 1 --2- 172.21.15.22:0/1155915290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 0x7f719010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.372+0000 7f71976b2640 1 --2- 172.21.15.22:0/1155915290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 0x7f719010d600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.372+0000 7f71976b2640 1 -- 172.21.15.22:0/1155915290 >> 172.21.15.22:0/1155915290 conn(0x7f719006c3d0 msgr2=0x7f719006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:04.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.380+0000 7f71976b2640 1 -- 172.21.15.22:0/1155915290 shutdown_connections 2024-03-09T00:08:04.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.382+0000 7f71976b2640 1 -- 172.21.15.22:0/1155915290 wait complete. 2024-03-09T00:08:04.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.383+0000 7f71976b2640 1 Processor -- start 2024-03-09T00:08:04.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.385+0000 7f71976b2640 1 -- start start 2024-03-09T00:08:04.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.385+0000 7f71976b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 0x7f71901180c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.385+0000 7f71976b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 0x7f7190111140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.385+0000 7f71976b2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f719011aad0 0x7f7190111680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.385+0000 7f71976b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71901204f0 con 0x7f719010db40 2024-03-09T00:08:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.385+0000 7f71976b2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190120370 con 0x7f719011aad0 2024-03-09T00:08:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.385+0000 7f71976b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190120670 con 0x7f719010fb10 2024-03-09T00:08:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.388+0000 7f719544e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 0x7f71901180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.388+0000 7f719544e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 0x7f71901180c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39868/0 (socket says 172.21.15.22:39868) 2024-03-09T00:08:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.388+0000 7f719544e640 1 -- 172.21.15.22:0/3477644965 learned_addr learned my addr 172.21.15.22:0/3477644965 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:04.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.389+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f719011aad0 0x7f7190111680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.391+0000 7f7194c4d640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 0x7f7190111140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.393+0000 7f719544e640 1 -- 172.21.15.22:0/3477644965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 msgr2=0x7f7190111140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:04.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.393+0000 7f719544e640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 0x7f7190111140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.393+0000 7f719544e640 1 -- 172.21.15.22:0/3477644965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f719011aad0 msgr2=0x7f7190111680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:04.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.393+0000 7f719544e640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f719011aad0 0x7f7190111680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.393+0000 7f719544e640 1 -- 172.21.15.22:0/3477644965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190111c70 con 0x7f719010db40 2024-03-09T00:08:04.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.397+0000 7f719544e640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 0x7f71901180c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f718002efd0 tx=0x7f71800045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:04.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.403+0000 7f71867fc640 1 -- 172.21.15.22:0/3477644965 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180041070 con 0x7f719010db40 2024-03-09T00:08:04.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.404+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7190111f00 con 0x7f719010db40 2024-03-09T00:08:04.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.404+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7190115e80 con 0x7f719010db40 2024-03-09T00:08:04.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.404+0000 7f71867fc640 1 -- 172.21.15.22:0/3477644965 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f718002ccc0 con 0x7f719010db40 2024-03-09T00:08:04.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.404+0000 7f71867fc640 1 -- 172.21.15.22:0/3477644965 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180034440 con 0x7f719010db40 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.407+0000 7f719a374740 1 Processor -- start 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.407+0000 7f719a374740 1 -- start start 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.408+0000 7f719a374740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e955660d80 0x55e955661160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.408+0000 7f719a374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9556616a0 0x55e955669d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.408+0000 7f719a374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e95566c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.408+0000 7f719a374740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9555c4b40 con 0x55e955660d80 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.408+0000 7f719a374740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95556dd80 con 0x55e95566a280 2024-03-09T00:08:04.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.408+0000 7f719a374740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9555c0be0 con 0x55e9556616a0 2024-03-09T00:08:04.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.411+0000 7f719544e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9556616a0 0x55e955669d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.411+0000 7f719544e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9556616a0 0x55e955669d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39816/0 (socket says 172.21.15.22:39816) 2024-03-09T00:08:04.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.411+0000 7f719544e640 1 -- 172.21.15.22:0/2730268878 learned_addr learned my addr 172.21.15.22:0/2730268878 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:04.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.413+0000 7f7195c4f640 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e955660d80 0x55e955661160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.414+0000 7f7194c4d640 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e95566c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.414+0000 7f719544e640 1 -- 172.21.15.22:0/2730268878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 msgr2=0x55e95566c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:04.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.414+0000 7f719544e640 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e95566c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.414+0000 7f719544e640 1 -- 172.21.15.22:0/2730268878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e955660d80 msgr2=0x55e955661160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:04.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.414+0000 7f719544e640 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e955660d80 0x55e955661160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.414+0000 7f719544e640 1 -- 172.21.15.22:0/2730268878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9553a17e0 con 0x55e9556616a0 2024-03-09T00:08:04.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.414+0000 7f719544e640 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9556616a0 0x55e955669d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f71800f37e0 tx=0x7f71800f8e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1ff1ca0ff9a33c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:04.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.417+0000 7f71867fc640 1 -- 172.21.15.22:0/3477644965 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7180044070 con 0x7f719010db40 2024-03-09T00:08:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.418+0000 7f71867fc640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f717c0a2b70 0x7f717c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.418+0000 7f71867fc640 1 -- 172.21.15.22:0/3477644965 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f71800f63e0 con 0x7f719010db40 2024-03-09T00:08:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.418+0000 7f717b7fe640 1 -- 172.21.15.22:0/2730268878 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180107070 con 0x55e9556616a0 2024-03-09T00:08:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.418+0000 7f717b7fe640 1 -- 172.21.15.22:0/2730268878 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71800f85a0 con 0x55e9556616a0 2024-03-09T00:08:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.419+0000 7f719a374740 1 -- 172.21.15.22:0/2730268878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9556616a0 msgr2=0x55e955669d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.419+0000 7f719a374740 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9556616a0 0x55e955669d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f71800f37e0 tx=0x7f71800f8e70 comp rx=0 tx=0).stop 2024-03-09T00:08:04.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.421+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f717c0a2b70 0x7f717c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.427+0000 7f719a374740 1 -- 172.21.15.22:0/2730268878 shutdown_connections 2024-03-09T00:08:04.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.427+0000 7f719a374740 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e95566c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.427+0000 7f719a374740 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9556616a0 0x55e955669d40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.427+0000 7f719a374740 1 --2- 172.21.15.22:0/2730268878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e955660d80 0x55e955661160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.427+0000 7f719a374740 1 -- 172.21.15.22:0/2730268878 >> 172.21.15.22:0/2730268878 conn(0x55e95565b4b0 msgr2=0x55e95565b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:04.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.429+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f717c0a2b70 0x7f717c0a5030 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f718c005fd0 tx=0x7f718c00b5b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:04.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.437+0000 7f719a374740 1 -- 172.21.15.22:0/2730268878 shutdown_connections 2024-03-09T00:08:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.442+0000 7f719a374740 1 -- 172.21.15.22:0/2730268878 wait complete. 2024-03-09T00:08:04.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.442+0000 7f719a374740 1 -- start start 2024-03-09T00:08:04.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.442+0000 7f719a374740 1 Processor -- start 2024-03-09T00:08:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.444+0000 7f719a374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e955669c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.444+0000 7f719a374740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e955666f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.444+0000 7f719a374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e955667510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.444+0000 7f719a374740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95566ce30 con 0x55e9556616a0 2024-03-09T00:08:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.444+0000 7f719a374740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9555811c0 con 0x55e95566a280 2024-03-09T00:08:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.444+0000 7f719a374740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95566cfb0 con 0x55e955660d80 2024-03-09T00:08:04.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.444+0000 7f719a374740 10 client.0 _ll_register_callbacks cb 0x55e9555707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.445+0000 7f719544e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e955669c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.445+0000 7f719544e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e955669c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39820/0 (socket says 172.21.15.22:39820) 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.445+0000 7f719544e640 1 -- 172.21.15.22:0/3616810537 learned_addr learned my addr 172.21.15.22:0/3616810537 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.447+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e955667510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.447+0000 7f719544e640 1 -- 172.21.15.22:0/3616810537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 msgr2=0x55e955667510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.447+0000 7f719544e640 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e955667510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.447+0000 7f719544e640 1 -- 172.21.15.22:0/3616810537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 msgr2=0x55e955666f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:08:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.447+0000 7f719544e640 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e955666f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:04.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.447+0000 7f719544e640 1 -- 172.21.15.22:0/3616810537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e955401f70 con 0x55e955660d80 2024-03-09T00:08:04.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.448+0000 7f719544e640 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e955669c00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f71800c23f0 tx=0x7f7180000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0330d45ee901eb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:04.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.448+0000 7f7178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:08:04.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.453+0000 7f7178ff9640 1 -- 172.21.15.22:0/3616810537 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718004b040 con 0x55e955660d80 2024-03-09T00:08:04.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.453+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71800f85a0 con 0x55e955660d80 2024-03-09T00:08:04.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.453+0000 7f7178ff9640 1 -- 172.21.15.22:0/3616810537 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f71800035f0 con 0x55e955660d80 2024-03-09T00:08:04.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.454+0000 7f7178ff9640 1 -- 172.21.15.22:0/3616810537 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71800be900 con 0x55e955660d80 2024-03-09T00:08:04.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.454+0000 7f719a374740 10 client.29299 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:04.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.454+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e955299170 con 0x55e955660d80 2024-03-09T00:08:04.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.458+0000 7f7178ff9640 1 -- 172.21.15.22:0/3616810537 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71800f85a0 con 0x55e955660d80 2024-03-09T00:08:04.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.458+0000 7f7178ff9640 1 -- 172.21.15.22:0/3616810537 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f71800c2950 con 0x55e955660d80 2024-03-09T00:08:04.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.459+0000 7f719a374740 10 client.29299 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:04.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.459+0000 7f719a374740 10 client.29299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:04.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.459+0000 7f719a374740 20 client.29299 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:04.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.459+0000 7f719a374740 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e955679fe0 0x55e95567c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:04.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.459+0000 7f719a374740 4 client.29299 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:04.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.459+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e95559a960 con 0x55e955679fe0 2024-03-09T00:08:04.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.460+0000 7f7194c4d640 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e955679fe0 0x55e95567c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.475+0000 7f7194c4d640 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e955679fe0 0x55e95567c3d0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56750f89e4cb7b2c server_cookie=b9251d9b02e3aae8 in_seq=0 out_seq=0 2024-03-09T00:08:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:04.476+0000 7f7178ff9640 10 client.29299 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:04.860 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:04 smithi062 ceph-mon[36129]: pgmap v426: 129 pgs: 129 active+clean; 227 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 469 op/s 2024-03-09T00:08:04.879 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:04 smithi192 ceph-mon[35995]: pgmap v426: 129 pgs: 129 active+clean; 227 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 469 op/s 2024-03-09T00:08:04.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:04 smithi022 ceph-mon[31894]: pgmap v426: 129 pgs: 129 active+clean; 227 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 469 op/s 2024-03-09T00:08:05.061 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3069 28.25 MB/sec warmup 12 sec latency 1215.840 ms 2024-03-09T00:08:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.118+0000 7f5b197fa640 1 -- 172.21.15.22:0/3410256183 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c573e6fe80 con 0x55c573f75ed0 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.118+0000 7f5b197fa640 10 client.19863 handle_command_reply: tid=0 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.118+0000 7f5b3abbd740 1 client.19863 shutdown 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.118+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 msgr2=0x55c573f13c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.118+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f13c80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5b20004700 tx=0x7f5b20010f00 comp rx=0 tx=0).stop 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 shutdown_connections 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c573f75ed0 0x55c573f782f0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c573f0d550 0x55c573f13c80 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c573f04d40 0x55c573f13740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 --2- 172.21.15.22:0/3410256183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c573f03fd0 0x55c573f13200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 >> 172.21.15.22:0/3410256183 conn(0x55c573ecb5b0 msgr2=0x55c573efd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b197fa640 0 client.19863 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 shutdown_connections 2024-03-09T00:08:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 1 -- 172.21.15.22:0/3410256183 wait complete. 2024-03-09T00:08:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.119+0000 7f5b3abbd740 20 client.19863 trim_cache size 0 max 16384 2024-03-09T00:08:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:08:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:08:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:08:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:08:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5b1c0a2b60 msgr2=0x7f5b1c0a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5b1c0a2b60 0x7f5b1c0a5020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005fd0 tx=0x7f5b20005dc0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3010d200 msgr2=0x7f5b30107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3010d200 0x7f5b30107d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c01fd70 tx=0x7f5b2c01d6b0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 shutdown_connections 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5b1c0a2b60 0x7f5b1c0a5020 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b30108370 0x7f5b30114c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3010d200 0x7f5b30107d20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 --2- 172.21.15.22:0/2665990764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b30105510 0x7f5b3010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 >> 172.21.15.22:0/2665990764 conn(0x7f5b30077310 msgr2=0x7f5b30072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 shutdown_connections 2024-03-09T00:08:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f5b37fe1640 1 -- 172.21.15.22:0/2665990764 wait complete. 2024-03-09T00:08:05.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f7178ff9640 1 -- 172.21.15.22:0/3616810537 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e95559a960 con 0x55e955679fe0 2024-03-09T00:08:05.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f7178ff9640 10 client.29299 handle_command_reply: tid=0 2024-03-09T00:08:05.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.120+0000 7f719a374740 1 client.29299 shutdown 2024-03-09T00:08:05.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 msgr2=0x55e955669c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e955669c00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f71800c23f0 tx=0x7f7180000bc0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 shutdown_connections 2024-03-09T00:08:05.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e955679fe0 0x55e95567c3d0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e95566a280 0x55e955667510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e955666f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 --2- 172.21.15.22:0/3616810537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e955669c00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 >> 172.21.15.22:0/3616810537 conn(0x55e95565b4b0 msgr2=0x55e95565d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f7178ff9640 0 client.29299 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 shutdown_connections 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 1 -- 172.21.15.22:0/3616810537 wait complete. 2024-03-09T00:08:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.121+0000 7f719a374740 20 client.29299 trim_cache size 0 max 16384 2024-03-09T00:08:05.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.151+0000 7f719a374740 1 Processor -- start 2024-03-09T00:08:05.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.151+0000 7f719a374740 1 -- start start 2024-03-09T00:08:05.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.151+0000 7f719a374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e9556696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.151+0000 7f719a374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9556616a0 0x55e955669c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f719544e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e9556696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f719544e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e9556696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:39822/0 (socket says 172.21.15.22:39822) 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f719544e640 1 -- 172.21.15.22:0/3853007248 learned_addr learned my addr 172.21.15.22:0/3853007248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f719a374740 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95566a280 0x55e95567bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f7194c4d640 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9556616a0 0x55e955669c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71740123f0 con 0x55e95566a280 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717400ff30 con 0x55e9556616a0 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717400dff0 con 0x55e955660d80 2024-03-09T00:08:05.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.152+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95566a280 0x55e95567bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f7195c4f640 1 -- 172.21.15.22:0/3853007248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 msgr2=0x55e9556696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e9556696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f7195c4f640 1 -- 172.21.15.22:0/3853007248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9556616a0 msgr2=0x55e955669c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9556616a0 0x55e955669c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f7195c4f640 1 -- 172.21.15.22:0/3853007248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e955665500 con 0x55e95566a280 2024-03-09T00:08:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f7195c4f640 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95566a280 0x55e95567bff0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x55e95565ffb0 tx=0x7f718c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a802700bd012620 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f717affd640 1 -- 172.21.15.22:0/3853007248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c029670 con 0x55e95566a280 2024-03-09T00:08:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f717affd640 1 -- 172.21.15.22:0/3853007248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f718c024070 con 0x55e95566a280 2024-03-09T00:08:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.153+0000 7f717affd640 1 -- 172.21.15.22:0/3853007248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c020c60 con 0x55e95566a280 2024-03-09T00:08:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.154+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95566a280 msgr2=0x55e95567bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.154+0000 7f719a374740 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95566a280 0x55e95567bff0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x55e95565ffb0 tx=0x7f718c014000 comp rx=0 tx=0).stop 2024-03-09T00:08:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.155+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 shutdown_connections 2024-03-09T00:08:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.155+0000 7f719a374740 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e95566a280 0x55e95567bff0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.155+0000 7f719a374740 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9556616a0 0x55e955669c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.155+0000 7f719a374740 1 --2- 172.21.15.22:0/3853007248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e955660d80 0x55e9556696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.155+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 >> 172.21.15.22:0/3853007248 conn(0x55e95565b4b0 msgr2=0x55e9556622e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.155+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 shutdown_connections 2024-03-09T00:08:05.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.155+0000 7f719a374740 1 -- 172.21.15.22:0/3853007248 wait complete. 2024-03-09T00:08:05.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.156+0000 7f719a374740 1 -- start start 2024-03-09T00:08:05.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.156+0000 7f719a374740 1 Processor -- start 2024-03-09T00:08:05.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.157+0000 7f719a374740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e955660d80 0x55e9556642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.157+0000 7f719a374740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e9556647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.157+0000 7f719a374740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e95566a280 0x55e9556d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.157+0000 7f719544e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e955660d80 0x55e9556642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.157+0000 7f719544e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e955660d80 0x55e9556642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35870/0 (socket says 172.21.15.22:35870) 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.158+0000 7f719544e640 1 -- 172.21.15.22:0/2112880973 learned_addr learned my addr 172.21.15.22:0/2112880973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.158+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717400e5f0 con 0x55e9556616a0 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.158+0000 7f7195c4f640 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e95566a280 0x55e9556d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.158+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717400f8d0 con 0x55e955660d80 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.158+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717400ec10 con 0x55e95566a280 2024-03-09T00:08:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.158+0000 7f719a374740 10 client.0 _ll_register_callbacks cb 0x7f71800f6710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.159+0000 7f7194c4d640 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e9556647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.159+0000 7f7194c4d640 1 -- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e95566a280 msgr2=0x55e9556d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.159+0000 7f7194c4d640 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e95566a280 0x55e9556d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.159+0000 7f7194c4d640 1 -- 172.21.15.22:0/2112880973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e955660d80 msgr2=0x55e9556642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.159+0000 7f7194c4d640 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e955660d80 0x55e9556642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.159+0000 7f7194c4d640 1 -- 172.21.15.22:0/2112880973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9556d0890 con 0x55e9556616a0 2024-03-09T00:08:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.160+0000 7f7194c4d640 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e9556647f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x55e95567cb80 tx=0x7f71880160b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d026fdd022beb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.160+0000 7f7178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.160+0000 7f7178ff9640 1 -- 172.21.15.22:0/2112880973 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188024060 con 0x55e9556616a0 2024-03-09T00:08:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.161+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f718c024070 con 0x55e9556616a0 2024-03-09T00:08:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.161+0000 7f7178ff9640 1 -- 172.21.15.22:0/2112880973 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f718800d040 con 0x55e9556616a0 2024-03-09T00:08:05.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.163+0000 7f7178ff9640 1 -- 172.21.15.22:0/2112880973 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f718802b050 con 0x55e9556616a0 2024-03-09T00:08:05.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.164+0000 7f719a374740 10 client.19875 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:05.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.164+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e9556d1af0 con 0x55e9556616a0 2024-03-09T00:08:05.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.166+0000 7f7178ff9640 1 -- 172.21.15.22:0/2112880973 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f718c024070 con 0x55e9556616a0 2024-03-09T00:08:05.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.166+0000 7f7178ff9640 1 -- 172.21.15.22:0/2112880973 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f718801f880 con 0x55e9556616a0 2024-03-09T00:08:05.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.168+0000 7f719a374740 10 client.19875 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.168+0000 7f719a374740 10 client.19875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.168+0000 7f719a374740 20 client.19875 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.168+0000 7f719a374740 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e9556d21a0 0x55e9556d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.168+0000 7f719a374740 4 client.19875 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:08:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.168+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e955586080 con 0x55e9556d21a0 2024-03-09T00:08:05.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.168+0000 7f719544e640 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e9556d21a0 0x55e9556d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.172+0000 7f719544e640 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e9556d21a0 0x55e9556d45c0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160d6d19beb62fca server_cookie=cb90c8026e367985 in_seq=0 out_seq=0 2024-03-09T00:08:05.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.172+0000 7f7178ff9640 10 client.19875 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.173+0000 7f7178ff9640 1 -- 172.21.15.22:0/2112880973 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e955586080 con 0x55e9556d21a0 2024-03-09T00:08:05.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.174+0000 7f7178ff9640 10 client.19875 handle_command_reply: tid=0 2024-03-09T00:08:05.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.174+0000 7f719a374740 1 client.19875 shutdown 2024-03-09T00:08:05.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.176+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 msgr2=0x55e9556647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.176+0000 7f719a374740 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e9556647f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x55e95567cb80 tx=0x7f71880160b0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.177+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 shutdown_connections 2024-03-09T00:08:05.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.177+0000 7f719a374740 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e9556d21a0 0x55e9556d45c0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.177+0000 7f719a374740 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e95566a280 0x55e9556d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.177+0000 7f719a374740 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9556616a0 0x55e9556647f0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.177+0000 7f719a374740 1 --2- 172.21.15.22:0/2112880973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e955660d80 0x55e9556642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.177+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 >> 172.21.15.22:0/2112880973 conn(0x55e95565b4b0 msgr2=0x55e95567bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.177+0000 7f7178ff9640 0 client.19875 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.179+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 shutdown_connections 2024-03-09T00:08:05.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.179+0000 7f719a374740 1 -- 172.21.15.22:0/2112880973 wait complete. 2024-03-09T00:08:05.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.181+0000 7f719a374740 20 client.19875 trim_cache size 0 max 16384 2024-03-09T00:08:05.176 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:08:05.176 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:08:05.176 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "9ea19535-eb06-4414-b589-e511e479ba7f", 2024-03-09T00:08:05.176 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:08:05.176 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:08:05.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.182+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f717c0a2b70 msgr2=0x7f717c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f717c0a2b70 0x7f717c0a5030 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f718c005fd0 tx=0x7f718c00b5b0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 msgr2=0x7f71901180c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 0x7f71901180c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f718002efd0 tx=0x7f71800045e0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 shutdown_connections 2024-03-09T00:08:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f717c0a2b70 0x7f717c0a5030 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f719011aad0 0x7f7190111680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f719010fb10 0x7f7190111140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 --2- 172.21.15.22:0/3477644965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f719010db40 0x7f71901180c0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 >> 172.21.15.22:0/3477644965 conn(0x7f719006c3d0 msgr2=0x7f7190071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 shutdown_connections 2024-03-09T00:08:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.183+0000 7f71976b2640 1 -- 172.21.15.22:0/3477644965 wait complete. 2024-03-09T00:08:05.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:08:05.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:08:05.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.376+0000 7fcbfa601640 1 -- 172.21.15.22:0/3228527684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 msgr2=0x7fcbf4100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.376+0000 7fcbfa601640 1 --2- 172.21.15.22:0/3228527684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 0x7fcbf4100950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009ca0 tx=0x7fcbec02c140 comp rx=0 tx=0).stop 2024-03-09T00:08:05.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.377+0000 7fcbfa601640 1 -- 172.21.15.22:0/3228527684 shutdown_connections 2024-03-09T00:08:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.377+0000 7fcbfa601640 1 --2- 172.21.15.22:0/3228527684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbf4100e90 0x7fcbf410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.377+0000 7fcbfa601640 1 --2- 172.21.15.22:0/3228527684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 0x7fcbf4100950 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.377+0000 7fcbfa601640 1 --2- 172.21.15.22:0/3228527684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 0x7fcbf4106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.377+0000 7fcbfa601640 1 -- 172.21.15.22:0/3228527684 >> 172.21.15.22:0/3228527684 conn(0x7fcbf40771d0 msgr2=0x7fcbf40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.378+0000 7fcbfa601640 1 -- 172.21.15.22:0/3228527684 shutdown_connections 2024-03-09T00:08:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.378+0000 7fcbfa601640 1 -- 172.21.15.22:0/3228527684 wait complete. 2024-03-09T00:08:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.379+0000 7fcbfa601640 1 Processor -- start 2024-03-09T00:08:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.379+0000 7fcbfa601640 1 -- start start 2024-03-09T00:08:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.380+0000 7fcbfa601640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 0x7fcbf419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.380+0000 7fcbfa601640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbf4100e90 0x7fcbf419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.380+0000 7fcbfa601640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 0x7fcbf41a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.380+0000 7fcbfa601640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4110660 con 0x7fcbf41063b0 2024-03-09T00:08:05.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.380+0000 7fcbfa601640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41104e0 con 0x7fcbf4100e90 2024-03-09T00:08:05.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.380+0000 7fcbfa601640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41107e0 con 0x7fcbf41004d0 2024-03-09T00:08:05.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.382+0000 7fcbf8b9e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 0x7fcbf41a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.382+0000 7fcbf8b9e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 0x7fcbf41a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39896/0 (socket says 172.21.15.22:39896) 2024-03-09T00:08:05.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.382+0000 7fcbf8b9e640 1 -- 172.21.15.22:0/1229962414 learned_addr learned my addr 172.21.15.22:0/1229962414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:05.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.382+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 0x7fcbf419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.382+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbf4100e90 0x7fcbf419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.383+0000 7fcbf8b9e640 1 -- 172.21.15.22:0/1229962414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 msgr2=0x7fcbf419e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.383+0000 7fcbf8b9e640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 0x7fcbf419e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.383+0000 7fcbf8b9e640 1 -- 172.21.15.22:0/1229962414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbf4100e90 msgr2=0x7fcbf419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.383+0000 7fcbf8b9e640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbf4100e90 0x7fcbf419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.383+0000 7fcbf8b9e640 1 -- 172.21.15.22:0/1229962414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf41a3230 con 0x7fcbf41063b0 2024-03-09T00:08:05.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.383+0000 7fcbf8b9e640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 0x7fcbf41a2b00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e970 tx=0x7fcbe400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.384+0000 7fcbf17fa640 1 -- 172.21.15.22:0/1229962414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400ccc0 con 0x7fcbf41063b0 2024-03-09T00:08:05.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.384+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf41a3520 con 0x7fcbf41063b0 2024-03-09T00:08:05.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.384+0000 7fcbf17fa640 1 -- 172.21.15.22:0/1229962414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbe400ce60 con 0x7fcbf41063b0 2024-03-09T00:08:05.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.384+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf41aad70 con 0x7fcbf41063b0 2024-03-09T00:08:05.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.384+0000 7fcbf17fa640 1 -- 172.21.15.22:0/1229962414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4010490 con 0x7fcbf41063b0 2024-03-09T00:08:05.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.387+0000 7fcbfd1d9740 1 Processor -- start 2024-03-09T00:08:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.388+0000 7fcbf17fa640 1 -- 172.21.15.22:0/1229962414 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fcbe4010770 con 0x7fcbf41063b0 2024-03-09T00:08:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.389+0000 7fcbfd1d9740 1 -- start start 2024-03-09T00:08:05.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.389+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 0x559da2f6cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.389+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f75720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.389+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f75c60 0x559da2f78050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.389+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da2ecfe00 con 0x559da2f6d080 2024-03-09T00:08:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.389+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da2e78b10 con 0x559da2f75c60 2024-03-09T00:08:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.389+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da2ecbea0 con 0x559da2f6c760 2024-03-09T00:08:05.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.390+0000 7fcbf17fa640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbe00a2ab0 0x7fcbe00a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.390+0000 7fcbf17fa640 1 -- 172.21.15.22:0/1229962414 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcbe40cfba0 con 0x7fcbf41063b0 2024-03-09T00:08:05.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.391+0000 7fcbf3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f75720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.391+0000 7fcbf3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f75720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39898/0 (socket says 172.21.15.22:39898) 2024-03-09T00:08:05.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.391+0000 7fcbf3fff640 1 -- 172.21.15.22:0/3166138953 learned_addr learned my addr 172.21.15.22:0/3166138953 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:05.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbf8b9e640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbe00a2ab0 0x7fcbe00a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbf3fff640 1 -- 172.21.15.22:0/3166138953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 msgr2=0x559da2f6cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/3166138953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 0x559da2f6cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbf3fff640 1 -- 172.21.15.22:0/3166138953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f75c60 msgr2=0x559da2f78050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/3166138953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f75c60 0x559da2f78050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbf3fff640 1 -- 172.21.15.22:0/3166138953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da2cae7d0 con 0x559da2f6d080 2024-03-09T00:08:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/3166138953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f75720 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009a40 tx=0x7fcbdc009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3b66bc58c19b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.392+0000 7fcbea7fc640 1 -- 172.21.15.22:0/3166138953 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc017070 con 0x559da2f6d080 2024-03-09T00:08:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.393+0000 7fcbea7fc640 1 -- 172.21.15.22:0/3166138953 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc013470 con 0x559da2f6d080 2024-03-09T00:08:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.393+0000 7fcbea7fc640 1 -- 172.21.15.22:0/3166138953 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc012610 con 0x559da2f6d080 2024-03-09T00:08:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.393+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/3166138953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 msgr2=0x559da2f75720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.393+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/3166138953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f75720 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009a40 tx=0x7fcbdc009f00 comp rx=0 tx=0).stop 2024-03-09T00:08:05.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/3166138953 shutdown_connections 2024-03-09T00:08:05.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/3166138953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f75c60 0x559da2f78050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/3166138953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f75720 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/3166138953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 0x559da2f6cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/3166138953 >> 172.21.15.22:0/3166138953 conn(0x559da2f66e90 msgr2=0x559da2f67270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/3166138953 shutdown_connections 2024-03-09T00:08:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbf8b9e640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbe00a2ab0 0x7fcbe00a4f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcbe409baa0 tx=0x7fcbe4010630 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.395+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/3166138953 wait complete. 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.396+0000 7fcbfd1d9740 1 -- start start 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.396+0000 7fcbfd1d9740 1 Processor -- start 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.396+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 0x559da2f752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.396+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6d080 0x559da2f72660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.396+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f75c60 0x559da2f72ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.396+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da2f78af0 con 0x559da2f75c60 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.396+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da2e8c0a0 con 0x559da2f6d080 2024-03-09T00:08:05.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.397+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da2f78c70 con 0x559da2f6c760 2024-03-09T00:08:05.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.397+0000 7fcbfd1d9740 10 client.0 _ll_register_callbacks cb 0x559da2e8ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6d080 0x559da2f72660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6d080 0x559da2f72660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35904/0 (socket says 172.21.15.22:35904) 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 -- 172.21.15.22:0/2972442971 learned_addr learned my addr 172.21.15.22:0/2972442971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 0x559da2f752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f75c60 0x559da2f72ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 -- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 msgr2=0x559da2f752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 0x559da2f752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 -- 172.21.15.22:0/2972442971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f75c60 msgr2=0x559da2f72ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f75c60 0x559da2f72ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.398+0000 7fcbf3fff640 1 -- 172.21.15.22:0/2972442971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559da2be43f0 con 0x559da2f6d080 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.399+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6d080 0x559da2f72660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc006c80 tx=0x7fcbdc00eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee2e58f29104252 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.399+0000 7fcbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:08:05.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.400+0000 7fcbd37fe640 1 -- 172.21.15.22:0/2972442971 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc017050 con 0x559da2f6d080 2024-03-09T00:08:05.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.400+0000 7fcbd37fe640 1 -- 172.21.15.22:0/2972442971 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcbdc013930 con 0x559da2f6d080 2024-03-09T00:08:05.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.400+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbdc013470 con 0x559da2f6d080 2024-03-09T00:08:05.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.401+0000 7fcbd37fe640 1 -- 172.21.15.22:0/2972442971 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbdc01c450 con 0x559da2f6d080 2024-03-09T00:08:05.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.401+0000 7fcbfd1d9740 10 client.29141 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:05.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.401+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559da2ba40f0 con 0x559da2f6d080 2024-03-09T00:08:05.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbd37fe640 1 -- 172.21.15.22:0/2972442971 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc013470 con 0x559da2f6d080 2024-03-09T00:08:05.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbd37fe640 1 -- 172.21.15.22:0/2972442971 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcbdc041830 con 0x559da2f6d080 2024-03-09T00:08:05.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbfd1d9740 10 client.29141 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:05.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbfd1d9740 10 client.29141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:05.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbfd1d9740 20 client.29141 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:05.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2f965d0 0x559da2f989c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbfd1d9740 4 client.29141 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:05.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.403+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559da2edd340 con 0x559da2f965d0 2024-03-09T00:08:05.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.404+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2f965d0 0x559da2f989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.405+0000 7fcbd37fe640 10 client.29141 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.405+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2f965d0 0x559da2f989c0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130e8b6c738d2810 server_cookie=d2666bd13acac75e in_seq=0 out_seq=0 2024-03-09T00:08:05.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.407+0000 7fcbd37fe640 1 -- 172.21.15.22:0/2972442971 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559da2edd340 con 0x559da2f965d0 2024-03-09T00:08:05.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.407+0000 7fcbd37fe640 10 client.29141 handle_command_reply: tid=0 2024-03-09T00:08:05.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.407+0000 7fcbfd1d9740 1 client.29141 shutdown 2024-03-09T00:08:05.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.407+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6d080 msgr2=0x559da2f72660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.407+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6d080 0x559da2f72660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc006c80 tx=0x7fcbdc00eed0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 shutdown_connections 2024-03-09T00:08:05.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2f965d0 0x559da2f989c0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f75c60 0x559da2f72ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6d080 0x559da2f72660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/2972442971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f6c760 0x559da2f752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 >> 172.21.15.22:0/2972442971 conn(0x559da2f66e90 msgr2=0x559da2f68be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbd37fe640 0 client.29141 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 shutdown_connections 2024-03-09T00:08:05.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.408+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/2972442971 wait complete. 2024-03-09T00:08:05.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.409+0000 7fcbfd1d9740 20 client.29141 trim_cache size 0 max 16384 2024-03-09T00:08:05.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.438+0000 7fcbfd1d9740 1 Processor -- start 2024-03-09T00:08:05.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.439+0000 7fcbfd1d9740 1 -- start start 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.439+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f6dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.439+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f74d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.439+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.439+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40124c0 con 0x559da2f6d080 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.440+0000 7fcbf37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f6dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.440+0000 7fcbf37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f6dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35920/0 (socket says 172.21.15.22:35920) 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.440+0000 7fcbf37fe640 1 -- 172.21.15.22:0/1163937952 learned_addr learned my addr 172.21.15.22:0/1163937952 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.440+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1163937952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4010000 con 0x559da2f6c760 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.440+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1163937952 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd400e0c0 con 0x559da2f75c60 2024-03-09T00:08:05.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.440+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.441+0000 7fcbf37fe640 1 -- 172.21.15.22:0/1163937952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 msgr2=0x559da2f752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.441+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.441+0000 7fcbf37fe640 1 -- 172.21.15.22:0/1163937952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 msgr2=0x559da2f74d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:08:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.441+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f74d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.441+0000 7fcbf37fe640 1 -- 172.21.15.22:0/1163937952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da2f6ceb0 con 0x559da2f6c760 2024-03-09T00:08:05.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.443+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f6dcb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbec02c720 tx=0x7fcbec0097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd07ad6d9213dfc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.444+0000 7fcbe9ffb640 1 -- 172.21.15.22:0/1163937952 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec039040 con 0x559da2f6c760 2024-03-09T00:08:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.444+0000 7fcbe9ffb640 1 -- 172.21.15.22:0/1163937952 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbec047e50 con 0x559da2f6c760 2024-03-09T00:08:05.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.444+0000 7fcbe9ffb640 1 -- 172.21.15.22:0/1163937952 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0477e0 con 0x559da2f6c760 2024-03-09T00:08:05.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.446+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1163937952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 msgr2=0x559da2f6dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.446+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f6dcb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbec02c720 tx=0x7fcbec0097e0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.448+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1163937952 shutdown_connections 2024-03-09T00:08:05.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.448+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.448+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f74d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.448+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1163937952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f6dcb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.448+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1163937952 >> 172.21.15.22:0/1163937952 conn(0x559da2f66e90 msgr2=0x559da2f6d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.449+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1163937952 shutdown_connections 2024-03-09T00:08:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.449+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1163937952 wait complete. 2024-03-09T00:08:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.450+0000 7fcbfd1d9740 1 -- start start 2024-03-09T00:08:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.450+0000 7fcbfd1d9740 1 Processor -- start 2024-03-09T00:08:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.450+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.451+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f6f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.451+0000 7fcbfd1d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.451+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd400e6c0 con 0x559da2f6d080 2024-03-09T00:08:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.451+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd400f9a0 con 0x559da2f6c760 2024-03-09T00:08:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.451+0000 7fcbfd1d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd400ece0 con 0x559da2f75c60 2024-03-09T00:08:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.451+0000 7fcbfd1d9740 10 client.0 _ll_register_callbacks cb 0x7fcbdc0098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:05.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f7d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:35934/0 (socket says 172.21.15.22:35934) 2024-03-09T00:08:05.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf37fe640 1 -- 172.21.15.22:0/1460146940 learned_addr learned my addr 172.21.15.22:0/1460146940 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:05.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f6f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf3fff640 1 -- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 msgr2=0x559da2f6f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f6f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf3fff640 1 -- 172.21.15.22:0/1460146940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 msgr2=0x559da2f7d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f7d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.452+0000 7fcbf3fff640 1 -- 172.21.15.22:0/1460146940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559da2fec820 con 0x559da2f6d080 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.453+0000 7fcbf3fff640 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f6f150 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x559da2f75b40 tx=0x7fcbdc041650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c71d30ab7e76a13 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.453+0000 7fcbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.453+0000 7fcbd37fe640 1 -- 172.21.15.22:0/1460146940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc017070 con 0x559da2f6d080 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.453+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbec047e50 con 0x559da2f6d080 2024-03-09T00:08:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.454+0000 7fcbd37fe640 1 -- 172.21.15.22:0/1460146940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fcbdc013b40 con 0x559da2f6d080 2024-03-09T00:08:05.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.454+0000 7fcbd37fe640 1 -- 172.21.15.22:0/1460146940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbdc013470 con 0x559da2f6d080 2024-03-09T00:08:05.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.455+0000 7fcbfd1d9740 10 client.19887 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:05.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.455+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559da2fed950 con 0x559da2f6d080 2024-03-09T00:08:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbd37fe640 1 -- 172.21.15.22:0/1460146940 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbec047e50 con 0x559da2f6d080 2024-03-09T00:08:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbd37fe640 1 -- 172.21.15.22:0/1460146940 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcbdc0036b0 con 0x559da2f6d080 2024-03-09T00:08:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbfd1d9740 10 client.19887 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbfd1d9740 10 client.19887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbfd1d9740 20 client.19887 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2fee050 0x559da2ff0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbfd1d9740 4 client.19887 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:08:05.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559da2ed7690 con 0x559da2fee050 2024-03-09T00:08:05.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.458+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2fee050 0x559da2ff0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:05.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.460+0000 7fcbf37fe640 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2fee050 0x559da2ff0470 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63191d8a7f63a16 server_cookie=ce95c23a59b178ce in_seq=0 out_seq=0 2024-03-09T00:08:05.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.460+0000 7fcbd37fe640 10 client.19887 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.461+0000 7fcbd37fe640 1 -- 172.21.15.22:0/1460146940 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x559da2ed7690 con 0x559da2fee050 2024-03-09T00:08:05.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.461+0000 7fcbd37fe640 10 client.19887 handle_command_reply: tid=0 2024-03-09T00:08:05.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.462+0000 7fcbfd1d9740 1 client.19887 shutdown 2024-03-09T00:08:05.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.463+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 msgr2=0x559da2f6f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.463+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f6f150 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x559da2f75b40 tx=0x7fcbdc041650 comp rx=0 tx=0).stop 2024-03-09T00:08:05.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.465+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 shutdown_connections 2024-03-09T00:08:05.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.465+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559da2fee050 0x559da2ff0470 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.465+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559da2f75c60 0x559da2f6f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.465+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559da2f6d080 0x559da2f6f150 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.465+0000 7fcbfd1d9740 1 --2- 172.21.15.22:0/1460146940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559da2f6c760 0x559da2f7d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.465+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 >> 172.21.15.22:0/1460146940 conn(0x559da2f66e90 msgr2=0x559da2f71ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.465+0000 7fcbd37fe640 0 client.19887 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:05.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.466+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 shutdown_connections 2024-03-09T00:08:05.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.466+0000 7fcbfd1d9740 1 -- 172.21.15.22:0/1460146940 wait complete. 2024-03-09T00:08:05.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.466+0000 7fcbfd1d9740 20 client.19887 trim_cache size 0 max 16384 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (455 inodes in the stack)", 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout: "9ea19535-eb06-4414-b589-e511e479ba7f": { 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "9ea19535-eb06-4414-b589-e511e479ba7f", 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:08:05.462 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:08:05.463 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:08:05.463 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:08:05.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.468+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbe00a2ab0 msgr2=0x7fcbe00a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.468+0000 7fcbfa601640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbe00a2ab0 0x7fcbe00a4f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcbe409baa0 tx=0x7fcbe4010630 comp rx=0 tx=0).stop 2024-03-09T00:08:05.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.468+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 msgr2=0x7fcbf41a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:05.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.468+0000 7fcbfa601640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 0x7fcbf41a2b00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e970 tx=0x7fcbe400ed70 comp rx=0 tx=0).stop 2024-03-09T00:08:05.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.469+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 shutdown_connections 2024-03-09T00:08:05.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.469+0000 7fcbfa601640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fcbe00a2ab0 0x7fcbe00a4f70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.469+0000 7fcbfa601640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf41063b0 0x7fcbf41a2b00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.469+0000 7fcbfa601640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbf4100e90 0x7fcbf419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.469+0000 7fcbfa601640 1 --2- 172.21.15.22:0/1229962414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbf41004d0 0x7fcbf419e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:05.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.469+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 >> 172.21.15.22:0/1229962414 conn(0x7fcbf40771d0 msgr2=0x7fcbf4069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:05.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.470+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 shutdown_connections 2024-03-09T00:08:05.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:05.470+0000 7fcbfa601640 1 -- 172.21.15.22:0/1229962414 wait complete. 2024-03-09T00:08:05.483 INFO:tasks.cephfs.filesystem:scrub status for tag:9ea19535-eb06-4414-b589-e511e479ba7f - {'path': '/', 'tag': '9ea19535-eb06-4414-b589-e511e479ba7f', 'options': 'recursive,force'} 2024-03-09T00:08:06.061 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3120 26.23 MB/sec warmup 13 sec latency 1238.175 ms 2024-03-09T00:08:06.879 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: pgmap v427: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 767 op/s 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:06 smithi192 ceph-mon[35995]: osdmap e113: 12 total, 12 up, 12 in 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: pgmap v427: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 767 op/s 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:06.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:06 smithi022 ceph-mon[31894]: osdmap e113: 12 total, 12 up, 12 in 2024-03-09T00:08:07.061 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3174 24.53 MB/sec warmup 14 sec latency 1176.841 ms 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: pgmap v427: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 18 MiB/s wr, 767 op/s 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:07.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:07.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:07.111 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:06 smithi062 ceph-mon[36129]: osdmap e113: 12 total, 12 up, 12 in 2024-03-09T00:08:07.879 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:07 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:07.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:07 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:08.061 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3211 23.20 MB/sec warmup 15 sec latency 1252.507 ms 2024-03-09T00:08:08.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:07 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:08.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:08 smithi022 ceph-mon[31894]: pgmap v429: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 MiB/s wr, 701 op/s 2024-03-09T00:08:09.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3264 22.00 MB/sec warmup 16 sec latency 1299.080 ms 2024-03-09T00:08:09.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:08 smithi062 ceph-mon[36129]: pgmap v429: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 MiB/s wr, 701 op/s 2024-03-09T00:08:09.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:08 smithi192 ceph-mon[35995]: pgmap v429: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 20 MiB/s wr, 701 op/s 2024-03-09T00:08:10.061 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3345 21.13 MB/sec warmup 17 sec latency 1203.023 ms 2024-03-09T00:08:10.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:10 smithi022 ceph-mon[31894]: pgmap v430: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 675 op/s 2024-03-09T00:08:11.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3530 20.95 MB/sec warmup 18 sec latency 975.206 ms 2024-03-09T00:08:11.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:10 smithi062 ceph-mon[36129]: pgmap v430: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 675 op/s 2024-03-09T00:08:11.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:10 smithi192 ceph-mon[35995]: pgmap v430: 129 pgs: 129 active+clean; 238 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 19 MiB/s wr, 675 op/s 2024-03-09T00:08:12.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3665 20.35 MB/sec warmup 19 sec latency 490.455 ms 2024-03-09T00:08:12.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:12 smithi022 ceph-mon[31894]: pgmap v431: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 226 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 738 op/s 2024-03-09T00:08:13.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3788 19.74 MB/sec warmup 20 sec latency 307.125 ms 2024-03-09T00:08:13.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:12 smithi062 ceph-mon[36129]: pgmap v431: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 226 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 738 op/s 2024-03-09T00:08:13.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:12 smithi192 ceph-mon[35995]: pgmap v431: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 226 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 738 op/s 2024-03-09T00:08:14.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 3918 19.03 MB/sec warmup 21 sec latency 79.185 ms 2024-03-09T00:08:14.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:14 smithi022 ceph-mon[31894]: pgmap v432: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 226 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 738 op/s 2024-03-09T00:08:15.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 4083 18.23 MB/sec warmup 22 sec latency 64.795 ms 2024-03-09T00:08:15.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:14 smithi062 ceph-mon[36129]: pgmap v432: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 226 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 738 op/s 2024-03-09T00:08:15.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:14 smithi192 ceph-mon[35995]: pgmap v432: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 226 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 738 op/s 2024-03-09T00:08:15.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:15 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:08:15.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:15 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:08:15.984 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:15 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:08:16.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 4230 17.51 MB/sec warmup 23 sec latency 330.263 ms 2024-03-09T00:08:16.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:15 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:08:16.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:15 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:08:16.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:15 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:08:16.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:15 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:08:16.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:15 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:08:16.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:15 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:08:16.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:16 smithi022 ceph-mon[31894]: pgmap v433: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-03-09T00:08:17.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 4481 17.11 MB/sec warmup 24 sec latency 295.853 ms 2024-03-09T00:08:17.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:16 smithi062 ceph-mon[36129]: pgmap v433: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-03-09T00:08:17.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:16 smithi192 ceph-mon[35995]: pgmap v433: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-03-09T00:08:18.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 4903 17.31 MB/sec warmup 25 sec latency 92.866 ms 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.233+0000 7febd1620640 1 -- 172.21.15.22:0/849021874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100de0 msgr2=0x7febcc104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.233+0000 7febd1620640 1 --2- 172.21.15.22:0/849021874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100de0 0x7febcc104e20 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7febc0009900 tx=0x7febc002bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 -- 172.21.15.22:0/849021874 shutdown_connections 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 --2- 172.21.15.22:0/849021874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febcc105500 0x7febcc10d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 --2- 172.21.15.22:0/849021874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100de0 0x7febcc104e20 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 --2- 172.21.15.22:0/849021874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febcc100430 0x7febcc100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 -- 172.21.15.22:0/849021874 >> 172.21.15.22:0/849021874 conn(0x7febcc0fbcb0 msgr2=0x7febcc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 -- 172.21.15.22:0/849021874 shutdown_connections 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 -- 172.21.15.22:0/849021874 wait complete. 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.234+0000 7febd1620640 1 Processor -- start 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febd1620640 1 -- start start 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febd1620640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100430 0x7febcc1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febd1620640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febcc105500 0x7febcc196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febd1620640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febcc197150 0x7febcc1a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febd1620640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1105e0 con 0x7febcc100430 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febd1620640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc110460 con 0x7febcc105500 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febd1620640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc110760 con 0x7febcc197150 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100430 0x7febcc1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100430 0x7febcc1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47602/0 (socket says 172.21.15.22:47602) 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 -- 172.21.15.22:0/307498742 learned_addr learned my addr 172.21.15.22:0/307498742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcb7fe640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febcc197150 0x7febcc1a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febca7fc640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febcc105500 0x7febcc196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 -- 172.21.15.22:0/307498742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febcc197150 msgr2=0x7febcc1a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febcc197150 0x7febcc1a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 -- 172.21.15.22:0/307498742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febcc105500 msgr2=0x7febcc196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febcc105500 0x7febcc196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.235+0000 7febcaffd640 1 -- 172.21.15.22:0/307498742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc1a6360 con 0x7febcc100430 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.236+0000 7febcaffd640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100430 0x7febcc1966d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ea20 tx=0x7febbc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.236+0000 7febb3fff640 1 -- 172.21.15.22:0/307498742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00cd80 con 0x7febcc100430 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.236+0000 7febb3fff640 1 -- 172.21.15.22:0/307498742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc005500 con 0x7febcc100430 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.236+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc1a6650 con 0x7febcc100430 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.236+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc1a6b20 con 0x7febcc100430 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.236+0000 7febb3fff640 1 -- 172.21.15.22:0/307498742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc010690 con 0x7febcc100430 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.238+0000 7febb3fff640 1 -- 172.21.15.22:0/307498742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7febbc010830 con 0x7febcc100430 2024-03-09T00:08:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.238+0000 7febd4202740 1 Processor -- start 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febd4202740 1 -- start start 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febd4202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35dd40 0x559d4d35ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febd4202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35cfd0 0x559d4d35d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febd4202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d368940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febd4202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d2c15f0 con 0x559d4d366550 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febd4202740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d26a930 con 0x559d4d35cfd0 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febd4202740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d2bd690 con 0x559d4d35dd40 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febb3fff640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febb40a2a10 0x7febb40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febca7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d368940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35cfd0 0x559d4d35d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febca7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d368940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47610/0 (socket says 172.21.15.22:47610) 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febca7fc640 1 -- 172.21.15.22:0/1441014346 learned_addr learned my addr 172.21.15.22:0/1441014346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcb7fe640 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35dd40 0x559d4d35ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcb7fe640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febb40a2a10 0x7febb40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febb3fff640 1 -- 172.21.15.22:0/307498742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9627+0+0 (secure 0 0 0) 0x7febbc0cf650 con 0x7febcc100430 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcaffd640 1 -- 172.21.15.22:0/1441014346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35dd40 msgr2=0x559d4d35ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcaffd640 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35dd40 0x559d4d35ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcaffd640 1 -- 172.21.15.22:0/1441014346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 msgr2=0x559d4d368940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcaffd640 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d368940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcaffd640 1 -- 172.21.15.22:0/1441014346 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d4cfd6210 con 0x559d4d35cfd0 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febca7fc640 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d368940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcaffd640 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35cfd0 0x559d4d35d450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7febbc0cc950 tx=0x7febbc06e540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f058429ea3a65ac server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.239+0000 7febcb7fe640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febb40a2a10 0x7febb40a4ed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7febc4009fd0 tx=0x7febc4009600 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febb17fa640 1 -- 172.21.15.22:0/1441014346 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc09b1c0 con 0x559d4d35cfd0 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febb17fa640 1 -- 172.21.15.22:0/1441014346 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc09bf60 con 0x559d4d35cfd0 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febb17fa640 1 -- 172.21.15.22:0/1441014346 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc0ccd30 con 0x559d4d35cfd0 2024-03-09T00:08:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 -- 172.21.15.22:0/1441014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35cfd0 msgr2=0x559d4d35d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35cfd0 0x559d4d35d450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7febbc0cc950 tx=0x7febbc06e540 comp rx=0 tx=0).stop 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 -- 172.21.15.22:0/1441014346 shutdown_connections 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d368940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35cfd0 0x559d4d35d450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 --2- 172.21.15.22:0/1441014346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35dd40 0x559d4d35ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 -- 172.21.15.22:0/1441014346 >> 172.21.15.22:0/1441014346 conn(0x559d4d3245b0 msgr2=0x559d4d357cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 -- 172.21.15.22:0/1441014346 shutdown_connections 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.240+0000 7febd4202740 1 -- 172.21.15.22:0/1441014346 wait complete. 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 -- start start 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 Processor -- start 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d363650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d363b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d3640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d369220 con 0x559d4d35cfd0 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d27dd90 con 0x559d4d35dd40 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4d3693a0 con 0x559d4d366550 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febd4202740 10 client.0 _ll_register_callbacks cb 0x559d4cfd5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d363650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d363650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47618/0 (socket says 172.21.15.22:47618) 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 -- 172.21.15.22:0/3572045186 learned_addr learned my addr 172.21.15.22:0/3572045186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febca7fc640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d363b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcb7fe640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d3640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 -- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 msgr2=0x559d4d3640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d3640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 -- 172.21.15.22:0/3572045186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 msgr2=0x559d4d363b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d363b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcaffd640 1 -- 172.21.15.22:0/3572045186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d4d0a2620 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.241+0000 7febcb7fe640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d3640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febcaffd640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d363650 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9f0 tx=0x7febbc097c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f38ce7278119c75 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febaeff5640 1 -- 172.21.15.22:0/3572045186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc09b860 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febaeff5640 1 -- 172.21.15.22:0/3572045186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9627+0+0 (secure 0 0 0) 0x7febbc0046c0 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febbc09bf60 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febaeff5640 1 -- 172.21.15.22:0/3572045186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7febbc09bac0 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 10 client.19899 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559d4d3755e0 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febaeff5640 1 -- 172.21.15.22:0/3572045186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc09bf60 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febaeff5640 1 -- 172.21.15.22:0/3572045186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7febbc09cb90 con 0x559d4d35cfd0 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 10 client.19899 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 10 client.19899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 20 client.19899 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d376930 0x559d4d378d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 4 client.19899 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d4d2ceb30 con 0x559d4d376930 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.242+0000 7febca7fc640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d376930 0x559d4d378d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.243+0000 7febaeff5640 10 client.19899 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.243+0000 7febca7fc640 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d376930 0x559d4d378d20 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb46cc81dd591d3 server_cookie=82f19492f8de7d5a in_seq=0 out_seq=0 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febaeff5640 1 -- 172.21.15.22:0/3572045186 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d4d2ceb30 con 0x559d4d376930 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febaeff5640 10 client.19899 handle_command_reply: tid=0 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 client.19899 shutdown 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 msgr2=0x559d4d363650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d363650 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9f0 tx=0x7febbc097c30 comp rx=0 tx=0).stop 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 shutdown_connections 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d376930 0x559d4d378d20 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d3640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d363b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 --2- 172.21.15.22:0/3572045186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d363650 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 >> 172.21.15.22:0/3572045186 conn(0x559d4d3245b0 msgr2=0x559d4d359a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febaeff5640 0 client.19899 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 shutdown_connections 2024-03-09T00:08:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.245+0000 7febd4202740 1 -- 172.21.15.22:0/3572045186 wait complete. 2024-03-09T00:08:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.246+0000 7febd4202740 20 client.19899 trim_cache size 0 max 16384 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.259+0000 7febd4202740 1 Processor -- start 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.259+0000 7febd4202740 1 -- start start 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febd4202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35cfd0 0x559d4d365d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febd4202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d37a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febd4202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d37ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febd4202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1b2f10 con 0x559d4d366550 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febd4202740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1b0a50 con 0x559d4d35dd40 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febd4202740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1aeb10 con 0x559d4d35cfd0 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d37ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febca7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d37a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d37ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47628/0 (socket says 172.21.15.22:47628) 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcaffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35cfd0 0x559d4d365d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 -- 172.21.15.22:0/2949590572 learned_addr learned my addr 172.21.15.22:0/2949590572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 -- 172.21.15.22:0/2949590572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35cfd0 msgr2=0x559d4d365d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35cfd0 0x559d4d365d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 -- 172.21.15.22:0/2949590572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 msgr2=0x559d4d37a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d37a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcb7fe640 1 -- 172.21.15.22:0/2949590572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d4d35c2a0 con 0x559d4d366550 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febcaffd640 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35cfd0 0x559d4d365d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.260+0000 7febca7fc640 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d37a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.261+0000 7febcb7fe640 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d37ae90 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7febc40128a0 tx=0x7febc400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0543b5152cd8fb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.261+0000 7febb0ff9640 1 -- 172.21.15.22:0/2949590572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc400acd0 con 0x559d4d366550 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.261+0000 7febb0ff9640 1 -- 172.21.15.22:0/2949590572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febc401e970 con 0x559d4d366550 2024-03-09T00:08:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.261+0000 7febb0ff9640 1 -- 172.21.15.22:0/2949590572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4005410 con 0x559d4d366550 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 -- 172.21.15.22:0/2949590572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 msgr2=0x559d4d37ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d37ae90 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7febc40128a0 tx=0x7febc400c000 comp rx=0 tx=0).stop 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 -- 172.21.15.22:0/2949590572 shutdown_connections 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d366550 0x559d4d37ae90 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d37a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 --2- 172.21.15.22:0/2949590572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d35cfd0 0x559d4d365d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 -- 172.21.15.22:0/2949590572 >> 172.21.15.22:0/2949590572 conn(0x559d4d3245b0 msgr2=0x559d4d361f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 -- 172.21.15.22:0/2949590572 shutdown_connections 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 -- 172.21.15.22:0/2949590572 wait complete. 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 -- start start 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.262+0000 7febd4202740 1 Processor -- start 2024-03-09T00:08:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febd4202740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d37a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febd4202740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d35f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febd4202740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d35fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febd4202740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1af110 con 0x559d4d35cfd0 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febd4202740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1b03f0 con 0x559d4d35dd40 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febd4202740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1af730 con 0x559d4d366550 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febd4202740 10 client.0 _ll_register_callbacks cb 0x7febbc004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d37a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d35fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d37a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47644/0 (socket says 172.21.15.22:47644) 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 -- 172.21.15.22:0/4125354412 learned_addr learned my addr 172.21.15.22:0/4125354412 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febca7fc640 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d35f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 -- 172.21.15.22:0/4125354412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 msgr2=0x559d4d35fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d35fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 -- 172.21.15.22:0/4125354412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 msgr2=0x559d4d35f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d35f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 -- 172.21.15.22:0/4125354412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d4d359fc0 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febca7fc640 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d35f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febcaffd640 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d37a960 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7febbc0cf1e0 tx=0x7febbc099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed347c626cd6216c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febaeff5640 1 -- 172.21.15.22:0/4125354412 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc09b630 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.263+0000 7febaeff5640 1 -- 172.21.15.22:0/4125354412 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9627+0+0 (secure 0 0 0) 0x7febbc09cb90 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febc401e970 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febaeff5640 1 -- 172.21.15.22:0/4125354412 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7febbc0cf300 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 10 client.19905 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559d4d3cd1d0 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febaeff5640 1 -- 172.21.15.22:0/4125354412 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febc401e970 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febaeff5640 1 -- 172.21.15.22:0/4125354412 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7febbc0661e0 con 0x559d4d35cfd0 2024-03-09T00:08:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 10 client.19905 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 10 client.19905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 20 client.19905 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d3cd700 0x559d4d3cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 4 client.19905 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d4d2c8e80 con 0x559d4d3cd700 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.264+0000 7febca7fc640 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d3cd700 0x559d4d3cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.265+0000 7febca7fc640 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d3cd700 0x559d4d3cfb20 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed1006f1ba3f277 server_cookie=d922a7327f969b50 in_seq=0 out_seq=0 2024-03-09T00:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.265+0000 7febaeff5640 10 client.19905 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.855+0000 7febaeff5640 1 -- 172.21.15.22:0/4125354412 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d4d2c8e80 con 0x559d4d3cd700 2024-03-09T00:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.855+0000 7febaeff5640 10 client.19905 handle_command_reply: tid=0 2024-03-09T00:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 client.19905 shutdown 2024-03-09T00:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 msgr2=0x559d4d37a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d37a960 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7febbc0cf1e0 tx=0x7febbc099d70 comp rx=0 tx=0).stop 2024-03-09T00:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 shutdown_connections 2024-03-09T00:08:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559d4d3cd700 0x559d4d3cfb20 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febaeff5640 0 client.19905 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d4d366550 0x559d4d35fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d4d35dd40 0x559d4d35f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 --2- 172.21.15.22:0/4125354412 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559d4d35cfd0 0x559d4d37a960 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 >> 172.21.15.22:0/4125354412 conn(0x559d4d3245b0 msgr2=0x559d4d371ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 shutdown_connections 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.856+0000 7febd4202740 1 -- 172.21.15.22:0/4125354412 wait complete. 2024-03-09T00:08:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd4202740 20 client.19905 trim_cache size 0 max 16384 2024-03-09T00:08:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:08:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:08:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:08:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:08:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febb40a2a10 msgr2=0x7febb40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febb40a2a10 0x7febb40a4ed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7febc4009fd0 tx=0x7febc4009600 comp rx=0 tx=0).stop 2024-03-09T00:08:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100430 msgr2=0x7febcc1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100430 0x7febcc1966d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7febbc00ea20 tx=0x7febbc00ee20 comp rx=0 tx=0).stop 2024-03-09T00:08:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 shutdown_connections 2024-03-09T00:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7febb40a2a10 0x7febb40a4ed0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febcc197150 0x7febcc1a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febcc105500 0x7febcc196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.857+0000 7febd1620640 1 --2- 172.21.15.22:0/307498742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febcc100430 0x7febcc1966d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.858+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 >> 172.21.15.22:0/307498742 conn(0x7febcc0fbcb0 msgr2=0x7febcc0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.858+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 shutdown_connections 2024-03-09T00:08:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:18.858+0000 7febd1620640 1 -- 172.21.15.22:0/307498742 wait complete. 2024-03-09T00:08:18.983 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:18 smithi022 ceph-mon[31894]: pgmap v434: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-03-09T00:08:19.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 5195 17.36 MB/sec warmup 26 sec latency 470.588 ms 2024-03-09T00:08:19.110 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:18 smithi062 ceph-mon[36129]: pgmap v434: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-03-09T00:08:19.129 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:18 smithi192 ceph-mon[35995]: pgmap v434: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-03-09T00:08:20.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 5783 17.31 MB/sec warmup 27 sec latency 180.183 ms 2024-03-09T00:08:20.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:19 smithi022 ceph-mon[31894]: osdmap e114: 12 total, 12 up, 12 in 2024-03-09T00:08:20.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:19 smithi022 ceph-mon[31894]: pgmap v436: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-03-09T00:08:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:19 smithi062 ceph-mon[36129]: osdmap e114: 12 total, 12 up, 12 in 2024-03-09T00:08:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:19 smithi062 ceph-mon[36129]: pgmap v436: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-03-09T00:08:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:19 smithi192 ceph-mon[35995]: osdmap e114: 12 total, 12 up, 12 in 2024-03-09T00:08:20.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:19 smithi192 ceph-mon[35995]: pgmap v436: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-03-09T00:08:21.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 5897 16.93 MB/sec warmup 28 sec latency 792.797 ms 2024-03-09T00:08:22.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 6001 16.53 MB/sec warmup 29 sec latency 1442.461 ms 2024-03-09T00:08:22.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:22 smithi022 ceph-mon[31894]: pgmap v437: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 367 op/s 2024-03-09T00:08:22.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:22 smithi062 ceph-mon[36129]: pgmap v437: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 367 op/s 2024-03-09T00:08:22.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:22 smithi192 ceph-mon[35995]: pgmap v437: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 367 op/s 2024-03-09T00:08:23.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 6314 16.96 MB/sec warmup 30 sec latency 284.445 ms 2024-03-09T00:08:24.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 6738 17.52 MB/sec warmup 31 sec latency 262.840 ms 2024-03-09T00:08:24.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:24 smithi022 ceph-mon[31894]: pgmap v438: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 367 op/s 2024-03-09T00:08:24.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:24 smithi062 ceph-mon[36129]: pgmap v438: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 367 op/s 2024-03-09T00:08:24.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:24 smithi192 ceph-mon[35995]: pgmap v438: 129 pgs: 129 active+clean; 231 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 367 op/s 2024-03-09T00:08:25.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 7007 17.56 MB/sec warmup 32 sec latency 251.343 ms 2024-03-09T00:08:26.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 7222 17.48 MB/sec warmup 33 sec latency 172.468 ms 2024-03-09T00:08:26.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:26 smithi022 ceph-mon[31894]: pgmap v439: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-09T00:08:26.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:26 smithi062 ceph-mon[36129]: pgmap v439: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-09T00:08:26.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:26 smithi192 ceph-mon[35995]: pgmap v439: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-09T00:08:27.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 7414 17.21 MB/sec warmup 34 sec latency 62.646 ms 2024-03-09T00:08:28.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 7587 16.87 MB/sec warmup 35 sec latency 159.045 ms 2024-03-09T00:08:28.431 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:28 smithi192 ceph-mon[35995]: pgmap v440: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-09T00:08:28.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:28 smithi022 ceph-mon[31894]: pgmap v440: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-09T00:08:28.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:28 smithi062 ceph-mon[36129]: pgmap v440: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-03-09T00:08:29.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 7780 16.49 MB/sec warmup 36 sec latency 299.804 ms 2024-03-09T00:08:30.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 8058 16.31 MB/sec warmup 37 sec latency 213.133 ms 2024-03-09T00:08:30.411 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:30 smithi062 ceph-mon[36129]: pgmap v441: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-03-09T00:08:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:30 smithi022 ceph-mon[31894]: pgmap v441: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-03-09T00:08:30.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:30 smithi192 ceph-mon[35995]: pgmap v441: 129 pgs: 129 active+clean; 294 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-03-09T00:08:31.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 8553 16.70 MB/sec warmup 38 sec latency 187.457 ms 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 -- 172.21.15.22:0/2121625092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100440 msgr2=0x7fae341008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 --2- 172.21.15.22:0/2121625092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100440 0x7fae341008a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fae24009900 tx=0x7fae2402bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 -- 172.21.15.22:0/2121625092 shutdown_connections 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 --2- 172.21.15.22:0/2121625092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae34100de0 0x7fae3410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 --2- 172.21.15.22:0/2121625092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100440 0x7fae341008a0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 --2- 172.21.15.22:0/2121625092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34105170 0x7fae34105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 -- 172.21.15.22:0/2121625092 >> 172.21.15.22:0/2121625092 conn(0x7fae34076fc0 msgr2=0x7fae340773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 -- 172.21.15.22:0/2121625092 shutdown_connections 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.950+0000 7fae3c35b640 1 -- 172.21.15.22:0/2121625092 wait complete. 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 Processor -- start 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 -- start start 2024-03-09T00:08:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34100440 0x7fae3419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100de0 0x7fae3419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae34105170 0x7fae34196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae34110620 con 0x7fae34100de0 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341104a0 con 0x7fae34100440 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3c35b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341107a0 con 0x7fae34105170 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100de0 0x7fae3419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100de0 0x7fae3419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54646/0 (socket says 172.21.15.22:54646) 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 -- 172.21.15.22:0/1993916576 learned_addr learned my addr 172.21.15.22:0/1993916576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae34105170 0x7fae34196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34100440 0x7fae3419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 -- 172.21.15.22:0/1993916576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae34105170 msgr2=0x7fae34196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae34105170 0x7fae34196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 -- 172.21.15.22:0/1993916576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34100440 msgr2=0x7fae3419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34100440 0x7fae3419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 -- 172.21.15.22:0/1993916576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34197080 con 0x7fae34100de0 2024-03-09T00:08:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34100440 0x7fae3419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:08:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.951+0000 7fae398f6640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100de0 0x7fae3419cc40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fae24006010 tx=0x7fae2402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.952+0000 7fae2b7fe640 1 -- 172.21.15.22:0/1993916576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24041070 con 0x7fae34100de0 2024-03-09T00:08:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.952+0000 7fae2b7fe640 1 -- 172.21.15.22:0/1993916576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae24004450 con 0x7fae34100de0 2024-03-09T00:08:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.952+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae34197310 con 0x7fae34100de0 2024-03-09T00:08:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.952+0000 7fae2b7fe640 1 -- 172.21.15.22:0/1993916576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24034440 con 0x7fae34100de0 2024-03-09T00:08:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.952+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae341a35c0 con 0x7fae34100de0 2024-03-09T00:08:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.954+0000 7fae2b7fe640 1 -- 172.21.15.22:0/1993916576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fae24044070 con 0x7fae34100de0 2024-03-09T00:08:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 Processor -- start 2024-03-09T00:08:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 -- start start 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae2b7fe640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fae200a2a90 0x7fae200a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345eed40 0x5624345eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345ee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345f7550 0x5624345f9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624345525f0 con 0x5624345edfd0 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624344fb930 con 0x5624345eed40 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3f018740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243454e690 con 0x5624345f7550 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.955+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fae200a2a90 0x7fae200a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345eed40 0x5624345eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae2b7fe640 1 -- 172.21.15.22:0/1993916576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fae240f0d20 con 0x7fae34100de0 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345eed40 0x5624345eda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:37832/0 (socket says 172.21.15.22:37832) 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3874425090 learned_addr learned my addr 172.21.15.22:0/3874425090 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345f7550 0x5624345f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae398f6640 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3874425090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345f7550 msgr2=0x5624345f9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345f7550 0x5624345f9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3874425090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 msgr2=0x5624345ee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345ee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3874425090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562434267210 con 0x5624345eed40 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345eed40 0x5624345eda90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae3000bcc0 tx=0x7fae3000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b53ca1f06a810b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fae200a2a90 0x7fae200a4f50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0092f0 tx=0x7fae2c009e20 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae28bfe640 1 -- 172.21.15.22:0/3874425090 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30005020 con 0x5624345eed40 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae28bfe640 1 -- 172.21.15.22:0/3874425090 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae300047a0 con 0x5624345eed40 2024-03-09T00:08:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.956+0000 7fae28bfe640 1 -- 172.21.15.22:0/3874425090 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30011680 con 0x5624345eed40 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.957+0000 7fae3f018740 1 -- 172.21.15.22:0/3874425090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345eed40 msgr2=0x5624345eda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.957+0000 7fae3f018740 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345eed40 0x5624345eda90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fae3000bcc0 tx=0x7fae3000f730 comp rx=0 tx=0).stop 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.957+0000 7fae3f018740 1 -- 172.21.15.22:0/3874425090 shutdown_connections 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.957+0000 7fae3f018740 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345f7550 0x5624345f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.957+0000 7fae3f018740 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345ee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.957+0000 7fae3f018740 1 --2- 172.21.15.22:0/3874425090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345eed40 0x5624345eda90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.957+0000 7fae3f018740 1 -- 172.21.15.22:0/3874425090 >> 172.21.15.22:0/3874425090 conn(0x5624345b55b0 msgr2=0x5624345e8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 -- 172.21.15.22:0/3874425090 shutdown_connections 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 -- 172.21.15.22:0/3874425090 wait complete. 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 -- start start 2024-03-09T00:08:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 Processor -- start 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345f3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624346075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624345fa180 con 0x5624345edfd0 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3a0f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243450ed90 con 0x5624345f7550 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3f018740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624345fa300 con 0x5624345eed40 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3a0f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54662/0 (socket says 172.21.15.22:54662) 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.958+0000 7fae3a0f7640 1 -- 172.21.15.22:0/1649668035 learned_addr learned my addr 172.21.15.22:0/1649668035 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624346075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae398f6640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3f018740 10 client.0 _ll_register_callbacks cb 0x562434266a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a8f8640 1 -- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 msgr2=0x5624345f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a8f8640 1 -- 172.21.15.22:0/1649668035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 msgr2=0x5624345f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a8f8640 1 -- 172.21.15.22:0/1649668035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562434333620 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624346075e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5624345eea70 tx=0x7fae2c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed1a45f6fb93307 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae1dffb640 1 -- 172.21.15.22:0/1649668035 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c01ee60 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae1dffb640 1 -- 172.21.15.22:0/1649668035 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fae2c02a410 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.959+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae300047a0 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae1dffb640 1 -- 172.21.15.22:0/1649668035 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae2c033050 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 10 client.29177 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56243460a900 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae1dffb640 1 -- 172.21.15.22:0/1649668035 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae300047a0 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae1dffb640 1 -- 172.21.15.22:0/1649668035 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fae2c01f070 con 0x5624345f7550 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 10 client.29177 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 10 client.29177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 20 client.29177 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56243460bdc0 0x56243460e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 4 client.29177 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56243455fb30 con 0x56243460bdc0 2024-03-09T00:08:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.960+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56243460bdc0 0x56243460e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.961+0000 7fae1dffb640 10 client.29177 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.961+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56243460bdc0 0x56243460e1d0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ee3412e7817f0f server_cookie=93771a3392d85031 in_seq=0 out_seq=0 2024-03-09T00:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae1dffb640 1 -- 172.21.15.22:0/1649668035 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56243455fb30 con 0x56243460bdc0 2024-03-09T00:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae1dffb640 10 client.29177 handle_command_reply: tid=0 2024-03-09T00:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 client.29177 shutdown 2024-03-09T00:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 msgr2=0x5624346075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624346075e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5624345eea70 tx=0x7fae2c009650 comp rx=0 tx=0).stop 2024-03-09T00:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 shutdown_connections 2024-03-09T00:08:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56243460bdc0 0x56243460e1d0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624346075e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 --2- 172.21.15.22:0/1649668035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624345f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae1dffb640 0 client.29177 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 >> 172.21.15.22:0/1649668035 conn(0x5624345b55b0 msgr2=0x5624345eaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 shutdown_connections 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 1 -- 172.21.15.22:0/1649668035 wait complete. 2024-03-09T00:08:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.963+0000 7fae3f018740 20 client.29177 trim_cache size 0 max 16384 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 Processor -- start 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 -- start start 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345edfd0 0x5624345f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345f7550 0x562434607070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341ae8a0 con 0x5624345f7550 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341ac3e0 con 0x5624345edfd0 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3f018740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341aa4a0 con 0x5624345eed40 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3a8f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345f7550 0x562434607070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae398f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3a8f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345f7550 0x562434607070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54664/0 (socket says 172.21.15.22:54664) 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae398f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51352/0 (socket says 172.21.15.22:51352) 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.978+0000 7fae3a8f8640 1 -- 172.21.15.22:0/3365945301 learned_addr learned my addr 172.21.15.22:0/3365945301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345edfd0 0x5624345f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a8f8640 1 -- 172.21.15.22:0/3365945301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 msgr2=0x5624345fb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a8f8640 1 -- 172.21.15.22:0/3365945301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345edfd0 msgr2=0x5624345f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345edfd0 0x5624345f0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a8f8640 1 -- 172.21.15.22:0/3365945301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624345f81c0 con 0x5624345f7550 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345edfd0 0x5624345f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345f7550 0x562434607070 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00a010 tx=0x7fae2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe23ca93c7210eb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae1ffff640 1 -- 172.21.15.22:0/3365945301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c03c070 con 0x5624345f7550 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae1ffff640 1 -- 172.21.15.22:0/3365945301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae2c012920 con 0x5624345f7550 2024-03-09T00:08:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.979+0000 7fae1ffff640 1 -- 172.21.15.22:0/3365945301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c01fe00 con 0x5624345f7550 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 -- 172.21.15.22:0/3365945301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345f7550 msgr2=0x562434607070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345f7550 0x562434607070 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00a010 tx=0x7fae2c004a40 comp rx=0 tx=0).stop 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 -- 172.21.15.22:0/3365945301 shutdown_connections 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345f7550 0x562434607070 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 --2- 172.21.15.22:0/3365945301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345edfd0 0x5624345f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 -- 172.21.15.22:0/3365945301 >> 172.21.15.22:0/3365945301 conn(0x5624345b55b0 msgr2=0x5624345f86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 -- 172.21.15.22:0/3365945301 shutdown_connections 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 -- 172.21.15.22:0/3365945301 wait complete. 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 -- start start 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.980+0000 7fae3f018740 1 Processor -- start 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3f018740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624346049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3f018740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3f018740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624345fab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3f018740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341aaaa0 con 0x5624345edfd0 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3f018740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341abd80 con 0x5624345f7550 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3f018740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341ab0c0 con 0x5624345eed40 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3f018740 10 client.0 _ll_register_callbacks cb 0x7fae3000b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624346049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624346049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54666/0 (socket says 172.21.15.22:54666) 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3079068032 learned_addr learned my addr 172.21.15.22:0/3079068032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae398f6640 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a8f8640 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624345fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 msgr2=0x5624345fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3079068032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 msgr2=0x5624345fab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624345fab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 -- 172.21.15.22:0/3079068032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624345fb350 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae3a0f7640 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624346049e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b840 tx=0x7fae3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e489af0e8467d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.981+0000 7fae1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae1dffb640 1 -- 172.21.15.22:0/3079068032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30028640 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae1dffb640 1 -- 172.21.15.22:0/3079068032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fae300287e0 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae2c012920 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae1dffb640 1 -- 172.21.15.22:0/3079068032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae30005020 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 10 client.19923 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562434663360 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae1dffb640 1 -- 172.21.15.22:0/3079068032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae2c012920 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae1dffb640 1 -- 172.21.15.22:0/3079068032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fae30028ce0 con 0x5624345edfd0 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 10 client.19923 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 10 client.19923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 20 client.19923 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5624346637b0 0x562434665ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 4 client.19923 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:08:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562434559e80 con 0x5624346637b0 2024-03-09T00:08:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.982+0000 7fae398f6640 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5624346637b0 0x562434665ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.983+0000 7fae398f6640 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5624346637b0 0x562434665ba0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35a517a0fd0c705 server_cookie=35502ddc70a5d139 in_seq=0 out_seq=0 2024-03-09T00:08:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:31.983+0000 7fae1dffb640 10 client.19923 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:32.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 9120 16.65 MB/sec warmup 39 sec latency 185.025 ms 2024-03-09T00:08:32.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:32 smithi022 ceph-mon[31894]: pgmap v442: 129 pgs: 129 active+clean; 300 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-03-09T00:08:32.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:32 smithi062 ceph-mon[36129]: pgmap v442: 129 pgs: 129 active+clean; 300 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-03-09T00:08:32.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:32 smithi192 ceph-mon[35995]: pgmap v442: 129 pgs: 129 active+clean; 300 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-03-09T00:08:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae1dffb640 1 -- 172.21.15.22:0/3079068032 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562434559e80 con 0x5624346637b0 2024-03-09T00:08:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae1dffb640 10 client.19923 handle_command_reply: tid=0 2024-03-09T00:08:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 client.19923 shutdown 2024-03-09T00:08:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 msgr2=0x5624346049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624346049e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b840 tx=0x7fae3000b960 comp rx=0 tx=0).stop 2024-03-09T00:08:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 shutdown_connections 2024-03-09T00:08:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5624346637b0 0x562434665ba0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624345f7550 0x5624345fab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae1dffb640 0 client.19923 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624345eed40 0x5624345fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 --2- 172.21.15.22:0/3079068032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5624345edfd0 0x5624346049e0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.753+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 >> 172.21.15.22:0/3079068032 conn(0x5624345b55b0 msgr2=0x562434607070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.754+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 shutdown_connections 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.754+0000 7fae3f018740 1 -- 172.21.15.22:0/3079068032 wait complete. 2024-03-09T00:08:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.754+0000 7fae3f018740 20 client.19923 trim_cache size 0 max 16384 2024-03-09T00:08:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:08:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:08:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:08:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.754+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fae200a2a90 msgr2=0x7fae200a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fae200a2a90 0x7fae200a4f50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0092f0 tx=0x7fae2c009e20 comp rx=0 tx=0).stop 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100de0 msgr2=0x7fae3419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100de0 0x7fae3419cc40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fae24006010 tx=0x7fae2402cc70 comp rx=0 tx=0).stop 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 shutdown_connections 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fae200a2a90 0x7fae200a4f50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae34105170 0x7fae34196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae34100de0 0x7fae3419cc40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 --2- 172.21.15.22:0/1993916576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34100440 0x7fae3419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 >> 172.21.15.22:0/1993916576 conn(0x7fae34076fc0 msgr2=0x7fae340fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 shutdown_connections 2024-03-09T00:08:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:32.755+0000 7fae3c35b640 1 -- 172.21.15.22:0/1993916576 wait complete. 2024-03-09T00:08:33.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 9300 16.51 MB/sec warmup 40 sec latency 569.560 ms 2024-03-09T00:08:34.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 9686 16.88 MB/sec warmup 41 sec latency 144.250 ms 2024-03-09T00:08:34.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:34 smithi022 ceph-mon[31894]: pgmap v443: 129 pgs: 129 active+clean; 300 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.9 MiB/s wr, 367 op/s 2024-03-09T00:08:34.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:34 smithi062 ceph-mon[36129]: pgmap v443: 129 pgs: 129 active+clean; 300 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.9 MiB/s wr, 367 op/s 2024-03-09T00:08:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:34 smithi192 ceph-mon[35995]: pgmap v443: 129 pgs: 129 active+clean; 300 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.9 MiB/s wr, 367 op/s 2024-03-09T00:08:35.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 10071 17.22 MB/sec warmup 42 sec latency 321.034 ms 2024-03-09T00:08:35.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:08:35.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:08:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 -- 172.21.15.22:0/1840336471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4301054e0 msgr2=0x7fe43010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 --2- 172.21.15.22:0/1840336471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4301054e0 0x7fe43010d0b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe428009ca0 tx=0x7fe42802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 -- 172.21.15.22:0/1840336471 shutdown_connections 2024-03-09T00:08:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 --2- 172.21.15.22:0/1840336471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4301054e0 0x7fe43010d0b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 --2- 172.21.15.22:0/1840336471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100dc0 0x7fe430104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 --2- 172.21.15.22:0/1840336471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe430100410 0x7fe4301007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 -- 172.21.15.22:0/1840336471 >> 172.21.15.22:0/1840336471 conn(0x7fe430076570 msgr2=0x7fe430076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 -- 172.21.15.22:0/1840336471 shutdown_connections 2024-03-09T00:08:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.601+0000 7fe43590a640 1 -- 172.21.15.22:0/1840336471 wait complete. 2024-03-09T00:08:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 Processor -- start 2024-03-09T00:08:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 -- start start 2024-03-09T00:08:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100410 0x7fe43019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4301054e0 0x7fe43019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe430196880 0x7fe430196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301105a0 con 0x7fe430196880 2024-03-09T00:08:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe430110420 con 0x7fe4301054e0 2024-03-09T00:08:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe43590a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe430110720 con 0x7fe430100410 2024-03-09T00:08:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe42f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe430196880 0x7fe430196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe42effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100410 0x7fe43019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe4267fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4301054e0 0x7fe43019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe42effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100410 0x7fe43019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51382/0 (socket says 172.21.15.22:51382) 2024-03-09T00:08:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe42f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe430196880 0x7fe430196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54678/0 (socket says 172.21.15.22:54678) 2024-03-09T00:08:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.602+0000 7fe42effd640 1 -- 172.21.15.22:0/1509876565 learned_addr learned my addr 172.21.15.22:0/1509876565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe42f7fe640 1 -- 172.21.15.22:0/1509876565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100410 msgr2=0x7fe43019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100410 0x7fe43019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe42f7fe640 1 -- 172.21.15.22:0/1509876565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4301054e0 msgr2=0x7fe43019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4301054e0 0x7fe43019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe42f7fe640 1 -- 172.21.15.22:0/1509876565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4301973a0 con 0x7fe430196880 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe42effd640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100410 0x7fe43019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe430196880 0x7fe430196d30 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe42802c700 tx=0x7fe428032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe434908640 1 -- 172.21.15.22:0/1509876565 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428041020 con 0x7fe430196880 2024-03-09T00:08:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe434908640 1 -- 172.21.15.22:0/1509876565 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe428034cd0 con 0x7fe430196880 2024-03-09T00:08:35.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe430197630 con 0x7fe430196880 2024-03-09T00:08:35.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe434908640 1 -- 172.21.15.22:0/1509876565 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428039040 con 0x7fe430196880 2024-03-09T00:08:35.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.603+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4301a3820 con 0x7fe430196880 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.606+0000 7fe434908640 1 -- 172.21.15.22:0/1509876565 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe428044020 con 0x7fe430196880 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.606+0000 7fe4385be740 1 Processor -- start 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.606+0000 7fe4385be740 1 -- start start 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.606+0000 7fe434908640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe41c0a2970 0x7fe41c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.606+0000 7fe4385be740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c6d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.606+0000 7fe4385be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.606+0000 7fe4385be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c7b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe4385be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76bd0e00 con 0x559a76c6dd40 2024-03-09T00:08:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe4385be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76b79b10 con 0x559a76c78e70 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe4385be740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76bccea0 con 0x559a76c6d420 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe4267fc640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe41c0a2970 0x7fe41c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe434908640 1 -- 172.21.15.22:0/1509876565 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe4280f0f30 con 0x7fe430196880 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c6d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe4267fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c6d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51384/0 (socket says 172.21.15.22:51384) 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 -- 172.21.15.22:0/1121841658 learned_addr learned my addr 172.21.15.22:0/1121841658 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c7b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 -- 172.21.15.22:0/1121841658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 msgr2=0x559a76c7b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c7b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 -- 172.21.15.22:0/1121841658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 msgr2=0x559a76c78930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 -- 172.21.15.22:0/1121841658 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a769af7d0 con 0x559a76c6d420 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c7b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe4267fc640 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe4267fc640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe41c0a2970 0x7fe41c0a4e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x559a76c68eb0 tx=0x7fe418004730 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe42effd640 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c6d800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe420010580 tx=0x7fe420010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9db75470676075b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.607+0000 7fe425ffb640 1 -- 172.21.15.22:0/1121841658 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42000e830 con 0x559a76c6d420 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe425ffb640 1 -- 172.21.15.22:0/1121841658 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe420011440 con 0x559a76c6d420 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe425ffb640 1 -- 172.21.15.22:0/1121841658 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42001b690 con 0x559a76c6d420 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 -- 172.21.15.22:0/1121841658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 msgr2=0x559a76c6d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c6d800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe420010580 tx=0x7fe420010a40 comp rx=0 tx=0).stop 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 -- 172.21.15.22:0/1121841658 shutdown_connections 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c7b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 --2- 172.21.15.22:0/1121841658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c6d800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 -- 172.21.15.22:0/1121841658 >> 172.21.15.22:0/1121841658 conn(0x559a76c67e90 msgr2=0x559a76c68270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 -- 172.21.15.22:0/1121841658 shutdown_connections 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.608+0000 7fe4385be740 1 -- 172.21.15.22:0/1121841658 wait complete. 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 -- start start 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 Processor -- start 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6d420 0x559a76c78530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6dd40 0x559a76c758c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c75e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76c7bc60 con 0x559a76c6d420 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76b8d0a0 con 0x559a76c6dd40 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76c7bde0 con 0x559a76c78e70 2024-03-09T00:08:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4385be740 10 client.0 _ll_register_callbacks cb 0x559a76b8ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe42effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6d420 0x559a76c78530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe42effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6d420 0x559a76c78530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54694/0 (socket says 172.21.15.22:54694) 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe4267fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6dd40 0x559a76c758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe42effd640 1 -- 172.21.15.22:0/44181178 learned_addr learned my addr 172.21.15.22:0/44181178 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.609+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c75e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe42effd640 1 -- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 msgr2=0x559a76c75e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe42effd640 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c75e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe42effd640 1 -- 172.21.15.22:0/44181178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6dd40 msgr2=0x559a76c758c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe42effd640 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6dd40 0x559a76c758c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe42effd640 1 -- 172.21.15.22:0/44181178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a768e53f0 con 0x559a76c6d420 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe42effd640 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6d420 0x559a76c78530 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe420011d40 tx=0x7fe420011820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c97634fbba00b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe40effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe40effd640 1 -- 172.21.15.22:0/44181178 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420004220 con 0x559a76c6d420 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe420011440 con 0x559a76c6d420 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe40effd640 1 -- 172.21.15.22:0/44181178 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe4200043c0 con 0x559a76c6d420 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe40effd640 1 -- 172.21.15.22:0/44181178 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe420030050 con 0x559a76c6d420 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe4385be740 10 client.19935 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:35.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559a768a50f0 con 0x559a76c6d420 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.610+0000 7fe40effd640 1 -- 172.21.15.22:0/44181178 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe420011440 con 0x559a76c6d420 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe40effd640 1 -- 172.21.15.22:0/44181178 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe420004a30 con 0x559a76c6d420 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4385be740 10 client.19935 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4385be740 10 client.19935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4385be740 20 client.19935 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4385be740 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76c88860 0x559a76c8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4385be740 4 client.19935 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:35.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a76bde340 con 0x559a76c88860 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4267fc640 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76c88860 0x559a76c8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe40effd640 10 client.19935 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.611+0000 7fe4267fc640 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76c88860 0x559a76c8ac50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce4b8f204c1fd4d server_cookie=5166d5c5271ef40a in_seq=0 out_seq=0 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe40effd640 1 -- 172.21.15.22:0/44181178 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a76bde340 con 0x559a76c88860 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe40effd640 10 client.19935 handle_command_reply: tid=0 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 client.19935 shutdown 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6d420 msgr2=0x559a76c78530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6d420 0x559a76c78530 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe420011d40 tx=0x7fe420011820 comp rx=0 tx=0).stop 2024-03-09T00:08:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 shutdown_connections 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76c88860 0x559a76c8ac50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c75e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6dd40 0x559a76c758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe40effd640 0 client.19935 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 --2- 172.21.15.22:0/44181178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6d420 0x559a76c78530 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 >> 172.21.15.22:0/44181178 conn(0x559a76c67e90 msgr2=0x559a76c69be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 shutdown_connections 2024-03-09T00:08:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 1 -- 172.21.15.22:0/44181178 wait complete. 2024-03-09T00:08:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.614+0000 7fe4385be740 20 client.19935 trim_cache size 0 max 16384 2024-03-09T00:08:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 Processor -- start 2024-03-09T00:08:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 -- start start 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c77ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c74190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301aea00 con 0x559a76c6dd40 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301ac540 con 0x559a76c78e70 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4385be740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301aa600 con 0x559a76c6d420 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54708/0 (socket says 172.21.15.22:54708) 2024-03-09T00:08:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 -- 172.21.15.22:0/1511858950 learned_addr learned my addr 172.21.15.22:0/1511858950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe42effd640 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c77ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c74190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 -- 172.21.15.22:0/1511858950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 msgr2=0x559a76c77ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c77ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 -- 172.21.15.22:0/1511858950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 msgr2=0x559a76c74190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c74190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.631+0000 7fe4267fc640 1 -- 172.21.15.22:0/1511858950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a76c8fc30 con 0x559a76c6dd40 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.632+0000 7fe4267fc640 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78530 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe4300ff9e0 tx=0x7fe418012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb5dbbef2059678 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.632+0000 7fe4257fa640 1 -- 172.21.15.22:0/1511858950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418019040 con 0x559a76c6dd40 2024-03-09T00:08:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.632+0000 7fe4257fa640 1 -- 172.21.15.22:0/1511858950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe418030070 con 0x559a76c6dd40 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.632+0000 7fe4257fa640 1 -- 172.21.15.22:0/1511858950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41801e410 con 0x559a76c6dd40 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 -- 172.21.15.22:0/1511858950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 msgr2=0x559a76c78530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78530 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe4300ff9e0 tx=0x7fe418012070 comp rx=0 tx=0).stop 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 -- 172.21.15.22:0/1511858950 shutdown_connections 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c78e70 0x559a76c74190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c78530 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 --2- 172.21.15.22:0/1511858950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c6d420 0x559a76c77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 -- 172.21.15.22:0/1511858950 >> 172.21.15.22:0/1511858950 conn(0x559a76c67e90 msgr2=0x559a76c75880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 -- 172.21.15.22:0/1511858950 shutdown_connections 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 -- 172.21.15.22:0/1511858950 wait complete. 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 -- start start 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 Processor -- start 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6d420 0x559a76c801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.633+0000 7fe4385be740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c7cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4385be740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301aac00 con 0x559a76c6dd40 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4385be740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301abee0 con 0x559a76c6d420 2024-03-09T00:08:35.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4385be740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4301ab220 con 0x559a76c78e70 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4385be740 10 client.0 _ll_register_callbacks cb 0x7fe418009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c7c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54720/0 (socket says 172.21.15.22:54720) 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 -- 172.21.15.22:0/3161399607 learned_addr learned my addr 172.21.15.22:0/3161399607 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe42effd640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6d420 0x559a76c801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe42f7fe640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c7cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 -- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 msgr2=0x559a76c7cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c7cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 -- 172.21.15.22:0/3161399607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6d420 msgr2=0x559a76c801f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6d420 0x559a76c801f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 -- 172.21.15.22:0/3161399607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a76c7d160 con 0x559a76c6dd40 2024-03-09T00:08:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe42effd640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6d420 0x559a76c801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4267fc640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c7c630 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe4180105a0 tx=0x7fe418005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64fdec1d2171666 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe40effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe40effd640 1 -- 172.21.15.22:0/3161399607 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418019040 con 0x559a76c6dd40 2024-03-09T00:08:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe40effd640 1 -- 172.21.15.22:0/3161399607 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fe41802b3b0 con 0x559a76c6dd40 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.634+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe418030070 con 0x559a76c6dd40 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe40effd640 1 -- 172.21.15.22:0/3161399607 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe418045050 con 0x559a76c6dd40 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 10 client.19941 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559a76cdf930 con 0x559a76c6dd40 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe40effd640 1 -- 172.21.15.22:0/3161399607 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe418030070 con 0x559a76c6dd40 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe40effd640 1 -- 172.21.15.22:0/3161399607 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe4180047a0 con 0x559a76c6dd40 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 10 client.19941 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 10 client.19941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 20 client.19941 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:35.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76ce0030 0x559a76ce2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 4 client.19941 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559a76bd8690 con 0x559a76ce0030 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.635+0000 7fe42effd640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76ce0030 0x559a76ce2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.636+0000 7fe40effd640 10 client.19941 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.636+0000 7fe42effd640 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76ce0030 0x559a76ce2450 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf8a2945911191e6 server_cookie=868fd8f2019ae186 in_seq=0 out_seq=0 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.639+0000 7fe40effd640 1 -- 172.21.15.22:0/3161399607 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559a76bd8690 con 0x559a76ce0030 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.639+0000 7fe40effd640 10 client.19941 handle_command_reply: tid=0 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.639+0000 7fe4385be740 1 client.19941 shutdown 2024-03-09T00:08:35.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 msgr2=0x559a76c7c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c7c630 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe4180105a0 tx=0x7fe418005f60 comp rx=0 tx=0).stop 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 shutdown_connections 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559a76ce0030 0x559a76ce2450 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559a76c78e70 0x559a76c7cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559a76c6dd40 0x559a76c7c630 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 --2- 172.21.15.22:0/3161399607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a76c6d420 0x559a76c801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe40effd640 0 client.19941 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 >> 172.21.15.22:0/3161399607 conn(0x559a76c67e90 msgr2=0x559a76c740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:35.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 shutdown_connections 2024-03-09T00:08:35.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 1 -- 172.21.15.22:0/3161399607 wait complete. 2024-03-09T00:08:35.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe4385be740 20 client.19941 trim_cache size 0 max 16384 2024-03-09T00:08:35.646 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:08:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:08:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:08:35.647 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe41c0a2970 msgr2=0x7fe41c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe41c0a2970 0x7fe41c0a4e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x559a76c68eb0 tx=0x7fe418004730 comp rx=0 tx=0).stop 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe430196880 msgr2=0x7fe430196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe430196880 0x7fe430196d30 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe42802c700 tx=0x7fe428032f40 comp rx=0 tx=0).stop 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 shutdown_connections 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe41c0a2970 0x7fe41c0a4e30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe430196880 0x7fe430196d30 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4301054e0 0x7fe43019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 --2- 172.21.15.22:0/1509876565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe430100410 0x7fe43019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 >> 172.21.15.22:0/1509876565 conn(0x7fe430076570 msgr2=0x7fe4300fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:35.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.640+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 shutdown_connections 2024-03-09T00:08:35.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.641+0000 7fe43590a640 1 -- 172.21.15.22:0/1509876565 wait complete. 2024-03-09T00:08:35.650 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:08:35.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:08:35.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 -- 172.21.15.22:0/2934923993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 msgr2=0x7fb25010c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 --2- 172.21.15.22:0/2934923993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25010c7b0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb24800aa10 tx=0x7fb24802cf70 comp rx=0 tx=0).stop 2024-03-09T00:08:35.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 -- 172.21.15.22:0/2934923993 shutdown_connections 2024-03-09T00:08:35.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 --2- 172.21.15.22:0/2934923993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25010c7b0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 --2- 172.21.15.22:0/2934923993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2500ffc10 0x7fb250100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 --2- 172.21.15.22:0/2934923993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2501059f0 0x7fb250105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 -- 172.21.15.22:0/2934923993 >> 172.21.15.22:0/2934923993 conn(0x7fb250075460 msgr2=0x7fb250075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 -- 172.21.15.22:0/2934923993 shutdown_connections 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.767+0000 7fb2555f1640 1 -- 172.21.15.22:0/2934923993 wait complete. 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 Processor -- start 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 -- start start 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2500ffc10 0x7fb25019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2501059f0 0x7fb250197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25010f4a0 con 0x7fb2501005d0 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25010f320 con 0x7fb2501059f0 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb2555f1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25010f620 con 0x7fb2500ffc10 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb24e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb24effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2500ffc10 0x7fb25019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb24e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25019dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54736/0 (socket says 172.21.15.22:54736) 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb24e7fc640 1 -- 172.21.15.22:0/3228021787 learned_addr learned my addr 172.21.15.22:0/3228021787 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:35.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.768+0000 7fb24f7fe640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2501059f0 0x7fb250197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb24effd640 1 -- 172.21.15.22:0/3228021787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2501059f0 msgr2=0x7fb250197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb24effd640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2501059f0 0x7fb250197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb24effd640 1 -- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 msgr2=0x7fb25019dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb24effd640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25019dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb24effd640 1 -- 172.21.15.22:0/3228021787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb250198010 con 0x7fb2500ffc10 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb24e7fc640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25019dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb24effd640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2500ffc10 0x7fb25019d6e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb24000eb00 tx=0x7fb24000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb247fff640 1 -- 172.21.15.22:0/3228021787 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240010430 con 0x7fb2500ffc10 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb250198300 con 0x7fb2500ffc10 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2501a45a0 con 0x7fb2500ffc10 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb247fff640 1 -- 172.21.15.22:0/3228021787 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb240010ab0 con 0x7fb2500ffc10 2024-03-09T00:08:35.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.769+0000 7fb247fff640 1 -- 172.21.15.22:0/3228021787 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240018b00 con 0x7fb2500ffc10 2024-03-09T00:08:35.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.770+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb250068530 con 0x7fb2500ffc10 2024-03-09T00:08:35.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.771+0000 7fb247fff640 1 -- 172.21.15.22:0/3228021787 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb240018ce0 con 0x7fb2500ffc10 2024-03-09T00:08:35.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.772+0000 7fb247fff640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb23c0a2970 0x7fb23c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:35.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.772+0000 7fb24e7fc640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb23c0a2970 0x7fb23c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:35.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.772+0000 7fb247fff640 1 -- 172.21.15.22:0/3228021787 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fb2400cfdc0 con 0x7fb2500ffc10 2024-03-09T00:08:35.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.772+0000 7fb24e7fc640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb23c0a2970 0x7fb23c0a4e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb250075980 tx=0x7fb238005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:35.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:35.779+0000 7fb247fff640 1 -- 172.21.15.22:0/3228021787 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb24009c750 con 0x7fb2500ffc10 2024-03-09T00:08:36.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.010+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2501a4880 con 0x7fb2500ffc10 2024-03-09T00:08:36.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.012+0000 7fb247fff640 1 -- 172.21.15.22:0/3228021787 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fb24009c050 con 0x7fb2500ffc10 2024-03-09T00:08:36.006 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:08:36.007 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:07:54.629831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:08:36.007 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-03-09T00:08:36.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb23c0a2970 msgr2=0x7fb23c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb23c0a2970 0x7fb23c0a4e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb250075980 tx=0x7fb238005f90 comp rx=0 tx=0).stop 2024-03-09T00:08:36.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2500ffc10 msgr2=0x7fb25019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2500ffc10 0x7fb25019d6e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb24000eb00 tx=0x7fb24000c2d0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 shutdown_connections 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb23c0a2970 0x7fb23c0a4e30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2501059f0 0x7fb250197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2501005d0 0x7fb25019dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 --2- 172.21.15.22:0/3228021787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2500ffc10 0x7fb25019d6e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 >> 172.21.15.22:0/3228021787 conn(0x7fb250075460 msgr2=0x7fb2500fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 shutdown_connections 2024-03-09T00:08:36.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.014+0000 7fb2555f1640 1 -- 172.21.15.22:0/3228021787 wait complete. 2024-03-09T00:08:36.020 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:08:36.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:08:36.062 INFO:tasks.workunit.client.0.smithi022.stdout: 10 10369 17.31 MB/sec warmup 43 sec latency 157.591 ms 2024-03-09T00:08:36.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 -- 172.21.15.22:0/1017874294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 msgr2=0x7f03d410b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 --2- 172.21.15.22:0/1017874294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d410b310 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009900 tx=0x7f03c002bcd0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 -- 172.21.15.22:0/1017874294 shutdown_connections 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 --2- 172.21.15.22:0/1017874294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d410b850 0x7f03d410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 --2- 172.21.15.22:0/1017874294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d410b310 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 --2- 172.21.15.22:0/1017874294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03d4102510 0x7f03d41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 -- 172.21.15.22:0/1017874294 >> 172.21.15.22:0/1017874294 conn(0x7f03d40773a0 msgr2=0x7f03d40777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 -- 172.21.15.22:0/1017874294 shutdown_connections 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 -- 172.21.15.22:0/1017874294 wait complete. 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 Processor -- start 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.136+0000 7f03db012640 1 -- start start 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03db012640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03d4102510 0x7f03d4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03db012640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03db012640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d410b850 0x7f03d419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03db012640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4111680 con 0x7f03d4102e30 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03db012640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4111500 con 0x7f03d410b850 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03db012640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4111800 con 0x7f03d4102510 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54750/0 (socket says 172.21.15.22:54750) 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 -- 172.21.15.22:0/2825785172 learned_addr learned my addr 172.21.15.22:0/2825785172 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d95af640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d410b850 0x7f03d419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d8dae640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03d4102510 0x7f03d4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 -- 172.21.15.22:0/2825785172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03d4102510 msgr2=0x7f03d4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03d4102510 0x7f03d4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 -- 172.21.15.22:0/2825785172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d410b850 msgr2=0x7f03d419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d410b850 0x7f03d419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 -- 172.21.15.22:0/2825785172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d419eda0 con 0x7f03d4102e30 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d95af640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d410b850 0x7f03d419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d3fff640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d419a2f0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f03c0000c00 tx=0x7f03c0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d1ffb640 1 -- 172.21.15.22:0/2825785172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0042020 con 0x7f03d4102e30 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d419f030 con 0x7f03d4102e30 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.137+0000 7f03d1ffb640 1 -- 172.21.15.22:0/2825785172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c00054e0 con 0x7f03d4102e30 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.138+0000 7f03d1ffb640 1 -- 172.21.15.22:0/2825785172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0003af0 con 0x7f03d4102e30 2024-03-09T00:08:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.138+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d41a6920 con 0x7f03d4102e30 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03d1ffb640 1 -- 172.21.15.22:0/2825785172 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f03c0045020 con 0x7f03d4102e30 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 Processor -- start 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 -- start start 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03d1ffb640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03c40a2970 0x7f03c40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944eb1a0 0x55d1944eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944ebac0 0x55d1944f4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1944b2360 con 0x55d1944eb1a0 2024-03-09T00:08:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d19444eec0 con 0x55d1944f46a0 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.140+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1943f7d30 con 0x55d1944ebac0 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944eb1a0 0x55d1944eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d1ffb640 1 -- 172.21.15.22:0/2825785172 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f03c00f0da0 con 0x7f03d4102e30 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d95af640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03c40a2970 0x7f03c40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944eb1a0 0x55d1944eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54756/0 (socket says 172.21.15.22:54756) 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 -- 172.21.15.22:0/2975330779 learned_addr learned my addr 172.21.15.22:0/2975330779 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d3fff640 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944ebac0 0x55d1944f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 -- 172.21.15.22:0/2975330779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944ebac0 msgr2=0x55d1944f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944ebac0 0x55d1944f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d95af640 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 -- 172.21.15.22:0/2975330779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 msgr2=0x55d1944f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 -- 172.21.15.22:0/2975330779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1943f6bc0 con 0x55d1944eb1a0 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d95af640 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d8dae640 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944eb1a0 0x55d1944eb580 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009d20 tx=0x7f03c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93448499a1ab4379 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03d95af640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03c40a2970 0x7f03c40a4e30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55d1944e6a60 tx=0x7f03cc009f60 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03beffd640 1 -- 172.21.15.22:0/2975330779 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c801e070 con 0x55d1944eb1a0 2024-03-09T00:08:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03beffd640 1 -- 172.21.15.22:0/2975330779 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c8019b10 con 0x55d1944eb1a0 2024-03-09T00:08:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.141+0000 7f03beffd640 1 -- 172.21.15.22:0/2975330779 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c800ebb0 con 0x55d1944eb1a0 2024-03-09T00:08:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 -- 172.21.15.22:0/2975330779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944eb1a0 msgr2=0x55d1944eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944eb1a0 0x55d1944eb580 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009d20 tx=0x7f03c8011810 comp rx=0 tx=0).stop 2024-03-09T00:08:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 -- 172.21.15.22:0/2975330779 shutdown_connections 2024-03-09T00:08:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944ebac0 0x55d1944f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/2975330779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944eb1a0 0x55d1944eb580 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 -- 172.21.15.22:0/2975330779 >> 172.21.15.22:0/2975330779 conn(0x55d1944e58d0 msgr2=0x55d1944e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 -- 172.21.15.22:0/2975330779 shutdown_connections 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 -- 172.21.15.22:0/2975330779 wait complete. 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 -- start start 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.142+0000 7f03ddbfc740 1 Processor -- start 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944ebac0 0x55d1944f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944f46a0 0x55d1944f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1943ed4a0 con 0x55d1944f46a0 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d19444af60 con 0x55d1944ebac0 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1944f7490 con 0x55d1944eb1a0 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03ddbfc740 10 client.0 _ll_register_callbacks cb 0x55d194448d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d95af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944f46a0 0x55d1944f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944ebac0 0x55d1944f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d8dae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d8dae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51450/0 (socket says 172.21.15.22:51450) 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944ebac0 0x55d1944f1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:37994/0 (socket says 172.21.15.22:37994) 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d8dae640 1 -- 172.21.15.22:0/3465731408 learned_addr learned my addr 172.21.15.22:0/3465731408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d95af640 1 -- 172.21.15.22:0/3465731408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 msgr2=0x55d1944f1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d95af640 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944f1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d95af640 1 -- 172.21.15.22:0/3465731408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944ebac0 msgr2=0x55d1944f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d95af640 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944ebac0 0x55d1944f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d95af640 1 -- 172.21.15.22:0/3465731408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1941633f0 con 0x55d1944f46a0 2024-03-09T00:08:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d3fff640 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944ebac0 0x55d1944f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03d95af640 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944f46a0 0x55d1944f20a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f03cc005fd0 tx=0x7f03cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d33c033ec5297d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.143+0000 7f03b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03b3fff640 1 -- 172.21.15.22:0/3465731408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc024070 con 0x55d1944f46a0 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03b3fff640 1 -- 172.21.15.22:0/3465731408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f03cc020800 con 0x55d1944f46a0 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03c8019b10 con 0x55d1944f46a0 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03b3fff640 1 -- 172.21.15.22:0/3465731408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03cc01a070 con 0x55d1944f46a0 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 10 client.19959 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d19422d7d0 con 0x55d1944f46a0 2024-03-09T00:08:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03b3fff640 1 -- 172.21.15.22:0/3465731408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c8019b10 con 0x55d1944f46a0 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03b3fff640 1 -- 172.21.15.22:0/3465731408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f03cc01fad0 con 0x55d1944f46a0 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 10 client.19959 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 10 client.19959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 20 client.19959 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d1945047f0 0x55d194506c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 4 client.19959 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.144+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d19445c400 con 0x55d1945047f0 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.145+0000 7f03d8dae640 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d1945047f0 0x55d194506c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.145+0000 7f03d8dae640 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d1945047f0 0x55d194506c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce622c90e532791a server_cookie=7116379a8ee1de67 in_seq=0 out_seq=0 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.145+0000 7f03b3fff640 10 client.19959 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03b3fff640 1 -- 172.21.15.22:0/3465731408 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d19445c400 con 0x55d1945047f0 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03b3fff640 10 client.19959 handle_command_reply: tid=0 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 client.19959 shutdown 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944f46a0 msgr2=0x55d1944f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944f46a0 0x55d1944f20a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f03cc005fd0 tx=0x7f03cc004a40 comp rx=0 tx=0).stop 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 shutdown_connections 2024-03-09T00:08:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d1945047f0 0x55d194506c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944f46a0 0x55d1944f20a0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944ebac0 0x55d1944f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03b3fff640 0 client.19959 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/3465731408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 >> 172.21.15.22:0/3465731408 conn(0x55d1944e58d0 msgr2=0x55d1944e7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 shutdown_connections 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 1 -- 172.21.15.22:0/3465731408 wait complete. 2024-03-09T00:08:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.146+0000 7f03ddbfc740 20 client.19959 trim_cache size 0 max 16384 2024-03-09T00:08:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.167+0000 7f03ddbfc740 1 Processor -- start 2024-03-09T00:08:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.167+0000 7f03ddbfc740 1 -- start start 2024-03-09T00:08:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.167+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944eb1a0 0x55d1944ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.167+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.167+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944f46a0 0x55d1944efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.167+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.167+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944ece80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54784/0 (socket says 172.21.15.22:54784) 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d3fff640 1 -- 172.21.15.22:0/1919955402 learned_addr learned my addr 172.21.15.22:0/1919955402 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d8dae640 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944eb1a0 0x55d1944ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41b2e30 con 0x55d1944ebac0 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41b0970 con 0x55d1944eb1a0 2024-03-09T00:08:36.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41aea30 con 0x55d1944f46a0 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d95af640 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944f46a0 0x55d1944efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d3fff640 1 -- 172.21.15.22:0/1919955402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944f46a0 msgr2=0x55d1944efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d3fff640 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944f46a0 0x55d1944efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d3fff640 1 -- 172.21.15.22:0/1919955402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944eb1a0 msgr2=0x55d1944ec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d3fff640 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944eb1a0 0x55d1944ec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.168+0000 7f03d3fff640 1 -- 172.21.15.22:0/1919955402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1944f6a90 con 0x55d1944ebac0 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.169+0000 7f03d3fff640 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944ece80 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f03c00b9660 tx=0x7f03c0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d554fd2b28219abe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.169+0000 7f03be7fc640 1 -- 172.21.15.22:0/1919955402 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00a0fc0 con 0x55d1944ebac0 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.169+0000 7f03be7fc640 1 -- 172.21.15.22:0/1919955402 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c0005690 con 0x55d1944ebac0 2024-03-09T00:08:36.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.169+0000 7f03be7fc640 1 -- 172.21.15.22:0/1919955402 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ee310 con 0x55d1944ebac0 2024-03-09T00:08:36.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.170+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 msgr2=0x55d1944ece80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.170+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944ece80 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f03c00b9660 tx=0x7f03c0005d50 comp rx=0 tx=0).stop 2024-03-09T00:08:36.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.170+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 shutdown_connections 2024-03-09T00:08:36.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.170+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944f46a0 0x55d1944efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.170+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944ece80 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.170+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/1919955402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944eb1a0 0x55d1944ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.170+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 >> 172.21.15.22:0/1919955402 conn(0x55d1944e58d0 msgr2=0x55d1944e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 shutdown_connections 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 -- 172.21.15.22:0/1919955402 wait complete. 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 -- start start 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 Processor -- start 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41af030 con 0x55d1944ebac0 2024-03-09T00:08:36.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41b0310 con 0x55d1944f46a0 2024-03-09T00:08:36.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41af650 con 0x55d1944eb1a0 2024-03-09T00:08:36.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.171+0000 7f03ddbfc740 10 client.0 _ll_register_callbacks cb 0x7f03c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:36.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54792/0 (socket says 172.21.15.22:54792) 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 -- 172.21.15.22:0/604532489 learned_addr learned my addr 172.21.15.22:0/604532489 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d95af640 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 -- 172.21.15.22:0/604532489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 msgr2=0x55d1944fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 -- 172.21.15.22:0/604532489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 msgr2=0x55d1944f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 -- 172.21.15.22:0/604532489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1944fa430 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03d3fff640 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944fbbd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f03c002c8c0 tx=0x7f03c00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fcba711a760dd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03b3fff640 1 -- 172.21.15.22:0/604532489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ee510 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03b3fff640 1 -- 172.21.15.22:0/604532489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f03c00dbbb0 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d194456750 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03b3fff640 1 -- 172.21.15.22:0/604532489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03c00d7050 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03ddbfc740 10 client.19965 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.172+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d19455b8e0 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03b3fff640 1 -- 172.21.15.22:0/604532489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d194456750 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03b3fff640 1 -- 172.21.15.22:0/604532489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f03c00bc0c0 con 0x55d1944ebac0 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03ddbfc740 10 client.19965 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03ddbfc740 10 client.19965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:08:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03ddbfc740 20 client.19965 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d19455c060 0x55d19455e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03ddbfc740 4 client.19965 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:08:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.173+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d1943d4640 con 0x55d19455c060 2024-03-09T00:08:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.175+0000 7f03d8dae640 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d19455c060 0x55d19455e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.175+0000 7f03d8dae640 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d19455c060 0x55d19455e480 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=375f184b2aec646c server_cookie=bfe0903ee0bcc138 in_seq=0 out_seq=0 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.175+0000 7f03b3fff640 10 client.19965 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.175+0000 7f03b3fff640 1 -- 172.21.15.22:0/604532489 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d1943d4640 con 0x55d19455c060 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.175+0000 7f03b3fff640 10 client.19965 handle_command_reply: tid=0 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.175+0000 7f03ddbfc740 1 client.19965 shutdown 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 msgr2=0x55d1944fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944fbbd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f03c002c8c0 tx=0x7f03c00035f0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 shutdown_connections 2024-03-09T00:08:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55d19455c060 0x55d19455e480 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1944f46a0 0x55d1944f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d1944ebac0 0x55d1944fbbd0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03b3fff640 0 client.19965 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 --2- 172.21.15.22:0/604532489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1944eb1a0 0x55d1944fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 >> 172.21.15.22:0/604532489 conn(0x55d1944e58d0 msgr2=0x55d1944ec940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 shutdown_connections 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 1 -- 172.21.15.22:0/604532489 wait complete. 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.176+0000 7f03ddbfc740 20 client.19965 trim_cache size 0 max 16384 2024-03-09T00:08:36.177 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:08:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03c40a2970 msgr2=0x7f03c40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03c40a2970 0x7f03c40a4e30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55d1944e6a60 tx=0x7f03cc009f60 comp rx=0 tx=0).stop 2024-03-09T00:08:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 msgr2=0x7f03d419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d419a2f0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f03c0000c00 tx=0x7f03c0004360 comp rx=0 tx=0).stop 2024-03-09T00:08:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 shutdown_connections 2024-03-09T00:08:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03c40a2970 0x7f03c40a4e30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d410b850 0x7f03d419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03d4102e30 0x7f03d419a2f0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 --2- 172.21.15.22:0/2825785172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03d4102510 0x7f03d4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 >> 172.21.15.22:0/2825785172 conn(0x7f03d40773a0 msgr2=0x7f03d4107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 shutdown_connections 2024-03-09T00:08:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.177+0000 7f03db012640 1 -- 172.21.15.22:0/2825785172 wait complete. 2024-03-09T00:08:36.191 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:08:36.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:08:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 -- 172.21.15.22:0/3275164245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 msgr2=0x7f01001007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 --2- 172.21.15.22:0/3275164245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f01001007c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009900 tx=0x7f00f002bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 -- 172.21.15.22:0/3275164245 shutdown_connections 2024-03-09T00:08:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 --2- 172.21.15.22:0/3275164245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01001054b0 0x7f010010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 --2- 172.21.15.22:0/3275164245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0100100d90 0x7f0100104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 --2- 172.21.15.22:0/3275164245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f01001007c0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 -- 172.21.15.22:0/3275164245 >> 172.21.15.22:0/3275164245 conn(0x7f01000773a0 msgr2=0x7f01000777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 -- 172.21.15.22:0/3275164245 shutdown_connections 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 -- 172.21.15.22:0/3275164245 wait complete. 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 Processor -- start 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.355+0000 7f0106924640 1 -- start start 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f0106924640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f010019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f0106924640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01001054b0 0x7f010019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f0106924640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0100197a30 0x7f0100197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f0106924640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0100110560 con 0x7f01001003e0 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f0106924640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001103e0 con 0x7f01001054b0 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f0106924640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001106e0 con 0x7f0100197a30 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f010019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f010019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54808/0 (socket says 172.21.15.22:54808) 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 -- 172.21.15.22:0/2209383848 learned_addr learned my addr 172.21.15.22:0/2209383848 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01001054b0 0x7f010019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f0104ec1640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0100197a30 0x7f0100197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 -- 172.21.15.22:0/2209383848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0100197a30 msgr2=0x7f0100197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0100197a30 0x7f0100197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 -- 172.21.15.22:0/2209383848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01001054b0 msgr2=0x7f010019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01001054b0 0x7f010019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 -- 172.21.15.22:0/2209383848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100198550 con 0x7f01001003e0 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01001054b0 0x7f010019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.356+0000 7f00fffff640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f010019d830 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f00f0006010 tx=0x7f00f002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.357+0000 7f00fd7fa640 1 -- 172.21.15.22:0/2209383848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0041070 con 0x7f01001003e0 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.357+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01001987e0 con 0x7f01001003e0 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.357+0000 7f00fd7fa640 1 -- 172.21.15.22:0/2209383848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f0004450 con 0x7f01001003e0 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.357+0000 7f00fd7fa640 1 -- 172.21.15.22:0/2209383848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0003e00 con 0x7f01001003e0 2024-03-09T00:08:36.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.357+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01001a49e0 con 0x7f01001003e0 2024-03-09T00:08:36.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.359+0000 7f00fd7fa640 1 -- 172.21.15.22:0/2209383848 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f00f0044070 con 0x7f01001003e0 2024-03-09T00:08:36.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.359+0000 7f01095d7740 1 Processor -- start 2024-03-09T00:08:36.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.359+0000 7f01095d7740 1 -- start start 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00fd7fa640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00e80a2970 0x7f00e80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f01095d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f01095d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a82160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f01095d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a84a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f01095d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335a40360 con 0x55f335a791a0 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f01095d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3359dcec0 con 0x55f335a79ac0 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f01095d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335985d30 con 0x55f335a826a0 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f0104ec1640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00e80a2970 0x7f00e80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a79580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54814/0 (socket says 172.21.15.22:54814) 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00fffff640 1 -- 172.21.15.22:0/1413323022 learned_addr learned my addr 172.21.15.22:0/1413323022 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a82160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f0104ec1640 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a84a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00fd7fa640 1 -- 172.21.15.22:0/2209383848 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f00f00f0bb0 con 0x7f01001003e0 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00ff7fe640 1 -- 172.21.15.22:0/1413323022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 msgr2=0x55f335a84a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a84a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00ff7fe640 1 -- 172.21.15.22:0/1413323022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 msgr2=0x55f335a79580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a79580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00ff7fe640 1 -- 172.21.15.22:0/1413323022 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f335984bc0 con 0x55f335a79ac0 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.360+0000 7f00fffff640 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a79580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.361+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a82160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f01001991d0 tx=0x7f00f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34733bb5a0dddc10 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.361+0000 7f0104ec1640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00e80a2970 0x7f00e80a4e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f00f8005fd0 tx=0x7f00f8009e70 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.361+0000 7f00e67fc640 1 -- 172.21.15.22:0/1413323022 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4004310 con 0x55f335a79ac0 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.361+0000 7f00e67fc640 1 -- 172.21.15.22:0/1413323022 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f40044b0 con 0x55f335a79ac0 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.361+0000 7f00e67fc640 1 -- 172.21.15.22:0/1413323022 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4003630 con 0x55f335a79ac0 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 -- 172.21.15.22:0/1413323022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 msgr2=0x55f335a82160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a82160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f01001991d0 tx=0x7f00f40118d0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 -- 172.21.15.22:0/1413323022 shutdown_connections 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a84a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a82160 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 --2- 172.21.15.22:0/1413323022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a79580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 -- 172.21.15.22:0/1413323022 >> 172.21.15.22:0/1413323022 conn(0x55f335a738d0 msgr2=0x55f335a73cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 -- 172.21.15.22:0/1413323022 shutdown_connections 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 -- 172.21.15.22:0/1413323022 wait complete. 2024-03-09T00:08:36.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 -- start start 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 Processor -- start 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a791a0 0x55f335a7f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a7fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.362+0000 7f01095d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a80130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f01095d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33597b4a0 con 0x55f335a79ac0 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f01095d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3359d8f60 con 0x55f335a791a0 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f01095d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f335a85490 con 0x55f335a826a0 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f01095d7740 10 client.0 _ll_register_callbacks cb 0x55f3359d6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a791a0 0x55f335a7f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a7fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a791a0 0x55f335a7f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:38064/0 (socket says 172.21.15.22:38064) 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a7fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54822/0 (socket says 172.21.15.22:54822) 2024-03-09T00:08:36.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 -- 172.21.15.22:0/1869032547 learned_addr learned my addr 172.21.15.22:0/1869032547 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f0104ec1640 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a80130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 -- 172.21.15.22:0/1869032547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 msgr2=0x55f335a80130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a80130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 -- 172.21.15.22:0/1869032547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 msgr2=0x55f335a7fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a7fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 -- 172.21.15.22:0/1869032547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3356f13f0 con 0x55f335a791a0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00fffff640 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a791a0 0x55f335a7f6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00f0005820 tx=0x7f00f00bce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3fb0192d86027d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00df7fe640 1 -- 172.21.15.22:0/1869032547 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f004b040 con 0x55f335a791a0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00df7fe640 1 -- 172.21.15.22:0/1869032547 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f00f00ee180 con 0x55f335a791a0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f40044b0 con 0x55f335a791a0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f00df7fe640 1 -- 172.21.15.22:0/1869032547 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f00cfb00 con 0x55f335a791a0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f01095d7740 10 client.29234 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.363+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f3357bb7d0 con 0x55f335a791a0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f00df7fe640 1 -- 172.21.15.22:0/1869032547 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f40044b0 con 0x55f335a791a0 2024-03-09T00:08:36.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f00df7fe640 1 -- 172.21.15.22:0/1869032547 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f00f00b9640 con 0x55f335a791a0 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f01095d7740 10 client.29234 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f01095d7740 10 client.29234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f01095d7740 20 client.29234 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f01095d7740 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335a92820 0x55f335a94c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f01095d7740 4 client.29234 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3359ea400 con 0x55f335a92820 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335a92820 0x55f335a94c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335a92820 0x55f335a94c10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a313b0b63746e0 server_cookie=619588f2ba4f59f0 in_seq=0 out_seq=0 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.364+0000 7f00df7fe640 10 client.29234 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f00df7fe640 1 -- 172.21.15.22:0/1869032547 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3359ea400 con 0x55f335a92820 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f00df7fe640 10 client.29234 handle_command_reply: tid=0 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 client.29234 shutdown 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a791a0 msgr2=0x55f335a7f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a791a0 0x55f335a7f6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00f0005820 tx=0x7f00f00bce20 comp rx=0 tx=0).stop 2024-03-09T00:08:36.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 shutdown_connections 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335a92820 0x55f335a94c10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a80130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a7fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f00df7fe640 0 client.29234 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 --2- 172.21.15.22:0/1869032547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a791a0 0x55f335a7f6b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 >> 172.21.15.22:0/1869032547 conn(0x55f335a738d0 msgr2=0x55f335a75620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 shutdown_connections 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 1 -- 172.21.15.22:0/1869032547 wait complete. 2024-03-09T00:08:36.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.365+0000 7f01095d7740 20 client.29234 trim_cache size 0 max 16384 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 Processor -- start 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 -- start start 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a81de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a7ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a72810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001b0b00 con 0x55f335a791a0 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001ae640 con 0x55f335a79ac0 2024-03-09T00:08:36.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.378+0000 7f01095d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001ac700 con 0x55f335a826a0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00fffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a81de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a7ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a7ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:38078/0 (socket says 172.21.15.22:38078) 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 -- 172.21.15.22:0/3440532134 learned_addr learned my addr 172.21.15.22:0/3440532134 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f0104ec1640 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a72810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 -- 172.21.15.22:0/3440532134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 msgr2=0x55f335a72810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a72810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 -- 172.21.15.22:0/3440532134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 msgr2=0x55f335a81de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a81de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 -- 172.21.15.22:0/3440532134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f335a744a0 con 0x55f335a79ac0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a7ef90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009710 tx=0x7f00f4004050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e36ad91f6a0a551 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00e5ffb640 1 -- 172.21.15.22:0/3440532134 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4004490 con 0x55f335a79ac0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00e5ffb640 1 -- 172.21.15.22:0/3440532134 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f402e020 con 0x55f335a79ac0 2024-03-09T00:08:36.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.379+0000 7f00e5ffb640 1 -- 172.21.15.22:0/3440532134 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f40035f0 con 0x55f335a79ac0 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 -- 172.21.15.22:0/3440532134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 msgr2=0x55f335a7ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a7ef90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009710 tx=0x7f00f4004050 comp rx=0 tx=0).stop 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 -- 172.21.15.22:0/3440532134 shutdown_connections 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a826a0 0x55f335a72810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a79ac0 0x55f335a7ef90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 --2- 172.21.15.22:0/3440532134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a791a0 0x55f335a81de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 -- 172.21.15.22:0/3440532134 >> 172.21.15.22:0/3440532134 conn(0x55f335a738d0 msgr2=0x55f335a72100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 -- 172.21.15.22:0/3440532134 shutdown_connections 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 -- 172.21.15.22:0/3440532134 wait complete. 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.380+0000 7f01095d7740 1 -- start start 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 1 Processor -- start 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a791a0 0x55f335a89b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a87ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a826a0 0x55f335a88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001acd00 con 0x55f335a79ac0 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001adfe0 con 0x55f335a826a0 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001ad320 con 0x55f335a791a0 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f01095d7740 10 client.0 _ll_register_callbacks cb 0x7f00f4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a87ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a87ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54838/0 (socket says 172.21.15.22:54838) 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 -- 172.21.15.22:0/2265510907 learned_addr learned my addr 172.21.15.22:0/2265510907 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00fffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a791a0 0x55f335a89b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f0104ec1640 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a826a0 0x55f335a88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 -- 172.21.15.22:0/2265510907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a791a0 msgr2=0x55f335a89b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a791a0 0x55f335a89b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 -- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a826a0 msgr2=0x55f335a88540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a826a0 0x55f335a88540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.381+0000 7f00ff7fe640 1 -- 172.21.15.22:0/2265510907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f335ae8d00 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f00ff7fe640 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a87ec0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f00f4004c20 tx=0x7f00f402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9813b3e00c34a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f00df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f00df7fe640 1 -- 172.21.15.22:0/2265510907 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4037070 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f00df7fe640 1 -- 172.21.15.22:0/2265510907 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f00f402a1d0 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f402e020 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f00df7fe640 1 -- 172.21.15.22:0/2265510907 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f4043050 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 10 client.19983 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f335ae9e30 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f00df7fe640 1 -- 172.21.15.22:0/2265510907 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f402e020 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f00df7fe640 1 -- 172.21.15.22:0/2265510907 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f00f4022040 con 0x55f335a79ac0 2024-03-09T00:08:36.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 10 client.19983 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 10 client.19983 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 20 client.19983 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335aea3c0 0x55f335aec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 4 client.19983 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.382+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3359e4750 con 0x55f335aea3c0 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f00fffff640 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335aea3c0 0x55f335aec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f00fffff640 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335aea3c0 0x55f335aec7e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae221a88497244c4 server_cookie=40a4e1f30f49f226 in_seq=0 out_seq=0 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f00df7fe640 10 client.19983 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f00df7fe640 1 -- 172.21.15.22:0/2265510907 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3359e4750 con 0x55f335aea3c0 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f00df7fe640 10 client.19983 handle_command_reply: tid=0 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 client.19983 shutdown 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 msgr2=0x55f335a87ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a87ec0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f00f4004c20 tx=0x7f00f402a610 comp rx=0 tx=0).stop 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 shutdown_connections 2024-03-09T00:08:36.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55f335aea3c0 0x55f335aec7e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f335a826a0 0x55f335a88540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f335a79ac0 0x55f335a87ec0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f00df7fe640 0 client.19983 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 --2- 172.21.15.22:0/2265510907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f335a791a0 0x55f335a89b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 >> 172.21.15.22:0/2265510907 conn(0x55f335a738d0 msgr2=0x55f335a72990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 shutdown_connections 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.383+0000 7f01095d7740 1 -- 172.21.15.22:0/2265510907 wait complete. 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f01095d7740 20 client.19983 trim_cache size 0 max 16384 2024-03-09T00:08:36.385 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:08:36.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00e80a2970 msgr2=0x7f00e80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00e80a2970 0x7f00e80a4e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f00f8005fd0 tx=0x7f00f8009e70 comp rx=0 tx=0).stop 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 msgr2=0x7f010019d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f010019d830 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f00f0006010 tx=0x7f00f002cb30 comp rx=0 tx=0).stop 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 shutdown_connections 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f00e80a2970 0x7f00e80a4e30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0100197a30 0x7f0100197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01001054b0 0x7f010019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 --2- 172.21.15.22:0/2209383848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01001003e0 0x7f010019d830 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 >> 172.21.15.22:0/2209383848 conn(0x7f01000773a0 msgr2=0x7f0100068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 shutdown_connections 2024-03-09T00:08:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.384+0000 7f0106924640 1 -- 172.21.15.22:0/2209383848 wait complete. 2024-03-09T00:08:36.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:36 smithi022 ceph-mon[31894]: pgmap v444: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 653 op/s 2024-03-09T00:08:36.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:36 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3228021787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:08:36.390 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:08:36.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:08:36.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 -- 172.21.15.22:0/251230764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc1003e0 msgr2=0x7f30bc1007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 --2- 172.21.15.22:0/251230764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc1003e0 0x7f30bc1007c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009900 tx=0x7f30b802bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 -- 172.21.15.22:0/251230764 shutdown_connections 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 --2- 172.21.15.22:0/251230764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30bc1054b0 0x7f30bc10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 --2- 172.21.15.22:0/251230764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30bc100d90 0x7f30bc104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 --2- 172.21.15.22:0/251230764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc1003e0 0x7f30bc1007c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 -- 172.21.15.22:0/251230764 >> 172.21.15.22:0/251230764 conn(0x7f30bc0773a0 msgr2=0x7f30bc0777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 -- 172.21.15.22:0/251230764 shutdown_connections 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.505+0000 7f30c353b640 1 -- 172.21.15.22:0/251230764 wait complete. 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 Processor -- start 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 -- start start 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30bc1003e0 0x7f30bc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30bc1054b0 0x7f30bc19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc197970 0x7f30bc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc110540 con 0x7f30bc197970 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1103c0 con 0x7f30bc1054b0 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c353b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1106c0 con 0x7f30bc1003e0 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc197970 0x7f30bc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc197970 0x7f30bc197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54860/0 (socket says 172.21.15.22:54860) 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1335560958 learned_addr learned my addr 172.21.15.22:0/1335560958 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30bc1003e0 0x7f30bc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30bc1054b0 0x7f30bc19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1335560958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30bc1003e0 msgr2=0x7f30bc19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30bc1003e0 0x7f30bc19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1335560958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30bc1054b0 msgr2=0x7f30bc19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30bc1054b0 0x7f30bc19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 -- 172.21.15.22:0/1335560958 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc198490 con 0x7f30bc197970 2024-03-09T00:08:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc197970 0x7f30bc197e20 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f30b400eb10 tx=0x7f30b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.506+0000 7f30b27fc640 1 -- 172.21.15.22:0/1335560958 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4004b20 con 0x7f30bc197970 2024-03-09T00:08:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.507+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc198780 con 0x7f30bc197970 2024-03-09T00:08:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.507+0000 7f30b27fc640 1 -- 172.21.15.22:0/1335560958 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30b4004cc0 con 0x7f30bc197970 2024-03-09T00:08:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.507+0000 7f30b27fc640 1 -- 172.21.15.22:0/1335560958 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4010760 con 0x7f30bc197970 2024-03-09T00:08:36.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.507+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc1a4980 con 0x7f30bc197970 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.509+0000 7f30b27fc640 1 -- 172.21.15.22:0/1335560958 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f30b4005020 con 0x7f30bc197970 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.509+0000 7f30c61fc740 1 Processor -- start 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.509+0000 7f30c61fc740 1 -- start start 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31bf62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c03570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31c05ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c61fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd31bbd360 con 0x55dd31c03ab0 2024-03-09T00:08:36.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c61fc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd31b59ec0 con 0x55dd31bf67f0 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c61fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd31b02d30 con 0x55dd31bf5ed0 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30b27fc640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30a80a2970 0x7f30a80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c03570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c0ad6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31bf62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30a80a2970 0x7f30a80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c0ad6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31bf62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:51518/0 (socket says 172.21.15.22:51518) 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c0ad6640 1 -- 172.21.15.22:0/2381464092 learned_addr learned my addr 172.21.15.22:0/2381464092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c03570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:38108/0 (socket says 172.21.15.22:38108) 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31c05ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30b27fc640 1 -- 172.21.15.22:0/1335560958 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f30b40cf770 con 0x7f30bc197970 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 -- 172.21.15.22:0/2381464092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 msgr2=0x55dd31bf62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31bf62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 -- 172.21.15.22:0/2381464092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 msgr2=0x55dd31c05ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31c05ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 -- 172.21.15.22:0/2381464092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd31b01bc0 con 0x55dd31bf67f0 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c1ad8640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30a80a2970 0x7f30a80a4e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f30b4097e30 tx=0x7f30b400efd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30c12d7640 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c03570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b8005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe320df817a20a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.510+0000 7f30a77fe640 1 -- 172.21.15.22:0/2381464092 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8042070 con 0x55dd31bf67f0 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30a77fe640 1 -- 172.21.15.22:0/2381464092 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30b8004030 con 0x55dd31bf67f0 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30a77fe640 1 -- 172.21.15.22:0/2381464092 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8004f00 con 0x55dd31bf67f0 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 -- 172.21.15.22:0/2381464092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 msgr2=0x55dd31c03570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c03570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b8005be0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 -- 172.21.15.22:0/2381464092 shutdown_connections 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31c05ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c03570 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2381464092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31bf62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 -- 172.21.15.22:0/2381464092 >> 172.21.15.22:0/2381464092 conn(0x55dd31bf08d0 msgr2=0x55dd31bf0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 -- 172.21.15.22:0/2381464092 shutdown_connections 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.511+0000 7f30c61fc740 1 -- 172.21.15.22:0/2381464092 wait complete. 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 -- start start 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 Processor -- start 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf5ed0 0x55dd31c00bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31c03ab0 0x55dd31c01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd31af84a0 con 0x55dd31bf5ed0 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd31b55f60 con 0x55dd31bf67f0 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd31c06810 con 0x55dd31c03ab0 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c61fc740 10 client.0 _ll_register_callbacks cb 0x55dd31b53d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c0ad6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf5ed0 0x55dd31c00bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c0ad6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf5ed0 0x55dd31c00bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54864/0 (socket says 172.21.15.22:54864) 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c0ad6640 1 -- 172.21.15.22:0/2046740344 learned_addr learned my addr 172.21.15.22:0/2046740344 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31c03ab0 0x55dd31c01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.512+0000 7f30c12d7640 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c0ad6640 1 -- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31c03ab0 msgr2=0x55dd31c01670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31c03ab0 0x55dd31c01670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c0ad6640 1 -- 172.21.15.22:0/2046740344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 msgr2=0x55dd31c01130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c01130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c0ad6640 1 -- 172.21.15.22:0/2046740344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd3186e3f0 con 0x55dd31bf5ed0 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf5ed0 0x55dd31c00bf0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e10 tx=0x7f30ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32182a9658c809b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:36.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30a4ff9640 1 -- 172.21.15.22:0/2046740344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac011ab0 con 0x55dd31bf5ed0 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30a4ff9640 1 -- 172.21.15.22:0/2046740344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f30ac011c50 con 0x55dd31bf5ed0 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30b8004030 con 0x55dd31bf5ed0 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30a4ff9640 1 -- 172.21.15.22:0/2046740344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30ac011e50 con 0x55dd31bf5ed0 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c61fc740 10 client.19995 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55dd319387d0 con 0x55dd31bf5ed0 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.513+0000 7f30a4ff9640 1 -- 172.21.15.22:0/2046740344 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30b8004030 con 0x55dd31bf5ed0 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30a4ff9640 1 -- 172.21.15.22:0/2046740344 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f30ac004ba0 con 0x55dd31bf5ed0 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c61fc740 10 client.19995 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:36.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c61fc740 10 client.19995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c61fc740 20 client.19995 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c13d70 0x55dd31c16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c61fc740 4 client.19995 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd31b67400 con 0x55dd31c13d70 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c12d7640 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c13d70 0x55dd31c16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30a4ff9640 10 client.19995 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.514+0000 7f30c12d7640 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c13d70 0x55dd31c16160 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a78d19d7a31936d server_cookie=2df542f9ca2e8d4 in_seq=0 out_seq=0 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30a4ff9640 1 -- 172.21.15.22:0/2046740344 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd31b67400 con 0x55dd31c13d70 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30a4ff9640 10 client.19995 handle_command_reply: tid=0 2024-03-09T00:08:36.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 client.19995 shutdown 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf5ed0 msgr2=0x55dd31c00bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf5ed0 0x55dd31c00bf0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e10 tx=0x7f30ac004730 comp rx=0 tx=0).stop 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 shutdown_connections 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c13d70 0x55dd31c16160 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31c03ab0 0x55dd31c01670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31c01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30a4ff9640 0 client.19995 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 --2- 172.21.15.22:0/2046740344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf5ed0 0x55dd31c00bf0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 >> 172.21.15.22:0/2046740344 conn(0x55dd31bf08d0 msgr2=0x55dd31bf2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 shutdown_connections 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 1 -- 172.21.15.22:0/2046740344 wait complete. 2024-03-09T00:08:36.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.517+0000 7f30c61fc740 20 client.19995 trim_cache size 0 max 16384 2024-03-09T00:08:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.529+0000 7f30c61fc740 1 Processor -- start 2024-03-09T00:08:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.529+0000 7f30c61fc740 1 -- start start 2024-03-09T00:08:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c03320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31bfde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31bfe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c61fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1ae960 con 0x55dd31c03ab0 2024-03-09T00:08:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c61fc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1ac4a0 con 0x55dd31bf67f0 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c61fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1aa560 con 0x55dd31bf5ed0 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c0ad6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31bfe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31bfde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31bfde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:38126/0 (socket says 172.21.15.22:38126) 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 -- 172.21.15.22:0/1693196130 learned_addr learned my addr 172.21.15.22:0/1693196130 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c0ad6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c03320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 -- 172.21.15.22:0/1693196130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 msgr2=0x55dd31c03320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1693196130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c03320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 -- 172.21.15.22:0/1693196130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 msgr2=0x55dd31bfe3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1693196130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31bfe3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 -- 172.21.15.22:0/1693196130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd31bff140 con 0x55dd31bf67f0 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30c12d7640 1 --2- 172.21.15.22:0/1693196130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31bfde70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdbcbac50c59866 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.530+0000 7f30a6ffd640 1 -- 172.21.15.22:0/1693196130 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8042070 con 0x55dd31bf67f0 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30a6ffd640 1 -- 172.21.15.22:0/1693196130 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30b8004030 con 0x55dd31bf67f0 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30a6ffd640 1 -- 172.21.15.22:0/1693196130 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b804e6b0 con 0x55dd31bf67f0 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 -- 172.21.15.22:0/1693196130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 msgr2=0x55dd31bfde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 --2- 172.21.15.22:0/1693196130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31bfde70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b802bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 -- 172.21.15.22:0/1693196130 shutdown_connections 2024-03-09T00:08:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 --2- 172.21.15.22:0/1693196130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31c03ab0 0x55dd31bfe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 --2- 172.21.15.22:0/1693196130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31bf67f0 0x55dd31bfde70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 --2- 172.21.15.22:0/1693196130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c03320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 -- 172.21.15.22:0/1693196130 >> 172.21.15.22:0/1693196130 conn(0x55dd31bf08d0 msgr2=0x55dd31bff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 -- 172.21.15.22:0/1693196130 shutdown_connections 2024-03-09T00:08:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 -- 172.21.15.22:0/1693196130 wait complete. 2024-03-09T00:08:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.531+0000 7f30c61fc740 1 -- start start 2024-03-09T00:08:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 1 Processor -- start 2024-03-09T00:08:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c0ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf67f0 0x55dd31c090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31c03ab0 0x55dd31c09610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1aab60 con 0x55dd31bf67f0 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1abe40 con 0x55dd31c03ab0 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc1ab180 con 0x55dd31bf5ed0 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c61fc740 10 client.0 _ll_register_callbacks cb 0x7f30ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf67f0 0x55dd31c090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf67f0 0x55dd31c090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54882/0 (socket says 172.21.15.22:54882) 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 -- 172.21.15.22:0/3896397408 learned_addr learned my addr 172.21.15.22:0/3896397408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c0ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31c03ab0 0x55dd31c09610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 -- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 msgr2=0x55dd31c0ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c0ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 -- 172.21.15.22:0/3896397408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31c03ab0 msgr2=0x55dd31c09610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31c03ab0 0x55dd31c09610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c12d7640 1 -- 172.21.15.22:0/3896397408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd31c09d50 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.532+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31c03ab0 0x55dd31c09610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c12d7640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf67f0 0x55dd31c090d0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009900 tx=0x7f30b80095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b1320385d2a6a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30a4ff9640 1 -- 172.21.15.22:0/3896397408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8042040 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30a4ff9640 1 -- 172.21.15.22:0/3896397408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f30b804ede0 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30b8004030 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30a4ff9640 1 -- 172.21.15.22:0/3896397408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30b805d050 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 10 client.20001 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55dd31c6b1e0 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30a4ff9640 1 -- 172.21.15.22:0/3896397408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30b8004030 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30a4ff9640 1 -- 172.21.15.22:0/3896397408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f30b80035f0 con 0x55dd31bf67f0 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 10 client.20001 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:36.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 10 client.20001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 20 client.20001 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c6b4a0 0x55dd31c6d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 4 client.20001 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:08:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.533+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd31b61750 con 0x55dd31c6b4a0 2024-03-09T00:08:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.534+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c6b4a0 0x55dd31c6d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.534+0000 7f30a4ff9640 10 client.20001 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:36.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.534+0000 7f30c0ad6640 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c6b4a0 0x55dd31c6d890 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b234d3210b51ae86 server_cookie=ec5a3af7fe45b843 in_seq=0 out_seq=0 2024-03-09T00:08:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30a4ff9640 1 -- 172.21.15.22:0/3896397408 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd31b61750 con 0x55dd31c6b4a0 2024-03-09T00:08:36.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30a4ff9640 10 client.20001 handle_command_reply: tid=0 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 client.20001 shutdown 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf67f0 msgr2=0x55dd31c090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf67f0 0x55dd31c090d0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009900 tx=0x7f30b80095f0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 shutdown_connections 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55dd31c6b4a0 0x55dd31c6d890 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd31c03ab0 0x55dd31c09610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dd31bf67f0 0x55dd31c090d0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 --2- 172.21.15.22:0/3896397408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd31bf5ed0 0x55dd31c0ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30a4ff9640 0 client.20001 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 >> 172.21.15.22:0/3896397408 conn(0x55dd31bf08d0 msgr2=0x55dd31bf1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 shutdown_connections 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 1 -- 172.21.15.22:0/3896397408 wait complete. 2024-03-09T00:08:36.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.539+0000 7f30c61fc740 20 client.20001 trim_cache size 0 max 16384 2024-03-09T00:08:36.538 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30a80a2970 msgr2=0x7f30a80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30a80a2970 0x7f30a80a4e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f30b4097e30 tx=0x7f30b400efd0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc197970 msgr2=0x7f30bc197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc197970 0x7f30bc197e20 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f30b400eb10 tx=0x7f30b400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 shutdown_connections 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f30a80a2970 0x7f30a80a4e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30bc197970 0x7f30bc197e20 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30bc1054b0 0x7f30bc19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 --2- 172.21.15.22:0/1335560958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30bc1003e0 0x7f30bc19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 >> 172.21.15.22:0/1335560958 conn(0x7f30bc0773a0 msgr2=0x7f30bc0687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 shutdown_connections 2024-03-09T00:08:36.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:36.540+0000 7f30c353b640 1 -- 172.21.15.22:0/1335560958 wait complete. 2024-03-09T00:08:36.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:36 smithi062 ceph-mon[36129]: pgmap v444: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 653 op/s 2024-03-09T00:08:36.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:36 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3228021787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:08:36.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:36 smithi192 ceph-mon[35995]: pgmap v444: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 653 op/s 2024-03-09T00:08:36.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:36 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3228021787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:08:37.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 10577 17.26 MB/sec warmup 44 sec latency 177.767 ms 2024-03-09T00:08:37.546 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:08:37.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:08:37.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:08:37.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 -- 172.21.15.22:0/4190529236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101003d0 msgr2=0x7f41101007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 --2- 172.21.15.22:0/4190529236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101003d0 0x7f41101007b0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f410c009900 tx=0x7f410c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:37.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 -- 172.21.15.22:0/4190529236 shutdown_connections 2024-03-09T00:08:37.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 --2- 172.21.15.22:0/4190529236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41101054a0 0x7f411010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 --2- 172.21.15.22:0/4190529236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4110100d80 0x7f4110104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 --2- 172.21.15.22:0/4190529236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101003d0 0x7f41101007b0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 -- 172.21.15.22:0/4190529236 >> 172.21.15.22:0/4190529236 conn(0x7f41100772f0 msgr2=0x7f4110077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 -- 172.21.15.22:0/4190529236 shutdown_connections 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.665+0000 7f4117f90640 1 -- 172.21.15.22:0/4190529236 wait complete. 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 Processor -- start 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 -- start start 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41101003d0 0x7f4110101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101054a0 0x7f41101023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41101028e0 0x7f41101a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4110110560 con 0x7f41101054a0 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101103e0 con 0x7f41101028e0 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4117f90640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101106e0 con 0x7f41101003d0 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101054a0 0x7f41101023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101054a0 0x7f41101023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54888/0 (socket says 172.21.15.22:54888) 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 -- 172.21.15.22:0/2676299735 learned_addr learned my addr 172.21.15.22:0/2676299735 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f4115d2c640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41101003d0 0x7f4110101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411652d640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41101028e0 0x7f41101a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 -- 172.21.15.22:0/2676299735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41101003d0 msgr2=0x7f4110101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41101003d0 0x7f4110101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 -- 172.21.15.22:0/2676299735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41101028e0 msgr2=0x7f41101a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41101028e0 0x7f41101a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.666+0000 7f411552b640 1 -- 172.21.15.22:0/2676299735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41101a63c0 con 0x7f41101054a0 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.667+0000 7f411552b640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101054a0 0x7f41101023a0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f410000c970 tx=0x7f410000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.667+0000 7f4106ffd640 1 -- 172.21.15.22:0/2676299735 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100014070 con 0x7f41101054a0 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.667+0000 7f4106ffd640 1 -- 172.21.15.22:0/2676299735 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4100004440 con 0x7f41101054a0 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.667+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41101a66b0 con 0x7f41101054a0 2024-03-09T00:08:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.667+0000 7f4106ffd640 1 -- 172.21.15.22:0/2676299735 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410000f450 con 0x7f41101054a0 2024-03-09T00:08:37.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.667+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41101a6c00 con 0x7f41101054a0 2024-03-09T00:08:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.669+0000 7f4106ffd640 1 -- 172.21.15.22:0/2676299735 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4100017070 con 0x7f41101054a0 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.669+0000 7f4106ffd640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f40fc0a29a0 0x7f40fc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.669+0000 7f4106ffd640 1 -- 172.21.15.22:0/2676299735 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f41000cecb0 con 0x7f41101054a0 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.669+0000 7f4115d2c640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f40fc0a29a0 0x7f40fc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 Processor -- start 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 -- start start 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e042d80 0x55bb3e043160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e04bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e04e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3dfa6b40 con 0x55bb3e0436a0 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3df4fd80 con 0x55bb3e042d80 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411ab71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3dfa2be0 con 0x55bb3e04c280 2024-03-09T00:08:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f4115d2c640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f40fc0a29a0 0x7f40fc0a4e60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f410c006010 tx=0x7f410c005dd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411552b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e04bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411552b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e04bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54904/0 (socket says 172.21.15.22:54904) 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411552b640 1 -- 172.21.15.22:0/1332932006 learned_addr learned my addr 172.21.15.22:0/1332932006 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411652d640 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e04e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f4115d2c640 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e042d80 0x55bb3e043160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411552b640 1 -- 172.21.15.22:0/1332932006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 msgr2=0x55bb3e04e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411552b640 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e04e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411552b640 1 -- 172.21.15.22:0/1332932006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e042d80 msgr2=0x55bb3e043160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.670+0000 7f411552b640 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e042d80 0x55bb3e043160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.671+0000 7f411552b640 1 -- 172.21.15.22:0/1332932006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb3dd837e0 con 0x55bb3e0436a0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.671+0000 7f411552b640 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e04bd40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f41000a3180 tx=0x7f410009a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0180f1996b11b47 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.671+0000 7f40fbfff640 1 -- 172.21.15.22:0/1332932006 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41000a36b0 con 0x55bb3e0436a0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.671+0000 7f40fbfff640 1 -- 172.21.15.22:0/1332932006 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41000035f0 con 0x55bb3e0436a0 2024-03-09T00:08:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.671+0000 7f40fbfff640 1 -- 172.21.15.22:0/1332932006 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410009a5c0 con 0x55bb3e0436a0 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 -- 172.21.15.22:0/1332932006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 msgr2=0x55bb3e04bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e04bd40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f41000a3180 tx=0x7f410009a930 comp rx=0 tx=0).stop 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 -- 172.21.15.22:0/1332932006 shutdown_connections 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e04e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e04bd40 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 --2- 172.21.15.22:0/1332932006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e042d80 0x55bb3e043160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 -- 172.21.15.22:0/1332932006 >> 172.21.15.22:0/1332932006 conn(0x55bb3e03d4b0 msgr2=0x55bb3e03d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 -- 172.21.15.22:0/1332932006 shutdown_connections 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 -- 172.21.15.22:0/1332932006 wait complete. 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 -- start start 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.672+0000 7f411ab71740 1 Processor -- start 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e04bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e0490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e04c280 0x55bb3e049790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3e04f070 con 0x55bb3e0436a0 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3df631c0 con 0x55bb3e04c280 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3e04f1f0 con 0x55bb3e042d80 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 10 client.0 _ll_register_callbacks cb 0x55bb3df527a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e0490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e0490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54916/0 (socket says 172.21.15.22:54916) 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 -- 172.21.15.22:0/647785753 learned_addr learned my addr 172.21.15.22:0/647785753 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411652d640 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e04c280 0x55bb3e049790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f4115d2c640 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e04bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 -- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 msgr2=0x55bb3e04bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e04bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 -- 172.21.15.22:0/647785753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e04c280 msgr2=0x55bb3e049790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e04c280 0x55bb3e049790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 -- 172.21.15.22:0/647785753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb3dde3f70 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411552b640 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e0490a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f41000cc600 tx=0x7f410009a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb792a71d1c1ba82 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f40f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f40f97fa640 1 -- 172.21.15.22:0/647785753 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41000a36b0 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f40f97fa640 1 -- 172.21.15.22:0/647785753 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f410009a570 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41000035f0 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f40f97fa640 1 -- 172.21.15.22:0/647785753 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4100096c60 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 10 client.20013 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.673+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bb3dc7b170 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f40f97fa640 1 -- 172.21.15.22:0/647785753 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41000035f0 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f40f97fa640 1 -- 172.21.15.22:0/647785753 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f41000b0030 con 0x55bb3e0436a0 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f411ab71740 10 client.20013 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f411ab71740 10 client.20013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f411ab71740 20 client.20013 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f411ab71740 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e05c280 0x55bb3e05e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f411ab71740 4 client.20013 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb3df7c960 con 0x55bb3e05c280 2024-03-09T00:08:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f4115d2c640 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e05c280 0x55bb3e05e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f4115d2c640 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e05c280 0x55bb3e05e670 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1451cc5abb37f9cd server_cookie=646e67f3071d1481 in_seq=0 out_seq=0 2024-03-09T00:08:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.674+0000 7f40f97fa640 10 client.20013 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f40f97fa640 1 -- 172.21.15.22:0/647785753 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb3df7c960 con 0x55bb3e05c280 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f40f97fa640 10 client.20013 handle_command_reply: tid=0 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 client.20013 shutdown 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 msgr2=0x55bb3e0490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e0490a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f41000cc600 tx=0x7f410009a9a0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 shutdown_connections 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e05c280 0x55bb3e05e670 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e04c280 0x55bb3e049790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e0436a0 0x55bb3e0490a0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f40f97fa640 0 client.20013 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 --2- 172.21.15.22:0/647785753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e04bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.682+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 >> 172.21.15.22:0/647785753 conn(0x55bb3e03d4b0 msgr2=0x55bb3e03f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.683+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 shutdown_connections 2024-03-09T00:08:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.683+0000 7f411ab71740 1 -- 172.21.15.22:0/647785753 wait complete. 2024-03-09T00:08:37.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.683+0000 7f411ab71740 20 client.20013 trim_cache size 0 max 16384 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 Processor -- start 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 -- start start 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e058870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e04b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e04c280 0x55bb3e04bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101b2d90 con 0x55bb3e04c280 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101b08d0 con 0x55bb3e0436a0 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411ab71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101ae990 con 0x55bb3e042d80 2024-03-09T00:08:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411652d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e04c280 0x55bb3e04bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411652d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e04c280 0x55bb3e04bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54922/0 (socket says 172.21.15.22:54922) 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411652d640 1 -- 172.21.15.22:0/2737692431 learned_addr learned my addr 172.21.15.22:0/2737692431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f4115d2c640 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e058870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.698+0000 7f411552b640 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e04b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f411652d640 1 -- 172.21.15.22:0/2737692431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 msgr2=0x55bb3e058870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f411652d640 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e058870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f411652d640 1 -- 172.21.15.22:0/2737692431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 msgr2=0x55bb3e04b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f411652d640 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e04b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f411652d640 1 -- 172.21.15.22:0/2737692431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb3e049b00 con 0x55bb3e04c280 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f411652d640 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e04c280 0x55bb3e04bd10 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4108003cc0 tx=0x7f4108004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed14cc0c2caca4e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f40fb7fe640 1 -- 172.21.15.22:0/2737692431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108004b20 con 0x55bb3e04c280 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f40fb7fe640 1 -- 172.21.15.22:0/2737692431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4108004cc0 con 0x55bb3e04c280 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.699+0000 7f40fb7fe640 1 -- 172.21.15.22:0/2737692431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108010720 con 0x55bb3e04c280 2024-03-09T00:08:37.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- 172.21.15.22:0/2737692431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e04c280 msgr2=0x55bb3e04bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e04c280 0x55bb3e04bd10 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4108003cc0 tx=0x7f4108004410 comp rx=0 tx=0).stop 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- 172.21.15.22:0/2737692431 shutdown_connections 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e04c280 0x55bb3e04bd10 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e04b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 --2- 172.21.15.22:0/2737692431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e042d80 0x55bb3e058870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- 172.21.15.22:0/2737692431 >> 172.21.15.22:0/2737692431 conn(0x55bb3e03d4b0 msgr2=0x55bb3e062c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- 172.21.15.22:0/2737692431 shutdown_connections 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- 172.21.15.22:0/2737692431 wait complete. 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- start start 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 Processor -- start 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e042d80 0x55bb3e0ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e0acef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e052960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101aef90 con 0x55bb3e042d80 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101b0270 con 0x55bb3e0436a0 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.700+0000 7f411ab71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101af5b0 con 0x55bb3e04c280 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f411ab71740 10 client.0 _ll_register_callbacks cb 0x7f4100003f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e042d80 0x55bb3e0ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e042d80 0x55bb3e0ac9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54932/0 (socket says 172.21.15.22:54932) 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 -- 172.21.15.22:0/2378169609 learned_addr learned my addr 172.21.15.22:0/2378169609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f411552b640 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e0acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f411652d640 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e052960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 -- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 msgr2=0x55bb3e052960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e052960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 -- 172.21.15.22:0/2378169609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 msgr2=0x55bb3e0acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e0acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 -- 172.21.15.22:0/2378169609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb3e05e9e0 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f4115d2c640 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e042d80 0x55bb3e0ac9b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f410c02cfd0 tx=0x7f410c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0f90cf9d9e21d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f40f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f40f97fa640 1 -- 172.21.15.22:0/2378169609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c004450 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f40f97fa640 1 -- 172.21.15.22:0/2378169609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f410c055320 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4108004cc0 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f40f97fa640 1 -- 172.21.15.22:0/2378169609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f410c066050 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f411ab71740 10 client.20019 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.701+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bb3e0434d0 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f40f97fa640 1 -- 172.21.15.22:0/2378169609 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4108004cc0 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f40f97fa640 1 -- 172.21.15.22:0/2378169609 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f410c004f50 con 0x55bb3e042d80 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411ab71740 10 client.20019 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411ab71740 10 client.20019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411ab71740 20 client.20019 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411ab71740 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e0b4650 0x55bb3e0b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411ab71740 4 client.20019 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:08:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bb3df68080 con 0x55bb3e0b4650 2024-03-09T00:08:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411552b640 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e0b4650 0x55bb3e0b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f40f97fa640 10 client.20019 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.702+0000 7f411552b640 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e0b4650 0x55bb3e0b6a70 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3dedffdeb0e2d0e server_cookie=bbd07c83f9fe57ff in_seq=0 out_seq=0 2024-03-09T00:08:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f40f97fa640 1 -- 172.21.15.22:0/2378169609 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bb3df68080 con 0x55bb3e0b4650 2024-03-09T00:08:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f40f97fa640 10 client.20019 handle_command_reply: tid=0 2024-03-09T00:08:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 client.20019 shutdown 2024-03-09T00:08:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e042d80 msgr2=0x55bb3e0ac9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e042d80 0x55bb3e0ac9b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f410c02cfd0 tx=0x7f410c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 shutdown_connections 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bb3e0b4650 0x55bb3e0b6a70 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb3e04c280 0x55bb3e052960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3e0436a0 0x55bb3e0acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 --2- 172.21.15.22:0/2378169609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb3e042d80 0x55bb3e0ac9b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f40f97fa640 0 client.20019 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 >> 172.21.15.22:0/2378169609 conn(0x55bb3e03d4b0 msgr2=0x55bb3e03bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 shutdown_connections 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 1 -- 172.21.15.22:0/2378169609 wait complete. 2024-03-09T00:08:37.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.704+0000 7f411ab71740 20 client.20019 trim_cache size 0 max 16384 2024-03-09T00:08:37.706 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:08:37.706 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:08:37.706 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "6a04c4cd-2b82-4a88-ad3f-eab86c4f9e04", 2024-03-09T00:08:37.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:08:37.706 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:08:37.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f40fc0a29a0 msgr2=0x7f40fc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f40fc0a29a0 0x7f40fc0a4e60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f410c006010 tx=0x7f410c005dd0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101054a0 msgr2=0x7f41101023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101054a0 0x7f41101023a0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f410000c970 tx=0x7f410000ce30 comp rx=0 tx=0).stop 2024-03-09T00:08:37.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 shutdown_connections 2024-03-09T00:08:37.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f40fc0a29a0 0x7f40fc0a4e60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41101028e0 0x7f41101a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41101054a0 0x7f41101023a0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 --2- 172.21.15.22:0/2676299735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41101003d0 0x7f4110101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 >> 172.21.15.22:0/2676299735 conn(0x7f41100772f0 msgr2=0x7f4110068790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 shutdown_connections 2024-03-09T00:08:37.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.705+0000 7f4117f90640 1 -- 172.21.15.22:0/2676299735 wait complete. 2024-03-09T00:08:37.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:08:37.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:08:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 -- 172.21.15.22:0/2434261291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd0101ea0 msgr2=0x7f6bd0105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2434261291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd0101ea0 0x7f6bd0105ee0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009900 tx=0x7f6bc402bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 -- 172.21.15.22:0/2434261291 shutdown_connections 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2434261291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd01065c0 0x7f6bd010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2434261291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd0101ea0 0x7f6bd0105ee0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2434261291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd01014f0 0x7f6bd01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 -- 172.21.15.22:0/2434261291 >> 172.21.15.22:0/2434261291 conn(0x7f6bd00786f0 msgr2=0x7f6bd0078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 -- 172.21.15.22:0/2434261291 shutdown_connections 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 -- 172.21.15.22:0/2434261291 wait complete. 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.836+0000 7f6bd5868640 1 Processor -- start 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bd5868640 1 -- start start 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bd5868640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd01014f0 0x7f6bd019d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bd5868640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01065c0 0x7f6bd019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bd5868640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd01979d0 0x7f6bd0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bd5868640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0111650 con 0x7f6bd01065c0 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bd5868640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01114d0 con 0x7f6bd01979d0 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bd5868640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01117d0 con 0x7f6bd01014f0 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01065c0 0x7f6bd019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01065c0 0x7f6bd019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54956/0 (socket says 172.21.15.22:54956) 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 -- 172.21.15.22:0/2154500307 learned_addr learned my addr 172.21.15.22:0/2154500307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bceffd640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd01014f0 0x7f6bd019d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd01979d0 0x7f6bd0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 -- 172.21.15.22:0/2154500307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd01014f0 msgr2=0x7f6bd019d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd01014f0 0x7f6bd019d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 -- 172.21.15.22:0/2154500307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd01979d0 msgr2=0x7f6bd0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd01979d0 0x7f6bd0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 -- 172.21.15.22:0/2154500307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd01984f0 con 0x7f6bd01065c0 2024-03-09T00:08:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.837+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01065c0 0x7f6bd019dd60 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4036040 tx=0x7f6bc402cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.838+0000 7f6bd4866640 1 -- 172.21.15.22:0/2154500307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4041070 con 0x7f6bd01065c0 2024-03-09T00:08:37.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.838+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd0198780 con 0x7f6bd01065c0 2024-03-09T00:08:37.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.838+0000 7f6bd4866640 1 -- 172.21.15.22:0/2154500307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc4004450 con 0x7f6bd01065c0 2024-03-09T00:08:37.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.838+0000 7f6bd4866640 1 -- 172.21.15.22:0/2154500307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4003dc0 con 0x7f6bd01065c0 2024-03-09T00:08:37.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.838+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd01a49d0 con 0x7f6bd01065c0 2024-03-09T00:08:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.840+0000 7f6bd4866640 1 -- 172.21.15.22:0/2154500307 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6bc4044070 con 0x7f6bd01065c0 2024-03-09T00:08:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 Processor -- start 2024-03-09T00:08:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 -- start start 2024-03-09T00:08:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48403b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48404080 0x559e4840c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e4840f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e48366e00 con 0x559e48403760 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4830fb10 con 0x559e48404080 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd8446740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e48362ea0 con 0x559e4840cc60 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd4866640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bb80a2970 0x7f6bb80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48403b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48404080 0x559e4840c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48403b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54960/0 (socket says 172.21.15.22:54960) 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bcf7fe640 1 -- 172.21.15.22:0/1321842345 learned_addr learned my addr 172.21.15.22:0/1321842345 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e4840f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bd4866640 1 -- 172.21.15.22:0/2154500307 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6bc40f0c00 con 0x7f6bd01065c0 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bb80a2970 0x7f6bb80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bceffd640 1 -- 172.21.15.22:0/1321842345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 msgr2=0x559e4840f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bceffd640 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e4840f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bceffd640 1 -- 172.21.15.22:0/1321842345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 msgr2=0x559e48403b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bceffd640 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48403b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bceffd640 1 -- 172.21.15.22:0/1321842345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e481457d0 con 0x559e48404080 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.841+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48403b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.842+0000 7f6bceffd640 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48404080 0x559e4840c720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009bd0 tx=0x7f6bc0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43894a8ec6fe7d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.842+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bb80a2970 0x7f6bb80a4e30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80049d0 tx=0x7f6bc8004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.842+0000 7f6bb5ffb640 1 -- 172.21.15.22:0/1321842345 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0004230 con 0x559e48404080 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.842+0000 7f6bb5ffb640 1 -- 172.21.15.22:0/1321842345 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc0009e40 con 0x559e48404080 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.842+0000 7f6bb5ffb640 1 -- 172.21.15.22:0/1321842345 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0004d20 con 0x559e48404080 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- 172.21.15.22:0/1321842345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48404080 msgr2=0x559e4840c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48404080 0x559e4840c720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009bd0 tx=0x7f6bc0011810 comp rx=0 tx=0).stop 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- 172.21.15.22:0/1321842345 shutdown_connections 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e4840f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48404080 0x559e4840c720 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1321842345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48403b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- 172.21.15.22:0/1321842345 >> 172.21.15.22:0/1321842345 conn(0x559e483fde90 msgr2=0x559e483fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- 172.21.15.22:0/1321842345 shutdown_connections 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- 172.21.15.22:0/1321842345 wait complete. 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- start start 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 Processor -- start 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48403760 0x559e4840c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e48409670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e48409bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4840fa50 con 0x559e48404080 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e483230a0 con 0x559e48403760 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.843+0000 7f6bd8446740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4840fbd0 con 0x559e4840cc60 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e48409670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bd8446740 10 client.0 _ll_register_callbacks cb 0x559e48322a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e48409670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54974/0 (socket says 172.21.15.22:54974) 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 -- 172.21.15.22:0/3116280823 learned_addr learned my addr 172.21.15.22:0/3116280823 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e48409bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bceffd640 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48403760 0x559e4840c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 -- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 msgr2=0x559e48409bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e48409bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 -- 172.21.15.22:0/3116280823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48403760 msgr2=0x559e4840c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48403760 0x559e4840c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 -- 172.21.15.22:0/3116280823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e4807b3f0 con 0x559e48404080 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e48409670 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40b99f0 tx=0x7f6bc403aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56995b560212c849 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6baeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6baeffd640 1 -- 172.21.15.22:0/3116280823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc404b040 con 0x559e48404080 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6baeffd640 1 -- 172.21.15.22:0/3116280823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6bc40b9690 con 0x559e48404080 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e48374340 con 0x559e48404080 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6baeffd640 1 -- 172.21.15.22:0/3116280823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc40bc980 con 0x559e48404080 2024-03-09T00:08:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bd8446740 10 client.20031 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.844+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559e4803b0f0 con 0x559e48404080 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6baeffd640 1 -- 172.21.15.22:0/3116280823 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e48374340 con 0x559e48404080 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6baeffd640 1 -- 172.21.15.22:0/3116280823 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6bc404b1e0 con 0x559e48404080 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bd8446740 10 client.20031 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bd8446740 10 client.20031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bd8446740 20 client.20031 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bd8446740 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e4841c560 0x559e4841ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bd8446740 4 client.20031 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:37.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e4836e690 con 0x559e4841c560 2024-03-09T00:08:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bceffd640 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e4841c560 0x559e4841ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6bceffd640 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e4841c560 0x559e4841ea00 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce27f191082c6ade server_cookie=2fc2a4b6688df64 in_seq=0 out_seq=0 2024-03-09T00:08:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.845+0000 7f6baeffd640 10 client.20031 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6baeffd640 1 -- 172.21.15.22:0/3116280823 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e4836e690 con 0x559e4841c560 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6baeffd640 10 client.20031 handle_command_reply: tid=0 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 client.20031 shutdown 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 msgr2=0x559e48409670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e48409670 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40b99f0 tx=0x7f6bc403aa90 comp rx=0 tx=0).stop 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 shutdown_connections 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e4841c560 0x559e4841ea00 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e4840cc60 0x559e48409bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e48409670 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6baeffd640 0 client.20031 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 --2- 172.21.15.22:0/3116280823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e48403760 0x559e4840c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 >> 172.21.15.22:0/3116280823 conn(0x559e483fde90 msgr2=0x559e483ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 shutdown_connections 2024-03-09T00:08:37.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 1 -- 172.21.15.22:0/3116280823 wait complete. 2024-03-09T00:08:37.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.849+0000 7f6bd8446740 20 client.20031 trim_cache size 0 max 16384 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.864+0000 7f6bd8446740 1 Processor -- start 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.864+0000 7f6bd8446740 1 -- start start 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48403760 0x559e48404cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e4840bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e4840c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bd8446740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01b0ba0 con 0x559e48404080 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bd8446740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01ae6e0 con 0x559e4840cc60 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bd8446740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01ac7a0 con 0x559e48403760 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e4840bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e4840bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54976/0 (socket says 172.21.15.22:54976) 2024-03-09T00:08:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 -- 172.21.15.22:0/450523770 learned_addr learned my addr 172.21.15.22:0/450523770 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bceffd640 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48403760 0x559e48404cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e4840c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 -- 172.21.15.22:0/450523770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48403760 msgr2=0x559e48404cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48403760 0x559e48404cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 -- 172.21.15.22:0/450523770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 msgr2=0x559e4840c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e4840c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 -- 172.21.15.22:0/450523770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e48407be0 con 0x559e48404080 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e4840c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bceffd640 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48403760 0x559e48404cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e4840bda0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4000c00 tx=0x7f6bc402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969e7beba33bba9d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bb57fa640 1 -- 172.21.15.22:0/450523770 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40c5710 con 0x559e48404080 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bb57fa640 1 -- 172.21.15.22:0/450523770 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc40da070 con 0x559e48404080 2024-03-09T00:08:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.865+0000 7f6bb57fa640 1 -- 172.21.15.22:0/450523770 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40c5280 con 0x559e48404080 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 -- 172.21.15.22:0/450523770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 msgr2=0x559e4840bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e4840bda0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4000c00 tx=0x7f6bc402bc30 comp rx=0 tx=0).stop 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 -- 172.21.15.22:0/450523770 shutdown_connections 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e4840c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48404080 0x559e4840bda0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 --2- 172.21.15.22:0/450523770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48403760 0x559e48404cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 -- 172.21.15.22:0/450523770 >> 172.21.15.22:0/450523770 conn(0x559e483fde90 msgr2=0x559e484045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 -- 172.21.15.22:0/450523770 shutdown_connections 2024-03-09T00:08:37.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 -- 172.21.15.22:0/450523770 wait complete. 2024-03-09T00:08:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.866+0000 7f6bd8446740 1 -- start start 2024-03-09T00:08:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 1 Processor -- start 2024-03-09T00:08:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48406930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48404080 0x559e48406e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e48472b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01acda0 con 0x559e48403760 2024-03-09T00:08:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01ae080 con 0x559e4840cc60 2024-03-09T00:08:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01ad3c0 con 0x559e48404080 2024-03-09T00:08:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bd8446740 10 client.0 _ll_register_callbacks cb 0x7f6bc40b9320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48406930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48406930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54980/0 (socket says 172.21.15.22:54980) 2024-03-09T00:08:37.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 -- 172.21.15.22:0/1574349930 learned_addr learned my addr 172.21.15.22:0/1574349930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bcf7fe640 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e48472b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48404080 0x559e48406e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 -- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48404080 msgr2=0x559e48406e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48404080 0x559e48406e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 -- 172.21.15.22:0/1574349930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 msgr2=0x559e48472b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e48472b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 -- 172.21.15.22:0/1574349930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e48473060 con 0x559e48403760 2024-03-09T00:08:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6bceffd640 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48406930 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0019410 tx=0x7f6bc0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11964045f852b9ed server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6baeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.867+0000 7f6baeffd640 1 -- 172.21.15.22:0/1574349930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc001e040 con 0x559e48403760 2024-03-09T00:08:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6baeffd640 1 -- 172.21.15.22:0/1574349930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6bc0004040 con 0x559e48403760 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bc40da070 con 0x559e48403760 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6baeffd640 1 -- 172.21.15.22:0/1574349930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc0038050 con 0x559e48403760 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 10 client.20037 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559e48474270 con 0x559e48403760 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6baeffd640 1 -- 172.21.15.22:0/1574349930 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc40da070 con 0x559e48403760 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6baeffd640 1 -- 172.21.15.22:0/1574349930 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6bc0018070 con 0x559e48403760 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 10 client.20037 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 10 client.20037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 20 client.20037 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e48474860 0x559e48476c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 4 client.20037 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:08:37.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559e482ec4e0 con 0x559e48474860 2024-03-09T00:08:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.868+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e48474860 0x559e48476c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.869+0000 7f6baeffd640 10 client.20037 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.869+0000 7f6bce7fc640 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e48474860 0x559e48476c80 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def14b2deb3e4b0 server_cookie=2647c441ca2fbc3 in_seq=0 out_seq=0 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6baeffd640 1 -- 172.21.15.22:0/1574349930 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x559e482ec4e0 con 0x559e48474860 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6baeffd640 10 client.20037 handle_command_reply: tid=0 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 client.20037 shutdown 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 msgr2=0x559e48406930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48406930 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0019410 tx=0x7f6bc0011e50 comp rx=0 tx=0).stop 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 shutdown_connections 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559e48474860 0x559e48476c80 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e4840cc60 0x559e48472b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6baeffd640 0 client.20037 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559e48404080 0x559e48406e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 --2- 172.21.15.22:0/1574349930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559e48403760 0x559e48406930 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 >> 172.21.15.22:0/1574349930 conn(0x559e483fde90 msgr2=0x559e483ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 shutdown_connections 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 1 -- 172.21.15.22:0/1574349930 wait complete. 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.874+0000 7f6bd8446740 20 client.20037 trim_cache size 0 max 16384 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (491 inodes in the stack)", 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout: "6a04c4cd-2b82-4a88-ad3f-eab86c4f9e04": { 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "6a04c4cd-2b82-4a88-ad3f-eab86c4f9e04", 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:08:37.876 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:08:37.877 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:08:37.877 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:08:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bb80a2970 msgr2=0x7f6bb80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bb80a2970 0x7f6bb80a4e30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80049d0 tx=0x7f6bc8004ab0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01065c0 msgr2=0x7f6bd019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01065c0 0x7f6bd019dd60 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4036040 tx=0x7f6bc402cb20 comp rx=0 tx=0).stop 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 shutdown_connections 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6bb80a2970 0x7f6bb80a4e30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd01979d0 0x7f6bd0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bd01065c0 0x7f6bd019dd60 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 --2- 172.21.15.22:0/2154500307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd01014f0 0x7f6bd019d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 >> 172.21.15.22:0/2154500307 conn(0x7f6bd00786f0 msgr2=0x7f6bd00ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 shutdown_connections 2024-03-09T00:08:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:08:37.875+0000 7f6bd5868640 1 -- 172.21.15.22:0/2154500307 wait complete. 2024-03-09T00:08:37.880 INFO:tasks.cephfs.filesystem:scrub status for tag:6a04c4cd-2b82-4a88-ad3f-eab86c4f9e04 - {'path': '/', 'tag': '6a04c4cd-2b82-4a88-ad3f-eab86c4f9e04', 'options': 'recursive,force'} 2024-03-09T00:08:38.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 10726 16.99 MB/sec warmup 45 sec latency 78.422 ms 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: pgmap v445: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:38 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: pgmap v445: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:38 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: pgmap v445: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-03-09T00:08:38.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:08:38.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:38.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:38.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:38.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:38.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:38 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 10889 16.72 MB/sec warmup 46 sec latency 185.352 ms 2024-03-09T00:08:39.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:39 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:39 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:39 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:39 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:39 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:39 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:39.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:39 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:39 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:08:39.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:39 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:39 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:39 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:39.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:39 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:39 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:08:39.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:39 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:39.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:39 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:08:40.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 11053 16.41 MB/sec warmup 47 sec latency 266.506 ms 2024-03-09T00:08:40.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:40 smithi022 ceph-mon[31894]: pgmap v446: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-03-09T00:08:40.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:40 smithi062 ceph-mon[36129]: pgmap v446: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-03-09T00:08:40.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:40 smithi192 ceph-mon[35995]: pgmap v446: 129 pgs: 129 active+clean; 296 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-03-09T00:08:41.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 11317 16.24 MB/sec warmup 48 sec latency 265.216 ms 2024-03-09T00:08:42.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 11718 16.35 MB/sec warmup 49 sec latency 210.234 ms 2024-03-09T00:08:42.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:42 smithi022 ceph-mon[31894]: pgmap v447: 129 pgs: 129 active+clean; 308 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.3 MiB/s wr, 541 op/s 2024-03-09T00:08:42.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:42 smithi062 ceph-mon[36129]: pgmap v447: 129 pgs: 129 active+clean; 308 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.3 MiB/s wr, 541 op/s 2024-03-09T00:08:42.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:42 smithi192 ceph-mon[35995]: pgmap v447: 129 pgs: 129 active+clean; 308 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.3 MiB/s wr, 541 op/s 2024-03-09T00:08:43.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 12301 16.48 MB/sec warmup 50 sec latency 204.163 ms 2024-03-09T00:08:44.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 12782 16.64 MB/sec warmup 51 sec latency 179.241 ms 2024-03-09T00:08:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:44 smithi022 ceph-mon[31894]: pgmap v448: 129 pgs: 129 active+clean; 308 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 443 op/s 2024-03-09T00:08:44.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:44 smithi062 ceph-mon[36129]: pgmap v448: 129 pgs: 129 active+clean; 308 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 443 op/s 2024-03-09T00:08:44.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:44 smithi192 ceph-mon[35995]: pgmap v448: 129 pgs: 129 active+clean; 308 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 443 op/s 2024-03-09T00:08:45.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 13301 16.99 MB/sec warmup 52 sec latency 154.209 ms 2024-03-09T00:08:45.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:45 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:08:45.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:45 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:08:45.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:45 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:08:45.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:45 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:08:45.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:45 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:08:45.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:45 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:08:45.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:45 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:08:45.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:45 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:08:45.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:45 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/3059574289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 msgr2=0x7f6a480ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/3059574289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a480ffe00 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00aa40 tx=0x7f6a3c02cf50 comp rx=0 tx=0).stop 2024-03-09T00:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/3059574289 shutdown_connections 2024-03-09T00:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/3059574289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a48100340 0x7f6a4810c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/3059574289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a480ffe00 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/3059574289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a48105940 0x7f6a48105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/3059574289 >> 172.21.15.22:0/3059574289 conn(0x7f6a48076710 msgr2=0x7f6a480fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/3059574289 shutdown_connections 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.851+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/3059574289 wait complete. 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 Processor -- start 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 -- start start 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a4819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a48100340 0x7f6a4819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a48105940 0x7f6a48197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4810f430 con 0x7f6a480ff980 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4810f2b0 con 0x7f6a48100340 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a4d2c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4810f5b0 con 0x7f6a48105940 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a46ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a4819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a46ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a4819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51714/0 (socket says 172.21.15.22:51714) 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a46ffd640 1 -- 172.21.15.22:0/1854204993 learned_addr learned my addr 172.21.15.22:0/1854204993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a467fc640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a48100340 0x7f6a4819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.852+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a48105940 0x7f6a48197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a46ffd640 1 -- 172.21.15.22:0/1854204993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a48105940 msgr2=0x7f6a48197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a46ffd640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a48105940 0x7f6a48197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a46ffd640 1 -- 172.21.15.22:0/1854204993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a48100340 msgr2=0x7f6a4819dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a46ffd640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a48100340 0x7f6a4819dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a46ffd640 1 -- 172.21.15.22:0/1854204993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48198180 con 0x7f6a480ff980 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a467fc640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a48100340 0x7f6a4819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a46ffd640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a4819d760 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800e9f0 tx=0x7f6a3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a2ffff640 1 -- 172.21.15.22:0/1854204993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800cd80 con 0x7f6a480ff980 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a2ffff640 1 -- 172.21.15.22:0/1854204993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a38010910 con 0x7f6a480ff980 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a48198470 con 0x7f6a480ff980 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a2ffff640 1 -- 172.21.15.22:0/1854204993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38018990 con 0x7f6a480ff980 2024-03-09T00:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.853+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a481a4620 con 0x7f6a480ff980 2024-03-09T00:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 Processor -- start 2024-03-09T00:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 -- start start 2024-03-09T00:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a2ffff640 1 -- 172.21.15.22:0/1854204993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6a38010ac0 con 0x7f6a480ff980 2024-03-09T00:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfcea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfcf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1df335f0 con 0x55bd1dfcfd40 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1dedc930 con 0x55bd1dfcefd0 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1df2f690 con 0x55bd1dfdc960 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfcea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a46ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfcf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfcea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51730/0 (socket says 172.21.15.22:51730) 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 -- 172.21.15.22:0/797179927 learned_addr learned my addr 172.21.15.22:0/797179927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a467fc640 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 -- 172.21.15.22:0/797179927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 msgr2=0x55bd1dfded50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfded50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 -- 172.21.15.22:0/797179927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 msgr2=0x55bd1dfcf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfcf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.856+0000 7f6a477fe640 1 -- 172.21.15.22:0/797179927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd1dc48210 con 0x55bd1dfcfd40 2024-03-09T00:08:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.857+0000 7f6a2ffff640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a300a2a10 0x7f6a300a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.857+0000 7f6a46ffd640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a300a2a10 0x7f6a300a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.857+0000 7f6a477fe640 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfcea90 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009fd0 tx=0x7f6a400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12e7df421e79b2d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.857+0000 7f6a2ffff640 1 -- 172.21.15.22:0/1854204993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6a380cf530 con 0x7f6a480ff980 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.857+0000 7f6a2d7fa640 1 -- 172.21.15.22:0/797179927 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40004d20 con 0x55bd1dfcfd40 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.857+0000 7f6a2d7fa640 1 -- 172.21.15.22:0/797179927 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a40004050 con 0x55bd1dfcfd40 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.857+0000 7f6a2d7fa640 1 -- 172.21.15.22:0/797179927 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40005260 con 0x55bd1dfcfd40 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 -- 172.21.15.22:0/797179927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 msgr2=0x55bd1dfcea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a46ffd640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a300a2a10 0x7f6a300a4ed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a380cc930 tx=0x7f6a380d50e0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfcea90 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009fd0 tx=0x7f6a400118d0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 -- 172.21.15.22:0/797179927 shutdown_connections 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfded50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfcf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/797179927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfcea90 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 -- 172.21.15.22:0/797179927 >> 172.21.15.22:0/797179927 conn(0x55bd1df965b0 msgr2=0x55bd1dfc9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 -- 172.21.15.22:0/797179927 shutdown_connections 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 -- 172.21.15.22:0/797179927 wait complete. 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 -- start start 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.858+0000 7f6a4ff86740 1 Processor -- start 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfdc960 0x55bd1dfda480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1dfdf690 con 0x55bd1dfcefd0 2024-03-09T00:08:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1deefd90 con 0x55bd1dfdc960 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1dfdf810 con 0x55bd1dfcfd40 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a4ff86740 10 client.0 _ll_register_callbacks cb 0x55bd1dc47a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfd9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51738/0 (socket says 172.21.15.22:51738) 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a477fe640 1 -- 172.21.15.22:0/1788086682 learned_addr learned my addr 172.21.15.22:0/1788086682 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a467fc640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfdc960 0x55bd1dfda480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a467fc640 1 -- 172.21.15.22:0/1788086682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfdc960 msgr2=0x55bd1dfda480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a467fc640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfdc960 0x55bd1dfda480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a467fc640 1 -- 172.21.15.22:0/1788086682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcefd0 msgr2=0x55bd1dfd9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a467fc640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfd9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a467fc640 1 -- 172.21.15.22:0/1788086682 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd1dd14620 con 0x55bd1dfcfd40 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a467fc640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd9f40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00a340 tx=0x7f6a3c035c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f7e371750e8e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.859+0000 7f6a2aff5640 1 -- 172.21.15.22:0/1788086682 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c004240 con 0x55bd1dfcfd40 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a40004050 con 0x55bd1dfcfd40 2024-03-09T00:08:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a2aff5640 1 -- 172.21.15.22:0/1788086682 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6a3c0043e0 con 0x55bd1dfcfd40 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a2aff5640 1 -- 172.21.15.22:0/1788086682 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a3c050050 con 0x55bd1dfcfd40 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 10 client.29476 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bd1dfeba20 con 0x55bd1dfcfd40 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a2aff5640 1 -- 172.21.15.22:0/1788086682 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a40004050 con 0x55bd1dfcfd40 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a2aff5640 1 -- 172.21.15.22:0/1788086682 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6a3c00b040 con 0x55bd1dfcfd40 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 10 client.29476 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 10 client.29476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 20 client.29476 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1dfece90 0x55bd1dfef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 4 client.29476 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd1df40b30 con 0x55bd1dfece90 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.860+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1dfece90 0x55bd1dfef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.861+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1dfece90 0x55bd1dfef280 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8445e3fa00076a5 server_cookie=d171a82233e76727 in_seq=0 out_seq=0 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.861+0000 7f6a2aff5640 10 client.29476 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a2aff5640 1 -- 172.21.15.22:0/1788086682 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd1df40b30 con 0x55bd1dfece90 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a2aff5640 10 client.29476 handle_command_reply: tid=0 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 client.29476 shutdown 2024-03-09T00:08:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcfd40 msgr2=0x55bd1dfd9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd9f40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00a340 tx=0x7f6a3c035c20 comp rx=0 tx=0).stop 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 shutdown_connections 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1dfece90 0x55bd1dfef280 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfdc960 0x55bd1dfda480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd9f40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a2aff5640 0 client.29476 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1788086682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 >> 172.21.15.22:0/1788086682 conn(0x55bd1df965b0 msgr2=0x55bd1dfcba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 shutdown_connections 2024-03-09T00:08:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1788086682 wait complete. 2024-03-09T00:08:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.863+0000 7f6a4ff86740 20 client.29476 trim_cache size 0 max 16384 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 Processor -- start 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 -- start start 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfdc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfe3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfdc960 0x55bd1dfe3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481ae9b0 con 0x55bd1dfdc960 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481ac4f0 con 0x55bd1dfcfd40 2024-03-09T00:08:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481aa5b0 con 0x55bd1dfcefd0 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfdc960 0x55bd1dfe3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfdc960 0x55bd1dfe3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51748/0 (socket says 172.21.15.22:51748) 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a477fe640 1 -- 172.21.15.22:0/1625854403 learned_addr learned my addr 172.21.15.22:0/1625854403 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a467fc640 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfe3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.877+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfdc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a477fe640 1 -- 172.21.15.22:0/1625854403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcefd0 msgr2=0x55bd1dfdc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfdc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a477fe640 1 -- 172.21.15.22:0/1625854403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcfd40 msgr2=0x55bd1dfe3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfe3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a477fe640 1 -- 172.21.15.22:0/1625854403 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd1dff4420 con 0x55bd1dfdc960 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a477fe640 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfdc960 0x55bd1dfe3fc0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009f70 tx=0x7f6a40009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30932f6b5bd06cb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a2cff9640 1 -- 172.21.15.22:0/1625854403 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40004050 con 0x55bd1dfdc960 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a2cff9640 1 -- 172.21.15.22:0/1625854403 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a40005520 con 0x55bd1dfdc960 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.878+0000 7f6a2cff9640 1 -- 172.21.15.22:0/1625854403 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000f900 con 0x55bd1dfdc960 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1625854403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfdc960 msgr2=0x55bd1dfe3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfdc960 0x55bd1dfe3fc0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009f70 tx=0x7f6a40009be0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1625854403 shutdown_connections 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfdc960 0x55bd1dfe3fc0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfe3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/1625854403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfdc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1625854403 >> 172.21.15.22:0/1625854403 conn(0x55bd1df965b0 msgr2=0x55bd1dfc97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1625854403 shutdown_connections 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 -- 172.21.15.22:0/1625854403 wait complete. 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 -- start start 2024-03-09T00:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 Processor -- start 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfe3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfd6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.879+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481aabb0 con 0x55bd1dfcfd40 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481abe90 con 0x55bd1dfcefd0 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a4ff86740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481ab1d0 con 0x55bd1dfdc960 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a4ff86740 10 client.0 _ll_register_callbacks cb 0x7f6a40009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfe3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51752/0 (socket says 172.21.15.22:51752) 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 -- 172.21.15.22:0/2410378478 learned_addr learned my addr 172.21.15.22:0/2410378478 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a477fe640 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfd6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 -- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 msgr2=0x55bd1dfd6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfd6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 -- 172.21.15.22:0/2410378478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 msgr2=0x55bd1dfe3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfe3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 -- 172.21.15.22:0/2410378478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd1dfd6cd0 con 0x55bd1dfcfd40 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a467fc640 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd6250 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c000c00 tx=0x7f6a3c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7c6241563e4922 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a2aff5640 1 -- 172.21.15.22:0/2410378478 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c059060 con 0x55bd1dfcfd40 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a2aff5640 1 -- 172.21.15.22:0/2410378478 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7f6a3c005060 con 0x55bd1dfcfd40 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a40005520 con 0x55bd1dfcfd40 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a2aff5640 1 -- 172.21.15.22:0/2410378478 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a3c060050 con 0x55bd1dfcfd40 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a4ff86740 10 client.20055 fetch_fsmap learned FSMap version 36 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.880+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bd1e044030 con 0x55bd1dfcfd40 2024-03-09T00:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a2aff5640 1 -- 172.21.15.22:0/2410378478 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a40005520 con 0x55bd1dfcfd40 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a2aff5640 1 -- 172.21.15.22:0/2410378478 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6a3c047400 con 0x55bd1dfcfd40 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a4ff86740 10 client.20055 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a4ff86740 10 client.20055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a4ff86740 20 client.20055 populate_metadata read hostname 'smithi022' 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1e0446d0 0x55bd1e046af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a4ff86740 4 client.20055 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bd1df3ae80 con 0x55bd1e0446d0 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a477fe640 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1e0446d0 0x55bd1e046af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a2aff5640 10 client.20055 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:45.881+0000 7f6a477fe640 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1e0446d0 0x55bd1e046af0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943702197dfc0b72 server_cookie=1874867eb2823873 in_seq=0 out_seq=0 2024-03-09T00:08:46.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 13687 17.39 MB/sec warmup 53 sec latency 190.554 ms 2024-03-09T00:08:46.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:46 smithi022 ceph-mon[31894]: pgmap v449: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 557 op/s 2024-03-09T00:08:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:46 smithi062 ceph-mon[36129]: pgmap v449: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 557 op/s 2024-03-09T00:08:46.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:46 smithi192 ceph-mon[35995]: pgmap v449: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 557 op/s 2024-03-09T00:08:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a2aff5640 1 -- 172.21.15.22:0/2410378478 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bd1df3ae80 con 0x55bd1e0446d0 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a2aff5640 10 client.20055 handle_command_reply: tid=0 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 client.20055 shutdown 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 msgr2=0x55bd1dfd6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd6250 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c000c00 tx=0x7f6a3c005e80 comp rx=0 tx=0).stop 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 shutdown_connections 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bd1e0446d0 0x55bd1e046af0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd1dfdc960 0x55bd1dfd6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bd1dfcfd40 0x55bd1dfd6250 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a2aff5640 0 client.20055 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:08:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 --2- 172.21.15.22:0/2410378478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd1dfcefd0 0x55bd1dfe3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 >> 172.21.15.22:0/2410378478 conn(0x55bd1df965b0 msgr2=0x55bd1dfd8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 shutdown_connections 2024-03-09T00:08:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.973+0000 7f6a4ff86740 1 -- 172.21.15.22:0/2410378478 wait complete. 2024-03-09T00:08:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4ff86740 20 client.20055 trim_cache size 0 max 16384 2024-03-09T00:08:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:08:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:08:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:08:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:08:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a300a2a10 msgr2=0x7f6a300a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a300a2a10 0x7f6a300a4ed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a380cc930 tx=0x7f6a380d50e0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 msgr2=0x7f6a4819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:08:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a4819d760 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800e9f0 tx=0x7f6a3800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 shutdown_connections 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a300a2a10 0x7f6a300a4ed0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a48105940 0x7f6a48197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a48100340 0x7f6a4819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 --2- 172.21.15.22:0/1854204993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a480ff980 0x7f6a4819d760 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.974+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 >> 172.21.15.22:0/1854204993 conn(0x7f6a48076710 msgr2=0x7f6a48102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.975+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 shutdown_connections 2024-03-09T00:08:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:08:46.975+0000 7f6a4d2c6640 1 -- 172.21.15.22:0/1854204993 wait complete. 2024-03-09T00:08:47.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 13778 17.18 MB/sec warmup 54 sec latency 798.662 ms 2024-03-09T00:08:48.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14003 17.20 MB/sec warmup 55 sec latency 164.468 ms 2024-03-09T00:08:48.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:48 smithi022 ceph-mon[31894]: pgmap v450: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-03-09T00:08:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:48 smithi062 ceph-mon[36129]: pgmap v450: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-03-09T00:08:48.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:48 smithi192 ceph-mon[35995]: pgmap v450: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-03-09T00:08:49.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14168 16.99 MB/sec warmup 56 sec latency 222.841 ms 2024-03-09T00:08:50.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14294 16.73 MB/sec warmup 57 sec latency 267.244 ms 2024-03-09T00:08:50.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:50 smithi022 ceph-mon[31894]: pgmap v451: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-03-09T00:08:50.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:50 smithi062 ceph-mon[36129]: pgmap v451: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-03-09T00:08:50.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:50 smithi192 ceph-mon[35995]: pgmap v451: 129 pgs: 129 active+clean; 319 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-03-09T00:08:51.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14451 16.48 MB/sec warmup 58 sec latency 282.144 ms 2024-03-09T00:08:52.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14465 16.21 MB/sec warmup 59 sec latency 707.980 ms 2024-03-09T00:08:52.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:52 smithi022 ceph-mon[31894]: pgmap v452: 129 pgs: 129 active+clean; 309 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.0 MiB/s wr, 629 op/s 2024-03-09T00:08:52.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:52 smithi062 ceph-mon[36129]: pgmap v452: 129 pgs: 129 active+clean; 309 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.0 MiB/s wr, 629 op/s 2024-03-09T00:08:52.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:52 smithi192 ceph-mon[35995]: pgmap v452: 129 pgs: 129 active+clean; 309 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.0 MiB/s wr, 629 op/s 2024-03-09T00:08:53.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14496 15.96 MB/sec warmup 60 sec latency 1089.435 ms 2024-03-09T00:08:54.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14669 15.78 MB/sec warmup 61 sec latency 340.494 ms 2024-03-09T00:08:54.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:54 smithi022 ceph-mon[31894]: pgmap v453: 129 pgs: 129 active+clean; 309 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 471 op/s 2024-03-09T00:08:54.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:54 smithi062 ceph-mon[36129]: pgmap v453: 129 pgs: 129 active+clean; 309 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 471 op/s 2024-03-09T00:08:54.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:54 smithi192 ceph-mon[35995]: pgmap v453: 129 pgs: 129 active+clean; 309 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 471 op/s 2024-03-09T00:08:55.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 14861 15.71 MB/sec warmup 62 sec latency 253.083 ms 2024-03-09T00:08:56.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 15133 15.68 MB/sec warmup 63 sec latency 176.836 ms 2024-03-09T00:08:56.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:56 smithi022 ceph-mon[31894]: pgmap v454: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.8 MiB/s wr, 751 op/s 2024-03-09T00:08:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:56 smithi062 ceph-mon[36129]: pgmap v454: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.8 MiB/s wr, 751 op/s 2024-03-09T00:08:56.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:56 smithi192 ceph-mon[35995]: pgmap v454: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.8 MiB/s wr, 751 op/s 2024-03-09T00:08:57.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 15675 15.73 MB/sec warmup 64 sec latency 190.953 ms 2024-03-09T00:08:57.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:57 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:08:57.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:57 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:08:57.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:57 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:08:58.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 16127 15.82 MB/sec warmup 65 sec latency 215.654 ms 2024-03-09T00:08:58.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:58 smithi022 ceph-mon[31894]: pgmap v455: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 637 op/s 2024-03-09T00:08:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:58 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:08:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:58 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:08:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:08:58 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:08:58.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:58 smithi192 ceph-mon[35995]: pgmap v455: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 637 op/s 2024-03-09T00:08:58.531 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:58 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:08:58.531 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:58 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:08:58.531 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:08:58 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:08:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:58 smithi062 ceph-mon[36129]: pgmap v455: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 637 op/s 2024-03-09T00:08:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:58 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:08:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:58 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:08:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:08:58 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:08:59.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 16638 16.19 MB/sec warmup 66 sec latency 50.518 ms 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.068+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/438167655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100440 msgr2=0x7fd1b01008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.068+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/438167655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100440 0x7fd1b01008c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009900 tx=0x7fd1a402bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/438167655 shutdown_connections 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/438167655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1b0100e00 0x7fd1b010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/438167655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0100440 0x7fd1b01008c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/438167655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b0105190 0x7fd1b0105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/438167655 >> 172.21.15.22:0/438167655 conn(0x7fd1b0077040 msgr2=0x7fd1b0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/438167655 shutdown_connections 2024-03-09T00:09:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/438167655 wait complete. 2024-03-09T00:09:00.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17011 16.39 MB/sec warmup 67 sec latency 201.842 ms 2024-03-09T00:09:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 Processor -- start 2024-03-09T00:09:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- start start 2024-03-09T00:09:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b0100440 0x7fd1b019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1b0100e00 0x7fd1b019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0105190 0x7fd1b0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01117d0 con 0x7fd1b0105190 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0111650 con 0x7fd1b0100440 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b6f7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0111950 con 0x7fd1b0100e00 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b551b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0105190 0x7fd1b0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1affff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1b0100e00 0x7fd1b019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b551b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0105190 0x7fd1b0197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56754/0 (socket says 172.21.15.22:56754) 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.069+0000 7fd1b551b640 1 -- 172.21.15.22:0/2955177674 learned_addr learned my addr 172.21.15.22:0/2955177674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b0100440 0x7fd1b019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b551b640 1 -- 172.21.15.22:0/2955177674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1b0100e00 msgr2=0x7fd1b019dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b551b640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1b0100e00 0x7fd1b019dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b551b640 1 -- 172.21.15.22:0/2955177674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b0100440 msgr2=0x7fd1b019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b551b640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b0100440 0x7fd1b019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b551b640 1 -- 172.21.15.22:0/2955177674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b01980f0 con 0x7fd1b0105190 2024-03-09T00:09:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b551b640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0105190 0x7fd1b0197970 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000e970 tx=0x7fd1a000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1adffb640 1 -- 172.21.15.22:0/2955177674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000ccc0 con 0x7fd1b0105190 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1adffb640 1 -- 172.21.15.22:0/2955177674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1a000ce60 con 0x7fd1b0105190 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b01983e0 con 0x7fd1b0105190 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1adffb640 1 -- 172.21.15.22:0/2955177674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0010430 con 0x7fd1b0105190 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.070+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b01a4680 con 0x7fd1b0105190 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.072+0000 7fd1adffb640 1 -- 172.21.15.22:0/2955177674 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd1a0005000 con 0x7fd1b0105190 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 Processor -- start 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 -- start start 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1adffb640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1980a2a10 0x7fd1980a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9911da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9911e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1affff640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1980a2a10 0x7fd1980a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed99129940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1adffb640 1 -- 172.21.15.22:0/2955177674 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd1a00cf6e0 con 0x7fd1b0105190 2024-03-09T00:09:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed990825f0 con 0x55ed9911ed40 2024-03-09T00:09:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed9902b930 con 0x55ed99127550 2024-03-09T00:09:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed9907e690 con 0x55ed9911dfd0 2024-03-09T00:09:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9911da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9911da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56768/0 (socket says 172.21.15.22:56768) 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/4211588385 learned_addr learned my addr 172.21.15.22:0/4211588385 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1affff640 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9911e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b551b640 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed99129940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/4211588385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 msgr2=0x55ed9911e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9911e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/4211588385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 msgr2=0x55ed99129940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed99129940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/4211588385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed98d97210 con 0x55ed9911ed40 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1affff640 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9911e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.073+0000 7fd1affff640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1980a2a10 0x7fd1980a4ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009230 tx=0x7fd1a40041c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9911da90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80103d0 tx=0x7fd1a8010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42d5792a8edfd71 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd196ffd640 1 -- 172.21.15.22:0/4211588385 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8011700 con 0x55ed9911ed40 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd196ffd640 1 -- 172.21.15.22:0/4211588385 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1a8011d80 con 0x55ed9911ed40 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd196ffd640 1 -- 172.21.15.22:0/4211588385 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a801a380 con 0x55ed9911ed40 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/4211588385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 msgr2=0x55ed9911da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9911da90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80103d0 tx=0x7fd1a8010890 comp rx=0 tx=0).stop 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/4211588385 shutdown_connections 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed99129940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9911e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/4211588385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9911da90 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/4211588385 >> 172.21.15.22:0/4211588385 conn(0x55ed990e55b0 msgr2=0x55ed99118cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.074+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/4211588385 shutdown_connections 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/4211588385 wait complete. 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 -- start start 2024-03-09T00:09:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 Processor -- start 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911dfd0 0x55ed991245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911ed40 0x55ed99124b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed99127550 0x55ed99125050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed9912a280 con 0x55ed99127550 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed9903ed90 con 0x55ed9911dfd0 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed9912a400 con 0x55ed9911ed40 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b9c3a740 10 client.0 _ll_register_callbacks cb 0x55ed98d96a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b551b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed99127550 0x55ed99125050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1affff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911ed40 0x55ed99124b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.075+0000 7fd1b4d1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911dfd0 0x55ed991245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911ed40 0x55ed99124b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:55796/0 (socket says 172.21.15.22:55796) 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1b4d1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911dfd0 0x55ed991245d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:47298/0 (socket says 172.21.15.22:47298) 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 -- 172.21.15.22:0/3245061458 learned_addr learned my addr 172.21.15.22:0/3245061458 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 -- 172.21.15.22:0/3245061458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911dfd0 msgr2=0x55ed991245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911dfd0 0x55ed991245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 -- 172.21.15.22:0/3245061458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed99127550 msgr2=0x55ed99125050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed99127550 0x55ed99125050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 -- 172.21.15.22:0/3245061458 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed98e63620 con 0x55ed9911ed40 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1affff640 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911ed40 0x55ed99124b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4035000 tx=0x7fd1a402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=974af7ce187c7247 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd18ffff640 1 -- 172.21.15.22:0/3245061458 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4052020 con 0x55ed9911ed40 2024-03-09T00:09:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1a8011d80 con 0x55ed9911ed40 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd18ffff640 1 -- 172.21.15.22:0/3245061458 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd1a40050c0 con 0x55ed9911ed40 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd18ffff640 1 -- 172.21.15.22:0/3245061458 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1a405d050 con 0x55ed9911ed40 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1b9c3a740 10 client.29494 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.076+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ed99136600 con 0x55ed9911ed40 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd18ffff640 1 -- 172.21.15.22:0/3245061458 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1a8011d80 con 0x55ed9911ed40 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd18ffff640 1 -- 172.21.15.22:0/3245061458 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd1a404d070 con 0x55ed9911ed40 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b9c3a740 10 client.29494 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b9c3a740 10 client.29494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b9c3a740 20 client.29494 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed99137b10 0x55ed99139f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b9c3a740 4 client.29494 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed9908fb30 con 0x55ed99137b10 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed99137b10 0x55ed99139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd18ffff640 10 client.29494 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.077+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed99137b10 0x55ed99139f30 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877c4c76c5111445 server_cookie=4cc2a88f4c873bab in_seq=0 out_seq=0 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd18ffff640 1 -- 172.21.15.22:0/3245061458 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed9908fb30 con 0x55ed99137b10 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd18ffff640 10 client.29494 handle_command_reply: tid=0 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 client.29494 shutdown 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911ed40 msgr2=0x55ed99124b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911ed40 0x55ed99124b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4035000 tx=0x7fd1a402bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 shutdown_connections 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed99137b10 0x55ed99139f30 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed99127550 0x55ed99125050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911ed40 0x55ed99124b10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd18ffff640 0 client.29494 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3245061458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911dfd0 0x55ed991245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 >> 172.21.15.22:0/3245061458 conn(0x55ed990e55b0 msgr2=0x55ed9911aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 shutdown_connections 2024-03-09T00:09:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.078+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3245061458 wait complete. 2024-03-09T00:09:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.079+0000 7fd1b9c3a740 20 client.29494 trim_cache size 0 max 16384 2024-03-09T00:09:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 Processor -- start 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 -- start start 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911dfd0 0x55ed99122ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911ed40 0x55ed991233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed99127550 0x55ed99121830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01b0b70 con 0x55ed9911dfd0 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01ae6b0 con 0x55ed9911ed40 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01ac770 con 0x55ed99127550 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b4d1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911dfd0 0x55ed99122ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b4d1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911dfd0 0x55ed99122ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56784/0 (socket says 172.21.15.22:56784) 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/3060355616 learned_addr learned my addr 172.21.15.22:0/3060355616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1affff640 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911ed40 0x55ed991233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.095+0000 7fd1b551b640 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed99127550 0x55ed99121830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/3060355616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed99127550 msgr2=0x55ed99121830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed99127550 0x55ed99121830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/3060355616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911ed40 msgr2=0x55ed991233e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911ed40 0x55ed991233e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1b4d1a640 1 -- 172.21.15.22:0/3060355616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed9913f120 con 0x55ed9911dfd0 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1affff640 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911ed40 0x55ed991233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911dfd0 0x55ed99122ea0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009f40 tx=0x7fd1a801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d201d5eba52351e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1967fc640 1 -- 172.21.15.22:0/3060355616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a802d070 con 0x55ed9911dfd0 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1967fc640 1 -- 172.21.15.22:0/3060355616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1a8003970 con 0x55ed9911dfd0 2024-03-09T00:09:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.096+0000 7fd1967fc640 1 -- 172.21.15.22:0/3060355616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8005340 con 0x55ed9911dfd0 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3060355616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911dfd0 msgr2=0x55ed99122ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911dfd0 0x55ed99122ea0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009f40 tx=0x7fd1a801cba0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3060355616 shutdown_connections 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed99127550 0x55ed99121830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9911ed40 0x55ed991233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/3060355616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911dfd0 0x55ed99122ea0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3060355616 >> 172.21.15.22:0/3060355616 conn(0x55ed990e55b0 msgr2=0x55ed9911f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3060355616 shutdown_connections 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/3060355616 wait complete. 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.097+0000 7fd1b9c3a740 1 -- start start 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 1 Processor -- start 2024-03-09T00:09:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9912ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9912ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed9912d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01acd70 con 0x55ed9911ed40 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01ae050 con 0x55ed99127550 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01ad390 con 0x55ed9911dfd0 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9912ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b9c3a740 10 client.0 _ll_register_callbacks cb 0x7fd1a8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9912ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56788/0 (socket says 172.21.15.22:56788) 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1affff640 1 -- 172.21.15.22:0/1986378462 learned_addr learned my addr 172.21.15.22:0/1986378462 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.098+0000 7fd1b551b640 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed9912d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b551b640 1 -- 172.21.15.22:0/1986378462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 msgr2=0x55ed9912ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b551b640 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9912ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b551b640 1 -- 172.21.15.22:0/1986378462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 msgr2=0x55ed9912ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b551b640 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9912ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b551b640 1 -- 172.21.15.22:0/1986378462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed9912dc00 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1affff640 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9912ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b551b640 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed9912d3d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00cc9d0 tx=0x7fd1a009c0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a161fe564b7c3996 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd18ffff640 1 -- 172.21.15.22:0/1986378462 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00a4080 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd18ffff640 1 -- 172.21.15.22:0/1986378462 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9539+0+0 (secure 0 0 0) 0x7fd1a00052f0 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1a8003970 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd18ffff640 1 -- 172.21.15.22:0/1986378462 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1a009bb40 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b9c3a740 10 client.29327 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ed99189190 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.099+0000 7fd18ffff640 1 -- 172.21.15.22:0/1986378462 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1a8003970 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.100+0000 7fd18ffff640 1 -- 172.21.15.22:0/1986378462 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd1a009ad30 con 0x55ed99127550 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.100+0000 7fd1b9c3a740 10 client.29327 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.100+0000 7fd1b9c3a740 10 client.29327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.100+0000 7fd1b9c3a740 20 client.29327 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.100+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed991898c0 0x55ed99196430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.100+0000 7fd1b9c3a740 4 client.29327 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:09:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.100+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed99089e80 con 0x55ed991898c0 2024-03-09T00:09:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.101+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed991898c0 0x55ed99196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.101+0000 7fd18ffff640 10 client.29327 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.101+0000 7fd1b4d1a640 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed991898c0 0x55ed99196430 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa75893c8b5c617 server_cookie=b3e97345777aa853 in_seq=0 out_seq=0 2024-03-09T00:09:00.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:00 smithi022 ceph-mon[31894]: pgmap v456: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 637 op/s 2024-03-09T00:09:00.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:00 smithi062 ceph-mon[36129]: pgmap v456: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 637 op/s 2024-03-09T00:09:00.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:00 smithi192 ceph-mon[35995]: pgmap v456: 129 pgs: 129 active+clean; 340 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.7 MiB/s wr, 637 op/s 2024-03-09T00:09:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd18ffff640 1 -- 172.21.15.22:0/1986378462 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed99089e80 con 0x55ed991898c0 2024-03-09T00:09:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd18ffff640 10 client.29327 handle_command_reply: tid=0 2024-03-09T00:09:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 client.29327 shutdown 2024-03-09T00:09:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 msgr2=0x55ed9912d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed9912d3d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00cc9d0 tx=0x7fd1a009c0a0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 shutdown_connections 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ed991898c0 0x55ed99196430 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed99127550 0x55ed9912d3d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed9911ed40 0x55ed9912ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 --2- 172.21.15.22:0/1986378462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed9911dfd0 0x55ed9912ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd18ffff640 0 client.29327 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 >> 172.21.15.22:0/1986378462 conn(0x55ed990e55b0 msgr2=0x55ed99122e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 shutdown_connections 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.801+0000 7fd1b9c3a740 1 -- 172.21.15.22:0/1986378462 wait complete. 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b9c3a740 20 client.29327 trim_cache size 0 max 16384 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:09:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:09:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:09:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:09:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1980a2a10 msgr2=0x7fd1980a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1980a2a10 0x7fd1980a4ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009230 tx=0x7fd1a40041c0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0105190 msgr2=0x7fd1b0197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0105190 0x7fd1b0197970 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000e970 tx=0x7fd1a000ee30 comp rx=0 tx=0).stop 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 shutdown_connections 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd1980a2a10 0x7fd1980a4ed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1b0105190 0x7fd1b0197970 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1b0100e00 0x7fd1b019dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 --2- 172.21.15.22:0/2955177674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b0100440 0x7fd1b019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.802+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 >> 172.21.15.22:0/2955177674 conn(0x7fd1b0077040 msgr2=0x7fd1b00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.803+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 shutdown_connections 2024-03-09T00:09:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:00.803+0000 7fd1b6f7e640 1 -- 172.21.15.22:0/2955177674 wait complete. 2024-03-09T00:09:01.063 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17120 16.32 MB/sec warmup 68 sec latency 606.548 ms 2024-03-09T00:09:02.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17254 16.14 MB/sec warmup 69 sec latency 287.879 ms 2024-03-09T00:09:02.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:02 smithi022 ceph-mon[31894]: pgmap v457: 129 pgs: 129 active+clean; 318 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 825 op/s 2024-03-09T00:09:02.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:02 smithi062 ceph-mon[36129]: pgmap v457: 129 pgs: 129 active+clean; 318 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 825 op/s 2024-03-09T00:09:02.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:02 smithi192 ceph-mon[35995]: pgmap v457: 129 pgs: 129 active+clean; 318 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.7 MiB/s wr, 825 op/s 2024-03-09T00:09:03.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17326 15.95 MB/sec warmup 70 sec latency 368.449 ms 2024-03-09T00:09:04.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17378 15.80 MB/sec warmup 71 sec latency 405.676 ms 2024-03-09T00:09:04.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:04 smithi022 ceph-mon[31894]: pgmap v458: 129 pgs: 129 active+clean; 318 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.0 MiB/s wr, 467 op/s 2024-03-09T00:09:04.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:04 smithi062 ceph-mon[36129]: pgmap v458: 129 pgs: 129 active+clean; 318 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.0 MiB/s wr, 467 op/s 2024-03-09T00:09:04.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:04 smithi192 ceph-mon[35995]: pgmap v458: 129 pgs: 129 active+clean; 318 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.0 MiB/s wr, 467 op/s 2024-03-09T00:09:05.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17409 15.63 MB/sec warmup 72 sec latency 422.808 ms 2024-03-09T00:09:06.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17447 15.43 MB/sec warmup 73 sec latency 850.686 ms 2024-03-09T00:09:06.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:06 smithi022 ceph-mon[31894]: pgmap v459: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 840 op/s 2024-03-09T00:09:06.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:06 smithi022 ceph-mon[31894]: osdmap e115: 12 total, 12 up, 12 in 2024-03-09T00:09:06.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:06 smithi062 ceph-mon[36129]: pgmap v459: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 840 op/s 2024-03-09T00:09:06.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:06 smithi062 ceph-mon[36129]: osdmap e115: 12 total, 12 up, 12 in 2024-03-09T00:09:06.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:06 smithi192 ceph-mon[35995]: pgmap v459: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 840 op/s 2024-03-09T00:09:06.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:06 smithi192 ceph-mon[35995]: osdmap e115: 12 total, 12 up, 12 in 2024-03-09T00:09:07.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17494 15.23 MB/sec warmup 74 sec latency 1225.296 ms 2024-03-09T00:09:07.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:09:07.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:09:08.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17557 15.04 MB/sec warmup 75 sec latency 1059.512 ms 2024-03-09T00:09:08.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.080+0000 7f4725873640 1 -- 172.21.15.22:0/1853193809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47201065c0 msgr2=0x7f472010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.080+0000 7f4725873640 1 --2- 172.21.15.22:0/1853193809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47201065c0 0x7f472010e190 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f4714009900 tx=0x7f471402bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:08.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.082+0000 7f4725873640 1 -- 172.21.15.22:0/1853193809 shutdown_connections 2024-03-09T00:09:08.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.082+0000 7f4725873640 1 --2- 172.21.15.22:0/1853193809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47201065c0 0x7f472010e190 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.082+0000 7f4725873640 1 --2- 172.21.15.22:0/1853193809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4720101ea0 0x7f4720105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.082+0000 7f4725873640 1 --2- 172.21.15.22:0/1853193809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47201014f0 0x7f47201018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.082+0000 7f4725873640 1 -- 172.21.15.22:0/1853193809 >> 172.21.15.22:0/1853193809 conn(0x7f47200786b0 msgr2=0x7f4720078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.084+0000 7f4725873640 1 -- 172.21.15.22:0/1853193809 shutdown_connections 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.085+0000 7f4725873640 1 -- 172.21.15.22:0/1853193809 wait complete. 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.085+0000 7f4725873640 1 Processor -- start 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f4725873640 1 -- start start 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f4725873640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47201014f0 0x7f4720074bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f4725873640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47201065c0 0x7f47200750f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f4725873640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f472006bc00 0x7f472006c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f4725873640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47201116d0 con 0x7f472006bc00 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f4725873640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4720111550 con 0x7f47201014f0 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f4725873640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4720111850 con 0x7f47201065c0 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f471e7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47201065c0 0x7f47200750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f471e7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47201065c0 0x7f47200750f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38584/0 (socket says 172.21.15.22:38584) 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.086+0000 7f471e7fc640 1 -- 172.21.15.22:0/1300631933 learned_addr learned my addr 172.21.15.22:0/1300631933 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.087+0000 7f471f7fe640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f472006bc00 0x7f472006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.088+0000 7f471effd640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47201014f0 0x7f4720074bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.088+0000 7f471e7fc640 1 -- 172.21.15.22:0/1300631933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47201014f0 msgr2=0x7f4720074bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.088+0000 7f471e7fc640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47201014f0 0x7f4720074bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.088+0000 7f471e7fc640 1 -- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f472006bc00 msgr2=0x7f472006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.088+0000 7f471e7fc640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f472006bc00 0x7f472006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.088+0000 7f471e7fc640 1 -- 172.21.15.22:0/1300631933 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472006c920 con 0x7f47201065c0 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.088+0000 7f471effd640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47201014f0 0x7f4720074bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:09:08.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.089+0000 7f471e7fc640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47201065c0 0x7f47200750f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f471000b480 tx=0x7f471000b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.092+0000 7f4724871640 1 -- 172.21.15.22:0/1300631933 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710014070 con 0x7f47201065c0 2024-03-09T00:09:08.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.092+0000 7f4724871640 1 -- 172.21.15.22:0/1300631933 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4710010d50 con 0x7f47201065c0 2024-03-09T00:09:08.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.092+0000 7f4724871640 1 -- 172.21.15.22:0/1300631933 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710017360 con 0x7f47201065c0 2024-03-09T00:09:08.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.094+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4720070650 con 0x7f47201065c0 2024-03-09T00:09:08.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.094+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4720070b50 con 0x7f47201065c0 2024-03-09T00:09:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.100+0000 7f4728452740 1 Processor -- start 2024-03-09T00:09:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.100+0000 7f4724871640 1 -- 172.21.15.22:0/1300631933 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4710017070 con 0x7f47201065c0 2024-03-09T00:09:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4728452740 1 -- start start 2024-03-09T00:09:08.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4728452740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d2760 0x5569ae4d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4728452740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4728452740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4728452740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ae435e00 con 0x5569ae4d3080 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4728452740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ae3deb10 con 0x5569ae4d2760 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4728452740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ae431ea0 con 0x5569ae4dbc60 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.101+0000 7f4724871640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f47080a2b70 0x7f47080a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.102+0000 7f4724871640 1 -- 172.21.15.22:0/1300631933 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f47100ce780 con 0x7f47201065c0 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.102+0000 7f471f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d2760 0x5569ae4d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.102+0000 7f471f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d2760 0x5569ae4d2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36620/0 (socket says 172.21.15.22:36620) 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.102+0000 7f471f7fe640 1 -- 172.21.15.22:0/2250532153 learned_addr learned my addr 172.21.15.22:0/2250532153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.102+0000 7f471effd640 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.102+0000 7f471e7fc640 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.102+0000 7f471f7fe640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f47080a2b70 0x7f47080a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.103+0000 7f471effd640 1 -- 172.21.15.22:0/2250532153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 msgr2=0x5569ae4de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.103+0000 7f471effd640 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4de050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.103+0000 7f471effd640 1 -- 172.21.15.22:0/2250532153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d2760 msgr2=0x5569ae4d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.103+0000 7f471effd640 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d2760 0x5569ae4d2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.103+0000 7f471effd640 1 -- 172.21.15.22:0/2250532153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569ae2147d0 con 0x5569ae4d3080 2024-03-09T00:09:08.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.103+0000 7f471effd640 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db720 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f4718005c40 tx=0x7f471800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c27d2dea46b8068 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.104+0000 7f4705ffb640 1 -- 172.21.15.22:0/2250532153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471800f830 con 0x5569ae4d3080 2024-03-09T00:09:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.104+0000 7f4705ffb640 1 -- 172.21.15.22:0/2250532153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471800de20 con 0x5569ae4d3080 2024-03-09T00:09:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.105+0000 7f4728452740 1 -- 172.21.15.22:0/2250532153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 msgr2=0x5569ae4db720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.105+0000 7f4728452740 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db720 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f4718005c40 tx=0x7f471800d710 comp rx=0 tx=0).stop 2024-03-09T00:09:08.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f4728452740 1 -- 172.21.15.22:0/2250532153 shutdown_connections 2024-03-09T00:09:08.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f4728452740 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f4728452740 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db720 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f4728452740 1 --2- 172.21.15.22:0/2250532153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d2760 0x5569ae4d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f4728452740 1 -- 172.21.15.22:0/2250532153 >> 172.21.15.22:0/2250532153 conn(0x5569ae4cce90 msgr2=0x5569ae4cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f471f7fe640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f47080a2b70 0x7f47080a5030 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f47140096c0 tx=0x7f471402bc30 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f4728452740 1 -- 172.21.15.22:0/2250532153 shutdown_connections 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.106+0000 7f4728452740 1 -- 172.21.15.22:0/2250532153 wait complete. 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.107+0000 7f4728452740 1 -- start start 2024-03-09T00:09:08.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.107+0000 7f4728452740 1 Processor -- start 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f4728452740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d2760 0x5569ae4db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f4728452740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d3080 0x5569ae4d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f4728452740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f4728452740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ae4deaa0 con 0x5569ae4d2760 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f4728452740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ae3f20a0 con 0x5569ae4d3080 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f471f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f471f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4d8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38602/0 (socket says 172.21.15.22:38602) 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f471f7fe640 1 -- 172.21.15.22:0/1867371859 learned_addr learned my addr 172.21.15.22:0/1867371859 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.108+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ae4dec20 con 0x5569ae4dbc60 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.109+0000 7f4728452740 10 client.0 _ll_register_callbacks cb 0x5569ae3f1a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.109+0000 7f471f7fe640 1 -- 172.21.15.22:0/1867371859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d3080 msgr2=0x5569ae4d8630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.109+0000 7f471f7fe640 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d3080 0x5569ae4d8630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.109+0000 7f471f7fe640 1 -- 172.21.15.22:0/1867371859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d2760 msgr2=0x5569ae4db2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:08.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.109+0000 7f471f7fe640 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d2760 0x5569ae4db2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.109+0000 7f471f7fe640 1 -- 172.21.15.22:0/1867371859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569ae14a3f0 con 0x5569ae4dbc60 2024-03-09T00:09:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.111+0000 7f471f7fe640 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4d8b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4714009850 tx=0x7f4714009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe056f82b5a61b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.111+0000 7f46feffd640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:09:08.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.112+0000 7f46feffd640 1 -- 172.21.15.22:0/1867371859 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714052070 con 0x5569ae4dbc60 2024-03-09T00:09:08.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.112+0000 7f46feffd640 1 -- 172.21.15.22:0/1867371859 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f4714004f10 con 0x5569ae4dbc60 2024-03-09T00:09:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.113+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f471800de20 con 0x5569ae4dbc60 2024-03-09T00:09:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.113+0000 7f46feffd640 1 -- 172.21.15.22:0/1867371859 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f471405c050 con 0x5569ae4dbc60 2024-03-09T00:09:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.113+0000 7f4728452740 10 client.29506 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.113+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5569ae10a0f0 con 0x5569ae4dbc60 2024-03-09T00:09:08.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.114+0000 7f46feffd640 1 -- 172.21.15.22:0/1867371859 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471800de20 con 0x5569ae4dbc60 2024-03-09T00:09:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.114+0000 7f46feffd640 1 -- 172.21.15.22:0/1867371859 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f471404d070 con 0x5569ae4dbc60 2024-03-09T00:09:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.115+0000 7f4728452740 10 client.29506 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.115+0000 7f4728452740 10 client.29506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:08.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.115+0000 7f4728452740 20 client.29506 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.115+0000 7f4728452740 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae4eb6f0 0x5569ae4edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.115+0000 7f4728452740 4 client.29506 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.115+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569ae443340 con 0x5569ae4eb6f0 2024-03-09T00:09:08.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.116+0000 7f471effd640 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae4eb6f0 0x5569ae4edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.133+0000 7f471effd640 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae4eb6f0 0x5569ae4edb60 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e91a4265d1b94d3 server_cookie=5bd3dc787510c91f in_seq=0 out_seq=0 2024-03-09T00:09:08.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.133+0000 7f46feffd640 10 client.29506 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:08.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.145+0000 7f46feffd640 1 -- 172.21.15.22:0/1867371859 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569ae443340 con 0x5569ae4eb6f0 2024-03-09T00:09:08.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.145+0000 7f46feffd640 10 client.29506 handle_command_reply: tid=0 2024-03-09T00:09:08.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.145+0000 7f4728452740 1 client.29506 shutdown 2024-03-09T00:09:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 msgr2=0x5569ae4d8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4d8b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4714009850 tx=0x7f4714009260 comp rx=0 tx=0).stop 2024-03-09T00:09:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 shutdown_connections 2024-03-09T00:09:08.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae4eb6f0 0x5569ae4edb60 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4dbc60 0x5569ae4d8b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4d3080 0x5569ae4d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 --2- 172.21.15.22:0/1867371859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d2760 0x5569ae4db2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 >> 172.21.15.22:0/1867371859 conn(0x5569ae4cce90 msgr2=0x5569ae4cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.151+0000 7f46feffd640 0 client.29506 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:08.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.153+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 shutdown_connections 2024-03-09T00:09:08.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.153+0000 7f4728452740 1 -- 172.21.15.22:0/1867371859 wait complete. 2024-03-09T00:09:08.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.154+0000 7f4728452740 20 client.29506 trim_cache size 0 max 16384 2024-03-09T00:09:08.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.186+0000 7f4728452740 1 Processor -- start 2024-03-09T00:09:08.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.187+0000 7f4728452740 1 -- start start 2024-03-09T00:09:08.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.187+0000 7f4728452740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 0x5569ae4dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.187+0000 7f4728452740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.187+0000 7f4728452740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.187+0000 7f4728452740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0125b0 con 0x5569ae4d3080 2024-03-09T00:09:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.187+0000 7f4728452740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0100f0 con 0x5569ae4dbc60 2024-03-09T00:09:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.188+0000 7f471f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.188+0000 7f471f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4d35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36634/0 (socket says 172.21.15.22:36634) 2024-03-09T00:09:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.188+0000 7f471f7fe640 1 -- 172.21.15.22:0/1431131451 learned_addr learned my addr 172.21.15.22:0/1431131451 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.188+0000 7f4728452740 1 -- 172.21.15.22:0/1431131451 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c00e1b0 con 0x5569ae4d2760 2024-03-09T00:09:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.189+0000 7f471e7fc640 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.189+0000 7f471e7fc640 1 -- 172.21.15.22:0/1431131451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 msgr2=0x5569ae4dad60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:08.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.189+0000 7f471e7fc640 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 0x5569ae4dad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.189+0000 7f471e7fc640 1 -- 172.21.15.22:0/1431131451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 msgr2=0x5569ae4d35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:08.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.189+0000 7f471e7fc640 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4d35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.189+0000 7f471e7fc640 1 -- 172.21.15.22:0/1431131451 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569ae4d2e70 con 0x5569ae4d3080 2024-03-09T00:09:08.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.193+0000 7f471f7fe640 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4d35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:08.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.194+0000 7f471e7fc640 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db2a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f47100cbb30 tx=0x7f471009b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79561a1e2242ed06 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.196+0000 7f47057fa640 1 -- 172.21.15.22:0/1431131451 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100b3070 con 0x5569ae4d3080 2024-03-09T00:09:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.196+0000 7f47057fa640 1 -- 172.21.15.22:0/1431131451 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4710098dd0 con 0x5569ae4d3080 2024-03-09T00:09:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.196+0000 7f47057fa640 1 -- 172.21.15.22:0/1431131451 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100cd030 con 0x5569ae4d3080 2024-03-09T00:09:08.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.197+0000 7f4728452740 1 -- 172.21.15.22:0/1431131451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 msgr2=0x5569ae4db2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.197+0000 7f4728452740 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db2a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f47100cbb30 tx=0x7f471009b320 comp rx=0 tx=0).stop 2024-03-09T00:09:08.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.198+0000 7f4728452740 1 -- 172.21.15.22:0/1431131451 shutdown_connections 2024-03-09T00:09:08.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.198+0000 7f4728452740 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4d35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.198+0000 7f4728452740 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae4db2a0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.198+0000 7f4728452740 1 --2- 172.21.15.22:0/1431131451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 0x5569ae4dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.198+0000 7f4728452740 1 -- 172.21.15.22:0/1431131451 >> 172.21.15.22:0/1431131451 conn(0x5569ae4cce90 msgr2=0x5569ae4d85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.203+0000 7f4728452740 1 -- 172.21.15.22:0/1431131451 shutdown_connections 2024-03-09T00:09:08.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.204+0000 7f4728452740 1 -- 172.21.15.22:0/1431131451 wait complete. 2024-03-09T00:09:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.205+0000 7f4728452740 1 -- start start 2024-03-09T00:09:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.205+0000 7f4728452740 1 Processor -- start 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f4728452740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 0x5569ae53c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f4728452740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae53c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f4728452740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f4728452740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c00e7b0 con 0x5569ae4d3080 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f4728452740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c00fa90 con 0x5569ae4dbc60 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f4728452740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c00edd0 con 0x5569ae4d2760 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f4728452740 10 client.0 _ll_register_callbacks cb 0x7f47140045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f471e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae53c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f471e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae53c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60874/0 (socket says 172.21.15.22:60874) 2024-03-09T00:09:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.208+0000 7f471e7fc640 1 -- 172.21.15.22:0/4273861795 learned_addr learned my addr 172.21.15.22:0/4273861795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.209+0000 7f471e7fc640 1 -- 172.21.15.22:0/4273861795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 msgr2=0x5569ae53c000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.209+0000 7f471e7fc640 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 0x5569ae53c000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.209+0000 7f471e7fc640 1 -- 172.21.15.22:0/4273861795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 msgr2=0x5569ae4e0140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.209+0000 7f471e7fc640 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4e0140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.209+0000 7f471e7fc640 1 -- 172.21.15.22:0/4273861795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569ae4e0830 con 0x5569ae4d3080 2024-03-09T00:09:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.209+0000 7f471e7fc640 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae53c540 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f471000fa40 tx=0x7f471009ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23d0cc8c33457b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.209+0000 7f46feffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.210+0000 7f46feffd640 1 -- 172.21.15.22:0/4273861795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100b8060 con 0x5569ae4d3080 2024-03-09T00:09:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.210+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4710098dd0 con 0x5569ae4d3080 2024-03-09T00:09:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.210+0000 7f46feffd640 1 -- 172.21.15.22:0/4273861795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f471009fd50 con 0x5569ae4d3080 2024-03-09T00:09:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.210+0000 7f46feffd640 1 -- 172.21.15.22:0/4273861795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47100cd070 con 0x5569ae4d3080 2024-03-09T00:09:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 10 client.20088 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5569ae5437b0 con 0x5569ae4d3080 2024-03-09T00:09:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f46feffd640 1 -- 172.21.15.22:0/4273861795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4710098dd0 con 0x5569ae4d3080 2024-03-09T00:09:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f46feffd640 1 -- 172.21.15.22:0/4273861795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4710099f20 con 0x5569ae4d3080 2024-03-09T00:09:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 10 client.20088 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 10 client.20088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 20 client.20088 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae543ec0 0x5569ae546300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 4 client.20088 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:09:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569ae43d690 con 0x5569ae543ec0 2024-03-09T00:09:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.211+0000 7f471effd640 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae543ec0 0x5569ae546300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.227+0000 7f471effd640 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae543ec0 0x5569ae546300 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d6ca5297a01512 server_cookie=42903f948e3ca94d in_seq=0 out_seq=0 2024-03-09T00:09:08.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.229+0000 7f46feffd640 10 client.20088 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.238+0000 7f46feffd640 1 -- 172.21.15.22:0/4273861795 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5569ae43d690 con 0x5569ae543ec0 2024-03-09T00:09:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.238+0000 7f46feffd640 10 client.20088 handle_command_reply: tid=0 2024-03-09T00:09:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.238+0000 7f4728452740 1 client.20088 shutdown 2024-03-09T00:09:08.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 msgr2=0x5569ae53c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae53c540 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f471000fa40 tx=0x7f471009ac50 comp rx=0 tx=0).stop 2024-03-09T00:09:08.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 shutdown_connections 2024-03-09T00:09:08.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5569ae543ec0 0x5569ae546300 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569ae4dbc60 0x5569ae4e0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5569ae4d3080 0x5569ae53c540 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 --2- 172.21.15.22:0/4273861795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569ae4d2760 0x5569ae53c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.239+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 >> 172.21.15.22:0/4273861795 conn(0x5569ae4cce90 msgr2=0x5569ae4d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.240+0000 7f46feffd640 0 client.20088 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:08.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.241+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 shutdown_connections 2024-03-09T00:09:08.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.241+0000 7f4728452740 1 -- 172.21.15.22:0/4273861795 wait complete. 2024-03-09T00:09:08.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.241+0000 7f4728452740 20 client.20088 trim_cache size 0 max 16384 2024-03-09T00:09:08.238 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:09:08.238 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:09:08.239 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:09:08.239 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:09:08.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f47080a2b70 msgr2=0x7f47080a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f47080a2b70 0x7f47080a5030 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f47140096c0 tx=0x7f471402bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47201065c0 msgr2=0x7f47200750f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47201065c0 0x7f47200750f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f471000b480 tx=0x7f471000b940 comp rx=0 tx=0).stop 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 shutdown_connections 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f47080a2b70 0x7f47080a5030 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f472006bc00 0x7f472006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47201065c0 0x7f47200750f0 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f471000b480 tx=0x7f471000b940 comp rx=0 tx=0).stop 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 --2- 172.21.15.22:0/1300631933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47201014f0 0x7f4720074bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.246+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 >> 172.21.15.22:0/1300631933 conn(0x7f47200786b0 msgr2=0x7f47200ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.247+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 shutdown_connections 2024-03-09T00:09:08.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.249+0000 7f4725873640 1 -- 172.21.15.22:0/1300631933 wait complete. 2024-03-09T00:09:08.263 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:09:08.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:09:08.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.458+0000 7f9d98e68640 1 -- 172.21.15.22:0/15338870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 msgr2=0x7f9d9411cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.458+0000 7f9d98e68640 1 --2- 172.21.15.22:0/15338870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d9411cae0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009890 tx=0x7f9d8402bb90 comp rx=0 tx=0).stop 2024-03-09T00:09:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.460+0000 7f9d98e68640 1 -- 172.21.15.22:0/15338870 shutdown_connections 2024-03-09T00:09:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.460+0000 7f9d98e68640 1 --2- 172.21.15.22:0/15338870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d9411cae0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.460+0000 7f9d98e68640 1 --2- 172.21.15.22:0/15338870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d9410d3d0 0x7f9d9410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.460+0000 7f9d98e68640 1 --2- 172.21.15.22:0/15338870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d9410f900 0x7f9d9410fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.460+0000 7f9d98e68640 1 -- 172.21.15.22:0/15338870 >> 172.21.15.22:0/15338870 conn(0x7f9d9406c220 msgr2=0x7f9d9406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.463+0000 7f9d98e68640 1 -- 172.21.15.22:0/15338870 shutdown_connections 2024-03-09T00:09:08.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.465+0000 7f9d98e68640 1 -- 172.21.15.22:0/15338870 wait complete. 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.466+0000 7f9d98e68640 1 Processor -- start 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d98e68640 1 -- start start 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d98e68640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d9410d3d0 0x7f9d941a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d98e68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d941a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d98e68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d9410f900 0x7f9d941aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d98e68640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94120330 con 0x7f9d9410df80 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d98e68640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941201b0 con 0x7f9d9410f900 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d98e68640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941204b0 con 0x7f9d9410d3d0 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d941a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d941a6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60896/0 (socket says 172.21.15.22:60896) 2024-03-09T00:09:08.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.467+0000 7f9d92ffd640 1 -- 172.21.15.22:0/2260983333 learned_addr learned my addr 172.21.15.22:0/2260983333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:08.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.468+0000 7f9d92ffd640 1 -- 172.21.15.22:0/2260983333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d9410d3d0 msgr2=0x7f9d941a6750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:09:08.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.468+0000 7f9d92ffd640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d9410d3d0 0x7f9d941a6750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.468+0000 7f9d92ffd640 1 -- 172.21.15.22:0/2260983333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d9410f900 msgr2=0x7f9d941aafc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:09:08.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.468+0000 7f9d92ffd640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d9410f900 0x7f9d941aafc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.468+0000 7f9d92ffd640 1 -- 172.21.15.22:0/2260983333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d941ab6f0 con 0x7f9d9410df80 2024-03-09T00:09:08.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.468+0000 7f9d92ffd640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d941a6c90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00ba50 tx=0x7f9d8c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.469+0000 7f9d90ff9640 1 -- 172.21.15.22:0/2260983333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c019070 con 0x7f9d9410df80 2024-03-09T00:09:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.470+0000 7f9d90ff9640 1 -- 172.21.15.22:0/2260983333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c00cbe0 con 0x7f9d9410df80 2024-03-09T00:09:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.470+0000 7f9d90ff9640 1 -- 172.21.15.22:0/2260983333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0076f0 con 0x7f9d9410df80 2024-03-09T00:09:08.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.472+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d941ab9e0 con 0x7f9d9410df80 2024-03-09T00:09:08.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.472+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d941b3270 con 0x7f9d9410df80 2024-03-09T00:09:08.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.477+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9406d480 con 0x7f9d9410df80 2024-03-09T00:09:08.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.478+0000 7f9d90ff9640 1 -- 172.21.15.22:0/2260983333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9d8c014a40 con 0x7f9d9410df80 2024-03-09T00:09:08.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.479+0000 7f9d90ff9640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:08.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.479+0000 7f9d90ff9640 1 -- 172.21.15.22:0/2260983333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f9d8c0ce640 con 0x7f9d9410df80 2024-03-09T00:09:08.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.481+0000 7f9d937fe640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:08.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.487+0000 7f9d937fe640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005fd0 tx=0x7f9d88009260 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:08.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.488+0000 7f9d90ff9640 1 -- 172.21.15.22:0/2260983333 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9d8c09b000 con 0x7f9d9410df80 2024-03-09T00:09:08.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:08 smithi022 ceph-mon[31894]: pgmap v461: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 672 op/s 2024-03-09T00:09:08.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:08 smithi062 ceph-mon[36129]: pgmap v461: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 672 op/s 2024-03-09T00:09:08.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:08 smithi192 ceph-mon[35995]: pgmap v461: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 672 op/s 2024-03-09T00:09:08.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.930+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d941b3470 con 0x7f9d9410df80 2024-03-09T00:09:08.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.933+0000 7f9d90ff9640 1 -- 172.21.15.22:0/2260983333 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f9d8c09a900 con 0x7f9d9410df80 2024-03-09T00:09:08.928 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:09:08.928 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:07:54.629831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:09:08.928 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-03-09T00:09:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9d7c0a2b80 msgr2=0x7f9d7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005fd0 tx=0x7f9d88009260 comp rx=0 tx=0).stop 2024-03-09T00:09:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 msgr2=0x7f9d941a6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d941a6c90 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00ba50 tx=0x7f9d8c00bf10 comp rx=0 tx=0).stop 2024-03-09T00:09:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 shutdown_connections 2024-03-09T00:09:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d9410f900 0x7f9d941aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d9410df80 0x7f9d941a6c90 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 --2- 172.21.15.22:0/2260983333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d9410d3d0 0x7f9d941a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 >> 172.21.15.22:0/2260983333 conn(0x7f9d9406c220 msgr2=0x7f9d940704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 shutdown_connections 2024-03-09T00:09:08.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:08.937+0000 7f9d98e68640 1 -- 172.21.15.22:0/2260983333 wait complete. 2024-03-09T00:09:08.951 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:09:08.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:09:09.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17655 14.91 MB/sec warmup 76 sec latency 1112.121 ms 2024-03-09T00:09:09.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.146+0000 7ff60f545640 1 -- 172.21.15.22:0/2632216953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff608100d90 msgr2=0x7ff608104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.146+0000 7ff60f545640 1 --2- 172.21.15.22:0/2632216953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff608100d90 0x7ff608104dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff60401a910 tx=0x7ff60403cc90 comp rx=0 tx=0).stop 2024-03-09T00:09:09.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.149+0000 7ff60f545640 1 -- 172.21.15.22:0/2632216953 shutdown_connections 2024-03-09T00:09:09.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.149+0000 7ff60f545640 1 --2- 172.21.15.22:0/2632216953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6081054b0 0x7ff60810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.149+0000 7ff60f545640 1 --2- 172.21.15.22:0/2632216953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff608100d90 0x7ff608104dd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.149+0000 7ff60f545640 1 --2- 172.21.15.22:0/2632216953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6081003e0 0x7ff6081007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.149+0000 7ff60f545640 1 -- 172.21.15.22:0/2632216953 >> 172.21.15.22:0/2632216953 conn(0x7ff6080773a0 msgr2=0x7ff6080777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.151+0000 7ff60f545640 1 -- 172.21.15.22:0/2632216953 shutdown_connections 2024-03-09T00:09:09.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.151+0000 7ff60f545640 1 -- 172.21.15.22:0/2632216953 wait complete. 2024-03-09T00:09:09.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.152+0000 7ff60f545640 1 Processor -- start 2024-03-09T00:09:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60f545640 1 -- start start 2024-03-09T00:09:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60f545640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6081003e0 0x7ff608072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60f545640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6081054b0 0x7ff60806dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60f545640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff60806e130 0x7ff60806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60f545640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60806c660 con 0x7ff6081054b0 2024-03-09T00:09:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60f545640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60806c4e0 con 0x7ff6081003e0 2024-03-09T00:09:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60f545640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60806c7e0 con 0x7ff60806e130 2024-03-09T00:09:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60cae0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6081054b0 0x7ff60806dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60cae0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6081054b0 0x7ff60806dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60914/0 (socket says 172.21.15.22:60914) 2024-03-09T00:09:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.153+0000 7ff60cae0640 1 -- 172.21.15.22:0/3370596661 learned_addr learned my addr 172.21.15.22:0/3370596661 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.154+0000 7ff60dae2640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff60806e130 0x7ff60806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.156+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6081003e0 0x7ff608072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.157+0000 7ff60dae2640 1 -- 172.21.15.22:0/3370596661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6081003e0 msgr2=0x7ff608072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.157+0000 7ff60dae2640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6081003e0 0x7ff608072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.157+0000 7ff60dae2640 1 -- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6081054b0 msgr2=0x7ff60806dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.157+0000 7ff60dae2640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6081054b0 0x7ff60806dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.157+0000 7ff60dae2640 1 -- 172.21.15.22:0/3370596661 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608199920 con 0x7ff60806e130 2024-03-09T00:09:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.158+0000 7ff60dae2640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff60806e130 0x7ff60806e5e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800b3e0 tx=0x7ff5f800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.158+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3370596661 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8015020 con 0x7ff60806e130 2024-03-09T00:09:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.158+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff608199c10 con 0x7ff60806e130 2024-03-09T00:09:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.159+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60819a1b0 con 0x7ff60806e130 2024-03-09T00:09:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.160+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3370596661 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5f8011d10 con 0x7ff60806e130 2024-03-09T00:09:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.161+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3370596661 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800f2e0 con 0x7ff60806e130 2024-03-09T00:09:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.162+0000 7ff6121fe740 1 Processor -- start 2024-03-09T00:09:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.162+0000 7ff6121fe740 1 -- start start 2024-03-09T00:09:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.163+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x563140032580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.163+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563140032ac0 0x56314003b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.163+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56314003b6a0 0x56314003da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.163+0000 7ff6121fe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313fff9360 con 0x5631400321a0 2024-03-09T00:09:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.163+0000 7ff6121fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313ff95ec0 con 0x56314003b6a0 2024-03-09T00:09:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.163+0000 7ff6121fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313ff3ed30 con 0x563140032ac0 2024-03-09T00:09:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.164+0000 7ff60d2e1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563140032ac0 0x56314003b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.164+0000 7ff60d2e1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563140032ac0 0x56314003b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38636/0 (socket says 172.21.15.22:38636) 2024-03-09T00:09:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.164+0000 7ff60d2e1640 1 -- 172.21.15.22:0/1406060360 learned_addr learned my addr 172.21.15.22:0/1406060360 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.165+0000 7ff60cae0640 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56314003b6a0 0x56314003da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.165+0000 7ff60cae0640 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x563140032580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.166+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3370596661 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff5f8018020 con 0x7ff60806e130 2024-03-09T00:09:09.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.166+0000 7ff5fe7fc640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5f40a2aa0 0x7ff5f40a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.167+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3370596661 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff5f80ce8e0 con 0x7ff60806e130 2024-03-09T00:09:09.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.167+0000 7ff60cae0640 1 -- 172.21.15.22:0/1406060360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563140032ac0 msgr2=0x56314003b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.167+0000 7ff60cae0640 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563140032ac0 0x56314003b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.167+0000 7ff60cae0640 1 -- 172.21.15.22:0/1406060360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 msgr2=0x563140032580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.167+0000 7ff60cae0640 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x563140032580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.167+0000 7ff60cae0640 1 -- 172.21.15.22:0/1406060360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56313ff3dbc0 con 0x56314003b6a0 2024-03-09T00:09:09.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.170+0000 7ff60dae2640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5f40a2aa0 0x7ff5f40a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.170+0000 7ff60cae0640 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56314003b6a0 0x56314003da90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff60806f440 tx=0x7ff604005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6ffe83059d4ed3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.170+0000 7ff5f37fe640 1 -- 172.21.15.22:0/1406060360 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604004040 con 0x56314003b6a0 2024-03-09T00:09:09.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.170+0000 7ff5f37fe640 1 -- 172.21.15.22:0/1406060360 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff604057070 con 0x56314003b6a0 2024-03-09T00:09:09.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.170+0000 7ff5f37fe640 1 -- 172.21.15.22:0/1406060360 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604005310 con 0x56314003b6a0 2024-03-09T00:09:09.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.172+0000 7ff60dae2640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5f40a2aa0 0x7ff5f40a4f60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff5f809a8a0 tx=0x7ff5f8004770 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.174+0000 7ff6121fe740 1 -- 172.21.15.22:0/1406060360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56314003b6a0 msgr2=0x56314003da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.174+0000 7ff6121fe740 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56314003b6a0 0x56314003da90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff60806f440 tx=0x7ff604005b50 comp rx=0 tx=0).stop 2024-03-09T00:09:09.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.176+0000 7ff6121fe740 1 -- 172.21.15.22:0/1406060360 shutdown_connections 2024-03-09T00:09:09.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.176+0000 7ff6121fe740 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56314003b6a0 0x56314003da90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.176+0000 7ff6121fe740 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563140032ac0 0x56314003b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.176+0000 7ff6121fe740 1 --2- 172.21.15.22:0/1406060360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x563140032580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.176+0000 7ff6121fe740 1 -- 172.21.15.22:0/1406060360 >> 172.21.15.22:0/1406060360 conn(0x56314002c8d0 msgr2=0x56314002ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.182+0000 7ff6121fe740 1 -- 172.21.15.22:0/1406060360 shutdown_connections 2024-03-09T00:09:09.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.182+0000 7ff6121fe740 1 -- 172.21.15.22:0/1406060360 wait complete. 2024-03-09T00:09:09.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.182+0000 7ff6121fe740 1 -- start start 2024-03-09T00:09:09.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 1 Processor -- start 2024-03-09T00:09:09.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631400321a0 0x563140038650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56314003b6a0 0x5631400390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313ff344a0 con 0x56314003b6a0 2024-03-09T00:09:09.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313ff91f60 con 0x563140032ac0 2024-03-09T00:09:09.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314003e430 con 0x5631400321a0 2024-03-09T00:09:09.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.183+0000 7ff6121fe740 10 client.0 _ll_register_callbacks cb 0x56313ff8fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.186+0000 7ff60d2e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.186+0000 7ff60cae0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631400321a0 0x563140038650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.186+0000 7ff60cae0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631400321a0 0x563140038650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38642/0 (socket says 172.21.15.22:38642) 2024-03-09T00:09:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.186+0000 7ff60cae0640 1 -- 172.21.15.22:0/3790286960 learned_addr learned my addr 172.21.15.22:0/3790286960 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.186+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56314003b6a0 0x5631400390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.189+0000 7ff60d2e1640 1 -- 172.21.15.22:0/3790286960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631400321a0 msgr2=0x563140038650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.189+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631400321a0 0x563140038650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.189+0000 7ff60d2e1640 1 -- 172.21.15.22:0/3790286960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56314003b6a0 msgr2=0x5631400390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.189+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56314003b6a0 0x5631400390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.189+0000 7ff60d2e1640 1 -- 172.21.15.22:0/3790286960 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56313fcaa3f0 con 0x563140032ac0 2024-03-09T00:09:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.196+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff600003f00 tx=0x7ff600004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca45edec412aa37 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.196+0000 7ff5f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:09:09.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.198+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3790286960 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600004d20 con 0x563140032ac0 2024-03-09T00:09:09.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.198+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3790286960 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff6000128e0 con 0x563140032ac0 2024-03-09T00:09:09.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.200+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff604057070 con 0x563140032ac0 2024-03-09T00:09:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.202+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3790286960 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff600012ae0 con 0x563140032ac0 2024-03-09T00:09:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.203+0000 7ff6121fe740 10 client.29348 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.203+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56313fd747d0 con 0x563140032ac0 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.208+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3790286960 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff604057070 con 0x563140032ac0 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.208+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3790286960 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff600024420 con 0x563140032ac0 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.209+0000 7ff6121fe740 10 client.29348 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.209+0000 7ff6121fe740 10 client.29348 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.209+0000 7ff6121fe740 20 client.29348 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.209+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x56314004bcd0 0x56314004e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.209+0000 7ff6121fe740 4 client.29348 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.209+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56313ffa3400 con 0x56314004bcd0 2024-03-09T00:09:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.209+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x56314004bcd0 0x56314004e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.210+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x56314004bcd0 0x56314004e0f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=898f37a98ce42c24 server_cookie=97280f22b155afb6 in_seq=0 out_seq=0 2024-03-09T00:09:09.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.210+0000 7ff5f0ff9640 10 client.29348 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:09.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.211+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3790286960 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56313ffa3400 con 0x56314004bcd0 2024-03-09T00:09:09.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.211+0000 7ff5f0ff9640 10 client.29348 handle_command_reply: tid=0 2024-03-09T00:09:09.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.211+0000 7ff6121fe740 1 client.29348 shutdown 2024-03-09T00:09:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.217+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 msgr2=0x563140038b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.217+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff600003f00 tx=0x7ff600004410 comp rx=0 tx=0).stop 2024-03-09T00:09:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.220+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 shutdown_connections 2024-03-09T00:09:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.220+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x56314004bcd0 0x56314004e0f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.220+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56314003b6a0 0x5631400390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.220+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.220+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3790286960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5631400321a0 0x563140038650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.220+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 >> 172.21.15.22:0/3790286960 conn(0x56314002c8d0 msgr2=0x56314002e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.221+0000 7ff5f0ff9640 0 client.29348 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:09.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.225+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 shutdown_connections 2024-03-09T00:09:09.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.226+0000 7ff6121fe740 1 -- 172.21.15.22:0/3790286960 wait complete. 2024-03-09T00:09:09.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.229+0000 7ff6121fe740 20 client.29348 trim_cache size 0 max 16384 2024-03-09T00:09:09.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.257+0000 7ff6121fe740 1 Processor -- start 2024-03-09T00:09:09.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.258+0000 7ff6121fe740 1 -- start start 2024-03-09T00:09:09.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.258+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x56314003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.258+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.258+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x563140052a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.258+0000 7ff6121fe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec0125d0 con 0x5631400321a0 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.258+0000 7ff6121fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec010110 con 0x563140032ac0 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.259+0000 7ff6121fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec00e1d0 con 0x56314003b6a0 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.259+0000 7ff60d2e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.259+0000 7ff60d2e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36684/0 (socket says 172.21.15.22:36684) 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.259+0000 7ff60d2e1640 1 -- 172.21.15.22:0/3622935015 learned_addr learned my addr 172.21.15.22:0/3622935015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.259+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x563140052a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.259+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x56314003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.262+0000 7ff60d2e1640 1 -- 172.21.15.22:0/3622935015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 msgr2=0x563140052a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.262+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x563140052a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.262+0000 7ff60d2e1640 1 -- 172.21.15.22:0/3622935015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 msgr2=0x56314003ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.262+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x56314003ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.262+0000 7ff60d2e1640 1 -- 172.21.15.22:0/3622935015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56314003da90 con 0x563140032ac0 2024-03-09T00:09:09.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.262+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff600003a80 tx=0x7ff600024990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15bc6f31b59948a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.263+0000 7ff5f2ffd640 1 -- 172.21.15.22:0/3622935015 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000128e0 con 0x563140032ac0 2024-03-09T00:09:09.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.263+0000 7ff5f2ffd640 1 -- 172.21.15.22:0/3622935015 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff600024b10 con 0x563140032ac0 2024-03-09T00:09:09.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.263+0000 7ff5f2ffd640 1 -- 172.21.15.22:0/3622935015 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60001ab40 con 0x563140032ac0 2024-03-09T00:09:09.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.264+0000 7ff6121fe740 1 -- 172.21.15.22:0/3622935015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 msgr2=0x563140038ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.264+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff600003a80 tx=0x7ff600024990 comp rx=0 tx=0).stop 2024-03-09T00:09:09.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 -- 172.21.15.22:0/3622935015 shutdown_connections 2024-03-09T00:09:09.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x563140052a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563140032ac0 0x563140038ef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3622935015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5631400321a0 0x56314003ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 -- 172.21.15.22:0/3622935015 >> 172.21.15.22:0/3622935015 conn(0x56314002c8d0 msgr2=0x56314002ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 -- 172.21.15.22:0/3622935015 shutdown_connections 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 -- 172.21.15.22:0/3622935015 wait complete. 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 -- start start 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.265+0000 7ff6121fe740 1 Processor -- start 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631400321a0 0x5631400404b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563140032ac0 0x5631400409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff6121fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x5631400a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff6121fe740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec00e7d0 con 0x563140032ac0 2024-03-09T00:09:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff6121fe740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec00fab0 con 0x5631400321a0 2024-03-09T00:09:09.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff6121fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec00edf0 con 0x56314003b6a0 2024-03-09T00:09:09.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff6121fe740 10 client.0 _ll_register_callbacks cb 0x7ff60401a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.266+0000 7ff60cae0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x5631400a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x5631400a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38656/0 (socket says 172.21.15.22:38656) 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 -- 172.21.15.22:0/3794841130 learned_addr learned my addr 172.21.15.22:0/3794841130 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631400321a0 0x5631400404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563140032ac0 0x5631400409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 -- 172.21.15.22:0/3794841130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631400321a0 msgr2=0x5631400404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631400321a0 0x5631400404b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 -- 172.21.15.22:0/3794841130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563140032ac0 msgr2=0x5631400409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563140032ac0 0x5631400409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.267+0000 7ff60cae0640 1 -- 172.21.15.22:0/3794841130 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631400a1710 con 0x56314003b6a0 2024-03-09T00:09:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.268+0000 7ff60cae0640 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x5631400a11d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff60403d2a0 tx=0x7ff6040047a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d964b85962e684 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.268+0000 7ff5f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:09:09.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.268+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3794841130 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60403d840 con 0x56314003b6a0 2024-03-09T00:09:09.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.268+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff600024b10 con 0x56314003b6a0 2024-03-09T00:09:09.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.268+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3794841130 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff60403d9e0 con 0x56314003b6a0 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.269+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3794841130 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff604078050 con 0x56314003b6a0 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.269+0000 7ff6121fe740 10 client.29521 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.269+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5631400a28b0 con 0x56314003b6a0 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.269+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3794841130 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff600024b10 con 0x56314003b6a0 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.270+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3794841130 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff604005020 con 0x56314003b6a0 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.270+0000 7ff6121fe740 10 client.29521 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.270+0000 7ff6121fe740 10 client.29521 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.270+0000 7ff6121fe740 20 client.29521 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.270+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5631400a3000 0x5631400a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.270+0000 7ff6121fe740 4 client.29521 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:09:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.270+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56313ff9d750 con 0x5631400a3000 2024-03-09T00:09:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.272+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5631400a3000 0x5631400a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.273+0000 7ff60d2e1640 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5631400a3000 0x5631400a5420 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=804e34241b4bcd8f server_cookie=1df2863bc2cdad5 in_seq=0 out_seq=0 2024-03-09T00:09:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.273+0000 7ff5f0ff9640 10 client.29521 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.274+0000 7ff5f0ff9640 1 -- 172.21.15.22:0/3794841130 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56313ff9d750 con 0x5631400a3000 2024-03-09T00:09:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.274+0000 7ff5f0ff9640 10 client.29521 handle_command_reply: tid=0 2024-03-09T00:09:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.274+0000 7ff6121fe740 1 client.29521 shutdown 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 msgr2=0x5631400a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x5631400a11d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff60403d2a0 tx=0x7ff6040047a0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 shutdown_connections 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5631400a3000 0x5631400a5420 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56314003b6a0 0x5631400a11d0 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff60403d2a0 tx=0x7ff6040047a0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563140032ac0 0x5631400409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 --2- 172.21.15.22:0/3794841130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631400321a0 0x5631400404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 >> 172.21.15.22:0/3794841130 conn(0x56314002c8d0 msgr2=0x563140052a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff5f0ff9640 0 client.29521 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 shutdown_connections 2024-03-09T00:09:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 1 -- 172.21.15.22:0/3794841130 wait complete. 2024-03-09T00:09:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.275+0000 7ff6121fe740 20 client.29521 trim_cache size 0 max 16384 2024-03-09T00:09:09.271 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.278+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5f40a2aa0 msgr2=0x7ff5f40a4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.278+0000 7ff60f545640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5f40a2aa0 0x7ff5f40a4f60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff5f809a8a0 tx=0x7ff5f8004770 comp rx=0 tx=0).stop 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.278+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff60806e130 msgr2=0x7ff60806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.278+0000 7ff60f545640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff60806e130 0x7ff60806e5e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800b3e0 tx=0x7ff5f800b8a0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.280+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 shutdown_connections 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.280+0000 7ff60f545640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff5f40a2aa0 0x7ff5f40a4f60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.280+0000 7ff60f545640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff60806e130 0x7ff60806e5e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.280+0000 7ff60f545640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6081054b0 0x7ff60806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.280+0000 7ff60f545640 1 --2- 172.21.15.22:0/3370596661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6081003e0 0x7ff608072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.280+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 >> 172.21.15.22:0/3370596661 conn(0x7ff6080773a0 msgr2=0x7ff6080fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.281+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 shutdown_connections 2024-03-09T00:09:09.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.281+0000 7ff60f545640 1 -- 172.21.15.22:0/3370596661 wait complete. 2024-03-09T00:09:09.296 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:09:09.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:09:09.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.491+0000 7fd03ffff640 1 -- 172.21.15.22:0/271656104 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400338f0 con 0x7fd048100300 2024-03-09T00:09:09.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.491+0000 7fd04f329640 1 -- 172.21.15.22:0/271656104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 msgr2=0x7fd0481006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.491+0000 7fd04f329640 1 --2- 172.21.15.22:0/271656104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 0x7fd0481006e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd040009be0 tx=0x7fd04002bfa0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.495+0000 7fd04f329640 1 -- 172.21.15.22:0/271656104 shutdown_connections 2024-03-09T00:09:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.495+0000 7fd04f329640 1 --2- 172.21.15.22:0/271656104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 0x7fd048105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.495+0000 7fd04f329640 1 --2- 172.21.15.22:0/271656104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd048100c20 0x7fd0481050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.495+0000 7fd04f329640 1 --2- 172.21.15.22:0/271656104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 0x7fd0481006e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.495+0000 7fd04f329640 1 -- 172.21.15.22:0/271656104 >> 172.21.15.22:0/271656104 conn(0x7fd0480773a0 msgr2=0x7fd0480777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.498+0000 7fd04f329640 1 -- 172.21.15.22:0/271656104 shutdown_connections 2024-03-09T00:09:09.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.499+0000 7fd04f329640 1 -- 172.21.15.22:0/271656104 wait complete. 2024-03-09T00:09:09.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.500+0000 7fd04f329640 1 Processor -- start 2024-03-09T00:09:09.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.500+0000 7fd04f329640 1 -- start start 2024-03-09T00:09:09.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.501+0000 7fd04f329640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 0x7fd048072a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.501+0000 7fd04f329640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd048100c20 0x7fd048072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.501+0000 7fd04f329640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 0x7fd04806bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.501+0000 7fd04f329640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048117ca0 con 0x7fd048105620 2024-03-09T00:09:09.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.501+0000 7fd04f329640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048117b20 con 0x7fd048100c20 2024-03-09T00:09:09.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.501+0000 7fd04f329640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048117e20 con 0x7fd048100300 2024-03-09T00:09:09.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.501+0000 7fd04d8c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 0x7fd04806bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.502+0000 7fd04d0c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 0x7fd048072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.502+0000 7fd04d8c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 0x7fd04806bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60976/0 (socket says 172.21.15.22:60976) 2024-03-09T00:09:09.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.502+0000 7fd04d8c6640 1 -- 172.21.15.22:0/2504606469 learned_addr learned my addr 172.21.15.22:0/2504606469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.504+0000 7fd04d8c6640 1 -- 172.21.15.22:0/2504606469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 msgr2=0x7fd048072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.504+0000 7fd04d8c6640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 0x7fd048072a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.504+0000 7fd04d8c6640 1 -- 172.21.15.22:0/2504606469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd048100c20 msgr2=0x7fd048072fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:09:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.504+0000 7fd04d8c6640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd048100c20 0x7fd048072fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.504+0000 7fd04d8c6640 1 -- 172.21.15.22:0/2504606469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04806c580 con 0x7fd048105620 2024-03-09T00:09:09.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.507+0000 7fd04d8c6640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 0x7fd04806bcc0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd03800d750 tx=0x7fd03800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.511+0000 7fd03e7fc640 1 -- 172.21.15.22:0/2504606469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038017070 con 0x7fd048105620 2024-03-09T00:09:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.511+0000 7fd03e7fc640 1 -- 172.21.15.22:0/2504606469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd038005500 con 0x7fd048105620 2024-03-09T00:09:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.511+0000 7fd03e7fc640 1 -- 172.21.15.22:0/2504606469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038004d90 con 0x7fd048105620 2024-03-09T00:09:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.512+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04806c870 con 0x7fd048105620 2024-03-09T00:09:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.512+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd048070820 con 0x7fd048105620 2024-03-09T00:09:09.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 Processor -- start 2024-03-09T00:09:09.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 -- start start 2024-03-09T00:09:09.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e585580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e585ac0 0x555a7e58e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e590a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e54c360 con 0x555a7e585ac0 2024-03-09T00:09:09.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e4e8ec0 con 0x555a7e5851a0 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd051fe3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e491d30 con 0x555a7e58e6a0 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.517+0000 7fd03e7fc640 1 -- 172.21.15.22:0/2504606469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd038010460 con 0x7fd048105620 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.518+0000 7fd03e7fc640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0340a2b40 0x7fd0340a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.519+0000 7fd04d8c6640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0340a2b40 0x7fd0340a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.519+0000 7fd04d0c5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e585ac0 0x555a7e58e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.519+0000 7fd04d0c5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e585ac0 0x555a7e58e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60984/0 (socket says 172.21.15.22:60984) 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.519+0000 7fd04d0c5640 1 -- 172.21.15.22:0/3161656715 learned_addr learned my addr 172.21.15.22:0/3161656715 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.519+0000 7fd04d8c6640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0340a2b40 0x7fd0340a5000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0380cbe40 tx=0x7fd0380d22b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.519+0000 7fd03e7fc640 1 -- 172.21.15.22:0/2504606469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fd0380ceb70 con 0x7fd048105620 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.520+0000 7fd04d0c5640 1 -- 172.21.15.22:0/3161656715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 msgr2=0x555a7e590a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:09.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.520+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/3161656715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e590a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.520+0000 7fd04d0c5640 1 -- 172.21.15.22:0/3161656715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 msgr2=0x555a7e585580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:09.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.520+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/3161656715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e585580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.520+0000 7fd04d0c5640 1 -- 172.21.15.22:0/3161656715 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7e490bc0 con 0x555a7e585ac0 2024-03-09T00:09:09.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.525+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/3161656715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e585ac0 0x555a7e58e160 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd040000f80 tx=0x7fd0400092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802cec2689d645a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.530+0000 7fd0337fe640 1 -- 172.21.15.22:0/3161656715 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040042070 con 0x555a7e585ac0 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.530+0000 7fd0337fe640 1 -- 172.21.15.22:0/3161656715 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd040033de0 con 0x555a7e585ac0 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.530+0000 7fd0337fe640 1 -- 172.21.15.22:0/3161656715 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04002cca0 con 0x555a7e585ac0 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.533+0000 7fd051fe3740 1 -- 172.21.15.22:0/3161656715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e585ac0 msgr2=0x555a7e58e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.533+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3161656715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e585ac0 0x555a7e58e160 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd040000f80 tx=0x7fd0400092f0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.534+0000 7fd051fe3740 1 -- 172.21.15.22:0/3161656715 shutdown_connections 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.534+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3161656715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e590a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.534+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3161656715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e585ac0 0x555a7e58e160 secure :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd040000f80 tx=0x7fd0400092f0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.534+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3161656715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e585580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.534+0000 7fd051fe3740 1 -- 172.21.15.22:0/3161656715 >> 172.21.15.22:0/3161656715 conn(0x555a7e57f8d0 msgr2=0x555a7e57fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.540+0000 7fd051fe3740 1 -- 172.21.15.22:0/3161656715 shutdown_connections 2024-03-09T00:09:09.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.542+0000 7fd051fe3740 1 -- 172.21.15.22:0/3161656715 wait complete. 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.542+0000 7fd051fe3740 1 -- start start 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.542+0000 7fd051fe3740 1 Processor -- start 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.543+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.543+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e58bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.543+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58c280 0x555a7e59a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.543+0000 7fd051fe3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e4874a0 con 0x555a7e5851a0 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.543+0000 7fd051fe3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e4e4f60 con 0x555a7e58c280 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.543+0000 7fd051fe3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e591430 con 0x555a7e58e6a0 2024-03-09T00:09:09.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.543+0000 7fd051fe3740 10 client.0 _ll_register_callbacks cb 0x555a7e4e2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.544+0000 7fd04c8c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.544+0000 7fd04c8c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60990/0 (socket says 172.21.15.22:60990) 2024-03-09T00:09:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.544+0000 7fd04c8c4640 1 -- 172.21.15.22:0/3737483713 learned_addr learned my addr 172.21.15.22:0/3737483713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.544+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e58bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.545+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58c280 0x555a7e59a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.546+0000 7fd04d0c5640 1 -- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58c280 msgr2=0x555a7e59a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.546+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58c280 0x555a7e59a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.546+0000 7fd04d0c5640 1 -- 172.21.15.22:0/3737483713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 msgr2=0x555a7e58b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.546+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.546+0000 7fd04d0c5640 1 -- 172.21.15.22:0/3737483713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a7e1fd3f0 con 0x555a7e58e6a0 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.550+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e58bd40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x555a7e5807d0 tx=0x7fd04002dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57554b17e451db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.551+0000 7fd030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.551+0000 7fd030ff9640 1 -- 172.21.15.22:0/3737483713 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040042070 con 0x555a7e58e6a0 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.551+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd040033de0 con 0x555a7e58e6a0 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.551+0000 7fd030ff9640 1 -- 172.21.15.22:0/3737483713 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fd040033700 con 0x555a7e58e6a0 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.551+0000 7fd030ff9640 1 -- 172.21.15.22:0/3737483713 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd040033900 con 0x555a7e58e6a0 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.552+0000 7fd051fe3740 10 client.29527 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.552+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x555a7e2c77d0 con 0x555a7e58e6a0 2024-03-09T00:09:09.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.554+0000 7fd030ff9640 1 -- 172.21.15.22:0/3737483713 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd040033de0 con 0x555a7e58e6a0 2024-03-09T00:09:09.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.554+0000 7fd030ff9640 1 -- 172.21.15.22:0/3737483713 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd0400475f0 con 0x555a7e58e6a0 2024-03-09T00:09:09.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.555+0000 7fd051fe3740 10 client.29527 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:09.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.555+0000 7fd051fe3740 10 client.29527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:09:09.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.555+0000 7fd051fe3740 20 client.29527 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:09.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.555+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e59ee70 0x555a7e5a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.555+0000 7fd051fe3740 4 client.29527 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.555+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a7e4f6400 con 0x555a7e59ee70 2024-03-09T00:09:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.557+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e59ee70 0x555a7e5a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.564+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e59ee70 0x555a7e5a12b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=749f901d5ce080e6 server_cookie=ea8f912444c92324 in_seq=0 out_seq=0 2024-03-09T00:09:09.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.565+0000 7fd030ff9640 10 client.29527 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:09.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.567+0000 7fd030ff9640 1 -- 172.21.15.22:0/3737483713 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a7e4f6400 con 0x555a7e59ee70 2024-03-09T00:09:09.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.567+0000 7fd030ff9640 10 client.29527 handle_command_reply: tid=0 2024-03-09T00:09:09.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.567+0000 7fd051fe3740 1 client.29527 shutdown 2024-03-09T00:09:09.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.575+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 msgr2=0x555a7e58bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.575+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e58bd40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x555a7e5807d0 tx=0x7fd04002dc60 comp rx=0 tx=0).stop 2024-03-09T00:09:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.578+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 shutdown_connections 2024-03-09T00:09:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.578+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e59ee70 0x555a7e5a12b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.578+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58c280 0x555a7e59a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.578+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58e6a0 0x555a7e58bd40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.578+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3737483713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.578+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 >> 172.21.15.22:0/3737483713 conn(0x555a7e57f8d0 msgr2=0x555a7e581620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.578+0000 7fd030ff9640 0 client.29527 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:09.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.580+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 shutdown_connections 2024-03-09T00:09:09.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.583+0000 7fd051fe3740 1 -- 172.21.15.22:0/3737483713 wait complete. 2024-03-09T00:09:09.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.584+0000 7fd051fe3740 20 client.29527 trim_cache size 0 max 16384 2024-03-09T00:09:09.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.610+0000 7fd051fe3740 1 Processor -- start 2024-03-09T00:09:09.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd051fe3740 1 -- start start 2024-03-09T00:09:09.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e59a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58e6a0 0x555a7e59a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd051fe3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c012600 con 0x555a7e5851a0 2024-03-09T00:09:09.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd051fe3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c010140 con 0x555a7e58e6a0 2024-03-09T00:09:09.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd051fe3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c00e200 con 0x555a7e58c280 2024-03-09T00:09:09.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd04c8c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd04c8c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60998/0 (socket says 172.21.15.22:60998) 2024-03-09T00:09:09.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.611+0000 7fd04c8c4640 1 -- 172.21.15.22:0/3257556138 learned_addr learned my addr 172.21.15.22:0/3257556138 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.612+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58e6a0 0x555a7e59a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.612+0000 7fd04c8c4640 1 -- 172.21.15.22:0/3257556138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 msgr2=0x555a7e59a1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:09.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.612+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e59a1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.612+0000 7fd04c8c4640 1 -- 172.21.15.22:0/3257556138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58e6a0 msgr2=0x555a7e59a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.612+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58e6a0 0x555a7e59a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.612+0000 7fd04c8c4640 1 -- 172.21.15.22:0/3257556138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7e590e00 con 0x555a7e5851a0 2024-03-09T00:09:09.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.613+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58e040 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd04400a010 tx=0x7fd044009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead5d275cc84b97b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:09 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2260983333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:09:09.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.614+0000 7fd032ffd640 1 -- 172.21.15.22:0/3257556138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044004050 con 0x555a7e5851a0 2024-03-09T00:09:09.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.614+0000 7fd032ffd640 1 -- 172.21.15.22:0/3257556138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd04401d760 con 0x555a7e5851a0 2024-03-09T00:09:09.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.614+0000 7fd032ffd640 1 -- 172.21.15.22:0/3257556138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044003bb0 con 0x555a7e5851a0 2024-03-09T00:09:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.617+0000 7fd051fe3740 1 -- 172.21.15.22:0/3257556138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 msgr2=0x555a7e58e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.617+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58e040 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd04400a010 tx=0x7fd044009650 comp rx=0 tx=0).stop 2024-03-09T00:09:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.620+0000 7fd051fe3740 1 -- 172.21.15.22:0/3257556138 shutdown_connections 2024-03-09T00:09:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.620+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e58e6a0 0x555a7e59a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.620+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e59a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.620+0000 7fd051fe3740 1 --2- 172.21.15.22:0/3257556138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e5851a0 0x555a7e58e040 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.620+0000 7fd051fe3740 1 -- 172.21.15.22:0/3257556138 >> 172.21.15.22:0/3257556138 conn(0x555a7e57f8d0 msgr2=0x555a7e5a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.626+0000 7fd051fe3740 1 -- 172.21.15.22:0/3257556138 shutdown_connections 2024-03-09T00:09:09.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.626+0000 7fd051fe3740 1 -- 172.21.15.22:0/3257556138 wait complete. 2024-03-09T00:09:09.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.626+0000 7fd051fe3740 1 -- start start 2024-03-09T00:09:09.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.626+0000 7fd051fe3740 1 Processor -- start 2024-03-09T00:09:09.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.627+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e595790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.627+0000 7fd051fe3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e593ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd04c8c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e595790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd04c8c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e595790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36734/0 (socket says 172.21.15.22:36734) 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd04d0c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e593ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd04d0c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e593ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38702/0 (socket says 172.21.15.22:38702) 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd04d0c5640 1 -- 172.21.15.22:0/4102208263 learned_addr learned my addr 172.21.15.22:0/4102208263 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd051fe3740 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e58e6a0 0x555a7e5941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c00e800 con 0x555a7e58e6a0 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c00fae0 con 0x555a7e5851a0 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c00ee20 con 0x555a7e58c280 2024-03-09T00:09:09.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.628+0000 7fd051fe3740 10 client.0 _ll_register_callbacks cb 0x7fd044005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:09.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.629+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e58e6a0 0x555a7e5941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.630+0000 7fd04d0c5640 1 -- 172.21.15.22:0/4102208263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 msgr2=0x555a7e595790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.630+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e595790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.630+0000 7fd04d0c5640 1 -- 172.21.15.22:0/4102208263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e58e6a0 msgr2=0x555a7e5941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.630+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e58e6a0 0x555a7e5941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.630+0000 7fd04d0c5640 1 -- 172.21.15.22:0/4102208263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a7e594720 con 0x555a7e58c280 2024-03-09T00:09:09.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.634+0000 7fd04d0c5640 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e593ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd040005f00 tx=0x7fd040005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d6d486cadcf7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:09 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2260983333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:09:09.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.638+0000 7fd030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:09:09.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.638+0000 7fd030ff9640 1 -- 172.21.15.22:0/4102208263 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040056020 con 0x555a7e58c280 2024-03-09T00:09:09.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.638+0000 7fd030ff9640 1 -- 172.21.15.22:0/4102208263 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fd040033d70 con 0x555a7e58c280 2024-03-09T00:09:09.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.638+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd04401d760 con 0x555a7e58c280 2024-03-09T00:09:09.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.642+0000 7fd030ff9640 1 -- 172.21.15.22:0/4102208263 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd040042040 con 0x555a7e58c280 2024-03-09T00:09:09.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.643+0000 7fd051fe3740 10 client.29530 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:09.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.643+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x555a7e5f60d0 con 0x555a7e58c280 2024-03-09T00:09:09.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.645+0000 7fd030ff9640 1 -- 172.21.15.22:0/4102208263 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd04401d760 con 0x555a7e58c280 2024-03-09T00:09:09.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.645+0000 7fd030ff9640 1 -- 172.21.15.22:0/4102208263 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd04002cc50 con 0x555a7e58c280 2024-03-09T00:09:09.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.646+0000 7fd051fe3740 10 client.29530 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:09.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.646+0000 7fd051fe3740 10 client.29530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:09:09.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.646+0000 7fd051fe3740 20 client.29530 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:09.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.646+0000 7fd051fe3740 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e5f67a0 0x555a7e5f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.646+0000 7fd051fe3740 4 client.29530 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:09:09.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.646+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555a7e4f0750 con 0x555a7e5f67a0 2024-03-09T00:09:09.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.648+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e5f67a0 0x555a7e5f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.655+0000 7fd04c8c4640 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e5f67a0 0x555a7e5f8bc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2296878b06dff5f5 server_cookie=cf20b1079a0d679e in_seq=0 out_seq=0 2024-03-09T00:09:09.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.655+0000 7fd030ff9640 10 client.29530 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:09.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.655+0000 7fd030ff9640 1 -- 172.21.15.22:0/4102208263 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555a7e4f0750 con 0x555a7e5f67a0 2024-03-09T00:09:09.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.656+0000 7fd030ff9640 10 client.29530 handle_command_reply: tid=0 2024-03-09T00:09:09.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.656+0000 7fd051fe3740 1 client.29530 shutdown 2024-03-09T00:09:09.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 msgr2=0x555a7e593ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e593ca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd040005f00 tx=0x7fd040005fd0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 shutdown_connections 2024-03-09T00:09:09.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x555a7e5f67a0 0x555a7e5f8bc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a7e58e6a0 0x555a7e5941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a7e58c280 0x555a7e593ca0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 --2- 172.21.15.22:0/4102208263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a7e5851a0 0x555a7e595790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 >> 172.21.15.22:0/4102208263 conn(0x555a7e57f8d0 msgr2=0x555a7e59a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd030ff9640 0 client.29530 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:09.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 shutdown_connections 2024-03-09T00:09:09.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.657+0000 7fd051fe3740 1 -- 172.21.15.22:0/4102208263 wait complete. 2024-03-09T00:09:09.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.658+0000 7fd051fe3740 20 client.29530 trim_cache size 0 max 16384 2024-03-09T00:09:09.653 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:09:09.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.661+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0340a2b40 msgr2=0x7fd0340a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.661+0000 7fd04f329640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0340a2b40 0x7fd0340a5000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0380cbe40 tx=0x7fd0380d22b0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.661+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 msgr2=0x7fd04806bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.661+0000 7fd04f329640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 0x7fd04806bcc0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd03800d750 tx=0x7fd03800dc10 comp rx=0 tx=0).stop 2024-03-09T00:09:09.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.662+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 shutdown_connections 2024-03-09T00:09:09.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.662+0000 7fd04f329640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0340a2b40 0x7fd0340a5000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.662+0000 7fd04f329640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd048105620 0x7fd04806bcc0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.662+0000 7fd04f329640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd048100c20 0x7fd048072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.662+0000 7fd04f329640 1 --2- 172.21.15.22:0/2504606469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048100300 0x7fd048072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.662+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 >> 172.21.15.22:0/2504606469 conn(0x7fd0480773a0 msgr2=0x7fd0481030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.664+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 shutdown_connections 2024-03-09T00:09:09.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.664+0000 7fd04f329640 1 -- 172.21.15.22:0/2504606469 wait complete. 2024-03-09T00:09:09.676 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:09 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2260983333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:09:09.678 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:09:09.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:09:09.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.869+0000 7efe584e8640 1 -- 172.21.15.22:0/1318876302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 msgr2=0x7efe5011cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.869+0000 7efe584e8640 1 --2- 172.21.15.22:0/1318876302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 0x7efe5011cec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009860 tx=0x7efe4c02bb90 comp rx=0 tx=0).stop 2024-03-09T00:09:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.869+0000 7efe584e8640 1 -- 172.21.15.22:0/1318876302 shutdown_connections 2024-03-09T00:09:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.869+0000 7efe584e8640 1 --2- 172.21.15.22:0/1318876302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 0x7efe5011cec0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.869+0000 7efe584e8640 1 --2- 172.21.15.22:0/1318876302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe5010fe80 0x7efe5010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.869+0000 7efe584e8640 1 --2- 172.21.15.22:0/1318876302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5010fa10 0x7efe5010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.869+0000 7efe584e8640 1 -- 172.21.15.22:0/1318876302 >> 172.21.15.22:0/1318876302 conn(0x7efe5006c3e0 msgr2=0x7efe5006c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.875+0000 7efe584e8640 1 -- 172.21.15.22:0/1318876302 shutdown_connections 2024-03-09T00:09:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.875+0000 7efe584e8640 1 -- 172.21.15.22:0/1318876302 wait complete. 2024-03-09T00:09:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.875+0000 7efe584e8640 1 Processor -- start 2024-03-09T00:09:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe584e8640 1 -- start start 2024-03-09T00:09:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe584e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe5010fa10 0x7efe501ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe584e8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5010fe80 0x7efe501acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe584e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 0x7efe501a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe584e8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe50120450 con 0x7efe5010fa10 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe584e8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe501202d0 con 0x7efe5011aad0 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe584e8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe501205d0 con 0x7efe5010fe80 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe56284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe5010fa10 0x7efe501ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe56284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe5010fa10 0x7efe501ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32806/0 (socket says 172.21.15.22:32806) 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.876+0000 7efe56284640 1 -- 172.21.15.22:0/4109015039 learned_addr learned my addr 172.21.15.22:0/4109015039 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.878+0000 7efe56a85640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 0x7efe501a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.879+0000 7efe55a83640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5010fe80 0x7efe501acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.880+0000 7efe55a83640 1 -- 172.21.15.22:0/4109015039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 msgr2=0x7efe501a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.880+0000 7efe55a83640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 0x7efe501a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.880+0000 7efe55a83640 1 -- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe5010fa10 msgr2=0x7efe501ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.880+0000 7efe55a83640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe5010fa10 0x7efe501ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.880+0000 7efe55a83640 1 -- 172.21.15.22:0/4109015039 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe501a6f80 con 0x7efe5010fe80 2024-03-09T00:09:09.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.881+0000 7efe55a83640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5010fe80 0x7efe501acb80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efe44005fd0 tx=0x7efe4400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.881+0000 7efe437fe640 1 -- 172.21.15.22:0/4109015039 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400ccc0 con 0x7efe5010fe80 2024-03-09T00:09:09.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.882+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe501a7270 con 0x7efe5010fe80 2024-03-09T00:09:09.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.882+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe501102b0 con 0x7efe5010fe80 2024-03-09T00:09:09.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.882+0000 7efe437fe640 1 -- 172.21.15.22:0/4109015039 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe4400ee70 con 0x7efe5010fe80 2024-03-09T00:09:09.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.882+0000 7efe437fe640 1 -- 172.21.15.22:0/4109015039 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44018780 con 0x7efe5010fe80 2024-03-09T00:09:09.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.886+0000 7efe5b19f740 1 Processor -- start 2024-03-09T00:09:09.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.890+0000 7efe5b19f740 1 -- start start 2024-03-09T00:09:09.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.890+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd032580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.890+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd032ac0 0x55f4fd03b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.890+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd03da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.890+0000 7efe56a85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd032580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.891+0000 7efe56a85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd032580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:38722/0 (socket says 172.21.15.22:38722) 2024-03-09T00:09:09.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.891+0000 7efe56a85640 1 -- 172.21.15.22:0/1454635153 learned_addr learned my addr 172.21.15.22:0/1454635153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.891+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fcff9360 con 0x55f4fd03b6a0 2024-03-09T00:09:09.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.891+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fcf95ec0 con 0x55f4fd032ac0 2024-03-09T00:09:09.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.891+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fcf3ed30 con 0x55f4fd0321a0 2024-03-09T00:09:09.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.891+0000 7efe56284640 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd032ac0 0x55f4fd03b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.892+0000 7efe55a83640 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd03da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.894+0000 7efe437fe640 1 -- 172.21.15.22:0/4109015039 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7efe44018920 con 0x7efe5010fe80 2024-03-09T00:09:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.895+0000 7efe437fe640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe3c0a2ab0 0x7efe3c0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.898+0000 7efe437fe640 1 -- 172.21.15.22:0/4109015039 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7efe440cf3c0 con 0x7efe5010fe80 2024-03-09T00:09:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.899+0000 7efe55a83640 1 -- 172.21.15.22:0/1454635153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 msgr2=0x55f4fd032580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.899+0000 7efe55a83640 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd032580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.899+0000 7efe55a83640 1 -- 172.21.15.22:0/1454635153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd032ac0 msgr2=0x55f4fd03b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.899+0000 7efe55a83640 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd032ac0 0x55f4fd03b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.899+0000 7efe55a83640 1 -- 172.21.15.22:0/1454635153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4fcf3dbc0 con 0x55f4fd03b6a0 2024-03-09T00:09:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.901+0000 7efe56a85640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe3c0a2ab0 0x7efe3c0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.903+0000 7efe55a83640 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd03da90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7efe440cc660 tx=0x7efe4409b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212bd35503d03ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.907+0000 7efe40bfe640 1 -- 172.21.15.22:0/1454635153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400ce60 con 0x55f4fd03b6a0 2024-03-09T00:09:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.907+0000 7efe40bfe640 1 -- 172.21.15.22:0/1454635153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe4409bc70 con 0x55f4fd03b6a0 2024-03-09T00:09:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.907+0000 7efe40bfe640 1 -- 172.21.15.22:0/1454635153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe440cc8e0 con 0x55f4fd03b6a0 2024-03-09T00:09:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.910+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 msgr2=0x55f4fd03da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.910+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd03da90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7efe440cc660 tx=0x7efe4409b0b0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.911+0000 7efe56a85640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe3c0a2ab0 0x7efe3c0a4f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005b60 tx=0x7efe4c02bb90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.921+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 shutdown_connections 2024-03-09T00:09:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.921+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd03da90 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.921+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd032ac0 0x55f4fd03b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.921+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1454635153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd032580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.921+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 >> 172.21.15.22:0/1454635153 conn(0x55f4fd02c8d0 msgr2=0x55f4fd02ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.924+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 shutdown_connections 2024-03-09T00:09:09.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.928+0000 7efe5b19f740 1 -- 172.21.15.22:0/1454635153 wait complete. 2024-03-09T00:09:09.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.928+0000 7efe5b19f740 1 -- start start 2024-03-09T00:09:09.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.928+0000 7efe5b19f740 1 Processor -- start 2024-03-09T00:09:09.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.929+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd038650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.929+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd032ac0 0x55f4fd038b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.929+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.929+0000 7efe5b19f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fcf344a0 con 0x55f4fd032ac0 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.929+0000 7efe5b19f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fcf91f60 con 0x55f4fd03b6a0 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.929+0000 7efe5b19f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fd03e3d0 con 0x55f4fd0321a0 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.930+0000 7efe5b19f740 10 client.0 _ll_register_callbacks cb 0x55f4fcf8fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.930+0000 7efe55a83640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd032ac0 0x55f4fd038b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.930+0000 7efe55a83640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd032ac0 0x55f4fd038b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32820/0 (socket says 172.21.15.22:32820) 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.930+0000 7efe55a83640 1 -- 172.21.15.22:0/1938604859 learned_addr learned my addr 172.21.15.22:0/1938604859 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:09.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.930+0000 7efe56284640 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd038650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.932+0000 7efe56a85640 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.934+0000 7efe56284640 1 -- 172.21.15.22:0/1938604859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 msgr2=0x55f4fd0390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.934+0000 7efe56284640 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.934+0000 7efe56284640 1 -- 172.21.15.22:0/1938604859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd032ac0 msgr2=0x55f4fd038b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:09.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.934+0000 7efe56284640 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd032ac0 0x55f4fd038b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.934+0000 7efe56284640 1 -- 172.21.15.22:0/1938604859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4fccaa3f0 con 0x55f4fd0321a0 2024-03-09T00:09:09.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.935+0000 7efe56284640 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd038650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe48012cf0 tx=0x7efe4800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0452b0a592f521 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:09.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.936+0000 7efe39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:09:09.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.938+0000 7efe39ffb640 1 -- 172.21.15.22:0/1938604859 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe48009ca0 con 0x55f4fd0321a0 2024-03-09T00:09:09.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.938+0000 7efe39ffb640 1 -- 172.21.15.22:0/1938604859 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7efe48003970 con 0x55f4fd0321a0 2024-03-09T00:09:09.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.940+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe4409bc70 con 0x55f4fd0321a0 2024-03-09T00:09:09.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.948+0000 7efe39ffb640 1 -- 172.21.15.22:0/1938604859 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe48009e40 con 0x55f4fd0321a0 2024-03-09T00:09:09.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.949+0000 7efe5b19f740 10 client.29542 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:09.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.949+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f4fcd747d0 con 0x55f4fd0321a0 2024-03-09T00:09:09.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.950+0000 7efe39ffb640 1 -- 172.21.15.22:0/1938604859 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe4409bc70 con 0x55f4fd0321a0 2024-03-09T00:09:09.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.950+0000 7efe39ffb640 1 -- 172.21.15.22:0/1938604859 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efe480104d0 con 0x55f4fd0321a0 2024-03-09T00:09:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.951+0000 7efe5b19f740 10 client.29542 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.951+0000 7efe5b19f740 10 client.29542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.951+0000 7efe5b19f740 20 client.29542 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.951+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd04bc40 0x55f4fd04e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.951+0000 7efe5b19f740 4 client.29542 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.951+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4fcfa3400 con 0x55f4fd04bc40 2024-03-09T00:09:09.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.953+0000 7efe55a83640 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd04bc40 0x55f4fd04e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:09.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.960+0000 7efe55a83640 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd04bc40 0x55f4fd04e0a0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6595289930e5b7ab server_cookie=ede03859ffaa1e27 in_seq=0 out_seq=0 2024-03-09T00:09:09.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.961+0000 7efe39ffb640 10 client.29542 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:09.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.965+0000 7efe39ffb640 1 -- 172.21.15.22:0/1938604859 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4fcfa3400 con 0x55f4fd04bc40 2024-03-09T00:09:09.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.965+0000 7efe39ffb640 10 client.29542 handle_command_reply: tid=0 2024-03-09T00:09:09.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.965+0000 7efe5b19f740 1 client.29542 shutdown 2024-03-09T00:09:09.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.974+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 msgr2=0x55f4fd038650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:09.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.974+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd038650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe48012cf0 tx=0x7efe4800f790 comp rx=0 tx=0).stop 2024-03-09T00:09:09.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.977+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 shutdown_connections 2024-03-09T00:09:09.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.977+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd04bc40 0x55f4fd04e0a0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.977+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.977+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd032ac0 0x55f4fd038b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.977+0000 7efe5b19f740 1 --2- 172.21.15.22:0/1938604859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd0321a0 0x55f4fd038650 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:09.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.977+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 >> 172.21.15.22:0/1938604859 conn(0x55f4fd02c8d0 msgr2=0x55f4fd02e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:09.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.977+0000 7efe39ffb640 0 client.29542 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:09.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.982+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 shutdown_connections 2024-03-09T00:09:09.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.984+0000 7efe5b19f740 1 -- 172.21.15.22:0/1938604859 wait complete. 2024-03-09T00:09:09.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:09.984+0000 7efe5b19f740 20 client.29542 trim_cache size 0 max 16384 2024-03-09T00:09:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.009+0000 7efe5b19f740 1 Processor -- start 2024-03-09T00:09:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.010+0000 7efe5b19f740 1 -- start start 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.010+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd0321a0 0x55f4fd02b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.010+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd042670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.010+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd042bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.010+0000 7efe5b19f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe34012580 con 0x55f4fd0321a0 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.010+0000 7efe5b19f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340100c0 con 0x55f4fd03b6a0 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.010+0000 7efe5b19f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3400e180 con 0x55f4fd032ac0 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.011+0000 7efe56a85640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd042bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.011+0000 7efe56a85640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd042bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36768/0 (socket says 172.21.15.22:36768) 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.011+0000 7efe56a85640 1 -- 172.21.15.22:0/677585529 learned_addr learned my addr 172.21.15.22:0/677585529 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.011+0000 7efe56284640 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd0321a0 0x55f4fd02b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.012+0000 7efe56284640 1 -- 172.21.15.22:0/677585529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 msgr2=0x55f4fd042670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.012+0000 7efe55a83640 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd042670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.013+0000 7efe56284640 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd042670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.013+0000 7efe56284640 1 -- 172.21.15.22:0/677585529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 msgr2=0x55f4fd042bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.013+0000 7efe56284640 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd042bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.013+0000 7efe56284640 1 -- 172.21.15.22:0/677585529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4fd03da90 con 0x55f4fd0321a0 2024-03-09T00:09:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.013+0000 7efe56a85640 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd042bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.013+0000 7efe56284640 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd0321a0 0x55f4fd02b810 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7efe480108d0 tx=0x7efe48005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c1d23bac9cea5f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.014+0000 7efe3bfff640 1 -- 172.21.15.22:0/677585529 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe48009bf0 con 0x55f4fd0321a0 2024-03-09T00:09:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.014+0000 7efe3bfff640 1 -- 172.21.15.22:0/677585529 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe48003e50 con 0x55f4fd0321a0 2024-03-09T00:09:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.014+0000 7efe3bfff640 1 -- 172.21.15.22:0/677585529 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe480050e0 con 0x55f4fd0321a0 2024-03-09T00:09:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.015+0000 7efe55a83640 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd042670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:09:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.015+0000 7efe5b19f740 1 -- 172.21.15.22:0/677585529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd0321a0 msgr2=0x55f4fd02b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.015+0000 7efe5b19f740 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd0321a0 0x55f4fd02b810 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7efe480108d0 tx=0x7efe48005b70 comp rx=0 tx=0).stop 2024-03-09T00:09:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.018+0000 7efe5b19f740 1 -- 172.21.15.22:0/677585529 shutdown_connections 2024-03-09T00:09:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.018+0000 7efe5b19f740 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd03b6a0 0x55f4fd042bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.018+0000 7efe5b19f740 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd042670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.018+0000 7efe5b19f740 1 --2- 172.21.15.22:0/677585529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd0321a0 0x55f4fd02b810 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.018+0000 7efe5b19f740 1 -- 172.21.15.22:0/677585529 >> 172.21.15.22:0/677585529 conn(0x55f4fd02c8d0 msgr2=0x55f4fd02b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.020+0000 7efe5b19f740 1 -- 172.21.15.22:0/677585529 shutdown_connections 2024-03-09T00:09:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.024+0000 7efe5b19f740 1 -- 172.21.15.22:0/677585529 wait complete. 2024-03-09T00:09:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.024+0000 7efe5b19f740 1 -- start start 2024-03-09T00:09:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.024+0000 7efe5b19f740 1 Processor -- start 2024-03-09T00:09:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd0321a0 0x55f4fd0429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd040d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe5b19f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe5b19f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3400e780 con 0x55f4fd03b6a0 2024-03-09T00:09:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe5b19f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3400fa60 con 0x55f4fd0321a0 2024-03-09T00:09:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe5b19f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3400eda0 con 0x55f4fd032ac0 2024-03-09T00:09:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe5b19f740 10 client.0 _ll_register_callbacks cb 0x7efe44097890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe56a85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe56a85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32840/0 (socket says 172.21.15.22:32840) 2024-03-09T00:09:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe56a85640 1 -- 172.21.15.22:0/952465676 learned_addr learned my addr 172.21.15.22:0/952465676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.025+0000 7efe56284640 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd0321a0 0x55f4fd0429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.026+0000 7efe56a85640 1 -- 172.21.15.22:0/952465676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 msgr2=0x55f4fd040d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.026+0000 7efe56a85640 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd040d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.026+0000 7efe56a85640 1 -- 172.21.15.22:0/952465676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd0321a0 msgr2=0x55f4fd0429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.026+0000 7efe56a85640 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd0321a0 0x55f4fd0429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.026+0000 7efe56a85640 1 -- 172.21.15.22:0/952465676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4fd041ab0 con 0x55f4fd03b6a0 2024-03-09T00:09:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.026+0000 7efe56a85640 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0412a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7efe4c037040 tx=0x7efe4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1006d950232f1337 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.027+0000 7efe39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.027+0000 7efe39ffb640 1 -- 172.21.15.22:0/952465676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c056060 con 0x55f4fd03b6a0 2024-03-09T00:09:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.027+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe48003e50 con 0x55f4fd03b6a0 2024-03-09T00:09:10.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.028+0000 7efe39ffb640 1 -- 172.21.15.22:0/952465676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7efe4c02c730 con 0x55f4fd03b6a0 2024-03-09T00:09:10.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.032+0000 7efe39ffb640 1 -- 172.21.15.22:0/952465676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe4c05d050 con 0x55f4fd03b6a0 2024-03-09T00:09:10.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.033+0000 7efe5b19f740 10 client.20148 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:10.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.033+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f4fd0a3140 con 0x55f4fd03b6a0 2024-03-09T00:09:10.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.034+0000 7efe39ffb640 1 -- 172.21.15.22:0/952465676 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe48003e50 con 0x55f4fd03b6a0 2024-03-09T00:09:10.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.035+0000 7efe39ffb640 1 -- 172.21.15.22:0/952465676 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efe4c004d20 con 0x55f4fd03b6a0 2024-03-09T00:09:10.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.035+0000 7efe5b19f740 10 client.20148 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:10.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.035+0000 7efe5b19f740 10 client.20148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:10.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.035+0000 7efe5b19f740 20 client.20148 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:10.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.035+0000 7efe5b19f740 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd0a3950 0x55f4fd0a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:10.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.035+0000 7efe5b19f740 4 client.20148 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:09:10.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.035+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f4fcf9d750 con 0x55f4fd0a3950 2024-03-09T00:09:10.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.037+0000 7efe56284640 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd0a3950 0x55f4fd0a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:10.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.042+0000 7efe56284640 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd0a3950 0x55f4fd0a5d70 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a100d4a1fabbae server_cookie=25455edec599f158 in_seq=0 out_seq=0 2024-03-09T00:09:10.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.042+0000 7efe39ffb640 10 client.20148 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:10.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.045+0000 7efe39ffb640 1 -- 172.21.15.22:0/952465676 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f4fcf9d750 con 0x55f4fd0a3950 2024-03-09T00:09:10.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.045+0000 7efe39ffb640 10 client.20148 handle_command_reply: tid=0 2024-03-09T00:09:10.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.045+0000 7efe5b19f740 1 client.20148 shutdown 2024-03-09T00:09:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.050+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 msgr2=0x55f4fd0412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:10.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.050+0000 7efe5b19f740 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0412a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7efe4c037040 tx=0x7efe4c005f90 comp rx=0 tx=0).stop 2024-03-09T00:09:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.054+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 shutdown_connections 2024-03-09T00:09:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.054+0000 7efe5b19f740 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f4fd0a3950 0x55f4fd0a5d70 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.054+0000 7efe5b19f740 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4fd03b6a0 0x55f4fd0412a0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.054+0000 7efe5b19f740 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4fd032ac0 0x55f4fd040d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.054+0000 7efe5b19f740 1 --2- 172.21.15.22:0/952465676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4fd0321a0 0x55f4fd0429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.054+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 >> 172.21.15.22:0/952465676 conn(0x55f4fd02c8d0 msgr2=0x55f4fd02b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.054+0000 7efe39ffb640 0 client.20148 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.062+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 shutdown_connections 2024-03-09T00:09:10.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.065+0000 7efe5b19f740 1 -- 172.21.15.22:0/952465676 wait complete. 2024-03-09T00:09:10.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.066+0000 7efe5b19f740 20 client.20148 trim_cache size 0 max 16384 2024-03-09T00:09:10.063 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:09:10.064 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17729 14.74 MB/sec warmup 77 sec latency 1054.469 ms 2024-03-09T00:09:10.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe3c0a2ab0 msgr2=0x7efe3c0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:10.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe3c0a2ab0 0x7efe3c0a4f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005b60 tx=0x7efe4c02bb90 comp rx=0 tx=0).stop 2024-03-09T00:09:10.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5010fe80 msgr2=0x7efe501acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:10.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5010fe80 0x7efe501acb80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efe44005fd0 tx=0x7efe4400ec20 comp rx=0 tx=0).stop 2024-03-09T00:09:10.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 shutdown_connections 2024-03-09T00:09:10.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe3c0a2ab0 0x7efe3c0a4f70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe5011aad0 0x7efe501a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5010fe80 0x7efe501acb80 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efe44005fd0 tx=0x7efe4400ec20 comp rx=0 tx=0).stop 2024-03-09T00:09:10.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 --2- 172.21.15.22:0/4109015039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe5010fa10 0x7efe501ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:10.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.070+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 >> 172.21.15.22:0/4109015039 conn(0x7efe5006c3e0 msgr2=0x7efe501194c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:10.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.071+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 shutdown_connections 2024-03-09T00:09:10.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:10.073+0000 7efe584e8640 1 -- 172.21.15.22:0/4109015039 wait complete. 2024-03-09T00:09:10.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:10 smithi022 ceph-mon[31894]: pgmap v462: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 672 op/s 2024-03-09T00:09:10.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:10 smithi062 ceph-mon[36129]: pgmap v462: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 672 op/s 2024-03-09T00:09:10.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:10 smithi192 ceph-mon[35995]: pgmap v462: 129 pgs: 129 active+clean; 300 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 672 op/s 2024-03-09T00:09:11.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17837 14.58 MB/sec warmup 78 sec latency 810.634 ms 2024-03-09T00:09:11.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:09:11.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:09:11.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:09:11.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.272+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3525215282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec073fb0 msgr2=0x7fa6ec111d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.272+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3525215282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec073fb0 0x7fa6ec111d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009900 tx=0x7fa6e802bd10 comp rx=0 tx=0).stop 2024-03-09T00:09:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.272+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3525215282 shutdown_connections 2024-03-09T00:09:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.272+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3525215282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ec112290 0x7fa6ec114680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.272+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3525215282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec073fb0 0x7fa6ec111d50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.272+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3525215282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6ec073690 0x7fa6ec073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.272+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3525215282 >> 172.21.15.22:0/3525215282 conn(0x7fa6ec0786b0 msgr2=0x7fa6ec078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.276+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3525215282 shutdown_connections 2024-03-09T00:09:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.279+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3525215282 wait complete. 2024-03-09T00:09:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.279+0000 7fa6f40a1640 1 Processor -- start 2024-03-09T00:09:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f40a1640 1 -- start start 2024-03-09T00:09:11.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f40a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ec073690 0x7fa6ec072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f40a1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6ec073fb0 0x7fa6ec072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f40a1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec112290 0x7fa6ec06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f40a1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec117d40 con 0x7fa6ec073690 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f40a1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec117bc0 con 0x7fa6ec112290 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f40a1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec117ec0 con 0x7fa6ec073fb0 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f263e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec112290 0x7fa6ec06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f263e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec112290 0x7fa6ec06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55008/0 (socket says 172.21.15.22:55008) 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.280+0000 7fa6f263e640 1 -- 172.21.15.22:0/3116544829 learned_addr learned my addr 172.21.15.22:0/3116544829 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.281+0000 7fa6f263e640 1 -- 172.21.15.22:0/3116544829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6ec073fb0 msgr2=0x7fa6ec072f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.281+0000 7fa6f263e640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6ec073fb0 0x7fa6ec072f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.281+0000 7fa6f263e640 1 -- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ec073690 msgr2=0x7fa6ec072a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.281+0000 7fa6f263e640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ec073690 0x7fa6ec072a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.281+0000 7fa6f263e640 1 -- 172.21.15.22:0/3116544829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec06c550 con 0x7fa6ec112290 2024-03-09T00:09:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.281+0000 7fa6f263e640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec112290 0x7fa6ec06bc90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e9d0 tx=0x7fa6e400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.283+0000 7fa6deffd640 1 -- 172.21.15.22:0/3116544829 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400cd80 con 0x7fa6ec112290 2024-03-09T00:09:11.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.283+0000 7fa6deffd640 1 -- 172.21.15.22:0/3116544829 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e4005020 con 0x7fa6ec112290 2024-03-09T00:09:11.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.283+0000 7fa6deffd640 1 -- 172.21.15.22:0/3116544829 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40116d0 con 0x7fa6ec112290 2024-03-09T00:09:11.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.283+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec06c840 con 0x7fa6ec112290 2024-03-09T00:09:11.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.283+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec070790 con 0x7fa6ec112290 2024-03-09T00:09:11.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.288+0000 7fa6f6922740 1 Processor -- start 2024-03-09T00:09:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.291+0000 7fa6deffd640 1 -- 172.21.15.22:0/3116544829 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa6e4004a40 con 0x7fa6ec112290 2024-03-09T00:09:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.292+0000 7fa6deffd640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa6d80a2b00 0x7fa6d80a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.292+0000 7fa6deffd640 1 -- 172.21.15.22:0/3116544829 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa6e40cf350 con 0x7fa6ec112290 2024-03-09T00:09:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.292+0000 7fa6f6922740 1 -- start start 2024-03-09T00:09:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db2900d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db29015d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29018670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f6922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db28f70b40 con 0x55db2900cd80 2024-03-09T00:09:11.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f6922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db28f19d80 con 0x55db29016280 2024-03-09T00:09:11.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f6922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db28f6cbe0 con 0x55db2900d6a0 2024-03-09T00:09:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f1e3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db2900d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f1e3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db2900d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34910/0 (socket says 172.21.15.22:34910) 2024-03-09T00:09:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f1e3d640 1 -- 172.21.15.22:0/2451275893 learned_addr learned my addr 172.21.15.22:0/2451275893 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f1e3d640 1 -- 172.21.15.22:0/2451275893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 msgr2=0x55db29015d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.293+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/2451275893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db29015d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6f1e3d640 1 -- 172.21.15.22:0/2451275893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 msgr2=0x55db29018670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/2451275893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29018670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6f1e3d640 1 -- 172.21.15.22:0/2451275893 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db28d4d7e0 con 0x55db2900cd80 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/2451275893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db2900d160 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009770 tx=0x7fa6e0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b580aec750b9228 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6d7fff640 1 -- 172.21.15.22:0/2451275893 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0015020 con 0x55db2900cd80 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6f163c640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa6d80a2b00 0x7fa6d80a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6d7fff640 1 -- 172.21.15.22:0/2451275893 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e0011420 con 0x55db2900cd80 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.294+0000 7fa6d7fff640 1 -- 172.21.15.22:0/2451275893 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00105e0 con 0x55db2900cd80 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.295+0000 7fa6f6922740 1 -- 172.21.15.22:0/2451275893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 msgr2=0x55db2900d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.295+0000 7fa6f6922740 1 --2- 172.21.15.22:0/2451275893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db2900d160 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009770 tx=0x7fa6e0009c30 comp rx=0 tx=0).stop 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.296+0000 7fa6f6922740 1 -- 172.21.15.22:0/2451275893 shutdown_connections 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.296+0000 7fa6f6922740 1 --2- 172.21.15.22:0/2451275893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29018670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.296+0000 7fa6f6922740 1 --2- 172.21.15.22:0/2451275893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db29015d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.296+0000 7fa6f6922740 1 --2- 172.21.15.22:0/2451275893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db2900d160 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.296+0000 7fa6f6922740 1 -- 172.21.15.22:0/2451275893 >> 172.21.15.22:0/2451275893 conn(0x55db290074b0 msgr2=0x55db29007890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.297+0000 7fa6f163c640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa6d80a2b00 0x7fa6d80a4fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa6e802c320 tx=0x7fa6e8005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.298+0000 7fa6f6922740 1 -- 172.21.15.22:0/2451275893 shutdown_connections 2024-03-09T00:09:11.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.298+0000 7fa6f6922740 1 -- 172.21.15.22:0/2451275893 wait complete. 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.298+0000 7fa6f6922740 1 -- start start 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.298+0000 7fa6f6922740 1 Processor -- start 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 0x55db29015c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db29012f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29013510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f6922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29019110 con 0x55db2900d6a0 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f6922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db28f2d1c0 con 0x55db29016280 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f6922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db29019290 con 0x55db2900cd80 2024-03-09T00:09:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f6922740 10 client.0 _ll_register_callbacks cb 0x55db28f1c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:11.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db29012f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db29012f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34920/0 (socket says 172.21.15.22:34920) 2024-03-09T00:09:11.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f163c640 1 -- 172.21.15.22:0/1252248493 learned_addr learned my addr 172.21.15.22:0/1252248493 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f263e640 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29013510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.299+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 0x55db29015c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.300+0000 7fa6f163c640 1 -- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 msgr2=0x55db29015c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:11.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.300+0000 7fa6f163c640 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 0x55db29015c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.300+0000 7fa6f163c640 1 -- 172.21.15.22:0/1252248493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 msgr2=0x55db29013510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:11.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.300+0000 7fa6f163c640 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29013510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.300+0000 7fa6f163c640 1 -- 172.21.15.22:0/1252248493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db28dadf70 con 0x55db2900d6a0 2024-03-09T00:09:11.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.301+0000 7fa6f163c640 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db29012f90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8035000 tx=0x7fa6e802bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15748f93592a52cc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.301+0000 7fa6d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.302+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1252248493 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e804d020 con 0x55db2900d6a0 2024-03-09T00:09:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.302+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1252248493 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa6e80035f0 con 0x55db2900d6a0 2024-03-09T00:09:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.302+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6e0011420 con 0x55db2900d6a0 2024-03-09T00:09:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.306+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1252248493 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6e8058050 con 0x55db2900d6a0 2024-03-09T00:09:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.312+0000 7fa6f6922740 10 client.20157 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.312+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55db28c45170 con 0x55db2900d6a0 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.317+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1252248493 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e0011420 con 0x55db2900d6a0 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.317+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1252248493 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa6e8048030 con 0x55db2900d6a0 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.317+0000 7fa6f6922740 10 client.20157 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.317+0000 7fa6f6922740 10 client.20157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.317+0000 7fa6f6922740 20 client.20157 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.318+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db29037470 0x55db29039860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.318+0000 7fa6f6922740 4 client.20157 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:11.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.318+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db28f46960 con 0x55db29037470 2024-03-09T00:09:11.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.319+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db29037470 0x55db29039860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.326+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db29037470 0x55db29039860 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d51e9c4444e3490 server_cookie=7179981a05fd0228 in_seq=0 out_seq=0 2024-03-09T00:09:11.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.326+0000 7fa6d57fa640 10 client.20157 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.336+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1252248493 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db28f46960 con 0x55db29037470 2024-03-09T00:09:11.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.336+0000 7fa6d57fa640 10 client.20157 handle_command_reply: tid=0 2024-03-09T00:09:11.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.336+0000 7fa6f6922740 1 client.20157 shutdown 2024-03-09T00:09:11.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.336+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 msgr2=0x55db29012f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.336+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db29012f90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8035000 tx=0x7fa6e802bd10 comp rx=0 tx=0).stop 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 shutdown_connections 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db29037470 0x55db29039860 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29013510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db29012f90 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1252248493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 0x55db29015c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 >> 172.21.15.22:0/1252248493 conn(0x55db290074b0 msgr2=0x55db29009200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6d57fa640 0 client.20157 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.337+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 shutdown_connections 2024-03-09T00:09:11.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.338+0000 7fa6f6922740 1 -- 172.21.15.22:0/1252248493 wait complete. 2024-03-09T00:09:11.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.338+0000 7fa6f6922740 20 client.20157 trim_cache size 0 max 16384 2024-03-09T00:09:11.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.369+0000 7fa6f6922740 1 Processor -- start 2024-03-09T00:09:11.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.370+0000 7fa6f6922740 1 -- start start 2024-03-09T00:09:11.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.370+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db29009010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.370+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db290156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.370+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29015c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.370+0000 7fa6f6922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00125c0 con 0x55db2900cd80 2024-03-09T00:09:11.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.371+0000 7fa6f6922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0010100 con 0x55db29016280 2024-03-09T00:09:11.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.371+0000 7fa6f6922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000e1c0 con 0x55db2900d6a0 2024-03-09T00:09:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.372+0000 7fa6f163c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db290156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.372+0000 7fa6f163c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db290156c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50700/0 (socket says 172.21.15.22:50700) 2024-03-09T00:09:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.372+0000 7fa6f163c640 1 -- 172.21.15.22:0/3671877646 learned_addr learned my addr 172.21.15.22:0/3671877646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.373+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db29009010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.373+0000 7fa6f163c640 1 -- 172.21.15.22:0/3671877646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 msgr2=0x55db29015c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.373+0000 7fa6f163c640 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29015c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.373+0000 7fa6f163c640 1 -- 172.21.15.22:0/3671877646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 msgr2=0x55db29009010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.373+0000 7fa6f163c640 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db29009010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.373+0000 7fa6f163c640 1 -- 172.21.15.22:0/3671877646 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db29011500 con 0x55db2900d6a0 2024-03-09T00:09:11.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.377+0000 7fa6f163c640 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db290156c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6e802c8c0 tx=0x7fa6e8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577ed92833cbdf8e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.379+0000 7fa6d77fe640 1 -- 172.21.15.22:0/3671877646 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e804d070 con 0x55db2900d6a0 2024-03-09T00:09:11.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.379+0000 7fa6d77fe640 1 -- 172.21.15.22:0/3671877646 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8009450 con 0x55db2900d6a0 2024-03-09T00:09:11.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.379+0000 7fa6d77fe640 1 -- 172.21.15.22:0/3671877646 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8045040 con 0x55db2900d6a0 2024-03-09T00:09:11.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.380+0000 7fa6f6922740 1 -- 172.21.15.22:0/3671877646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 msgr2=0x55db290156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.380+0000 7fa6f6922740 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db290156c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6e802c8c0 tx=0x7fa6e8009260 comp rx=0 tx=0).stop 2024-03-09T00:09:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.383+0000 7fa6f6922740 1 -- 172.21.15.22:0/3671877646 shutdown_connections 2024-03-09T00:09:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.383+0000 7fa6f6922740 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db29015c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.383+0000 7fa6f6922740 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900d6a0 0x55db290156c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.383+0000 7fa6f6922740 1 --2- 172.21.15.22:0/3671877646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900cd80 0x55db29009010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.383+0000 7fa6f6922740 1 -- 172.21.15.22:0/3671877646 >> 172.21.15.22:0/3671877646 conn(0x55db290074b0 msgr2=0x55db29008900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.386+0000 7fa6f6922740 1 -- 172.21.15.22:0/3671877646 shutdown_connections 2024-03-09T00:09:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.386+0000 7fa6f6922740 1 -- 172.21.15.22:0/3671877646 wait complete. 2024-03-09T00:09:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.387+0000 7fa6f6922740 1 -- start start 2024-03-09T00:09:11.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.387+0000 7fa6f6922740 1 Processor -- start 2024-03-09T00:09:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 0x55db2901d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db2901daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f6922740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db2901b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f6922740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000e7c0 con 0x55db2900d6a0 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f6922740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000faa0 con 0x55db29016280 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f6922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000ede0 con 0x55db2900cd80 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f6922740 10 client.0 _ll_register_callbacks cb 0x7fa6e0006f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db2901daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db2901daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34932/0 (socket says 172.21.15.22:34932) 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 -- 172.21.15.22:0/1501748843 learned_addr learned my addr 172.21.15.22:0/1501748843 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 -- 172.21.15.22:0/1501748843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 msgr2=0x55db2901d560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:11.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 0x55db2901d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 -- 172.21.15.22:0/1501748843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 msgr2=0x55db2901b540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:11.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db2901b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.388+0000 7fa6f163c640 1 -- 172.21.15.22:0/1501748843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db2901bd40 con 0x55db2900d6a0 2024-03-09T00:09:11.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.389+0000 7fa6f163c640 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db2901daa0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80096c0 tx=0x7fa6e8004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02f492f2020f67f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.389+0000 7fa6d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.392+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1501748843 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e804d040 con 0x55db2900d6a0 2024-03-09T00:09:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.392+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1501748843 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa6e803f7c0 con 0x55db2900d6a0 2024-03-09T00:09:11.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.392+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6e8009450 con 0x55db2900d6a0 2024-03-09T00:09:11.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.396+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1501748843 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6e8003770 con 0x55db2900d6a0 2024-03-09T00:09:11.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.401+0000 7fa6f6922740 10 client.20163 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:11.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.401+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55db2908f410 con 0x55db2900d6a0 2024-03-09T00:09:11.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.403+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1501748843 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8009450 con 0x55db2900d6a0 2024-03-09T00:09:11.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.403+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1501748843 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa6e8048040 con 0x55db2900d6a0 2024-03-09T00:09:11.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.404+0000 7fa6f6922740 10 client.20163 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:11.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.404+0000 7fa6f6922740 10 client.20163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:11.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.404+0000 7fa6f6922740 20 client.20163 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:11.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.404+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db2908f7d0 0x55db29091bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.404+0000 7fa6f6922740 4 client.20163 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:09:11.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.404+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55db28f32080 con 0x55db2908f7d0 2024-03-09T00:09:11.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.405+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db2908f7d0 0x55db29091bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.414+0000 7fa6f1e3d640 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db2908f7d0 0x55db29091bf0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7693a8f3d9ec7d server_cookie=a4629d4c7b02536c in_seq=0 out_seq=0 2024-03-09T00:09:11.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.415+0000 7fa6d57fa640 10 client.20163 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.432+0000 7fa6d57fa640 1 -- 172.21.15.22:0/1501748843 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55db28f32080 con 0x55db2908f7d0 2024-03-09T00:09:11.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.432+0000 7fa6d57fa640 10 client.20163 handle_command_reply: tid=0 2024-03-09T00:09:11.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.432+0000 7fa6f6922740 1 client.20163 shutdown 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 msgr2=0x55db2901daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db2901daa0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80096c0 tx=0x7fa6e8004120 comp rx=0 tx=0).stop 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 shutdown_connections 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55db2908f7d0 0x55db29091bf0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db29016280 0x55db2901b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55db2900d6a0 0x55db2901daa0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 --2- 172.21.15.22:0/1501748843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db2900cd80 0x55db2901d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.435+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 >> 172.21.15.22:0/1501748843 conn(0x55db290074b0 msgr2=0x55db29005bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.438+0000 7fa6d57fa640 0 client.20163 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.438+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 shutdown_connections 2024-03-09T00:09:11.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.441+0000 7fa6f6922740 1 -- 172.21.15.22:0/1501748843 wait complete. 2024-03-09T00:09:11.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.443+0000 7fa6f6922740 20 client.20163 trim_cache size 0 max 16384 2024-03-09T00:09:11.438 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:09:11.438 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:09:11.438 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "64d87ddf-e319-4cd6-ad1c-90bba8a40a9b", 2024-03-09T00:09:11.438 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:09:11.439 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:09:11.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa6d80a2b00 msgr2=0x7fa6d80a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa6d80a2b00 0x7fa6d80a4fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa6e802c320 tx=0x7fa6e8005fd0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec112290 msgr2=0x7fa6ec06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec112290 0x7fa6ec06bc90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e9d0 tx=0x7fa6e400ee90 comp rx=0 tx=0).stop 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 shutdown_connections 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa6d80a2b00 0x7fa6d80a4fc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec112290 0x7fa6ec06bc90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6ec073fb0 0x7fa6ec072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 --2- 172.21.15.22:0/3116544829 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6ec073690 0x7fa6ec072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 >> 172.21.15.22:0/3116544829 conn(0x7fa6ec0786b0 msgr2=0x7fa6ec10dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 shutdown_connections 2024-03-09T00:09:11.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.445+0000 7fa6f40a1640 1 -- 172.21.15.22:0/3116544829 wait complete. 2024-03-09T00:09:11.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:09:11.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:09:11.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92c789a640 1 -- 172.21.15.22:0/1882955184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c011aad0 msgr2=0x7f92c011cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92b7fff640 1 -- 172.21.15.22:0/1882955184 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc043880 con 0x7f92c011aad0 2024-03-09T00:09:11.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92c789a640 1 --2- 172.21.15.22:0/1882955184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c011aad0 0x7f92c011cec0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f92bc01acb0 tx=0x7f92bc03d150 comp rx=0 tx=0).stop 2024-03-09T00:09:11.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92c789a640 1 -- 172.21.15.22:0/1882955184 shutdown_connections 2024-03-09T00:09:11.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92c789a640 1 --2- 172.21.15.22:0/1882955184 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c011aad0 0x7f92c011cec0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92c789a640 1 --2- 172.21.15.22:0/1882955184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92c010db40 0x7f92c010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92c789a640 1 --2- 172.21.15.22:0/1882955184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c010fb10 0x7f92c010d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.651+0000 7f92c789a640 1 -- 172.21.15.22:0/1882955184 >> 172.21.15.22:0/1882955184 conn(0x7f92c006c3d0 msgr2=0x7f92c006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.659+0000 7f92c789a640 1 -- 172.21.15.22:0/1882955184 shutdown_connections 2024-03-09T00:09:11.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.660+0000 7f92c789a640 1 -- 172.21.15.22:0/1882955184 wait complete. 2024-03-09T00:09:11.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.660+0000 7f92c789a640 1 Processor -- start 2024-03-09T00:09:11.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.663+0000 7f92c789a640 1 -- start start 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.663+0000 7f92c789a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c010db40 0x7f92c011a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.663+0000 7f92c789a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92c010fb10 0x7f92c01161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.663+0000 7f92c789a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c011aad0 0x7f92c0116720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.663+0000 7f92c789a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c0112cb0 con 0x7f92c010db40 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.663+0000 7f92c789a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c0112b30 con 0x7f92c010fb10 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.663+0000 7f92c789a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c0112e30 con 0x7f92c011aad0 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.664+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c011aad0 0x7f92c0116720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.664+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c011aad0 0x7f92c0116720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50730/0 (socket says 172.21.15.22:50730) 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.664+0000 7f92c5e37640 1 -- 172.21.15.22:0/2043322409 learned_addr learned my addr 172.21.15.22:0/2043322409 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.664+0000 7f92c4e35640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92c010fb10 0x7f92c01161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.665+0000 7f92c5e37640 1 -- 172.21.15.22:0/2043322409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92c010fb10 msgr2=0x7f92c01161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.665+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92c010fb10 0x7f92c01161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.665+0000 7f92c5e37640 1 -- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c010db40 msgr2=0x7f92c011a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.665+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c010db40 0x7f92c011a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.665+0000 7f92c5e37640 1 -- 172.21.15.22:0/2043322409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c0116e50 con 0x7f92c011aad0 2024-03-09T00:09:11.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.668+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c011aad0 0x7f92c0116720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f92bc003ba0 tx=0x7f92bc003bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.669+0000 7f92b67fc640 1 -- 172.21.15.22:0/2043322409 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc04a040 con 0x7f92c011aad0 2024-03-09T00:09:11.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.670+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c01aa840 con 0x7f92c011aad0 2024-03-09T00:09:11.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.670+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c01aabf0 con 0x7f92c011aad0 2024-03-09T00:09:11.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.670+0000 7f92b67fc640 1 -- 172.21.15.22:0/2043322409 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92bc003cf0 con 0x7f92c011aad0 2024-03-09T00:09:11.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.670+0000 7f92b67fc640 1 -- 172.21.15.22:0/2043322409 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc045900 con 0x7f92c011aad0 2024-03-09T00:09:11.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.674+0000 7f92ca481740 1 Processor -- start 2024-03-09T00:09:11.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.682+0000 7f92b67fc640 1 -- 172.21.15.22:0/2043322409 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f92bc004040 con 0x7f92c011aad0 2024-03-09T00:09:11.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.683+0000 7f92b67fc640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f92ac0a2b80 0x7f92ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.683+0000 7f92b67fc640 1 -- 172.21.15.22:0/2043322409 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f92bc102510 con 0x7f92c011aad0 2024-03-09T00:09:11.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92ca481740 1 -- start start 2024-03-09T00:09:11.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92ca481740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b762760 0x55b52b762b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92ca481740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b763080 0x55b52b76b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92ca481740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b76e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92ca481740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52b6c5e00 con 0x55b52b76bc60 2024-03-09T00:09:11.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92ca481740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52b66eb10 con 0x55b52b762760 2024-03-09T00:09:11.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92ca481740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52b6c1ea0 con 0x55b52b763080 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b76e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b76e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34970/0 (socket says 172.21.15.22:34970) 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.684+0000 7f92c5e37640 1 -- 172.21.15.22:0/2953909108 learned_addr learned my addr 172.21.15.22:0/2953909108 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.685+0000 7f92c4e35640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f92ac0a2b80 0x7f92ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.685+0000 7f92c5e37640 1 -- 172.21.15.22:0/2953909108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b763080 msgr2=0x55b52b76b720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.685+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2953909108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b763080 0x55b52b76b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.685+0000 7f92c5e37640 1 -- 172.21.15.22:0/2953909108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b762760 msgr2=0x55b52b762b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:11.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.685+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2953909108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b762760 0x55b52b762b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.685+0000 7f92c5e37640 1 -- 172.21.15.22:0/2953909108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b52b4a47d0 con 0x55b52b76bc60 2024-03-09T00:09:11.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.686+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2953909108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b76e050 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0d6a40 tx=0x7f92bc0ce1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13fa0accba0c29b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.686+0000 7f92ab7fe640 1 -- 172.21.15.22:0/2953909108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0e7070 con 0x55b52b76bc60 2024-03-09T00:09:11.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.686+0000 7f92ab7fe640 1 -- 172.21.15.22:0/2953909108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92bc0ce860 con 0x55b52b76bc60 2024-03-09T00:09:11.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.687+0000 7f92ca481740 1 -- 172.21.15.22:0/2953909108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 msgr2=0x55b52b76e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.687+0000 7f92ca481740 1 --2- 172.21.15.22:0/2953909108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b76e050 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0d6a40 tx=0x7f92bc0ce1c0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.687+0000 7f92c4e35640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f92ac0a2b80 0x7f92ac0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55b52b762230 tx=0x7f92b8008040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.690+0000 7f92ca481740 1 -- 172.21.15.22:0/2953909108 shutdown_connections 2024-03-09T00:09:11.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.690+0000 7f92ca481740 1 --2- 172.21.15.22:0/2953909108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b76e050 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.690+0000 7f92ca481740 1 --2- 172.21.15.22:0/2953909108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b763080 0x55b52b76b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.690+0000 7f92ca481740 1 --2- 172.21.15.22:0/2953909108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b762760 0x55b52b762b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.690+0000 7f92ca481740 1 -- 172.21.15.22:0/2953909108 >> 172.21.15.22:0/2953909108 conn(0x55b52b75ce90 msgr2=0x55b52b75d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.694+0000 7f92ca481740 1 -- 172.21.15.22:0/2953909108 shutdown_connections 2024-03-09T00:09:11.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.695+0000 7f92ca481740 1 -- 172.21.15.22:0/2953909108 wait complete. 2024-03-09T00:09:11.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.695+0000 7f92ca481740 1 -- start start 2024-03-09T00:09:11.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.695+0000 7f92ca481740 1 Processor -- start 2024-03-09T00:09:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.696+0000 7f92ca481740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b762760 0x55b52b76b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.696+0000 7f92ca481740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b768650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.696+0000 7f92ca481740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b768b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.696+0000 7f92ca481740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52b76e810 con 0x55b52b76bc60 2024-03-09T00:09:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.696+0000 7f92ca481740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52b6820a0 con 0x55b52b763080 2024-03-09T00:09:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.696+0000 7f92ca481740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52b76e990 con 0x55b52b762760 2024-03-09T00:09:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.696+0000 7f92ca481740 10 client.0 _ll_register_callbacks cb 0x55b52b681a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b768b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b768b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34980/0 (socket says 172.21.15.22:34980) 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 -- 172.21.15.22:0/2805546407 learned_addr learned my addr 172.21.15.22:0/2805546407 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5636640 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b762760 0x55b52b76b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c4e35640 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b768650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 -- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b762760 msgr2=0x55b52b76b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b762760 0x55b52b76b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 -- 172.21.15.22:0/2805546407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 msgr2=0x55b52b768650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b768650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.697+0000 7f92c5e37640 1 -- 172.21.15.22:0/2805546407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b52b3da3f0 con 0x55b52b76bc60 2024-03-09T00:09:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.698+0000 7f92c5e37640 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b768b90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0cb230 tx=0x7f92bc003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78f62e94f139202 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.698+0000 7f92a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.698+0000 7f92a8ff9640 1 -- 172.21.15.22:0/2805546407 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0e7040 con 0x55b52b76bc60 2024-03-09T00:09:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.699+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92bc0ce860 con 0x55b52b76bc60 2024-03-09T00:09:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.699+0000 7f92a8ff9640 1 -- 172.21.15.22:0/2805546407 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f92bc0ffa30 con 0x55b52b76bc60 2024-03-09T00:09:11.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.699+0000 7f92a8ff9640 1 -- 172.21.15.22:0/2805546407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92bc0e0050 con 0x55b52b76bc60 2024-03-09T00:09:11.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.699+0000 7f92ca481740 10 client.20172 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:11.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.699+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b52b39a0f0 con 0x55b52b76bc60 2024-03-09T00:09:11.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.700+0000 7f92a8ff9640 1 -- 172.21.15.22:0/2805546407 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92bc0ce860 con 0x55b52b76bc60 2024-03-09T00:09:11.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.701+0000 7f92a8ff9640 1 -- 172.21.15.22:0/2805546407 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f92bc0db080 con 0x55b52b76bc60 2024-03-09T00:09:11.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.701+0000 7f92ca481740 10 client.20172 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:11.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.701+0000 7f92ca481740 10 client.20172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:11.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.701+0000 7f92ca481740 20 client.20172 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:11.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.701+0000 7f92ca481740 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b77b4e0 0x55b52b77d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.701+0000 7f92ca481740 4 client.20172 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:11.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.701+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b52b6d3340 con 0x55b52b77b4e0 2024-03-09T00:09:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.706+0000 7f92c5636640 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b77b4e0 0x55b52b77d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.710+0000 7f92c5636640 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b77b4e0 0x55b52b77d900 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f863535820603201 server_cookie=8e852b5bd7b0fa54 in_seq=0 out_seq=0 2024-03-09T00:09:11.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.710+0000 7f92a8ff9640 10 client.20172 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.716+0000 7f92a8ff9640 1 -- 172.21.15.22:0/2805546407 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b52b6d3340 con 0x55b52b77b4e0 2024-03-09T00:09:11.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.716+0000 7f92a8ff9640 10 client.20172 handle_command_reply: tid=0 2024-03-09T00:09:11.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.718+0000 7f92ca481740 1 client.20172 shutdown 2024-03-09T00:09:11.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.724+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 msgr2=0x55b52b768b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.724+0000 7f92ca481740 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b768b90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0cb230 tx=0x7f92bc003ea0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.728+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 shutdown_connections 2024-03-09T00:09:11.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.728+0000 7f92ca481740 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b77b4e0 0x55b52b77d900 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.728+0000 7f92ca481740 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b76bc60 0x55b52b768b90 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.728+0000 7f92ca481740 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b768650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.728+0000 7f92ca481740 1 --2- 172.21.15.22:0/2805546407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b762760 0x55b52b76b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.728+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 >> 172.21.15.22:0/2805546407 conn(0x55b52b75ce90 msgr2=0x55b52b75ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.728+0000 7f92a8ff9640 0 client.20172 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.729+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 shutdown_connections 2024-03-09T00:09:11.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.729+0000 7f92ca481740 1 -- 172.21.15.22:0/2805546407 wait complete. 2024-03-09T00:09:11.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.731+0000 7f92ca481740 20 client.20172 trim_cache size 0 max 16384 2024-03-09T00:09:11.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.762+0000 7f92ca481740 1 Processor -- start 2024-03-09T00:09:11.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.762+0000 7f92ca481740 1 -- start start 2024-03-09T00:09:11.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92ca481740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b76ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92ca481740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b76b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92ca481740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b772c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92ca481740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40123f0 con 0x55b52b762760 2024-03-09T00:09:11.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92ca481740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a400ff30 con 0x55b52b763080 2024-03-09T00:09:11.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92ca481740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a400e080 con 0x55b52b76bc60 2024-03-09T00:09:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b772c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92c5e37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b772c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50768/0 (socket says 172.21.15.22:50768) 2024-03-09T00:09:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.763+0000 7f92c5e37640 1 -- 172.21.15.22:0/3232211082 learned_addr learned my addr 172.21.15.22:0/3232211082 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.765+0000 7f92c5e37640 1 -- 172.21.15.22:0/3232211082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 msgr2=0x55b52b76b2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:11.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.765+0000 7f92c5e37640 1 --2- 172.21.15.22:0/3232211082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b76b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.765+0000 7f92c5e37640 1 -- 172.21.15.22:0/3232211082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 msgr2=0x55b52b76ad80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.765+0000 7f92c5e37640 1 --2- 172.21.15.22:0/3232211082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b76ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.765+0000 7f92c5e37640 1 -- 172.21.15.22:0/3232211082 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b52b767590 con 0x55b52b76bc60 2024-03-09T00:09:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.765+0000 7f92c5e37640 1 --2- 172.21.15.22:0/3232211082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b772c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92bc03d730 tx=0x7f92bc003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a668132d5c0378a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.766+0000 7f92aaffd640 1 -- 172.21.15.22:0/3232211082 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0e7040 con 0x55b52b76bc60 2024-03-09T00:09:11.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.766+0000 7f92aaffd640 1 -- 172.21.15.22:0/3232211082 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92bc0ec070 con 0x55b52b76bc60 2024-03-09T00:09:11.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.766+0000 7f92aaffd640 1 -- 172.21.15.22:0/3232211082 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0d6ae0 con 0x55b52b76bc60 2024-03-09T00:09:11.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 -- 172.21.15.22:0/3232211082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 msgr2=0x55b52b772c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 --2- 172.21.15.22:0/3232211082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b772c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92bc03d730 tx=0x7f92bc003f10 comp rx=0 tx=0).stop 2024-03-09T00:09:11.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 -- 172.21.15.22:0/3232211082 shutdown_connections 2024-03-09T00:09:11.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 --2- 172.21.15.22:0/3232211082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b772c90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 --2- 172.21.15.22:0/3232211082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b76b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 --2- 172.21.15.22:0/3232211082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b76ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 -- 172.21.15.22:0/3232211082 >> 172.21.15.22:0/3232211082 conn(0x55b52b75ce90 msgr2=0x55b52b782240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 -- 172.21.15.22:0/3232211082 shutdown_connections 2024-03-09T00:09:11.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.767+0000 7f92ca481740 1 -- 172.21.15.22:0/3232211082 wait complete. 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 -- start start 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 Processor -- start 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b772ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b7731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b7648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a400e5f0 con 0x55b52b762760 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a400f8d0 con 0x55b52b763080 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a400ec10 con 0x55b52b76bc60 2024-03-09T00:09:11.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.768+0000 7f92ca481740 10 client.0 _ll_register_callbacks cb 0x7f92bc0e5070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.769+0000 7f92c5636640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b772ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.769+0000 7f92c5636640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b772ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34986/0 (socket says 172.21.15.22:34986) 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.769+0000 7f92c5636640 1 -- 172.21.15.22:0/4186955531 learned_addr learned my addr 172.21.15.22:0/4186955531 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.770+0000 7f92c4e35640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b7731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.770+0000 7f92c5636640 1 -- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 msgr2=0x55b52b7648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.770+0000 7f92c5e37640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b7648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.770+0000 7f92c5636640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b7648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.770+0000 7f92c5636640 1 -- 172.21.15.22:0/4186955531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 msgr2=0x55b52b7731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:11.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.770+0000 7f92c5636640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b7731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.770+0000 7f92c5636640 1 -- 172.21.15.22:0/4186955531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b52b764eb0 con 0x55b52b762760 2024-03-09T00:09:11.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.771+0000 7f92c4e35640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b7731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:09:11.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.771+0000 7f92c5e37640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b7648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:09:11.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.777+0000 7f92c5636640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b772ca0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f92b000bd50 tx=0x7f92b000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552e34df446b0514 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:11.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.778+0000 7f92a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.779+0000 7f92a8ff9640 1 -- 172.21.15.22:0/4186955531 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b001a070 con 0x55b52b762760 2024-03-09T00:09:11.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.779+0000 7f92a8ff9640 1 -- 172.21.15.22:0/4186955531 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f92b00162c0 con 0x55b52b762760 2024-03-09T00:09:11.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.781+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92bc0ec070 con 0x55b52b762760 2024-03-09T00:09:11.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.787+0000 7f92a8ff9640 1 -- 172.21.15.22:0/4186955531 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92b0024050 con 0x55b52b762760 2024-03-09T00:09:11.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.789+0000 7f92ca481740 10 client.20175 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:11.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.789+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b52b7d2310 con 0x55b52b762760 2024-03-09T00:09:11.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92a8ff9640 1 -- 172.21.15.22:0/4186955531 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92bc0ec070 con 0x55b52b762760 2024-03-09T00:09:11.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92a8ff9640 1 -- 172.21.15.22:0/4186955531 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f92b001fac0 con 0x55b52b762760 2024-03-09T00:09:11.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92ca481740 10 client.20175 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:11.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92ca481740 10 client.20175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:11.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92ca481740 20 client.20175 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:11.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92ca481740 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b7d2ad0 0x55b52b7d4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:11.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92ca481740 4 client.20175 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:09:11.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.790+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b52b6cd690 con 0x55b52b7d2ad0 2024-03-09T00:09:11.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.794+0000 7f92c4e35640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b7d2ad0 0x55b52b7d4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.804+0000 7f92c4e35640 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b7d2ad0 0x55b52b7d4ef0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed83355fcc415384 server_cookie=84fad150499ec512 in_seq=0 out_seq=0 2024-03-09T00:09:11.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.804+0000 7f92a8ff9640 10 client.20175 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.809+0000 7f92a8ff9640 1 -- 172.21.15.22:0/4186955531 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55b52b6cd690 con 0x55b52b7d2ad0 2024-03-09T00:09:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.809+0000 7f92a8ff9640 10 client.20175 handle_command_reply: tid=0 2024-03-09T00:09:11.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.810+0000 7f92ca481740 1 client.20175 shutdown 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 msgr2=0x55b52b772ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b772ca0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f92b000bd50 tx=0x7f92b000f7f0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 shutdown_connections 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55b52b7d2ad0 0x55b52b7d4ef0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b52b76bc60 0x55b52b7648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b52b763080 0x55b52b7731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 --2- 172.21.15.22:0/4186955531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b52b762760 0x55b52b772ca0 secure :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f92b000bd50 tx=0x7f92b000f7f0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 >> 172.21.15.22:0/4186955531 conn(0x55b52b75ce90 msgr2=0x55b52b7635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92a8ff9640 0 client.20175 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.814+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 shutdown_connections 2024-03-09T00:09:11.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.815+0000 7f92ca481740 1 -- 172.21.15.22:0/4186955531 wait complete. 2024-03-09T00:09:11.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.817+0000 7f92ca481740 20 client.20175 trim_cache size 0 max 16384 2024-03-09T00:09:11.815 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:09:11.815 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (563 inodes in the stack)", 2024-03-09T00:09:11.815 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:09:11.815 INFO:teuthology.orchestra.run.smithi022.stdout: "64d87ddf-e319-4cd6-ad1c-90bba8a40a9b": { 2024-03-09T00:09:11.815 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:09:11.815 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "64d87ddf-e319-4cd6-ad1c-90bba8a40a9b", 2024-03-09T00:09:11.816 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:09:11.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:09:11.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:09:11.816 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:09:11.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.821+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f92ac0a2b80 msgr2=0x7f92ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.821+0000 7f92c789a640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f92ac0a2b80 0x7f92ac0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55b52b762230 tx=0x7f92b8008040 comp rx=0 tx=0).stop 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.821+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c011aad0 msgr2=0x7f92c0116720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.821+0000 7f92c789a640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c011aad0 0x7f92c0116720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f92bc003ba0 tx=0x7f92bc003bd0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.822+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 shutdown_connections 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.822+0000 7f92c789a640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f92ac0a2b80 0x7f92ac0a5040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.822+0000 7f92c789a640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92c011aad0 0x7f92c0116720 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.822+0000 7f92c789a640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92c010fb10 0x7f92c01161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.822+0000 7f92c789a640 1 --2- 172.21.15.22:0/2043322409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92c010db40 0x7f92c011a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:11.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.822+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 >> 172.21.15.22:0/2043322409 conn(0x7f92c006c3d0 msgr2=0x7f92c011c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:11.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.823+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 shutdown_connections 2024-03-09T00:09:11.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:11.823+0000 7f92c789a640 1 -- 172.21.15.22:0/2043322409 wait complete. 2024-03-09T00:09:11.836 INFO:tasks.cephfs.filesystem:scrub status for tag:64d87ddf-e319-4cd6-ad1c-90bba8a40a9b - {'path': '/', 'tag': '64d87ddf-e319-4cd6-ad1c-90bba8a40a9b', 'options': 'recursive,force'} 2024-03-09T00:09:12.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 17943 14.46 MB/sec warmup 79 sec latency 153.482 ms 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: pgmap v463: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 296 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 548 op/s 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:12 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: pgmap v463: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 296 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 548 op/s 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:12 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: pgmap v463: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 296 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 548 op/s 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:12.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:12 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:13.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 18087 14.33 MB/sec warmup 80 sec latency 121.327 ms 2024-03-09T00:09:13.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:13 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:13.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:13 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:13.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:13 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.949+0000 7fb6dccab640 1 -- 172.21.15.22:0/811526274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6d8101eb0 msgr2=0x7fb6d8105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.949+0000 7fb6dccab640 1 --2- 172.21.15.22:0/811526274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6d8101eb0 0x7fb6d8105ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009be0 tx=0x7fb6d002bf10 comp rx=0 tx=0).stop 2024-03-09T00:09:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.949+0000 7fb6dccab640 1 -- 172.21.15.22:0/811526274 shutdown_connections 2024-03-09T00:09:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.949+0000 7fb6dccab640 1 --2- 172.21.15.22:0/811526274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 0x7fb6d810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.949+0000 7fb6dccab640 1 --2- 172.21.15.22:0/811526274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6d8101eb0 0x7fb6d8105ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.949+0000 7fb6dccab640 1 --2- 172.21.15.22:0/811526274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d8101500 0x7fb6d81018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.949+0000 7fb6dccab640 1 -- 172.21.15.22:0/811526274 >> 172.21.15.22:0/811526274 conn(0x7fb6d80fcd80 msgr2=0x7fb6d80ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.953+0000 7fb6dccab640 1 -- 172.21.15.22:0/811526274 shutdown_connections 2024-03-09T00:09:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.953+0000 7fb6dccab640 1 -- 172.21.15.22:0/811526274 wait complete. 2024-03-09T00:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.954+0000 7fb6dccab640 1 Processor -- start 2024-03-09T00:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.954+0000 7fb6dccab640 1 -- start start 2024-03-09T00:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6dccab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6d8101500 0x7fb6d819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6dccab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 0x7fb6d81a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6dccab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d810d5d0 0x7fb6d810da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6dccab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d81116e0 con 0x7fb6d81065d0 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6dccab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8111560 con 0x7fb6d8101500 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6dccab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8111860 con 0x7fb6d810d5d0 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6d5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 0x7fb6d81a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6d6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d810d5d0 0x7fb6d810da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6d5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 0x7fb6d81a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35010/0 (socket says 172.21.15.22:35010) 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6d6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d810d5d0 0x7fb6d810da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50796/0 (socket says 172.21.15.22:50796) 2024-03-09T00:09:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.955+0000 7fb6d5d9b640 1 -- 172.21.15.22:0/958352299 learned_addr learned my addr 172.21.15.22:0/958352299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6d5d9b640 1 -- 172.21.15.22:0/958352299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d810d5d0 msgr2=0x7fb6d810da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d810d5d0 0x7fb6d810da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6d5d9b640 1 -- 172.21.15.22:0/958352299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6d8101500 msgr2=0x7fb6d819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6d8101500 0x7fb6d819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6d5d9b640 1 -- 172.21.15.22:0/958352299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d810e1b0 con 0x7fb6d81065d0 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6d6d9d640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d810d5d0 0x7fb6d810da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 0x7fb6d81a0220 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0033420 tx=0x7fb6d0033750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6c77fe640 1 -- 172.21.15.22:0/958352299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0043020 con 0x7fb6d81065d0 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d81a6b30 con 0x7fb6d81065d0 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d81a6ee0 con 0x7fb6d81065d0 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.956+0000 7fb6c77fe640 1 -- 172.21.15.22:0/958352299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6d0033b50 con 0x7fb6d81065d0 2024-03-09T00:09:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.957+0000 7fb6c77fe640 1 -- 172.21.15.22:0/958352299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0043020 con 0x7fb6d81065d0 2024-03-09T00:09:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.961+0000 7fb6c77fe640 1 -- 172.21.15.22:0/958352299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb6d0046020 con 0x7fb6d81065d0 2024-03-09T00:09:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 Processor -- start 2024-03-09T00:09:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 -- start start 2024-03-09T00:09:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465765a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6c77fe640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6c00a2ea0 0x7fb6c00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465766450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465780560 0x562465782950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624656ca5f0 con 0x562465766d40 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562465673930 con 0x562465780560 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.962+0000 7fb6df96f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624656c6690 con 0x562465765fd0 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6c77fe640 1 -- 172.21.15.22:0/958352299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fb6d003a040 con 0x7fb6d81065d0 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465765a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465765a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35014/0 (socket says 172.21.15.22:35014) 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 -- 172.21.15.22:0/1996149782 learned_addr learned my addr 172.21.15.22:0/1996149782 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465766450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 -- 172.21.15.22:0/1996149782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 msgr2=0x562465766450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465766450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 -- 172.21.15.22:0/1996149782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465780560 msgr2=0x562465782950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465780560 0x562465782950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.963+0000 7fb6d6d9d640 1 -- 172.21.15.22:0/1996149782 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624653df210 con 0x562465766d40 2024-03-09T00:09:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.964+0000 7fb6d659c640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6c00a2ea0 0x7fb6c00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.964+0000 7fb6d6d9d640 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465765a90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800bfc0 tx=0x7fb6c8012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d7e7b39ffc5c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.965+0000 7fb6c4ff9640 1 -- 172.21.15.22:0/1996149782 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8005220 con 0x562465766d40 2024-03-09T00:09:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.965+0000 7fb6c4ff9640 1 -- 172.21.15.22:0/1996149782 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6c8004c80 con 0x562465766d40 2024-03-09T00:09:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.965+0000 7fb6c4ff9640 1 -- 172.21.15.22:0/1996149782 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8010b50 con 0x562465766d40 2024-03-09T00:09:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 -- 172.21.15.22:0/1996149782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 msgr2=0x562465765a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465765a90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800bfc0 tx=0x7fb6c8012980 comp rx=0 tx=0).stop 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6d659c640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6c00a2ea0 0x7fb6c00a5360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0096f0 tx=0x7fb6cc006c40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 -- 172.21.15.22:0/1996149782 shutdown_connections 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465780560 0x562465782950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465766450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1996149782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465765a90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 -- 172.21.15.22:0/1996149782 >> 172.21.15.22:0/1996149782 conn(0x56246572d5b0 msgr2=0x562465760cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 -- 172.21.15.22:0/1996149782 shutdown_connections 2024-03-09T00:09:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 -- 172.21.15.22:0/1996149782 wait complete. 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.966+0000 7fb6df96f740 1 -- start start 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 1 Processor -- start 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465765fd0 0x562465768500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x562465768a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 0x562465768f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562465783230 con 0x562465765fd0 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562465686d90 con 0x562465766d40 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624657833b0 con 0x562465780560 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6df96f740 10 client.0 _ll_register_callbacks cb 0x5624653dea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6d659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465765fd0 0x562465768500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6d659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465765fd0 0x562465768500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35026/0 (socket says 172.21.15.22:35026) 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6d659c640 1 -- 172.21.15.22:0/1859713964 learned_addr learned my addr 172.21.15.22:0/1859713964 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.967+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x562465768a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6d659c640 1 -- 172.21.15.22:0/1859713964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 msgr2=0x562465768f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6d659c640 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 0x562465768f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6d659c640 1 -- 172.21.15.22:0/1859713964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 msgr2=0x562465768a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6d659c640 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x562465768a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6d659c640 1 -- 172.21.15.22:0/1859713964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624654ab620 con 0x562465765fd0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6d659c640 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465765fd0 0x562465768500 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00b430 tx=0x7fb6cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f620b2c9e27f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6bdffb640 1 -- 172.21.15.22:0/1859713964 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc023060 con 0x562465765fd0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6bdffb640 1 -- 172.21.15.22:0/1859713964 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fb6cc0150d0 con 0x562465765fd0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.968+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6c8004c80 con 0x562465765fd0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6bdffb640 1 -- 172.21.15.22:0/1859713964 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6cc02a050 con 0x562465765fd0 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 10 client.20187 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56246578f570 con 0x562465765fd0 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6bdffb640 1 -- 172.21.15.22:0/1859713964 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6c8004c80 con 0x562465765fd0 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6bdffb640 1 -- 172.21.15.22:0/1859713964 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb6cc012070 con 0x562465765fd0 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 10 client.20187 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 10 client.20187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 20 client.20187 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562465790ab0 0x562465792f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 4 client.20187 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624656d7b30 con 0x562465790ab0 2024-03-09T00:09:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.969+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562465790ab0 0x562465792f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.970+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562465790ab0 0x562465792f30 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e60bdc2c914115e server_cookie=146af8f0244f1bf8 in_seq=0 out_seq=0 2024-03-09T00:09:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.970+0000 7fb6bdffb640 10 client.20187 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.974+0000 7fb6bdffb640 1 -- 172.21.15.22:0/1859713964 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5624656d7b30 con 0x562465790ab0 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.974+0000 7fb6bdffb640 10 client.20187 handle_command_reply: tid=0 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.974+0000 7fb6df96f740 1 client.20187 shutdown 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465765fd0 msgr2=0x562465768500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465765fd0 0x562465768500 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00b430 tx=0x7fb6cc004770 comp rx=0 tx=0).stop 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 shutdown_connections 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562465790ab0 0x562465792f30 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 0x562465768f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x562465768a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1859713964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465765fd0 0x562465768500 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 >> 172.21.15.22:0/1859713964 conn(0x56246572d5b0 msgr2=0x562465762a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6bdffb640 0 client.20187 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 shutdown_connections 2024-03-09T00:09:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 1 -- 172.21.15.22:0/1859713964 wait complete. 2024-03-09T00:09:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.975+0000 7fb6df96f740 20 client.20187 trim_cache size 0 max 16384 2024-03-09T00:09:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.994+0000 7fb6df96f740 1 Processor -- start 2024-03-09T00:09:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.994+0000 7fb6df96f740 1 -- start start 2024-03-09T00:09:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465797890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x56246577c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465797890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465797890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:50824/0 (socket says 172.21.15.22:50824) 2024-03-09T00:09:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465780560 0x56246577cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x56246577c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x56246577c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55094/0 (socket says 172.21.15.22:55094) 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d659c640 1 -- 172.21.15.22:0/1389773860 learned_addr learned my addr 172.21.15.22:0/1389773860 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80124b0 con 0x562465780560 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b800fff0 con 0x562465766d40 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b800e0b0 con 0x562465765fd0 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d6d9d640 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465780560 0x56246577cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d5d9b640 1 -- 172.21.15.22:0/1389773860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 msgr2=0x562465797890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.995+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465797890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.996+0000 7fb6d5d9b640 1 -- 172.21.15.22:0/1389773860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465780560 msgr2=0x56246577cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.996+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465780560 0x56246577cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.996+0000 7fb6d5d9b640 1 -- 172.21.15.22:0/1389773860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624657652a0 con 0x562465766d40 2024-03-09T00:09:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.996+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x56246577c840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00ef590 tx=0x7fb6d00de040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bd2a5ec9605aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.996+0000 7fb6bffff640 1 -- 172.21.15.22:0/1389773860 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0043040 con 0x562465766d40 2024-03-09T00:09:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.996+0000 7fb6bffff640 1 -- 172.21.15.22:0/1389773860 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6d00beba0 con 0x562465766d40 2024-03-09T00:09:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.996+0000 7fb6bffff640 1 -- 172.21.15.22:0/1389773860 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0033d00 con 0x562465766d40 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 msgr2=0x56246577c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x56246577c840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00ef590 tx=0x7fb6d00de040 comp rx=0 tx=0).stop 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 shutdown_connections 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465780560 0x56246577cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465766d40 0x56246577c840 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 --2- 172.21.15.22:0/1389773860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465765fd0 0x562465797890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 >> 172.21.15.22:0/1389773860 conn(0x56246572d5b0 msgr2=0x562465767a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 shutdown_connections 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.997+0000 7fb6df96f740 1 -- 172.21.15.22:0/1389773860 wait complete. 2024-03-09T00:09:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 -- start start 2024-03-09T00:09:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 Processor -- start 2024-03-09T00:09:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465765fd0 0x562465787440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465787980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 0x562465785890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b800e6b0 con 0x562465766d40 2024-03-09T00:09:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b800f990 con 0x562465765fd0 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b800ecd0 con 0x562465780560 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6df96f740 10 client.0 _ll_register_callbacks cb 0x7fb6c800bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6d659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465765fd0 0x562465787440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6d659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465765fd0 0x562465787440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55102/0 (socket says 172.21.15.22:55102) 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.998+0000 7fb6d659c640 1 -- 172.21.15.22:0/2044139132 learned_addr learned my addr 172.21.15.22:0/2044139132 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6d6d9d640 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 0x562465785890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6d659c640 1 -- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 msgr2=0x562465785890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6d659c640 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 0x562465785890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6d659c640 1 -- 172.21.15.22:0/2044139132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 msgr2=0x562465787980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6d659c640 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465787980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6d659c640 1 -- 172.21.15.22:0/2044139132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562465786070 con 0x562465765fd0 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6d659c640 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465765fd0 0x562465787440 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc012550 tx=0x7fb6cc00b160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898792582115ae0d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:09:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:13.999+0000 7fb6bdffb640 1 -- 172.21.15.22:0/2044139132 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc056070 con 0x562465765fd0 2024-03-09T00:09:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.000+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6d00beba0 con 0x562465765fd0 2024-03-09T00:09:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.000+0000 7fb6bdffb640 1 -- 172.21.15.22:0/2044139132 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fb6cc012070 con 0x562465765fd0 2024-03-09T00:09:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.000+0000 7fb6bdffb640 1 -- 172.21.15.22:0/2044139132 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6cc0122b0 con 0x562465765fd0 2024-03-09T00:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.000+0000 7fb6df96f740 10 client.29411 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56246576b530 con 0x562465765fd0 2024-03-09T00:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6bdffb640 1 -- 172.21.15.22:0/2044139132 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6cc0152e0 con 0x562465765fd0 2024-03-09T00:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6bdffb640 1 -- 172.21.15.22:0/2044139132 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb6cc059030 con 0x562465765fd0 2024-03-09T00:09:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6df96f740 10 client.29411 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6df96f740 10 client.29411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6df96f740 20 client.29411 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6df96f740 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56246576bcf0 0x56246576e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6df96f740 4 client.29411 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.001+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fb6d00beba0 con 0x56246576bcf0 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.002+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56246576bcf0 0x56246576e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.003+0000 7fb6d5d9b640 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56246576bcf0 0x56246576e1b0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446a87efca5c2681 server_cookie=6147a11046216aa7 in_seq=0 out_seq=0 2024-03-09T00:09:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.003+0000 7fb6bdffb640 10 client.29411 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:14.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 18363 14.30 MB/sec warmup 81 sec latency 114.895 ms 2024-03-09T00:09:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6bdffb640 1 -- 172.21.15.22:0/2044139132 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fb6d00beba0 con 0x56246576bcf0 2024-03-09T00:09:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6bdffb640 10 client.29411 handle_command_reply: tid=0 2024-03-09T00:09:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 client.29411 shutdown 2024-03-09T00:09:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465765fd0 msgr2=0x562465787440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465765fd0 0x562465787440 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc012550 tx=0x7fb6cc00b160 comp rx=0 tx=0).stop 2024-03-09T00:09:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 shutdown_connections 2024-03-09T00:09:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56246576bcf0 0x56246576e1b0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6bdffb640 0 client.29411 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562465780560 0x562465785890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562465766d40 0x562465787980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 --2- 172.21.15.22:0/2044139132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562465765fd0 0x562465787440 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 >> 172.21.15.22:0/2044139132 conn(0x56246572d5b0 msgr2=0x56246577c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 shutdown_connections 2024-03-09T00:09:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.609+0000 7fb6df96f740 1 -- 172.21.15.22:0/2044139132 wait complete. 2024-03-09T00:09:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.610+0000 7fb6df96f740 20 client.29411 trim_cache size 0 max 16384 2024-03-09T00:09:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:09:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:09:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:09:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:09:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.610+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6c00a2ea0 msgr2=0x7fb6c00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.610+0000 7fb6dccab640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6c00a2ea0 0x7fb6c00a5360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0096f0 tx=0x7fb6cc006c40 comp rx=0 tx=0).stop 2024-03-09T00:09:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.610+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 msgr2=0x7fb6d81a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.610+0000 7fb6dccab640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 0x7fb6d81a0220 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0033420 tx=0x7fb6d0033750 comp rx=0 tx=0).stop 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 shutdown_connections 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb6c00a2ea0 0x7fb6c00a5360 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d810d5d0 0x7fb6d810da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb6d81065d0 0x7fb6d81a0220 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 --2- 172.21.15.22:0/958352299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6d8101500 0x7fb6d819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 >> 172.21.15.22:0/958352299 conn(0x7fb6d80fcd80 msgr2=0x7fb6d8073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 shutdown_connections 2024-03-09T00:09:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:14.611+0000 7fb6dccab640 1 -- 172.21.15.22:0/958352299 wait complete. 2024-03-09T00:09:14.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:14 smithi062 ceph-mon[36129]: pgmap v464: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 296 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 547 op/s 2024-03-09T00:09:14.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:14 smithi192 ceph-mon[35995]: pgmap v464: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 296 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 547 op/s 2024-03-09T00:09:14.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:14 smithi022 ceph-mon[31894]: pgmap v464: 129 pgs: 20 active+clean+snaptrim, 49 active+clean+snaptrim_wait, 60 active+clean; 296 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 547 op/s 2024-03-09T00:09:15.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 18566 14.28 MB/sec warmup 82 sec latency 525.549 ms 2024-03-09T00:09:16.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 19048 14.41 MB/sec warmup 83 sec latency 173.488 ms 2024-03-09T00:09:16.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:16 smithi062 ceph-mon[36129]: pgmap v465: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-03-09T00:09:16.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:16 smithi192 ceph-mon[35995]: pgmap v465: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-03-09T00:09:16.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:16 smithi022 ceph-mon[31894]: pgmap v465: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-03-09T00:09:17.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 19453 14.44 MB/sec warmup 84 sec latency 184.542 ms 2024-03-09T00:09:18.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 19937 14.61 MB/sec warmup 85 sec latency 186.326 ms 2024-03-09T00:09:18.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:18 smithi062 ceph-mon[36129]: pgmap v466: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 320 op/s 2024-03-09T00:09:18.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:18 smithi192 ceph-mon[35995]: pgmap v466: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 320 op/s 2024-03-09T00:09:18.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:18 smithi022 ceph-mon[31894]: pgmap v466: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 320 op/s 2024-03-09T00:09:19.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 20337 14.79 MB/sec warmup 86 sec latency 178.205 ms 2024-03-09T00:09:20.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 20604 14.85 MB/sec warmup 87 sec latency 170.239 ms 2024-03-09T00:09:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:19 smithi062 ceph-mon[36129]: osdmap e116: 12 total, 12 up, 12 in 2024-03-09T00:09:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:19 smithi062 ceph-mon[36129]: pgmap v468: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-03-09T00:09:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:19 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:09:20.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:19 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:09:20.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:19 smithi192 ceph-mon[35995]: osdmap e116: 12 total, 12 up, 12 in 2024-03-09T00:09:20.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:19 smithi192 ceph-mon[35995]: pgmap v468: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-03-09T00:09:20.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:19 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:09:20.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:19 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:09:20.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:19 smithi022 ceph-mon[31894]: osdmap e116: 12 total, 12 up, 12 in 2024-03-09T00:09:20.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:19 smithi022 ceph-mon[31894]: pgmap v468: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 293 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-03-09T00:09:20.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:19 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:09:20.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:19 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:09:21.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 20856 14.88 MB/sec warmup 88 sec latency 80.008 ms 2024-03-09T00:09:21.359 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:21 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:09:21.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:21 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:09:21.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:20 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:09:22.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 21043 14.80 MB/sec warmup 89 sec latency 134.519 ms 2024-03-09T00:09:22.359 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:22 smithi062 ceph-mon[36129]: pgmap v469: 129 pgs: 129 active+clean; 298 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-03-09T00:09:22.379 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:22 smithi192 ceph-mon[35995]: pgmap v469: 129 pgs: 129 active+clean; 298 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-03-09T00:09:22.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:22 smithi022 ceph-mon[31894]: pgmap v469: 129 pgs: 129 active+clean; 298 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-03-09T00:09:23.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 21229 14.69 MB/sec warmup 90 sec latency 270.539 ms 2024-03-09T00:09:24.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 21490 14.63 MB/sec warmup 91 sec latency 68.281 ms 2024-03-09T00:09:24.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:24 smithi022 ceph-mon[31894]: pgmap v470: 129 pgs: 129 active+clean; 298 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-03-09T00:09:24.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:24 smithi022 ceph-mon[31894]: osdmap e117: 12 total, 12 up, 12 in 2024-03-09T00:09:24.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:24 smithi062 ceph-mon[36129]: pgmap v470: 129 pgs: 129 active+clean; 298 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-03-09T00:09:24.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:24 smithi062 ceph-mon[36129]: osdmap e117: 12 total, 12 up, 12 in 2024-03-09T00:09:24.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:24 smithi192 ceph-mon[35995]: pgmap v470: 129 pgs: 129 active+clean; 298 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-03-09T00:09:24.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:24 smithi192 ceph-mon[35995]: osdmap e117: 12 total, 12 up, 12 in 2024-03-09T00:09:25.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 21699 14.57 MB/sec warmup 92 sec latency 238.060 ms 2024-03-09T00:09:26.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 22017 14.54 MB/sec warmup 93 sec latency 251.644 ms 2024-03-09T00:09:26.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:26 smithi022 ceph-mon[31894]: pgmap v472: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 359 op/s 2024-03-09T00:09:26.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:26 smithi062 ceph-mon[36129]: pgmap v472: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 359 op/s 2024-03-09T00:09:26.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:26 smithi192 ceph-mon[35995]: pgmap v472: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 359 op/s 2024-03-09T00:09:27.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 22404 14.58 MB/sec warmup 94 sec latency 204.100 ms 2024-03-09T00:09:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.769+0000 7fe8b0861640 1 -- 172.21.15.22:0/3210424004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106650 msgr2=0x7fe8ac06a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.769+0000 7fe8b0861640 1 --2- 172.21.15.22:0/3210424004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106650 0x7fe8ac06a0b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009900 tx=0x7fe89c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.770+0000 7fe8b0861640 1 -- 172.21.15.22:0/3210424004 shutdown_connections 2024-03-09T00:09:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.770+0000 7fe8b0861640 1 --2- 172.21.15.22:0/3210424004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8ac06a790 0x7fe8ac103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.770+0000 7fe8b0861640 1 --2- 172.21.15.22:0/3210424004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106650 0x7fe8ac06a0b0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.770+0000 7fe8b0861640 1 --2- 172.21.15.22:0/3210424004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac106270 0x7fe8ac069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.770+0000 7fe8b0861640 1 -- 172.21.15.22:0/3210424004 >> 172.21.15.22:0/3210424004 conn(0x7fe8ac077030 msgr2=0x7fe8ac077440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.771+0000 7fe8b0861640 1 -- 172.21.15.22:0/3210424004 shutdown_connections 2024-03-09T00:09:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.771+0000 7fe8b0861640 1 -- 172.21.15.22:0/3210424004 wait complete. 2024-03-09T00:09:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.772+0000 7fe8b0861640 1 Processor -- start 2024-03-09T00:09:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.774+0000 7fe8b0861640 1 -- start start 2024-03-09T00:09:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.774+0000 7fe8b0861640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106270 0x7fe8ac114770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.774+0000 7fe8b0861640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8ac106650 0x7fe8ac114cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.774+0000 7fe8b0861640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac10d8e0 0x7fe8ac10dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.774+0000 7fe8b0861640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac117cd0 con 0x7fe8ac106270 2024-03-09T00:09:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.774+0000 7fe8b0861640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac117b50 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.774+0000 7fe8b0861640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac117e50 con 0x7fe8ac106650 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.775+0000 7fe8aad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac10d8e0 0x7fe8ac10dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac10d8e0 0x7fe8ac10dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33110/0 (socket says 172.21.15.22:33110) 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 -- 172.21.15.22:0/4085947042 learned_addr learned my addr 172.21.15.22:0/4085947042 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106270 0x7fe8ac114770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 -- 172.21.15.22:0/4085947042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8ac106650 msgr2=0x7fe8ac114cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8ac106650 0x7fe8ac114cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 -- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106270 msgr2=0x7fe8ac114770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106270 0x7fe8ac114770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 -- 172.21.15.22:0/4085947042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac10e620 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.776+0000 7fe8aad9d640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac10d8e0 0x7fe8ac10dd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400eb10 tx=0x7fe8a400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.777+0000 7fe89b7fe640 1 -- 172.21.15.22:0/4085947042 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4004b20 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.777+0000 7fe89b7fe640 1 -- 172.21.15.22:0/4085947042 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8a4004cc0 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.777+0000 7fe89b7fe640 1 -- 172.21.15.22:0/4085947042 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4010790 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.777+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac112350 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.777+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac1127f0 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.781+0000 7fe89b7fe640 1 -- 172.21.15.22:0/4085947042 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe8a4005020 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.781+0000 7fe89b7fe640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8940a2a60 0x7fe8940a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.782+0000 7fe89b7fe640 1 -- 172.21.15.22:0/4085947042 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fe8a40cf9a0 con 0x7fe8ac10d8e0 2024-03-09T00:09:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.782+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8940a2a60 0x7fe8940a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.778 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.784+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8940a2a60 0x7fe8940a4f20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009ae0 tx=0x7fe8a0005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.785+0000 7fe8b3440740 1 Processor -- start 2024-03-09T00:09:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.785+0000 7fe8b3440740 1 -- start start 2024-03-09T00:09:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee30fd40 0x558bee30ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30efd0 0x558bee30f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee318550 0x558bee31a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8aa59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee30fd40 0x558bee30ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8b3440740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bee2735f0 con 0x558bee318550 2024-03-09T00:09:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8b3440740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bee21c930 con 0x558bee30efd0 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8b3440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bee26f690 con 0x558bee30fd40 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30efd0 0x558bee30f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8aa59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee30fd40 0x558bee30ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:43020/0 (socket says 172.21.15.22:43020) 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8aa59c640 1 -- 172.21.15.22:0/242496845 learned_addr learned my addr 172.21.15.22:0/242496845 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30efd0 0x558bee30f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33112/0 (socket says 172.21.15.22:33112) 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.786+0000 7fe8aad9d640 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee318550 0x558bee31a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.787+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/242496845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee30fd40 msgr2=0x558bee30ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.787+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee30fd40 0x558bee30ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.787+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/242496845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee318550 msgr2=0x558bee31a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.787+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee318550 0x558bee31a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.787+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/242496845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bedf88210 con 0x558bee30efd0 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.787+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30efd0 0x558bee30f450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac100460 tx=0x7fe89c005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40d6269e53e7d1d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.787+0000 7fe898ff9640 1 -- 172.21.15.22:0/242496845 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c042020 con 0x558bee30efd0 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.788+0000 7fe898ff9640 1 -- 172.21.15.22:0/242496845 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe89c02c730 con 0x558bee30efd0 2024-03-09T00:09:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.788+0000 7fe898ff9640 1 -- 172.21.15.22:0/242496845 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003a80 con 0x558bee30efd0 2024-03-09T00:09:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 -- 172.21.15.22:0/242496845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30efd0 msgr2=0x558bee30f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30efd0 0x558bee30f450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac100460 tx=0x7fe89c005730 comp rx=0 tx=0).stop 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 -- 172.21.15.22:0/242496845 shutdown_connections 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee318550 0x558bee31a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30efd0 0x558bee30f450 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 --2- 172.21.15.22:0/242496845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee30fd40 0x558bee30ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 -- 172.21.15.22:0/242496845 >> 172.21.15.22:0/242496845 conn(0x558bee2d65b0 msgr2=0x558bee309cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 -- 172.21.15.22:0/242496845 shutdown_connections 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 -- 172.21.15.22:0/242496845 wait complete. 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 -- start start 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.789+0000 7fe8b3440740 1 Processor -- start 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee3156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee315bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8b3440740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bee31b220 con 0x558bee316130 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8b3440740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bee22fd90 con 0x558bee30fd40 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8b3440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bee31b3a0 con 0x558bee318550 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8b3440740 10 client.0 _ll_register_callbacks cb 0x558bedf87a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee315bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee315bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:43026/0 (socket says 172.21.15.22:43026) 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/351912654 learned_addr learned my addr 172.21.15.22:0/351912654 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8aad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3246d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57118/0 (socket says 172.21.15.22:57118) 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.790+0000 7fe8aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee3156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/351912654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 msgr2=0x558bee3156b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee3156b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/351912654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 msgr2=0x558bee3246d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3246d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/351912654 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bee054620 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee315bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0057c0 tx=0x7fe89c02dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c628c73e25c1adac server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe891ffb640 1 -- 172.21.15.22:0/351912654 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003a00 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bee280b30 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe891ffb640 1 -- 172.21.15.22:0/351912654 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fe89c003ba0 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe891ffb640 1 -- 172.21.15.22:0/351912654 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89c004470 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8b3440740 10 client.29575 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.791+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558bee31b270 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe891ffb640 1 -- 172.21.15.22:0/351912654 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558bee280b30 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe891ffb640 1 -- 172.21.15.22:0/351912654 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe89c047420 con 0x558bee318550 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe8b3440740 10 client.29575 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe8b3440740 10 client.29575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe8b3440740 20 client.29575 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe8b3440740 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee328c80 0x558bee32b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe8b3440740 4 client.29575 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bee27ae80 con 0x558bee328c80 2024-03-09T00:09:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.792+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee328c80 0x558bee32b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.793+0000 7fe891ffb640 10 client.29575 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.793+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee328c80 0x558bee32b070 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14e8e71139c5670 server_cookie=f3aac73ad3549422 in_seq=0 out_seq=0 2024-03-09T00:09:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe891ffb640 1 -- 172.21.15.22:0/351912654 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558bee27ae80 con 0x558bee328c80 2024-03-09T00:09:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe891ffb640 10 client.29575 handle_command_reply: tid=0 2024-03-09T00:09:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 client.29575 shutdown 2024-03-09T00:09:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 msgr2=0x558bee315bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee315bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0057c0 tx=0x7fe89c02dbe0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 shutdown_connections 2024-03-09T00:09:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee328c80 0x558bee32b070 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee315bf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe891ffb640 0 client.29575 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 --2- 172.21.15.22:0/351912654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee3156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 >> 172.21.15.22:0/351912654 conn(0x558bee2d65b0 msgr2=0x558bee30ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 shutdown_connections 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 1 -- 172.21.15.22:0/351912654 wait complete. 2024-03-09T00:09:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.796+0000 7fe8b3440740 20 client.29575 trim_cache size 0 max 16384 2024-03-09T00:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.810+0000 7fe8b3440740 1 Processor -- start 2024-03-09T00:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.810+0000 7fe8b3440740 1 -- start start 2024-03-09T00:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee313f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee3240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8b3440740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1bf9b0 con 0x558bee316130 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8b3440740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1bd4f0 con 0x558bee30fd40 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee313f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8b3440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1bb5b0 con 0x558bee318550 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee313f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33128/0 (socket says 172.21.15.22:33128) 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 -- 172.21.15.22:0/1190938452 learned_addr learned my addr 172.21.15.22:0/1190938452 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aad9d640 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee3240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 -- 172.21.15.22:0/1190938452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 msgr2=0x558bee3240d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee3240d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 -- 172.21.15.22:0/1190938452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 msgr2=0x558bee3144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aa59c640 1 -- 172.21.15.22:0/1190938452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bee30e2a0 con 0x558bee30fd40 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.811+0000 7fe8aad9d640 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee3240d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.812+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee313f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x558bee32b890 tx=0x7fe8a0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c7516db64a3055 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.812+0000 7fe893fff640 1 -- 172.21.15.22:0/1190938452 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a002d070 con 0x558bee30fd40 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.812+0000 7fe893fff640 1 -- 172.21.15.22:0/1190938452 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8a00295f0 con 0x558bee30fd40 2024-03-09T00:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.812+0000 7fe893fff640 1 -- 172.21.15.22:0/1190938452 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a001e610 con 0x558bee30fd40 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 -- 172.21.15.22:0/1190938452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 msgr2=0x558bee313f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee313f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x558bee32b890 tx=0x7fe8a0014000 comp rx=0 tx=0).stop 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 -- 172.21.15.22:0/1190938452 shutdown_connections 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee3240d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee316130 0x558bee3144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1190938452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee30fd40 0x558bee313f80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 -- 172.21.15.22:0/1190938452 >> 172.21.15.22:0/1190938452 conn(0x558bee2d65b0 msgr2=0x558bee3097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 -- 172.21.15.22:0/1190938452 shutdown_connections 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 -- 172.21.15.22:0/1190938452 wait complete. 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 -- start start 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.813+0000 7fe8b3440740 1 Processor -- start 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee30fd40 0x558bee31c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee316130 0x558bee31c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8b3440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee31caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8b3440740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1bbbb0 con 0x558bee30fd40 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8b3440740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1bce90 con 0x558bee316130 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8b3440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1bc1d0 con 0x558bee318550 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee316130 0x558bee31c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8aad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee31caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee316130 0x558bee31c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33140/0 (socket says 172.21.15.22:33140) 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/1162269646 learned_addr learned my addr 172.21.15.22:0/1162269646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8b3440740 10 client.0 _ll_register_callbacks cb 0x7fe8ac1151f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee30fd40 0x558bee31c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/1162269646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 msgr2=0x558bee31caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee31caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/1162269646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee30fd40 msgr2=0x558bee31c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee30fd40 0x558bee31c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 -- 172.21.15.22:0/1162269646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bee37f0b0 con 0x558bee316130 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe8a9d9b640 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee316130 0x558bee31c5b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558bee316c80 tx=0x7fe89c004350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467653fd6b304ea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.814+0000 7fe891ffb640 1 -- 172.21.15.22:0/1162269646 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c042070 con 0x558bee316130 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8a00295f0 con 0x558bee316130 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe891ffb640 1 -- 172.21.15.22:0/1162269646 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fe89c005230 con 0x558bee316130 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe891ffb640 1 -- 172.21.15.22:0/1162269646 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89c067050 con 0x558bee316130 2024-03-09T00:09:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 10 client.29429 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558bee380380 con 0x558bee316130 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe891ffb640 1 -- 172.21.15.22:0/1162269646 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8a00295f0 con 0x558bee316130 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe891ffb640 1 -- 172.21.15.22:0/1162269646 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe89c004c10 con 0x558bee316130 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 10 client.29429 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 10 client.29429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 20 client.29429 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee380a40 0x558bee382e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 4 client.29429 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558bee1f9710 con 0x558bee380a40 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.815+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee380a40 0x558bee382e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.816+0000 7fe891ffb640 10 client.29429 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:27.816+0000 7fe8aa59c640 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee380a40 0x558bee382e60 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a508fd17ce1ee5c4 server_cookie=60db784ece029dc9 in_seq=0 out_seq=0 2024-03-09T00:09:28.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 22651 14.61 MB/sec warmup 95 sec latency 359.837 ms 2024-03-09T00:09:28.432 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:28 smithi192 ceph-mon[35995]: pgmap v473: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 354 op/s 2024-03-09T00:09:28.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:28 smithi022 ceph-mon[31894]: pgmap v473: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 354 op/s 2024-03-09T00:09:28.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:28 smithi062 ceph-mon[36129]: pgmap v473: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 354 op/s 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.821+0000 7fe891ffb640 1 -- 172.21.15.22:0/1162269646 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558bee1f9710 con 0x558bee380a40 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.821+0000 7fe891ffb640 10 client.29429 handle_command_reply: tid=0 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.821+0000 7fe8b3440740 1 client.29429 shutdown 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee316130 msgr2=0x558bee31c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee316130 0x558bee31c5b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558bee316c80 tx=0x7fe89c004350 comp rx=0 tx=0).stop 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 shutdown_connections 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558bee380a40 0x558bee382e60 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe891ffb640 0 client.29429 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bee318550 0x558bee31caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bee316130 0x558bee31c5b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 --2- 172.21.15.22:0/1162269646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558bee30fd40 0x558bee31c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 >> 172.21.15.22:0/1162269646 conn(0x558bee2d65b0 msgr2=0x558bee3097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 shutdown_connections 2024-03-09T00:09:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.822+0000 7fe8b3440740 1 -- 172.21.15.22:0/1162269646 wait complete. 2024-03-09T00:09:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.823+0000 7fe8b3440740 20 client.29429 trim_cache size 0 max 16384 2024-03-09T00:09:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:09:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:09:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:09:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:09:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8940a2a60 msgr2=0x7fe8940a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8940a2a60 0x7fe8940a4f20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009ae0 tx=0x7fe8a0005f90 comp rx=0 tx=0).stop 2024-03-09T00:09:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac10d8e0 msgr2=0x7fe8ac10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac10d8e0 0x7fe8ac10dd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400eb10 tx=0x7fe8a400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 shutdown_connections 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe8940a2a60 0x7fe8940a4f20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8ac10d8e0 0x7fe8ac10dd60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8ac106650 0x7fe8ac114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 --2- 172.21.15.22:0/4085947042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8ac106270 0x7fe8ac114770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 >> 172.21.15.22:0/4085947042 conn(0x7fe8ac077030 msgr2=0x7fe8ac0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 shutdown_connections 2024-03-09T00:09:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:28.824+0000 7fe8b0861640 1 -- 172.21.15.22:0/4085947042 wait complete. 2024-03-09T00:09:29.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 22902 14.61 MB/sec warmup 96 sec latency 674.345 ms 2024-03-09T00:09:30.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 23370 14.68 MB/sec warmup 97 sec latency 196.072 ms 2024-03-09T00:09:30.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:30 smithi022 ceph-mon[31894]: pgmap v474: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.4 MiB/s wr, 287 op/s 2024-03-09T00:09:30.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:30 smithi062 ceph-mon[36129]: pgmap v474: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.4 MiB/s wr, 287 op/s 2024-03-09T00:09:30.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:30 smithi192 ceph-mon[35995]: pgmap v474: 129 pgs: 129 active+clean; 315 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.4 MiB/s wr, 287 op/s 2024-03-09T00:09:31.065 INFO:tasks.workunit.client.0.smithi022.stdout: 10 23763 14.84 MB/sec warmup 98 sec latency 189.534 ms 2024-03-09T00:09:32.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 24079 14.94 MB/sec warmup 99 sec latency 169.238 ms 2024-03-09T00:09:32.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:32 smithi022 ceph-mon[31894]: pgmap v475: 129 pgs: 129 active+clean; 316 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-03-09T00:09:32.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:32 smithi062 ceph-mon[36129]: pgmap v475: 129 pgs: 129 active+clean; 316 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-03-09T00:09:32.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:32 smithi192 ceph-mon[35995]: pgmap v475: 129 pgs: 129 active+clean; 316 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-03-09T00:09:33.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 24263 14.92 MB/sec warmup 100 sec latency 245.431 ms 2024-03-09T00:09:34.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 24494 14.89 MB/sec warmup 101 sec latency 75.672 ms 2024-03-09T00:09:34.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:34 smithi022 ceph-mon[31894]: pgmap v476: 129 pgs: 129 active+clean; 316 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-03-09T00:09:34.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:34 smithi062 ceph-mon[36129]: pgmap v476: 129 pgs: 129 active+clean; 316 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-03-09T00:09:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:34 smithi192 ceph-mon[35995]: pgmap v476: 129 pgs: 129 active+clean; 316 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-03-09T00:09:35.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 24673 14.81 MB/sec warmup 102 sec latency 299.368 ms 2024-03-09T00:09:36.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 24879 14.74 MB/sec warmup 103 sec latency 240.062 ms 2024-03-09T00:09:36.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:36 smithi022 ceph-mon[31894]: pgmap v477: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 647 op/s 2024-03-09T00:09:36.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:36 smithi062 ceph-mon[36129]: pgmap v477: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 647 op/s 2024-03-09T00:09:36.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:36 smithi192 ceph-mon[35995]: pgmap v477: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 647 op/s 2024-03-09T00:09:37.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 25164 14.70 MB/sec warmup 104 sec latency 247.212 ms 2024-03-09T00:09:38.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 25535 14.67 MB/sec warmup 105 sec latency 291.232 ms 2024-03-09T00:09:38.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:38 smithi022 ceph-mon[31894]: pgmap v478: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 489 op/s 2024-03-09T00:09:38.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:38 smithi062 ceph-mon[36129]: pgmap v478: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 489 op/s 2024-03-09T00:09:38.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:38 smithi192 ceph-mon[35995]: pgmap v478: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 489 op/s 2024-03-09T00:09:39.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 25986 14.78 MB/sec warmup 106 sec latency 176.716 ms 2024-03-09T00:09:40.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 26406 14.91 MB/sec warmup 107 sec latency 152.159 ms 2024-03-09T00:09:40.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:40 smithi022 ceph-mon[31894]: pgmap v479: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 491 op/s 2024-03-09T00:09:40.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:40 smithi062 ceph-mon[36129]: pgmap v479: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 491 op/s 2024-03-09T00:09:40.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:40 smithi192 ceph-mon[35995]: pgmap v479: 129 pgs: 129 active+clean; 324 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 491 op/s 2024-03-09T00:09:41.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 26891 15.01 MB/sec warmup 108 sec latency 79.148 ms 2024-03-09T00:09:41.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:09:41.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:09:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 -- 172.21.15.22:0/1551581581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03501082a0 msgr2=0x7f03501147b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 --2- 172.21.15.22:0/1551581581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03501082a0 0x7f03501147b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0348009be0 tx=0x7f034802bef0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 -- 172.21.15.22:0/1551581581 shutdown_connections 2024-03-09T00:09:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 --2- 172.21.15.22:0/1551581581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03501082a0 0x7f03501147b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 --2- 172.21.15.22:0/1551581581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501078e0 0x7f0350107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 --2- 172.21.15.22:0/1551581581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f035010d8f0 0x7f035010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 -- 172.21.15.22:0/1551581581 >> 172.21.15.22:0/1551581581 conn(0x7f0350075860 msgr2=0x7f0350077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 -- 172.21.15.22:0/1551581581 shutdown_connections 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.928+0000 7f03573ab640 1 -- 172.21.15.22:0/1551581581 wait complete. 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.929+0000 7f03573ab640 1 Processor -- start 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.929+0000 7f03573ab640 1 -- start start 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f03573ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03501078e0 0x7f0350081c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f03573ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501082a0 0x7f0350079c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f03573ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035010d8f0 0x7f035007a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f03573ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0350117d50 con 0x7f03501082a0 2024-03-09T00:09:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f03573ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0350117bd0 con 0x7f035010d8f0 2024-03-09T00:09:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f03573ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0350117ed0 con 0x7f03501078e0 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501082a0 0x7f0350079c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0355147640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03501078e0 0x7f0350081c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501082a0 0x7f0350079c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40850/0 (socket says 172.21.15.22:40850) 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 -- 172.21.15.22:0/191779349 learned_addr learned my addr 172.21.15.22:0/191779349 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0355948640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035010d8f0 0x7f035007a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 -- 172.21.15.22:0/191779349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03501078e0 msgr2=0x7f0350081c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03501078e0 0x7f0350081c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 -- 172.21.15.22:0/191779349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035010d8f0 msgr2=0x7f035007a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035010d8f0 0x7f035007a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 -- 172.21.15.22:0/191779349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035007a970 con 0x7f03501082a0 2024-03-09T00:09:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.930+0000 7f0354946640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501082a0 0x7f0350079c10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f034000e9c0 tx=0x7f034000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.931+0000 7f03467fc640 1 -- 172.21.15.22:0/191779349 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034000ccc0 con 0x7f03501082a0 2024-03-09T00:09:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.931+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03501b2e90 con 0x7f03501082a0 2024-03-09T00:09:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.931+0000 7f03467fc640 1 -- 172.21.15.22:0/191779349 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0340005500 con 0x7f03501082a0 2024-03-09T00:09:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.931+0000 7f03467fc640 1 -- 172.21.15.22:0/191779349 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340010430 con 0x7f03501082a0 2024-03-09T00:09:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.931+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03501b33a0 con 0x7f03501082a0 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.934+0000 7f03467fc640 1 -- 172.21.15.22:0/191779349 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f03400046c0 con 0x7f03501082a0 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.934+0000 7f0359f81740 1 Processor -- start 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0359f81740 1 -- start start 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0359f81740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c64d40 0x5572d0c63a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0359f81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c64450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0359f81740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c6f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0359f81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572d0bc85f0 con 0x5572d0c63fd0 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0359f81740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572d0b71930 con 0x5572d0c64d40 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0359f81740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572d0bc4690 con 0x5572d0c6d550 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c64d40 0x5572d0c63a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c64d40 0x5572d0c63a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:49266/0 (socket says 172.21.15.22:49266) 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 -- 172.21.15.22:0/1746868711 learned_addr learned my addr 172.21.15.22:0/1746868711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0354946640 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c6f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355147640 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c64450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f03467fc640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f033c0a2a10 0x7f033c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 -- 172.21.15.22:0/1746868711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 msgr2=0x5572d0c6f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c6f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 -- 172.21.15.22:0/1746868711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 msgr2=0x5572d0c64450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c64450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 -- 172.21.15.22:0/1746868711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572d08dd210 con 0x5572d0c64d40 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.935+0000 7f0355948640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f033c0a2a10 0x7f033c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.936+0000 7f0355948640 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c64d40 0x5572d0c63a90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0348009be0 tx=0x7f03480098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c025f67f781f934b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.936+0000 7f033b7fe640 1 -- 172.21.15.22:0/1746868711 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348003e40 con 0x5572d0c64d40 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.936+0000 7f03467fc640 1 -- 172.21.15.22:0/191779349 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f03400cf540 con 0x7f03501082a0 2024-03-09T00:09:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.936+0000 7f033b7fe640 1 -- 172.21.15.22:0/1746868711 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034802cd70 con 0x5572d0c64d40 2024-03-09T00:09:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.936+0000 7f033b7fe640 1 -- 172.21.15.22:0/1746868711 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348041460 con 0x5572d0c64d40 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 -- 172.21.15.22:0/1746868711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c64d40 msgr2=0x5572d0c63a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0355948640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f033c0a2a10 0x7f033c0a4ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f034804d000 tx=0x7f034804a000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c64d40 0x5572d0c63a90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0348009be0 tx=0x7f03480098f0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 -- 172.21.15.22:0/1746868711 shutdown_connections 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c6f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c64450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 --2- 172.21.15.22:0/1746868711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c64d40 0x5572d0c63a90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 -- 172.21.15.22:0/1746868711 >> 172.21.15.22:0/1746868711 conn(0x5572d0c2b5b0 msgr2=0x5572d0c5ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 -- 172.21.15.22:0/1746868711 shutdown_connections 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 -- 172.21.15.22:0/1746868711 wait complete. 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 -- start start 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.937+0000 7f0359f81740 1 Processor -- start 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0359f81740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c63fd0 0x5572d0c6a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0359f81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c6aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0359f81740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c6afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0359f81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572d0c70280 con 0x5572d0c64d40 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0359f81740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572d0b84d90 con 0x5572d0c6d550 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0359f81740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572d0c70400 con 0x5572d0c63fd0 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c6aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0355147640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c63fd0 0x5572d0c6a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c6aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40876/0 (socket says 172.21.15.22:40876) 2024-03-09T00:09:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 -- 172.21.15.22:0/463196228 learned_addr learned my addr 172.21.15.22:0/463196228 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0355948640 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c6afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0359f81740 10 client.0 _ll_register_callbacks cb 0x5572d08dca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 -- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c63fd0 msgr2=0x5572d0c6a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c63fd0 0x5572d0c6a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 -- 172.21.15.22:0/463196228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 msgr2=0x5572d0c6afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c6afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 -- 172.21.15.22:0/463196228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572d09a9620 con 0x5572d0c64d40 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0355147640 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c63fd0 0x5572d0c6a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0354946640 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c6aa90 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f034000c6a0 tx=0x7f03400d2c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2132aeed082a7cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0338ff9640 1 -- 172.21.15.22:0/463196228 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034009b580 con 0x5572d0c64d40 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.938+0000 7f0338ff9640 1 -- 172.21.15.22:0/463196228 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f034009f5b0 con 0x5572d0c64d40 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f034802cd70 con 0x5572d0c64d40 2024-03-09T00:09:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0338ff9640 1 -- 172.21.15.22:0/463196228 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034009b110 con 0x5572d0c64d40 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 10 client.20220 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5572d0c7c550 con 0x5572d0c64d40 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0338ff9640 1 -- 172.21.15.22:0/463196228 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034802cd70 con 0x5572d0c64d40 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0338ff9640 1 -- 172.21.15.22:0/463196228 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0340097e20 con 0x5572d0c64d40 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 10 client.20220 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 10 client.20220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 20 client.20220 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0c7d9c0 0x5572d0c7fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 4 client.20220 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572d0bd5b30 con 0x5572d0c7d9c0 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.939+0000 7f0355147640 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0c7d9c0 0x5572d0c7fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.940+0000 7f0355147640 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0c7d9c0 0x5572d0c7fdb0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e5a945ae038377 server_cookie=79f5ff73a2d5f8e1 in_seq=0 out_seq=0 2024-03-09T00:09:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.940+0000 7f0338ff9640 10 client.20220 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0338ff9640 1 -- 172.21.15.22:0/463196228 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5572d0bd5b30 con 0x5572d0c7d9c0 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0338ff9640 10 client.20220 handle_command_reply: tid=0 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 client.20220 shutdown 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 msgr2=0x5572d0c6aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c6aa90 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f034000c6a0 tx=0x7f03400d2c80 comp rx=0 tx=0).stop 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 shutdown_connections 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0c7d9c0 0x5572d0c7fdb0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c6afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0338ff9640 0 client.20220 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c6aa90 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 --2- 172.21.15.22:0/463196228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c63fd0 0x5572d0c6a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.944+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 >> 172.21.15.22:0/463196228 conn(0x5572d0c2b5b0 msgr2=0x5572d0c60a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.945+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 shutdown_connections 2024-03-09T00:09:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.945+0000 7f0359f81740 1 -- 172.21.15.22:0/463196228 wait complete. 2024-03-09T00:09:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.945+0000 7f0359f81740 20 client.20220 trim_cache size 0 max 16384 2024-03-09T00:09:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.960+0000 7f0359f81740 1 Processor -- start 2024-03-09T00:09:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.960+0000 7f0359f81740 1 -- start start 2024-03-09T00:09:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0359f81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c6cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0359f81740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c64d40 0x5572d0c68e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0359f81740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c69360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0359f81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501bf8c0 con 0x5572d0c63fd0 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0359f81740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501bd400 con 0x5572d0c6d550 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0359f81740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501bb4c0 con 0x5572d0c64d40 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c6cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0354946640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c64d40 0x5572d0c68e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c6cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40878/0 (socket says 172.21.15.22:40878) 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 -- 172.21.15.22:0/1162670951 learned_addr learned my addr 172.21.15.22:0/1162670951 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355948640 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c69360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 -- 172.21.15.22:0/1162670951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c64d40 msgr2=0x5572d0c68e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c64d40 0x5572d0c68e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 -- 172.21.15.22:0/1162670951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 msgr2=0x5572d0c69360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c69360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 -- 172.21.15.22:0/1162670951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572d0c84ec0 con 0x5572d0c63fd0 2024-03-09T00:09:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f0355147640 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c6cc80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x5572d0c805b0 tx=0x7f034c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6310da6642a30158 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.961+0000 7f033affd640 1 -- 172.21.15.22:0/1162670951 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0122d0 con 0x5572d0c63fd0 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f033affd640 1 -- 172.21.15.22:0/1162670951 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034c005020 con 0x5572d0c63fd0 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f033affd640 1 -- 172.21.15.22:0/1162670951 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00fcb0 con 0x5572d0c63fd0 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 -- 172.21.15.22:0/1162670951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 msgr2=0x5572d0c6cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c6cc80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x5572d0c805b0 tx=0x7f034c005f90 comp rx=0 tx=0).stop 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 -- 172.21.15.22:0/1162670951 shutdown_connections 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c6d550 0x5572d0c69360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c64d40 0x5572d0c68e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 --2- 172.21.15.22:0/1162670951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c63fd0 0x5572d0c6cc80 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 -- 172.21.15.22:0/1162670951 >> 172.21.15.22:0/1162670951 conn(0x5572d0c2b5b0 msgr2=0x5572d0c65830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 -- 172.21.15.22:0/1162670951 shutdown_connections 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.962+0000 7f0359f81740 1 -- 172.21.15.22:0/1162670951 wait complete. 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 -- start start 2024-03-09T00:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 Processor -- start 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c63fd0 0x5572d0c74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c74b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c72bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501bbac0 con 0x5572d0c64d40 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501bcda0 con 0x5572d0c63fd0 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501bc0e0 con 0x5572d0c6d550 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0359f81740 10 client.0 _ll_register_callbacks cb 0x7f034c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0354946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c74b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0355147640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c63fd0 0x5572d0c74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0355147640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c63fd0 0x5572d0c74650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:49296/0 (socket says 172.21.15.22:49296) 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0355147640 1 -- 172.21.15.22:0/7577948 learned_addr learned my addr 172.21.15.22:0/7577948 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.963+0000 7f0355948640 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c72bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0355147640 1 -- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 msgr2=0x5572d0c72bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0355147640 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c72bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0355147640 1 -- 172.21.15.22:0/7577948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 msgr2=0x5572d0c74b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0355147640 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c74b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0355147640 1 -- 172.21.15.22:0/7577948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572d0c733f0 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0355147640 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c63fd0 0x5572d0c74650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f034c009e50 tx=0x7f034c009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02176c96978e923 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0338ff9640 1 -- 172.21.15.22:0/7577948 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0051d0 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0338ff9640 1 -- 172.21.15.22:0/7577948 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f034c005370 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f034c005020 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0338ff9640 1 -- 172.21.15.22:0/7577948 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034c03c050 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0359f81740 10 client.29450 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.964+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5572d0cd4a00 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0338ff9640 1 -- 172.21.15.22:0/7577948 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034c005020 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0338ff9640 1 -- 172.21.15.22:0/7577948 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f034c0210f0 con 0x5572d0c63fd0 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0359f81740 10 client.29450 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0359f81740 10 client.29450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0359f81740 20 client.29450 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0359f81740 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0cd5070 0x5572d0cd7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0359f81740 4 client.29450 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:09:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5572d0bcfe80 con 0x5572d0cd5070 2024-03-09T00:09:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.965+0000 7f0354946640 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0cd5070 0x5572d0cd7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.965+0000 7fd0ac5df640 1 -- 172.21.15.22:0/464372871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a4101e90 msgr2=0x7fd0a4105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.965+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/464372871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a4101e90 0x7fd0a4105ed0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd094009900 tx=0x7fd09402bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.966+0000 7f0338ff9640 10 client.29450 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:41.966+0000 7f0354946640 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0cd5070 0x5572d0cd7490 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7124708d5a22adb4 server_cookie=53062b6e2803338e in_seq=0 out_seq=0 2024-03-09T00:09:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 -- 172.21.15.22:0/464372871 shutdown_connections 2024-03-09T00:09:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/464372871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0a41065b0 0x7fd0a410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/464372871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a4101e90 0x7fd0a4105ed0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/464372871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a41014e0 0x7fd0a41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 -- 172.21.15.22:0/464372871 >> 172.21.15.22:0/464372871 conn(0x7fd0a40fcd80 msgr2=0x7fd0a40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 -- 172.21.15.22:0/464372871 shutdown_connections 2024-03-09T00:09:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 -- 172.21.15.22:0/464372871 wait complete. 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 Processor -- start 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.966+0000 7fd0ac5df640 1 -- start start 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0ac5df640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a41014e0 0x7fd0a419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0ac5df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a41065b0 0x7fd0a419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0aa37b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a41014e0 0x7fd0a419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0ac5df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0a41979e0 0x7fd0a4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0aa37b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a41014e0 0x7fd0a419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40912/0 (socket says 172.21.15.22:40912) 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0ac5df640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4111620 con 0x7fd0a41014e0 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0aa37b640 1 -- 172.21.15.22:0/328750344 learned_addr learned my addr 172.21.15.22:0/328750344 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41114a0 con 0x7fd0a41979e0 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41117a0 con 0x7fd0a41065b0 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a41065b0 0x7fd0a419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0aa37b640 1 -- 172.21.15.22:0/328750344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a41065b0 msgr2=0x7fd0a419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0aa37b640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a41065b0 0x7fd0a419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.967+0000 7fd0aa37b640 1 -- 172.21.15.22:0/328750344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0a41979e0 msgr2=0x7fd0a4197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0a41979e0 0x7fd0a4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd0aa37b640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0a41979e0 0x7fd0a4197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd0aa37b640 1 -- 172.21.15.22:0/328750344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4198500 con 0x7fd0a41014e0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0a41979e0 0x7fd0a4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd0aa37b640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a41014e0 0x7fd0a419d830 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009c20 tx=0x7fd0a000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd09b7fe640 1 -- 172.21.15.22:0/328750344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a001e070 con 0x7fd0a41014e0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd09b7fe640 1 -- 172.21.15.22:0/328750344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0a00198f0 con 0x7fd0a41014e0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd09b7fe640 1 -- 172.21.15.22:0/328750344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00187f0 con 0x7fd0a41014e0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a41987f0 con 0x7fd0a41014e0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.968+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a40738c0 con 0x7fd0a41014e0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.971+0000 7fd0af1c3740 1 Processor -- start 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.971+0000 7fd09b7fe640 1 -- 172.21.15.22:0/328750344 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd0a0021070 con 0x7fd0a41014e0 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0af1c3740 1 -- start start 2024-03-09T00:09:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878d760 0x55a74878db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748796720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a748796c60 0x55a748799050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7486f0e00 con 0x55a74878e080 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a748699b10 con 0x55a748796c60 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7486ecea0 con 0x55a74878d760 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0aab7c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878d760 0x55a74878db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0a9b7a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a748796c60 0x55a748799050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0a9b7a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a748796c60 0x55a748799050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:49308/0 (socket says 172.21.15.22:49308) 2024-03-09T00:09:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0aab7c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878d760 0x55a74878db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59704/0 (socket says 172.21.15.22:59704) 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0a9b7a640 1 -- 172.21.15.22:0/616014299 learned_addr learned my addr 172.21.15.22:0/616014299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0aa37b640 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748796720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0aab7c640 1 -- 172.21.15.22:0/616014299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a748796c60 msgr2=0x55a748799050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a748796c60 0x55a748799050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.972+0000 7fd0aab7c640 1 -- 172.21.15.22:0/616014299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 msgr2=0x55a748796720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748796720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd0aab7c640 1 -- 172.21.15.22:0/616014299 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7484cf7d0 con 0x55a74878d760 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd09b7fe640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0900a29f0 0x7fd0900a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878d760 0x55a74878db40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009d80 tx=0x7fd09c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=123d1b7f0abc4412 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd0aa37b640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0900a29f0 0x7fd0900a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd098bfe640 1 -- 172.21.15.22:0/616014299 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c004b20 con 0x55a74878d760 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd098bfe640 1 -- 172.21.15.22:0/616014299 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd09c004cc0 con 0x55a74878d760 2024-03-09T00:09:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd098bfe640 1 -- 172.21.15.22:0/616014299 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c011830 con 0x55a74878d760 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.973+0000 7fd09b7fe640 1 -- 172.21.15.22:0/328750344 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd0a00d2b50 con 0x7fd0a41014e0 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 -- 172.21.15.22:0/616014299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878d760 msgr2=0x55a74878db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0aa37b640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0900a29f0 0x7fd0900a4eb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00cff60 tx=0x7fd0a00d6690 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878d760 0x55a74878db40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009d80 tx=0x7fd09c00f730 comp rx=0 tx=0).stop 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 -- 172.21.15.22:0/616014299 shutdown_connections 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a748796c60 0x55a748799050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748796720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/616014299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878d760 0x55a74878db40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 -- 172.21.15.22:0/616014299 >> 172.21.15.22:0/616014299 conn(0x55a748787e90 msgr2=0x55a748788270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 -- 172.21.15.22:0/616014299 shutdown_connections 2024-03-09T00:09:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 -- 172.21.15.22:0/616014299 wait complete. 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 -- start start 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.974+0000 7fd0af1c3740 1 Processor -- start 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748796420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878e080 0x55a7487937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a748796c60 0x55a748793cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a748799a50 con 0x55a748796c60 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7486ad0a0 con 0x55a74878d760 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a748799bd0 con 0x55a74878e080 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 10 client.0 _ll_register_callbacks cb 0x55a7486aca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a748796c60 0x55a748793cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a748796c60 0x55a748793cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40934/0 (socket says 172.21.15.22:40934) 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2779204674 learned_addr learned my addr 172.21.15.22:0/2779204674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878e080 0x55a7487937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748796420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878e080 msgr2=0x55a7487937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878e080 0x55a7487937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2779204674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 msgr2=0x55a748796420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748796420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2779204674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7484053f0 con 0x55a748796c60 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a748796c60 0x55a748793cf0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009b40 tx=0x7fd09c005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be68716e7c62cb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd08dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd08dffb640 1 -- 172.21.15.22:0/2779204674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c022040 con 0x55a748796c60 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd08dffb640 1 -- 172.21.15.22:0/2779204674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd09c005410 con 0x55a748796c60 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.975+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd09c004cc0 con 0x55a748796c60 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd08dffb640 1 -- 172.21.15.22:0/2779204674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd09c005610 con 0x55a748796c60 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 10 client.20238 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a7483c50f0 con 0x55a748796c60 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd08dffb640 1 -- 172.21.15.22:0/2779204674 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd09c004cc0 con 0x55a748796c60 2024-03-09T00:09:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd08dffb640 1 -- 172.21.15.22:0/2779204674 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd09c019550 con 0x55a748796c60 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 10 client.20238 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 10 client.20238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 20 client.20238 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487a6650 0x55a7487a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 4 client.20238 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7486fe340 con 0x55a7487a6650 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.976+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487a6650 0x55a7487a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.977+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487a6650 0x55a7487a8a40 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e13b667a80c0db2 server_cookie=d71855192097d159 in_seq=0 out_seq=0 2024-03-09T00:09:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:41.977+0000 7fd08dffb640 10 client.20238 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:42.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 27177 15.04 MB/sec warmup 109 sec latency 121.429 ms 2024-03-09T00:09:42.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:42 smithi022 ceph-mon[31894]: pgmap v480: 129 pgs: 129 active+clean; 335 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 605 op/s 2024-03-09T00:09:42.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:42 smithi062 ceph-mon[36129]: pgmap v480: 129 pgs: 129 active+clean; 335 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 605 op/s 2024-03-09T00:09:42.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:42 smithi192 ceph-mon[35995]: pgmap v480: 129 pgs: 129 active+clean; 335 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 605 op/s 2024-03-09T00:09:43.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 27211 14.92 MB/sec warmup 110 sec latency 767.355 ms 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0338ff9640 1 -- 172.21.15.22:0/7577948 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5572d0bcfe80 con 0x5572d0cd5070 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0338ff9640 10 client.29450 handle_command_reply: tid=0 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 client.29450 shutdown 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c63fd0 msgr2=0x5572d0c74650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c63fd0 0x5572d0c74650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f034c009e50 tx=0x7f034c009e80 comp rx=0 tx=0).stop 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 shutdown_connections 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5572d0cd5070 0x5572d0cd7490 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572d0c6d550 0x5572d0c72bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5572d0c64d40 0x5572d0c74b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 --2- 172.21.15.22:0/7577948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572d0c63fd0 0x5572d0c74650 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0338ff9640 0 client.29450 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.211+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 >> 172.21.15.22:0/7577948 conn(0x5572d0c2b5b0 msgr2=0x5572d0c68de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 shutdown_connections 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f0359f81740 1 -- 172.21.15.22:0/7577948 wait complete. 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f0359f81740 20 client.29450 trim_cache size 0 max 16384 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:09:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:09:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f033c0a2a10 msgr2=0x7f033c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f033c0a2a10 0x7f033c0a4ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f034804d000 tx=0x7f034804a000 comp rx=0 tx=0).stop 2024-03-09T00:09:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501082a0 msgr2=0x7f0350079c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501082a0 0x7f0350079c10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f034000e9c0 tx=0x7f034000ee80 comp rx=0 tx=0).stop 2024-03-09T00:09:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 shutdown_connections 2024-03-09T00:09:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f033c0a2a10 0x7f033c0a4ed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035010d8f0 0x7f035007a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03501082a0 0x7f0350079c10 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 --2- 172.21.15.22:0/191779349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03501078e0 0x7f0350081c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 >> 172.21.15.22:0/191779349 conn(0x7f0350075860 msgr2=0x7f0350075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 shutdown_connections 2024-03-09T00:09:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:43.212+0000 7f03573ab640 1 -- 172.21.15.22:0/191779349 wait complete. 2024-03-09T00:09:43.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.212+0000 7fd08dffb640 1 -- 172.21.15.22:0/2779204674 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7486fe340 con 0x55a7487a6650 2024-03-09T00:09:43.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.212+0000 7fd08dffb640 10 client.20238 handle_command_reply: tid=0 2024-03-09T00:09:43.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 client.20238 shutdown 2024-03-09T00:09:43.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a748796c60 msgr2=0x55a748793cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a748796c60 0x55a748793cf0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009b40 tx=0x7fd09c005aa0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 shutdown_connections 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487a6650 0x55a7487a8a40 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a748796c60 0x55a748793cf0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd08dffb640 0 client.20238 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a74878e080 0x55a7487937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2779204674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748796420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 >> 172.21.15.22:0/2779204674 conn(0x55a748787e90 msgr2=0x55a748789be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 shutdown_connections 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2779204674 wait complete. 2024-03-09T00:09:43.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.213+0000 7fd0af1c3740 20 client.20238 trim_cache size 0 max 16384 2024-03-09T00:09:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.229+0000 7fd0af1c3740 1 Processor -- start 2024-03-09T00:09:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.229+0000 7fd0af1c3740 1 -- start start 2024-03-09T00:09:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748792790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748795ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a748796420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41aeb30 con 0x55a74878e080 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41ac670 con 0x55a74878d760 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41aa730 con 0x55a748796c60 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748795ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748795ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40950/0 (socket says 172.21.15.22:40950) 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 -- 172.21.15.22:0/2519782019 learned_addr learned my addr 172.21.15.22:0/2519782019 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748792790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a748796420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 -- 172.21.15.22:0/2519782019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 msgr2=0x55a748796420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a748796420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 -- 172.21.15.22:0/2519782019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 msgr2=0x55a748792790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748792790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 -- 172.21.15.22:0/2519782019 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a748788c20 con 0x55a74878e080 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748792790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748795ee0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd0a406a960 tx=0x7fd094005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291febce38e54107 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd08ffff640 1 -- 172.21.15.22:0/2519782019 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094049020 con 0x55a74878e080 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd08ffff640 1 -- 172.21.15.22:0/2519782019 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd094004040 con 0x55a74878e080 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.230+0000 7fd08ffff640 1 -- 172.21.15.22:0/2519782019 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094045720 con 0x55a74878e080 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2519782019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 msgr2=0x55a748795ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748795ee0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd0a406a960 tx=0x7fd094005a20 comp rx=0 tx=0).stop 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2519782019 shutdown_connections 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a748796420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878e080 0x55a748795ee0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2519782019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878d760 0x55a748792790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2519782019 >> 172.21.15.22:0/2519782019 conn(0x55a748787e90 msgr2=0x55a748792080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2519782019 shutdown_connections 2024-03-09T00:09:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2519782019 wait complete. 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.231+0000 7fd0af1c3740 1 -- start start 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 1 Processor -- start 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878d760 0x55a74879e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878e080 0x55a74879c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a74879cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41aad30 con 0x55a74878d760 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41ac010 con 0x55a74878e080 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41ab350 con 0x55a748796c60 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0af1c3740 10 client.0 _ll_register_callbacks cb 0x7fd0a00d6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878d760 0x55a74879e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878d760 0x55a74879e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40956/0 (socket says 172.21.15.22:40956) 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2278330628 learned_addr learned my addr 172.21.15.22:0/2278330628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878e080 0x55a74879c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a74879cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 msgr2=0x55a74879cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a74879cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2278330628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878e080 msgr2=0x55a74879c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878e080 0x55a74879c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 -- 172.21.15.22:0/2278330628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a74879d350 con 0x55a74878d760 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878e080 0x55a74879c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd0aab7c640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878d760 0x55a74879e1b0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0058d0 tx=0x7fd09c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b5b8b84c0edf64 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.232+0000 7fd08dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd08dffb640 1 -- 172.21.15.22:0/2278330628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c03c020 con 0x55a74878d760 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd08dffb640 1 -- 172.21.15.22:0/2278330628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd09c00fe40 con 0x55a74878d760 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd094004040 con 0x55a74878d760 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd08dffb640 1 -- 172.21.15.22:0/2278330628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd09c044050 con 0x55a74878d760 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 10 client.20244 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a7487fda30 con 0x55a74878d760 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd08dffb640 1 -- 172.21.15.22:0/2278330628 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd094004040 con 0x55a74878d760 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd08dffb640 1 -- 172.21.15.22:0/2278330628 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd09c011920 con 0x55a74878d760 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 10 client.20244 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 10 client.20244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 20 client.20244 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487fdfe0 0x55a748800400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 4 client.20244 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:09:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7486f8690 con 0x55a7487fdfe0 2024-03-09T00:09:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.233+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487fdfe0 0x55a748800400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.234+0000 7fd0a9b7a640 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487fdfe0 0x55a748800400 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c399af10af056565 server_cookie=d135d56d05c9df98 in_seq=0 out_seq=0 2024-03-09T00:09:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.234+0000 7fd08dffb640 10 client.20244 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd08dffb640 1 -- 172.21.15.22:0/2278330628 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7486f8690 con 0x55a7487fdfe0 2024-03-09T00:09:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd08dffb640 10 client.20244 handle_command_reply: tid=0 2024-03-09T00:09:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 client.20244 shutdown 2024-03-09T00:09:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878d760 msgr2=0x55a74879e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878d760 0x55a74879e1b0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0058d0 tx=0x7fd09c009f90 comp rx=0 tx=0).stop 2024-03-09T00:09:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 shutdown_connections 2024-03-09T00:09:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55a7487fdfe0 0x55a748800400 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a748796c60 0x55a74879cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd08dffb640 0 client.20244 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a74878e080 0x55a74879c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 --2- 172.21.15.22:0/2278330628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a74878d760 0x55a74879e1b0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 >> 172.21.15.22:0/2278330628 conn(0x55a748787e90 msgr2=0x55a748792910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 shutdown_connections 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 1 -- 172.21.15.22:0/2278330628 wait complete. 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.236+0000 7fd0af1c3740 20 client.20244 trim_cache size 0 max 16384 2024-03-09T00:09:43.238 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:09:43.239 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:09:43.239 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:09:43.239 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:09:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0900a29f0 msgr2=0x7fd0900a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0900a29f0 0x7fd0900a4eb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00cff60 tx=0x7fd0a00d6690 comp rx=0 tx=0).stop 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a41014e0 msgr2=0x7fd0a419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a41014e0 0x7fd0a419d830 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009c20 tx=0x7fd0a000b690 comp rx=0 tx=0).stop 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 shutdown_connections 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0900a29f0 0x7fd0900a4eb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0a41979e0 0x7fd0a4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a41065b0 0x7fd0a419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 --2- 172.21.15.22:0/328750344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a41014e0 0x7fd0a419d830 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 >> 172.21.15.22:0/328750344 conn(0x7fd0a40fcd80 msgr2=0x7fd0a40fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 shutdown_connections 2024-03-09T00:09:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.237+0000 7fd0ac5df640 1 -- 172.21.15.22:0/328750344 wait complete. 2024-03-09T00:09:43.243 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:09:43.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:09:43.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 -- 172.21.15.22:0/5194427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c101ea0 msgr2=0x7f740c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 --2- 172.21.15.22:0/5194427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c101ea0 0x7f740c105ee0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7400009900 tx=0x7f740002bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:43.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 -- 172.21.15.22:0/5194427 shutdown_connections 2024-03-09T00:09:43.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 --2- 172.21.15.22:0/5194427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c1065c0 0x7f740c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 --2- 172.21.15.22:0/5194427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c101ea0 0x7f740c105ee0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 --2- 172.21.15.22:0/5194427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c1014f0 0x7f740c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 -- 172.21.15.22:0/5194427 >> 172.21.15.22:0/5194427 conn(0x7f740c0fcd90 msgr2=0x7f740c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 -- 172.21.15.22:0/5194427 shutdown_connections 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.357+0000 7f7411169640 1 -- 172.21.15.22:0/5194427 wait complete. 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.358+0000 7f7411169640 1 Processor -- start 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f7411169640 1 -- start start 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f7411169640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c1014f0 0x7f740c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f7411169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c1065c0 0x7f740c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f7411169640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c197a10 0x7f740c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f7411169640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c111650 con 0x7f740c1065c0 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f7411169640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c1114d0 con 0x7f740c1014f0 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f7411169640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c1117d0 con 0x7f740c197a10 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c1065c0 0x7f740c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c1014f0 0x7f740c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c1065c0 0x7f740c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40974/0 (socket says 172.21.15.22:40974) 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 -- 172.21.15.22:0/1855607113 learned_addr learned my addr 172.21.15.22:0/1855607113 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740b59e640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c197a10 0x7f740c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 -- 172.21.15.22:0/1855607113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c197a10 msgr2=0x7f740c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c197a10 0x7f740c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 -- 172.21.15.22:0/1855607113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c1014f0 msgr2=0x7f740c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c1014f0 0x7f740c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.359+0000 7f740a59c640 1 -- 172.21.15.22:0/1855607113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c198530 con 0x7f740c1065c0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.360+0000 7f740a59c640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c1065c0 0x7f740c19dd50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7400036040 tx=0x7f740002cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.360+0000 7f73f3fff640 1 -- 172.21.15.22:0/1855607113 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400041070 con 0x7f740c1065c0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.360+0000 7f73f3fff640 1 -- 172.21.15.22:0/1855607113 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7400004450 con 0x7f740c1065c0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.360+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c1987c0 con 0x7f740c1065c0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.360+0000 7f73f3fff640 1 -- 172.21.15.22:0/1855607113 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400003dc0 con 0x7f740c1065c0 2024-03-09T00:09:43.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.360+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c1a4960 con 0x7f740c1065c0 2024-03-09T00:09:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.361+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c068530 con 0x7f740c1065c0 2024-03-09T00:09:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.362+0000 7f73f3fff640 1 -- 172.21.15.22:0/1855607113 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7400044070 con 0x7f740c1065c0 2024-03-09T00:09:43.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.363+0000 7f73f3fff640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f73f40a2970 0x7f73f40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.363+0000 7f73f3fff640 1 -- 172.21.15.22:0/1855607113 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f74000f0c50 con 0x7f740c1065c0 2024-03-09T00:09:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.363+0000 7f740ad9d640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f73f40a2970 0x7f73f40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.363+0000 7f740ad9d640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f73f40a2970 0x7f73f40a4e30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f740c1004d0 tx=0x7f73fc005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.368+0000 7f73f3fff640 1 -- 172.21.15.22:0/1855607113 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f74000bd020 con 0x7f740c1065c0 2024-03-09T00:09:43.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.693+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f740c101ea0 con 0x7f740c1065c0 2024-03-09T00:09:43.690 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:09:43.690 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:07:54.629831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:09:43.691 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-03-09T00:09:43.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.695+0000 7f73f3fff640 1 -- 172.21.15.22:0/1855607113 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f74000c10c0 con 0x7f740c1065c0 2024-03-09T00:09:43.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f73f40a2970 msgr2=0x7f73f40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f73f40a2970 0x7f73f40a4e30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f740c1004d0 tx=0x7f73fc005f90 comp rx=0 tx=0).stop 2024-03-09T00:09:43.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c1065c0 msgr2=0x7f740c19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c1065c0 0x7f740c19dd50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7400036040 tx=0x7f740002cb20 comp rx=0 tx=0).stop 2024-03-09T00:09:43.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 shutdown_connections 2024-03-09T00:09:43.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f73f40a2970 0x7f73f40a4e30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c197a10 0x7f740c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f740c1065c0 0x7f740c19dd50 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 --2- 172.21.15.22:0/1855607113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c1014f0 0x7f740c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 >> 172.21.15.22:0/1855607113 conn(0x7f740c0fcd90 msgr2=0x7f740c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.699+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 shutdown_connections 2024-03-09T00:09:43.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.700+0000 7f7411169640 1 -- 172.21.15.22:0/1855607113 wait complete. 2024-03-09T00:09:43.714 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:09:43.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:09:43.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 -- 172.21.15.22:0/1935506297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc102510 msgr2=0x7fbfdc1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1935506297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc102510 0x7fbfdc1028f0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009900 tx=0x7fbfd802bcd0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 -- 172.21.15.22:0/1935506297 shutdown_connections 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1935506297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfdc10b850 0x7fbfdc10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1935506297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfdc102e30 0x7fbfdc10b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1935506297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc102510 0x7fbfdc1028f0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 -- 172.21.15.22:0/1935506297 >> 172.21.15.22:0/1935506297 conn(0x7fbfdc0773a0 msgr2=0x7fbfdc0777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 -- 172.21.15.22:0/1935506297 shutdown_connections 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.891+0000 7fbfe4769640 1 -- 172.21.15.22:0/1935506297 wait complete. 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 Processor -- start 2024-03-09T00:09:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 -- start start 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfdc102510 0x7fbfdc100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfdc102e30 0x7fbfdc1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc10b850 0x7fbfdc100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc111610 con 0x7fbfdc10b850 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc111490 con 0x7fbfdc102510 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe4769640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc111790 con 0x7fbfdc102e30 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc10b850 0x7fbfdc100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc10b850 0x7fbfdc100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40988/0 (socket says 172.21.15.22:40988) 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 -- 172.21.15.22:0/1286659368 learned_addr learned my addr 172.21.15.22:0/1286659368 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfdc102e30 0x7fbfdc1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2505640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfdc102510 0x7fbfdc100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 -- 172.21.15.22:0/1286659368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfdc102e30 msgr2=0x7fbfdc1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfdc102e30 0x7fbfdc1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 -- 172.21.15.22:0/1286659368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfdc102510 msgr2=0x7fbfdc100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfdc102510 0x7fbfdc100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.892+0000 7fbfe2d06640 1 -- 172.21.15.22:0/1286659368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc1a6010 con 0x7fbfdc10b850 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.893+0000 7fbfe2d06640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc10b850 0x7fbfdc100be0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400eb10 tx=0x7fbfd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.893+0000 7fbfd37fe640 1 -- 172.21.15.22:0/1286659368 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4004b20 con 0x7fbfdc10b850 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.893+0000 7fbfd37fe640 1 -- 172.21.15.22:0/1286659368 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd4004cc0 con 0x7fbfdc10b850 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.893+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc1a6300 con 0x7fbfdc10b850 2024-03-09T00:09:43.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.893+0000 7fbfd37fe640 1 -- 172.21.15.22:0/1286659368 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4010760 con 0x7fbfdc10b850 2024-03-09T00:09:43.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.893+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc1a6850 con 0x7fbfdc10b850 2024-03-09T00:09:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.895+0000 7fbfd37fe640 1 -- 172.21.15.22:0/1286659368 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbfd4005020 con 0x7fbfdc10b850 2024-03-09T00:09:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 Processor -- start 2024-03-09T00:09:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 -- start start 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e30160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e32a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561698dee360 con 0x561698e271a0 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561698d8aec0 con 0x561698e306a0 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe7427740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561698d33d30 con 0x561698e27ac0 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe1d04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe2505640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e30160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe1d04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e27580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40998/0 (socket says 172.21.15.22:40998) 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfd37fe640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbfc80a29f0 0x7fbfc80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.896+0000 7fbfe1d04640 1 -- 172.21.15.22:0/732759961 learned_addr learned my addr 172.21.15.22:0/732759961 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2d06640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbfc80a29f0 0x7fbfc80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e32a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2505640 1 -- 172.21.15.22:0/732759961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 msgr2=0x561698e32a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2505640 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e32a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2505640 1 -- 172.21.15.22:0/732759961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 msgr2=0x561698e27580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2505640 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e27580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2505640 1 -- 172.21.15.22:0/732759961 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561698d32bc0 con 0x561698e27ac0 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfd37fe640 1 -- 172.21.15.22:0/1286659368 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fbfd40cf7b0 con 0x7fbfdc10b850 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2505640 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e30160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009230 tx=0x7fbfd8005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc394db97b59d681 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfe2d06640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbfc80a29f0 0x7fbfc80a4eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4097e80 tx=0x7fbfd4097d40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfd0ff9640 1 -- 172.21.15.22:0/732759961 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8042070 con 0x561698e27ac0 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfd0ff9640 1 -- 172.21.15.22:0/732759961 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd802cb20 con 0x561698e27ac0 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.897+0000 7fbfd0ff9640 1 -- 172.21.15.22:0/732759961 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80046c0 con 0x561698e27ac0 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 -- 172.21.15.22:0/732759961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 msgr2=0x561698e30160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e30160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009230 tx=0x7fbfd8005d50 comp rx=0 tx=0).stop 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 -- 172.21.15.22:0/732759961 shutdown_connections 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e32a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e30160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 --2- 172.21.15.22:0/732759961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e27580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 -- 172.21.15.22:0/732759961 >> 172.21.15.22:0/732759961 conn(0x561698e218d0 msgr2=0x561698e21cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 -- 172.21.15.22:0/732759961 shutdown_connections 2024-03-09T00:09:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 -- 172.21.15.22:0/732759961 wait complete. 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 -- start start 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.898+0000 7fbfe7427740 1 Processor -- start 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e271a0 0x561698e2d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e27ac0 0x561698e2db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e306a0 0x561698e2e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561698d294a0 con 0x561698e27ac0 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561698d86f60 con 0x561698e271a0 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e27ac0 0x561698e2db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e27ac0 0x561698e2db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41004/0 (socket says 172.21.15.22:41004) 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561698e33430 con 0x561698e306a0 2024-03-09T00:09:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 -- 172.21.15.22:0/2352658981 learned_addr learned my addr 172.21.15.22:0/2352658981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e306a0 0x561698e2e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 10 client.0 _ll_register_callbacks cb 0x561698d84d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e271a0 0x561698e2d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 -- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e306a0 msgr2=0x561698e2e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e306a0 0x561698e2e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 -- 172.21.15.22:0/2352658981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e271a0 msgr2=0x561698e2d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e271a0 0x561698e2d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 -- 172.21.15.22:0/2352658981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561698a9f3f0 con 0x561698e27ac0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e306a0 0x561698e2e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe2505640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e27ac0 0x561698e2db20 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbfd802cf00 tx=0x7fbfd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47fe570ff63db97 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/2352658981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8042040 con 0x561698e27ac0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/2352658981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fbfd8004ad0 con 0x561698e27ac0 2024-03-09T00:09:43.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.899+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfd802cb20 con 0x561698e27ac0 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/2352658981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd8004cd0 con 0x561698e27ac0 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 10 client.20262 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561698b697d0 con 0x561698e27ac0 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/2352658981 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfd802cb20 con 0x561698e27ac0 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/2352658981 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbfd8034440 con 0x561698e27ac0 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 10 client.20262 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 10 client.20262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 20 client.20262 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 4 client.20262 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561698d98400 con 0x561698e405d0 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.900+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.901+0000 7fbfc5ffb640 10 client.20262 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:43.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.901+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e429c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=707c8973be57d329 server_cookie=66327a9edf583e63 in_seq=0 out_seq=0 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.901+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/2352658981 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561698d98400 con 0x561698e405d0 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.901+0000 7fbfc5ffb640 10 client.20262 handle_command_reply: tid=0 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 client.20262 shutdown 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e27ac0 msgr2=0x561698e2db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e27ac0 0x561698e2db20 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbfd802cf00 tx=0x7fbfd8009260 comp rx=0 tx=0).stop 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 shutdown_connections 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e429c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e306a0 0x561698e2e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfc5ffb640 0 client.20262 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e27ac0 0x561698e2db20 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2352658981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e271a0 0x561698e2d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 >> 172.21.15.22:0/2352658981 conn(0x561698e218d0 msgr2=0x561698e23620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 shutdown_connections 2024-03-09T00:09:43.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 1 -- 172.21.15.22:0/2352658981 wait complete. 2024-03-09T00:09:43.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.902+0000 7fbfe7427740 20 client.20262 trim_cache size 0 max 16384 2024-03-09T00:09:43.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 Processor -- start 2024-03-09T00:09:43.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 -- start start 2024-03-09T00:09:43.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e2a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e2ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1b2ce0 con 0x561698e271a0 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1b0820 con 0x561698e306a0 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe7427740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1ae8e0 con 0x561698e27ac0 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe1d04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe1d04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e2fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41016/0 (socket says 172.21.15.22:41016) 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe1d04640 1 -- 172.21.15.22:0/2237431034 learned_addr learned my addr 172.21.15.22:0/2237431034 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe2505640 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e2a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.914+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e2ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe1d04640 1 -- 172.21.15.22:0/2237431034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 msgr2=0x561698e2a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e2a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe1d04640 1 -- 172.21.15.22:0/2237431034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 msgr2=0x561698e2ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e2ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe1d04640 1 -- 172.21.15.22:0/2237431034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561698e278f0 con 0x561698e271a0 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe2505640 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e2a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e2fd10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00b6c0 tx=0x7fbfcc00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44aee6855aad7c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfc7fff640 1 -- 172.21.15.22:0/2237431034 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc005590 con 0x561698e271a0 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfc7fff640 1 -- 172.21.15.22:0/2237431034 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc01f750 con 0x561698e271a0 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfc7fff640 1 -- 172.21.15.22:0/2237431034 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc004800 con 0x561698e271a0 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe7427740 1 -- 172.21.15.22:0/2237431034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 msgr2=0x561698e2fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e2fd10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00b6c0 tx=0x7fbfcc00b650 comp rx=0 tx=0).stop 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe7427740 1 -- 172.21.15.22:0/2237431034 shutdown_connections 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e2ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e2a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe7427740 1 --2- 172.21.15.22:0/2237431034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e2fd10 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.915+0000 7fbfe7427740 1 -- 172.21.15.22:0/2237431034 >> 172.21.15.22:0/2237431034 conn(0x561698e218d0 msgr2=0x561698e21cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 -- 172.21.15.22:0/2237431034 shutdown_connections 2024-03-09T00:09:43.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 -- 172.21.15.22:0/2237431034 wait complete. 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 -- start start 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 Processor -- start 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e35d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1aeee0 con 0x561698e271a0 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1b01c0 con 0x561698e306a0 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1af500 con 0x561698e27ac0 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe7427740 10 client.0 _ll_register_callbacks cb 0x7fbfd40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe1d04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe1d04640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41028/0 (socket says 172.21.15.22:41028) 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe1d04640 1 -- 172.21.15.22:0/1672209271 learned_addr learned my addr 172.21.15.22:0/1672209271 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe2505640 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e35d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.916+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe1d04640 1 -- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 msgr2=0x561698e35d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e35d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe1d04640 1 -- 172.21.15.22:0/1672209271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 msgr2=0x561698e362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe1d04640 1 -- 172.21.15.22:0/1672209271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561698e36af0 con 0x561698e271a0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe1d04640 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e379e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00ba60 tx=0x7fbfcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4136cd3fe51bdc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/1672209271 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc004220 con 0x561698e271a0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/1672209271 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fbfcc0043c0 con 0x561698e271a0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfcc01f750 con 0x561698e271a0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/1672209271 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfcc004730 con 0x561698e271a0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe7427740 10 client.20268 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561698e976f0 con 0x561698e271a0 2024-03-09T00:09:43.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/1672209271 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc01f750 con 0x561698e271a0 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/1672209271 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbfcc0038c0 con 0x561698e271a0 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe7427740 10 client.20268 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.917+0000 7fbfe7427740 10 client.20268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 20 client.20268 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e99cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 4 client.20268 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561698d92750 con 0x561698e405d0 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe2505640 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e99cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfc5ffb640 10 client.20268 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe2505640 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e99cf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b80b3bc6f2e19e9 server_cookie=217da09df781abad in_seq=0 out_seq=0 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfc5ffb640 1 -- 172.21.15.22:0/1672209271 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561698d92750 con 0x561698e405d0 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfc5ffb640 10 client.20268 handle_command_reply: tid=0 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 1 client.20268 shutdown 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 msgr2=0x561698e379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e379e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00ba60 tx=0x7fbfcc005830 comp rx=0 tx=0).stop 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 shutdown_connections 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.918+0000 7fbfe7427740 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x561698e405d0 0x561698e99cf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe7427740 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561698e306a0 0x561698e362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe7427740 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561698e27ac0 0x561698e35d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfc5ffb640 0 client.20268 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe7427740 1 --2- 172.21.15.22:0/1672209271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561698e271a0 0x561698e379e0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 >> 172.21.15.22:0/1672209271 conn(0x561698e218d0 msgr2=0x561698e2beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 shutdown_connections 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe7427740 1 -- 172.21.15.22:0/1672209271 wait complete. 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe7427740 20 client.20268 trim_cache size 0 max 16384 2024-03-09T00:09:43.920 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:09:43.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbfc80a29f0 msgr2=0x7fbfc80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbfc80a29f0 0x7fbfc80a4eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4097e80 tx=0x7fbfd4097d40 comp rx=0 tx=0).stop 2024-03-09T00:09:43.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc10b850 msgr2=0x7fbfdc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc10b850 0x7fbfdc100be0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400eb10 tx=0x7fbfd400c2d0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 shutdown_connections 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbfc80a29f0 0x7fbfc80a4eb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbfdc10b850 0x7fbfdc100be0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfdc102e30 0x7fbfdc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 --2- 172.21.15.22:0/1286659368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfdc102510 0x7fbfdc100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 >> 172.21.15.22:0/1286659368 conn(0x7fbfdc0773a0 msgr2=0x7fbfdc0fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 shutdown_connections 2024-03-09T00:09:43.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:43.919+0000 7fbfe4769640 1 -- 172.21.15.22:0/1286659368 wait complete. 2024-03-09T00:09:43.925 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:09:43.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:09:44.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 -- 172.21.15.22:0/1961575755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 msgr2=0x7f17b81007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 --2- 172.21.15.22:0/1961575755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b81007b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009900 tx=0x7f17a802bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:44.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 -- 172.21.15.22:0/1961575755 shutdown_connections 2024-03-09T00:09:44.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 --2- 172.21.15.22:0/1961575755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81054a0 0x7f17b810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 --2- 172.21.15.22:0/1961575755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b8100d80 0x7f17b8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 --2- 172.21.15.22:0/1961575755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b81007b0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 -- 172.21.15.22:0/1961575755 >> 172.21.15.22:0/1961575755 conn(0x7f17b8077320 msgr2=0x7f17b8077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 -- 172.21.15.22:0/1961575755 shutdown_connections 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.048+0000 7f17be09a640 1 -- 172.21.15.22:0/1961575755 wait complete. 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 Processor -- start 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 -- start start 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b8101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81054a0 0x7f17b81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b81028e0 0x7f17b81a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8110560 con 0x7f17b81003d0 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81103e0 con 0x7f17b81028e0 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17be09a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81106e0 con 0x7f17b81054a0 2024-03-09T00:09:44.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b8101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b8101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41036/0 (socket says 172.21.15.22:41036) 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 -- 172.21.15.22:0/2362070080 learned_addr learned my addr 172.21.15.22:0/2362070080 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81054a0 0x7f17b81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b7fff640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b81028e0 0x7f17b81a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 -- 172.21.15.22:0/2362070080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81054a0 msgr2=0x7f17b81023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81054a0 0x7f17b81023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 -- 172.21.15.22:0/2362070080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b81028e0 msgr2=0x7f17b81a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b81028e0 0x7f17b81a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b77fe640 1 -- 172.21.15.22:0/2362070080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b81a6410 con 0x7f17b81003d0 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.049+0000 7f17b7fff640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b81028e0 0x7f17b81a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.050+0000 7f17b77fe640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b8101e60 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006010 tx=0x7f17a802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.050+0000 7f17b4ff9640 1 -- 172.21.15.22:0/2362070080 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8041070 con 0x7f17b81003d0 2024-03-09T00:09:44.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.050+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b81a66a0 con 0x7f17b81003d0 2024-03-09T00:09:44.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.050+0000 7f17b4ff9640 1 -- 172.21.15.22:0/2362070080 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17a8004450 con 0x7f17b81003d0 2024-03-09T00:09:44.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.050+0000 7f17b4ff9640 1 -- 172.21.15.22:0/2362070080 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8003e00 con 0x7f17b81003d0 2024-03-09T00:09:44.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.050+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b81a6be0 con 0x7f17b81003d0 2024-03-09T00:09:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.052+0000 7f17b4ff9640 1 -- 172.21.15.22:0/2362070080 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f17a8044070 con 0x7f17b81003d0 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.052+0000 7f17c0c83740 1 Processor -- start 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.052+0000 7f17c0c83740 1 -- start start 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.052+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9cfc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.052+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d05160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.052+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d07a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17c0c83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9cc3360 con 0x5595b9cfcac0 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17c0c83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9c5fec0 con 0x5595b9d056a0 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17c0c83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9c08d30 con 0x5595b9cfc1a0 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b4ff9640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17a00a29a0 0x7f17a00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9cfc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9cfc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59800/0 (socket says 172.21.15.22:59800) 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b7fff640 1 -- 172.21.15.22:0/1974791958 learned_addr learned my addr 172.21.15.22:0/1974791958 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b77fe640 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d05160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d07a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b4ff9640 1 -- 172.21.15.22:0/2362070080 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f17a80f0d20 con 0x7f17b81003d0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b7fff640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17a00a29a0 0x7f17a00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1974791958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 msgr2=0x5595b9cfc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9cfc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1974791958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 msgr2=0x5595b9d05160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d05160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1974791958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b9c07bc0 con 0x5595b9d056a0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b7fff640 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9cfc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b77fe640 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d05160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b7fff640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17a00a29a0 0x7f17a00a4e60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009fd0 tx=0x7f17b0004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.053+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d07a90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005d00 tx=0x7f17ac0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639c8b56e68f38ec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f179dffb640 1 -- 172.21.15.22:0/1974791958 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac01c480 con 0x5595b9d056a0 2024-03-09T00:09:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f179dffb640 1 -- 172.21.15.22:0/1974791958 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17ac01cb00 con 0x5595b9d056a0 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f179dffb640 1 -- 172.21.15.22:0/1974791958 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac011b50 con 0x5595b9d056a0 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 -- 172.21.15.22:0/1974791958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 msgr2=0x5595b9d07a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d07a90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005d00 tx=0x7f17ac0107a0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 -- 172.21.15.22:0/1974791958 shutdown_connections 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d07a90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d05160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1974791958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9cfc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 -- 172.21.15.22:0/1974791958 >> 172.21.15.22:0/1974791958 conn(0x5595b9cf68d0 msgr2=0x5595b9cf6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 -- 172.21.15.22:0/1974791958 shutdown_connections 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.054+0000 7f17c0c83740 1 -- 172.21.15.22:0/1974791958 wait complete. 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 -- start start 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 Processor -- start 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9d02630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d02b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9bfe4a0 con 0x5595b9cfcac0 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9c5bf60 con 0x5595b9d056a0 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9d08490 con 0x5595b9cfc1a0 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17c0c83740 10 client.0 _ll_register_callbacks cb 0x5595b9c59d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d02b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d02b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41042/0 (socket says 172.21.15.22:41042) 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17b6ffd640 1 -- 172.21.15.22:0/2537337573 learned_addr learned my addr 172.21.15.22:0/2537337573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17b77fe640 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9d02630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.055+0000 7f17b7fff640 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17b6ffd640 1 -- 172.21.15.22:0/2537337573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 msgr2=0x5595b9d02630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9d02630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17b6ffd640 1 -- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 msgr2=0x5595b9d030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17b6ffd640 1 -- 172.21.15.22:0/2537337573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b99743f0 con 0x5595b9cfcac0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d02b70 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00bfb0 tx=0x7f17ac011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dc3b9b0e2a4760 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f1796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f1796ffd640 1 -- 172.21.15.22:0/2537337573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac02a060 con 0x5595b9cfcac0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f1796ffd640 1 -- 172.21.15.22:0/2537337573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f17ac01c690 con 0x5595b9cfcac0 2024-03-09T00:09:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17ac01cb00 con 0x5595b9cfcac0 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f1796ffd640 1 -- 172.21.15.22:0/2537337573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17ac031050 con 0x5595b9cfcac0 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17c0c83740 10 client.20280 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5595b9a3e7d0 con 0x5595b9cfcac0 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.056+0000 7f1796ffd640 1 -- 172.21.15.22:0/2537337573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17ac01cb00 con 0x5595b9cfcac0 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f1796ffd640 1 -- 172.21.15.22:0/2537337573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f17ac011bd0 con 0x5595b9cfcac0 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17c0c83740 10 client.20280 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17c0c83740 10 client.20280 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17c0c83740 20 client.20280 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17c0c83740 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d157a0 0x5595b9d17b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17c0c83740 4 client.20280 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595b9c6d400 con 0x5595b9d157a0 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17b77fe640 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d157a0 0x5595b9d17b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f1796ffd640 10 client.20280 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:44.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.057+0000 7f17b77fe640 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d157a0 0x5595b9d17b90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b5760bdbdc39f08 server_cookie=261bb6d23290f917 in_seq=0 out_seq=0 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f1796ffd640 1 -- 172.21.15.22:0/2537337573 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595b9c6d400 con 0x5595b9d157a0 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f1796ffd640 10 client.20280 handle_command_reply: tid=0 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 client.20280 shutdown 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 msgr2=0x5595b9d02b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d02b70 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00bfb0 tx=0x7f17ac011630 comp rx=0 tx=0).stop 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 shutdown_connections 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d157a0 0x5595b9d17b90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9d02b70 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f1796ffd640 0 client.20280 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 --2- 172.21.15.22:0/2537337573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfc1a0 0x5595b9d02630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 >> 172.21.15.22:0/2537337573 conn(0x5595b9cf68d0 msgr2=0x5595b9cf8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 shutdown_connections 2024-03-09T00:09:44.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 1 -- 172.21.15.22:0/2537337573 wait complete. 2024-03-09T00:09:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.058+0000 7f17c0c83740 20 client.20280 trim_cache size 0 max 16384 2024-03-09T00:09:44.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.070+0000 7f17c0c83740 1 Processor -- start 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.070+0000 7f17c0c83740 1 -- start start 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9cfc1a0 0x5595b9d04d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9cff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9d056a0 0x5595b9cffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17c0c83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81b2e20 con 0x5595b9cfcac0 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17c0c83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81b0960 con 0x5595b9cfc1a0 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17c0c83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81aea20 con 0x5595b9d056a0 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9cff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9cff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41048/0 (socket says 172.21.15.22:41048) 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 -- 172.21.15.22:0/3979325849 learned_addr learned my addr 172.21.15.22:0/3979325849 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b7fff640 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9d056a0 0x5595b9cffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b77fe640 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9cfc1a0 0x5595b9d04d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 -- 172.21.15.22:0/3979325849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9d056a0 msgr2=0x5595b9cffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9d056a0 0x5595b9cffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 -- 172.21.15.22:0/3979325849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9cfc1a0 msgr2=0x5595b9d04d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9cfc1a0 0x5595b9d04d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 -- 172.21.15.22:0/3979325849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b9cf6700 con 0x5595b9cfcac0 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b77fe640 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9cfc1a0 0x5595b9d04d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9cff890 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00b470 tx=0x7f17ac0058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287c87006d14f479 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f179d7fa640 1 -- 172.21.15.22:0/3979325849 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac02a070 con 0x5595b9cfcac0 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f179d7fa640 1 -- 172.21.15.22:0/3979325849 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17ac011440 con 0x5595b9cfcac0 2024-03-09T00:09:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.071+0000 7f179d7fa640 1 -- 172.21.15.22:0/3979325849 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004c30 con 0x5595b9cfcac0 2024-03-09T00:09:44.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 27475 14.97 MB/sec warmup 111 sec latency 76.812 ms 2024-03-09T00:09:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 -- 172.21.15.22:0/3979325849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 msgr2=0x5595b9cff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9cff890 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00b470 tx=0x7f17ac0058c0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 -- 172.21.15.22:0/3979325849 shutdown_connections 2024-03-09T00:09:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9d056a0 0x5595b9cffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfcac0 0x5595b9cff890 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 --2- 172.21.15.22:0/3979325849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9cfc1a0 0x5595b9d04d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 -- 172.21.15.22:0/3979325849 >> 172.21.15.22:0/3979325849 conn(0x5595b9cf68d0 msgr2=0x5595b9d00f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 -- 172.21.15.22:0/3979325849 shutdown_connections 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 -- 172.21.15.22:0/3979325849 wait complete. 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.072+0000 7f17c0c83740 1 -- start start 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 1 Processor -- start 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfc1a0 0x5595b9d0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfcac0 0x5595b9d0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d0bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81af020 con 0x5595b9cfc1a0 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81b0300 con 0x5595b9d056a0 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81af640 con 0x5595b9cfcac0 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17c0c83740 10 client.0 _ll_register_callbacks cb 0x7f17ac00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfc1a0 0x5595b9d0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfc1a0 0x5595b9d0b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41058/0 (socket says 172.21.15.22:41058) 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 -- 172.21.15.22:0/1805037496 learned_addr learned my addr 172.21.15.22:0/1805037496 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfcac0 0x5595b9d0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b7fff640 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d0bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 -- 172.21.15.22:0/1805037496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfcac0 msgr2=0x5595b9d0b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfcac0 0x5595b9d0b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 -- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 msgr2=0x5595b9d0bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d0bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b77fe640 1 -- 172.21.15.22:0/1805037496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b9d07a90 con 0x5595b9cfc1a0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.073+0000 7f17b7fff640 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d0bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17b77fe640 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfc1a0 0x5595b9d0b340 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f17a802cfd0 tx=0x7f17a808cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89a3d23a9e2977d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f1796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f1796ffd640 1 -- 172.21.15.22:0/1805037496 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a804b040 con 0x5595b9cfc1a0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f1796ffd640 1 -- 172.21.15.22:0/1805037496 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f17a80bb3a0 con 0x5595b9cfc1a0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17ac011440 con 0x5595b9cfc1a0 2024-03-09T00:09:44.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f1796ffd640 1 -- 172.21.15.22:0/1805037496 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17a8004c80 con 0x5595b9cfc1a0 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 10 client.20286 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5595b9d6cd10 con 0x5595b9cfc1a0 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f1796ffd640 1 -- 172.21.15.22:0/1805037496 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17ac011440 con 0x5595b9cfc1a0 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f1796ffd640 1 -- 172.21.15.22:0/1805037496 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f17a80ee300 con 0x5595b9cfc1a0 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 10 client.20286 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 10 client.20286 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 20 client.20286 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d6d390 0x5595b9d6f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 4 client.20286 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5595b9c67750 con 0x5595b9d6d390 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.074+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d6d390 0x5595b9d6f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f1796ffd640 10 client.20286 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d6d390 0x5595b9d6f7b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c92728b7af2b0a5d server_cookie=d9f43507f7bddc5e in_seq=0 out_seq=0 2024-03-09T00:09:44.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f1796ffd640 1 -- 172.21.15.22:0/1805037496 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5595b9c67750 con 0x5595b9d6d390 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f1796ffd640 10 client.20286 handle_command_reply: tid=0 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 client.20286 shutdown 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfc1a0 msgr2=0x5595b9d0b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfc1a0 0x5595b9d0b340 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f17a802cfd0 tx=0x7f17a808cb70 comp rx=0 tx=0).stop 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 shutdown_connections 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5595b9d6d390 0x5595b9d6f7b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595b9d056a0 0x5595b9d0bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f1796ffd640 0 client.20286 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5595b9cfcac0 0x5595b9d0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 --2- 172.21.15.22:0/1805037496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5595b9cfc1a0 0x5595b9d0b340 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 >> 172.21.15.22:0/1805037496 conn(0x5595b9cf68d0 msgr2=0x5595b9d01390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 shutdown_connections 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 1 -- 172.21.15.22:0/1805037496 wait complete. 2024-03-09T00:09:44.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.075+0000 7f17c0c83740 20 client.20286 trim_cache size 0 max 16384 2024-03-09T00:09:44.078 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17a00a29a0 msgr2=0x7f17a00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17a00a29a0 0x7f17a00a4e60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009fd0 tx=0x7f17b0004ab0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 msgr2=0x7f17b8101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b8101e60 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006010 tx=0x7f17a802cb30 comp rx=0 tx=0).stop 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 shutdown_connections 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f17a00a29a0 0x7f17a00a4e60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17b81028e0 0x7f17b81a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17b81054a0 0x7f17b81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 --2- 172.21.15.22:0/2362070080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b81003d0 0x7f17b8101e60 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 >> 172.21.15.22:0/2362070080 conn(0x7f17b8077320 msgr2=0x7f17b8069900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 shutdown_connections 2024-03-09T00:09:44.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.076+0000 7f17be09a640 1 -- 172.21.15.22:0/2362070080 wait complete. 2024-03-09T00:09:44.081 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:09:44.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 -- 172.21.15.22:0/1117477405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 msgr2=0x7f9f541066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/1117477405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541066a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f4802bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 -- 172.21.15.22:0/1117477405 shutdown_connections 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/1117477405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100500 0x7f9f5410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/1117477405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f540ffb40 0x7f9f540fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/1117477405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541066a0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 -- 172.21.15.22:0/1117477405 >> 172.21.15.22:0/1117477405 conn(0x7f9f540754f0 msgr2=0x7f9f54075900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 -- 172.21.15.22:0/1117477405 shutdown_connections 2024-03-09T00:09:44.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 -- 172.21.15.22:0/1117477405 wait complete. 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 Processor -- start 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 -- start start 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f540ffb40 0x7f9f54100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.195+0000 7f9f5afa9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100500 0x7f9f54101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f5afa9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f5afa9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54111640 con 0x7f9f541062c0 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f5afa9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541114c0 con 0x7f9f54100500 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f5afa9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541117c0 con 0x7f9f540ffb40 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41078/0 (socket says 172.21.15.22:41078) 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 -- 172.21.15.22:0/4069673316 learned_addr learned my addr 172.21.15.22:0/4069673316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f58d45640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f540ffb40 0x7f9f54100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f53fff640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100500 0x7f9f54101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 -- 172.21.15.22:0/4069673316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f540ffb40 msgr2=0x7f9f54100d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f540ffb40 0x7f9f54100d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 -- 172.21.15.22:0/4069673316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100500 msgr2=0x7f9f54101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100500 0x7f9f54101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 -- 172.21.15.22:0/4069673316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f541a6020 con 0x7f9f541062c0 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f59546640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541017a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00eb10 tx=0x7f9f4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f51ffb640 1 -- 172.21.15.22:0/4069673316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004b20 con 0x7f9f541062c0 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f51ffb640 1 -- 172.21.15.22:0/4069673316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c004cc0 con 0x7f9f541062c0 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f541a6310 con 0x7f9f541062c0 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.196+0000 7f9f51ffb640 1 -- 172.21.15.22:0/4069673316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c010760 con 0x7f9f541062c0 2024-03-09T00:09:44.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.197+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a6860 con 0x7f9f541062c0 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.199+0000 7f9f51ffb640 1 -- 172.21.15.22:0/4069673316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9f4c005020 con 0x7f9f541062c0 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.199+0000 7f9f51ffb640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f440a2a20 0x7f9f440a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 Processor -- start 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f58d45640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f440a2a20 0x7f9f440a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f51ffb640 1 -- 172.21.15.22:0/4069673316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f9f4c0cf6c0 con 0x7f9f541062c0 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 -- start start 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccc8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5bcc8f360 con 0x55f5bccd16a0 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5bcc2bec0 con 0x55f5bccc81a0 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f5db85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5bcbd4d30 con 0x55f5bccc8ac0 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f59546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f58d45640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f440a2a20 0x7f9f440a4ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009230 tx=0x7f9f48005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59866/0 (socket says 172.21.15.22:59866) 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f59546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41092/0 (socket says 172.21.15.22:41092) 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f53fff640 1 -- 172.21.15.22:0/1043779518 learned_addr learned my addr 172.21.15.22:0/1043779518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.200+0000 7f9f58d45640 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccc8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f59546640 1 -- 172.21.15.22:0/1043779518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 msgr2=0x55f5bccd1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f59546640 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f59546640 1 -- 172.21.15.22:0/1043779518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 msgr2=0x55f5bccc8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f59546640 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccc8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f59546640 1 -- 172.21.15.22:0/1043779518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5bcbd3bc0 con 0x55f5bccd16a0 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f53fff640 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f58d45640 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccc8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f59546640 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd3a90 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0cca20 tx=0x7f9f4c09b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18dee204be163f9c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f3effd640 1 -- 172.21.15.22:0/1043779518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0ce070 con 0x55f5bccd16a0 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f3effd640 1 -- 172.21.15.22:0/1043779518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c09b930 con 0x55f5bccd16a0 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.201+0000 7f9f3effd640 1 -- 172.21.15.22:0/1043779518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0cf2b0 con 0x55f5bccd16a0 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 -- 172.21.15.22:0/1043779518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 msgr2=0x55f5bccd3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd3a90 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0cca20 tx=0x7f9f4c09b470 comp rx=0 tx=0).stop 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 -- 172.21.15.22:0/1043779518 shutdown_connections 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd3a90 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 --2- 172.21.15.22:0/1043779518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccc8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 -- 172.21.15.22:0/1043779518 >> 172.21.15.22:0/1043779518 conn(0x55f5bccc28d0 msgr2=0x55f5bccc2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 -- 172.21.15.22:0/1043779518 shutdown_connections 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 -- 172.21.15.22:0/1043779518 wait complete. 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 -- start start 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.202+0000 7f9f5db85740 1 Processor -- start 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f5db85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5bcbca4a0 con 0x55f5bccc8ac0 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f5db85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5bcc27f60 con 0x55f5bccd16a0 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f5db85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5bccd4490 con 0x55f5bccc81a0 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f5db85740 10 client.0 _ll_register_callbacks cb 0x55f5bcc25d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41108/0 (socket says 172.21.15.22:41108) 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 -- 172.21.15.22:0/2631250092 learned_addr learned my addr 172.21.15.22:0/2631250092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f59546640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f58d45640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 -- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 msgr2=0x55f5bccce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 -- 172.21.15.22:0/2631250092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 msgr2=0x55f5bcccf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 -- 172.21.15.22:0/2631250092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5bc9403f0 con 0x55f5bccc8ac0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f59546640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f53fff640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccec10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9f40012770 tx=0x7f9f40012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcef1f337a1a34f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.203+0000 7f9f33fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f33fff640 1 -- 172.21.15.22:0/2631250092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40004020 con 0x55f5bccc8ac0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f33fff640 1 -- 172.21.15.22:0/2631250092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f9f400041c0 con 0x55f5bccc8ac0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f4c09b930 con 0x55f5bccc8ac0 2024-03-09T00:09:44.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f33fff640 1 -- 172.21.15.22:0/2631250092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f400043c0 con 0x55f5bccc8ac0 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 10 client.20298 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f5bca0a7d0 con 0x55f5bccc8ac0 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f33fff640 1 -- 172.21.15.22:0/2631250092 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c09b930 con 0x55f5bccc8ac0 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f33fff640 1 -- 172.21.15.22:0/2631250092 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9f40003bf0 con 0x55f5bccc8ac0 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 10 client.20298 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 20 client.20298 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcce1900 0x55f5bcce3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 4 client.20298 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.204+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5bcc39400 con 0x55f5bcce1900 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.205+0000 7f9f58d45640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcce1900 0x55f5bcce3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.205+0000 7f9f58d45640 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcce1900 0x55f5bcce3cf0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac10770ba9b00982 server_cookie=749d1b2f9a54ea17 in_seq=0 out_seq=0 2024-03-09T00:09:44.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.205+0000 7f9f33fff640 10 client.20298 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:44.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f33fff640 1 -- 172.21.15.22:0/2631250092 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5bcc39400 con 0x55f5bcce1900 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f33fff640 10 client.20298 handle_command_reply: tid=0 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 client.20298 shutdown 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 msgr2=0x55f5bcccec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccec10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9f40012770 tx=0x7f9f40012c30 comp rx=0 tx=0).stop 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 shutdown_connections 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcce1900 0x55f5bcce3cf0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccec10 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 --2- 172.21.15.22:0/2631250092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f33fff640 0 client.20298 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 >> 172.21.15.22:0/2631250092 conn(0x55f5bccc28d0 msgr2=0x55f5bccc4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 shutdown_connections 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 1 -- 172.21.15.22:0/2631250092 wait complete. 2024-03-09T00:09:44.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.208+0000 7f9f5db85740 20 client.20298 trim_cache size 0 max 16384 2024-03-09T00:09:44.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.220+0000 7f9f5db85740 1 Processor -- start 2024-03-09T00:09:44.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.220+0000 7f9f5db85740 1 -- start start 2024-03-09T00:09:44.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccd0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f5db85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541b2e20 con 0x55f5bccc8ac0 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f5db85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541b0960 con 0x55f5bccd16a0 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f5db85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541aea20 con 0x55f5bccc81a0 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f53fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41110/0 (socket says 172.21.15.22:41110) 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f53fff640 1 -- 172.21.15.22:0/541452723 learned_addr learned my addr 172.21.15.22:0/541452723 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f59546640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f58d45640 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccd0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f59546640 1 -- 172.21.15.22:0/541452723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 msgr2=0x55f5bccd0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f59546640 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccd0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f59546640 1 -- 172.21.15.22:0/541452723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 msgr2=0x55f5bcccb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f59546640 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f59546640 1 -- 172.21.15.22:0/541452723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5bccbcda0 con 0x55f5bccd16a0 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f53fff640 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f59546640 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccbe70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c09fc40 tx=0x7f9f4c097fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0c2db86aee1516 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f3e7fc640 1 -- 172.21.15.22:0/541452723 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0ce070 con 0x55f5bccd16a0 2024-03-09T00:09:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f3e7fc640 1 -- 172.21.15.22:0/541452723 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c09fa80 con 0x55f5bccd16a0 2024-03-09T00:09:44.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.221+0000 7f9f3e7fc640 1 -- 172.21.15.22:0/541452723 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0a3c40 con 0x55f5bccd16a0 2024-03-09T00:09:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 -- 172.21.15.22:0/541452723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 msgr2=0x55f5bcccbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccbe70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c09fc40 tx=0x7f9f4c097fa0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 -- 172.21.15.22:0/541452723 shutdown_connections 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccd16a0 0x55f5bcccbe70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccc8ac0 0x55f5bcccb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 --2- 172.21.15.22:0/541452723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc81a0 0x55f5bccd0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 -- 172.21.15.22:0/541452723 >> 172.21.15.22:0/541452723 conn(0x55f5bccc28d0 msgr2=0x55f5bcce8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 -- 172.21.15.22:0/541452723 shutdown_connections 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.222+0000 7f9f5db85740 1 -- 172.21.15.22:0/541452723 wait complete. 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 -- start start 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 Processor -- start 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccd7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541af020 con 0x55f5bccd16a0 2024-03-09T00:09:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541b0300 con 0x55f5bccc81a0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541af640 con 0x55f5bccc8ac0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f5db85740 10 client.0 _ll_register_callbacks cb 0x7f9f40009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f59546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f59546640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41124/0 (socket says 172.21.15.22:41124) 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f59546640 1 -- 172.21.15.22:0/3516114855 learned_addr learned my addr 172.21.15.22:0/3516114855 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f53fff640 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.223+0000 7f9f58d45640 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccd7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f59546640 1 -- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 msgr2=0x55f5bccd7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f59546640 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f59546640 1 -- 172.21.15.22:0/3516114855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 msgr2=0x55f5bccd7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f59546640 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccd7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f59546640 1 -- 172.21.15.22:0/3516114855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5bcd38020 con 0x55f5bccd16a0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f59546640 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd7dc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0049d0 tx=0x7f9f4c09b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd291967f5ecb4e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f33fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:44.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f33fff640 1 -- 172.21.15.22:0/3516114855 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0ce030 con 0x55f5bccd16a0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f33fff640 1 -- 172.21.15.22:0/3516114855 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f9f4c0862d0 con 0x55f5bccd16a0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5bcc33750 con 0x55f5bccd16a0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f33fff640 1 -- 172.21.15.22:0/3516114855 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4c09b930 con 0x55f5bccd16a0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f5db85740 10 client.20304 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f5bcd39300 con 0x55f5bccd16a0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f33fff640 1 -- 172.21.15.22:0/3516114855 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c0a40d0 con 0x55f5bccd16a0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.224+0000 7f9f33fff640 1 -- 172.21.15.22:0/3516114855 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9f4c09adf0 con 0x55f5bccd16a0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f5db85740 10 client.20304 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f5db85740 10 client.20304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f5db85740 20 client.20304 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f5db85740 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcd39580 0x55f5bcd3b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f5db85740 4 client.20304 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5bcc33750 con 0x55f5bcd39580 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f58d45640 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcd39580 0x55f5bcd3b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f58d45640 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcd39580 0x55f5bcd3b970 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54992b76d5b0647 server_cookie=304ccfdc4492a015 in_seq=0 out_seq=0 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.225+0000 7f9f33fff640 10 client.20304 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f33fff640 1 -- 172.21.15.22:0/3516114855 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5bcc33750 con 0x55f5bcd39580 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f33fff640 10 client.20304 handle_command_reply: tid=0 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 client.20304 shutdown 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 msgr2=0x55f5bccd7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd7dc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0049d0 tx=0x7f9f4c09b5a0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 shutdown_connections 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f5bcd39580 0x55f5bcd3b970 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f5bccd16a0 0x55f5bccd7dc0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5bccc8ac0 0x55f5bccd7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f33fff640 0 client.20304 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 --2- 172.21.15.22:0/3516114855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5bccc81a0 0x55f5bccd7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.226+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 >> 172.21.15.22:0/3516114855 conn(0x55f5bccc28d0 msgr2=0x55f5bccc1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 shutdown_connections 2024-03-09T00:09:44.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5db85740 1 -- 172.21.15.22:0/3516114855 wait complete. 2024-03-09T00:09:44.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5db85740 20 client.20304 trim_cache size 0 max 16384 2024-03-09T00:09:44.227 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f440a2a20 msgr2=0x7f9f440a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f440a2a20 0x7f9f440a4ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009230 tx=0x7f9f48005fd0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 msgr2=0x7f9f541017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541017a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00eb10 tx=0x7f9f4c00c2d0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 shutdown_connections 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f9f440a2a20 0x7f9f440a4ee0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f541062c0 0x7f9f541017a0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100500 0x7f9f54101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 --2- 172.21.15.22:0/4069673316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f540ffb40 0x7f9f54100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:44.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 >> 172.21.15.22:0/4069673316 conn(0x7f9f540754f0 msgr2=0x7f9f540fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:44.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 shutdown_connections 2024-03-09T00:09:44.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:44.227+0000 7f9f5afa9640 1 -- 172.21.15.22:0/4069673316 wait complete. 2024-03-09T00:09:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:44 smithi022 ceph-mon[31894]: pgmap v481: 129 pgs: 129 active+clean; 335 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-03-09T00:09:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:44 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1855607113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:09:44.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:44 smithi062 ceph-mon[36129]: pgmap v481: 129 pgs: 129 active+clean; 335 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-03-09T00:09:44.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:44 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1855607113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:09:44.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:44 smithi192 ceph-mon[35995]: pgmap v481: 129 pgs: 129 active+clean; 335 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-03-09T00:09:44.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:44 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1855607113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:09:45.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 27728 14.99 MB/sec warmup 112 sec latency 127.858 ms 2024-03-09T00:09:45.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:09:45.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:09:45.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:09:45.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 -- 172.21.15.22:0/4087066226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024100d80 msgr2=0x7fa024104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 --2- 172.21.15.22:0/4087066226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024100d80 0x7fa024104dc0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa018009900 tx=0x7fa01802bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 -- 172.21.15.22:0/4087066226 shutdown_connections 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 --2- 172.21.15.22:0/4087066226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0241054a0 0x7fa02410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 --2- 172.21.15.22:0/4087066226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa024100d80 0x7fa024104dc0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 --2- 172.21.15.22:0/4087066226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0241003d0 0x7fa0241007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 -- 172.21.15.22:0/4087066226 >> 172.21.15.22:0/4087066226 conn(0x7fa0240772f0 msgr2=0x7fa024077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 -- 172.21.15.22:0/4087066226 shutdown_connections 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.358+0000 7fa029aa4640 1 -- 172.21.15.22:0/4087066226 wait complete. 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 Processor -- start 2024-03-09T00:09:45.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 -- start start 2024-03-09T00:09:45.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0241003d0 0x7fa02419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0241054a0 0x7fa02419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0241968d0 0x7fa024196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa024110530 con 0x7fa0241054a0 2024-03-09T00:09:45.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241103b0 con 0x7fa0241968d0 2024-03-09T00:09:45.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa029aa4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241106b0 con 0x7fa0241003d0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa022ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0241054a0 0x7fa02419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa022ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0241054a0 0x7fa02419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41138/0 (socket says 172.21.15.22:41138) 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa022ffd640 1 -- 172.21.15.22:0/2047397210 learned_addr learned my addr 172.21.15.22:0/2047397210 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa0237fe640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0241003d0 0x7fa02419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.359+0000 7fa023fff640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0241968d0 0x7fa024196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa022ffd640 1 -- 172.21.15.22:0/2047397210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0241003d0 msgr2=0x7fa02419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa022ffd640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0241003d0 0x7fa02419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa022ffd640 1 -- 172.21.15.22:0/2047397210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0241968d0 msgr2=0x7fa024196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa022ffd640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0241968d0 0x7fa024196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa022ffd640 1 -- 172.21.15.22:0/2047397210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0241973f0 con 0x7fa0241054a0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa022ffd640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0241054a0 0x7fa02419cc60 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa018036040 tx=0x7fa01802cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa020ff9640 1 -- 172.21.15.22:0/2047397210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018041070 con 0x7fa0241054a0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa020ff9640 1 -- 172.21.15.22:0/2047397210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa018004450 con 0x7fa0241054a0 2024-03-09T00:09:45.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa024197680 con 0x7fa0241054a0 2024-03-09T00:09:45.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa020ff9640 1 -- 172.21.15.22:0/2047397210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018003dc0 con 0x7fa0241054a0 2024-03-09T00:09:45.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.360+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0241a3870 con 0x7fa0241054a0 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.362+0000 7fa020ff9640 1 -- 172.21.15.22:0/2047397210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa018044070 con 0x7fa0241054a0 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.362+0000 7fa02c763740 1 Processor -- start 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.362+0000 7fa02c763740 1 -- start start 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa020ff9640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa00c0a2970 0x7fa00c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa02c763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fca690 0x556298fcaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa02c763740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa02c763740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fd8880 0x556298fdac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa02c763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556298f2eb40 con 0x556298fca690 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa02c763740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556298ed7d80 con 0x556298fcafb0 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa02c763740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556298f2abe0 con 0x556298fd8880 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa023fff640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa00c0a2970 0x7fa00c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa020ff9640 1 -- 172.21.15.22:0/2047397210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa0180f0bf0 con 0x7fa0241054a0 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fca690 0x556298fcaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fca690 0x556298fcaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41154/0 (socket says 172.21.15.22:41154) 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa022ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 -- 172.21.15.22:0/739303726 learned_addr learned my addr 172.21.15.22:0/739303726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa023fff640 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fd8880 0x556298fdac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 -- 172.21.15.22:0/739303726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fd8880 msgr2=0x556298fdac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fd8880 0x556298fdac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 -- 172.21.15.22:0/739303726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 msgr2=0x556298fd8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 -- 172.21.15.22:0/739303726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556298d0b7e0 con 0x556298fca690 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa022ffd640 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa023fff640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa00c0a2970 0x7fa00c0a4e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009fd0 tx=0x7fa01c009d80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.363+0000 7fa0237fe640 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fca690 0x556298fcaa70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa014009c90 tx=0x7fa014011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b68aacc7be6715 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa009ffb640 1 -- 172.21.15.22:0/739303726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01401e070 con 0x556298fca690 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa009ffb640 1 -- 172.21.15.22:0/739303726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa014009e40 con 0x556298fca690 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa009ffb640 1 -- 172.21.15.22:0/739303726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140186c0 con 0x556298fca690 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 -- 172.21.15.22:0/739303726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fca690 msgr2=0x556298fcaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fca690 0x556298fcaa70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa014009c90 tx=0x7fa014011810 comp rx=0 tx=0).stop 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 -- 172.21.15.22:0/739303726 shutdown_connections 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fd8880 0x556298fdac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 --2- 172.21.15.22:0/739303726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fca690 0x556298fcaa70 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 -- 172.21.15.22:0/739303726 >> 172.21.15.22:0/739303726 conn(0x556298fc54b0 msgr2=0x556298fc5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 -- 172.21.15.22:0/739303726 shutdown_connections 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.364+0000 7fa02c763740 1 -- 172.21.15.22:0/739303726 wait complete. 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 -- start start 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 Processor -- start 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fd5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556298fdb4b0 con 0x556298fd8880 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556298eeb1c0 con 0x556298fcafb0 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556298fdb630 con 0x556298fca690 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa02c763740 10 client.0 _ll_register_callbacks cb 0x556298eda7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fd5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fd5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41158/0 (socket says 172.21.15.22:41158) 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 -- 172.21.15.22:0/3233560737 learned_addr learned my addr 172.21.15.22:0/3233560737 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa022ffd640 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 -- 172.21.15.22:0/3233560737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 msgr2=0x556298fd8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 -- 172.21.15.22:0/3233560737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 msgr2=0x556298fd55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.365+0000 7fa023fff640 1 -- 172.21.15.22:0/3233560737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556298d6bf70 con 0x556298fd8880 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa023fff640 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fd5b50 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00fec0 tx=0x7fa01c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a234d6e36d3efa30 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa002ffd640 1 -- 172.21.15.22:0/3233560737 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c013330 con 0x556298fd8880 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556298f04960 con 0x556298fd8880 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa002ffd640 1 -- 172.21.15.22:0/3233560737 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa01c01c040 con 0x556298fd8880 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa002ffd640 1 -- 172.21.15.22:0/3233560737 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01c010e40 con 0x556298fd8880 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 10 client.20316 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556298c03170 con 0x556298fd8880 2024-03-09T00:09:45.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa002ffd640 1 -- 172.21.15.22:0/3233560737 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556298f04960 con 0x556298fd8880 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa002ffd640 1 -- 172.21.15.22:0/3233560737 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa01c020050 con 0x556298fd8880 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 10 client.20316 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 10 client.20316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 20 client.20316 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556298feab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 4 client.20316 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.366+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556298ef0080 con 0x556298fe8730 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.367+0000 7fa0237fe640 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556298feab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.367+0000 7fa0237fe640 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556298feab70 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d43c78364e3c0c server_cookie=189a6fb123cc78b3 in_seq=0 out_seq=0 2024-03-09T00:09:45.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.367+0000 7fa002ffd640 10 client.20316 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa002ffd640 1 -- 172.21.15.22:0/3233560737 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556298ef0080 con 0x556298fe8730 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa002ffd640 10 client.20316 handle_command_reply: tid=0 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 client.20316 shutdown 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 msgr2=0x556298fd5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fd5b50 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00fec0 tx=0x7fa01c01d000 comp rx=0 tx=0).stop 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 shutdown_connections 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556298feab70 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa002ffd640 0 client.20316 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fd5b50 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 --2- 172.21.15.22:0/3233560737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 >> 172.21.15.22:0/3233560737 conn(0x556298fc54b0 msgr2=0x556298fc7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 shutdown_connections 2024-03-09T00:09:45.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 1 -- 172.21.15.22:0/3233560737 wait complete. 2024-03-09T00:09:45.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.371+0000 7fa02c763740 20 client.20316 trim_cache size 0 max 16384 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.386+0000 7fa02c763740 1 Processor -- start 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.386+0000 7fa02c763740 1 -- start start 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.386+0000 7fa02c763740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.386+0000 7fa02c763740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.386+0000 7fa02c763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.386+0000 7fa02c763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241ae9e0 con 0x556298fd8880 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.386+0000 7fa02c763740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241ac520 con 0x556298fcafb0 2024-03-09T00:09:45.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa02c763740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241aa5e0 con 0x556298fca690 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41170/0 (socket says 172.21.15.22:41170) 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 -- 172.21.15.22:0/2638949980 learned_addr learned my addr 172.21.15.22:0/2638949980 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa0237fe640 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa022ffd640 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 -- 172.21.15.22:0/2638949980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 msgr2=0x556298fd7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 -- 172.21.15.22:0/2638949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 msgr2=0x556298fd8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 -- 172.21.15.22:0/2638949980 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556298fca4b0 con 0x556298fd8880 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa0237fe640 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa023fff640 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdf8b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00ffd0 tx=0x7fa01c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bc17c5968a4900 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa0097fa640 1 -- 172.21.15.22:0/2638949980 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c010bd0 con 0x556298fd8880 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa0097fa640 1 -- 172.21.15.22:0/2638949980 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa01c013330 con 0x556298fd8880 2024-03-09T00:09:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.387+0000 7fa0097fa640 1 -- 172.21.15.22:0/2638949980 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c01fa10 con 0x556298fd8880 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 -- 172.21.15.22:0/2638949980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 msgr2=0x556298fdf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdf8b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00ffd0 tx=0x7fa01c016000 comp rx=0 tx=0).stop 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 -- 172.21.15.22:0/2638949980 shutdown_connections 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdf8b0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fcafb0 0x556298fd8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 --2- 172.21.15.22:0/2638949980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fca690 0x556298fd7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 -- 172.21.15.22:0/2638949980 >> 172.21.15.22:0/2638949980 conn(0x556298fc54b0 msgr2=0x556298fda6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 -- 172.21.15.22:0/2638949980 shutdown_connections 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 -- 172.21.15.22:0/2638949980 wait complete. 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 -- start start 2024-03-09T00:09:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.388+0000 7fa02c763740 1 Processor -- start 2024-03-09T00:09:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa02c763740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fca690 0x556298fde490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa02c763740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fcafb0 0x556298fde9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa02c763740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa02c763740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241aabe0 con 0x556298fd8880 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa02c763740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241abec0 con 0x556298fca690 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa02c763740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241ab200 con 0x556298fcafb0 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fca690 0x556298fde490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa02c763740 10 client.0 _ll_register_callbacks cb 0x7fa0140097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41180/0 (socket says 172.21.15.22:41180) 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 -- 172.21.15.22:0/2970540788 learned_addr learned my addr 172.21.15.22:0/2970540788 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa022ffd640 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fcafb0 0x556298fde9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 -- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fcafb0 msgr2=0x556298fde9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fcafb0 0x556298fde9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 -- 172.21.15.22:0/2970540788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fca690 msgr2=0x556298fde490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fca690 0x556298fde490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 -- 172.21.15.22:0/2970540788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55629903ee30 con 0x556298fd8880 2024-03-09T00:09:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa0237fe640 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fca690 0x556298fde490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa023fff640 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdef10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa01c004730 tx=0x7fa01c01f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18ceed7f0f95c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.389+0000 7fa002ffd640 1 -- 172.21.15.22:0/2970540788 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c01f540 con 0x556298fd8880 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa002ffd640 1 -- 172.21.15.22:0/2970540788 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa01c01f870 con 0x556298fd8880 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa01c013330 con 0x556298fd8880 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa002ffd640 1 -- 172.21.15.22:0/2970540788 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01c047050 con 0x556298fd8880 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 10 client.20322 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556299040150 con 0x556298fd8880 2024-03-09T00:09:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa002ffd640 1 -- 172.21.15.22:0/2970540788 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa01c013330 con 0x556298fd8880 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa002ffd640 1 -- 172.21.15.22:0/2970540788 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa01c03d030 con 0x556298fd8880 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 10 client.20322 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 10 client.20322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 20 client.20322 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556299042820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 4 client.20322 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556298f3c080 con 0x556298fe8730 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.390+0000 7fa0237fe640 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556299042820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.391+0000 7fa002ffd640 10 client.20322 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.391+0000 7fa0237fe640 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556299042820 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b209d8a480d3014 server_cookie=c8c967a61a45466e in_seq=0 out_seq=0 2024-03-09T00:09:45.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa002ffd640 1 -- 172.21.15.22:0/2970540788 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556298f3c080 con 0x556298fe8730 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa002ffd640 10 client.20322 handle_command_reply: tid=0 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 client.20322 shutdown 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 msgr2=0x556298fdef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdef10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa01c004730 tx=0x7fa01c01f730 comp rx=0 tx=0).stop 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 shutdown_connections 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556298fe8730 0x556299042820 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556298fd8880 0x556298fdef10 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa002ffd640 0 client.20322 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556298fcafb0 0x556298fde9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 --2- 172.21.15.22:0/2970540788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556298fca690 0x556298fde490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.394+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 >> 172.21.15.22:0/2970540788 conn(0x556298fc54b0 msgr2=0x556298fd5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 shutdown_connections 2024-03-09T00:09:45.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa02c763740 1 -- 172.21.15.22:0/2970540788 wait complete. 2024-03-09T00:09:45.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa02c763740 20 client.20322 trim_cache size 0 max 16384 2024-03-09T00:09:45.394 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:09:45.394 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:09:45.394 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "9674e6c4-a492-4bb1-a1ae-af3232fa5fa2", 2024-03-09T00:09:45.394 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:09:45.394 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa00c0a2970 msgr2=0x7fa00c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa00c0a2970 0x7fa00c0a4e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009fd0 tx=0x7fa01c009d80 comp rx=0 tx=0).stop 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0241054a0 msgr2=0x7fa02419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0241054a0 0x7fa02419cc60 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa018036040 tx=0x7fa01802cb20 comp rx=0 tx=0).stop 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 shutdown_connections 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa00c0a2970 0x7fa00c0a4e30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0241968d0 0x7fa024196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0241054a0 0x7fa02419cc60 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 --2- 172.21.15.22:0/2047397210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0241003d0 0x7fa02419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 >> 172.21.15.22:0/2047397210 conn(0x7fa0240772f0 msgr2=0x7fa0240fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 shutdown_connections 2024-03-09T00:09:45.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.395+0000 7fa029aa4640 1 -- 172.21.15.22:0/2047397210 wait complete. 2024-03-09T00:09:45.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:09:45.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:09:45.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.542+0000 7fa720148640 1 -- 172.21.15.22:0/790982207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 msgr2=0x7fa71810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.542+0000 7fa720148640 1 --2- 172.21.15.22:0/790982207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71810dfc0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa70801a910 tx=0x7fa70803cc90 comp rx=0 tx=0).stop 2024-03-09T00:09:45.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.547+0000 7fa720148640 1 -- 172.21.15.22:0/790982207 shutdown_connections 2024-03-09T00:09:45.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.547+0000 7fa720148640 1 --2- 172.21.15.22:0/790982207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71811aad0 0x7fa71811cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.547+0000 7fa720148640 1 --2- 172.21.15.22:0/790982207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71810dfc0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.547+0000 7fa720148640 1 --2- 172.21.15.22:0/790982207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa71810fb10 0x7fa71810d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.547+0000 7fa720148640 1 -- 172.21.15.22:0/790982207 >> 172.21.15.22:0/790982207 conn(0x7fa71806c3d0 msgr2=0x7fa71806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.549+0000 7fa720148640 1 -- 172.21.15.22:0/790982207 shutdown_connections 2024-03-09T00:09:45.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.549+0000 7fa720148640 1 -- 172.21.15.22:0/790982207 wait complete. 2024-03-09T00:09:45.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.550+0000 7fa720148640 1 Processor -- start 2024-03-09T00:09:45.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.550+0000 7fa720148640 1 -- start start 2024-03-09T00:09:45.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa720148640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71811a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa720148640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71810fb10 0x7fa7181162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa71dee4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71811a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa71dee4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71811a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41200/0 (socket says 172.21.15.22:41200) 2024-03-09T00:09:45.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa720148640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa71811aad0 0x7fa718116800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa71dee4640 1 -- 172.21.15.22:0/3993380755 learned_addr learned my addr 172.21.15.22:0/3993380755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718112cb0 con 0x7fa71810db40 2024-03-09T00:09:45.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718112b30 con 0x7fa71811aad0 2024-03-09T00:09:45.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.551+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718112e30 con 0x7fa71810fb10 2024-03-09T00:09:45.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.552+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71810fb10 0x7fa7181162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.552+0000 7fa71dee4640 1 -- 172.21.15.22:0/3993380755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71810fb10 msgr2=0x7fa7181162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.552+0000 7fa71dee4640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71810fb10 0x7fa7181162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.552+0000 7fa71dee4640 1 -- 172.21.15.22:0/3993380755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa71811aad0 msgr2=0x7fa718116800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.552+0000 7fa71dee4640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa71811aad0 0x7fa718116800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.552+0000 7fa71dee4640 1 -- 172.21.15.22:0/3993380755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718116f30 con 0x7fa71810db40 2024-03-09T00:09:45.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.553+0000 7fa71dee4640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71811a1e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b450 tx=0x7fa71400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.554+0000 7fa70effd640 1 -- 172.21.15.22:0/3993380755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714015020 con 0x7fa71810db40 2024-03-09T00:09:45.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.554+0000 7fa70effd640 1 -- 172.21.15.22:0/3993380755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa714011d70 con 0x7fa71810db40 2024-03-09T00:09:45.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.554+0000 7fa70effd640 1 -- 172.21.15.22:0/3993380755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71400f340 con 0x7fa71810db40 2024-03-09T00:09:45.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.556+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7181aa840 con 0x7fa71810db40 2024-03-09T00:09:45.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.556+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7181aac50 con 0x7fa71810db40 2024-03-09T00:09:45.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.560+0000 7fa70effd640 1 -- 172.21.15.22:0/3993380755 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa714018020 con 0x7fa71810db40 2024-03-09T00:09:45.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.561+0000 7fa70effd640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa7040a2af0 0x7fa7040a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.562+0000 7fa70effd640 1 -- 172.21.15.22:0/3993380755 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa7140cea60 con 0x7fa71810db40 2024-03-09T00:09:45.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.562+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa7040a2af0 0x7fa7040a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.565+0000 7fa722dfd740 1 Processor -- start 2024-03-09T00:09:45.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.565+0000 7fa722dfd740 1 -- start start 2024-03-09T00:09:45.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.566+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380a760 0x55e69380ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.566+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380b080 0x55e693813720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.566+0000 7fa71dee4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380a760 0x55e69380ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.566+0000 7fa71dee4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380a760 0x55e69380ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41214/0 (socket says 172.21.15.22:41214) 2024-03-09T00:09:45.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.566+0000 7fa71dee4640 1 -- 172.21.15.22:0/987081634 learned_addr learned my addr 172.21.15.22:0/987081634 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.566+0000 7fa722dfd740 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e693816050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.566+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69376de00 con 0x55e69380a760 2024-03-09T00:09:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e693716b10 con 0x55e69380b080 2024-03-09T00:09:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e693769ea0 con 0x55e693813c60 2024-03-09T00:09:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e693816050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa7040a2af0 0x7fa7040a4fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa718117ac0 tx=0x7fa70803f040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380b080 0x55e693813720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71e6e5640 1 -- 172.21.15.22:0/987081634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380b080 msgr2=0x55e693813720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380b080 0x55e693813720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71e6e5640 1 -- 172.21.15.22:0/987081634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380a760 msgr2=0x55e69380ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380a760 0x55e69380ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.567+0000 7fa71e6e5640 1 -- 172.21.15.22:0/987081634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e69354c7d0 con 0x55e693813c60 2024-03-09T00:09:45.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.568+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e693816050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa710009d90 tx=0x7fa71000d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2fb36bfc9a4d848 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.569+0000 7fa703fff640 1 -- 172.21.15.22:0/987081634 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710003970 con 0x55e693813c60 2024-03-09T00:09:45.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.569+0000 7fa703fff640 1 -- 172.21.15.22:0/987081634 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa710004030 con 0x55e693813c60 2024-03-09T00:09:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.570+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 msgr2=0x55e693816050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.570+0000 7fa722dfd740 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e693816050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa710009d90 tx=0x7fa71000d6b0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 shutdown_connections 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e693816050 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380b080 0x55e693813720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 --2- 172.21.15.22:0/987081634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380a760 0x55e69380ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 >> 172.21.15.22:0/987081634 conn(0x55e693804e90 msgr2=0x55e693805270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 shutdown_connections 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 -- 172.21.15.22:0/987081634 wait complete. 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.571+0000 7fa722dfd740 1 -- start start 2024-03-09T00:09:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 1 Processor -- start 2024-03-09T00:09:45.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e6938133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693810740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e693810c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e693816aa0 con 0x55e693813c60 2024-03-09T00:09:45.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69372a0a0 con 0x55e69380a760 2024-03-09T00:09:45.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e693816c20 con 0x55e69380b080 2024-03-09T00:09:45.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.572+0000 7fa722dfd740 10 client.0 _ll_register_callbacks cb 0x55e693729a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:45.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.573+0000 7fa71e6e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e693810c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.573+0000 7fa71e6e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e693810c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41230/0 (socket says 172.21.15.22:41230) 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.573+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1101434386 learned_addr learned my addr 172.21.15.22:0/1101434386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.573+0000 7fa71dee4640 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e6938133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.574+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693810740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.574+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 msgr2=0x55e693810740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.574+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693810740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.574+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1101434386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 msgr2=0x55e6938133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.575+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e6938133b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.575+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1101434386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6934823f0 con 0x55e693813c60 2024-03-09T00:09:45.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.575+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e693810c80 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa710005f40 tx=0x7fa710011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ae3ac81a7407d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.576+0000 7fa7017fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:45.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.576+0000 7fa7017fa640 1 -- 172.21.15.22:0/1101434386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710011e20 con 0x55e693813c60 2024-03-09T00:09:45.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.576+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e69377b340 con 0x55e693813c60 2024-03-09T00:09:45.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.576+0000 7fa7017fa640 1 -- 172.21.15.22:0/1101434386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa710004b20 con 0x55e693813c60 2024-03-09T00:09:45.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa7017fa640 1 -- 172.21.15.22:0/1101434386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa71002e050 con 0x55e693813c60 2024-03-09T00:09:45.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 10 client.20334 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e6934420f0 con 0x55e693813c60 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa7017fa640 1 -- 172.21.15.22:0/1101434386 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e69377b340 con 0x55e693813c60 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa7017fa640 1 -- 172.21.15.22:0/1101434386 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa71001b440 con 0x55e693813c60 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 10 client.20334 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 10 client.20334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 20 client.20334 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e693823ad0 0x55e693825f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 4 client.20334 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.577+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e693775690 con 0x55e693823ad0 2024-03-09T00:09:45.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.578+0000 7fa71dee4640 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e693823ad0 0x55e693825f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.579+0000 7fa7017fa640 10 client.20334 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.579+0000 7fa71dee4640 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e693823ad0 0x55e693825f10 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4d0838057964d1 server_cookie=9753f0c4b27aceb7 in_seq=0 out_seq=0 2024-03-09T00:09:45.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.584+0000 7fa7017fa640 1 -- 172.21.15.22:0/1101434386 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e693775690 con 0x55e693823ad0 2024-03-09T00:09:45.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.584+0000 7fa7017fa640 10 client.20334 handle_command_reply: tid=0 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.584+0000 7fa722dfd740 1 client.20334 shutdown 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.584+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 msgr2=0x55e693810c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.584+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e693810c80 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa710005f40 tx=0x7fa710011b00 comp rx=0 tx=0).stop 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 shutdown_connections 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e693823ad0 0x55e693825f10 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e693810c80 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693810740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1101434386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e6938133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 >> 172.21.15.22:0/1101434386 conn(0x55e693804e90 msgr2=0x55e693806be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa7017fa640 0 client.20334 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 shutdown_connections 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 1 -- 172.21.15.22:0/1101434386 wait complete. 2024-03-09T00:09:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.585+0000 7fa722dfd740 20 client.20334 trim_cache size 0 max 16384 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.612+0000 7fa722dfd740 1 Processor -- start 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.612+0000 7fa722dfd740 1 -- start start 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.612+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e693812e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.612+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380b080 0x55e6938133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.612+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e69381aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.613+0000 7fa722dfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181b7350 con 0x55e69380b080 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.613+0000 7fa722dfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181b4e90 con 0x55e69380a760 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.613+0000 7fa722dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181b2f50 con 0x55e693813c60 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.613+0000 7fa71e6e5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e69381aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.613+0000 7fa71e6e5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e69381aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:60028/0 (socket says 172.21.15.22:60028) 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.613+0000 7fa71e6e5640 1 -- 172.21.15.22:0/761526214 learned_addr learned my addr 172.21.15.22:0/761526214 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.613+0000 7fa71dee4640 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e693812e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380b080 0x55e6938133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71e6e5640 1 -- 172.21.15.22:0/761526214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 msgr2=0x55e693812e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e693812e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71e6e5640 1 -- 172.21.15.22:0/761526214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380b080 msgr2=0x55e6938133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380b080 0x55e6938133b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71e6e5640 1 -- 172.21.15.22:0/761526214 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e69380aeb0 con 0x55e693813c60 2024-03-09T00:09:45.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380b080 0x55e6938133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:09:45.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.614+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e69381aea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa71001b840 tx=0x7fa71000d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da801623345e0161 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.615+0000 7fa7037fe640 1 -- 172.21.15.22:0/761526214 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100104d0 con 0x55e693813c60 2024-03-09T00:09:45.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.615+0000 7fa7037fe640 1 -- 172.21.15.22:0/761526214 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa710037070 con 0x55e693813c60 2024-03-09T00:09:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.616+0000 7fa722dfd740 1 -- 172.21.15.22:0/761526214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 msgr2=0x55e69381aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.616+0000 7fa722dfd740 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e69381aea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa71001b840 tx=0x7fa71000d6b0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.617+0000 7fa722dfd740 1 -- 172.21.15.22:0/761526214 shutdown_connections 2024-03-09T00:09:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.617+0000 7fa722dfd740 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e693813c60 0x55e69381aea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.617+0000 7fa722dfd740 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e69380b080 0x55e6938133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.617+0000 7fa722dfd740 1 --2- 172.21.15.22:0/761526214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e693812e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.617+0000 7fa722dfd740 1 -- 172.21.15.22:0/761526214 >> 172.21.15.22:0/761526214 conn(0x55e693804e90 msgr2=0x55e69380f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.617+0000 7fa722dfd740 1 -- 172.21.15.22:0/761526214 shutdown_connections 2024-03-09T00:09:45.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.617+0000 7fa722dfd740 1 -- 172.21.15.22:0/761526214 wait complete. 2024-03-09T00:09:45.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 -- start start 2024-03-09T00:09:45.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 Processor -- start 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e69381b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693816e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e6938173a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181b3550 con 0x55e693813c60 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181b4830 con 0x55e69380a760 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181b3b70 con 0x55e69380b080 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.618+0000 7fa722dfd740 10 client.0 _ll_register_callbacks cb 0x7fa714097020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e6938173a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e6938173a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41250/0 (socket says 172.21.15.22:41250) 2024-03-09T00:09:45.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1574922850 learned_addr learned my addr 172.21.15.22:0/1574922850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:45.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71d6e3640 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693816e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 msgr2=0x55e693816e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:45.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693816e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1574922850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 msgr2=0x55e69381b090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:09:45.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e69381b090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 -- 172.21.15.22:0/1574922850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e693817990 con 0x55e693813c60 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.619+0000 7fa71e6e5640 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e6938173a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa710009480 tx=0x7fa710009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6cf1c721380e8a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa7017fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa7017fa640 1 -- 172.21.15.22:0/1574922850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100104d0 con 0x55e693813c60 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa710037070 con 0x55e693813c60 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa7017fa640 1 -- 172.21.15.22:0/1574922850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fa710004b20 con 0x55e693813c60 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa7017fa640 1 -- 172.21.15.22:0/1574922850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa710048050 con 0x55e693813c60 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa722dfd740 10 client.20337 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e693879da0 con 0x55e693813c60 2024-03-09T00:09:45.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.620+0000 7fa7017fa640 1 -- 172.21.15.22:0/1574922850 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa710037070 con 0x55e693813c60 2024-03-09T00:09:45.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.622+0000 7fa7017fa640 1 -- 172.21.15.22:0/1574922850 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa710025b80 con 0x55e693813c60 2024-03-09T00:09:45.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.622+0000 7fa722dfd740 10 client.20337 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:45.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.622+0000 7fa722dfd740 10 client.20337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:45.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.622+0000 7fa722dfd740 20 client.20337 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:45.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.622+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e69387a390 0x55e69387c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:45.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.622+0000 7fa722dfd740 4 client.20337 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:09:45.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.622+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e6936f34e0 con 0x55e69387a390 2024-03-09T00:09:45.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.624+0000 7fa71dee4640 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e69387a390 0x55e69387c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:45.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.626+0000 7fa71dee4640 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e69387a390 0x55e69387c7b0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a43cb7ea8bbe0d7 server_cookie=ad996ae2300d4a96 in_seq=0 out_seq=0 2024-03-09T00:09:45.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.627+0000 7fa7017fa640 10 client.20337 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa7017fa640 1 -- 172.21.15.22:0/1574922850 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55e6936f34e0 con 0x55e69387a390 2024-03-09T00:09:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa7017fa640 10 client.20337 handle_command_reply: tid=0 2024-03-09T00:09:45.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 client.20337 shutdown 2024-03-09T00:09:45.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 msgr2=0x55e6938173a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e6938173a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa710009480 tx=0x7fa710009a90 comp rx=0 tx=0).stop 2024-03-09T00:09:45.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 shutdown_connections 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e69387a390 0x55e69387c7b0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e693813c60 0x55e6938173a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e69380b080 0x55e693816e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 --2- 172.21.15.22:0/1574922850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e69380a760 0x55e69381b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 >> 172.21.15.22:0/1574922850 conn(0x55e693804e90 msgr2=0x55e69380ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.633+0000 7fa7017fa640 0 client.20337 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.634+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 shutdown_connections 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.634+0000 7fa722dfd740 1 -- 172.21.15.22:0/1574922850 wait complete. 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.634+0000 7fa722dfd740 20 client.20337 trim_cache size 0 max 16384 2024-03-09T00:09:45.629 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (470 inodes in the stack)", 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: "9674e6c4-a492-4bb1-a1ae-af3232fa5fa2": { 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "9674e6c4-a492-4bb1-a1ae-af3232fa5fa2", 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:09:45.630 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:09:45.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa7040a2af0 msgr2=0x7fa7040a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa7040a2af0 0x7fa7040a4fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa718117ac0 tx=0x7fa70803f040 comp rx=0 tx=0).stop 2024-03-09T00:09:45.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 msgr2=0x7fa71811a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:45.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71811a1e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b450 tx=0x7fa71400b910 comp rx=0 tx=0).stop 2024-03-09T00:09:45.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 shutdown_connections 2024-03-09T00:09:45.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa7040a2af0 0x7fa7040a4fb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa71811aad0 0x7fa718116800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71810fb10 0x7fa7181162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 --2- 172.21.15.22:0/3993380755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa71810db40 0x7fa71811a1e0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:45.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 >> 172.21.15.22:0/3993380755 conn(0x7fa71806c3d0 msgr2=0x7fa71811c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:45.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.635+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 shutdown_connections 2024-03-09T00:09:45.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:09:45.636+0000 7fa720148640 1 -- 172.21.15.22:0/3993380755 wait complete. 2024-03-09T00:09:45.644 INFO:tasks.cephfs.filesystem:scrub status for tag:9674e6c4-a492-4bb1-a1ae-af3232fa5fa2 - {'path': '/', 'tag': '9674e6c4-a492-4bb1-a1ae-af3232fa5fa2', 'options': 'recursive,force'} 2024-03-09T00:09:46.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 27893 14.91 MB/sec warmup 113 sec latency 262.092 ms 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: pgmap v482: 129 pgs: 129 active+clean; 326 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.2 MiB/s wr, 591 op/s 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:09:46.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:46 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: pgmap v482: 129 pgs: 129 active+clean; 326 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.2 MiB/s wr, 591 op/s 2024-03-09T00:09:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:09:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:46.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:09:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:46 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: pgmap v482: 129 pgs: 129 active+clean; 326 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.2 MiB/s wr, 591 op/s 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:46.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:09:46.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:46 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:47.066 INFO:tasks.workunit.client.0.smithi022.stdout: 10 28081 14.84 MB/sec warmup 114 sec latency 224.082 ms 2024-03-09T00:09:47.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:47 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:47.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:47 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:47.629 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:47 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:09:48.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 28244 14.78 MB/sec warmup 115 sec latency 270.708 ms 2024-03-09T00:09:48.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:48 smithi022 ceph-mon[31894]: pgmap v483: 129 pgs: 129 active+clean; 326 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.0 MiB/s wr, 460 op/s 2024-03-09T00:09:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:48 smithi062 ceph-mon[36129]: pgmap v483: 129 pgs: 129 active+clean; 326 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.0 MiB/s wr, 460 op/s 2024-03-09T00:09:48.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:48 smithi192 ceph-mon[35995]: pgmap v483: 129 pgs: 129 active+clean; 326 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.0 MiB/s wr, 460 op/s 2024-03-09T00:09:49.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 28495 14.70 MB/sec warmup 116 sec latency 303.350 ms 2024-03-09T00:09:50.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 28799 14.68 MB/sec warmup 117 sec latency 269.958 ms 2024-03-09T00:09:50.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:50 smithi022 ceph-mon[31894]: pgmap v484: 129 pgs: 129 active+clean; 328 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 470 op/s 2024-03-09T00:09:50.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:50 smithi062 ceph-mon[36129]: pgmap v484: 129 pgs: 129 active+clean; 328 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 470 op/s 2024-03-09T00:09:50.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:50 smithi192 ceph-mon[35995]: pgmap v484: 129 pgs: 129 active+clean; 328 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 470 op/s 2024-03-09T00:09:51.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 29293 14.76 MB/sec warmup 118 sec latency 175.621 ms 2024-03-09T00:09:52.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 29832 14.92 MB/sec warmup 119 sec latency 152.090 ms 2024-03-09T00:09:52.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:52 smithi022 ceph-mon[31894]: pgmap v485: 129 pgs: 129 active+clean; 334 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 669 op/s 2024-03-09T00:09:52.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:52 smithi062 ceph-mon[36129]: pgmap v485: 129 pgs: 129 active+clean; 334 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 669 op/s 2024-03-09T00:09:52.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:52 smithi192 ceph-mon[35995]: pgmap v485: 129 pgs: 129 active+clean; 334 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 669 op/s 2024-03-09T00:09:54.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 30524 27.58 MB/sec execute 1 sec latency 308.801 ms 2024-03-09T00:09:54.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:54 smithi022 ceph-mon[31894]: pgmap v486: 129 pgs: 129 active+clean; 334 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 555 op/s 2024-03-09T00:09:54.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:54 smithi062 ceph-mon[36129]: pgmap v486: 129 pgs: 129 active+clean; 334 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 555 op/s 2024-03-09T00:09:54.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:54 smithi192 ceph-mon[35995]: pgmap v486: 129 pgs: 129 active+clean; 334 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 555 op/s 2024-03-09T00:09:55.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 30853 27.39 MB/sec execute 2 sec latency 180.736 ms 2024-03-09T00:09:55.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:55 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:09:55.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:55 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:09:55.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:55 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:09:55.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:55 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:09:55.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:55 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:09:55.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:55 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:09:55.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:55 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:09:55.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:55 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:09:55.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:55 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:09:56.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31078 21.79 MB/sec execute 3 sec latency 235.203 ms 2024-03-09T00:09:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/2049930045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24101ee0 msgr2=0x7f6a24105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/2049930045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24101ee0 0x7f6a24105f20 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009900 tx=0x7f6a2002bc30 comp rx=0 tx=0).stop 2024-03-09T00:09:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/2049930045 shutdown_connections 2024-03-09T00:09:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/2049930045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a24106600 0x7f6a2410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/2049930045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24101ee0 0x7f6a24105f20 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/2049930045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24101530 0x7f6a24101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/2049930045 >> 172.21.15.22:0/2049930045 conn(0x7f6a240fcdf0 msgr2=0x7f6a240ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/2049930045 shutdown_connections 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.307+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/2049930045 wait complete. 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 Processor -- start 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 -- start start 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24101530 0x7f6a2419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24106600 0x7f6a2419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a24197930 0x7f6a24197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24111690 con 0x7f6a24106600 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24111510 con 0x7f6a24197930 2024-03-09T00:09:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2c1c4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24111810 con 0x7f6a24101530 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2975f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24106600 0x7f6a2419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2975f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24106600 0x7f6a2419dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57416/0 (socket says 172.21.15.22:57416) 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2975f640 1 -- 172.21.15.22:0/1327774794 learned_addr learned my addr 172.21.15.22:0/1327774794 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a29f60640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24101530 0x7f6a2419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.308+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a24197930 0x7f6a24197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2975f640 1 -- 172.21.15.22:0/1327774794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24101530 msgr2=0x7f6a2419d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24101530 0x7f6a2419d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2975f640 1 -- 172.21.15.22:0/1327774794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a24197930 msgr2=0x7f6a24197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a24197930 0x7f6a24197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2975f640 1 -- 172.21.15.22:0/1327774794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24198450 con 0x7f6a24106600 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24106600 0x7f6a2419dcc0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6a20036040 tx=0x7f6a2002cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a1affd640 1 -- 172.21.15.22:0/1327774794 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20041070 con 0x7f6a24106600 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a1affd640 1 -- 172.21.15.22:0/1327774794 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a20004450 con 0x7f6a24106600 2024-03-09T00:09:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a1affd640 1 -- 172.21.15.22:0/1327774794 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20003dc0 con 0x7f6a24106600 2024-03-09T00:09:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a241986e0 con 0x7f6a24106600 2024-03-09T00:09:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.309+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a241a48d0 con 0x7f6a24106600 2024-03-09T00:09:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.311+0000 7f6a1affd640 1 -- 172.21.15.22:0/1327774794 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6a20044070 con 0x7f6a24106600 2024-03-09T00:09:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 Processor -- start 2024-03-09T00:09:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 -- start start 2024-03-09T00:09:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a1affd640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a100a2a10 0x7f6a100a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f7da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f7e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f89940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347ee25f0 con 0x559347f87550 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347e8b930 con 0x559347f7ed40 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347ede690 con 0x559347f7dfd0 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a100a2a10 0x7f6a100a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a29f60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f7da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2975f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f7e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a29f60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f7da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:46418/0 (socket says 172.21.15.22:46418) 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a29f60640 1 -- 172.21.15.22:0/1240787334 learned_addr learned my addr 172.21.15.22:0/1240787334 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a1affd640 1 -- 172.21.15.22:0/1327774794 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f6a200f0c50 con 0x7f6a24106600 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f89940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2975f640 1 -- 172.21.15.22:0/1240787334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 msgr2=0x559347f7da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f7da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2975f640 1 -- 172.21.15.22:0/1240787334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 msgr2=0x559347f89940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f89940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a2975f640 1 -- 172.21.15.22:0/1240787334 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559347bf7210 con 0x559347f7dfd0 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.312+0000 7f6a29f60640 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f7da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.313+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a100a2a10 0x7f6a100a4ed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005fd0 tx=0x7f6a1c009be0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.313+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f7e450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a200ee060 tx=0x7f6a20042f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae64b017ef6d38c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.313+0000 7f6a0ffff640 1 -- 172.21.15.22:0/1240787334 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2004b020 con 0x559347f7dfd0 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.313+0000 7f6a0ffff640 1 -- 172.21.15.22:0/1240787334 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a200d4460 con 0x559347f7dfd0 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.313+0000 7f6a0ffff640 1 -- 172.21.15.22:0/1240787334 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a200c5710 con 0x559347f7dfd0 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1240787334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 msgr2=0x559347f7e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f7e450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a200ee060 tx=0x7f6a20042f50 comp rx=0 tx=0).stop 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1240787334 shutdown_connections 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f89940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f7e450 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1240787334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f7da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1240787334 >> 172.21.15.22:0/1240787334 conn(0x559347f455b0 msgr2=0x559347f78cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1240787334 shutdown_connections 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1240787334 wait complete. 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- start start 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 Processor -- start 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f84b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f85070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347f8a040 con 0x559347f7dfd0 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.314+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347e9ed90 con 0x559347f87550 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347f8a1c0 con 0x559347f7ed40 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2ed9f740 10 client.0 _ll_register_callbacks cb 0x559347bf6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a29f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a29f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f845f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57434/0 (socket says 172.21.15.22:57434) 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a29f60640 1 -- 172.21.15.22:0/1487023302 learned_addr learned my addr 172.21.15.22:0/1487023302 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f85070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f84b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2a761640 1 -- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 msgr2=0x559347f84b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f84b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2a761640 1 -- 172.21.15.22:0/1487023302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 msgr2=0x559347f845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2a761640 1 -- 172.21.15.22:0/1487023302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559347cc3620 con 0x559347f87550 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a29f60640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2975f640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f84b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2a761640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f85070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x559347f79e50 tx=0x7f6a1c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b09a7e3c1b69e7b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/1487023302 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c024070 con 0x559347f87550 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a200d4460 con 0x559347f87550 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/1487023302 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f6a1c003e00 con 0x559347f87550 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.315+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/1487023302 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a1c033050 con 0x559347f87550 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 10 client.29558 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559347f96310 con 0x559347f87550 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/1487023302 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a200d4460 con 0x559347f87550 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/1487023302 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6a1c01fc00 con 0x559347f87550 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 10 client.29558 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 10 client.29558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 20 client.29558 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347f97780 0x559347f99b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 4 client.29558 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:09:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559347eefb30 con 0x559347f97780 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a29f60640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347f97780 0x559347f99b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a29f60640 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347f97780 0x559347f99b70 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdebe8699e223ca5 server_cookie=287450e5b7057fa2 in_seq=0 out_seq=0 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.316+0000 7f6a0d7fa640 10 client.29558 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/1487023302 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559347eefb30 con 0x559347f97780 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a0d7fa640 10 client.29558 handle_command_reply: tid=0 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 client.29558 shutdown 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 msgr2=0x559347f85070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f85070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x559347f79e50 tx=0x7f6a1c01d000 comp rx=0 tx=0).stop 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 shutdown_connections 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347f97780 0x559347f99b70 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f85070 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a0d7fa640 0 client.29558 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f84b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/1487023302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 >> 172.21.15.22:0/1487023302 conn(0x559347f455b0 msgr2=0x559347f7aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 shutdown_connections 2024-03-09T00:09:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.319+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/1487023302 wait complete. 2024-03-09T00:09:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.320+0000 7f6a2ed9f740 20 client.29558 trim_cache size 0 max 16384 2024-03-09T00:09:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 Processor -- start 2024-03-09T00:09:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 -- start start 2024-03-09T00:09:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f86d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f8e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f8e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241b0b50 con 0x559347f87550 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241ae690 con 0x559347f7ed40 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241ac750 con 0x559347f7dfd0 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f8e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f8e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57438/0 (socket says 172.21.15.22:57438) 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 -- 172.21.15.22:0/3619799006 learned_addr learned my addr 172.21.15.22:0/3619799006 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a29f60640 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f86d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2975f640 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f8e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 -- 172.21.15.22:0/3619799006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 msgr2=0x559347f86d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f86d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 -- 172.21.15.22:0/3619799006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 msgr2=0x559347f8e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f8e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 -- 172.21.15.22:0/3619799006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559347f7d2a0 con 0x559347f87550 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a29f60640 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f86d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2975f640 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f8e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.333+0000 7f6a2a761640 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f8e930 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009bb0 tx=0x7f6a1c010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be5ba05de9a9f11 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a0f7fe640 1 -- 172.21.15.22:0/3619799006 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c03b070 con 0x559347f87550 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a0f7fe640 1 -- 172.21.15.22:0/3619799006 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c009e90 con 0x559347f87550 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a0f7fe640 1 -- 172.21.15.22:0/3619799006 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c024070 con 0x559347f87550 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/3619799006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 msgr2=0x559347f8e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f8e930 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009bb0 tx=0x7f6a1c010520 comp rx=0 tx=0).stop 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/3619799006 shutdown_connections 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f87550 0x559347f8e930 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f7ed40 0x559347f8e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/3619799006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7dfd0 0x559347f86d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/3619799006 >> 172.21.15.22:0/3619799006 conn(0x559347f455b0 msgr2=0x559347f82ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/3619799006 shutdown_connections 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.334+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/3619799006 wait complete. 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 -- start start 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 Processor -- start 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f8e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f81830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241acd50 con 0x559347f7dfd0 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241ae030 con 0x559347f87550 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241ad370 con 0x559347f7ed40 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2ed9f740 10 client.0 _ll_register_callbacks cb 0x7f6a20005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f8e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f8e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57442/0 (socket says 172.21.15.22:57442) 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 -- 172.21.15.22:0/4172721905 learned_addr learned my addr 172.21.15.22:0/4172721905 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:09:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2975f640 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a2a761640 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f81830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 -- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 msgr2=0x559347f810f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f810f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 -- 172.21.15.22:0/4172721905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 msgr2=0x559347f81830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f81830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.335+0000 7f6a29f60640 1 -- 172.21.15.22:0/4172721905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559347fedc20 con 0x559347f7dfd0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a29f60640 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f8e8e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x559347f9a450 tx=0x7f6a140044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e865086e6088bbf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/4172721905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14010c40 con 0x559347f7dfd0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a1c009e90 con 0x559347f7dfd0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/4172721905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f6a14010de0 con 0x559347f7dfd0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/4172721905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a1402f050 con 0x559347f7dfd0 2024-03-09T00:09:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a2ed9f740 10 client.20355 fetch_fsmap learned FSMap version 36 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559347feed70 con 0x559347f7dfd0 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/4172721905 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c009e90 con 0x559347f7dfd0 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/4172721905 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6a1401e7e0 con 0x559347f7dfd0 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a2ed9f740 10 client.20355 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a2ed9f740 10 client.20355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.336+0000 7f6a2ed9f740 20 client.20355 populate_metadata read hostname 'smithi022' 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.337+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347fef4b0 0x559347ff18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.337+0000 7f6a2ed9f740 4 client.20355 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.337+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559347f7de20 con 0x559347fef4b0 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.337+0000 7f6a2975f640 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347fef4b0 0x559347ff18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.337+0000 7f6a2975f640 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347fef4b0 0x559347ff18d0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e3b6949c42b3d server_cookie=7c91dc0a25c4c28f in_seq=0 out_seq=0 2024-03-09T00:09:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:56.337+0000 7f6a0d7fa640 10 client.20355 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:56 smithi022 ceph-mon[31894]: pgmap v487: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 703 op/s 2024-03-09T00:09:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:56 smithi062 ceph-mon[36129]: pgmap v487: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 703 op/s 2024-03-09T00:09:56.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:56 smithi192 ceph-mon[35995]: pgmap v487: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 703 op/s 2024-03-09T00:09:57.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31167 18.00 MB/sec execute 4 sec latency 553.701 ms 2024-03-09T00:09:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a0d7fa640 1 -- 172.21.15.22:0/4172721905 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559347f7de20 con 0x559347fef4b0 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a0d7fa640 10 client.20355 handle_command_reply: tid=0 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 client.20355 shutdown 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 msgr2=0x559347f8e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f8e8e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x559347f9a450 tx=0x7f6a140044d0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 shutdown_connections 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x559347fef4b0 0x559347ff18d0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559347f87550 0x559347f81830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a0d7fa640 0 client.20355 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:09:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559347f7ed40 0x559347f810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 --2- 172.21.15.22:0/4172721905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559347f7dfd0 0x559347f8e8e0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 >> 172.21.15.22:0/4172721905 conn(0x559347f455b0 msgr2=0x559347f845b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 shutdown_connections 2024-03-09T00:09:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.471+0000 7f6a2ed9f740 1 -- 172.21.15.22:0/4172721905 wait complete. 2024-03-09T00:09:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2ed9f740 20 client.20355 trim_cache size 0 max 16384 2024-03-09T00:09:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:09:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:09:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:09:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:09:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a100a2a10 msgr2=0x7f6a100a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a100a2a10 0x7f6a100a4ed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005fd0 tx=0x7f6a1c009be0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24106600 msgr2=0x7f6a2419dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24106600 0x7f6a2419dcc0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6a20036040 tx=0x7f6a2002cb20 comp rx=0 tx=0).stop 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 shutdown_connections 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f6a100a2a10 0x7f6a100a4ed0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a24197930 0x7f6a24197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a24106600 0x7f6a2419dcc0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 --2- 172.21.15.22:0/1327774794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a24101530 0x7f6a2419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 >> 172.21.15.22:0/1327774794 conn(0x7f6a240fcdf0 msgr2=0x7f6a240ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 shutdown_connections 2024-03-09T00:09:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:09:57.472+0000 7f6a2c1c4640 1 -- 172.21.15.22:0/1327774794 wait complete. 2024-03-09T00:09:58.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31251 15.37 MB/sec execute 5 sec latency 579.356 ms 2024-03-09T00:09:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:58 smithi022 ceph-mon[31894]: pgmap v488: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 358 op/s 2024-03-09T00:09:58.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:58 smithi062 ceph-mon[36129]: pgmap v488: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 358 op/s 2024-03-09T00:09:58.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:58 smithi192 ceph-mon[35995]: pgmap v488: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 358 op/s 2024-03-09T00:09:59.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31401 13.55 MB/sec execute 6 sec latency 244.812 ms 2024-03-09T00:09:59.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:59 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:09:59.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:59 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:09:59.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:59 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:09:59.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:09:59 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:09:59.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:59 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:09:59.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:59 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:09:59.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:59 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:09:59.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:09:59 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:09:59.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:59 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:09:59.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:59 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:09:59.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:59 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:09:59.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:09:59 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:10:00.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31624 13.20 MB/sec execute 7 sec latency 277.317 ms 2024-03-09T00:10:00.511 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:00 smithi062 ceph-mon[36129]: pgmap v489: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 383 op/s 2024-03-09T00:10:00.511 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:00 smithi062 ceph-mon[36129]: overall HEALTH_OK 2024-03-09T00:10:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:00 smithi192 ceph-mon[35995]: pgmap v489: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 383 op/s 2024-03-09T00:10:00.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:00 smithi192 ceph-mon[35995]: overall HEALTH_OK 2024-03-09T00:10:00.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:00 smithi022 ceph-mon[31894]: pgmap v489: 129 pgs: 129 active+clean; 342 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 383 op/s 2024-03-09T00:10:00.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:00 smithi022 ceph-mon[31894]: overall HEALTH_OK 2024-03-09T00:10:01.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31655 11.58 MB/sec execute 8 sec latency 905.944 ms 2024-03-09T00:10:02.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31655 10.29 MB/sec execute 9 sec latency 1906.060 ms 2024-03-09T00:10:02.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:02 smithi062 ceph-mon[36129]: pgmap v490: 129 pgs: 129 active+clean; 325 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 736 op/s 2024-03-09T00:10:02.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:02 smithi192 ceph-mon[35995]: pgmap v490: 129 pgs: 129 active+clean; 325 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 736 op/s 2024-03-09T00:10:02.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:02 smithi022 ceph-mon[31894]: pgmap v490: 129 pgs: 129 active+clean; 325 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 736 op/s 2024-03-09T00:10:03.067 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31655 9.27 MB/sec execute 10 sec latency 2906.149 ms 2024-03-09T00:10:04.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 31853 8.81 MB/sec execute 11 sec latency 2956.197 ms 2024-03-09T00:10:04.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:04 smithi062 ceph-mon[36129]: pgmap v491: 129 pgs: 129 active+clean; 325 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 535 op/s 2024-03-09T00:10:04.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:04 smithi192 ceph-mon[35995]: pgmap v491: 129 pgs: 129 active+clean; 325 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 535 op/s 2024-03-09T00:10:04.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:04 smithi022 ceph-mon[31894]: pgmap v491: 129 pgs: 129 active+clean; 325 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 535 op/s 2024-03-09T00:10:05.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 32143 9.12 MB/sec execute 12 sec latency 103.192 ms 2024-03-09T00:10:06.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 32309 8.82 MB/sec execute 13 sec latency 244.989 ms 2024-03-09T00:10:06.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:06 smithi062 ceph-mon[36129]: pgmap v492: 129 pgs: 129 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 654 op/s 2024-03-09T00:10:06.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:06 smithi062 ceph-mon[36129]: osdmap e118: 12 total, 12 up, 12 in 2024-03-09T00:10:06.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:06 smithi192 ceph-mon[35995]: pgmap v492: 129 pgs: 129 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 654 op/s 2024-03-09T00:10:06.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:06 smithi192 ceph-mon[35995]: osdmap e118: 12 total, 12 up, 12 in 2024-03-09T00:10:06.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:06 smithi022 ceph-mon[31894]: pgmap v492: 129 pgs: 129 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.6 MiB/s wr, 654 op/s 2024-03-09T00:10:06.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:06 smithi022 ceph-mon[31894]: osdmap e118: 12 total, 12 up, 12 in 2024-03-09T00:10:07.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 32511 9.19 MB/sec execute 14 sec latency 261.096 ms 2024-03-09T00:10:08.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 32688 9.12 MB/sec execute 15 sec latency 286.669 ms 2024-03-09T00:10:08.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:08 smithi062 ceph-mon[36129]: pgmap v494: 129 pgs: 129 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 608 op/s 2024-03-09T00:10:08.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:08 smithi192 ceph-mon[35995]: pgmap v494: 129 pgs: 129 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 608 op/s 2024-03-09T00:10:08.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:08 smithi022 ceph-mon[31894]: pgmap v494: 129 pgs: 129 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 608 op/s 2024-03-09T00:10:09.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 32834 8.86 MB/sec execute 16 sec latency 438.129 ms 2024-03-09T00:10:10.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 32993 8.70 MB/sec execute 17 sec latency 716.131 ms 2024-03-09T00:10:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.612+0000 7f933f169640 1 -- 172.21.15.22:0/1601806615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933810fea0 msgr2=0x7f933810db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.612+0000 7f933f169640 1 --2- 172.21.15.22:0/1601806615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933810fea0 0x7f933810db90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9324009860 tx=0x7f932402bbb0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.613+0000 7f933f169640 1 -- 172.21.15.22:0/1601806615 shutdown_connections 2024-03-09T00:10:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.613+0000 7f933f169640 1 --2- 172.21.15.22:0/1601806615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f933811aad0 0x7f933811cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.613+0000 7f933f169640 1 --2- 172.21.15.22:0/1601806615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933810fea0 0x7f933810db90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.613+0000 7f933f169640 1 --2- 172.21.15.22:0/1601806615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f933810fa30 0x7f933810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.613+0000 7f933f169640 1 -- 172.21.15.22:0/1601806615 >> 172.21.15.22:0/1601806615 conn(0x7f933806c390 msgr2=0x7f933806c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.614+0000 7f933f169640 1 -- 172.21.15.22:0/1601806615 shutdown_connections 2024-03-09T00:10:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.614+0000 7f933f169640 1 -- 172.21.15.22:0/1601806615 wait complete. 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.614+0000 7f933f169640 1 Processor -- start 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.614+0000 7f933f169640 1 -- start start 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933f169640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f933810fa30 0x7f933811a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933f169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f933810fea0 0x7f9338116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933f169640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933811aad0 0x7f9338116860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933cf05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f933810fa30 0x7f933811a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933cf05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f933810fa30 0x7f933811a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35526/0 (socket says 172.21.15.22:35526) 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933cf05640 1 -- 172.21.15.22:0/235261137 learned_addr learned my addr 172.21.15.22:0/235261137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338112cd0 con 0x7f933810fea0 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338112b50 con 0x7f933811aad0 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.615+0000 7f933d706640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933811aad0 0x7f9338116860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9338112e50 con 0x7f933810fa30 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f9337fff640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f933810fea0 0x7f9338116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f9337fff640 1 -- 172.21.15.22:0/235261137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f933810fa30 msgr2=0x7f933811a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f9337fff640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f933810fa30 0x7f933811a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f9337fff640 1 -- 172.21.15.22:0/235261137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933811aad0 msgr2=0x7f9338116860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f9337fff640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933811aad0 0x7f9338116860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f9337fff640 1 -- 172.21.15.22:0/235261137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338116f60 con 0x7f933810fea0 2024-03-09T00:10:10.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:10 smithi062 ceph-mon[36129]: pgmap v495: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 586 op/s 2024-03-09T00:10:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.616+0000 7f9337fff640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f933810fea0 0x7f9338116320 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f9324037040 tx=0x7f93240045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.617+0000 7f9335ffb640 1 -- 172.21.15.22:0/235261137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324039040 con 0x7f933810fea0 2024-03-09T00:10:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.617+0000 7f9335ffb640 1 -- 172.21.15.22:0/235261137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f932402c550 con 0x7f933810fea0 2024-03-09T00:10:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.617+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93381aa870 con 0x7f933810fea0 2024-03-09T00:10:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.617+0000 7f9335ffb640 1 -- 172.21.15.22:0/235261137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324005520 con 0x7f933810fea0 2024-03-09T00:10:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.618+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93381aac70 con 0x7f933810fea0 2024-03-09T00:10:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.621+0000 7f9341e27740 1 Processor -- start 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.621+0000 7f9341e27740 1 -- start start 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9341e27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9341e27740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd42e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9341e27740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd439940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9341e27740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bd3925f0 con 0x5603bd42dfd0 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9341e27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bd33b930 con 0x5603bd42ed40 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9341e27740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bd38e690 con 0x5603bd437550 2024-03-09T00:10:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9335ffb640 1 -- 172.21.15.22:0/235261137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9324044020 con 0x7f933810fea0 2024-03-09T00:10:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f933d706640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f933d706640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36106/0 (socket says 172.21.15.22:36106) 2024-03-09T00:10:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f933d706640 1 -- 172.21.15.22:0/3591738133 learned_addr learned my addr 172.21.15.22:0/3591738133 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f9337fff640 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd439940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.622+0000 7f933cf05640 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd42e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f9335ffb640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f93280a2b60 0x7f93280a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f9335ffb640 1 -- 172.21.15.22:0/235261137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f93240eff00 con 0x7f933810fea0 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f933d706640 1 -- 172.21.15.22:0/3591738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 msgr2=0x5603bd439940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f933d706640 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd439940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f933d706640 1 -- 172.21.15.22:0/3591738133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 msgr2=0x5603bd42e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f933d706640 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd42e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f933d706640 1 -- 172.21.15.22:0/3591738133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603bd0a7210 con 0x5603bd42ed40 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f9337fff640 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd439940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:10:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.623+0000 7f933cf05640 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd42e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:10:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.624+0000 7f933d706640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f93280a2b60 0x7f93280a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.624+0000 7f933d706640 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42da90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f933000a400 tx=0x7f9330009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33febc64e20adcd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.624+0000 7f9322ffd640 1 -- 172.21.15.22:0/3591738133 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330010b70 con 0x5603bd42ed40 2024-03-09T00:10:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.624+0000 7f9322ffd640 1 -- 172.21.15.22:0/3591738133 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9330010d90 con 0x5603bd42ed40 2024-03-09T00:10:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.624+0000 7f9322ffd640 1 -- 172.21.15.22:0/3591738133 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300205d0 con 0x5603bd42ed40 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.625+0000 7f933d706640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f93280a2b60 0x7f93280a5020 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f933001e000 tx=0x7f9330021000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.625+0000 7f9341e27740 1 -- 172.21.15.22:0/3591738133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 msgr2=0x5603bd42da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42da90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f933000a400 tx=0x7f9330009650 comp rx=0 tx=0).stop 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 -- 172.21.15.22:0/3591738133 shutdown_connections 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd439940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd42e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 --2- 172.21.15.22:0/3591738133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42da90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 -- 172.21.15.22:0/3591738133 >> 172.21.15.22:0/3591738133 conn(0x5603bd3f55b0 msgr2=0x5603bd428cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 -- 172.21.15.22:0/3591738133 shutdown_connections 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.626+0000 7f9341e27740 1 -- 172.21.15.22:0/3591738133 wait complete. 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.627+0000 7f9341e27740 1 -- start start 2024-03-09T00:10:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.627+0000 7f9341e27740 1 Processor -- start 2024-03-09T00:10:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f9341e27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42dfd0 0x5603bd434590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f9341e27740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd42ed40 0x5603bd434ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f9341e27740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd437550 0x5603bd435010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f933cf05640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42dfd0 0x5603bd434590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f933cf05640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42dfd0 0x5603bd434590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36116/0 (socket says 172.21.15.22:36116) 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f9337fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd42ed40 0x5603bd434ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f9337fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd42ed40 0x5603bd434ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35536/0 (socket says 172.21.15.22:35536) 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f933d706640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd437550 0x5603bd435010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f933d706640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd437550 0x5603bd435010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46424/0 (socket says 172.21.15.22:46424) 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f933cf05640 1 -- 172.21.15.22:0/1914636878 learned_addr learned my addr 172.21.15.22:0/1914636878 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.628+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bd43a280 con 0x5603bd437550 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bd34ed90 con 0x5603bd42dfd0 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bd43a400 con 0x5603bd42ed40 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9341e27740 10 client.0 _ll_register_callbacks cb 0x5603bd0a6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9337fff640 1 -- 172.21.15.22:0/1914636878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42dfd0 msgr2=0x5603bd434590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9337fff640 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42dfd0 0x5603bd434590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9337fff640 1 -- 172.21.15.22:0/1914636878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd437550 msgr2=0x5603bd435010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9337fff640 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd437550 0x5603bd435010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f9337fff640 1 -- 172.21.15.22:0/1914636878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603bd173620 con 0x5603bd42ed40 2024-03-09T00:10:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.629+0000 7f933d706640 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd437550 0x5603bd435010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.630+0000 7f9337fff640 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd42ed40 0x5603bd434ad0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f93240ed2b0 tx=0x7f932402c0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0051a335de5221 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.631+0000 7f9317fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:10:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.631+0000 7f9317fff640 1 -- 172.21.15.22:0/1914636878 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240bc1c0 con 0x5603bd42ed40 2024-03-09T00:10:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.631+0000 7f9317fff640 1 -- 172.21.15.22:0/1914636878 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f93240b8830 con 0x5603bd42ed40 2024-03-09T00:10:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.631+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9330010d90 con 0x5603bd42ed40 2024-03-09T00:10:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.632+0000 7f9317fff640 1 -- 172.21.15.22:0/1914636878 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93240c0310 con 0x5603bd42ed40 2024-03-09T00:10:10.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:10 smithi192 ceph-mon[35995]: pgmap v495: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 586 op/s 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.635+0000 7f9341e27740 10 client.29740 fetch_fsmap learned FSMap version 36 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.635+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5603bd43a280 con 0x5603bd42ed40 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9317fff640 1 -- 172.21.15.22:0/1914636878 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93240d4020 con 0x5603bd42ed40 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9317fff640 1 -- 172.21.15.22:0/1914636878 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f932403c070 con 0x5603bd42ed40 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9341e27740 10 client.29740 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9341e27740 10 client.29740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9341e27740 20 client.29740 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9341e27740 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd4479a0 0x5603bd449e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9341e27740 4 client.29740 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.636+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603bd39fb30 con 0x5603bd4479a0 2024-03-09T00:10:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.639+0000 7f933cf05640 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd4479a0 0x5603bd449e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.648+0000 7f933cf05640 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd4479a0 0x5603bd449e00 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85503f4b199800f8 server_cookie=a2e768ca01865cae in_seq=0 out_seq=0 2024-03-09T00:10:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.648+0000 7f9317fff640 10 client.29740 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.652+0000 7f9317fff640 1 -- 172.21.15.22:0/1914636878 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5603bd39fb30 con 0x5603bd4479a0 2024-03-09T00:10:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.652+0000 7f9317fff640 10 client.29740 handle_command_reply: tid=0 2024-03-09T00:10:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.652+0000 7f9341e27740 1 client.29740 shutdown 2024-03-09T00:10:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.652+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd42ed40 msgr2=0x5603bd434ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.652+0000 7f9341e27740 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd42ed40 0x5603bd434ad0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f93240ed2b0 tx=0x7f932402c0d0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.652+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 shutdown_connections 2024-03-09T00:10:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.652+0000 7f9341e27740 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd4479a0 0x5603bd449e00 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9341e27740 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd437550 0x5603bd435010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9341e27740 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd42ed40 0x5603bd434ad0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9341e27740 1 --2- 172.21.15.22:0/1914636878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42dfd0 0x5603bd434590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 >> 172.21.15.22:0/1914636878 conn(0x5603bd3f55b0 msgr2=0x5603bd42aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9317fff640 0 client.29740 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 shutdown_connections 2024-03-09T00:10:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9341e27740 1 -- 172.21.15.22:0/1914636878 wait complete. 2024-03-09T00:10:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.653+0000 7f9341e27740 20 client.29740 trim_cache size 0 max 16384 2024-03-09T00:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.679+0000 7f9341e27740 1 Processor -- start 2024-03-09T00:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.679+0000 7f9341e27740 1 -- start start 2024-03-09T00:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.680+0000 7f9341e27740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd436cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.680+0000 7f9341e27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.680+0000 7f9341e27740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd44e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.680+0000 7f9341e27740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93381b7450 con 0x5603bd42dfd0 2024-03-09T00:10:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.680+0000 7f9341e27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93381b4f90 con 0x5603bd42ed40 2024-03-09T00:10:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.680+0000 7f9341e27740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93381b3050 con 0x5603bd437550 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.681+0000 7f9337fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.681+0000 7f933cf05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd436cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.681+0000 7f933cf05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd436cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46436/0 (socket says 172.21.15.22:46436) 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.681+0000 7f933cf05640 1 -- 172.21.15.22:0/3910281162 learned_addr learned my addr 172.21.15.22:0/3910281162 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.682+0000 7f933cf05640 1 -- 172.21.15.22:0/3910281162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 msgr2=0x5603bd44e760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.682+0000 7f933cf05640 1 --2- 172.21.15.22:0/3910281162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd44e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.682+0000 7f933cf05640 1 -- 172.21.15.22:0/3910281162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 msgr2=0x5603bd42f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.682+0000 7f933cf05640 1 --2- 172.21.15.22:0/3910281162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.682+0000 7f933cf05640 1 -- 172.21.15.22:0/3910281162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603bd432ae0 con 0x5603bd42dfd0 2024-03-09T00:10:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.683+0000 7f933cf05640 1 --2- 172.21.15.22:0/3910281162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd436cc0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f932c01a910 tx=0x7f932c01aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba1ffca02ba050a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.684+0000 7f93227fc640 1 -- 172.21.15.22:0/3910281162 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c003d60 con 0x5603bd42dfd0 2024-03-09T00:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.685+0000 7f93227fc640 1 -- 172.21.15.22:0/3910281162 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f932c022450 con 0x5603bd42dfd0 2024-03-09T00:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.685+0000 7f93227fc640 1 -- 172.21.15.22:0/3910281162 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c020690 con 0x5603bd42dfd0 2024-03-09T00:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.687+0000 7f9341e27740 1 -- 172.21.15.22:0/3910281162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 msgr2=0x5603bd436cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.687+0000 7f9341e27740 1 --2- 172.21.15.22:0/3910281162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd436cc0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f932c01a910 tx=0x7f932c01aee0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.688+0000 7f9341e27740 1 -- 172.21.15.22:0/3910281162 shutdown_connections 2024-03-09T00:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.688+0000 7f9341e27740 1 --2- 172.21.15.22:0/3910281162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd437550 0x5603bd44e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.688+0000 7f9341e27740 1 --2- 172.21.15.22:0/3910281162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd42ed40 0x5603bd42f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.688+0000 7f9341e27740 1 --2- 172.21.15.22:0/3910281162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42dfd0 0x5603bd436cc0 secure :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f932c01a910 tx=0x7f932c01aee0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.688+0000 7f9341e27740 1 -- 172.21.15.22:0/3910281162 >> 172.21.15.22:0/3910281162 conn(0x5603bd3f55b0 msgr2=0x5603bd4287f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.693+0000 7f9341e27740 1 -- 172.21.15.22:0/3910281162 shutdown_connections 2024-03-09T00:10:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.697+0000 7f9341e27740 1 -- 172.21.15.22:0/3910281162 wait complete. 2024-03-09T00:10:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.697+0000 7f9341e27740 1 -- start start 2024-03-09T00:10:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 1 Processor -- start 2024-03-09T00:10:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42ed40 0x5603bd43e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd437550 0x5603bd43cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd43d220 0x5603bd43d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93381b3650 con 0x5603bd42ed40 2024-03-09T00:10:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93381b4930 con 0x5603bd437550 2024-03-09T00:10:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93381b3c70 con 0x5603bd43d220 2024-03-09T00:10:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.698+0000 7f9341e27740 10 client.0 _ll_register_callbacks cb 0x7f9330009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.699+0000 7f9337fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd437550 0x5603bd43cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.699+0000 7f9337fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd437550 0x5603bd43cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36132/0 (socket says 172.21.15.22:36132) 2024-03-09T00:10:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.699+0000 7f9337fff640 1 -- 172.21.15.22:0/566415201 learned_addr learned my addr 172.21.15.22:0/566415201 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.700+0000 7f933d706640 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd43d220 0x5603bd43d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.700+0000 7f9337fff640 1 -- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd43d220 msgr2=0x5603bd43d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.700+0000 7f9337fff640 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd43d220 0x5603bd43d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.700+0000 7f9337fff640 1 -- 172.21.15.22:0/566415201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42ed40 msgr2=0x5603bd43e990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:10:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.700+0000 7f9337fff640 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42ed40 0x5603bd43e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.700+0000 7f9337fff640 1 -- 172.21.15.22:0/566415201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603bd49dff0 con 0x5603bd437550 2024-03-09T00:10:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.703+0000 7f9337fff640 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd437550 0x5603bd43cce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9324009780 tx=0x7f932402bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559525110f15c84c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.703+0000 7f9317fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.706+0000 7f9317fff640 1 -- 172.21.15.22:0/566415201 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240bc1c0 con 0x5603bd437550 2024-03-09T00:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.706+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f932c022450 con 0x5603bd437550 2024-03-09T00:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.706+0000 7f9317fff640 1 -- 172.21.15.22:0/566415201 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f932402ca90 con 0x5603bd437550 2024-03-09T00:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.707+0000 7f9317fff640 1 -- 172.21.15.22:0/566415201 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93240a4050 con 0x5603bd437550 2024-03-09T00:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.708+0000 7f9341e27740 10 client.29582 fetch_fsmap learned FSMap version 36 2024-03-09T00:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.708+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5603bd49f300 con 0x5603bd437550 2024-03-09T00:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.709+0000 7f9317fff640 1 -- 172.21.15.22:0/566415201 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f932c022450 con 0x5603bd437550 2024-03-09T00:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.710+0000 7f9317fff640 1 -- 172.21.15.22:0/566415201 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f932403c040 con 0x5603bd437550 2024-03-09T00:10:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.710+0000 7f9341e27740 10 client.29582 fetch_fsmap finished waiting for FSMap version 36 2024-03-09T00:10:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.710+0000 7f9341e27740 10 client.29582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.710+0000 7f9341e27740 20 client.29582 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.710+0000 7f9341e27740 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd49f7c0 0x5603bd4a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.710+0000 7f9341e27740 4 client.29582 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:10:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.710+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603bd399e80 con 0x5603bd49f7c0 2024-03-09T00:10:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.715+0000 7f933cf05640 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd49f7c0 0x5603bd4a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.722+0000 7f933cf05640 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd49f7c0 0x5603bd4a1be0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b2ad1f0240b0fe server_cookie=b6eb9006eacdffd8 in_seq=0 out_seq=0 2024-03-09T00:10:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:10.722+0000 7f9317fff640 10 client.29582 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:10.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:10 smithi022 ceph-mon[31894]: pgmap v495: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 329 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 586 op/s 2024-03-09T00:10:11.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 33186 8.96 MB/sec execute 18 sec latency 780.820 ms 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9317fff640 1 -- 172.21.15.22:0/566415201 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603bd399e80 con 0x5603bd49f7c0 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9317fff640 10 client.29582 handle_command_reply: tid=0 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 client.29582 shutdown 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd437550 msgr2=0x5603bd43cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd437550 0x5603bd43cce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9324009780 tx=0x7f932402bbb0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 shutdown_connections 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5603bd49f7c0 0x5603bd4a1be0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603bd43d220 0x5603bd43d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603bd437550 0x5603bd43cce0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 --2- 172.21.15.22:0/566415201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5603bd42ed40 0x5603bd43e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.924+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 >> 172.21.15.22:0/566415201 conn(0x5603bd3f55b0 msgr2=0x5603bd42f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.925+0000 7f9317fff640 0 client.29582 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.925+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 shutdown_connections 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.925+0000 7f9341e27740 1 -- 172.21.15.22:0/566415201 wait complete. 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.925+0000 7f9341e27740 20 client.29582 trim_cache size 0 max 16384 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:10:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:10:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:10:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:10:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f93280a2b60 msgr2=0x7f93280a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f93280a2b60 0x7f93280a5020 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f933001e000 tx=0x7f9330021000 comp rx=0 tx=0).stop 2024-03-09T00:10:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f933810fea0 msgr2=0x7f9338116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f933810fea0 0x7f9338116320 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f9324037040 tx=0x7f93240045e0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 shutdown_connections 2024-03-09T00:10:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f93280a2b60 0x7f93280a5020 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f933811aad0 0x7f9338116860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f933810fea0 0x7f9338116320 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 --2- 172.21.15.22:0/235261137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f933810fa30 0x7f933811a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 >> 172.21.15.22:0/235261137 conn(0x7f933806c390 msgr2=0x7f933811b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 shutdown_connections 2024-03-09T00:10:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:11.926+0000 7f933f169640 1 -- 172.21.15.22:0/235261137 wait complete. 2024-03-09T00:10:12.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 33236 8.67 MB/sec execute 19 sec latency 1230.097 ms 2024-03-09T00:10:12.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:12 smithi022 ceph-mon[31894]: pgmap v496: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 338 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-03-09T00:10:12.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:12 smithi062 ceph-mon[36129]: pgmap v496: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 338 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-03-09T00:10:12.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:12 smithi192 ceph-mon[35995]: pgmap v496: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 338 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-03-09T00:10:13.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 33383 8.81 MB/sec execute 20 sec latency 1238.029 ms 2024-03-09T00:10:14.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 33487 8.56 MB/sec execute 21 sec latency 1047.875 ms 2024-03-09T00:10:14.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:14 smithi062 ceph-mon[36129]: pgmap v497: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 338 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-03-09T00:10:14.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:14 smithi192 ceph-mon[35995]: pgmap v497: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 338 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-03-09T00:10:14.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:14 smithi022 ceph-mon[31894]: pgmap v497: 129 pgs: 20 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 58 active+clean; 338 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2024-03-09T00:10:15.068 INFO:tasks.workunit.client.0.smithi022.stdout: 10 33671 8.90 MB/sec execute 22 sec latency 896.413 ms 2024-03-09T00:10:15.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:10:15.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:10:15.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.845+0000 7f54ad28e640 1 -- 172.21.15.22:0/1880236157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810fb10 msgr2=0x7f54a810d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.845+0000 7f54ad28e640 1 --2- 172.21.15.22:0/1880236157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810fb10 0x7f54a810d600 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009ca0 tx=0x7f54a002c120 comp rx=0 tx=0).stop 2024-03-09T00:10:15.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.848+0000 7f54ad28e640 1 -- 172.21.15.22:0/1880236157 shutdown_connections 2024-03-09T00:10:15.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.848+0000 7f54ad28e640 1 --2- 172.21.15.22:0/1880236157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a811aad0 0x7f54a811cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.848+0000 7f54ad28e640 1 --2- 172.21.15.22:0/1880236157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a810db40 0x7f54a810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.848+0000 7f54ad28e640 1 --2- 172.21.15.22:0/1880236157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810fb10 0x7f54a810d600 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.848+0000 7f54ad28e640 1 -- 172.21.15.22:0/1880236157 >> 172.21.15.22:0/1880236157 conn(0x7f54a806c3d0 msgr2=0x7f54a806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:15.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.852+0000 7f54ad28e640 1 -- 172.21.15.22:0/1880236157 shutdown_connections 2024-03-09T00:10:15.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.854+0000 7f54ad28e640 1 -- 172.21.15.22:0/1880236157 wait complete. 2024-03-09T00:10:15.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.854+0000 7f54ad28e640 1 Processor -- start 2024-03-09T00:10:15.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54ad28e640 1 -- start start 2024-03-09T00:10:15.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54ad28e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810db40 0x7f54a811a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54ad28e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a810fb10 0x7f54a811a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54ad28e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a811aad0 0x7f54a81132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54ad28e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a81204f0 con 0x7f54a810db40 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54ad28e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a8120370 con 0x7f54a811aad0 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54ad28e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a8120670 con 0x7f54a810fb10 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54a67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a810fb10 0x7f54a811a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54a67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a810fb10 0x7f54a811a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35554/0 (socket says 172.21.15.22:35554) 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.856+0000 7f54a67fc640 1 -- 172.21.15.22:0/3653998049 learned_addr learned my addr 172.21.15.22:0/3653998049 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.857+0000 7f54a77fe640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a811aad0 0x7f54a81132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.858+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810db40 0x7f54a811a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.858+0000 7f54a6ffd640 1 -- 172.21.15.22:0/3653998049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a810fb10 msgr2=0x7f54a811a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.858+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a810fb10 0x7f54a811a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.858+0000 7f54a6ffd640 1 -- 172.21.15.22:0/3653998049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a811aad0 msgr2=0x7f54a81132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.858+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a811aad0 0x7f54a81132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.858+0000 7f54a6ffd640 1 -- 172.21.15.22:0/3653998049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8113b70 con 0x7f54a810db40 2024-03-09T00:10:15.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.859+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810db40 0x7f54a811a050 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f54a002c730 tx=0x7f54a0035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:15.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.863+0000 7f548ffff640 1 -- 172.21.15.22:0/3653998049 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0039040 con 0x7f54a810db40 2024-03-09T00:10:15.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.863+0000 7f548ffff640 1 -- 172.21.15.22:0/3653998049 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54a0003970 con 0x7f54a810db40 2024-03-09T00:10:15.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.863+0000 7f548ffff640 1 -- 172.21.15.22:0/3653998049 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a00042c0 con 0x7f54a810db40 2024-03-09T00:10:15.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.866+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a8113e00 con 0x7f54a810db40 2024-03-09T00:10:15.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.866+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a8117d50 con 0x7f54a810db40 2024-03-09T00:10:15.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.870+0000 7f548ffff640 1 -- 172.21.15.22:0/3653998049 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f54a0003b20 con 0x7f54a810db40 2024-03-09T00:10:15.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.871+0000 7f548ffff640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54900a2af0 0x7f54900a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.871+0000 7f548ffff640 1 -- 172.21.15.22:0/3653998049 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f54a00bb240 con 0x7f54a810db40 2024-03-09T00:10:15.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.873+0000 7f54afe5f740 1 Processor -- start 2024-03-09T00:10:15.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.875+0000 7f54a67fc640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54900a2af0 0x7f54900a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.875+0000 7f54a67fc640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54900a2af0 0x7f54900a4fb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54a8114890 tx=0x7f5498009190 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:15.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.876+0000 7f54afe5f740 1 -- start start 2024-03-09T00:10:15.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.876+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d4db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.876+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d56720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.877+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d59050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.877+0000 7f54afe5f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbd5cb0e00 con 0x55bbd5d4e080 2024-03-09T00:10:15.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.877+0000 7f54afe5f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbd5c59b10 con 0x55bbd5d56c60 2024-03-09T00:10:15.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.877+0000 7f54afe5f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbd5cacea0 con 0x55bbd5d4d760 2024-03-09T00:10:15.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.878+0000 7f54a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d56720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.878+0000 7f54a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d56720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46464/0 (socket says 172.21.15.22:46464) 2024-03-09T00:10:15.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.878+0000 7f54a67fc640 1 -- 172.21.15.22:0/2206335000 learned_addr learned my addr 172.21.15.22:0/2206335000 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:15.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.879+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d4db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.880+0000 7f54a67fc640 1 -- 172.21.15.22:0/2206335000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 msgr2=0x55bbd5d4db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.880+0000 7f54a67fc640 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d4db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.880+0000 7f54a67fc640 1 -- 172.21.15.22:0/2206335000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 msgr2=0x55bbd5d59050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.880+0000 7f54a67fc640 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d59050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.880+0000 7f54a67fc640 1 -- 172.21.15.22:0/2206335000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbd5a8f7d0 con 0x55bbd5d4e080 2024-03-09T00:10:15.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.881+0000 7f54a67fc640 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d56720 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f549800ebc0 tx=0x7f5498011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b46644947e5e22 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:15.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.883+0000 7f548d7fa640 1 -- 172.21.15.22:0/2206335000 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549801b040 con 0x55bbd5d4e080 2024-03-09T00:10:15.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.883+0000 7f548d7fa640 1 -- 172.21.15.22:0/2206335000 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5498024070 con 0x55bbd5d4e080 2024-03-09T00:10:15.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.883+0000 7f548d7fa640 1 -- 172.21.15.22:0/2206335000 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549801f3f0 con 0x55bbd5d4e080 2024-03-09T00:10:15.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.884+0000 7f54afe5f740 1 -- 172.21.15.22:0/2206335000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 msgr2=0x55bbd5d56720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.884+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d56720 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f549800ebc0 tx=0x7f5498011000 comp rx=0 tx=0).stop 2024-03-09T00:10:15.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.886+0000 7f54afe5f740 1 -- 172.21.15.22:0/2206335000 shutdown_connections 2024-03-09T00:10:15.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.886+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d59050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.887+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d56720 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.887+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2206335000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d4db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.887+0000 7f54afe5f740 1 -- 172.21.15.22:0/2206335000 >> 172.21.15.22:0/2206335000 conn(0x55bbd5d47e90 msgr2=0x55bbd5d48270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:15.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.889+0000 7f54afe5f740 1 -- 172.21.15.22:0/2206335000 shutdown_connections 2024-03-09T00:10:15.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.889+0000 7f54afe5f740 1 -- 172.21.15.22:0/2206335000 wait complete. 2024-03-09T00:10:15.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.889+0000 7f54afe5f740 1 -- start start 2024-03-09T00:10:15.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.889+0000 7f54afe5f740 1 Processor -- start 2024-03-09T00:10:15.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d56450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d53d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54afe5f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbd5d59a50 con 0x55bbd5d4e080 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54afe5f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbd5c6d0a0 con 0x55bbd5d56c60 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54afe5f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbd5d59bd0 con 0x55bbd5d4d760 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54afe5f740 10 client.0 _ll_register_callbacks cb 0x55bbd5c6ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d53d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d53d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36146/0 (socket says 172.21.15.22:36146) 2024-03-09T00:10:15.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.890+0000 7f54a77fe640 1 -- 172.21.15.22:0/453331231 learned_addr learned my addr 172.21.15.22:0/453331231 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:15.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d56450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a67fc640 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a77fe640 1 -- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 msgr2=0x55bbd5d56450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a77fe640 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d56450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a77fe640 1 -- 172.21.15.22:0/453331231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 msgr2=0x55bbd5d537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a77fe640 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a77fe640 1 -- 172.21.15.22:0/453331231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbd59c53f0 con 0x55bbd5d56c60 2024-03-09T00:10:15.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.891+0000 7f54a77fe640 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d53d20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f549c004700 tx=0x7f549c00bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f144b55e66af926f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:15.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.893+0000 7f548aff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:15.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.893+0000 7f548aff5640 1 -- 172.21.15.22:0/453331231 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c005020 con 0x55bbd5d56c60 2024-03-09T00:10:15.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.893+0000 7f548aff5640 1 -- 172.21.15.22:0/453331231 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f549c0051c0 con 0x55bbd5d56c60 2024-03-09T00:10:15.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.893+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5498024070 con 0x55bbd5d56c60 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.893+0000 7f548aff5640 1 -- 172.21.15.22:0/453331231 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f549c0053c0 con 0x55bbd5d56c60 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.894+0000 7f54afe5f740 10 client.29591 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.894+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55bbd59850f0 con 0x55bbd5d56c60 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f548aff5640 1 -- 172.21.15.22:0/453331231 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f549c015070 con 0x55bbd5d56c60 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f548aff5640 1 -- 172.21.15.22:0/453331231 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f549c0117c0 con 0x55bbd5d56c60 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f54afe5f740 10 client.29591 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f54afe5f740 10 client.29591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f54afe5f740 20 client.29591 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f54afe5f740 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5d66800 0x55bbd5d68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f54afe5f740 4 client.29591 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:15.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.895+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5498024070 con 0x55bbd5d66800 2024-03-09T00:10:15.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.896+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5d66800 0x55bbd5d68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.901+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5d66800 0x55bbd5d68bf0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121c8859d6089e7 server_cookie=3488a5259c02a686 in_seq=0 out_seq=0 2024-03-09T00:10:15.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.901+0000 7f548aff5640 10 client.29591 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:15.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.906+0000 7f548aff5640 1 -- 172.21.15.22:0/453331231 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f5498024070 con 0x55bbd5d66800 2024-03-09T00:10:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.906+0000 7f548aff5640 10 client.29591 handle_command_reply: tid=0 2024-03-09T00:10:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.906+0000 7f54afe5f740 1 client.29591 shutdown 2024-03-09T00:10:15.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.913+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 msgr2=0x55bbd5d53d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.913+0000 7f54afe5f740 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d53d20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f549c004700 tx=0x7f549c00bcc0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.916+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 shutdown_connections 2024-03-09T00:10:15.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.916+0000 7f54afe5f740 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5d66800 0x55bbd5d68bf0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.916+0000 7f54afe5f740 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d56c60 0x55bbd5d53d20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.916+0000 7f54afe5f740 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4e080 0x55bbd5d537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.916+0000 7f54afe5f740 1 --2- 172.21.15.22:0/453331231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d4d760 0x55bbd5d56450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.916+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 >> 172.21.15.22:0/453331231 conn(0x55bbd5d47e90 msgr2=0x55bbd5d49be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:15.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.916+0000 7f548aff5640 0 client.29591 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:15.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.921+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 shutdown_connections 2024-03-09T00:10:15.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.921+0000 7f54afe5f740 1 -- 172.21.15.22:0/453331231 wait complete. 2024-03-09T00:10:15.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.922+0000 7f54afe5f740 20 client.29591 trim_cache size 0 max 16384 2024-03-09T00:10:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.951+0000 7f54afe5f740 1 Processor -- start 2024-03-09T00:10:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54afe5f740 1 -- start start 2024-03-09T00:10:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d46d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d55f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d56450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54a67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d55f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d46d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54a67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d55f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36150/0 (socket says 172.21.15.22:36150) 2024-03-09T00:10:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d46d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46494/0 (socket says 172.21.15.22:46494) 2024-03-09T00:10:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54afe5f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a81c8420 con 0x55bbd5d4d760 2024-03-09T00:10:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54afe5f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a81c5fc0 con 0x55bbd5d4e080 2024-03-09T00:10:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54afe5f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a81c4080 con 0x55bbd5d56c60 2024-03-09T00:10:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.952+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d56450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d56450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35592/0 (socket says 172.21.15.22:35592) 2024-03-09T00:10:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a77fe640 1 -- 172.21.15.22:0/2325294070 learned_addr learned my addr 172.21.15.22:0/2325294070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:15.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a67fc640 1 -- 172.21.15.22:0/2325294070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 msgr2=0x55bbd5d56450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a67fc640 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d56450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a67fc640 1 -- 172.21.15.22:0/2325294070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 msgr2=0x55bbd5d46d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a67fc640 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d46d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a67fc640 1 -- 172.21.15.22:0/2325294070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbd5d51d50 con 0x55bbd5d4e080 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.953+0000 7f54a67fc640 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d55f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5498006e50 tx=0x7f5498009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cbada616bc46a66 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.954+0000 7f548cff9640 1 -- 172.21.15.22:0/2325294070 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498015b00 con 0x55bbd5d4e080 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.954+0000 7f548cff9640 1 -- 172.21.15.22:0/2325294070 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5498024040 con 0x55bbd5d4e080 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.954+0000 7f548cff9640 1 -- 172.21.15.22:0/2325294070 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54980036b0 con 0x55bbd5d4e080 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.954+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d46d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.955+0000 7f54afe5f740 1 -- 172.21.15.22:0/2325294070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 msgr2=0x55bbd5d55f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.955+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d55f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5498006e50 tx=0x7f5498009750 comp rx=0 tx=0).stop 2024-03-09T00:10:15.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.958+0000 7f54afe5f740 1 -- 172.21.15.22:0/2325294070 shutdown_connections 2024-03-09T00:10:15.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.958+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d56450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.958+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d55f10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.958+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2325294070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d46d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.958+0000 7f54afe5f740 1 -- 172.21.15.22:0/2325294070 >> 172.21.15.22:0/2325294070 conn(0x55bbd5d47e90 msgr2=0x55bbd5d46660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:15.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.960+0000 7f54afe5f740 1 -- 172.21.15.22:0/2325294070 shutdown_connections 2024-03-09T00:10:15.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.964+0000 7f54afe5f740 1 -- 172.21.15.22:0/2325294070 wait complete. 2024-03-09T00:10:15.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.964+0000 7f54afe5f740 1 -- start start 2024-03-09T00:10:15.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.964+0000 7f54afe5f740 1 Processor -- start 2024-03-09T00:10:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.967+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.967+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d5cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.967+0000 7f54afe5f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d5d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.967+0000 7f54afe5f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a81c4680 con 0x55bbd5d4d760 2024-03-09T00:10:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.967+0000 7f54afe5f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a81c5960 con 0x55bbd5d4e080 2024-03-09T00:10:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.967+0000 7f54afe5f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a81c4ca0 con 0x55bbd5d56c60 2024-03-09T00:10:15.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.967+0000 7f54afe5f740 10 client.0 _ll_register_callbacks cb 0x7f549c004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:15.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.969+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d5d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.969+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d5d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35606/0 (socket says 172.21.15.22:35606) 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.969+0000 7f54a77fe640 1 -- 172.21.15.22:0/2967984831 learned_addr learned my addr 172.21.15.22:0/2967984831 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.970+0000 7f54a77fe640 1 -- 172.21.15.22:0/2967984831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 msgr2=0x55bbd5d5cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.970+0000 7f54a77fe640 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d5cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.970+0000 7f54a77fe640 1 -- 172.21.15.22:0/2967984831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 msgr2=0x55bbd5d5ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.970+0000 7f54a77fe640 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.970+0000 7f54a77fe640 1 -- 172.21.15.22:0/2967984831 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbd5dbced0 con 0x55bbd5d56c60 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.970+0000 7f54a77fe640 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d5d490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f549c011b60 tx=0x7f549c0119f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c248fc6d825c9a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:15.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.971+0000 7f548aff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:10:15.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.975+0000 7f548aff5640 1 -- 172.21.15.22:0/2967984831 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c032060 con 0x55bbd5d56c60 2024-03-09T00:10:15.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.975+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5498024040 con 0x55bbd5d56c60 2024-03-09T00:10:15.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.975+0000 7f548aff5640 1 -- 172.21.15.22:0/2967984831 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f549c003d60 con 0x55bbd5d56c60 2024-03-09T00:10:15.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.980+0000 7f548aff5640 1 -- 172.21.15.22:0/2967984831 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f549c005b90 con 0x55bbd5d56c60 2024-03-09T00:10:15.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.981+0000 7f54afe5f740 10 client.29752 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:15.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.981+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55bbd5dbe150 con 0x55bbd5d56c60 2024-03-09T00:10:15.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.982+0000 7f548aff5640 1 -- 172.21.15.22:0/2967984831 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5498024040 con 0x55bbd5d56c60 2024-03-09T00:10:15.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.982+0000 7f548aff5640 1 -- 172.21.15.22:0/2967984831 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f549c019510 con 0x55bbd5d56c60 2024-03-09T00:10:15.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.983+0000 7f54afe5f740 10 client.29752 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:15.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.983+0000 7f54afe5f740 10 client.29752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:15.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.983+0000 7f54afe5f740 20 client.29752 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:15.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.983+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5dbe830 0x55bbd5dc0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:15.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.983+0000 7f54afe5f740 4 client.29752 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:10:15.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.983+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bbd5cbe340 con 0x55bbd5dbe830 2024-03-09T00:10:15.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.984+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5dbe830 0x55bbd5dc0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:15.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.986+0000 7f548aff5640 10 client.29752 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:15.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.987+0000 7f54a6ffd640 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5dbe830 0x55bbd5dc0c90 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c00119b6146bae server_cookie=fd0139c4c2ee579a in_seq=0 out_seq=0 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.989+0000 7f548aff5640 1 -- 172.21.15.22:0/2967984831 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bbd5cbe340 con 0x55bbd5dbe830 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.989+0000 7f548aff5640 10 client.29752 handle_command_reply: tid=0 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.989+0000 7f54afe5f740 1 client.29752 shutdown 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 msgr2=0x55bbd5d5d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d5d490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f549c011b60 tx=0x7f549c0119f0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 shutdown_connections 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55bbd5dbe830 0x55bbd5dc0c90 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bbd5d56c60 0x55bbd5d5d490 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbd5d4e080 0x55bbd5d5cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 --2- 172.21.15.22:0/2967984831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bbd5d4d760 0x55bbd5d5ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.990+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 >> 172.21.15.22:0/2967984831 conn(0x55bbd5d47e90 msgr2=0x55bbd5d4ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:15.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.991+0000 7f548aff5640 0 client.29752 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.993+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 shutdown_connections 2024-03-09T00:10:15.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.994+0000 7f54afe5f740 1 -- 172.21.15.22:0/2967984831 wait complete. 2024-03-09T00:10:15.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:15.994+0000 7f54afe5f740 20 client.29752 trim_cache size 0 max 16384 2024-03-09T00:10:15.993 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:10:15.993 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:10:15.993 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:10:15.993 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:10:15.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54900a2af0 msgr2=0x7f54900a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54900a2af0 0x7f54900a4fb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54a8114890 tx=0x7f5498009190 comp rx=0 tx=0).stop 2024-03-09T00:10:15.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810db40 msgr2=0x7f54a811a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:15.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810db40 0x7f54a811a050 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f54a002c730 tx=0x7f54a0035d10 comp rx=0 tx=0).stop 2024-03-09T00:10:15.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 shutdown_connections 2024-03-09T00:10:15.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54900a2af0 0x7f54900a4fb0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f54a8114890 tx=0x7f5498009190 comp rx=0 tx=0).stop 2024-03-09T00:10:15.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a811aad0 0x7f54a81132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a810fb10 0x7f54a811a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 --2- 172.21.15.22:0/3653998049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54a810db40 0x7f54a811a050 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:15.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.002+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 >> 172.21.15.22:0/3653998049 conn(0x7f54a806c3d0 msgr2=0x7f54a811c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.006+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 shutdown_connections 2024-03-09T00:10:16.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.007+0000 7f54ad28e640 1 -- 172.21.15.22:0/3653998049 wait complete. 2024-03-09T00:10:16.020 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:10:16.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:10:16.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 33919 9.21 MB/sec execute 23 sec latency 467.442 ms 2024-03-09T00:10:16.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.217+0000 7ff1b759e640 1 -- 172.21.15.22:0/3792440391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b8100c60 msgr2=0x7ff1b8104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.217+0000 7ff1b759e640 1 --2- 172.21.15.22:0/3792440391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b8100c60 0x7ff1b8104ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009860 tx=0x7ff1a002bb70 comp rx=0 tx=0).stop 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.217+0000 7ff1b759e640 1 -- 172.21.15.22:0/3792440391 shutdown_connections 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.217+0000 7ff1b759e640 1 --2- 172.21.15.22:0/3792440391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b8105380 0x7ff1b810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.217+0000 7ff1b759e640 1 --2- 172.21.15.22:0/3792440391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b8100c60 0x7ff1b8104ca0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.217+0000 7ff1b759e640 1 --2- 172.21.15.22:0/3792440391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b81002b0 0x7ff1b8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.217+0000 7ff1b759e640 1 -- 172.21.15.22:0/3792440391 >> 172.21.15.22:0/3792440391 conn(0x7ff1b80fbb30 msgr2=0x7ff1b80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.218+0000 7ff1b759e640 1 -- 172.21.15.22:0/3792440391 shutdown_connections 2024-03-09T00:10:16.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.218+0000 7ff1b759e640 1 -- 172.21.15.22:0/3792440391 wait complete. 2024-03-09T00:10:16.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.220+0000 7ff1b759e640 1 Processor -- start 2024-03-09T00:10:16.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.220+0000 7ff1b759e640 1 -- start start 2024-03-09T00:10:16.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.220+0000 7ff1b759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b81002b0 0x7ff1b8196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.220+0000 7ff1b759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b8105380 0x7ff1b8196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b81002b0 0x7ff1b8196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b81002b0 0x7ff1b8196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35626/0 (socket says 172.21.15.22:35626) 2024-03-09T00:10:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b659c640 1 -- 172.21.15.22:0/2749201759 learned_addr learned my addr 172.21.15.22:0/2749201759 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b759e640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b819b010 0x7ff1b819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b81103f0 con 0x7ff1b8105380 2024-03-09T00:10:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b6d9d640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b819b010 0x7ff1b819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8110270 con 0x7ff1b819b010 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.221+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8110570 con 0x7ff1b81002b0 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1affff640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b8105380 0x7ff1b8196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1b6d9d640 1 -- 172.21.15.22:0/2749201759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b81002b0 msgr2=0x7ff1b8196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1b6d9d640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b81002b0 0x7ff1b8196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1b6d9d640 1 -- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b8105380 msgr2=0x7ff1b8196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1b6d9d640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b8105380 0x7ff1b8196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1b6d9d640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b819bb00 con 0x7ff1b819b010 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1b6d9d640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b819b010 0x7ff1b819b4c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000ef00 tx=0x7ff1b000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1af7fe640 1 -- 172.21.15.22:0/2749201759 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0004b20 con 0x7ff1b819b010 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1af7fe640 1 -- 172.21.15.22:0/2749201759 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1b0003e50 con 0x7ff1b819b010 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.222+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b819bdf0 con 0x7ff1b819b010 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.223+0000 7ff1af7fe640 1 -- 172.21.15.22:0/2749201759 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000b570 con 0x7ff1b819b010 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.223+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b80694e0 con 0x7ff1b819b010 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.224+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b8067770 con 0x7ff1b819b010 2024-03-09T00:10:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.225+0000 7ff1af7fe640 1 -- 172.21.15.22:0/2749201759 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff1b001d020 con 0x7ff1b819b010 2024-03-09T00:10:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.226+0000 7ff1af7fe640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1a40a2970 0x7ff1a40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.226+0000 7ff1af7fe640 1 -- 172.21.15.22:0/2749201759 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff1b00cf390 con 0x7ff1b819b010 2024-03-09T00:10:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.226+0000 7ff1b659c640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1a40a2970 0x7ff1a40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.226+0000 7ff1b659c640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1a40a2970 0x7ff1a40a4e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8005fd0 tx=0x7ff1a800b480 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.231+0000 7ff1af7fe640 1 -- 172.21.15.22:0/2749201759 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff1b009bca0 con 0x7ff1b819b010 2024-03-09T00:10:16.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.557+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1b8100db0 con 0x7ff1b819b010 2024-03-09T00:10:16.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.558+0000 7ff1af7fe640 1 -- 172.21.15.22:0/2749201759 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff1b009b5a0 con 0x7ff1b819b010 2024-03-09T00:10:16.554 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:10:16.554 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:10:14.710788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:10:16.554 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-03-09T00:10:16.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1a40a2970 msgr2=0x7ff1a40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1a40a2970 0x7ff1a40a4e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8005fd0 tx=0x7ff1a800b480 comp rx=0 tx=0).stop 2024-03-09T00:10:16.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b819b010 msgr2=0x7ff1b819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b819b010 0x7ff1b819b4c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000ef00 tx=0x7ff1b000c740 comp rx=0 tx=0).stop 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 shutdown_connections 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1a40a2970 0x7ff1a40a4e30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1b819b010 0x7ff1b819b4c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1b8105380 0x7ff1b8196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 --2- 172.21.15.22:0/2749201759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1b81002b0 0x7ff1b8196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 >> 172.21.15.22:0/2749201759 conn(0x7ff1b80fbb30 msgr2=0x7ff1b80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 shutdown_connections 2024-03-09T00:10:16.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.560+0000 7ff1b759e640 1 -- 172.21.15.22:0/2749201759 wait complete. 2024-03-09T00:10:16.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:16 smithi022 ceph-mon[31894]: pgmap v498: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 444 op/s 2024-03-09T00:10:16.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:16 smithi022 ceph-mon[31894]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:10:16.567 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:10:16.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:10:16.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:16 smithi062 ceph-mon[36129]: pgmap v498: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 444 op/s 2024-03-09T00:10:16.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:16 smithi062 ceph-mon[36129]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:10:16.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:16 smithi192 ceph-mon[35995]: pgmap v498: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 444 op/s 2024-03-09T00:10:16.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:16 smithi192 ceph-mon[35995]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 -- 172.21.15.22:0/98071029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc368006560 msgr2=0x7fc3680069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 --2- 172.21.15.22:0/98071029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc368006560 0x7fc3680069e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc360009930 tx=0x7fc36002bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 -- 172.21.15.22:0/98071029 shutdown_connections 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 --2- 172.21.15.22:0/98071029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368006f20 0x7fc3680a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 --2- 172.21.15.22:0/98071029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc368006560 0x7fc3680069e0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 --2- 172.21.15.22:0/98071029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36809be10 0x7fc36809c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 -- 172.21.15.22:0/98071029 >> 172.21.15.22:0/98071029 conn(0x7fc36800a740 msgr2=0x7fc36800ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 -- 172.21.15.22:0/98071029 shutdown_connections 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.775+0000 7fc37646b640 1 -- 172.21.15.22:0/98071029 wait complete. 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 Processor -- start 2024-03-09T00:10:16.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 -- start start 2024-03-09T00:10:16.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368006560 0x7fc36812c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368006f20 0x7fc36812cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc36809be10 0x7fc368130e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680a60a0 con 0x7fc36809be10 2024-03-09T00:10:16.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680a5f20 con 0x7fc368006560 2024-03-09T00:10:16.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc37646b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680a6220 con 0x7fc368006f20 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc375c6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc36809be10 0x7fc368130e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc375c6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc36809be10 0x7fc368130e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46532/0 (socket says 172.21.15.22:46532) 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.776+0000 7fc375c6a640 1 -- 172.21.15.22:0/4052449694 learned_addr learned my addr 172.21.15.22:0/4052449694 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc374c68640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368006f20 0x7fc36812cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc375c6a640 1 -- 172.21.15.22:0/4052449694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368006f20 msgr2=0x7fc36812cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc375c6a640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368006f20 0x7fc36812cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc375c6a640 1 -- 172.21.15.22:0/4052449694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368006560 msgr2=0x7fc36812c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc375c6a640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368006560 0x7fc36812c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc375c6a640 1 -- 172.21.15.22:0/4052449694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368131590 con 0x7fc36809be10 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc375c6a640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc36809be10 0x7fc368130e60 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc37004fe00 tx=0x7fc37006a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc3667fc640 1 -- 172.21.15.22:0/4052449694 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370086070 con 0x7fc36809be10 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368131880 con 0x7fc36809be10 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc3667fc640 1 -- 172.21.15.22:0/4052449694 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3700762d0 con 0x7fc36809be10 2024-03-09T00:10:16.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc3667fc640 1 -- 172.21.15.22:0/4052449694 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370081860 con 0x7fc36809be10 2024-03-09T00:10:16.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.777+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc368139110 con 0x7fc36809be10 2024-03-09T00:10:16.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.780+0000 7fc3667fc640 1 -- 172.21.15.22:0/4052449694 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc37007f070 con 0x7fc36809be10 2024-03-09T00:10:16.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 Processor -- start 2024-03-09T00:10:16.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 -- start start 2024-03-09T00:10:16.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb72d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb73a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb73cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eb6f4360 con 0x55a4eb72cff0 2024-03-09T00:10:16.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eb690ec0 con 0x55a4eb72d910 2024-03-09T00:10:16.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc37a382740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eb639d30 con 0x55a4eb73abd0 2024-03-09T00:10:16.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc3667fc640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc35c0a2970 0x7fc35c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc375469640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb73a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc375469640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb73a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36192/0 (socket says 172.21.15.22:36192) 2024-03-09T00:10:16.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc375469640 1 -- 172.21.15.22:0/5684608 learned_addr learned my addr 172.21.15.22:0/5684608 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc374c68640 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb72d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.781+0000 7fc3667fc640 1 -- 172.21.15.22:0/4052449694 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc370139be0 con 0x7fc36809be10 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc374c68640 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb73cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc375469640 1 -- 172.21.15.22:0/5684608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 msgr2=0x55a4eb73cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc375469640 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb73cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc375469640 1 -- 172.21.15.22:0/5684608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 msgr2=0x55a4eb72d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc375469640 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb72d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc375469640 1 -- 172.21.15.22:0/5684608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4eb638bc0 con 0x55a4eb72d910 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc375469640 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb73a690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009c40 tx=0x7fc36c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74c1bc2ae383cb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc35b7fe640 1 -- 172.21.15.22:0/5684608 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c005020 con 0x55a4eb72d910 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc35b7fe640 1 -- 172.21.15.22:0/5684608 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc36c0047a0 con 0x55a4eb72d910 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc35b7fe640 1 -- 172.21.15.22:0/5684608 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c010760 con 0x55a4eb72d910 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.782+0000 7fc375c6a640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc35c0a2970 0x7fc35c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc375c6a640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc35c0a2970 0x7fc35c0a4e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc370050950 tx=0x7fc370078040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc37a382740 1 -- 172.21.15.22:0/5684608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 msgr2=0x55a4eb73a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc37a382740 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb73a690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009c40 tx=0x7fc36c00e720 comp rx=0 tx=0).stop 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc37a382740 1 -- 172.21.15.22:0/5684608 shutdown_connections 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc37a382740 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb73cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc37a382740 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb73a690 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc37a382740 1 --2- 172.21.15.22:0/5684608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb72d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.783+0000 7fc37a382740 1 -- 172.21.15.22:0/5684608 >> 172.21.15.22:0/5684608 conn(0x55a4eb7279f0 msgr2=0x55a4eb727dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.784+0000 7fc37a382740 1 -- 172.21.15.22:0/5684608 shutdown_connections 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.784+0000 7fc37a382740 1 -- 172.21.15.22:0/5684608 wait complete. 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.784+0000 7fc37a382740 1 -- start start 2024-03-09T00:10:16.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.784+0000 7fc37a382740 1 Processor -- start 2024-03-09T00:10:16.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc37a382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb737c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc37a382740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72d910 0x55a4eb738140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc37a382740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb738680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc375469640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72d910 0x55a4eb738140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc375469640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72d910 0x55a4eb738140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46550/0 (socket says 172.21.15.22:46550) 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc374c68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb737c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc375469640 1 -- 172.21.15.22:0/1080912803 learned_addr learned my addr 172.21.15.22:0/1080912803 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eb62f4a0 con 0x55a4eb72d910 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eb68cf60 con 0x55a4eb72cff0 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eb73d990 con 0x55a4eb73abd0 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc37a382740 10 client.0 _ll_register_callbacks cb 0x55a4eb68ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.785+0000 7fc374c68640 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb738680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc375469640 1 -- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 msgr2=0x55a4eb738680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc375469640 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb738680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc375469640 1 -- 172.21.15.22:0/1080912803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 msgr2=0x55a4eb737c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc375469640 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb737c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc375469640 1 -- 172.21.15.22:0/1080912803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4eb3a53f0 con 0x55a4eb72d910 2024-03-09T00:10:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc374c68640 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb737c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc375469640 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72d910 0x55a4eb738140 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc36c000c00 tx=0x7fc36c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f231dde589cf12 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc36c0047a0 con 0x55a4eb72d910 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc358ff9640 1 -- 172.21.15.22:0/1080912803 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c026020 con 0x55a4eb72d910 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc358ff9640 1 -- 172.21.15.22:0/1080912803 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc36c005020 con 0x55a4eb72d910 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc358ff9640 1 -- 172.21.15.22:0/1080912803 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc36c01d040 con 0x55a4eb72d910 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc37a382740 10 client.20394 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.786+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=37}) v3 -- 0x55a4eb46f7d0 con 0x55a4eb72d910 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc358ff9640 1 -- 172.21.15.22:0/1080912803 <== mon.0 v2:172.21.15.22:3300/0 4 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc36c005260 con 0x55a4eb72d910 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc37a382740 10 client.20394 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc37a382740 10 client.20394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc37a382740 20 client.20394 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc37a382740 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb74cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc37a382740 4 client.20394 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc36c0047a0 con 0x55a4eb74ab70 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.787+0000 7fc374c68640 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb74cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc374c68640 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb74cf60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4940e4b761f8abe server_cookie=e6036b9fbd1378ad in_seq=0 out_seq=0 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc358ff9640 10 client.20394 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc358ff9640 1 -- 172.21.15.22:0/1080912803 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fc36c0047a0 con 0x55a4eb74ab70 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc358ff9640 10 client.20394 handle_command_reply: tid=0 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 client.20394 shutdown 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72d910 msgr2=0x55a4eb738140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72d910 0x55a4eb738140 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc36c000c00 tx=0x7fc36c0049d0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 shutdown_connections 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb74cf60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc358ff9640 0 client.20394 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb738680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72d910 0x55a4eb738140 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.788+0000 7fc37a382740 1 --2- 172.21.15.22:0/1080912803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb737c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.789+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 >> 172.21.15.22:0/1080912803 conn(0x55a4eb7279f0 msgr2=0x55a4eb729740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.789+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 shutdown_connections 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.789+0000 7fc37a382740 1 -- 172.21.15.22:0/1080912803 wait complete. 2024-03-09T00:10:16.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.789+0000 7fc37a382740 20 client.20394 trim_cache size 0 max 16384 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.803+0000 7fc37a382740 1 Processor -- start 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.803+0000 7fc37a382740 1 -- start start 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.803+0000 7fc37a382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb73a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc37a382740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb72d910 0x55a4eb734e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc37a382740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb73abd0 0x55a4eb7353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc37a382740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368143590 con 0x55a4eb73abd0 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc37a382740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368141130 con 0x55a4eb72cff0 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc37a382740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36813f1f0 con 0x55a4eb72d910 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb72d910 0x55a4eb734e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb72d910 0x55a4eb734e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35668/0 (socket says 172.21.15.22:35668) 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc374c68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb73a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 -- 172.21.15.22:0/2172781466 learned_addr learned my addr 172.21.15.22:0/2172781466 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc374c68640 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb73abd0 0x55a4eb7353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 -- 172.21.15.22:0/2172781466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 msgr2=0x55a4eb73a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb73a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 -- 172.21.15.22:0/2172781466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb73abd0 msgr2=0x55a4eb7353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb73abd0 0x55a4eb7353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 -- 172.21.15.22:0/2172781466 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4eb721e40 con 0x55a4eb72d910 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc374c68640 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb73a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.804+0000 7fc375469640 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb72d910 0x55a4eb734e60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc36c004fd0 tx=0x7fc36c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c4ef146546dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc35affd640 1 -- 172.21.15.22:0/2172781466 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c026070 con 0x55a4eb72d910 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc35affd640 1 -- 172.21.15.22:0/2172781466 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc36c039070 con 0x55a4eb72d910 2024-03-09T00:10:16.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc35affd640 1 -- 172.21.15.22:0/2172781466 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c010880 con 0x55a4eb72d910 2024-03-09T00:10:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc37a382740 1 -- 172.21.15.22:0/2172781466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb72d910 msgr2=0x55a4eb734e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc37a382740 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb72d910 0x55a4eb734e60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc36c004fd0 tx=0x7fc36c01e000 comp rx=0 tx=0).stop 2024-03-09T00:10:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc37a382740 1 -- 172.21.15.22:0/2172781466 shutdown_connections 2024-03-09T00:10:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc37a382740 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb73abd0 0x55a4eb7353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc37a382740 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb72d910 0x55a4eb734e60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc37a382740 1 --2- 172.21.15.22:0/2172781466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72cff0 0x55a4eb73a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.805+0000 7fc37a382740 1 -- 172.21.15.22:0/2172781466 >> 172.21.15.22:0/2172781466 conn(0x55a4eb7279f0 msgr2=0x55a4eb7364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.806+0000 7fc37a382740 1 -- 172.21.15.22:0/2172781466 shutdown_connections 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.806+0000 7fc37a382740 1 -- 172.21.15.22:0/2172781466 wait complete. 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.806+0000 7fc37a382740 1 -- start start 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.806+0000 7fc37a382740 1 Processor -- start 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.806+0000 7fc37a382740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb741c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc37a382740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb740020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc37a382740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb740560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc37a382740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36813f7f0 con 0x55a4eb72cff0 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc37a382740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368140ad0 con 0x55a4eb72d910 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc37a382740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36813fe10 con 0x55a4eb73abd0 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc374c68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb741c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc374c68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb741c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46560/0 (socket says 172.21.15.22:46560) 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc375469640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb740020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc374c68640 1 -- 172.21.15.22:0/2453918188 learned_addr learned my addr 172.21.15.22:0/2453918188 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc37a382740 10 client.0 _ll_register_callbacks cb 0x7fc36c009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc374c68640 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb740560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc375469640 1 -- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 msgr2=0x55a4eb740560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc375469640 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb740560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc375469640 1 -- 172.21.15.22:0/2453918188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 msgr2=0x55a4eb741c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc375469640 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb741c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc375469640 1 -- 172.21.15.22:0/2453918188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4eb740aa0 con 0x55a4eb72d910 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc375469640 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb740020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0054d0 tx=0x7fc36c0049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a3eb888aecc129 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.807+0000 7fc358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc358ff9640 1 -- 172.21.15.22:0/2453918188 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c042060 con 0x55a4eb72d910 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc358ff9640 1 -- 172.21.15.22:0/2453918188 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fc36c010840 con 0x55a4eb72d910 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc36c039070 con 0x55a4eb72d910 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc358ff9640 1 -- 172.21.15.22:0/2453918188 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc36c026030 con 0x55a4eb72d910 2024-03-09T00:10:16.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc37a382740 10 client.29615 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a4eb7a1c40 con 0x55a4eb72d910 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc358ff9640 1 -- 172.21.15.22:0/2453918188 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc36c039070 con 0x55a4eb72d910 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.808+0000 7fc358ff9640 1 -- 172.21.15.22:0/2453918188 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc36c0346c0 con 0x55a4eb72d910 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc37a382740 10 client.29615 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc37a382740 10 client.29615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc37a382740 20 client.29615 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc37a382740 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb7a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc37a382740 4 client.29615 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a4eb69e400 con 0x55a4eb74ab70 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc374c68640 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb7a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc358ff9640 10 client.29615 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.809+0000 7fc374c68640 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb7a4210 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8acbc4ff64b80a45 server_cookie=28cfc50b1de44357 in_seq=0 out_seq=0 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc358ff9640 1 -- 172.21.15.22:0/2453918188 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a4eb69e400 con 0x55a4eb74ab70 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc358ff9640 10 client.29615 handle_command_reply: tid=0 2024-03-09T00:10:16.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 client.29615 shutdown 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 msgr2=0x55a4eb740020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb740020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0054d0 tx=0x7fc36c0049d0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 shutdown_connections 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55a4eb74ab70 0x55a4eb7a4210 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a4eb73abd0 0x55a4eb740560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc358ff9640 0 client.29615 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4eb72d910 0x55a4eb740020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 --2- 172.21.15.22:0/2453918188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4eb72cff0 0x55a4eb741c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 >> 172.21.15.22:0/2453918188 conn(0x55a4eb7279f0 msgr2=0x55a4eb726100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 shutdown_connections 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 1 -- 172.21.15.22:0/2453918188 wait complete. 2024-03-09T00:10:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.810+0000 7fc37a382740 20 client.29615 trim_cache size 0 max 16384 2024-03-09T00:10:16.812 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:10:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc35c0a2970 msgr2=0x7fc35c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc35c0a2970 0x7fc35c0a4e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc370050950 tx=0x7fc370078040 comp rx=0 tx=0).stop 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc36809be10 msgr2=0x7fc368130e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc36809be10 0x7fc368130e60 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc37004fe00 tx=0x7fc37006a410 comp rx=0 tx=0).stop 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 shutdown_connections 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fc35c0a2970 0x7fc35c0a4e30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc36809be10 0x7fc368130e60 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc368006f20 0x7fc36812cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 --2- 172.21.15.22:0/4052449694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc368006560 0x7fc36812c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 >> 172.21.15.22:0/4052449694 conn(0x7fc36800a740 msgr2=0x7fc368093b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 shutdown_connections 2024-03-09T00:10:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.811+0000 7fc37646b640 1 -- 172.21.15.22:0/4052449694 wait complete. 2024-03-09T00:10:16.817 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:10:16.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:10:16.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 -- 172.21.15.22:0/312146506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181054c0 msgr2=0x7f8a1810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 --2- 172.21.15.22:0/312146506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181054c0 0x7f8a1810d090 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009ca0 tx=0x7f8a1002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 -- 172.21.15.22:0/312146506 shutdown_connections 2024-03-09T00:10:16.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 --2- 172.21.15.22:0/312146506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181054c0 0x7f8a1810d090 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 --2- 172.21.15.22:0/312146506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a18100da0 0x7f8a18104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 --2- 172.21.15.22:0/312146506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a181003f0 0x7f8a181007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 -- 172.21.15.22:0/312146506 >> 172.21.15.22:0/312146506 conn(0x7f8a180772e0 msgr2=0x7f8a180776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 -- 172.21.15.22:0/312146506 shutdown_connections 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 -- 172.21.15.22:0/312146506 wait complete. 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.955+0000 7f8a1d789640 1 Processor -- start 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a1d789640 1 -- start start 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a1d789640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a181003f0 0x7f8a1819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a1d789640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a181054c0 0x7f8a1819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a1d789640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181978d0 0x7f8a18197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a1d789640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18110560 con 0x7f8a181978d0 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a1d789640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181103e0 con 0x7f8a181054c0 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a1d789640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181106e0 con 0x7f8a181003f0 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181978d0 0x7f8a18197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181978d0 0x7f8a18197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46576/0 (socket says 172.21.15.22:46576) 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 -- 172.21.15.22:0/3660497430 learned_addr learned my addr 172.21.15.22:0/3660497430 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a181054c0 0x7f8a1819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a181003f0 0x7f8a1819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 -- 172.21.15.22:0/3660497430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a181003f0 msgr2=0x7f8a1819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a181003f0 0x7f8a1819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 -- 172.21.15.22:0/3660497430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a181054c0 msgr2=0x7f8a1819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a181054c0 0x7f8a1819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 -- 172.21.15.22:0/3660497430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a181983f0 con 0x7f8a181978d0 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.956+0000 7f8a177fe640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181978d0 0x7f8a18197d80 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f8a10037040 tx=0x7f8a10032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.957+0000 7f8a14ff9640 1 -- 172.21.15.22:0/3660497430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10041020 con 0x7f8a181978d0 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.957+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a18198680 con 0x7f8a181978d0 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.957+0000 7f8a14ff9640 1 -- 172.21.15.22:0/3660497430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a10032e80 con 0x7f8a181978d0 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.957+0000 7f8a14ff9640 1 -- 172.21.15.22:0/3660497430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10039040 con 0x7f8a181978d0 2024-03-09T00:10:16.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.957+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181a48d0 con 0x7f8a181978d0 2024-03-09T00:10:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.959+0000 7f8a14ff9640 1 -- 172.21.15.22:0/3660497430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8a10044020 con 0x7f8a181978d0 2024-03-09T00:10:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a14ff9640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a040a2970 0x7f8a040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a14ff9640 1 -- 172.21.15.22:0/3660497430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f8a100f0e40 con 0x7f8a181978d0 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a040a2970 0x7f8a040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 Processor -- start 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 -- start start 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a040a2970 0x7f8a040a4e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8a18198a10 tx=0x7f8a08005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9c91800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91d40 0x5644c9cab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c9c5e360 con 0x5644c9ca3830 2024-03-09T00:10:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c9bfaec0 con 0x5644c9c91d40 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.960+0000 7f8a2036a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c9ba3d30 con 0x5644c9c91380 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46584/0 (socket says 172.21.15.22:46584) 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 -- 172.21.15.22:0/974091035 learned_addr learned my addr 172.21.15.22:0/974091035 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a177fe640 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91d40 0x5644c9cab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9c91800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 -- 172.21.15.22:0/974091035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 msgr2=0x5644c9c91800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9c91800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 -- 172.21.15.22:0/974091035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91d40 msgr2=0x5644c9cab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91d40 0x5644c9cab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 -- 172.21.15.22:0/974091035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644c9ba2bc0 con 0x5644c9ca3830 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca3c10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009580 tx=0x7f8a08014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c242a4c14195ba8a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a0d7fa640 1 -- 172.21.15.22:0/974091035 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08012e60 con 0x5644c9ca3830 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a0d7fa640 1 -- 172.21.15.22:0/974091035 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a0801fb00 con 0x5644c9ca3830 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.961+0000 7f8a0d7fa640 1 -- 172.21.15.22:0/974091035 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08027d50 con 0x5644c9ca3830 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 -- 172.21.15.22:0/974091035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 msgr2=0x5644c9ca3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca3c10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009580 tx=0x7f8a08014000 comp rx=0 tx=0).stop 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 -- 172.21.15.22:0/974091035 shutdown_connections 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91d40 0x5644c9cab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9c91800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 --2- 172.21.15.22:0/974091035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca3c10 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 -- 172.21.15.22:0/974091035 >> 172.21.15.22:0/974091035 conn(0x5644c9c93e40 msgr2=0x5644c9c94220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 -- 172.21.15.22:0/974091035 shutdown_connections 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.962+0000 7f8a2036a740 1 -- 172.21.15.22:0/974091035 wait complete. 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 -- start start 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 Processor -- start 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91380 0x5644c9ca8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9ca9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c9b994a0 con 0x5644c9ca3830 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c9bf6f60 con 0x5644c9c91380 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c9cabd60 con 0x5644c9c91d40 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a2036a740 10 client.0 _ll_register_callbacks cb 0x5644c9bf4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46592/0 (socket says 172.21.15.22:46592) 2024-03-09T00:10:16.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a177fe640 1 -- 172.21.15.22:0/1176865038 learned_addr learned my addr 172.21.15.22:0/1176865038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91380 0x5644c9ca8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.963+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9ca9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a177fe640 1 -- 172.21.15.22:0/1176865038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 msgr2=0x5644c9ca9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a177fe640 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9ca9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a177fe640 1 -- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91380 msgr2=0x5644c9ca8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a177fe640 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91380 0x5644c9ca8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a177fe640 1 -- 172.21.15.22:0/1176865038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644c990f3f0 con 0x5644c9ca3830 2024-03-09T00:10:16.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a177fe640 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca95d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005e30 tx=0x7f8a100bcac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66145b94ecc16906 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f89fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f89fe7fc640 1 -- 172.21.15.22:0/1176865038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100bd860 con 0x5644c9ca3830 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f89fe7fc640 1 -- 172.21.15.22:0/1176865038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f8a100c5470 con 0x5644c9ca3830 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a0801fb00 con 0x5644c9ca3830 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f89fe7fc640 1 -- 172.21.15.22:0/1176865038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a10003e20 con 0x5644c9ca3830 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a2036a740 10 client.20412 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5644c99d97d0 con 0x5644c9ca3830 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.964+0000 7f89fe7fc640 1 -- 172.21.15.22:0/1176865038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a0801fb00 con 0x5644c9ca3830 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f89fe7fc640 1 -- 172.21.15.22:0/1176865038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a1003c030 con 0x5644c9ca3830 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a2036a740 10 client.20412 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a2036a740 10 client.20412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a2036a740 20 client.20412 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a2036a740 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9cb8ff0 0x5644c9cbb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a2036a740 4 client.20412 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644c9c08400 con 0x5644c9cb8ff0 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9cb8ff0 0x5644c9cbb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f89fe7fc640 10 client.20412 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.965+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9cb8ff0 0x5644c9cbb410 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=738e51e0bb0b8cd1 server_cookie=4a09d4506e80b58 in_seq=0 out_seq=0 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f89fe7fc640 1 -- 172.21.15.22:0/1176865038 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644c9c08400 con 0x5644c9cb8ff0 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f89fe7fc640 10 client.20412 handle_command_reply: tid=0 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 client.20412 shutdown 2024-03-09T00:10:16.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 msgr2=0x5644c9ca95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca95d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005e30 tx=0x7f8a100bcac0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 shutdown_connections 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9cb8ff0 0x5644c9cbb410 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9ca3830 0x5644c9ca95d0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9ca9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f89fe7fc640 0 client.20412 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 --2- 172.21.15.22:0/1176865038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9c91380 0x5644c9ca8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.966+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 >> 172.21.15.22:0/1176865038 conn(0x5644c9c93e40 msgr2=0x5644c9c935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.967+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 shutdown_connections 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.967+0000 7f8a2036a740 1 -- 172.21.15.22:0/1176865038 wait complete. 2024-03-09T00:10:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.967+0000 7f8a2036a740 20 client.20412 trim_cache size 0 max 16384 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.980+0000 7f8a2036a740 1 Processor -- start 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.980+0000 7f8a2036a740 1 -- start start 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9cab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91d40 0x5644c9ca1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9caff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a2036a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181aeaf0 con 0x5644c9c91d40 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a2036a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181ac690 con 0x5644c9ca3830 2024-03-09T00:10:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a2036a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181aa750 con 0x5644c9c91380 2024-03-09T00:10:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91d40 0x5644c9ca1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91d40 0x5644c9ca1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46606/0 (socket says 172.21.15.22:46606) 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 -- 172.21.15.22:0/993113917 learned_addr learned my addr 172.21.15.22:0/993113917 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9cab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a177fe640 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9caff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 -- 172.21.15.22:0/993113917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 msgr2=0x5644c9cab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9cab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 -- 172.21.15.22:0/993113917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 msgr2=0x5644c9caff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9caff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.981+0000 7f8a0ffff640 1 -- 172.21.15.22:0/993113917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644c9cc0600 con 0x5644c9c91d40 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.982+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91d40 0x5644c9ca1310 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009d00 tx=0x7f8a00010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28841e2b51d6be server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.982+0000 7f8a0cff9640 1 -- 172.21.15.22:0/993113917 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00021070 con 0x5644c9c91d40 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.982+0000 7f8a0cff9640 1 -- 172.21.15.22:0/993113917 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a000035f0 con 0x5644c9c91d40 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.982+0000 7f8a0cff9640 1 -- 172.21.15.22:0/993113917 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0001c650 con 0x5644c9c91d40 2024-03-09T00:10:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 -- 172.21.15.22:0/993113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91d40 msgr2=0x5644c9ca1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91d40 0x5644c9ca1310 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009d00 tx=0x7f8a00010800 comp rx=0 tx=0).stop 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 -- 172.21.15.22:0/993113917 shutdown_connections 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9caff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91d40 0x5644c9ca1310 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 --2- 172.21.15.22:0/993113917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91380 0x5644c9cab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 -- 172.21.15.22:0/993113917 >> 172.21.15.22:0/993113917 conn(0x5644c9c93e40 msgr2=0x5644c9ca1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 -- 172.21.15.22:0/993113917 shutdown_connections 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 -- 172.21.15.22:0/993113917 wait complete. 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 -- start start 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.983+0000 7f8a2036a740 1 Processor -- start 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91380 0x5644c9caf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9caf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a2036a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9d094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a2036a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181aad50 con 0x5644c9c91380 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a2036a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181ac030 con 0x5644c9ca3830 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a2036a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181ab370 con 0x5644c9c91d40 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a2036a740 10 client.0 _ll_register_callbacks cb 0x7f8a08009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91380 0x5644c9caf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91380 0x5644c9caf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46608/0 (socket says 172.21.15.22:46608) 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 -- 172.21.15.22:0/4122090448 learned_addr learned my addr 172.21.15.22:0/4122090448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9caf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a177fe640 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9d094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 -- 172.21.15.22:0/4122090448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 msgr2=0x5644c9caf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9caf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 -- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 msgr2=0x5644c9d094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9d094d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 -- 172.21.15.22:0/4122090448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644c9d09ac0 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f8a16ffd640 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91380 0x5644c9caf1d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8a08027880 tx=0x7f8a08011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b9dce563c5aff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.984+0000 7f89fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f89fe7fc640 1 -- 172.21.15.22:0/4122090448 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08037070 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f89fe7fc640 1 -- 172.21.15.22:0/4122090448 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f8a08027b20 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a000035f0 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f89fe7fc640 1 -- 172.21.15.22:0/4122090448 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a08042050 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 10 client.20418 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5644c9d10e40 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f89fe7fc640 1 -- 172.21.15.22:0/4122090448 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a000035f0 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f89fe7fc640 1 -- 172.21.15.22:0/4122090448 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a08033ce0 con 0x5644c9c91380 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 10 client.20418 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:16.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 10 client.20418 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:10:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 20 client.20418 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9d111e0 0x5644c9d13600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 4 client.20418 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:10:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.985+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5644c9c02750 con 0x5644c9d111e0 2024-03-09T00:10:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.986+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9d111e0 0x5644c9d13600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.986+0000 7f89fe7fc640 10 client.20418 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.986+0000 7f8a0ffff640 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9d111e0 0x5644c9d13600 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=211998fcaab013bd server_cookie=55fd9bbba2015580 in_seq=0 out_seq=0 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.986+0000 7f89fe7fc640 1 -- 172.21.15.22:0/4122090448 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5644c9c02750 con 0x5644c9d111e0 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.986+0000 7f89fe7fc640 10 client.20418 handle_command_reply: tid=0 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.986+0000 7f8a2036a740 1 client.20418 shutdown 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91380 msgr2=0x5644c9caf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91380 0x5644c9caf1d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8a08027880 tx=0x7f8a08011b80 comp rx=0 tx=0).stop 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 shutdown_connections 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x5644c9d111e0 0x5644c9d13600 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5644c9ca3830 0x5644c9d094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5644c9c91d40 0x5644c9caf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 --2- 172.21.15.22:0/4122090448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644c9c91380 0x5644c9caf1d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 >> 172.21.15.22:0/4122090448 conn(0x5644c9c93e40 msgr2=0x5644c9ca7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f89fe7fc640 0 client.20418 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 shutdown_connections 2024-03-09T00:10:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 1 -- 172.21.15.22:0/4122090448 wait complete. 2024-03-09T00:10:16.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a2036a740 20 client.20418 trim_cache size 0 max 16384 2024-03-09T00:10:16.989 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a040a2970 msgr2=0x7f8a040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a040a2970 0x7f8a040a4e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8a18198a10 tx=0x7f8a08005f90 comp rx=0 tx=0).stop 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181978d0 msgr2=0x7f8a18197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181978d0 0x7f8a18197d80 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f8a10037040 tx=0x7f8a10032c90 comp rx=0 tx=0).stop 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 shutdown_connections 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8a040a2970 0x7f8a040a4e30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a181978d0 0x7f8a18197d80 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a181054c0 0x7f8a1819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 --2- 172.21.15.22:0/3660497430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a181003f0 0x7f8a1819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 >> 172.21.15.22:0/3660497430 conn(0x7f8a180772e0 msgr2=0x7f8a18069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.987+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 shutdown_connections 2024-03-09T00:10:16.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:16.988+0000 7f8a1d789640 1 -- 172.21.15.22:0/3660497430 wait complete. 2024-03-09T00:10:16.997 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:10:16.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:10:17.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 34132 9.57 MB/sec execute 24 sec latency 342.060 ms 2024-03-09T00:10:17.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 -- 172.21.15.22:0/230208153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80102e30 msgr2=0x7faa8010b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 --2- 172.21.15.22:0/230208153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80102e30 0x7faa8010b310 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7faa74009900 tx=0x7faa7402bcd0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 -- 172.21.15.22:0/230208153 shutdown_connections 2024-03-09T00:10:17.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 --2- 172.21.15.22:0/230208153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa8010b850 0x7faa8010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 --2- 172.21.15.22:0/230208153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa80102e30 0x7faa8010b310 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 --2- 172.21.15.22:0/230208153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa80102510 0x7faa801028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 -- 172.21.15.22:0/230208153 >> 172.21.15.22:0/230208153 conn(0x7faa800773a0 msgr2=0x7faa800777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 -- 172.21.15.22:0/230208153 shutdown_connections 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.126+0000 7faa84fb9640 1 -- 172.21.15.22:0/230208153 wait complete. 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 Processor -- start 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 -- start start 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa80102510 0x7faa80199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80102e30 0x7faa8019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa8010b850 0x7faa8019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa80111680 con 0x7faa8010b850 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa80111500 con 0x7faa80102e30 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa84fb9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa80111800 con 0x7faa80102510 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa7ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa8010b850 0x7faa8019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa7e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa80102510 0x7faa80199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa7ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa8010b850 0x7faa8019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46628/0 (socket says 172.21.15.22:46628) 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa7ed9d640 1 -- 172.21.15.22:0/4235068875 learned_addr learned my addr 172.21.15.22:0/4235068875 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.127+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80102e30 0x7faa8019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa7ed9d640 1 -- 172.21.15.22:0/4235068875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa80102510 msgr2=0x7faa80199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa7ed9d640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa80102510 0x7faa80199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa7ed9d640 1 -- 172.21.15.22:0/4235068875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80102e30 msgr2=0x7faa8019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa7ed9d640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80102e30 0x7faa8019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa7ed9d640 1 -- 172.21.15.22:0/4235068875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8019ee50 con 0x7faa8010b850 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa7ed9d640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa8010b850 0x7faa8019e720 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7faa7800eb10 tx=0x7faa7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa6f7fe640 1 -- 172.21.15.22:0/4235068875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78004b20 con 0x7faa8010b850 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8019f140 con 0x7faa8010b850 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa6f7fe640 1 -- 172.21.15.22:0/4235068875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa78004cc0 con 0x7faa8010b850 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa801a69d0 con 0x7faa8010b850 2024-03-09T00:10:17.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.128+0000 7faa6f7fe640 1 -- 172.21.15.22:0/4235068875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78010760 con 0x7faa8010b850 2024-03-09T00:10:17.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.130+0000 7faa6f7fe640 1 -- 172.21.15.22:0/4235068875 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7faa78005020 con 0x7faa8010b850 2024-03-09T00:10:17.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.131+0000 7faa6f7fe640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faa680a2970 0x7faa680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.131+0000 7faa87b96740 1 Processor -- start 2024-03-09T00:10:17.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.131+0000 7faa7e59c640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faa680a2970 0x7faa680a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.131+0000 7faa87b96740 1 -- start start 2024-03-09T00:10:17.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.131+0000 7faa6f7fe640 1 -- 172.21.15.22:0/4235068875 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7faa780cf820 con 0x7faa8010b850 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa87b96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a635580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa87b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a63e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa87b96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a640a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa87b96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393a5fc360 con 0x56393a635ac0 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa87b96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393a598ec0 con 0x56393a6351a0 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa87b96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393a541d30 con 0x56393a63e6a0 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a63e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a63e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46630/0 (socket says 172.21.15.22:46630) 2024-03-09T00:10:17.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3584333982 learned_addr learned my addr 172.21.15.22:0/3584333982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:17.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7ed9d640 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a640a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7e59c640 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a635580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7e59c640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faa680a2970 0x7faa680a4e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa70009c20 tx=0x7faa70005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:17.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3584333982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 msgr2=0x56393a640a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a640a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3584333982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 msgr2=0x56393a635580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a635580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.132+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3584333982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393a540bc0 con 0x56393a635ac0 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.133+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a63e160 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7faa74009900 tx=0x7faa740095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d384d02d9aeedf1b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.133+0000 7faa6cff9640 1 -- 172.21.15.22:0/3584333982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74043070 con 0x56393a635ac0 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.133+0000 7faa6cff9640 1 -- 172.21.15.22:0/3584333982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa7402c670 con 0x56393a635ac0 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.133+0000 7faa6cff9640 1 -- 172.21.15.22:0/3584333982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa740049c0 con 0x56393a635ac0 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.133+0000 7faa87b96740 1 -- 172.21.15.22:0/3584333982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 msgr2=0x56393a63e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.133+0000 7faa87b96740 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a63e160 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7faa74009900 tx=0x7faa740095e0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- 172.21.15.22:0/3584333982 shutdown_connections 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a640a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a63e160 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 --2- 172.21.15.22:0/3584333982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a635580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- 172.21.15.22:0/3584333982 >> 172.21.15.22:0/3584333982 conn(0x56393a62f8d0 msgr2=0x56393a62fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- 172.21.15.22:0/3584333982 shutdown_connections 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- 172.21.15.22:0/3584333982 wait complete. 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- start start 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 Processor -- start 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a6351a0 0x56393a63b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a635ac0 0x56393a63bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393a5374a0 con 0x56393a6351a0 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393a594f60 con 0x56393a635ac0 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.134+0000 7faa87b96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393a641430 con 0x56393a63e6a0 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa87b96740 10 client.0 _ll_register_callbacks cb 0x56393a592d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a6351a0 0x56393a63b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a635ac0 0x56393a63bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a635ac0 0x56393a63bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36334/0 (socket says 172.21.15.22:36334) 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a6351a0 0x56393a63b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46634/0 (socket says 172.21.15.22:46634) 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7dd9b640 1 -- 172.21.15.22:0/1648544196 learned_addr learned my addr 172.21.15.22:0/1648544196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7ed9d640 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 -- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 msgr2=0x56393a63c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 -- 172.21.15.22:0/1648544196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a635ac0 msgr2=0x56393a63bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a635ac0 0x56393a63bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 -- 172.21.15.22:0/1648544196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56393a2ad3f0 con 0x56393a6351a0 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a635ac0 0x56393a63bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa7e59c640 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a6351a0 0x56393a63b700 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7faa70009a90 tx=0x7faa70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd902bc152141e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:17.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa65ffb640 1 -- 172.21.15.22:0/1648544196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7001b670 con 0x56393a6351a0 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa7402c670 con 0x56393a6351a0 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.135+0000 7faa65ffb640 1 -- 172.21.15.22:0/1648544196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7faa7001b810 con 0x56393a6351a0 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa65ffb640 1 -- 172.21.15.22:0/1648544196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa7001ba10 con 0x56393a6351a0 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 10 client.20430 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56393a3777d0 con 0x56393a6351a0 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa65ffb640 1 -- 172.21.15.22:0/1648544196 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa7402c670 con 0x56393a6351a0 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa65ffb640 1 -- 172.21.15.22:0/1648544196 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa70027420 con 0x56393a6351a0 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 10 client.20430 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 10 client.20430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 20 client.20430 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a64e9f0 0x56393a650de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 4 client.20430 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:17.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56393a5a6400 con 0x56393a64e9f0 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.136+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a64e9f0 0x56393a650de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.137+0000 7faa65ffb640 10 client.20430 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.137+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a64e9f0 0x56393a650de0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8d09bea0c30635 server_cookie=47cb83646511b8cb in_seq=0 out_seq=0 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa65ffb640 1 -- 172.21.15.22:0/1648544196 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56393a5a6400 con 0x56393a64e9f0 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa65ffb640 10 client.20430 handle_command_reply: tid=0 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 client.20430 shutdown 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a6351a0 msgr2=0x56393a63b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a6351a0 0x56393a63b700 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7faa70009a90 tx=0x7faa70004730 comp rx=0 tx=0).stop 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 shutdown_connections 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a64e9f0 0x56393a650de0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa65ffb640 0 client.20430 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a635ac0 0x56393a63bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 --2- 172.21.15.22:0/1648544196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a6351a0 0x56393a63b700 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 >> 172.21.15.22:0/1648544196 conn(0x56393a62f8d0 msgr2=0x56393a631620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:17.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 shutdown_connections 2024-03-09T00:10:17.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.139+0000 7faa87b96740 1 -- 172.21.15.22:0/1648544196 wait complete. 2024-03-09T00:10:17.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.140+0000 7faa87b96740 20 client.20430 trim_cache size 0 max 16384 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.154+0000 7faa87b96740 1 Processor -- start 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.154+0000 7faa87b96740 1 -- start start 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa87b96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a63de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa87b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a639fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa87b96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a639fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a639fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46648/0 (socket says 172.21.15.22:46648) 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 -- 172.21.15.22:0/79998447 learned_addr learned my addr 172.21.15.22:0/79998447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801b2db0 con 0x56393a635ac0 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801b0950 con 0x56393a6351a0 2024-03-09T00:10:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801aea10 con 0x56393a63e6a0 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7e59c640 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a63de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7ed9d640 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 -- 172.21.15.22:0/79998447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 msgr2=0x56393a63a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 -- 172.21.15.22:0/79998447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 msgr2=0x56393a63de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a63de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 -- 172.21.15.22:0/79998447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393a640dc0 con 0x56393a635ac0 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.155+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a639fd0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7faa74005ec0 tx=0x7faa7402cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6738be78dfe06a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.156+0000 7faa67fff640 1 -- 172.21.15.22:0/79998447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74043040 con 0x56393a635ac0 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.156+0000 7faa67fff640 1 -- 172.21.15.22:0/79998447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa74050be0 con 0x56393a635ac0 2024-03-09T00:10:17.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.156+0000 7faa67fff640 1 -- 172.21.15.22:0/79998447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74004c40 con 0x56393a635ac0 2024-03-09T00:10:17.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.156+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 msgr2=0x56393a639fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.156+0000 7faa87b96740 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a639fd0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7faa74005ec0 tx=0x7faa7402cf90 comp rx=0 tx=0).stop 2024-03-09T00:10:17.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 shutdown_connections 2024-03-09T00:10:17.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a63a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a639fd0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 --2- 172.21.15.22:0/79998447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a63de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 >> 172.21.15.22:0/79998447 conn(0x56393a62f8d0 msgr2=0x56393a655760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:17.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 shutdown_connections 2024-03-09T00:10:17.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 -- 172.21.15.22:0/79998447 wait complete. 2024-03-09T00:10:17.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 -- start start 2024-03-09T00:10:17.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 Processor -- start 2024-03-09T00:10:17.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.157+0000 7faa87b96740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a645930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa87b96740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a643cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa87b96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a644210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa87b96740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801af010 con 0x56393a635ac0 2024-03-09T00:10:17.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa87b96740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801b02f0 con 0x56393a6351a0 2024-03-09T00:10:17.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa87b96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801af630 con 0x56393a63e6a0 2024-03-09T00:10:17.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa87b96740 10 client.0 _ll_register_callbacks cb 0x7faa70005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:17.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a643cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a643cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46662/0 (socket says 172.21.15.22:46662) 2024-03-09T00:10:17.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3606655307 learned_addr learned my addr 172.21.15.22:0/3606655307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:17.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7e59c640 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a645930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7ed9d640 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a644210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 msgr2=0x56393a644210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a644210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3606655307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 msgr2=0x56393a645930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:17.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a645930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 -- 172.21.15.22:0/3606655307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56393a644950 con 0x56393a635ac0 2024-03-09T00:10:17.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa7dd9b640 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a643cd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7faa74005c40 tx=0x7faa7402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e519249196fec3b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:17.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:17.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa65ffb640 1 -- 172.21.15.22:0/3606655307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74043070 con 0x56393a635ac0 2024-03-09T00:10:17.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa65ffb640 1 -- 172.21.15.22:0/3606655307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7faa74004840 con 0x56393a635ac0 2024-03-09T00:10:17.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.158+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa74050be0 con 0x56393a635ac0 2024-03-09T00:10:17.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa65ffb640 1 -- 172.21.15.22:0/3606655307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa74061050 con 0x56393a635ac0 2024-03-09T00:10:17.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 10 client.20436 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:17.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56393a6a5e60 con 0x56393a635ac0 2024-03-09T00:10:17.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa65ffb640 1 -- 172.21.15.22:0/3606655307 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa74050be0 con 0x56393a635ac0 2024-03-09T00:10:17.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa65ffb640 1 -- 172.21.15.22:0/3606655307 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa74005210 con 0x56393a635ac0 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 10 client.20436 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 10 client.20436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 20 client.20436 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a6a60f0 0x56393a6a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 4 client.20436 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56393a5a0750 con 0x56393a6a60f0 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.159+0000 7faa7e59c640 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a6a60f0 0x56393a6a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.160+0000 7faa7e59c640 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a6a60f0 0x56393a6a84e0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b2355d6ad395ab server_cookie=7725802ea1709b66 in_seq=0 out_seq=0 2024-03-09T00:10:17.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.160+0000 7faa65ffb640 10 client.20436 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa65ffb640 1 -- 172.21.15.22:0/3606655307 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56393a5a0750 con 0x56393a6a60f0 2024-03-09T00:10:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa65ffb640 10 client.20436 handle_command_reply: tid=0 2024-03-09T00:10:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 client.20436 shutdown 2024-03-09T00:10:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 msgr2=0x56393a643cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a643cd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7faa74005c40 tx=0x7faa7402c7e0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 shutdown_connections 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56393a6a60f0 0x56393a6a84e0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56393a63e6a0 0x56393a644210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56393a635ac0 0x56393a643cd0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 --2- 172.21.15.22:0/3606655307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393a6351a0 0x56393a645930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa65ffb640 0 client.20436 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.208+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 >> 172.21.15.22:0/3606655307 conn(0x56393a62f8d0 msgr2=0x56393a639f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 shutdown_connections 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa87b96740 1 -- 172.21.15.22:0/3606655307 wait complete. 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa87b96740 20 client.20436 trim_cache size 0 max 16384 2024-03-09T00:10:17.206 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:10:17.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faa680a2970 msgr2=0x7faa680a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faa680a2970 0x7faa680a4e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa70009c20 tx=0x7faa70005f90 comp rx=0 tx=0).stop 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa8010b850 msgr2=0x7faa8019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa8010b850 0x7faa8019e720 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7faa7800eb10 tx=0x7faa7800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 shutdown_connections 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7faa680a2970 0x7faa680a4e30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa8010b850 0x7faa8019e720 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80102e30 0x7faa8019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 --2- 172.21.15.22:0/4235068875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa80102510 0x7faa80199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 >> 172.21.15.22:0/4235068875 conn(0x7faa800773a0 msgr2=0x7faa80107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 shutdown_connections 2024-03-09T00:10:17.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:17.209+0000 7faa84fb9640 1 -- 172.21.15.22:0/4235068875 wait complete. 2024-03-09T00:10:17.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2749201759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:10:17.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2749201759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:10:17.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2749201759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:10:18.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 34313 9.44 MB/sec execute 25 sec latency 280.317 ms 2024-03-09T00:10:18.220 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:10:18.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:10:18.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:10:18.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 -- 172.21.15.22:0/516711474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c101ea0 msgr2=0x7fe79c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 --2- 172.21.15.22:0/516711474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c101ea0 0x7fe79c105ee0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe790009900 tx=0x7fe79002bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:18.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 -- 172.21.15.22:0/516711474 shutdown_connections 2024-03-09T00:10:18.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 --2- 172.21.15.22:0/516711474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe79c1065c0 0x7fe79c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 --2- 172.21.15.22:0/516711474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c101ea0 0x7fe79c105ee0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 --2- 172.21.15.22:0/516711474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe79c1014f0 0x7fe79c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 -- 172.21.15.22:0/516711474 >> 172.21.15.22:0/516711474 conn(0x7fe79c0fcd90 msgr2=0x7fe79c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 -- 172.21.15.22:0/516711474 shutdown_connections 2024-03-09T00:10:18.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.356+0000 7fe7a2977640 1 -- 172.21.15.22:0/516711474 wait complete. 2024-03-09T00:10:18.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.357+0000 7fe7a2977640 1 Processor -- start 2024-03-09T00:10:18.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.357+0000 7fe7a2977640 1 -- start start 2024-03-09T00:10:18.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe7a2977640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe79c1014f0 0x7fe79c115910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe7a2977640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c1065c0 0x7fe79c115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe7a2977640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe79c110a20 0x7fe79c110ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe7a2977640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10f4d0 con 0x7fe79c1065c0 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe7a2977640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10f350 con 0x7fe79c110a20 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe7a2977640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10f650 con 0x7fe79c1014f0 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c1065c0 0x7fe79c115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c1065c0 0x7fe79c115e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46674/0 (socket says 172.21.15.22:46674) 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79b7fe640 1 -- 172.21.15.22:0/2609549178 learned_addr learned my addr 172.21.15.22:0/2609549178 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79bfff640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe79c1014f0 0x7fe79c115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe79c110a20 0x7fe79c110ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79bfff640 1 -- 172.21.15.22:0/2609549178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe79c110a20 msgr2=0x7fe79c110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79bfff640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe79c110a20 0x7fe79c110ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79bfff640 1 -- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c1065c0 msgr2=0x7fe79c115e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79bfff640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c1065c0 0x7fe79c115e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.358+0000 7fe79bfff640 1 -- 172.21.15.22:0/2609549178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c111410 con 0x7fe79c1014f0 2024-03-09T00:10:18.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.359+0000 7fe79bfff640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe79c1014f0 0x7fe79c115910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c970 tx=0x7fe78c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.359+0000 7fe7997fa640 1 -- 172.21.15.22:0/2609549178 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c007b50 con 0x7fe79c1014f0 2024-03-09T00:10:18.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.359+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c111700 con 0x7fe79c1014f0 2024-03-09T00:10:18.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.359+0000 7fe7997fa640 1 -- 172.21.15.22:0/2609549178 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe78c007cf0 con 0x7fe79c1014f0 2024-03-09T00:10:18.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.359+0000 7fe7997fa640 1 -- 172.21.15.22:0/2609549178 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c018820 con 0x7fe79c1014f0 2024-03-09T00:10:18.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.359+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c1b3870 con 0x7fe79c1014f0 2024-03-09T00:10:18.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 Processor -- start 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 -- start start 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f84160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210f8cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f8d280 0x55c210f8f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210ee7b40 con 0x55c210f8d280 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210e90d80 con 0x55c210f83d80 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.362+0000 7fe7a5632740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210ee3be0 con 0x55c210f846a0 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f8d280 0x55c210f8f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f8d280 0x55c210f8f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46680/0 (socket says 172.21.15.22:46680) 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79b7fe640 1 -- 172.21.15.22:0/3830982887 learned_addr learned my addr 172.21.15.22:0/3830982887 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f84160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79bfff640 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210f8cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe7997fa640 1 -- 172.21.15.22:0/2609549178 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe78c0189c0 con 0x7fe79c1014f0 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79bfff640 1 -- 172.21.15.22:0/3830982887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 msgr2=0x55c210f84160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79bfff640 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f84160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79bfff640 1 -- 172.21.15.22:0/3830982887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f8d280 msgr2=0x55c210f8f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79bfff640 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f8d280 0x55c210f8f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79bfff640 1 -- 172.21.15.22:0/3830982887 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c210cc47e0 con 0x55c210f846a0 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.363+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f8d280 0x55c210f8f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:18.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.364+0000 7fe79bfff640 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210f8cd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0cc5d0 tx=0x7fe78c0d7540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d72b87f77945c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.364+0000 7fe7827fc640 1 -- 172.21.15.22:0/3830982887 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0e2070 con 0x55c210f846a0 2024-03-09T00:10:18.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.364+0000 7fe7827fc640 1 -- 172.21.15.22:0/3830982887 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe78c0cfc40 con 0x55c210f846a0 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.364+0000 7fe7827fc640 1 -- 172.21.15.22:0/3830982887 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0dd7e0 con 0x55c210f846a0 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.364+0000 7fe7997fa640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe7840a2a30 0x7fe7840a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.364+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe7840a2a30 0x7fe7840a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7997fa640 1 -- 172.21.15.22:0/2609549178 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe78c0cf330 con 0x7fe79c1014f0 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 -- 172.21.15.22:0/3830982887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 msgr2=0x55c210f8cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210f8cd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0cc5d0 tx=0x7fe78c0d7540 comp rx=0 tx=0).stop 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 -- 172.21.15.22:0/3830982887 shutdown_connections 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f8d280 0x55c210f8f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210f8cd40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3830982887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f84160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 -- 172.21.15.22:0/3830982887 >> 172.21.15.22:0/3830982887 conn(0x55c210f7e4b0 msgr2=0x55c210f7e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 -- 172.21.15.22:0/3830982887 shutdown_connections 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe7840a2a30 0x7fe7840a4ef0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe794005fd0 tx=0x7fe794009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 -- 172.21.15.22:0/3830982887 wait complete. 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.365+0000 7fe7a5632740 1 -- start start 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 1 Processor -- start 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f8ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210f8a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210f8a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210f90070 con 0x55c210f846a0 2024-03-09T00:10:18.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210ea41c0 con 0x55c210f83d80 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210f901f0 con 0x55c210f8d280 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f8ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f8ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:36388/0 (socket says 172.21.15.22:36388) 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe79bfff640 1 -- 172.21.15.22:0/737950604 learned_addr learned my addr 172.21.15.22:0/737950604 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210f8a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210f8a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.366+0000 7fe7a5632740 10 client.0 _ll_register_callbacks cb 0x55c210e937a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe79bfff640 1 -- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 msgr2=0x55c210f8a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe79bfff640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210f8a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe79bfff640 1 -- 172.21.15.22:0/737950604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 msgr2=0x55c210f8a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe79bfff640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210f8a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe79bfff640 1 -- 172.21.15.22:0/737950604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c210d24f70 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210f8a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe77b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe79bfff640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f8ccc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe78c097eb0 tx=0x7fe78c0d7f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5942a9b5ea1e92c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe77b7fe640 1 -- 172.21.15.22:0/737950604 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c097780 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.367+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe78c0cfc40 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe77b7fe640 1 -- 172.21.15.22:0/737950604 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe78c0997e0 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe77b7fe640 1 -- 172.21.15.22:0/737950604 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe78c097780 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe7a5632740 10 client.29660 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c210bbc170 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe77b7fe640 1 -- 172.21.15.22:0/737950604 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe78c0cfc40 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe77b7fe640 1 -- 172.21.15.22:0/737950604 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe78c097b50 con 0x55c210f83d80 2024-03-09T00:10:18.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe7a5632740 10 client.29660 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe7a5632740 10 client.29660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.368+0000 7fe7a5632740 20 client.29660 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.369+0000 7fe7a5632740 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210f9f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.369+0000 7fe7a5632740 4 client.29660 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.369+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c210ebd960 con 0x55c210f9d200 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.369+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210f9f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.369+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210f9f5f0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7adae05f8417805e server_cookie=cdf8ae294b04f843 in_seq=0 out_seq=0 2024-03-09T00:10:18.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.369+0000 7fe77b7fe640 10 client.29660 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.373+0000 7fe77b7fe640 1 -- 172.21.15.22:0/737950604 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c210ebd960 con 0x55c210f9d200 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.373+0000 7fe77b7fe640 10 client.29660 handle_command_reply: tid=0 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.373+0000 7fe7a5632740 1 client.29660 shutdown 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 msgr2=0x55c210f8ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f8ccc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe78c097eb0 tx=0x7fe78c0d7f50 comp rx=0 tx=0).stop 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 shutdown_connections 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210f9f5f0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe77b7fe640 0 client.29660 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210f8a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210f8a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 --2- 172.21.15.22:0/737950604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f8ccc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 >> 172.21.15.22:0/737950604 conn(0x55c210f7e4b0 msgr2=0x55c210f80200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 shutdown_connections 2024-03-09T00:10:18.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 1 -- 172.21.15.22:0/737950604 wait complete. 2024-03-09T00:10:18.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.374+0000 7fe7a5632740 20 client.29660 trim_cache size 0 max 16384 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.399+0000 7fe7a5632740 1 Processor -- start 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.399+0000 7fe7a5632740 1 -- start start 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f98b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210fa1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210fa1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a5632740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1bfb20 con 0x55c210f846a0 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a5632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1bd6c0 con 0x55c210f83d80 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a5632740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1bb780 con 0x55c210f8d280 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210fa1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210fa1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:35866/0 (socket says 172.21.15.22:35866) 2024-03-09T00:10:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 -- 172.21.15.22:0/785565515 learned_addr learned my addr 172.21.15.22:0/785565515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 -- 172.21.15.22:0/785565515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 msgr2=0x55c210f98b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/785565515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f98b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 -- 172.21.15.22:0/785565515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 msgr2=0x55c210fa1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/785565515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210fa1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.400+0000 7fe7a0f14640 1 -- 172.21.15.22:0/785565515 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c210f8ab00 con 0x55c210f8d280 2024-03-09T00:10:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.401+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/785565515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210fa1b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe794003ee0 tx=0x7fe79401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4232fc5439dd36c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.402+0000 7fe781ffb640 1 -- 172.21.15.22:0/785565515 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794004b20 con 0x55c210f8d280 2024-03-09T00:10:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.402+0000 7fe781ffb640 1 -- 172.21.15.22:0/785565515 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe794004cc0 con 0x55c210f8d280 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.402+0000 7fe781ffb640 1 -- 172.21.15.22:0/785565515 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79400f870 con 0x55c210f8d280 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 -- 172.21.15.22:0/785565515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 msgr2=0x55c210fa1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 --2- 172.21.15.22:0/785565515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210fa1b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe794003ee0 tx=0x7fe79401a000 comp rx=0 tx=0).stop 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 -- 172.21.15.22:0/785565515 shutdown_connections 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 --2- 172.21.15.22:0/785565515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f8d280 0x55c210fa1b50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 --2- 172.21.15.22:0/785565515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f846a0 0x55c210fa1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 --2- 172.21.15.22:0/785565515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f83d80 0x55c210f98b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 -- 172.21.15.22:0/785565515 >> 172.21.15.22:0/785565515 conn(0x55c210f7e4b0 msgr2=0x55c210f98440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 -- 172.21.15.22:0/785565515 shutdown_connections 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 -- 172.21.15.22:0/785565515 wait complete. 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 -- start start 2024-03-09T00:10:18.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.403+0000 7fe7a5632740 1 Processor -- start 2024-03-09T00:10:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f83d80 0x55c210fa0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210fa1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a5632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f8d280 0x55c210fec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a5632740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1bbd80 con 0x55c210f83d80 2024-03-09T00:10:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a5632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1bd060 con 0x55c210f8d280 2024-03-09T00:10:18.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a5632740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1bc3a0 con 0x55c210f846a0 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a5632740 10 client.0 _ll_register_callbacks cb 0x7fe78c0d2c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f83d80 0x55c210fa0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f83d80 0x55c210fa0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46716/0 (socket says 172.21.15.22:46716) 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 -- 172.21.15.22:0/3809800614 learned_addr learned my addr 172.21.15.22:0/3809800614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210fa1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe7a0f14640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f8d280 0x55c210fec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 -- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 msgr2=0x55c210fa1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210fa1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 -- 172.21.15.22:0/3809800614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f8d280 msgr2=0x55c210fec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f8d280 0x55c210fec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79bfff640 1 -- 172.21.15.22:0/3809800614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c210f84e50 con 0x55c210f83d80 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.404+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210fa1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:18.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe79bfff640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f83d80 0x55c210fa0d30 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0a3750 tx=0x7fe78c00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e76a4902481f495 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe77b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe77b7fe640 1 -- 172.21.15.22:0/3809800614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c018cb0 con 0x55c210f83d80 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe77b7fe640 1 -- 172.21.15.22:0/3809800614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe78c09edc0 con 0x55c210f83d80 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe794004cc0 con 0x55c210f83d80 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe77b7fe640 1 -- 172.21.15.22:0/3809800614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe78c09ae40 con 0x55c210f83d80 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe7a5632740 10 client.20451 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c210ff4f60 con 0x55c210f83d80 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.405+0000 7fe77b7fe640 1 -- 172.21.15.22:0/3809800614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe794004cc0 con 0x55c210f83d80 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe77b7fe640 1 -- 172.21.15.22:0/3809800614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe78c0b4140 con 0x55c210f83d80 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe7a5632740 10 client.20451 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe7a5632740 10 client.20451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe7a5632740 20 client.20451 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210ff74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe7a5632740 4 client.20451 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:10:18.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c210ea9080 con 0x55c210f9d200 2024-03-09T00:10:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210ff74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe79b7fe640 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210ff74b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734535e13c95b62b server_cookie=bbb275b513d4b4b0 in_seq=0 out_seq=0 2024-03-09T00:10:18.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.406+0000 7fe77b7fe640 10 client.20451 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe77b7fe640 1 -- 172.21.15.22:0/3809800614 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c210ea9080 con 0x55c210f9d200 2024-03-09T00:10:18.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe77b7fe640 10 client.20451 handle_command_reply: tid=0 2024-03-09T00:10:18.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 client.20451 shutdown 2024-03-09T00:10:18.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f83d80 msgr2=0x55c210fa0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f83d80 0x55c210fa0d30 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0a3750 tx=0x7fe78c00fd60 comp rx=0 tx=0).stop 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 shutdown_connections 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c210f9d200 0x55c210ff74b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c210f8d280 0x55c210fec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c210f846a0 0x55c210fa1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 --2- 172.21.15.22:0/3809800614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c210f83d80 0x55c210fa0d30 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 >> 172.21.15.22:0/3809800614 conn(0x55c210f7e4b0 msgr2=0x55c210f98cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.410+0000 7fe77b7fe640 0 client.20451 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.411+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 shutdown_connections 2024-03-09T00:10:18.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.411+0000 7fe7a5632740 1 -- 172.21.15.22:0/3809800614 wait complete. 2024-03-09T00:10:18.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.411+0000 7fe7a5632740 20 client.20451 trim_cache size 0 max 16384 2024-03-09T00:10:18.408 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:10:18.408 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:10:18.408 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "66b4120b-310d-46ea-92fe-debf681ae262", 2024-03-09T00:10:18.408 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:10:18.408 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:10:18.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.411+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe7840a2a30 msgr2=0x7fe7840a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.411+0000 7fe7a2977640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe7840a2a30 0x7fe7840a4ef0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe794005fd0 tx=0x7fe794009500 comp rx=0 tx=0).stop 2024-03-09T00:10:18.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.411+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe79c1014f0 msgr2=0x7fe79c115910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.411+0000 7fe7a2977640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe79c1014f0 0x7fe79c115910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c970 tx=0x7fe78c00ce30 comp rx=0 tx=0).stop 2024-03-09T00:10:18.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 shutdown_connections 2024-03-09T00:10:18.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe7840a2a30 0x7fe7840a4ef0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe79c110a20 0x7fe79c110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe79c1065c0 0x7fe79c115e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 --2- 172.21.15.22:0/2609549178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe79c1014f0 0x7fe79c115910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 >> 172.21.15.22:0/2609549178 conn(0x7fe79c0fcd90 msgr2=0x7fe79c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 shutdown_connections 2024-03-09T00:10:18.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.412+0000 7fe7a2977640 1 -- 172.21.15.22:0/2609549178 wait complete. 2024-03-09T00:10:18.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:10:18.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:10:18.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 -- 172.21.15.22:0/3126452941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538101e80 msgr2=0x7f1538105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3126452941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538101e80 0x7f1538105ec0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f1520009900 tx=0x7f152002bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:18.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 -- 172.21.15.22:0/3126452941 shutdown_connections 2024-03-09T00:10:18.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3126452941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15381065a0 0x7f153810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3126452941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538101e80 0x7f1538105ec0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3126452941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15381014d0 0x7f15381018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 -- 172.21.15.22:0/3126452941 >> 172.21.15.22:0/3126452941 conn(0x7f15380fcd50 msgr2=0x7f15380ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 -- 172.21.15.22:0/3126452941 shutdown_connections 2024-03-09T00:10:18.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.535+0000 7f153cbc4640 1 -- 172.21.15.22:0/3126452941 wait complete. 2024-03-09T00:10:18.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 Processor -- start 2024-03-09T00:10:18.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 -- start start 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15381014d0 0x7f153819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15381065a0 0x7f153819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538197af0 0x7f1538197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1538111630 con 0x7f1538197af0 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381114b0 con 0x7f15381065a0 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.536+0000 7f153cbc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381117b0 con 0x7f15381014d0 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538197af0 0x7f1538197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538197af0 0x7f1538197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46724/0 (socket says 172.21.15.22:46724) 2024-03-09T00:10:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 -- 172.21.15.22:0/3173625166 learned_addr learned my addr 172.21.15.22:0/3173625166 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f153659c640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15381014d0 0x7f153819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f152ffff640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15381065a0 0x7f153819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 -- 172.21.15.22:0/3173625166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15381014d0 msgr2=0x7f153819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15381014d0 0x7f153819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 -- 172.21.15.22:0/3173625166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15381065a0 msgr2=0x7f153819de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15381065a0 0x7f153819de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 -- 172.21.15.22:0/3173625166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538198610 con 0x7f1538197af0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f1536d9d640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538197af0 0x7f1538197fa0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f153000eaa0 tx=0x7f153000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f152f7fe640 1 -- 172.21.15.22:0/3173625166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000cdb0 con 0x7f1538197af0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f152f7fe640 1 -- 172.21.15.22:0/3173625166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1530005020 con 0x7f1538197af0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f152f7fe640 1 -- 172.21.15.22:0/3173625166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15300116b0 con 0x7f1538197af0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1538198900 con 0x7f1538197af0 2024-03-09T00:10:18.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.537+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15381a4ab0 con 0x7f1538197af0 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.539+0000 7f152f7fe640 1 -- 172.21.15.22:0/3173625166 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1530005560 con 0x7f1538197af0 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 Processor -- start 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 -- start start 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f152f7fe640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15240a2970 0x7f15240a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622450 0x55cb36622830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622d70 0x55cb3662fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb36632460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f152f7fe640 1 -- 172.21.15.22:0/3173625166 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f15300cf420 con 0x7f1538197af0 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36585e00 con 0x55cb36622450 2024-03-09T00:10:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb3652eb10 con 0x55cb36630070 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153f7a7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36581ea0 con 0x55cb36622d70 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f152ffff640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15240a2970 0x7f15240a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622450 0x55cb36622830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622450 0x55cb36622830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46726/0 (socket says 172.21.15.22:46726) 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f152ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622d70 0x55cb3662fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f153659c640 1 -- 172.21.15.22:0/1789382580 learned_addr learned my addr 172.21.15.22:0/1789382580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.540+0000 7f1536d9d640 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb36632460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f153659c640 1 -- 172.21.15.22:0/1789382580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622d70 msgr2=0x55cb3662fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f153659c640 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622d70 0x55cb3662fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f153659c640 1 -- 172.21.15.22:0/1789382580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 msgr2=0x55cb36632460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f153659c640 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb36632460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f153659c640 1 -- 172.21.15.22:0/1789382580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb363647d0 con 0x55cb36622450 2024-03-09T00:10:18.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f1536d9d640 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb36632460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f152ffff640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15240a2970 0x7f15240a4e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1520009230 tx=0x7f152002c5d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f153659c640 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622450 0x55cb36622830 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1528009ca0 tx=0x7f1528011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc65971a71dacb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f152cff9640 1 -- 172.21.15.22:0/1789382580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528009e50 con 0x55cb36622450 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f152cff9640 1 -- 172.21.15.22:0/1789382580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1528005500 con 0x55cb36622450 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.541+0000 7f152cff9640 1 -- 172.21.15.22:0/1789382580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528019610 con 0x55cb36622450 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 -- 172.21.15.22:0/1789382580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622450 msgr2=0x55cb36622830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622450 0x55cb36622830 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1528009ca0 tx=0x7f1528011810 comp rx=0 tx=0).stop 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 -- 172.21.15.22:0/1789382580 shutdown_connections 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb36632460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622d70 0x55cb3662fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1789382580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622450 0x55cb36622830 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 -- 172.21.15.22:0/1789382580 >> 172.21.15.22:0/1789382580 conn(0x55cb3661ce90 msgr2=0x55cb3661d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 -- 172.21.15.22:0/1789382580 shutdown_connections 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 -- 172.21.15.22:0/1789382580 wait complete. 2024-03-09T00:10:18.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 -- start start 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.542+0000 7f153f7a7740 1 Processor -- start 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622d70 0x55cb3662cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb3662d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36632e60 con 0x55cb36622d70 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb365420a0 con 0x55cb36630070 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb36632fe0 con 0x55cb36622450 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 10 client.0 _ll_register_callbacks cb 0x55cb36541a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622d70 0x55cb3662cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622d70 0x55cb3662cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46738/0 (socket says 172.21.15.22:46738) 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 -- 172.21.15.22:0/1340596422 learned_addr learned my addr 172.21.15.22:0/1340596422 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f1536d9d640 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb3662d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 -- 172.21.15.22:0/1340596422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 msgr2=0x55cb3662f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 -- 172.21.15.22:0/1340596422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 msgr2=0x55cb3662d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb3662d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 -- 172.21.15.22:0/1340596422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb3629a3f0 con 0x55cb36622d70 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f152ffff640 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622d70 0x55cb3662cac0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f152002cfd0 tx=0x7f152002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d4061e626bba69 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f151dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f151dffb640 1 -- 172.21.15.22:0/1340596422 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520053060 con 0x55cb36622d70 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f151dffb640 1 -- 172.21.15.22:0/1340596422 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f1520004ec0 con 0x55cb36622d70 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.543+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1528005500 con 0x55cb36622d70 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f151dffb640 1 -- 172.21.15.22:0/1340596422 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f152005a050 con 0x55cb36622d70 2024-03-09T00:10:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 10 client.20463 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cb3625a0f0 con 0x55cb36622d70 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f151dffb640 1 -- 172.21.15.22:0/1340596422 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1528005500 con 0x55cb36622d70 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f151dffb640 1 -- 172.21.15.22:0/1340596422 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1520003bc0 con 0x55cb36622d70 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 10 client.20463 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 10 client.20463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 20 client.20463 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb3663fac0 0x55cb36641eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 4 client.20463 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb36593340 con 0x55cb3663fac0 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153659c640 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb3663fac0 0x55cb36641eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f153659c640 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb3663fac0 0x55cb36641eb0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef63d3ab486bbb8 server_cookie=154898a64d19ab1d in_seq=0 out_seq=0 2024-03-09T00:10:18.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.544+0000 7f151dffb640 10 client.20463 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.548+0000 7f151dffb640 1 -- 172.21.15.22:0/1340596422 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb36593340 con 0x55cb3663fac0 2024-03-09T00:10:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.548+0000 7f151dffb640 10 client.20463 handle_command_reply: tid=0 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 client.20463 shutdown 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622d70 msgr2=0x55cb3662cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622d70 0x55cb3662cac0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f152002cfd0 tx=0x7f152002bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 shutdown_connections 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb3663fac0 0x55cb36641eb0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36630070 0x55cb3662d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36622d70 0x55cb3662cac0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 --2- 172.21.15.22:0/1340596422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f151dffb640 0 client.20463 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 >> 172.21.15.22:0/1340596422 conn(0x55cb3661ce90 msgr2=0x55cb3661ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 shutdown_connections 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 1 -- 172.21.15.22:0/1340596422 wait complete. 2024-03-09T00:10:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.549+0000 7f153f7a7740 20 client.20463 trim_cache size 0 max 16384 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 Processor -- start 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 -- start start 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb3662f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb3662ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381aea10 con 0x55cb36630070 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381ac5b0 con 0x55cb36622d70 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153f7a7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381aa670 con 0x55cb36622450 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb3662ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f152ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb3662f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb3662ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46748/0 (socket says 172.21.15.22:46748) 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f1536d9d640 1 -- 172.21.15.22:0/192374759 learned_addr learned my addr 172.21.15.22:0/192374759 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.565+0000 7f153659c640 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f1536d9d640 1 -- 172.21.15.22:0/192374759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 msgr2=0x55cb3662f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f1536d9d640 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f1536d9d640 1 -- 172.21.15.22:0/192374759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 msgr2=0x55cb3662f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f1536d9d640 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb3662f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f1536d9d640 1 -- 172.21.15.22:0/192374759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb3662b010 con 0x55cb36630070 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f152ffff640 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb3662f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f1536d9d640 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb3662ca80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f153000efd0 tx=0x7f1530086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec3b71dab49c77b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f151ffff640 1 -- 172.21.15.22:0/192374759 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153001f040 con 0x55cb36630070 2024-03-09T00:10:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f151ffff640 1 -- 172.21.15.22:0/192374759 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15300051d0 con 0x55cb36630070 2024-03-09T00:10:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.566+0000 7f151ffff640 1 -- 172.21.15.22:0/192374759 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15300b4270 con 0x55cb36630070 2024-03-09T00:10:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- 172.21.15.22:0/192374759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 msgr2=0x55cb3662ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb3662ca80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f153000efd0 tx=0x7f1530086040 comp rx=0 tx=0).stop 2024-03-09T00:10:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- 172.21.15.22:0/192374759 shutdown_connections 2024-03-09T00:10:18.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb3662ca80 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb3662f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 --2- 172.21.15.22:0/192374759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb3662f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- 172.21.15.22:0/192374759 >> 172.21.15.22:0/192374759 conn(0x55cb3661ce90 msgr2=0x55cb3661d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- 172.21.15.22:0/192374759 shutdown_connections 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- 172.21.15.22:0/192374759 wait complete. 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- start start 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 Processor -- start 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb36637520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb366338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb36633e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381aac70 con 0x55cb36630070 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381abf50 con 0x55cb36622d70 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381ab290 con 0x55cb36622450 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f153f7a7740 10 client.0 _ll_register_callbacks cb 0x7f1520005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.567+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb36633e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb36633e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46756/0 (socket says 172.21.15.22:46756) 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 -- 172.21.15.22:0/2625846242 learned_addr learned my addr 172.21.15.22:0/2625846242 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f153659c640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb36637520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f152ffff640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb366338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 -- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 msgr2=0x55cb36637520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb36637520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 -- 172.21.15.22:0/2625846242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 msgr2=0x55cb366338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb366338d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 -- 172.21.15.22:0/2625846242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb36634610 con 0x55cb36630070 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f152ffff640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb366338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f1536d9d640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb36633e10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f15300cc7d0 tx=0x7f1530099bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03b978c2fc9d88d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f151dffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f151dffb640 1 -- 172.21.15.22:0/2625846242 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153007f060 con 0x55cb36630070 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f151dffb640 1 -- 172.21.15.22:0/2625846242 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f15300b4450 con 0x55cb36630070 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15300051d0 con 0x55cb36630070 2024-03-09T00:10:18.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f151dffb640 1 -- 172.21.15.22:0/2625846242 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f153001f040 con 0x55cb36630070 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f153f7a7740 10 client.20469 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cb36696eb0 con 0x55cb36630070 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.568+0000 7f151dffb640 1 -- 172.21.15.22:0/2625846242 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15300051d0 con 0x55cb36630070 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f151dffb640 1 -- 172.21.15.22:0/2625846242 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f153000f040 con 0x55cb36630070 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153f7a7740 10 client.20469 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153f7a7740 10 client.20469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153f7a7740 20 client.20469 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153f7a7740 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb36697560 0x55cb36699980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153f7a7740 4 client.20469 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:10:18.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cb3658d690 con 0x55cb36697560 2024-03-09T00:10:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153659c640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb36697560 0x55cb36699980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f153659c640 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb36697560 0x55cb36699980 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1c36252fdd463d server_cookie=208f9693638f6430 in_seq=0 out_seq=0 2024-03-09T00:10:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.569+0000 7f151dffb640 10 client.20469 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f151dffb640 1 -- 172.21.15.22:0/2625846242 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55cb3658d690 con 0x55cb36697560 2024-03-09T00:10:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f151dffb640 10 client.20469 handle_command_reply: tid=0 2024-03-09T00:10:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 client.20469 shutdown 2024-03-09T00:10:18.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 msgr2=0x55cb36633e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb36633e10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f15300cc7d0 tx=0x7f1530099bf0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 shutdown_connections 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55cb36697560 0x55cb36699980 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cb36630070 0x55cb36633e10 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb36622d70 0x55cb366338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f151dffb640 0 client.20469 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 --2- 172.21.15.22:0/2625846242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb36622450 0x55cb36637520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 >> 172.21.15.22:0/2625846242 conn(0x55cb3661ce90 msgr2=0x55cb3662ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 shutdown_connections 2024-03-09T00:10:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 1 -- 172.21.15.22:0/2625846242 wait complete. 2024-03-09T00:10:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153f7a7740 20 client.20469 trim_cache size 0 max 16384 2024-03-09T00:10:18.579 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:10:18.579 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (407 inodes in the stack)", 2024-03-09T00:10:18.579 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:10:18.579 INFO:teuthology.orchestra.run.smithi022.stdout: "66b4120b-310d-46ea-92fe-debf681ae262": { 2024-03-09T00:10:18.580 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:10:18.580 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "66b4120b-310d-46ea-92fe-debf681ae262", 2024-03-09T00:10:18.580 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:10:18.580 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:10:18.580 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:10:18.580 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:10:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15240a2970 msgr2=0x7f15240a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15240a2970 0x7f15240a4e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1520009230 tx=0x7f152002c5d0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538197af0 msgr2=0x7f1538197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:18.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538197af0 0x7f1538197fa0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f153000eaa0 tx=0x7f153000ef60 comp rx=0 tx=0).stop 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 shutdown_connections 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f15240a2970 0x7f15240a4e30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1538197af0 0x7f1538197fa0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15381065a0 0x7f153819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.574+0000 7f153cbc4640 1 --2- 172.21.15.22:0/3173625166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15381014d0 0x7f153819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.575+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 >> 172.21.15.22:0/3173625166 conn(0x7f15380fcd50 msgr2=0x7f15380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.575+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 shutdown_connections 2024-03-09T00:10:18.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:18.575+0000 7f153cbc4640 1 -- 172.21.15.22:0/3173625166 wait complete. 2024-03-09T00:10:18.583 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:18 smithi022 ceph-mon[31894]: pgmap v499: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-03-09T00:10:18.584 INFO:tasks.cephfs.filesystem:scrub status for tag:66b4120b-310d-46ea-92fe-debf681ae262 - {'path': '/', 'tag': '66b4120b-310d-46ea-92fe-debf681ae262', 'options': 'recursive,force'} 2024-03-09T00:10:18.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:18 smithi062 ceph-mon[36129]: pgmap v499: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-03-09T00:10:18.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:18 smithi192 ceph-mon[35995]: pgmap v499: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-03-09T00:10:19.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 34446 9.39 MB/sec execute 26 sec latency 337.874 ms 2024-03-09T00:10:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:19 smithi062 ceph-mon[36129]: osdmap e119: 12 total, 12 up, 12 in 2024-03-09T00:10:19.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:10:19.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:19 smithi192 ceph-mon[35995]: osdmap e119: 12 total, 12 up, 12 in 2024-03-09T00:10:19.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:10:19.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:19.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:19 smithi022 ceph-mon[31894]: osdmap e119: 12 total, 12 up, 12 in 2024-03-09T00:10:20.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 34611 9.23 MB/sec execute 27 sec latency 223.118 ms 2024-03-09T00:10:20.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:20 smithi062 ceph-mon[36129]: pgmap v501: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-03-09T00:10:20.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:20 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:20.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:20 smithi192 ceph-mon[35995]: pgmap v501: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-03-09T00:10:20.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:20 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:20.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:20 smithi022 ceph-mon[31894]: pgmap v501: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 332 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 449 op/s 2024-03-09T00:10:20.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:20 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:21.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 34870 9.39 MB/sec execute 28 sec latency 114.474 ms 2024-03-09T00:10:22.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 35185 9.38 MB/sec execute 29 sec latency 82.472 ms 2024-03-09T00:10:22.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:22 smithi062 ceph-mon[36129]: pgmap v502: 129 pgs: 129 active+clean; 340 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 573 op/s 2024-03-09T00:10:22.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:22 smithi192 ceph-mon[35995]: pgmap v502: 129 pgs: 129 active+clean; 340 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 573 op/s 2024-03-09T00:10:22.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:22 smithi022 ceph-mon[31894]: pgmap v502: 129 pgs: 129 active+clean; 340 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 573 op/s 2024-03-09T00:10:23.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 35515 9.73 MB/sec execute 30 sec latency 191.781 ms 2024-03-09T00:10:24.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 35687 9.70 MB/sec execute 31 sec latency 373.184 ms 2024-03-09T00:10:24.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:24 smithi062 ceph-mon[36129]: pgmap v503: 129 pgs: 129 active+clean; 340 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 573 op/s 2024-03-09T00:10:24.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:24 smithi062 ceph-mon[36129]: osdmap e120: 12 total, 12 up, 12 in 2024-03-09T00:10:24.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:24 smithi192 ceph-mon[35995]: pgmap v503: 129 pgs: 129 active+clean; 340 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 573 op/s 2024-03-09T00:10:24.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:24 smithi192 ceph-mon[35995]: osdmap e120: 12 total, 12 up, 12 in 2024-03-09T00:10:24.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:24 smithi022 ceph-mon[31894]: pgmap v503: 129 pgs: 129 active+clean; 340 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 573 op/s 2024-03-09T00:10:24.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:24 smithi022 ceph-mon[31894]: osdmap e120: 12 total, 12 up, 12 in 2024-03-09T00:10:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 -- 172.21.15.22:0/2834331314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680107400 msgr2=0x7f56801077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 --2- 172.21.15.22:0/2834331314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680107400 0x7f56801077e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f567c009900 tx=0x7f567c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 -- 172.21.15.22:0/2834331314 shutdown_connections 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 --2- 172.21.15.22:0/2834331314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680069330 0x7f568010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 --2- 172.21.15.22:0/2834331314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5680068970 0x7f5680068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 --2- 172.21.15.22:0/2834331314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680107400 0x7f56801077e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 -- 172.21.15.22:0/2834331314 >> 172.21.15.22:0/2834331314 conn(0x7f5680077040 msgr2=0x7f5680077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 -- 172.21.15.22:0/2834331314 shutdown_connections 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.028+0000 7f5686be6640 1 -- 172.21.15.22:0/2834331314 wait complete. 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 Processor -- start 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 -- start start 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680068970 0x7f568019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680069330 0x7f568019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5680107400 0x7f56801978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5680111660 con 0x7f5680069330 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801114e0 con 0x7f5680068970 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5686be6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801117e0 con 0x7f5680107400 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5684982640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680068970 0x7f568019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5677fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680069330 0x7f568019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5684982640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680068970 0x7f568019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51554/0 (socket says 172.21.15.22:51554) 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5684982640 1 -- 172.21.15.22:0/2547261194 learned_addr learned my addr 172.21.15.22:0/2547261194 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5685183640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5680107400 0x7f56801978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5677fff640 1 -- 172.21.15.22:0/2547261194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5680107400 msgr2=0x7f56801978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5677fff640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5680107400 0x7f56801978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5677fff640 1 -- 172.21.15.22:0/2547261194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680068970 msgr2=0x7f568019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5677fff640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680068970 0x7f568019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5677fff640 1 -- 172.21.15.22:0/2547261194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680198020 con 0x7f5680069330 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.029+0000 7f5684982640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680068970 0x7f568019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.030+0000 7f5677fff640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680069330 0x7f568019dc30 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f567000e9f0 tx=0x7f567000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.030+0000 7f5675ffb640 1 -- 172.21.15.22:0/2547261194 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000ccc0 con 0x7f5680069330 2024-03-09T00:10:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.030+0000 7f5675ffb640 1 -- 172.21.15.22:0/2547261194 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5670005020 con 0x7f5680069330 2024-03-09T00:10:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.030+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5680198310 con 0x7f5680069330 2024-03-09T00:10:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.030+0000 7f5675ffb640 1 -- 172.21.15.22:0/2547261194 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700049e0 con 0x7f5680069330 2024-03-09T00:10:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.030+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56801a45b0 con 0x7f5680069330 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.032+0000 7f56897c6740 1 Processor -- start 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f56897c6740 1 -- start start 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f56897c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f56897c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f56897c6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 0x55f4739f4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f56897c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47394d5f0 con 0x55f4739e8fd0 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f56897c6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4738f6930 con 0x55f4739f2550 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f56897c6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f473949690 con 0x55f4739e9d40 2024-03-09T00:10:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5684982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5685183640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5684982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739e9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47856/0 (socket says 172.21.15.22:47856) 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5684982640 1 -- 172.21.15.22:0/2650672837 learned_addr learned my addr 172.21.15.22:0/2650672837 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5675ffb640 1 -- 172.21.15.22:0/2547261194 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5670005560 con 0x7f5680069330 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5685183640 1 -- 172.21.15.22:0/2650672837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 msgr2=0x55f4739f4940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5685183640 1 --2- 172.21.15.22:0/2650672837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 0x55f4739f4940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5685183640 1 -- 172.21.15.22:0/2650672837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 msgr2=0x55f4739e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5685183640 1 --2- 172.21.15.22:0/2650672837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.033+0000 7f5685183640 1 -- 172.21.15.22:0/2650672837 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f473662210 con 0x55f4739e9d40 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.034+0000 7f5685183640 1 --2- 172.21.15.22:0/2650672837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739e8a90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5678009ef0 tx=0x7f5678011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff2af8aebfb419a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.034+0000 7f5666ffd640 1 -- 172.21.15.22:0/2650672837 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678004d20 con 0x55f4739e9d40 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.034+0000 7f5675ffb640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f56680a2a90 0x7f56680a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.034+0000 7f5666ffd640 1 -- 172.21.15.22:0/2650672837 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f567801a070 con 0x55f4739e9d40 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.034+0000 7f5666ffd640 1 -- 172.21.15.22:0/2650672837 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678016510 con 0x55f4739e9d40 2024-03-09T00:10:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.034+0000 7f5684982640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f56680a2a90 0x7f56680a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.034+0000 7f5675ffb640 1 -- 172.21.15.22:0/2547261194 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f56700cf080 con 0x7f5680069330 2024-03-09T00:10:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 -- 172.21.15.22:0/2650672837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 msgr2=0x55f4739e8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f5684982640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f56680a2a90 0x7f56680a4f50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f567c0096c0 tx=0x7f567c02e040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 --2- 172.21.15.22:0/2650672837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739e8a90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5678009ef0 tx=0x7f5678011810 comp rx=0 tx=0).stop 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 -- 172.21.15.22:0/2650672837 shutdown_connections 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 --2- 172.21.15.22:0/2650672837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 0x55f4739f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 --2- 172.21.15.22:0/2650672837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 --2- 172.21.15.22:0/2650672837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739e8a90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 -- 172.21.15.22:0/2650672837 >> 172.21.15.22:0/2650672837 conn(0x55f4739b05b0 msgr2=0x55f4739e3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 -- 172.21.15.22:0/2650672837 shutdown_connections 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.035+0000 7f56897c6740 1 -- 172.21.15.22:0/2650672837 wait complete. 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 -- start start 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 Processor -- start 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739efba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739f2550 0x55f4739f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4739f5280 con 0x55f4739e8fd0 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f473909d90 con 0x55f4739e9d40 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4739f5400 con 0x55f4739f2550 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f56897c6740 10 client.0 _ll_register_callbacks cb 0x55f473661a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f5684982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f5685183640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739f2550 0x55f4739f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f5677fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f5684982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739ef660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47870/0 (socket says 172.21.15.22:47870) 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.036+0000 7f5684982640 1 -- 172.21.15.22:0/3652714991 learned_addr learned my addr 172.21.15.22:0/3652714991 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5684982640 1 -- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739f2550 msgr2=0x55f4739f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5684982640 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739f2550 0x55f4739f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5684982640 1 -- 172.21.15.22:0/3652714991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 msgr2=0x55f4739efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5684982640 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5684982640 1 -- 172.21.15.22:0/3652714991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f47372e620 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5677fff640 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5685183640 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739f2550 0x55f4739f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f5684982640 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739ef660 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f567c005940 tx=0x7f567c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e0af0f7a87ec5b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f565ffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f565ffff640 1 -- 172.21.15.22:0/3652714991 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c051020 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f565ffff640 1 -- 172.21.15.22:0/3652714991 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f567c004b70 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f567801a070 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f565ffff640 1 -- 172.21.15.22:0/3652714991 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f567c05c050 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f56897c6740 10 client.20481 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.037+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f473a01690 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f565ffff640 1 -- 172.21.15.22:0/3652714991 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f567801a070 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f565ffff640 1 -- 172.21.15.22:0/3652714991 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f567c003ae0 con 0x55f4739e8fd0 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f56897c6740 10 client.20481 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f56897c6740 10 client.20481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f56897c6740 20 client.20481 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f56897c6740 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a02f80 0x55f473a053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f56897c6740 4 client.20481 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f47395ab30 con 0x55f473a02f80 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.038+0000 7f5677fff640 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a02f80 0x55f473a053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.039+0000 7f565ffff640 10 client.20481 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.039+0000 7f5677fff640 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a02f80 0x55f473a053a0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bebee35cd48d1 server_cookie=4f487da2033f8ec0 in_seq=0 out_seq=0 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.040+0000 7f565ffff640 1 -- 172.21.15.22:0/3652714991 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f47395ab30 con 0x55f473a02f80 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.040+0000 7f565ffff640 10 client.20481 handle_command_reply: tid=0 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.040+0000 7f56897c6740 1 client.20481 shutdown 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 msgr2=0x55f4739ef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739ef660 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f567c005940 tx=0x7f567c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 shutdown_connections 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a02f80 0x55f473a053a0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739f2550 0x55f4739f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f565ffff640 0 client.20481 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 --2- 172.21.15.22:0/3652714991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739ef660 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 >> 172.21.15.22:0/3652714991 conn(0x55f4739b05b0 msgr2=0x55f4739e5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 shutdown_connections 2024-03-09T00:10:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 1 -- 172.21.15.22:0/3652714991 wait complete. 2024-03-09T00:10:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.041+0000 7f56897c6740 20 client.20481 trim_cache size 0 max 16384 2024-03-09T00:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 Processor -- start 2024-03-09T00:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 -- start start 2024-03-09T00:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739f9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 0x55f4739f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801ae870 con 0x55f4739e8fd0 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801ac410 con 0x55f4739f2550 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f56897c6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801aa4d0 con 0x55f4739e9d40 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47882/0 (socket says 172.21.15.22:47882) 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 -- 172.21.15.22:0/2849932137 learned_addr learned my addr 172.21.15.22:0/2849932137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5685183640 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 0x55f4739f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5677fff640 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 -- 172.21.15.22:0/2849932137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 msgr2=0x55f4739f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 -- 172.21.15.22:0/2849932137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 msgr2=0x55f4739f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 0x55f4739f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5684982640 1 -- 172.21.15.22:0/2849932137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4739e4890 con 0x55f4739e8fd0 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.055+0000 7f5677fff640 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.056+0000 7f5684982640 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739f1d90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f567c003e80 tx=0x7f567c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b70c02a8122ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.056+0000 7f56667fc640 1 -- 172.21.15.22:0/2849932137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c064020 con 0x55f4739e8fd0 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.056+0000 7f56667fc640 1 -- 172.21.15.22:0/2849932137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f567c02c730 con 0x55f4739e8fd0 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.056+0000 7f56667fc640 1 -- 172.21.15.22:0/2849932137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0039e0 con 0x55f4739e8fd0 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- 172.21.15.22:0/2849932137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 msgr2=0x55f4739f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739f1d90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f567c003e80 tx=0x7f567c009780 comp rx=0 tx=0).stop 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- 172.21.15.22:0/2849932137 shutdown_connections 2024-03-09T00:10:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739f2550 0x55f4739f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e9d40 0x55f4739f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 --2- 172.21.15.22:0/2849932137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739e8fd0 0x55f4739f1d90 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- 172.21.15.22:0/2849932137 >> 172.21.15.22:0/2849932137 conn(0x55f4739b05b0 msgr2=0x55f4739e37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- 172.21.15.22:0/2849932137 shutdown_connections 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- 172.21.15.22:0/2849932137 wait complete. 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- start start 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 Processor -- start 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e8fd0 0x55f4739f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739f2550 0x55f4739f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801aaad0 con 0x55f4739f2550 2024-03-09T00:10:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801abdb0 con 0x55f4739e9d40 2024-03-09T00:10:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801ab0f0 con 0x55f4739e8fd0 2024-03-09T00:10:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.057+0000 7f56897c6740 10 client.0 _ll_register_callbacks cb 0x7f5678009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5685183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739f2550 0x55f4739f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51588/0 (socket says 172.21.15.22:51588) 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 -- 172.21.15.22:0/2219188277 learned_addr learned my addr 172.21.15.22:0/2219188277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5685183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739f2550 0x55f4739f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47890/0 (socket says 172.21.15.22:47890) 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5684982640 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e8fd0 0x55f4739f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 -- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e8fd0 msgr2=0x55f4739f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e8fd0 0x55f4739f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 -- 172.21.15.22:0/2219188277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739f2550 msgr2=0x55f4739f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739f2550 0x55f4739f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 -- 172.21.15.22:0/2219188277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4739f64e0 con 0x55f4739e9d40 2024-03-09T00:10:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f5677fff640 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739f5a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f567000e9c0 tx=0x7f5670009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d29a2cf3c7ecd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f565ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f565ffff640 1 -- 172.21.15.22:0/2219188277 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567001c040 con 0x55f4739e9d40 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f567c02c730 con 0x55f4739e9d40 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f565ffff640 1 -- 172.21.15.22:0/2219188277 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f567009f120 con 0x55f4739e9d40 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f565ffff640 1 -- 172.21.15.22:0/2219188277 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5670095e60 con 0x55f4739e9d40 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f56897c6740 10 client.29696 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.058+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f473a5a190 con 0x55f4739e9d40 2024-03-09T00:10:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f565ffff640 1 -- 172.21.15.22:0/2219188277 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f567c02c730 con 0x55f4739e9d40 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f565ffff640 1 -- 172.21.15.22:0/2219188277 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f56700bb0b0 con 0x55f4739e9d40 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f56897c6740 10 client.29696 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f56897c6740 10 client.29696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f56897c6740 20 client.29696 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f56897c6740 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a5a8c0 0x55f473a5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f56897c6740 4 client.29696 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f473954e80 con 0x55f473a5a8c0 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f5684982640 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a5a8c0 0x55f473a5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f5684982640 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a5a8c0 0x55f473a5cce0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980037a20c8a6249 server_cookie=b500d2ed3e2f4802 in_seq=0 out_seq=0 2024-03-09T00:10:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.059+0000 7f565ffff640 10 client.29696 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:25.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 36075 10.16 MB/sec execute 32 sec latency 258.750 ms 2024-03-09T00:10:25.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:25 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:10:25.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:25 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:10:25.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:25 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:10:25.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:25 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:10:25.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:25 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:10:25.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:25 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:10:25.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:25 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:10:25.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:25 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:10:25.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:25 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:10:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.938+0000 7f565ffff640 1 -- 172.21.15.22:0/2219188277 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f473954e80 con 0x55f473a5a8c0 2024-03-09T00:10:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.938+0000 7f565ffff640 10 client.29696 handle_command_reply: tid=0 2024-03-09T00:10:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.938+0000 7f56897c6740 1 client.29696 shutdown 2024-03-09T00:10:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.938+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 msgr2=0x55f4739f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.938+0000 7f56897c6740 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739f5a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f567000e9c0 tx=0x7f5670009f60 comp rx=0 tx=0).stop 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 shutdown_connections 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55f473a5a8c0 0x55f473a5cce0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f4739f2550 0x55f4739f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f4739e9d40 0x55f4739f5a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f565ffff640 0 client.29696 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 --2- 172.21.15.22:0/2219188277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4739e8fd0 0x55f4739f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 >> 172.21.15.22:0/2219188277 conn(0x55f4739b05b0 msgr2=0x55f4739edef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 shutdown_connections 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 1 -- 172.21.15.22:0/2219188277 wait complete. 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f56897c6740 20 client.29696 trim_cache size 0 max 16384 2024-03-09T00:10:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:10:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:10:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:10:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:10:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f56680a2a90 msgr2=0x7f56680a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f5686be6640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f56680a2a90 0x7f56680a4f50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f567c0096c0 tx=0x7f567c02e040 comp rx=0 tx=0).stop 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680069330 msgr2=0x7f568019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.939+0000 7f5686be6640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680069330 0x7f568019dc30 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f567000e9f0 tx=0x7f567000eeb0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 shutdown_connections 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f56680a2a90 0x7f56680a4f50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5680107400 0x7f56801978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5680069330 0x7f568019dc30 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 --2- 172.21.15.22:0/2547261194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5680068970 0x7f568019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 >> 172.21.15.22:0/2547261194 conn(0x7f5680077040 msgr2=0x7f56800ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 shutdown_connections 2024-03-09T00:10:25.938 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:25.940+0000 7f5686be6640 1 -- 172.21.15.22:0/2547261194 wait complete. 2024-03-09T00:10:26.069 INFO:tasks.workunit.client.0.smithi022.stdout: 10 36186 9.95 MB/sec execute 33 sec latency 573.968 ms 2024-03-09T00:10:26.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:26 smithi062 ceph-mon[36129]: pgmap v505: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 565 op/s 2024-03-09T00:10:26.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:26 smithi192 ceph-mon[35995]: pgmap v505: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 565 op/s 2024-03-09T00:10:26.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:26 smithi022 ceph-mon[31894]: pgmap v505: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 565 op/s 2024-03-09T00:10:27.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 36631 10.48 MB/sec execute 34 sec latency 218.637 ms 2024-03-09T00:10:28.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 36930 10.67 MB/sec execute 35 sec latency 219.682 ms 2024-03-09T00:10:28.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:28 smithi062 ceph-mon[36129]: pgmap v506: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 542 op/s 2024-03-09T00:10:28.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:28 smithi192 ceph-mon[35995]: pgmap v506: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 542 op/s 2024-03-09T00:10:28.734 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:28 smithi022 ceph-mon[31894]: pgmap v506: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 542 op/s 2024-03-09T00:10:29.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 37210 10.89 MB/sec execute 36 sec latency 113.895 ms 2024-03-09T00:10:30.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 37428 10.98 MB/sec execute 37 sec latency 189.339 ms 2024-03-09T00:10:30.360 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:30 smithi062 ceph-mon[36129]: pgmap v507: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-03-09T00:10:30.380 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:30 smithi192 ceph-mon[35995]: pgmap v507: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-03-09T00:10:30.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:30 smithi022 ceph-mon[31894]: pgmap v507: 129 pgs: 129 active+clean; 352 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-03-09T00:10:31.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 37633 11.00 MB/sec execute 38 sec latency 67.963 ms 2024-03-09T00:10:32.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 37864 11.04 MB/sec execute 39 sec latency 225.502 ms 2024-03-09T00:10:32.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:32 smithi022 ceph-mon[31894]: pgmap v508: 129 pgs: 129 active+clean; 340 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 553 op/s 2024-03-09T00:10:32.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:32 smithi062 ceph-mon[36129]: pgmap v508: 129 pgs: 129 active+clean; 340 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 553 op/s 2024-03-09T00:10:32.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:32 smithi192 ceph-mon[35995]: pgmap v508: 129 pgs: 129 active+clean; 340 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 553 op/s 2024-03-09T00:10:33.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 38095 11.00 MB/sec execute 40 sec latency 252.365 ms 2024-03-09T00:10:34.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 38351 10.86 MB/sec execute 41 sec latency 241.965 ms 2024-03-09T00:10:34.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:34 smithi022 ceph-mon[31894]: pgmap v509: 129 pgs: 129 active+clean; 340 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 553 op/s 2024-03-09T00:10:34.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:34 smithi062 ceph-mon[36129]: pgmap v509: 129 pgs: 129 active+clean; 340 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 553 op/s 2024-03-09T00:10:34.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:34 smithi192 ceph-mon[35995]: pgmap v509: 129 pgs: 129 active+clean; 340 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 553 op/s 2024-03-09T00:10:35.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 38680 10.95 MB/sec execute 42 sec latency 242.067 ms 2024-03-09T00:10:36.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 39079 11.24 MB/sec execute 43 sec latency 171.846 ms 2024-03-09T00:10:36.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:36 smithi022 ceph-mon[31894]: pgmap v510: 129 pgs: 129 active+clean; 350 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 597 op/s 2024-03-09T00:10:36.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:36 smithi062 ceph-mon[36129]: pgmap v510: 129 pgs: 129 active+clean; 350 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 597 op/s 2024-03-09T00:10:36.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:36 smithi192 ceph-mon[35995]: pgmap v510: 129 pgs: 129 active+clean; 350 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.8 MiB/s wr, 597 op/s 2024-03-09T00:10:37.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 39491 11.53 MB/sec execute 44 sec latency 217.295 ms 2024-03-09T00:10:38.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 39934 11.80 MB/sec execute 45 sec latency 222.643 ms 2024-03-09T00:10:38.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:38 smithi022 ceph-mon[31894]: pgmap v511: 129 pgs: 129 active+clean; 350 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.4 MiB/s wr, 396 op/s 2024-03-09T00:10:38.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:38 smithi062 ceph-mon[36129]: pgmap v511: 129 pgs: 129 active+clean; 350 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.4 MiB/s wr, 396 op/s 2024-03-09T00:10:38.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:38 smithi192 ceph-mon[35995]: pgmap v511: 129 pgs: 129 active+clean; 350 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.4 MiB/s wr, 396 op/s 2024-03-09T00:10:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 -- 172.21.15.22:0/2081330745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd038101500 msgr2=0x7fd0381018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 --2- 172.21.15.22:0/2081330745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd038101500 0x7fd0381018e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02802bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 -- 172.21.15.22:0/2081330745 shutdown_connections 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 --2- 172.21.15.22:0/2081330745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0381065d0 0x7fd03810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 --2- 172.21.15.22:0/2081330745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd038101eb0 0x7fd038105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 --2- 172.21.15.22:0/2081330745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd038101500 0x7fd0381018e0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 -- 172.21.15.22:0/2081330745 >> 172.21.15.22:0/2081330745 conn(0x7fd0380fcd80 msgr2=0x7fd0380ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 -- 172.21.15.22:0/2081330745 shutdown_connections 2024-03-09T00:10:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.037+0000 7fd03d080640 1 -- 172.21.15.22:0/2081330745 wait complete. 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.038+0000 7fd03d080640 1 Processor -- start 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03d080640 1 -- start start 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03d080640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101500 0x7fd03819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03d080640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0381065d0 0x7fd03819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03d080640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0381979a0 0x7fd038197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03d080640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038111680 con 0x7fd0381979a0 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03d080640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038111500 con 0x7fd038101500 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03d080640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038111800 con 0x7fd0381065d0 2024-03-09T00:10:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd036d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101500 0x7fd03819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0381065d0 0x7fd03819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0381065d0 0x7fd03819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:33850/0 (socket says 172.21.15.22:33850) 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03659c640 1 -- 172.21.15.22:0/2338883629 learned_addr learned my addr 172.21.15.22:0/2338883629 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.039+0000 7fd03759e640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0381979a0 0x7fd038197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd036d9d640 1 -- 172.21.15.22:0/2338883629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0381065d0 msgr2=0x7fd03819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd036d9d640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0381065d0 0x7fd03819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd036d9d640 1 -- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0381979a0 msgr2=0x7fd038197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd036d9d640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0381979a0 0x7fd038197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd036d9d640 1 -- 172.21.15.22:0/2338883629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0381984c0 con 0x7fd038101500 2024-03-09T00:10:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd03759e640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0381979a0 0x7fd038197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd036d9d640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101500 0x7fd03819d7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd028036040 tx=0x7fd02802cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd01ffff640 1 -- 172.21.15.22:0/2338883629 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028049070 con 0x7fd038101500 2024-03-09T00:10:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd038198750 con 0x7fd038101500 2024-03-09T00:10:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd01ffff640 1 -- 172.21.15.22:0/2338883629 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd028004450 con 0x7fd038101500 2024-03-09T00:10:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd01ffff640 1 -- 172.21.15.22:0/2338883629 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028003dc0 con 0x7fd038101500 2024-03-09T00:10:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.040+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0380737b0 con 0x7fd038101500 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.043+0000 7fd01ffff640 1 -- 172.21.15.22:0/2338883629 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd02804c070 con 0x7fd038101500 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.043+0000 7fd03fc5b740 1 Processor -- start 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.043+0000 7fd03fc5b740 1 -- start start 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c491d40 0x555f3c490a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c491450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd01ffff640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0200a2a10 0x7fd0200a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c49a550 0x555f3c49c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3c3f55f0 con 0x555f3c491d40 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3c39e930 con 0x555f3c490fd0 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3c3f1690 con 0x555f3c49a550 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd036d9d640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0200a2a10 0x7fd0200a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd01ffff640 1 -- 172.21.15.22:0/2338883629 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd0280f4390 con 0x7fd038101500 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c49a550 0x555f3c49c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c491d40 0x555f3c490a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c491d40 0x555f3c490a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58520/0 (socket says 172.21.15.22:58520) 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.044+0000 7fd03759e640 1 -- 172.21.15.22:0/129883883 learned_addr learned my addr 172.21.15.22:0/129883883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03659c640 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c491450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03759e640 1 -- 172.21.15.22:0/129883883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c49a550 msgr2=0x555f3c49c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03759e640 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c49a550 0x555f3c49c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03759e640 1 -- 172.21.15.22:0/129883883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 msgr2=0x555f3c491450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03759e640 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c491450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03759e640 1 -- 172.21.15.22:0/129883883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3c10a210 con 0x555f3c491d40 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03659c640 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c491450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd036d9d640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0200a2a10 0x7fd0200a4ed0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd0280bcc20 tx=0x7fd028000bc0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd03759e640 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c491d40 0x555f3c490a90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd030010fd0 tx=0x7fd030009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8ca68ba3e33f70 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd01d7fa640 1 -- 172.21.15.22:0/129883883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030016030 con 0x555f3c491d40 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd01d7fa640 1 -- 172.21.15.22:0/129883883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd03001e890 con 0x555f3c491d40 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.045+0000 7fd01d7fa640 1 -- 172.21.15.22:0/129883883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030019040 con 0x555f3c491d40 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 -- 172.21.15.22:0/129883883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c491d40 msgr2=0x555f3c490a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c491d40 0x555f3c490a90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd030010fd0 tx=0x7fd030009500 comp rx=0 tx=0).stop 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 -- 172.21.15.22:0/129883883 shutdown_connections 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c49a550 0x555f3c49c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c491450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/129883883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c491d40 0x555f3c490a90 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 -- 172.21.15.22:0/129883883 >> 172.21.15.22:0/129883883 conn(0x555f3c4585b0 msgr2=0x555f3c48bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 -- 172.21.15.22:0/129883883 shutdown_connections 2024-03-09T00:10:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.046+0000 7fd03fc5b740 1 -- 172.21.15.22:0/129883883 wait complete. 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 -- start start 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 Processor -- start 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c497550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c497fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3c49d280 con 0x555f3c490fd0 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3c3b1d90 con 0x555f3c49a550 2024-03-09T00:10:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3c49d400 con 0x555f3c491d40 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03fc5b740 10 client.0 _ll_register_callbacks cb 0x555f3c109a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.047+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c497550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c497550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58528/0 (socket says 172.21.15.22:58528) 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 -- 172.21.15.22:0/3615680625 learned_addr learned my addr 172.21.15.22:0/3615680625 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03659c640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c497fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 -- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 msgr2=0x555f3c497a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 -- 172.21.15.22:0/3615680625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 msgr2=0x555f3c497fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c497fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 -- 172.21.15.22:0/3615680625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f3c1d6620 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03659c640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03759e640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c497550 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd03001e860 tx=0x7fd0300095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95984a842c29bccb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd01aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd01aff5640 1 -- 172.21.15.22:0/3615680625 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030023030 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd01aff5640 1 -- 172.21.15.22:0/3615680625 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd030004dd0 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.048+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd03001e890 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd01aff5640 1 -- 172.21.15.22:0/3615680625 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd030035050 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 10 client.20499 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555f3c4b1ca0 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd01aff5640 1 -- 172.21.15.22:0/3615680625 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd030029040 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd01aff5640 1 -- 172.21.15.22:0/3615680625 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd030004020 con 0x555f3c490fd0 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 10 client.20499 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 10 client.20499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 20 client.20499 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c4b3050 0x555f3c4b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 4 client.20499 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.049+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd03001e890 con 0x555f3c4b3050 2024-03-09T00:10:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.050+0000 7fd03659c640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c4b3050 0x555f3c4b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.050+0000 7fd01aff5640 10 client.20499 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.050+0000 7fd03659c640 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c4b3050 0x555f3c4b5440 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc005805852816b7 server_cookie=9ed3ea805a43f91b in_seq=0 out_seq=0 2024-03-09T00:10:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd01aff5640 1 -- 172.21.15.22:0/3615680625 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fd03001e890 con 0x555f3c4b3050 2024-03-09T00:10:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd01aff5640 10 client.20499 handle_command_reply: tid=0 2024-03-09T00:10:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 client.20499 shutdown 2024-03-09T00:10:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 msgr2=0x555f3c497550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c497550 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd03001e860 tx=0x7fd0300095d0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 shutdown_connections 2024-03-09T00:10:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c4b3050 0x555f3c4b5440 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c497fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/3615680625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c497550 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 >> 172.21.15.22:0/3615680625 conn(0x555f3c4585b0 msgr2=0x555f3c48da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.053+0000 7fd01aff5640 0 client.20499 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.054+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 shutdown_connections 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.054+0000 7fd03fc5b740 1 -- 172.21.15.22:0/3615680625 wait complete. 2024-03-09T00:10:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.054+0000 7fd03fc5b740 20 client.20499 trim_cache size 0 max 16384 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 Processor -- start 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 -- start start 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c499c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c49a550 0x555f3c495e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0381ae960 con 0x555f3c49a550 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0381ac500 con 0x555f3c490fd0 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0381aa5c0 con 0x555f3c491d40 2024-03-09T00:10:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c49a550 0x555f3c495e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.068+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c49a550 0x555f3c495e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58538/0 (socket says 172.21.15.22:58538) 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:33878/0 (socket says 172.21.15.22:33878) 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 -- 172.21.15.22:0/587088279 learned_addr learned my addr 172.21.15.22:0/587088279 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c499c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 -- 172.21.15.22:0/587088279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 msgr2=0x555f3c497df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 -- 172.21.15.22:0/587088279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 msgr2=0x555f3c499c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c499c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 -- 172.21.15.22:0/587088279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3c4902a0 con 0x555f3c49a550 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd03759e640 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c49a550 0x555f3c495e20 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd030005cc0 tx=0x7fd0300106a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643d3669cc8b660f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd01cff9640 1 -- 172.21.15.22:0/587088279 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030023070 con 0x555f3c49a550 2024-03-09T00:10:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd01cff9640 1 -- 172.21.15.22:0/587088279 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd030010830 con 0x555f3c49a550 2024-03-09T00:10:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.069+0000 7fd01cff9640 1 -- 172.21.15.22:0/587088279 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030016070 con 0x555f3c49a550 2024-03-09T00:10:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 -- 172.21.15.22:0/587088279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c49a550 msgr2=0x555f3c495e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c49a550 0x555f3c495e20 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd030005cc0 tx=0x7fd0300106a0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 -- 172.21.15.22:0/587088279 shutdown_connections 2024-03-09T00:10:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c49a550 0x555f3c495e20 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c497df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/587088279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c490fd0 0x555f3c499c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 -- 172.21.15.22:0/587088279 >> 172.21.15.22:0/587088279 conn(0x555f3c4585b0 msgr2=0x555f3c492830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 -- 172.21.15.22:0/587088279 shutdown_connections 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.070+0000 7fd03fc5b740 1 -- 172.21.15.22:0/587088279 wait complete. 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 -- start start 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 Processor -- start 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c4a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c49fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c4a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0381aabc0 con 0x555f3c490fd0 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0381abea0 con 0x555f3c49a550 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0381ab1e0 con 0x555f3c491d40 2024-03-09T00:10:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03fc5b740 10 client.0 _ll_register_callbacks cb 0x7fd0300105f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c4a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c4a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58542/0 (socket says 172.21.15.22:58542) 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 -- 172.21.15.22:0/2980392434 learned_addr learned my addr 172.21.15.22:0/2980392434 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03659c640 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c49fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c4a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 -- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 msgr2=0x555f3c49fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c49fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 -- 172.21.15.22:0/2980392434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 msgr2=0x555f3c4a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c4a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 -- 172.21.15.22:0/2980392434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f3c4a0900 con 0x555f3c490fd0 2024-03-09T00:10:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd03759e640 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c4a1930 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x555f3c39ea10 tx=0x7fd030004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964e490b020423e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.071+0000 7fd01aff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd01aff5640 1 -- 172.21.15.22:0/2980392434 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030016040 con 0x555f3c490fd0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd01aff5640 1 -- 172.21.15.22:0/2980392434 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd030002030 con 0x555f3c490fd0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd030010830 con 0x555f3c490fd0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd01aff5640 1 -- 172.21.15.22:0/2980392434 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd030023040 con 0x555f3c490fd0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 10 client.20505 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555f3c50a120 con 0x555f3c490fd0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd01aff5640 1 -- 172.21.15.22:0/2980392434 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd030010830 con 0x555f3c490fd0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd01aff5640 1 -- 172.21.15.22:0/2980392434 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd030004bd0 con 0x555f3c490fd0 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 10 client.20505 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 10 client.20505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 20 client.20505 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c50a710 0x555f3c50cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 4 client.20505 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:10:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f3c4b5830 con 0x555f3c50a710 2024-03-09T00:10:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.072+0000 7fd03659c640 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c50a710 0x555f3c50cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.073+0000 7fd03659c640 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c50a710 0x555f3c50cb30 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bddc9b20c5298a7b server_cookie=df9153347d99607d in_seq=0 out_seq=0 2024-03-09T00:10:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:39.073+0000 7fd01aff5640 10 client.20505 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:39.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 40154 11.81 MB/sec execute 46 sec latency 372.048 ms 2024-03-09T00:10:40.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 40206 11.61 MB/sec execute 47 sec latency 842.482 ms 2024-03-09T00:10:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.165+0000 7fd01aff5640 1 -- 172.21.15.22:0/2980392434 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f3c4b5830 con 0x555f3c50a710 2024-03-09T00:10:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.165+0000 7fd01aff5640 10 client.20505 handle_command_reply: tid=0 2024-03-09T00:10:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.165+0000 7fd03fc5b740 1 client.20505 shutdown 2024-03-09T00:10:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 msgr2=0x555f3c4a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c4a1930 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x555f3c39ea10 tx=0x7fd030004590 comp rx=0 tx=0).stop 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 shutdown_connections 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x555f3c50a710 0x555f3c50cb30 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3c49a550 0x555f3c4a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f3c491d40 0x555f3c49fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd01aff5640 0 client.20505 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 --2- 172.21.15.22:0/2980392434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555f3c490fd0 0x555f3c4a1930 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 >> 172.21.15.22:0/2980392434 conn(0x555f3c4585b0 msgr2=0x555f3c495da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 shutdown_connections 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 1 -- 172.21.15.22:0/2980392434 wait complete. 2024-03-09T00:10:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03fc5b740 20 client.20505 trim_cache size 0 max 16384 2024-03-09T00:10:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:10:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:10:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:10:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0200a2a10 msgr2=0x7fd0200a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0200a2a10 0x7fd0200a4ed0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd0280bcc20 tx=0x7fd028000bc0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101500 msgr2=0x7fd03819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101500 0x7fd03819d7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd028036040 tx=0x7fd02802cb20 comp rx=0 tx=0).stop 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 shutdown_connections 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd0200a2a10 0x7fd0200a4ed0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0381979a0 0x7fd038197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0381065d0 0x7fd03819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 --2- 172.21.15.22:0/2338883629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101500 0x7fd03819d7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.166+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 >> 172.21.15.22:0/2338883629 conn(0x7fd0380fcd80 msgr2=0x7fd0380ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.167+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 shutdown_connections 2024-03-09T00:10:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:40.167+0000 7fd03d080640 1 -- 172.21.15.22:0/2338883629 wait complete. 2024-03-09T00:10:40.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:40 smithi022 ceph-mon[31894]: pgmap v512: 129 pgs: 129 active+clean; 352 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-03-09T00:10:40.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:40 smithi062 ceph-mon[36129]: pgmap v512: 129 pgs: 129 active+clean; 352 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-03-09T00:10:40.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:40 smithi192 ceph-mon[35995]: pgmap v512: 129 pgs: 129 active+clean; 352 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-03-09T00:10:41.070 INFO:tasks.workunit.client.0.smithi022.stdout: 10 40560 11.87 MB/sec execute 48 sec latency 325.423 ms 2024-03-09T00:10:42.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 40828 11.98 MB/sec execute 49 sec latency 207.289 ms 2024-03-09T00:10:42.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:42 smithi022 ceph-mon[31894]: pgmap v513: 129 pgs: 129 active+clean; 371 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.3 MiB/s wr, 644 op/s 2024-03-09T00:10:42.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:42 smithi062 ceph-mon[36129]: pgmap v513: 129 pgs: 129 active+clean; 371 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.3 MiB/s wr, 644 op/s 2024-03-09T00:10:42.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:42 smithi192 ceph-mon[35995]: pgmap v513: 129 pgs: 129 active+clean; 371 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.3 MiB/s wr, 644 op/s 2024-03-09T00:10:43.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 41075 11.98 MB/sec execute 50 sec latency 185.806 ms 2024-03-09T00:10:44.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 41287 11.98 MB/sec execute 51 sec latency 238.103 ms 2024-03-09T00:10:44.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:44 smithi022 ceph-mon[31894]: pgmap v514: 129 pgs: 129 active+clean; 371 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 366 op/s 2024-03-09T00:10:44.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:44 smithi062 ceph-mon[36129]: pgmap v514: 129 pgs: 129 active+clean; 371 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 366 op/s 2024-03-09T00:10:44.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:44 smithi192 ceph-mon[35995]: pgmap v514: 129 pgs: 129 active+clean; 371 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 366 op/s 2024-03-09T00:10:45.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 41509 11.96 MB/sec execute 52 sec latency 238.942 ms 2024-03-09T00:10:46.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 41884 12.02 MB/sec execute 53 sec latency 101.771 ms 2024-03-09T00:10:46.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:46 smithi022 ceph-mon[31894]: pgmap v515: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-03-09T00:10:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:46 smithi062 ceph-mon[36129]: pgmap v515: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-03-09T00:10:46.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:46 smithi192 ceph-mon[35995]: pgmap v515: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-03-09T00:10:47.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 42204 12.10 MB/sec execute 54 sec latency 244.878 ms 2024-03-09T00:10:48.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 42622 12.24 MB/sec execute 55 sec latency 190.265 ms 2024-03-09T00:10:48.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:48 smithi022 ceph-mon[31894]: pgmap v516: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 443 op/s 2024-03-09T00:10:48.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:10:48.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:10:48.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:48 smithi062 ceph-mon[36129]: pgmap v516: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 443 op/s 2024-03-09T00:10:48.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:48 smithi192 ceph-mon[35995]: pgmap v516: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 443 op/s 2024-03-09T00:10:48.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.698+0000 7f65a88d1640 1 -- 172.21.15.22:0/1029623237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4100d80 msgr2=0x7f65a4104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.698+0000 7f65a88d1640 1 --2- 172.21.15.22:0/1029623237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4100d80 0x7f65a4104dc0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f658c009900 tx=0x7f658c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.698+0000 7f65a88d1640 1 -- 172.21.15.22:0/1029623237 shutdown_connections 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.698+0000 7f65a88d1640 1 --2- 172.21.15.22:0/1029623237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a41054a0 0x7f65a410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.698+0000 7f65a88d1640 1 --2- 172.21.15.22:0/1029623237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4100d80 0x7f65a4104dc0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.698+0000 7f65a88d1640 1 --2- 172.21.15.22:0/1029623237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a41003d0 0x7f65a41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.699+0000 7f65a88d1640 1 -- 172.21.15.22:0/1029623237 >> 172.21.15.22:0/1029623237 conn(0x7f65a40772f0 msgr2=0x7f65a4077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.699+0000 7f65a88d1640 1 -- 172.21.15.22:0/1029623237 shutdown_connections 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.699+0000 7f65a88d1640 1 -- 172.21.15.22:0/1029623237 wait complete. 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.699+0000 7f65a88d1640 1 Processor -- start 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.699+0000 7f65a88d1640 1 -- start start 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.699+0000 7f65a88d1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a41003d0 0x7f65a419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.699+0000 7f65a88d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a41054a0 0x7f65a419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a88d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4197a00 0x7f65a4197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a88d1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a4110530 con 0x7f65a4197a00 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a88d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41103b0 con 0x7f65a41054a0 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a88d1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41106b0 con 0x7f65a41003d0 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4197a00 0x7f65a4197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4197a00 0x7f65a4197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38700/0 (socket says 172.21.15.22:38700) 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 -- 172.21.15.22:0/2628934037 learned_addr learned my addr 172.21.15.22:0/2628934037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a259c640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a41003d0 0x7f65a419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f659bfff640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a41054a0 0x7f65a419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 -- 172.21.15.22:0/2628934037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a41003d0 msgr2=0x7f65a419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a41003d0 0x7f65a419d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 -- 172.21.15.22:0/2628934037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a41054a0 msgr2=0x7f65a419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a41054a0 0x7f65a419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 -- 172.21.15.22:0/2628934037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4198520 con 0x7f65a4197a00 2024-03-09T00:10:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4197a00 0x7f65a4197eb0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f659c00efd0 tx=0x7f659c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f659b7fe640 1 -- 172.21.15.22:0/2628934037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c003b70 con 0x7f65a4197a00 2024-03-09T00:10:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f659b7fe640 1 -- 172.21.15.22:0/2628934037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f659c003d10 con 0x7f65a4197a00 2024-03-09T00:10:48.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a4198810 con 0x7f65a4197a00 2024-03-09T00:10:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f659b7fe640 1 -- 172.21.15.22:0/2628934037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0109f0 con 0x7f65a4197a00 2024-03-09T00:10:48.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.700+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a41a49c0 con 0x7f65a4197a00 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.702+0000 7f659b7fe640 1 -- 172.21.15.22:0/2628934037 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f659c004020 con 0x7f65a4197a00 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 Processor -- start 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 -- start start 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f659b7fe640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65900a2970 0x7f65900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55791b329e00 con 0x55791b3c6760 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55791b2d2b10 con 0x55791b3cfc60 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65ab590740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55791b325ea0 con 0x55791b3c7080 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f659bfff640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65900a2970 0x7f65900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65a259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f659b7fe640 1 -- 172.21.15.22:0/2628934037 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f659c0cf680 con 0x7f65a4197a00 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65a259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38702/0 (socket says 172.21.15.22:38702) 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65a259c640 1 -- 172.21.15.22:0/23154692 learned_addr learned my addr 172.21.15.22:0/23154692 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.703+0000 7f659bfff640 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f65a259c640 1 -- 172.21.15.22:0/23154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 msgr2=0x55791b3cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f65a259c640 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f65a259c640 1 -- 172.21.15.22:0/23154692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 msgr2=0x55791b3d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f65a259c640 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f65a259c640 1 -- 172.21.15.22:0/23154692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55791b1087d0 con 0x55791b3c6760 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f65a259c640 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3c6b40 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f6594009ca0 tx=0x7f6594011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07942140fe11e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f659bfff640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65900a2970 0x7f65900a4e30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f658c009230 tx=0x7f658c02c5d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f6598ff9640 1 -- 172.21.15.22:0/23154692 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594005220 con 0x55791b3c6760 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f6598ff9640 1 -- 172.21.15.22:0/23154692 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6594004c80 con 0x55791b3c6760 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.704+0000 7f6598ff9640 1 -- 172.21.15.22:0/23154692 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940187d0 con 0x55791b3c6760 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 -- 172.21.15.22:0/23154692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 msgr2=0x55791b3c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3c6b40 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f6594009ca0 tx=0x7f6594011810 comp rx=0 tx=0).stop 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 -- 172.21.15.22:0/23154692 shutdown_connections 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 --2- 172.21.15.22:0/23154692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3c6b40 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 -- 172.21.15.22:0/23154692 >> 172.21.15.22:0/23154692 conn(0x55791b3c0e90 msgr2=0x55791b3c1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 -- 172.21.15.22:0/23154692 shutdown_connections 2024-03-09T00:10:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 -- 172.21.15.22:0/23154692 wait complete. 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 -- start start 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 Processor -- start 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c6760 0x55791b3cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.705+0000 7f65ab590740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c7080 0x55791b3cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55791b3d2a50 con 0x55791b3c7080 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55791b2e60a0 con 0x55791b3c6760 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55791b3d2bd0 con 0x55791b3cfc60 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 10 client.0 _ll_register_callbacks cb 0x55791b2e5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c7080 0x55791b3cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c7080 0x55791b3cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38710/0 (socket says 172.21.15.22:38710) 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 -- 172.21.15.22:0/1998259842 learned_addr learned my addr 172.21.15.22:0/1998259842 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65a259c640 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c6760 0x55791b3cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 -- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 msgr2=0x55791b3ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 -- 172.21.15.22:0/1998259842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c6760 msgr2=0x55791b3cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c6760 0x55791b3cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 -- 172.21.15.22:0/1998259842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55791b03e3f0 con 0x55791b3c7080 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f659bfff640 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c7080 0x55791b3cc730 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f658c02cfd0 tx=0x7f658c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759f8da2a59c635 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f6589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f6589ffb640 1 -- 172.21.15.22:0/1998259842 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c02c880 con 0x55791b3c7080 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f6589ffb640 1 -- 172.21.15.22:0/1998259842 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f658c004ee0 con 0x55791b3c7080 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6594004c80 con 0x55791b3c7080 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f6589ffb640 1 -- 172.21.15.22:0/1998259842 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f658c05c050 con 0x55791b3c7080 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 10 client.20517 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.706+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55791affe0f0 con 0x55791b3c7080 2024-03-09T00:10:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f6589ffb640 1 -- 172.21.15.22:0/1998259842 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6594004c80 con 0x55791b3c7080 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f6589ffb640 1 -- 172.21.15.22:0/1998259842 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f658c003ae0 con 0x55791b3c7080 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65ab590740 10 client.20517 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65ab590740 10 client.20517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65ab590740 20 client.20517 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65ab590740 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b3df6f0 0x55791b3e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65ab590740 4 client.20517 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55791b337340 con 0x55791b3df6f0 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65a259c640 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b3df6f0 0x55791b3e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f6589ffb640 10 client.20517 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.707+0000 7f65a259c640 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b3df6f0 0x55791b3e1b00 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70b1beea0814a97b server_cookie=dd227bce1155e883 in_seq=0 out_seq=0 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f6589ffb640 1 -- 172.21.15.22:0/1998259842 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55791b337340 con 0x55791b3df6f0 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f6589ffb640 10 client.20517 handle_command_reply: tid=0 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 client.20517 shutdown 2024-03-09T00:10:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c7080 msgr2=0x55791b3cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c7080 0x55791b3cc730 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f658c02cfd0 tx=0x7f658c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 shutdown_connections 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b3df6f0 0x55791b3e1b00 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c7080 0x55791b3cc730 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 --2- 172.21.15.22:0/1998259842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c6760 0x55791b3cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f6589ffb640 0 client.20517 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 >> 172.21.15.22:0/1998259842 conn(0x55791b3c0e90 msgr2=0x55791b3c2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 shutdown_connections 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 1 -- 172.21.15.22:0/1998259842 wait complete. 2024-03-09T00:10:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.709+0000 7f65ab590740 20 client.20517 trim_cache size 0 max 16384 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 Processor -- start 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 -- start start 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41b0a30 con 0x55791b3c6760 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41ae5d0 con 0x55791b3cfc60 2024-03-09T00:10:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65ab590740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41ac690 con 0x55791b3c7080 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65a259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65a259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38716/0 (socket says 172.21.15.22:38716) 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65a259c640 1 -- 172.21.15.22:0/3123663015 learned_addr learned my addr 172.21.15.22:0/3123663015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f659bfff640 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.725+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f65a259c640 1 -- 172.21.15.22:0/3123663015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 msgr2=0x55791b3cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f65a259c640 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f65a259c640 1 -- 172.21.15.22:0/3123663015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 msgr2=0x55791b3cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f65a259c640 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f65a259c640 1 -- 172.21.15.22:0/3123663015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55791b3cac80 con 0x55791b3c6760 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f65a259c640 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3cee60 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6594009c70 tx=0x7f6594005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97aa787712ee1e83 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f658bfff640 1 -- 172.21.15.22:0/3123663015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659401e040 con 0x55791b3c6760 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f658bfff640 1 -- 172.21.15.22:0/3123663015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65940054e0 con 0x55791b3c6760 2024-03-09T00:10:48.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.726+0000 7f658bfff640 1 -- 172.21.15.22:0/3123663015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940035f0 con 0x55791b3c6760 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- 172.21.15.22:0/3123663015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 msgr2=0x55791b3cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3cee60 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6594009c70 tx=0x7f6594005700 comp rx=0 tx=0).stop 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- 172.21.15.22:0/3123663015 shutdown_connections 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3cfc60 0x55791b3cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3c7080 0x55791b3cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 --2- 172.21.15.22:0/3123663015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3cee60 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- 172.21.15.22:0/3123663015 >> 172.21.15.22:0/3123663015 conn(0x55791b3c0e90 msgr2=0x55791b3cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- 172.21.15.22:0/3123663015 shutdown_connections 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- 172.21.15.22:0/3123663015 wait complete. 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- start start 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 Processor -- start 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c7080 0x55791b3d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41acc90 con 0x55791b3c6760 2024-03-09T00:10:48.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41adf70 con 0x55791b3c7080 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a41ad2b0 con 0x55791b3cfc60 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65ab590740 10 client.0 _ll_register_callbacks cb 0x7f658c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65a259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65a259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3d7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38718/0 (socket says 172.21.15.22:38718) 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65a259c640 1 -- 172.21.15.22:0/3315079270 learned_addr learned my addr 172.21.15.22:0/3315079270 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f65a2d9d640 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.727+0000 7f659bfff640 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c7080 0x55791b3d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65a259c640 1 -- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 msgr2=0x55791b3d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65a259c640 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65a259c640 1 -- 172.21.15.22:0/3315079270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c7080 msgr2=0x55791b3d3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65a259c640 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c7080 0x55791b3d3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65a259c640 1 -- 172.21.15.22:0/3315079270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55791b3d4260 con 0x55791b3c6760 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65a259c640 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3d7150 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6594011850 tx=0x7f6594011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dada4786f5f2b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f6589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f6589ffb640 1 -- 172.21.15.22:0/3315079270 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940044d0 con 0x55791b3c6760 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f6589ffb640 1 -- 172.21.15.22:0/3315079270 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f65940035f0 con 0x55791b3c6760 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65940054e0 con 0x55791b3c6760 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f6589ffb640 1 -- 172.21.15.22:0/3315079270 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65940037f0 con 0x55791b3c6760 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 10 client.20523 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55791b43afd0 con 0x55791b3c6760 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f6589ffb640 1 -- 172.21.15.22:0/3315079270 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65940054e0 con 0x55791b3c6760 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f6589ffb640 1 -- 172.21.15.22:0/3315079270 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f659402b070 con 0x55791b3c6760 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 10 client.20523 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 10 client.20523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 20 client.20523 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b43b500 0x55791b43d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 4 client.20523 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.728+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55791b331690 con 0x55791b43b500 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.729+0000 7f659bfff640 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b43b500 0x55791b43d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.729+0000 7f659bfff640 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b43b500 0x55791b43d8f0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81463b35ce9bce1c server_cookie=58b4bf0ea20548ed in_seq=0 out_seq=0 2024-03-09T00:10:48.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.729+0000 7f6589ffb640 10 client.20523 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f6589ffb640 1 -- 172.21.15.22:0/3315079270 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55791b331690 con 0x55791b43b500 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f6589ffb640 10 client.20523 handle_command_reply: tid=0 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 client.20523 shutdown 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 msgr2=0x55791b3d7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3d7150 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6594011850 tx=0x7f6594011e20 comp rx=0 tx=0).stop 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 shutdown_connections 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55791b43b500 0x55791b43d8f0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55791b3cfc60 0x55791b3d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55791b3c7080 0x55791b3d3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f6589ffb640 0 client.20523 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 --2- 172.21.15.22:0/3315079270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55791b3c6760 0x55791b3d7150 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 >> 172.21.15.22:0/3315079270 conn(0x55791b3c0e90 msgr2=0x55791b3caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 shutdown_connections 2024-03-09T00:10:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.732+0000 7f65ab590740 1 -- 172.21.15.22:0/3315079270 wait complete. 2024-03-09T00:10:48.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65ab590740 20 client.20523 trim_cache size 0 max 16384 2024-03-09T00:10:48.732 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:10:48.732 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:10:48.732 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:10:48.732 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:10:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65900a2970 msgr2=0x7f65900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65900a2970 0x7f65900a4e30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f658c009230 tx=0x7f658c02c5d0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4197a00 msgr2=0x7f65a4197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4197a00 0x7f65a4197eb0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f659c00efd0 tx=0x7f659c00c6a0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 shutdown_connections 2024-03-09T00:10:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f65900a2970 0x7f65900a4e30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65a4197a00 0x7f65a4197eb0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a41054a0 0x7f65a419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 --2- 172.21.15.22:0/2628934037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a41003d0 0x7f65a419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 >> 172.21.15.22:0/2628934037 conn(0x7f65a40772f0 msgr2=0x7f65a40687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 shutdown_connections 2024-03-09T00:10:48.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.733+0000 7f65a88d1640 1 -- 172.21.15.22:0/2628934037 wait complete. 2024-03-09T00:10:48.740 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:10:48.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:10:48.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.948+0000 7ff08ad9d640 1 -- 172.21.15.22:0/2998896720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410df80 msgr2=0x7ff08411cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.948+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/2998896720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410df80 0x7ff08411cae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009ca0 tx=0x7ff07c02c120 comp rx=0 tx=0).stop 2024-03-09T00:10:48.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.950+0000 7ff08ad9d640 1 -- 172.21.15.22:0/2998896720 shutdown_connections 2024-03-09T00:10:48.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.950+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/2998896720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410df80 0x7ff08411cae0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.950+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/2998896720 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff08410d3f0 0x7ff08410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.950+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/2998896720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff08410f890 0x7ff08410fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.950+0000 7ff08ad9d640 1 -- 172.21.15.22:0/2998896720 >> 172.21.15.22:0/2998896720 conn(0x7ff08406c1d0 msgr2=0x7ff08406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:48.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.953+0000 7ff08ad9d640 1 -- 172.21.15.22:0/2998896720 shutdown_connections 2024-03-09T00:10:48.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.955+0000 7ff08ad9d640 1 -- 172.21.15.22:0/2998896720 wait complete. 2024-03-09T00:10:48.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.956+0000 7ff08ad9d640 1 Processor -- start 2024-03-09T00:10:48.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.956+0000 7ff08ad9d640 1 -- start start 2024-03-09T00:10:48.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff08ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410d3f0 0x7ff0841ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff08ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff08410df80 0x7ff0841ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff08ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff08410f890 0x7ff0841a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff08ad9d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0841202d0 con 0x7ff08410df80 2024-03-09T00:10:48.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff08ad9d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084120150 con 0x7ff08410d3f0 2024-03-09T00:10:48.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff08ad9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084120450 con 0x7ff08410f890 2024-03-09T00:10:48.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff08a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff08410f890 0x7ff0841a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.957+0000 7ff089d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410d3f0 0x7ff0841ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410d3f0 0x7ff0841ac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:43390/0 (socket says 172.21.15.22:43390) 2024-03-09T00:10:48.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 -- 172.21.15.22:0/1646591324 learned_addr learned my addr 172.21.15.22:0/1646591324 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:48.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 -- 172.21.15.22:0/1646591324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff08410f890 msgr2=0x7ff0841a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff08410f890 0x7ff0841a65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 -- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff08410df80 msgr2=0x7ff0841ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:48.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff08410df80 0x7ff0841ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:48.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 -- 172.21.15.22:0/1646591324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0841a6d50 con 0x7ff08410d3f0 2024-03-09T00:10:48.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.958+0000 7ff089d9b640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410d3f0 0x7ff0841ac420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c920 tx=0x7ff07400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.959+0000 7ff07affd640 1 -- 172.21.15.22:0/1646591324 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074007c70 con 0x7ff08410d3f0 2024-03-09T00:10:48.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.959+0000 7ff07affd640 1 -- 172.21.15.22:0/1646591324 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff074007e10 con 0x7ff08410d3f0 2024-03-09T00:10:48.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.959+0000 7ff07affd640 1 -- 172.21.15.22:0/1646591324 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0740188a0 con 0x7ff08410d3f0 2024-03-09T00:10:48.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.959+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0841a7040 con 0x7ff08410d3f0 2024-03-09T00:10:48.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.959+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0841b32e0 con 0x7ff08410d3f0 2024-03-09T00:10:48.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.960+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff068005290 con 0x7ff08410d3f0 2024-03-09T00:10:48.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.962+0000 7ff07affd640 1 -- 172.21.15.22:0/1646591324 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff07400fbc0 con 0x7ff08410d3f0 2024-03-09T00:10:48.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.963+0000 7ff07affd640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0700a2a30 0x7ff0700a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:48.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.963+0000 7ff07affd640 1 -- 172.21.15.22:0/1646591324 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7ff0740cf340 con 0x7ff08410d3f0 2024-03-09T00:10:48.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.963+0000 7ff08959a640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0700a2a30 0x7ff0700a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:48.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.964+0000 7ff08959a640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0700a2a30 0x7ff0700a4ef0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff0800058c0 tx=0x7ff080005850 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:48.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:48.969+0000 7ff07affd640 1 -- 172.21.15.22:0/1646591324 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff074014030 con 0x7ff08410d3f0 2024-03-09T00:10:49.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 42901 12.34 MB/sec execute 56 sec latency 350.254 ms 2024-03-09T00:10:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.205+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff068005580 con 0x7ff08410d3f0 2024-03-09T00:10:49.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.207+0000 7ff07affd640 1 -- 172.21.15.22:0/1646591324 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff07409bd30 con 0x7ff08410d3f0 2024-03-09T00:10:49.202 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:10:49.203 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:10:14.710788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:10:49.203 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-03-09T00:10:49.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.208+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0700a2a30 msgr2=0x7ff0700a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.208+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0700a2a30 0x7ff0700a4ef0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff0800058c0 tx=0x7ff080005850 comp rx=0 tx=0).stop 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.208+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410d3f0 msgr2=0x7ff0841ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.208+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410d3f0 0x7ff0841ac420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c920 tx=0x7ff07400cde0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 shutdown_connections 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff0700a2a30 0x7ff0700a4ef0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff08410f890 0x7ff0841a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff08410df80 0x7ff0841ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 --2- 172.21.15.22:0/1646591324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff08410d3f0 0x7ff0841ac420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 >> 172.21.15.22:0/1646591324 conn(0x7ff08406c1d0 msgr2=0x7ff084070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 shutdown_connections 2024-03-09T00:10:49.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.209+0000 7ff08ad9d640 1 -- 172.21.15.22:0/1646591324 wait complete. 2024-03-09T00:10:49.216 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:10:49.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:10:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.332+0000 7fd07b93b640 1 -- 172.21.15.22:0/3870029781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741003f0 msgr2=0x7fd0741007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.332+0000 7fd07b93b640 1 --2- 172.21.15.22:0/3870029781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741003f0 0x7fd0741007d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009900 tx=0x7fd06c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 -- 172.21.15.22:0/3870029781 shutdown_connections 2024-03-09T00:10:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 --2- 172.21.15.22:0/3870029781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0741054c0 0x7fd07410d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 --2- 172.21.15.22:0/3870029781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd074100da0 0x7fd074104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 --2- 172.21.15.22:0/3870029781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741003f0 0x7fd0741007d0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 -- 172.21.15.22:0/3870029781 >> 172.21.15.22:0/3870029781 conn(0x7fd074076570 msgr2=0x7fd074076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 -- 172.21.15.22:0/3870029781 shutdown_connections 2024-03-09T00:10:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 -- 172.21.15.22:0/3870029781 wait complete. 2024-03-09T00:10:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 Processor -- start 2024-03-09T00:10:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.333+0000 7fd07b93b640 1 -- start start 2024-03-09T00:10:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0741003f0 0x7fd07419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741054c0 0x7fd07419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd074197950 0x7fd074197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd074110550 con 0x7fd0741054c0 2024-03-09T00:10:49.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741103d0 con 0x7fd0741003f0 2024-03-09T00:10:49.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741106d0 con 0x7fd074197950 2024-03-09T00:10:49.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741054c0 0x7fd07419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741054c0 0x7fd07419dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38744/0 (socket says 172.21.15.22:38744) 2024-03-09T00:10:49.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 -- 172.21.15.22:0/2717039982 learned_addr learned my addr 172.21.15.22:0/2717039982 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd079ed8640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd074197950 0x7fd074197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd0796d7640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0741003f0 0x7fd07419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 -- 172.21.15.22:0/2717039982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd074197950 msgr2=0x7fd074197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd074197950 0x7fd074197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 -- 172.21.15.22:0/2717039982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0741003f0 msgr2=0x7fd07419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0741003f0 0x7fd07419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 -- 172.21.15.22:0/2717039982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074198470 con 0x7fd0741054c0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd0796d7640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0741003f0 0x7fd07419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741054c0 0x7fd07419dce0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fd068005fd0 tx=0x7fd06800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd0727fc640 1 -- 172.21.15.22:0/2717039982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068017070 con 0x7fd0741054c0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd0727fc640 1 -- 172.21.15.22:0/2717039982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd068009c30 con 0x7fd0741054c0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd074198760 con 0x7fd0741054c0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd0727fc640 1 -- 172.21.15.22:0/2717039982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068010460 con 0x7fd0741054c0 2024-03-09T00:10:49.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.334+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0741a49b0 con 0x7fd0741054c0 2024-03-09T00:10:49.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.336+0000 7fd0727fc640 1 -- 172.21.15.22:0/2717039982 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd068003e70 con 0x7fd0741054c0 2024-03-09T00:10:49.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 Processor -- start 2024-03-09T00:10:49.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 -- start start 2024-03-09T00:10:49.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd0727fc640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd05c0a2970 0x7fd05c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4d17f0 0x559b5a4de570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4deab0 0x559b5a4e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd0727fc640 1 -- 172.21.15.22:0/2717039982 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd0680cef20 con 0x7fd0741054c0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5a498360 con 0x559b5a4d17f0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd079ed8640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd05c0a2970 0x7fd05c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5a434ec0 con 0x559b5a4deab0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5a3ddd30 con 0x559b5a4d0ed0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd078ed6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4d17f0 0x559b5a4de570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd0796d7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd078ed6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4d17f0 0x559b5a4de570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38758/0 (socket says 172.21.15.22:38758) 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.337+0000 7fd078ed6640 1 -- 172.21.15.22:0/2853196735 learned_addr learned my addr 172.21.15.22:0/2853196735 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd079ed8640 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4deab0 0x559b5a4e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd078ed6640 1 -- 172.21.15.22:0/2853196735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 msgr2=0x559b5a4d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd078ed6640 1 -- 172.21.15.22:0/2853196735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4deab0 msgr2=0x559b5a4e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4deab0 0x559b5a4e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd078ed6640 1 -- 172.21.15.22:0/2853196735 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5a3dcbc0 con 0x559b5a4d17f0 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd079ed8640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd05c0a2970 0x7fd05c0a4e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd064005fd0 tx=0x7fd064011810 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4d17f0 0x559b5a4de570 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fd0680a3400 tx=0x7fd06809b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebf404da1319646 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd05b7fe640 1 -- 172.21.15.22:0/2853196735 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06809f2e0 con 0x559b5a4d17f0 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd05b7fe640 1 -- 172.21.15.22:0/2853196735 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0680a3930 con 0x559b5a4d17f0 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.338+0000 7fd05b7fe640 1 -- 172.21.15.22:0/2853196735 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06809a5e0 con 0x559b5a4d17f0 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2853196735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4d17f0 msgr2=0x559b5a4de570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4d17f0 0x559b5a4de570 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fd0680a3400 tx=0x7fd06809b950 comp rx=0 tx=0).stop 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2853196735 shutdown_connections 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4deab0 0x559b5a4e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4d17f0 0x559b5a4de570 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2853196735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2853196735 >> 172.21.15.22:0/2853196735 conn(0x559b5a4cb8d0 msgr2=0x559b5a4cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2853196735 shutdown_connections 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2853196735 wait complete. 2024-03-09T00:10:49.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 -- start start 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.339+0000 7fd07e5fa740 1 Processor -- start 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d17f0 0x559b5a4dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5a3d34a0 con 0x559b5a4deab0 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5a430f60 con 0x559b5a4d17f0 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5a4e1870 con 0x559b5a4d0ed0 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 10 client.0 _ll_register_callbacks cb 0x559b5a42ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd0796d7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38766/0 (socket says 172.21.15.22:38766) 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 -- 172.21.15.22:0/313117470 learned_addr learned my addr 172.21.15.22:0/313117470 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd078ed6640 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d17f0 0x559b5a4dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 -- 172.21.15.22:0/313117470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 msgr2=0x559b5a4dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4dbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 -- 172.21.15.22:0/313117470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d17f0 msgr2=0x559b5a4dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d17f0 0x559b5a4dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 -- 172.21.15.22:0/313117470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b5a1493f0 con 0x559b5a4deab0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd079ed8640 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dc5d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd064004120 tx=0x7fd064004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdebca748ecffa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd058ff9640 1 -- 172.21.15.22:0/313117470 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064017040 con 0x559b5a4deab0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0680a3930 con 0x559b5a4deab0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.340+0000 7fd058ff9640 1 -- 172.21.15.22:0/313117470 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd06401e680 con 0x559b5a4deab0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd058ff9640 1 -- 172.21.15.22:0/313117470 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06401e880 con 0x559b5a4deab0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 10 client.20538 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x559b5a2137d0 con 0x559b5a4deab0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd058ff9640 1 -- 172.21.15.22:0/313117470 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0680a3930 con 0x559b5a4deab0 2024-03-09T00:10:49.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd058ff9640 1 -- 172.21.15.22:0/313117470 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd064027ca0 con 0x559b5a4deab0 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 10 client.20538 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 10 client.20538 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 20 client.20538 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a4eed80 0x559b5a4f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 4 client.20538 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b5a442400 con 0x559b5a4eed80 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.341+0000 7fd0796d7640 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a4eed80 0x559b5a4f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.342+0000 7fd058ff9640 10 client.20538 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.342+0000 7fd0796d7640 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a4eed80 0x559b5a4f1170 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a6fa264c460038f server_cookie=3c67ded09d0c1cad in_seq=0 out_seq=0 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.342+0000 7fd058ff9640 1 -- 172.21.15.22:0/313117470 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b5a442400 con 0x559b5a4eed80 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.342+0000 7fd058ff9640 10 client.20538 handle_command_reply: tid=0 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.342+0000 7fd07e5fa740 1 client.20538 shutdown 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 msgr2=0x559b5a4dc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dc5d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd064004120 tx=0x7fd064004ab0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 shutdown_connections 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a4eed80 0x559b5a4f1170 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dc5d0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd058ff9640 0 client.20538 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d17f0 0x559b5a4dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/313117470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d0ed0 0x559b5a4dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 >> 172.21.15.22:0/313117470 conn(0x559b5a4cb8d0 msgr2=0x559b5a4cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 shutdown_connections 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 1 -- 172.21.15.22:0/313117470 wait complete. 2024-03-09T00:10:49.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.343+0000 7fd07e5fa740 20 client.20538 trim_cache size 0 max 16384 2024-03-09T00:10:49.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 Processor -- start 2024-03-09T00:10:49.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 -- start start 2024-03-09T00:10:49.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741aea70 con 0x559b5a4deab0 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741ac610 con 0x559b5a4d0ed0 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741aa6d0 con 0x559b5a4d17f0 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38776/0 (socket says 172.21.15.22:38776) 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd0796d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 -- 172.21.15.22:0/3894572382 learned_addr learned my addr 172.21.15.22:0/3894572382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd078ed6640 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 -- 172.21.15.22:0/3894572382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 msgr2=0x559b5a4d9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4d9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 -- 172.21.15.22:0/3894572382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 msgr2=0x559b5a4d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 -- 172.21.15.22:0/3894572382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5a4f6320 con 0x559b5a4deab0 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd0796d7640 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.356+0000 7fd079ed8640 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dbb10 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd06400a010 tx=0x7fd064009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b9ceaa0479245e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd05affd640 1 -- 172.21.15.22:0/3894572382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064017050 con 0x559b5a4deab0 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd05affd640 1 -- 172.21.15.22:0/3894572382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd064019070 con 0x559b5a4deab0 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd05affd640 1 -- 172.21.15.22:0/3894572382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064015070 con 0x559b5a4deab0 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 -- 172.21.15.22:0/3894572382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 msgr2=0x559b5a4dbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dbb10 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd06400a010 tx=0x7fd064009500 comp rx=0 tx=0).stop 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 -- 172.21.15.22:0/3894572382 shutdown_connections 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a4dbb10 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4d9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/3894572382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 -- 172.21.15.22:0/3894572382 >> 172.21.15.22:0/3894572382 conn(0x559b5a4cb8d0 msgr2=0x559b5a4da460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 -- 172.21.15.22:0/3894572382 shutdown_connections 2024-03-09T00:10:49.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.357+0000 7fd07e5fa740 1 -- 172.21.15.22:0/3894572382 wait complete. 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 -- start start 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 Processor -- start 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4e5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a53f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741aacd0 con 0x559b5a4deab0 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741abfb0 con 0x559b5a4d0ed0 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741ab2f0 con 0x559b5a4d17f0 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd07e5fa740 10 client.0 _ll_register_callbacks cb 0x7fd068097430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd079ed8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a53f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd0796d7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd079ed8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a53f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38784/0 (socket says 172.21.15.22:38784) 2024-03-09T00:10:49.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd079ed8640 1 -- 172.21.15.22:0/2627984774 learned_addr learned my addr 172.21.15.22:0/2627984774 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.358+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd0796d7640 1 -- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 msgr2=0x559b5a4e5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd0796d7640 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4e5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd0796d7640 1 -- 172.21.15.22:0/2627984774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 msgr2=0x559b5a53f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd0796d7640 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a53f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd0796d7640 1 -- 172.21.15.22:0/2627984774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b5a53f7b0 con 0x559b5a4d0ed0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd079ed8640 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a53f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd0796d7640 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4e4d10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036040 tx=0x7fd06c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293c3c643e514e80 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd058ff9640 1 -- 172.21.15.22:0/2627984774 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c048070 con 0x559b5a4d0ed0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd064019070 con 0x559b5a4d0ed0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd058ff9640 1 -- 172.21.15.22:0/2627984774 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fd06c044d70 con 0x559b5a4d0ed0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd058ff9640 1 -- 172.21.15.22:0/2627984774 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06c054050 con 0x559b5a4d0ed0 2024-03-09T00:10:49.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd07e5fa740 10 client.29747 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x559b5a546b40 con 0x559b5a4d0ed0 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.359+0000 7fd058ff9640 1 -- 172.21.15.22:0/2627984774 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd064019070 con 0x559b5a4d0ed0 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd058ff9640 1 -- 172.21.15.22:0/2627984774 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd06c0442b0 con 0x559b5a4d0ed0 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 10 client.29747 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 10 client.29747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 20 client.29747 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a5471f0 0x559b5a549610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 4 client.29747 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b5a43c750 con 0x559b5a5471f0 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a5471f0 0x559b5a549610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd078ed6640 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a5471f0 0x559b5a549610 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31dbb559d752c10 server_cookie=73838d2eb5479b52 in_seq=0 out_seq=0 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd058ff9640 10 client.29747 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:49.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd058ff9640 1 -- 172.21.15.22:0/2627984774 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b5a43c750 con 0x559b5a5471f0 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd058ff9640 10 client.29747 handle_command_reply: tid=0 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 1 client.29747 shutdown 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 msgr2=0x559b5a4e4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.360+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4e4d10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036040 tx=0x7fd06c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 shutdown_connections 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x559b5a5471f0 0x559b5a549610 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559b5a4deab0 0x559b5a53f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559b5a4d17f0 0x559b5a4e5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 --2- 172.21.15.22:0/2627984774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b5a4d0ed0 0x559b5a4e4d10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd058ff9640 0 client.29747 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 >> 172.21.15.22:0/2627984774 conn(0x559b5a4cb8d0 msgr2=0x559b5a4ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 shutdown_connections 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 1 -- 172.21.15.22:0/2627984774 wait complete. 2024-03-09T00:10:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07e5fa740 20 client.29747 trim_cache size 0 max 16384 2024-03-09T00:10:49.368 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:10:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd05c0a2970 msgr2=0x7fd05c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd05c0a2970 0x7fd05c0a4e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd064005fd0 tx=0x7fd064011810 comp rx=0 tx=0).stop 2024-03-09T00:10:49.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741054c0 msgr2=0x7fd07419dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741054c0 0x7fd07419dce0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fd068005fd0 tx=0x7fd06800dde0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 shutdown_connections 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd05c0a2970 0x7fd05c0a4e30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd074197950 0x7fd074197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0741054c0 0x7fd07419dce0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 --2- 172.21.15.22:0/2717039982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0741003f0 0x7fd07419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 >> 172.21.15.22:0/2717039982 conn(0x7fd074076570 msgr2=0x7fd074069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 shutdown_connections 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.361+0000 7fd07b93b640 1 -- 172.21.15.22:0/2717039982 wait complete. 2024-03-09T00:10:49.370 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:10:49.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:10:49.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.489+0000 7fade4a59640 1 -- 172.21.15.22:0/2786019783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 msgr2=0x7fade00ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.489+0000 7fade4a59640 1 --2- 172.21.15.22:0/2786019783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade00ffde0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009900 tx=0x7fadd802bc30 comp rx=0 tx=0).stop 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 -- 172.21.15.22:0/2786019783 shutdown_connections 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 --2- 172.21.15.22:0/2786019783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade0106670 0x7fade010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 --2- 172.21.15.22:0/2786019783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade01003b0 0x7fade0105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 --2- 172.21.15.22:0/2786019783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade00ffde0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 -- 172.21.15.22:0/2786019783 >> 172.21.15.22:0/2786019783 conn(0x7fade00766e0 msgr2=0x7fade00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 -- 172.21.15.22:0/2786019783 shutdown_connections 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 -- 172.21.15.22:0/2786019783 wait complete. 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 Processor -- start 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.490+0000 7fade4a59640 1 -- start start 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0106670 0x7fade019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade0197a40 0x7fade0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0111720 con 0x7fade00ffa00 2024-03-09T00:10:49.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01115a0 con 0x7fade0197a40 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01118a0 con 0x7fade0106670 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade019d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38802/0 (socket says 172.21.15.22:38802) 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 -- 172.21.15.22:0/947838562 learned_addr learned my addr 172.21.15.22:0/947838562 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadddd9b640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0106670 0x7fade019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadded9d640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade0197a40 0x7fade0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 -- 172.21.15.22:0/947838562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0106670 msgr2=0x7fade019dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0106670 0x7fade019dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 -- 172.21.15.22:0/947838562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade0197a40 msgr2=0x7fade0197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade0197a40 0x7fade0197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 -- 172.21.15.22:0/947838562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0198560 con 0x7fade00ffa00 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadddd9b640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0106670 0x7fade019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadde59c640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade019d840 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fadd802c240 tx=0x7fadd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadd77fe640 1 -- 172.21.15.22:0/947838562 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8043020 con 0x7fade00ffa00 2024-03-09T00:10:49.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadd77fe640 1 -- 172.21.15.22:0/947838562 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd80054e0 con 0x7fade00ffa00 2024-03-09T00:10:49.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade01987f0 con 0x7fade00ffa00 2024-03-09T00:10:49.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fadd77fe640 1 -- 172.21.15.22:0/947838562 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8003b30 con 0x7fade00ffa00 2024-03-09T00:10:49.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.491+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade01a49f0 con 0x7fade00ffa00 2024-03-09T00:10:49.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.494+0000 7fadd77fe640 1 -- 172.21.15.22:0/947838562 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fadd8046020 con 0x7fade00ffa00 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.494+0000 7fade763a740 1 Processor -- start 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.494+0000 7fade763a740 1 -- start start 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fade763a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fade763a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d17160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fade763a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d176a0 0x55c4f4d19a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fade763a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4cd5360 con 0x55c4f4d176a0 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fade763a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4c71ec0 con 0x55c4f4d0eac0 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fade763a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4c1ad30 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d176a0 0x55c4f4d19a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadd77fe640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadcc0a2970 0x7fadcc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d176a0 0x55c4f4d19a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38814/0 (socket says 172.21.15.22:38814) 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadde59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d17160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 -- 172.21.15.22:0/3150564278 learned_addr learned my addr 172.21.15.22:0/3150564278 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadded9d640 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadded9d640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadcc0a2970 0x7fadcc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 -- 172.21.15.22:0/3150564278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 msgr2=0x55c4f4d0e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadd77fe640 1 -- 172.21.15.22:0/947838562 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fadd80f10f0 con 0x7fade00ffa00 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d0e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 -- 172.21.15.22:0/3150564278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 msgr2=0x55c4f4d17160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d17160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 -- 172.21.15.22:0/3150564278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f4c19bc0 con 0x55c4f4d176a0 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadddd9b640 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d176a0 0x55c4f4d19a90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009fd0 tx=0x7fadc80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de68d8eef15476b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.495+0000 7fadded9d640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadcc0a2970 0x7fadcc0a4e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009990 tx=0x7fadd00096a0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fadd4ff9640 1 -- 172.21.15.22:0/3150564278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8004310 con 0x55c4f4d176a0 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fadd4ff9640 1 -- 172.21.15.22:0/3150564278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc80044b0 con 0x55c4f4d176a0 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fadd4ff9640 1 -- 172.21.15.22:0/3150564278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80035f0 con 0x55c4f4d176a0 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 -- 172.21.15.22:0/3150564278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d176a0 msgr2=0x55c4f4d19a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d176a0 0x55c4f4d19a90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009fd0 tx=0x7fadc80118d0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 -- 172.21.15.22:0/3150564278 shutdown_connections 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d176a0 0x55c4f4d19a90 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d17160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 --2- 172.21.15.22:0/3150564278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 -- 172.21.15.22:0/3150564278 >> 172.21.15.22:0/3150564278 conn(0x55c4f4d088d0 msgr2=0x55c4f4d08cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 -- 172.21.15.22:0/3150564278 shutdown_connections 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.496+0000 7fade763a740 1 -- 172.21.15.22:0/3150564278 wait complete. 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 -- start start 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 Processor -- start 2024-03-09T00:10:49.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d14620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d14b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d176a0 0x55c4f4d150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4c104a0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4c6df60 con 0x55c4f4d0eac0 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4d1a490 con 0x55c4f4d176a0 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fade763a740 10 client.0 _ll_register_callbacks cb 0x55c4f4c6bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d14620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d14620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38824/0 (socket says 172.21.15.22:38824) 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 -- 172.21.15.22:0/2233967977 learned_addr learned my addr 172.21.15.22:0/2233967977 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadddd9b640 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d14b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadded9d640 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d176a0 0x55c4f4d150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 -- 172.21.15.22:0/2233967977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d176a0 msgr2=0x55c4f4d150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d176a0 0x55c4f4d150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 -- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 msgr2=0x55c4f4d14b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d14b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.497+0000 7fadde59c640 1 -- 172.21.15.22:0/2233967977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4f49863f0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fadde59c640 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d14620 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fadd80bd400 tx=0x7fadd8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f1c761daeb2184 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fadc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fadc5ffb640 1 -- 172.21.15.22:0/2233967977 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8004c80 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fadc5ffb640 1 -- 172.21.15.22:0/2233967977 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fadd80ee6d0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc80044b0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fadc5ffb640 1 -- 172.21.15.22:0/2233967977 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd80ee8d0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 10 client.20556 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c4f4a507d0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fadc5ffb640 1 -- 172.21.15.22:0/2233967977 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc80044b0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fadc5ffb640 1 -- 172.21.15.22:0/2233967977 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fadd80c1170 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 10 client.20556 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 10 client.20556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:10:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 20 client.20556 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d276c0 0x55c4f4d29af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 4 client.20556 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.498+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4f4c7f400 con 0x55c4f4d276c0 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.499+0000 7fadddd9b640 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d276c0 0x55c4f4d29af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.499+0000 7fadc5ffb640 10 client.20556 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.499+0000 7fadddd9b640 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d276c0 0x55c4f4d29af0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1141f1f9bc17425 server_cookie=c8332dcf0f3f88ae in_seq=0 out_seq=0 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fadc5ffb640 1 -- 172.21.15.22:0/2233967977 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4f4c7f400 con 0x55c4f4d276c0 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fadc5ffb640 10 client.20556 handle_command_reply: tid=0 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 client.20556 shutdown 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0e1a0 msgr2=0x55c4f4d14620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d14620 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fadd80bd400 tx=0x7fadd8009560 comp rx=0 tx=0).stop 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 shutdown_connections 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d276c0 0x55c4f4d29af0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d176a0 0x55c4f4d150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d14b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fadc5ffb640 0 client.20556 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 --2- 172.21.15.22:0/2233967977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d14620 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 >> 172.21.15.22:0/2233967977 conn(0x55c4f4d088d0 msgr2=0x55c4f4d0a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 shutdown_connections 2024-03-09T00:10:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 1 -- 172.21.15.22:0/2233967977 wait complete. 2024-03-09T00:10:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.500+0000 7fade763a740 20 client.20556 trim_cache size 0 max 16384 2024-03-09T00:10:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 Processor -- start 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 -- start start 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d16d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01b0d50 con 0x55c4f4d0eac0 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01ae8f0 con 0x55c4f4d176a0 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fade763a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01ac9b0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d13660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38836/0 (socket says 172.21.15.22:38836) 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fadddd9b640 1 -- 172.21.15.22:0/809470267 learned_addr learned my addr 172.21.15.22:0/809470267 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fadde59c640 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d16d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fadded9d640 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fadddd9b640 1 -- 172.21.15.22:0/809470267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 msgr2=0x55c4f4d16d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.512+0000 7fadddd9b640 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d16d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadddd9b640 1 -- 172.21.15.22:0/809470267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 msgr2=0x55c4f4d11900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadddd9b640 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d11900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadddd9b640 1 -- 172.21.15.22:0/809470267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f4d19a90 con 0x55c4f4d0eac0 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadded9d640 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadddd9b640 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d13660 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x55c4f4d09a60 tx=0x7fadc8004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c8b0b37c70a21d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadc7fff640 1 -- 172.21.15.22:0/809470267 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc802dd80 con 0x55c4f4d0eac0 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadc7fff640 1 -- 172.21.15.22:0/809470267 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc8032070 con 0x55c4f4d0eac0 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fadc7fff640 1 -- 172.21.15.22:0/809470267 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800f740 con 0x55c4f4d0eac0 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fade763a740 1 -- 172.21.15.22:0/809470267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 msgr2=0x55c4f4d13660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fade763a740 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d13660 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x55c4f4d09a60 tx=0x7fadc8004ba0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fade763a740 1 -- 172.21.15.22:0/809470267 shutdown_connections 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fade763a740 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fade763a740 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d13660 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fade763a740 1 --2- 172.21.15.22:0/809470267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d16d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.513+0000 7fade763a740 1 -- 172.21.15.22:0/809470267 >> 172.21.15.22:0/809470267 conn(0x55c4f4d088d0 msgr2=0x55c4f4d12f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 -- 172.21.15.22:0/809470267 shutdown_connections 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 -- 172.21.15.22:0/809470267 wait complete. 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 -- start start 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 Processor -- start 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d1dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01acfb0 con 0x55c4f4d0eac0 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01ae290 con 0x55c4f4d176a0 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01ad5d0 con 0x55c4f4d0e1a0 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fade763a740 10 client.0 _ll_register_callbacks cb 0x7fadc8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:49.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d1dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fadddd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d1dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38852/0 (socket says 172.21.15.22:38852) 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fadddd9b640 1 -- 172.21.15.22:0/270170546 learned_addr learned my addr 172.21.15.22:0/270170546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fadde59c640 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.514+0000 7fadded9d640 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadddd9b640 1 -- 172.21.15.22:0/270170546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 msgr2=0x55c4f4d1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadddd9b640 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadddd9b640 1 -- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 msgr2=0x55c4f4d7dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadddd9b640 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d7dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadddd9b640 1 -- 172.21.15.22:0/270170546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4f4d7e0f0 con 0x55c4f4d0eac0 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadddd9b640 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d1dc60 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fadc8011dd0 tx=0x7fadc8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667a8d56d5c1cfdd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:49.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadc5ffb640 1 -- 172.21.15.22:0/270170546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc802dd80 con 0x55c4f4d0eac0 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadc5ffb640 1 -- 172.21.15.22:0/270170546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fadc8004660 con 0x55c4f4d0eac0 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc8032070 con 0x55c4f4d0eac0 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadc5ffb640 1 -- 172.21.15.22:0/270170546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc8048050 con 0x55c4f4d0eac0 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 10 client.20562 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c4f4d7f2f0 con 0x55c4f4d0eac0 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadc5ffb640 1 -- 172.21.15.22:0/270170546 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc8032070 con 0x55c4f4d0eac0 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fadc5ffb640 1 -- 172.21.15.22:0/270170546 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fadc800fae0 con 0x55c4f4d0eac0 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 10 client.20562 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 10 client.20562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 20 client.20562 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d7f880 0x55c4f4d81ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 4 client.20562 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.515+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c4f4c79750 con 0x55c4f4d7f880 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fadde59c640 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d7f880 0x55c4f4d81ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fadc5ffb640 10 client.20562 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fadde59c640 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d7f880 0x55c4f4d81ca0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33b479b6cc748c3c server_cookie=3ffcc49bd7ad6e07 in_seq=0 out_seq=0 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fadc5ffb640 1 -- 172.21.15.22:0/270170546 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c4f4c79750 con 0x55c4f4d7f880 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fadc5ffb640 10 client.20562 handle_command_reply: tid=0 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 client.20562 shutdown 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 msgr2=0x55c4f4d1dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d1dc60 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fadc8011dd0 tx=0x7fadc8009f60 comp rx=0 tx=0).stop 2024-03-09T00:10:49.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 shutdown_connections 2024-03-09T00:10:49.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c4f4d7f880 0x55c4f4d81ca0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4f4d176a0 0x55c4f4d7dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c4f4d0eac0 0x55c4f4d1dc60 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fadc5ffb640 0 client.20562 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:10:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 --2- 172.21.15.22:0/270170546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c4f4d0e1a0 0x55c4f4d1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.516+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 >> 172.21.15.22:0/270170546 conn(0x55c4f4d088d0 msgr2=0x55c4f4d07100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 shutdown_connections 2024-03-09T00:10:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade763a740 1 -- 172.21.15.22:0/270170546 wait complete. 2024-03-09T00:10:49.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade763a740 20 client.20562 trim_cache size 0 max 16384 2024-03-09T00:10:49.522 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:10:49.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadcc0a2970 msgr2=0x7fadcc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadcc0a2970 0x7fadcc0a4e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009990 tx=0x7fadd00096a0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 msgr2=0x7fade019d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade019d840 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fadd802c240 tx=0x7fadd80042f0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 shutdown_connections 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fadcc0a2970 0x7fadcc0a4e30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fade0197a40 0x7fade0197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade0106670 0x7fade019dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 --2- 172.21.15.22:0/947838562 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade00ffa00 0x7fade019d840 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 >> 172.21.15.22:0/947838562 conn(0x7fade00766e0 msgr2=0x7fade00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 shutdown_connections 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.517+0000 7fade4a59640 1 -- 172.21.15.22:0/947838562 wait complete. 2024-03-09T00:10:49.525 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:10:49.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 -- 172.21.15.22:0/564113473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90941062d0 msgr2=0x7f90941066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 --2- 172.21.15.22:0/564113473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90941062d0 0x7f90941066b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f9084009900 tx=0x7f908402bc80 comp rx=0 tx=0).stop 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 -- 172.21.15.22:0/564113473 shutdown_connections 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 --2- 172.21.15.22:0/564113473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9094100e00 0x7f909410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 --2- 172.21.15.22:0/564113473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9094100440 0x7f90941008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 --2- 172.21.15.22:0/564113473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90941062d0 0x7f90941066b0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 -- 172.21.15.22:0/564113473 >> 172.21.15.22:0/564113473 conn(0x7f9094076fd0 msgr2=0x7f90940773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 -- 172.21.15.22:0/564113473 shutdown_connections 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.639+0000 7f90998ea640 1 -- 172.21.15.22:0/564113473 wait complete. 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 Processor -- start 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 -- start start 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9094100440 0x7f909419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9094100e00 0x7f909419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90941062d0 0x7f9094197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941105d0 con 0x7f9094100e00 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094110450 con 0x7f9094100440 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90998ea640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094110750 con 0x7f90941062d0 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9094100e00 0x7f909419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9094100e00 0x7f909419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38876/0 (socket says 172.21.15.22:38876) 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 -- 172.21.15.22:0/2167913809 learned_addr learned my addr 172.21.15.22:0/2167913809 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f90937fe640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90941062d0 0x7f9094197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f9092ffd640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9094100440 0x7f909419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 -- 172.21.15.22:0/2167913809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90941062d0 msgr2=0x7f9094197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90941062d0 0x7f9094197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 -- 172.21.15.22:0/2167913809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9094100440 msgr2=0x7f909419d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9094100440 0x7f909419d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 -- 172.21.15.22:0/2167913809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094198110 con 0x7f9094100e00 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f9092ffd640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9094100440 0x7f909419d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.640+0000 7f908bfff640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9094100e00 0x7f909419dd20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9f0 tx=0x7f907c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.641+0000 7f90988e8640 1 -- 172.21.15.22:0/2167913809 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00cd80 con 0x7f9094100e00 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.641+0000 7f90988e8640 1 -- 172.21.15.22:0/2167913809 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f907c010910 con 0x7f9094100e00 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.641+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094198400 con 0x7f9094100e00 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.641+0000 7f90988e8640 1 -- 172.21.15.22:0/2167913809 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c018990 con 0x7f9094100e00 2024-03-09T00:10:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.641+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90941a46a0 con 0x7f9094100e00 2024-03-09T00:10:49.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f90988e8640 1 -- 172.21.15.22:0/2167913809 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f907c010ac0 con 0x7f9094100e00 2024-03-09T00:10:49.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 Processor -- start 2024-03-09T00:10:49.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 -- start start 2024-03-09T00:10:49.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f90988e8640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f90800a2970 0x7f90800a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e15580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c26ddc360 con 0x557c26e1e6a0 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c26d78ec0 con 0x557c26e15ac0 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.643+0000 7f909c5a4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c26d21d30 con 0x557c26e151a0 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f90937fe640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f90800a2970 0x7f90800a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f90988e8640 1 -- 172.21.15.22:0/2167913809 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f907c0cf6a0 con 0x7f9094100e00 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f90937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e15580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:43546/0 (socket says 172.21.15.22:43546) 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 -- 172.21.15.22:0/1058444598 learned_addr learned my addr 172.21.15.22:0/1058444598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 -- 172.21.15.22:0/1058444598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 msgr2=0x557c26e15580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 --2- 172.21.15.22:0/1058444598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e15580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 -- 172.21.15.22:0/1058444598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 msgr2=0x557c26e20a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 --2- 172.21.15.22:0/1058444598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e20a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 -- 172.21.15.22:0/1058444598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c26d20bc0 con 0x557c26e15ac0 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f90937fe640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f90800a2970 0x7f90800a4e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f908c005fd0 tx=0x7f908c009f10 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f908bfff640 1 --2- 172.21.15.22:0/1058444598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1e160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f907c0a3b20 tx=0x7f907c09c070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb8f40fa65e41fc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f9089ffb640 1 -- 172.21.15.22:0/1058444598 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c09b8b0 con 0x557c26e15ac0 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f9089ffb640 1 -- 172.21.15.22:0/1058444598 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f907c09fa00 con 0x557c26e15ac0 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.644+0000 7f9089ffb640 1 -- 172.21.15.22:0/1058444598 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0860b0 con 0x557c26e15ac0 2024-03-09T00:10:49.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 -- 172.21.15.22:0/1058444598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 msgr2=0x557c26e1e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1058444598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1e160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f907c0a3b20 tx=0x7f907c09c070 comp rx=0 tx=0).stop 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 -- 172.21.15.22:0/1058444598 shutdown_connections 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1058444598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e20a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1058444598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1e160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1058444598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e15580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 -- 172.21.15.22:0/1058444598 >> 172.21.15.22:0/1058444598 conn(0x557c26e0f8d0 msgr2=0x557c26e0fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 -- 172.21.15.22:0/1058444598 shutdown_connections 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.645+0000 7f909c5a4740 1 -- 172.21.15.22:0/1058444598 wait complete. 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 -- start start 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 Processor -- start 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e1b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e1c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c26d174a0 con 0x557c26e1e6a0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c26d74f60 con 0x557c26e15ac0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c26e21490 con 0x557c26e151a0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f909c5a4740 10 client.0 _ll_register_callbacks cb 0x557c26d72d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f90937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e1c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e1b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e1b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:36026/0 (socket says 172.21.15.22:36026) 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f90937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e1c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38896/0 (socket says 172.21.15.22:38896) 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f9092ffd640 1 -- 172.21.15.22:0/1496655500 learned_addr learned my addr 172.21.15.22:0/1496655500 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f908bfff640 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f90937fe640 1 -- 172.21.15.22:0/1496655500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 msgr2=0x557c26e1b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f90937fe640 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e1b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f90937fe640 1 -- 172.21.15.22:0/1496655500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 msgr2=0x557c26e1bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.646+0000 7f90937fe640 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f90937fe640 1 -- 172.21.15.22:0/1496655500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c26a8d3f0 con 0x557c26e1e6a0 2024-03-09T00:10:49.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f908bfff640 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f90937fe640 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e1c180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f908c00fc20 tx=0x7f908c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce2d3328ba49570 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f9072ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f9072ffd640 1 -- 172.21.15.22:0/1496655500 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c024070 con 0x557c26e1e6a0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f907c09fa00 con 0x557c26e1e6a0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f9072ffd640 1 -- 172.21.15.22:0/1496655500 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f908c003970 con 0x557c26e1e6a0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f9072ffd640 1 -- 172.21.15.22:0/1496655500 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908c033050 con 0x557c26e1e6a0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 10 client.20574 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x557c26b577d0 con 0x557c26e1e6a0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f9072ffd640 1 -- 172.21.15.22:0/1496655500 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f907c09fa00 con 0x557c26e1e6a0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f9072ffd640 1 -- 172.21.15.22:0/1496655500 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f908c020600 con 0x557c26e1e6a0 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 10 client.20574 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 10 client.20574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:49.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 20 client.20574 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e2e920 0x557c26e30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 4 client.20574 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.647+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c26d86400 con 0x557c26e2e920 2024-03-09T00:10:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.648+0000 7f9092ffd640 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e2e920 0x557c26e30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.648+0000 7f9092ffd640 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e2e920 0x557c26e30d40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cba4350fe7fcfb server_cookie=19c33ae7d9941b24 in_seq=0 out_seq=0 2024-03-09T00:10:49.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.648+0000 7f9072ffd640 10 client.20574 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.651+0000 7f9072ffd640 1 -- 172.21.15.22:0/1496655500 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c26d86400 con 0x557c26e2e920 2024-03-09T00:10:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.651+0000 7f9072ffd640 10 client.20574 handle_command_reply: tid=0 2024-03-09T00:10:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.651+0000 7f909c5a4740 1 client.20574 shutdown 2024-03-09T00:10:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 msgr2=0x557c26e1c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e1c180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f908c00fc20 tx=0x7f908c01d000 comp rx=0 tx=0).stop 2024-03-09T00:10:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 shutdown_connections 2024-03-09T00:10:49.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e2e920 0x557c26e30d40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e1e6a0 0x557c26e1c180 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e15ac0 0x557c26e1bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f9072ffd640 0 client.20574 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 --2- 172.21.15.22:0/1496655500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e151a0 0x557c26e1b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 >> 172.21.15.22:0/1496655500 conn(0x557c26e0f8d0 msgr2=0x557c26e11620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 shutdown_connections 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 1 -- 172.21.15.22:0/1496655500 wait complete. 2024-03-09T00:10:49.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.652+0000 7f909c5a4740 20 client.20574 trim_cache size 0 max 16384 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 Processor -- start 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 -- start start 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e151a0 0x557c26e1de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e15ac0 0x557c26e19fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e1e6a0 0x557c26e1a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941b0b90 con 0x557c26e151a0 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941ae730 con 0x557c26e1e6a0 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f909c5a4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941ac7f0 con 0x557c26e15ac0 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e151a0 0x557c26e1de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e151a0 0x557c26e1de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38904/0 (socket says 172.21.15.22:38904) 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f9092ffd640 1 -- 172.21.15.22:0/4253757112 learned_addr learned my addr 172.21.15.22:0/4253757112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.664+0000 7f90937fe640 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e1e6a0 0x557c26e1a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f908bfff640 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e15ac0 0x557c26e19fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f9092ffd640 1 -- 172.21.15.22:0/4253757112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e15ac0 msgr2=0x557c26e19fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f9092ffd640 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e15ac0 0x557c26e19fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f9092ffd640 1 -- 172.21.15.22:0/4253757112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e1e6a0 msgr2=0x557c26e1a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f9092ffd640 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e1e6a0 0x557c26e1a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f9092ffd640 1 -- 172.21.15.22:0/4253757112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c26e35f50 con 0x557c26e151a0 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f908bfff640 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e15ac0 0x557c26e19fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f9092ffd640 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e151a0 0x557c26e1de30 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f9084000c00 tx=0x7f9084005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa6c79a4c5d9567 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f90897fa640 1 -- 172.21.15.22:0/4253757112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084045de0 con 0x557c26e151a0 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f90897fa640 1 -- 172.21.15.22:0/4253757112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f908404a070 con 0x557c26e151a0 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.665+0000 7f90897fa640 1 -- 172.21.15.22:0/4253757112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840457c0 con 0x557c26e151a0 2024-03-09T00:10:49.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 -- 172.21.15.22:0/4253757112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e151a0 msgr2=0x557c26e1de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e151a0 0x557c26e1de30 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f9084000c00 tx=0x7f9084005b50 comp rx=0 tx=0).stop 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 -- 172.21.15.22:0/4253757112 shutdown_connections 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e1e6a0 0x557c26e1a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e15ac0 0x557c26e19fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 --2- 172.21.15.22:0/4253757112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e151a0 0x557c26e1de30 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 -- 172.21.15.22:0/4253757112 >> 172.21.15.22:0/4253757112 conn(0x557c26e0f8d0 msgr2=0x557c26e169e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 -- 172.21.15.22:0/4253757112 shutdown_connections 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 -- 172.21.15.22:0/4253757112 wait complete. 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 -- start start 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.666+0000 7f909c5a4740 1 Processor -- start 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e151a0 0x557c26e240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e15ac0 0x557c26e24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f909c5a4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e1e6a0 0x557c26e24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f909c5a4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941acdf0 con 0x557c26e15ac0 2024-03-09T00:10:49.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f909c5a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941ae0d0 con 0x557c26e151a0 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f909c5a4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941ad410 con 0x557c26e1e6a0 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f909c5a4740 10 client.0 _ll_register_callbacks cb 0x7f907c097430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e15ac0 0x557c26e24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e151a0 0x557c26e240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e15ac0 0x557c26e24600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38916/0 (socket says 172.21.15.22:38916) 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f9092ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e151a0 0x557c26e240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:43586/0 (socket says 172.21.15.22:43586) 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f90937fe640 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e1e6a0 0x557c26e24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 -- 172.21.15.22:0/182343221 learned_addr learned my addr 172.21.15.22:0/182343221 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 -- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e1e6a0 msgr2=0x557c26e24b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e1e6a0 0x557c26e24b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 -- 172.21.15.22:0/182343221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e151a0 msgr2=0x557c26e240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e151a0 0x557c26e240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 -- 172.21.15.22:0/182343221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c26e84e50 con 0x557c26e15ac0 2024-03-09T00:10:49.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f9092ffd640 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e151a0 0x557c26e240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f908bfff640 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e15ac0 0x557c26e24600 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f907c09b5b0 tx=0x7f907c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ef575992e3ba19 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f9072ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.667+0000 7f9072ffd640 1 -- 172.21.15.22:0/182343221 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b3070 con 0x557c26e15ac0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f9072ffd640 1 -- 172.21.15.22:0/182343221 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f907c0860b0 con 0x557c26e15ac0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f908404a070 con 0x557c26e15ac0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f9072ffd640 1 -- 172.21.15.22:0/182343221 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f907c0862b0 con 0x557c26e15ac0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 10 client.20580 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x557c26e86100 con 0x557c26e15ac0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f9072ffd640 1 -- 172.21.15.22:0/182343221 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f908404a070 con 0x557c26e15ac0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f9072ffd640 1 -- 172.21.15.22:0/182343221 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f907c09cbe0 con 0x557c26e15ac0 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 10 client.20580 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 10 client.20580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 20 client.20580 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e86430 0x557c26e88820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 4 client.20580 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:10:49.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c26d80750 con 0x557c26e86430 2024-03-09T00:10:49.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.668+0000 7f9092ffd640 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e86430 0x557c26e88820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:49.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.669+0000 7f9072ffd640 10 client.20580 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:49.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.669+0000 7f9092ffd640 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e86430 0x557c26e88820 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac740cc93c4c11a server_cookie=21bc092a790bca0a in_seq=0 out_seq=0 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f9072ffd640 1 -- 172.21.15.22:0/182343221 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c26d80750 con 0x557c26e86430 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f9072ffd640 10 client.20580 handle_command_reply: tid=0 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 client.20580 shutdown 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e15ac0 msgr2=0x557c26e24600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e15ac0 0x557c26e24600 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f907c09b5b0 tx=0x7f907c009700 comp rx=0 tx=0).stop 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 shutdown_connections 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x557c26e86430 0x557c26e88820 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c26e1e6a0 0x557c26e24b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557c26e15ac0 0x557c26e24600 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f9072ffd640 0 client.20580 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:49.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 --2- 172.21.15.22:0/182343221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c26e151a0 0x557c26e240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 >> 172.21.15.22:0/182343221 conn(0x557c26e0f8d0 msgr2=0x557c26e169e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.670+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 shutdown_connections 2024-03-09T00:10:49.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f909c5a4740 1 -- 172.21.15.22:0/182343221 wait complete. 2024-03-09T00:10:49.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f909c5a4740 20 client.20580 trim_cache size 0 max 16384 2024-03-09T00:10:49.672 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:10:49.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f90800a2970 msgr2=0x7f90800a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f90800a2970 0x7f90800a4e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f908c005fd0 tx=0x7f908c009f10 comp rx=0 tx=0).stop 2024-03-09T00:10:49.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9094100e00 msgr2=0x7f909419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:49.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9094100e00 0x7f909419dd20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9f0 tx=0x7f907c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 shutdown_connections 2024-03-09T00:10:49.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f90800a2970 0x7f90800a4e30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90941062d0 0x7f9094197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9094100e00 0x7f909419dd20 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 --2- 172.21.15.22:0/2167913809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9094100440 0x7f909419d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:49.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 >> 172.21.15.22:0/2167913809 conn(0x7f9094076fd0 msgr2=0x7f9094069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:49.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 shutdown_connections 2024-03-09T00:10:49.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:49.671+0000 7f90998ea640 1 -- 172.21.15.22:0/2167913809 wait complete. 2024-03-09T00:10:50.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 43233 12.51 MB/sec execute 57 sec latency 351.291 ms 2024-03-09T00:10:50.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:50 smithi022 ceph-mon[31894]: pgmap v517: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 446 op/s 2024-03-09T00:10:50.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:50 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1646591324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:10:50.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:50 smithi062 ceph-mon[36129]: pgmap v517: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 446 op/s 2024-03-09T00:10:50.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:50 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1646591324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:10:50.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:50 smithi192 ceph-mon[35995]: pgmap v517: 129 pgs: 129 active+clean; 361 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 446 op/s 2024-03-09T00:10:50.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:50 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1646591324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:10:50.679 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:10:50.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:10:50.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:10:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.815+0000 7f1b8aea4640 1 -- 172.21.15.22:0/226537664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410c8a0 msgr2=0x7f1b841108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.815+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/226537664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410c8a0 0x7f1b841108e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009be0 tx=0x7f1b7c02bef0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.816+0000 7f1b8aea4640 1 -- 172.21.15.22:0/226537664 shutdown_connections 2024-03-09T00:10:50.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.816+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/226537664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b84110fc0 0x7f1b84118b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.816+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/226537664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410c8a0 0x7f1b841108e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.816+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/226537664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b8410bef0 0x7f1b8410c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.816+0000 7f1b8aea4640 1 -- 172.21.15.22:0/226537664 >> 172.21.15.22:0/226537664 conn(0x7f1b8406c2f0 msgr2=0x7f1b8410a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:50.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.819+0000 7f1b8aea4640 1 -- 172.21.15.22:0/226537664 shutdown_connections 2024-03-09T00:10:50.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.819+0000 7f1b8aea4640 1 -- 172.21.15.22:0/226537664 wait complete. 2024-03-09T00:10:50.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.819+0000 7f1b8aea4640 1 Processor -- start 2024-03-09T00:10:50.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b8aea4640 1 -- start start 2024-03-09T00:10:50.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b8aea4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410bef0 0x7f1b841a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b8aea4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b84110fc0 0x7f1b841a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b8aea4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b841a6e80 0x7f1b841a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b8aea4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8411c080 con 0x7f1b841a6e80 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b8aea4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8411bf00 con 0x7f1b84110fc0 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b8aea4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8411c200 con 0x7f1b8410bef0 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b89441640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b841a6e80 0x7f1b841a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b89441640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b841a6e80 0x7f1b841a7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38162/0 (socket says 172.21.15.22:38162) 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b89441640 1 -- 172.21.15.22:0/2548520321 learned_addr learned my addr 172.21.15.22:0/2548520321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b88c40640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410bef0 0x7f1b841a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.820+0000 7f1b83fff640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b84110fc0 0x7f1b841a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.821+0000 7f1b88c40640 1 -- 172.21.15.22:0/2548520321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b84110fc0 msgr2=0x7f1b841a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.821+0000 7f1b88c40640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b84110fc0 0x7f1b841a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.821+0000 7f1b88c40640 1 -- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b841a6e80 msgr2=0x7f1b841a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.821+0000 7f1b88c40640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b841a6e80 0x7f1b841a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.821+0000 7f1b88c40640 1 -- 172.21.15.22:0/2548520321 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b841a7970 con 0x7f1b8410bef0 2024-03-09T00:10:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.821+0000 7f1b89441640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b841a6e80 0x7f1b841a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:50.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.822+0000 7f1b88c40640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410bef0 0x7f1b841a25c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e790 tx=0x7f1b7400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:50.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.822+0000 7f1b81ffb640 1 -- 172.21.15.22:0/2548520321 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400eea0 con 0x7f1b8410bef0 2024-03-09T00:10:50.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.823+0000 7f1b81ffb640 1 -- 172.21.15.22:0/2548520321 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b7400ccc0 con 0x7f1b8410bef0 2024-03-09T00:10:50.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.823+0000 7f1b81ffb640 1 -- 172.21.15.22:0/2548520321 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74018780 con 0x7f1b8410bef0 2024-03-09T00:10:50.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.824+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b841a7c60 con 0x7f1b8410bef0 2024-03-09T00:10:50.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.824+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b841af3d0 con 0x7f1b8410bef0 2024-03-09T00:10:50.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.829+0000 7f1b81ffb640 1 -- 172.21.15.22:0/2548520321 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1b7401f020 con 0x7f1b8410bef0 2024-03-09T00:10:50.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.830+0000 7f1b8db64740 1 Processor -- start 2024-03-09T00:10:50.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.830+0000 7f1b81ffb640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1b6c0a2a30 0x7f1b6c0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.830+0000 7f1b81ffb640 1 -- 172.21.15.22:0/2548520321 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1b740cf280 con 0x7f1b8410bef0 2024-03-09T00:10:50.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.831+0000 7f1b8db64740 1 -- start start 2024-03-09T00:10:50.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.831+0000 7f1b89441640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1b6c0a2a30 0x7f1b6c0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073ffed80 0x564073fff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564073fff6a0 0x564074007d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 0x56407400a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b8db64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564073f62b40 con 0x564073ffed80 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b8db64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564073f0bd80 con 0x564074008280 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b8db64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564073f5ebe0 con 0x564073fff6a0 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073ffed80 0x564073fff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073ffed80 0x564073fff160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38174/0 (socket says 172.21.15.22:38174) 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 -- 172.21.15.22:0/261004137 learned_addr learned my addr 172.21.15.22:0/261004137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b83fff640 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564073fff6a0 0x564074007d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 -- 172.21.15.22:0/261004137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564073fff6a0 msgr2=0x564074007d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564073fff6a0 0x564074007d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 -- 172.21.15.22:0/261004137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 msgr2=0x56407400a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 0x56407400a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 -- 172.21.15.22:0/261004137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564073d3f7e0 con 0x564073ffed80 2024-03-09T00:10:50.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b88c40640 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073ffed80 0x564073fff160 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1b7409b480 tx=0x7f1b7409af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2392231610330e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:50.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.832+0000 7f1b89441640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1b6c0a2a30 0x7f1b6c0a4ef0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1b78011550 tx=0x7f1b78011950 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:50.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.833+0000 7f1b6affd640 1 -- 172.21.15.22:0/261004137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7409f640 con 0x564073ffed80 2024-03-09T00:10:50.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.833+0000 7f1b6affd640 1 -- 172.21.15.22:0/261004137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b740a3c90 con 0x564073ffed80 2024-03-09T00:10:50.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.833+0000 7f1b6affd640 1 -- 172.21.15.22:0/261004137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740a3800 con 0x564073ffed80 2024-03-09T00:10:50.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 -- 172.21.15.22:0/261004137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073ffed80 msgr2=0x564073fff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073ffed80 0x564073fff160 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1b7409b480 tx=0x7f1b7409af10 comp rx=0 tx=0).stop 2024-03-09T00:10:50.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 -- 172.21.15.22:0/261004137 shutdown_connections 2024-03-09T00:10:50.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 0x56407400a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564073fff6a0 0x564074007d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 --2- 172.21.15.22:0/261004137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073ffed80 0x564073fff160 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 -- 172.21.15.22:0/261004137 >> 172.21.15.22:0/261004137 conn(0x564073ff94b0 msgr2=0x564073ff9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 -- 172.21.15.22:0/261004137 shutdown_connections 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 -- 172.21.15.22:0/261004137 wait complete. 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 -- start start 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.834+0000 7f1b8db64740 1 Processor -- start 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073ffed80 0x564074007d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x5640740057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b8db64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407400b070 con 0x564073fff6a0 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b8db64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564073f1f1c0 con 0x564073ffed80 2024-03-09T00:10:50.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b8db64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407400b1f0 con 0x564074008280 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b8db64740 10 client.0 _ll_register_callbacks cb 0x564073f0e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b88c40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073ffed80 0x564074007d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b88c40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073ffed80 0x564074007d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:37832/0 (socket says 172.21.15.22:37832) 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b88c40640 1 -- 172.21.15.22:0/3169832693 learned_addr learned my addr 172.21.15.22:0/3169832693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.835+0000 7f1b89441640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x5640740057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b83fff640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b88c40640 1 -- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 msgr2=0x5640740057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b88c40640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x5640740057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b88c40640 1 -- 172.21.15.22:0/3169832693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 msgr2=0x5640740050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b88c40640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b88c40640 1 -- 172.21.15.22:0/3169832693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564073d9ff70 con 0x564073ffed80 2024-03-09T00:10:50.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b83fff640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:10:50.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b88c40640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073ffed80 0x564074007d20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b7409b1f0 tx=0x7f1b74010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da60df44324407cb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:50.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.836+0000 7f1b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:50.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.837+0000 7f1b63fff640 1 -- 172.21.15.22:0/3169832693 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740b3070 con 0x564073ffed80 2024-03-09T00:10:50.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.837+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b740a3c90 con 0x564073ffed80 2024-03-09T00:10:50.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.837+0000 7f1b63fff640 1 -- 172.21.15.22:0/3169832693 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1b740a3a10 con 0x564073ffed80 2024-03-09T00:10:50.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.837+0000 7f1b63fff640 1 -- 172.21.15.22:0/3169832693 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b7409b4f0 con 0x564073ffed80 2024-03-09T00:10:50.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 10 client.29792 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x564073c37170 con 0x564073ffed80 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b63fff640 1 -- 172.21.15.22:0/3169832693 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b740a3c90 con 0x564073ffed80 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b63fff640 1 -- 172.21.15.22:0/3169832693 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1b7409aad0 con 0x564073ffed80 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 10 client.29792 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 10 client.29792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 20 client.29792 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56407400f4c0 0x564074011910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 4 client.29792 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.838+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564073f38960 con 0x56407400f4c0 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.839+0000 7f1b83fff640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56407400f4c0 0x564074011910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.840+0000 7f1b63fff640 10 client.29792 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:50.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.840+0000 7f1b83fff640 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56407400f4c0 0x564074011910 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7994d46850906b server_cookie=8b3e64092d3bdff9 in_seq=0 out_seq=0 2024-03-09T00:10:50.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b63fff640 1 -- 172.21.15.22:0/3169832693 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564073f38960 con 0x56407400f4c0 2024-03-09T00:10:50.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b63fff640 10 client.29792 handle_command_reply: tid=0 2024-03-09T00:10:50.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 client.29792 shutdown 2024-03-09T00:10:50.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073ffed80 msgr2=0x564074007d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073ffed80 0x564074007d20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b7409b1f0 tx=0x7f1b74010c60 comp rx=0 tx=0).stop 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 shutdown_connections 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56407400f4c0 0x564074011910 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x5640740057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 --2- 172.21.15.22:0/3169832693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073ffed80 0x564074007d20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 >> 172.21.15.22:0/3169832693 conn(0x564073ff94b0 msgr2=0x564073ffb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b63fff640 0 client.29792 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.842+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 shutdown_connections 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.843+0000 7f1b8db64740 1 -- 172.21.15.22:0/3169832693 wait complete. 2024-03-09T00:10:50.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.843+0000 7f1b8db64740 20 client.29792 trim_cache size 0 max 16384 2024-03-09T00:10:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.873+0000 7f1b8db64740 1 Processor -- start 2024-03-09T00:10:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.874+0000 7f1b8db64740 1 -- start start 2024-03-09T00:10:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.874+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.874+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 0x564074007d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.874+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640740003e0 0x564074000840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.874+0000 7f1b8db64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841b71c0 con 0x564073fff6a0 2024-03-09T00:10:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.874+0000 7f1b8db64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841b4d60 con 0x564074008280 2024-03-09T00:10:50.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.874+0000 7f1b8db64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841b2e20 con 0x5640740003e0 2024-03-09T00:10:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640740003e0 0x564074000840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640740003e0 0x564074000840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:37834/0 (socket says 172.21.15.22:37834) 2024-03-09T00:10:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 -- 172.21.15.22:0/593121153 learned_addr learned my addr 172.21.15.22:0/593121153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 -- 172.21.15.22:0/593121153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 msgr2=0x564074007d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:10:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 --2- 172.21.15.22:0/593121153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 0x564074007d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 -- 172.21.15.22:0/593121153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 msgr2=0x5640740077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:50.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 --2- 172.21.15.22:0/593121153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.875+0000 7f1b89441640 1 -- 172.21.15.22:0/593121153 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564074005b10 con 0x5640740003e0 2024-03-09T00:10:50.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.878+0000 7f1b89441640 1 --2- 172.21.15.22:0/593121153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640740003e0 0x564074000840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800f920 tx=0x7f1b7801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7912cfe69fd51236 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.879+0000 7f1b6a7fc640 1 -- 172.21.15.22:0/593121153 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78017030 con 0x5640740003e0 2024-03-09T00:10:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.879+0000 7f1b6a7fc640 1 -- 172.21.15.22:0/593121153 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b78019050 con 0x5640740003e0 2024-03-09T00:10:50.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.879+0000 7f1b6a7fc640 1 -- 172.21.15.22:0/593121153 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b780204e0 con 0x5640740003e0 2024-03-09T00:10:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.880+0000 7f1b8db64740 1 -- 172.21.15.22:0/593121153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640740003e0 msgr2=0x564074000840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.880+0000 7f1b8db64740 1 --2- 172.21.15.22:0/593121153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640740003e0 0x564074000840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800f920 tx=0x7f1b7801a000 comp rx=0 tx=0).stop 2024-03-09T00:10:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.881+0000 7f1b8db64740 1 -- 172.21.15.22:0/593121153 shutdown_connections 2024-03-09T00:10:50.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.881+0000 7f1b8db64740 1 --2- 172.21.15.22:0/593121153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640740003e0 0x564074000840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.881+0000 7f1b8db64740 1 --2- 172.21.15.22:0/593121153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564074008280 0x564074007d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.881+0000 7f1b8db64740 1 --2- 172.21.15.22:0/593121153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564073fff6a0 0x5640740077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.881+0000 7f1b8db64740 1 -- 172.21.15.22:0/593121153 >> 172.21.15.22:0/593121153 conn(0x564073ff94b0 msgr2=0x564073ff9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:50.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.881+0000 7f1b8db64740 1 -- 172.21.15.22:0/593121153 shutdown_connections 2024-03-09T00:10:50.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.883+0000 7f1b8db64740 1 -- 172.21.15.22:0/593121153 wait complete. 2024-03-09T00:10:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.883+0000 7f1b8db64740 1 -- start start 2024-03-09T00:10:50.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.883+0000 7f1b8db64740 1 Processor -- start 2024-03-09T00:10:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073fff6a0 0x564074016250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640740003e0 0x564074016790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b8db64740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x564074014230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b8db64740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841b3420 con 0x5640740003e0 2024-03-09T00:10:50.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b8db64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841b4700 con 0x564073fff6a0 2024-03-09T00:10:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b8db64740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841b3a40 con 0x564074008280 2024-03-09T00:10:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b8db64740 10 client.0 _ll_register_callbacks cb 0x7f1b740970d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b89441640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x564074014230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.884+0000 7f1b89441640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x564074014230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:37838/0 (socket says 172.21.15.22:37838) 2024-03-09T00:10:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b89441640 1 -- 172.21.15.22:0/815233467 learned_addr learned my addr 172.21.15.22:0/815233467 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b83fff640 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640740003e0 0x564074016790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b88c40640 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073fff6a0 0x564074016250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b88c40640 1 -- 172.21.15.22:0/815233467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 msgr2=0x564074014230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b88c40640 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x564074014230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b88c40640 1 -- 172.21.15.22:0/815233467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640740003e0 msgr2=0x564074016790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b88c40640 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640740003e0 0x564074016790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.885+0000 7f1b88c40640 1 -- 172.21.15.22:0/815233467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564074014a60 con 0x564073fff6a0 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.886+0000 7f1b88c40640 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073fff6a0 0x564074016250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b74005fd0 tx=0x7f1b7409bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5ceb0c7d25b5b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.886+0000 7f1b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.886+0000 7f1b63fff640 1 -- 172.21.15.22:0/815233467 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740b3070 con 0x564073fff6a0 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.886+0000 7f1b63fff640 1 -- 172.21.15.22:0/815233467 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1b740b4260 con 0x564073fff6a0 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.886+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b78019050 con 0x564073fff6a0 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.887+0000 7f1b63fff640 1 -- 172.21.15.22:0/815233467 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b7409b030 con 0x564073fff6a0 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.887+0000 7f1b8db64740 10 client.29795 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:50.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.887+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x564074080f80 con 0x564073fff6a0 2024-03-09T00:10:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.887+0000 7f1b63fff640 1 -- 172.21.15.22:0/815233467 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b78019050 con 0x564073fff6a0 2024-03-09T00:10:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.887+0000 7f1b63fff640 1 -- 172.21.15.22:0/815233467 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1b7409a940 con 0x564073fff6a0 2024-03-09T00:10:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b8db64740 10 client.29795 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b8db64740 10 client.29795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:50.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b8db64740 20 client.29795 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:50.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b8db64740 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564074081400 0x5640740837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:50.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b8db64740 4 client.29795 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:10:50.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564073ffff30 con 0x564074081400 2024-03-09T00:10:50.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b83fff640 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564074081400 0x5640740837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:50.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b63fff640 10 client.29795 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:50.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.888+0000 7f1b83fff640 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564074081400 0x5640740837f0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7c5620300550e2 server_cookie=51c77880b653f120 in_seq=0 out_seq=0 2024-03-09T00:10:50.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b63fff640 1 -- 172.21.15.22:0/815233467 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564073ffff30 con 0x564074081400 2024-03-09T00:10:50.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b63fff640 10 client.29795 handle_command_reply: tid=0 2024-03-09T00:10:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 client.29795 shutdown 2024-03-09T00:10:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073fff6a0 msgr2=0x564074016250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073fff6a0 0x564074016250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b74005fd0 tx=0x7f1b7409bd30 comp rx=0 tx=0).stop 2024-03-09T00:10:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 shutdown_connections 2024-03-09T00:10:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x564074081400 0x5640740837f0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564074008280 0x564074014230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640740003e0 0x564074016790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 --2- 172.21.15.22:0/815233467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564073fff6a0 0x564074016250 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b74005fd0 tx=0x7f1b7409bd30 comp rx=0 tx=0).stop 2024-03-09T00:10:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.896+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 >> 172.21.15.22:0/815233467 conn(0x564073ff94b0 msgr2=0x564073ffa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.897+0000 7f1b63fff640 0 client.29795 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.897+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 shutdown_connections 2024-03-09T00:10:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.897+0000 7f1b8db64740 1 -- 172.21.15.22:0/815233467 wait complete. 2024-03-09T00:10:50.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.897+0000 7f1b8db64740 20 client.29795 trim_cache size 0 max 16384 2024-03-09T00:10:50.894 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:10:50.894 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:10:50.894 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "c0feb732-e3b5-450f-be27-a0ad34ae8828", 2024-03-09T00:10:50.894 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:10:50.894 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:10:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1b6c0a2a30 msgr2=0x7f1b6c0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1b6c0a2a30 0x7f1b6c0a4ef0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1b78011550 tx=0x7f1b78011950 comp rx=0 tx=0).stop 2024-03-09T00:10:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410bef0 msgr2=0x7f1b841a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410bef0 0x7f1b841a25c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e790 tx=0x7f1b7400ec50 comp rx=0 tx=0).stop 2024-03-09T00:10:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 shutdown_connections 2024-03-09T00:10:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1b6c0a2a30 0x7f1b6c0a4ef0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b841a6e80 0x7f1b841a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b84110fc0 0x7f1b841a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 --2- 172.21.15.22:0/2548520321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b8410bef0 0x7f1b841a25c0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e790 tx=0x7f1b7400ec50 comp rx=0 tx=0).stop 2024-03-09T00:10:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.898+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 >> 172.21.15.22:0/2548520321 conn(0x7f1b8406c2f0 msgr2=0x7f1b8410f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.899+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 shutdown_connections 2024-03-09T00:10:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:50.899+0000 7f1b8aea4640 1 -- 172.21.15.22:0/2548520321 wait complete. 2024-03-09T00:10:50.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:10:50.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:10:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 -- 172.21.15.22:0/4128341545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6388102e50 msgr2=0x7f638810b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 --2- 172.21.15.22:0/4128341545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6388102e50 0x7f638810b330 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f6378009900 tx=0x7f637802bcd0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 -- 172.21.15.22:0/4128341545 shutdown_connections 2024-03-09T00:10:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 --2- 172.21.15.22:0/4128341545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f638810b900 0x7f638810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 --2- 172.21.15.22:0/4128341545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6388102e50 0x7f638810b330 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 --2- 172.21.15.22:0/4128341545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6388102530 0x7f6388102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 -- 172.21.15.22:0/4128341545 >> 172.21.15.22:0/4128341545 conn(0x7f6388076570 msgr2=0x7f6388076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 -- 172.21.15.22:0/4128341545 shutdown_connections 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.035+0000 7f639031b640 1 -- 172.21.15.22:0/4128341545 wait complete. 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 Processor -- start 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 -- start start 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6388102530 0x7f638819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6388102e50 0x7f638819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638810b900 0x7f6388197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388111670 con 0x7f638810b900 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881114f0 con 0x7f6388102530 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f639031b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881117f0 con 0x7f6388102e50 2024-03-09T00:10:51.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638810b900 0x7f6388197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638810b900 0x7f6388197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38230/0 (socket says 172.21.15.22:38230) 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 -- 172.21.15.22:0/324658962 learned_addr learned my addr 172.21.15.22:0/324658962 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638d8b6640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6388102e50 0x7f638819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e0b7640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6388102530 0x7f638819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 -- 172.21.15.22:0/324658962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6388102e50 msgr2=0x7f638819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6388102e50 0x7f638819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 -- 172.21.15.22:0/324658962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6388102530 msgr2=0x7f638819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6388102530 0x7f638819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.036+0000 7f638e8b8640 1 -- 172.21.15.22:0/324658962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388198110 con 0x7f638810b900 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.037+0000 7f638e8b8640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638810b900 0x7f6388197990 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f638000e9d0 tx=0x7f638000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.037+0000 7f637f7fe640 1 -- 172.21.15.22:0/324658962 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638000cd80 con 0x7f638810b900 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.037+0000 7f637f7fe640 1 -- 172.21.15.22:0/324658962 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6380005020 con 0x7f638810b900 2024-03-09T00:10:51.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.037+0000 7f637f7fe640 1 -- 172.21.15.22:0/324658962 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800116d0 con 0x7f638810b900 2024-03-09T00:10:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.037+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388198400 con 0x7f638810b900 2024-03-09T00:10:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.037+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63881a46a0 con 0x7f638810b900 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.039+0000 7f637f7fe640 1 -- 172.21.15.22:0/324658962 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f63800051d0 con 0x7f638810b900 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 Processor -- start 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 -- start start 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f637f7fe640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63740a29f0 0x7f63740a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f638d8b6640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63740a29f0 0x7f63740a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597acbd4e00 con 0x5597acc7ac60 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597acb7db10 con 0x5597acc72080 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f6392fdb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597acbd0ea0 con 0x5597acc71760 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f637f7fe640 1 -- 172.21.15.22:0/324658962 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f63800cf6e0 con 0x7f638810b900 2024-03-09T00:10:51.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f638e0b7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f638e8b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.040+0000 7f638e0b7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc71b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:37868/0 (socket says 172.21.15.22:37868) 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638e0b7640 1 -- 172.21.15.22:0/1359830349 learned_addr learned my addr 172.21.15.22:0/1359830349 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638d8b6640 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638e8b8640 1 -- 172.21.15.22:0/1359830349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 msgr2=0x5597acc71b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638e8b8640 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc71b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638e8b8640 1 -- 172.21.15.22:0/1359830349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 msgr2=0x5597acc7a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638e8b8640 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638e8b8640 1 -- 172.21.15.22:0/1359830349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597ac9b37d0 con 0x5597acc7ac60 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638d8b6640 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638d8b6640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63740a29f0 0x7f63740a4eb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f637802efd0 tx=0x7f637802ca40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f638e8b8640 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc7d050 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f63800cc9e0 tx=0x7f638009fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b65a976711d0ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:51.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f637cbfe640 1 -- 172.21.15.22:0/1359830349 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638001f040 con 0x5597acc7ac60 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f637cbfe640 1 -- 172.21.15.22:0/1359830349 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6380005540 con 0x5597acc7ac60 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.041+0000 7f637cbfe640 1 -- 172.21.15.22:0/1359830349 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63800cf4b0 con 0x5597acc7ac60 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 -- 172.21.15.22:0/1359830349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 msgr2=0x5597acc7d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc7d050 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f63800cc9e0 tx=0x7f638009fa40 comp rx=0 tx=0).stop 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 -- 172.21.15.22:0/1359830349 shutdown_connections 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc7d050 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1359830349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc71b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 -- 172.21.15.22:0/1359830349 >> 172.21.15.22:0/1359830349 conn(0x5597acc6be90 msgr2=0x5597acc6c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 -- 172.21.15.22:0/1359830349 shutdown_connections 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 -- 172.21.15.22:0/1359830349 wait complete. 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 -- start start 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.042+0000 7f6392fdb740 1 Processor -- start 2024-03-09T00:10:51.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc7a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc72080 0x5597acc77740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc7ac60 0x5597acc77c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597acc7da50 con 0x5597acc72080 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597acb910a0 con 0x5597acc7ac60 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597acc7dbd0 con 0x5597acc71760 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 10 client.0 _ll_register_callbacks cb 0x5597acb90a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc72080 0x5597acc77740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638e0b7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc7a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc72080 0x5597acc77740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38250/0 (socket says 172.21.15.22:38250) 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 -- 172.21.15.22:0/3549038098 learned_addr learned my addr 172.21.15.22:0/3549038098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638e8b8640 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc7ac60 0x5597acc77c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 -- 172.21.15.22:0/3549038098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 msgr2=0x5597acc7a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc7a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 -- 172.21.15.22:0/3549038098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc7ac60 msgr2=0x5597acc77c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc7ac60 0x5597acc77c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 -- 172.21.15.22:0/3549038098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597ac8e93f0 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638e8b8640 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc7ac60 0x5597acc77c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f638d8b6640 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc72080 0x5597acc77740 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f6378009900 tx=0x7f637802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68546cbd5c457ea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6371ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6371ffb640 1 -- 172.21.15.22:0/3549038098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378052060 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.043+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6380005540 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6371ffb640 1 -- 172.21.15.22:0/3549038098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f6378004ad0 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6371ffb640 1 -- 172.21.15.22:0/3549038098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6378059050 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 10 client.20604 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5597ac8a90f0 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6371ffb640 1 -- 172.21.15.22:0/3549038098 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6380005540 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6371ffb640 1 -- 172.21.15.22:0/3549038098 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6378003860 con 0x5597acc72080 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 10 client.20604 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 10 client.20604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 20 client.20604 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acc8ab60 0x5597acc8cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 4 client.20604 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597acbe2340 con 0x5597acc8ab60 2024-03-09T00:10:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.044+0000 7f638e0b7640 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acc8ab60 0x5597acc8cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.045+0000 7f6371ffb640 10 client.20604 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:51.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.045+0000 7f638e0b7640 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acc8ab60 0x5597acc8cfa0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4c5cee21a1fba8 server_cookie=aa214898b5c7c148 in_seq=0 out_seq=0 2024-03-09T00:10:51.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.049+0000 7f6371ffb640 1 -- 172.21.15.22:0/3549038098 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597acbe2340 con 0x5597acc8ab60 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.049+0000 7f6371ffb640 10 client.20604 handle_command_reply: tid=0 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.049+0000 7f6392fdb740 1 client.20604 shutdown 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc72080 msgr2=0x5597acc77740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc72080 0x5597acc77740 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f6378009900 tx=0x7f637802bcd0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 shutdown_connections 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acc8ab60 0x5597acc8cfa0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc7ac60 0x5597acc77c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc72080 0x5597acc77740 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 --2- 172.21.15.22:0/3549038098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc7a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6371ffb640 0 client.20604 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 >> 172.21.15.22:0/3549038098 conn(0x5597acc6be90 msgr2=0x5597acc6dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:51.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 shutdown_connections 2024-03-09T00:10:51.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 1 -- 172.21.15.22:0/3549038098 wait complete. 2024-03-09T00:10:51.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.050+0000 7f6392fdb740 20 client.20604 trim_cache size 0 max 16384 2024-03-09T00:10:51.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 Processor -- start 2024-03-09T00:10:51.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 -- start start 2024-03-09T00:10:51.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc79f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc77700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881b0c10 con 0x5597acc7ac60 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881ae7b0 con 0x5597acc72080 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f6392fdb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881ac870 con 0x5597acc71760 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f638e8b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f638e8b8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc77c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38254/0 (socket says 172.21.15.22:38254) 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f638e8b8640 1 -- 172.21.15.22:0/1459991656 learned_addr learned my addr 172.21.15.22:0/1459991656 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f638d8b6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc77700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.065+0000 7f638e0b7640 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc79f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f638e8b8640 1 -- 172.21.15.22:0/1459991656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 msgr2=0x5597acc79f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f638e8b8640 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc79f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f638e8b8640 1 -- 172.21.15.22:0/1459991656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 msgr2=0x5597acc77700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f638e8b8640 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc77700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f638e8b8640 1 -- 172.21.15.22:0/1459991656 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597acc71eb0 con 0x5597acc7ac60 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f638e0b7640 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc79f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f638e8b8640 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc77c40 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f6380097e00 tx=0x7f6380099eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f67171c94adda8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f6373fff640 1 -- 172.21.15.22:0/1459991656 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638001f040 con 0x5597acc7ac60 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f6373fff640 1 -- 172.21.15.22:0/1459991656 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6380098010 con 0x5597acc7ac60 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f6373fff640 1 -- 172.21.15.22:0/1459991656 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638009adf0 con 0x5597acc7ac60 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f6392fdb740 1 -- 172.21.15.22:0/1459991656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 msgr2=0x5597acc77c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.066+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc77c40 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f6380097e00 tx=0x7f6380099eb0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- 172.21.15.22:0/1459991656 shutdown_connections 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc7ac60 0x5597acc77c40 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc77700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 --2- 172.21.15.22:0/1459991656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc71760 0x5597acc79f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- 172.21.15.22:0/1459991656 >> 172.21.15.22:0/1459991656 conn(0x5597acc6be90 msgr2=0x5597acc91940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- 172.21.15.22:0/1459991656 shutdown_connections 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- 172.21.15.22:0/1459991656 wait complete. 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- start start 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 Processor -- start 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc71760 0x5597acc7e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc7ac60 0x5597acce00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881ace70 con 0x5597acc71760 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881ae150 con 0x5597acc72080 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881ad490 con 0x5597acc7ac60 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f6392fdb740 10 client.0 _ll_register_callbacks cb 0x7f637802c840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f638e0b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc71760 0x5597acc7e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f638e0b7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc71760 0x5597acc7e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38270/0 (socket says 172.21.15.22:38270) 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f638e0b7640 1 -- 172.21.15.22:0/2411043195 learned_addr learned my addr 172.21.15.22:0/2411043195 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f638d8b6640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.067+0000 7f638e8b8640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc7ac60 0x5597acce00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f638e0b7640 1 -- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc7ac60 msgr2=0x5597acce00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f638e0b7640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc7ac60 0x5597acce00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f638e0b7640 1 -- 172.21.15.22:0/2411043195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 msgr2=0x5597acc7e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f638e0b7640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f638e0b7640 1 -- 172.21.15.22:0/2411043195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597acce06a0 con 0x5597acc71760 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f638d8b6640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:51.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f638e0b7640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc71760 0x5597acc7e010 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f6384000c00 tx=0x7f6384011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2121c5d15bfdc6b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6371ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6371ffb640 1 -- 172.21.15.22:0/2411043195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638401f670 con 0x5597acc71760 2024-03-09T00:10:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6371ffb640 1 -- 172.21.15.22:0/2411043195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f638401f810 con 0x5597acc71760 2024-03-09T00:10:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6380098010 con 0x5597acc71760 2024-03-09T00:10:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6371ffb640 1 -- 172.21.15.22:0/2411043195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6384030050 con 0x5597acc71760 2024-03-09T00:10:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 10 client.20610 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:51.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5597acce19f0 con 0x5597acc71760 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6371ffb640 1 -- 172.21.15.22:0/2411043195 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6380098010 con 0x5597acc71760 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6371ffb640 1 -- 172.21.15.22:0/2411043195 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6384028680 con 0x5597acc71760 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 10 client.20610 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 10 client.20610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 20 client.20610 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acce2150 0x5597acce4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 4 client.20610 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.068+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5597acbdc690 con 0x5597acce2150 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.069+0000 7f638d8b6640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acce2150 0x5597acce4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:51.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.069+0000 7f638d8b6640 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acce2150 0x5597acce4570 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57dc6967afa810c9 server_cookie=2b6c4a707765c5dc in_seq=0 out_seq=0 2024-03-09T00:10:51.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.069+0000 7f6371ffb640 10 client.20610 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6371ffb640 1 -- 172.21.15.22:0/2411043195 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5597acbdc690 con 0x5597acce2150 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6371ffb640 10 client.20610 handle_command_reply: tid=0 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 client.20610 shutdown 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc71760 msgr2=0x5597acc7e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc71760 0x5597acc7e010 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f6384000c00 tx=0x7f6384011960 comp rx=0 tx=0).stop 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 shutdown_connections 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5597acce2150 0x5597acce4570 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597acc7ac60 0x5597acce00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5597acc72080 0x5597acc7e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 --2- 172.21.15.22:0/2411043195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5597acc71760 0x5597acc7e010 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6371ffb640 0 client.20610 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 >> 172.21.15.22:0/2411043195 conn(0x5597acc6be90 msgr2=0x5597acc77700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 shutdown_connections 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 1 -- 172.21.15.22:0/2411043195 wait complete. 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f6392fdb740 20 client.20610 trim_cache size 0 max 16384 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (639 inodes in the stack)", 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stdout: "c0feb732-e3b5-450f-be27-a0ad34ae8828": { 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:10:51.074 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "c0feb732-e3b5-450f-be27-a0ad34ae8828", 2024-03-09T00:10:51.075 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:10:51.075 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:10:51.075 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:10:51.075 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63740a29f0 msgr2=0x7f63740a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63740a29f0 0x7f63740a4eb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f637802efd0 tx=0x7f637802ca40 comp rx=0 tx=0).stop 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638810b900 msgr2=0x7f6388197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638810b900 0x7f6388197990 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f638000e9d0 tx=0x7f638000ee90 comp rx=0 tx=0).stop 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 shutdown_connections 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f63740a29f0 0x7f63740a4eb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638810b900 0x7f6388197990 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.073+0000 7f639031b640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6388102e50 0x7f638819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.074+0000 7f639031b640 1 --2- 172.21.15.22:0/324658962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6388102530 0x7f638819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.074+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 >> 172.21.15.22:0/324658962 conn(0x7f6388076570 msgr2=0x7f63880fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.074+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 shutdown_connections 2024-03-09T00:10:51.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:10:51.074+0000 7f639031b640 1 -- 172.21.15.22:0/324658962 wait complete. 2024-03-09T00:10:51.077 INFO:tasks.workunit.client.0.smithi022.stdout: 10 43566 12.62 MB/sec execute 58 sec latency 229.068 ms 2024-03-09T00:10:51.079 INFO:tasks.cephfs.filesystem:scrub status for tag:c0feb732-e3b5-450f-be27-a0ad34ae8828 - {'path': '/', 'tag': 'c0feb732-e3b5-450f-be27-a0ad34ae8828', 'options': 'recursive,force'} 2024-03-09T00:10:51.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:51 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:51.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:51 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:51.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:51 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:51.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:51 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:51.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:51 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:51.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:51 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:51.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:51 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:51.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:51 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:51.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:51 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:51.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:51 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:51.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:51 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:51.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:51 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 43893 12.76 MB/sec execute 59 sec latency 76.924 ms 2024-03-09T00:10:52.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: pgmap v518: 129 pgs: 129 active+clean; 376 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2024-03-09T00:10:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:52 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: pgmap v518: 129 pgs: 129 active+clean; 376 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2024-03-09T00:10:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:52 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: pgmap v518: 129 pgs: 129 active+clean; 376 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 572 op/s 2024-03-09T00:10:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:52 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:10:53.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 44124 12.70 MB/sec execute 60 sec latency 107.017 ms 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.265+0000 7feb60b6f640 1 -- 172.21.15.22:0/3925139350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c1073e0 msgr2=0x7feb5c1077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.265+0000 7feb60b6f640 1 --2- 172.21.15.22:0/3925139350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c1073e0 0x7feb5c1077c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009900 tx=0x7feb4c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 -- 172.21.15.22:0/3925139350 shutdown_connections 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 --2- 172.21.15.22:0/3925139350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb5c069310 0x7feb5c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 --2- 172.21.15.22:0/3925139350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb5c068970 0x7feb5c068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 --2- 172.21.15.22:0/3925139350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c1073e0 0x7feb5c1077c0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 -- 172.21.15.22:0/3925139350 >> 172.21.15.22:0/3925139350 conn(0x7feb5c077040 msgr2=0x7feb5c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 -- 172.21.15.22:0/3925139350 shutdown_connections 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 -- 172.21.15.22:0/3925139350 wait complete. 2024-03-09T00:10:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 Processor -- start 2024-03-09T00:10:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.266+0000 7feb60b6f640 1 -- start start 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb60b6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c068970 0x7feb5c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb60b6f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb5c069310 0x7feb5c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb60b6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb5c1073e0 0x7feb5c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb60b6f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c111660 con 0x7feb5c068970 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb60b6f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1114e0 con 0x7feb5c1073e0 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb60b6f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1117e0 con 0x7feb5c069310 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c068970 0x7feb5c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c068970 0x7feb5c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38288/0 (socket says 172.21.15.22:38288) 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 -- 172.21.15.22:0/1211375620 learned_addr learned my addr 172.21.15.22:0/1211375620 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb5c1073e0 0x7feb5c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb51d9b640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb5c069310 0x7feb5c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 -- 172.21.15.22:0/1211375620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb5c069310 msgr2=0x7feb5c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb5c069310 0x7feb5c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 -- 172.21.15.22:0/1211375620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb5c1073e0 msgr2=0x7feb5c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb5c1073e0 0x7feb5c197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 -- 172.21.15.22:0/1211375620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c198000 con 0x7feb5c068970 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb5c1073e0 0x7feb5c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb51d9b640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb5c069310 0x7feb5c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb5a59c640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c068970 0x7feb5c19d6d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7feb4c005940 tx=0x7feb4c02cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb53fff640 1 -- 172.21.15.22:0/1211375620 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c041070 con 0x7feb5c068970 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.267+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c198290 con 0x7feb5c068970 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.268+0000 7feb53fff640 1 -- 172.21.15.22:0/1211375620 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb4c004450 con 0x7feb5c068970 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.268+0000 7feb53fff640 1 -- 172.21.15.22:0/1211375620 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c034440 con 0x7feb5c068970 2024-03-09T00:10:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.268+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c1a4590 con 0x7feb5c068970 2024-03-09T00:10:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.270+0000 7feb53fff640 1 -- 172.21.15.22:0/1211375620 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7feb4c044070 con 0x7feb5c068970 2024-03-09T00:10:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.270+0000 7feb6381f740 1 Processor -- start 2024-03-09T00:10:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.270+0000 7feb6381f740 1 -- start start 2024-03-09T00:10:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.270+0000 7feb53fff640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feb480a2a90 0x7feb480a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb6381f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be845a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb6381f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be845fd0 0x5629be846450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb6381f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be851940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb6381f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be7aa5f0 con 0x5629be845fd0 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb6381f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be753930 con 0x5629be84f550 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb6381f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be7a6690 con 0x5629be846d40 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb53fff640 1 -- 172.21.15.22:0/1211375620 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7feb4c0f0d00 con 0x7feb5c068970 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feb480a2a90 0x7feb480a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be845fd0 0x5629be846450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be845fd0 0x5629be846450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38302/0 (socket says 172.21.15.22:38302) 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 -- 172.21.15.22:0/3633225687 learned_addr learned my addr 172.21.15.22:0/3633225687 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb5a59c640 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be845a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be851940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 -- 172.21.15.22:0/3633225687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 msgr2=0x5629be845a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be845a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 -- 172.21.15.22:0/3633225687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 msgr2=0x5629be851940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be851940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 -- 172.21.15.22:0/3633225687 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629be4bf210 con 0x5629be845fd0 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feb480a2a90 0x7feb480a4f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feb54009800 tx=0x7feb54012e80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb51d9b640 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be845fd0 0x5629be846450 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7feb4400bd00 tx=0x7feb4400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf0981f9b67dcc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb50d99640 1 -- 172.21.15.22:0/3633225687 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44005020 con 0x5629be845fd0 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb50d99640 1 -- 172.21.15.22:0/3633225687 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb440047a0 con 0x5629be845fd0 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.271+0000 7feb50d99640 1 -- 172.21.15.22:0/3633225687 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44011730 con 0x5629be845fd0 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 -- 172.21.15.22:0/3633225687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be845fd0 msgr2=0x5629be846450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be845fd0 0x5629be846450 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7feb4400bd00 tx=0x7feb4400f730 comp rx=0 tx=0).stop 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 -- 172.21.15.22:0/3633225687 shutdown_connections 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be851940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be845fd0 0x5629be846450 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 --2- 172.21.15.22:0/3633225687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be845a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 -- 172.21.15.22:0/3633225687 >> 172.21.15.22:0/3633225687 conn(0x5629be80d5b0 msgr2=0x5629be840cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 -- 172.21.15.22:0/3633225687 shutdown_connections 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.272+0000 7feb6381f740 1 -- 172.21.15.22:0/3633225687 wait complete. 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 -- start start 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 Processor -- start 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be845fd0 0x5629be84c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be846d40 0x5629be84ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be84cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be852280 con 0x5629be846d40 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be766d90 con 0x5629be84f550 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629be852400 con 0x5629be845fd0 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb6381f740 10 client.0 _ll_register_callbacks cb 0x5629be4bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb51d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be846d40 0x5629be84ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb51d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be846d40 0x5629be84ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38310/0 (socket says 172.21.15.22:38310) 2024-03-09T00:10:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be845fd0 0x5629be84c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb51d9b640 1 -- 172.21.15.22:0/2693871006 learned_addr learned my addr 172.21.15.22:0/2693871006 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.273+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be84cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb5a59c640 1 -- 172.21.15.22:0/2693871006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 msgr2=0x5629be84cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb5a59c640 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be84cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb5a59c640 1 -- 172.21.15.22:0/2693871006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be846d40 msgr2=0x5629be84ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb5a59c640 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be846d40 0x5629be84ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb5a59c640 1 -- 172.21.15.22:0/2693871006 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629be58b620 con 0x5629be845fd0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb51d9b640 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be846d40 0x5629be84ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb5a59c640 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be845fd0 0x5629be84c520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0ee170 tx=0x7feb4c0bcf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb32029c93d8dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb41ffb640 1 -- 172.21.15.22:0/2693871006 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c04b040 con 0x5629be845fd0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb41ffb640 1 -- 172.21.15.22:0/2693871006 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7feb4c0c5390 con 0x5629be845fd0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb440047a0 con 0x5629be845fd0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb41ffb640 1 -- 172.21.15.22:0/2693871006 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb4c0bd080 con 0x5629be845fd0 2024-03-09T00:10:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb6381f740 10 client.30001 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5629be852280 con 0x5629be845fd0 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.274+0000 7feb41ffb640 1 -- 172.21.15.22:0/2693871006 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb440047a0 con 0x5629be845fd0 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb41ffb640 1 -- 172.21.15.22:0/2693871006 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feb4c0dbb80 con 0x5629be845fd0 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb6381f740 10 client.30001 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb6381f740 10 client.30001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb6381f740 20 client.30001 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb6381f740 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be85f980 0x5629be861dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb6381f740 4 client.30001 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629be7b7b30 con 0x5629be85f980 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb51d9b640 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be85f980 0x5629be861dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb41ffb640 10 client.30001 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.275+0000 7feb51d9b640 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be85f980 0x5629be861dd0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45b072c9702a89 server_cookie=38c1637eddd17d3a in_seq=0 out_seq=0 2024-03-09T00:10:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb41ffb640 1 -- 172.21.15.22:0/2693871006 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629be7b7b30 con 0x5629be85f980 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb41ffb640 10 client.30001 handle_command_reply: tid=0 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 client.30001 shutdown 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be845fd0 msgr2=0x5629be84c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be845fd0 0x5629be84c520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0ee170 tx=0x7feb4c0bcf10 comp rx=0 tx=0).stop 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 shutdown_connections 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be85f980 0x5629be861dd0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be84f550 0x5629be84cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb41ffb640 0 client.30001 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be846d40 0x5629be84ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 --2- 172.21.15.22:0/2693871006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be845fd0 0x5629be84c520 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 >> 172.21.15.22:0/2693871006 conn(0x5629be80d5b0 msgr2=0x5629be842a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 shutdown_connections 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.278+0000 7feb6381f740 1 -- 172.21.15.22:0/2693871006 wait complete. 2024-03-09T00:10:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.279+0000 7feb6381f740 20 client.30001 trim_cache size 0 max 16384 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 Processor -- start 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 -- start start 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be84ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be856be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1b0ae0 con 0x5629be84f550 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1ae680 con 0x5629be845fd0 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb6381f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1ac740 con 0x5629be846d40 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be856be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be84ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be84ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:37914/0 (socket says 172.21.15.22:37914) 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 -- 172.21.15.22:0/228510422 learned_addr learned my addr 172.21.15.22:0/228510422 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be856be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38314/0 (socket says 172.21.15.22:38314) 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb51d9b640 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 -- 172.21.15.22:0/228510422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 msgr2=0x5629be8566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 -- 172.21.15.22:0/228510422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 msgr2=0x5629be856be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be856be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.292+0000 7feb5a59c640 1 -- 172.21.15.22:0/228510422 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629be8452a0 con 0x5629be845fd0 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb5a59c640 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be84ec50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000c00 tx=0x7feb4c02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e2f89cced7afa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb43fff640 1 -- 172.21.15.22:0/228510422 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c04b050 con 0x5629be845fd0 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb43fff640 1 -- 172.21.15.22:0/228510422 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb4c0c11d0 con 0x5629be845fd0 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb43fff640 1 -- 172.21.15.22:0/228510422 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0c5390 con 0x5629be845fd0 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb6381f740 1 -- 172.21.15.22:0/228510422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 msgr2=0x5629be84ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb6381f740 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be84ec50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000c00 tx=0x7feb4c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb6381f740 1 -- 172.21.15.22:0/228510422 shutdown_connections 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb6381f740 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be856be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb6381f740 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb6381f740 1 --2- 172.21.15.22:0/228510422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be84ec50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.293+0000 7feb6381f740 1 -- 172.21.15.22:0/228510422 >> 172.21.15.22:0/228510422 conn(0x5629be80d5b0 msgr2=0x5629be847830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 -- 172.21.15.22:0/228510422 shutdown_connections 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 -- 172.21.15.22:0/228510422 wait complete. 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 -- start start 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 Processor -- start 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be853080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be853b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1acd40 con 0x5629be84f550 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1ae020 con 0x5629be845fd0 2024-03-09T00:10:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1ad360 con 0x5629be846d40 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb6381f740 10 client.0 _ll_register_callbacks cb 0x7feb4400b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb5ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be853b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb5a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be853080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb5ad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be853b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38324/0 (socket says 172.21.15.22:38324) 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb5ad9d640 1 -- 172.21.15.22:0/2710567614 learned_addr learned my addr 172.21.15.22:0/2710567614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.294+0000 7feb51d9b640 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb5ad9d640 1 -- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 msgr2=0x5629be8535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb5ad9d640 1 -- 172.21.15.22:0/2710567614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 msgr2=0x5629be853080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be853080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb5ad9d640 1 -- 172.21.15.22:0/2710567614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629be8b5ef0 con 0x5629be84f550 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb5ad9d640 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be853b00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7feb540225e0 tx=0x7feb54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b1ce7b1a11018b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:10:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb41ffb640 1 -- 172.21.15.22:0/2710567614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54030060 con 0x5629be84f550 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb41ffb640 1 -- 172.21.15.22:0/2710567614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9523+0+0 (secure 0 0 0) 0x7feb540216c0 con 0x5629be84f550 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb4c0c11d0 con 0x5629be84f550 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb41ffb640 1 -- 172.21.15.22:0/2710567614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb54037050 con 0x5629be84f550 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb6381f740 10 client.20628 fetch_fsmap learned FSMap version 37 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5629be8b71e0 con 0x5629be84f550 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb41ffb640 1 -- 172.21.15.22:0/2710567614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb4c0c11d0 con 0x5629be84f550 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.295+0000 7feb41ffb640 1 -- 172.21.15.22:0/2710567614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feb5400f040 con 0x5629be84f550 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb6381f740 10 client.20628 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb6381f740 10 client.20628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb6381f740 20 client.20628 populate_metadata read hostname 'smithi022' 2024-03-09T00:10:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb6381f740 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be8b78f0 0x5629be8b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:10:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb6381f740 4 client.20628 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:10:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629be7b1e80 con 0x5629be8b78f0 2024-03-09T00:10:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb5a59c640 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be8b78f0 0x5629be8b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:10:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb41ffb640 10 client.20628 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:53.296+0000 7feb5a59c640 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be8b78f0 0x5629be8b9d10 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fdf9e5d04afbd01 server_cookie=d629d339cf3128ef in_seq=0 out_seq=0 2024-03-09T00:10:54.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 44228 12.62 MB/sec execute 61 sec latency 560.386 ms 2024-03-09T00:10:54.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:54 smithi022 ceph-mon[31894]: pgmap v519: 129 pgs: 129 active+clean; 376 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 308 op/s 2024-03-09T00:10:54.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:54 smithi062 ceph-mon[36129]: pgmap v519: 129 pgs: 129 active+clean; 376 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 308 op/s 2024-03-09T00:10:54.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:54 smithi192 ceph-mon[35995]: pgmap v519: 129 pgs: 129 active+clean; 376 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 308 op/s 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb41ffb640 1 -- 172.21.15.22:0/2710567614 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629be7b1e80 con 0x5629be8b78f0 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb41ffb640 10 client.20628 handle_command_reply: tid=0 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 client.20628 shutdown 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 msgr2=0x5629be853b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be853b00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7feb540225e0 tx=0x7feb54004ab0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 shutdown_connections 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5629be8b78f0 0x5629be8b9d10 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629be84f550 0x5629be853b00 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629be846d40 0x5629be8535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 --2- 172.21.15.22:0/2710567614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629be845fd0 0x5629be853080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 >> 172.21.15.22:0/2710567614 conn(0x5629be80d5b0 msgr2=0x5629be84ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb41ffb640 0 client.20628 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 shutdown_connections 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 1 -- 172.21.15.22:0/2710567614 wait complete. 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.735+0000 7feb6381f740 20 client.20628 trim_cache size 0 max 16384 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:10:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:10:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feb480a2a90 msgr2=0x7feb480a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feb480a2a90 0x7feb480a4f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feb54009800 tx=0x7feb54012e80 comp rx=0 tx=0).stop 2024-03-09T00:10:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c068970 msgr2=0x7feb5c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:10:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c068970 0x7feb5c19d6d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7feb4c005940 tx=0x7feb4c02cd10 comp rx=0 tx=0).stop 2024-03-09T00:10:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 shutdown_connections 2024-03-09T00:10:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7feb480a2a90 0x7feb480a4f50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb5c1073e0 0x7feb5c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb5c069310 0x7feb5c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 --2- 172.21.15.22:0/1211375620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb5c068970 0x7feb5c19d6d0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:10:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.736+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 >> 172.21.15.22:0/1211375620 conn(0x7feb5c077040 msgr2=0x7feb5c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:10:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.737+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 shutdown_connections 2024-03-09T00:10:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:10:54.737+0000 7feb60b6f640 1 -- 172.21.15.22:0/1211375620 wait complete. 2024-03-09T00:10:55.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 44353 12.56 MB/sec execute 62 sec latency 662.889 ms 2024-03-09T00:10:56.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 44547 12.50 MB/sec execute 63 sec latency 211.268 ms 2024-03-09T00:10:56.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:56 smithi022 ceph-mon[31894]: pgmap v520: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2024-03-09T00:10:56.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:56 smithi062 ceph-mon[36129]: pgmap v520: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2024-03-09T00:10:56.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:56 smithi192 ceph-mon[35995]: pgmap v520: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2024-03-09T00:10:57.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 44797 12.51 MB/sec execute 64 sec latency 188.106 ms 2024-03-09T00:10:58.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 45094 12.45 MB/sec execute 65 sec latency 292.420 ms 2024-03-09T00:10:58.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:10:58 smithi022 ceph-mon[31894]: pgmap v521: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 511 op/s 2024-03-09T00:10:58.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:10:58 smithi062 ceph-mon[36129]: pgmap v521: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 511 op/s 2024-03-09T00:10:58.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:10:58 smithi192 ceph-mon[35995]: pgmap v521: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 511 op/s 2024-03-09T00:10:59.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 45369 12.43 MB/sec execute 66 sec latency 154.175 ms 2024-03-09T00:11:00.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 45554 12.47 MB/sec execute 67 sec latency 262.065 ms 2024-03-09T00:11:00.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:00 smithi022 ceph-mon[31894]: pgmap v522: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 524 op/s 2024-03-09T00:11:00.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:00 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:11:00.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:00 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:11:00.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:00 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:11:00.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:00 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:11:00.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:00 smithi062 ceph-mon[36129]: pgmap v522: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 524 op/s 2024-03-09T00:11:00.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:00 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:11:00.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:00 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:11:00.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:00 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:11:00.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:00 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:11:00.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:00 smithi192 ceph-mon[35995]: pgmap v522: 129 pgs: 129 active+clean; 376 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 524 op/s 2024-03-09T00:11:00.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:00 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:11:00.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:00 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:11:00.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:00 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:11:00.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:00 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:11:01.071 INFO:tasks.workunit.client.0.smithi022.stdout: 10 45851 12.44 MB/sec execute 68 sec latency 280.849 ms 2024-03-09T00:11:01.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:01 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:11:01.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:01 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:11:01.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:01 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:11:01.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:01 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:11:01.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:01 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:11:01.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:01 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:11:01.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:01 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:11:01.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:01 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:11:01.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:01 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:11:02.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46028 12.33 MB/sec execute 69 sec latency 404.375 ms 2024-03-09T00:11:02.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:02 smithi062 ceph-mon[36129]: pgmap v523: 129 pgs: 129 active+clean; 374 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 667 op/s 2024-03-09T00:11:02.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:02 smithi192 ceph-mon[35995]: pgmap v523: 129 pgs: 129 active+clean; 374 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 667 op/s 2024-03-09T00:11:02.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:02 smithi022 ceph-mon[31894]: pgmap v523: 129 pgs: 129 active+clean; 374 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 667 op/s 2024-03-09T00:11:03.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46083 12.20 MB/sec execute 70 sec latency 444.041 ms 2024-03-09T00:11:04.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46091 12.05 MB/sec execute 71 sec latency 463.291 ms 2024-03-09T00:11:04.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:04 smithi062 ceph-mon[36129]: pgmap v524: 129 pgs: 129 active+clean; 374 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-03-09T00:11:04.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:04 smithi192 ceph-mon[35995]: pgmap v524: 129 pgs: 129 active+clean; 374 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-03-09T00:11:04.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:04 smithi022 ceph-mon[31894]: pgmap v524: 129 pgs: 129 active+clean; 374 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2024-03-09T00:11:05.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46137 11.95 MB/sec execute 72 sec latency 640.729 ms 2024-03-09T00:11:06.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46161 11.82 MB/sec execute 73 sec latency 977.567 ms 2024-03-09T00:11:06.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:06 smithi062 ceph-mon[36129]: pgmap v525: 129 pgs: 129 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 784 op/s 2024-03-09T00:11:06.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:06 smithi062 ceph-mon[36129]: osdmap e121: 12 total, 12 up, 12 in 2024-03-09T00:11:06.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:06 smithi192 ceph-mon[35995]: pgmap v525: 129 pgs: 129 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 784 op/s 2024-03-09T00:11:06.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:06 smithi192 ceph-mon[35995]: osdmap e121: 12 total, 12 up, 12 in 2024-03-09T00:11:06.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:06 smithi022 ceph-mon[31894]: pgmap v525: 129 pgs: 129 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 MiB/s wr, 784 op/s 2024-03-09T00:11:06.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:06 smithi022 ceph-mon[31894]: osdmap e121: 12 total, 12 up, 12 in 2024-03-09T00:11:07.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46174 11.67 MB/sec execute 74 sec latency 1242.484 ms 2024-03-09T00:11:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.889+0000 7f7e2519d640 1 -- 172.21.15.22:0/768227738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e201120e0 msgr2=0x7f7e201144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.889+0000 7f7e2519d640 1 --2- 172.21.15.22:0/768227738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e201120e0 0x7f7e201144d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009ca0 tx=0x7f7e1802c140 comp rx=0 tx=0).stop 2024-03-09T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.891+0000 7f7e2519d640 1 -- 172.21.15.22:0/768227738 shutdown_connections 2024-03-09T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.891+0000 7f7e2519d640 1 --2- 172.21.15.22:0/768227738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e201120e0 0x7f7e201144d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.891+0000 7f7e2519d640 1 --2- 172.21.15.22:0/768227738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 0x7f7e20111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.891+0000 7f7e2519d640 1 --2- 172.21.15.22:0/768227738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e200ff3c0 0x7f7e200ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.891+0000 7f7e2519d640 1 -- 172.21.15.22:0/768227738 >> 172.21.15.22:0/768227738 conn(0x7f7e20076380 msgr2=0x7f7e200fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.896+0000 7f7e2519d640 1 -- 172.21.15.22:0/768227738 shutdown_connections 2024-03-09T00:11:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.897+0000 7f7e2519d640 1 -- 172.21.15.22:0/768227738 wait complete. 2024-03-09T00:11:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 Processor -- start 2024-03-09T00:11:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 -- start start 2024-03-09T00:11:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e200ff3c0 0x7f7e20072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 0x7f7e20072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e201120e0 0x7f7e2006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20117ca0 con 0x7f7e200ffce0 2024-03-09T00:11:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20117b20 con 0x7f7e201120e0 2024-03-09T00:11:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.898+0000 7f7e2519d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20117e20 con 0x7f7e200ff3c0 2024-03-09T00:11:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 0x7f7e20072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 0x7f7e20072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39160/0 (socket says 172.21.15.22:39160) 2024-03-09T00:11:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 -- 172.21.15.22:0/4202667676 learned_addr learned my addr 172.21.15.22:0/4202667676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 -- 172.21.15.22:0/4202667676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e200ff3c0 msgr2=0x7f7e20072a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:11:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e200ff3c0 0x7f7e20072a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 -- 172.21.15.22:0/4202667676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e201120e0 msgr2=0x7f7e2006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e201120e0 0x7f7e2006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.901+0000 7f7e1e59c640 1 -- 172.21.15.22:0/4202667676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2006c5e0 con 0x7f7e200ffce0 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.902+0000 7f7e1e59c640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 0x7f7e20072f60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000c890 tx=0x7f7e1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.903+0000 7f7e07fff640 1 -- 172.21.15.22:0/4202667676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10004220 con 0x7f7e200ffce0 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.903+0000 7f7e07fff640 1 -- 172.21.15.22:0/4202667676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e100043c0 con 0x7f7e200ffce0 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.903+0000 7f7e07fff640 1 -- 172.21.15.22:0/4202667676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e100049c0 con 0x7f7e200ffce0 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.903+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2006c870 con 0x7f7e200ffce0 2024-03-09T00:11:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.903+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e20070780 con 0x7f7e200ffce0 2024-03-09T00:11:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.910+0000 7f7e07fff640 1 -- 172.21.15.22:0/4202667676 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7e100079b0 con 0x7f7e200ffce0 2024-03-09T00:11:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.912+0000 7f7e27d77740 1 Processor -- start 2024-03-09T00:11:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e27d77740 1 -- start start 2024-03-09T00:11:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x561048943a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x561048944450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 0x56104894f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e27d77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610488a85f0 con 0x561048944d40 2024-03-09T00:11:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e27d77740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561048851930 con 0x56104894d550 2024-03-09T00:11:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e27d77740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610488a4690 con 0x561048943fd0 2024-03-09T00:11:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.913+0000 7f7e07fff640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7e080a2c40 0x7f7e080a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e07fff640 1 -- 172.21.15.22:0/4202667676 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f7e100cf1b0 con 0x7f7e200ffce0 2024-03-09T00:11:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x561048943a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x561048943a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39172/0 (socket says 172.21.15.22:39172) 2024-03-09T00:11:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 -- 172.21.15.22:0/3196273642 learned_addr learned my addr 172.21.15.22:0/3196273642 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7e080a2c40 0x7f7e080a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 -- 172.21.15.22:0/3196273642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 msgr2=0x561048944450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:11:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/3196273642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x561048944450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 -- 172.21.15.22:0/3196273642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 msgr2=0x56104894f940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:11:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/3196273642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 0x56104894f940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.914+0000 7f7e1f59e640 1 -- 172.21.15.22:0/3196273642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610485bd210 con 0x561048944d40 2024-03-09T00:11:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.915+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/3196273642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x561048943a90 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009a60 tx=0x7f7e1802ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bfe960e7e8d146 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.916+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7e080a2c40 0x7f7e080a5100 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009bf0 tx=0x7f7e18068000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.916+0000 7f7e057fa640 1 -- 172.21.15.22:0/3196273642 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18039040 con 0x561048944d40 2024-03-09T00:11:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.916+0000 7f7e057fa640 1 -- 172.21.15.22:0/3196273642 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e18068070 con 0x561048944d40 2024-03-09T00:11:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.916+0000 7f7e057fa640 1 -- 172.21.15.22:0/3196273642 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18034a70 con 0x561048944d40 2024-03-09T00:11:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.918+0000 7f7e27d77740 1 -- 172.21.15.22:0/3196273642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 msgr2=0x561048943a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.918+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3196273642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x561048943a90 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009a60 tx=0x7f7e1802ef70 comp rx=0 tx=0).stop 2024-03-09T00:11:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.920+0000 7f7e27d77740 1 -- 172.21.15.22:0/3196273642 shutdown_connections 2024-03-09T00:11:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.920+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3196273642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 0x56104894f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.920+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3196273642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x561048944450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.920+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3196273642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x561048943a90 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.920+0000 7f7e27d77740 1 -- 172.21.15.22:0/3196273642 >> 172.21.15.22:0/3196273642 conn(0x56104890b5b0 msgr2=0x56104893ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.927+0000 7f7e27d77740 1 -- 172.21.15.22:0/3196273642 shutdown_connections 2024-03-09T00:11:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 -- 172.21.15.22:0/3196273642 wait complete. 2024-03-09T00:11:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 -- start start 2024-03-09T00:11:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 Processor -- start 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048943fd0 0x56104894a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048944d40 0x56104894aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56104894d550 0x56104894b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561048950220 con 0x56104894d550 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561048864d90 con 0x561048943fd0 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.931+0000 7f7e27d77740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610489503a0 con 0x561048944d40 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e27d77740 10 client.0 _ll_register_callbacks cb 0x5610485bca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56104894d550 0x56104894b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56104894d550 0x56104894b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39182/0 (socket says 172.21.15.22:39182) 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 -- 172.21.15.22:0/1188139053 learned_addr learned my addr 172.21.15.22:0/1188139053 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048943fd0 0x56104894a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 -- 172.21.15.22:0/1188139053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048944d40 msgr2=0x56104894aba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048944d40 0x56104894aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 -- 172.21.15.22:0/1188139053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048943fd0 msgr2=0x56104894a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048943fd0 0x56104894a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.932+0000 7f7e1f59e640 1 -- 172.21.15.22:0/1188139053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561048689620 con 0x56104894d550 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.933+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56104894d550 0x56104894b0e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7e180095c0 tx=0x7f7e18004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982a705db3c6fccd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.933+0000 7f7e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.936+0000 7f7e02ff5640 1 -- 172.21.15.22:0/1188139053 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18003970 con 0x56104894d550 2024-03-09T00:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.936+0000 7f7e02ff5640 1 -- 172.21.15.22:0/1188139053 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f7e18003b10 con 0x56104894d550 2024-03-09T00:11:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.937+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e18068070 con 0x56104894d550 2024-03-09T00:11:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.939+0000 7f7e02ff5640 1 -- 172.21.15.22:0/1188139053 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e18003d10 con 0x56104894d550 2024-03-09T00:11:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.939+0000 7f7e27d77740 10 client.20637 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.939+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56104895c5e0 con 0x56104894d550 2024-03-09T00:11:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e02ff5640 1 -- 172.21.15.22:0/1188139053 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e18068070 con 0x56104894d550 2024-03-09T00:11:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e02ff5640 1 -- 172.21.15.22:0/1188139053 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7e1803f050 con 0x56104894d550 2024-03-09T00:11:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e27d77740 10 client.20637 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e27d77740 10 client.20637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e27d77740 20 client.20637 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e27d77740 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56104895dc70 0x561048960090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e27d77740 4 client.20637 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.940+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610488b5b30 con 0x56104895dc70 2024-03-09T00:11:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.941+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56104895dc70 0x561048960090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.943+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56104895dc70 0x561048960090 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15225bab30d79968 server_cookie=8467b640583fc1db in_seq=0 out_seq=0 2024-03-09T00:11:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.943+0000 7f7e02ff5640 10 client.20637 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.945+0000 7f7e02ff5640 1 -- 172.21.15.22:0/1188139053 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610488b5b30 con 0x56104895dc70 2024-03-09T00:11:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.945+0000 7f7e02ff5640 10 client.20637 handle_command_reply: tid=0 2024-03-09T00:11:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.945+0000 7f7e27d77740 1 client.20637 shutdown 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56104894d550 msgr2=0x56104894b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56104894d550 0x56104894b0e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7e180095c0 tx=0x7f7e18004dc0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 shutdown_connections 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56104895dc70 0x561048960090 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56104894d550 0x56104894b0e0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048944d40 0x56104894aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 --2- 172.21.15.22:0/1188139053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048943fd0 0x56104894a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.946+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 >> 172.21.15.22:0/1188139053 conn(0x56104890b5b0 msgr2=0x561048940a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.950+0000 7f7e02ff5640 0 client.20637 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.953+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 shutdown_connections 2024-03-09T00:11:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.953+0000 7f7e27d77740 1 -- 172.21.15.22:0/1188139053 wait complete. 2024-03-09T00:11:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.953+0000 7f7e27d77740 20 client.20637 trim_cache size 0 max 16384 2024-03-09T00:11:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.983+0000 7f7e27d77740 1 Processor -- start 2024-03-09T00:11:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.984+0000 7f7e27d77740 1 -- start start 2024-03-09T00:11:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.984+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x56104894cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.984+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x56104894af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.984+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 0x561048954610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.984+0000 7f7e27d77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201a6420 con 0x561048944d40 2024-03-09T00:11:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.984+0000 7f7e27d77740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201a3fc0 con 0x56104894d550 2024-03-09T00:11:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.984+0000 7f7e27d77740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2004f050 con 0x561048943fd0 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.986+0000 7f7e1ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x56104894cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.986+0000 7f7e1ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x56104894cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59472/0 (socket says 172.21.15.22:59472) 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.986+0000 7f7e1ed9d640 1 -- 172.21.15.22:0/3479968971 learned_addr learned my addr 172.21.15.22:0/3479968971 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.986+0000 7f7e1e59c640 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x56104894af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.986+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 0x561048954610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.987+0000 7f7e1ed9d640 1 -- 172.21.15.22:0/3479968971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 msgr2=0x561048954610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.987+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 0x561048954610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.987+0000 7f7e1ed9d640 1 -- 172.21.15.22:0/3479968971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 msgr2=0x56104894af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.987+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x56104894af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.987+0000 7f7e1ed9d640 1 -- 172.21.15.22:0/3479968971 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610489432a0 con 0x561048943fd0 2024-03-09T00:11:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.987+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x56104894cd90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e140095b0 tx=0x7f7e14005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f90bc25f01fa014 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.996+0000 7f7e04ff9640 1 -- 172.21.15.22:0/3479968971 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e140139c0 con 0x561048943fd0 2024-03-09T00:11:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.996+0000 7f7e04ff9640 1 -- 172.21.15.22:0/3479968971 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e1401e030 con 0x561048943fd0 2024-03-09T00:11:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.996+0000 7f7e04ff9640 1 -- 172.21.15.22:0/3479968971 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e140118d0 con 0x561048943fd0 2024-03-09T00:11:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.998+0000 7f7e27d77740 1 -- 172.21.15.22:0/3479968971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 msgr2=0x56104894cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:07.998+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x56104894cd90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e140095b0 tx=0x7f7e14005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.003+0000 7f7e27d77740 1 -- 172.21.15.22:0/3479968971 shutdown_connections 2024-03-09T00:11:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.003+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56104894d550 0x561048954610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.003+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048944d40 0x56104894af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.003+0000 7f7e27d77740 1 --2- 172.21.15.22:0/3479968971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561048943fd0 0x56104894cd90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.003+0000 7f7e27d77740 1 -- 172.21.15.22:0/3479968971 >> 172.21.15.22:0/3479968971 conn(0x56104890b5b0 msgr2=0x56104893e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.006+0000 7f7e27d77740 1 -- 172.21.15.22:0/3479968971 shutdown_connections 2024-03-09T00:11:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.006+0000 7f7e27d77740 1 -- 172.21.15.22:0/3479968971 wait complete. 2024-03-09T00:11:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.006+0000 7f7e27d77740 1 -- start start 2024-03-09T00:11:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 1 Processor -- start 2024-03-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048943fd0 0x5610489505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048944d40 0x561048950b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56104894d550 0x561048951070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2010f1a0 con 0x561048943fd0 2024-03-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201a3960 con 0x561048944d40 2024-03-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2010f7a0 con 0x56104894d550 2024-03-09T00:11:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.007+0000 7f7e27d77740 10 client.0 _ll_register_callbacks cb 0x7f7e18037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.008+0000 7f7e1f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56104894d550 0x561048951070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.009+0000 7f7e1f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56104894d550 0x561048951070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:59476/0 (socket says 172.21.15.22:59476) 2024-03-09T00:11:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.009+0000 7f7e1f59e640 1 -- 172.21.15.22:0/10995158 learned_addr learned my addr 172.21.15.22:0/10995158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.009+0000 7f7e1e59c640 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048944d40 0x561048950b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.009+0000 7f7e1f59e640 1 -- 172.21.15.22:0/10995158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048944d40 msgr2=0x561048950b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.009+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048944d40 0x561048950b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.009+0000 7f7e1f59e640 1 -- 172.21.15.22:0/10995158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048943fd0 msgr2=0x5610489505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.009+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048943fd0 0x5610489505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.010+0000 7f7e1f59e640 1 -- 172.21.15.22:0/10995158 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610489b3350 con 0x56104894d550 2024-03-09T00:11:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.011+0000 7f7e1f59e640 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56104894d550 0x561048951070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009870 tx=0x7f7e18032c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79339864701f268 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.011+0000 7f7e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:11:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.012+0000 7f7e02ff5640 1 -- 172.21.15.22:0/10995158 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18083070 con 0x56104894d550 2024-03-09T00:11:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.012+0000 7f7e02ff5640 1 -- 172.21.15.22:0/10995158 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f7e180040b0 con 0x56104894d550 2024-03-09T00:11:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.013+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e1401e030 con 0x56104894d550 2024-03-09T00:11:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.015+0000 7f7e02ff5640 1 -- 172.21.15.22:0/10995158 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e1808d050 con 0x56104894d550 2024-03-09T00:11:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.015+0000 7f7e27d77740 10 client.30016 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.015+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5610489b4620 con 0x56104894d550 2024-03-09T00:11:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.016+0000 7f7e02ff5640 1 -- 172.21.15.22:0/10995158 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e1401e030 con 0x56104894d550 2024-03-09T00:11:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.017+0000 7f7e02ff5640 1 -- 172.21.15.22:0/10995158 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7e1803f040 con 0x56104894d550 2024-03-09T00:11:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.018+0000 7f7e27d77740 10 client.30016 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.018+0000 7f7e27d77740 10 client.30016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.018+0000 7f7e27d77740 20 client.30016 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.018+0000 7f7e27d77740 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5610489b4c80 0x5610489b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.018+0000 7f7e27d77740 4 client.30016 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:11:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.018+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5610488afe80 con 0x5610489b4c80 2024-03-09T00:11:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.018+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5610489b4c80 0x5610489b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.019+0000 7f7e1ed9d640 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5610489b4c80 0x5610489b70c0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3171cea207a356d8 server_cookie=28ab20fc535929db in_seq=0 out_seq=0 2024-03-09T00:11:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:08.019+0000 7f7e02ff5640 10 client.30016 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:08.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46239 11.53 MB/sec execute 75 sec latency 1326.584 ms 2024-03-09T00:11:08.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:08 smithi062 ceph-mon[36129]: pgmap v527: 129 pgs: 129 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 486 op/s 2024-03-09T00:11:08.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:08 smithi192 ceph-mon[35995]: pgmap v527: 129 pgs: 129 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 486 op/s 2024-03-09T00:11:08.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:08 smithi022 ceph-mon[31894]: pgmap v527: 129 pgs: 129 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 486 op/s 2024-03-09T00:11:09.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46260 11.42 MB/sec execute 76 sec latency 1736.720 ms 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.151+0000 7f7e02ff5640 1 -- 172.21.15.22:0/10995158 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5610488afe80 con 0x5610489b4c80 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.151+0000 7f7e02ff5640 10 client.30016 handle_command_reply: tid=0 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.151+0000 7f7e27d77740 1 client.30016 shutdown 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56104894d550 msgr2=0x561048951070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56104894d550 0x561048951070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009870 tx=0x7f7e18032c50 comp rx=0 tx=0).stop 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 shutdown_connections 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5610489b4c80 0x5610489b70c0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56104894d550 0x561048951070 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561048944d40 0x561048950b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 --2- 172.21.15.22:0/10995158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561048943fd0 0x5610489505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 >> 172.21.15.22:0/10995158 conn(0x56104890b5b0 msgr2=0x56104893d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e02ff5640 0 client.30016 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 shutdown_connections 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 1 -- 172.21.15.22:0/10995158 wait complete. 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.152+0000 7f7e27d77740 20 client.30016 trim_cache size 0 max 16384 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:11:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:11:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.153+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7e080a2c40 msgr2=0x7f7e080a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.153+0000 7f7e2519d640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7e080a2c40 0x7f7e080a5100 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009bf0 tx=0x7f7e18068000 comp rx=0 tx=0).stop 2024-03-09T00:11:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.153+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 msgr2=0x7f7e20072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.153+0000 7f7e2519d640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 0x7f7e20072f60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000c890 tx=0x7f7e1000cd50 comp rx=0 tx=0).stop 2024-03-09T00:11:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.153+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 shutdown_connections 2024-03-09T00:11:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.153+0000 7f7e2519d640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7e080a2c40 0x7f7e080a5100 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.154+0000 7f7e2519d640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e201120e0 0x7f7e2006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.154+0000 7f7e2519d640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7e200ffce0 0x7f7e20072f60 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.154+0000 7f7e2519d640 1 --2- 172.21.15.22:0/4202667676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e200ff3c0 0x7f7e20072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.154+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 >> 172.21.15.22:0/4202667676 conn(0x7f7e20076380 msgr2=0x7f7e201068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.154+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 shutdown_connections 2024-03-09T00:11:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:09.154+0000 7f7e2519d640 1 -- 172.21.15.22:0/4202667676 wait complete. 2024-03-09T00:11:10.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46309 11.28 MB/sec execute 77 sec latency 2153.088 ms 2024-03-09T00:11:10.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:10 smithi062 ceph-mon[36129]: pgmap v528: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-03-09T00:11:10.630 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:10 smithi192 ceph-mon[35995]: pgmap v528: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-03-09T00:11:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:10 smithi022 ceph-mon[31894]: pgmap v528: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 371 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-03-09T00:11:11.072 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46383 11.19 MB/sec execute 78 sec latency 2181.463 ms 2024-03-09T00:11:12.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46542 11.19 MB/sec execute 79 sec latency 1302.238 ms 2024-03-09T00:11:12.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:12 smithi192 ceph-mon[35995]: pgmap v529: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 363 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-03-09T00:11:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:12 smithi022 ceph-mon[31894]: pgmap v529: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 363 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-03-09T00:11:12.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:12 smithi062 ceph-mon[36129]: pgmap v529: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 363 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-03-09T00:11:13.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 46772 11.22 MB/sec execute 80 sec latency 1033.978 ms 2024-03-09T00:11:14.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 47011 11.22 MB/sec execute 81 sec latency 967.958 ms 2024-03-09T00:11:14.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:14 smithi192 ceph-mon[35995]: pgmap v530: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 363 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-03-09T00:11:14.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:14 smithi022 ceph-mon[31894]: pgmap v530: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 363 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-03-09T00:11:14.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:14 smithi062 ceph-mon[36129]: pgmap v530: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 363 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 605 op/s 2024-03-09T00:11:15.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 47110 11.16 MB/sec execute 82 sec latency 372.720 ms 2024-03-09T00:11:16.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 47314 11.27 MB/sec execute 83 sec latency 420.565 ms 2024-03-09T00:11:16.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:16 smithi192 ceph-mon[35995]: pgmap v531: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 458 op/s 2024-03-09T00:11:16.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:16 smithi022 ceph-mon[31894]: pgmap v531: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 458 op/s 2024-03-09T00:11:16.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:16 smithi062 ceph-mon[36129]: pgmap v531: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 458 op/s 2024-03-09T00:11:17.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 47600 11.27 MB/sec execute 84 sec latency 117.324 ms 2024-03-09T00:11:18.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 47827 11.34 MB/sec execute 85 sec latency 250.454 ms 2024-03-09T00:11:18.635 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:18 smithi192 ceph-mon[35995]: pgmap v532: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 388 op/s 2024-03-09T00:11:18.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:18 smithi022 ceph-mon[31894]: pgmap v532: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 388 op/s 2024-03-09T00:11:18.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:18 smithi062 ceph-mon[36129]: pgmap v532: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 388 op/s 2024-03-09T00:11:19.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 48014 11.34 MB/sec execute 86 sec latency 261.546 ms 2024-03-09T00:11:20.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 48130 11.31 MB/sec execute 87 sec latency 658.442 ms 2024-03-09T00:11:20.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:20 smithi022 ceph-mon[31894]: pgmap v533: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 386 op/s 2024-03-09T00:11:20.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:20 smithi062 ceph-mon[36129]: pgmap v533: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 386 op/s 2024-03-09T00:11:20.880 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:20 smithi192 ceph-mon[35995]: pgmap v533: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 369 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 386 op/s 2024-03-09T00:11:21.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 48333 11.28 MB/sec execute 88 sec latency 424.527 ms 2024-03-09T00:11:21.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:11:21.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:11:21.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 -- 172.21.15.22:0/1632453533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8478101eb0 msgr2=0x7f8478105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 --2- 172.21.15.22:0/1632453533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8478101eb0 0x7f8478105ef0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f8468009900 tx=0x7f846802bc30 comp rx=0 tx=0).stop 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 -- 172.21.15.22:0/1632453533 shutdown_connections 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 --2- 172.21.15.22:0/1632453533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84781065d0 0x7f847810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 --2- 172.21.15.22:0/1632453533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8478101eb0 0x7f8478105ef0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 --2- 172.21.15.22:0/1632453533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 0x7f84781018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 -- 172.21.15.22:0/1632453533 >> 172.21.15.22:0/1632453533 conn(0x7f84780fcd20 msgr2=0x7f84780ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 -- 172.21.15.22:0/1632453533 shutdown_connections 2024-03-09T00:11:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.240+0000 7f847ed5b640 1 -- 172.21.15.22:0/1632453533 wait complete. 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.241+0000 7f847ed5b640 1 Processor -- start 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.241+0000 7f847ed5b640 1 -- start start 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.241+0000 7f847ed5b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 0x7f847819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.241+0000 7f847ed5b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84781065d0 0x7f847819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.241+0000 7f847ed5b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84781958a0 0x7f8478195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.241+0000 7f847ed5b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847810f4e0 con 0x7f84781958a0 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 0x7f847819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 0x7f847819b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51120/0 (socket says 172.21.15.22:51120) 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 -- 172.21.15.22:0/2656981394 learned_addr learned my addr 172.21.15.22:0/2656981394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847810f360 con 0x7f8478101500 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847810f660 con 0x7f84781065d0 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847d2f8640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84781958a0 0x7f8478195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f846ffff640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84781065d0 0x7f847819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 -- 172.21.15.22:0/2656981394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84781065d0 msgr2=0x7f847819bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84781065d0 0x7f847819bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 -- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84781958a0 msgr2=0x7f8478195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84781958a0 0x7f8478195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 -- 172.21.15.22:0/2656981394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478196430 con 0x7f8478101500 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f846ffff640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84781065d0 0x7f847819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.242+0000 7f847caf7640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 0x7f847819b6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f847400d950 tx=0x7f847400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.243+0000 7f846dffb640 1 -- 172.21.15.22:0/2656981394 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474017070 con 0x7f8478101500 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.243+0000 7f846dffb640 1 -- 172.21.15.22:0/2656981394 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8474009c30 con 0x7f8478101500 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.243+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8478196720 con 0x7f8478101500 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.243+0000 7f846dffb640 1 -- 172.21.15.22:0/2656981394 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400f6b0 con 0x7f8478101500 2024-03-09T00:11:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.243+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84781a27c0 con 0x7f8478101500 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.245+0000 7f846dffb640 1 -- 172.21.15.22:0/2656981394 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f84740109d0 con 0x7f8478101500 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 Processor -- start 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 -- start start 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7710760 0x5556b7710b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7719c60 0x5556b771c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b7673e00 con 0x5556b7711080 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b761cb10 con 0x5556b7719c60 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f8481a1a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b766fea0 con 0x5556b7710760 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f847caf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f847caf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49434/0 (socket says 172.21.15.22:49434) 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f847caf7640 1 -- 172.21.15.22:0/1961233476 learned_addr learned my addr 172.21.15.22:0/1961233476 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f847d2f8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7710760 0x5556b7710b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.246+0000 7f846ffff640 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7719c60 0x5556b771c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847caf7640 1 -- 172.21.15.22:0/1961233476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7710760 msgr2=0x5556b7710b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847caf7640 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7710760 0x5556b7710b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847caf7640 1 -- 172.21.15.22:0/1961233476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7719c60 msgr2=0x5556b771c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847caf7640 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7719c60 0x5556b771c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847caf7640 1 -- 172.21.15.22:0/1961233476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556b74527d0 con 0x5556b7711080 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f846dffb640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84600a6ed0 0x7f84600a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f846ffff640 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7719c60 0x5556b771c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847d2f8640 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7710760 0x5556b7710b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847caf7640 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719720 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f84740cc050 tx=0x7f84740d7570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d8ccc6a37682ed server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f847d2f8640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84600a6ed0 0x7f84600a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f845effd640 1 -- 172.21.15.22:0/1961233476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740972c0 con 0x5556b7711080 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f845effd640 1 -- 172.21.15.22:0/1961233476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f847409b720 con 0x5556b7711080 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f845effd640 1 -- 172.21.15.22:0/1961233476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740b1100 con 0x5556b7711080 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.247+0000 7f846dffb640 1 -- 172.21.15.22:0/2656981394 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f84740cee70 con 0x7f8478101500 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f847d2f8640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84600a6ed0 0x7f84600a9390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8470005fd0 tx=0x7f84700096a0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 -- 172.21.15.22:0/1961233476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 msgr2=0x5556b7719720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719720 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f84740cc050 tx=0x7f84740d7570 comp rx=0 tx=0).stop 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 -- 172.21.15.22:0/1961233476 shutdown_connections 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7719c60 0x5556b771c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719720 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 --2- 172.21.15.22:0/1961233476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7710760 0x5556b7710b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 -- 172.21.15.22:0/1961233476 >> 172.21.15.22:0/1961233476 conn(0x5556b770ae90 msgr2=0x5556b770b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 -- 172.21.15.22:0/1961233476 shutdown_connections 2024-03-09T00:11:21.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 -- 172.21.15.22:0/1961233476 wait complete. 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 -- start start 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.248+0000 7f8481a1a740 1 Processor -- start 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7719320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b77166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7716bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8481a1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b771ca50 con 0x5556b7711080 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8481a1a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b76300a0 con 0x5556b7710760 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8481a1a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556b771cbd0 con 0x5556b7719c60 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8481a1a740 10 client.0 _ll_register_callbacks cb 0x5556b762fa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b77166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b77166b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49448/0 (socket says 172.21.15.22:49448) 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 -- 172.21.15.22:0/570906392 learned_addr learned my addr 172.21.15.22:0/570906392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f847caf7640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7719320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f847d2f8640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7716bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 -- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 msgr2=0x5556b7716bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7716bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 -- 172.21.15.22:0/570906392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 msgr2=0x5556b7719320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7719320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 -- 172.21.15.22:0/570906392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556b73883f0 con 0x5556b7711080 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f847caf7640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7719320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f846ffff640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b77166b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f847806a940 tx=0x7f8468005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9647f3b842c0ba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.249+0000 7f8457fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8457fff640 1 -- 172.21.15.22:0/570906392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468005030 con 0x5556b7711080 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8457fff640 1 -- 172.21.15.22:0/570906392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f84680051d0 con 0x5556b7711080 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f847409b720 con 0x5556b7711080 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8457fff640 1 -- 172.21.15.22:0/570906392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84680053d0 con 0x5556b7711080 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 10 client.20652 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:21.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5556b73480f0 con 0x5556b7711080 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8457fff640 1 -- 172.21.15.22:0/570906392 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f847409b720 con 0x5556b7711080 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8457fff640 1 -- 172.21.15.22:0/570906392 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8468003830 con 0x5556b7711080 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 10 client.20652 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 10 client.20652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 20 client.20652 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b772ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 4 client.20652 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.250+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556b7681340 con 0x5556b77296a0 2024-03-09T00:11:21.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.251+0000 7f847caf7640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b772ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.254+0000 7f847caf7640 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b772ba90 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99eba44e135f289 server_cookie=d59350ca9ef06599 in_seq=0 out_seq=0 2024-03-09T00:11:21.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.254+0000 7f8457fff640 10 client.20652 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.259+0000 7f8457fff640 1 -- 172.21.15.22:0/570906392 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556b7681340 con 0x5556b77296a0 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.259+0000 7f8457fff640 10 client.20652 handle_command_reply: tid=0 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.259+0000 7f8481a1a740 1 client.20652 shutdown 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 msgr2=0x5556b77166b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b77166b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f847806a940 tx=0x7f8468005e10 comp rx=0 tx=0).stop 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 shutdown_connections 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b772ba90 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8457fff640 0 client.20652 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:21.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7716bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b77166b0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 --2- 172.21.15.22:0/570906392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7719320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 >> 172.21.15.22:0/570906392 conn(0x5556b770ae90 msgr2=0x5556b770cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 shutdown_connections 2024-03-09T00:11:21.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.260+0000 7f8481a1a740 1 -- 172.21.15.22:0/570906392 wait complete. 2024-03-09T00:11:21.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.261+0000 7f8481a1a740 20 client.20652 trim_cache size 0 max 16384 2024-03-09T00:11:21.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.283+0000 7f8481a1a740 1 Processor -- start 2024-03-09T00:11:21.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f8481a1a740 1 -- start start 2024-03-09T00:11:21.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7718de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7714f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f8481a1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84781aeb70 con 0x5556b7711080 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f8481a1a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84781ac710 con 0x5556b7710760 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f8481a1a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84781aa7d0 con 0x5556b7719c60 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f847caf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7718de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f847caf7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7718de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51154/0 (socket says 172.21.15.22:51154) 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f847caf7640 1 -- 172.21.15.22:0/2922327674 learned_addr learned my addr 172.21.15.22:0/2922327674 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.284+0000 7f847d2f8640 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7714f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f847caf7640 1 -- 172.21.15.22:0/2922327674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 msgr2=0x5556b7714f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f847caf7640 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7714f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f847caf7640 1 -- 172.21.15.22:0/2922327674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 msgr2=0x5556b7719320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:11:21.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f847caf7640 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f847caf7640 1 -- 172.21.15.22:0/2922327674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556b7710eb0 con 0x5556b7710760 2024-03-09T00:11:21.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f847caf7640 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7718de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f847409ad20 tx=0x7f84740d7f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0fe5c75d2dae572 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f845e7fc640 1 -- 172.21.15.22:0/2922327674 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474020040 con 0x5556b7710760 2024-03-09T00:11:21.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.286+0000 7f845e7fc640 1 -- 172.21.15.22:0/2922327674 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8474009de0 con 0x5556b7710760 2024-03-09T00:11:21.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.287+0000 7f845e7fc640 1 -- 172.21.15.22:0/2922327674 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740b12d0 con 0x5556b7710760 2024-03-09T00:11:21.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 -- 172.21.15.22:0/2922327674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 msgr2=0x5556b7718de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7718de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f847409ad20 tx=0x7f84740d7f50 comp rx=0 tx=0).stop 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 -- 172.21.15.22:0/2922327674 shutdown_connections 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b7714f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7711080 0x5556b7719320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 --2- 172.21.15.22:0/2922327674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7710760 0x5556b7718de0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 -- 172.21.15.22:0/2922327674 >> 172.21.15.22:0/2922327674 conn(0x5556b770ae90 msgr2=0x5556b7716670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 -- 172.21.15.22:0/2922327674 shutdown_connections 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.288+0000 7f8481a1a740 1 -- 172.21.15.22:0/2922327674 wait complete. 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.289+0000 7f8481a1a740 1 -- start start 2024-03-09T00:11:21.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.289+0000 7f8481a1a740 1 Processor -- start 2024-03-09T00:11:21.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.289+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7710760 0x5556b7720ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7711080 0x5556b771d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f8481a1a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b771d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f8481a1a740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84781aadd0 con 0x5556b7710760 2024-03-09T00:11:21.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f8481a1a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84781ac0b0 con 0x5556b7711080 2024-03-09T00:11:21.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f8481a1a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84781ab3f0 con 0x5556b7719c60 2024-03-09T00:11:21.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f8481a1a740 10 client.0 _ll_register_callbacks cb 0x7f8468009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:21.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f847d2f8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b771d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f847d2f8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b771d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41556/0 (socket says 172.21.15.22:41556) 2024-03-09T00:11:21.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f847d2f8640 1 -- 172.21.15.22:0/726176317 learned_addr learned my addr 172.21.15.22:0/726176317 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.290+0000 7f847caf7640 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7710760 0x5556b7720ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.291+0000 7f847caf7640 1 -- 172.21.15.22:0/726176317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 msgr2=0x5556b771d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.291+0000 7f847caf7640 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b771d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.291+0000 7f847caf7640 1 -- 172.21.15.22:0/726176317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7711080 msgr2=0x5556b771d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.291+0000 7f847caf7640 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7711080 0x5556b771d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.291+0000 7f847caf7640 1 -- 172.21.15.22:0/726176317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556b771dca0 con 0x5556b7710760 2024-03-09T00:11:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.292+0000 7f847caf7640 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7710760 0x5556b7720ea0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f8474009610 tx=0x7f847400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b3ba07bc295d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.292+0000 7f8457fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:21.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.292+0000 7f8457fff640 1 -- 172.21.15.22:0/726176317 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474020070 con 0x5556b7710760 2024-03-09T00:11:21.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.292+0000 7f8457fff640 1 -- 172.21.15.22:0/726176317 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f84740d2450 con 0x5556b7710760 2024-03-09T00:11:21.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.293+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8474009de0 con 0x5556b7710760 2024-03-09T00:11:21.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.293+0000 7f8457fff640 1 -- 172.21.15.22:0/726176317 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84740af1a0 con 0x5556b7710760 2024-03-09T00:11:21.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.294+0000 7f8481a1a740 10 client.20655 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:21.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.294+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5556b7789050 con 0x5556b7710760 2024-03-09T00:11:21.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.296+0000 7f8457fff640 1 -- 172.21.15.22:0/726176317 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84740b6020 con 0x5556b7710760 2024-03-09T00:11:21.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.296+0000 7f8457fff640 1 -- 172.21.15.22:0/726176317 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8474097690 con 0x5556b7710760 2024-03-09T00:11:21.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.296+0000 7f8481a1a740 10 client.20655 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:21.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.296+0000 7f8481a1a740 10 client.20655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:21.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.296+0000 7f8481a1a740 20 client.20655 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:21.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.296+0000 7f8481a1a740 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b778b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.297+0000 7f8481a1a740 4 client.20655 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:11:21.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.297+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f8474009de0 con 0x5556b77296a0 2024-03-09T00:11:21.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.298+0000 7f846ffff640 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b778b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.304+0000 7f846ffff640 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b778b230 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf97c7f90d374c57 server_cookie=79e05f828bbd287b in_seq=0 out_seq=0 2024-03-09T00:11:21.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.304+0000 7f8457fff640 10 client.20655 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:21.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.307+0000 7f8457fff640 1 -- 172.21.15.22:0/726176317 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f8474009de0 con 0x5556b77296a0 2024-03-09T00:11:21.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.307+0000 7f8457fff640 10 client.20655 handle_command_reply: tid=0 2024-03-09T00:11:21.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.307+0000 7f8481a1a740 1 client.20655 shutdown 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7710760 msgr2=0x5556b7720ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7710760 0x5556b7720ea0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f8474009610 tx=0x7f847400bf60 comp rx=0 tx=0).stop 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 shutdown_connections 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5556b77296a0 0x5556b778b230 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556b7719c60 0x5556b771d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556b7711080 0x5556b771d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 --2- 172.21.15.22:0/726176317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5556b7710760 0x5556b7720ea0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 >> 172.21.15.22:0/726176317 conn(0x5556b770ae90 msgr2=0x5556b7714f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8457fff640 0 client.20655 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 shutdown_connections 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.308+0000 7f8481a1a740 1 -- 172.21.15.22:0/726176317 wait complete. 2024-03-09T00:11:21.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.309+0000 7f8481a1a740 20 client.20655 trim_cache size 0 max 16384 2024-03-09T00:11:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:11:21.306 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:11:21.306 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:11:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:11:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84600a6ed0 msgr2=0x7f84600a9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84600a6ed0 0x7f84600a9390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8470005fd0 tx=0x7f84700096a0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 msgr2=0x7f847819b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 0x7f847819b6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f847400d950 tx=0x7f847400de10 comp rx=0 tx=0).stop 2024-03-09T00:11:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 shutdown_connections 2024-03-09T00:11:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f84600a6ed0 0x7f84600a9390 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84781958a0 0x7f8478195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84781065d0 0x7f847819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 --2- 172.21.15.22:0/2656981394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8478101500 0x7f847819b6e0 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f847400d950 tx=0x7f847400de10 comp rx=0 tx=0).stop 2024-03-09T00:11:21.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.310+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 >> 172.21.15.22:0/2656981394 conn(0x7f84780fcd20 msgr2=0x7f84780ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.311+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 shutdown_connections 2024-03-09T00:11:21.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.311+0000 7f847ed5b640 1 -- 172.21.15.22:0/2656981394 wait complete. 2024-03-09T00:11:21.319 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:11:21.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:11:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 -- 172.21.15.22:0/3147409248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 msgr2=0x7ff31410d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 --2- 172.21.15.22:0/3147409248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31410d100 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009ca0 tx=0x7ff30c02c0f0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 -- 172.21.15.22:0/3147409248 shutdown_connections 2024-03-09T00:11:21.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 --2- 172.21.15.22:0/3147409248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31410d100 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 --2- 172.21.15.22:0/3147409248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff314100e10 0x7ff314104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 --2- 172.21.15.22:0/3147409248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314100460 0x7ff314100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 -- 172.21.15.22:0/3147409248 >> 172.21.15.22:0/3147409248 conn(0x7ff3140772f0 msgr2=0x7ff314077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 -- 172.21.15.22:0/3147409248 shutdown_connections 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.461+0000 7ff318da9640 1 -- 172.21.15.22:0/3147409248 wait complete. 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 Processor -- start 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 -- start start 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314100460 0x7ff31419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff314197a40 0x7ff314197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141105f0 con 0x7ff314105530 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314110470 con 0x7ff314197a40 2024-03-09T00:11:21.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff318da9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314110770 con 0x7ff314100460 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff311d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff311d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49476/0 (socket says 172.21.15.22:49476) 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.462+0000 7ff311d9b640 1 -- 172.21.15.22:0/1442332102 learned_addr learned my addr 172.21.15.22:0/1442332102 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff312d9d640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff314197a40 0x7ff314197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff31259c640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314100460 0x7ff31419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff311d9b640 1 -- 172.21.15.22:0/1442332102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314100460 msgr2=0x7ff31419d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff311d9b640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314100460 0x7ff31419d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff311d9b640 1 -- 172.21.15.22:0/1442332102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff314197a40 msgr2=0x7ff314197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff311d9b640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff314197a40 0x7ff314197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff311d9b640 1 -- 172.21.15.22:0/1442332102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314198560 con 0x7ff314105530 2024-03-09T00:11:21.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff31259c640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314100460 0x7ff31419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:21.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff311d9b640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31419ddd0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9c0 tx=0x7ff30800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff3037fe640 1 -- 172.21.15.22:0/1442332102 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30800cd80 con 0x7ff314105530 2024-03-09T00:11:21.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff3037fe640 1 -- 172.21.15.22:0/1442332102 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff308005500 con 0x7ff314105530 2024-03-09T00:11:21.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff3037fe640 1 -- 172.21.15.22:0/1442332102 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308010690 con 0x7ff314105530 2024-03-09T00:11:21.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.463+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff314198850 con 0x7ff314105530 2024-03-09T00:11:21.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.464+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141a4a50 con 0x7ff314105530 2024-03-09T00:11:21.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.465+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3140680e0 con 0x7ff314105530 2024-03-09T00:11:21.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.466+0000 7ff3037fe640 1 -- 172.21.15.22:0/1442332102 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff3080046c0 con 0x7ff314105530 2024-03-09T00:11:21.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.466+0000 7ff3037fe640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff2fc0a29f0 0x7ff2fc0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.466+0000 7ff31259c640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff2fc0a29f0 0x7ff2fc0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.467+0000 7ff3037fe640 1 -- 172.21.15.22:0/1442332102 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff3080cf610 con 0x7ff314105530 2024-03-09T00:11:21.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.467+0000 7ff31259c640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff2fc0a29f0 0x7ff2fc0a4eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff314198b80 tx=0x7ff304005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.471+0000 7ff3037fe640 1 -- 172.21.15.22:0/1442332102 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff30809bf20 con 0x7ff314105530 2024-03-09T00:11:21.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.709+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff314198f70 con 0x7ff314105530 2024-03-09T00:11:21.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.710+0000 7ff3037fe640 1 -- 172.21.15.22:0/1442332102 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11343 (secure 0 0 0) 0x7ff30809b820 con 0x7ff314105530 2024-03-09T00:11:21.706 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:11:21.706 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:10:14.710788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:11:21.707 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-03-09T00:11:21.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff2fc0a29f0 msgr2=0x7ff2fc0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff2fc0a29f0 0x7ff2fc0a4eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff314198b80 tx=0x7ff304005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:21.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 msgr2=0x7ff31419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31419ddd0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9c0 tx=0x7ff30800ee80 comp rx=0 tx=0).stop 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 shutdown_connections 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff2fc0a29f0 0x7ff2fc0a4eb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff314197a40 0x7ff314197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff314105530 0x7ff31419ddd0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 --2- 172.21.15.22:0/1442332102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314100460 0x7ff31419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 >> 172.21.15.22:0/1442332102 conn(0x7ff3140772f0 msgr2=0x7ff314068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 shutdown_connections 2024-03-09T00:11:21.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.712+0000 7ff318da9640 1 -- 172.21.15.22:0/1442332102 wait complete. 2024-03-09T00:11:21.719 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:11:21.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:11:21.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 -- 172.21.15.22:0/2696486942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 msgr2=0x7f107410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 --2- 172.21.15.22:0/2696486942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 0x7f107410d080 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f106c009ca0 tx=0x7f106c02c0f0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 -- 172.21.15.22:0/2696486942 shutdown_connections 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 --2- 172.21.15.22:0/2696486942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 0x7f107410d080 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 --2- 172.21.15.22:0/2696486942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1074100d90 0x7f1074104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 --2- 172.21.15.22:0/2696486942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10741003e0 0x7f10741007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 -- 172.21.15.22:0/2696486942 >> 172.21.15.22:0/2696486942 conn(0x7f10740773a0 msgr2=0x7f10740777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 -- 172.21.15.22:0/2696486942 shutdown_connections 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 -- 172.21.15.22:0/2696486942 wait complete. 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 Processor -- start 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.840+0000 7f107abf1640 1 -- start start 2024-03-09T00:11:21.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107abf1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10741003e0 0x7f107419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107abf1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 0x7f107419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107abf1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10741979a0 0x7f1074197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107abf1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1074110570 con 0x7f10741054b0 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107abf1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741103f0 con 0x7f10741003e0 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107abf1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741106f0 con 0x7f10741979a0 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f1073fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 0x7f107419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107898d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10741003e0 0x7f107419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f1073fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 0x7f107419dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49500/0 (socket says 172.21.15.22:49500) 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f1073fff640 1 -- 172.21.15.22:0/2251758278 learned_addr learned my addr 172.21.15.22:0/2251758278 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107918e640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10741979a0 0x7f1074197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107898d640 1 -- 172.21.15.22:0/2251758278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10741979a0 msgr2=0x7f1074197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107898d640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10741979a0 0x7f1074197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107898d640 1 -- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 msgr2=0x7f107419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107898d640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 0x7f107419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.841+0000 7f107898d640 1 -- 172.21.15.22:0/2251758278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10741984c0 con 0x7f10741003e0 2024-03-09T00:11:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.842+0000 7f107898d640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10741003e0 0x7f107419d7f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f106800e9f0 tx=0x7f106800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.842+0000 7f1071ffb640 1 -- 172.21.15.22:0/2251758278 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106800cd00 con 0x7f10741003e0 2024-03-09T00:11:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.842+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10741987b0 con 0x7f10741003e0 2024-03-09T00:11:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.842+0000 7f1071ffb640 1 -- 172.21.15.22:0/2251758278 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10680035f0 con 0x7f10741003e0 2024-03-09T00:11:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.842+0000 7f1071ffb640 1 -- 172.21.15.22:0/2251758278 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068010690 con 0x7f10741003e0 2024-03-09T00:11:21.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.842+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10741a4a00 con 0x7f10741003e0 2024-03-09T00:11:21.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.845+0000 7f1071ffb640 1 -- 172.21.15.22:0/2251758278 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f10680037a0 con 0x7f10741003e0 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.845+0000 7f1071ffb640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10640a29b0 0x7f10640a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.845+0000 7f107d7d2740 1 Processor -- start 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107d7d2740 1 -- start start 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f1071ffb640 1 -- 172.21.15.22:0/2251758278 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f10680cf4b0 con 0x7f10741003e0 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f1073fff640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10640a29b0 0x7f10640a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce5b80 0x55dbfcce5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce6530 0x55dbfccf7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107d7d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbfccad360 con 0x55dbfccf7eb0 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107d7d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbfcc49ec0 con 0x55dbfcce6530 2024-03-09T00:11:21.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107d7d2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbfcbf2d30 con 0x55dbfcce5b80 2024-03-09T00:11:21.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107918e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107918e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49514/0 (socket says 172.21.15.22:49514) 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f1073fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce6530 0x55dbfccf7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107918e640 1 -- 172.21.15.22:0/1193709420 learned_addr learned my addr 172.21.15.22:0/1193709420 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.846+0000 7f107898d640 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce5b80 0x55dbfcce5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f1073fff640 1 -- 172.21.15.22:0/1193709420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce5b80 msgr2=0x55dbfcce5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f1073fff640 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce5b80 0x55dbfcce5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f1073fff640 1 -- 172.21.15.22:0/1193709420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 msgr2=0x55dbfccfa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f1073fff640 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f1073fff640 1 -- 172.21.15.22:0/1193709420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbfcbf1bc0 con 0x55dbfcce6530 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f1073fff640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10640a29b0 0x7f10640a4e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f10740ffb10 tx=0x7f1060013460 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f1073fff640 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce6530 0x55dbfccf7970 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1060005870 tx=0x7f106001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6da4a93f74362391 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f105effd640 1 -- 172.21.15.22:0/1193709420 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000fe20 con 0x55dbfcce6530 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f105effd640 1 -- 172.21.15.22:0/1193709420 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1060005500 con 0x55dbfcce6530 2024-03-09T00:11:21.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.847+0000 7f105effd640 1 -- 172.21.15.22:0/1193709420 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000e970 con 0x55dbfcce6530 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 -- 172.21.15.22:0/1193709420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce6530 msgr2=0x55dbfccf7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce6530 0x55dbfccf7970 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1060005870 tx=0x7f106001d000 comp rx=0 tx=0).stop 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 -- 172.21.15.22:0/1193709420 shutdown_connections 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce6530 0x55dbfccf7970 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1193709420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce5b80 0x55dbfcce5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 -- 172.21.15.22:0/1193709420 >> 172.21.15.22:0/1193709420 conn(0x55dbfcce08d0 msgr2=0x55dbfcce0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 -- 172.21.15.22:0/1193709420 shutdown_connections 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 -- 172.21.15.22:0/1193709420 wait complete. 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 -- start start 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.848+0000 7f107d7d2740 1 Processor -- start 2024-03-09T00:11:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccf4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccf53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccf58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbfcbe84a0 con 0x55dbfccf7eb0 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbfcc45f60 con 0x55dbfcce5b80 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbfccfac10 con 0x55dbfcce6530 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 10 client.0 _ll_register_callbacks cb 0x55dbfcc43d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccf58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccf58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49526/0 (socket says 172.21.15.22:49526) 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 -- 172.21.15.22:0/3078451278 learned_addr learned my addr 172.21.15.22:0/3078451278 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f1073fff640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccf53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107898d640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccf4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 -- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 msgr2=0x55dbfccf53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccf53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 -- 172.21.15.22:0/3078451278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 msgr2=0x55dbfccf4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccf4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 -- 172.21.15.22:0/3078451278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbfc95e3f0 con 0x55dbfccf7eb0 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f1073fff640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccf53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107898d640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccf4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107918e640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccf58f0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f106c037040 tx=0x7f106c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a11a99cf2e8b78d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f1053fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f1053fff640 1 -- 172.21.15.22:0/3078451278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c044070 con 0x55dbfccf7eb0 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.849+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1060005500 con 0x55dbfccf7eb0 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f1053fff640 1 -- 172.21.15.22:0/3078451278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f106c032790 con 0x55dbfccf7eb0 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f1053fff640 1 -- 172.21.15.22:0/3078451278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f106c039040 con 0x55dbfccf7eb0 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 10 client.20673 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55dbfca287d0 con 0x55dbfccf7eb0 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f1053fff640 1 -- 172.21.15.22:0/3078451278 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1060005500 con 0x55dbfccf7eb0 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f1053fff640 1 -- 172.21.15.22:0/3078451278 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f106c004390 con 0x55dbfccf7eb0 2024-03-09T00:11:21.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 10 client.20673 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 10 client.20673 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 20 client.20673 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd07ff0 0x55dbfcd0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 4 client.20673 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbfcc57400 con 0x55dbfcd07ff0 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.850+0000 7f107898d640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd07ff0 0x55dbfcd0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.851+0000 7f107898d640 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd07ff0 0x55dbfcd0a410 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdc11b1b4ab44bfe server_cookie=9cc056983beb7dea in_seq=0 out_seq=0 2024-03-09T00:11:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.851+0000 7f1053fff640 10 client.20673 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:21.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.851+0000 7f1053fff640 1 -- 172.21.15.22:0/3078451278 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbfcc57400 con 0x55dbfcd07ff0 2024-03-09T00:11:21.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.851+0000 7f1053fff640 10 client.20673 handle_command_reply: tid=0 2024-03-09T00:11:21.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.851+0000 7f107d7d2740 1 client.20673 shutdown 2024-03-09T00:11:21.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 msgr2=0x55dbfccf58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccf58f0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f106c037040 tx=0x7f106c009960 comp rx=0 tx=0).stop 2024-03-09T00:11:21.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 shutdown_connections 2024-03-09T00:11:21.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd07ff0 0x55dbfcd0a410 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfccf58f0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccf53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f1053fff640 0 client.20673 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3078451278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccf4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 >> 172.21.15.22:0/3078451278 conn(0x55dbfcce08d0 msgr2=0x55dbfcce2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 shutdown_connections 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 1 -- 172.21.15.22:0/3078451278 wait complete. 2024-03-09T00:11:21.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.852+0000 7f107d7d2740 20 client.20673 trim_cache size 0 max 16384 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.864+0000 7f107d7d2740 1 Processor -- start 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.864+0000 7f107d7d2740 1 -- start start 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfcce5b80 0x55dbfccf75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccdf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107d7d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741aead0 con 0x55dbfcce5b80 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107d7d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741ac670 con 0x55dbfccf7eb0 2024-03-09T00:11:21.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107d7d2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741aa730 con 0x55dbfcce6530 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfcce5b80 0x55dbfccf75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfcce5b80 0x55dbfccf75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49540/0 (socket says 172.21.15.22:49540) 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 -- 172.21.15.22:0/1615939603 learned_addr learned my addr 172.21.15.22:0/1615939603 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107918e640 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f1073fff640 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccdf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 -- 172.21.15.22:0/1615939603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 msgr2=0x55dbfccdf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccdf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 -- 172.21.15.22:0/1615939603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfccf7eb0 msgr2=0x55dbfccfeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 -- 172.21.15.22:0/1615939603 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbfccefc80 con 0x55dbfcce5b80 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f1073fff640 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccdf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f107898d640 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfcce5b80 0x55dbfccf75a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f106809b650 tx=0x7f1068009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7210e617a497a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f105e7fc640 1 -- 172.21.15.22:0/1615939603 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10680b8070 con 0x55dbfcce5b80 2024-03-09T00:11:21.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f105e7fc640 1 -- 172.21.15.22:0/1615939603 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f106809f810 con 0x55dbfcce5b80 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.865+0000 7f105e7fc640 1 -- 172.21.15.22:0/1615939603 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068097d30 con 0x55dbfcce5b80 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 -- 172.21.15.22:0/1615939603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfcce5b80 msgr2=0x55dbfccf75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfcce5b80 0x55dbfccf75a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f106809b650 tx=0x7f1068009510 comp rx=0 tx=0).stop 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 -- 172.21.15.22:0/1615939603 shutdown_connections 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfccf7eb0 0x55dbfccfeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccdf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 --2- 172.21.15.22:0/1615939603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfcce5b80 0x55dbfccf75a0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 -- 172.21.15.22:0/1615939603 >> 172.21.15.22:0/1615939603 conn(0x55dbfcce08d0 msgr2=0x55dbfccdf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 -- 172.21.15.22:0/1615939603 shutdown_connections 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 -- 172.21.15.22:0/1615939603 wait complete. 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.866+0000 7f107d7d2740 1 -- start start 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 1 Processor -- start 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccfe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccfe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfcd58540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741aad30 con 0x55dbfccf7eb0 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741ac010 con 0x55dbfcce5b80 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741ab350 con 0x55dbfcce6530 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107d7d2740 10 client.0 _ll_register_callbacks cb 0x7f1060009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfcd58540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107898d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccfe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfcd58540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49542/0 (socket says 172.21.15.22:49542) 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 -- 172.21.15.22:0/3257573279 learned_addr learned my addr 172.21.15.22:0/3257573279 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f1073fff640 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccfe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 -- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 msgr2=0x55dbfccfe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccfe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 -- 172.21.15.22:0/3257573279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 msgr2=0x55dbfccfe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccfe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 -- 172.21.15.22:0/3257573279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbfcd58b80 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f1073fff640 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccfe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f107918e640 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfcd58540 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f106c0099d0 tx=0x7f106c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace78af7770d9097 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.867+0000 7f1053fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f1053fff640 1 -- 172.21.15.22:0/3257573279 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c044070 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f1053fff640 1 -- 172.21.15.22:0/3257573279 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f106c02c770 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f106809f810 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f1053fff640 1 -- 172.21.15.22:0/3257573279 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f106c005bd0 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 10 client.20679 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55dbfcd5ff30 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f1053fff640 1 -- 172.21.15.22:0/3257573279 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f106809f810 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f1053fff640 1 -- 172.21.15.22:0/3257573279 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f106c040910 con 0x55dbfccf7eb0 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 10 client.20679 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 10 client.20679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 20 client.20679 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:21.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd604c0 0x55dbfcd628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 4 client.20679 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dbfcc51750 con 0x55dbfcd604c0 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.868+0000 7f107898d640 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd604c0 0x55dbfcd628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f1053fff640 10 client.20679 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107898d640 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd604c0 0x55dbfcd628e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c316e5cf1c3ab370 server_cookie=b92f69d44adf2012 in_seq=0 out_seq=0 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f1053fff640 1 -- 172.21.15.22:0/3257573279 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dbfcc51750 con 0x55dbfcd604c0 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f1053fff640 10 client.20679 handle_command_reply: tid=0 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 client.20679 shutdown 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 msgr2=0x55dbfcd58540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfcd58540 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f106c0099d0 tx=0x7f106c034670 comp rx=0 tx=0).stop 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 shutdown_connections 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55dbfcd604c0 0x55dbfcd628e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbfccf7eb0 0x55dbfcd58540 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dbfcce6530 0x55dbfccfe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f1053fff640 0 client.20679 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 --2- 172.21.15.22:0/3257573279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dbfcce5b80 0x55dbfccfe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 >> 172.21.15.22:0/3257573279 conn(0x55dbfcce08d0 msgr2=0x55dbfccdf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 shutdown_connections 2024-03-09T00:11:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 1 -- 172.21.15.22:0/3257573279 wait complete. 2024-03-09T00:11:21.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.869+0000 7f107d7d2740 20 client.20679 trim_cache size 0 max 16384 2024-03-09T00:11:21.872 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:11:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10640a29b0 msgr2=0x7f10640a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10640a29b0 0x7f10640a4e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f10740ffb10 tx=0x7f1060013460 comp rx=0 tx=0).stop 2024-03-09T00:11:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10741003e0 msgr2=0x7f107419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10741003e0 0x7f107419d7f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f106800e9f0 tx=0x7f106800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 shutdown_connections 2024-03-09T00:11:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f10640a29b0 0x7f10640a4e70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10741979a0 0x7f1074197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10741054b0 0x7f107419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 --2- 172.21.15.22:0/2251758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10741003e0 0x7f107419d7f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 >> 172.21.15.22:0/2251758278 conn(0x7f10740773a0 msgr2=0x7f10740feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 shutdown_connections 2024-03-09T00:11:21.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.870+0000 7f107abf1640 1 -- 172.21.15.22:0/2251758278 wait complete. 2024-03-09T00:11:21.877 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:11:21.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.989+0000 7f09ad71b640 1 -- 172.21.15.22:0/2016640919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a8102510 msgr2=0x7f09a81028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.989+0000 7f09ad71b640 1 --2- 172.21.15.22:0/2016640919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a8102510 0x7f09a81028f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009900 tx=0x7f09a002bcd0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 -- 172.21.15.22:0/2016640919 shutdown_connections 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 --2- 172.21.15.22:0/2016640919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a810b850 0x7f09a810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 --2- 172.21.15.22:0/2016640919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8102e30 0x7f09a810b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 --2- 172.21.15.22:0/2016640919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a8102510 0x7f09a81028f0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 -- 172.21.15.22:0/2016640919 >> 172.21.15.22:0/2016640919 conn(0x7f09a80773a0 msgr2=0x7f09a80777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 -- 172.21.15.22:0/2016640919 shutdown_connections 2024-03-09T00:11:21.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 -- 172.21.15.22:0/2016640919 wait complete. 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 Processor -- start 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 -- start start 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.990+0000 7f09ad71b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8102510 0x7f09a8100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09ad71b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a8102e30 0x7f09a81006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09ad71b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a810b850 0x7f09a8100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09ad71b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8111610 con 0x7f09a810b850 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09ad71b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8111490 con 0x7f09a8102e30 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09ad71b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8111790 con 0x7f09a8102510 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a810b850 0x7f09a8100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a810b850 0x7f09a8100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49568/0 (socket says 172.21.15.22:49568) 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 -- 172.21.15.22:0/39949801 learned_addr learned my addr 172.21.15.22:0/39949801 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a67fc640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a8102e30 0x7f09a81006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8102510 0x7f09a8100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 -- 172.21.15.22:0/39949801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8102510 msgr2=0x7f09a8100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8102510 0x7f09a8100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 -- 172.21.15.22:0/39949801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a8102e30 msgr2=0x7f09a81006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a8102e30 0x7f09a81006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 -- 172.21.15.22:0/39949801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a81a6010 con 0x7f09a810b850 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09a77fe640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a810b850 0x7f09a8100be0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f099800eb10 tx=0x7f099800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f099ffff640 1 -- 172.21.15.22:0/39949801 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998004b20 con 0x7f09a810b850 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f099ffff640 1 -- 172.21.15.22:0/39949801 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0998004cc0 con 0x7f09a810b850 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a81a6300 con 0x7f09a810b850 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f099ffff640 1 -- 172.21.15.22:0/39949801 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998010760 con 0x7f09a810b850 2024-03-09T00:11:21.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.991+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a81a6800 con 0x7f09a810b850 2024-03-09T00:11:21.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f099ffff640 1 -- 172.21.15.22:0/39949801 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0998005020 con 0x7f09a810b850 2024-03-09T00:11:21.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 Processor -- start 2024-03-09T00:11:21.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 -- start start 2024-03-09T00:11:21.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee41a0 0x563505ee4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee4ac0 0x563505eed160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505eed6a0 0x563505eefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563505eab360 con 0x563505ee41a0 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563505e47ec0 con 0x563505ee4ac0 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09b03d4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563505df0d30 con 0x563505eed6a0 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f099ffff640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f09940a2970 0x7f09940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee41a0 0x563505ee4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09a67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee41a0 0x563505ee4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49582/0 (socket says 172.21.15.22:49582) 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.994+0000 7f09a67fc640 1 -- 172.21.15.22:0/2502123689 learned_addr learned my addr 172.21.15.22:0/2502123689 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee4ac0 0x563505eed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f099ffff640 1 -- 172.21.15.22:0/39949801 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f09980cf7c0 con 0x7f09a810b850 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a77fe640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f09940a2970 0x7f09940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a6ffd640 1 -- 172.21.15.22:0/2502123689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505eed6a0 msgr2=0x563505eefa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505eed6a0 0x563505eefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a6ffd640 1 -- 172.21.15.22:0/2502123689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee41a0 msgr2=0x563505ee4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee41a0 0x563505ee4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a6ffd640 1 -- 172.21.15.22:0/2502123689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563505defbc0 con 0x563505ee4ac0 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a67fc640 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee41a0 0x563505ee4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a77fe640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f09940a2970 0x7f09940a4e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09980ccb20 tx=0x7f0998086040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.995+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee4ac0 0x563505eed160 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f09a0000c00 tx=0x7f09a0009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea827e8290791be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.996+0000 7f099d7fa640 1 -- 172.21.15.22:0/2502123689 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0042070 con 0x563505ee4ac0 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.996+0000 7f099d7fa640 1 -- 172.21.15.22:0/2502123689 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09a002c730 con 0x563505ee4ac0 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.996+0000 7f099d7fa640 1 -- 172.21.15.22:0/2502123689 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0004f10 con 0x563505ee4ac0 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 -- 172.21.15.22:0/2502123689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee4ac0 msgr2=0x563505eed160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee4ac0 0x563505eed160 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f09a0000c00 tx=0x7f09a0009630 comp rx=0 tx=0).stop 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 -- 172.21.15.22:0/2502123689 shutdown_connections 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505eed6a0 0x563505eefa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee4ac0 0x563505eed160 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 --2- 172.21.15.22:0/2502123689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee41a0 0x563505ee4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:21.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 -- 172.21.15.22:0/2502123689 >> 172.21.15.22:0/2502123689 conn(0x563505ede8d0 msgr2=0x563505edecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:21.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 -- 172.21.15.22:0/2502123689 shutdown_connections 2024-03-09T00:11:21.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.997+0000 7f09b03d4740 1 -- 172.21.15.22:0/2502123689 wait complete. 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 -- start start 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 Processor -- start 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505eea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505eeabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505eeb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563505de64a0 con 0x563505eed6a0 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563505e43f60 con 0x563505ee41a0 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563505ef04d0 con 0x563505ee4ac0 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.998+0000 7f09b03d4740 10 client.0 _ll_register_callbacks cb 0x563505e41d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505eea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505eeabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505eea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51276/0 (socket says 172.21.15.22:51276) 2024-03-09T00:11:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a67fc640 1 -- 172.21.15.22:0/4280652913 learned_addr learned my addr 172.21.15.22:0/4280652913 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a67fc640 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505eeb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a6ffd640 1 -- 172.21.15.22:0/4280652913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 msgr2=0x563505eea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505eea680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a6ffd640 1 -- 172.21.15.22:0/4280652913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 msgr2=0x563505eeb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505eeb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a6ffd640 1 -- 172.21.15.22:0/4280652913 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563505b5c3f0 con 0x563505ee4ac0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:21.999+0000 7f09a67fc640 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505eeb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505eeabc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09a0036040 tx=0x7f09a002cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a809837371125da server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f098e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f098e7fc640 1 -- 172.21.15.22:0/4280652913 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0042040 con 0x563505ee4ac0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563505e55400 con 0x563505ee4ac0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f098e7fc640 1 -- 172.21.15.22:0/4280652913 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f09a0005120 con 0x563505ee4ac0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f098e7fc640 1 -- 172.21.15.22:0/4280652913 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a0058050 con 0x563505ee4ac0 2024-03-09T00:11:22.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f09b03d4740 10 client.30067 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.000+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x563505c267d0 con 0x563505ee4ac0 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.001+0000 7f098e7fc640 1 -- 172.21.15.22:0/4280652913 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563505e55400 con 0x563505ee4ac0 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.002+0000 7f098e7fc640 1 -- 172.21.15.22:0/4280652913 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f09a0047c40 con 0x563505ee4ac0 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.002+0000 7f09b03d4740 10 client.30067 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.002+0000 7f09b03d4740 10 client.30067 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.002+0000 7f09b03d4740 20 client.30067 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.002+0000 7f09b03d4740 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f06180 0x563505f08570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.002+0000 7f09b03d4740 4 client.30067 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:22.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.002+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563505e4f750 con 0x563505f06180 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.003+0000 7f09a67fc640 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f06180 0x563505f08570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.004+0000 7f098e7fc640 10 client.30067 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.004+0000 7f09a67fc640 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f06180 0x563505f08570 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebbc1ca4670c38fe server_cookie=270c35ac08ff3312 in_seq=0 out_seq=0 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f098e7fc640 1 -- 172.21.15.22:0/4280652913 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563505e4f750 con 0x563505f06180 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f098e7fc640 10 client.30067 handle_command_reply: tid=0 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 client.30067 shutdown 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 msgr2=0x563505eeabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505eeabc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09a0036040 tx=0x7f09a002cb50 comp rx=0 tx=0).stop 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 shutdown_connections 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f06180 0x563505f08570 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505eeb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f098e7fc640 0 client.30067 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505eeabc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 --2- 172.21.15.22:0/4280652913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505eea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.005+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 >> 172.21.15.22:0/4280652913 conn(0x563505ede8d0 msgr2=0x563505ee0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.006+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 shutdown_connections 2024-03-09T00:11:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.006+0000 7f09b03d4740 1 -- 172.21.15.22:0/4280652913 wait complete. 2024-03-09T00:11:22.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.006+0000 7f09b03d4740 20 client.30067 trim_cache size 0 max 16384 2024-03-09T00:11:22.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.029+0000 7f09b03d4740 1 Processor -- start 2024-03-09T00:11:22.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.029+0000 7f09b03d4740 1 -- start start 2024-03-09T00:11:22.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.029+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee41a0 0x563505eecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.029+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee4ac0 0x563505eeaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505eed6a0 0x563505ee5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09b03d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81ae980 con 0x563505ee4ac0 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09b03d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81ac520 con 0x563505eed6a0 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09a67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee41a0 0x563505eecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09a67fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee41a0 0x563505eecdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41660/0 (socket says 172.21.15.22:41660) 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09a67fc640 1 -- 172.21.15.22:0/426045103 learned_addr learned my addr 172.21.15.22:0/426045103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee4ac0 0x563505eeaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.030+0000 7f09b03d4740 1 -- 172.21.15.22:0/426045103 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81aa600 con 0x563505ee41a0 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.031+0000 7f09a6ffd640 1 -- 172.21.15.22:0/426045103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee41a0 msgr2=0x563505eecdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.031+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee41a0 0x563505eecdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.031+0000 7f09a6ffd640 1 -- 172.21.15.22:0/426045103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505eed6a0 msgr2=0x563505ee5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.031+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505eed6a0 0x563505ee5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.031+0000 7f09a6ffd640 1 -- 172.21.15.22:0/426045103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563505f05f30 con 0x563505ee4ac0 2024-03-09T00:11:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.031+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee4ac0 0x563505eeaf20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f09a0005c40 tx=0x7f09a0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82394c71f6b07e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.031+0000 7f099cff9640 1 -- 172.21.15.22:0/426045103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0054070 con 0x563505ee4ac0 2024-03-09T00:11:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.032+0000 7f099cff9640 1 -- 172.21.15.22:0/426045103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09a0034e40 con 0x563505ee4ac0 2024-03-09T00:11:22.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.032+0000 7f099cff9640 1 -- 172.21.15.22:0/426045103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0042040 con 0x563505ee4ac0 2024-03-09T00:11:22.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 -- 172.21.15.22:0/426045103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee4ac0 msgr2=0x563505eeaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee4ac0 0x563505eeaf20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f09a0005c40 tx=0x7f09a0001e50 comp rx=0 tx=0).stop 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 -- 172.21.15.22:0/426045103 shutdown_connections 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505eed6a0 0x563505ee5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505ee4ac0 0x563505eeaf20 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 --2- 172.21.15.22:0/426045103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee41a0 0x563505eecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 -- 172.21.15.22:0/426045103 >> 172.21.15.22:0/426045103 conn(0x563505ede8d0 msgr2=0x563505edecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.033+0000 7f09b03d4740 1 -- 172.21.15.22:0/426045103 shutdown_connections 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.034+0000 7f09b03d4740 1 -- 172.21.15.22:0/426045103 wait complete. 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.034+0000 7f09b03d4740 1 -- start start 2024-03-09T00:11:22.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.034+0000 7f09b03d4740 1 Processor -- start 2024-03-09T00:11:22.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.035+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505efacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.035+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505efb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.035+0000 7f09b03d4740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505f5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.035+0000 7f09b03d4740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81aac00 con 0x563505eed6a0 2024-03-09T00:11:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.035+0000 7f09b03d4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81abec0 con 0x563505ee41a0 2024-03-09T00:11:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.035+0000 7f09b03d4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81ab200 con 0x563505ee4ac0 2024-03-09T00:11:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.035+0000 7f09b03d4740 10 client.0 _ll_register_callbacks cb 0x7f09a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:22.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.036+0000 7f09a6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505efb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.037+0000 7f09a6ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505efb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41662/0 (socket says 172.21.15.22:41662) 2024-03-09T00:11:22.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.037+0000 7f09a6ffd640 1 -- 172.21.15.22:0/1071727336 learned_addr learned my addr 172.21.15.22:0/1071727336 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.038+0000 7f09a6ffd640 1 -- 172.21.15.22:0/1071727336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 msgr2=0x563505efacc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:11:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.038+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505efacc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.038+0000 7f09a6ffd640 1 -- 172.21.15.22:0/1071727336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 msgr2=0x563505f5b650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:11:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.038+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505f5b650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.038+0000 7f09a6ffd640 1 -- 172.21.15.22:0/1071727336 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563505efbad0 con 0x563505ee4ac0 2024-03-09T00:11:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.040+0000 7f09a6ffd640 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505efb200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f09a0004ae0 tx=0x7f09a00345b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b20df62e9d20229 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.040+0000 7f098e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:11:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.041+0000 7f098e7fc640 1 -- 172.21.15.22:0/1071727336 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0054040 con 0x563505ee4ac0 2024-03-09T00:11:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.041+0000 7f098e7fc640 1 -- 172.21.15.22:0/1071727336 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f09a0034b60 con 0x563505ee4ac0 2024-03-09T00:11:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.041+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563505dcd640 con 0x563505ee4ac0 2024-03-09T00:11:22.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.041+0000 7f098e7fc640 1 -- 172.21.15.22:0/1071727336 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a006b050 con 0x563505ee4ac0 2024-03-09T00:11:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.042+0000 7f09b03d4740 10 client.30073 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:22.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.042+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x563505f5cc60 con 0x563505ee4ac0 2024-03-09T00:11:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f098e7fc640 1 -- 172.21.15.22:0/1071727336 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563505dcd640 con 0x563505ee4ac0 2024-03-09T00:11:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f098e7fc640 1 -- 172.21.15.22:0/1071727336 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f09a00035f0 con 0x563505ee4ac0 2024-03-09T00:11:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f09b03d4740 10 client.30073 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f09b03d4740 10 client.30073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:11:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f09b03d4740 20 client.30073 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f09b03d4740 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f5d2a0 0x563505f5f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f09b03d4740 4 client.30073 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:11:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.046+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563505e19c70 con 0x563505f5d2a0 2024-03-09T00:11:22.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.048+0000 7f09a67fc640 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f5d2a0 0x563505f5f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09a67fc640 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f5d2a0 0x563505f5f6c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61c09eded2fb7454 server_cookie=dd7c93fec5c1b864 in_seq=0 out_seq=0 2024-03-09T00:11:22.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f098e7fc640 10 client.30073 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f098e7fc640 1 -- 172.21.15.22:0/1071727336 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563505e19c70 con 0x563505f5d2a0 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f098e7fc640 10 client.30073 handle_command_reply: tid=0 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 client.30073 shutdown 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 msgr2=0x563505efb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505efb200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f09a0004ae0 tx=0x7f09a00345b0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 shutdown_connections 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x563505f5d2a0 0x563505f5f6c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563505eed6a0 0x563505f5b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f098e7fc640 0 client.30073 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563505ee4ac0 0x563505efb200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 --2- 172.21.15.22:0/1071727336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563505ee41a0 0x563505efacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 >> 172.21.15.22:0/1071727336 conn(0x563505ede8d0 msgr2=0x563505ee5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 shutdown_connections 2024-03-09T00:11:22.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 1 -- 172.21.15.22:0/1071727336 wait complete. 2024-03-09T00:11:22.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.049+0000 7f09b03d4740 20 client.30073 trim_cache size 0 max 16384 2024-03-09T00:11:22.047 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f09940a2970 msgr2=0x7f09940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f09940a2970 0x7f09940a4e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09980ccb20 tx=0x7f0998086040 comp rx=0 tx=0).stop 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a810b850 msgr2=0x7f09a8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a810b850 0x7f09a8100be0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f099800eb10 tx=0x7f099800c2d0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 shutdown_connections 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f09940a2970 0x7f09940a4e30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09a810b850 0x7f09a8100be0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a8102e30 0x7f09a81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 --2- 172.21.15.22:0/39949801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a8102510 0x7f09a8100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 >> 172.21.15.22:0/39949801 conn(0x7f09a80773a0 msgr2=0x7f09a80fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 shutdown_connections 2024-03-09T00:11:22.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.050+0000 7f09ad71b640 1 -- 172.21.15.22:0/39949801 wait complete. 2024-03-09T00:11:22.061 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:11:22.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:11:22.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 48545 11.28 MB/sec execute 89 sec latency 287.719 ms 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 -- 172.21.15.22:0/680191332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c8100d90 msgr2=0x7f43c8104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 --2- 172.21.15.22:0/680191332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c8100d90 0x7f43c8104dd0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009900 tx=0x7f43bc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 -- 172.21.15.22:0/680191332 shutdown_connections 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 --2- 172.21.15.22:0/680191332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c81054b0 0x7f43c810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 --2- 172.21.15.22:0/680191332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c8100d90 0x7f43c8104dd0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 --2- 172.21.15.22:0/680191332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c81003e0 0x7f43c81007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 -- 172.21.15.22:0/680191332 >> 172.21.15.22:0/680191332 conn(0x7f43c80773a0 msgr2=0x7f43c80777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 -- 172.21.15.22:0/680191332 shutdown_connections 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.188+0000 7f43ce540640 1 -- 172.21.15.22:0/680191332 wait complete. 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 Processor -- start 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 -- start start 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c81003e0 0x7f43c819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c81054b0 0x7f43c819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c8197900 0x7f43c8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c8110520 con 0x7f43c81054b0 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81103a0 con 0x7f43c8197900 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ce540640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81106a0 con 0x7f43c81003e0 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c81054b0 0x7f43c819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c81054b0 0x7f43c819dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49620/0 (socket says 172.21.15.22:49620) 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43c77fe640 1 -- 172.21.15.22:0/126525821 learned_addr learned my addr 172.21.15.22:0/126525821 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43c7fff640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c81003e0 0x7f43c819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43ccadd640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c8197900 0x7f43c8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.189+0000 7f43c77fe640 1 -- 172.21.15.22:0/126525821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c81003e0 msgr2=0x7f43c819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c77fe640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c81003e0 0x7f43c819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c77fe640 1 -- 172.21.15.22:0/126525821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c8197900 msgr2=0x7f43c8197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c77fe640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c8197900 0x7f43c8197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c77fe640 1 -- 172.21.15.22:0/126525821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c8198420 con 0x7f43c81054b0 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c7fff640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c81003e0 0x7f43c819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c77fe640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c81054b0 0x7f43c819dc90 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f43bc036040 tx=0x7f43bc02cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c57fa640 1 -- 172.21.15.22:0/126525821 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc041070 con 0x7f43c81054b0 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c81986b0 con 0x7f43c81054b0 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c57fa640 1 -- 172.21.15.22:0/126525821 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43bc004450 con 0x7f43c81054b0 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43c57fa640 1 -- 172.21.15.22:0/126525821 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003dc0 con 0x7f43c81054b0 2024-03-09T00:11:22.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.190+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c81a48a0 con 0x7f43c81054b0 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.192+0000 7f43c57fa640 1 -- 172.21.15.22:0/126525821 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f43bc044070 con 0x7f43c81054b0 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 Processor -- start 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 -- start start 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19611580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e19611ac0 0x562e1961a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e195d8360 con 0x562e19611ac0 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e19574ec0 con 0x562e196111a0 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43d11f1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1951dd30 con 0x562e1961a6a0 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c57fa640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43b00a2970 0x7f43b00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e19611ac0 0x562e1961a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43ccadd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19611580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e19611ac0 0x562e1961a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49624/0 (socket says 172.21.15.22:49624) 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c7fff640 1 -- 172.21.15.22:0/3407888085 learned_addr learned my addr 172.21.15.22:0/3407888085 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43ccadd640 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19611580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51326/0 (socket says 172.21.15.22:51326) 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43ccadd640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43b00a2970 0x7f43b00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c57fa640 1 -- 172.21.15.22:0/126525821 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f43bc0f0d00 con 0x7f43c81054b0 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c77fe640 1 -- 172.21.15.22:0/3407888085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 msgr2=0x562e19611580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c77fe640 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19611580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c77fe640 1 -- 172.21.15.22:0/3407888085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e19611ac0 msgr2=0x562e1961a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c77fe640 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e19611ac0 0x562e1961a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c77fe640 1 -- 172.21.15.22:0/3407888085 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e1951cbc0 con 0x562e1961a6a0 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43c7fff640 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e19611ac0 0x562e1961a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.193+0000 7f43ccadd640 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19611580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.194+0000 7f43c77fe640 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961ca90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0ee110 tx=0x7f43bc004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68871dc2db4ad52f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.194+0000 7f43ccadd640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43b00a2970 0x7f43b00a4e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f43c0005c40 tx=0x7f43c0009500 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.194+0000 7f43ae7fc640 1 -- 172.21.15.22:0/3407888085 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc04b070 con 0x562e1961a6a0 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.194+0000 7f43ae7fc640 1 -- 172.21.15.22:0/3407888085 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43bc0bd720 con 0x562e1961a6a0 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.194+0000 7f43ae7fc640 1 -- 172.21.15.22:0/3407888085 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0035f0 con 0x562e1961a6a0 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 -- 172.21.15.22:0/3407888085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 msgr2=0x562e1961ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961ca90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0ee110 tx=0x7f43bc004ee0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 -- 172.21.15.22:0/3407888085 shutdown_connections 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961ca90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e19611ac0 0x562e1961a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3407888085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19611580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 -- 172.21.15.22:0/3407888085 >> 172.21.15.22:0/3407888085 conn(0x562e1960b8d0 msgr2=0x562e1960bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 -- 172.21.15.22:0/3407888085 shutdown_connections 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 -- 172.21.15.22:0/3407888085 wait complete. 2024-03-09T00:11:22.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 -- start start 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.195+0000 7f43d11f1740 1 Processor -- start 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19617610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19617b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19618090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e195134a0 con 0x562e1961a6a0 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e19570f60 con 0x562e196111a0 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1961d250 con 0x562e19611ac0 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 10 client.0 _ll_register_callbacks cb 0x562e1956ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19618090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19617b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19618090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49640/0 (socket says 172.21.15.22:49640) 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19617610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 -- 172.21.15.22:0/342268610 learned_addr learned my addr 172.21.15.22:0/342268610 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 -- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 msgr2=0x562e19617b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19617b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 -- 172.21.15.22:0/342268610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 msgr2=0x562e19617610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19617610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 -- 172.21.15.22:0/342268610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e192893f0 con 0x562e1961a6a0 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43c7fff640 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19617610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43c77fe640 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19617b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43ccadd640 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19618090 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f43c000dbe0 tx=0x7f43c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61816d286a46b1d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43a77fe640 1 -- 172.21.15.22:0/342268610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0025030 con 0x562e1961a6a0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43a77fe640 1 -- 172.21.15.22:0/342268610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f43c0009040 con 0x562e1961a6a0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.196+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43bc0bd720 con 0x562e1961a6a0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43a77fe640 1 -- 172.21.15.22:0/342268610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43c002f050 con 0x562e1961a6a0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 10 client.20706 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562e193537d0 con 0x562e1961a6a0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43a77fe640 1 -- 172.21.15.22:0/342268610 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43bc0bd720 con 0x562e1961a6a0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43a77fe640 1 -- 172.21.15.22:0/342268610 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f43c00055b0 con 0x562e1961a6a0 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 10 client.20706 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:22.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 10 client.20706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 20 client.20706 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e1962a5b0 0x562e1962c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 4 client.20706 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e19582400 con 0x562e1962a5b0 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.197+0000 7f43c7fff640 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e1962a5b0 0x562e1962c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.198+0000 7f43c7fff640 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e1962a5b0 0x562e1962c9a0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97128a042834f6d server_cookie=13887ebd7e4ff0f7 in_seq=0 out_seq=0 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.198+0000 7f43a77fe640 10 client.20706 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.199+0000 7f43a77fe640 1 -- 172.21.15.22:0/342268610 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e19582400 con 0x562e1962a5b0 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.199+0000 7f43a77fe640 10 client.20706 handle_command_reply: tid=0 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.199+0000 7f43d11f1740 1 client.20706 shutdown 2024-03-09T00:11:22.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 msgr2=0x562e19618090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19618090 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f43c000dbe0 tx=0x7f43c0004a40 comp rx=0 tx=0).stop 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 shutdown_connections 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e1962a5b0 0x562e1962c9a0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19618090 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19617b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43a77fe640 0 client.20706 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 --2- 172.21.15.22:0/342268610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19617610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 >> 172.21.15.22:0/342268610 conn(0x562e1960b8d0 msgr2=0x562e1960d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 shutdown_connections 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 1 -- 172.21.15.22:0/342268610 wait complete. 2024-03-09T00:11:22.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.200+0000 7f43d11f1740 20 client.20706 trim_cache size 0 max 16384 2024-03-09T00:11:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.211+0000 7f43d11f1740 1 Processor -- start 2024-03-09T00:11:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.211+0000 7f43d11f1740 1 -- start start 2024-03-09T00:11:22.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19619d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19614890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19614dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43d11f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81b0cc0 con 0x562e1961a6a0 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43d11f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81ae860 con 0x562e196111a0 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43d11f1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81ac920 con 0x562e19611ac0 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43ccadd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19614dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19614890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19619d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43ccadd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19614dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49644/0 (socket says 172.21.15.22:49644) 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19614890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41724/0 (socket says 172.21.15.22:41724) 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19619d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51352/0 (socket says 172.21.15.22:51352) 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c77fe640 1 -- 172.21.15.22:0/3823042626 learned_addr learned my addr 172.21.15.22:0/3823042626 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c7fff640 1 -- 172.21.15.22:0/3823042626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 msgr2=0x562e19614890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c7fff640 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19614890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c7fff640 1 -- 172.21.15.22:0/3823042626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 msgr2=0x562e19614dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c7fff640 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19614dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c7fff640 1 -- 172.21.15.22:0/3823042626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e19631b90 con 0x562e196111a0 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43c77fe640 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19614890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.212+0000 7f43ccadd640 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19614dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43c7fff640 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19619d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x562e1962cfa0 tx=0x7f43b8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8216257af056a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43adffb640 1 -- 172.21.15.22:0/3823042626 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800fc80 con 0x562e196111a0 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43adffb640 1 -- 172.21.15.22:0/3823042626 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b800fe20 con 0x562e196111a0 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43adffb640 1 -- 172.21.15.22:0/3823042626 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80210f0 con 0x562e196111a0 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43d11f1740 1 -- 172.21.15.22:0/3823042626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 msgr2=0x562e19619d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19619d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x562e1962cfa0 tx=0x7f43b8005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43d11f1740 1 -- 172.21.15.22:0/3823042626 shutdown_connections 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.213+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1961a6a0 0x562e19614dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e19611ac0 0x562e19614890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 --2- 172.21.15.22:0/3823042626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e196111a0 0x562e19619d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 -- 172.21.15.22:0/3823042626 >> 172.21.15.22:0/3823042626 conn(0x562e1960b8d0 msgr2=0x562e1960bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 -- 172.21.15.22:0/3823042626 shutdown_connections 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 -- 172.21.15.22:0/3823042626 wait complete. 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 -- start start 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 Processor -- start 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e196111a0 0x562e196210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e19611ac0 0x562e196215e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81acf20 con 0x562e196111a0 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81ae200 con 0x562e19611ac0 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c81ad540 con 0x562e1961a6a0 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43d11f1740 10 client.0 _ll_register_callbacks cb 0x7f43bc0b9320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:22.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e196111a0 0x562e196210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43c7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e196111a0 0x562e196210a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49648/0 (socket says 172.21.15.22:49648) 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43c7fff640 1 -- 172.21.15.22:0/2494634320 learned_addr learned my addr 172.21.15.22:0/2494634320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.214+0000 7f43ccadd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43c77fe640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e19611ac0 0x562e196215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43ccadd640 1 -- 172.21.15.22:0/2494634320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e19611ac0 msgr2=0x562e196215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43ccadd640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e19611ac0 0x562e196215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43ccadd640 1 -- 172.21.15.22:0/2494634320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e196111a0 msgr2=0x562e196210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43ccadd640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e196111a0 0x562e196210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43ccadd640 1 -- 172.21.15.22:0/2494634320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e1961fe40 con 0x562e1961a6a0 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43c7fff640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e196111a0 0x562e196210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43c77fe640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e19611ac0 0x562e196215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43ccadd640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961f750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43c00059d0 tx=0x7f43c000f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a735cf7ddebdeb7b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-09T00:11:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43a77fe640 1 -- 172.21.15.22:0/2494634320 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0025040 con 0x562e1961a6a0 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43b800fe20 con 0x562e1961a6a0 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43a77fe640 1 -- 172.21.15.22:0/2494634320 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f43c00046c0 con 0x562e1961a6a0 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43a77fe640 1 -- 172.21.15.22:0/2494634320 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43c0043050 con 0x562e1961a6a0 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43d11f1740 10 client.30091 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.215+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562e19681510 con 0x562e1961a6a0 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43a77fe640 1 -- 172.21.15.22:0/2494634320 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b800fe20 con 0x562e1961a6a0 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43a77fe640 1 -- 172.21.15.22:0/2494634320 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f43c003c850 con 0x562e1961a6a0 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43d11f1740 10 client.30091 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43d11f1740 10 client.30091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43d11f1740 20 client.30091 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43d11f1740 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e19681b50 0x562e19683f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43d11f1740 4 client.30091 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:11:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e1957c750 con 0x562e19681b50 2024-03-09T00:11:22.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43c7fff640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e19681b50 0x562e19683f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43a77fe640 10 client.30091 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:22.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.216+0000 7f43c7fff640 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e19681b50 0x562e19683f70 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434ae062c47a5b3 server_cookie=ac6f7d9ce5846473 in_seq=0 out_seq=0 2024-03-09T00:11:22.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43a77fe640 1 -- 172.21.15.22:0/2494634320 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e1957c750 con 0x562e19681b50 2024-03-09T00:11:22.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43a77fe640 10 client.30091 handle_command_reply: tid=0 2024-03-09T00:11:22.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 client.30091 shutdown 2024-03-09T00:11:22.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 msgr2=0x562e1961f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961f750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43c00059d0 tx=0x7f43c000f680 comp rx=0 tx=0).stop 2024-03-09T00:11:22.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 shutdown_connections 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562e19681b50 0x562e19683f70 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562e1961a6a0 0x562e1961f750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e19611ac0 0x562e196215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43a77fe640 0 client.30091 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 --2- 172.21.15.22:0/2494634320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e196111a0 0x562e196210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 >> 172.21.15.22:0/2494634320 conn(0x562e1960b8d0 msgr2=0x562e1960cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 shutdown_connections 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 1 -- 172.21.15.22:0/2494634320 wait complete. 2024-03-09T00:11:22.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.218+0000 7f43d11f1740 20 client.30091 trim_cache size 0 max 16384 2024-03-09T00:11:22.223 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:11:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43b00a2970 msgr2=0x7f43b00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43b00a2970 0x7f43b00a4e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f43c0005c40 tx=0x7f43c0009500 comp rx=0 tx=0).stop 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c81054b0 msgr2=0x7f43c819dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c81054b0 0x7f43c819dc90 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f43bc036040 tx=0x7f43bc02cb20 comp rx=0 tx=0).stop 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 shutdown_connections 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f43b00a2970 0x7f43b00a4e30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c8197900 0x7f43c8197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43c81054b0 0x7f43c819dc90 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 --2- 172.21.15.22:0/126525821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c81003e0 0x7f43c819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 >> 172.21.15.22:0/126525821 conn(0x7f43c80773a0 msgr2=0x7f43c8068840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 shutdown_connections 2024-03-09T00:11:22.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:22.219+0000 7f43ce540640 1 -- 172.21.15.22:0/126525821 wait complete. 2024-03-09T00:11:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 -- 172.21.15.22:0/2309794287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471010dc90 msgr2=0x7f4710110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 --2- 172.21.15.22:0/2309794287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471010dc90 0x7f4710110080 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4708009ca0 tx=0x7f470802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 -- 172.21.15.22:0/2309794287 shutdown_connections 2024-03-09T00:11:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 --2- 172.21.15.22:0/2309794287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471010dc90 0x7f4710110080 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 --2- 172.21.15.22:0/2309794287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710100a00 0x7f471010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 --2- 172.21.15.22:0/2309794287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47101000e0 0x7f47101004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 -- 172.21.15.22:0/2309794287 >> 172.21.15.22:0/2309794287 conn(0x7f4710077350 msgr2=0x7f4710077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 -- 172.21.15.22:0/2309794287 shutdown_connections 2024-03-09T00:11:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 -- 172.21.15.22:0/2309794287 wait complete. 2024-03-09T00:11:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 Processor -- start 2024-03-09T00:11:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.256+0000 7f4715a80640 1 -- start start 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f4715a80640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47101000e0 0x7f471019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f4715a80640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710100a00 0x7f471019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f4715a80640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471010dc90 0x7f4710199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f4715a80640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101138c0 con 0x7f47101000e0 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f4715a80640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710113740 con 0x7f471010dc90 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f4715a80640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710113a40 con 0x7f4710100a00 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47101000e0 0x7f471019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47101000e0 0x7f471019f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49664/0 (socket says 172.21.15.22:49664) 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 -- 172.21.15.22:0/2407809151 learned_addr learned my addr 172.21.15.22:0/2407809151 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470ffff640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471010dc90 0x7f4710199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470effd640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710100a00 0x7f471019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 -- 172.21.15.22:0/2407809151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710100a00 msgr2=0x7f471019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710100a00 0x7f471019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 -- 172.21.15.22:0/2407809151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471010dc90 msgr2=0x7f4710199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471010dc90 0x7f4710199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 -- 172.21.15.22:0/2407809151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471019a260 con 0x7f47101000e0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.257+0000 7f470f7fe640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47101000e0 0x7f471019f930 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f470000ea60 tx=0x7f470000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.258+0000 7f470cff9640 1 -- 172.21.15.22:0/2407809151 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000cde0 con 0x7f47101000e0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.258+0000 7f470cff9640 1 -- 172.21.15.22:0/2407809151 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47000035f0 con 0x7f47101000e0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.258+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471019a550 con 0x7f47101000e0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.258+0000 7f470cff9640 1 -- 172.21.15.22:0/2407809151 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700010670 con 0x7f47101000e0 2024-03-09T00:11:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.258+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47101a67f0 con 0x7f47101000e0 2024-03-09T00:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.260+0000 7f470cff9640 1 -- 172.21.15.22:0/2407809151 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f47000037a0 con 0x7f47101000e0 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.260+0000 7f4718656740 1 Processor -- start 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f4718656740 1 -- start start 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f4718656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbe03a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f4718656740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe04450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f4718656740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe0d550 0x55eefbe0f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f4718656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefbd685f0 con 0x55eefbe04d40 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f4718656740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefbd11930 con 0x55eefbe03fd0 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f4718656740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefbd64690 con 0x55eefbe0d550 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470cff9640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f80a2a90 0x7f46f80a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe04450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbe03a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe0d550 0x55eefbe0f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbe03a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49680/0 (socket says 172.21.15.22:49680) 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 -- 172.21.15.22:0/4031549322 learned_addr learned my addr 172.21.15.22:0/4031549322 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f80a2a90 0x7f46f80a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470cff9640 1 -- 172.21.15.22:0/2407809151 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f47000cf520 con 0x7f47101000e0 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 -- 172.21.15.22:0/4031549322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe0d550 msgr2=0x55eefbe0f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe0d550 0x55eefbe0f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 -- 172.21.15.22:0/4031549322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 msgr2=0x55eefbe04450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe04450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470ffff640 1 -- 172.21.15.22:0/4031549322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eefba7d210 con 0x55eefbe04d40 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470f7fe640 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe04450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.261+0000 7f470effd640 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe0d550 0x55eefbe0f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.262+0000 7f470ffff640 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbe03a90 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4708004af0 tx=0x7f4708032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0df9a34cca9c48 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.262+0000 7f470ffff640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f80a2a90 0x7f46f80a4f50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f470802c630 tx=0x7f4708042000 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.262+0000 7f46f5ffb640 1 -- 172.21.15.22:0/4031549322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708051070 con 0x55eefbe04d40 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.262+0000 7f46f5ffb640 1 -- 172.21.15.22:0/4031549322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f470802ce40 con 0x55eefbe04d40 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.262+0000 7f46f5ffb640 1 -- 172.21.15.22:0/4031549322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708039040 con 0x55eefbe04d40 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 -- 172.21.15.22:0/4031549322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 msgr2=0x55eefbe03a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbe03a90 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4708004af0 tx=0x7f4708032eb0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 -- 172.21.15.22:0/4031549322 shutdown_connections 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe0d550 0x55eefbe0f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe04450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 --2- 172.21.15.22:0/4031549322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbe03a90 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 -- 172.21.15.22:0/4031549322 >> 172.21.15.22:0/4031549322 conn(0x55eefbdcb5b0 msgr2=0x55eefbdfecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 -- 172.21.15.22:0/4031549322 shutdown_connections 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 -- 172.21.15.22:0/4031549322 wait complete. 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 -- start start 2024-03-09T00:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.263+0000 7f4718656740 1 Processor -- start 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f4718656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe03fd0 0x55eefbe0a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f4718656740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe0ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f4718656740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f4718656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefbe10220 con 0x55eefbe03fd0 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f4718656740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefbd24d90 con 0x55eefbe0d550 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f4718656740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eefbe103a0 con 0x55eefbe04d40 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f4718656740 10 client.0 _ll_register_callbacks cb 0x55eefba7ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe03fd0 0x55eefbe0a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe03fd0 0x55eefbe0a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49686/0 (socket says 172.21.15.22:49686) 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470f7fe640 1 -- 172.21.15.22:0/2947542586 learned_addr learned my addr 172.21.15.22:0/2947542586 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470ffff640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470effd640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe0ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470ffff640 1 -- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 msgr2=0x55eefbe0ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470ffff640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe0ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470ffff640 1 -- 172.21.15.22:0/2947542586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe03fd0 msgr2=0x55eefbe0a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470ffff640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe03fd0 0x55eefbe0a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470ffff640 1 -- 172.21.15.22:0/2947542586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eefbb49620 con 0x55eefbe0d550 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470f7fe640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe03fd0 0x55eefbe0a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f470ffff640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe0b040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4708009bf0 tx=0x7f4708005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c826adc1a900fa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.264+0000 7f46eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f46eeffd640 1 -- 172.21.15.22:0/2947542586 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708039030 con 0x55eefbe0d550 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eefbd75b30 con 0x55eefbe0d550 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f46eeffd640 1 -- 172.21.15.22:0/2947542586 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f4708004d40 con 0x55eefbe0d550 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f46eeffd640 1 -- 172.21.15.22:0/2947542586 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4708051070 con 0x55eefbe0d550 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 10 client.29924 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55eefbe10220 con 0x55eefbe0d550 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f46eeffd640 1 -- 172.21.15.22:0/2947542586 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55eefbd75b30 con 0x55eefbe0d550 2024-03-09T00:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f46eeffd640 1 -- 172.21.15.22:0/2947542586 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f470804b030 con 0x55eefbe0d550 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 10 client.29924 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 10 client.29924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 20 client.29924 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe1fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 4 client.29924 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eefbd6fe80 con 0x55eefbe1d6d0 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.265+0000 7f470f7fe640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe1fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.266+0000 7f470f7fe640 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe1fac0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cece6778669ff6f4 server_cookie=34bb7f2330446fc in_seq=0 out_seq=0 2024-03-09T00:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.266+0000 7f46eeffd640 10 client.29924 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.269+0000 7f46eeffd640 1 -- 172.21.15.22:0/2947542586 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eefbd6fe80 con 0x55eefbe1d6d0 2024-03-09T00:11:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.269+0000 7f46eeffd640 10 client.29924 handle_command_reply: tid=0 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.269+0000 7f4718656740 1 client.29924 shutdown 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 msgr2=0x55eefbe0b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe0b040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4708009bf0 tx=0x7f4708005d80 comp rx=0 tx=0).stop 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 shutdown_connections 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe1fac0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe0b040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe0ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 --2- 172.21.15.22:0/2947542586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe03fd0 0x55eefbe0a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f46eeffd640 0 client.29924 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 >> 172.21.15.22:0/2947542586 conn(0x55eefbdcb5b0 msgr2=0x55eefbe00a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 shutdown_connections 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 1 -- 172.21.15.22:0/2947542586 wait complete. 2024-03-09T00:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.270+0000 7f4718656740 20 client.29924 trim_cache size 0 max 16384 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 Processor -- start 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 -- start start 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe03fd0 0x55eefbe0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbdfdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe14610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101b2eb0 con 0x55eefbe04d40 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101b0a50 con 0x55eefbe0d550 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f4718656740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101aeb60 con 0x55eefbe03fd0 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f470effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbdfdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f470effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbdfdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49698/0 (socket says 172.21.15.22:49698) 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f470effd640 1 -- 172.21.15.22:0/667540071 learned_addr learned my addr 172.21.15.22:0/667540071 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f470f7fe640 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe03fd0 0x55eefbe0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.289+0000 7f470ffff640 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe14610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f470effd640 1 -- 172.21.15.22:0/667540071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe03fd0 msgr2=0x55eefbe0ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f470effd640 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe03fd0 0x55eefbe0ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f470effd640 1 -- 172.21.15.22:0/667540071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 msgr2=0x55eefbe14610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f470effd640 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe14610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f470effd640 1 -- 172.21.15.22:0/667540071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eefbe032a0 con 0x55eefbe04d40 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f470ffff640 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe14610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f470effd640 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbdfdcc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f471019ac20 tx=0x7f4704005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268d076acf904f20 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f46f57fa640 1 -- 172.21.15.22:0/667540071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704022070 con 0x55eefbe04d40 2024-03-09T00:11:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f46f57fa640 1 -- 172.21.15.22:0/667540071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47040035f0 con 0x55eefbe04d40 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.290+0000 7f46f57fa640 1 -- 172.21.15.22:0/667540071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470401d6c0 con 0x55eefbe04d40 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 -- 172.21.15.22:0/667540071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 msgr2=0x55eefbdfdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbdfdcc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f471019ac20 tx=0x7f4704005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 -- 172.21.15.22:0/667540071 shutdown_connections 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe0d550 0x55eefbe14610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe04d40 0x55eefbdfdcc0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 --2- 172.21.15.22:0/667540071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe03fd0 0x55eefbe0ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 -- 172.21.15.22:0/667540071 >> 172.21.15.22:0/667540071 conn(0x55eefbdcb5b0 msgr2=0x55eefbdfd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 -- 172.21.15.22:0/667540071 shutdown_connections 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 -- 172.21.15.22:0/667540071 wait complete. 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 -- start start 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.291+0000 7f4718656740 1 Processor -- start 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe11610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe0d550 0x55eefbe11b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101af110 con 0x55eefbe0d550 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101b03f0 con 0x55eefbe03fd0 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101af730 con 0x55eefbe04d40 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 10 client.0 _ll_register_callbacks cb 0x7f4700005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe0d550 0x55eefbe11b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe0d550 0x55eefbe11b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49710/0 (socket says 172.21.15.22:49710) 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 -- 172.21.15.22:0/1439769749 learned_addr learned my addr 172.21.15.22:0/1439769749 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470effd640 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe11610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 -- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 msgr2=0x55eefbe11610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe11610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 -- 172.21.15.22:0/1439769749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 msgr2=0x55eefbe110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 -- 172.21.15.22:0/1439769749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eefbe73ae0 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f470ffff640 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe0d550 0x55eefbe11b50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f470802c130 tx=0x7f4708004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daeb617dbb04138 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f46eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f46eeffd640 1 -- 172.21.15.22:0/1439769749 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708039040 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f46eeffd640 1 -- 172.21.15.22:0/1439769749 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f4708034440 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47040035f0 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.292+0000 7f46eeffd640 1 -- 172.21.15.22:0/1439769749 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47080323c0 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 10 client.20730 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55eefbe74dd0 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f46eeffd640 1 -- 172.21.15.22:0/1439769749 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47040035f0 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f46eeffd640 1 -- 172.21.15.22:0/1439769749 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f47080447e0 con 0x55eefbe0d550 2024-03-09T00:11:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 10 client.20730 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 10 client.20730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 20 client.20730 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe77470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 4 client.20730 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eefbcee710 con 0x55eefbe1d6d0 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.293+0000 7f470f7fe640 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe77470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.294+0000 7f46eeffd640 10 client.20730 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:22.294+0000 7f470f7fe640 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe77470 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51987c249d9bea58 server_cookie=db9cdf31c1928c36 in_seq=0 out_seq=0 2024-03-09T00:11:22.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:22 smithi022 ceph-mon[31894]: pgmap v534: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 379 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 500 op/s 2024-03-09T00:11:22.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:22 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1442332102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:11:22.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:22 smithi062 ceph-mon[36129]: pgmap v534: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 379 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 500 op/s 2024-03-09T00:11:22.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:22 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1442332102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:11:22.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:22 smithi192 ceph-mon[35995]: pgmap v534: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 379 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 500 op/s 2024-03-09T00:11:22.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:22 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1442332102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:11:23.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 48672 11.20 MB/sec execute 90 sec latency 572.685 ms 2024-03-09T00:11:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.218+0000 7f46eeffd640 1 -- 172.21.15.22:0/1439769749 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eefbcee710 con 0x55eefbe1d6d0 2024-03-09T00:11:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.218+0000 7f46eeffd640 10 client.20730 handle_command_reply: tid=0 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.218+0000 7f4718656740 1 client.20730 shutdown 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe0d550 msgr2=0x55eefbe11b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe0d550 0x55eefbe11b50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f470802c130 tx=0x7f4708004a40 comp rx=0 tx=0).stop 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 shutdown_connections 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55eefbe1d6d0 0x55eefbe77470 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f46eeffd640 0 client.20730 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eefbe0d550 0x55eefbe11b50 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55eefbe04d40 0x55eefbe11610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 --2- 172.21.15.22:0/1439769749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eefbe03fd0 0x55eefbe110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 >> 172.21.15.22:0/1439769749 conn(0x55eefbdcb5b0 msgr2=0x55eefbdfde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 shutdown_connections 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 1 -- 172.21.15.22:0/1439769749 wait complete. 2024-03-09T00:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4718656740 20 client.20730 trim_cache size 0 max 16384 2024-03-09T00:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f80a2a90 msgr2=0x7f46f80a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4715a80640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f80a2a90 0x7f46f80a4f50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f470802c630 tx=0x7f4708042000 comp rx=0 tx=0).stop 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47101000e0 msgr2=0x7f471019f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.219+0000 7f4715a80640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47101000e0 0x7f471019f930 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f470000ea60 tx=0x7f470000ef20 comp rx=0 tx=0).stop 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 shutdown_connections 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f46f80a2a90 0x7f46f80a4f50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471010dc90 0x7f4710199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710100a00 0x7f471019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 --2- 172.21.15.22:0/2407809151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47101000e0 0x7f471019f930 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 >> 172.21.15.22:0/2407809151 conn(0x7f4710077350 msgr2=0x7f471010efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 shutdown_connections 2024-03-09T00:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:23.220+0000 7f4715a80640 1 -- 172.21.15.22:0/2407809151 wait complete. 2024-03-09T00:11:23.227 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:11:23.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:11:23.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:11:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 -- 172.21.15.22:0/4190958148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff124101ea0 msgr2=0x7ff124105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/4190958148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff124101ea0 0x7ff124105ee0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff114009900 tx=0x7ff11402bc30 comp rx=0 tx=0).stop 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 -- 172.21.15.22:0/4190958148 shutdown_connections 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/4190958148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1241065c0 0x7ff12410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/4190958148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff124101ea0 0x7ff124105ee0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/4190958148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1241014f0 0x7ff1241018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 -- 172.21.15.22:0/4190958148 >> 172.21.15.22:0/4190958148 conn(0x7ff1240fcd90 msgr2=0x7ff1240ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 -- 172.21.15.22:0/4190958148 shutdown_connections 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.349+0000 7ff12bbe1640 1 -- 172.21.15.22:0/4190958148 wait complete. 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 Processor -- start 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 -- start start 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1241014f0 0x7ff12419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1241065c0 0x7ff12419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff124197a30 0x7ff124197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff124111630 con 0x7ff1241014f0 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241114b0 con 0x7ff124197a30 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12bbe1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241117b0 con 0x7ff1241065c0 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12997d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1241014f0 0x7ff12419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12997d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1241014f0 0x7ff12419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49718/0 (socket says 172.21.15.22:49718) 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12997d640 1 -- 172.21.15.22:0/2987922678 learned_addr learned my addr 172.21.15.22:0/2987922678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.350+0000 7ff12a17e640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff124197a30 0x7ff124197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12917c640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1241065c0 0x7ff12419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12997d640 1 -- 172.21.15.22:0/2987922678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1241065c0 msgr2=0x7ff12419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12997d640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1241065c0 0x7ff12419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12997d640 1 -- 172.21.15.22:0/2987922678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff124197a30 msgr2=0x7ff124197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12997d640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff124197a30 0x7ff124197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12997d640 1 -- 172.21.15.22:0/2987922678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124198550 con 0x7ff1241014f0 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12917c640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1241065c0 0x7ff12419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12997d640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1241014f0 0x7ff12419d830 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff12000e9f0 tx=0x7ff12000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff11affd640 1 -- 172.21.15.22:0/2987922678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000cd80 con 0x7ff1241014f0 2024-03-09T00:11:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff124198840 con 0x7ff1241014f0 2024-03-09T00:11:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff11affd640 1 -- 172.21.15.22:0/2987922678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff120010910 con 0x7ff1241014f0 2024-03-09T00:11:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff11affd640 1 -- 172.21.15.22:0/2987922678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200188f0 con 0x7ff1241014f0 2024-03-09T00:11:23.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.351+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1241a4a40 con 0x7ff1241014f0 2024-03-09T00:11:23.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.353+0000 7ff11affd640 1 -- 172.21.15.22:0/2987922678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff120010ac0 con 0x7ff1241014f0 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff11affd640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1100a2970 0x7ff1100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 Processor -- start 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12917c640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1100a2970 0x7ff1100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff11affd640 1 -- 172.21.15.22:0/2987922678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff1200cf720 con 0x7ff1241014f0 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 -- start start 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b6a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039b75670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e039acdb40 con 0x55e039b6a6a0 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e039a76d80 con 0x55e039b73280 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.354+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e039ac9be0 con 0x55e039b69d80 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12997d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b6a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12997d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b6a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41820/0 (socket says 172.21.15.22:41820) 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12997d640 1 -- 172.21.15.22:0/2279801065 learned_addr learned my addr 172.21.15.22:0/2279801065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12a17e640 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039b75670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1100a2970 0x7ff1100a4e30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff114006010 tx=0x7ff11402c710 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 -- 172.21.15.22:0/2279801065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 msgr2=0x55e039b6a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b6a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 -- 172.21.15.22:0/2279801065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 msgr2=0x55e039b75670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039b75670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 -- 172.21.15.22:0/2279801065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0398aa7e0 con 0x55e039b6a6a0 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12a17e640 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039b75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff12917c640 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b72d40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7ff114036040 tx=0x7ff11403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27944628d0c3ad1a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff10ffff640 1 -- 172.21.15.22:0/2279801065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11404e070 con 0x55e039b6a6a0 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff10ffff640 1 -- 172.21.15.22:0/2279801065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff114044070 con 0x55e039b6a6a0 2024-03-09T00:11:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.355+0000 7ff10ffff640 1 -- 172.21.15.22:0/2279801065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114004d00 con 0x55e039b6a6a0 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 -- 172.21.15.22:0/2279801065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 msgr2=0x55e039b72d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b72d40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7ff114036040 tx=0x7ff11403d000 comp rx=0 tx=0).stop 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 -- 172.21.15.22:0/2279801065 shutdown_connections 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039b75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b72d40 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/2279801065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b6a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 -- 172.21.15.22:0/2279801065 >> 172.21.15.22:0/2279801065 conn(0x55e039b644b0 msgr2=0x55e039b64890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 -- 172.21.15.22:0/2279801065 shutdown_connections 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.356+0000 7ff12e7bd740 1 -- 172.21.15.22:0/2279801065 wait complete. 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 -- start start 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 Processor -- start 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b69d80 0x55e039b72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e039b76010 con 0x55e039b6a6a0 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e039a8a1c0 con 0x55e039b69d80 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e039b76190 con 0x55e039b73280 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12e7bd740 10 client.0 _ll_register_callbacks cb 0x55e039a797a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12917c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12997d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b69d80 0x55e039b72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12917c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49736/0 (socket says 172.21.15.22:49736) 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12917c640 1 -- 172.21.15.22:0/4135677831 learned_addr learned my addr 172.21.15.22:0/4135677831 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12a17e640 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12917c640 1 -- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 msgr2=0x55e039b70810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.357+0000 7ff12917c640 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b70810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff12917c640 1 -- 172.21.15.22:0/4135677831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b69d80 msgr2=0x55e039b72d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff12917c640 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b69d80 0x55e039b72d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff12917c640 1 -- 172.21.15.22:0/4135677831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e03990af70 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff12917c640 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b700d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff11402c170 tx=0x7ff114004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a739b559de7df117 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff10d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff10d7fa640 1 -- 172.21.15.22:0/4135677831 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114057070 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff10d7fa640 1 -- 172.21.15.22:0/4135677831 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff114005080 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff114044070 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff10d7fa640 1 -- 172.21.15.22:0/4135677831 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff114065050 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff12e7bd740 10 client.20742 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e0397a2170 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.358+0000 7ff10d7fa640 1 -- 172.21.15.22:0/4135677831 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff114044070 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff10d7fa640 1 -- 172.21.15.22:0/4135677831 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff114049070 con 0x55e039b6a6a0 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff12e7bd740 10 client.20742 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff12e7bd740 10 client.20742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff12e7bd740 20 client.20742 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039b83240 0x55e039b85630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff12e7bd740 4 client.20742 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e039aa3960 con 0x55e039b83240 2024-03-09T00:11:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.359+0000 7ff12997d640 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039b83240 0x55e039b85630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.360+0000 7ff12997d640 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039b83240 0x55e039b85630 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0fcd07cc7c7a1fe server_cookie=e433f3d88cd0024c in_seq=0 out_seq=0 2024-03-09T00:11:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.360+0000 7ff10d7fa640 10 client.20742 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.364+0000 7ff10d7fa640 1 -- 172.21.15.22:0/4135677831 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e039aa3960 con 0x55e039b83240 2024-03-09T00:11:23.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.364+0000 7ff10d7fa640 10 client.20742 handle_command_reply: tid=0 2024-03-09T00:11:23.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 client.20742 shutdown 2024-03-09T00:11:23.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 msgr2=0x55e039b700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b700d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff11402c170 tx=0x7ff114004a80 comp rx=0 tx=0).stop 2024-03-09T00:11:23.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 shutdown_connections 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039b83240 0x55e039b85630 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b70810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff10d7fa640 0 client.20742 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b700d0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4135677831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b69d80 0x55e039b72d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 >> 172.21.15.22:0/4135677831 conn(0x55e039b644b0 msgr2=0x55e039b66200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 shutdown_connections 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4135677831 wait complete. 2024-03-09T00:11:23.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.365+0000 7ff12e7bd740 20 client.20742 trim_cache size 0 max 16384 2024-03-09T00:11:23.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.381+0000 7ff12e7bd740 1 Processor -- start 2024-03-09T00:11:23.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12e7bd740 1 -- start start 2024-03-09T00:11:23.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b69d80 0x55e039b7ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b6a6a0 0x55e039b72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241b0c70 con 0x55e039b69d80 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241ae810 con 0x55e039b6a6a0 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241ac8d0 con 0x55e039b73280 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12997d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b69d80 0x55e039b7ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b6a6a0 0x55e039b72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12997d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b69d80 0x55e039b7ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49740/0 (socket says 172.21.15.22:49740) 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12997d640 1 -- 172.21.15.22:0/4075765109 learned_addr learned my addr 172.21.15.22:0/4075765109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b6a6a0 0x55e039b72800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:51442/0 (socket says 172.21.15.22:51442) 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12a17e640 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 -- 172.21.15.22:0/4075765109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 msgr2=0x55e039b72d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b72d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 -- 172.21.15.22:0/4075765109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b69d80 msgr2=0x55e039b7ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b69d80 0x55e039b7ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 -- 172.21.15.22:0/4075765109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e039b859a0 con 0x55e039b6a6a0 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12997d640 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b69d80 0x55e039b7ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.382+0000 7ff12917c640 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b6a6a0 0x55e039b72800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff114009900 tx=0x7ff11402cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6855e43d9ca1258b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff10f7fe640 1 -- 172.21.15.22:0/4075765109 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114057040 con 0x55e039b6a6a0 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff10f7fe640 1 -- 172.21.15.22:0/4075765109 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff114044070 con 0x55e039b6a6a0 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff10f7fe640 1 -- 172.21.15.22:0/4075765109 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11404e040 con 0x55e039b6a6a0 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4075765109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b6a6a0 msgr2=0x55e039b72800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b6a6a0 0x55e039b72800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff114009900 tx=0x7ff11402cde0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4075765109 shutdown_connections 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b73280 0x55e039b72d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b6a6a0 0x55e039b72800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/4075765109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b69d80 0x55e039b7ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4075765109 >> 172.21.15.22:0/4075765109 conn(0x55e039b644b0 msgr2=0x55e039b6b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4075765109 shutdown_connections 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.383+0000 7ff12e7bd740 1 -- 172.21.15.22:0/4075765109 wait complete. 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 -- start start 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 Processor -- start 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b79be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039bd33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241aced0 con 0x55e039b6a6a0 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241ae1b0 con 0x55e039b73280 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241ad4f0 con 0x55e039b69d80 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12917c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b79be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12917c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b79be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49754/0 (socket says 172.21.15.22:49754) 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12a17e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039bd33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12917c640 1 -- 172.21.15.22:0/617475782 learned_addr learned my addr 172.21.15.22:0/617475782 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12e7bd740 10 client.0 _ll_register_callbacks cb 0x7ff120005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.384+0000 7ff12997d640 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12a17e640 1 -- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 msgr2=0x55e039b796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12a17e640 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12a17e640 1 -- 172.21.15.22:0/617475782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 msgr2=0x55e039b79be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12a17e640 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b79be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12a17e640 1 -- 172.21.15.22:0/617475782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e039bd39a0 con 0x55e039b73280 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12917c640 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b79be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12a17e640 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039bd33e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55e039b65640 tx=0x7ff11c012c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9522048984539bd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff10d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff10d7fa640 1 -- 172.21.15.22:0/617475782 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c010b70 con 0x55e039b73280 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff114044070 con 0x55e039b73280 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff10d7fa640 1 -- 172.21.15.22:0/617475782 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7ff11c010d10 con 0x55e039b73280 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff10d7fa640 1 -- 172.21.15.22:0/617475782 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff11c02a050 con 0x55e039b73280 2024-03-09T00:11:23.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12e7bd740 10 client.29948 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.385+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e039bdae40 con 0x55e039b73280 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff10d7fa640 1 -- 172.21.15.22:0/617475782 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff114044070 con 0x55e039b73280 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff10d7fa640 1 -- 172.21.15.22:0/617475782 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff11c01c6b0 con 0x55e039b73280 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12e7bd740 10 client.29948 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12e7bd740 10 client.29948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12e7bd740 20 client.29948 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039bdb5a0 0x55e039bdd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12e7bd740 4 client.29948 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e039a8f080 con 0x55e039bdb5a0 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12997d640 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039bdb5a0 0x55e039bdd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff12997d640 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039bdb5a0 0x55e039bdd9c0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ce7db9d007e047 server_cookie=24c7c5575bbb392c in_seq=0 out_seq=0 2024-03-09T00:11:23.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.386+0000 7ff10d7fa640 10 client.29948 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff10d7fa640 1 -- 172.21.15.22:0/617475782 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e039a8f080 con 0x55e039bdb5a0 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff10d7fa640 10 client.29948 handle_command_reply: tid=0 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 client.29948 shutdown 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 msgr2=0x55e039bd33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039bd33e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55e039b65640 tx=0x7ff11c012c50 comp rx=0 tx=0).stop 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 shutdown_connections 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55e039bdb5a0 0x55e039bdd9c0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e039b73280 0x55e039bd33e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e039b6a6a0 0x55e039b79be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff10d7fa640 0 client.29948 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 --2- 172.21.15.22:0/617475782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e039b69d80 0x55e039b796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 >> 172.21.15.22:0/617475782 conn(0x55e039b644b0 msgr2=0x55e039b62bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 shutdown_connections 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 1 -- 172.21.15.22:0/617475782 wait complete. 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.391+0000 7ff12e7bd740 20 client.29948 trim_cache size 0 max 16384 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "5033d74d-68f1-46ee-af6b-c6792ea77b78", 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:11:23.390 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1100a2970 msgr2=0x7ff1100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1100a2970 0x7ff1100a4e30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff114006010 tx=0x7ff11402c710 comp rx=0 tx=0).stop 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1241014f0 msgr2=0x7ff12419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1241014f0 0x7ff12419d830 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff12000e9f0 tx=0x7ff12000eeb0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 shutdown_connections 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1100a2970 0x7ff1100a4e30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff124197a30 0x7ff124197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1241065c0 0x7ff12419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 --2- 172.21.15.22:0/2987922678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1241014f0 0x7ff12419d830 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 >> 172.21.15.22:0/2987922678 conn(0x7ff1240fcd90 msgr2=0x7ff1240fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 shutdown_connections 2024-03-09T00:11:23.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.392+0000 7ff12bbe1640 1 -- 172.21.15.22:0/2987922678 wait complete. 2024-03-09T00:11:23.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:11:23.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:11:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 -- 172.21.15.22:0/141130587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 msgr2=0x7f68881007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 --2- 172.21.15.22:0/141130587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f68881007b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f6878009900 tx=0x7f687802bc30 comp rx=0 tx=0).stop 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 -- 172.21.15.22:0/141130587 shutdown_connections 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 --2- 172.21.15.22:0/141130587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68881054a0 0x7f688810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 --2- 172.21.15.22:0/141130587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6888100d80 0x7f6888104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 --2- 172.21.15.22:0/141130587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f68881007b0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 -- 172.21.15.22:0/141130587 >> 172.21.15.22:0/141130587 conn(0x7f68880772f0 msgr2=0x7f6888077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 -- 172.21.15.22:0/141130587 shutdown_connections 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 -- 172.21.15.22:0/141130587 wait complete. 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 Processor -- start 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.515+0000 7f688dfd4640 1 -- start start 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f688dfd4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f6888199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f688dfd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68881054a0 0x7f688819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f688dfd4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688819e610 0x7f688819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f688dfd4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888110530 con 0x7f68881003d0 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f688dfd4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881103b0 con 0x7f68881054a0 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f688dfd4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881106b0 con 0x7f688819e610 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f6888199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f6888199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49760/0 (socket says 172.21.15.22:49760) 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 -- 172.21.15.22:0/3948878791 learned_addr learned my addr 172.21.15.22:0/3948878791 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f6886ffd640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68881054a0 0x7f688819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f6887fff640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688819e610 0x7f688819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 -- 172.21.15.22:0/3948878791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688819e610 msgr2=0x7f688819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688819e610 0x7f688819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 -- 172.21.15.22:0/3948878791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68881054a0 msgr2=0x7f688819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68881054a0 0x7f688819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 -- 172.21.15.22:0/3948878791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688819f100 con 0x7f68881003d0 2024-03-09T00:11:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f68877fe640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f6888199d50 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f6878006010 tx=0x7f687802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f6884ff9640 1 -- 172.21.15.22:0/3948878791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878041070 con 0x7f68881003d0 2024-03-09T00:11:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f6884ff9640 1 -- 172.21.15.22:0/3948878791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6878004450 con 0x7f68881003d0 2024-03-09T00:11:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688819f390 con 0x7f68881003d0 2024-03-09T00:11:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.516+0000 7f6884ff9640 1 -- 172.21.15.22:0/3948878791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878034440 con 0x7f68881003d0 2024-03-09T00:11:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.517+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68881a6bb0 con 0x7f68881003d0 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.519+0000 7f6884ff9640 1 -- 172.21.15.22:0/3948878791 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6878044070 con 0x7f68881003d0 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.519+0000 7f6884ff9640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f68700a29f0 0x7f68700a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.519+0000 7f6884ff9640 1 -- 172.21.15.22:0/3948878791 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f68780f0cc0 con 0x7f68881003d0 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.519+0000 7f6886ffd640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f68700a29f0 0x7f68700a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.519+0000 7f6890bb2740 1 Processor -- start 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6890bb2740 1 -- start start 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a94760 0x55c0f6a94b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6890bb2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f69f7e00 con 0x55c0f6a9dc60 2024-03-09T00:11:23.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6890bb2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f69a0b10 con 0x55c0f6a94760 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6890bb2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f69f3ea0 con 0x55c0f6a95080 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6886ffd640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f68700a29f0 0x7f68700a4eb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6888069530 tx=0x7f687c00b410 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6886ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6887fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6886ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:41876/0 (socket says 172.21.15.22:41876) 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6886ffd640 1 -- 172.21.15.22:0/3254608947 learned_addr learned my addr 172.21.15.22:0/3254608947 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f68877fe640 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a94760 0x55c0f6a94b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6887fff640 1 -- 172.21.15.22:0/3254608947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 msgr2=0x55c0f6a9d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6887fff640 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6887fff640 1 -- 172.21.15.22:0/3254608947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a94760 msgr2=0x55c0f6a94b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6887fff640 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a94760 0x55c0f6a94b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6887fff640 1 -- 172.21.15.22:0/3254608947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0f67d67d0 con 0x55c0f6a9dc60 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f6886ffd640 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:23.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.520+0000 7f68877fe640 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a94760 0x55c0f6a94b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.521+0000 7f6887fff640 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa0050 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6880009eb0 tx=0x7f68800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d60927bc8e0e02a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.521+0000 7f686dffb640 1 -- 172.21.15.22:0/3254608947 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880003b70 con 0x55c0f6a9dc60 2024-03-09T00:11:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.521+0000 7f686dffb640 1 -- 172.21.15.22:0/3254608947 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6880015070 con 0x55c0f6a9dc60 2024-03-09T00:11:23.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.521+0000 7f686dffb640 1 -- 172.21.15.22:0/3254608947 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688001a7a0 con 0x55c0f6a9dc60 2024-03-09T00:11:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.521+0000 7f6890bb2740 1 -- 172.21.15.22:0/3254608947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a9dc60 msgr2=0x55c0f6aa0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.521+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa0050 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6880009eb0 tx=0x7f68800107a0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- 172.21.15.22:0/3254608947 shutdown_connections 2024-03-09T00:11:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa0050 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3254608947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a94760 0x55c0f6a94b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- 172.21.15.22:0/3254608947 >> 172.21.15.22:0/3254608947 conn(0x55c0f6a8ee90 msgr2=0x55c0f6a8f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- 172.21.15.22:0/3254608947 shutdown_connections 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- 172.21.15.22:0/3254608947 wait complete. 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- start start 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 Processor -- start 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a94760 0x55c0f6a9d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a95080 0x55c0f6a9a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a9ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f6aa0a50 con 0x55c0f6a95080 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f69b40a0 con 0x55c0f6a9dc60 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0f6aa0bd0 con 0x55c0f6a94760 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.522+0000 7f6890bb2740 10 client.0 _ll_register_callbacks cb 0x55c0f69b3a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a95080 0x55c0f6a9a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a95080 0x55c0f6a9a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49780/0 (socket says 172.21.15.22:49780) 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 -- 172.21.15.22:0/1762543792 learned_addr learned my addr 172.21.15.22:0/1762543792 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f68877fe640 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a94760 0x55c0f6a9d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6887fff640 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a9ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 -- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a94760 msgr2=0x55c0f6a9d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a94760 0x55c0f6a9d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 -- 172.21.15.22:0/1762543792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 msgr2=0x55c0f6a9ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a9ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 -- 172.21.15.22:0/1762543792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0f670c3f0 con 0x55c0f6a95080 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6886ffd640 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a95080 0x55c0f6a9a6e0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f687c0116c0 tx=0x7f687c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b838b818494250 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6866ffd640 1 -- 172.21.15.22:0/1762543792 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c01e780 con 0x55c0f6a95080 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6866ffd640 1 -- 172.21.15.22:0/1762543792 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f687c01e920 con 0x55c0f6a95080 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.523+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6880015070 con 0x55c0f6a95080 2024-03-09T00:11:23.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6866ffd640 1 -- 172.21.15.22:0/1762543792 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f687c01eb20 con 0x55c0f6a95080 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 10 client.20760 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c0f66cc0f0 con 0x55c0f6a95080 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6866ffd640 1 -- 172.21.15.22:0/1762543792 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6880015070 con 0x55c0f6a95080 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6866ffd640 1 -- 172.21.15.22:0/1762543792 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f687c01f430 con 0x55c0f6a95080 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 10 client.20760 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 10 client.20760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 20 client.20760 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6aad790 0x55c0f6aafb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 4 client.20760 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0f6a05340 con 0x55c0f6aad790 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.524+0000 7f68877fe640 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6aad790 0x55c0f6aafb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.525+0000 7f68877fe640 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6aad790 0x55c0f6aafb80 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da693c8afc8ae4c server_cookie=93a002dc8dba0c35 in_seq=0 out_seq=0 2024-03-09T00:11:23.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.525+0000 7f6866ffd640 10 client.20760 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.528+0000 7f6866ffd640 1 -- 172.21.15.22:0/1762543792 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0f6a05340 con 0x55c0f6aad790 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.528+0000 7f6866ffd640 10 client.20760 handle_command_reply: tid=0 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.528+0000 7f6890bb2740 1 client.20760 shutdown 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a95080 msgr2=0x55c0f6a9a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a95080 0x55c0f6a9a6e0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f687c0116c0 tx=0x7f687c004730 comp rx=0 tx=0).stop 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 shutdown_connections 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6aad790 0x55c0f6aafb80 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a9ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a95080 0x55c0f6a9a6e0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 --2- 172.21.15.22:0/1762543792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a94760 0x55c0f6a9d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 >> 172.21.15.22:0/1762543792 conn(0x55c0f6a8ee90 msgr2=0x55c0f6a90be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6866ffd640 0 client.20760 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 shutdown_connections 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 1 -- 172.21.15.22:0/1762543792 wait complete. 2024-03-09T00:11:23.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.529+0000 7f6890bb2740 20 client.20760 trim_cache size 0 max 16384 2024-03-09T00:11:23.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 Processor -- start 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 -- start start 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6a9ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881b2e50 con 0x55c0f6a94760 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881b09f0 con 0x55c0f6a9dc60 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.552+0000 7f6890bb2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881aeab0 con 0x55c0f6a95080 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6a9ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6a9ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49796/0 (socket says 172.21.15.22:49796) 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 -- 172.21.15.22:0/3371978109 learned_addr learned my addr 172.21.15.22:0/3371978109 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f6887fff640 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f6886ffd640 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 -- 172.21.15.22:0/3371978109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 msgr2=0x55c0f6a9d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 -- 172.21.15.22:0/3371978109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 msgr2=0x55c0f6aa4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 -- 172.21.15.22:0/3371978109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0f6a98c50 con 0x55c0f6a94760 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f68877fe640 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6a9ce10 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f687802cfd0 tx=0x7f687802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841e32c4a4b78d5d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f686d7fa640 1 -- 172.21.15.22:0/3371978109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687804b040 con 0x55c0f6a94760 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f686d7fa640 1 -- 172.21.15.22:0/3371978109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6878004c80 con 0x55c0f6a94760 2024-03-09T00:11:23.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.553+0000 7f686d7fa640 1 -- 172.21.15.22:0/3371978109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68780b9700 con 0x55c0f6a94760 2024-03-09T00:11:23.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 -- 172.21.15.22:0/3371978109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 msgr2=0x55c0f6a9ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6a9ce10 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f687802cfd0 tx=0x7f687802bc30 comp rx=0 tx=0).stop 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 -- 172.21.15.22:0/3371978109 shutdown_connections 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a9dc60 0x55c0f6aa4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a95080 0x55c0f6a9d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 --2- 172.21.15.22:0/3371978109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6a9ce10 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 -- 172.21.15.22:0/3371978109 >> 172.21.15.22:0/3371978109 conn(0x55c0f6a8ee90 msgr2=0x55c0f6a9a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 -- 172.21.15.22:0/3371978109 shutdown_connections 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.554+0000 7f6890bb2740 1 -- 172.21.15.22:0/3371978109 wait complete. 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 -- start start 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 Processor -- start 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6aa4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a95080 0x55c0f6aa5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881af0b0 con 0x55c0f6a94760 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881b0390 con 0x55c0f6a95080 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881af6d0 con 0x55c0f6a9dc60 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6890bb2740 10 client.0 _ll_register_callbacks cb 0x7f687c005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f68877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6aa4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f68877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6aa4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49810/0 (socket says 172.21.15.22:49810) 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f68877fe640 1 -- 172.21.15.22:0/4095400280 learned_addr learned my addr 172.21.15.22:0/4095400280 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6886ffd640 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a95080 0x55c0f6aa5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.555+0000 7f6887fff640 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f68877fe640 1 -- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a9dc60 msgr2=0x55c0f6a96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f68877fe640 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f68877fe640 1 -- 172.21.15.22:0/4095400280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a95080 msgr2=0x55c0f6aa5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f68877fe640 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a95080 0x55c0f6aa5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f68877fe640 1 -- 172.21.15.22:0/4095400280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0f6a97150 con 0x55c0f6a94760 2024-03-09T00:11:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f68877fe640 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6aa4e40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f68780b9a60 tx=0x7f68780c1220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc581cdf4842299d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6866ffd640 1 -- 172.21.15.22:0/4095400280 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687804b040 con 0x55c0f6a94760 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6866ffd640 1 -- 172.21.15.22:0/4095400280 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f68780b9700 con 0x55c0f6a94760 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6878004c80 con 0x55c0f6a94760 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6866ffd640 1 -- 172.21.15.22:0/4095400280 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68780035f0 con 0x55c0f6a94760 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 10 client.20766 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c0f6b04670 con 0x55c0f6a94760 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6866ffd640 1 -- 172.21.15.22:0/4095400280 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6878004c80 con 0x55c0f6a94760 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6866ffd640 1 -- 172.21.15.22:0/4095400280 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f68780a60d0 con 0x55c0f6a94760 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 10 client.20766 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 10 client.20766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 20 client.20766 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6b04c80 0x55c0f6b070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 4 client.20766 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:11:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.556+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c0f69ff690 con 0x55c0f6b04c80 2024-03-09T00:11:23.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.557+0000 7f6886ffd640 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6b04c80 0x55c0f6b070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:23.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.557+0000 7f6886ffd640 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6b04c80 0x55c0f6b070a0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7913d95b27f86b64 server_cookie=34d5ee2bbeb81637 in_seq=0 out_seq=0 2024-03-09T00:11:23.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.557+0000 7f6866ffd640 10 client.20766 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.562+0000 7f6866ffd640 1 -- 172.21.15.22:0/4095400280 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55c0f69ff690 con 0x55c0f6b04c80 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.562+0000 7f6866ffd640 10 client.20766 handle_command_reply: tid=0 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.562+0000 7f6890bb2740 1 client.20766 shutdown 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 msgr2=0x55c0f6aa4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6aa4e40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f68780b9a60 tx=0x7f68780c1220 comp rx=0 tx=0).stop 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 shutdown_connections 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55c0f6b04c80 0x55c0f6b070a0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f6a9dc60 0x55c0f6a96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6866ffd640 0 client.20766 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0f6a95080 0x55c0f6aa5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 --2- 172.21.15.22:0/4095400280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c0f6a94760 0x55c0f6aa4e40 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 >> 172.21.15.22:0/4095400280 conn(0x55c0f6a8ee90 msgr2=0x55c0f6a98f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 shutdown_connections 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 1 -- 172.21.15.22:0/4095400280 wait complete. 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.563+0000 7f6890bb2740 20 client.20766 trim_cache size 0 max 16384 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (541 inodes in the stack)", 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:11:23.560 INFO:teuthology.orchestra.run.smithi022.stdout: "5033d74d-68f1-46ee-af6b-c6792ea77b78": { 2024-03-09T00:11:23.561 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:11:23.561 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "5033d74d-68f1-46ee-af6b-c6792ea77b78", 2024-03-09T00:11:23.561 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:11:23.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:11:23.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:11:23.561 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f68700a29f0 msgr2=0x7f68700a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f68700a29f0 0x7f68700a4eb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6888069530 tx=0x7f687c00b410 comp rx=0 tx=0).stop 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 msgr2=0x7f6888199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f6888199d50 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f6878006010 tx=0x7f687802cc70 comp rx=0 tx=0).stop 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 shutdown_connections 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f68700a29f0 0x7f68700a4eb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688819e610 0x7f688819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68881054a0 0x7f688819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 --2- 172.21.15.22:0/3948878791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68881003d0 0x7f6888199d50 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 >> 172.21.15.22:0/3948878791 conn(0x7f68880772f0 msgr2=0x7f68880fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 shutdown_connections 2024-03-09T00:11:23.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:23.564+0000 7f688dfd4640 1 -- 172.21.15.22:0/3948878791 wait complete. 2024-03-09T00:11:23.570 INFO:tasks.cephfs.filesystem:scrub status for tag:5033d74d-68f1-46ee-af6b-c6792ea77b78 - {'path': '/', 'tag': '5033d74d-68f1-46ee-af6b-c6792ea77b78', 'options': 'recursive,force'} 2024-03-09T00:11:24.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 48893 11.20 MB/sec execute 91 sec latency 596.087 ms 2024-03-09T00:11:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: pgmap v535: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 379 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-09T00:11:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:11:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:24.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:24.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:24.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:24.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:24 smithi022 ceph-mon[31894]: osdmap e122: 12 total, 12 up, 12 in 2024-03-09T00:11:24.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: pgmap v535: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 379 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-09T00:11:24.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:11:24.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:24 smithi062 ceph-mon[36129]: osdmap e122: 12 total, 12 up, 12 in 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: pgmap v535: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 379 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:24.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:24.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:24.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:24.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:24 smithi192 ceph-mon[35995]: osdmap e122: 12 total, 12 up, 12 in 2024-03-09T00:11:25.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 49045 11.12 MB/sec execute 92 sec latency 323.380 ms 2024-03-09T00:11:25.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:25 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:25.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:25 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:25.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:25 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:26.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 49354 11.18 MB/sec execute 93 sec latency 110.494 ms 2024-03-09T00:11:26.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:26 smithi022 ceph-mon[31894]: pgmap v537: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-03-09T00:11:26.861 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:26 smithi062 ceph-mon[36129]: pgmap v537: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-03-09T00:11:26.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:26 smithi192 ceph-mon[35995]: pgmap v537: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-03-09T00:11:27.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 49794 11.29 MB/sec execute 94 sec latency 149.580 ms 2024-03-09T00:11:28.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 50111 11.34 MB/sec execute 95 sec latency 252.080 ms 2024-03-09T00:11:28.683 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:28 smithi192 ceph-mon[35995]: pgmap v538: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-03-09T00:11:28.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:28 smithi022 ceph-mon[31894]: pgmap v538: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-03-09T00:11:28.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:28 smithi062 ceph-mon[36129]: pgmap v538: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 606 op/s 2024-03-09T00:11:29.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 50518 11.52 MB/sec execute 96 sec latency 163.986 ms 2024-03-09T00:11:30.073 INFO:tasks.workunit.client.0.smithi022.stdout: 10 50738 11.57 MB/sec execute 97 sec latency 405.778 ms 2024-03-09T00:11:30.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:30 smithi062 ceph-mon[36129]: osdmap e123: 12 total, 12 up, 12 in 2024-03-09T00:11:30.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:30 smithi062 ceph-mon[36129]: pgmap v540: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 577 op/s 2024-03-09T00:11:30.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:30 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:11:30.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:30 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:11:30.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:30 smithi192 ceph-mon[35995]: osdmap e123: 12 total, 12 up, 12 in 2024-03-09T00:11:30.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:30 smithi192 ceph-mon[35995]: pgmap v540: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 577 op/s 2024-03-09T00:11:30.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:30 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:11:30.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:30 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:11:30.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:30 smithi022 ceph-mon[31894]: osdmap e123: 12 total, 12 up, 12 in 2024-03-09T00:11:30.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:30 smithi022 ceph-mon[31894]: pgmap v540: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 577 op/s 2024-03-09T00:11:30.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:30 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:11:30.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:30 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:11:31.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 51024 11.59 MB/sec execute 98 sec latency 564.057 ms 2024-03-09T00:11:31.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:31 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:11:31.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:31 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:11:31.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:31 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:11:32.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 51219 11.58 MB/sec execute 99 sec latency 240.379 ms 2024-03-09T00:11:32.361 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:32 smithi062 ceph-mon[36129]: pgmap v541: 129 pgs: 129 active+clean; 375 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 778 op/s 2024-03-09T00:11:32.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:32 smithi192 ceph-mon[35995]: pgmap v541: 129 pgs: 129 active+clean; 375 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 778 op/s 2024-03-09T00:11:32.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:32 smithi022 ceph-mon[31894]: pgmap v541: 129 pgs: 129 active+clean; 375 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 778 op/s 2024-03-09T00:11:33.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 51472 11.60 MB/sec execute 100 sec latency 200.172 ms 2024-03-09T00:11:34.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 51708 11.60 MB/sec execute 101 sec latency 247.233 ms 2024-03-09T00:11:34.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:34 smithi022 ceph-mon[31894]: pgmap v542: 129 pgs: 129 active+clean; 375 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 659 op/s 2024-03-09T00:11:34.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:34 smithi062 ceph-mon[36129]: pgmap v542: 129 pgs: 129 active+clean; 375 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 659 op/s 2024-03-09T00:11:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:34 smithi192 ceph-mon[35995]: pgmap v542: 129 pgs: 129 active+clean; 375 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 659 op/s 2024-03-09T00:11:35.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 51934 11.58 MB/sec execute 102 sec latency 339.572 ms 2024-03-09T00:11:36.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 52285 11.62 MB/sec execute 103 sec latency 245.602 ms 2024-03-09T00:11:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 -- 172.21.15.22:0/3107183319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 msgr2=0x7f889810e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 --2- 172.21.15.22:0/3107183319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889810e180 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8890009cd0 tx=0x7f889002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 -- 172.21.15.22:0/3107183319 shutdown_connections 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 --2- 172.21.15.22:0/3107183319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889810e180 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 --2- 172.21.15.22:0/3107183319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88980ff7f0 0x7f88980ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 --2- 172.21.15.22:0/3107183319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88981062d0 0x7f88981066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 -- 172.21.15.22:0/3107183319 >> 172.21.15.22:0/3107183319 conn(0x7f8898076730 msgr2=0x7f88980fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 -- 172.21.15.22:0/3107183319 shutdown_connections 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.310+0000 7f889e9d4640 1 -- 172.21.15.22:0/3107183319 wait complete. 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 Processor -- start 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 -- start start 2024-03-09T00:11:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88980ff7f0 0x7f8898199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88981062d0 0x7f889819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898111790 con 0x7f88981001b0 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898111610 con 0x7f88981062d0 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889e9d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898111910 con 0x7f88980ff7f0 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f88977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f8897fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88980ff7f0 0x7f8898199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f88977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44722/0 (socket says 172.21.15.22:44722) 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f88977fe640 1 -- 172.21.15.22:0/2826033693 learned_addr learned my addr 172.21.15.22:0/2826033693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.311+0000 7f889cf71640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88981062d0 0x7f889819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88977fe640 1 -- 172.21.15.22:0/2826033693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88980ff7f0 msgr2=0x7f8898199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88977fe640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88980ff7f0 0x7f8898199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88977fe640 1 -- 172.21.15.22:0/2826033693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88981062d0 msgr2=0x7f889819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88977fe640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88981062d0 0x7f889819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88977fe640 1 -- 172.21.15.22:0/2826033693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889819ed20 con 0x7f88981001b0 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f889cf71640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88981062d0 0x7f889819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88977fe640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889819a270 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e9f0 tx=0x7f888c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88957fa640 1 -- 172.21.15.22:0/2826033693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00cd80 con 0x7f88981001b0 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889819f010 con 0x7f88981001b0 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88957fa640 1 -- 172.21.15.22:0/2826033693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f888c005500 con 0x7f88981001b0 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f88957fa640 1 -- 172.21.15.22:0/2826033693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c010690 con 0x7f88981001b0 2024-03-09T00:11:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.312+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88981a68a0 con 0x7f88981001b0 2024-03-09T00:11:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.314+0000 7f88957fa640 1 -- 172.21.15.22:0/2826033693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f888c0046c0 con 0x7f88981001b0 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88957fa640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f88800a2a10 0x7f88800a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 Processor -- start 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 -- start start 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f8897fff640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f88800a2a10 0x7f88800a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88957fa640 1 -- 172.21.15.22:0/2826033693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f888c0cf620 con 0x7f88981001b0 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229fd40 0x55ff7229ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff7229f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff722035f0 con 0x55ff7229fd40 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff721ac930 con 0x55ff7229efd0 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88a15b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff721ff690 con 0x55ff722a8550 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88977fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff7229f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88977fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff7229f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:38644/0 (socket says 172.21.15.22:38644) 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f88977fe640 1 -- 172.21.15.22:0/895596810 learned_addr learned my addr 172.21.15.22:0/895596810 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f889cf71640 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.315+0000 7f8897fff640 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229fd40 0x55ff7229ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f88977fe640 1 -- 172.21.15.22:0/895596810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 msgr2=0x55ff722aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f88977fe640 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f88977fe640 1 -- 172.21.15.22:0/895596810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229fd40 msgr2=0x55ff7229ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f88977fe640 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229fd40 0x55ff7229ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f88977fe640 1 -- 172.21.15.22:0/895596810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff71f18210 con 0x55ff7229efd0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f8897fff640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f88800a2a10 0x7f88800a4ed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8888009c20 tx=0x7f8888011810 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f88977fe640 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff7229f450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f888c009380 tx=0x7f888c09b210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45951bc5ea51a735 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f887e7fc640 1 -- 172.21.15.22:0/895596810 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c09fa00 con 0x55ff7229efd0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f887e7fc640 1 -- 172.21.15.22:0/895596810 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f888c0a4050 con 0x55ff7229efd0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.316+0000 7f887e7fc640 1 -- 172.21.15.22:0/895596810 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c0cf470 con 0x55ff7229efd0 2024-03-09T00:11:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 -- 172.21.15.22:0/895596810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 msgr2=0x55ff7229f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff7229f450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f888c009380 tx=0x7f888c09b210 comp rx=0 tx=0).stop 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 -- 172.21.15.22:0/895596810 shutdown_connections 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff7229f450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 --2- 172.21.15.22:0/895596810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229fd40 0x55ff7229ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 -- 172.21.15.22:0/895596810 >> 172.21.15.22:0/895596810 conn(0x55ff722665b0 msgr2=0x55ff72299cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 -- 172.21.15.22:0/895596810 shutdown_connections 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 -- 172.21.15.22:0/895596810 wait complete. 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 -- start start 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.317+0000 7f88a15b8740 1 Processor -- start 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88a15b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff722ab280 con 0x55ff722a8550 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88a15b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff721bfd90 con 0x55ff7229efd0 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88a15b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff722ab400 con 0x55ff7229fd40 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88a15b8740 10 client.0 _ll_register_callbacks cb 0x55ff71f17a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f889cf71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88977fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:38660/0 (socket says 172.21.15.22:38660) 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 -- 172.21.15.22:0/1222950598 learned_addr learned my addr 172.21.15.22:0/1222950598 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 -- 172.21.15.22:0/1222950598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 msgr2=0x55ff722a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 -- 172.21.15.22:0/1222950598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 msgr2=0x55ff722a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 -- 172.21.15.22:0/1222950598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff71fe4620 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f8897fff640 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722a5610 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f888801e940 tx=0x7f8888004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eaf74f0e790eadb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.318+0000 7f88777fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88777fe640 1 -- 172.21.15.22:0/1222950598 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888017040 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f888c0a4050 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88777fe640 1 -- 172.21.15.22:0/1222950598 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f8888005220 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88777fe640 1 -- 172.21.15.22:0/1222950598 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8888005420 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 10 client.29978 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ff722b7650 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88777fe640 1 -- 172.21.15.22:0/1222950598 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f888c0a4050 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88777fe640 1 -- 172.21.15.22:0/1222950598 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f888800fb00 con 0x55ff7229efd0 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 10 client.29978 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 10 client.29978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 20 client.29978 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff722b8b10 0x55ff722baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 4 client.29978 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.319+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff72210b30 con 0x55ff722b8b10 2024-03-09T00:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.320+0000 7f88977fe640 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff722b8b10 0x55ff722baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.320+0000 7f88977fe640 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff722b8b10 0x55ff722baf00 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c125c3ea2835612 server_cookie=8f7463f606ce25cd in_seq=0 out_seq=0 2024-03-09T00:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.320+0000 7f88777fe640 10 client.29978 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88777fe640 1 -- 172.21.15.22:0/1222950598 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff72210b30 con 0x55ff722b8b10 2024-03-09T00:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88777fe640 10 client.29978 handle_command_reply: tid=0 2024-03-09T00:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 client.29978 shutdown 2024-03-09T00:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 msgr2=0x55ff722a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722a5610 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f888801e940 tx=0x7f8888004730 comp rx=0 tx=0).stop 2024-03-09T00:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 shutdown_connections 2024-03-09T00:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff722b8b10 0x55ff722baf00 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 --2- 172.21.15.22:0/1222950598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722a5610 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88777fe640 0 client.29978 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 >> 172.21.15.22:0/1222950598 conn(0x55ff722665b0 msgr2=0x55ff7229ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 shutdown_connections 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 1 -- 172.21.15.22:0/1222950598 wait complete. 2024-03-09T00:11:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.323+0000 7f88a15b8740 20 client.29978 trim_cache size 0 max 16384 2024-03-09T00:11:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 Processor -- start 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 -- start start 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229efd0 0x55ff722a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229fd40 0x55ff722a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722a4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981b2c40 con 0x55ff7229efd0 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981b07e0 con 0x55ff7229fd40 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88a15b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981ae8a0 con 0x55ff722a8550 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229efd0 0x55ff722a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229efd0 0x55ff722a7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44742/0 (socket says 172.21.15.22:44742) 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 -- 172.21.15.22:0/2355288393 learned_addr learned my addr 172.21.15.22:0/2355288393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f88977fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229fd40 0x55ff722a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f889cf71640 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 -- 172.21.15.22:0/2355288393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 msgr2=0x55ff722a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 -- 172.21.15.22:0/2355288393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229fd40 msgr2=0x55ff722a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229fd40 0x55ff722a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 -- 172.21.15.22:0/2355288393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff7229e2a0 con 0x55ff7229efd0 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f889cf71640 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f8897fff640 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229efd0 0x55ff722a7d40 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f8888006010 tx=0x7f88880096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1ce390fcce620c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.337+0000 7f887dffb640 1 -- 172.21.15.22:0/2355288393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888017040 con 0x55ff7229efd0 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f887dffb640 1 -- 172.21.15.22:0/2355288393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8888019050 con 0x55ff7229efd0 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f887dffb640 1 -- 172.21.15.22:0/2355288393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888800f8f0 con 0x55ff7229efd0 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 -- 172.21.15.22:0/2355288393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229efd0 msgr2=0x55ff722a7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229efd0 0x55ff722a7d40 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f8888006010 tx=0x7f88880096c0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 -- 172.21.15.22:0/2355288393 shutdown_connections 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff722a8550 0x55ff722a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229fd40 0x55ff722a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 --2- 172.21.15.22:0/2355288393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff7229efd0 0x55ff722a7d40 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 -- 172.21.15.22:0/2355288393 >> 172.21.15.22:0/2355288393 conn(0x55ff722665b0 msgr2=0x55ff722997f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 -- 172.21.15.22:0/2355288393 shutdown_connections 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.338+0000 7f88a15b8740 1 -- 172.21.15.22:0/2355288393 wait complete. 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 -- start start 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 Processor -- start 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722ae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981aeea0 con 0x55ff722a8550 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981b0180 con 0x55ff7229efd0 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981af4c0 con 0x55ff7229fd40 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88a15b8740 10 client.0 _ll_register_callbacks cb 0x7f888c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f889cf71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f889cf71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44754/0 (socket says 172.21.15.22:44754) 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f889cf71640 1 -- 172.21.15.22:0/3315234577 learned_addr learned my addr 172.21.15.22:0/3315234577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f8897fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.339+0000 7f88977fe640 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f8897fff640 1 -- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 msgr2=0x55ff722ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f8897fff640 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f8897fff640 1 -- 172.21.15.22:0/3315234577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 msgr2=0x55ff722aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f8897fff640 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f8897fff640 1 -- 172.21.15.22:0/3315234577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff7230efc0 con 0x55ff7229efd0 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f889cf71640 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f8897fff640 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722ae020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8888028420 tx=0x7f8888011de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=590cc6c6476a6ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f88777fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:11:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f88777fe640 1 -- 172.21.15.22:0/3315234577 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888045060 con 0x55ff7229efd0 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8888019050 con 0x55ff7229efd0 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f88777fe640 1 -- 172.21.15.22:0/3315234577 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f8888009040 con 0x55ff7229efd0 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f88777fe640 1 -- 172.21.15.22:0/3315234577 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888804c050 con 0x55ff7229efd0 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f88a15b8740 10 client.29984 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.340+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ff723101f0 con 0x55ff7229efd0 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88777fe640 1 -- 172.21.15.22:0/3315234577 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8888019050 con 0x55ff7229efd0 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88777fe640 1 -- 172.21.15.22:0/3315234577 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f888803f030 con 0x55ff7229efd0 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88a15b8740 10 client.29984 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88a15b8740 10 client.29984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88a15b8740 20 client.29984 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88a15b8740 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff72310750 0x55ff72312b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88a15b8740 4 client.29984 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff7220ae80 con 0x55ff72310750 2024-03-09T00:11:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.341+0000 7f88977fe640 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff72310750 0x55ff72312b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.342+0000 7f88977fe640 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff72310750 0x55ff72312b40 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af45e36f6db16da server_cookie=c0940591e9a66695 in_seq=0 out_seq=0 2024-03-09T00:11:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:36.342+0000 7f88777fe640 10 client.29984 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:36.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:36 smithi022 ceph-mon[31894]: pgmap v543: 129 pgs: 129 active+clean; 383 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-03-09T00:11:36.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:36 smithi062 ceph-mon[36129]: pgmap v543: 129 pgs: 129 active+clean; 383 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-03-09T00:11:36.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:36 smithi192 ceph-mon[35995]: pgmap v543: 129 pgs: 129 active+clean; 383 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-03-09T00:11:37.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 52380 11.55 MB/sec execute 104 sec latency 591.670 ms 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.418+0000 7f88777fe640 1 -- 172.21.15.22:0/3315234577 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff7220ae80 con 0x55ff72310750 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.418+0000 7f88777fe640 10 client.29984 handle_command_reply: tid=0 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.418+0000 7f88a15b8740 1 client.29984 shutdown 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 msgr2=0x55ff722ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722ae020 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8888028420 tx=0x7f8888011de0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 shutdown_connections 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55ff72310750 0x55ff72312b40 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ff722a8550 0x55ff722aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ff7229fd40 0x55ff722ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88777fe640 0 client.29984 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 --2- 172.21.15.22:0/3315234577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff7229efd0 0x55ff722ae020 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 >> 172.21.15.22:0/3315234577 conn(0x55ff722665b0 msgr2=0x55ff722a3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 shutdown_connections 2024-03-09T00:11:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 1 -- 172.21.15.22:0/3315234577 wait complete. 2024-03-09T00:11:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f88a15b8740 20 client.29984 trim_cache size 0 max 16384 2024-03-09T00:11:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:11:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:11:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:11:37.417 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:11:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f88800a2a10 msgr2=0x7f88800a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f88800a2a10 0x7f88800a4ed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8888009c20 tx=0x7f8888011810 comp rx=0 tx=0).stop 2024-03-09T00:11:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 msgr2=0x7f889819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889819a270 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f888c00e9f0 tx=0x7f888c00edf0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.418 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 shutdown_connections 2024-03-09T00:11:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f88800a2a10 0x7f88800a4ed0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88981062d0 0x7f889819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88981001b0 0x7f889819a270 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 --2- 172.21.15.22:0/2826033693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88980ff7f0 0x7f8898199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.419+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 >> 172.21.15.22:0/2826033693 conn(0x7f8898076730 msgr2=0x7f8898104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.420+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 shutdown_connections 2024-03-09T00:11:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:37.420+0000 7f889e9d4640 1 -- 172.21.15.22:0/2826033693 wait complete. 2024-03-09T00:11:38.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 52733 11.65 MB/sec execute 105 sec latency 785.645 ms 2024-03-09T00:11:38.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:38 smithi022 ceph-mon[31894]: pgmap v544: 129 pgs: 129 active+clean; 383 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-03-09T00:11:38.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:38 smithi062 ceph-mon[36129]: pgmap v544: 129 pgs: 129 active+clean; 383 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-03-09T00:11:38.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:38 smithi192 ceph-mon[35995]: pgmap v544: 129 pgs: 129 active+clean; 383 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 316 op/s 2024-03-09T00:11:39.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 53194 11.71 MB/sec execute 106 sec latency 212.592 ms 2024-03-09T00:11:40.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 53506 11.78 MB/sec execute 107 sec latency 98.857 ms 2024-03-09T00:11:40.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:40 smithi022 ceph-mon[31894]: pgmap v545: 129 pgs: 129 active+clean; 383 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-03-09T00:11:40.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:40 smithi062 ceph-mon[36129]: pgmap v545: 129 pgs: 129 active+clean; 383 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-03-09T00:11:40.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:40 smithi192 ceph-mon[35995]: pgmap v545: 129 pgs: 129 active+clean; 383 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-03-09T00:11:41.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 53901 11.92 MB/sec execute 108 sec latency 91.787 ms 2024-03-09T00:11:42.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 54261 12.08 MB/sec execute 109 sec latency 167.172 ms 2024-03-09T00:11:42.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:42 smithi022 ceph-mon[31894]: pgmap v546: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.5 MiB/s wr, 584 op/s 2024-03-09T00:11:42.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:42 smithi062 ceph-mon[36129]: pgmap v546: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.5 MiB/s wr, 584 op/s 2024-03-09T00:11:42.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:42 smithi192 ceph-mon[35995]: pgmap v546: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.5 MiB/s wr, 584 op/s 2024-03-09T00:11:43.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 54519 12.05 MB/sec execute 110 sec latency 222.877 ms 2024-03-09T00:11:44.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 54727 12.06 MB/sec execute 111 sec latency 262.571 ms 2024-03-09T00:11:44.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:44 smithi022 ceph-mon[31894]: pgmap v547: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.9 MiB/s wr, 450 op/s 2024-03-09T00:11:44.806 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:44 smithi192 ceph-mon[35995]: pgmap v547: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.9 MiB/s wr, 450 op/s 2024-03-09T00:11:44.806 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:44 smithi062 ceph-mon[36129]: pgmap v547: 129 pgs: 129 active+clean; 369 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.9 MiB/s wr, 450 op/s 2024-03-09T00:11:45.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 54919 12.01 MB/sec execute 112 sec latency 256.857 ms 2024-03-09T00:11:46.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 55117 12.02 MB/sec execute 113 sec latency 441.612 ms 2024-03-09T00:11:46.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:46 smithi022 ceph-mon[31894]: pgmap v548: 129 pgs: 129 active+clean; 371 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 491 op/s 2024-03-09T00:11:46.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:46 smithi062 ceph-mon[36129]: pgmap v548: 129 pgs: 129 active+clean; 371 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 491 op/s 2024-03-09T00:11:46.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:46 smithi192 ceph-mon[35995]: pgmap v548: 129 pgs: 129 active+clean; 371 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 491 op/s 2024-03-09T00:11:47.074 INFO:tasks.workunit.client.0.smithi022.stdout: 10 55364 11.95 MB/sec execute 114 sec latency 177.201 ms 2024-03-09T00:11:48.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 55698 11.98 MB/sec execute 115 sec latency 95.129 ms 2024-03-09T00:11:48.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:48 smithi022 ceph-mon[31894]: pgmap v549: 129 pgs: 129 active+clean; 381 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 432 op/s 2024-03-09T00:11:48.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:48 smithi062 ceph-mon[36129]: pgmap v549: 129 pgs: 129 active+clean; 381 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 432 op/s 2024-03-09T00:11:48.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:48 smithi192 ceph-mon[35995]: pgmap v549: 129 pgs: 129 active+clean; 381 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 432 op/s 2024-03-09T00:11:49.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 56086 12.10 MB/sec execute 116 sec latency 225.392 ms 2024-03-09T00:11:50.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 56562 12.21 MB/sec execute 117 sec latency 205.223 ms 2024-03-09T00:11:50.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:50 smithi022 ceph-mon[31894]: pgmap v550: 129 pgs: 129 active+clean; 381 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 435 op/s 2024-03-09T00:11:50.527 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 -- 172.21.15.22:0/3995343420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c10dff0 msgr2=0x7fa84c1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:50.527 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3995343420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c10dff0 0x7fa84c1103e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ca0 tx=0x7fa84402c180 comp rx=0 tx=0).stop 2024-03-09T00:11:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 -- 172.21.15.22:0/3995343420 shutdown_connections 2024-03-09T00:11:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3995343420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c10dff0 0x7fa84c1103e0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3995343420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84c069290 0x7fa84c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3995343420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c068970 0x7fa84c068d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 -- 172.21.15.22:0/3995343420 >> 172.21.15.22:0/3995343420 conn(0x7fa84c077040 msgr2=0x7fa84c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 -- 172.21.15.22:0/3995343420 shutdown_connections 2024-03-09T00:11:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.527+0000 7fa8540d1640 1 -- 172.21.15.22:0/3995343420 wait complete. 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 Processor -- start 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 -- start start 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c068970 0x7fa84c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c069290 0x7fa84c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84c10dff0 0x7fa84c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c113870 con 0x7fa84c069290 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1136f0 con 0x7fa84c10dff0 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa8540d1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1139f0 con 0x7fa84c068970 2024-03-09T00:11:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c069290 0x7fa84c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c069290 0x7fa84c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47376/0 (socket says 172.21.15.22:47376) 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 -- 172.21.15.22:0/3945458785 learned_addr learned my addr 172.21.15.22:0/3945458785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa851e6d640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c068970 0x7fa84c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85266e640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84c10dff0 0x7fa84c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 -- 172.21.15.22:0/3945458785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c068970 msgr2=0x7fa84c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c068970 0x7fa84c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 -- 172.21.15.22:0/3945458785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84c10dff0 msgr2=0x7fa84c1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84c10dff0 0x7fa84c1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.528+0000 7fa85166c640 1 -- 172.21.15.22:0/3945458785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c19a130 con 0x7fa84c069290 2024-03-09T00:11:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.529+0000 7fa85166c640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c069290 0x7fa84c19fd40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c920 tx=0x7fa83c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.529+0000 7fa842ffd640 1 -- 172.21.15.22:0/3945458785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007c70 con 0x7fa84c069290 2024-03-09T00:11:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.529+0000 7fa842ffd640 1 -- 172.21.15.22:0/3945458785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa83c007e10 con 0x7fa84c069290 2024-03-09T00:11:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.529+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c19a420 con 0x7fa84c069290 2024-03-09T00:11:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.529+0000 7fa842ffd640 1 -- 172.21.15.22:0/3945458785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c018890 con 0x7fa84c069290 2024-03-09T00:11:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.529+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c1a66c0 con 0x7fa84c069290 2024-03-09T00:11:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.531+0000 7fa842ffd640 1 -- 172.21.15.22:0/3945458785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa83c018b30 con 0x7fa84c069290 2024-03-09T00:11:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.531+0000 7fa856d8e740 1 Processor -- start 2024-03-09T00:11:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.531+0000 7fa856d8e740 1 -- start start 2024-03-09T00:11:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461cfea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cfefd0 0x562461cff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa842ffd640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8380a2a90 0x7fa8380a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461d08550 0x562461d0a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa856d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562461c635f0 con 0x562461cfefd0 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa856d8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562461c0c930 con 0x562461cffd40 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa856d8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562461c5f690 con 0x562461d08550 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa851e6d640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8380a2a90 0x7fa8380a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cfefd0 0x562461cff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cfefd0 0x562461cff450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47390/0 (socket says 172.21.15.22:47390) 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 -- 172.21.15.22:0/174000008 learned_addr learned my addr 172.21.15.22:0/174000008 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85266e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461d08550 0x562461d0a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85266e640 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461cfea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa842ffd640 1 -- 172.21.15.22:0/3945458785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7fa83c0cf240 con 0x7fa84c069290 2024-03-09T00:11:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 -- 172.21.15.22:0/174000008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461d08550 msgr2=0x562461d0a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461d08550 0x562461d0a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 -- 172.21.15.22:0/174000008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 msgr2=0x562461cfea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461cfea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 -- 172.21.15.22:0/174000008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562461978210 con 0x562461cfefd0 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85266e640 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461cfea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa851e6d640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8380a2a90 0x7fa8380a4f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa84c19aaf0 tx=0x7fa848005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.532+0000 7fa85166c640 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cfefd0 0x562461cff450 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0b9fd0 tx=0x7fa83c0c5f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ccea5e292d869d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa837fff640 1 -- 172.21.15.22:0/174000008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0152b0 con 0x562461cfefd0 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa837fff640 1 -- 172.21.15.22:0/174000008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa83c09bbd0 con 0x562461cfefd0 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa837fff640 1 -- 172.21.15.22:0/174000008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00fb30 con 0x562461cfefd0 2024-03-09T00:11:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa856d8e740 1 -- 172.21.15.22:0/174000008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cfefd0 msgr2=0x562461cff450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa856d8e740 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cfefd0 0x562461cff450 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0b9fd0 tx=0x7fa83c0c5f20 comp rx=0 tx=0).stop 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa856d8e740 1 -- 172.21.15.22:0/174000008 shutdown_connections 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa856d8e740 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461d08550 0x562461d0a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa856d8e740 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cfefd0 0x562461cff450 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa856d8e740 1 --2- 172.21.15.22:0/174000008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461cfea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.533+0000 7fa856d8e740 1 -- 172.21.15.22:0/174000008 >> 172.21.15.22:0/174000008 conn(0x562461cc65b0 msgr2=0x562461cf9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 -- 172.21.15.22:0/174000008 shutdown_connections 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 -- 172.21.15.22:0/174000008 wait complete. 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 -- start start 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 Processor -- start 2024-03-09T00:11:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d05620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d05b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461d08550 0x562461d060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562461d0b280 con 0x562461d08550 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562461c1fd90 con 0x562461cffd40 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.534+0000 7fa856d8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562461d0b400 con 0x562461cfefd0 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461d08550 0x562461d060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa851e6d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d05620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461d08550 0x562461d060a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47398/0 (socket says 172.21.15.22:47398) 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 -- 172.21.15.22:0/3330156887 learned_addr learned my addr 172.21.15.22:0/3330156887 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa856d8e740 10 client.0 _ll_register_callbacks cb 0x562461977a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85166c640 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d05b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 -- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 msgr2=0x562461d05620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d05620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 -- 172.21.15.22:0/3330156887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 msgr2=0x562461d05b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d05b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 -- 172.21.15.22:0/3330156887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562461a44620 con 0x562461d08550 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa851e6d640 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d05620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa85266e640 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461d08550 0x562461d060a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fa844004af0 tx=0x7fa844009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc681c68dca55383 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa8357fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa8357fa640 1 -- 172.21.15.22:0/3330156887 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844047070 con 0x562461d08550 2024-03-09T00:11:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa8357fa640 1 -- 172.21.15.22:0/3330156887 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7fa844004400 con 0x562461d08550 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa83c09bbd0 con 0x562461d08550 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa8357fa640 1 -- 172.21.15.22:0/3330156887 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa844053050 con 0x562461d08550 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa856d8e740 10 client.20796 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.535+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562461d1fe30 con 0x562461d08550 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa8357fa640 1 -- 172.21.15.22:0/3330156887 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa83c09bbd0 con 0x562461d08550 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa8357fa640 1 -- 172.21.15.22:0/3330156887 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa84403f040 con 0x562461d08550 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa856d8e740 10 client.20796 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa856d8e740 10 client.20796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa856d8e740 20 client.20796 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa856d8e740 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d212a0 0x562461d23690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa856d8e740 4 client.20796 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562461c70b30 con 0x562461d212a0 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa851e6d640 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d212a0 0x562461d23690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa851e6d640 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d212a0 0x562461d23690 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a3433a789b23dd server_cookie=3152b670954503e9 in_seq=0 out_seq=0 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.536+0000 7fa8357fa640 10 client.20796 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa8357fa640 1 -- 172.21.15.22:0/3330156887 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562461c70b30 con 0x562461d212a0 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa8357fa640 10 client.20796 handle_command_reply: tid=0 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 client.20796 shutdown 2024-03-09T00:11:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461d08550 msgr2=0x562461d060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461d08550 0x562461d060a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fa844004af0 tx=0x7fa844009800 comp rx=0 tx=0).stop 2024-03-09T00:11:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 shutdown_connections 2024-03-09T00:11:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d212a0 0x562461d23690 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa8357fa640 0 client.20796 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461d08550 0x562461d060a0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d05b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 --2- 172.21.15.22:0/3330156887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d05620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.540+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 >> 172.21.15.22:0/3330156887 conn(0x562461cc65b0 msgr2=0x562461cfba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.541+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 shutdown_connections 2024-03-09T00:11:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.541+0000 7fa856d8e740 1 -- 172.21.15.22:0/3330156887 wait complete. 2024-03-09T00:11:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.541+0000 7fa856d8e740 20 client.20796 trim_cache size 0 max 16384 2024-03-09T00:11:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 Processor -- start 2024-03-09T00:11:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 -- start start 2024-03-09T00:11:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cc65b0 0x562461d07d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cfefd0 0x562461d05ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cffd40 0x562461d0f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1ae770 con 0x562461cffd40 2024-03-09T00:11:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1ac310 con 0x562461cfefd0 2024-03-09T00:11:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa856d8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1aa410 con 0x562461cc65b0 2024-03-09T00:11:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.554+0000 7fa85266e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cffd40 0x562461d0f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cffd40 0x562461d0f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47410/0 (socket says 172.21.15.22:47410) 2024-03-09T00:11:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa851e6d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cc65b0 0x562461d07d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 -- 172.21.15.22:0/2111340112 learned_addr learned my addr 172.21.15.22:0/2111340112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85166c640 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cfefd0 0x562461d05ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 -- 172.21.15.22:0/2111340112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cc65b0 msgr2=0x562461d07d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cc65b0 0x562461d07d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 -- 172.21.15.22:0/2111340112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cfefd0 msgr2=0x562461d05ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cfefd0 0x562461d05ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 -- 172.21.15.22:0/2111340112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562461cfe2a0 con 0x562461cffd40 2024-03-09T00:11:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85166c640 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cfefd0 0x562461d05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa85266e640 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cffd40 0x562461d0f670 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ca0 tx=0x7fa844009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e19a66e33b1b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa8377fe640 1 -- 172.21.15.22:0/2111340112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844050070 con 0x562461cffd40 2024-03-09T00:11:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa8377fe640 1 -- 172.21.15.22:0/2111340112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8440323c0 con 0x562461cffd40 2024-03-09T00:11:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.555+0000 7fa8377fe640 1 -- 172.21.15.22:0/2111340112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844047040 con 0x562461cffd40 2024-03-09T00:11:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 -- 172.21.15.22:0/2111340112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cffd40 msgr2=0x562461d0f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cffd40 0x562461d0f670 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ca0 tx=0x7fa844009ba0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 -- 172.21.15.22:0/2111340112 shutdown_connections 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cffd40 0x562461d0f670 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cfefd0 0x562461d05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 --2- 172.21.15.22:0/2111340112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cc65b0 0x562461d07d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 -- 172.21.15.22:0/2111340112 >> 172.21.15.22:0/2111340112 conn(0x562461d0ab20 msgr2=0x562461cf97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 -- 172.21.15.22:0/2111340112 shutdown_connections 2024-03-09T00:11:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.556+0000 7fa856d8e740 1 -- 172.21.15.22:0/2111340112 wait complete. 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 -- start start 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 Processor -- start 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cc65b0 0x562461d0f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d01e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1aaa10 con 0x562461cc65b0 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1abcb0 con 0x562461cffd40 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c1ab010 con 0x562461cfefd0 2024-03-09T00:11:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa856d8e740 10 client.0 _ll_register_callbacks cb 0x7fa83c09b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa851e6d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cc65b0 0x562461d0f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa851e6d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cc65b0 0x562461d0f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47426/0 (socket says 172.21.15.22:47426) 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa851e6d640 1 -- 172.21.15.22:0/551855437 learned_addr learned my addr 172.21.15.22:0/551855437 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa85166c640 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d01e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.557+0000 7fa85266e640 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa851e6d640 1 -- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 msgr2=0x562461d01e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa851e6d640 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d01e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa851e6d640 1 -- 172.21.15.22:0/551855437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 msgr2=0x562461d023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa851e6d640 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa851e6d640 1 -- 172.21.15.22:0/551855437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562461d02910 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa851e6d640 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cc65b0 0x562461d0f870 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa8480118d0 tx=0x7fa848013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6d8e4d937e1dd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa8357fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa8357fa640 1 -- 172.21.15.22:0/551855437 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84800ed90 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa8357fa640 1 -- 172.21.15.22:0/551855437 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7fa848005050 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8440323c0 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa8357fa640 1 -- 172.21.15.22:0/551855437 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa848005250 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa856d8e740 10 client.20802 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562461d78540 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.558+0000 7fa8357fa640 1 -- 172.21.15.22:0/551855437 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8440323c0 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa8357fa640 1 -- 172.21.15.22:0/551855437 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa848031550 con 0x562461cc65b0 2024-03-09T00:11:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa856d8e740 10 client.20802 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa856d8e740 10 client.20802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa856d8e740 20 client.20802 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa856d8e740 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d08550 0x562461d7ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa856d8e740 4 client.20802 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562461c6ae80 con 0x562461d08550 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa85166c640 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d08550 0x562461d7ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa8357fa640 10 client.20802 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:50.559+0000 7fa85166c640 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d08550 0x562461d7ad60 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb3ccff21e0c3f9 server_cookie=421837e3ff9b1e4d in_seq=0 out_seq=0 2024-03-09T00:11:50.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:50 smithi062 ceph-mon[36129]: pgmap v550: 129 pgs: 129 active+clean; 381 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 435 op/s 2024-03-09T00:11:50.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:50 smithi192 ceph-mon[35995]: pgmap v550: 129 pgs: 129 active+clean; 381 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 435 op/s 2024-03-09T00:11:51.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 56659 12.15 MB/sec execute 118 sec latency 570.103 ms 2024-03-09T00:11:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.735+0000 7fa8357fa640 1 -- 172.21.15.22:0/551855437 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562461c6ae80 con 0x562461d08550 2024-03-09T00:11:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.735+0000 7fa8357fa640 10 client.20802 handle_command_reply: tid=0 2024-03-09T00:11:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.735+0000 7fa856d8e740 1 client.20802 shutdown 2024-03-09T00:11:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cc65b0 msgr2=0x562461d0f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cc65b0 0x562461d0f870 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa8480118d0 tx=0x7fa848013040 comp rx=0 tx=0).stop 2024-03-09T00:11:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 shutdown_connections 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x562461d08550 0x562461d7ad60 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562461cffd40 0x562461d023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa8357fa640 0 client.20802 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562461cfefd0 0x562461d01e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 --2- 172.21.15.22:0/551855437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562461cc65b0 0x562461d0f870 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 >> 172.21.15.22:0/551855437 conn(0x562461d0ab20 msgr2=0x562461d03e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.736+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 shutdown_connections 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.737+0000 7fa856d8e740 1 -- 172.21.15.22:0/551855437 wait complete. 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.737+0000 7fa856d8e740 20 client.20802 trim_cache size 0 max 16384 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:{ 2024-03-09T00:11:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "message": "", 2024-03-09T00:11:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout: "return_code": 0 2024-03-09T00:11:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stdout:} 2024-03-09T00:11:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8380a2a90 msgr2=0x7fa8380a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8380a2a90 0x7fa8380a4f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa84c19aaf0 tx=0x7fa848005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c069290 msgr2=0x7fa84c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c069290 0x7fa84c19fd40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c920 tx=0x7fa83c00cde0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 shutdown_connections 2024-03-09T00:11:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa8380a2a90 0x7fa8380a4f50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84c10dff0 0x7fa84c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.738+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa84c069290 0x7fa84c19fd40 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.739+0000 7fa8540d1640 1 --2- 172.21.15.22:0/3945458785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa84c068970 0x7fa84c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.739+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 >> 172.21.15.22:0/3945458785 conn(0x7fa84c077040 msgr2=0x7fa84c10f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.739+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 shutdown_connections 2024-03-09T00:11:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi022.front.sepia.ceph.com.smithi022.stderr:2024-03-09T00:11:51.739+0000 7fa8540d1640 1 -- 172.21.15.22:0/3945458785 wait complete. 2024-03-09T00:11:52.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 56822 12.10 MB/sec execute 119 sec latency 731.797 ms 2024-03-09T00:11:52.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:52 smithi022 ceph-mon[31894]: pgmap v551: 129 pgs: 129 active+clean; 389 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 536 op/s 2024-03-09T00:11:52.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:52 smithi062 ceph-mon[36129]: pgmap v551: 129 pgs: 129 active+clean; 389 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 536 op/s 2024-03-09T00:11:52.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:52 smithi192 ceph-mon[35995]: pgmap v551: 129 pgs: 129 active+clean; 389 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 536 op/s 2024-03-09T00:11:53.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 57196 12.22 MB/sec execute 120 sec latency 163.840 ms 2024-03-09T00:11:53.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:11:53.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:11:53.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 -- 172.21.15.22:0/339645629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 msgr2=0x7fd91410b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 --2- 172.21.15.22:0/339645629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 0x7fd91410b380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd900009900 tx=0x7fd90002bcd0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 -- 172.21.15.22:0/339645629 shutdown_connections 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 --2- 172.21.15.22:0/339645629 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd91410b8c0 0x7fd91410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 --2- 172.21.15.22:0/339645629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 0x7fd91410b380 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 --2- 172.21.15.22:0/339645629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd914102580 0x7fd914102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 -- 172.21.15.22:0/339645629 >> 172.21.15.22:0/339645629 conn(0x7fd914077350 msgr2=0x7fd914077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 -- 172.21.15.22:0/339645629 shutdown_connections 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 -- 172.21.15.22:0/339645629 wait complete. 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 Processor -- start 2024-03-09T00:11:53.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.690+0000 7fd91b153640 1 -- start start 2024-03-09T00:11:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd91b153640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd914102580 0x7fd91419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd91b153640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 0x7fd91419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd91b153640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91410b8c0 0x7fd9141979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd91b153640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141116a0 con 0x7fd914102580 2024-03-09T00:11:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd91b153640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914111520 con 0x7fd914102ea0 2024-03-09T00:11:53.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd91b153640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914111820 con 0x7fd91410b8c0 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd914102580 0x7fd91419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd914102580 0x7fd91419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47436/0 (socket says 172.21.15.22:47436) 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 -- 172.21.15.22:0/2716234219 learned_addr learned my addr 172.21.15.22:0/2716234219 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd9196f0640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91410b8c0 0x7fd9141979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd913fff640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 0x7fd91419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 -- 172.21.15.22:0/2716234219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91410b8c0 msgr2=0x7fd9141979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91410b8c0 0x7fd9141979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 -- 172.21.15.22:0/2716234219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 msgr2=0x7fd91419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 0x7fd91419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 -- 172.21.15.22:0/2716234219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914198160 con 0x7fd914102580 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd913fff640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 0x7fd91419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.691+0000 7fd918eef640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd914102580 0x7fd91419d830 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd90800ea60 tx=0x7fd90800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.692+0000 7fd911ffb640 1 -- 172.21.15.22:0/2716234219 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800cde0 con 0x7fd914102580 2024-03-09T00:11:53.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.692+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd914198450 con 0x7fd914102580 2024-03-09T00:11:53.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.692+0000 7fd911ffb640 1 -- 172.21.15.22:0/2716234219 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9080035f0 con 0x7fd914102580 2024-03-09T00:11:53.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.692+0000 7fd911ffb640 1 -- 172.21.15.22:0/2716234219 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908010670 con 0x7fd914102580 2024-03-09T00:11:53.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.692+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9141a46f0 con 0x7fd914102580 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.694+0000 7fd911ffb640 1 -- 172.21.15.22:0/2716234219 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd9080037a0 con 0x7fd914102580 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 Processor -- start 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 -- start start 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af38b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af41720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af44050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050ae9be00 con 0x56050af38760 2024-03-09T00:11:53.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050ae44b10 con 0x56050af39080 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd91de12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050ae97ea0 con 0x56050af41c60 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd911ffb640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9040a2970 0x7fd9040a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd9196f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af38b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af44050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd918eef640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af41720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af44050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48440/0 (socket says 172.21.15.22:48440) 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 -- 172.21.15.22:0/1570061159 learned_addr learned my addr 172.21.15.22:0/1570061159 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd911ffb640 1 -- 172.21.15.22:0/2716234219 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7fd9080cf490 con 0x7fd914102580 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd9196f0640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9040a2970 0x7fd9040a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 -- 172.21.15.22:0/1570061159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 msgr2=0x56050af41720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af41720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 -- 172.21.15.22:0/1570061159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 msgr2=0x56050af38b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af38b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd913fff640 1 -- 172.21.15.22:0/1570061159 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56050ac7a7d0 con 0x56050af41c60 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.695+0000 7fd918eef640 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af41720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.696+0000 7fd913fff640 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af44050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd900009230 tx=0x7fd900009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33781a8d564ecd7f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.696+0000 7fd9196f0640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9040a2970 0x7fd9040a4e30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005fd0 tx=0x7fd90c004ab0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.696+0000 7fd8feffd640 1 -- 172.21.15.22:0/1570061159 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900043070 con 0x56050af41c60 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.696+0000 7fd8feffd640 1 -- 172.21.15.22:0/1570061159 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90002c6f0 con 0x56050af41c60 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.696+0000 7fd8feffd640 1 -- 172.21.15.22:0/1570061159 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900004ee0 con 0x56050af41c60 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 -- 172.21.15.22:0/1570061159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 msgr2=0x56050af44050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af44050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd900009230 tx=0x7fd900009560 comp rx=0 tx=0).stop 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 -- 172.21.15.22:0/1570061159 shutdown_connections 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af44050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af41720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 --2- 172.21.15.22:0/1570061159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af38b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 -- 172.21.15.22:0/1570061159 >> 172.21.15.22:0/1570061159 conn(0x56050af32e90 msgr2=0x56050af33270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:53.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 -- 172.21.15.22:0/1570061159 shutdown_connections 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 -- 172.21.15.22:0/1570061159 wait complete. 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 -- start start 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 Processor -- start 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.697+0000 7fd91de12740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd91de12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd91de12740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd91de12740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050af449f0 con 0x56050af41c60 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd91de12740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050ae580a0 con 0x56050af38760 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd91de12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56050af44b70 con 0x56050af39080 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd91de12740 10 client.0 _ll_register_callbacks cb 0x56050ae57a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd9196f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd913fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd918eef640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd9196f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47464/0 (socket says 172.21.15.22:47464) 2024-03-09T00:11:53.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd9196f0640 1 -- 172.21.15.22:0/764609567 learned_addr learned my addr 172.21.15.22:0/764609567 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd918eef640 1 -- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 msgr2=0x56050af3e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd918eef640 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd918eef640 1 -- 172.21.15.22:0/764609567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 msgr2=0x56050af3ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd918eef640 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd918eef640 1 -- 172.21.15.22:0/764609567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56050abb03f0 con 0x56050af38760 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd913fff640 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd918eef640 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af412f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd9080cc8b0 tx=0x7fd908097b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb9348bf1704a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd8f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd8f3fff640 1 -- 172.21.15.22:0/764609567 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90809bec0 con 0x56050af38760 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd90002c6f0 con 0x56050af38760 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.698+0000 7fd8f3fff640 1 -- 172.21.15.22:0/764609567 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7fd9080cf170 con 0x56050af38760 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd8f3fff640 1 -- 172.21.15.22:0/764609567 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90809b7c0 con 0x56050af38760 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 10 client.30011 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:53.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56050ab700f0 con 0x56050af38760 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd8f3fff640 1 -- 172.21.15.22:0/764609567 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90002c6f0 con 0x56050af38760 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd8f3fff640 1 -- 172.21.15.22:0/764609567 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd908097e30 con 0x56050af38760 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 10 client.30011 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 10 client.30011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 20 client.30011 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050af51650 0x56050af53a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 4 client.30011 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56050aea9340 con 0x56050af51650 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd913fff640 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050af51650 0x56050af53a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd8f3fff640 10 client.30011 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:53.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.699+0000 7fd913fff640 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050af51650 0x56050af53a80 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0e1fbbe0532245 server_cookie=7370ebc027492428 in_seq=0 out_seq=0 2024-03-09T00:11:53.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.702+0000 7fd8f3fff640 1 -- 172.21.15.22:0/764609567 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56050aea9340 con 0x56050af51650 2024-03-09T00:11:53.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.702+0000 7fd8f3fff640 10 client.30011 handle_command_reply: tid=0 2024-03-09T00:11:53.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.702+0000 7fd91de12740 1 client.30011 shutdown 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.702+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 msgr2=0x56050af412f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.702+0000 7fd91de12740 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af412f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd9080cc8b0 tx=0x7fd908097b50 comp rx=0 tx=0).stop 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 shutdown_connections 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050af51650 0x56050af53a80 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 --2- 172.21.15.22:0/764609567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af412f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd8f3fff640 0 client.30011 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 >> 172.21.15.22:0/764609567 conn(0x56050af32e90 msgr2=0x56050af34be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 shutdown_connections 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 1 -- 172.21.15.22:0/764609567 wait complete. 2024-03-09T00:11:53.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.703+0000 7fd91de12740 20 client.30011 trim_cache size 0 max 16384 2024-03-09T00:11:53.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 Processor -- start 2024-03-09T00:11:53.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 -- start start 2024-03-09T00:11:53.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af40e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af48da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141ae950 con 0x56050af38760 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141ac4f0 con 0x56050af39080 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd91de12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141aa5b0 con 0x56050af41c60 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af40e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af40e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47470/0 (socket says 172.21.15.22:47470) 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 -- 172.21.15.22:0/3587127965 learned_addr learned my addr 172.21.15.22:0/3587127965 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd913fff640 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af48da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd9196f0640 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 -- 172.21.15.22:0/3587127965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 msgr2=0x56050af492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 -- 172.21.15.22:0/3587127965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 msgr2=0x56050af48da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af48da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.719+0000 7fd918eef640 1 -- 172.21.15.22:0/3587127965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56050af38080 con 0x56050af38760 2024-03-09T00:11:53.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.720+0000 7fd918eef640 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af40e20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd908000c00 tx=0x7fd90809b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d709db4e3433ff server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.720+0000 7fd8fe7fc640 1 -- 172.21.15.22:0/3587127965 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90809b300 con 0x56050af38760 2024-03-09T00:11:53.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.720+0000 7fd8fe7fc640 1 -- 172.21.15.22:0/3587127965 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9080a3f60 con 0x56050af38760 2024-03-09T00:11:53.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.720+0000 7fd8fe7fc640 1 -- 172.21.15.22:0/3587127965 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90809af10 con 0x56050af38760 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- 172.21.15.22:0/3587127965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 msgr2=0x56050af40e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af40e20 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd908000c00 tx=0x7fd90809b200 comp rx=0 tx=0).stop 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- 172.21.15.22:0/3587127965 shutdown_connections 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af41c60 0x56050af492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af39080 0x56050af48da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 --2- 172.21.15.22:0/3587127965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af38760 0x56050af40e20 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- 172.21.15.22:0/3587127965 >> 172.21.15.22:0/3587127965 conn(0x56050af32e90 msgr2=0x56050af3e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- 172.21.15.22:0/3587127965 shutdown_connections 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- 172.21.15.22:0/3587127965 wait complete. 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- start start 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 Processor -- start 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af49110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141aabb0 con 0x56050af41c60 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141abe90 con 0x56050af38760 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141ab1d0 con 0x56050af39080 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd91de12740 10 client.0 _ll_register_callbacks cb 0x7fd90800e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.721+0000 7fd9196f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47472/0 (socket says 172.21.15.22:47472) 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 -- 172.21.15.22:0/3134041125 learned_addr learned my addr 172.21.15.22:0/3134041125 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd918eef640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af49110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd913fff640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 -- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 msgr2=0x56050af3b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 -- 172.21.15.22:0/3134041125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 msgr2=0x56050af49110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af49110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 -- 172.21.15.22:0/3134041125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56050af3bd80 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd918eef640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af49110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd9196f0640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3b5a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010710 tx=0x7fd90c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6baeef91a55db50 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd8f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd8f3fff640 1 -- 172.21.15.22:0/3134041125 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c004b20 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd8f3fff640 1 -- 172.21.15.22:0/3134041125 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7fd90c004cc0 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56050aea3690 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd8f3fff640 1 -- 172.21.15.22:0/3134041125 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90c020790 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd91de12740 10 client.20820 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56050afa8b80 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.722+0000 7fd8f3fff640 1 -- 172.21.15.22:0/3134041125 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56050aea3690 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.723+0000 7fd8f3fff640 1 -- 172.21.15.22:0/3134041125 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd90c005570 con 0x56050af41c60 2024-03-09T00:11:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.723+0000 7fd91de12740 10 client.20820 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.723+0000 7fd91de12740 10 client.20820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.723+0000 7fd91de12740 20 client.20820 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.723+0000 7fd91de12740 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050afa9300 0x56050afab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.723+0000 7fd91de12740 4 client.20820 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.723+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56050ae214e0 con 0x56050afa9300 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.724+0000 7fd918eef640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050afa9300 0x56050afab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.724+0000 7fd918eef640 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050afa9300 0x56050afab720 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd411417821bab34 server_cookie=48625e03428c06b6 in_seq=0 out_seq=0 2024-03-09T00:11:53.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.724+0000 7fd8f3fff640 10 client.20820 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd8f3fff640 1 -- 172.21.15.22:0/3134041125 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56050ae214e0 con 0x56050afa9300 2024-03-09T00:11:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd8f3fff640 10 client.20820 handle_command_reply: tid=0 2024-03-09T00:11:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 client.20820 shutdown 2024-03-09T00:11:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 msgr2=0x56050af3b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3b5a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010710 tx=0x7fd90c009500 comp rx=0 tx=0).stop 2024-03-09T00:11:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 shutdown_connections 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56050afa9300 0x56050afab720 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56050af41c60 0x56050af3b5a0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56050af39080 0x56050af3b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd8f3fff640 0 client.20820 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 --2- 172.21.15.22:0/3134041125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050af38760 0x56050af49110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 >> 172.21.15.22:0/3134041125 conn(0x56050af32e90 msgr2=0x56050af3cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 shutdown_connections 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 1 -- 172.21.15.22:0/3134041125 wait complete. 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.728+0000 7fd91de12740 20 client.20820 trim_cache size 0 max 16384 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:11:53.727 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:11:53.728 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:11:53.728 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:11:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9040a2970 msgr2=0x7fd9040a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9040a2970 0x7fd9040a4e30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005fd0 tx=0x7fd90c004ab0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd914102580 msgr2=0x7fd91419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd914102580 0x7fd91419d830 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd90800ea60 tx=0x7fd90800ef20 comp rx=0 tx=0).stop 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 shutdown_connections 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fd9040a2970 0x7fd9040a4e30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91410b8c0 0x7fd9141979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd914102ea0 0x7fd91419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 --2- 172.21.15.22:0/2716234219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd914102580 0x7fd91419d830 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 >> 172.21.15.22:0/2716234219 conn(0x7fd914077350 msgr2=0x7fd9140fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 shutdown_connections 2024-03-09T00:11:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.729+0000 7fd91b153640 1 -- 172.21.15.22:0/2716234219 wait complete. 2024-03-09T00:11:53.737 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:11:53.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:11:53.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 -- 172.21.15.22:0/1841006307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c4810b930 msgr2=0x7f5c4810ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 --2- 172.21.15.22:0/1841006307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c4810b930 0x7f5c4810ddd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009ca0 tx=0x7f5c4002c0f0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 -- 172.21.15.22:0/1841006307 shutdown_connections 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 --2- 172.21.15.22:0/1841006307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c4810b930 0x7f5c4810ddd0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 --2- 172.21.15.22:0/1841006307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c48102e80 0x7f5c4810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 --2- 172.21.15.22:0/1841006307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c48102560 0x7f5c48102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 -- 172.21.15.22:0/1841006307 >> 172.21.15.22:0/1841006307 conn(0x7f5c480772f0 msgr2=0x7f5c48077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 -- 172.21.15.22:0/1841006307 shutdown_connections 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 -- 172.21.15.22:0/1841006307 wait complete. 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 Processor -- start 2024-03-09T00:11:53.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.853+0000 7f5c4e243640 1 -- start start 2024-03-09T00:11:53.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c4e243640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c48102560 0x7f5c48199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c4e243640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c48102e80 0x7f5c4819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c4e243640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c4810b930 0x7f5c4819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c4e243640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c481116f0 con 0x7f5c48102e80 2024-03-09T00:11:53.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c4e243640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48111570 con 0x7f5c48102560 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c4e243640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48111870 con 0x7f5c4810b930 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c46ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c48102e80 0x7f5c4819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c46ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c48102e80 0x7f5c4819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47492/0 (socket says 172.21.15.22:47492) 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c46ffd640 1 -- 172.21.15.22:0/2238035457 learned_addr learned my addr 172.21.15.22:0/2238035457 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c477fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c48102560 0x7f5c48199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c47fff640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c4810b930 0x7f5c4819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c477fe640 1 -- 172.21.15.22:0/2238035457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c4810b930 msgr2=0x7f5c4819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c477fe640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c4810b930 0x7f5c4819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c477fe640 1 -- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c48102e80 msgr2=0x7f5c4819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c477fe640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c48102e80 0x7f5c4819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c477fe640 1 -- 172.21.15.22:0/2238035457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4819ed40 con 0x7f5c48102560 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c46ffd640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c48102e80 0x7f5c4819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c47fff640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c4810b930 0x7f5c4819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.854+0000 7f5c477fe640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c48102560 0x7f5c48199d50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9f0 tx=0x7f5c3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.855+0000 7f5c44ff9640 1 -- 172.21.15.22:0/2238035457 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800ccc0 con 0x7f5c48102560 2024-03-09T00:11:53.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.855+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4819f030 con 0x7f5c48102560 2024-03-09T00:11:53.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.855+0000 7f5c44ff9640 1 -- 172.21.15.22:0/2238035457 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c38010910 con 0x7f5c48102560 2024-03-09T00:11:53.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.855+0000 7f5c44ff9640 1 -- 172.21.15.22:0/2238035457 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380189c0 con 0x7f5c48102560 2024-03-09T00:11:53.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.855+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c481a68c0 con 0x7f5c48102560 2024-03-09T00:11:53.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.856+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c480680e0 con 0x7f5c48102560 2024-03-09T00:11:53.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.857+0000 7f5c44ff9640 1 -- 172.21.15.22:0/2238035457 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5c38010ac0 con 0x7f5c48102560 2024-03-09T00:11:53.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.858+0000 7f5c44ff9640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5c300a2970 0x7f5c300a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:53.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.858+0000 7f5c46ffd640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5c300a2970 0x7f5c300a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:53.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.858+0000 7f5c44ff9640 1 -- 172.21.15.22:0/2238035457 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f5c380cf700 con 0x7f5c48102560 2024-03-09T00:11:53.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.858+0000 7f5c46ffd640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5c300a2970 0x7f5c300a4e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0095f0 tx=0x7f5c3c005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:53.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:53.862+0000 7f5c44ff9640 1 -- 172.21.15.22:0/2238035457 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5c3809c030 con 0x7f5c48102560 2024-03-09T00:11:54.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 57525 12.33 MB/sec execute 121 sec latency 152.851 ms 2024-03-09T00:11:54.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.106+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c481a6af0 con 0x7f5c48102560 2024-03-09T00:11:54.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.107+0000 7f5c44ff9640 1 -- 172.21.15.22:0/2238035457 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f5c3809b930 con 0x7f5c48102560 2024-03-09T00:11:54.104 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:11:54.104 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:10:14.710788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:11:54.105 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-03-09T00:11:54.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5c300a2970 msgr2=0x7f5c300a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5c300a2970 0x7f5c300a4e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0095f0 tx=0x7f5c3c005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:54.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c48102560 msgr2=0x7f5c48199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c48102560 0x7f5c48199d50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9f0 tx=0x7f5c3800eeb0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 shutdown_connections 2024-03-09T00:11:54.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5c300a2970 0x7f5c300a4e30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c4810b930 0x7f5c4819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c48102e80 0x7f5c4819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 --2- 172.21.15.22:0/2238035457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c48102560 0x7f5c48199d50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 >> 172.21.15.22:0/2238035457 conn(0x7f5c480772f0 msgr2=0x7f5c48107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 shutdown_connections 2024-03-09T00:11:54.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.109+0000 7f5c4e243640 1 -- 172.21.15.22:0/2238035457 wait complete. 2024-03-09T00:11:54.117 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:11:54.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 -- 172.21.15.22:0/1133669638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 msgr2=0x7f54d011cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 --2- 172.21.15.22:0/1133669638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 0x7f54d011cec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54c801acb0 tx=0x7f54c803d170 comp rx=0 tx=0).stop 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 -- 172.21.15.22:0/1133669638 shutdown_connections 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 --2- 172.21.15.22:0/1133669638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 0x7f54d011cec0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 --2- 172.21.15.22:0/1133669638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fe80 0x7f54d010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 --2- 172.21.15.22:0/1133669638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d010fa10 0x7f54d010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 -- 172.21.15.22:0/1133669638 >> 172.21.15.22:0/1133669638 conn(0x7f54d006c3e0 msgr2=0x7f54d006c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 -- 172.21.15.22:0/1133669638 shutdown_connections 2024-03-09T00:11:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.260+0000 7f54d5c32640 1 -- 172.21.15.22:0/1133669638 wait complete. 2024-03-09T00:11:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.261+0000 7f54d5c32640 1 Processor -- start 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54d5c32640 1 -- start start 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54d5c32640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fa10 0x7f54d011a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54d5c32640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d010fe80 0x7f54d01163f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54d5c32640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 0x7f54d0116930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54d5c32640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0112d50 con 0x7f54d010fa10 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54d5c32640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0112bd0 con 0x7f54d011aad0 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54d5c32640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0112ed0 con 0x7f54d010fe80 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.262+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fa10 0x7f54d011a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fa10 0x7f54d011a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47506/0 (socket says 172.21.15.22:47506) 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cf7fe640 1 -- 172.21.15.22:0/2383233249 learned_addr learned my addr 172.21.15.22:0/2383233249 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cffff640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 0x7f54d0116930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54ceffd640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d010fe80 0x7f54d01163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cffff640 1 -- 172.21.15.22:0/2383233249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d010fe80 msgr2=0x7f54d01163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cffff640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d010fe80 0x7f54d01163f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cffff640 1 -- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fa10 msgr2=0x7f54d011a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cffff640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fa10 0x7f54d011a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cffff640 1 -- 172.21.15.22:0/2383233249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d01171a0 con 0x7f54d011aad0 2024-03-09T00:11:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.263+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fa10 0x7f54d011a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.264+0000 7f54cffff640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 0x7f54d0116930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f54c8048040 tx=0x7f54c8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.264+0000 7f54ccff9640 1 -- 172.21.15.22:0/2383233249 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8052020 con 0x7f54d011aad0 2024-03-09T00:11:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.264+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d01aa870 con 0x7f54d011aad0 2024-03-09T00:11:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.264+0000 7f54ccff9640 1 -- 172.21.15.22:0/2383233249 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54c8003c90 con 0x7f54d011aad0 2024-03-09T00:11:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.264+0000 7f54ccff9640 1 -- 172.21.15.22:0/2383233249 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c804a040 con 0x7f54d011aad0 2024-03-09T00:11:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.264+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d01aad80 con 0x7f54d011aad0 2024-03-09T00:11:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.267+0000 7f54d880e740 1 Processor -- start 2024-03-09T00:11:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54ccff9640 1 -- 172.21.15.22:0/2383233249 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f54c8055020 con 0x7f54d011aad0 2024-03-09T00:11:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54d880e740 1 -- start start 2024-03-09T00:11:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54d880e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x555967296580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54d880e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x55596729f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54d880e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54d880e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596725d360 con 0x555967296ac0 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54d880e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559671f9ec0 con 0x5559672961a0 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54d880e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559671a2d30 con 0x55596729f6a0 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x55596729f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x55596729f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47508/0 (socket says 172.21.15.22:47508) 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54cf7fe640 1 -- 172.21.15.22:0/1234286641 learned_addr learned my addr 172.21.15.22:0/1234286641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.269+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x555967296580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54ccff9640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54b80a2ab0 0x7f54b80a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54ceffd640 1 -- 172.21.15.22:0/1234286641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 msgr2=0x5559672a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54ceffd640 1 -- 172.21.15.22:0/1234286641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 msgr2=0x55596729f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x55596729f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54ceffd640 1 -- 172.21.15.22:0/1234286641 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559671a1bc0 con 0x5559672961a0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.270+0000 7f54cffff640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54b80a2ab0 0x7f54b80a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.271+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x555967296580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f54d0117b60 tx=0x7f54c0006f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6a4fd8fd9874b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.271+0000 7f54b5ffb640 1 -- 172.21.15.22:0/1234286641 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c000f040 con 0x5559672961a0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.271+0000 7f54b5ffb640 1 -- 172.21.15.22:0/1234286641 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54c0004500 con 0x5559672961a0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.271+0000 7f54b5ffb640 1 -- 172.21.15.22:0/1234286641 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0017410 con 0x5559672961a0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.271+0000 7f54ccff9640 1 -- 172.21.15.22:0/2383233249 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f54c8101ed0 con 0x7f54d011aad0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54cffff640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54b80a2ab0 0x7f54b80a4f70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f54c80cabe0 tx=0x7f54c8105610 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54d880e740 1 -- 172.21.15.22:0/1234286641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 msgr2=0x555967296580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54d880e740 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x555967296580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f54d0117b60 tx=0x7f54c0006f40 comp rx=0 tx=0).stop 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54d880e740 1 -- 172.21.15.22:0/1234286641 shutdown_connections 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54d880e740 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54d880e740 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x55596729f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54d880e740 1 --2- 172.21.15.22:0/1234286641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x555967296580 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.272+0000 7f54d880e740 1 -- 172.21.15.22:0/1234286641 >> 172.21.15.22:0/1234286641 conn(0x5559672908d0 msgr2=0x555967290cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.273+0000 7f54d880e740 1 -- 172.21.15.22:0/1234286641 shutdown_connections 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.274+0000 7f54d880e740 1 -- 172.21.15.22:0/1234286641 wait complete. 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.274+0000 7f54d880e740 1 -- start start 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.275+0000 7f54d880e740 1 Processor -- start 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54d880e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54d880e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555967296ac0 0x55596729cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54d880e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x55596729d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54d880e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559671984a0 con 0x5559672961a0 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54d880e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559671f5f60 con 0x555967296ac0 2024-03-09T00:11:54.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54d880e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559672a2430 con 0x55596729f6a0 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54d880e740 10 client.0 _ll_register_callbacks cb 0x5559671f3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54ceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54ceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47518/0 (socket says 172.21.15.22:47518) 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54ceffd640 1 -- 172.21.15.22:0/1511210993 learned_addr learned my addr 172.21.15.22:0/1511210993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x55596729d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54ceffd640 1 -- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 msgr2=0x55596729d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x55596729d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54ceffd640 1 -- 172.21.15.22:0/1511210993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555967296ac0 msgr2=0x55596729cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.276+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555967296ac0 0x55596729cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.277+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555967296ac0 0x55596729cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.277+0000 7f54ceffd640 1 -- 172.21.15.22:0/1511210993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555966f0e3f0 con 0x5559672961a0 2024-03-09T00:11:54.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.278+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729c710 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f54c0015460 tx=0x7f54c000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1745b70266cd87bf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.278+0000 7f54aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:54.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54aeffd640 1 -- 172.21.15.22:0/1511210993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0003fe0 con 0x5559672961a0 2024-03-09T00:11:54.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54aeffd640 1 -- 172.21.15.22:0/1511210993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f54c0004180 con 0x5559672961a0 2024-03-09T00:11:54.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54c0004500 con 0x5559672961a0 2024-03-09T00:11:54.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54aeffd640 1 -- 172.21.15.22:0/1511210993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54c0034050 con 0x5559672961a0 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54d880e740 10 client.20835 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555966fd87d0 con 0x5559672961a0 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54aeffd640 1 -- 172.21.15.22:0/1511210993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54c0004500 con 0x5559672961a0 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.279+0000 7f54aeffd640 1 -- 172.21.15.22:0/1511210993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f54c0015040 con 0x5559672961a0 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.280+0000 7f54d880e740 10 client.20835 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.280+0000 7f54d880e740 10 client.20835 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.280+0000 7f54d880e740 20 client.20835 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.280+0000 7f54d880e740 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5559672afda0 0x5559672b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.280+0000 7f54d880e740 4 client.20835 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.280+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555967207400 con 0x5559672afda0 2024-03-09T00:11:54.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.281+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5559672afda0 0x5559672b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.283+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5559672afda0 0x5559672b2200 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8d065347009d907 server_cookie=f9fef9bb3d82f545 in_seq=0 out_seq=0 2024-03-09T00:11:54.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.283+0000 7f54aeffd640 10 client.20835 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:54.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.286+0000 7f54aeffd640 1 -- 172.21.15.22:0/1511210993 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555967207400 con 0x5559672afda0 2024-03-09T00:11:54.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.286+0000 7f54aeffd640 10 client.20835 handle_command_reply: tid=0 2024-03-09T00:11:54.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.286+0000 7f54d880e740 1 client.20835 shutdown 2024-03-09T00:11:54.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 msgr2=0x55596729c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729c710 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f54c0015460 tx=0x7f54c000e7e0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 shutdown_connections 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x5559672afda0 0x5559672b2200 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x55596729d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555967296ac0 0x55596729cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 --2- 172.21.15.22:0/1511210993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729c710 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 >> 172.21.15.22:0/1511210993 conn(0x5559672908d0 msgr2=0x555967292620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54aeffd640 0 client.20835 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.287+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 shutdown_connections 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.288+0000 7f54d880e740 1 -- 172.21.15.22:0/1511210993 wait complete. 2024-03-09T00:11:54.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.288+0000 7f54d880e740 20 client.20835 trim_cache size 0 max 16384 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 Processor -- start 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 -- start start 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555967296ac0 0x55596729cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55596729f6a0 0x5559672b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01b7340 con 0x5559672961a0 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01b4ee0 con 0x55596729f6a0 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.309+0000 7f54d880e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01b2fa0 con 0x555967296ac0 2024-03-09T00:11:54.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555967296ac0 0x55596729cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555967296ac0 0x55596729cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48524/0 (socket says 172.21.15.22:48524) 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 -- 172.21.15.22:0/2155268582 learned_addr learned my addr 172.21.15.22:0/2155268582 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54ceffd640 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54ceffd640 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55596729f6a0 0x5559672b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 -- 172.21.15.22:0/2155268582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55596729f6a0 msgr2=0x5559672b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55596729f6a0 0x5559672b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 -- 172.21.15.22:0/2155268582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 msgr2=0x55596729ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 -- 172.21.15.22:0/2155268582 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596728c190 con 0x555967296ac0 2024-03-09T00:11:54.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.310+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555967296ac0 0x55596729cfb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009e50 tx=0x7f54c4008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4ad1918ce16993 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.312+0000 7f54b57fa640 1 -- 172.21.15.22:0/2155268582 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4023070 con 0x555967296ac0 2024-03-09T00:11:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.312+0000 7f54b57fa640 1 -- 172.21.15.22:0/2155268582 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54c401c1d0 con 0x555967296ac0 2024-03-09T00:11:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.313+0000 7f54d880e740 1 -- 172.21.15.22:0/2155268582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555967296ac0 msgr2=0x55596729cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.313+0000 7f54d880e740 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555967296ac0 0x55596729cfb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009e50 tx=0x7f54c4008040 comp rx=0 tx=0).stop 2024-03-09T00:11:54.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 -- 172.21.15.22:0/2155268582 shutdown_connections 2024-03-09T00:11:54.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55596729f6a0 0x5559672b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555967296ac0 0x55596729cfb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 --2- 172.21.15.22:0/2155268582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5559672961a0 0x55596729ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 -- 172.21.15.22:0/2155268582 >> 172.21.15.22:0/2155268582 conn(0x5559672908d0 msgr2=0x555967297a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 -- 172.21.15.22:0/2155268582 shutdown_connections 2024-03-09T00:11:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 -- 172.21.15.22:0/2155268582 wait complete. 2024-03-09T00:11:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.314+0000 7f54d880e740 1 -- start start 2024-03-09T00:11:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 1 Processor -- start 2024-03-09T00:11:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x5559672a4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x5559672a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01b35a0 con 0x555967296ac0 2024-03-09T00:11:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01b4880 con 0x5559672961a0 2024-03-09T00:11:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01b3bc0 con 0x55596729f6a0 2024-03-09T00:11:54.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.315+0000 7f54d880e740 10 client.0 _ll_register_callbacks cb 0x7f54c000e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.317+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x5559672a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.317+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x5559672a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47538/0 (socket says 172.21.15.22:47538) 2024-03-09T00:11:54.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.317+0000 7f54cf7fe640 1 -- 172.21.15.22:0/3737626322 learned_addr learned my addr 172.21.15.22:0/3737626322 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.317+0000 7f54ceffd640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x5559672a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54ceffd640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54cf7fe640 1 -- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 msgr2=0x5559672a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54cf7fe640 1 -- 172.21.15.22:0/3737626322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 msgr2=0x5559672a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x5559672a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54cf7fe640 1 -- 172.21.15.22:0/3737626322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559673052e0 con 0x555967296ac0 2024-03-09T00:11:54.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54ceffd640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54cf7fe640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x5559672a4b90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f54c40091d0 tx=0x7f54c4017cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3cc657cf13de32 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.318+0000 7f54aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.319+0000 7f54aeffd640 1 -- 172.21.15.22:0/3737626322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c402c030 con 0x555967296ac0 2024-03-09T00:11:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.319+0000 7f54aeffd640 1 -- 172.21.15.22:0/3737626322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f54c4015b00 con 0x555967296ac0 2024-03-09T00:11:54.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.319+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54c401c1d0 con 0x555967296ac0 2024-03-09T00:11:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.319+0000 7f54aeffd640 1 -- 172.21.15.22:0/3737626322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54c4023040 con 0x555967296ac0 2024-03-09T00:11:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.319+0000 7f54d880e740 10 client.20841 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:54.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.319+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555967306580 con 0x555967296ac0 2024-03-09T00:11:54.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54aeffd640 1 -- 172.21.15.22:0/3737626322 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54c401c1d0 con 0x555967296ac0 2024-03-09T00:11:54.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54aeffd640 1 -- 172.21.15.22:0/3737626322 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f54c4028920 con 0x555967296ac0 2024-03-09T00:11:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54d880e740 10 client.20841 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54d880e740 10 client.20841 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:11:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54d880e740 20 client.20841 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54d880e740 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555967306c20 0x555967309040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54d880e740 4 client.20841 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:11:54.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.324+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555967201750 con 0x555967306c20 2024-03-09T00:11:54.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.325+0000 7f54ceffd640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555967306c20 0x555967309040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.328+0000 7f54ceffd640 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555967306c20 0x555967309040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89500ddba83e2fb server_cookie=5ecc5807f81ef226 in_seq=0 out_seq=0 2024-03-09T00:11:54.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.328+0000 7f54aeffd640 10 client.20841 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:54.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.328+0000 7f54aeffd640 1 -- 172.21.15.22:0/3737626322 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555967201750 con 0x555967306c20 2024-03-09T00:11:54.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.328+0000 7f54aeffd640 10 client.20841 handle_command_reply: tid=0 2024-03-09T00:11:54.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.328+0000 7f54d880e740 1 client.20841 shutdown 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 msgr2=0x5559672a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x5559672a4b90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f54c40091d0 tx=0x7f54c4017cd0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 shutdown_connections 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x555967306c20 0x555967309040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596729f6a0 0x5559672a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555967296ac0 0x5559672a4b90 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 --2- 172.21.15.22:0/3737626322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559672961a0 0x5559672a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 >> 172.21.15.22:0/3737626322 conn(0x5559672908d0 msgr2=0x5559672b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54aeffd640 0 client.20841 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:11:54.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 shutdown_connections 2024-03-09T00:11:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.329+0000 7f54d880e740 1 -- 172.21.15.22:0/3737626322 wait complete. 2024-03-09T00:11:54.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.330+0000 7f54d880e740 20 client.20841 trim_cache size 0 max 16384 2024-03-09T00:11:54.327 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:11:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.331+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54b80a2ab0 msgr2=0x7f54b80a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.331+0000 7f54d5c32640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54b80a2ab0 0x7f54b80a4f70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f54c80cabe0 tx=0x7f54c8105610 comp rx=0 tx=0).stop 2024-03-09T00:11:54.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.331+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 msgr2=0x7f54d0116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.331+0000 7f54d5c32640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 0x7f54d0116930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f54c8048040 tx=0x7f54c8003ba0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.332+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 shutdown_connections 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.332+0000 7f54d5c32640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54b80a2ab0 0x7f54b80a4f70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.332+0000 7f54d5c32640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54d011aad0 0x7f54d0116930 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.332+0000 7f54d5c32640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d010fe80 0x7f54d01163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.332+0000 7f54d5c32640 1 --2- 172.21.15.22:0/2383233249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d010fa10 0x7f54d011a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.332+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 >> 172.21.15.22:0/2383233249 conn(0x7f54d006c3e0 msgr2=0x7f54d011b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.333+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 shutdown_connections 2024-03-09T00:11:54.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.333+0000 7f54d5c32640 1 -- 172.21.15.22:0/2383233249 wait complete. 2024-03-09T00:11:54.344 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:11:54.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:11:54.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:54 smithi022 ceph-mon[31894]: pgmap v552: 129 pgs: 129 active+clean; 389 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-03-09T00:11:54.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:54 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2238035457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 -- 172.21.15.22:0/2216081589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e01065a0 msgr2=0x7f18e010e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 --2- 172.21.15.22:0/2216081589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e01065a0 0x7f18e010e170 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009ca0 tx=0x7f18d802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 -- 172.21.15.22:0/2216081589 shutdown_connections 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 --2- 172.21.15.22:0/2216081589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e01065a0 0x7f18e010e170 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 --2- 172.21.15.22:0/2216081589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e0101e80 0x7f18e0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 --2- 172.21.15.22:0/2216081589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18e01014d0 0x7f18e01018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 -- 172.21.15.22:0/2216081589 >> 172.21.15.22:0/2216081589 conn(0x7f18e00fcd70 msgr2=0x7f18e00ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 -- 172.21.15.22:0/2216081589 shutdown_connections 2024-03-09T00:11:54.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.466+0000 7f18e8374640 1 -- 172.21.15.22:0/2216081589 wait complete. 2024-03-09T00:11:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 Processor -- start 2024-03-09T00:11:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 -- start start 2024-03-09T00:11:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e01014d0 0x7f18e00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18e01065a0 0x7f18e0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e0074070 0x7f18e01a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e0111640 con 0x7f18e0074070 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01114c0 con 0x7f18e01014d0 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e8374640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01117c0 con 0x7f18e01065a0 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e6911640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e0074070 0x7f18e01a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e6110640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e01014d0 0x7f18e00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e590f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18e01065a0 0x7f18e0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e6911640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e0074070 0x7f18e01a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47564/0 (socket says 172.21.15.22:47564) 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.467+0000 7f18e6911640 1 -- 172.21.15.22:0/3420423685 learned_addr learned my addr 172.21.15.22:0/3420423685 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e6911640 1 -- 172.21.15.22:0/3420423685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18e01065a0 msgr2=0x7f18e0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e6911640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18e01065a0 0x7f18e0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e6911640 1 -- 172.21.15.22:0/3420423685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e01014d0 msgr2=0x7f18e00735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e6911640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e01014d0 0x7f18e00735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e6911640 1 -- 172.21.15.22:0/3420423685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e01a6410 con 0x7f18e0074070 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e6110640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e01014d0 0x7f18e00735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e6911640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e0074070 0x7f18e01a5d70 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f18d802c700 tx=0x7f18d8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18d77fe640 1 -- 172.21.15.22:0/3420423685 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8041020 con 0x7f18e0074070 2024-03-09T00:11:54.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18d77fe640 1 -- 172.21.15.22:0/3420423685 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18d8034cd0 con 0x7f18e0074070 2024-03-09T00:11:54.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e01a66a0 con 0x7f18e0074070 2024-03-09T00:11:54.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18d77fe640 1 -- 172.21.15.22:0/3420423685 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8039040 con 0x7f18e0074070 2024-03-09T00:11:54.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.468+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e01a6b90 con 0x7f18e0074070 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.470+0000 7f18d77fe640 1 -- 172.21.15.22:0/3420423685 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f18d8044020 con 0x7f18e0074070 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 Processor -- start 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 -- start start 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a269e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2a2665360 con 0x55c2a269e1a0 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2a2601ec0 con 0x55c2a26a76a0 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18eb02d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2a25aad30 con 0x55c2a269eac0 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.471+0000 7f18d77fe640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18cc0a2970 0x7f18cc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e590f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a269e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6110640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e590f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a269e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47570/0 (socket says 172.21.15.22:47570) 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e590f640 1 -- 172.21.15.22:0/345770955 learned_addr learned my addr 172.21.15.22:0/345770955 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6911640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18cc0a2970 0x7f18cc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e590f640 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18d77fe640 1 -- 172.21.15.22:0/3420423685 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f18d80f0f40 con 0x7f18e0074070 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6110640 1 -- 172.21.15.22:0/345770955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 msgr2=0x55c2a26a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6110640 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6110640 1 -- 172.21.15.22:0/345770955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 msgr2=0x55c2a269e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6110640 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a269e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6110640 1 -- 172.21.15.22:0/345770955 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2a25a9bc0 con 0x55c2a269eac0 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6110640 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a7160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005fd0 tx=0x7f18dc00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773b04edaa96f958 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18e6911640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18cc0a2970 0x7f18cc0a4e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f18d80bd270 tx=0x7f18d80884f0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18d4bfe640 1 -- 172.21.15.22:0/345770955 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc011700 con 0x55c2a269eac0 2024-03-09T00:11:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18d4bfe640 1 -- 172.21.15.22:0/345770955 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc011d80 con 0x55c2a269eac0 2024-03-09T00:11:54.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.472+0000 7f18d4bfe640 1 -- 172.21.15.22:0/345770955 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc01ce60 con 0x55c2a269eac0 2024-03-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 -- 172.21.15.22:0/345770955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 msgr2=0x55c2a26a7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a7160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f18dc005fd0 tx=0x7f18dc00bf90 comp rx=0 tx=0).stop 2024-03-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 -- 172.21.15.22:0/345770955 shutdown_connections 2024-03-09T00:11:54.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a7160 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 --2- 172.21.15.22:0/345770955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a269e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 -- 172.21.15.22:0/345770955 >> 172.21.15.22:0/345770955 conn(0x55c2a26988d0 msgr2=0x55c2a2698cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 -- 172.21.15.22:0/345770955 shutdown_connections 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.473+0000 7f18eb02d740 1 -- 172.21.15.22:0/345770955 wait complete. 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 -- start start 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 Processor -- start 2024-03-09T00:11:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2a25a04a0 con 0x55c2a269e1a0 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2a25fdf60 con 0x55c2a26a76a0 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2a26aa490 con 0x55c2a269eac0 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18eb02d740 10 client.0 _ll_register_callbacks cb 0x55c2a25fbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47574/0 (socket says 172.21.15.22:47574) 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e6110640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 -- 172.21.15.22:0/2368360969 learned_addr learned my addr 172.21.15.22:0/2368360969 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 -- 172.21.15.22:0/2368360969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 msgr2=0x55c2a26a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 -- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 msgr2=0x55c2a26a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.474+0000 7f18e590f640 1 -- 172.21.15.22:0/2368360969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2a23163f0 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18e590f640 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26a46e0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009fd0 tx=0x7f18d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e875d23d7edba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2368360969 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0004050 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2368360969 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f18d00041f0 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18dc011d80 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2368360969 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18d0029050 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 10 client.20853 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c2a23e07d0 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2368360969 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc011d80 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2368360969 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f18d000e3b0 con 0x55c2a269e1a0 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 10 client.20853 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 10 client.20853 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:11:54.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 20 client.20853 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a26b7890 0x55c2a26b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 4 client.20853 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.475+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2a260f400 con 0x55c2a26b7890 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.476+0000 7f18e6110640 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a26b7890 0x55c2a26b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.476+0000 7f18e6110640 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a26b7890 0x55c2a26b9cd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e0ef484777434f0 server_cookie=68436997a286bf7c in_seq=0 out_seq=0 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.476+0000 7f18c9ffb640 10 client.20853 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.476+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2368360969 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2a260f400 con 0x55c2a26b7890 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.476+0000 7f18c9ffb640 10 client.20853 handle_command_reply: tid=0 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.476+0000 7f18eb02d740 1 client.20853 shutdown 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 msgr2=0x55c2a26a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26a46e0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009fd0 tx=0x7f18d0005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 shutdown_connections 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a26b7890 0x55c2a26b9cd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2368360969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26a46e0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 >> 172.21.15.22:0/2368360969 conn(0x55c2a26988d0 msgr2=0x55c2a269a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18c9ffb640 0 client.20853 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 shutdown_connections 2024-03-09T00:11:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 1 -- 172.21.15.22:0/2368360969 wait complete. 2024-03-09T00:11:54.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.477+0000 7f18eb02d740 20 client.20853 trim_cache size 0 max 16384 2024-03-09T00:11:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 Processor -- start 2024-03-09T00:11:54.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 -- start start 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269e1a0 0x55c2a26a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269eac0 0x55c2a26ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01b2c40 con 0x55c2a269eac0 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01b07e0 con 0x55c2a26a76a0 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18eb02d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01ae8a0 con 0x55c2a269e1a0 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18e6110640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269eac0 0x55c2a26ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18e6110640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269eac0 0x55c2a26ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47588/0 (socket says 172.21.15.22:47588) 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18e6110640 1 -- 172.21.15.22:0/3744626421 learned_addr learned my addr 172.21.15.22:0/3744626421 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.490+0000 7f18e590f640 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269e1a0 0x55c2a26a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e590f640 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e6110640 1 -- 172.21.15.22:0/3744626421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269e1a0 msgr2=0x55c2a26a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e6110640 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269e1a0 0x55c2a26a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e6110640 1 -- 172.21.15.22:0/3744626421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 msgr2=0x55c2a26aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e6110640 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e6110640 1 -- 172.21.15.22:0/3744626421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2a269e8f0 con 0x55c2a269eac0 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e590f640 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18e6110640 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269eac0 0x55c2a26ae6f0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x55c2a269dc70 tx=0x7f18dc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae958912fc87901 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18cbfff640 1 -- 172.21.15.22:0/3744626421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc01c680 con 0x55c2a269eac0 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18cbfff640 1 -- 172.21.15.22:0/3744626421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc005040 con 0x55c2a269eac0 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.491+0000 7f18cbfff640 1 -- 172.21.15.22:0/3744626421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0048c0 con 0x55c2a269eac0 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- 172.21.15.22:0/3744626421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269eac0 msgr2=0x55c2a26ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269eac0 0x55c2a26ae6f0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x55c2a269dc70 tx=0x7f18dc01c940 comp rx=0 tx=0).stop 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- 172.21.15.22:0/3744626421 shutdown_connections 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a26aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269eac0 0x55c2a26ae6f0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 --2- 172.21.15.22:0/3744626421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269e1a0 0x55c2a26a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- 172.21.15.22:0/3744626421 >> 172.21.15.22:0/3744626421 conn(0x55c2a26988d0 msgr2=0x55c2a269eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- 172.21.15.22:0/3744626421 shutdown_connections 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- 172.21.15.22:0/3744626421 wait complete. 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- start start 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 Processor -- start 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a270cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01aeea0 con 0x55c2a269e1a0 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01b0180 con 0x55c2a26a76a0 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01af4c0 con 0x55c2a269eac0 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.492+0000 7f18eb02d740 10 client.0 _ll_register_callbacks cb 0x7f18dc00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e6110640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47590/0 (socket says 172.21.15.22:47590) 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 -- 172.21.15.22:0/2780769191 learned_addr learned my addr 172.21.15.22:0/2780769191 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a270cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 -- 172.21.15.22:0/2780769191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 msgr2=0x55c2a26ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 -- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 msgr2=0x55c2a270cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a270cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 -- 172.21.15.22:0/2780769191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2a270d3c0 con 0x55c2a269e1a0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18e590f640 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26ac8d0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f18d0013040 tx=0x7f18d0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273e9f6163a9a23d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2780769191 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000e940 con 0x55c2a269e1a0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2780769191 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f18d001c040 con 0x55c2a269e1a0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18dc005040 con 0x55c2a269e1a0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2780769191 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18d003d050 con 0x55c2a269e1a0 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18eb02d740 10 client.20859 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:54.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.493+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c2a270e6c0 con 0x55c2a269e1a0 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2780769191 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc005040 con 0x55c2a269e1a0 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2780769191 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f18d0003710 con 0x55c2a269e1a0 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18eb02d740 10 client.20859 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18eb02d740 10 client.20859 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18eb02d740 20 client.20859 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a270ed00 0x55c2a2711120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18eb02d740 4 client.20859 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c2a2609750 con 0x55c2a270ed00 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18e6110640 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a270ed00 0x55c2a2711120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18c9ffb640 10 client.20859 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18e6110640 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a270ed00 0x55c2a2711120 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d256d7cd3da65bf server_cookie=b7f627e4a3948116 in_seq=0 out_seq=0 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18c9ffb640 1 -- 172.21.15.22:0/2780769191 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c2a2609750 con 0x55c2a270ed00 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18c9ffb640 10 client.20859 handle_command_reply: tid=0 2024-03-09T00:11:54.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.494+0000 7f18eb02d740 1 client.20859 shutdown 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 msgr2=0x55c2a26ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26ac8d0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f18d0013040 tx=0x7f18d0010860 comp rx=0 tx=0).stop 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 shutdown_connections 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x55c2a270ed00 0x55c2a2711120 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2a26a76a0 0x55c2a270cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c2a269eac0 0x55c2a26ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18c9ffb640 0 client.20859 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 --2- 172.21.15.22:0/2780769191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2a269e1a0 0x55c2a26ac8d0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 >> 172.21.15.22:0/2780769191 conn(0x55c2a26988d0 msgr2=0x55c2a269f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 shutdown_connections 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 1 -- 172.21.15.22:0/2780769191 wait complete. 2024-03-09T00:11:54.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18eb02d740 20 client.20859 trim_cache size 0 max 16384 2024-03-09T00:11:54.499 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18cc0a2970 msgr2=0x7f18cc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18cc0a2970 0x7f18cc0a4e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f18d80bd270 tx=0x7f18d80884f0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e0074070 msgr2=0x7f18e01a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e0074070 0x7f18e01a5d70 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f18d802c700 tx=0x7f18d8032f20 comp rx=0 tx=0).stop 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 shutdown_connections 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f18cc0a2970 0x7f18cc0a4e30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18e0074070 0x7f18e01a5d70 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18e01065a0 0x7f18e0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 --2- 172.21.15.22:0/3420423685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e01014d0 0x7f18e00735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 >> 172.21.15.22:0/3420423685 conn(0x7f18e00fcd70 msgr2=0x7f18e00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 shutdown_connections 2024-03-09T00:11:54.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.495+0000 7f18e8374640 1 -- 172.21.15.22:0/3420423685 wait complete. 2024-03-09T00:11:54.504 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:11:54.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:11:54.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:54 smithi062 ceph-mon[36129]: pgmap v552: 129 pgs: 129 active+clean; 389 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-03-09T00:11:54.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:54 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2238035457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:11:54.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:54 smithi192 ceph-mon[35995]: pgmap v552: 129 pgs: 129 active+clean; 389 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-03-09T00:11:54.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:54 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2238035457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:11:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 -- 172.21.15.22:0/3124615814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992806a300 msgr2=0x7f992810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 --2- 172.21.15.22:0/3124615814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992806a300 0x7f992810d750 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f991c009900 tx=0x7f991c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 -- 172.21.15.22:0/3124615814 shutdown_connections 2024-03-09T00:11:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 --2- 172.21.15.22:0/3124615814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992810dc90 0x7f9928110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 --2- 172.21.15.22:0/3124615814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992806a300 0x7f992810d750 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 --2- 172.21.15.22:0/3124615814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 0x7f9928069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 -- 172.21.15.22:0/3124615814 >> 172.21.15.22:0/3124615814 conn(0x7f9928076570 msgr2=0x7f9928076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 -- 172.21.15.22:0/3124615814 shutdown_connections 2024-03-09T00:11:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.655+0000 7f992de68640 1 -- 172.21.15.22:0/3124615814 wait complete. 2024-03-09T00:11:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.656+0000 7f992de68640 1 Processor -- start 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.656+0000 7f992de68640 1 -- start start 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.656+0000 7f992de68640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 0x7f9928199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.656+0000 7f992de68640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992806a300 0x7f992819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.656+0000 7f992de68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992810dc90 0x7f992819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f992de68640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928113890 con 0x7f992810dc90 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f992de68640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928113710 con 0x7f992806a300 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f992de68640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928113a10 con 0x7f99280699e0 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f99277fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 0x7f9928199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f99277fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 0x7f9928199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48588/0 (socket says 172.21.15.22:48588) 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f99277fe640 1 -- 172.21.15.22:0/1682512343 learned_addr learned my addr 172.21.15.22:0/1682512343 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9927fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992810dc90 0x7f992819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9926ffd640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992806a300 0x7f992819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9927fff640 1 -- 172.21.15.22:0/1682512343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 msgr2=0x7f9928199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9927fff640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 0x7f9928199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9927fff640 1 -- 172.21.15.22:0/1682512343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992806a300 msgr2=0x7f992819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9927fff640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992806a300 0x7f992819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9927fff640 1 -- 172.21.15.22:0/1682512343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992819ed10 con 0x7f992810dc90 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f99277fe640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 0x7f9928199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9927fff640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992810dc90 0x7f992819e5e0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f9920009fd0 tx=0x7f992000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9924ff9640 1 -- 172.21.15.22:0/1682512343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000ce20 con 0x7f992810dc90 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992819f000 con 0x7f992810dc90 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9924ff9640 1 -- 172.21.15.22:0/1682512343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9920005020 con 0x7f992810dc90 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f9924ff9640 1 -- 172.21.15.22:0/1682512343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920010720 con 0x7f992810dc90 2024-03-09T00:11:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.657+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99281a6890 con 0x7f992810dc90 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.659+0000 7f9924ff9640 1 -- 172.21.15.22:0/1682512343 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9920005540 con 0x7f992810dc90 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 Processor -- start 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 -- start start 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877d9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d9ac0 0x5620877e2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620877a0360 con 0x5620877e26a0 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56208773cec0 con 0x5620877d91a0 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9930a47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620876e5d30 con 0x5620877d9ac0 2024-03-09T00:11:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9924ff9640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99100a2970 0x7f99100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f99277fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d9ac0 0x5620877e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9927fff640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99100a2970 0x7f99100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f99277fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d9ac0 0x5620877e2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48590/0 (socket says 172.21.15.22:48590) 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47622/0 (socket says 172.21.15.22:47622) 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f99277fe640 1 -- 172.21.15.22:0/3162926104 learned_addr learned my addr 172.21.15.22:0/3162926104 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9924ff9640 1 -- 172.21.15.22:0/1682512343 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f99200cf8a0 con 0x7f992810dc90 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.660+0000 7f9926ffd640 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877d9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f99277fe640 1 -- 172.21.15.22:0/3162926104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 msgr2=0x5620877d9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f99277fe640 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877d9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f99277fe640 1 -- 172.21.15.22:0/3162926104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 msgr2=0x5620877e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f99277fe640 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f99277fe640 1 -- 172.21.15.22:0/3162926104 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620876e4bc0 con 0x5620877d9ac0 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f9926ffd640 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f9927fff640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99100a2970 0x7f99100a4e30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99200cccd0 tx=0x7f9920090fc0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f99277fe640 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d9ac0 0x5620877e2160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9918011630 tx=0x7f9918011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d728bc278a12071b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f990dffb640 1 -- 172.21.15.22:0/3162926104 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991801e070 con 0x5620877d9ac0 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f990dffb640 1 -- 172.21.15.22:0/3162926104 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f991800fbd0 con 0x5620877d9ac0 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.661+0000 7f990dffb640 1 -- 172.21.15.22:0/3162926104 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918019c90 con 0x5620877d9ac0 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 -- 172.21.15.22:0/3162926104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d9ac0 msgr2=0x5620877e2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d9ac0 0x5620877e2160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9918011630 tx=0x7f9918011af0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 -- 172.21.15.22:0/3162926104 shutdown_connections 2024-03-09T00:11:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d9ac0 0x5620877e2160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 --2- 172.21.15.22:0/3162926104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877d9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 -- 172.21.15.22:0/3162926104 >> 172.21.15.22:0/3162926104 conn(0x5620877d38d0 msgr2=0x5620877d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 -- 172.21.15.22:0/3162926104 shutdown_connections 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 -- 172.21.15.22:0/3162926104 wait complete. 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 -- start start 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.662+0000 7f9930a47740 1 Processor -- start 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d91a0 0x5620877df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d9ac0 0x5620877dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620876db4a0 con 0x5620877e26a0 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562087738f60 con 0x5620877d9ac0 2024-03-09T00:11:54.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620877e5490 con 0x5620877d91a0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 10 client.0 _ll_register_callbacks cb 0x562087736d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47630/0 (socket says 172.21.15.22:47630) 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 -- 172.21.15.22:0/3490058666 learned_addr learned my addr 172.21.15.22:0/3490058666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d91a0 0x5620877df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f99277fe640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d9ac0 0x5620877dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 -- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d91a0 msgr2=0x5620877df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d91a0 0x5620877df630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 -- 172.21.15.22:0/3490058666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d9ac0 msgr2=0x5620877dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d9ac0 0x5620877dfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 -- 172.21.15.22:0/3490058666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620874513f0 con 0x5620877e26a0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f99277fe640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d9ac0 0x5620877dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9926ffd640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e00b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f991c009230 tx=0x7f991c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4da95a6c1fb620 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9906ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:54.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9906ffd640 1 -- 172.21.15.22:0/3490058666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c004780 con 0x5620877e26a0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9906ffd640 1 -- 172.21.15.22:0/3490058666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f991c004920 con 0x5620877e26a0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.663+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f991800fbd0 con 0x5620877e26a0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9906ffd640 1 -- 172.21.15.22:0/3490058666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f991c004b20 con 0x5620877e26a0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 10 client.20871 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56208751b7d0 con 0x5620877e26a0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9906ffd640 1 -- 172.21.15.22:0/3490058666 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f991800fbd0 con 0x5620877e26a0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9906ffd640 1 -- 172.21.15.22:0/3490058666 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f991c003710 con 0x5620877e26a0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 10 client.20871 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 10 client.20871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 20 client.20871 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5620877f2800 0x5620877f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 4 client.20871 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56208774a400 con 0x5620877f2800 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.664+0000 7f99277fe640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5620877f2800 0x5620877f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.665+0000 7f99277fe640 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5620877f2800 0x5620877f4bf0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9b105a08317a18 server_cookie=bed1b31cf7486e4e in_seq=0 out_seq=0 2024-03-09T00:11:54.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.665+0000 7f9906ffd640 10 client.20871 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.666+0000 7f9906ffd640 1 -- 172.21.15.22:0/3490058666 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56208774a400 con 0x5620877f2800 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.666+0000 7f9906ffd640 10 client.20871 handle_command_reply: tid=0 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.666+0000 7f9930a47740 1 client.20871 shutdown 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 msgr2=0x5620877e00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e00b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f991c009230 tx=0x7f991c02bcd0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 shutdown_connections 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5620877f2800 0x5620877f4bf0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877e26a0 0x5620877e00b0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d9ac0 0x5620877dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9906ffd640 0 client.20871 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 --2- 172.21.15.22:0/3490058666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877d91a0 0x5620877df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 >> 172.21.15.22:0/3490058666 conn(0x5620877d38d0 msgr2=0x5620877d5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 shutdown_connections 2024-03-09T00:11:54.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 1 -- 172.21.15.22:0/3490058666 wait complete. 2024-03-09T00:11:54.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.667+0000 7f9930a47740 20 client.20871 trim_cache size 0 max 16384 2024-03-09T00:11:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 Processor -- start 2024-03-09T00:11:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 -- start start 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281b2c90 con 0x5620877d9ac0 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281b0830 con 0x5620877d91a0 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.679+0000 7f9930a47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281ae8f0 con 0x5620877e26a0 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877dc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47632/0 (socket says 172.21.15.22:47632) 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 -- 172.21.15.22:0/1054069408 learned_addr learned my addr 172.21.15.22:0/1054069408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f9926ffd640 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f9926ffd640 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 -- 172.21.15.22:0/1054069408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 msgr2=0x5620877dcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877dcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 -- 172.21.15.22:0/1054069408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 msgr2=0x5620877e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 -- 172.21.15.22:0/1054069408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620877f9d70 con 0x5620877d9ac0 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f99277fe640 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877dc890 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9918009250 tx=0x7f99180195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd30a1ddb2e6857d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f990d7fa640 1 -- 172.21.15.22:0/1054069408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991802c020 con 0x5620877d9ac0 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f990d7fa640 1 -- 172.21.15.22:0/1054069408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9918019de0 con 0x5620877d9ac0 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.680+0000 7f990d7fa640 1 -- 172.21.15.22:0/1054069408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991801e040 con 0x5620877d9ac0 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 -- 172.21.15.22:0/1054069408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 msgr2=0x5620877dc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877dc890 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9918009250 tx=0x7f99180195a0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 -- 172.21.15.22:0/1054069408 shutdown_connections 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877dc890 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 --2- 172.21.15.22:0/1054069408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 -- 172.21.15.22:0/1054069408 >> 172.21.15.22:0/1054069408 conn(0x5620877d38d0 msgr2=0x5620877d9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 -- 172.21.15.22:0/1054069408 shutdown_connections 2024-03-09T00:11:54.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 -- 172.21.15.22:0/1054069408 wait complete. 2024-03-09T00:11:54.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 -- start start 2024-03-09T00:11:54.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.681+0000 7f9930a47740 1 Processor -- start 2024-03-09T00:11:54.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281aeef0 con 0x5620877d9ac0 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281b01d0 con 0x5620877d91a0 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281af510 con 0x5620877e26a0 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 10 client.0 _ll_register_callbacks cb 0x7f9918009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9926ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47644/0 (socket says 172.21.15.22:47644) 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 -- 172.21.15.22:0/721789425 learned_addr learned my addr 172.21.15.22:0/721789425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9926ffd640 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 -- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 msgr2=0x5620877e8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877e8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 -- 172.21.15.22:0/721789425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 msgr2=0x5620877e9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:54.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 -- 172.21.15.22:0/721789425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620877e8b40 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f99277fe640 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877e7dd0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f991801c040 tx=0x7f9918005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3615fa5b6177ebba server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9906ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9906ffd640 1 -- 172.21.15.22:0/721789425 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991802c030 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9906ffd640 1 -- 172.21.15.22:0/721789425 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f9918028e00 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.682+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9918019de0 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9906ffd640 1 -- 172.21.15.22:0/721789425 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9918041050 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 10 client.20877 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562087849c50 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9906ffd640 1 -- 172.21.15.22:0/721789425 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9918019de0 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9906ffd640 1 -- 172.21.15.22:0/721789425 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f991800f920 con 0x5620877d9ac0 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 10 client.20877 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 10 client.20877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 20 client.20877 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:54.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56208784a0c0 0x56208784c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 4 client.20877 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:11:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562087744750 con 0x56208784a0c0 2024-03-09T00:11:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.683+0000 7f9926ffd640 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56208784a0c0 0x56208784c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.684+0000 7f9906ffd640 10 client.20877 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:54.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.684+0000 7f9926ffd640 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56208784a0c0 0x56208784c4b0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4dee37f96db6c32 server_cookie=20e11acd31c94b48 in_seq=0 out_seq=0 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9906ffd640 1 -- 172.21.15.22:0/721789425 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562087744750 con 0x56208784a0c0 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9906ffd640 10 client.20877 handle_command_reply: tid=0 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 client.20877 shutdown 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 msgr2=0x5620877e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877e7dd0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f991801c040 tx=0x7f9918005db0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 shutdown_connections 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x56208784a0c0 0x56208784c4b0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5620877e26a0 0x5620877e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620877d9ac0 0x5620877e7dd0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 --2- 172.21.15.22:0/721789425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620877d91a0 0x5620877e9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 >> 172.21.15.22:0/721789425 conn(0x5620877d38d0 msgr2=0x5620877ddf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9906ffd640 0 client.20877 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:54.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 shutdown_connections 2024-03-09T00:11:54.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 1 -- 172.21.15.22:0/721789425 wait complete. 2024-03-09T00:11:54.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.689+0000 7f9930a47740 20 client.20877 trim_cache size 0 max 16384 2024-03-09T00:11:54.687 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99100a2970 msgr2=0x7f99100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99100a2970 0x7f99100a4e30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99200cccd0 tx=0x7f9920090fc0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992810dc90 msgr2=0x7f992819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992810dc90 0x7f992819e5e0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f9920009fd0 tx=0x7f992000ef10 comp rx=0 tx=0).stop 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 shutdown_connections 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f99100a2970 0x7f99100a4e30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f992810dc90 0x7f992819e5e0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f992806a300 0x7f992819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 --2- 172.21.15.22:0/1682512343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99280699e0 0x7f9928199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:54.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 >> 172.21.15.22:0/1682512343 conn(0x7f9928076570 msgr2=0x7f992810f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 shutdown_connections 2024-03-09T00:11:54.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:54.690+0000 7f992de68640 1 -- 172.21.15.22:0/1682512343 wait complete. 2024-03-09T00:11:55.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 57795 12.33 MB/sec execute 122 sec latency 201.256 ms 2024-03-09T00:11:55.699 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-09T00:11:55.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:11:55.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-09T00:11:55.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3954221372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df8068760 msgr2=0x7f4df8068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3954221372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df8068760 0x7f4df8068b40 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009ca0 tx=0x7f4de802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3954221372 shutdown_connections 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3954221372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df8106630 0x7f4df810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3954221372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8069110 0x7f4df8105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3954221372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df8068760 0x7f4df8068b40 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3954221372 >> 172.21.15.22:0/3954221372 conn(0x7f4df80772f0 msgr2=0x7f4df8077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3954221372 shutdown_connections 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3954221372 wait complete. 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 Processor -- start 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.825+0000 7f4dff7cd640 1 -- start start 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8068760 0x7f4df819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df8106630 0x7f4df819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df81979b0 0x7f4df8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df8111690 con 0x7f4df81979b0 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df8111510 con 0x7f4df8068760 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df8111810 con 0x7f4df8106630 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df81979b0 0x7f4df8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df81979b0 0x7f4df8197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47656/0 (socket says 172.21.15.22:47656) 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/3336610573 learned_addr learned my addr 172.21.15.22:0/3336610573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df8106630 0x7f4df819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfd569640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8068760 0x7f4df819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/3336610573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df8106630 msgr2=0x7f4df819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df8106630 0x7f4df819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/3336610573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8068760 msgr2=0x7f4df819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8068760 0x7f4df819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/3336610573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df81984d0 con 0x7f4df81979b0 2024-03-09T00:11:55.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df81979b0 0x7f4df8197e60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e970 tx=0x7f4df000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:55.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4df67fc640 1 -- 172.21.15.22:0/3336610573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000ccc0 con 0x7f4df81979b0 2024-03-09T00:11:55.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4df67fc640 1 -- 172.21.15.22:0/3336610573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4df000ce60 con 0x7f4df81979b0 2024-03-09T00:11:55.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4df67fc640 1 -- 172.21.15.22:0/3336610573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0010490 con 0x7f4df81979b0 2024-03-09T00:11:55.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df81987c0 con 0x7f4df81979b0 2024-03-09T00:11:55.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.826+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df81a4a10 con 0x7f4df81979b0 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4df67fc640 1 -- 172.21.15.22:0/3336610573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4df0005020 con 0x7f4df81979b0 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 Processor -- start 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 -- start start 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4df67fc640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4de00a2970 0x7f4de00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5c43fb40 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5c3e8d80 con 0x55aa5c4dbd80 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4e023af740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5c43bbe0 con 0x55aa5c4e5280 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4de00a2970 0x7f4de00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4df67fc640 1 -- 172.21.15.22:0/3336610573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f4df00cf630 con 0x7f4df81979b0 2024-03-09T00:11:55.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.829+0000 7f4dfdd6a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:48636/0 (socket says 172.21.15.22:48636) 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfcd68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/569044129 learned_addr learned my addr 172.21.15.22:0/569044129 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfd569640 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/569044129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dbd80 msgr2=0x55aa5c4dc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4dc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/569044129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 msgr2=0x55aa5c4e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 -- 172.21.15.22:0/569044129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa5c21c7e0 con 0x55aa5c4e5280 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfd569640 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4dc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4de00a2970 0x7f4de00a4e30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b620 tx=0x7f4dec00bb10 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e7670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4df0098290 tx=0x7f4df009b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477f5e0e57f3a9fb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4ddf7fe640 1 -- 172.21.15.22:0/569044129 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0078d00 con 0x55aa5c4e5280 2024-03-09T00:11:55.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4ddf7fe640 1 -- 172.21.15.22:0/569044129 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4df009bfc0 con 0x55aa5c4e5280 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.830+0000 7f4ddf7fe640 1 -- 172.21.15.22:0/569044129 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df00075f0 con 0x55aa5c4e5280 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 -- 172.21.15.22:0/569044129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 msgr2=0x55aa5c4e7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e7670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4df0098290 tx=0x7f4df009b7c0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 -- 172.21.15.22:0/569044129 shutdown_connections 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e7670 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 --2- 172.21.15.22:0/569044129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4dc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 -- 172.21.15.22:0/569044129 >> 172.21.15.22:0/569044129 conn(0x55aa5c4d64b0 msgr2=0x55aa5c4d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 -- 172.21.15.22:0/569044129 shutdown_connections 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 -- 172.21.15.22:0/569044129 wait complete. 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 -- start start 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.831+0000 7f4e023af740 1 Processor -- start 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4e023af740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4e023af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4e023af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4e023af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5c4e8070 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4e023af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5c3fc1c0 con 0x55aa5c4e5280 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4e023af740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5c4e81f0 con 0x55aa5c4dbd80 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4e023af740 10 client.0 _ll_register_callbacks cb 0x55aa5c3eb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfd569640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47680/0 (socket says 172.21.15.22:47680) 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 -- 172.21.15.22:0/1890180382 learned_addr learned my addr 172.21.15.22:0/1890180382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 -- 172.21.15.22:0/1890180382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dbd80 msgr2=0x55aa5c4e4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 -- 172.21.15.22:0/1890180382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 msgr2=0x55aa5c4e2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.832+0000 7f4dfcd68640 1 -- 172.21.15.22:0/1890180382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa5c27cf70 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e1fa0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f4dec012440 tx=0x7f4dec00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb8d1feb6e4ac66 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4ddcff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1890180382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec01eb10 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4df009bfc0 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1890180382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f4dec01ecb0 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1890180382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dec033050 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4e023af740 10 client.20889 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.833+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55aa5c114170 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1890180382 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4df009bfc0 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1890180382 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4dec01f070 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4e023af740 10 client.20889 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4e023af740 10 client.20889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:55.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4e023af740 20 client.20889 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4e023af740 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c4f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4e023af740 4 client.20889 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa5c415960 con 0x55aa5c4f52c0 2024-03-09T00:11:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4dfd569640 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c4f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4ddcff9640 10 client.20889 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.834+0000 7f4dfd569640 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c4f76b0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851606f43def3296 server_cookie=1eec22f2462492f8 in_seq=0 out_seq=0 2024-03-09T00:11:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.836+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1890180382 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa5c415960 con 0x55aa5c4f52c0 2024-03-09T00:11:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.836+0000 7f4ddcff9640 10 client.20889 handle_command_reply: tid=0 2024-03-09T00:11:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.836+0000 7f4e023af740 1 client.20889 shutdown 2024-03-09T00:11:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 msgr2=0x55aa5c4e1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e1fa0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f4dec012440 tx=0x7f4dec00b390 comp rx=0 tx=0).stop 2024-03-09T00:11:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 shutdown_connections 2024-03-09T00:11:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c4f76b0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4ddcff9640 0 client.20889 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e1fa0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 --2- 172.21.15.22:0/1890180382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 >> 172.21.15.22:0/1890180382 conn(0x55aa5c4d64b0 msgr2=0x55aa5c4d8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 shutdown_connections 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.837+0000 7f4e023af740 1 -- 172.21.15.22:0/1890180382 wait complete. 2024-03-09T00:11:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.839+0000 7f4e023af740 20 client.20889 trim_cache size 0 max 16384 2024-03-09T00:11:55.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.862+0000 7f4e023af740 1 Processor -- start 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4e023af740 1 -- start start 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4e023af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4e023af740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4e023af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4e023af740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81b0bf0 con 0x55aa5c4dbd80 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4e023af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81ae790 con 0x55aa5c4e5280 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4e023af740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81ac850 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfcd68640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47684/0 (socket says 172.21.15.22:47684) 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 -- 172.21.15.22:0/2750980286 learned_addr learned my addr 172.21.15.22:0/2750980286 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 -- 172.21.15.22:0/2750980286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dc6a0 msgr2=0x55aa5c4e46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 -- 172.21.15.22:0/2750980286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 msgr2=0x55aa5c4e4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.863+0000 7f4dfd569640 1 -- 172.21.15.22:0/2750980286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa5c4e2890 con 0x55aa5c4dbd80 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.864+0000 7f4dfdd6a640 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.864+0000 7f4dfd569640 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e75c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4de8004af0 tx=0x7f4de8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d415265da0c2598f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.864+0000 7f4ddeffd640 1 -- 172.21.15.22:0/2750980286 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8039040 con 0x55aa5c4dbd80 2024-03-09T00:11:55.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.864+0000 7f4ddeffd640 1 -- 172.21.15.22:0/2750980286 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4de8032c90 con 0x55aa5c4dbd80 2024-03-09T00:11:55.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.864+0000 7f4ddeffd640 1 -- 172.21.15.22:0/2750980286 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8003e60 con 0x55aa5c4dbd80 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.865+0000 7f4e023af740 1 -- 172.21.15.22:0/2750980286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 msgr2=0x55aa5c4e75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.865+0000 7f4e023af740 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e75c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4de8004af0 tx=0x7f4de8005c50 comp rx=0 tx=0).stop 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.865+0000 7f4e023af740 1 -- 172.21.15.22:0/2750980286 shutdown_connections 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.866+0000 7f4e023af740 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.866+0000 7f4e023af740 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.866+0000 7f4e023af740 1 --2- 172.21.15.22:0/2750980286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4e75c0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.866+0000 7f4e023af740 1 -- 172.21.15.22:0/2750980286 >> 172.21.15.22:0/2750980286 conn(0x55aa5c4d64b0 msgr2=0x55aa5c4dd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.866+0000 7f4e023af740 1 -- 172.21.15.22:0/2750980286 shutdown_connections 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.866+0000 7f4e023af740 1 -- 172.21.15.22:0/2750980286 wait complete. 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.866+0000 7f4e023af740 1 -- start start 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.867+0000 7f4e023af740 1 Processor -- start 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.867+0000 7f4e023af740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4ec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4e023af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4e023af740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4ec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47686/0 (socket says 172.21.15.22:47686) 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 -- 172.21.15.22:0/1760693528 learned_addr learned my addr 172.21.15.22:0/1760693528 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81ace50 con 0x55aa5c4dbd80 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81ae130 con 0x55aa5c4dc6a0 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81ad470 con 0x55aa5c4e5280 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4e023af740 10 client.0 _ll_register_callbacks cb 0x7f4df0005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 -- 172.21.15.22:0/1760693528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 msgr2=0x55aa5c4e9e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e9e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 -- 172.21.15.22:0/1760693528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dc6a0 msgr2=0x55aa5c4e98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.868+0000 7f4dfd569640 1 -- 172.21.15.22:0/1760693528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa5c4ea350 con 0x55aa5c4dbd80 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.869+0000 7f4dfd569640 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4ec540 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f4de8037040 tx=0x7f4de8032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4d9d654400b79 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.869+0000 7f4ddcff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.869+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1760693528 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8039040 con 0x55aa5c4dbd80 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.869+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1760693528 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f4de8003970 con 0x55aa5c4dbd80 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.869+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4de8032c90 con 0x55aa5c4dbd80 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.870+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1760693528 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4de8003b70 con 0x55aa5c4dbd80 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.870+0000 7f4e023af740 10 client.20895 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:55.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.870+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55aa5c54d4a0 con 0x55aa5c4dbd80 2024-03-09T00:11:55.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.871+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1760693528 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4de8032c90 con 0x55aa5c4dbd80 2024-03-09T00:11:55.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.871+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1760693528 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4de80044c0 con 0x55aa5c4dbd80 2024-03-09T00:11:55.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.872+0000 7f4e023af740 10 client.20895 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:55.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.872+0000 7f4e023af740 10 client.20895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:55.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.872+0000 7f4e023af740 20 client.20895 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:55.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.872+0000 7f4e023af740 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c54fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:55.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.872+0000 7f4e023af740 4 client.20895 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-09T00:11:55.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.872+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55aa5c401080 con 0x55aa5c4f52c0 2024-03-09T00:11:55.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.875+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c54fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:55.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.876+0000 7f4dfcd68640 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c54fb20 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e2d310a7931b5a server_cookie=10b75d8ca9e042e9 in_seq=0 out_seq=0 2024-03-09T00:11:55.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.876+0000 7f4ddcff9640 10 client.20895 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.881+0000 7f4ddcff9640 1 -- 172.21.15.22:0/1760693528 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55aa5c401080 con 0x55aa5c4f52c0 2024-03-09T00:11:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.881+0000 7f4ddcff9640 10 client.20895 handle_command_reply: tid=0 2024-03-09T00:11:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.881+0000 7f4e023af740 1 client.20895 shutdown 2024-03-09T00:11:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.881+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 msgr2=0x55aa5c4ec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.881+0000 7f4e023af740 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4ec540 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f4de8037040 tx=0x7f4de8032f50 comp rx=0 tx=0).stop 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 shutdown_connections 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x55aa5c4f52c0 0x55aa5c54fb20 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4ddcff9640 0 client.20895 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55aa5c4e5280 0x55aa5c4e9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa5c4dc6a0 0x55aa5c4e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 --2- 172.21.15.22:0/1760693528 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa5c4dbd80 0x55aa5c4ec540 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 >> 172.21.15.22:0/1760693528 conn(0x55aa5c4d64b0 msgr2=0x55aa5c4d7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 shutdown_connections 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 1 -- 172.21.15.22:0/1760693528 wait complete. 2024-03-09T00:11:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4e023af740 20 client.20895 trim_cache size 0 max 16384 2024-03-09T00:11:55.880 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:11:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-03-09T00:11:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "be23f2b7-c58c-4d88-901f-bd52e441e1d0", 2024-03-09T00:11:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-03-09T00:11:55.880 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4de00a2970 msgr2=0x7f4de00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4de00a2970 0x7f4de00a4e30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b620 tx=0x7f4dec00bb10 comp rx=0 tx=0).stop 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df81979b0 msgr2=0x7f4df8197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.882+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df81979b0 0x7f4df8197e60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e970 tx=0x7f4df000ed70 comp rx=0 tx=0).stop 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 shutdown_connections 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f4de00a2970 0x7f4de00a4e30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4df81979b0 0x7f4df8197e60 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df8106630 0x7f4df819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 --2- 172.21.15.22:0/3336610573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8068760 0x7f4df819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:55.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 >> 172.21.15.22:0/3336610573 conn(0x7f4df80772f0 msgr2=0x7f4df80fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:55.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 shutdown_connections 2024-03-09T00:11:55.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:55.883+0000 7f4dff7cd640 1 -- 172.21.15.22:0/3336610573 wait complete. 2024-03-09T00:11:55.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:11:55.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 -- 172.21.15.22:0/757438950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d4101e90 msgr2=0x7f87d4105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 --2- 172.21.15.22:0/757438950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d4101e90 0x7f87d4105ed0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009900 tx=0x7f87bc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 -- 172.21.15.22:0/757438950 shutdown_connections 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 --2- 172.21.15.22:0/757438950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d41065b0 0x7f87d410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 --2- 172.21.15.22:0/757438950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d4101e90 0x7f87d4105ed0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 --2- 172.21.15.22:0/757438950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d41014e0 0x7f87d41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 -- 172.21.15.22:0/757438950 >> 172.21.15.22:0/757438950 conn(0x7f87d40fcd80 msgr2=0x7f87d40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 -- 172.21.15.22:0/757438950 shutdown_connections 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.031+0000 7f87d9560640 1 -- 172.21.15.22:0/757438950 wait complete. 2024-03-09T00:11:56.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 Processor -- start 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 -- start start 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d41014e0 0x7f87d419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d41065b0 0x7f87d419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d41979e0 0x7f87d4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4111640 con 0x7f87d41065b0 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41114c0 con 0x7f87d41979e0 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d9560640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41117c0 con 0x7f87d41014e0 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d41065b0 0x7f87d419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d41065b0 0x7f87d419dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47714/0 (socket says 172.21.15.22:47714) 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 -- 172.21.15.22:0/599884857 learned_addr learned my addr 172.21.15.22:0/599884857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d41014e0 0x7f87d419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87d37fe640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d41979e0 0x7f87d4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 -- 172.21.15.22:0/599884857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d41014e0 msgr2=0x7f87d419d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d41014e0 0x7f87d419d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 -- 172.21.15.22:0/599884857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d41979e0 msgr2=0x7f87d4197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d41979e0 0x7f87d4197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 -- 172.21.15.22:0/599884857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4198500 con 0x7f87d41065b0 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.032+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d41065b0 0x7f87d419dd70 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f87bc036040 tx=0x7f87bc02cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.033+0000 7f87d0ff9640 1 -- 172.21.15.22:0/599884857 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc041070 con 0x7f87d41065b0 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.033+0000 7f87d0ff9640 1 -- 172.21.15.22:0/599884857 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87bc004450 con 0x7f87d41065b0 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.033+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d4198790 con 0x7f87d41065b0 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.033+0000 7f87d0ff9640 1 -- 172.21.15.22:0/599884857 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc003dc0 con 0x7f87d41065b0 2024-03-09T00:11:56.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.033+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d41a49e0 con 0x7f87d41065b0 2024-03-09T00:11:56.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.035+0000 7f87d0ff9640 1 -- 172.21.15.22:0/599884857 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f87bc044070 con 0x7f87d41065b0 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.035+0000 7f87dc217740 1 Processor -- start 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.035+0000 7f87dc217740 1 -- start start 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87dc217740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87d0ff9640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87c00a29f0 0x7f87c00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87c00a29f0 0x7f87c00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87d0ff9640 1 -- 172.21.15.22:0/599884857 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f87bc0f0c80 con 0x7f87d41065b0 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2b6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33846/0 (socket says 172.21.15.22:33846) 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87dc217740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87d37fe640 1 -- 172.21.15.22:0/1659938334 learned_addr learned my addr 172.21.15.22:0/1659938334 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87dc217740 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1e219e00 con 0x558b1e2bfc60 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1e1c2b10 con 0x558b1e2b6760 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1e215ea0 con 0x558b1e2b7080 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.036+0000 7f87d37fe640 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87d37fe640 1 -- 172.21.15.22:0/1659938334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 msgr2=0x558b1e2bf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87d37fe640 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87d37fe640 1 -- 172.21.15.22:0/1659938334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 msgr2=0x558b1e2c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87d37fe640 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87d37fe640 1 -- 172.21.15.22:0/1659938334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b1dff87d0 con 0x558b1e2b6760 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87c00a29f0 0x7f87c00a4eb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f87d4198b20 tx=0x7f87c4005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87d37fe640 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2b6b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87cc012a60 tx=0x7f87cc009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6888256be2b4ca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87c97fa640 1 -- 172.21.15.22:0/1659938334 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc012110 con 0x558b1e2b6760 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87c97fa640 1 -- 172.21.15.22:0/1659938334 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87cc003e50 con 0x558b1e2b6760 2024-03-09T00:11:56.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.037+0000 7f87c97fa640 1 -- 172.21.15.22:0/1659938334 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc004030 con 0x558b1e2b6760 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 msgr2=0x558b1e2b6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2b6b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87cc012a60 tx=0x7f87cc009b20 comp rx=0 tx=0).stop 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 shutdown_connections 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 --2- 172.21.15.22:0/1659938334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2b6b40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 >> 172.21.15.22:0/1659938334 conn(0x558b1e2b0e90 msgr2=0x558b1e2b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 shutdown_connections 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 -- 172.21.15.22:0/1659938334 wait complete. 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 -- start start 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.038+0000 7f87dc217740 1 Processor -- start 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87dc217740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2bf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87dc217740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87dc217740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87dc217740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1e2c6e60 con 0x558b1e2bfc60 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87dc217740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1e1d60a0 con 0x558b1e2b6760 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87dc217740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1e2c6fe0 con 0x558b1e2b7080 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2bf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33850/0 (socket says 172.21.15.22:33850) 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87dc217740 10 client.0 _ll_register_callbacks cb 0x558b1e1d5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d2ffd640 1 -- 172.21.15.22:0/889496701 learned_addr learned my addr 172.21.15.22:0/889496701 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d37fe640 1 -- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 msgr2=0x558b1e2bc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d37fe640 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d37fe640 1 -- 172.21.15.22:0/889496701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 msgr2=0x558b1e2bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d37fe640 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2bf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d37fe640 1 -- 172.21.15.22:0/889496701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b1df2e3f0 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87d37fe640 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2bcc50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f87cc012a30 tx=0x7f87cc00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554540ba96258379 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.039+0000 7f87ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87ba7fc640 1 -- 172.21.15.22:0/889496701 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc034060 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87ba7fc640 1 -- 172.21.15.22:0/889496701 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f87cc003b70 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87cc003e50 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87ba7fc640 1 -- 172.21.15.22:0/889496701 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87cc03b050 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 10 client.20907 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558b1deee0f0 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87ba7fc640 1 -- 172.21.15.22:0/889496701 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87cc003e50 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87ba7fc640 1 -- 172.21.15.22:0/889496701 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87cc0105e0 con 0x558b1e2bfc60 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 10 client.20907 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 10 client.20907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 20 client.20907 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:56.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e2c39c0 0x558b1e2c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 4 client.20907 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:11:56.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b1e227340 con 0x558b1e2c39c0 2024-03-09T00:11:56.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.040+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e2c39c0 0x558b1e2c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.041+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e2c39c0 0x558b1e2c5db0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94be632bebca92d6 server_cookie=6ab58b663ed1332f in_seq=0 out_seq=0 2024-03-09T00:11:56.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.041+0000 7f87ba7fc640 10 client.20907 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:56.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87ba7fc640 1 -- 172.21.15.22:0/889496701 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b1e227340 con 0x558b1e2c39c0 2024-03-09T00:11:56.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87ba7fc640 10 client.20907 handle_command_reply: tid=0 2024-03-09T00:11:56.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 client.20907 shutdown 2024-03-09T00:11:56.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 msgr2=0x558b1e2bcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2bcc50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f87cc012a30 tx=0x7f87cc00eed0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 shutdown_connections 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e2c39c0 0x558b1e2c5db0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2bfc60 0x558b1e2bcc50 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2bc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 --2- 172.21.15.22:0/889496701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b6760 0x558b1e2bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 >> 172.21.15.22:0/889496701 conn(0x558b1e2b0e90 msgr2=0x558b1e2b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87ba7fc640 0 client.20907 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 shutdown_connections 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 1 -- 172.21.15.22:0/889496701 wait complete. 2024-03-09T00:11:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.047+0000 7f87dc217740 20 client.20907 trim_cache size 0 max 16384 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 Processor -- start 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 -- start start 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b7080 0x558b1e2b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2bfc60 0x558b1e2bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41ae9d0 con 0x558b1e2b6760 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41ac570 con 0x558b1e2b7080 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.062+0000 7f87dc217740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41aa630 con 0x558b1e2bfc60 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87ca7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b7080 0x558b1e2b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2bbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47746/0 (socket says 172.21.15.22:47746) 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87ca7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b7080 0x558b1e2b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:33856/0 (socket says 172.21.15.22:33856) 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 -- 172.21.15.22:0/1657963947 learned_addr learned my addr 172.21.15.22:0/1657963947 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d37fe640 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2bfc60 0x558b1e2bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 -- 172.21.15.22:0/1657963947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2bfc60 msgr2=0x558b1e2bee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2bfc60 0x558b1e2bee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 -- 172.21.15.22:0/1657963947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b7080 msgr2=0x558b1e2b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b7080 0x558b1e2b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 -- 172.21.15.22:0/1657963947 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b1e2b5850 con 0x558b1e2b6760 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b7080 0x558b1e2b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2bbff0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f87c401d650 tx=0x7f87c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae28cf6739fa274 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87c8ff9640 1 -- 172.21.15.22:0/1657963947 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4028420 con 0x558b1e2b6760 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87c8ff9640 1 -- 172.21.15.22:0/1657963947 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87c4005500 con 0x558b1e2b6760 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.063+0000 7f87c8ff9640 1 -- 172.21.15.22:0/1657963947 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4004dd0 con 0x558b1e2b6760 2024-03-09T00:11:56.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 -- 172.21.15.22:0/1657963947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 msgr2=0x558b1e2bbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2bbff0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f87c401d650 tx=0x7f87c4014000 comp rx=0 tx=0).stop 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 -- 172.21.15.22:0/1657963947 shutdown_connections 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2bfc60 0x558b1e2bee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2b7080 0x558b1e2b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 --2- 172.21.15.22:0/1657963947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2bbff0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 -- 172.21.15.22:0/1657963947 >> 172.21.15.22:0/1657963947 conn(0x558b1e2b0e90 msgr2=0x558b1e2b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 -- 172.21.15.22:0/1657963947 shutdown_connections 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 -- 172.21.15.22:0/1657963947 wait complete. 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.064+0000 7f87dc217740 1 -- start start 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 1 Processor -- start 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2bfc60 0x558b1e2c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41aac30 con 0x558b1e2b6760 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41abf10 con 0x558b1e2bfc60 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41ab250 con 0x558b1e2b7080 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87dc217740 10 client.0 _ll_register_callbacks cb 0x7f87cc012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47748/0 (socket says 172.21.15.22:47748) 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 -- 172.21.15.22:0/3923564428 learned_addr learned my addr 172.21.15.22:0/3923564428 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d37fe640 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2bfc60 0x558b1e2c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 -- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 msgr2=0x558b1e2cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 -- 172.21.15.22:0/3923564428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2bfc60 msgr2=0x558b1e2c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2bfc60 0x558b1e2c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.065+0000 7f87d2ffd640 1 -- 172.21.15.22:0/3923564428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b1e2c7fa0 con 0x558b1e2b6760 2024-03-09T00:11:56.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87d2ffd640 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2cb1d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f87c4004770 tx=0x7f87c40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ba9a6fecfb6433 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87ba7fc640 1 -- 172.21.15.22:0/3923564428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4035020 con 0x558b1e2b6760 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87ba7fc640 1 -- 172.21.15.22:0/3923564428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f87c4004050 con 0x558b1e2b6760 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87c4005500 con 0x558b1e2b6760 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87ba7fc640 1 -- 172.21.15.22:0/3923564428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c4040050 con 0x558b1e2b6760 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 10 client.20913 fetch_fsmap learned FSMap version 37 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558b1e326b10 con 0x558b1e2b6760 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87ba7fc640 1 -- 172.21.15.22:0/3923564428 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87c4005500 con 0x558b1e2b6760 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87ba7fc640 1 -- 172.21.15.22:0/3923564428 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87c4003680 con 0x558b1e2b6760 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 10 client.20913 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 10 client.20913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 20 client.20913 populate_metadata read hostname 'smithi022' 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e327180 0x558b1e3295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 4 client.20913 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:11:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.066+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b1e221690 con 0x558b1e327180 2024-03-09T00:11:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.067+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e327180 0x558b1e3295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.067+0000 7f87ca7fc640 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e327180 0x558b1e3295a0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1677b64fc0e5fbd3 server_cookie=e0b2e314fded1bb1 in_seq=0 out_seq=0 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.067+0000 7f87ba7fc640 10 client.20913 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87ba7fc640 1 -- 172.21.15.22:0/3923564428 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x558b1e221690 con 0x558b1e327180 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87ba7fc640 10 client.20913 handle_command_reply: tid=0 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 client.20913 shutdown 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 msgr2=0x558b1e2cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2cb1d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f87c4004770 tx=0x7f87c40047a0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 shutdown_connections 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x558b1e327180 0x558b1e3295a0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b1e2bfc60 0x558b1e2c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558b1e2b7080 0x558b1e2cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87ba7fc640 0 client.20913 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 --2- 172.21.15.22:0/3923564428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558b1e2b6760 0x558b1e2cb1d0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 >> 172.21.15.22:0/3923564428 conn(0x558b1e2b0e90 msgr2=0x558b1e2b7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 shutdown_connections 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 1 -- 172.21.15.22:0/3923564428 wait complete. 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.072+0000 7f87dc217740 20 client.20913 trim_cache size 0 max 16384 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (627 inodes in the stack)", 2024-03-09T00:11:56.070 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-03-09T00:11:56.071 INFO:teuthology.orchestra.run.smithi022.stdout: "be23f2b7-c58c-4d88-901f-bd52e441e1d0": { 2024-03-09T00:11:56.071 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-03-09T00:11:56.071 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "be23f2b7-c58c-4d88-901f-bd52e441e1d0", 2024-03-09T00:11:56.071 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2024-03-09T00:11:56.071 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:11:56.071 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:11:56.071 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87c00a29f0 msgr2=0x7f87c00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87c00a29f0 0x7f87c00a4eb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f87d4198b20 tx=0x7f87c4005f90 comp rx=0 tx=0).stop 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d41065b0 msgr2=0x7f87d419dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d41065b0 0x7f87d419dd70 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f87bc036040 tx=0x7f87bc02cb20 comp rx=0 tx=0).stop 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 shutdown_connections 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f87c00a29f0 0x7f87c00a4eb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87d41979e0 0x7f87d4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87d41065b0 0x7f87d419dd70 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 --2- 172.21.15.22:0/599884857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d41014e0 0x7f87d419d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 >> 172.21.15.22:0/599884857 conn(0x7f87d40fcd80 msgr2=0x7f87d40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 shutdown_connections 2024-03-09T00:11:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:11:56.073+0000 7f87d9560640 1 -- 172.21.15.22:0/599884857 wait complete. 2024-03-09T00:11:56.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 57928 12.27 MB/sec execute 123 sec latency 370.209 ms 2024-03-09T00:11:56.080 INFO:tasks.cephfs.filesystem:scrub status for tag:be23f2b7-c58c-4d88-901f-bd52e441e1d0 - {'path': '/', 'tag': 'be23f2b7-c58c-4d88-901f-bd52e441e1d0', 'options': 'recursive,force'} 2024-03-09T00:11:56.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: pgmap v553: 129 pgs: 129 active+clean; 386 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-03-09T00:11:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: scrub queued for path: / 2024-03-09T00:11:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:56 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: pgmap v553: 129 pgs: 129 active+clean; 386 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: scrub queued for path: / 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:56 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: pgmap v553: 129 pgs: 129 active+clean; 386 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-03-09T00:11:56.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: scrub queued for path: / 2024-03-09T00:11:56.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:56.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:56.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:56.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:56.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:56 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 58106 12.24 MB/sec execute 124 sec latency 299.119 ms 2024-03-09T00:11:57.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:57 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:57.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:57 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:57 smithi062 ceph-mon[36129]: scrub summary: active paths [/] 2024-03-09T00:11:57.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:57 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:57 smithi062 ceph-mon[36129]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:57 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:57 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:57 smithi192 ceph-mon[35995]: scrub summary: active paths [/] 2024-03-09T00:11:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:57 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:57 smithi192 ceph-mon[35995]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:57 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:57 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:57 smithi022 ceph-mon[31894]: scrub summary: active paths [/] 2024-03-09T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:57 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:57 smithi022 ceph-mon[31894]: scrub summary: idle+waiting paths [/] 2024-03-09T00:11:58.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 58389 12.25 MB/sec execute 125 sec latency 229.753 ms 2024-03-09T00:11:58.532 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:11:58 smithi192 ceph-mon[35995]: pgmap v554: 129 pgs: 129 active+clean; 373 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-03-09T00:11:58.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:11:58 smithi062 ceph-mon[36129]: pgmap v554: 129 pgs: 129 active+clean; 373 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-03-09T00:11:58.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:11:58 smithi022 ceph-mon[31894]: pgmap v554: 129 pgs: 129 active+clean; 373 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-03-09T00:11:59.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 58665 12.22 MB/sec execute 126 sec latency 270.495 ms 2024-03-09T00:12:00.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 59029 12.27 MB/sec execute 127 sec latency 197.284 ms 2024-03-09T00:12:00.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:00 smithi062 ceph-mon[36129]: pgmap v555: 129 pgs: 129 active+clean; 373 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-09T00:12:00.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:00 smithi192 ceph-mon[35995]: pgmap v555: 129 pgs: 129 active+clean; 373 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-09T00:12:00.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:00 smithi022 ceph-mon[31894]: pgmap v555: 129 pgs: 129 active+clean; 373 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-09T00:12:01.075 INFO:tasks.workunit.client.0.smithi022.stdout: 10 59212 12.22 MB/sec execute 128 sec latency 518.819 ms 2024-03-09T00:12:01.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:01 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:12:01.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:01 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:12:01.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:01 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:12:02.006 INFO:tasks.workunit.client.0.smithi022.stdout:[58884] write failed on handle 14062 (Resource temporarily unavailable) 2024-03-09T00:12:02.006 INFO:tasks.workunit.client.0.smithi022.stdout:Child failed with status 1 2024-03-09T00:12:02.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T00:12:02.020 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-09T00:12:02.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-09T00:12:02.299 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:02 smithi022 ceph-mon[31894]: pgmap v556: 129 pgs: 129 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.6 MiB/s wr, 636 op/s 2024-03-09T00:12:02.299 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:02 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:12:02.299 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:02 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:12:02.299 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:02 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:12:02.300 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-09T00:12:02.560 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c633aec26d6e4cef9d88206a3079132d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-09T00:12:02.563 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-09T00:12:02.580 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi022.front.sepia.ceph.com 2024-03-09T00:12:02.596 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-09T00:12:02.606 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-09T00:12:02.606 INFO:teuthology.task.exec:Executing custom commands... 2024-03-09T00:12:02.607 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-03-09T00:12:02.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-09T00:12:02.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:02 smithi062 ceph-mon[36129]: pgmap v556: 129 pgs: 129 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.6 MiB/s wr, 636 op/s 2024-03-09T00:12:02.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:02 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:12:02.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:02 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:12:02.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:02 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:12:02.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:02 smithi192 ceph-mon[35995]: pgmap v556: 129 pgs: 129 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.6 MiB/s wr, 636 op/s 2024-03-09T00:12:02.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:02 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:12:02.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:02 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:12:02.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:02 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:12:02.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-09T00:12:02.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-09T00:12:02.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 -- 172.21.15.22:0/1838519673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc1065e0 msgr2=0x7fafbc10e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:02.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/1838519673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc1065e0 0x7fafbc10e1b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009ca0 tx=0x7fafb402c0f0 comp rx=0 tx=0).stop 2024-03-09T00:12:02.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 -- 172.21.15.22:0/1838519673 shutdown_connections 2024-03-09T00:12:02.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/1838519673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc1065e0 0x7fafbc10e1b0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:02.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/1838519673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafbc101ec0 0x7fafbc105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:02.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/1838519673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc101510 0x7fafbc1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 -- 172.21.15.22:0/1838519673 >> 172.21.15.22:0/1838519673 conn(0x7fafbc0fcdb0 msgr2=0x7fafbc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 -- 172.21.15.22:0/1838519673 shutdown_connections 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 -- 172.21.15.22:0/1838519673 wait complete. 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.895+0000 7fafc1ea7640 1 Processor -- start 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafc1ea7640 1 -- start start 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafc1ea7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc101510 0x7fafbc19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafc1ea7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafbc1065e0 0x7fafbc19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafc1ea7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc1978a0 0x7fafbc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:02.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafc1ea7640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc111680 con 0x7fafbc101510 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafc1ea7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc111500 con 0x7fafbc1065e0 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafc1ea7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc111800 con 0x7fafbc1978a0 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc101510 0x7fafbc19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc101510 0x7fafbc19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47664/0 (socket says 172.21.15.22:47664) 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 -- 172.21.15.22:0/2937460189 learned_addr learned my addr 172.21.15.22:0/2937460189 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbaffd640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafbc1065e0 0x7fafbc19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbbfff640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc1978a0 0x7fafbc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 -- 172.21.15.22:0/2937460189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc1978a0 msgr2=0x7fafbc197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:02.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc1978a0 0x7fafbc197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 -- 172.21.15.22:0/2937460189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafbc1065e0 msgr2=0x7fafbc19dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafbc1065e0 0x7fafbc19dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 -- 172.21.15.22:0/2937460189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc1983c0 con 0x7fafbc101510 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.896+0000 7fafbb7fe640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc101510 0x7fafbc19d6f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fafb000d950 tx=0x7fafb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.897+0000 7fafb8ff9640 1 -- 172.21.15.22:0/2937460189 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0017070 con 0x7fafbc101510 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.897+0000 7fafb8ff9640 1 -- 172.21.15.22:0/2937460189 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafb0009c30 con 0x7fafbc101510 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.897+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc1986b0 con 0x7fafbc101510 2024-03-09T00:12:02.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.897+0000 7fafb8ff9640 1 -- 172.21.15.22:0/2937460189 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000f6b0 con 0x7fafbc101510 2024-03-09T00:12:02.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.897+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc1a4950 con 0x7fafbc101510 2024-03-09T00:12:02.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.898+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc04f050 con 0x7fafbc101510 2024-03-09T00:12:02.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.899+0000 7fafb8ff9640 1 -- 172.21.15.22:0/2937460189 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fafb00109d0 con 0x7fafbc101510 2024-03-09T00:12:02.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.899+0000 7fafb8ff9640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fafa40a2a90 0x7fafa40a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:02.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.899+0000 7fafb8ff9640 1 -- 172.21.15.22:0/2937460189 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7fafb00ced50 con 0x7fafbc101510 2024-03-09T00:12:02.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.899+0000 7fafbaffd640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fafa40a2a90 0x7fafa40a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:02.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.900+0000 7fafbaffd640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fafa40a2a90 0x7fafa40a4f50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fafbc1003d0 tx=0x7fafac005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:02.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:02.903+0000 7fafb8ff9640 1 -- 172.21.15.22:0/2937460189 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fafb0009de0 con 0x7fafbc101510 2024-03-09T00:12:03.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.108+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 --> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fafbc102040 con 0x7fafa40a2a90 2024-03-09T00:12:03.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.118+0000 7fafb8ff9640 1 -- 172.21.15.22:0/2937460189 <== mgr.15660 v2:172.21.15.22:6800/2739225064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fafbc102040 con 0x7fafa40a2a90 2024-03-09T00:12:03.115 INFO:teuthology.orchestra.run.smithi022.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-08T00:00:00", "created": "2024-03-08T23:54:33", "first": "2024-03-08T23:55:00", "last": "2024-03-09T00:12:00", "last_pruned": "2024-03-09T00:12:01", "created_count": 18, "pruned_count": 11, "active": true} 2024-03-09T00:12:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fafa40a2a90 msgr2=0x7fafa40a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fafa40a2a90 0x7fafa40a4f50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fafbc1003d0 tx=0x7fafac005f90 comp rx=0 tx=0).stop 2024-03-09T00:12:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc101510 msgr2=0x7fafbc19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc101510 0x7fafbc19d6f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fafb000d950 tx=0x7fafb000de10 comp rx=0 tx=0).stop 2024-03-09T00:12:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 shutdown_connections 2024-03-09T00:12:03.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fafa40a2a90 0x7fafa40a4f50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc1978a0 0x7fafbc197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafbc1065e0 0x7fafbc19dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 --2- 172.21.15.22:0/2937460189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fafbc101510 0x7fafbc19d6f0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 >> 172.21.15.22:0/2937460189 conn(0x7fafbc0fcdb0 msgr2=0x7fafbc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 shutdown_connections 2024-03-09T00:12:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.120+0000 7fafc1ea7640 1 -- 172.21.15.22:0/2937460189 wait complete. 2024-03-09T00:12:03.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-09T00:12:03.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 -- 172.21.15.22:0/1379847594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4101510 msgr2=0x7f77b41018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:03.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 --2- 172.21.15.22:0/1379847594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4101510 0x7f77b41018f0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009900 tx=0x7f77a402bc30 comp rx=0 tx=0).stop 2024-03-09T00:12:03.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 -- 172.21.15.22:0/1379847594 shutdown_connections 2024-03-09T00:12:03.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 --2- 172.21.15.22:0/1379847594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b41065e0 0x7f77b410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 --2- 172.21.15.22:0/1379847594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b4101ec0 0x7f77b4105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 --2- 172.21.15.22:0/1379847594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4101510 0x7f77b41018f0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 -- 172.21.15.22:0/1379847594 >> 172.21.15.22:0/1379847594 conn(0x7f77b40fcdb0 msgr2=0x7f77b40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 -- 172.21.15.22:0/1379847594 shutdown_connections 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.236+0000 7f77b886c640 1 -- 172.21.15.22:0/1379847594 wait complete. 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 Processor -- start 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 -- start start 2024-03-09T00:12:03.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b4101510 0x7f77b40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b41065e0 0x7f77b4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4074070 0x7f77b41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41116a0 con 0x7f77b4074070 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b4111520 con 0x7f77b41065e0 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b886c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b4111820 con 0x7f77b4101510 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4074070 0x7f77b41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4074070 0x7f77b41a5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47690/0 (socket says 172.21.15.22:47690) 2024-03-09T00:12:03.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 -- 172.21.15.22:0/2718578769 learned_addr learned my addr 172.21.15.22:0/2718578769 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b259c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b4101510 0x7f77b40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77abfff640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b41065e0 0x7f77b4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 -- 172.21.15.22:0/2718578769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b4101510 msgr2=0x7f77b40735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b4101510 0x7f77b40735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 -- 172.21.15.22:0/2718578769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b41065e0 msgr2=0x7f77b4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b41065e0 0x7f77b4073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 -- 172.21.15.22:0/2718578769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b41a63d0 con 0x7f77b4074070 2024-03-09T00:12:03.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.237+0000 7f77b2d9d640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4074070 0x7f77b41a5d30 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00eaa0 tx=0x7f77ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.238+0000 7f77ab7fe640 1 -- 172.21.15.22:0/2718578769 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac00cdb0 con 0x7f77b4074070 2024-03-09T00:12:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.238+0000 7f77ab7fe640 1 -- 172.21.15.22:0/2718578769 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77ac005020 con 0x7f77b4074070 2024-03-09T00:12:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.238+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b41a66c0 con 0x7f77b4074070 2024-03-09T00:12:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.238+0000 7f77ab7fe640 1 -- 172.21.15.22:0/2718578769 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0116b0 con 0x7f77b4074070 2024-03-09T00:12:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.238+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b41a6c60 con 0x7f77b4074070 2024-03-09T00:12:03.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.239+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b404f050 con 0x7f77b4074070 2024-03-09T00:12:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.240+0000 7f77ab7fe640 1 -- 172.21.15.22:0/2718578769 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f77ac005560 con 0x7f77b4074070 2024-03-09T00:12:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.241+0000 7f77ab7fe640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77a00a2a90 0x7f77a00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:03.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.241+0000 7f77ab7fe640 1 -- 172.21.15.22:0/2718578769 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 9495+0+0 (secure 0 0 0) 0x7f77ac0cf6a0 con 0x7f77b4074070 2024-03-09T00:12:03.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.241+0000 7f77b259c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77a00a2a90 0x7f77a00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:03.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.242+0000 7f77b259c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77a00a2a90 0x7f77a00a4f50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006010 tx=0x7f77a4005dd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:03.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.245+0000 7f77ab7fe640 1 -- 172.21.15.22:0/2718578769 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f77ac016030 con 0x7f77b4074070 2024-03-09T00:12:03.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.450+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 --> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f77b4101ec0 con 0x7f77a00a2a90 2024-03-09T00:12:03.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.459+0000 7f77ab7fe640 1 -- 172.21.15.22:0/2718578769 <== mgr.15660 v2:172.21.15.22:6800/2739225064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f77b4101ec0 con 0x7f77a00a2a90 2024-03-09T00:12:03.456 INFO:teuthology.orchestra.run.smithi022.stdout:/ 1m 6m3h 2024-03-09T00:12:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77a00a2a90 msgr2=0x7f77a00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77a00a2a90 0x7f77a00a4f50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006010 tx=0x7f77a4005dd0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4074070 msgr2=0x7f77b41a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4074070 0x7f77b41a5d30 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00eaa0 tx=0x7f77ac00ef60 comp rx=0 tx=0).stop 2024-03-09T00:12:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 shutdown_connections 2024-03-09T00:12:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77a00a2a90 0x7f77a00a4f50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77b4074070 0x7f77b41a5d30 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b41065e0 0x7f77b4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 --2- 172.21.15.22:0/2718578769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b4101510 0x7f77b40735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 >> 172.21.15.22:0/2718578769 conn(0x7f77b40fcdb0 msgr2=0x7f77b40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 shutdown_connections 2024-03-09T00:12:03.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:03.461+0000 7f77b886c640 1 -- 172.21.15.22:0/2718578769 wait complete. 2024-03-09T00:12:03.469 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-09T00:12:03.479 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-09T00:12:04.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:04 smithi062 ceph-mon[36129]: from='client.20919 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-09T00:12:04.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:04 smithi062 ceph-mon[36129]: pgmap v557: 129 pgs: 129 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 518 op/s 2024-03-09T00:12:04.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:04 smithi062 ceph-mon[36129]: from='client.20925 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-09T00:12:04.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:04 smithi192 ceph-mon[35995]: from='client.20919 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-09T00:12:04.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:04 smithi192 ceph-mon[35995]: pgmap v557: 129 pgs: 129 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 518 op/s 2024-03-09T00:12:04.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:04 smithi192 ceph-mon[35995]: from='client.20925 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-09T00:12:04.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:04 smithi022 ceph-mon[31894]: from='client.20919 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-09T00:12:04.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:04 smithi022 ceph-mon[31894]: pgmap v557: 129 pgs: 129 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 518 op/s 2024-03-09T00:12:04.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:04 smithi022 ceph-mon[31894]: from='client.20925 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-09T00:12:05.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:05 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:12:05.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:05 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:12:05.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:05 smithi062 ceph-mon[36129]: scrub summary: idle 2024-03-09T00:12:05.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:05 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:12:05.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:05 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:12:05.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:05 smithi192 ceph-mon[35995]: scrub summary: idle 2024-03-09T00:12:05.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:05 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:12:05.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:05 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:12:05.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:05 smithi022 ceph-mon[31894]: scrub summary: idle 2024-03-09T00:12:06.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:06 smithi062 ceph-mon[36129]: pgmap v558: 129 pgs: 129 active+clean; 385 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 616 op/s 2024-03-09T00:12:06.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:06 smithi062 ceph-mon[36129]: osdmap e124: 12 total, 12 up, 12 in 2024-03-09T00:12:06.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:06 smithi192 ceph-mon[35995]: pgmap v558: 129 pgs: 129 active+clean; 385 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 616 op/s 2024-03-09T00:12:06.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:06 smithi192 ceph-mon[35995]: osdmap e124: 12 total, 12 up, 12 in 2024-03-09T00:12:06.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:06 smithi022 ceph-mon[31894]: pgmap v558: 129 pgs: 129 active+clean; 385 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 616 op/s 2024-03-09T00:12:06.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:06 smithi022 ceph-mon[31894]: osdmap e124: 12 total, 12 up, 12 in 2024-03-09T00:12:08.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:08 smithi062 ceph-mon[36129]: pgmap v560: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 350 op/s 2024-03-09T00:12:08.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:08 smithi192 ceph-mon[35995]: pgmap v560: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 350 op/s 2024-03-09T00:12:08.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:08 smithi022 ceph-mon[31894]: pgmap v560: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 350 op/s 2024-03-09T00:12:10.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:10 smithi062 ceph-mon[36129]: pgmap v561: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-03-09T00:12:10.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:10 smithi192 ceph-mon[35995]: pgmap v561: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-03-09T00:12:10.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:10 smithi022 ceph-mon[31894]: pgmap v561: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 384 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-03-09T00:12:12.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:12 smithi062 ceph-mon[36129]: pgmap v562: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 384 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-09T00:12:12.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:12 smithi192 ceph-mon[35995]: pgmap v562: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 384 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-09T00:12:12.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:12 smithi022 ceph-mon[31894]: pgmap v562: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 384 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-09T00:12:14.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:14 smithi062 ceph-mon[36129]: pgmap v563: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 384 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-09T00:12:14.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:14 smithi192 ceph-mon[35995]: pgmap v563: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 384 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-09T00:12:14.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:14 smithi022 ceph-mon[31894]: pgmap v563: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 384 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-09T00:12:16.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:16 smithi062 ceph-mon[36129]: pgmap v564: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.6 MiB/s wr, 110 op/s 2024-03-09T00:12:16.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:16 smithi192 ceph-mon[35995]: pgmap v564: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.6 MiB/s wr, 110 op/s 2024-03-09T00:12:16.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:16 smithi022 ceph-mon[31894]: pgmap v564: 129 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 109 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.6 MiB/s wr, 110 op/s 2024-03-09T00:12:18.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:18 smithi062 ceph-mon[36129]: pgmap v565: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.4 MiB/s wr, 100 op/s 2024-03-09T00:12:18.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:18 smithi192 ceph-mon[35995]: pgmap v565: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.4 MiB/s wr, 100 op/s 2024-03-09T00:12:18.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:18 smithi022 ceph-mon[31894]: pgmap v565: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 396 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.4 MiB/s wr, 100 op/s 2024-03-09T00:12:20.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:20 smithi062 ceph-mon[36129]: osdmap e125: 12 total, 12 up, 12 in 2024-03-09T00:12:20.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:20 smithi062 ceph-mon[36129]: pgmap v567: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 394 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.5 MiB/s wr, 106 op/s 2024-03-09T00:12:20.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:20 smithi192 ceph-mon[35995]: osdmap e125: 12 total, 12 up, 12 in 2024-03-09T00:12:20.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:20 smithi192 ceph-mon[35995]: pgmap v567: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 394 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.5 MiB/s wr, 106 op/s 2024-03-09T00:12:20.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:20 smithi022 ceph-mon[31894]: osdmap e125: 12 total, 12 up, 12 in 2024-03-09T00:12:20.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:20 smithi022 ceph-mon[31894]: pgmap v567: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 394 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.5 MiB/s wr, 106 op/s 2024-03-09T00:12:22.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:22 smithi022 ceph-mon[31894]: pgmap v568: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.6 MiB/s wr, 140 op/s 2024-03-09T00:12:22.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:22 smithi062 ceph-mon[36129]: pgmap v568: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.6 MiB/s wr, 140 op/s 2024-03-09T00:12:22.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:22 smithi192 ceph-mon[35995]: pgmap v568: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.6 MiB/s wr, 140 op/s 2024-03-09T00:12:24.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:24 smithi022 ceph-mon[31894]: pgmap v569: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.6 MiB/s wr, 140 op/s 2024-03-09T00:12:24.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:24 smithi022 ceph-mon[31894]: osdmap e126: 12 total, 12 up, 12 in 2024-03-09T00:12:24.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:24 smithi062 ceph-mon[36129]: pgmap v569: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.6 MiB/s wr, 140 op/s 2024-03-09T00:12:24.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:24 smithi062 ceph-mon[36129]: osdmap e126: 12 total, 12 up, 12 in 2024-03-09T00:12:24.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:24 smithi192 ceph-mon[35995]: pgmap v569: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.6 MiB/s wr, 140 op/s 2024-03-09T00:12:24.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:24 smithi192 ceph-mon[35995]: osdmap e126: 12 total, 12 up, 12 in 2024-03-09T00:12:26.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-09T00:12:26.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-09T00:12:26.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.289+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/648542139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74112730 msgr2=0x7fbf74114b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.289+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/648542139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74112730 0x7fbf74114b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009370 tx=0x7fbf6402dfa0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.294+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/648542139 shutdown_connections 2024-03-09T00:12:26.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.294+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/648542139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74112730 0x7fbf74114b20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.294+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/648542139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf74108ed0 0x7fbf7410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.294+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/648542139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 0x7fbf7410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.294+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/648542139 >> 172.21.15.22:0/648542139 conn(0x7fbf7406c3d0 msgr2=0x7fbf7406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.297+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/648542139 shutdown_connections 2024-03-09T00:12:26.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.299+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/648542139 wait complete. 2024-03-09T00:12:26.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 Processor -- start 2024-03-09T00:12:26.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 -- start start 2024-03-09T00:12:26.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 0x7fbf7419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74108ed0 0x7fbf7419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf74112730 0x7fbf741a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74117d60 con 0x7fbf74108a60 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74117be0 con 0x7fbf74108ed0 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.300+0000 7fbf7a8ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74117ee0 con 0x7fbf74112730 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.301+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74108ed0 0x7fbf7419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.301+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 0x7fbf7419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.301+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 0x7fbf7419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53360/0 (socket says 172.21.15.22:53360) 2024-03-09T00:12:26.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.301+0000 7fbf73fff640 1 -- 172.21.15.22:0/826653185 learned_addr learned my addr 172.21.15.22:0/826653185 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.302+0000 7fbf737fe640 1 -- 172.21.15.22:0/826653185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf74112730 msgr2=0x7fbf741a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.302+0000 7fbf78e48640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf74112730 0x7fbf741a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.302+0000 7fbf737fe640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf74112730 0x7fbf741a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.302+0000 7fbf737fe640 1 -- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 msgr2=0x7fbf7419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.302+0000 7fbf737fe640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 0x7fbf7419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.302+0000 7fbf737fe640 1 -- 172.21.15.22:0/826653185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf741a32f0 con 0x7fbf74108ed0 2024-03-09T00:12:26.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.302+0000 7fbf73fff640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 0x7fbf7419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:12:26.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.306+0000 7fbf737fe640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74108ed0 0x7fbf7419e840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbf7406d2a0 tx=0x7fbf6c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.310+0000 7fbf717fa640 1 -- 172.21.15.22:0/826653185 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c015710 con 0x7fbf74108ed0 2024-03-09T00:12:26.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.310+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf741a35e0 con 0x7fbf74108ed0 2024-03-09T00:12:26.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.310+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf74079550 con 0x7fbf74108ed0 2024-03-09T00:12:26.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.310+0000 7fbf717fa640 1 -- 172.21.15.22:0/826653185 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf6c015d90 con 0x7fbf74108ed0 2024-03-09T00:12:26.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.310+0000 7fbf717fa640 1 -- 172.21.15.22:0/826653185 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c015710 con 0x7fbf74108ed0 2024-03-09T00:12:26.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.314+0000 7fbf7d56b740 1 Processor -- start 2024-03-09T00:12:26.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.314+0000 7fbf717fa640 1 -- 172.21.15.22:0/826653185 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbf6c0158b0 con 0x7fbf74108ed0 2024-03-09T00:12:26.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.315+0000 7fbf717fa640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbf5c0a2db0 0x7fbf5c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.315+0000 7fbf717fa640 1 -- 172.21.15.22:0/826653185 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fbf6c015510 con 0x7fbf74108ed0 2024-03-09T00:12:26.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.316+0000 7fbf78e48640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbf5c0a2db0 0x7fbf5c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.316+0000 7fbf78e48640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbf5c0a2db0 0x7fbf5c0a5270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009f50 tx=0x7fbf64005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.317+0000 7fbf7d56b740 1 -- start start 2024-03-09T00:12:26.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.317+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a863db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.317+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a8646720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.317+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a8649050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.317+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a85a0e00 con 0x5608a863e080 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.317+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a8549b10 con 0x5608a8646c60 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.317+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a859cea0 con 0x5608a863d760 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a8646720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf78e48640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a8649050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a8646720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53372/0 (socket says 172.21.15.22:53372) 2024-03-09T00:12:26.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 -- 172.21.15.22:0/864857822 learned_addr learned my addr 172.21.15.22:0/864857822 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 -- 172.21.15.22:0/864857822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 msgr2=0x5608a863db40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T00:12:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 --2- 172.21.15.22:0/864857822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a863db40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 -- 172.21.15.22:0/864857822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 msgr2=0x5608a8649050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 --2- 172.21.15.22:0/864857822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a8649050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 -- 172.21.15.22:0/864857822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a837f7d0 con 0x5608a863e080 2024-03-09T00:12:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.318+0000 7fbf737fe640 1 --2- 172.21.15.22:0/864857822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a8646720 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c09b880 tx=0x7fbf6c0a3760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ec318fc7a56b01 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.319+0000 7fbf5a7fc640 1 -- 172.21.15.22:0/864857822 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c09be50 con 0x5608a863e080 2024-03-09T00:12:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.319+0000 7fbf5a7fc640 1 -- 172.21.15.22:0/864857822 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf6c017a40 con 0x5608a863e080 2024-03-09T00:12:26.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.319+0000 7fbf5a7fc640 1 -- 172.21.15.22:0/864857822 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c09bff0 con 0x5608a863e080 2024-03-09T00:12:26.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.320+0000 7fbf7d56b740 1 -- 172.21.15.22:0/864857822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 msgr2=0x5608a8646720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.320+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/864857822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a8646720 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c09b880 tx=0x7fbf6c0a3760 comp rx=0 tx=0).stop 2024-03-09T00:12:26.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.320+0000 7fbf7d56b740 1 -- 172.21.15.22:0/864857822 shutdown_connections 2024-03-09T00:12:26.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.320+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/864857822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a8649050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.320+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/864857822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a8646720 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.320+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/864857822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a863db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.320+0000 7fbf7d56b740 1 -- 172.21.15.22:0/864857822 >> 172.21.15.22:0/864857822 conn(0x5608a8637e90 msgr2=0x5608a8638270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.321+0000 7fbf7d56b740 1 -- 172.21.15.22:0/864857822 shutdown_connections 2024-03-09T00:12:26.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.321+0000 7fbf7d56b740 1 -- 172.21.15.22:0/864857822 wait complete. 2024-03-09T00:12:26.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.321+0000 7fbf7d56b740 1 -- start start 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 1 Processor -- start 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a8646490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863e080 0x5608a8643820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a8643d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a8649990 con 0x5608a8646c60 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a855d0a0 con 0x5608a863e080 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a8649b10 con 0x5608a863d760 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf7d56b740 10 client.0 _ll_register_callbacks cb 0x5608a855ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:26.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863e080 0x5608a8643820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863e080 0x5608a8643820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:56898/0 (socket says 172.21.15.22:56898) 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf78e48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a8643d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf737fe640 1 -- 172.21.15.22:0/3963482137 learned_addr learned my addr 172.21.15.22:0/3963482137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.322+0000 7fbf73fff640 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a8646490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf737fe640 1 -- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 msgr2=0x5608a8646490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf737fe640 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a8646490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf737fe640 1 -- 172.21.15.22:0/3963482137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 msgr2=0x5608a8643d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf737fe640 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a8643d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf737fe640 1 -- 172.21.15.22:0/3963482137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608a82b53f0 con 0x5608a863e080 2024-03-09T00:12:26.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf737fe640 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863e080 0x5608a8643820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00bfd0 tx=0x7fbf6c09f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72b698ee9941d89 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf537fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-03-09T00:12:26.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf537fe640 1 -- 172.21.15.22:0/3963482137 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0b7020 con 0x5608a863e080 2024-03-09T00:12:26.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.323+0000 7fbf537fe640 1 -- 172.21.15.22:0/3963482137 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fbf6c0d0790 con 0x5608a863e080 2024-03-09T00:12:26.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf6c017a40 con 0x5608a863e080 2024-03-09T00:12:26.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf537fe640 1 -- 172.21.15.22:0/3963482137 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf6c0bd050 con 0x5608a863e080 2024-03-09T00:12:26.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 10 client.30122 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:26.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5608a82750f0 con 0x5608a863e080 2024-03-09T00:12:26.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf537fe640 1 -- 172.21.15.22:0/3963482137 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf6c017a40 con 0x5608a863e080 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf537fe640 1 -- 172.21.15.22:0/3963482137 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbf6c09b4a0 con 0x5608a863e080 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 10 client.30122 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 10 client.30122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 20 client.30122 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a8656620 0x5608a8658a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 4 client.30122 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.324+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608a85ae340 con 0x5608a8656620 2024-03-09T00:12:26.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.325+0000 7fbf73fff640 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a8656620 0x5608a8658a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.325+0000 7fbf73fff640 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a8656620 0x5608a8658a10 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0b03b46a173bb3 server_cookie=d9f268f30bca1f4a in_seq=0 out_seq=0 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.325+0000 7fbf537fe640 10 client.30122 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.326+0000 7fbf537fe640 1 -- 172.21.15.22:0/3963482137 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608a85ae340 con 0x5608a8656620 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.326+0000 7fbf537fe640 10 client.30122 handle_command_reply: tid=0 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.326+0000 7fbf7d56b740 1 client.30122 shutdown 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863e080 msgr2=0x5608a8643820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863e080 0x5608a8643820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00bfd0 tx=0x7fbf6c09f520 comp rx=0 tx=0).stop 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 shutdown_connections 2024-03-09T00:12:26.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a8656620 0x5608a8658a10 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a8643d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863e080 0x5608a8643820 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/3963482137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a8646490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf537fe640 0 client.30122 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 >> 172.21.15.22:0/3963482137 conn(0x5608a8637e90 msgr2=0x5608a8639be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 shutdown_connections 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 1 -- 172.21.15.22:0/3963482137 wait complete. 2024-03-09T00:12:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.327+0000 7fbf7d56b740 20 client.30122 trim_cache size 0 max 16384 2024-03-09T00:12:26.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 Processor -- start 2024-03-09T00:12:26.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 -- start start 2024-03-09T00:12:26.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863d760 0x5608a8645f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863e080 0x5608a8646490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a865a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600123e0 con 0x5608a8646c60 2024-03-09T00:12:26.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6000ff80 con 0x5608a863d760 2024-03-09T00:12:26.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6000e040 con 0x5608a863e080 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863d760 0x5608a8645f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf78e48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a865a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863d760 0x5608a8645f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:56912/0 (socket says 172.21.15.22:56912) 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 -- 172.21.15.22:0/4027235927 learned_addr learned my addr 172.21.15.22:0/4027235927 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf737fe640 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863e080 0x5608a8646490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 -- 172.21.15.22:0/4027235927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863e080 msgr2=0x5608a8646490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863e080 0x5608a8646490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 -- 172.21.15.22:0/4027235927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 msgr2=0x5608a865a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a865a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.343+0000 7fbf73fff640 1 -- 172.21.15.22:0/4027235927 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a86499f0 con 0x5608a863d760 2024-03-09T00:12:26.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.344+0000 7fbf73fff640 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863d760 0x5608a8645f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf68008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765fa4a528e80a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.344+0000 7fbf59ffb640 1 -- 172.21.15.22:0/4027235927 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68023430 con 0x5608a863d760 2024-03-09T00:12:26.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.344+0000 7fbf59ffb640 1 -- 172.21.15.22:0/4027235927 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5608a85ae340 con 0x5608a863d760 2024-03-09T00:12:26.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.344+0000 7fbf59ffb640 1 -- 172.21.15.22:0/4027235927 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680227f0 con 0x5608a863d760 2024-03-09T00:12:26.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.344+0000 7fbf7d56b740 1 -- 172.21.15.22:0/4027235927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863d760 msgr2=0x5608a8645f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.344+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863d760 0x5608a8645f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf68008040 comp rx=0 tx=0).stop 2024-03-09T00:12:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 -- 172.21.15.22:0/4027235927 shutdown_connections 2024-03-09T00:12:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a8646c60 0x5608a865a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863e080 0x5608a8646490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/4027235927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a863d760 0x5608a8645f50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 -- 172.21.15.22:0/4027235927 >> 172.21.15.22:0/4027235927 conn(0x5608a8637e90 msgr2=0x5608a863ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 -- 172.21.15.22:0/4027235927 shutdown_connections 2024-03-09T00:12:26.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 -- 172.21.15.22:0/4027235927 wait complete. 2024-03-09T00:12:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 -- start start 2024-03-09T00:12:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 Processor -- start 2024-03-09T00:12:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.345+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a865a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a865a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf7d56b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a86588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6000e640 con 0x5608a863e080 2024-03-09T00:12:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a865a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6000f920 con 0x5608a8646c60 2024-03-09T00:12:26.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a865a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53402/0 (socket says 172.21.15.22:53402) 2024-03-09T00:12:26.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf7d56b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6000ec60 con 0x5608a863d760 2024-03-09T00:12:26.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 -- 172.21.15.22:0/2333376624 learned_addr learned my addr 172.21.15.22:0/2333376624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf73fff640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a865a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf78e48640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a86588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf7d56b740 10 client.0 _ll_register_callbacks cb 0x7fbf6c06a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 -- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 msgr2=0x5608a865a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a865a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 -- 172.21.15.22:0/2333376624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 msgr2=0x5608a86588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a86588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 -- 172.21.15.22:0/2333376624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608a8658f90 con 0x5608a863e080 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf73fff640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a865a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:12:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf78e48640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a86588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf737fe640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a865a890 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c000f80 tx=0x7fbf6c0070b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a678886bccaf92c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf537fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf537fe640 1 -- 172.21.15.22:0/2333376624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0a4040 con 0x5608a863e080 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf537fe640 1 -- 172.21.15.22:0/2333376624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fbf6c0d0790 con 0x5608a863e080 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.346+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a85ae340 con 0x5608a863e080 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf537fe640 1 -- 172.21.15.22:0/2333376624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf6c0ac080 con 0x5608a863e080 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 10 client.20940 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:26.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5608a86b5a70 con 0x5608a863e080 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf537fe640 1 -- 172.21.15.22:0/2333376624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5608a85ae340 con 0x5608a863e080 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf537fe640 1 -- 172.21.15.22:0/2333376624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbf6c09b440 con 0x5608a863e080 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 10 client.20940 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 10 client.20940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 20 client.20940 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a86b5f20 0x5608a86b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 4 client.20940 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-09T00:12:26.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608a85a8690 con 0x5608a86b5f20 2024-03-09T00:12:26.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.347+0000 7fbf73fff640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a86b5f20 0x5608a86b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf537fe640 10 client.20940 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:26.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf73fff640 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a86b5f20 0x5608a86b8340 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda535939c2faca0 server_cookie=e1b804efe9bbb565 in_seq=0 out_seq=0 2024-03-09T00:12:26.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf537fe640 1 -- 172.21.15.22:0/2333376624 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5608a85a8690 con 0x5608a86b5f20 2024-03-09T00:12:26.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf537fe640 10 client.20940 handle_command_reply: tid=0 2024-03-09T00:12:26.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 client.20940 shutdown 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 msgr2=0x5608a865a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a865a890 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c000f80 tx=0x7fbf6c0070b0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 shutdown_connections 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x5608a86b5f20 0x5608a86b8340 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a8646c60 0x5608a86588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf537fe640 0 client.20940 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608a863e080 0x5608a865a890 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 --2- 172.21.15.22:0/2333376624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5608a863d760 0x5608a865a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 >> 172.21.15.22:0/2333376624 conn(0x5608a8637e90 msgr2=0x5608a86391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 shutdown_connections 2024-03-09T00:12:26.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 1 -- 172.21.15.22:0/2333376624 wait complete. 2024-03-09T00:12:26.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.348+0000 7fbf7d56b740 20 client.20940 trim_cache size 0 max 16384 2024-03-09T00:12:26.360 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:12:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-03-09T00:12:26.361 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2024-03-09T00:12:26.361 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:12:26.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbf5c0a2db0 msgr2=0x7fbf5c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbf5c0a2db0 0x7fbf5c0a5270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009f50 tx=0x7fbf64005fd0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74108ed0 msgr2=0x7fbf7419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74108ed0 0x7fbf7419e840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbf7406d2a0 tx=0x7fbf6c009f70 comp rx=0 tx=0).stop 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 shutdown_connections 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fbf5c0a2db0 0x7fbf5c0a5270 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf74112730 0x7fbf741a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf74108ed0 0x7fbf7419e840 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 --2- 172.21.15.22:0/826653185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf74108a60 0x7fbf7419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 >> 172.21.15.22:0/826653185 conn(0x7fbf7406c3d0 msgr2=0x7fbf7406fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 shutdown_connections 2024-03-09T00:12:26.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.349+0000 7fbf7a8ab640 1 -- 172.21.15.22:0/826653185 wait complete. 2024-03-09T00:12:26.364 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-09T00:12:26.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:12:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 -- 172.21.15.22:0/2277844322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 msgr2=0x7f01241018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 --2- 172.21.15.22:0/2277844322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f01241018d0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f0120009900 tx=0x7f012002bc30 comp rx=0 tx=0).stop 2024-03-09T00:12:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 -- 172.21.15.22:0/2277844322 shutdown_connections 2024-03-09T00:12:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 --2- 172.21.15.22:0/2277844322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01241065c0 0x7f012410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 --2- 172.21.15.22:0/2277844322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0124101ea0 0x7f0124105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 --2- 172.21.15.22:0/2277844322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f01241018d0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 -- 172.21.15.22:0/2277844322 >> 172.21.15.22:0/2277844322 conn(0x7f01240fcd90 msgr2=0x7f01240ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 -- 172.21.15.22:0/2277844322 shutdown_connections 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.470+0000 7f012c2a4640 1 -- 172.21.15.22:0/2277844322 wait complete. 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 Processor -- start 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 -- start start 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f012419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01241065c0 0x7f012419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0124197ab0 0x7f0124197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124111650 con 0x7f01241014f0 2024-03-09T00:12:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01241114d0 con 0x7f01241065c0 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012c2a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01241117d0 con 0x7f0124197ab0 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f012419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f012419d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53414/0 (socket says 172.21.15.22:53414) 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 -- 172.21.15.22:0/4203838779 learned_addr learned my addr 172.21.15.22:0/4203838779 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012983f640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01241065c0 0x7f012419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a841640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0124197ab0 0x7f0124197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 -- 172.21.15.22:0/4203838779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0124197ab0 msgr2=0x7f0124197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0124197ab0 0x7f0124197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 -- 172.21.15.22:0/4203838779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01241065c0 msgr2=0x7f012419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01241065c0 0x7f012419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.471+0000 7f012a040640 1 -- 172.21.15.22:0/4203838779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01241985d0 con 0x7f01241014f0 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.472+0000 7f012a040640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f012419d8b0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f012002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.472+0000 7f011b7fe640 1 -- 172.21.15.22:0/4203838779 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120041070 con 0x7f01241014f0 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.472+0000 7f011b7fe640 1 -- 172.21.15.22:0/4203838779 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0120004450 con 0x7f01241014f0 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.472+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0124198860 con 0x7f01241014f0 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.472+0000 7f011b7fe640 1 -- 172.21.15.22:0/4203838779 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120003e00 con 0x7f01241014f0 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.472+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241a4a00 con 0x7f01241014f0 2024-03-09T00:12:26.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.473+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0124068530 con 0x7f01241014f0 2024-03-09T00:12:26.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.474+0000 7f011b7fe640 1 -- 172.21.15.22:0/4203838779 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0120044070 con 0x7f01241014f0 2024-03-09T00:12:26.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.474+0000 7f011b7fe640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01100a2970 0x7f01100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.474+0000 7f012983f640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01100a2970 0x7f01100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.474+0000 7f011b7fe640 1 -- 172.21.15.22:0/4203838779 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f01200f0ca0 con 0x7f01241014f0 2024-03-09T00:12:26.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.475+0000 7f012983f640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01100a2970 0x7f01100a4e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f01240fd2f0 tx=0x7f0114005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.479+0000 7f011b7fe640 1 -- 172.21.15.22:0/4203838779 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f01200bd720 con 0x7f01241014f0 2024-03-09T00:12:26.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:26 smithi022 ceph-mon[31894]: pgmap v571: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 2.9 MiB/s wr, 203 op/s 2024-03-09T00:12:26.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:26 smithi062 ceph-mon[36129]: pgmap v571: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 2.9 MiB/s wr, 203 op/s 2024-03-09T00:12:26.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:26 smithi192 ceph-mon[35995]: pgmap v571: 129 pgs: 129 active+clean; 380 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 2.9 MiB/s wr, 203 op/s 2024-03-09T00:12:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.700+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0124101ea0 con 0x7f01241014f0 2024-03-09T00:12:26.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.701+0000 7f011b7fe640 1 -- 172.21.15.22:0/4203838779 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f01200bd020 con 0x7f01241014f0 2024-03-09T00:12:26.698 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:12:26.699 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:10:14.710788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:12:26.699 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-03-09T00:12:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01100a2970 msgr2=0x7f01100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01100a2970 0x7f01100a4e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f01240fd2f0 tx=0x7f0114005f90 comp rx=0 tx=0).stop 2024-03-09T00:12:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 msgr2=0x7f012419d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f012419d8b0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f012002cb30 comp rx=0 tx=0).stop 2024-03-09T00:12:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 shutdown_connections 2024-03-09T00:12:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f01100a2970 0x7f01100a4e30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0124197ab0 0x7f0124197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01241065c0 0x7f012419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 --2- 172.21.15.22:0/4203838779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01241014f0 0x7f012419d8b0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 >> 172.21.15.22:0/4203838779 conn(0x7f01240fcd90 msgr2=0x7f01240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 shutdown_connections 2024-03-09T00:12:26.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.703+0000 7f012c2a4640 1 -- 172.21.15.22:0/4203838779 wait complete. 2024-03-09T00:12:26.710 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:12:26.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-09T00:12:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 -- 172.21.15.22:0/2885332936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa40ff9f0 msgr2=0x7f7aa40ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/2885332936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa40ff9f0 0x7f7aa40ffdd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009900 tx=0x7f7a9402bc50 comp rx=0 tx=0).stop 2024-03-09T00:12:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 -- 172.21.15.22:0/2885332936 shutdown_connections 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/2885332936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa4106660 0x7f7aa410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/2885332936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7aa41003a0 0x7f7aa4105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/2885332936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa40ff9f0 0x7f7aa40ffdd0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 -- 172.21.15.22:0/2885332936 >> 172.21.15.22:0/2885332936 conn(0x7f7aa4076700 msgr2=0x7f7aa40fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 -- 172.21.15.22:0/2885332936 shutdown_connections 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.819+0000 7f7aa8b97640 1 -- 172.21.15.22:0/2885332936 wait complete. 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 Processor -- start 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 -- start start 2024-03-09T00:12:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7aa40ff9f0 0x7f7aa419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa4106660 0x7f7aa419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa4197b00 0x7f7aa4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4111700 con 0x7f7aa4106660 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4111580 con 0x7f7aa4197b00 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa8b97640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4111880 con 0x7f7aa40ff9f0 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa4106660 0x7f7aa419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa4106660 0x7f7aa419de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53436/0 (socket says 172.21.15.22:53436) 2024-03-09T00:12:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 -- 172.21.15.22:0/981365523 learned_addr learned my addr 172.21.15.22:0/981365523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa259c640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7aa40ff9f0 0x7f7aa419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa4197b00 0x7f7aa4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 -- 172.21.15.22:0/981365523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7aa40ff9f0 msgr2=0x7f7aa419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7aa40ff9f0 0x7f7aa419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 -- 172.21.15.22:0/981365523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa4197b00 msgr2=0x7f7aa4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa4197b00 0x7f7aa4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 -- 172.21.15.22:0/981365523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4198620 con 0x7f7aa4106660 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.820+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa4106660 0x7f7aa419de90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00b430 tx=0x7f7a8c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.821+0000 7f7a9b7fe640 1 -- 172.21.15.22:0/981365523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c015020 con 0x7f7aa4106660 2024-03-09T00:12:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.821+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa4198910 con 0x7f7aa4106660 2024-03-09T00:12:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.821+0000 7f7a9b7fe640 1 -- 172.21.15.22:0/981365523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a8c011d10 con 0x7f7aa4106660 2024-03-09T00:12:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.821+0000 7f7a9b7fe640 1 -- 172.21.15.22:0/981365523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00f2e0 con 0x7f7aa4106660 2024-03-09T00:12:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.821+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa41a4b60 con 0x7f7aa4106660 2024-03-09T00:12:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7a9b7fe640 1 -- 172.21.15.22:0/981365523 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7a8c018020 con 0x7f7aa4106660 2024-03-09T00:12:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 Processor -- start 2024-03-09T00:12:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 -- start start 2024-03-09T00:12:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d3104580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d30cb360 con 0x55f2d3104ac0 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d3067ec0 con 0x55f2d310d6a0 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aab778740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d3010d30 con 0x55f2d31041a0 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7a9b7fe640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7a900a2970 0x7f7a900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.823+0000 7f7aa259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53440/0 (socket says 172.21.15.22:53440) 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7a900a2970 0x7f7a900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 -- 172.21.15.22:0/278849356 learned_addr learned my addr 172.21.15.22:0/278849356 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d3104580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7a9b7fe640 1 -- 172.21.15.22:0/981365523 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f7a8c0cea50 con 0x7f7aa4106660 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 -- 172.21.15.22:0/278849356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 msgr2=0x55f2d3104580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d3104580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 -- 172.21.15.22:0/278849356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 msgr2=0x55f2d310fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 -- 172.21.15.22:0/278849356 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2d300fbc0 con 0x55f2d3104ac0 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7a900a2970 0x7f7a900a4e30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c005c40 tx=0x7f7a9c00ac90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7aa259c640 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310d160 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009900 tx=0x7f7a94005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f624a0c664dfdb7f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7a98ff9640 1 -- 172.21.15.22:0/278849356 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94042070 con 0x55f2d3104ac0 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7a98ff9640 1 -- 172.21.15.22:0/278849356 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a9402caa0 con 0x55f2d3104ac0 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.824+0000 7f7a98ff9640 1 -- 172.21.15.22:0/278849356 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94004c50 con 0x55f2d3104ac0 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 -- 172.21.15.22:0/278849356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 msgr2=0x55f2d310d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310d160 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009900 tx=0x7f7a94005f10 comp rx=0 tx=0).stop 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 -- 172.21.15.22:0/278849356 shutdown_connections 2024-03-09T00:12:26.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310d160 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 --2- 172.21.15.22:0/278849356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d3104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 -- 172.21.15.22:0/278849356 >> 172.21.15.22:0/278849356 conn(0x55f2d30fe8d0 msgr2=0x55f2d30fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 -- 172.21.15.22:0/278849356 shutdown_connections 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 -- 172.21.15.22:0/278849356 wait complete. 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 -- start start 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.825+0000 7f7aab778740 1 Processor -- start 2024-03-09T00:12:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d310a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d30064a0 con 0x55f2d3104ac0 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d3063f60 con 0x55f2d310d6a0 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d3110430 con 0x55f2d31041a0 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 10 client.0 _ll_register_callbacks cb 0x55f2d3061d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53450/0 (socket says 172.21.15.22:53450) 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 -- 172.21.15.22:0/3712252345 learned_addr learned my addr 172.21.15.22:0/3712252345 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aa259c640 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d310a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 -- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 msgr2=0x55f2d310a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d310a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 -- 172.21.15.22:0/3712252345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 msgr2=0x55f2d310b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 -- 172.21.15.22:0/3712252345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2d2d7c3f0 con 0x55f2d3104ac0 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310ab60 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0093b0 tx=0x7f7a8c09aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f23bb34c9481bd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3712252345 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0b8060 con 0x55f2d3104ac0 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3712252345 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f7a8c011830 con 0x55f2d3104ac0 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.826+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a9402caa0 con 0x55f2d3104ac0 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3712252345 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a8c09ef20 con 0x55f2d3104ac0 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 10 client.20958 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:26.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f2d2e467d0 con 0x55f2d3104ac0 2024-03-09T00:12:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3712252345 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a9402caa0 con 0x55f2d3104ac0 2024-03-09T00:12:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3712252345 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a8c0b2030 con 0x55f2d3104ac0 2024-03-09T00:12:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 10 client.20958 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 10 client.20958 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:12:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 20 client.20958 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d311d7e0 0x55f2d311fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 4 client.20958 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:12:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2d3075400 con 0x55f2d311d7e0 2024-03-09T00:12:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aa259c640 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d311d7e0 0x55f2d311fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7aa259c640 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d311d7e0 0x55f2d311fbd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c83888f54e7053af server_cookie=5d24a2a82bd4a459 in_seq=0 out_seq=0 2024-03-09T00:12:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.827+0000 7f7a89ffb640 10 client.20958 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:12:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3712252345 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2d3075400 con 0x55f2d311d7e0 2024-03-09T00:12:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7a89ffb640 10 client.20958 handle_command_reply: tid=0 2024-03-09T00:12:26.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 client.20958 shutdown 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 msgr2=0x55f2d310ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310ab60 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0093b0 tx=0x7f7a8c09aa20 comp rx=0 tx=0).stop 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 shutdown_connections 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d311d7e0 0x55f2d311fbd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d310d6a0 0x55f2d310b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d3104ac0 0x55f2d310ab60 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 --2- 172.21.15.22:0/3712252345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d31041a0 0x55f2d310a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7a89ffb640 0 client.20958 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:12:26.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 >> 172.21.15.22:0/3712252345 conn(0x55f2d30fe8d0 msgr2=0x55f2d3100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 shutdown_connections 2024-03-09T00:12:26.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 1 -- 172.21.15.22:0/3712252345 wait complete. 2024-03-09T00:12:26.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.828+0000 7f7aab778740 20 client.20958 trim_cache size 0 max 16384 2024-03-09T00:12:26.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 Processor -- start 2024-03-09T00:12:26.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 -- start start 2024-03-09T00:12:26.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d31041a0 0x55f2d310cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d3104ac0 0x55f2d3109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d310d6a0 0x55f2d3107880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41b0c30 con 0x55f2d31041a0 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41ae7d0 con 0x55f2d3104ac0 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aab778740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41ac890 con 0x55f2d310d6a0 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aa259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d31041a0 0x55f2d310cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aa259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d31041a0 0x55f2d310cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53454/0 (socket says 172.21.15.22:53454) 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7aa259c640 1 -- 172.21.15.22:0/4034934549 learned_addr learned my addr 172.21.15.22:0/4034934549 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.840+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d3104ac0 0x55f2d3109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aa259c640 1 -- 172.21.15.22:0/4034934549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d310d6a0 msgr2=0x55f2d3107880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aa259c640 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d310d6a0 0x55f2d3107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aa259c640 1 -- 172.21.15.22:0/4034934549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d3104ac0 msgr2=0x55f2d3109600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aa259c640 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d3104ac0 0x55f2d3109600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aa259c640 1 -- 172.21.15.22:0/4034934549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2d30fa190 con 0x55f2d31041a0 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d3104ac0 0x55f2d3109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aa259c640 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d31041a0 0x55f2d310cd50 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7a9402f020 tx=0x7f7a94005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a82fe2211eb3573 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7a8bfff640 1 -- 172.21.15.22:0/4034934549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94042040 con 0x55f2d31041a0 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7a8bfff640 1 -- 172.21.15.22:0/4034934549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a94052070 con 0x55f2d31041a0 2024-03-09T00:12:26.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7a8bfff640 1 -- 172.21.15.22:0/4034934549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940039f0 con 0x55f2d31041a0 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 -- 172.21.15.22:0/4034934549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d31041a0 msgr2=0x55f2d310cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d31041a0 0x55f2d310cd50 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7a9402f020 tx=0x7f7a94005a20 comp rx=0 tx=0).stop 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 -- 172.21.15.22:0/4034934549 shutdown_connections 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d310d6a0 0x55f2d3107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d3104ac0 0x55f2d3109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 --2- 172.21.15.22:0/4034934549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d31041a0 0x55f2d310cd50 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 -- 172.21.15.22:0/4034934549 >> 172.21.15.22:0/4034934549 conn(0x55f2d30fe8d0 msgr2=0x55f2d3108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 -- 172.21.15.22:0/4034934549 shutdown_connections 2024-03-09T00:12:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.841+0000 7f7aab778740 1 -- 172.21.15.22:0/4034934549 wait complete. 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 -- start start 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 Processor -- start 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d31041a0 0x55f2d3113420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d3104ac0 0x55f2d3113960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d310d6a0 0x55f2d3173b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41ace90 con 0x55f2d310d6a0 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41ae170 con 0x55f2d31041a0 2024-03-09T00:12:26.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41ad4b0 con 0x55f2d3104ac0 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aab778740 10 client.0 _ll_register_callbacks cb 0x7f7a94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d310d6a0 0x55f2d3173b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d310d6a0 0x55f2d3173b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53462/0 (socket says 172.21.15.22:53462) 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 -- 172.21.15.22:0/3481838808 learned_addr learned my addr 172.21.15.22:0/3481838808 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa259c640 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d31041a0 0x55f2d3113420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7a9bfff640 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d3104ac0 0x55f2d3113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 -- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d3104ac0 msgr2=0x55f2d3113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d3104ac0 0x55f2d3113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 -- 172.21.15.22:0/3481838808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d31041a0 msgr2=0x55f2d3113420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d31041a0 0x55f2d3113420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.842+0000 7f7aa2d9d640 1 -- 172.21.15.22:0/3481838808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2d31740a0 con 0x55f2d310d6a0 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aa2d9d640 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d310d6a0 0x55f2d3173b60 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c010cc0 tx=0x7f7a9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e53bd7c5e2d4550 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7a89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3481838808 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c02d060 con 0x55f2d310d6a0 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3481838808 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f7a9c020440 con 0x55f2d310d6a0 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a94052070 con 0x55f2d310d6a0 2024-03-09T00:12:26.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3481838808 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a9c034050 con 0x55f2d310d6a0 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 10 client.20964 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f2d31751d0 con 0x55f2d310d6a0 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3481838808 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a94052070 con 0x55f2d310d6a0 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3481838808 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a9c01f640 con 0x55f2d310d6a0 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 10 client.20964 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 10 client.20964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 20 client.20964 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d3175900 0x55f2d3177d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 4 client.20964 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:12:26.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 --> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2d306f750 con 0x55f2d3175900 2024-03-09T00:12:26.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.843+0000 7f7aa259c640 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d3175900 0x55f2d3177d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aa259c640 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d3175900 0x55f2d3177d20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aff78ec28b9f8d69 server_cookie=cf9b12a7dd62c201 in_seq=0 out_seq=0 2024-03-09T00:12:26.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7a89ffb640 10 client.20964 ms_handle_connect on v2:172.21.15.192:6836/1326709093 2024-03-09T00:12:26.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7a89ffb640 1 -- 172.21.15.22:0/3481838808 <== mds.2 v2:172.21.15.192:6836/1326709093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2d306f750 con 0x55f2d3175900 2024-03-09T00:12:26.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7a89ffb640 10 client.20964 handle_command_reply: tid=0 2024-03-09T00:12:26.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 client.20964 shutdown 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d310d6a0 msgr2=0x55f2d3173b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d310d6a0 0x55f2d3173b60 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c010cc0 tx=0x7f7a9c004a40 comp rx=0 tx=0).stop 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 shutdown_connections 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:6836/1326709093,v1:172.21.15.192:6837/1326709093] conn(0x55f2d3175900 0x55f2d3177d20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f2d310d6a0 0x55f2d3173b60 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2d3104ac0 0x55f2d3113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 --2- 172.21.15.22:0/3481838808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d31041a0 0x55f2d3113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 >> 172.21.15.22:0/3481838808 conn(0x55f2d30fe8d0 msgr2=0x55f2d30fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7a89ffb640 0 client.20964 ms_handle_reset on v2:172.21.15.192:6836/1326709093 2024-03-09T00:12:26.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 shutdown_connections 2024-03-09T00:12:26.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 1 -- 172.21.15.22:0/3481838808 wait complete. 2024-03-09T00:12:26.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.844+0000 7f7aab778740 20 client.20964 trim_cache size 0 max 16384 2024-03-09T00:12:26.860 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:12:26.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7a900a2970 msgr2=0x7f7a900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7a900a2970 0x7f7a900a4e30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c005c40 tx=0x7f7a9c00ac90 comp rx=0 tx=0).stop 2024-03-09T00:12:26.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa4106660 msgr2=0x7f7aa419de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa4106660 0x7f7aa419de90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00b430 tx=0x7f7a8c00b8f0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 shutdown_connections 2024-03-09T00:12:26.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f7a900a2970 0x7f7a900a4e30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa4197b00 0x7f7aa4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7aa4106660 0x7f7aa419de90 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 --2- 172.21.15.22:0/981365523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7aa40ff9f0 0x7f7aa419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 >> 172.21.15.22:0/981365523 conn(0x7f7aa4076700 msgr2=0x7f7aa4104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 shutdown_connections 2024-03-09T00:12:26.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.845+0000 7f7aa8b97640 1 -- 172.21.15.22:0/981365523 wait complete. 2024-03-09T00:12:26.863 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:12:26.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-09T00:12:26.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 -- 172.21.15.22:0/626488963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1210100410 msgr2=0x7f1210105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 --2- 172.21.15.22:0/626488963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1210100410 0x7f1210105f40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009930 tx=0x7f11f802bc80 comp rx=0 tx=0).stop 2024-03-09T00:12:26.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 -- 172.21.15.22:0/626488963 shutdown_connections 2024-03-09T00:12:26.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 --2- 172.21.15.22:0/626488963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210106670 0x7f121010e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 --2- 172.21.15.22:0/626488963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1210100410 0x7f1210105f40 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 --2- 172.21.15.22:0/626488963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12100ffa60 0x7f12100ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 -- 172.21.15.22:0/626488963 >> 172.21.15.22:0/626488963 conn(0x7f1210076720 msgr2=0x7f12100fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 -- 172.21.15.22:0/626488963 shutdown_connections 2024-03-09T00:12:26.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.971+0000 7f1214c44640 1 -- 172.21.15.22:0/626488963 wait complete. 2024-03-09T00:12:26.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 Processor -- start 2024-03-09T00:12:26.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 -- start start 2024-03-09T00:12:26.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12100ffa60 0x7f121019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210106670 0x7f121019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210197940 0x7f1210197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1210111640 con 0x7f12100ffa60 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101114c0 con 0x7f1210197940 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f1214c44640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101117c0 con 0x7f1210106670 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12100ffa60 0x7f121019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12100ffa60 0x7f121019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53494/0 (socket says 172.21.15.22:53494) 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.972+0000 7f120e59c640 1 -- 172.21.15.22:0/966389115 learned_addr learned my addr 172.21.15.22:0/966389115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f120e59c640 1 -- 172.21.15.22:0/966389115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210106670 msgr2=0x7f121019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f120e59c640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210106670 0x7f121019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f120ed9d640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210197940 0x7f1210197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f120e59c640 1 -- 172.21.15.22:0/966389115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210197940 msgr2=0x7f1210197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f120e59c640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210197940 0x7f1210197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f120e59c640 1 -- 172.21.15.22:0/966389115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210198460 con 0x7f12100ffa60 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f120e59c640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12100ffa60 0x7f121019d790 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f120800d950 tx=0x7f120800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f12077fe640 1 -- 172.21.15.22:0/966389115 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208017070 con 0x7f12100ffa60 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f12077fe640 1 -- 172.21.15.22:0/966389115 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1208009c30 con 0x7f12100ffa60 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1210198750 con 0x7f12100ffa60 2024-03-09T00:12:26.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f12077fe640 1 -- 172.21.15.22:0/966389115 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120800f6b0 con 0x7f12100ffa60 2024-03-09T00:12:26.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.973+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12101a49a0 con 0x7f12100ffa60 2024-03-09T00:12:26.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.975+0000 7f12077fe640 1 -- 172.21.15.22:0/966389115 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f12080109d0 con 0x7f12100ffa60 2024-03-09T00:12:26.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 Processor -- start 2024-03-09T00:12:26.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 -- start start 2024-03-09T00:12:26.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f12077fe640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f11fc0a2970 0x7f11fc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d49230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d56570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d58ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db7d10360 con 0x560db7d48e50 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db7cacec0 con 0x560db7d56ab0 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f1217821740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db7c55d30 con 0x560db7d49770 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120ed9d640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f11fc0a2970 0x7f11fc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f12077fe640 1 -- 172.21.15.22:0/966389115 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f12080cefe0 con 0x7f12100ffa60 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d49230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d49230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53496/0 (socket says 172.21.15.22:53496) 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 -- 172.21.15.22:0/1957160539 learned_addr learned my addr 172.21.15.22:0/1957160539 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120ed9d640 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d58ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120dd9b640 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d56570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 -- 172.21.15.22:0/1957160539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 msgr2=0x560db7d56570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d56570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 -- 172.21.15.22:0/1957160539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 msgr2=0x560db7d58ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d58ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120e59c640 1 -- 172.21.15.22:0/1957160539 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db7c54bc0 con 0x560db7d48e50 2024-03-09T00:12:26.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120ed9d640 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d58ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.976+0000 7f120dd9b640 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d56570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f120ed9d640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f11fc0a2970 0x7f11fc0a4e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1200005fd0 tx=0x7f12000139c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f120e59c640 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d49230 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f1208097c00 tx=0x7f1208005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63463ed1a477186 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1204ff9640 1 -- 172.21.15.22:0/1957160539 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208020070 con 0x560db7d48e50 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1204ff9640 1 -- 172.21.15.22:0/1957160539 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f120809f160 con 0x560db7d48e50 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1204ff9640 1 -- 172.21.15.22:0/1957160539 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120800e760 con 0x560db7d48e50 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1217821740 1 -- 172.21.15.22:0/1957160539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 msgr2=0x560db7d49230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1217821740 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d49230 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f1208097c00 tx=0x7f1208005880 comp rx=0 tx=0).stop 2024-03-09T00:12:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1217821740 1 -- 172.21.15.22:0/1957160539 shutdown_connections 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1217821740 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d58ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1217821740 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d56570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1217821740 1 --2- 172.21.15.22:0/1957160539 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d49230 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.977+0000 7f1217821740 1 -- 172.21.15.22:0/1957160539 >> 172.21.15.22:0/1957160539 conn(0x560db7d438d0 msgr2=0x560db7d43cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 -- 172.21.15.22:0/1957160539 shutdown_connections 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 -- 172.21.15.22:0/1957160539 wait complete. 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 -- start start 2024-03-09T00:12:26.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 Processor -- start 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d53a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d49770 0x560db7d53f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d56ab0 0x560db7d544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db7c4b4a0 con 0x560db7d48e50 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db7ca8f60 con 0x560db7d49770 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db7d598f0 con 0x560db7d56ab0 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f1217821740 10 client.0 _ll_register_callbacks cb 0x560db7ca6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:26.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d53a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d53a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53510/0 (socket says 172.21.15.22:53510) 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f120e59c640 1 -- 172.21.15.22:0/2844170438 learned_addr learned my addr 172.21.15.22:0/2844170438 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f120dd9b640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d49770 0x560db7d53f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f120ed9d640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d56ab0 0x560db7d544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f120e59c640 1 -- 172.21.15.22:0/2844170438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d56ab0 msgr2=0x560db7d544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.978+0000 7f120e59c640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d56ab0 0x560db7d544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f120e59c640 1 -- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d49770 msgr2=0x560db7d53f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f120e59c640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d49770 0x560db7d53f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f120e59c640 1 -- 172.21.15.22:0/2844170438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db79c13f0 con 0x560db7d48e50 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f120ed9d640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d56ab0 0x560db7d544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f120e59c640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d53a50 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f12080a34a0 tx=0x7f1208000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f31b74fd54203f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f11f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f11f5ffb640 1 -- 172.21.15.22:0/2844170438 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208020040 con 0x560db7d48e50 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f11f5ffb640 1 -- 172.21.15.22:0/2844170438 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f12080bb0d0 con 0x560db7d48e50 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560db7cba400 con 0x560db7d48e50 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f11f5ffb640 1 -- 172.21.15.22:0/2844170438 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f120809b230 con 0x560db7d48e50 2024-03-09T00:12:26.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 10 client.20976 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x560db7a8b7d0 con 0x560db7d48e50 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f11f5ffb640 1 -- 172.21.15.22:0/2844170438 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560db7cba400 con 0x560db7d48e50 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f11f5ffb640 1 -- 172.21.15.22:0/2844170438 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f12080b5070 con 0x560db7d48e50 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 10 client.20976 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 10 client.20976 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 20 client.20976 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7d66aa0 0x560db7d68e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 4 client.20976 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:12:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.979+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560db7cb4750 con 0x560db7d66aa0 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.980+0000 7f120dd9b640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7d66aa0 0x560db7d68e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.980+0000 7f11f5ffb640 10 client.20976 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.980+0000 7f120dd9b640 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7d66aa0 0x560db7d68e90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5157a35cdbef29eb server_cookie=4b230c7c27866383 in_seq=0 out_seq=0 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f11f5ffb640 1 -- 172.21.15.22:0/2844170438 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560db7cb4750 con 0x560db7d66aa0 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f11f5ffb640 10 client.20976 handle_command_reply: tid=0 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 client.20976 shutdown 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 msgr2=0x560db7d53a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:26.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d53a50 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f12080a34a0 tx=0x7f1208000bc0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 shutdown_connections 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7d66aa0 0x560db7d68e90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d56ab0 0x560db7d544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d49770 0x560db7d53f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f11f5ffb640 0 client.20976 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 --2- 172.21.15.22:0/2844170438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d53a50 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 >> 172.21.15.22:0/2844170438 conn(0x560db7d438d0 msgr2=0x560db7d45620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 shutdown_connections 2024-03-09T00:12:26.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 1 -- 172.21.15.22:0/2844170438 wait complete. 2024-03-09T00:12:26.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.981+0000 7f1217821740 20 client.20976 trim_cache size 0 max 16384 2024-03-09T00:12:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 Processor -- start 2024-03-09T00:12:26.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 -- start start 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d48e50 0x560db7d56180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d49770 0x560db7d50cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101aea60 con 0x560db7d49770 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101ac600 con 0x560db7d56ab0 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f1217821740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101aa6c0 con 0x560db7d48e50 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f120dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d49770 0x560db7d50cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f120dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d49770 0x560db7d50cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53526/0 (socket says 172.21.15.22:53526) 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.993+0000 7f120dd9b640 1 -- 172.21.15.22:0/1822137609 learned_addr learned my addr 172.21.15.22:0/1822137609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120dd9b640 1 -- 172.21.15.22:0/1822137609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d48e50 msgr2=0x560db7d56180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120dd9b640 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d48e50 0x560db7d56180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120dd9b640 1 -- 172.21.15.22:0/1822137609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 msgr2=0x560db7d511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120ed9d640 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120dd9b640 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120dd9b640 1 -- 172.21.15.22:0/1822137609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db7d6dd10 con 0x560db7d49770 2024-03-09T00:12:26.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120ed9d640 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f120dd9b640 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d49770 0x560db7d50cb0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f11f802dfd0 tx=0x7f11f802c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee5de236f21c57a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f11f7fff640 1 -- 172.21.15.22:0/1822137609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f802caf0 con 0x560db7d49770 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f11f7fff640 1 -- 172.21.15.22:0/1822137609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11f802cc90 con 0x560db7d49770 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.994+0000 7f11f7fff640 1 -- 172.21.15.22:0/1822137609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8004920 con 0x560db7d49770 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- 172.21.15.22:0/1822137609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d49770 msgr2=0x560db7d50cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d49770 0x560db7d50cb0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f11f802dfd0 tx=0x7f11f802c790 comp rx=0 tx=0).stop 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- 172.21.15.22:0/1822137609 shutdown_connections 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d49770 0x560db7d50cb0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 --2- 172.21.15.22:0/1822137609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d48e50 0x560db7d56180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- 172.21.15.22:0/1822137609 >> 172.21.15.22:0/1822137609 conn(0x560db7d438d0 msgr2=0x560db7d4ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- 172.21.15.22:0/1822137609 shutdown_connections 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- 172.21.15.22:0/1822137609 wait complete. 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- start start 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 Processor -- start 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d5dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d5c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d5c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101aacc0 con 0x560db7d48e50 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101abfa0 con 0x560db7d56ab0 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101ab2e0 con 0x560db7d49770 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f1217821740 10 client.0 _ll_register_callbacks cb 0x7f12080974d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d5dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f120e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d5dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53536/0 (socket says 172.21.15.22:53536) 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.995+0000 7f120e59c640 1 -- 172.21.15.22:0/1285270130 learned_addr learned my addr 172.21.15.22:0/1285270130 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:27.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f120e59c640 1 -- 172.21.15.22:0/1285270130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 msgr2=0x560db7d5c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f120e59c640 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d5c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f120e59c640 1 -- 172.21.15.22:0/1285270130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 msgr2=0x560db7d5c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f120e59c640 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d5c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f120e59c640 1 -- 172.21.15.22:0/1285270130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560db7d5cd20 con 0x560db7d48e50 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f120e59c640 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d5dd80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f120809b1c0 tx=0x7f1208005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c13e10c09e9f8fc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f11f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f11f5ffb640 1 -- 172.21.15.22:0/1285270130 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208020070 con 0x560db7d48e50 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f11f5ffb640 1 -- 172.21.15.22:0/1285270130 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1208019310 con 0x560db7d48e50 2024-03-09T00:12:27.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11f802cc90 con 0x560db7d48e50 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f11f5ffb640 1 -- 172.21.15.22:0/1285270130 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12080b40d0 con 0x560db7d48e50 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 10 client.20982 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x560db7dbda70 con 0x560db7d48e50 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f11f5ffb640 1 -- 172.21.15.22:0/1285270130 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11f802cc90 con 0x560db7d48e50 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f11f5ffb640 1 -- 172.21.15.22:0/1285270130 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f12080b5040 con 0x560db7d48e50 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 10 client.20982 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 10 client.20982 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 20 client.20982 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:27.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7dbe000 0x560db7dc0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 4 client.20982 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.996+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 --> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560db7c32640 con 0x560db7dbe000 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f120dd9b640 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7dbe000 0x560db7dc0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f120dd9b640 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7dbe000 0x560db7dc0440 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5884beb283f5d87e server_cookie=23f259be67fad317 in_seq=0 out_seq=0 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f11f5ffb640 10 client.20982 ms_handle_connect on v2:172.21.15.62:6838/2804197530 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f11f5ffb640 1 -- 172.21.15.22:0/1285270130 <== mds.1 v2:172.21.15.62:6838/2804197530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560db7c32640 con 0x560db7dbe000 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f11f5ffb640 10 client.20982 handle_command_reply: tid=0 2024-03-09T00:12:27.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 client.20982 shutdown 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 msgr2=0x560db7d5dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d5dd80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f120809b1c0 tx=0x7f1208005880 comp rx=0 tx=0).stop 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 shutdown_connections 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.62:6838/2804197530,v1:172.21.15.62:6839/2804197530] conn(0x560db7dbe000 0x560db7dc0440 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560db7d56ab0 0x560db7d5c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560db7d49770 0x560db7d5c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f11f5ffb640 0 client.20982 ms_handle_reset on v2:172.21.15.62:6838/2804197530 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 --2- 172.21.15.22:0/1285270130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560db7d48e50 0x560db7d5dd80 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 >> 172.21.15.22:0/1285270130 conn(0x560db7d438d0 msgr2=0x560db7d52320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.997+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 shutdown_connections 2024-03-09T00:12:27.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1217821740 1 -- 172.21.15.22:0/1285270130 wait complete. 2024-03-09T00:12:27.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1217821740 20 client.20982 trim_cache size 0 max 16384 2024-03-09T00:12:27.007 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f11fc0a2970 msgr2=0x7f11fc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f11fc0a2970 0x7f11fc0a4e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1200005fd0 tx=0x7f12000139c0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12100ffa60 msgr2=0x7f121019d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12100ffa60 0x7f121019d790 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f120800d950 tx=0x7f120800de10 comp rx=0 tx=0).stop 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 shutdown_connections 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f11fc0a2970 0x7f11fc0a4e30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1210197940 0x7f1210197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210106670 0x7f121019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 --2- 172.21.15.22:0/966389115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12100ffa60 0x7f121019d790 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 >> 172.21.15.22:0/966389115 conn(0x7f1210076720 msgr2=0x7f12100fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 shutdown_connections 2024-03-09T00:12:27.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:26.998+0000 7f1214c44640 1 -- 172.21.15.22:0/966389115 wait complete. 2024-03-09T00:12:27.010 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-09T00:12:27.010 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-09T00:12:27.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 -- 172.21.15.22:0/1896762819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c100e00 msgr2=0x7fb10c10cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 --2- 172.21.15.22:0/1896762819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c100e00 0x7fb10c10cff0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb108009ca0 tx=0x7fb10802c140 comp rx=0 tx=0).stop 2024-03-09T00:12:27.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 -- 172.21.15.22:0/1896762819 shutdown_connections 2024-03-09T00:12:27.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 --2- 172.21.15.22:0/1896762819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c100e00 0x7fb10c10cff0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 --2- 172.21.15.22:0/1896762819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb10c100440 0x7fb10c1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 --2- 172.21.15.22:0/1896762819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c1062c0 0x7fb10c1066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 -- 172.21.15.22:0/1896762819 >> 172.21.15.22:0/1896762819 conn(0x7fb10c076fc0 msgr2=0x7fb10c0773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 -- 172.21.15.22:0/1896762819 shutdown_connections 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.118+0000 7fb113e75640 1 -- 172.21.15.22:0/1896762819 wait complete. 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 Processor -- start 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 -- start start 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c100440 0x7fb10c196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb10c100e00 0x7fb10c196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c1062c0 0x7fb10c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1105a0 con 0x7fb10c1062c0 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c110420 con 0x7fb10c100e00 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb113e75640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c110720 con 0x7fb10c100440 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb112412640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c1062c0 0x7fb10c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb112412640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c1062c0 0x7fb10c197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53550/0 (socket says 172.21.15.22:53550) 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb112412640 1 -- 172.21.15.22:0/3101873826 learned_addr learned my addr 172.21.15.22:0/3101873826 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.119+0000 7fb112412640 1 -- 172.21.15.22:0/3101873826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c100440 msgr2=0x7fb10c196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb112412640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c100440 0x7fb10c196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb112412640 1 -- 172.21.15.22:0/3101873826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb10c100e00 msgr2=0x7fb10c196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb112412640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb10c100e00 0x7fb10c196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb112412640 1 -- 172.21.15.22:0/3101873826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c1a5f00 con 0x7fb10c1062c0 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb112412640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c1062c0 0x7fb10c197110 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb10802c750 tx=0x7fb108035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb102ffd640 1 -- 172.21.15.22:0/3101873826 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108039040 con 0x7fb10c1062c0 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb102ffd640 1 -- 172.21.15.22:0/3101873826 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb108044070 con 0x7fb10c1062c0 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb102ffd640 1 -- 172.21.15.22:0/3101873826 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1080044e0 con 0x7fb10c1062c0 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c1a6190 con 0x7fb10c1062c0 2024-03-09T00:12:27.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.120+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c1a66d0 con 0x7fb10c1062c0 2024-03-09T00:12:27.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.122+0000 7fb102ffd640 1 -- 172.21.15.22:0/3101873826 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb108004680 con 0x7fb10c1062c0 2024-03-09T00:12:27.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.122+0000 7fb116a4b740 1 Processor -- start 2024-03-09T00:12:27.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb116a4b740 1 -- start start 2024-03-09T00:12:27.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb116a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568909b1360 con 0x5568909eaac0 2024-03-09T00:12:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb116a4b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689094dec0 con 0x5568909ea1a0 2024-03-09T00:12:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb116a4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568908f6d30 con 0x5568909f36a0 2024-03-09T00:12:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb102ffd640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb0f80a29f0 0x7fb0f80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53564/0 (socket says 172.21.15.22:53564) 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 -- 172.21.15.22:0/4048934384 learned_addr learned my addr 172.21.15.22:0/4048934384 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb112412640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb0f80a29f0 0x7fb0f80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111410640 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb102ffd640 1 -- 172.21.15.22:0/3101873826 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fb1080f1830 con 0x7fb10c1062c0 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111410640 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 -- 172.21.15.22:0/4048934384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 msgr2=0x5568909f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 -- 172.21.15.22:0/4048934384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 msgr2=0x5568909ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909ea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111c11640 1 -- 172.21.15.22:0/4048934384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568908f5bc0 con 0x5568909eaac0 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.123+0000 7fb111410640 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb111c11640 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f3160 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb10400bb10 tx=0x7fb10400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de01a6d104da9fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb112412640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb0f80a29f0 0x7fb0f80a4eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1080bd8b0 tx=0x7fb10808d680 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb0f7fff640 1 -- 172.21.15.22:0/4048934384 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104016070 con 0x5568909eaac0 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb0f7fff640 1 -- 172.21.15.22:0/4048934384 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb104005500 con 0x5568909eaac0 2024-03-09T00:12:27.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb0f7fff640 1 -- 172.21.15.22:0/4048934384 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104010460 con 0x5568909eaac0 2024-03-09T00:12:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb116a4b740 1 -- 172.21.15.22:0/4048934384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 msgr2=0x5568909f3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb116a4b740 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f3160 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb10400bb10 tx=0x7fb10400d2e0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb116a4b740 1 -- 172.21.15.22:0/4048934384 shutdown_connections 2024-03-09T00:12:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb116a4b740 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb116a4b740 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f3160 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb116a4b740 1 --2- 172.21.15.22:0/4048934384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.124+0000 7fb116a4b740 1 -- 172.21.15.22:0/4048934384 >> 172.21.15.22:0/4048934384 conn(0x5568909e48d0 msgr2=0x5568909e4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 -- 172.21.15.22:0/4048934384 shutdown_connections 2024-03-09T00:12:27.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 -- 172.21.15.22:0/4048934384 wait complete. 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 -- start start 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 Processor -- start 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909ea1a0 0x5568909f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909f36a0 0x5568909f1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568908ec4a0 con 0x5568909eaac0 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556890949f60 con 0x5568909f36a0 2024-03-09T00:12:27.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568909f6490 con 0x5568909ea1a0 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb116a4b740 10 client.0 _ll_register_callbacks cb 0x556890947d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb111c11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb111c11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53570/0 (socket says 172.21.15.22:53570) 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.125+0000 7fb111c11640 1 -- 172.21.15.22:0/3750466589 learned_addr learned my addr 172.21.15.22:0/3750466589 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb111410640 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909ea1a0 0x5568909f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb111c11640 1 -- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909ea1a0 msgr2=0x5568909f06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb111c11640 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909ea1a0 0x5568909f06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb111c11640 1 -- 172.21.15.22:0/3750466589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909f36a0 msgr2=0x5568909f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb111c11640 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909f36a0 0x5568909f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb111c11640 1 -- 172.21.15.22:0/3750466589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568906623f0 con 0x5568909eaac0 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb111c11640 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f0c10 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb104017000 tx=0x7fb104005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafcd331ad2b1392 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb0f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb0f57fa640 1 -- 172.21.15.22:0/3750466589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104016030 con 0x5568909eaac0 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb0f57fa640 1 -- 172.21.15.22:0/3750466589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fb104005200 con 0x5568909eaac0 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb104005500 con 0x5568909eaac0 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb0f57fa640 1 -- 172.21.15.22:0/3750466589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb10402d050 con 0x5568909eaac0 2024-03-09T00:12:27.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb116a4b740 10 client.20994 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55689072c7d0 con 0x5568909eaac0 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.126+0000 7fb0f57fa640 1 -- 172.21.15.22:0/3750466589 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb104005500 con 0x5568909eaac0 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb0f57fa640 1 -- 172.21.15.22:0/3750466589 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb104003830 con 0x5568909eaac0 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb116a4b740 10 client.20994 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb116a4b740 10 client.20994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb116a4b740 20 client.20994 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb116a4b740 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a03860 0x556890a05c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb116a4b740 4 client.20994 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T00:12:27.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55689095b400 con 0x556890a03860 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb111410640 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a03860 0x556890a05c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb0f57fa640 10 client.20994 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.127+0000 7fb111410640 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a03860 0x556890a05c50 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9ffeb6bb9a659c server_cookie=e01b94a6e50ff0e in_seq=0 out_seq=0 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb0f57fa640 1 -- 172.21.15.22:0/3750466589 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55689095b400 con 0x556890a03860 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb0f57fa640 10 client.20994 handle_command_reply: tid=0 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 client.20994 shutdown 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 msgr2=0x5568909f0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f0c10 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb104017000 tx=0x7fb104005780 comp rx=0 tx=0).stop 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 shutdown_connections 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a03860 0x556890a05c50 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909f36a0 0x5568909f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x5568909f0c10 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb0f57fa640 0 client.20994 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 --2- 172.21.15.22:0/3750466589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909ea1a0 0x5568909f06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 >> 172.21.15.22:0/3750466589 conn(0x5568909e48d0 msgr2=0x5568909e6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 shutdown_connections 2024-03-09T00:12:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.128+0000 7fb116a4b740 1 -- 172.21.15.22:0/3750466589 wait complete. 2024-03-09T00:12:27.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.129+0000 7fb116a4b740 20 client.20994 trim_cache size 0 max 16384 2024-03-09T00:12:27.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 Processor -- start 2024-03-09T00:12:27.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 -- start start 2024-03-09T00:12:27.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x556890a0a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909ed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1b2b70 con 0x5568909eaac0 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1b0710 con 0x5568909ea1a0 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.140+0000 7fb116a4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1ae7d0 con 0x5568909f36a0 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x556890a0a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x556890a0a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53572/0 (socket says 172.21.15.22:53572) 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 -- 172.21.15.22:0/2878983103 learned_addr learned my addr 172.21.15.22:0/2878983103 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 -- 172.21.15.22:0/2878983103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 msgr2=0x5568909ed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111410640 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909ed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 -- 172.21.15.22:0/2878983103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 msgr2=0x5568909f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 -- 172.21.15.22:0/2878983103 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568909ea8f0 con 0x5568909eaac0 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111410640 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb111c11640 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x556890a0a2d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fb104004cc0 tx=0x7fb104024470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89ee14ccb3aa3e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb0f77fe640 1 -- 172.21.15.22:0/2878983103 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104035020 con 0x5568909eaac0 2024-03-09T00:12:27.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb0f77fe640 1 -- 172.21.15.22:0/2878983103 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb10401f070 con 0x5568909eaac0 2024-03-09T00:12:27.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.141+0000 7fb0f77fe640 1 -- 172.21.15.22:0/2878983103 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104016040 con 0x5568909eaac0 2024-03-09T00:12:27.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- 172.21.15.22:0/2878983103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 msgr2=0x556890a0a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x556890a0a2d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fb104004cc0 tx=0x7fb104024470 comp rx=0 tx=0).stop 2024-03-09T00:12:27.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- 172.21.15.22:0/2878983103 shutdown_connections 2024-03-09T00:12:27.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x5568909ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909eaac0 0x556890a0a2d0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2878983103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909ea1a0 0x5568909f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- 172.21.15.22:0/2878983103 >> 172.21.15.22:0/2878983103 conn(0x5568909e48d0 msgr2=0x5568909e4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- 172.21.15.22:0/2878983103 shutdown_connections 2024-03-09T00:12:27.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- 172.21.15.22:0/2878983103 wait complete. 2024-03-09T00:12:27.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- start start 2024-03-09T00:12:27.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 Processor -- start 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909ea1a0 0x5568909f9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909eaac0 0x5568909f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x556890a59a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1aedd0 con 0x5568909ea1a0 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1b00b0 con 0x5568909eaac0 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1af3f0 con 0x5568909f36a0 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb116a4b740 10 client.0 _ll_register_callbacks cb 0x7fb10400b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T00:12:27.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb111410640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909ea1a0 0x5568909f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.142+0000 7fb111410640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909ea1a0 0x5568909f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53586/0 (socket says 172.21.15.22:53586) 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb111410640 1 -- 172.21.15.22:0/2899352192 learned_addr learned my addr 172.21.15.22:0/2899352192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb111410640 1 -- 172.21.15.22:0/2899352192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 msgr2=0x556890a59a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb111410640 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x556890a59a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb111410640 1 -- 172.21.15.22:0/2899352192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909eaac0 msgr2=0x5568909f9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb111410640 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909eaac0 0x5568909f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb111410640 1 -- 172.21.15.22:0/2899352192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556890a5a020 con 0x5568909ea1a0 2024-03-09T00:12:27.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb111410640 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909ea1a0 0x5568909f9800 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x556890a06280 tx=0x7fb0fc018ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4d74c0c525a53f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb0f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb0f57fa640 1 -- 172.21.15.22:0/2899352192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0187f0 con 0x5568909ea1a0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb0f57fa640 1 -- 172.21.15.22:0/2899352192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fb0fc004020 con 0x5568909ea1a0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb10401f070 con 0x5568909ea1a0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb0f57fa640 1 -- 172.21.15.22:0/2899352192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0fc004220 con 0x5568909ea1a0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb116a4b740 10 client.21000 fetch_fsmap learned FSMap version 37 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556890a5b2d0 con 0x5568909ea1a0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.143+0000 7fb0f57fa640 1 -- 172.21.15.22:0/2899352192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb10401f070 con 0x5568909ea1a0 2024-03-09T00:12:27.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb0f57fa640 1 -- 172.21.15.22:0/2899352192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 37) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb0fc01f030 con 0x5568909ea1a0 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 10 client.21000 fetch_fsmap finished waiting for FSMap version 37 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 10 client.21000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 20 client.21000 populate_metadata read hostname 'smithi022' 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a5b9a0 0x556890a5ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 4 client.21000 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 --> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556890955750 con 0x556890a5b9a0 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb111c11640 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a5b9a0 0x556890a5ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:27.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb0f57fa640 10 client.21000 ms_handle_connect on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb111c11640 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a5b9a0 0x556890a5ddc0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e94b1d60e2e9e7 server_cookie=e726d8e5323ba06f in_seq=0 out_seq=0 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb0f57fa640 1 -- 172.21.15.22:0/2899352192 <== mds.0 v2:172.21.15.192:6838/1126404206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556890955750 con 0x556890a5b9a0 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb0f57fa640 10 client.21000 handle_command_reply: tid=0 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 1 client.21000 shutdown 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909ea1a0 msgr2=0x5568909f9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.144+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909ea1a0 0x5568909f9800 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x556890a06280 tx=0x7fb0fc018ed0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 shutdown_connections 2024-03-09T00:12:27.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.192:6838/1126404206,v1:172.21.15.192:6839/1126404206] conn(0x556890a5b9a0 0x556890a5ddc0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5568909f36a0 0x556890a59a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568909eaac0 0x5568909f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb0f57fa640 0 client.21000 ms_handle_reset on v2:172.21.15.192:6838/1126404206 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 --2- 172.21.15.22:0/2899352192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5568909ea1a0 0x5568909f9800 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 >> 172.21.15.22:0/2899352192 conn(0x5568909e48d0 msgr2=0x556890a0a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 shutdown_connections 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 1 -- 172.21.15.22:0/2899352192 wait complete. 2024-03-09T00:12:27.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb116a4b740 20 client.21000 trim_cache size 0 max 16384 2024-03-09T00:12:27.157 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-03-09T00:12:27.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb0f80a29f0 msgr2=0x7fb0f80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb0f80a29f0 0x7fb0f80a4eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1080bd8b0 tx=0x7fb10808d680 comp rx=0 tx=0).stop 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c1062c0 msgr2=0x7fb10c197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c1062c0 0x7fb10c197110 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb10802c750 tx=0x7fb108035d90 comp rx=0 tx=0).stop 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 shutdown_connections 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb0f80a29f0 0x7fb0f80a4eb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb10c1062c0 0x7fb10c197110 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb10c100e00 0x7fb10c196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 --2- 172.21.15.22:0/3101873826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb10c100440 0x7fb10c196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 >> 172.21.15.22:0/3101873826 conn(0x7fb10c076fc0 msgr2=0x7fb10c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:27.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 shutdown_connections 2024-03-09T00:12:27.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:27.145+0000 7fb113e75640 1 -- 172.21.15.22:0/3101873826 wait complete. 2024-03-09T00:12:27.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:27 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4203838779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:12:27.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:27 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4203838779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:12:27.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:27 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4203838779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:12:28.161 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-09T00:12:28.161 INFO:tasks.fwd_scrub:done joining 2024-03-09T00:12:28.161 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-09T00:12:28.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T00:12:28.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 -- 172.21.15.22:0/1291877745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102520 msgr2=0x7f77e8102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 --2- 172.21.15.22:0/1291877745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102520 0x7f77e8102900 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009900 tx=0x7f77d802bcd0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 -- 172.21.15.22:0/1291877745 shutdown_connections 2024-03-09T00:12:28.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 --2- 172.21.15.22:0/1291877745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77e810b8f0 0x7f77e810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 --2- 172.21.15.22:0/1291877745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8102e40 0x7f77e810b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 --2- 172.21.15.22:0/1291877745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102520 0x7f77e8102900 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 -- 172.21.15.22:0/1291877745 >> 172.21.15.22:0/1291877745 conn(0x7f77e8076570 msgr2=0x7f77e8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.293+0000 7f77ed861640 1 -- 172.21.15.22:0/1291877745 shutdown_connections 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 -- 172.21.15.22:0/1291877745 wait complete. 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 Processor -- start 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 -- start start 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77e8102520 0x7f77e819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102e40 0x7f77e819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e810b8f0 0x7f77e81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8111690 con 0x7f77e8102e40 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8111510 con 0x7f77e8102520 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.294+0000 7f77ed861640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8111810 con 0x7f77e810b8f0 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102e40 0x7f77e819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102e40 0x7f77e819dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53592/0 (socket says 172.21.15.22:53592) 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 -- 172.21.15.22:0/1318719515 learned_addr learned my addr 172.21.15.22:0/1318719515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e77fe640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e810b8f0 0x7f77e81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e6ffd640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77e8102520 0x7f77e819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 -- 172.21.15.22:0/1318719515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e810b8f0 msgr2=0x7f77e81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e810b8f0 0x7f77e81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 -- 172.21.15.22:0/1318719515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77e8102520 msgr2=0x7f77e819d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77e8102520 0x7f77e819d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 -- 172.21.15.22:0/1318719515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8198170 con 0x7f77e8102e40 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e6ffd640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77e8102520 0x7f77e819d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77e67fc640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102e40 0x7f77e819dd80 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00d770 tx=0x7f77dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77ec85f640 1 -- 172.21.15.22:0/1318719515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc017070 con 0x7f77e8102e40 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e8198460 con 0x7f77e8102e40 2024-03-09T00:12:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77ec85f640 1 -- 172.21.15.22:0/1318719515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77dc005500 con 0x7f77e8102e40 2024-03-09T00:12:28.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77ec85f640 1 -- 172.21.15.22:0/1318719515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc004e00 con 0x7f77e8102e40 2024-03-09T00:12:28.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.295+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e81a4700 con 0x7f77e8102e40 2024-03-09T00:12:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.296+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e806a140 con 0x7f77e8102e40 2024-03-09T00:12:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.297+0000 7f77ec85f640 1 -- 172.21.15.22:0/1318719515 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f77dc004fa0 con 0x7f77e8102e40 2024-03-09T00:12:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.298+0000 7f77ec85f640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77d00a2970 0x7f77d00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.298+0000 7f77e6ffd640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77d00a2970 0x7f77d00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.298+0000 7f77ec85f640 1 -- 172.21.15.22:0/1318719515 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f77dc0ced40 con 0x7f77e8102e40 2024-03-09T00:12:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.299+0000 7f77e6ffd640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77d00a2970 0x7f77d00a4e30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006010 tx=0x7f77d8005a20 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.303+0000 7f77ec85f640 1 -- 172.21.15.22:0/1318719515 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f77dc09b6b0 con 0x7f77e8102e40 2024-03-09T00:12:28.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:28 smithi022 ceph-mon[31894]: pgmap v572: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.3 MiB/s wr, 211 op/s 2024-03-09T00:12:28.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.491+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77e8106450 con 0x7f77e8102e40 2024-03-09T00:12:28.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.493+0000 7f77ec85f640 1 -- 172.21.15.22:0/1318719515 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+25258 (secure 0 0 0) 0x7f77dc09afb0 con 0x7f77e8102e40 2024-03-09T00:12:28.490 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:12:28.490 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":126,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-09T00:12:24.039477+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3461287105":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/1632997532":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3477373324":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1865576829":"2024-03-09T23:54:32.857800+0000","172.21.15.22:6800/2813827512":"2024-03-09T23:54:32.857800+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1683058859":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/2813827512":"2024-03-09T23:54:32.857800+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":30,"length":1}]},{"pool":4,"snaps":[{"begin":17,"length":1},{"begin":30,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77d00a2970 msgr2=0x7f77d00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77d00a2970 0x7f77d00a4e30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006010 tx=0x7f77d8005a20 comp rx=0 tx=0).stop 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102e40 msgr2=0x7f77e819dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102e40 0x7f77e819dd80 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00d770 tx=0x7f77dc00dc30 comp rx=0 tx=0).stop 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 shutdown_connections 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f77d00a2970 0x7f77d00a4e30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e810b8f0 0x7f77e81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77e8102e40 0x7f77e819dd80 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 --2- 172.21.15.22:0/1318719515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77e8102520 0x7f77e819d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 >> 172.21.15.22:0/1318719515 conn(0x7f77e8076570 msgr2=0x7f77e80fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 shutdown_connections 2024-03-09T00:12:28.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.495+0000 7f77ed861640 1 -- 172.21.15.22:0/1318719515 wait complete. 2024-03-09T00:12:28.503 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-09T00:12:28.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T00:12:28.533 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:28 smithi192 ceph-mon[35995]: pgmap v572: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.3 MiB/s wr, 211 op/s 2024-03-09T00:12:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 -- 172.21.15.22:0/1633491350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc0690f0 msgr2=0x7f8ddc105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 --2- 172.21.15.22:0/1633491350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc0690f0 0x7f8ddc105ec0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009900 tx=0x7f8dcc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:12:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 -- 172.21.15.22:0/1633491350 shutdown_connections 2024-03-09T00:12:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 --2- 172.21.15.22:0/1633491350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ddc1065a0 0x7f8ddc10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 --2- 172.21.15.22:0/1633491350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc0690f0 0x7f8ddc105ec0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 --2- 172.21.15.22:0/1633491350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ddc068740 0x7f8ddc068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 -- 172.21.15.22:0/1633491350 >> 172.21.15.22:0/1633491350 conn(0x7f8ddc0772f0 msgr2=0x7f8ddc077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 -- 172.21.15.22:0/1633491350 shutdown_connections 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 -- 172.21.15.22:0/1633491350 wait complete. 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 Processor -- start 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.613+0000 7f8de37a9640 1 -- start start 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de37a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ddc068740 0x7f8ddc1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de37a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ddc1065a0 0x7f8ddc1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de37a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc100c30 0x7f8ddc1a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de37a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc111630 con 0x7f8ddc100c30 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de37a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1114b0 con 0x7f8ddc068740 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de37a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1117b0 con 0x7f8ddc1065a0 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc100c30 0x7f8ddc1a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc100c30 0x7f8ddc1a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53612/0 (socket says 172.21.15.22:53612) 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 -- 172.21.15.22:0/662153277 learned_addr learned my addr 172.21.15.22:0/662153277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1545640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ddc068740 0x7f8ddc1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de0d44640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ddc1065a0 0x7f8ddc1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 -- 172.21.15.22:0/662153277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ddc1065a0 msgr2=0x7f8ddc1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ddc1065a0 0x7f8ddc1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 -- 172.21.15.22:0/662153277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ddc068740 msgr2=0x7f8ddc1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ddc068740 0x7f8ddc1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 -- 172.21.15.22:0/662153277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc1a63c0 con 0x7f8ddc100c30 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1545640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ddc068740 0x7f8ddc1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de1d46640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc100c30 0x7f8ddc1a5cc0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400cad0 tx=0x7f8dd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8dd27fc640 1 -- 172.21.15.22:0/662153277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400ae20 con 0x7f8ddc100c30 2024-03-09T00:12:28.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8dd27fc640 1 -- 172.21.15.22:0/662153277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8dd4005020 con 0x7f8ddc100c30 2024-03-09T00:12:28.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc1a66b0 con 0x7f8ddc100c30 2024-03-09T00:12:28.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.614+0000 7f8dd27fc640 1 -- 172.21.15.22:0/662153277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400f6f0 con 0x7f8ddc100c30 2024-03-09T00:12:28.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.615+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc1a6bd0 con 0x7f8ddc100c30 2024-03-09T00:12:28.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.615+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc0680e0 con 0x7f8ddc100c30 2024-03-09T00:12:28.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:28 smithi062 ceph-mon[36129]: pgmap v572: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.3 MiB/s wr, 211 op/s 2024-03-09T00:12:28.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.616+0000 7f8dd27fc640 1 -- 172.21.15.22:0/662153277 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8dd40051d0 con 0x7f8ddc100c30 2024-03-09T00:12:28.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.617+0000 7f8dd27fc640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8dc80a29a0 0x7f8dc80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.617+0000 7f8de1545640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8dc80a29a0 0x7f8dc80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.617+0000 7f8dd27fc640 1 -- 172.21.15.22:0/662153277 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f8dd40cf2f0 con 0x7f8ddc100c30 2024-03-09T00:12:28.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.617+0000 7f8de1545640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8dc80a29a0 0x7f8dc80a4e60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800abe0 tx=0x7f8dd800a5c0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:28.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.621+0000 7f8dd27fc640 1 -- 172.21.15.22:0/662153277 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8dd4014030 con 0x7f8ddc100c30 2024-03-09T00:12:28.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.804+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ddc0690f0 con 0x7f8ddc100c30 2024-03-09T00:12:28.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.805+0000 7f8dd27fc640 1 -- 172.21.15.22:0/662153277 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dd409bc00 con 0x7f8ddc100c30 2024-03-09T00:12:28.802 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8dc80a29a0 msgr2=0x7f8dc80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8dc80a29a0 0x7f8dc80a4e60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800abe0 tx=0x7f8dd800a5c0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc100c30 msgr2=0x7f8ddc1a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc100c30 0x7f8ddc1a5cc0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400cad0 tx=0x7f8dd400cf90 comp rx=0 tx=0).stop 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 shutdown_connections 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f8dc80a29a0 0x7f8dc80a4e60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ddc100c30 0x7f8ddc1a5cc0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ddc1065a0 0x7f8ddc1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 --2- 172.21.15.22:0/662153277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ddc068740 0x7f8ddc1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 >> 172.21.15.22:0/662153277 conn(0x7f8ddc0772f0 msgr2=0x7f8ddc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:28.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 shutdown_connections 2024-03-09T00:12:28.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.807+0000 7f8de37a9640 1 -- 172.21.15.22:0/662153277 wait complete. 2024-03-09T00:12:28.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-09T00:12:28.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3335819367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df81054e0 msgr2=0x7f5df810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3335819367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df81054e0 0x7f5df810d0b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009ca0 tx=0x7f5df002c100 comp rx=0 tx=0).stop 2024-03-09T00:12:28.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3335819367 shutdown_connections 2024-03-09T00:12:28.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3335819367 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df81054e0 0x7f5df810d0b0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3335819367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df8100dc0 0x7f5df8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3335819367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df8100410 0x7f5df81007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3335819367 >> 172.21.15.22:0/3335819367 conn(0x7f5df8076570 msgr2=0x7f5df8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:28.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3335819367 shutdown_connections 2024-03-09T00:12:28.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3335819367 wait complete. 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 Processor -- start 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.936+0000 7f5dfe88c640 1 -- start start 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfe88c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df8100410 0x7f5df8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfe88c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df81054e0 0x7f5df8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfe88c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df8199400 0x7f5df81a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfe88c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81105a0 con 0x7f5df8199400 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfe88c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8110420 con 0x7f5df81054e0 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfe88c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8110720 con 0x7f5df8100410 2024-03-09T00:12:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df8199400 0x7f5df81a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df8199400 0x7f5df81a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53640/0 (socket says 172.21.15.22:53640) 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 -- 172.21.15.22:0/3310612790 learned_addr learned my addr 172.21.15.22:0/3310612790 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5df77fe640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df81054e0 0x7f5df8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5df7fff640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df8100410 0x7f5df8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 -- 172.21.15.22:0/3310612790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df8100410 msgr2=0x7f5df8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df8100410 0x7f5df8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 -- 172.21.15.22:0/3310612790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df81054e0 msgr2=0x7f5df8198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:28.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df81054e0 0x7f5df8198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 -- 172.21.15.22:0/3310612790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df81a4180 con 0x7f5df8199400 2024-03-09T00:12:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.937+0000 7f5dfce29640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df8199400 0x7f5df81a3a00 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5df002c710 tx=0x7f5df0035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.938+0000 7f5df57fa640 1 -- 172.21.15.22:0/3310612790 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0039040 con 0x7f5df8199400 2024-03-09T00:12:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.938+0000 7f5df57fa640 1 -- 172.21.15.22:0/3310612790 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5df0044070 con 0x7f5df8199400 2024-03-09T00:12:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.938+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df81a4410 con 0x7f5df8199400 2024-03-09T00:12:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.938+0000 7f5df57fa640 1 -- 172.21.15.22:0/3310612790 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0004330 con 0x7f5df8199400 2024-03-09T00:12:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.938+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df81a4950 con 0x7f5df8199400 2024-03-09T00:12:28.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.939+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df81018e0 con 0x7f5df8199400 2024-03-09T00:12:28.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.940+0000 7f5df57fa640 1 -- 172.21.15.22:0/3310612790 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5df00044d0 con 0x7f5df8199400 2024-03-09T00:12:28.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.940+0000 7f5df57fa640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5de00a29f0 0x7f5de00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:28.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.941+0000 7f5df7fff640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5de00a29f0 0x7f5de00a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:28.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.941+0000 7f5df57fa640 1 -- 172.21.15.22:0/3310612790 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f5df00f1690 con 0x7f5df8199400 2024-03-09T00:12:28.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.941+0000 7f5df7fff640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5de00a29f0 0x7f5de00a4eb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5de800abe0 tx=0x7f5de800a2d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:28.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:28.945+0000 7f5df57fa640 1 -- 172.21.15.22:0/3310612790 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5df00be0b0 con 0x7f5df8199400 2024-03-09T00:12:29.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.127+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5df8108710 con 0x7f5df8199400 2024-03-09T00:12:29.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.127+0000 7f5df57fa640 1 -- 172.21.15.22:0/3310612790 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5df00bd9b0 con 0x7f5df8199400 2024-03-09T00:12:29.125 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-09T00:12:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.129+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5de00a29f0 msgr2=0x7f5de00a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.129+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5de00a29f0 0x7f5de00a4eb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5de800abe0 tx=0x7f5de800a2d0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.129+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df8199400 msgr2=0x7f5df81a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.129+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df8199400 0x7f5df81a3a00 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5df002c710 tx=0x7f5df0035d90 comp rx=0 tx=0).stop 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 shutdown_connections 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f5de00a29f0 0x7f5de00a4eb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5df8199400 0x7f5df81a3a00 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df81054e0 0x7f5df8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 --2- 172.21.15.22:0/3310612790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df8100410 0x7f5df8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 >> 172.21.15.22:0/3310612790 conn(0x7f5df8076570 msgr2=0x7f5df80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:29.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 shutdown_connections 2024-03-09T00:12:29.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.130+0000 7f5dfe88c640 1 -- 172.21.15.22:0/3310612790 wait complete. 2024-03-09T00:12:29.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-09T00:12:29.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 -- 172.21.15.22:0/480254093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 msgr2=0x7f689410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 --2- 172.21.15.22:0/480254093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f689410d070 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ca0 tx=0x7f688c02c0f0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 -- 172.21.15.22:0/480254093 shutdown_connections 2024-03-09T00:12:29.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 --2- 172.21.15.22:0/480254093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f689410d070 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 --2- 172.21.15.22:0/480254093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6894100d80 0x7f6894104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 --2- 172.21.15.22:0/480254093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68941003d0 0x7f68941007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 -- 172.21.15.22:0/480254093 >> 172.21.15.22:0/480254093 conn(0x7f68940772f0 msgr2=0x7f6894077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:29.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 -- 172.21.15.22:0/480254093 shutdown_connections 2024-03-09T00:12:29.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.247+0000 7f689a859640 1 -- 172.21.15.22:0/480254093 wait complete. 2024-03-09T00:12:29.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 Processor -- start 2024-03-09T00:12:29.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 -- start start 2024-03-09T00:12:29.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68941003d0 0x7f68941989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f6894198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6894199440 0x7f68941a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894110560 con 0x7f68941054a0 2024-03-09T00:12:29.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941103e0 con 0x7f6894199440 2024-03-09T00:12:29.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f689a859640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941106e0 con 0x7f68941003d0 2024-03-09T00:12:29.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.248+0000 7f68937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f6894198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f6894198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53656/0 (socket says 172.21.15.22:53656) 2024-03-09T00:12:29.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 -- 172.21.15.22:0/211041690 learned_addr learned my addr 172.21.15.22:0/211041690 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f6898df6640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6894199440 0x7f68941a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f6893fff640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68941003d0 0x7f68941989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 -- 172.21.15.22:0/211041690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68941003d0 msgr2=0x7f68941989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68941003d0 0x7f68941989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 -- 172.21.15.22:0/211041690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6894199440 msgr2=0x7f68941a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6894199440 0x7f68941a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 -- 172.21.15.22:0/211041690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68941a41e0 con 0x7f68941054a0 2024-03-09T00:12:29.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f6898df6640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6894199440 0x7f68941a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68937fe640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f6894198f00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f688800c970 tx=0x7f688800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68917fa640 1 -- 172.21.15.22:0/211041690 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888007bb0 con 0x7f68941054a0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68917fa640 1 -- 172.21.15.22:0/211041690 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6888007d50 con 0x7f68941054a0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68941a44d0 con 0x7f68941054a0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f68917fa640 1 -- 172.21.15.22:0/211041690 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880187a0 con 0x7f68941054a0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.249+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68941a49d0 con 0x7f68941054a0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.250+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68940680e0 con 0x7f68941054a0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.251+0000 7f68917fa640 1 -- 172.21.15.22:0/211041690 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f688800f450 con 0x7f68941054a0 2024-03-09T00:12:29.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.252+0000 7f68917fa640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f687c0a29a0 0x7f687c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.252+0000 7f68917fa640 1 -- 172.21.15.22:0/211041690 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f68880cf320 con 0x7f68941054a0 2024-03-09T00:12:29.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.252+0000 7f6893fff640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f687c0a29a0 0x7f687c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.252+0000 7f6893fff640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f687c0a29a0 0x7f687c0a4e60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6884009b60 tx=0x7f6884005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:29.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.256+0000 7f68917fa640 1 -- 172.21.15.22:0/211041690 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6888014030 con 0x7f68941054a0 2024-03-09T00:12:29.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.439+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6894108720 con 0x7f68941054a0 2024-03-09T00:12:29.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.439+0000 7f68917fa640 1 -- 172.21.15.22:0/211041690 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f688809bc30 con 0x7f68941054a0 2024-03-09T00:12:29.436 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-09T00:12:29.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f687c0a29a0 msgr2=0x7f687c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f687c0a29a0 0x7f687c0a4e60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6884009b60 tx=0x7f6884005f90 comp rx=0 tx=0).stop 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 msgr2=0x7f6894198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f6894198f00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f688800c970 tx=0x7f688800ce30 comp rx=0 tx=0).stop 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 shutdown_connections 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f687c0a29a0 0x7f687c0a4e60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6894199440 0x7f68941a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68941054a0 0x7f6894198f00 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 --2- 172.21.15.22:0/211041690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68941003d0 0x7f68941989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.441+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 >> 172.21.15.22:0/211041690 conn(0x7f68940772f0 msgr2=0x7f68940fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:29.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.442+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 shutdown_connections 2024-03-09T00:12:29.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.442+0000 7f689a859640 1 -- 172.21.15.22:0/211041690 wait complete. 2024-03-09T00:12:29.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:29 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1318719515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:12:29.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:29 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/662153277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-09T00:12:29.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:29 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3310612790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-09T00:12:29.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-09T00:12:29.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.557+0000 7fdfb33a9640 1 -- 172.21.15.22:0/1129988550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1003d0 msgr2=0x7fdfac1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.557+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/1129988550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1003d0 0x7fdfac1007b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009900 tx=0x7fdfa802bc30 comp rx=0 tx=0).stop 2024-03-09T00:12:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- 172.21.15.22:0/1129988550 shutdown_connections 2024-03-09T00:12:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/1129988550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1054a0 0x7fdfac10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/1129988550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac100d80 0x7fdfac104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/1129988550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1003d0 0x7fdfac1007b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- 172.21.15.22:0/1129988550 >> 172.21.15.22:0/1129988550 conn(0x7fdfac0772f0 msgr2=0x7fdfac077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- 172.21.15.22:0/1129988550 shutdown_connections 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- 172.21.15.22:0/1129988550 wait complete. 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 Processor -- start 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- start start 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1003d0 0x7fdfac1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac1054a0 0x7fdfac198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1994a0 0x7fdfac1a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac110550 con 0x7fdfac1994a0 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1103d0 con 0x7fdfac1003d0 2024-03-09T00:12:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.558+0000 7fdfb33a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1106d0 con 0x7fdfac1054a0 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1994a0 0x7fdfac1a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1994a0 0x7fdfac1a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53684/0 (socket says 172.21.15.22:53684) 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 -- 172.21.15.22:0/4006810395 learned_addr learned my addr 172.21.15.22:0/4006810395 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1145640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1003d0 0x7fdfac1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb0944640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac1054a0 0x7fdfac198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 -- 172.21.15.22:0/4006810395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac1054a0 msgr2=0x7fdfac198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac1054a0 0x7fdfac198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 -- 172.21.15.22:0/4006810395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1003d0 msgr2=0x7fdfac1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1003d0 0x7fdfac1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 -- 172.21.15.22:0/4006810395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac1a4220 con 0x7fdfac1994a0 2024-03-09T00:12:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb1946640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1994a0 0x7fdfac1a3a50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009fd0 tx=0x7fdfa400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4006810395 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4017070 con 0x7fdfac1994a0 2024-03-09T00:12:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4006810395 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa4003e50 con 0x7fdfac1994a0 2024-03-09T00:12:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac1a4510 con 0x7fdfac1994a0 2024-03-09T00:12:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4006810395 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40106c0 con 0x7fdfac1994a0 2024-03-09T00:12:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.559+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac1a4a10 con 0x7fdfac1994a0 2024-03-09T00:12:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.560+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac0680e0 con 0x7fdfac1994a0 2024-03-09T00:12:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.561+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4006810395 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdfa4003970 con 0x7fdfac1994a0 2024-03-09T00:12:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.562+0000 7fdfa27fc640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.562+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4006810395 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7fdfa40cefe0 con 0x7fdfac1994a0 2024-03-09T00:12:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.562+0000 7fdfb1145640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.562+0000 7fdfb1145640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006010 tx=0x7fdfa8005dd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.566+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4006810395 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdfa409b910 con 0x7fdfac1994a0 2024-03-09T00:12:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:29 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1318719515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:12:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:29 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/662153277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-09T00:12:29.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:29 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3310612790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-09T00:12:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:29 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1318719515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:12:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:29 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/662153277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-09T00:12:29.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:29 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3310612790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-09T00:12:29.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.749+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdfac100d80 con 0x7fdfac1994a0 2024-03-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.749+0000 7fdfa27fc640 1 -- 172.21.15.22:0/4006810395 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v126) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdfa409b210 con 0x7fdfac1994a0 2024-03-09T00:12:29.746 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 msgr2=0x7fdf980a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006010 tx=0x7fdfa8005dd0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1994a0 msgr2=0x7fdfac1a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1994a0 0x7fdfac1a3a50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009fd0 tx=0x7fdfa400b260 comp rx=0 tx=0).stop 2024-03-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 shutdown_connections 2024-03-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdf980a2970 0x7fdf980a4e30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfac1994a0 0x7fdfac1a3a50 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfac1054a0 0x7fdfac198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 --2- 172.21.15.22:0/4006810395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac1003d0 0x7fdfac1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 >> 172.21.15.22:0/4006810395 conn(0x7fdfac0772f0 msgr2=0x7fdfac0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 shutdown_connections 2024-03-09T00:12:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.751+0000 7fdfb33a9640 1 -- 172.21.15.22:0/4006810395 wait complete. 2024-03-09T00:12:29.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 -- 172.21.15.22:0/3912857980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad81054f0 msgr2=0x7f1ad810d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/3912857980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad81054f0 0x7f1ad810d0c0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009ca0 tx=0x7f1ac802c0f0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 -- 172.21.15.22:0/3912857980 shutdown_connections 2024-03-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/3912857980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad81054f0 0x7f1ad810d0c0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/3912857980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ad8100dd0 0x7f1ad8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/3912857980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad8100420 0x7f1ad8100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 -- 172.21.15.22:0/3912857980 >> 172.21.15.22:0/3912857980 conn(0x7f1ad8077290 msgr2=0x7f1ad80776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 -- 172.21.15.22:0/3912857980 shutdown_connections 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.865+0000 7f1ae03cd640 1 -- 172.21.15.22:0/3912857980 wait complete. 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 Processor -- start 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 -- start start 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad8100420 0x7f1ad8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81054f0 0x7f1ad8198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ad8199530 0x7f1ad81a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8110590 con 0x7f1ad8100420 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8110410 con 0x7f1ad81054f0 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ae03cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8110710 con 0x7f1ad8199530 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ade169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad8100420 0x7f1ad8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ade169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad8100420 0x7f1ad8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53698/0 (socket says 172.21.15.22:53698) 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1ade169640 1 -- 172.21.15.22:0/890112462 learned_addr learned my addr 172.21.15.22:0/890112462 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.866+0000 7f1add968640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81054f0 0x7f1ad8198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ade96a640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ad8199530 0x7f1ad81a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ade169640 1 -- 172.21.15.22:0/890112462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ad8199530 msgr2=0x7f1ad81a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ade169640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ad8199530 0x7f1ad81a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ade169640 1 -- 172.21.15.22:0/890112462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81054f0 msgr2=0x7f1ad8198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ade169640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81054f0 0x7f1ad8198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ade169640 1 -- 172.21.15.22:0/890112462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad81a42b0 con 0x7f1ad8100420 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1add968640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81054f0 0x7f1ad8198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ade169640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad8100420 0x7f1ad8198ab0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000d950 tx=0x7f1ad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ad77fe640 1 -- 172.21.15.22:0/890112462 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0017070 con 0x7f1ad8100420 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ad77fe640 1 -- 172.21.15.22:0/890112462 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ad0009c30 con 0x7f1ad8100420 2024-03-09T00:12:29.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad81a45a0 con 0x7f1ad8100420 2024-03-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ad77fe640 1 -- 172.21.15.22:0/890112462 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000f6b0 con 0x7f1ad8100420 2024-03-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.867+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad81a4af0 con 0x7f1ad8100420 2024-03-09T00:12:29.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.868+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad8068530 con 0x7f1ad8100420 2024-03-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.869+0000 7f1ad77fe640 1 -- 172.21.15.22:0/890112462 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1ad00109d0 con 0x7f1ad8100420 2024-03-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.870+0000 7f1ad77fe640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1ac00a2970 0x7f1ac00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.870+0000 7f1add968640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1ac00a2970 0x7f1ac00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.870+0000 7f1ad77fe640 1 -- 172.21.15.22:0/890112462 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9523+0+0 (secure 0 0 0) 0x7f1ad00cefc0 con 0x7f1ad8100420 2024-03-09T00:12:29.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.870+0000 7f1add968640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1ac00a2970 0x7f1ac00a4e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1acc005fd0 tx=0x7f1acc005850 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:12:29.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:29.874+0000 7f1ad77fe640 1 -- 172.21.15.22:0/890112462 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1ad0009de0 con 0x7f1ad8100420 2024-03-09T00:12:30.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.106+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f1ad8108780 con 0x7f1ad8100420 2024-03-09T00:12:30.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.114+0000 7f1ad77fe640 1 -- 172.21.15.22:0/890112462 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) v1 ==== 81+0+740362 (secure 0 0 0) 0x7f1ad009b930 con 0x7f1ad8100420 2024-03-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:12:30.122 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 27, 2024-03-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi022.stdout: "active_gid": 15660, 2024-03-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi022.stdout: "active_name": "x", 2024-03-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addrs": { 2024-03-09T00:12:30.123 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-03-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-03-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6800", 2024-03-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2739225064 2024-03-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2739225064 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addr": "172.21.15.22:6801/2739225064", 2024-03-09T00:12:30.125 INFO:teuthology.orchestra.run.smithi022.stdout: "active_change": "2024-03-08T23:54:32.858039+0000", 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: "active_mgr_features": 4540701547738038271, 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: "available": true, 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: "standbys": [ 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 25219, 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "z", 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540701547738038271, 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-03-09T00:12:30.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-03-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.127 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.129 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.130 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.134 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.135 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.136 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.137 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.138 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.139 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.140 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.141 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.142 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.144 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.146 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.147 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.148 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.149 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.150 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-03-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-03-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.151 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.153 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.155 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.156 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-03-09T00:12:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-03-09T00:12:30.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.159 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.160 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.161 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.162 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.163 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-03-09T00:12:30.164 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.165 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-09T00:12:30.166 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.167 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.169 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.170 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.171 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.172 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.173 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.174 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.175 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.176 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.178 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.179 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-03-09T00:12:30.180 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.181 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.182 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.183 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.184 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.186 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.187 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.188 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-03-09T00:12:30.190 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-09T00:12:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.192 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-03-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.193 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.194 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-03-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-03-09T00:12:30.196 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.197 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-03-09T00:12:30.198 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-03-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.199 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.200 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.201 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.202 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-03-09T00:12:30.203 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-03-09T00:12:30.204 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-09T00:12:30.205 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.206 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-03-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-03-09T00:12:30.208 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.209 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.210 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-03-09T00:12:30.211 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:30.212 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.215 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-03-09T00:12:30.216 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.217 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-03-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-03-09T00:12:30.218 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-03-09T00:12:30.219 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.220 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.222 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-03-09T00:12:30.223 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-03-09T00:12:30.224 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.225 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.227 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-03-09T00:12:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.230 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-03-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.231 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.232 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.234 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-03-09T00:12:30.235 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.236 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.237 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.239 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-03-09T00:12:30.240 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-09T00:12:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.242 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-03-09T00:12:30.245 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-09T00:12:30.246 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.247 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.249 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-03-09T00:12:30.250 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.251 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.252 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.253 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.254 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.255 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.258 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.259 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.260 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.261 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.264 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-09T00:12:30.265 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-09T00:12:30.266 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-03-09T00:12:30.270 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-03-09T00:12:30.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.272 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-03-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-03-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-03-09T00:12:30.275 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.276 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-03-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-09T00:12:30.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.278 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.279 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.282 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.283 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.284 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-03-09T00:12:30.285 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-09T00:12:30.286 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.287 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.288 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-03-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-03-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-03-09T00:12:30.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-03-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-03-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-03-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-03-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-03-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.295 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-03-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-03-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.297 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-03-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-03-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.299 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-03-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.301 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.302 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-03-09T00:12:30.304 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.306 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.308 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.309 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.310 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.311 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.314 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-03-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-03-09T00:12:30.315 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-03-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.316 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-03-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-03-09T00:12:30.317 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-03-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.318 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-03-09T00:12:30.319 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-03-09T00:12:30.321 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.322 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.323 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.324 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.325 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.326 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.327 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.328 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.329 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.330 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.332 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.333 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.334 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.335 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.336 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.337 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-09T00:12:30.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-09T00:12:30.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.341 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-09T00:12:30.342 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-09T00:12:30.344 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.345 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.346 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.348 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-03-09T00:12:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-03-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-09T00:12:30.351 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-09T00:12:30.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.354 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.356 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.357 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.358 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-09T00:12:30.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "MULTICLUSTER_CONFIG": { 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.361 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "{}", 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.362 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.363 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-09T00:12:30.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.365 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.366 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.367 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.368 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.369 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.370 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.372 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.373 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.374 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.375 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.376 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.377 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.378 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.380 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.381 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.382 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.384 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.385 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-03-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.386 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-09T00:12:30.387 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-03-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.388 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-03-09T00:12:30.389 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.390 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.391 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-09T00:12:30.394 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.395 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "UNSAFE_TLS_v1_2": { 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.396 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-09T00:12:30.397 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.398 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.399 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.401 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.402 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.403 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.404 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-03-09T00:12:30.405 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.406 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-03-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-03-09T00:12:30.407 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-03-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-03-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.409 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-03-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-03-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.413 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.415 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.416 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.417 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.418 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.419 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.420 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.421 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-03-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-03-09T00:12:30.422 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-03-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-03-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-03-09T00:12:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.425 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:30.426 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.427 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.428 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-03-09T00:12:30.431 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.432 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.433 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.434 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-03-09T00:12:30.435 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-03-09T00:12:30.436 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.437 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-03-09T00:12:30.438 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.440 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.441 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.442 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.443 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.444 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.445 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.446 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.447 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.449 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-03-09T00:12:30.450 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-03-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-03-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.451 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-03-09T00:12:30.452 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.453 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-03-09T00:12:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-03-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-03-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-03-09T00:12:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-03-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.457 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-03-09T00:12:30.458 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-03-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.459 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:30.460 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.461 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-03-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-03-09T00:12:30.462 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-03-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-03-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.463 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.464 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.465 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.466 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.468 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.469 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.470 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.472 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.473 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.474 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:30.476 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.477 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-03-09T00:12:30.478 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.479 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-03-09T00:12:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-03-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-03-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.482 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-03-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-03-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.483 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-03-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.484 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.485 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-03-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-03-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.486 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.487 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.488 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.489 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.490 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.493 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.494 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.497 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-03-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.498 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-03-09T00:12:30.499 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-03-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.500 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-03-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-03-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-03-09T00:12:30.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-03-09T00:12:30.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-09T00:12:30.503 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.504 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.505 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-03-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-03-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-03-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.506 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.507 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.508 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.509 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.510 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.511 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.512 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.513 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.514 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.515 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.516 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-03-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.517 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.518 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.519 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.522 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.523 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.524 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.525 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.526 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.527 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-03-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.528 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-03-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-03-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-03-09T00:12:30.529 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-03-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-03-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.531 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-03-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.532 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.533 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.534 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.535 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.536 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.537 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.538 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.542 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.543 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.544 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.545 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.547 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.548 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.549 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.550 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.551 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.552 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.553 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.554 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.555 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.556 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-03-09T00:12:30.557 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.558 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-03-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.559 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-03-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-03-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-03-09T00:12:30.560 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-03-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.561 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-03-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-03-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.562 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-03-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-03-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-03-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.564 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-03-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.565 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-03-09T00:12:30.566 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.567 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.568 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.571 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.572 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.573 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-03-09T00:12:30.577 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.578 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.579 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.580 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.581 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.582 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.583 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.584 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.585 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-03-09T00:12:30.586 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.587 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.588 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.589 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.590 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.591 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.592 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.593 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.594 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-03-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.596 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.597 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.598 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.599 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.600 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.601 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.602 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.603 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.604 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.605 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.606 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.607 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.609 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.610 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-03-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-03-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.611 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-03-09T00:12:30.612 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-03-09T00:12:30.613 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.614 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.617 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.619 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.620 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.621 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.622 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.623 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.624 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.625 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.626 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.628 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.629 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.630 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.631 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.632 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.633 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.634 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.635 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.636 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.637 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.638 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.639 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.640 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.641 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:30.642 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.643 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-03-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-03-09T00:12:30.644 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-03-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-03-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.645 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-03-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-03-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-03-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-03-09T00:12:30.647 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.648 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-03-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-03-09T00:12:30.649 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.650 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.651 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.652 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.653 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.654 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.656 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.657 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.658 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.660 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.661 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.662 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.663 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.664 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-03-09T00:12:30.665 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-09T00:12:30.666 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.667 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.669 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.670 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.671 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.672 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.673 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.674 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.675 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.676 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-09T00:12:30.677 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.679 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-03-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.681 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.682 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.684 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-03-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-03-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.685 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-03-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-03-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-03-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.687 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.688 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-03-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-03-09T00:12:30.689 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-03-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-03-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-03-09T00:12:30.690 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.692 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.693 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.694 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.695 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.696 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.697 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.698 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.699 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.700 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.701 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-03-09T00:12:30.702 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.703 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-03-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-03-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.704 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.705 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.706 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-03-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-03-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-03-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.707 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.708 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-03-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-03-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.709 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.710 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.711 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.713 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.714 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.715 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.716 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.717 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.718 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.719 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.720 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.721 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.722 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.724 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.725 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-03-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.726 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.727 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.728 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.730 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.731 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.732 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.736 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.737 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.738 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.739 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.740 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.741 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.742 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.743 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-03-09T00:12:30.744 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-03-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-03-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.745 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-03-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.747 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.748 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.749 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.750 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.751 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.752 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.753 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.754 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.756 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-03-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-03-09T00:12:30.757 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.758 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-03-09T00:12:30.759 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-03-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.760 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-03-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-03-09T00:12:30.761 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.762 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-03-09T00:12:30.763 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.764 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-03-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-03-09T00:12:30.765 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.766 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.767 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.769 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-03-09T00:12:30.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.771 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.772 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-03-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-03-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.775 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.776 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.777 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.778 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-03-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-03-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-03-09T00:12:30.780 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.781 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-03-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-03-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-03-09T00:12:30.782 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.783 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-03-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.784 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.785 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.786 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.787 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.789 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.790 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.792 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.793 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-03-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.794 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.795 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.796 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.797 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.798 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.800 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.801 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.802 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.803 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.804 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.805 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.806 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.808 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.810 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.812 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.813 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.814 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.816 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.817 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.819 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.820 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.821 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.823 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.824 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.826 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-03-09T00:12:30.828 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.829 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-03-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-03-09T00:12:30.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.831 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-03-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-03-09T00:12:30.832 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-09T00:12:30.833 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-03-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-03-09T00:12:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.835 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-03-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-03-09T00:12:30.836 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.837 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-03-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-03-09T00:12:30.838 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.839 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.840 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.841 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-09T00:12:30.842 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-03-09T00:12:30.843 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.844 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.845 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.846 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.847 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.849 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.850 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.851 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.852 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.853 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.854 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.854 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.854 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.854 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.854 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.855 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.855 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.855 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.855 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.855 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.855 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.855 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.856 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.856 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.856 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.856 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.856 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.856 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.857 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.857 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.857 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.857 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.857 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.857 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.857 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.858 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.858 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.858 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.858 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.858 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.858 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.859 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.860 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.860 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.860 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.860 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.860 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.860 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.860 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-03-09T00:12:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-03-09T00:12:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.861 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.862 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.862 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.862 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.862 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.862 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.862 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.862 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.863 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.865 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.865 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-03-09T00:12:30.865 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-03-09T00:12:30.865 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.865 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.865 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.866 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-09T00:12:30.866 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.866 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.866 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.866 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.866 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.866 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.867 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.868 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.869 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.869 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.869 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.869 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.869 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.869 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.869 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.870 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.870 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.870 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.870 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.870 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.871 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.871 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.871 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.871 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.871 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.871 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.872 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.872 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.872 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.872 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.872 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.872 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.872 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.873 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.873 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.873 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.873 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.873 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.873 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.874 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.874 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.874 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.874 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.874 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.874 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.875 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.876 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.877 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.878 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.879 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.880 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.881 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.882 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.883 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.884 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.885 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.886 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.887 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.890 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-03-09T00:12:30.891 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-09T00:12:30.892 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.893 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.893 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.893 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.893 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.893 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-03-09T00:12:30.894 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.894 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.894 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.894 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-03-09T00:12:30.894 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-03-09T00:12:30.894 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.895 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.895 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.895 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-03-09T00:12:30.895 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.895 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.895 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.895 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.896 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.896 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.896 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.896 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-03-09T00:12:30.896 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-03-09T00:12:30.896 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.897 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.897 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.897 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.897 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.897 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.897 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.897 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.898 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.898 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.898 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.898 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:30.898 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:30.899 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.899 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.899 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.899 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:30.899 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.899 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.899 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.900 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.900 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.900 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.900 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.900 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.900 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.901 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.901 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.901 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.901 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.901 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.901 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.901 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.902 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.902 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.902 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.902 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.902 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.902 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.902 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.903 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.903 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.903 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.903 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.903 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.903 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.904 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.904 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.904 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.904 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.904 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.904 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.904 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.906 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.906 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.906 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.906 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.906 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.906 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.906 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.907 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.907 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.907 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.907 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.907 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.907 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.907 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.908 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.908 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.908 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.908 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.908 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.908 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.909 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.909 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.909 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.909 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.909 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.909 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.909 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.910 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.910 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.910 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.910 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.910 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-03-09T00:12:30.910 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-03-09T00:12:30.911 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.911 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.911 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.911 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.911 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.911 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.911 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.912 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.912 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.912 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.912 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.912 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-03-09T00:12:30.912 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-03-09T00:12:30.913 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.913 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.913 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.913 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-03-09T00:12:30.913 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.913 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.913 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-03-09T00:12:30.914 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.915 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 25241, 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "y", 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540701547738038271, 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:30.918 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.919 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.920 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.921 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.922 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.923 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.924 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.925 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.926 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-03-09T00:12:30.927 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-03-09T00:12:30.928 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.928 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.928 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.928 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.928 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.928 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.928 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.929 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-03-09T00:12:30.929 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.929 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.929 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.929 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-03-09T00:12:30.929 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-03-09T00:12:30.930 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.930 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.930 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.930 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-03-09T00:12:30.930 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.930 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.931 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.931 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-03-09T00:12:30.931 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.931 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.931 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.931 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-03-09T00:12:30.932 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-03-09T00:12:30.932 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.932 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.932 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.932 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.932 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.932 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-03-09T00:12:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-03-09T00:12:30.934 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.934 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.934 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.934 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.934 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.934 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.934 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.935 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-03-09T00:12:30.935 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.935 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.935 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.935 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-03-09T00:12:30.935 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-03-09T00:12:30.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.936 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.936 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.936 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-03-09T00:12:30.936 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.936 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.936 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-03-09T00:12:30.937 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.938 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.938 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.938 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.938 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.938 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.938 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.938 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.939 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.940 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.940 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.940 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.940 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.940 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.940 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-09T00:12:30.940 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.941 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.941 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.941 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-03-09T00:12:30.941 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-03-09T00:12:30.941 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.942 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.942 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.942 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.942 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.942 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.942 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.942 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-03-09T00:12:30.943 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.943 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.943 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-03-09T00:12:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-03-09T00:12:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-03-09T00:12:30.944 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-03-09T00:12:30.945 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-03-09T00:12:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-03-09T00:12:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.947 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.947 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-03-09T00:12:30.947 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.947 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.950 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.950 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-03-09T00:12:30.950 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.951 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.953 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.954 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.955 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-03-09T00:12:30.956 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-03-09T00:12:30.957 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-03-09T00:12:30.958 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.959 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.961 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.962 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.963 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.965 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.966 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.967 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.969 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.970 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.970 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-09T00:12:30.970 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.970 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.970 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.970 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-03-09T00:12:30.971 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-03-09T00:12:30.971 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.971 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.971 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.971 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-03-09T00:12:30.971 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.971 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.972 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.972 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-03-09T00:12:30.972 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-03-09T00:12:30.972 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-03-09T00:12:30.972 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-03-09T00:12:30.972 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-03-09T00:12:30.972 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-03-09T00:12:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-03-09T00:12:30.973 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-03-09T00:12:30.974 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.974 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.974 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.974 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.974 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.974 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.974 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.975 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-09T00:12:30.975 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.975 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.975 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.975 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:30.976 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:30.976 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.976 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.976 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.976 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:30.976 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.976 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.977 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.977 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-09T00:12:30.977 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.977 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.977 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.977 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-03-09T00:12:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-03-09T00:12:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.979 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.979 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-09T00:12:30.979 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-09T00:12:30.979 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.979 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.979 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-03-09T00:12:30.980 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-03-09T00:12:30.980 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:30.980 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.980 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:30.980 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:30.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.980 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.981 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.981 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-09T00:12:30.981 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.981 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.981 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.981 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:30.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:30.982 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-03-09T00:12:30.982 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:30.982 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:30.982 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:30.983 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-03-09T00:12:30.983 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-03-09T00:12:30.983 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.983 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.983 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.983 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-03-09T00:12:30.983 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.984 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.984 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.984 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-09T00:12:30.984 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.984 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.984 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.985 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-03-09T00:12:30.985 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-03-09T00:12:30.985 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.985 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.985 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.985 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-03-09T00:12:30.985 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-09T00:12:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.987 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-03-09T00:12:30.987 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-03-09T00:12:30.987 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:30.987 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.987 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.987 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-03-09T00:12:30.987 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.988 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.988 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.988 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-09T00:12:30.988 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.988 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.988 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.989 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-03-09T00:12:30.990 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-09T00:12:30.991 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.992 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-03-09T00:12:30.993 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.993 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.993 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.993 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-09T00:12:30.993 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.993 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.993 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.994 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-03-09T00:12:30.994 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-03-09T00:12:30.994 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:30.994 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.994 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.994 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-09T00:12:30.995 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.996 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.997 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:30.998 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:30.999 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.000 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.001 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-03-09T00:12:31.002 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-03-09T00:12:31.003 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.004 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.006 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-03-09T00:12:31.006 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-03-09T00:12:31.006 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.006 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.006 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.006 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-09T00:12:31.006 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.007 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-03-09T00:12:31.008 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-03-09T00:12:31.008 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.008 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.008 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.008 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-09T00:12:31.008 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.008 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.009 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.009 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-03-09T00:12:31.009 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.009 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.009 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.009 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-03-09T00:12:31.010 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-03-09T00:12:31.010 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.010 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.010 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.010 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-09T00:12:31.010 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.011 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.011 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.011 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-03-09T00:12:31.011 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.011 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.011 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.012 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-03-09T00:12:31.012 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-03-09T00:12:31.012 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.012 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.012 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.012 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-09T00:12:31.012 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.013 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.013 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.013 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-03-09T00:12:31.013 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.013 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.013 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.014 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-03-09T00:12:31.014 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-03-09T00:12:31.014 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.014 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.014 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.014 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-09T00:12:31.015 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.015 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.015 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.015 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-03-09T00:12:31.015 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.016 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.016 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.016 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-03-09T00:12:31.016 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-03-09T00:12:31.017 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.017 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.017 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.017 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-09T00:12:31.017 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.017 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.018 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.018 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-03-09T00:12:31.018 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.018 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.018 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.019 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-03-09T00:12:31.019 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-03-09T00:12:31.019 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.019 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.019 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.019 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-09T00:12:31.020 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.020 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.020 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.020 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-03-09T00:12:31.020 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.021 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.021 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.021 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-03-09T00:12:31.021 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-03-09T00:12:31.021 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.022 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.022 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-09T00:12:31.022 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.022 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.022 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.022 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.023 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.025 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.026 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.026 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.026 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.026 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.026 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-03-09T00:12:31.027 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-03-09T00:12:31.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.027 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.027 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.027 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-09T00:12:31.027 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-03-09T00:12:31.029 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-03-09T00:12:31.029 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.029 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.029 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.029 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-09T00:12:31.029 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.030 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.030 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.030 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-03-09T00:12:31.030 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.030 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.030 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.031 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-03-09T00:12:31.031 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-03-09T00:12:31.031 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.031 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.031 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.031 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.032 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.032 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.032 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.032 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-03-09T00:12:31.032 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.032 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.032 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.033 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-03-09T00:12:31.034 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.035 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.035 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.035 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-03-09T00:12:31.035 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.035 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.035 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.035 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.036 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.038 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.039 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.040 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.041 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-03-09T00:12:31.042 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.043 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.044 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.045 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.045 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.045 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.045 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.045 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-09T00:12:31.045 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.045 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.046 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.046 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-03-09T00:12:31.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-03-09T00:12:31.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.046 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.047 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.048 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.049 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.050 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.050 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.050 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.050 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.050 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.050 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-03-09T00:12:31.051 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-03-09T00:12:31.051 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.051 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.051 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.051 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.051 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.052 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.052 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-09T00:12:31.052 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.052 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.052 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.053 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.053 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.053 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.053 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.053 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.053 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.053 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.054 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.054 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.054 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-09T00:12:31.054 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.054 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.054 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.055 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.055 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.055 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.055 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.055 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.055 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.055 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.056 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.056 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.056 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.056 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.056 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.056 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.056 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.057 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.057 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.057 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.057 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.057 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.057 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.058 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.058 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.058 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.058 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.058 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.058 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.059 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.059 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.059 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.059 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.059 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.059 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.059 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.060 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.061 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.061 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.061 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-09T00:12:31.061 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.061 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.061 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.062 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-09T00:12:31.062 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-09T00:12:31.062 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.062 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.062 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.062 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-03-09T00:12:31.063 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.063 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.063 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.063 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-09T00:12:31.063 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.063 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.063 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.064 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-03-09T00:12:31.064 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-03-09T00:12:31.064 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.064 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.064 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.065 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.065 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.065 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.065 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-03-09T00:12:31.065 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.065 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.065 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.066 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-03-09T00:12:31.066 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-03-09T00:12:31.066 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.066 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.066 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.066 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.067 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.067 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.067 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.067 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-09T00:12:31.067 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.068 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.070 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.071 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.072 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.073 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.074 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-03-09T00:12:31.076 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-09T00:12:31.077 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.078 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.079 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.079 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.079 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.079 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-09T00:12:31.079 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.079 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.080 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.080 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-03-09T00:12:31.080 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-03-09T00:12:31.080 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.080 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.080 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.081 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.081 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-09T00:12:31.081 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.081 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.082 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.083 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.083 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.083 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.083 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-09T00:12:31.083 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.083 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.083 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.084 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-03-09T00:12:31.084 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-03-09T00:12:31.084 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.084 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.084 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.085 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-03-09T00:12:31.085 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.085 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.085 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.085 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-03-09T00:12:31.085 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.085 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.086 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.086 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-03-09T00:12:31.086 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-03-09T00:12:31.086 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.086 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.086 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.087 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.087 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.087 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.087 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.087 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-09T00:12:31.087 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.087 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-03-09T00:12:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-03-09T00:12:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.088 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.089 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.089 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.089 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.089 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.089 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-09T00:12:31.089 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.090 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.090 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-03-09T00:12:31.090 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-03-09T00:12:31.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.090 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.091 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.091 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.091 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.091 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.091 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-09T00:12:31.091 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.092 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.092 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.092 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-03-09T00:12:31.092 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-03-09T00:12:31.092 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.092 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.093 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.093 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.093 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.093 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.093 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.093 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-03-09T00:12:31.093 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.094 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.094 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.094 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-03-09T00:12:31.094 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-03-09T00:12:31.094 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.094 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.095 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.095 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.095 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.095 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.095 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.095 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-09T00:12:31.095 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.096 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.096 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.096 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-03-09T00:12:31.096 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-03-09T00:12:31.096 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.096 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.097 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.097 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.097 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.097 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.097 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.097 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-09T00:12:31.097 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.098 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.098 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.098 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.098 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.098 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.099 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.100 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.100 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.100 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.100 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.100 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.100 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.101 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.101 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.101 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.101 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.101 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.101 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.101 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.102 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.102 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.102 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.102 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.102 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.102 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.103 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.104 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.105 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.106 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.107 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-03-09T00:12:31.108 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-03-09T00:12:31.109 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.110 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-03-09T00:12:31.111 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.112 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-03-09T00:12:31.113 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.114 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.115 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.116 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.117 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-03-09T00:12:31.117 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-03-09T00:12:31.117 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.117 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.117 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.117 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.118 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.118 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.118 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.118 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.118 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.118 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.118 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.119 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-09T00:12:31.120 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.121 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.122 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.123 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.124 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.125 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.125 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.125 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.125 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.125 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-09T00:12:31.125 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-09T00:12:31.126 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.126 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.126 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.126 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.126 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.126 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.126 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.127 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.127 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.127 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.127 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.127 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-09T00:12:31.127 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-09T00:12:31.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.128 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.128 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.128 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.128 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.128 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.129 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.129 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.129 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.129 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.129 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.129 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-09T00:12:31.130 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-09T00:12:31.130 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.130 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.130 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.130 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.130 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.130 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.131 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.131 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.131 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.131 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.131 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.131 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-09T00:12:31.132 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-09T00:12:31.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.132 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.132 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.132 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.132 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.132 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.133 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.133 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.133 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.133 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.133 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.133 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-09T00:12:31.134 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-09T00:12:31.134 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.134 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.134 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.134 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.134 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.134 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.135 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.135 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.135 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.135 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.135 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.135 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-09T00:12:31.136 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-09T00:12:31.136 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.136 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.136 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.136 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.136 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.136 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-03-09T00:12:31.137 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-09T00:12:31.138 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.138 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.138 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:31.138 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.138 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.139 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.139 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.139 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.140 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-03-09T00:12:31.141 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-03-09T00:12:31.142 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.142 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.142 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.142 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.142 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.142 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.142 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-09T00:12:31.143 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.145 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-09T00:12:31.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-09T00:12:31.147 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.147 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.147 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.147 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.147 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.147 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.148 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.150 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-09T00:12:31.150 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-09T00:12:31.150 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.150 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.150 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.150 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.150 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.151 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.154 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.154 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.154 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.154 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.154 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: "MULTICLUSTER_CONFIG": { 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "{}", 2024-03-09T00:12:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.156 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.157 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.158 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.159 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.159 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.159 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.160 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.161 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.161 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.161 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.161 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.161 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.161 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.161 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.162 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.162 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-09T00:12:31.162 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-09T00:12:31.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.162 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.163 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.163 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.163 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.163 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.163 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.163 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.163 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.164 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.164 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.164 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-09T00:12:31.164 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-09T00:12:31.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.165 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.165 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.165 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.165 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.165 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.165 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.165 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.166 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.166 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.166 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.166 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-09T00:12:31.166 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-09T00:12:31.166 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.167 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.167 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.167 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.167 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.167 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.167 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.168 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.168 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.168 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.168 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.168 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-09T00:12:31.168 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.169 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.170 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.170 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.170 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.170 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-09T00:12:31.170 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-09T00:12:31.171 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.171 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.171 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.171 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.171 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.171 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.172 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.172 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.172 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.172 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.172 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.172 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-09T00:12:31.173 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-09T00:12:31.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.173 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.173 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.173 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.174 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.174 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.174 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.174 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.175 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-03-09T00:12:31.175 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-09T00:12:31.175 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.175 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.175 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.175 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.175 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.176 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.176 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.176 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.176 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.176 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.176 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.177 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-09T00:12:31.177 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-09T00:12:31.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.177 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.177 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-09T00:12:31.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.178 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.178 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.178 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.178 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.178 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.178 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-09T00:12:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-09T00:12:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.179 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.180 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.180 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.180 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.180 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.180 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.180 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.181 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.181 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-09T00:12:31.181 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-09T00:12:31.182 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.182 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.183 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.183 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-03-09T00:12:31.183 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.184 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.184 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.184 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.184 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.184 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.185 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-09T00:12:31.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-09T00:12:31.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.185 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.185 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.186 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-03-09T00:12:31.186 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.186 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.186 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.186 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.186 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.186 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.187 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.189 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-09T00:12:31.189 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-09T00:12:31.189 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.189 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:31.189 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.190 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.191 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.192 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.193 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "UNSAFE_TLS_v1_2": { 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.194 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.195 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.195 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.195 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.195 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.195 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.195 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-09T00:12:31.196 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-09T00:12:31.196 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.196 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.196 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.196 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:31.196 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.196 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.197 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.198 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.199 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.199 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.199 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.199 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.199 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-09T00:12:31.199 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-09T00:12:31.200 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.200 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.200 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.200 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:31.200 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.200 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.200 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-03-09T00:12:31.201 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.202 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.202 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.202 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.202 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.202 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.202 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.202 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.203 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.203 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.203 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.203 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-03-09T00:12:31.203 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-03-09T00:12:31.203 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.204 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.204 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.204 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.204 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.204 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.205 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.205 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.205 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.205 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.205 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.206 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-03-09T00:12:31.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-03-09T00:12:31.206 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.206 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.206 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.206 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.207 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.207 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.207 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-03-09T00:12:31.207 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.207 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.207 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-03-09T00:12:31.208 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.209 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.209 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.209 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.209 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.209 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.210 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-03-09T00:12:31.210 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-03-09T00:12:31.210 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.210 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.210 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.210 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.212 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.212 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.212 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.212 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.212 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.212 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.213 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.213 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.213 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.213 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.213 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.213 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.214 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.215 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.216 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.216 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.216 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.216 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.216 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.216 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.216 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.217 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.217 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.217 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.218 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.219 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.220 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.221 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.222 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.223 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:31.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-03-09T00:12:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-03-09T00:12:31.227 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.228 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.229 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.231 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.233 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-03-09T00:12:31.233 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-03-09T00:12:31.233 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.233 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-03-09T00:12:31.234 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-03-09T00:12:31.234 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-03-09T00:12:31.234 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.234 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.234 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.234 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.236 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-03-09T00:12:31.237 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-03-09T00:12:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.238 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.239 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.240 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.242 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.243 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.244 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.245 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.246 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-03-09T00:12:31.247 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-09T00:12:31.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-03-09T00:12:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-09T00:12:31.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.254 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.255 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.256 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-03-09T00:12:31.257 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-09T00:12:31.258 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-03-09T00:12:31.259 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.260 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.261 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.262 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.265 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.267 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.268 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.270 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:31.271 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.272 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-03-09T00:12:31.273 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.274 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-03-09T00:12:31.275 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.276 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.277 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-03-09T00:12:31.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.280 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.281 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.282 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.283 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.284 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.285 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.286 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.286 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.286 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.286 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.286 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.286 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.286 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.288 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.289 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.290 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.291 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-03-09T00:12:31.292 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.293 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.294 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-03-09T00:12:31.297 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-09T00:12:31.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-03-09T00:12:31.299 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.300 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.300 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.300 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.300 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.300 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.300 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.301 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.302 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.304 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.305 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.306 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.307 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.309 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.310 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.311 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.314 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.315 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.316 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.317 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.317 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.317 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.317 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.317 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.318 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.319 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.320 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.321 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.322 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.323 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.325 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.326 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.327 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.328 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.329 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.330 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.331 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.333 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.334 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.337 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.338 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.339 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.340 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.341 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.343 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-03-09T00:12:31.344 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-03-09T00:12:31.345 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.346 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.346 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-03-09T00:12:31.346 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-03-09T00:12:31.346 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.349 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.351 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.353 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.354 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.355 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.356 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.357 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.358 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.359 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.361 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.362 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.363 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.364 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.365 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.366 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.367 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.368 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.369 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.370 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.371 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.372 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.373 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.374 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.376 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-03-09T00:12:31.377 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.379 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.380 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.382 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.383 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.384 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.385 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.386 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.390 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.391 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.392 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.395 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.396 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.397 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.398 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.399 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.400 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.401 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.402 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.402 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.402 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.402 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.402 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.403 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.405 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.406 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.407 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.410 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.410 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.410 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.410 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.413 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.414 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.415 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.416 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.417 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:31.418 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-03-09T00:12:31.419 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.420 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-03-09T00:12:31.421 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.422 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.422 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.422 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-03-09T00:12:31.422 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-03-09T00:12:31.422 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.422 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.422 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.424 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.425 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.426 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.427 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.428 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.430 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.431 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.432 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:31.435 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-03-09T00:12:31.436 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.437 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.439 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.442 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.445 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.446 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.447 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.448 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-03-09T00:12:31.449 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.451 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.452 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.453 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:31.454 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-09T00:12:31.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.456 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.458 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-03-09T00:12:31.459 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.461 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.462 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.463 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.464 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.465 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.466 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.467 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.468 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.469 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.471 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-03-09T00:12:31.472 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.473 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.474 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.475 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.476 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.477 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.478 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.480 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.481 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.482 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.483 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.485 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.486 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.487 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.488 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.489 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.490 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:31.491 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.492 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.493 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.494 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.495 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.496 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.497 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.499 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.500 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.501 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-03-09T00:12:31.502 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.503 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.504 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.505 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.507 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.508 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.510 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.511 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.513 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.514 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.515 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.516 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.517 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.518 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.519 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.520 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.521 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.522 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-03-09T00:12:31.523 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.524 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.525 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.526 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.527 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-03-09T00:12:31.528 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.529 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.530 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.531 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.532 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-03-09T00:12:31.533 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.534 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.535 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.537 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-03-09T00:12:31.538 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.539 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.540 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.541 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-03-09T00:12:31.542 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.543 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.544 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.545 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.546 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.547 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.548 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.549 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.550 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.551 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.552 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.553 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-03-09T00:12:31.554 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.556 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.557 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.559 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.560 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.562 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.562 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.562 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.562 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.562 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.562 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.562 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.566 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.566 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.566 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.566 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.568 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.570 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.571 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.572 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.573 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.574 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.575 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.576 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.577 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.578 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.579 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.580 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.581 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.582 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.583 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.584 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.585 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.586 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.587 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.588 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.589 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.591 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-03-09T00:12:31.592 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.593 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.594 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.595 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.596 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.597 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.598 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.599 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-03-09T00:12:31.600 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.601 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.602 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-03-09T00:12:31.604 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-03-09T00:12:31.605 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.606 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.607 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.609 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.610 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.611 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.613 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.614 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.615 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.617 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-03-09T00:12:31.618 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.619 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.620 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.622 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.623 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.625 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.627 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.628 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.629 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.631 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.633 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.634 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.636 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.637 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.638 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.640 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-03-09T00:12:31.641 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-09T00:12:31.642 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.643 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.645 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.646 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-09T00:12:31.646 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.646 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.646 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.646 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-03-09T00:12:31.646 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-09T00:12:31.647 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-03-09T00:12:31.648 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-03-09T00:12:31.649 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.650 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.651 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.652 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.653 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.654 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.655 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.656 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.657 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.658 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.659 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.661 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-03-09T00:12:31.662 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.663 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.664 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.666 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.667 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: "modules": [ 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard", 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: "iostat", 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: "nfs", 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: "restful", 2024-03-09T00:12:31.668 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_schedule" 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.669 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.670 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.671 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.673 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.674 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.675 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.676 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.677 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.678 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.679 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.680 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.681 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.682 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.683 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2024-03-09T00:12:31.684 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.685 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.686 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.687 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.688 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2024-03-09T00:12:31.689 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.690 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.691 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2024-03-09T00:12:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.694 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.695 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-03-09T00:12:31.697 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2024-03-09T00:12:31.698 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-09T00:12:31.699 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.700 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2024-03-09T00:12:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2024-03-09T00:12:31.703 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2024-03-09T00:12:31.704 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.705 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.706 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.707 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.708 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.709 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.710 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.711 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.712 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.713 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.714 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.715 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.717 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: "read", 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap", 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap-read" 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2024-03-09T00:12:31.718 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.719 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.721 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2024-03-09T00:12:31.722 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-09T00:12:31.723 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.724 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2024-03-09T00:12:31.726 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-09T00:12:31.727 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.728 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.730 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2024-03-09T00:12:31.731 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-09T00:12:31.732 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.733 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.735 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_log_destination": { 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm_log_destination", 2024-03-09T00:12:31.736 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "file", 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "file,syslog", 2024-03-09T00:12:31.737 INFO:teuthology.orchestra.run.smithi022.stdout: "syslog" 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2024-03-09T00:12:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.739 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.740 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.742 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.743 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.744 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.745 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.746 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-09T00:12:31.747 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.748 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.749 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.750 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.751 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-09T00:12:31.752 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2024-03-09T00:12:31.753 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.754 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.755 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.756 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-09T00:12:31.757 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2024-03-09T00:12:31.758 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.759 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.760 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.761 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.762 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2024-03-09T00:12:31.763 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.764 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.765 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.766 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.767 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.768 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.769 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2024-03-09T00:12:31.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.771 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.771 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.771 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.771 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.771 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.771 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.771 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-09T00:12:31.774 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.774 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.774 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.774 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.775 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.776 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.777 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.779 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.780 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.781 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.782 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.783 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.784 INFO:teuthology.orchestra.run.smithi022.stdout: "hw_monitoring": { 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hw_monitoring", 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.785 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2024-03-09T00:12:31.786 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.788 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.789 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2024-03-09T00:12:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.792 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.793 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.794 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.795 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.797 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.798 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.799 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.799 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.799 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.799 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-09T00:12:31.799 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.799 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.799 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.800 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2024-03-09T00:12:31.801 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.802 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.803 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.804 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.805 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2024-03-09T00:12:31.806 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2024-03-09T00:12:31.807 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: "oob_default_addr": { 2024-03-09T00:12:31.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "oob_default_addr", 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "169.254.1.1", 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-09T00:12:31.809 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.810 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.812 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2024-03-09T00:12:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-09T00:12:31.814 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.815 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.816 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.817 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2024-03-09T00:12:31.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.819 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.820 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.822 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2024-03-09T00:12:31.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.824 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.825 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.826 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.827 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.828 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.829 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.830 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.834 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.835 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.836 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.837 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.838 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.839 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.841 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2024-03-09T00:12:31.842 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-09T00:12:31.843 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.844 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.845 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.847 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.848 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.849 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.850 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.850 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2024-03-09T00:12:31.850 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2024-03-09T00:12:31.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.850 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.852 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-09T00:12:31.853 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.854 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.857 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-09T00:12:31.858 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.859 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.860 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.862 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-09T00:12:31.863 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.864 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.865 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.866 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.867 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2024-03-09T00:12:31.868 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.869 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.870 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.871 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.872 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-09T00:12:31.873 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.874 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.875 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.876 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.877 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-09T00:12:31.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.879 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.880 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.881 INFO:teuthology.orchestra.run.smithi022.stdout: "MULTICLUSTER_CONFIG": { 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "{}", 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.882 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.883 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.886 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.887 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.888 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.889 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.890 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-09T00:12:31.890 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-09T00:12:31.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.890 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.890 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.890 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.891 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.892 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.893 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.894 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.895 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.896 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-09T00:12:31.898 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.899 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.900 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.901 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.902 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-09T00:12:31.903 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.904 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.905 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.906 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.907 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.908 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.909 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.910 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.911 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.912 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "UNSAFE_TLS_v1_2": { 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.913 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.914 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:31.915 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.916 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.917 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.918 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.919 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.920 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.921 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.922 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.923 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.924 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.925 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.926 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.927 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.928 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.929 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.930 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.931 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.932 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.933 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.934 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.935 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.936 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.937 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2024-03-09T00:12:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.939 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.940 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.941 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.942 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.943 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.944 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.945 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-03-09T00:12:31.946 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.947 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-03-09T00:12:31.948 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.949 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.950 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.951 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.952 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.953 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.954 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.955 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.956 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.957 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.958 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.959 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.960 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.961 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2024-03-09T00:12:31.962 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.963 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.964 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.965 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2024-03-09T00:12:31.966 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2024-03-09T00:12:31.967 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.968 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.969 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.970 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.971 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2024-03-09T00:12:31.972 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-09T00:12:31.973 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2024-03-09T00:12:31.974 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.975 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.976 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.977 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.978 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.979 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.980 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:31.981 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.983 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.985 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2024-03-09T00:12:31.985 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2024-03-09T00:12:31.985 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.985 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.985 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.985 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.987 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:31.989 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.990 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2024-03-09T00:12:31.991 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2024-03-09T00:12:31.992 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.993 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:31.994 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.995 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.996 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:31.997 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:31.998 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:31.999 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.000 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.001 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.002 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.003 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.004 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.006 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2024-03-09T00:12:32.007 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.008 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:32.009 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.010 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.011 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2024-03-09T00:12:32.012 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.013 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.014 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.016 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.017 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.018 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.019 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.020 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.021 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.022 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.023 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.024 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.025 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.026 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.027 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.029 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.031 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.032 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.033 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2024-03-09T00:12:32.035 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.036 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.037 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.038 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.039 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.040 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.041 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.042 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.043 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.044 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.045 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.046 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.047 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.048 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.050 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.051 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.052 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.053 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.054 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2024-03-09T00:12:32.055 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.056 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.057 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.058 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2024-03-09T00:12:32.059 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2024-03-09T00:12:32.060 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2024-03-09T00:12:32.061 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.062 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.063 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.065 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.066 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.068 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.070 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.071 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.072 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2024-03-09T00:12:32.073 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.074 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.075 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.076 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.077 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.078 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.078 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.078 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.078 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.078 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.078 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.079 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.080 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.081 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.082 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.083 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.084 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.085 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.086 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.087 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.088 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.090 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.092 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.093 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.094 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.095 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.096 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.097 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.098 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.099 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.101 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2024-03-09T00:12:32.102 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2024-03-09T00:12:32.103 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.106 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.107 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.109 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.110 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.111 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.112 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.113 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.114 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.115 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.116 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.117 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.119 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.120 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.121 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.123 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.124 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.125 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.126 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.127 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.128 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.129 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.130 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.131 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:32.132 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.133 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2024-03-09T00:12:32.134 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2024-03-09T00:12:32.136 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-09T00:12:32.137 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.140 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.141 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.142 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.143 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.144 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.145 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.146 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.147 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.148 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.150 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.152 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.153 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.154 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.155 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.156 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.157 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.159 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.160 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.161 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.163 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.164 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.166 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2024-03-09T00:12:32.167 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.168 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.169 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.170 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.171 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2024-03-09T00:12:32.172 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.173 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.174 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.175 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.176 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.177 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.178 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.178 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.178 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-03-09T00:12:32.178 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.178 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.182 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.182 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.182 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.182 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.183 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.186 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.187 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.188 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.191 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2024-03-09T00:12:32.192 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.194 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.195 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2024-03-09T00:12:32.196 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.197 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.198 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.199 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.201 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.202 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.203 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.204 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.205 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.207 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.208 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.210 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.212 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.213 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.214 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.214 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.214 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.214 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.214 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.214 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.214 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.215 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.216 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.217 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.218 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.220 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.221 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.222 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.224 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.225 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.226 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.227 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.228 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.229 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2024-03-09T00:12:32.230 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.232 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.233 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.235 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.236 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.237 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.238 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.239 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2024-03-09T00:12:32.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.241 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.243 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2024-03-09T00:12:32.245 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.246 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.248 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.249 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2024-03-09T00:12:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.252 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2024-03-09T00:12:32.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2024-03-09T00:12:32.255 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.256 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.257 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.259 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.261 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.262 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.263 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.264 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.266 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.270 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.271 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.272 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.273 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.274 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.275 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.276 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.278 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.279 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.280 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.281 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.283 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.284 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.286 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.287 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.288 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.289 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.290 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.291 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.292 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.294 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.295 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.296 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.297 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.298 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.300 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.301 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.302 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.303 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2024-03-09T00:12:32.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.305 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.306 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.308 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.309 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.310 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2024-03-09T00:12:32.311 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.312 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.313 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.315 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.318 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.320 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2024-03-09T00:12:32.321 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.322 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.322 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2024-03-09T00:12:32.322 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.322 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.322 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.322 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.323 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.326 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.327 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.328 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.329 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.331 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.333 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.334 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.335 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2024-03-09T00:12:32.336 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.337 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.338 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.339 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.340 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.341 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.342 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.345 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.347 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.348 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.349 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.350 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.351 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.352 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.356 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.357 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2024-03-09T00:12:32.358 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.359 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.360 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.361 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.362 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2024-03-09T00:12:32.363 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2024-03-09T00:12:32.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.367 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.369 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2024-03-09T00:12:32.370 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.371 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.372 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.375 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.376 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2024-03-09T00:12:32.377 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.378 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.379 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.380 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.381 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2024-03-09T00:12:32.382 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2024-03-09T00:12:32.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: "services": { 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard": "https://172.21.15.22:8443/" 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.384 INFO:teuthology.orchestra.run.smithi022.stdout: "always_on_modules": { 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "octopus": [ 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-03-09T00:12:32.385 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: "pacific": [ 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-03-09T00:12:32.386 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.387 INFO:teuthology.orchestra.run.smithi022.stdout: "quincy": [ 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-03-09T00:12:32.388 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: "reef": [ 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-03-09T00:12:32.389 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: "squid": [ 2024-03-09T00:12:32.390 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2024-03-09T00:12:32.391 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: "last_failure_osd_epoch": 94, 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: "active_clients": [ 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-03-09T00:12:32.392 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3869424893 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "libcephsqlite", 2024-03-09T00:12:32.393 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3277218220 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.394 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 992656147 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3422701724 2024-03-09T00:12:32.396 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-03-09T00:12:32.397 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1773788910 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-03-09T00:12:32.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2024-03-09T00:12:32.399 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2024-03-09T00:12:32.399 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1804211785 2024-03-09T00:12:32.399 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.399 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.399 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:32.399 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-03-09T00:12:32.399 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:12:32.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:30 smithi192 ceph-mon[35995]: pgmap v573: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-03-09T00:12:32.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:30 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/211041690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-09T00:12:32.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:30 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/4006810395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-09T00:12:32.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:30 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/890112462' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-09T00:12:32.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:30 smithi062 ceph-mon[36129]: pgmap v573: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-03-09T00:12:32.402 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:30 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/211041690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-09T00:12:32.402 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:30 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/4006810395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-09T00:12:32.402 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:30 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/890112462' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-09T00:12:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1ac00a2970 msgr2=0x7f1ac00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1ac00a2970 0x7f1ac00a4e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1acc005fd0 tx=0x7f1acc005850 comp rx=0 tx=0).stop 2024-03-09T00:12:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad8100420 msgr2=0x7f1ad8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:12:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad8100420 0x7f1ad8198ab0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000d950 tx=0x7f1ad000de10 comp rx=0 tx=0).stop 2024-03-09T00:12:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 shutdown_connections 2024-03-09T00:12:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f1ac00a2970 0x7f1ac00a4e30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ad8199530 0x7f1ad81a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81054f0 0x7f1ad8198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 --2- 172.21.15.22:0/890112462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ad8100420 0x7f1ad8198ab0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:12:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 >> 172.21.15.22:0/890112462 conn(0x7f1ad8077290 msgr2=0x7f1ad80fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:12:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 shutdown_connections 2024-03-09T00:12:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:12:30.119+0000 7f1ae03cd640 1 -- 172.21.15.22:0/890112462 wait complete. 2024-03-09T00:12:32.411 DEBUG:tasks.check_counter:Getting stats from a 2024-03-09T00:12:32.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-09T00:12:32.412 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:30 smithi022 ceph-mon[31894]: pgmap v573: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.6 MiB/s wr, 166 op/s 2024-03-09T00:12:32.412 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:30 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/211041690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-09T00:12:32.413 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:30 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/4006810395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-09T00:12:32.413 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:30 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/890112462' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-09T00:12:32.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:32 smithi062 ceph-mon[36129]: pgmap v574: 129 pgs: 129 active+clean; 389 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.9 MiB/s wr, 104 op/s 2024-03-09T00:12:32.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:32 smithi192 ceph-mon[35995]: pgmap v574: 129 pgs: 129 active+clean; 389 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.9 MiB/s wr, 104 op/s 2024-03-09T00:12:32.666 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-09T00:12:32.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:32 smithi022 ceph-mon[31894]: pgmap v574: 129 pgs: 129 active+clean; 389 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.9 MiB/s wr, 104 op/s 2024-03-09T00:12:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:34 smithi022 ceph-mon[31894]: pgmap v575: 129 pgs: 129 active+clean; 387 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 3.0 MiB/s wr, 110 op/s 2024-03-09T00:12:34.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:34 smithi062 ceph-mon[36129]: pgmap v575: 129 pgs: 129 active+clean; 387 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 3.0 MiB/s wr, 110 op/s 2024-03-09T00:12:34.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:34 smithi192 ceph-mon[35995]: pgmap v575: 129 pgs: 129 active+clean; 387 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 3.0 MiB/s wr, 110 op/s 2024-03-09T00:12:35.243 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:12:35.243 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:35.243 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 344, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 332, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1746752, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 200336, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.053871077, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.017574597, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.038674144, 2024-03-09T00:12:35.244 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.006160989, 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 331, 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.011910641, 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000035983 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:35.245 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1746752, 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 200336 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 7, 2024-03-09T00:12:35.246 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 258, 2024-03-09T00:12:35.247 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3552, 2024-03-09T00:12:35.247 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 278544, 2024-03-09T00:12:35.247 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 6, 2024-03-09T00:12:35.247 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-03-09T00:12:35.247 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.017138616, 2024-03-09T00:12:35.248 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.011965090, 2024-03-09T00:12:35.248 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.002523029, 2024-03-09T00:12:35.248 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:35.248 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:35.248 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 256, 2024-03-09T00:12:35.248 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.020500060, 2024-03-09T00:12:35.249 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000080078 2024-03-09T00:12:35.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.249 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:35.249 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:35.249 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:35.249 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 278544 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-03-09T00:12:35.250 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.000189204, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000118896, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:35.251 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:35.253 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:35.254 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:35.255 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:35.256 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:35.257 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148709, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:35.258 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:35.259 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:35.259 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-03-09T00:12:35.259 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-03-09T00:12:35.259 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:35.259 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:35.260 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 347, 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1698224, 2024-03-09T00:12:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:35.262 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 347, 2024-03-09T00:12:35.262 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-03-09T00:12:35.262 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-03-09T00:12:35.262 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 347, 2024-03-09T00:12:35.262 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1698224, 2024-03-09T00:12:35.263 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-03-09T00:12:35.263 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:35.263 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:35.263 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:35.263 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:35.263 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:35.263 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:12:35.950 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-09T00:12:35.951 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-09T00:12:35.951 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-03-09T00:12:35.951 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-03-09T00:12:35.951 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-09T00:12:35.951 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-09T00:12:35.951 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-09T00:12:35.951 DEBUG:tasks.check_counter:Getting stats from d 2024-03-09T00:12:35.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-09T00:12:36.201 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-09T00:12:36.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:36 smithi022 ceph-mon[31894]: pgmap v576: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 3.7 MiB/s wr, 143 op/s 2024-03-09T00:12:36.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:36 smithi062 ceph-mon[36129]: pgmap v576: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 3.7 MiB/s wr, 143 op/s 2024-03-09T00:12:36.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:36 smithi192 ceph-mon[35995]: pgmap v576: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 3.7 MiB/s wr, 143 op/s 2024-03-09T00:12:38.079 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-03-09T00:12:38.080 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.001038906, 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000182500, 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:38.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-03-09T00:12:38.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 352, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 340, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1749616, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 204576, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.053916589, 2024-03-09T00:12:38.083 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.018105986, 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.038698137, 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.006204920, 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 339, 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.013297441, 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000039225 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.084 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1749616, 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 204576 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:38.085 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 7, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 262, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3552, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 279248, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 6, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.017735610, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.012717063, 2024-03-09T00:12:38.086 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.002343773, 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 260, 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.024596812, 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000094603 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:38.087 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 279248 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-03-09T00:12:38.088 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:38.089 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:38.090 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:38.091 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:38.092 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:38.092 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:38.092 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:38.092 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:38.092 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:38.092 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:38.092 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148692, 2024-03-09T00:12:38.093 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 30, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:38.094 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-03-09T00:12:38.095 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:38.096 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-03-09T00:12:38.096 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:38.096 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-03-09T00:12:38.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:38.097 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:38.097 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-03-09T00:12:38.097 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-03-09T00:12:38.097 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 355, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1699965, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 355, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 355, 2024-03-09T00:12:38.098 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1699965, 2024-03-09T00:12:38.099 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-03-09T00:12:38.099 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:38.099 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:38.099 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:38.099 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:38.099 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:38.100 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:12:38.485 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:38 smithi022 ceph-mon[31894]: pgmap v577: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 2.4 MiB/s wr, 83 op/s 2024-03-09T00:12:38.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:38 smithi062 ceph-mon[36129]: pgmap v577: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 2.4 MiB/s wr, 83 op/s 2024-03-09T00:12:38.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:38 smithi192 ceph-mon[35995]: pgmap v577: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 2.4 MiB/s wr, 83 op/s 2024-03-09T00:12:38.943 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-09T00:12:38.943 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-09T00:12:38.944 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-09T00:12:38.944 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-09T00:12:38.944 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-09T00:12:38.944 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-09T00:12:38.944 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-09T00:12:38.944 DEBUG:tasks.check_counter:Getting stats from g 2024-03-09T00:12:38.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-09T00:12:39.196 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-09T00:12:40.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:40 smithi022 ceph-mon[31894]: pgmap v578: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 2.1 MiB/s wr, 75 op/s 2024-03-09T00:12:40.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:40 smithi062 ceph-mon[36129]: pgmap v578: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 2.1 MiB/s wr, 75 op/s 2024-03-09T00:12:40.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:40 smithi192 ceph-mon[35995]: pgmap v578: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 2.1 MiB/s wr, 75 op/s 2024-03-09T00:12:40.891 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:12:40.891 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:40.891 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 347, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 335, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1747632, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 201744, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.053966823, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.017949178, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.038260842, 2024-03-09T00:12:40.892 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.006242299, 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 334, 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.012211455, 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000036561 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:40.893 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1747632, 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 201744 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2, 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 256, 2024-03-09T00:12:40.894 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 320, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 200240, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 5, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.017294372, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.012583644, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.001954554, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:40.895 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 253, 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.016937467, 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000066946 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:40.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 200240 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 4, 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 4, 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 3072, 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 448, 2024-03-09T00:12:40.897 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.000714624, 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000031227, 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000642964, 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 3, 2024-03-09T00:12:40.898 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000692496, 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000230832 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-09T00:12:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 448 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:40.900 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:40.901 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:40.902 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:40.903 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:40.904 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:40.904 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:40.904 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:40.904 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-03-09T00:12:40.904 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:40.904 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:40.905 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:40.905 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:40.905 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148994, 2024-03-09T00:12:40.905 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-03-09T00:12:40.905 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:12:40.905 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-03-09T00:12:40.905 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-03-09T00:12:40.906 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-03-09T00:12:40.906 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:40.906 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:40.906 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:40.906 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:40.906 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-03-09T00:12:40.906 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-03-09T00:12:40.907 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:40.907 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:40.907 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:40.907 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-03-09T00:12:40.907 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-03-09T00:12:40.907 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-03-09T00:12:40.907 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-03-09T00:12:40.908 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 349, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1698651, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 349, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 349, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1698651, 2024-03-09T00:12:40.909 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-03-09T00:12:40.910 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:40.910 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:40.910 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:40.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:40.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:40.910 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:12:41.816 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-09T00:12:41.816 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-09T00:12:41.816 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-09T00:12:41.816 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-09T00:12:41.816 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-09T00:12:41.816 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-09T00:12:41.816 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-09T00:12:41.816 DEBUG:tasks.check_counter:Getting stats from j 2024-03-09T00:12:41.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-09T00:12:42.062 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.a/config 2024-03-09T00:12:42.449 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:42 smithi022 ceph-mon[31894]: pgmap v579: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 2.1 MiB/s wr, 78 op/s 2024-03-09T00:12:42.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:42 smithi062 ceph-mon[36129]: pgmap v579: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 2.1 MiB/s wr, 78 op/s 2024-03-09T00:12:42.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:42 smithi192 ceph-mon[35995]: pgmap v579: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 2.1 MiB/s wr, 78 op/s 2024-03-09T00:12:43.823 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-03-09T00:12:43.823 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:43.823 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2024-03-09T00:12:43.823 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.001504687, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000368399, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:43.824 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:43.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.826 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-09T00:12:43.826 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2024-03-09T00:12:43.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.826 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:43.826 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 3, 2024-03-09T00:12:43.826 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 261, 2024-03-09T00:12:43.826 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 480, 2024-03-09T00:12:43.827 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 278976, 2024-03-09T00:12:43.827 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 6, 2024-03-09T00:12:43.829 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2024-03-09T00:12:43.830 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.018420850, 2024-03-09T00:12:43.830 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.013351423, 2024-03-09T00:12:43.830 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.001980461, 2024-03-09T00:12:43.830 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:43.830 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:43.830 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 258, 2024-03-09T00:12:43.831 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.012948611, 2024-03-09T00:12:43.831 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000050188 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 278976 2024-03-09T00:12:43.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 358, 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 344, 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1753232, 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 206272, 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2024-03-09T00:12:43.833 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.058282111, 2024-03-09T00:12:43.834 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.018321841, 2024-03-09T00:12:43.834 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.042447619, 2024-03-09T00:12:43.834 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.006325931, 2024-03-09T00:12:43.834 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:43.834 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 344, 2024-03-09T00:12:43.834 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.016824014, 2024-03-09T00:12:43.834 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000048907 2024-03-09T00:12:43.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.835 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:43.835 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:43.835 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:43.835 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:43.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.835 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1753232, 2024-03-09T00:12:43.836 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 206272 2024-03-09T00:12:43.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.837 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2024-03-09T00:12:43.837 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2024-03-09T00:12:43.837 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2024-03-09T00:12:43.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.838 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2024-03-09T00:12:43.838 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:43.838 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:43.839 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:43.840 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:43.841 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:43.842 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148754, 2024-03-09T00:12:43.843 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 31, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:43.844 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:43.845 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2024-03-09T00:12:43.845 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2024-03-09T00:12:43.845 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:43.845 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:43.845 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:43.845 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:43.846 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 357, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1700225, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 357, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2024-03-09T00:12:43.847 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 357, 2024-03-09T00:12:43.848 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1700225, 2024-03-09T00:12:43.848 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2024-03-09T00:12:43.848 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2024-03-09T00:12:43.848 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2024-03-09T00:12:43.848 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2024-03-09T00:12:43.848 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:43.849 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-03-09T00:12:43.849 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-03-09T00:12:44.492 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-09T00:12:44.492 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-09T00:12:44.492 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-09T00:12:44.492 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-09T00:12:44.492 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-09T00:12:44.492 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-09T00:12:44.493 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-09T00:12:44.493 DEBUG:tasks.check_counter:Getting stats from b 2024-03-09T00:12:44.493 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-09T00:12:44.525 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:44 smithi022 ceph-mon[31894]: pgmap v580: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1010 KiB/s wr, 54 op/s 2024-03-09T00:12:44.527 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:44 smithi062 ceph-mon[36129]: pgmap v580: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1010 KiB/s wr, 54 op/s 2024-03-09T00:12:44.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:44 smithi192 ceph-mon[35995]: pgmap v580: 129 pgs: 129 active+clean; 382 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1010 KiB/s wr, 54 op/s 2024-03-09T00:12:44.759 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-09T00:12:46.594 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:46 smithi062 ceph-mon[36129]: pgmap v581: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 961 KiB/s wr, 54 op/s 2024-03-09T00:12:46.634 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:46 smithi192 ceph-mon[35995]: pgmap v581: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 961 KiB/s wr, 54 op/s 2024-03-09T00:12:47.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:46 smithi022 ceph-mon[31894]: pgmap v581: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 961 KiB/s wr, 54 op/s 2024-03-09T00:12:47.234 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-09T00:12:47.234 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:47.234 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 405860, 2024-03-09T00:12:47.234 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 205886, 2024-03-09T00:12:47.234 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 184082297, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 42117518, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 11, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 17.235063504, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 6.070020239, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 15.705955731, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.104133137, 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:47.235 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 205886, 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.703141505, 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000017986 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 605519, 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.072045233, 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000118 2024-03-09T00:12:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1861280, 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 243552 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 50288, 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 53252, 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 12847370, 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 13970536, 2024-03-09T00:12:47.237 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 86, 2024-03-09T00:12:47.238 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-03-09T00:12:47.238 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 2.983629568, 2024-03-09T00:12:47.238 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 1.232004127, 2024-03-09T00:12:47.238 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 2.415597309, 2024-03-09T00:12:47.238 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.020822850, 2024-03-09T00:12:47.238 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:47.238 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 53249, 2024-03-09T00:12:47.239 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.366769023, 2024-03-09T00:12:47.239 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000025667 2024-03-09T00:12:47.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.239 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:47.239 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 52260, 2024-03-09T00:12:47.239 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.026695012, 2024-03-09T00:12:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000510 2024-03-09T00:12:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 5008, 2024-03-09T00:12:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 140032 2024-03-09T00:12:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:47.240 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 810, 2024-03-09T00:12:47.241 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1031, 2024-03-09T00:12:47.241 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 265050, 2024-03-09T00:12:47.241 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 934777, 2024-03-09T00:12:47.241 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 14, 2024-03-09T00:12:47.241 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-03-09T00:12:47.241 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.067462786, 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.020865488, 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.018973029, 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.022246763, 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1029, 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.077093944, 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000074921 2024-03-09T00:12:47.242 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4, 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000001179, 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000294 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 476528 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.243 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2733, 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.522783346, 2024-03-09T00:12:47.244 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000557183 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1323, 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.081897270, 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000061902 2024-03-09T00:12:47.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 46500, 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 3880, 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3880, 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 17.391142738, 2024-03-09T00:12:47.246 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004482253 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 42620, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 20, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 1, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-03-09T00:12:47.247 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 309, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 126, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 183, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 128, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 1, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 17, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 17, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 11, 2024-03-09T00:12:47.248 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 51252, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 3978, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 42615, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 110, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 119, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-03-09T00:12:47.249 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 2, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 22, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 245, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 747, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 212221307, 2024-03-09T00:12:47.250 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 7, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 245, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 245, 2024-03-09T00:12:47.251 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 225, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 2939, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 1958, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 788, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 336, 2024-03-09T00:12:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 3525 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 114, 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 412, 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 257, 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 325, 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 120, 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-03-09T00:12:47.253 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 561, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 561, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-03-09T00:12:47.254 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 1, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-03-09T00:12:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 5104, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 5104, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-03-09T00:12:47.256 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 7, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 7, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 12033689, 2024-03-09T00:12:47.257 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1207, 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 11.657899712, 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.009658574 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.258 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 269, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 830, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 561, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 169, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 239, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 70, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 309, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 1460, 2024-03-09T00:12:47.259 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 1151, 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 17, 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 790, 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 773, 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 59880, 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 207228 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 49414, 2024-03-09T00:12:47.260 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 46500, 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 128, 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 869, 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 563, 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 9.530961738, 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.016928884 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.261 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-03-09T00:12:47.262 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 5, 2024-03-09T00:12:47.262 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.001265512, 2024-03-09T00:12:47.262 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000253102 2024-03-09T00:12:47.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.262 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-03-09T00:12:47.262 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.263 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1151, 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.139165378, 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000989717 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.265 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-03-09T00:12:47.266 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.267 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.268 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.269 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1474, 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.304072488, 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000884716 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.270 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 119, 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.047350186, 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.017204623 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.272 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-03-09T00:12:47.273 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 7, 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.063294408, 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.009042058 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.274 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.276 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-03-09T00:12:47.277 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 561, 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.305033028, 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.005891324 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.278 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 5, 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 3, 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 2, 2024-03-09T00:12:47.279 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 1505, 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 752, 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 1036, 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:47.280 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:47.281 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:47.282 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:47.283 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 310884, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 291, 2024-03-09T00:12:47.284 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 792, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 9, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 9720, 2024-03-09T00:12:47.285 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 202, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 1074685, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 7483, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:47.286 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 3092, 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 8131908, 2024-03-09T00:12:47.287 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 27, 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 3092, 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3092, 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 37.724427979, 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.012200655 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-03-09T00:12:47.288 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3092, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3322 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 3092, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 357, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 2735, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-03-09T00:12:47.289 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 10, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 112, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 1319, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 90, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 1, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 1135, 2024-03-09T00:12:47.290 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 245, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 20, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-03-09T00:12:47.291 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-03-09T00:12:47.292 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-03-09T00:12:47.293 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 126, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 37, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 12, 2024-03-09T00:12:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 17, 2024-03-09T00:12:47.295 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 5, 2024-03-09T00:12:47.295 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-03-09T00:12:47.295 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 316, 2024-03-09T00:12:47.295 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-03-09T00:12:47.295 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 74 2024-03-09T00:12:47.295 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.295 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-03-09T00:12:47.297 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-03-09T00:12:47.297 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-03-09T00:12:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-03-09T00:12:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-03-09T00:12:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-03-09T00:12:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-03-09T00:12:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 16, 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 8, 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 1122, 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 561, 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.299 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 456954, 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 156829455, 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 456954, 2024-03-09T00:12:47.300 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 456954, 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 156829455, 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:47.302 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3092, 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 8193225, 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3092, 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 8193225, 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.303 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:47.304 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3092, 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 3092, 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3092, 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3092, 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.305 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:47.306 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 561, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 61317, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 561, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 112, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 61317, 2024-03-09T00:12:47.307 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x55cacf6306e0": { 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 4592, 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-03-09T00:12:47.308 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 5104, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 7839385, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 5104, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1207, 2024-03-09T00:12:47.309 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 7834793, 2024-03-09T00:12:47.310 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:47.310 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:47.310 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:47.310 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:47.310 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.310 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:47.310 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-09T00:12:47.425 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-09T00:12:47.425 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-03-09T00:12:47.425 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=114 2024-03-09T00:12:47.425 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=412 2024-03-09T00:12:47.426 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-03-09T00:12:47.426 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-03-09T00:12:47.426 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-03-09T00:12:47.426 DEBUG:tasks.check_counter:Getting stats from e 2024-03-09T00:12:47.426 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-09T00:12:47.724 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-09T00:12:48.591 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:48 smithi062 ceph-mon[36129]: pgmap v582: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-03-09T00:12:48.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:48 smithi192 ceph-mon[35995]: pgmap v582: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-03-09T00:12:48.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:48 smithi022 ceph-mon[31894]: pgmap v582: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-03-09T00:12:49.573 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-09T00:12:49.574 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:49.574 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 410, 2024-03-09T00:12:49.574 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 385, 2024-03-09T00:12:49.574 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1842306, 2024-03-09T00:12:49.574 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 213492, 2024-03-09T00:12:49.574 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.062422802, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.019805554, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.044130284, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.007634507, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 384, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.021884328, 2024-03-09T00:12:49.575 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000056990 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1841472, 2024-03-09T00:12:49.576 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 213168 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1184, 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1443, 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1879017, 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 988809, 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-03-09T00:12:49.577 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.102045537, 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.033187979, 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.023138836, 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.034561208, 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1440, 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.057848522, 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000040172 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.578 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 612928 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.579 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2563, 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2563, 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 4955835, 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 814268, 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.207595691, 2024-03-09T00:12:49.580 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.069106080, 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.053156254, 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.066828818, 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2562, 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.111375800, 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000043472 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:49.581 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.582 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.582 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.582 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.582 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-09T00:12:49.582 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-03-09T00:12:49.582 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6219, 2024-03-09T00:12:49.583 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.150210818, 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000024153 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.584 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.585 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-03-09T00:12:49.586 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 874, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 874, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 874, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-03-09T00:12:49.587 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 4, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-03-09T00:12:49.588 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-03-09T00:12:49.589 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-03-09T00:12:49.590 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-09T00:12:49.591 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-03-09T00:12:49.592 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-03-09T00:12:49.593 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-03-09T00:12:49.594 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-03-09T00:12:49.595 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 3848, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 5, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-03-09T00:12:49.596 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 9974719, 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 9974120, 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.597 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 3848 2024-03-09T00:12:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-03-09T00:12:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 62, 2024-03-09T00:12:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 851, 2024-03-09T00:12:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 789, 2024-03-09T00:12:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 17, 2024-03-09T00:12:49.599 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 17, 2024-03-09T00:12:49.599 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-03-09T00:12:49.599 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 874, 2024-03-09T00:12:49.599 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 874, 2024-03-09T00:12:49.599 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 0, 2024-03-09T00:12:49.599 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-03-09T00:12:49.600 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-03-09T00:12:49.600 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-03-09T00:12:49.600 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 51336, 2024-03-09T00:12:49.600 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215420 2024-03-09T00:12:49.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.600 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-03-09T00:12:49.600 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-03-09T00:12:49.601 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-03-09T00:12:49.601 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-03-09T00:12:49.601 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-03-09T00:12:49.601 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-03-09T00:12:49.601 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-03-09T00:12:49.601 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.602 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.602 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.602 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.602 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-03-09T00:12:49.602 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.602 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.602 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.603 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-03-09T00:12:49.603 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.603 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.603 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.604 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-03-09T00:12:49.604 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.604 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.604 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.604 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-03-09T00:12:49.604 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.605 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.605 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.605 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-03-09T00:12:49.605 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.605 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.605 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.606 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-03-09T00:12:49.606 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.606 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.606 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.606 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-03-09T00:12:49.607 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.607 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.607 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.607 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-03-09T00:12:49.607 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.608 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.608 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.608 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.608 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-03-09T00:12:49.608 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.608 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.609 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.609 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-03-09T00:12:49.609 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.609 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.609 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.610 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-03-09T00:12:49.610 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.610 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.611 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.611 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-03-09T00:12:49.611 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.611 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.611 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-03-09T00:12:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-03-09T00:12:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-03-09T00:12:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.614 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.615 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.616 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.618 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.619 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.620 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-03-09T00:12:49.621 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-03-09T00:12:49.622 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 3, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 2, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 2, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 4227, 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-03-09T00:12:49.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:49.624 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:49.625 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:49.626 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:49.627 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 201813, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 114, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-03-09T00:12:49.628 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 9720, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 202, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:49.629 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 792153, 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 4904, 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.630 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 3742, 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 3742, 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3742, 2024-03-09T00:12:49.631 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 4.172935767, 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001115161 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3742, 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3743 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 3742, 2024-03-09T00:12:49.632 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 3742, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 2368, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 1372, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-03-09T00:12:49.633 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-03-09T00:12:49.634 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-03-09T00:12:49.635 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-03-09T00:12:49.636 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-03-09T00:12:49.637 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 126, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 37, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 6, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 7, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 1, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-03-09T00:12:49.638 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-03-09T00:12:49.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-03-09T00:12:49.640 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 4153, 2024-03-09T00:12:49.641 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 8279542, 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 4153, 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4153, 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 8279542, 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:49.642 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:49.643 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3742, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 5780884, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3742, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 5780884, 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:49.644 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-03-09T00:12:49.645 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3742, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 3742, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3742, 2024-03-09T00:12:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3742, 2024-03-09T00:12:49.647 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:49.647 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.647 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.647 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.647 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.647 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-03-09T00:12:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-03-09T00:12:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:49.648 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:49.649 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:49.649 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:49.649 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-03-09T00:12:49.649 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-03-09T00:12:49.649 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:49.649 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x55e3fcc8e960": { 2024-03-09T00:12:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:49.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-03-09T00:12:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:49.651 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:49.652 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-09T00:12:50.362 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-09T00:12:50.362 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-09T00:12:50.362 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-09T00:12:50.362 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-09T00:12:50.362 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-09T00:12:50.363 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-09T00:12:50.363 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-09T00:12:50.363 DEBUG:tasks.check_counter:Getting stats from h 2024-03-09T00:12:50.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-09T00:12:50.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:50 smithi062 ceph-mon[36129]: pgmap v583: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-03-09T00:12:50.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:50 smithi192 ceph-mon[35995]: pgmap v583: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-03-09T00:12:50.642 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-09T00:12:50.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:50 smithi022 ceph-mon[31894]: pgmap v583: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-03-09T00:12:52.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:52 smithi062 ceph-mon[36129]: pgmap v584: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-03-09T00:12:52.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:52 smithi192 ceph-mon[35995]: pgmap v584: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-03-09T00:12:52.663 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-09T00:12:52.664 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:52.664 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1190, 2024-03-09T00:12:52.664 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1192, 2024-03-09T00:12:52.664 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 323275870, 2024-03-09T00:12:52.664 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 378436, 2024-03-09T00:12:52.664 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 8, 2024-03-09T00:12:52.664 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-03-09T00:12:52.665 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.243611559, 2024-03-09T00:12:52.665 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.020223348, 2024-03-09T00:12:52.665 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.186427127, 2024-03-09T00:12:52.665 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.028141478, 2024-03-09T00:12:52.665 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:52.665 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1190, 2024-03-09T00:12:52.665 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.047420249, 2024-03-09T00:12:52.666 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000039848 2024-03-09T00:12:52.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.666 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:52.666 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.666 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.666 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1739, 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1872, 2024-03-09T00:12:52.667 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 297868313, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 872537, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.283452009, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.044171886, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.184009617, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.042650989, 2024-03-09T00:12:52.668 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:52.669 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1870, 2024-03-09T00:12:52.669 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.083670281, 2024-03-09T00:12:52.669 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000044743 2024-03-09T00:12:52.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.669 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:52.669 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.670 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.670 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.670 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-09T00:12:52.670 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 319792 2024-03-09T00:12:52.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.671 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:52.671 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 628, 2024-03-09T00:12:52.671 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 623, 2024-03-09T00:12:52.671 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 117263118, 2024-03-09T00:12:52.672 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 242902, 2024-03-09T00:12:52.672 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-03-09T00:12:52.672 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-03-09T00:12:52.672 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.106897743, 2024-03-09T00:12:52.672 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.019276150, 2024-03-09T00:12:52.672 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.071178788, 2024-03-09T00:12:52.673 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.013130105, 2024-03-09T00:12:52.673 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:52.673 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 623, 2024-03-09T00:12:52.673 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.032604445, 2024-03-09T00:12:52.673 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000052334 2024-03-09T00:12:52.674 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.674 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:52.674 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.674 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.674 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 236592, 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 111568 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-03-09T00:12:52.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.676 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-03-09T00:12:52.676 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-03-09T00:12:52.676 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-03-09T00:12:52.676 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 5130, 2024-03-09T00:12:52.676 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.480389622, 2024-03-09T00:12:52.676 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000093643 2024-03-09T00:12:52.677 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.677 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.677 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-03-09T00:12:52.677 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-03-09T00:12:52.677 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-03-09T00:12:52.677 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.677 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.678 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.678 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.678 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-03-09T00:12:52.678 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-03-09T00:12:52.678 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-03-09T00:12:52.678 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-03-09T00:12:52.679 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.679 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.679 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.679 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-03-09T00:12:52.679 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-03-09T00:12:52.679 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-03-09T00:12:52.680 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-03-09T00:12:52.680 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-03-09T00:12:52.680 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-03-09T00:12:52.680 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-03-09T00:12:52.680 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 42388, 2024-03-09T00:12:52.680 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 27213, 2024-03-09T00:12:52.680 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 15175, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 3513, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 13, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-03-09T00:12:52.681 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-03-09T00:12:52.682 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 674, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 217410574, 2024-03-09T00:12:52.683 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 7, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-03-09T00:12:52.684 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-03-09T00:12:52.685 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-03-09T00:12:52.685 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-03-09T00:12:52.685 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-03-09T00:12:52.685 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-03-09T00:12:52.685 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-09T00:12:52.685 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-03-09T00:12:52.686 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-03-09T00:12:52.687 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-03-09T00:12:52.687 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-03-09T00:12:52.687 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-03-09T00:12:52.687 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-03-09T00:12:52.687 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-03-09T00:12:52.687 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-03-09T00:12:52.687 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-03-09T00:12:52.688 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.689 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-03-09T00:12:52.690 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-03-09T00:12:52.690 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-03-09T00:12:52.690 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-03-09T00:12:52.690 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-03-09T00:12:52.690 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 186288, 2024-03-09T00:12:52.690 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-03-09T00:12:52.690 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 12, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 44, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-03-09T00:12:52.691 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 1143178338, 2024-03-09T00:12:52.692 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 1169759241, 2024-03-09T00:12:52.692 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 1169745600, 2024-03-09T00:12:52.692 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-03-09T00:12:52.692 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.692 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.692 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 186288 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 3541, 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 41732, 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 38191, 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 206, 2024-03-09T00:12:52.693 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 311, 2024-03-09T00:12:52.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 105, 2024-03-09T00:12:52.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 42389, 2024-03-09T00:12:52.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 42417, 2024-03-09T00:12:52.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 28, 2024-03-09T00:12:52.694 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-03-09T00:12:52.695 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-03-09T00:12:52.695 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-03-09T00:12:52.695 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 98192, 2024-03-09T00:12:52.695 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215396 2024-03-09T00:12:52.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.695 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-03-09T00:12:52.695 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.696 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-03-09T00:12:52.697 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.699 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-03-09T00:12:52.699 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.699 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.699 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.699 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.700 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-03-09T00:12:52.701 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.703 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-03-09T00:12:52.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.703 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.703 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-03-09T00:12:52.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.704 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.704 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.705 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.706 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-03-09T00:12:52.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.707 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.707 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-03-09T00:12:52.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.707 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.708 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-03-09T00:12:52.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.712 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-03-09T00:12:52.713 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-03-09T00:12:52.715 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-03-09T00:12:52.716 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 2, 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 0, 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 2669, 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-03-09T00:12:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:52.720 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:52.721 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 392370, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 4223, 2024-03-09T00:12:52.722 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:52.723 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 9720, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 202, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 38839473, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 267393, 2024-03-09T00:12:52.724 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-03-09T00:12:52.725 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 3341, 2024-03-09T00:12:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 26, 2024-03-09T00:12:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 4, 2024-03-09T00:12:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 3341, 2024-03-09T00:12:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-03-09T00:12:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3341, 2024-03-09T00:12:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 13.833789794, 2024-03-09T00:12:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004140613 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3341, 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3342 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 3341, 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 3341, 2024-03-09T00:12:52.727 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 1958, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 1380, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-03-09T00:12:52.728 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-03-09T00:12:52.729 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 2, 2024-03-09T00:12:52.730 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-03-09T00:12:52.730 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-03-09T00:12:52.730 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-03-09T00:12:52.730 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-03-09T00:12:52.730 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-03-09T00:12:52.730 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-03-09T00:12:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-03-09T00:12:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 126, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-03-09T00:12:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 25, 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 11, 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 11, 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-03-09T00:12:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-03-09T00:12:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-03-09T00:12:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 3553, 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 738073719, 2024-03-09T00:12:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 3553, 2024-03-09T00:12:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3553, 2024-03-09T00:12:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 738073719, 2024-03-09T00:12:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3341, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 737257144, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3341, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 737257144, 2024-03-09T00:12:52.740 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:52.741 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3341, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 3341, 2024-03-09T00:12:52.742 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3341, 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3341, 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-03-09T00:12:52.743 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:52.744 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x55c78299e960": { 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:52.746 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:52.747 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.748 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:52.748 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-09T00:12:52.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:52 smithi022 ceph-mon[31894]: pgmap v584: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 13 op/s 2024-03-09T00:12:53.458 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-09T00:12:53.458 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-09T00:12:53.458 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-09T00:12:53.458 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-03-09T00:12:53.458 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-03-09T00:12:53.458 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-09T00:12:53.459 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-09T00:12:53.459 DEBUG:tasks.check_counter:Getting stats from k 2024-03-09T00:12:53.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-09T00:12:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.b/config 2024-03-09T00:12:54.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:54 smithi062 ceph-mon[36129]: pgmap v585: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-03-09T00:12:54.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:54 smithi192 ceph-mon[35995]: pgmap v585: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-03-09T00:12:54.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:54 smithi022 ceph-mon[31894]: pgmap v585: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 8 op/s 2024-03-09T00:12:55.454 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-09T00:12:55.454 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.001168929, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-03-09T00:12:55.455 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000267067, 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:55.456 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 3, 2024-03-09T00:12:55.457 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 265, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 480, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 279680, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.014461580, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.009731300, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001863981, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:12:55.458 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 262, 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.016329049, 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000062324 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:55.459 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 279680 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 362, 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 348, 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1754320, 2024-03-09T00:12:55.460 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 208768, 2024-03-09T00:12:55.461 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-03-09T00:12:55.461 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-03-09T00:12:55.461 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.060228783, 2024-03-09T00:12:55.461 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.017734027, 2024-03-09T00:12:55.461 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.044741483, 2024-03-09T00:12:55.461 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.007538057, 2024-03-09T00:12:55.461 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 348, 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.016618176, 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000047753 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:55.462 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1754320, 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 208768 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-03-09T00:12:55.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:55.464 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:55.465 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:55.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:55.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:55.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:55.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:55.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:55.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:55.466 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:55.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:55.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:55.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:55.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:55.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:55.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:55.467 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:55.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:55.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-03-09T00:12:55.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:55.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:55.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:55.468 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148794, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 32, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:55.469 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:55.470 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:55.471 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 361, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1700745, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 361, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-03-09T00:12:55.472 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 361, 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1700745, 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-09T00:12:55.473 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-09T00:12:56.188 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-09T00:12:56.188 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-09T00:12:56.189 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-03-09T00:12:56.189 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-03-09T00:12:56.189 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-09T00:12:56.189 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-09T00:12:56.189 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-09T00:12:56.189 DEBUG:tasks.check_counter:Getting stats from c 2024-03-09T00:12:56.189 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-09T00:12:56.555 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-09T00:12:56.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:56 smithi062 ceph-mon[36129]: pgmap v586: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-03-09T00:12:56.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:56 smithi192 ceph-mon[35995]: pgmap v586: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-03-09T00:12:56.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:56 smithi022 ceph-mon[31894]: pgmap v586: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-03-09T00:12:58.599 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:12:58 smithi192 ceph-mon[35995]: pgmap v587: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-03-09T00:12:58.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:12:58 smithi062 ceph-mon[36129]: pgmap v587: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-03-09T00:12:58.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:12:58 smithi022 ceph-mon[31894]: pgmap v587: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-03-09T00:12:58.949 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-09T00:12:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:12:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 49912, 2024-03-09T00:12:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 54008, 2024-03-09T00:12:58.949 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 12737892, 2024-03-09T00:12:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 15282829, 2024-03-09T00:12:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 31, 2024-03-09T00:12:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-03-09T00:12:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 2.273389234, 2024-03-09T00:12:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.969466242, 2024-03-09T00:12:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 1.675435874, 2024-03-09T00:12:58.950 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.043445660, 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 54006, 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.442613731, 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000026712 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 51696, 2024-03-09T00:12:58.951 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.022591096, 2024-03-09T00:12:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000436 2024-03-09T00:12:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 928, 2024-03-09T00:12:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 736 2024-03-09T00:12:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:12:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1639, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1612, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 2159777, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 2317978, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 11, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.152916120, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.047376517, 2024-03-09T00:12:58.953 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.067926332, 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.042992920, 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1611, 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.152723250, 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000094800 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:58.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 8, 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000001304, 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000163 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1853808, 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 227104 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 404915, 2024-03-09T00:12:58.955 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 205254, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 182186513, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 34799894, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 65, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 16.342662582, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 7.038938065, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 13.038231571, 2024-03-09T00:12:58.956 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.025711034, 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 205252, 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 3.044781224, 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000014834 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 607035, 2024-03-09T00:12:58.957 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.073135888, 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000120 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 617168 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-03-09T00:12:58.958 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2382, 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.577045409, 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000662067 2024-03-09T00:12:58.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1808, 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.120747477, 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000066785 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:58.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-03-09T00:12:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 47174, 2024-03-09T00:12:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 4421, 2024-03-09T00:12:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-03-09T00:12:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 4421, 2024-03-09T00:12:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 19.426019646, 2024-03-09T00:12:58.961 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.004394032 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 42753, 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 11, 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-03-09T00:12:58.962 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 221, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 78, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 143, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 109, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 4, 2024-03-09T00:12:58.963 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 4, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 4, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 52454, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 4532, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 42753, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 108, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 108, 2024-03-09T00:12:58.964 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 2, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 6, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 63, 2024-03-09T00:12:58.965 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 747, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 212221307, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 7, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 63, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-03-09T00:12:58.966 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 63, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 54, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 2706, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-03-09T00:12:58.967 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 439, 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 147, 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 125, 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 4303 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 53, 2024-03-09T00:12:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 534, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 318, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 445, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 59, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 789, 2024-03-09T00:12:58.969 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 789, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-03-09T00:12:58.970 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-03-09T00:12:58.971 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 3848, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 3848, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 5, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-03-09T00:12:58.972 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-03-09T00:12:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 5, 2024-03-09T00:12:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-03-09T00:12:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-03-09T00:12:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-03-09T00:12:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-03-09T00:12:58.973 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 9974719, 2024-03-09T00:12:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-03-09T00:12:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-03-09T00:12:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1063, 2024-03-09T00:12:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 5.656958635, 2024-03-09T00:12:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005321692 2024-03-09T00:12:58.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 175, 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 964, 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 789, 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 162, 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 291, 2024-03-09T00:12:58.975 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 129, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 221, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 1458, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 1237, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 4, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 830, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 826, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 59524, 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 207228 2024-03-09T00:12:58.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 50625, 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 47174, 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 22, 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 974, 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-03-09T00:12:58.977 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 791, 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 13.805883055, 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.017453708 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 21, 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.035907520, 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001709881 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.978 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1101, 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.881443299, 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000800584 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.980 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000192286, 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000192286 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.982 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.983 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.984 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.984 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.985 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.986 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-03-09T00:12:58.986 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.986 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.986 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.986 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1610, 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.174654233, 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000729598 2024-03-09T00:12:58.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 108, 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.722022430, 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.015944652 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.988 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-03-09T00:12:58.989 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.992 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.994 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 789, 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.805916823, 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002288867 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-03-09T00:12:58.995 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-03-09T00:12:58.996 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 1, 2024-03-09T00:12:58.997 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 3, 2024-03-09T00:12:58.997 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 2, 2024-03-09T00:12:58.997 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 1, 2024-03-09T00:12:58.997 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-03-09T00:12:58.997 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 1654, 2024-03-09T00:12:58.997 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 1654, 2024-03-09T00:12:58.997 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 1122, 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:12:58.998 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:12:58.999 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:12:58.999 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:12:58.999 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:12:58.999 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:12:58.999 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:12:58.999 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:12:58.999 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:12:59.000 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:12:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:12:59.002 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 306770, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 261, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 440, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 5, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-09T00:12:59.003 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 9720, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 202, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 786152, 2024-03-09T00:12:59.004 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 5296, 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-03-09T00:12:59.005 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 3253, 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 5967063, 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 27, 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 3253, 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3253, 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 39.495299557, 2024-03-09T00:12:59.006 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.012141192 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3253, 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 3480 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 3253, 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 193, 2024-03-09T00:12:59.007 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 3060, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 0, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 10, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 130, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 1193, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 101, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-03-09T00:12:59.008 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 1591, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 63, 2024-03-09T00:12:59.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 20, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-03-09T00:12:59.010 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-03-09T00:12:59.011 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-03-09T00:12:59.012 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-03-09T00:12:59.012 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-03-09T00:12:59.012 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-03-09T00:12:59.012 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-03-09T00:12:59.012 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-03-09T00:12:59.012 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-03-09T00:12:59.012 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 126, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 37, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 12, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 14, 2024-03-09T00:12:59.013 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 2, 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 293, 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 79 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-03-09T00:12:59.014 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-03-09T00:12:59.015 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-03-09T00:12:59.015 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-03-09T00:12:59.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.015 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-03-09T00:12:59.015 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-03-09T00:12:59.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.015 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 16, 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 8, 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 1578, 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 789, 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.016 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 456462, 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 156750930, 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 456462, 2024-03-09T00:12:59.017 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 456462, 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 156750930, 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:59.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:12:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:59.020 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:59.020 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 3253, 2024-03-09T00:12:59.020 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 6053128, 2024-03-09T00:12:59.020 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 3253, 2024-03-09T00:12:59.020 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 6053128, 2024-03-09T00:12:59.020 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:12:59.021 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:59.021 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:59.021 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:59.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:59.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.021 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-03-09T00:12:59.022 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:12:59.022 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-03-09T00:12:59.022 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:12:59.022 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:12:59.022 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:12:59.022 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:59.023 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:12:59.023 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 3253, 2024-03-09T00:12:59.023 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 3253, 2024-03-09T00:12:59.023 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 3253, 2024-03-09T00:12:59.023 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 3253, 2024-03-09T00:12:59.024 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:12:59.024 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:59.024 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:59.024 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:59.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:59.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-03-09T00:12:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:12:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-09T00:12:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:12:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 789, 2024-03-09T00:12:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 86065, 2024-03-09T00:12:59.025 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:59.026 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 789, 2024-03-09T00:12:59.026 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:12:59.026 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:12:59.026 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 130, 2024-03-09T00:12:59.026 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 86065, 2024-03-09T00:12:59.026 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55b07301a6e0": { 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-09T00:12:59.027 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 3848, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 5780415, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 3848, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1063, 2024-03-09T00:12:59.028 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 5780415, 2024-03-09T00:12:59.029 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:12:59.029 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:12:59.029 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:12:59.029 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:12:59.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:59.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:12:59.029 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-09T00:12:59.794 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-09T00:12:59.794 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-03-09T00:12:59.795 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=53 2024-03-09T00:12:59.795 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=534 2024-03-09T00:12:59.795 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-03-09T00:12:59.795 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-09T00:12:59.795 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-09T00:12:59.795 DEBUG:tasks.check_counter:Getting stats from f 2024-03-09T00:12:59.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-09T00:13:00.040 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-09T00:13:00.478 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:00 smithi192 ceph-mon[35995]: pgmap v588: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-03-09T00:13:00.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:00 smithi062 ceph-mon[36129]: pgmap v588: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-03-09T00:13:00.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:00 smithi022 ceph-mon[31894]: pgmap v588: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-03-09T00:13:01.874 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1648, 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1623, 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 2257648, 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 607782, 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2024-03-09T00:13:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.163241147, 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.049275433, 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.071897861, 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.042875083, 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1622, 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.071289219, 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000043951 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1842016, 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 214416 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 9, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 267, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 4108, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 702986, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 8, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.013590119, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.008469693, 2024-03-09T00:13:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.002819905, 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000065002, 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 265, 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.024214807, 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000091376 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 702288 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2536, 2024-03-09T00:13:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2538, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 8483934, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 806455, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.153090328, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.042551019, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.041782429, 2024-03-09T00:13:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.059918846, 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2536, 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.069293805, 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000027324 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-09T00:13:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-03-09T00:13:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 6332, 2024-03-09T00:13:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.141286200, 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000022313 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-03-09T00:13:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 746, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 746, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 746, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-03-09T00:13:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 11, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-03-09T00:13:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-03-09T00:13:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-03-09T00:13:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-09T00:13:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-03-09T00:13:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-03-09T00:13:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-03-09T00:13:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-03-09T00:13:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-03-09T00:13:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-03-09T00:13:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-03-09T00:13:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-03-09T00:13:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 5104, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-03-09T00:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 7, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 12033689, 2024-03-09T00:13:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 12033115, 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 5104 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-03-09T00:13:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 160, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 721, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 561, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 28, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 29, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 1, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 746, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 746, 2024-03-09T00:13:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 0, 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 51092, 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215420 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-03-09T00:13:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-03-09T00:13:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-03-09T00:13:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-03-09T00:13:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-03-09T00:13:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-03-09T00:13:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-03-09T00:13:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 5, 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 3, 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 3, 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-03-09T00:13:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 3463, 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:13:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:13:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:13:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-09T00:13:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 188381, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 199, 2024-03-09T00:13:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 9720, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 202, 2024-03-09T00:13:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 944041, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 6751, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-09T00:13:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 3775, 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 21, 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 3, 2024-03-09T00:13:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 3775, 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3775, 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 9.330832998, 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002471743 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3775, 2024-03-09T00:13:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 3776 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 3775, 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 3775, 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 2388, 2024-03-09T00:13:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 1385, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-03-09T00:13:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 1, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-03-09T00:13:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-03-09T00:13:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-03-09T00:13:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 126, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 37, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 4, 2024-03-09T00:13:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 6, 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 2, 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-03-09T00:13:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-03-09T00:13:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 4189, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 10344687, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 4189, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:13:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 4189, 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 10344687, 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-03-09T00:13:01.941 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 3775, 2024-03-09T00:13:01.942 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 7839390, 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 3775, 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 7839390, 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:13:01.944 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 3775, 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 3775, 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 3775, 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 3775, 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:01.946 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.947 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x5648c9e90960": { 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:01.948 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:01.949 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:01.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:01.950 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-09T00:13:02.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:02 smithi062 ceph-mon[36129]: pgmap v589: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-03-09T00:13:02.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:02 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:13:02.631 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:02 smithi192 ceph-mon[35995]: pgmap v589: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-03-09T00:13:02.632 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:02 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:13:02.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:02 smithi022 ceph-mon[31894]: pgmap v589: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-03-09T00:13:02.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:02 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-09T00:13:03.006 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-09T00:13:03.006 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-09T00:13:03.006 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-09T00:13:03.006 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-09T00:13:03.006 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-09T00:13:03.007 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-09T00:13:03.007 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-09T00:13:03.007 DEBUG:tasks.check_counter:Getting stats from i 2024-03-09T00:13:03.007 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-09T00:13:03.255 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-09T00:13:03.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:03 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:13:03.539 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:03 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:13:03.539 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:03 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:13:03.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:03 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:13:03.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:03 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:13:03.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:03 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:13:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:03 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-09T00:13:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:03 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-09T00:13:03.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:03 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' 2024-03-09T00:13:04.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:04 smithi022 ceph-mon[31894]: pgmap v590: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 9 op/s 2024-03-09T00:13:05.001 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:04 smithi192 ceph-mon[35995]: pgmap v590: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 9 op/s 2024-03-09T00:13:05.094 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-09T00:13:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 836427, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1138546, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 244020685, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 1448981709, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 435, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 9, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 41.619682280, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 19.658293916, 2024-03-09T00:13:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 22.114096059, 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 4.748487282, 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1138543, 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 128.951520618, 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000113260 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 928573, 2024-03-09T00:13:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.182768848, 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000196 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 106400, 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 35232 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 305866, 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 506015, 2024-03-09T00:13:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 67120852, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 602070544, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 12, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 26.980305861, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 12.933240671, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 10.690932869, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 3.623480270, 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 506016, 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 84.847016944, 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000167676 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 601525, 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.092887985, 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000154 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1880464, 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 755040 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 302730, 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 501751, 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 61797144, 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 478123148, 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 26, 2024-03-09T00:13:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 22.155627161, 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 8.788298618, 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 9.411382210, 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 3.817008943, 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 501748, 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 94.781757837, 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000188903 2024-03-09T00:13:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 599979, 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.098726591, 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000164 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 625232, 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 342704 2024-03-09T00:13:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 200816, 2024-03-09T00:13:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 216.373313209, 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001077470 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 112027, 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 26.088275738, 2024-03-09T00:13:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000232874 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 485846, 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 484982, 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 484982, 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2498.104936424, 2024-03-09T00:13:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005150922 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 864, 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 6226, 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 30, 2024-03-09T00:13:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 41, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 6704, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 2818, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 3886, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 2678, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 28, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 466, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 467, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 23, 2024-03-09T00:13:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 710569, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 437430, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 857, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 4550, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-03-09T00:13:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 4, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 28, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 9087, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 674, 2024-03-09T00:13:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 217410574, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 7, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 9087, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 1508, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 36, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 1472, 2024-03-09T00:13:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 7579, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 4, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 99065, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 234135, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 197106, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 54204, 2024-03-09T00:13:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 22379, 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 333042 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 204, 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-03-09T00:13:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 591, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 64683, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 64683, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-03-09T00:13:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 36, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 4, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 71, 2024-03-09T00:13:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 312168, 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 304554, 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 304554, 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 7614, 2024-03-09T00:13:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 5887, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 501, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 488, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 488, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 13, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 1, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 9, 2024-03-09T00:13:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 1143178338, 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 1171402300, 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 103661, 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 831.710980494, 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.008023374 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-03-09T00:13:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 3979, 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 68724, 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 64745, 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 208, 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 420, 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 212, 2024-03-09T00:13:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 6704, 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 154747, 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 148043, 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 467, 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 71801, 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 71334, 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 95516, 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215420 2024-03-09T00:13:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 721852, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 1843, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 485846, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 8662, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 3686, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 65353, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 291.766214789, 2024-03-09T00:13:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.004464465 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3132, 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 79.149849037, 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.025271343 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-03-09T00:13:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 152880, 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 554.011282021, 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.003623830 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000178406, 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000178406 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-03-09T00:13:05.126 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.126 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:04 smithi062 ceph-mon[36129]: pgmap v590: 129 pgs: 129 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 9 op/s 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 19, 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.012925639, 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000680296 2024-03-09T00:13:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 119, 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.194941581, 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.010041525 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 19, 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 11.744885881, 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.618151888 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-03-09T00:13:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 9, 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.014330867, 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001592318 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 182768, 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1151.647092649, 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.006301141 2024-03-09T00:13:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 13660, 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 63.818873804, 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.004671952 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-03-09T00:13:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.001763432, 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001763432 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 12, 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 4.031143958, 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.335928663 2024-03-09T00:13:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 191, 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 7.099392999, 2024-03-09T00:13:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.037169596 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2126, 2024-03-09T00:13:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 7.420816031, 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.003490506 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-03-09T00:13:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 9, 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.021674525, 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002408280 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 64683, 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 326.169570805, 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005042585 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 7, 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 5, 2024-03-09T00:13:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 2, 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 22305, 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 11152, 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 1106, 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:13:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:13:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-09T00:13:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:13:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:13:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 2384967, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 2311, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 88, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 1, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-09T00:13:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 9720, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 202, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-09T00:13:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 15123030, 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 132950, 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-03-09T00:13:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 319393, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 1235203345, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 22, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 319393, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 319393, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 17760.176478087, 2024-03-09T00:13:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.055606029 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 319393, 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 433473 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 319393, 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 10887, 2024-03-09T00:13:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 308506, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 5818, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 64002, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 1764, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 105389, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 1104, 2024-03-09T00:13:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 29, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 130611, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-03-09T00:13:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 9123, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 97107, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-03-09T00:13:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-03-09T00:13:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-03-09T00:13:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 126, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 42, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 12, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 12, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-03-09T00:13:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 13714, 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 4812 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-03-09T00:13:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 16, 2024-03-09T00:13:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 8, 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 129366, 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 64683, 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 1445019, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 254222280, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 1445019, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1445019, 2024-03-09T00:13:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 254222280, 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-03-09T00:13:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 319393, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 1242265032, 2024-03-09T00:13:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 319393, 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1242265032, 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-03-09T00:13:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-03-09T00:13:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-09T00:13:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-09T00:13:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-09T00:13:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 319393, 2024-03-09T00:13:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 319393, 2024-03-09T00:13:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 319393, 2024-03-09T00:13:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 319393, 2024-03-09T00:13:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 64683, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 7061687, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 64683, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1728, 2024-03-09T00:13:05.166 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 7061687, 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x559b4a5c86e0": { 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:05.167 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 312168, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1167207996, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 312168, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 103661, 2024-03-09T00:13:05.168 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1167207996, 2024-03-09T00:13:05.169 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:05.169 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:05.169 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:05.169 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:05.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:05.169 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-09T00:13:05.948 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-03-09T00:13:05.948 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-09T00:13:05.948 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=204 2024-03-09T00:13:05.948 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-03-09T00:13:05.949 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-03-09T00:13:05.949 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=19 2024-03-09T00:13:05.949 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=12 2024-03-09T00:13:05.949 DEBUG:tasks.check_counter:Getting stats from l 2024-03-09T00:13:05.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-09T00:13:06.370 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/mon.c/config 2024-03-09T00:13:06.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:06 smithi192 ceph-mon[35995]: pgmap v591: 129 pgs: 129 active+clean; 384 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 119 KiB/s wr, 27 op/s 2024-03-09T00:13:06.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:06 smithi192 ceph-mon[35995]: osdmap e127: 12 total, 12 up, 12 in 2024-03-09T00:13:06.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:06 smithi022 ceph-mon[31894]: pgmap v591: 129 pgs: 129 active+clean; 384 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 119 KiB/s wr, 27 op/s 2024-03-09T00:13:06.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:06 smithi022 ceph-mon[31894]: osdmap e127: 12 total, 12 up, 12 in 2024-03-09T00:13:07.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:06 smithi062 ceph-mon[36129]: pgmap v591: 129 pgs: 129 active+clean; 384 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 119 KiB/s wr, 27 op/s 2024-03-09T00:13:07.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:06 smithi062 ceph-mon[36129]: osdmap e127: 12 total, 12 up, 12 in 2024-03-09T00:13:09.036 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:08 smithi192 ceph-mon[35995]: pgmap v593: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2024-03-09T00:13:09.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:08 smithi062 ceph-mon[36129]: pgmap v593: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2024-03-09T00:13:09.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:08 smithi022 ceph-mon[31894]: pgmap v593: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 384 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2024-03-09T00:13:10.197 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-09T00:13:10.197 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-09T00:13:10.197 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 4, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 4, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 3072, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 448, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.001360735, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000042232, 2024-03-09T00:13:10.198 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000539856, 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3, 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000694069, 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000231356 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:10.199 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 362, 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 350, 2024-03-09T00:13:10.200 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1752336, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 210816, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.059209094, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.016999283, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.044620642, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.007397133, 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:10.201 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 349, 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.017214271, 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000049324 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1752336, 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 210816 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 3, 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 268, 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 480, 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 280208, 2024-03-09T00:13:10.203 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 6, 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.012791917, 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.008939309, 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001481775, 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 265, 2024-03-09T00:13:10.204 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.012687296, 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000047876 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-09T00:13:10.205 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 280208 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-09T00:13:10.206 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-09T00:13:10.207 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-09T00:13:10.208 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-09T00:13:10.209 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-09T00:13:10.210 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 148692, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 30, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-09T00:13:10.211 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-09T00:13:10.212 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-09T00:13:10.212 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2024-03-09T00:13:10.212 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2024-03-09T00:13:10.212 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-09T00:13:10.212 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-09T00:13:10.212 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-09T00:13:10.212 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-09T00:13:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 365, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1701265, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 365, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-09T00:13:10.214 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 365, 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1701265, 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-09T00:13:10.215 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-09T00:13:11.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:10 smithi062 ceph-mon[36129]: pgmap v594: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2024-03-09T00:13:11.123 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:10 smithi192 ceph-mon[35995]: pgmap v594: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2024-03-09T00:13:11.169 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-09T00:13:11.169 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-09T00:13:11.170 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-09T00:13:11.170 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-09T00:13:11.170 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-09T00:13:11.170 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-09T00:13:11.170 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-09T00:13:11.170 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-09T00:13:11.185 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-09T00:13:11.185 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-09T00:13:11.185 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T00:13:11.216 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-03-09T00:13:11.216 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T00:13:11.236 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:10 smithi022 ceph-mon[31894]: pgmap v594: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 383 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2024-03-09T00:13:11.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T00:13:11.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.466+0000 7fa2bea38640 1 -- 172.21.15.22:0/1413184197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 msgr2=0x7fa2b811cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:11.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.466+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1413184197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b811cec0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009cd0 tx=0x7fa2b002c0e0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.466+0000 7fa2bea38640 1 -- 172.21.15.22:0/1413184197 shutdown_connections 2024-03-09T00:13:11.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.466+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1413184197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b811cec0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.466+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1413184197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b810db50 0x7fa2b810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.466+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1413184197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b810fb20 0x7fa2b810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.466+0000 7fa2bea38640 1 -- 172.21.15.22:0/1413184197 >> 172.21.15.22:0/1413184197 conn(0x7fa2b806c3e0 msgr2=0x7fa2b806c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:11.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.468+0000 7fa2bea38640 1 -- 172.21.15.22:0/1413184197 shutdown_connections 2024-03-09T00:13:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.470+0000 7fa2bea38640 1 -- 172.21.15.22:0/1413184197 wait complete. 2024-03-09T00:13:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.470+0000 7fa2bea38640 1 Processor -- start 2024-03-09T00:13:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.470+0000 7fa2bea38640 1 -- start start 2024-03-09T00:13:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bea38640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b810db50 0x7fa2b81ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bea38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b810fb20 0x7fa2b81acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bea38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b81a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bea38640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8120490 con 0x7fa2b810fb20 2024-03-09T00:13:11.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bea38640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8120310 con 0x7fa2b811aad0 2024-03-09T00:13:11.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bea38640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8120610 con 0x7fa2b810db50 2024-03-09T00:13:11.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bcfd5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b81a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:11.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bcfd5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b81a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55952/0 (socket says 172.21.15.22:55952) 2024-03-09T00:13:11.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.471+0000 7fa2bcfd5640 1 -- 172.21.15.22:0/1491784755 learned_addr learned my addr 172.21.15.22:0/1491784755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:11.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.472+0000 7fa2bcfd5640 1 -- 172.21.15.22:0/1491784755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b810db50 msgr2=0x7fa2b81ac680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.472+0000 7fa2bcfd5640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b810db50 0x7fa2b81ac680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.472+0000 7fa2bcfd5640 1 -- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b810fb20 msgr2=0x7fa2b81acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.472+0000 7fa2bcfd5640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b810fb20 0x7fa2b81acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.472+0000 7fa2bcfd5640 1 -- 172.21.15.22:0/1491784755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b81a6fc0 con 0x7fa2b811aad0 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.472+0000 7fa2bcfd5640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b81a67f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009ca0 tx=0x7fa2b0035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.474+0000 7fa2b57fa640 1 -- 172.21.15.22:0/1491784755 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0039040 con 0x7fa2b811aad0 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.474+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b81a7250 con 0x7fa2b811aad0 2024-03-09T00:13:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.474+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b8079550 con 0x7fa2b811aad0 2024-03-09T00:13:11.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.474+0000 7fa2b57fa640 1 -- 172.21.15.22:0/1491784755 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2b0044070 con 0x7fa2b811aad0 2024-03-09T00:13:11.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.474+0000 7fa2b57fa640 1 -- 172.21.15.22:0/1491784755 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00044a0 con 0x7fa2b811aad0 2024-03-09T00:13:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.480+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a4005290 con 0x7fa2b811aad0 2024-03-09T00:13:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.481+0000 7fa2b57fa640 1 -- 172.21.15.22:0/1491784755 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa2b0032790 con 0x7fa2b811aad0 2024-03-09T00:13:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.482+0000 7fa2b57fa640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2a00a2b40 0x7fa2a00a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.482+0000 7fa2b57fa640 1 -- 172.21.15.22:0/1491784755 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fa2b00f1180 con 0x7fa2b811aad0 2024-03-09T00:13:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.483+0000 7fa2b7fff640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2a00a2b40 0x7fa2a00a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.484+0000 7fa2b7fff640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2a00a2b40 0x7fa2a00a5000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00b990 tx=0x7fa2ac00b370 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:11.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.494+0000 7fa2b57fa640 1 -- 172.21.15.22:0/1491784755 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa2b004d020 con 0x7fa2b811aad0 2024-03-09T00:13:11.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.853+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2a4005850 con 0x7fa2b811aad0 2024-03-09T00:13:11.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.855+0000 7fa2b57fa640 1 -- 172.21.15.22:0/1491784755 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+25419 (secure 0 0 0) 0x7fa2b00bdb40 con 0x7fa2b811aad0 2024-03-09T00:13:11.853 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:13:11.853 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":127,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-09T00:13:05.727861+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3461287105":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/1632997532":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3477373324":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1865576829":"2024-03-09T23:54:32.857800+0000","172.21.15.22:6800/2813827512":"2024-03-09T23:54:32.857800+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1683058859":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/2813827512":"2024-03-09T23:54:32.857800+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]},{"pool":4,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]},{"pool":4,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T00:13:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2a00a2b40 msgr2=0x7fa2a00a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2a00a2b40 0x7fa2a00a5000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00b990 tx=0x7fa2ac00b370 comp rx=0 tx=0).stop 2024-03-09T00:13:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 msgr2=0x7fa2b81a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b81a67f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009ca0 tx=0x7fa2b0035d10 comp rx=0 tx=0).stop 2024-03-09T00:13:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 shutdown_connections 2024-03-09T00:13:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fa2a00a2b40 0x7fa2a00a5000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b811aad0 0x7fa2b81a67f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b810fb20 0x7fa2b81acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 --2- 172.21.15.22:0/1491784755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2b810db50 0x7fa2b81ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:11.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 >> 172.21.15.22:0/1491784755 conn(0x7fa2b806c3e0 msgr2=0x7fa2b8071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:11.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 shutdown_connections 2024-03-09T00:13:11.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:11.859+0000 7fa2bea38640 1 -- 172.21.15.22:0/1491784755 wait complete. 2024-03-09T00:13:11.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T23:47:12.051601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T23:51:50.171325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T23:51:50.798188+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 127, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-08T23:51:54.496659+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 127, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-09T00:13:11.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T00:13:12.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.051+0000 7f0417d09640 1 -- 172.21.15.22:0/2966424079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410114440 msgr2=0x7f0410116830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:12.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.051+0000 7f0417d09640 1 --2- 172.21.15.22:0/2966424079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410114440 0x7f0410116830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0408009ca0 tx=0x7f040802c120 comp rx=0 tx=0).stop 2024-03-09T00:13:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.055+0000 7f0417d09640 1 -- 172.21.15.22:0/2966424079 shutdown_connections 2024-03-09T00:13:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.055+0000 7f0417d09640 1 --2- 172.21.15.22:0/2966424079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410114440 0x7f0410116830 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.055+0000 7f0417d09640 1 --2- 172.21.15.22:0/2966424079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410108df0 0x7f0410113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.055+0000 7f0417d09640 1 --2- 172.21.15.22:0/2966424079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04101084d0 0x7f04101088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.055+0000 7f0417d09640 1 -- 172.21.15.22:0/2966424079 >> 172.21.15.22:0/2966424079 conn(0x7f041006c3d0 msgr2=0x7f041006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:12.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.060+0000 7f0417d09640 1 -- 172.21.15.22:0/2966424079 shutdown_connections 2024-03-09T00:13:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.061+0000 7f0417d09640 1 -- 172.21.15.22:0/2966424079 wait complete. 2024-03-09T00:13:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.061+0000 7f0417d09640 1 Processor -- start 2024-03-09T00:13:12.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.066+0000 7f0417d09640 1 -- start start 2024-03-09T00:13:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.066+0000 7f0417d09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04101084d0 0x7f0410113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.066+0000 7f0417d09640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410108df0 0x7f0410113da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.066+0000 7f0417d09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410114440 0x7f041010a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.066+0000 7f0417d09640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410119ef0 con 0x7f0410114440 2024-03-09T00:13:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.066+0000 7f0417d09640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410119d70 con 0x7f04101084d0 2024-03-09T00:13:12.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.066+0000 7f0417d09640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041011a070 con 0x7f0410108df0 2024-03-09T00:13:12.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.067+0000 7f0415aa5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04101084d0 0x7f0410113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:12.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.070+0000 7f04162a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410114440 0x7f041010a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:12.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.070+0000 7f04162a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410114440 0x7f041010a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33756/0 (socket says 172.21.15.22:33756) 2024-03-09T00:13:12.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.070+0000 7f04162a6640 1 -- 172.21.15.22:0/3582520611 learned_addr learned my addr 172.21.15.22:0/3582520611 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:12.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.072+0000 7f04152a4640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410108df0 0x7f0410113da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.072+0000 7f0415aa5640 1 -- 172.21.15.22:0/3582520611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410108df0 msgr2=0x7f0410113da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.072+0000 7f0415aa5640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410108df0 0x7f0410113da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.072+0000 7f0415aa5640 1 -- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410114440 msgr2=0x7f041010a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.072+0000 7f0415aa5640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410114440 0x7f041010a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.072+0000 7f0415aa5640 1 -- 172.21.15.22:0/3582520611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041010b180 con 0x7f04101084d0 2024-03-09T00:13:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.073+0000 7f0415aa5640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04101084d0 0x7f0410113860 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0404009fd0 tx=0x7f040400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:12.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.075+0000 7f0402ffd640 1 -- 172.21.15.22:0/3582520611 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404015020 con 0x7f04101084d0 2024-03-09T00:13:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.075+0000 7f0402ffd640 1 -- 172.21.15.22:0/3582520611 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0404011d10 con 0x7f04101084d0 2024-03-09T00:13:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.075+0000 7f0402ffd640 1 -- 172.21.15.22:0/3582520611 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040400f2e0 con 0x7f04101084d0 2024-03-09T00:13:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.075+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041010b470 con 0x7f04101084d0 2024-03-09T00:13:12.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.075+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041010f480 con 0x7f04101084d0 2024-03-09T00:13:12.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.079+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041006cf90 con 0x7f04101084d0 2024-03-09T00:13:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.083+0000 7f0402ffd640 1 -- 172.21.15.22:0/3582520611 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0404018020 con 0x7f04101084d0 2024-03-09T00:13:12.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.084+0000 7f0402ffd640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03fc0a2b80 0x7f03fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.084+0000 7f0402ffd640 1 -- 172.21.15.22:0/3582520611 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f04040ce800 con 0x7f04101084d0 2024-03-09T00:13:12.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.085+0000 7f04152a4640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03fc0a2b80 0x7f03fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.086+0000 7f04152a4640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03fc0a2b80 0x7f03fc0a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f041010bfd0 tx=0x7f040c005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.089+0000 7f0402ffd640 1 -- 172.21.15.22:0/3582520611 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f040409b1c0 con 0x7f04101084d0 2024-03-09T00:13:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.451+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04100680e0 con 0x7f04101084d0 2024-03-09T00:13:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.451+0000 7f0402ffd640 1 -- 172.21.15.22:0/3582520611 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f040409aac0 con 0x7f04101084d0 2024-03-09T00:13:12.449 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-03-09T00:13:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03fc0a2b80 msgr2=0x7f03fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03fc0a2b80 0x7f03fc0a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f041010bfd0 tx=0x7f040c005f90 comp rx=0 tx=0).stop 2024-03-09T00:13:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04101084d0 msgr2=0x7f0410113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04101084d0 0x7f0410113860 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0404009fd0 tx=0x7f040400b870 comp rx=0 tx=0).stop 2024-03-09T00:13:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 shutdown_connections 2024-03-09T00:13:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f03fc0a2b80 0x7f03fc0a5040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0410114440 0x7f041010a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0410108df0 0x7f0410113da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 --2- 172.21.15.22:0/3582520611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04101084d0 0x7f0410113860 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 >> 172.21.15.22:0/3582520611 conn(0x7f041006c3d0 msgr2=0x7f041006ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 shutdown_connections 2024-03-09T00:13:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.455+0000 7f0417d09640 1 -- 172.21.15.22:0/3582520611 wait complete. 2024-03-09T00:13:12.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-09T00:13:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.675+0000 7efe30bc4640 1 -- 172.21.15.22:0/311404505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe2c10db40 msgr2=0x7efe2c10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.675+0000 7efe30bc4640 1 --2- 172.21.15.22:0/311404505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe2c10db40 0x7efe2c10dfc0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7efe20009900 tx=0x7efe2002bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.676+0000 7efe30bc4640 1 -- 172.21.15.22:0/311404505 shutdown_connections 2024-03-09T00:13:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.676+0000 7efe30bc4640 1 --2- 172.21.15.22:0/311404505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c11aad0 0x7efe2c11cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.676+0000 7efe30bc4640 1 --2- 172.21.15.22:0/311404505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe2c10db40 0x7efe2c10dfc0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.676+0000 7efe30bc4640 1 --2- 172.21.15.22:0/311404505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2c10fb10 0x7efe2c10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.676+0000 7efe30bc4640 1 -- 172.21.15.22:0/311404505 >> 172.21.15.22:0/311404505 conn(0x7efe2c06c3d0 msgr2=0x7efe2c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.679+0000 7efe30bc4640 1 -- 172.21.15.22:0/311404505 shutdown_connections 2024-03-09T00:13:12.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.679+0000 7efe30bc4640 1 -- 172.21.15.22:0/311404505 wait complete. 2024-03-09T00:13:12.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.680+0000 7efe30bc4640 1 Processor -- start 2024-03-09T00:13:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.680+0000 7efe30bc4640 1 -- start start 2024-03-09T00:13:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe30bc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c10db40 0x7efe2c1ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe30bc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe2c10fb10 0x7efe2c1acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe30bc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2c11aad0 0x7efe2c1a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe30bc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c120450 con 0x7efe2c10fb10 2024-03-09T00:13:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe30bc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c1202d0 con 0x7efe2c10db40 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe30bc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c1205d0 con 0x7efe2c11aad0 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c10db40 0x7efe2c1ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c10db40 0x7efe2c1ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:55976/0 (socket says 172.21.15.22:55976) 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2a59c640 1 -- 172.21.15.22:0/3768909670 learned_addr learned my addr 172.21.15.22:0/3768909670 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2ad9d640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2c11aad0 0x7efe2c1a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2a59c640 1 -- 172.21.15.22:0/3768909670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2c11aad0 msgr2=0x7efe2c1a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2a59c640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2c11aad0 0x7efe2c1a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2a59c640 1 -- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe2c10fb10 msgr2=0x7efe2c1acb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T00:13:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.681+0000 7efe2a59c640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe2c10fb10 0x7efe2c1acb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.682+0000 7efe2a59c640 1 -- 172.21.15.22:0/3768909670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c1a6f80 con 0x7efe2c10db40 2024-03-09T00:13:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.682+0000 7efe2a59c640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c10db40 0x7efe2c1ac640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe2400ebc0 tx=0x7efe2400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.682+0000 7efe1b7fe640 1 -- 172.21.15.22:0/3768909670 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24010630 con 0x7efe2c10db40 2024-03-09T00:13:12.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.682+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c1a7270 con 0x7efe2c10db40 2024-03-09T00:13:12.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.682+0000 7efe1b7fe640 1 -- 172.21.15.22:0/3768909670 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe24010cb0 con 0x7efe2c10db40 2024-03-09T00:13:12.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.682+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c1102b0 con 0x7efe2c10db40 2024-03-09T00:13:12.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.683+0000 7efe1b7fe640 1 -- 172.21.15.22:0/3768909670 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24010630 con 0x7efe2c10db40 2024-03-09T00:13:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.684+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c06d670 con 0x7efe2c10db40 2024-03-09T00:13:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.695+0000 7efe1b7fe640 1 -- 172.21.15.22:0/3768909670 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7efe240107d0 con 0x7efe2c10db40 2024-03-09T00:13:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.696+0000 7efe1b7fe640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe140a2db0 0x7efe140a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.696+0000 7efe1b7fe640 1 -- 172.21.15.22:0/3768909670 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7efe24010430 con 0x7efe2c10db40 2024-03-09T00:13:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.697+0000 7efe1b7fe640 1 -- 172.21.15.22:0/3768909670 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7efe24052260 con 0x7efe2c10db40 2024-03-09T00:13:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.698+0000 7efe29d9b640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe140a2db0 0x7efe140a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:12.700+0000 7efe29d9b640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe140a2db0 0x7efe140a5270 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe20005c20 tx=0x7efe20005fd0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:12.922 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:12 smithi022 ceph-mon[31894]: pgmap v595: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 375 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 212 KiB/s wr, 31 op/s 2024-03-09T00:13:12.922 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:12 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/1491784755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:13:12.922 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:12 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3582520611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-09T00:13:13.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.043+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe2c0680e0 con 0x7efe2c10db40 2024-03-09T00:13:13.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.045+0000 7efe1b7fe640 1 -- 172.21.15.22:0/3768909670 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe2c0680e0 con 0x7efe2c10db40 2024-03-09T00:13:13.042 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-09T00:13:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe140a2db0 msgr2=0x7efe140a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe140a2db0 0x7efe140a5270 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe20005c20 tx=0x7efe20005fd0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c10db40 msgr2=0x7efe2c1ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c10db40 0x7efe2c1ac640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe2400ebc0 tx=0x7efe2400c6a0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 shutdown_connections 2024-03-09T00:13:13.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7efe140a2db0 0x7efe140a5270 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe20005c20 tx=0x7efe20005fd0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2c11aad0 0x7efe2c1a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe2c10fb10 0x7efe2c1acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 --2- 172.21.15.22:0/3768909670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe2c10db40 0x7efe2c1ac640 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe2400ebc0 tx=0x7efe2400c6a0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.048+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 >> 172.21.15.22:0/3768909670 conn(0x7efe2c06c3d0 msgr2=0x7efe2c11c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:13.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.049+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 shutdown_connections 2024-03-09T00:13:13.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.049+0000 7efe30bc4640 1 -- 172.21.15.22:0/3768909670 wait complete. 2024-03-09T00:13:13.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-09T00:13:13.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:12 smithi062 ceph-mon[36129]: pgmap v595: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 375 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 212 KiB/s wr, 31 op/s 2024-03-09T00:13:13.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:12 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/1491784755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:13:13.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:12 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3582520611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-09T00:13:13.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:12 smithi192 ceph-mon[35995]: pgmap v595: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 375 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 212 KiB/s wr, 31 op/s 2024-03-09T00:13:13.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:12 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/1491784755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:13:13.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:12 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3582520611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-09T00:13:13.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.243+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2625709509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 msgr2=0x7fe9e410d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.243+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2625709509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 0x7fe9e410d600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009900 tx=0x7fe9d802bc50 comp rx=0 tx=0).stop 2024-03-09T00:13:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.245+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2625709509 shutdown_connections 2024-03-09T00:13:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.245+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2625709509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e411aad0 0x7fe9e411cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.245+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2625709509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e410db40 0x7fe9e410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.245+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2625709509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 0x7fe9e410d600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.245+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2625709509 >> 172.21.15.22:0/2625709509 conn(0x7fe9e406c3d0 msgr2=0x7fe9e406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.246+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2625709509 shutdown_connections 2024-03-09T00:13:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.246+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2625709509 wait complete. 2024-03-09T00:13:13.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.247+0000 7fe9ea0b1640 1 Processor -- start 2024-03-09T00:13:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.247+0000 7fe9ea0b1640 1 -- start start 2024-03-09T00:13:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9ea0b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e410db40 0x7fe9e41ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9ea0b1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 0x7fe9e41acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9ea0b1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e411aad0 0x7fe9e41a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9ea0b1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4120510 con 0x7fe9e410db40 2024-03-09T00:13:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9ea0b1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4120390 con 0x7fe9e411aad0 2024-03-09T00:13:13.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9ea0b1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4120690 con 0x7fe9e410fb10 2024-03-09T00:13:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e410db40 0x7fe9e41ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e410db40 0x7fe9e41ac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33784/0 (socket says 172.21.15.22:33784) 2024-03-09T00:13:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9e37fe640 1 -- 172.21.15.22:0/2634638067 learned_addr learned my addr 172.21.15.22:0/2634638067 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.248+0000 7fe9e2ffd640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 0x7fe9e41acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9e2ffd640 1 -- 172.21.15.22:0/2634638067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e411aad0 msgr2=0x7fe9e41a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9e2ffd640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e411aad0 0x7fe9e41a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9e2ffd640 1 -- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e410db40 msgr2=0x7fe9e41ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9e2ffd640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e410db40 0x7fe9e41ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9e2ffd640 1 -- 172.21.15.22:0/2634638067 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e41a6ff0 con 0x7fe9e410fb10 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9e2ffd640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 0x7fe9e41acc00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009fd0 tx=0x7fe9d400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9e0ff9640 1 -- 172.21.15.22:0/2634638067 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4015020 con 0x7fe9e410fb10 2024-03-09T00:13:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.249+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e41a72e0 con 0x7fe9e410fb10 2024-03-09T00:13:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.250+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e41b3580 con 0x7fe9e410fb10 2024-03-09T00:13:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.250+0000 7fe9e0ff9640 1 -- 172.21.15.22:0/2634638067 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9d4011d10 con 0x7fe9e410fb10 2024-03-09T00:13:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.251+0000 7fe9e0ff9640 1 -- 172.21.15.22:0/2634638067 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400f2e0 con 0x7fe9e410fb10 2024-03-09T00:13:13.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.253+0000 7fe9e0ff9640 1 -- 172.21.15.22:0/2634638067 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe9d4018020 con 0x7fe9e410fb10 2024-03-09T00:13:13.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.254+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d0005290 con 0x7fe9e410fb10 2024-03-09T00:13:13.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.254+0000 7fe9e0ff9640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a2990 0x7fe9cc0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.254+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a2990 0x7fe9cc0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:13.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.254+0000 7fe9e0ff9640 1 -- 172.21.15.22:0/2634638067 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe9d40ce9c0 con 0x7fe9e410fb10 2024-03-09T00:13:13.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.255+0000 7fe9e37fe640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a2990 0x7fe9cc0a4e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005c40 tx=0x7fe9d8005b50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:13.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.267+0000 7fe9e0ff9640 1 -- 172.21.15.22:0/2634638067 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe9d401f020 con 0x7fe9e410fb10 2024-03-09T00:13:13.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.623+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9d0005850 con 0x7fe9e410fb10 2024-03-09T00:13:13.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.624+0000 7fe9e0ff9640 1 -- 172.21.15.22:0/2634638067 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9d409b380 con 0x7fe9e410fb10 2024-03-09T00:13:13.621 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-03-09T00:13:13.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a2990 msgr2=0x7fe9cc0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a2990 0x7fe9cc0a4e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005c40 tx=0x7fe9d8005b50 comp rx=0 tx=0).stop 2024-03-09T00:13:13.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 msgr2=0x7fe9e41acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 0x7fe9e41acc00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009fd0 tx=0x7fe9d400b870 comp rx=0 tx=0).stop 2024-03-09T00:13:13.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 shutdown_connections 2024-03-09T00:13:13.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe9cc0a2990 0x7fe9cc0a4e50 secure :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005c40 tx=0x7fe9d8005b50 comp rx=0 tx=0).stop 2024-03-09T00:13:13.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e411aad0 0x7fe9e41a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9e410fb10 0x7fe9e41acc00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 --2- 172.21.15.22:0/2634638067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9e410db40 0x7fe9e41ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 >> 172.21.15.22:0/2634638067 conn(0x7fe9e406c3d0 msgr2=0x7fe9e4071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:13.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.628+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 shutdown_connections 2024-03-09T00:13:13.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.629+0000 7fe9ea0b1640 1 -- 172.21.15.22:0/2634638067 wait complete. 2024-03-09T00:13:13.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-09T00:13:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 -- 172.21.15.22:0/2329876438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 msgr2=0x7f50a81088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 --2- 172.21.15.22:0/2329876438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 0x7f50a81088b0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f5090009900 tx=0x7f509002bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 -- 172.21.15.22:0/2329876438 shutdown_connections 2024-03-09T00:13:13.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 --2- 172.21.15.22:0/2329876438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 0x7f50a8116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 --2- 172.21.15.22:0/2329876438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a8108df0 0x7f50a8113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 --2- 172.21.15.22:0/2329876438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 0x7f50a81088b0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 -- 172.21.15.22:0/2329876438 >> 172.21.15.22:0/2329876438 conn(0x7f50a806c3d0 msgr2=0x7f50a806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:13.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 -- 172.21.15.22:0/2329876438 shutdown_connections 2024-03-09T00:13:13.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.826+0000 7f50acb90640 1 -- 172.21.15.22:0/2329876438 wait complete. 2024-03-09T00:13:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.827+0000 7f50acb90640 1 Processor -- start 2024-03-09T00:13:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.827+0000 7f50acb90640 1 -- start start 2024-03-09T00:13:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50acb90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 0x7f50a81138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50acb90640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a8108df0 0x7f50a8113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50acb90640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 0x7f50a810b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50acb90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8119e40 con 0x7f50a81084d0 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50acb90640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8119cc0 con 0x7f50a8108df0 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50acb90640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8119fc0 con 0x7f50a8114440 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 0x7f50a810b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 0x7f50a810b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:49736/0 (socket says 172.21.15.22:49736) 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 -- 172.21.15.22:0/2310637493 learned_addr learned my addr 172.21.15.22:0/2310637493 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 -- 172.21.15.22:0/2310637493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a8108df0 msgr2=0x7f50a8113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f509ffff640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a8108df0 0x7f50a8113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a659c640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 0x7f50a81138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a8108df0 0x7f50a8113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 -- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 msgr2=0x7f50a81138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 0x7f50a81138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a6d9d640 1 -- 172.21.15.22:0/2310637493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a810bb10 con 0x7f50a8114440 2024-03-09T00:13:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.828+0000 7f50a659c640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 0x7f50a81138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:13:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.830+0000 7f50a6d9d640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 0x7f50a810b250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50a000eb40 tx=0x7f50a000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.831+0000 7f509f7fe640 1 -- 172.21.15.22:0/2310637493 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0004b20 con 0x7f50a8114440 2024-03-09T00:13:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.831+0000 7f509f7fe640 1 -- 172.21.15.22:0/2310637493 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50a0004cc0 con 0x7f50a8114440 2024-03-09T00:13:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.831+0000 7f509f7fe640 1 -- 172.21.15.22:0/2310637493 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0010850 con 0x7f50a8114440 2024-03-09T00:13:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.831+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a810be00 con 0x7f50a8114440 2024-03-09T00:13:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.831+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a810f500 con 0x7f50a8114440 2024-03-09T00:13:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.834+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5088005290 con 0x7f50a8114440 2024-03-09T00:13:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.835+0000 7f509f7fe640 1 -- 172.21.15.22:0/2310637493 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f50a0005020 con 0x7f50a8114440 2024-03-09T00:13:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.836+0000 7f509f7fe640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50940a2b40 0x7f50940a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.836+0000 7f509f7fe640 1 -- 172.21.15.22:0/2310637493 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f50a00cf980 con 0x7f50a8114440 2024-03-09T00:13:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.840+0000 7f50a659c640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50940a2b40 0x7f50940a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.844+0000 7f509f7fe640 1 -- 172.21.15.22:0/2310637493 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f50a009c230 con 0x7f50a8114440 2024-03-09T00:13:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:13.847+0000 7f50a659c640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50940a2b40 0x7f50940a5000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5090036040 tx=0x7f509002c750 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:13 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3768909670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-09T00:13:13.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:13 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2634638067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-09T00:13:14.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:13 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3768909670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-09T00:13:14.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:13 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2634638067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-09T00:13:14.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:13 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3768909670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-09T00:13:14.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:13 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2634638067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-09T00:13:14.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.196+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5088005850 con 0x7f50a8114440 2024-03-09T00:13:14.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.200+0000 7f509f7fe640 1 -- 172.21.15.22:0/2310637493 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v127) v1 ==== 103+0+11 (secure 0 0 0) 0x7f50a009bb30 con 0x7f50a8114440 2024-03-09T00:13:14.197 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-03-09T00:13:14.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50940a2b40 msgr2=0x7f50940a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:14.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50940a2b40 0x7f50940a5000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5090036040 tx=0x7f509002c750 comp rx=0 tx=0).stop 2024-03-09T00:13:14.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 msgr2=0x7f50a810b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:14.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 0x7f50a810b250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50a000eb40 tx=0x7f50a000c2d0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 shutdown_connections 2024-03-09T00:13:14.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f50940a2b40 0x7f50940a5000 secure :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5090036040 tx=0x7f509002c750 comp rx=0 tx=0).stop 2024-03-09T00:13:14.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8114440 0x7f50a810b250 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a8108df0 0x7f50a8113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 --2- 172.21.15.22:0/2310637493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50a81084d0 0x7f50a81138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 >> 172.21.15.22:0/2310637493 conn(0x7f50a806c3d0 msgr2=0x7f50a8109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:14.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.204+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 shutdown_connections 2024-03-09T00:13:14.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.205+0000 7f50acb90640 1 -- 172.21.15.22:0/2310637493 wait complete. 2024-03-09T00:13:14.222 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-09T00:13:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.415+0000 7f05be2ea640 1 -- 172.21.15.22:0/1169948127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b810db40 msgr2=0x7f05b810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.415+0000 7f05be2ea640 1 --2- 172.21.15.22:0/1169948127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b810db40 0x7f05b810dfc0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009900 tx=0x7f05ac02bc80 comp rx=0 tx=0).stop 2024-03-09T00:13:14.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.417+0000 7f05be2ea640 1 -- 172.21.15.22:0/1169948127 shutdown_connections 2024-03-09T00:13:14.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.417+0000 7f05be2ea640 1 --2- 172.21.15.22:0/1169948127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b811aad0 0x7f05b811cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.417+0000 7f05be2ea640 1 --2- 172.21.15.22:0/1169948127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b810db40 0x7f05b810dfc0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.417+0000 7f05be2ea640 1 --2- 172.21.15.22:0/1169948127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b810fb10 0x7f05b810d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.417+0000 7f05be2ea640 1 -- 172.21.15.22:0/1169948127 >> 172.21.15.22:0/1169948127 conn(0x7f05b806c3d0 msgr2=0x7f05b806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:14.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.419+0000 7f05be2ea640 1 -- 172.21.15.22:0/1169948127 shutdown_connections 2024-03-09T00:13:14.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.420+0000 7f05be2ea640 1 -- 172.21.15.22:0/1169948127 wait complete. 2024-03-09T00:13:14.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.421+0000 7f05be2ea640 1 Processor -- start 2024-03-09T00:13:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.425+0000 7f05be2ea640 1 -- start start 2024-03-09T00:13:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.425+0000 7f05be2ea640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b810db40 0x7f05b811a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:14.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.425+0000 7f05be2ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b810fb10 0x7f05b81142c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.425+0000 7f05be2ea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b811aad0 0x7f05b8114800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.425+0000 7f05be2ea640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8110ce0 con 0x7f05b811aad0 2024-03-09T00:13:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.425+0000 7f05be2ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8110b60 con 0x7f05b810fb10 2024-03-09T00:13:14.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.425+0000 7f05be2ea640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8110e60 con 0x7f05b810db40 2024-03-09T00:13:14.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.429+0000 7f05b77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b810fb10 0x7f05b81142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.429+0000 7f05b77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b810fb10 0x7f05b81142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:56032/0 (socket says 172.21.15.22:56032) 2024-03-09T00:13:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.429+0000 7f05b77fe640 1 -- 172.21.15.22:0/352344951 learned_addr learned my addr 172.21.15.22:0/352344951 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:14.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.429+0000 7f05bc887640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b811aad0 0x7f05b8114800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:14.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.430+0000 7f05b77fe640 1 -- 172.21.15.22:0/352344951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b810db40 msgr2=0x7f05b811a250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T00:13:14.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.430+0000 7f05b77fe640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b810db40 0x7f05b811a250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.431+0000 7f05b77fe640 1 -- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b811aad0 msgr2=0x7f05b8114800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:14.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.431+0000 7f05b77fe640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b811aad0 0x7f05b8114800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.431+0000 7f05b77fe640 1 -- 172.21.15.22:0/352344951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8114f80 con 0x7f05b810fb10 2024-03-09T00:13:14.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.435+0000 7f05b77fe640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b810fb10 0x7f05b81142c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac02ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:14.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.438+0000 7f05b57fa640 1 -- 172.21.15.22:0/352344951 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac041020 con 0x7f05b810fb10 2024-03-09T00:13:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.438+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b81bb770 con 0x7f05b810fb10 2024-03-09T00:13:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.438+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b81bbbd0 con 0x7f05b810fb10 2024-03-09T00:13:14.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.439+0000 7f05b57fa640 1 -- 172.21.15.22:0/352344951 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05ac02c620 con 0x7f05b810fb10 2024-03-09T00:13:14.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.439+0000 7f05b57fa640 1 -- 172.21.15.22:0/352344951 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac004ec0 con 0x7f05b810fb10 2024-03-09T00:13:14.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.442+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a40052c0 con 0x7f05b810fb10 2024-03-09T00:13:14.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.445+0000 7f05b57fa640 1 -- 172.21.15.22:0/352344951 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f05ac044020 con 0x7f05b810fb10 2024-03-09T00:13:14.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.446+0000 7f05b57fa640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f05a00a2b70 0x7f05a00a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:14.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.446+0000 7f05b57fa640 1 -- 172.21.15.22:0/352344951 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f05ac0f0980 con 0x7f05b810fb10 2024-03-09T00:13:14.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.450+0000 7f05b7fff640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f05a00a2b70 0x7f05a00a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:14.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.451+0000 7f05b57fa640 1 -- 172.21.15.22:0/352344951 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f05ac04b020 con 0x7f05b810fb10 2024-03-09T00:13:14.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.457+0000 7f05b7fff640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f05a00a2b70 0x7f05a00a5030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f05a800a890 tx=0x7f05a8005b80 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.824+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f05a4005880 con 0x7f05b810fb10 2024-03-09T00:13:14.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.825+0000 7f05b57fa640 1 -- 172.21.15.22:0/352344951 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v127) v1 ==== 81+0+1002 (secure 0 0 0) 0x7f05ac0bd340 con 0x7f05b810fb10 2024-03-09T00:13:14.822 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3461287105 2024-03-09T23:54:32.857800+0000 2024-03-09T00:13:14.822 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2337825839 2024-03-09T23:44:43.705970+0000 2024-03-09T00:13:14.822 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/4028157579 2024-03-09T23:44:43.705970+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3384641534 2024-03-09T23:44:16.515766+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1632997532 2024-03-09T23:54:32.857800+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1968010400 2024-03-09T23:44:16.515766+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3762004902 2024-03-09T23:44:16.515766+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3477373324 2024-03-09T23:54:32.857800+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/99443044 2024-03-09T23:44:43.705970+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3384641534 2024-03-09T23:44:16.515766+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3201080518 2024-03-09T23:44:43.705970+0000 2024-03-09T00:13:14.823 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1865576829 2024-03-09T23:54:32.857800+0000 2024-03-09T00:13:14.824 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/2813827512 2024-03-09T23:54:32.857800+0000 2024-03-09T00:13:14.824 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/4028157579 2024-03-09T23:44:43.705970+0000 2024-03-09T00:13:14.824 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1683058859 2024-03-09T23:54:32.857800+0000 2024-03-09T00:13:14.824 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1133673382 2024-03-09T23:44:16.515766+0000 2024-03-09T00:13:14.824 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/2813827512 2024-03-09T23:54:32.857800+0000 2024-03-09T00:13:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:listed 17 entries 2024-03-09T00:13:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f05a00a2b70 msgr2=0x7f05a00a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f05a00a2b70 0x7f05a00a5030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f05a800a890 tx=0x7f05a8005b80 comp rx=0 tx=0).stop 2024-03-09T00:13:14.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b810fb10 msgr2=0x7f05b81142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b810fb10 0x7f05b81142c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c00 tx=0x7f05ac02ec50 comp rx=0 tx=0).stop 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 shutdown_connections 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f05a00a2b70 0x7f05a00a5030 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05b811aad0 0x7f05b8114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b810fb10 0x7f05b81142c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 --2- 172.21.15.22:0/352344951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b810db40 0x7f05b811a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 >> 172.21.15.22:0/352344951 conn(0x7f05b806c3d0 msgr2=0x7f05b811c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:14.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 shutdown_connections 2024-03-09T00:13:14.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:14.827+0000 7f05be2ea640 1 -- 172.21.15.22:0/352344951 wait complete. 2024-03-09T00:13:14.842 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-09T00:13:14.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:13:14.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-09T00:13:15.021 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:14 smithi022 ceph-mon[31894]: pgmap v596: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 374 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 185 KiB/s wr, 28 op/s 2024-03-09T00:13:15.022 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:14 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2310637493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-09T00:13:15.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-03-09T00:13:15.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:13:15.023 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-09T00:13:15.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:13:15.058 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-03-09T00:13:15.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:14 smithi062 ceph-mon[36129]: pgmap v596: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 374 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 185 KiB/s wr, 28 op/s 2024-03-09T00:13:15.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:14 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2310637493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-09T00:13:15.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:14 smithi192 ceph-mon[35995]: pgmap v596: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 374 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 185 KiB/s wr, 28 op/s 2024-03-09T00:13:15.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:14 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2310637493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-09T00:13:15.143 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-09T00:13:15.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:13:15.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-09T00:13:15.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T00:13:15.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-03-09T00:13:15.274 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-09T00:13:15.289 INFO:tasks.cephadm:Teardown begin 2024-03-09T00:13:15.289 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-09T00:13:15.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:13:15.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 -- 172.21.15.22:0/1501275059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181001f0 msgr2=0x7f5418100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 --2- 172.21.15.22:0/1501275059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181001f0 0x7f5418100650 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f5404009900 tx=0x7f540402bcf0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 -- 172.21.15.22:0/1501275059 shutdown_connections 2024-03-09T00:13:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 --2- 172.21.15.22:0/1501275059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5418100b90 0x7f541810d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 --2- 172.21.15.22:0/1501275059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181001f0 0x7f5418100650 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 --2- 172.21.15.22:0/1501275059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54181062b0 0x7f5418106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 -- 172.21.15.22:0/1501275059 >> 172.21.15.22:0/1501275059 conn(0x7f5418077320 msgr2=0x7f5418077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 -- 172.21.15.22:0/1501275059 shutdown_connections 2024-03-09T00:13:15.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.429+0000 7f541f1e6640 1 -- 172.21.15.22:0/1501275059 wait complete. 2024-03-09T00:13:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.430+0000 7f541f1e6640 1 Processor -- start 2024-03-09T00:13:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.430+0000 7f541f1e6640 1 -- start start 2024-03-09T00:13:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.430+0000 7f541f1e6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54181001f0 0x7f541819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.430+0000 7f541f1e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418100b90 0x7f541819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.430+0000 7f541f1e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181062b0 0x7f5418197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54181001f0 0x7f541819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f5417fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418100b90 0x7f541819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54181001f0 0x7f541819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.22:49756/0 (socket says 172.21.15.22:49756) 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f5417fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418100b90 0x7f541819dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:56052/0 (socket says 172.21.15.22:56052) 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 -- 172.21.15.22:0/2320545076 learned_addr learned my addr 172.21.15.22:0/2320545076 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54181116e0 con 0x7f54181062b0 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418111560 con 0x7f5418100b90 2024-03-09T00:13:15.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418111860 con 0x7f54181001f0 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541d783640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181062b0 0x7f5418197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 -- 172.21.15.22:0/2320545076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418100b90 msgr2=0x7f541819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418100b90 0x7f541819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 -- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181062b0 msgr2=0x7f5418197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181062b0 0x7f5418197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54181980a0 con 0x7f54181001f0 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541d783640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181062b0 0x7f5418197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.431+0000 7f541cf82640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54181001f0 0x7f541819d720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f540c00e9f0 tx=0x7f540c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:15.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.432+0000 7f5415ffb640 1 -- 172.21.15.22:0/2320545076 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c00cd80 con 0x7f54181001f0 2024-03-09T00:13:15.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.432+0000 7f5415ffb640 1 -- 172.21.15.22:0/2320545076 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f540c010910 con 0x7f54181001f0 2024-03-09T00:13:15.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.432+0000 7f5415ffb640 1 -- 172.21.15.22:0/2320545076 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c018990 con 0x7f54181001f0 2024-03-09T00:13:15.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.432+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5418198390 con 0x7f54181001f0 2024-03-09T00:13:15.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.432+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54181a45e0 con 0x7f54181001f0 2024-03-09T00:13:15.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.433+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5418067850 con 0x7f54181001f0 2024-03-09T00:13:15.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.435+0000 7f5415ffb640 1 -- 172.21.15.22:0/2320545076 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f540c010ac0 con 0x7f54181001f0 2024-03-09T00:13:15.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.436+0000 7f5415ffb640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54080a2970 0x7f54080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.436+0000 7f5415ffb640 1 -- 172.21.15.22:0/2320545076 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f540c0cf6c0 con 0x7f54181001f0 2024-03-09T00:13:15.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.436+0000 7f5417fff640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54080a2970 0x7f54080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.436+0000 7f5417fff640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54080a2970 0x7f54080a4e30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5404005ab0 tx=0x7f5404005a40 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:15.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.440+0000 7f5415ffb640 1 -- 172.21.15.22:0/2320545076 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f540c09bfd0 con 0x7f54181001f0 2024-03-09T00:13:15.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.788+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f541802ce70 con 0x7f54181001f0 2024-03-09T00:13:15.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.788+0000 7f5415ffb640 1 -- 172.21.15.22:0/2320545076 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f540c09b8d0 con 0x7f54181001f0 2024-03-09T00:13:15.786 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:13:15.786 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:10:14.710788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:13:15.787 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-03-09T00:13:15.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54080a2970 msgr2=0x7f54080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54080a2970 0x7f54080a4e30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5404005ab0 tx=0x7f5404005a40 comp rx=0 tx=0).stop 2024-03-09T00:13:15.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54181001f0 msgr2=0x7f541819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54181001f0 0x7f541819d720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f540c00e9f0 tx=0x7f540c00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 shutdown_connections 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f54080a2970 0x7f54080a4e30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54181062b0 0x7f5418197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418100b90 0x7f541819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 --2- 172.21.15.22:0/2320545076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54181001f0 0x7f541819d720 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 >> 172.21.15.22:0/2320545076 conn(0x7f5418077320 msgr2=0x7f54180fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 shutdown_connections 2024-03-09T00:13:15.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.791+0000 7f541f1e6640 1 -- 172.21.15.22:0/2320545076 wait complete. 2024-03-09T00:13:15.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T00:13:15.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 -- 172.21.15.22:0/460013969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c100460 msgr2=0x7fe33c100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 --2- 172.21.15.22:0/460013969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c100460 0x7fe33c100840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009860 tx=0x7fe32c02bb70 comp rx=0 tx=0).stop 2024-03-09T00:13:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 -- 172.21.15.22:0/460013969 shutdown_connections 2024-03-09T00:13:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 --2- 172.21.15.22:0/460013969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c105530 0x7fe33c068450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 --2- 172.21.15.22:0/460013969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe33c100e10 0x7fe33c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 --2- 172.21.15.22:0/460013969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c100460 0x7fe33c100840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 -- 172.21.15.22:0/460013969 >> 172.21.15.22:0/460013969 conn(0x7fe33c0772f0 msgr2=0x7fe33c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 -- 172.21.15.22:0/460013969 shutdown_connections 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.926+0000 7fe3422ab640 1 -- 172.21.15.22:0/460013969 wait complete. 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 Processor -- start 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 -- start start 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe33c100460 0x7fe33c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c105530 0x7fe33c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c197ac0 0x7fe33c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c1116e0 con 0x7fe33c100460 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c111560 con 0x7fe33c197ac0 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.927+0000 7fe3422ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c111860 con 0x7fe33c105530 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe33bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe33c100460 0x7fe33c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c197ac0 0x7fe33c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c197ac0 0x7fe33c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:56076/0 (socket says 172.21.15.22:56076) 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 -- 172.21.15.22:0/3756486602 learned_addr learned my addr 172.21.15.22:0/3756486602 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe333fff640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c105530 0x7fe33c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 -- 172.21.15.22:0/3756486602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c105530 msgr2=0x7fe33c19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c105530 0x7fe33c19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 -- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe33c100460 msgr2=0x7fe33c19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe33c100460 0x7fe33c19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 -- 172.21.15.22:0/3756486602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c1985e0 con 0x7fe33c197ac0 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe340848640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c197ac0 0x7fe33c197f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe33400efc0 tx=0x7fe33400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe339ffb640 1 -- 172.21.15.22:0/3756486602 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334003970 con 0x7fe33c197ac0 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.928+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c1988d0 con 0x7fe33c197ac0 2024-03-09T00:13:15.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.929+0000 7fe339ffb640 1 -- 172.21.15.22:0/3756486602 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe334004020 con 0x7fe33c197ac0 2024-03-09T00:13:15.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.929+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c107420 con 0x7fe33c197ac0 2024-03-09T00:13:15.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.929+0000 7fe339ffb640 1 -- 172.21.15.22:0/3756486602 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33400b9f0 con 0x7fe33c197ac0 2024-03-09T00:13:15.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.929+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c005290 con 0x7fe33c197ac0 2024-03-09T00:13:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.931+0000 7fe339ffb640 1 -- 172.21.15.22:0/3756486602 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe33401d020 con 0x7fe33c197ac0 2024-03-09T00:13:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.932+0000 7fe339ffb640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3280a29f0 0x7fe3280a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.932+0000 7fe339ffb640 1 -- 172.21.15.22:0/3756486602 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7fe3340cf760 con 0x7fe33c197ac0 2024-03-09T00:13:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.932+0000 7fe33bfff640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3280a29f0 0x7fe3280a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.932+0000 7fe33bfff640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3280a29f0 0x7fe3280a4eb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe32c02c180 tx=0x7fe32c0058b0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:15.936+0000 7fe339ffb640 1 -- 172.21.15.22:0/3756486602 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe33409c010 con 0x7fe33c197ac0 2024-03-09T00:13:16.125 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:15 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/352344951' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-09T00:13:16.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:15 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/352344951' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-09T00:13:16.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.220+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe31c005580 con 0x7fe33c197ac0 2024-03-09T00:13:16.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.221+0000 7fe339ffb640 1 -- 172.21.15.22:0/3756486602 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11343 (secure 0 0 0) 0x7fe33409b910 con 0x7fe33c197ac0 2024-03-09T00:13:16.219 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:13:16.219 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1092846184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":1092846184},{"type":"v1","addr":"172.21.15.22:6835","nonce":1092846184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/1255227444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":1255227444},{"type":"v1","addr":"172.21.15.22:6837","nonce":1255227444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/1763775736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":1763775736},{"type":"v1","addr":"172.21.15.22:6839","nonce":1763775736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/2885682684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":2885682684},{"type":"v1","addr":"172.21.15.22:6841","nonce":2885682684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/292631857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":292631857},{"type":"v1","addr":"172.21.15.62:6833","nonce":292631857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3599891094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3599891094},{"type":"v1","addr":"172.21.15.192:6833","nonce":3599891094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T23:51:51.610292+0000","modified":"2024-03-09T00:10:14.710788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24494,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24437":{"gid":24437,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/1656345994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1656345994},{"type":"v1","addr":"172.21.15.62:6835","nonce":1656345994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1768340207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1768340207},{"type":"v1","addr":"172.21.15.192:6835","nonce":1768340207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.192:6837/1326709093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1326709093},{"type":"v1","addr":"172.21.15.192:6837","nonce":1326709093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.62:6839/2804197530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":2804197530},{"type":"v1","addr":"172.21.15.62:6839","nonce":2804197530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.192:6839/1126404206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1126404206},{"type":"v1","addr":"172.21.15.192:6839","nonce":1126404206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_27395":{"gid":27395,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2391693225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2391693225},{"type":"v1","addr":"172.21.15.62:6837","nonce":2391693225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-09T00:13:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 37 2024-03-09T00:13:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3280a29f0 msgr2=0x7fe3280a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3280a29f0 0x7fe3280a4eb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe32c02c180 tx=0x7fe32c0058b0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c197ac0 msgr2=0x7fe33c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c197ac0 0x7fe33c197f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe33400efc0 tx=0x7fe33400c800 comp rx=0 tx=0).stop 2024-03-09T00:13:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 shutdown_connections 2024-03-09T00:13:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe3280a29f0 0x7fe3280a4eb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe33c197ac0 0x7fe33c197f70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c105530 0x7fe33c19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 --2- 172.21.15.22:0/3756486602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe33c100460 0x7fe33c19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 >> 172.21.15.22:0/3756486602 conn(0x7fe33c0772f0 msgr2=0x7fe33c103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:16.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 shutdown_connections 2024-03-09T00:13:16.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.224+0000 7fe3422ab640 1 -- 172.21.15.22:0/3756486602 wait complete. 2024-03-09T00:13:16.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:15 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/352344951' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-09T00:13:16.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T00:13:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 -- 172.21.15.22:0/506769490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec100db0 msgr2=0x7f76ec104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 --2- 172.21.15.22:0/506769490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec100db0 0x7f76ec104df0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009900 tx=0x7f76e002bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 -- 172.21.15.22:0/506769490 shutdown_connections 2024-03-09T00:13:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 --2- 172.21.15.22:0/506769490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ec1054d0 0x7f76ec10d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 --2- 172.21.15.22:0/506769490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec100db0 0x7f76ec104df0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 --2- 172.21.15.22:0/506769490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 0x7f76ec1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 -- 172.21.15.22:0/506769490 >> 172.21.15.22:0/506769490 conn(0x7f76ec076570 msgr2=0x7f76ec076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 -- 172.21.15.22:0/506769490 shutdown_connections 2024-03-09T00:13:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.361+0000 7f76f1c81640 1 -- 172.21.15.22:0/506769490 wait complete. 2024-03-09T00:13:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.362+0000 7f76f1c81640 1 Processor -- start 2024-03-09T00:13:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.362+0000 7f76f1c81640 1 -- start start 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.362+0000 7f76f1c81640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 0x7f76ec19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.362+0000 7f76f1c81640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ec1054d0 0x7f76ec19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76f1c81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec1979c0 0x7f76ec197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76f1c81640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec110560 con 0x7f76ec1979c0 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76f1c81640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec1103e0 con 0x7f76ec100400 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76f1c81640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec1106e0 con 0x7f76ec1054d0 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 0x7f76ec19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 0x7f76ec19d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:56094/0 (socket says 172.21.15.22:56094) 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2178530577 learned_addr learned my addr 172.21.15.22:0/2178530577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76ebfff640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec1979c0 0x7f76ec197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eaffd640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ec1054d0 0x7f76ec19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2178530577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ec1054d0 msgr2=0x7f76ec19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ec1054d0 0x7f76ec19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec1979c0 msgr2=0x7f76ec197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec1979c0 0x7f76ec197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 -- 172.21.15.22:0/2178530577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec1984e0 con 0x7f76ec100400 2024-03-09T00:13:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76eb7fe640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 0x7f76ec19d810 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e9f0 tx=0x7f76dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76e8ff9640 1 -- 172.21.15.22:0/2178530577 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00cd80 con 0x7f76ec100400 2024-03-09T00:13:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76e8ff9640 1 -- 172.21.15.22:0/2178530577 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76dc010910 con 0x7f76ec100400 2024-03-09T00:13:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec1987d0 con 0x7f76ec100400 2024-03-09T00:13:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.363+0000 7f76e8ff9640 1 -- 172.21.15.22:0/2178530577 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc018990 con 0x7f76ec100400 2024-03-09T00:13:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.364+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec102110 con 0x7f76ec100400 2024-03-09T00:13:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.364+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ec1018d0 con 0x7f76ec100400 2024-03-09T00:13:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.366+0000 7f76e8ff9640 1 -- 172.21.15.22:0/2178530577 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f76dc010ac0 con 0x7f76ec100400 2024-03-09T00:13:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.367+0000 7f76e8ff9640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f76d40a29f0 0x7f76d40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.367+0000 7f76eaffd640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f76d40a29f0 0x7f76d40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.367+0000 7f76e8ff9640 1 -- 172.21.15.22:0/2178530577 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f76dc0cfa40 con 0x7f76ec100400 2024-03-09T00:13:16.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.368+0000 7f76eaffd640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f76d40a29f0 0x7f76d40a4eb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009230 tx=0x7f76e003e040 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:16.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.372+0000 7f76e8ff9640 1 -- 172.21.15.22:0/2178530577 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f76dc09bca0 con 0x7f76ec100400 2024-03-09T00:13:16.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.639+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76ec100e40 con 0x7f76ec100400 2024-03-09T00:13:16.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.641+0000 7f76e8ff9640 1 -- 172.21.15.22:0/2178530577 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+25419 (secure 0 0 0) 0x7f76dc09fd40 con 0x7f76ec100400 2024-03-09T00:13:16.641 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-03-09T00:13:16.642 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":127,"fsid":"8c5a3016-dda5-11ee-95c6-87774f69a715","created":"2024-03-08T23:43:41.144090+0000","modified":"2024-03-09T00:13:05.727861+0000","last_up_change":"2024-03-08T23:51:04.554776+0000","last_in_change":"2024-03-08T23:50:44.505218+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T23:47:12.051601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T23:51:50.171325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T23:51:50.798188+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-08T23:51:54.496659+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3bc68a9c-ec9a-4342-aa98-70c2835d2f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6803","nonce":103047658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6805","nonce":103047658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6809","nonce":103047658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":103047658},{"type":"v1","addr":"172.21.15.22:6807","nonce":103047658}]},"public_addr":"172.21.15.22:6803/103047658","cluster_addr":"172.21.15.22:6805/103047658","heartbeat_back_addr":"172.21.15.22:6809/103047658","heartbeat_front_addr":"172.21.15.22:6807/103047658","state":["exists","up"]},{"osd":1,"uuid":"06bad818-44be-4c08-a916-5e7cf0548753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6801","nonce":782240245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6803","nonce":782240245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6807","nonce":782240245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":782240245},{"type":"v1","addr":"172.21.15.62:6805","nonce":782240245}]},"public_addr":"172.21.15.62:6801/782240245","cluster_addr":"172.21.15.62:6803/782240245","heartbeat_back_addr":"172.21.15.62:6807/782240245","heartbeat_front_addr":"172.21.15.62:6805/782240245","state":["exists","up"]},{"osd":2,"uuid":"d5674ba8-0f77-4cca-baa2-8d27058e2259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6801","nonce":316512323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6803","nonce":316512323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6807","nonce":316512323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":316512323},{"type":"v1","addr":"172.21.15.192:6805","nonce":316512323}]},"public_addr":"172.21.15.192:6801/316512323","cluster_addr":"172.21.15.192:6803/316512323","heartbeat_back_addr":"172.21.15.192:6807/316512323","heartbeat_front_addr":"172.21.15.192:6805/316512323","state":["exists","up"]},{"osd":3,"uuid":"da0b079a-88ee-483c-b094-40c3cc5fdcd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6811","nonce":2127248159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6813","nonce":2127248159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6817","nonce":2127248159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":2127248159},{"type":"v1","addr":"172.21.15.22:6815","nonce":2127248159}]},"public_addr":"172.21.15.22:6811/2127248159","cluster_addr":"172.21.15.22:6813/2127248159","heartbeat_back_addr":"172.21.15.22:6817/2127248159","heartbeat_front_addr":"172.21.15.22:6815/2127248159","state":["exists","up"]},{"osd":4,"uuid":"9b0bac61-2019-4ebf-8688-8ae47e9457ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6809","nonce":590253654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6811","nonce":590253654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6815","nonce":590253654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":590253654},{"type":"v1","addr":"172.21.15.62:6813","nonce":590253654}]},"public_addr":"172.21.15.62:6809/590253654","cluster_addr":"172.21.15.62:6811/590253654","heartbeat_back_addr":"172.21.15.62:6815/590253654","heartbeat_front_addr":"172.21.15.62:6813/590253654","state":["exists","up"]},{"osd":5,"uuid":"45fe7b13-24f3-4e73-984b-04ff364a0d6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6809","nonce":3950935286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6811","nonce":3950935286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6815","nonce":3950935286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3950935286},{"type":"v1","addr":"172.21.15.192:6813","nonce":3950935286}]},"public_addr":"172.21.15.192:6809/3950935286","cluster_addr":"172.21.15.192:6811/3950935286","heartbeat_back_addr":"172.21.15.192:6815/3950935286","heartbeat_front_addr":"172.21.15.192:6813/3950935286","state":["exists","up"]},{"osd":6,"uuid":"be6cb9f2-a8f1-410a-a997-76adcb1fa023","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6819","nonce":1438134132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6821","nonce":1438134132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6825","nonce":1438134132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1438134132},{"type":"v1","addr":"172.21.15.22:6823","nonce":1438134132}]},"public_addr":"172.21.15.22:6819/1438134132","cluster_addr":"172.21.15.22:6821/1438134132","heartbeat_back_addr":"172.21.15.22:6825/1438134132","heartbeat_front_addr":"172.21.15.22:6823/1438134132","state":["exists","up"]},{"osd":7,"uuid":"db83755e-a0a0-435c-963a-41f2eeb41266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6817","nonce":92390165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6819","nonce":92390165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6823","nonce":92390165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":92390165},{"type":"v1","addr":"172.21.15.62:6821","nonce":92390165}]},"public_addr":"172.21.15.62:6817/92390165","cluster_addr":"172.21.15.62:6819/92390165","heartbeat_back_addr":"172.21.15.62:6823/92390165","heartbeat_front_addr":"172.21.15.62:6821/92390165","state":["exists","up"]},{"osd":8,"uuid":"766c75f7-7820-4cb6-ac6b-ab7596735fc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6817","nonce":3107982312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6819","nonce":3107982312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6823","nonce":3107982312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3107982312},{"type":"v1","addr":"172.21.15.192:6821","nonce":3107982312}]},"public_addr":"172.21.15.192:6817/3107982312","cluster_addr":"172.21.15.192:6819/3107982312","heartbeat_back_addr":"172.21.15.192:6823/3107982312","heartbeat_front_addr":"172.21.15.192:6821/3107982312","state":["exists","up"]},{"osd":9,"uuid":"029fa555-d0bd-4863-9199-b610bec7e240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6827","nonce":460204078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6829","nonce":460204078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6833","nonce":460204078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":460204078},{"type":"v1","addr":"172.21.15.22:6831","nonce":460204078}]},"public_addr":"172.21.15.22:6827/460204078","cluster_addr":"172.21.15.22:6829/460204078","heartbeat_back_addr":"172.21.15.22:6833/460204078","heartbeat_front_addr":"172.21.15.22:6831/460204078","state":["exists","up"]},{"osd":10,"uuid":"0094504e-2371-485f-b601-d2e36f3062ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6825","nonce":1674155650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6827","nonce":1674155650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6831","nonce":1674155650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1674155650},{"type":"v1","addr":"172.21.15.62:6829","nonce":1674155650}]},"public_addr":"172.21.15.62:6825/1674155650","cluster_addr":"172.21.15.62:6827/1674155650","heartbeat_back_addr":"172.21.15.62:6831/1674155650","heartbeat_front_addr":"172.21.15.62:6829/1674155650","state":["exists","up"]},{"osd":11,"uuid":"182eae10-50a1-43f2-b776-13486e2939b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6825","nonce":3259500463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6827","nonce":3259500463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6831","nonce":3259500463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":3259500463},{"type":"v1","addr":"172.21.15.192:6829","nonce":3259500463}]},"public_addr":"172.21.15.192:6825/3259500463","cluster_addr":"172.21.15.192:6827/3259500463","heartbeat_back_addr":"172.21.15.192:6831/3259500463","heartbeat_front_addr":"172.21.15.192:6829/3259500463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:21.919085+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:46:46.036214+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:09.586253+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:47:35.323917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:00.495036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:25.763295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:48:52.112049+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:18.410789+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:49:44.324619+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:10.187177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:50:36.874546+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T23:51:02.913197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/3461287105":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/2337825839":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6800/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/1632997532":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/1968010400":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3762004902":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3477373324":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/99443044":"2024-03-09T23:44:43.705970+0000","172.21.15.22:6801/3384641534":"2024-03-09T23:44:16.515766+0000","172.21.15.22:0/3201080518":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1865576829":"2024-03-09T23:54:32.857800+0000","172.21.15.22:6800/2813827512":"2024-03-09T23:54:32.857800+0000","172.21.15.22:6801/4028157579":"2024-03-09T23:44:43.705970+0000","172.21.15.22:0/1683058859":"2024-03-09T23:54:32.857800+0000","172.21.15.22:0/1133673382":"2024-03-09T23:44:16.515766+0000","172.21.15.22:6801/2813827512":"2024-03-09T23:54:32.857800+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]},{"pool":4,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]},{"pool":4,"snaps":[{"begin":19,"length":1},{"begin":32,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T00:13:16.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f76d40a29f0 msgr2=0x7f76d40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f76d40a29f0 0x7f76d40a4eb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009230 tx=0x7f76e003e040 comp rx=0 tx=0).stop 2024-03-09T00:13:16.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 msgr2=0x7f76ec19d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 0x7f76ec19d810 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e9f0 tx=0x7f76dc00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 shutdown_connections 2024-03-09T00:13:16.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f76d40a29f0 0x7f76d40a4eb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76ec1979c0 0x7f76ec197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ec1054d0 0x7f76ec19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 --2- 172.21.15.22:0/2178530577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec100400 0x7f76ec19d810 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e9f0 tx=0x7f76dc00eeb0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.644+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 >> 172.21.15.22:0/2178530577 conn(0x7f76ec076570 msgr2=0x7f76ec104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:16.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.645+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 shutdown_connections 2024-03-09T00:13:16.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.645+0000 7f76f1c81640 1 -- 172.21.15.22:0/2178530577 wait complete. 2024-03-09T00:13:16.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-09T00:13:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 -- 172.21.15.22:0/3386060199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 msgr2=0x7f31481055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 --2- 172.21.15.22:0/3386060199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481055c0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f3138009900 tx=0x7f313802bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 -- 172.21.15.22:0/3386060199 shutdown_connections 2024-03-09T00:13:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 --2- 172.21.15.22:0/3386060199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 0x7f314810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 --2- 172.21.15.22:0/3386060199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3148100900 0x7f3148100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 --2- 172.21.15.22:0/3386060199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481055c0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 -- 172.21.15.22:0/3386060199 >> 172.21.15.22:0/3386060199 conn(0x7f31480767b0 msgr2=0x7f31480fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 -- 172.21.15.22:0/3386060199 shutdown_connections 2024-03-09T00:13:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.791+0000 7f314cf16640 1 -- 172.21.15.22:0/3386060199 wait complete. 2024-03-09T00:13:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 Processor -- start 2024-03-09T00:13:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 -- start start 2024-03-09T00:13:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3148100900 0x7f31481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 0x7f3148198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3148110590 con 0x7f31481051e0 2024-03-09T00:13:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3148110410 con 0x7f31481012c0 2024-03-09T00:13:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314cf16640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3148110710 con 0x7f3148100900 2024-03-09T00:13:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f3145d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 0x7f3148198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f3146d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f3145d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 0x7f3148198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.22:56124/0 (socket says 172.21.15.22:56124) 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f3146d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33910/0 (socket says 172.21.15.22:33910) 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f3145d9b640 1 -- 172.21.15.22:0/222353449 learned_addr learned my addr 172.21.15.22:0/222353449 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.792+0000 7f314659c640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3148100900 0x7f31481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f3145d9b640 1 -- 172.21.15.22:0/222353449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3148100900 msgr2=0x7f31481989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f3145d9b640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3148100900 0x7f31481989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f3145d9b640 1 -- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 msgr2=0x7f31481a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f3145d9b640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f3145d9b640 1 -- 172.21.15.22:0/222353449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31481a3f90 con 0x7f31481012c0 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f3146d9d640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f3145d9b640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 0x7f3148198f10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f313c00d950 tx=0x7f313c00dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f31377fe640 1 -- 172.21.15.22:0/222353449 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c017070 con 0x7f31481012c0 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31481a4160 con 0x7f31481012c0 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f31377fe640 1 -- 172.21.15.22:0/222353449 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f313c00de80 con 0x7f31481012c0 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f31377fe640 1 -- 172.21.15.22:0/222353449 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00f620 con 0x7f31481012c0 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.793+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31481a4680 con 0x7f31481012c0 2024-03-09T00:13:16.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.794+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3148068530 con 0x7f31481012c0 2024-03-09T00:13:16.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.796+0000 7f31377fe640 1 -- 172.21.15.22:0/222353449 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f313c009c30 con 0x7f31481012c0 2024-03-09T00:13:16.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.796+0000 7f31377fe640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f31300a2970 0x7f31300a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:16.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.796+0000 7f31377fe640 1 -- 172.21.15.22:0/222353449 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9627+0+0 (secure 0 0 0) 0x7f313c0cef00 con 0x7f31481012c0 2024-03-09T00:13:16.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.797+0000 7f314659c640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f31300a2970 0x7f31300a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:16.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.797+0000 7f314659c640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f31300a2970 0x7f31300a4e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f313802efd0 tx=0x7f3138000bc0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:16.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:16.801+0000 7f31377fe640 1 -- 172.21.15.22:0/222353449 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f313c09b7b0 con 0x7f31481012c0 2024-03-09T00:13:16.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:16 smithi022 ceph-mon[31894]: pgmap v597: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 366 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 107 KiB/s wr, 10 op/s 2024-03-09T00:13:16.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:16 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2320545076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:13:16.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:16 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/3756486602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:13:16.987 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:16 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2178530577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:13:17.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.096+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3148108710 con 0x7f31481012c0 2024-03-09T00:13:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.106+0000 7f31377fe640 1 -- 172.21.15.22:0/222353449 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7f313c09b0b0 con 0x7f31481012c0 2024-03-09T00:13:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-09T00:13:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f31300a2970 msgr2=0x7f31300a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:17.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f31300a2970 0x7f31300a4e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f313802efd0 tx=0x7f3138000bc0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 msgr2=0x7f3148198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 0x7f3148198f10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f313c00d950 tx=0x7f313c00dd50 comp rx=0 tx=0).stop 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 shutdown_connections 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7f31300a2970 0x7f31300a4e30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31481051e0 0x7f31481a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31481012c0 0x7f3148198f10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 --2- 172.21.15.22:0/222353449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3148100900 0x7f31481989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 >> 172.21.15.22:0/222353449 conn(0x7f31480767b0 msgr2=0x7f3148103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 shutdown_connections 2024-03-09T00:13:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.108+0000 7f314cf16640 1 -- 172.21.15.22:0/222353449 wait complete. 2024-03-09T00:13:17.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-09T00:13:17.126 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:16 smithi062 ceph-mon[36129]: pgmap v597: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 366 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 107 KiB/s wr, 10 op/s 2024-03-09T00:13:17.126 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:16 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2320545076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:13:17.126 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:16 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/3756486602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:13:17.126 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:16 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2178530577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:13:17.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:16 smithi192 ceph-mon[35995]: pgmap v597: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 366 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 107 KiB/s wr, 10 op/s 2024-03-09T00:13:17.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:16 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2320545076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:13:17.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:16 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/3756486602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-09T00:13:17.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:16 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2178530577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-09T00:13:17.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/3793923857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1003d0 msgr2=0x7ff19c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/3793923857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1003d0 0x7ff19c1007b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009900 tx=0x7ff18c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/3793923857 shutdown_connections 2024-03-09T00:13:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/3793923857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c1054a0 0x7ff19c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/3793923857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff19c100d80 0x7ff19c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/3793923857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1003d0 0x7ff19c1007b0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/3793923857 >> 172.21.15.22:0/3793923857 conn(0x7ff19c0772f0 msgr2=0x7ff19c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/3793923857 shutdown_connections 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.246+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/3793923857 wait complete. 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 Processor -- start 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 -- start start 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff19c1003d0 0x7ff19c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c1054a0 0x7ff19c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1994a0 0x7ff19c1a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1105b0 con 0x7ff19c1994a0 2024-03-09T00:13:17.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c110430 con 0x7ff19c1003d0 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff1a1dc4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c110730 con 0x7ff19c1054a0 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff19bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1994a0 0x7ff19c1a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff19b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff19c1003d0 0x7ff19c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff19bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1994a0 0x7ff19c1a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33938/0 (socket says 172.21.15.22:33938) 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.247+0000 7ff19bfff640 1 -- 172.21.15.22:0/2185769715 learned_addr learned my addr 172.21.15.22:0/2185769715 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff192ffd640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c1054a0 0x7ff19c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff19b7fe640 1 -- 172.21.15.22:0/2185769715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c1054a0 msgr2=0x7ff19c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff19b7fe640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c1054a0 0x7ff19c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff19b7fe640 1 -- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1994a0 msgr2=0x7ff19c1a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff19b7fe640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1994a0 0x7ff19c1a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff19b7fe640 1 -- 172.21.15.22:0/2185769715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c1a4220 con 0x7ff19c1003d0 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff19b7fe640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff19c1003d0 0x7ff19c1989d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff18c02c170 tx=0x7ff18c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff1997fa640 1 -- 172.21.15.22:0/2185769715 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c043020 con 0x7ff19c1003d0 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff1997fa640 1 -- 172.21.15.22:0/2185769715 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff18c0054e0 con 0x7ff19c1003d0 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c1967c0 con 0x7ff19c1003d0 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff1997fa640 1 -- 172.21.15.22:0/2185769715 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c003b80 con 0x7ff19c1003d0 2024-03-09T00:13:17.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.248+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c196d30 con 0x7ff19c1003d0 2024-03-09T00:13:17.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.250+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c0680e0 con 0x7ff19c1003d0 2024-03-09T00:13:17.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.251+0000 7ff1997fa640 1 -- 172.21.15.22:0/2185769715 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff18c046020 con 0x7ff19c1003d0 2024-03-09T00:13:17.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.251+0000 7ff1997fa640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1880a29f0 0x7ff1880a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:17.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.252+0000 7ff192ffd640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1880a29f0 0x7ff1880a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:17.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.252+0000 7ff1997fa640 1 -- 172.21.15.22:0/2185769715 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9797+0+0 (secure 0 0 0) 0x7ff18c0f1250 con 0x7ff19c1003d0 2024-03-09T00:13:17.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.252+0000 7ff192ffd640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1880a29f0 0x7ff1880a4eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff19c068fa0 tx=0x7ff184005f90 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:17.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.257+0000 7ff1997fa640 1 -- 172.21.15.22:0/2185769715 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff18c04d020 con 0x7ff19c1003d0 2024-03-09T00:13:17.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a[31890]: 2024-03-09T00:13:17.097+0000 7ff351574700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-09T00:13:17.626 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b[66615]: ignoring --setuser ceph since I am not root 2024-03-09T00:13:17.626 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b[66615]: ignoring --setgroup ceph since I am not root 2024-03-09T00:13:17.626 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mds[66619]: -- 172.21.15.62:0/518697412 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c968dd7a40 con 0x55c968daac00 2024-03-09T00:13:17.627 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b[66615]: starting mds.b at 2024-03-09T00:13:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:17.571+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff19c1086d0 con 0x7ff19c1003d0 2024-03-09T00:13:17.627 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e[65998]: ignoring --setuser ceph since I am not root 2024-03-09T00:13:17.627 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e[65998]: ignoring --setgroup ceph since I am not root 2024-03-09T00:13:17.628 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mds[66002]: -- 172.21.15.62:0/1930503480 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f66d229a40 con 0x55f66d1fc800 2024-03-09T00:13:17.628 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e[65998]: starting mds.e at 2024-03-09T00:13:17.628 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setuser ceph since I am not root 2024-03-09T00:13:17.628 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: ignoring --setgroup ceph since I am not root 2024-03-09T00:13:17.628 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mds[70095]: -- 172.21.15.62:0/1178262422 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x565032959a40 con 0x56503292cc00 2024-03-09T00:13:17.628 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: starting mds.h at 2024-03-09T00:13:17.632 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: ignoring --setuser ceph since I am not root 2024-03-09T00:13:17.632 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: ignoring --setgroup ceph since I am not root 2024-03-09T00:13:17.632 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mds[67827]: -- 172.21.15.192:0/2882412545 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562eed1c1a40 con 0x562eed194c00 2024-03-09T00:13:17.632 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: starting mds.i at 2024-03-09T00:13:17.633 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f[67202]: ignoring --setuser ceph since I am not root 2024-03-09T00:13:17.633 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f[67202]: ignoring --setgroup ceph since I am not root 2024-03-09T00:13:17.633 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mds[67206]: -- 172.21.15.192:0/2309471176 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c02cc51a40 con 0x55c02cc25000 2024-03-09T00:13:17.633 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f[67202]: starting mds.f at 2024-03-09T00:13:17.634 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c[67511]: ignoring --setuser ceph since I am not root 2024-03-09T00:13:17.634 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c[67511]: ignoring --setgroup ceph since I am not root 2024-03-09T00:13:17.634 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mds[67515]: -- 172.21.15.192:0/697173457 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b82e75a40 con 0x561b82e48c00 2024-03-09T00:13:17.634 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c[67511]: starting mds.c at 2024-03-09T00:13:18.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.108+0000 7ff1997fa640 1 -- 172.21.15.22:0/2185769715 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff18c0bdbc0 con 0x7ff19c1003d0 2024-03-09T00:13:18.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1880a29f0 msgr2=0x7ff1880a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:18.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1880a29f0 0x7ff1880a4eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff19c068fa0 tx=0x7ff184005f90 comp rx=0 tx=0).stop 2024-03-09T00:13:18.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff19c1003d0 msgr2=0x7ff19c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:18.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff19c1003d0 0x7ff19c1989d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff18c02c170 tx=0x7ff18c004360 comp rx=0 tx=0).stop 2024-03-09T00:13:18.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 shutdown_connections 2024-03-09T00:13:18.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7ff1880a29f0 0x7ff1880a4eb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff19c1994a0 0x7ff19c1a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c1054a0 0x7ff19c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 --2- 172.21.15.22:0/2185769715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff19c1003d0 0x7ff19c1989d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 >> 172.21.15.22:0/2185769715 conn(0x7ff19c0772f0 msgr2=0x7ff19c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:18.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 shutdown_connections 2024-03-09T00:13:18.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.111+0000 7ff1a1dc4640 1 -- 172.21.15.22:0/2185769715 wait complete. 2024-03-09T00:13:18.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/222353449' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-09T00:13:18.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-09T00:13:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-09T00:13:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-09T00:13:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-09T00:13:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: osdmap e128: 12 total, 12 up, 12 in 2024-03-09T00:13:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-09T00:13:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2185769715' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-09T00:13:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:17 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-09T00:13:18.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/222353449' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-09T00:13:18.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-09T00:13:18.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-09T00:13:18.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-09T00:13:18.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-09T00:13:18.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: osdmap e128: 12 total, 12 up, 12 in 2024-03-09T00:13:18.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-09T00:13:18.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2185769715' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-09T00:13:18.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:17 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-09T00:13:18.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-09T00:13:18.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/222353449' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-09T00:13:18.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-09T00:13:18.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-09T00:13:18.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-09T00:13:18.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-09T00:13:18.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: osdmap e128: 12 total, 12 up, 12 in 2024-03-09T00:13:18.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-09T00:13:18.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2185769715' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-09T00:13:18.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:17 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-09T00:13:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.232+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/3015256105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec0690f0 msgr2=0x7fdcec105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.232+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/3015256105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec0690f0 0x7fdcec105ec0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009900 tx=0x7fdcdc02bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.232+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/3015256105 shutdown_connections 2024-03-09T00:13:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.232+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/3015256105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec1065a0 0x7fdcec10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.232+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/3015256105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec0690f0 0x7fdcec105ec0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.232+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/3015256105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcec068740 0x7fdcec068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.232+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/3015256105 >> 172.21.15.22:0/3015256105 conn(0x7fdcec0772f0 msgr2=0x7fdcec077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/3015256105 shutdown_connections 2024-03-09T00:13:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/3015256105 wait complete. 2024-03-09T00:13:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 Processor -- start 2024-03-09T00:13:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 -- start start 2024-03-09T00:13:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec068740 0x7fdcec19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcec1065a0 0x7fdcec19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec197970 0x7fdcec197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec111630 con 0x7fdcec068740 2024-03-09T00:13:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec1114b0 con 0x7fdcec1065a0 2024-03-09T00:13:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf3cf1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec1117b0 con 0x7fdcec197970 2024-03-09T00:13:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf1a8d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec068740 0x7fdcec19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf1a8d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec068740 0x7fdcec19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33966/0 (socket says 172.21.15.22:33966) 2024-03-09T00:13:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.233+0000 7fdcf1a8d640 1 -- 172.21.15.22:0/301917799 learned_addr learned my addr 172.21.15.22:0/301917799 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf128c640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcec1065a0 0x7fdcec19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf228e640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec197970 0x7fdcec197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf1a8d640 1 -- 172.21.15.22:0/301917799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec197970 msgr2=0x7fdcec197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf1a8d640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec197970 0x7fdcec197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf1a8d640 1 -- 172.21.15.22:0/301917799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcec1065a0 msgr2=0x7fdcec19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf1a8d640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcec1065a0 0x7fdcec19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf1a8d640 1 -- 172.21.15.22:0/301917799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec198490 con 0x7fdcec068740 2024-03-09T00:13:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf128c640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcec1065a0 0x7fdcec19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:13:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf228e640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec197970 0x7fdcec197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:13:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf1a8d640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec068740 0x7fdcec19d770 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c920 tx=0x7fdce800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdce2ffd640 1 -- 172.21.15.22:0/301917799 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8007c70 con 0x7fdcec068740 2024-03-09T00:13:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec198780 con 0x7fdcec068740 2024-03-09T00:13:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdce2ffd640 1 -- 172.21.15.22:0/301917799 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdce8007e10 con 0x7fdcec068740 2024-03-09T00:13:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdce2ffd640 1 -- 172.21.15.22:0/301917799 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8018890 con 0x7fdcec068740 2024-03-09T00:13:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.234+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec1a4980 con 0x7fdcec068740 2024-03-09T00:13:18.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.235+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec0680e0 con 0x7fdcec068740 2024-03-09T00:13:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.236+0000 7fdce2ffd640 1 -- 172.21.15.22:0/301917799 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fdce8018a50 con 0x7fdcec068740 2024-03-09T00:13:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.237+0000 7fdce2ffd640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdcd80a2970 0x7fdcd80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.237+0000 7fdcf128c640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdcd80a2970 0x7fdcd80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.237+0000 7fdce2ffd640 1 -- 172.21.15.22:0/301917799 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9797+0+0 (secure 0 0 0) 0x7fdce80cf820 con 0x7fdcec068740 2024-03-09T00:13:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.237+0000 7fdcf128c640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdcd80a2970 0x7fdcd80a4e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc006010 tx=0x7fdcdc02c7d0 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:18.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.240+0000 7fdce2ffd640 1 -- 172.21.15.22:0/301917799 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdce8014030 con 0x7fdcec068740 2024-03-09T00:13:18.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:18.425+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdcec198e50 con 0x7fdcec068740 2024-03-09T00:13:19.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.053+0000 7fdce2ffd640 1 -- 172.21.15.22:0/301917799 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdce809c110 con 0x7fdcec068740 2024-03-09T00:13:19.051 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-03-09T00:13:19.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.057+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdcd80a2970 msgr2=0x7fdcd80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:19.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.057+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdcd80a2970 0x7fdcd80a4e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc006010 tx=0x7fdcdc02c7d0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.057+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec068740 msgr2=0x7fdcec19d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:19.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.057+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec068740 0x7fdcec19d770 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c920 tx=0x7fdce800cde0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 shutdown_connections 2024-03-09T00:13:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fdcd80a2970 0x7fdcd80a4e30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec197970 0x7fdcec197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcec1065a0 0x7fdcec19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 --2- 172.21.15.22:0/301917799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcec068740 0x7fdcec19d770 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 >> 172.21.15.22:0/301917799 conn(0x7fdcec0772f0 msgr2=0x7fdcec0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 shutdown_connections 2024-03-09T00:13:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.058+0000 7fdcf3cf1640 1 -- 172.21.15.22:0/301917799 wait complete. 2024-03-09T00:13:19.071 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: pgmap v599: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 363 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 107 KiB/s wr, 10 op/s 2024-03-09T00:13:19.071 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: Cluster is now healthy 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6838/3301789151,v1:172.21.15.62:6839/3301789151] up:boot 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6834/3634346225,v1:172.21.15.62:6836/3634346225] up:boot 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.62:6835/1538399209,v1:172.21.15.62:6837/1538399209] up:boot 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6837/3214599949,v1:172.21.15.192:6839/3214599949] up:boot 2024-03-09T00:13:19.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6834/3190235287,v1:172.21.15.192:6835/3190235287] up:boot 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: mds.? [v2:172.21.15.192:6836/2246819644,v1:172.21.15.192:6838/2246819644] up:boot 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: fsmap 12 up:standby 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-09T00:13:19.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-09T00:13:19.074 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:18 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/301917799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:19.075 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-09T00:13:19.112 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: pgmap v599: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 363 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 107 KiB/s wr, 10 op/s 2024-03-09T00:13:19.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-09T00:13:19.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-09T00:13:19.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: Cluster is now healthy 2024-03-09T00:13:19.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-09T00:13:19.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6838/3301789151,v1:172.21.15.62:6839/3301789151] up:boot 2024-03-09T00:13:19.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6834/3634346225,v1:172.21.15.62:6836/3634346225] up:boot 2024-03-09T00:13:19.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.62:6835/1538399209,v1:172.21.15.62:6837/1538399209] up:boot 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6837/3214599949,v1:172.21.15.192:6839/3214599949] up:boot 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6834/3190235287,v1:172.21.15.192:6835/3190235287] up:boot 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: mds.? [v2:172.21.15.192:6836/2246819644,v1:172.21.15.192:6838/2246819644] up:boot 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: fsmap 12 up:standby 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-09T00:13:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-09T00:13:19.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-09T00:13:19.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-09T00:13:19.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:18 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/301917799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:19.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: pgmap v599: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 363 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 107 KiB/s wr, 10 op/s 2024-03-09T00:13:19.132 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: Cluster is now healthy 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6838/3301789151,v1:172.21.15.62:6839/3301789151] up:boot 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6834/3634346225,v1:172.21.15.62:6836/3634346225] up:boot 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.62:6835/1538399209,v1:172.21.15.62:6837/1538399209] up:boot 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6837/3214599949,v1:172.21.15.192:6839/3214599949] up:boot 2024-03-09T00:13:19.133 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6834/3190235287,v1:172.21.15.192:6835/3190235287] up:boot 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: mds.? [v2:172.21.15.192:6836/2246819644,v1:172.21.15.192:6838/2246819644] up:boot 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: fsmap 12 up:standby 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='mgr.15660 172.21.15.22:0/1044185034' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-09T00:13:19.134 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:18 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/301917799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 -- 172.21.15.22:0/1071737807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 msgr2=0x7fb43006a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 --2- 172.21.15.22:0/1071737807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43006a030 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009900 tx=0x7fb42c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 -- 172.21.15.22:0/1071737807 shutdown_connections 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 --2- 172.21.15.22:0/1071737807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb430106640 0x7fb43010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 --2- 172.21.15.22:0/1071737807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb43006a600 0x7fb430105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 --2- 172.21.15.22:0/1071737807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43006a030 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 -- 172.21.15.22:0/1071737807 >> 172.21.15.22:0/1071737807 conn(0x7fb430076570 msgr2=0x7fb430076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 -- 172.21.15.22:0/1071737807 shutdown_connections 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.198+0000 7fb4376d1640 1 -- 172.21.15.22:0/1071737807 wait complete. 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 Processor -- start 2024-03-09T00:13:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 -- start start 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430106640 0x7fb43019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb430197970 0x7fb430197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430111690 con 0x7fb430069c50 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430111510 con 0x7fb430197970 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb4376d1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430111810 con 0x7fb430106640 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:19.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33998/0 (socket says 172.21.15.22:33998) 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 -- 172.21.15.22:0/2873105977 learned_addr learned my addr 172.21.15.22:0/2873105977 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb434c6c640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430106640 0x7fb43019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 -- 172.21.15.22:0/2873105977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430106640 msgr2=0x7fb43019dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb435c6e640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb430197970 0x7fb430197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430106640 0x7fb43019dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 -- 172.21.15.22:0/2873105977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb430197970 msgr2=0x7fb430197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb430197970 0x7fb430197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:19.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb43546d640 1 -- 172.21.15.22:0/2873105977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb430198490 con 0x7fb430069c50 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb434c6c640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430106640 0x7fb43019dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.199+0000 7fb435c6e640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb430197970 0x7fb430197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.200+0000 7fb43546d640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43019d7c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.200+0000 7fb4267fc640 1 -- 172.21.15.22:0/2873105977 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c041070 con 0x7fb430069c50 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.200+0000 7fb4267fc640 1 -- 172.21.15.22:0/2873105977 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42c004450 con 0x7fb430069c50 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.200+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb430198720 con 0x7fb430069c50 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.200+0000 7fb4267fc640 1 -- 172.21.15.22:0/2873105977 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c034440 con 0x7fb430069c50 2024-03-09T00:13:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.200+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4301a4970 con 0x7fb430069c50 2024-03-09T00:13:19.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.201+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4301029d0 con 0x7fb430069c50 2024-03-09T00:13:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.202+0000 7fb4267fc640 1 -- 172.21.15.22:0/2873105977 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb42c044070 con 0x7fb430069c50 2024-03-09T00:13:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.202+0000 7fb4267fc640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb41c0a29f0 0x7fb41c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.202+0000 7fb434c6c640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb41c0a29f0 0x7fb41c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.203+0000 7fb4267fc640 1 -- 172.21.15.22:0/2873105977 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9284+0+0 (secure 0 0 0) 0x7fb42c0f0f50 con 0x7fb430069c50 2024-03-09T00:13:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.203+0000 7fb434c6c640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb41c0a29f0 0x7fb41c0a4eb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4300ffae0 tx=0x7fb420005e50 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.206+0000 7fb4267fc640 1 -- 172.21.15.22:0/2873105977 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb42c0bda60 con 0x7fb430069c50 2024-03-09T00:13:19.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:19.406+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb430068530 con 0x7fb430069c50 2024-03-09T00:13:20.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.059+0000 7fb4267fc640 1 -- 172.21.15.22:0/2873105977 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb42c0bd360 con 0x7fb430069c50 2024-03-09T00:13:20.056 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-03-09T00:13:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb41c0a29f0 msgr2=0x7fb41c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb41c0a29f0 0x7fb41c0a4eb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4300ffae0 tx=0x7fb420005e50 comp rx=0 tx=0).stop 2024-03-09T00:13:20.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 msgr2=0x7fb43019d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43019d7c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c02cc70 comp rx=0 tx=0).stop 2024-03-09T00:13:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 shutdown_connections 2024-03-09T00:13:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fb41c0a29f0 0x7fb41c0a4eb0 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4300ffae0 tx=0x7fb420005e50 comp rx=0 tx=0).stop 2024-03-09T00:13:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb430197970 0x7fb430197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430106640 0x7fb43019dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 --2- 172.21.15.22:0/2873105977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb430069c50 0x7fb43019d7c0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.063+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 >> 172.21.15.22:0/2873105977 conn(0x7fb430076570 msgr2=0x7fb4300fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.064+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 shutdown_connections 2024-03-09T00:13:20.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.064+0000 7fb4376d1640 1 -- 172.21.15.22:0/2873105977 wait complete. 2024-03-09T00:13:20.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-09T00:13:20.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 -- 172.21.15.22:0/902583579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1003d0 msgr2=0x7fe49c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:20.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 --2- 172.21.15.22:0/902583579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1003d0 0x7fe49c1007b0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009900 tx=0x7fe48c02bc30 comp rx=0 tx=0).stop 2024-03-09T00:13:20.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 -- 172.21.15.22:0/902583579 shutdown_connections 2024-03-09T00:13:20.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 --2- 172.21.15.22:0/902583579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c1054a0 0x7fe49c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 --2- 172.21.15.22:0/902583579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c100d80 0x7fe49c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 --2- 172.21.15.22:0/902583579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1003d0 0x7fe49c1007b0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 -- 172.21.15.22:0/902583579 >> 172.21.15.22:0/902583579 conn(0x7fe49c0772f0 msgr2=0x7fe49c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 -- 172.21.15.22:0/902583579 shutdown_connections 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.207+0000 7fe4a1790640 1 -- 172.21.15.22:0/902583579 wait complete. 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 Processor -- start 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 -- start start 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c1003d0 0x7fe49c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1054a0 0x7fe49c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c196870 0x7fe49c196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c110550 con 0x7fe49c1054a0 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c1103d0 con 0x7fe49c196870 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe4a1790640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c1106d0 con 0x7fe49c1003d0 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe49a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1054a0 0x7fe49c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe49a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1054a0 0x7fe49c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34008/0 (socket says 172.21.15.22:34008) 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe49a7fc640 1 -- 172.21.15.22:0/564006319 learned_addr learned my addr 172.21.15.22:0/564006319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-03-09T00:13:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe49affd640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c1003d0 0x7fe49c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.208+0000 7fe49b7fe640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c196870 0x7fe49c196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49a7fc640 1 -- 172.21.15.22:0/564006319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c1003d0 msgr2=0x7fe49c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49a7fc640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c1003d0 0x7fe49c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49a7fc640 1 -- 172.21.15.22:0/564006319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c196870 msgr2=0x7fe49c196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49a7fc640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c196870 0x7fe49c196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49a7fc640 1 -- 172.21.15.22:0/564006319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c197390 con 0x7fe49c1054a0 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49b7fe640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c196870 0x7fe49c196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T00:13:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49affd640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c1003d0 0x7fe49c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T00:13:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe49a7fc640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1054a0 0x7fe49c19cc00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d740 tx=0x7fe49000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe483fff640 1 -- 172.21.15.22:0/564006319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490017070 con 0x7fe49c1054a0 2024-03-09T00:13:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe483fff640 1 -- 172.21.15.22:0/564006319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe49000de50 con 0x7fe49c1054a0 2024-03-09T00:13:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c197680 con 0x7fe49c1054a0 2024-03-09T00:13:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe483fff640 1 -- 172.21.15.22:0/564006319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490004e00 con 0x7fe49c1054a0 2024-03-09T00:13:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.209+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c1a3880 con 0x7fe49c1054a0 2024-03-09T00:13:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.210+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c0680e0 con 0x7fe49c1054a0 2024-03-09T00:13:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.211+0000 7fe483fff640 1 -- 172.21.15.22:0/564006319 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 27) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe490004fa0 con 0x7fe49c1054a0 2024-03-09T00:13:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.212+0000 7fe483fff640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4840a2970 0x7fe4840a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T00:13:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.212+0000 7fe49affd640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4840a2970 0x7fe4840a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T00:13:20.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.212+0000 7fe483fff640 1 -- 172.21.15.22:0/564006319 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8697+0+0 (secure 0 0 0) 0x7fe4900cedb0 con 0x7fe49c1054a0 2024-03-09T00:13:20.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.212+0000 7fe49affd640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4840a2970 0x7fe4840a4e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe48c006010 tx=0x7fe48c009610 comp rx=0 tx=0).ready entity=mgr.15660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T00:13:20.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.215+0000 7fe483fff640 1 -- 172.21.15.22:0/564006319 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe49009ba60 con 0x7fe49c1054a0 2024-03-09T00:13:20.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:20 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/301917799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-09T00:13:20.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:20 smithi062 ceph-mon[36129]: osdmap e129: 12 total, 12 up, 12 in 2024-03-09T00:13:20.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:20 smithi062 ceph-mon[36129]: pgmap v601: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 318 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-09T00:13:20.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:20 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2873105977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:20.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:20 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/301917799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-09T00:13:20.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:20 smithi192 ceph-mon[35995]: osdmap e129: 12 total, 12 up, 12 in 2024-03-09T00:13:20.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:20 smithi192 ceph-mon[35995]: pgmap v601: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 318 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-09T00:13:20.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:20 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2873105977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:20.400+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe49c100d80 con 0x7fe49c1054a0 2024-03-09T00:13:20.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:20 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/301917799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-09T00:13:20.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:20 smithi022 ceph-mon[31894]: osdmap e129: 12 total, 12 up, 12 in 2024-03-09T00:13:20.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:20 smithi022 ceph-mon[31894]: pgmap v601: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 318 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-09T00:13:20.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:20 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2873105977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:21.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.064+0000 7fe483fff640 1 -- 172.21.15.22:0/564006319 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v131) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe49009b360 con 0x7fe49c1054a0 2024-03-09T00:13:21.062 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' removed 2024-03-09T00:13:21.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.068+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4840a2970 msgr2=0x7fe4840a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:21.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.068+0000 7fe4a1790640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4840a2970 0x7fe4840a4e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe48c006010 tx=0x7fe48c009610 comp rx=0 tx=0).stop 2024-03-09T00:13:21.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.068+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1054a0 msgr2=0x7fe49c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T00:13:21.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.068+0000 7fe4a1790640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1054a0 0x7fe49c19cc00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d740 tx=0x7fe49000dc00 comp rx=0 tx=0).stop 2024-03-09T00:13:21.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 shutdown_connections 2024-03-09T00:13:21.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:6800/2739225064,v1:172.21.15.22:6801/2739225064] conn(0x7fe4840a2970 0x7fe4840a4e30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:21.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c196870 0x7fe49c196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:21.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe49c1054a0 0x7fe49c19cc00 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:21.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 --2- 172.21.15.22:0/564006319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c1003d0 0x7fe49c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T00:13:21.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 >> 172.21.15.22:0/564006319 conn(0x7fe49c0772f0 msgr2=0x7fe49c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T00:13:21.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 shutdown_connections 2024-03-09T00:13:21.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-03-09T00:13:21.069+0000 7fe4a1790640 1 -- 172.21.15.22:0/564006319 wait complete. 2024-03-09T00:13:21.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-09T00:13:21.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-09T00:13:21.301 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-09T00:13:21.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:21 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/2873105977' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-09T00:13:21.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:21 smithi062 ceph-mon[36129]: osdmap e130: 12 total, 12 up, 12 in 2024-03-09T00:13:21.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:21 smithi062 ceph-mon[36129]: from='client.? 172.21.15.22:0/564006319' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:21.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:21 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/2873105977' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-09T00:13:21.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:21 smithi192 ceph-mon[35995]: osdmap e130: 12 total, 12 up, 12 in 2024-03-09T00:13:21.368 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:21 smithi192 ceph-mon[35995]: from='client.? 172.21.15.22:0/564006319' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:21.379 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-09T00:13:21.379 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-09T00:13:21.396 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-09T00:13:21.413 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-09T00:13:21.430 INFO:tasks.cephadm:Stopping all daemons... 2024-03-09T00:13:21.430 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-09T00:13:21.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a 2024-03-09T00:13:21.463 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:21 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/2873105977' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-09T00:13:21.463 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:21 smithi022 ceph-mon[31894]: osdmap e130: 12 total, 12 up, 12 in 2024-03-09T00:13:21.463 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:21 smithi022 ceph-mon[31894]: from='client.? 172.21.15.22:0/564006319' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-09T00:13:21.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:21 smithi022 systemd[1]: Stopping Ceph mon.a for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:22.189 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a[31890]: 2024-03-09T00:13:21.926+0000 7ff356d7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:22.189 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a[31890]: 2024-03-09T00:13:21.926+0000 7ff356d7f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-09T00:13:22.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:22 smithi022 podman[112708]: 2024-03-09 00:13:22.192354775 +0000 UTC m=+0.484041566 container died 6c0c3ed4de50525bd6007c4b9cf987d119edf28a0ff2400f4590441d21e0515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:13:22.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:22 smithi022 podman[112708]: 2024-03-09 00:13:22.607796187 +0000 UTC m=+0.899482974 container cleanup 6c0c3ed4de50525bd6007c4b9cf987d119edf28a0ff2400f4590441d21e0515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD) 2024-03-09T00:13:22.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Mar 09 00:13:22 smithi022 bash[112708]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-a 2024-03-09T00:13:23.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.a.service' 2024-03-09T00:13:23.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:23.186 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-09T00:13:23.187 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-09T00:13:23.187 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.b 2024-03-09T00:13:23.556 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:23 smithi062 systemd[1]: Stopping Ceph mon.b for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:23.862 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-b[36125]: 2024-03-09T00:13:23.558+0000 7fb7e6c3d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-b[36125]: 2024-03-09T00:13:23.558+0000 7fb7e6c3d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-09T00:13:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:23 smithi062 podman[75022]: 2024-03-09 00:13:23.784116242 +0000 UTC m=+0.367992108 container died 18fb8795a3875eb2e06a95d576156ce87425adfe3c3ab8e6b09feb229e8aac24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-09T00:13:24.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:24 smithi062 podman[75022]: 2024-03-09 00:13:24.131750438 +0000 UTC m=+0.715626294 container cleanup 18fb8795a3875eb2e06a95d576156ce87425adfe3c3ab8e6b09feb229e8aac24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-09T00:13:24.362 INFO:journalctl@ceph.mon.b.smithi062.stdout:Mar 09 00:13:24 smithi062 bash[75022]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-b 2024-03-09T00:13:24.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.b.service' 2024-03-09T00:13:24.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:24.757 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-09T00:13:24.757 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-09T00:13:24.757 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.c 2024-03-09T00:13:25.118 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:24 smithi192 systemd[1]: Stopping Ceph mon.c for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:25.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-c[35991]: 2024-03-09T00:13:25.120+0000 7fdd2ce03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:25.382 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-c[35991]: 2024-03-09T00:13:25.120+0000 7fdd2ce03700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-09T00:13:25.383 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:25 smithi192 podman[74955]: 2024-03-09 00:13:25.230186014 +0000 UTC m=+0.222496622 container died 00f2a2c5a4e4e21aa84e3642407c5b0221f2264d31779f7b53898932f05a8706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-09T00:13:25.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:25 smithi192 podman[74955]: 2024-03-09 00:13:25.66357376 +0000 UTC m=+0.655884362 container cleanup 00f2a2c5a4e4e21aa84e3642407c5b0221f2264d31779f7b53898932f05a8706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:13:25.883 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 09 00:13:25 smithi192 bash[74955]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mon-c 2024-03-09T00:13:26.132 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mon.c.service' 2024-03-09T00:13:26.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:26.181 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-09T00:13:26.181 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-09T00:13:26.181 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.x 2024-03-09T00:13:26.486 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 09 00:13:26 smithi022 systemd[1]: Stopping Ceph mgr.x for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:26.853 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 09 00:13:26 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x[32123]: 2024-03-09T00:13:26.528+0000 7f1216ace700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:26.853 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 09 00:13:26 smithi022 podman[112822]: 2024-03-09 00:13:26.560941895 +0000 UTC m=+0.180260675 container died a551c7f49a0d85e91168d17d13c8a58552d118edc947d693e1188bbf37d03628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-09T00:13:27.236 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 09 00:13:26 smithi022 podman[112822]: 2024-03-09 00:13:26.957693967 +0000 UTC m=+0.577012789 container cleanup a551c7f49a0d85e91168d17d13c8a58552d118edc947d693e1188bbf37d03628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, ceph=True) 2024-03-09T00:13:27.236 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Mar 09 00:13:26 smithi022 bash[112822]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-x 2024-03-09T00:13:27.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.x.service' 2024-03-09T00:13:27.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:27.565 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-09T00:13:27.566 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-09T00:13:27.566 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.y 2024-03-09T00:13:27.862 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:27 smithi062 systemd[1]: Stopping Ceph mgr.y for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:28.238 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:27 smithi062 podman[75144]: 2024-03-09 00:13:27.933413344 +0000 UTC m=+0.144654595 container died 6914d87866581bc40829ae48c7214f31d713977b0fea3636e20a6cbaf7612357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-09T00:13:28.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:28 smithi062 podman[75144]: 2024-03-09 00:13:28.44908198 +0000 UTC m=+0.660323241 container cleanup 6914d87866581bc40829ae48c7214f31d713977b0fea3636e20a6cbaf7612357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:13:28.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:28 smithi062 bash[75144]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y 2024-03-09T00:13:28.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.y.service' 2024-03-09T00:13:28.956 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:28 smithi062 podman[75158]: 2024-03-09 00:13:28.701126778 +0000 UTC m=+0.763726937 container remove 6914d87866581bc40829ae48c7214f31d713977b0fea3636e20a6cbaf7612357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:13:28.957 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:28 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-09T00:13:28.957 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:28 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-09T00:13:28.957 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:28 smithi062 systemd[1]: Stopped Ceph mgr.y for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-09T00:13:28.957 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 09 00:13:28 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.y.service: Consumed 22.769s CPU time. 2024-03-09T00:13:28.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:28.970 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-09T00:13:28.970 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-09T00:13:28.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.z 2024-03-09T00:13:29.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 09 00:13:29 smithi192 systemd[1]: Stopping Ceph mgr.z for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:29.632 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 09 00:13:29 smithi192 podman[75075]: 2024-03-09 00:13:29.322567549 +0000 UTC m=+0.153589577 container died 44bdf83cbba1eb68f6a986d2c1516afbe64e02f8ad0820f5d812c051db11f0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-09T00:13:30.133 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 09 00:13:29 smithi192 podman[75075]: 2024-03-09 00:13:29.777740377 +0000 UTC m=+0.608762399 container cleanup 44bdf83cbba1eb68f6a986d2c1516afbe64e02f8ad0820f5d812c051db11f0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-09T00:13:30.133 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 09 00:13:29 smithi192 bash[75075]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mgr-z 2024-03-09T00:13:30.249 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mgr.z.service' 2024-03-09T00:13:30.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:30.288 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-09T00:13:30.289 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-09T00:13:30.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.0 2024-03-09T00:13:30.624 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:30 smithi022 systemd[1]: Stopping Ceph osd.0 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:30.986 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0[46570]: 2024-03-09T00:13:30.626+0000 7efd1475b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:30.986 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0[46570]: 2024-03-09T00:13:30.626+0000 7efd1475b700 -1 osd.0 131 *** Got signal Terminated *** 2024-03-09T00:13:30.986 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0[46570]: 2024-03-09T00:13:30.626+0000 7efd1475b700 -1 osd.0 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:13:36.567 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:36 smithi022 podman[112941]: 2024-03-09 00:13:36.269136792 +0000 UTC m=+5.782458783 container died 5cf56178b529723e7ea5aea3b0feaac68d8157f9d150ac518aa9b42103d14f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD) 2024-03-09T00:13:36.951 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:36 smithi022 podman[112941]: 2024-03-09 00:13:36.644289057 +0000 UTC m=+6.157611052 container cleanup 5cf56178b529723e7ea5aea3b0feaac68d8157f9d150ac518aa9b42103d14f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:13:36.951 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:36 smithi022 bash[112941]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0 2024-03-09T00:13:37.236 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:36 smithi022 podman[112962]: 2024-03-09 00:13:36.954245002 +0000 UTC m=+0.681617545 container remove 5cf56178b529723e7ea5aea3b0feaac68d8157f9d150ac518aa9b42103d14f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, ceph=True) 2024-03-09T00:13:37.571 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:37 smithi022 podman[113059]: 2024-03-09 00:13:37.201887823 +0000 UTC m=+0.018004102 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:13:37.571 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:37 smithi022 podman[113059]: 2024-03-09 00:13:37.33030802 +0000 UTC m=+0.146424324 container create e1ac3c59ea9b272c3bf46a2bedc898686b8d580b15087c082c737240b3666863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-09T00:13:37.571 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:37 smithi022 podman[113059]: 2024-03-09 00:13:37.574414037 +0000 UTC m=+0.390530309 container init e1ac3c59ea9b272c3bf46a2bedc898686b8d580b15087c082c737240b3666863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-09T00:13:37.866 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:37 smithi022 podman[113059]: 2024-03-09 00:13:37.577739623 +0000 UTC m=+0.393855883 container start e1ac3c59ea9b272c3bf46a2bedc898686b8d580b15087c082c737240b3666863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-09T00:13:37.867 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:37 smithi022 podman[113059]: 2024-03-09 00:13:37.674518165 +0000 UTC m=+0.490634439 container attach e1ac3c59ea9b272c3bf46a2bedc898686b8d580b15087c082c737240b3666863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-09T00:13:38.216 INFO:journalctl@ceph.osd.0.smithi022.stdout:Mar 09 00:13:37 smithi022 podman[113059]: 2024-03-09 00:13:37.870423729 +0000 UTC m=+0.686539996 container died e1ac3c59ea9b272c3bf46a2bedc898686b8d580b15087c082c737240b3666863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:13:38.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.0.service' 2024-03-09T00:13:38.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:38.880 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-09T00:13:38.880 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-09T00:13:38.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.1 2024-03-09T00:13:39.113 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:38 smithi062 systemd[1]: Stopping Ceph osd.1 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:39.614 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1[40734]: 2024-03-09T00:13:39.346+0000 7f1145043700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:39.615 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1[40734]: 2024-03-09T00:13:39.346+0000 7f1145043700 -1 osd.1 131 *** Got signal Terminated *** 2024-03-09T00:13:39.615 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1[40734]: 2024-03-09T00:13:39.346+0000 7f1145043700 -1 osd.1 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:13:45.053 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:44 smithi062 podman[75281]: 2024-03-09 00:13:44.768814213 +0000 UTC m=+5.686139490 container died d4195be68f00ecbdd449fa02a6cd728a029a472695b3dc9c03c90a2a82ecc58f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:13:45.362 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:45 smithi062 podman[75281]: 2024-03-09 00:13:45.14662736 +0000 UTC m=+6.063952637 container cleanup d4195be68f00ecbdd449fa02a6cd728a029a472695b3dc9c03c90a2a82ecc58f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-09T00:13:45.363 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:45 smithi062 bash[75281]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1 2024-03-09T00:13:45.747 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:45 smithi062 podman[75294]: 2024-03-09 00:13:45.440175035 +0000 UTC m=+0.674302676 container remove d4195be68f00ecbdd449fa02a6cd728a029a472695b3dc9c03c90a2a82ecc58f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-03-09T00:13:46.112 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:45 smithi062 podman[75384]: 2024-03-09 00:13:45.653233041 +0000 UTC m=+0.019460178 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:13:46.443 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:46 smithi062 podman[75384]: 2024-03-09 00:13:46.175368697 +0000 UTC m=+0.541595820 container create f6df5dbc0c9dc4a8f13e5ba583bce945544e6873513f9ff6cd0fcd64182c55e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-09T00:13:46.818 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:46 smithi062 podman[75384]: 2024-03-09 00:13:46.446501626 +0000 UTC m=+0.812728757 container init f6df5dbc0c9dc4a8f13e5ba583bce945544e6873513f9ff6cd0fcd64182c55e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-09T00:13:46.818 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:46 smithi062 podman[75384]: 2024-03-09 00:13:46.450498725 +0000 UTC m=+0.816725854 container start f6df5dbc0c9dc4a8f13e5ba583bce945544e6873513f9ff6cd0fcd64182c55e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:13:46.818 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:46 smithi062 podman[75384]: 2024-03-09 00:13:46.537576562 +0000 UTC m=+0.903803707 container attach f6df5dbc0c9dc4a8f13e5ba583bce945544e6873513f9ff6cd0fcd64182c55e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-09T00:13:47.113 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 09 00:13:46 smithi062 podman[75384]: 2024-03-09 00:13:46.820796289 +0000 UTC m=+1.187023420 container died f6df5dbc0c9dc4a8f13e5ba583bce945544e6873513f9ff6cd0fcd64182c55e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:13:47.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.1.service' 2024-03-09T00:13:47.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:47.503 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-09T00:13:47.503 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-09T00:13:47.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.2 2024-03-09T00:13:47.829 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:47 smithi192 systemd[1]: Stopping Ceph osd.2 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:48.132 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2[41283]: 2024-03-09T00:13:47.831+0000 7fd3ba66c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:48.132 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2[41283]: 2024-03-09T00:13:47.831+0000 7fd3ba66c700 -1 osd.2 131 *** Got signal Terminated *** 2024-03-09T00:13:48.132 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2[41283]: 2024-03-09T00:13:47.831+0000 7fd3ba66c700 -1 osd.2 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:13:53.632 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:53 smithi192 podman[75209]: 2024-03-09 00:13:53.334072744 +0000 UTC m=+5.628262084 container died 8ca3932f492679a136ed89d3b4089284beb3b4234d4c2861d71df70425c2405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:13:54.022 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:53 smithi192 podman[75209]: 2024-03-09 00:13:53.777302003 +0000 UTC m=+6.071491353 container cleanup 8ca3932f492679a136ed89d3b4089284beb3b4234d4c2861d71df70425c2405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-09T00:13:54.023 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:53 smithi192 bash[75209]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2 2024-03-09T00:13:54.382 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:54 smithi192 podman[75226]: 2024-03-09 00:13:54.025942459 +0000 UTC m=+0.681386672 container remove 8ca3932f492679a136ed89d3b4089284beb3b4234d4c2861d71df70425c2405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:13:54.735 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:54 smithi192 podman[75313]: 2024-03-09 00:13:54.355420288 +0000 UTC m=+0.018036318 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:13:54.735 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:54 smithi192 podman[75313]: 2024-03-09 00:13:54.493839669 +0000 UTC m=+0.156455684 container create 2897baadaae838cf9b35639bffbb6b9b84cbf81df2d674342205f8a632a6ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:13:55.132 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:54 smithi192 podman[75313]: 2024-03-09 00:13:54.738207354 +0000 UTC m=+0.400823392 container init 2897baadaae838cf9b35639bffbb6b9b84cbf81df2d674342205f8a632a6ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-09T00:13:55.133 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:54 smithi192 podman[75313]: 2024-03-09 00:13:54.743186993 +0000 UTC m=+0.405803015 container start 2897baadaae838cf9b35639bffbb6b9b84cbf81df2d674342205f8a632a6ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-09T00:13:55.133 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:54 smithi192 podman[75313]: 2024-03-09 00:13:54.880240385 +0000 UTC m=+0.542856509 container attach 2897baadaae838cf9b35639bffbb6b9b84cbf81df2d674342205f8a632a6ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-09T00:13:55.739 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 09 00:13:55 smithi192 podman[75313]: 2024-03-09 00:13:55.464941478 +0000 UTC m=+1.127557493 container died 2897baadaae838cf9b35639bffbb6b9b84cbf81df2d674342205f8a632a6ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:13:56.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.2.service' 2024-03-09T00:13:56.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:13:56.166 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-09T00:13:56.166 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-09T00:13:56.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.3 2024-03-09T00:13:56.486 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:13:56 smithi022 systemd[1]: Stopping Ceph osd.3 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:13:56.801 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:13:56 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3[53373]: 2024-03-09T00:13:56.488+0000 7ff90c789700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:13:56.802 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:13:56 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3[53373]: 2024-03-09T00:13:56.488+0000 7ff90c789700 -1 osd.3 131 *** Got signal Terminated *** 2024-03-09T00:13:56.802 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:13:56 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3[53373]: 2024-03-09T00:13:56.488+0000 7ff90c789700 -1 osd.3 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:13:57.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:13:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:13:57.047+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:13:57.632 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:13:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:13:57.159+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:13:57.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:13:57 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:13:57.711+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:13:58.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:13:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:13:58.090+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:13:58.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:13:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:13:58.133+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:13:58.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:13:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:13:58.597+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:13:58.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:13:58 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:13:58.692+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:13:59.362 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:13:59 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:13:59.079+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:13:59.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:13:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:13:59.083+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:13:59.725 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:13:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:13:59.599+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:13:59.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:13:59 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:13:59.681+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:00.105 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:13:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:13:59.727+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:00.362 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:00 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:00.087+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:00.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:00.108+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:00.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:00 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:00.689+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:01.099 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:00.699+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:01.100 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:00.637+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:01.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:01.101+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:01.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:01 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:01.120+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:01.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:01 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:01.683+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:02.078 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:01.732+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:02.079 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:01.636+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:02.337 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:02 smithi022 podman[113164]: 2024-03-09 00:14:02.058482104 +0000 UTC m=+5.692918290 container died 33a88f6a2a0bd8f92c9508f70715f24eebab0cc66ffc467d9d37690160b0b7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-09T00:14:02.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:02.080+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:02.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:02 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:02.126+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:02.667 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:02 smithi022 podman[113164]: 2024-03-09 00:14:02.43105308 +0000 UTC m=+6.065489265 container cleanup 33a88f6a2a0bd8f92c9508f70715f24eebab0cc66ffc467d9d37690160b0b7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS) 2024-03-09T00:14:02.667 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:02 smithi022 bash[113164]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3 2024-03-09T00:14:02.986 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:02 smithi022 podman[113193]: 2024-03-09 00:14:02.690832204 +0000 UTC m=+0.628133229 container remove 33a88f6a2a0bd8f92c9508f70715f24eebab0cc66ffc467d9d37690160b0b7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:02.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:02 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:02.669+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:03.048 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:02.702+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:03.048 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:02.653+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:03.366 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:03 smithi022 podman[113287]: 2024-03-09 00:14:02.952358367 +0000 UTC m=+0.017719352 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:14:03.366 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:03 smithi022 podman[113287]: 2024-03-09 00:14:03.075233826 +0000 UTC m=+0.140594850 container create e5273ce27ae3ccda85c6201d2d3befc793ae3c17bf4fe375680f42521b0912ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-09T00:14:03.366 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:03 smithi022 podman[113287]: 2024-03-09 00:14:03.369575006 +0000 UTC m=+0.434935998 container init e5273ce27ae3ccda85c6201d2d3befc793ae3c17bf4fe375680f42521b0912ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-09T00:14:03.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:03.051+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:03.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:03 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:03.171+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:03.668 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:03 smithi022 podman[113287]: 2024-03-09 00:14:03.373079932 +0000 UTC m=+0.438440913 container start e5273ce27ae3ccda85c6201d2d3befc793ae3c17bf4fe375680f42521b0912ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-09T00:14:03.668 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:03 smithi022 podman[113287]: 2024-03-09 00:14:03.477827104 +0000 UTC m=+0.543188097 container attach e5273ce27ae3ccda85c6201d2d3befc793ae3c17bf4fe375680f42521b0912ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3-deactivate, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-09T00:14:03.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:03 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:03.672+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:03.987 INFO:journalctl@ceph.osd.3.smithi022.stdout:Mar 09 00:14:03 smithi022 podman[113287]: 2024-03-09 00:14:03.671767594 +0000 UTC m=+0.737128583 container died e5273ce27ae3ccda85c6201d2d3befc793ae3c17bf4fe375680f42521b0912ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-09T00:14:04.067 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:03.665+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:04.068 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:03.657+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:04.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.3.service' 2024-03-09T00:14:04.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:14:04.354 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-09T00:14:04.354 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-09T00:14:04.354 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.4 2024-03-09T00:14:04.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:04.070+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:04.409 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:04.160+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:04.675 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:04 smithi062 systemd[1]: Stopping Ceph osd.4 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:14:04.882 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:04.706+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:04.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:04.626+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:04.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:04 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:04.635+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:05.113 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-09T00:14:04.677+0000 7f87786d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:14:05.113 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-09T00:14:04.677+0000 7f87786d1700 -1 osd.4 131 *** Got signal Terminated *** 2024-03-09T00:14:05.113 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:04 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-09T00:14:04.677+0000 7f87786d1700 -1 osd.4 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:14:05.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:05.021+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:05.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:05 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:05.200+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:05.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:05 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:05.681+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:06.003 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:05.678+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:06.004 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:05.636+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:06.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:06.007+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:06.612 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:06 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:06.158+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:06.882 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:06.725+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:06.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:06.627+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:06.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:06 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:06.697+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:06.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:06 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:06.697+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:07.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:07 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:07.151+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:07.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:07 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:07.151+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:07.363 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:07 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-09T00:14:07.104+0000 7f8774cc5700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.568675+0000 front 2024-03-09T00:13:42.568713+0000 (oldest deadline 2024-03-09T00:14:06.668800+0000) 2024-03-09T00:14:07.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:07 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:07.023+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:07.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:07 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:07.614+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:07.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:07 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:07.682+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:07.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:07 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:07.682+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:07.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:07 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:07.685+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:07.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:07 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:07.685+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:08.363 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:08 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-09T00:14:08.063+0000 7f8774cc5700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.568675+0000 front 2024-03-09T00:13:42.568713+0000 (oldest deadline 2024-03-09T00:14:06.668800+0000) 2024-03-09T00:14:08.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:08 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:08.167+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:08.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:08 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:08.167+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:08.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:08 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:08.024+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:08.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:08 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:08.585+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:08.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:08 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:08.721+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:08.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:08 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:08.721+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:08.987 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:08 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:08.532+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:08.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:08 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:08.728+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:08.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:08 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:08.729+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:09.363 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:09 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4[47710]: 2024-03-09T00:14:09.060+0000 7f8774cc5700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.568675+0000 front 2024-03-09T00:13:42.568713+0000 (oldest deadline 2024-03-09T00:14:06.668800+0000) 2024-03-09T00:14:09.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:09 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:09.179+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:09.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:09 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:09.179+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:09.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:09 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:09.025+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:09.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:09 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:09.624+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:09.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:09 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:09.672+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:09.884 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:09 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:09.672+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:09.986 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:09 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:09.497+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:09.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:09 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:09.686+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:09.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:09 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:09.686+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:10.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:10 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:10.044+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:10.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:10 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:10.044+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:10.483 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:10 smithi062 podman[75491]: 2024-03-09 00:14:10.251894078 +0000 UTC m=+5.690605054 container died 95d2bdb2b314d1340acc7d7d35849f6bd4cc725e552816ffd6e7dd3611126799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-09T00:14:10.484 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:10 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:10.129+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:10.484 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:10 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:10.129+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:10.853 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:10 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:10.485+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:10.853 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:10 smithi062 podman[75491]: 2024-03-09 00:14:10.645964834 +0000 UTC m=+6.084675801 container cleanup 95d2bdb2b314d1340acc7d7d35849f6bd4cc725e552816ffd6e7dd3611126799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:14:10.854 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:10 smithi062 bash[75491]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4 2024-03-09T00:14:10.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:10 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:10.612+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:10.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:10 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:10.690+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:10.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:10 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:10.690+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:10.986 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:10.522+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:10.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:10.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:10.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:10 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:10.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:11.113 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:10 smithi062 podman[75509]: 2024-03-09 00:14:10.855808809 +0000 UTC m=+0.604777105 container remove 95d2bdb2b314d1340acc7d7d35849f6bd4cc725e552816ffd6e7dd3611126799 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:14:11.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:11 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:11.094+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:11.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:11 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:11.094+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:11.427 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:11 smithi062 podman[75598]: 2024-03-09 00:14:11.161121209 +0000 UTC m=+0.018115550 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:14:11.427 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:11 smithi062 podman[75598]: 2024-03-09 00:14:11.290304296 +0000 UTC m=+0.147298667 container create 064d9e4cc9d19578a89685757675554ba3f4c9f3f510c68ebbc6b0ba2aaad0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-09T00:14:11.427 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:11 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:11.161+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:11.428 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:11 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:11.161+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:11.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:11.477+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:11.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:11.648+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:11.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:11 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:11.648+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:11.833 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:11 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:11.517+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:11.834 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:11 smithi062 podman[75598]: 2024-03-09 00:14:11.534886845 +0000 UTC m=+0.391881170 container init 064d9e4cc9d19578a89685757675554ba3f4c9f3f510c68ebbc6b0ba2aaad0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-09T00:14:11.834 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:11 smithi062 podman[75598]: 2024-03-09 00:14:11.538101437 +0000 UTC m=+0.395095765 container start 064d9e4cc9d19578a89685757675554ba3f4c9f3f510c68ebbc6b0ba2aaad0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:14:11.834 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:11 smithi062 podman[75598]: 2024-03-09 00:14:11.626628944 +0000 UTC m=+0.483623278 container attach 064d9e4cc9d19578a89685757675554ba3f4c9f3f510c68ebbc6b0ba2aaad0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-09T00:14:11.834 INFO:journalctl@ceph.osd.4.smithi062.stdout:Mar 09 00:14:11 smithi062 podman[75598]: 2024-03-09 00:14:11.836698925 +0000 UTC m=+0.693693268 container died 064d9e4cc9d19578a89685757675554ba3f4c9f3f510c68ebbc6b0ba2aaad0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:12.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:11 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:11.638+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:12.133 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:11 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:11.650+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:12.133 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:11 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:11.650+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:12.489 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:12 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:12.153+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:12.489 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:12 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:12.153+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:12.611 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:12 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:12.135+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:12.611 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:12 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:12.135+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:12.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.4.service' 2024-03-09T00:14:12.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:14:12.681 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-09T00:14:12.681 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-09T00:14:12.682 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.5 2024-03-09T00:14:12.736 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:12 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:12.483+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:12.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:12 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:12.604+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:12.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:12 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:12.604+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:12.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:12 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:12.491+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:12.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:12 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:12.613+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:12.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:12 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:12.647+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:12.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:12 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:12.647+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:12.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:12 smithi192 systemd[1]: Stopping Ceph osd.5 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:14:13.382 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:13.138+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:13.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:13.138+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:13.383 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:13.003+0000 7f1bb4d87700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:14:13.383 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:13.003+0000 7f1bb4d87700 -1 osd.5 131 *** Got signal Terminated *** 2024-03-09T00:14:13.383 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:13.003+0000 7f1bb4d87700 -1 osd.5 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:14:13.482 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:13 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:13.185+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:13.482 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:13 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:13.185+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:13.736 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:13.470+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:13.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:13.561+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:13.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:13 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:13.561+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:13.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:13 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:13.484+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:13.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:13.565+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:13.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:13.610+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:13.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:13 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:13.610+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:14.470 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:14 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:14.194+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:14.470 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:14 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:14.194+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:14.470 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:14 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:14.470+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:14.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:14 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:14.151+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:14.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:14 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:14.151+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:14.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:14 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:14.151+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:14.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:14 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:14.584+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:14.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:14 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:14.628+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:14.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:14 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:14.628+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:14.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:14 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:14.628+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.649372+0000 front 2024-03-09T00:13:51.649549+0000 (oldest deadline 2024-03-09T00:14:13.949555+0000) 2024-03-09T00:14:14.987 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:14.514+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:14.987 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:14.514+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:14.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:14.564+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:14.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:14.564+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:15.489 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:15 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:15.221+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:15.489 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:15 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:15.221+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:15.583 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:15.195+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:15.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:15.195+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:15.584 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:15.195+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:15.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:15 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:15.491+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:15.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:15 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:15.491+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:15.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:15.585+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:15.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:15.585+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:15.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:15.600+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:15.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:15.600+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:15.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:15 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:15.600+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.649372+0000 front 2024-03-09T00:13:51.649549+0000 (oldest deadline 2024-03-09T00:14:13.949555+0000) 2024-03-09T00:14:15.987 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:15 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:15.525+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:15.987 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:15 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:15.525+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:15.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:15 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:15.520+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:15.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:15 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:15.520+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:15.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:15 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:15.520+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:16.612 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:16.229+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:16.612 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:16.229+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:16.612 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:16.229+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:16.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:16 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:16.472+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:16.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:16 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:16.472+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:16.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:16 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:16.258+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:16.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:16 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:16.258+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:16.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:16 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:16.258+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:16.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:16.478+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:16.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:16.478+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:16.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:16.526+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:16.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:16.526+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:16.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:16 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:16.526+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:16.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:16.614+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:16.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:16.614+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:16.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:16.633+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:16.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:16.633+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:16.884 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:16 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:16.633+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.649372+0000 front 2024-03-09T00:13:51.649549+0000 (oldest deadline 2024-03-09T00:14:13.949555+0000) 2024-03-09T00:14:17.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:17.463+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:17.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:17.463+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:17.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:17.236+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:17.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:17.236+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:17.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:17 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:17.236+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:17.625 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:17.207+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:17.625 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:17.207+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:17.625 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:17.207+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:17.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:17 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:17.459+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:17.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:17 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:17.459+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:17.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:17 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:17.538+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:17.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:17 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:17.538+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:17.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:17 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:17.538+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:17.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:17.630+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:17.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:17.630+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:17.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:17.627+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.346724+0000 front 2024-03-09T00:13:35.346683+0000 (oldest deadline 2024-03-09T00:13:58.846638+0000) 2024-03-09T00:14:17.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:17.627+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:42.347669+0000 front 2024-03-09T00:13:42.347728+0000 (oldest deadline 2024-03-09T00:14:07.047719+0000) 2024-03-09T00:14:17.883 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:17 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5[48086]: 2024-03-09T00:14:17.627+0000 7f1bb137b700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.649372+0000 front 2024-03-09T00:13:51.649549+0000 (oldest deadline 2024-03-09T00:14:13.949555+0000) 2024-03-09T00:14:18.521 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:18 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:18.200+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:18.521 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:18 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:18.200+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:18.521 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:18 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:18.200+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:18.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:18 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:18.478+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:18.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:18 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:18.478+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:18.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:18 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:18.270+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:18.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:18 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:18.270+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:18.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:18 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:18.270+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:18.785 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:18 smithi192 podman[75434]: 2024-03-09 00:14:18.523758178 +0000 UTC m=+5.640378694 container died e21f7cb1c6f2bc6e2a1bcf783fb97b0a4f0cdc3a393e775a3c39702bf7fde812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-09T00:14:18.785 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:18 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:18.667+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:18.786 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:18 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:18.667+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:18.987 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:18.504+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:18.987 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:18.504+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:18.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:18.570+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:18.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:18.570+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:18.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:18.570+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:19.085 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:18 smithi192 podman[75434]: 2024-03-09 00:14:18.853288474 +0000 UTC m=+5.969908976 container cleanup e21f7cb1c6f2bc6e2a1bcf783fb97b0a4f0cdc3a393e775a3c39702bf7fde812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-09T00:14:19.085 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:18 smithi192 bash[75434]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5 2024-03-09T00:14:19.382 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:19 smithi192 podman[75451]: 2024-03-09 00:14:19.087873434 +0000 UTC m=+0.567699176 container remove e21f7cb1c6f2bc6e2a1bcf783fb97b0a4f0cdc3a393e775a3c39702bf7fde812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:14:19.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:19 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:19.223+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:19.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:19 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:19.223+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:19.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:19 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:19.223+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:19.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:19 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:19.522+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:19.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:19 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:19.522+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:19.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:19 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:19.277+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:19.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:19 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:19.277+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:19.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:19 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:19.277+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:19.736 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:19 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:19.469+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:19.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:19 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:19.469+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:19.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:19 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:19.608+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:19.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:19 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:19.608+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:19.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:19 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:19.608+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:19.771 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:19 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:19.677+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:19.772 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:19 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:19.677+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:19.772 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:19 smithi192 podman[75539]: 2024-03-09 00:14:19.405731524 +0000 UTC m=+0.016768904 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:14:19.772 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:19 smithi192 podman[75539]: 2024-03-09 00:14:19.522465491 +0000 UTC m=+0.133502858 container create 41defc97de21b81905d4394cd243a73851acf6992bd454b29bdefa02630e155f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:14:20.063 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:19 smithi192 podman[75539]: 2024-03-09 00:14:19.774933961 +0000 UTC m=+0.385971340 container init 41defc97de21b81905d4394cd243a73851acf6992bd454b29bdefa02630e155f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-09T00:14:20.063 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:19 smithi192 podman[75539]: 2024-03-09 00:14:19.778083286 +0000 UTC m=+0.389120655 container start 41defc97de21b81905d4394cd243a73851acf6992bd454b29bdefa02630e155f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:14:20.063 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:19 smithi192 podman[75539]: 2024-03-09 00:14:19.891763416 +0000 UTC m=+0.502800801 container attach 41defc97de21b81905d4394cd243a73851acf6992bd454b29bdefa02630e155f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-09T00:14:20.383 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 09 00:14:20 smithi192 podman[75539]: 2024-03-09 00:14:20.066682712 +0000 UTC m=+0.677720098 container died 41defc97de21b81905d4394cd243a73851acf6992bd454b29bdefa02630e155f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:14:20.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:20 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:20.270+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:20.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:20 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:20.270+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:20.383 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:20 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:20.270+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:20.535 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:20 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:20.282+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:20.535 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:20 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:20.282+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:20.535 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:20 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:20.282+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:20.692 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:20 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:20.638+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:20.693 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:20 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:20.638+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:20.725 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.5.service' 2024-03-09T00:14:20.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:20 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:20.460+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:20.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:20 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:20.460+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:20.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:20 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:20.589+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:20.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:20 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:20.589+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:20.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:20 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:20.589+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:20.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:14:20.767 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-09T00:14:20.767 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-09T00:14:20.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.6 2024-03-09T00:14:20.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:20 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:20.537+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:20.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:20 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:20.537+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:21.181 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:20 smithi022 systemd[1]: Stopping Ceph osd.6 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:14:21.486 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:21.183+0000 7f02edb24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:14:21.487 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:21.183+0000 7f02edb24700 -1 osd.6 131 *** Got signal Terminated *** 2024-03-09T00:14:21.487 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:21.183+0000 7f02edb24700 -1 osd.6 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:14:21.487 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:21.418+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:21.487 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:21.418+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:21.546 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:21 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:21.269+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:21.547 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:21 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:21.269+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:21.547 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:21 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:21.269+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:21.594 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:21 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:21.304+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:21.594 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:21 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:21.304+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:21.594 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:21 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:21.304+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:21.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:21 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:21.549+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:21.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:21 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:21.549+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:21.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:21 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:21.595+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:21.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:21 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:21.595+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:21.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:21.614+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:21.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:21.614+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:21.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:21 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:21.614+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:22.534 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:22 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:22.253+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:22.535 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:22 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:22.253+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:22.535 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:22 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:22.253+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:22.535 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:22 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:22.253+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:22.633 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:22 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:22.584+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:22.633 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:22 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:22.584+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:22.633 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:22 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:22.338+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:22.633 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:22 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:22.338+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:22.634 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:22 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:22.338+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:22.736 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:22 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:22.460+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:22.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:22 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:22.460+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:22.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:22 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:22.604+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:22.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:22 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:22.604+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:22.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:22 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:22.604+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:22.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:22 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:22.537+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:22.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:22 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:22.537+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:22.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:22 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:22.537+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:23.550 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:23.248+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:23.550 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:23.248+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:23.550 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:23.248+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:23.551 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:23.248+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:23.593 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:23 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:23.320+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:23.593 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:23 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:23.320+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:23.593 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:23 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:23.320+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:23.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:23.422+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:23.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:23.422+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:23.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:23.422+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:57.303548+0000 front 2024-03-09T00:13:57.303588+0000 (oldest deadline 2024-03-09T00:14:22.504224+0000) 2024-03-09T00:14:23.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:23.569+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:23.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:23.569+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:23.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:23.569+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:23.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:23.552+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:23.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:23.552+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:23.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:23 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:23.552+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:23.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:23 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:23.595+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:23.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:23 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:23.595+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:23.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:23 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:23.595+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:24.566 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:24 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:24.199+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:24.566 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:24 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:24.199+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:24.567 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:24 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:24.199+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:24.567 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:24 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:24.199+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:24.633 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:24 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:24.349+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:24.633 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:24 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:24.349+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:24.633 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:24 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:24.349+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:24.633 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:24 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:24.596+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:24.633 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:24 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:24.596+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:24.634 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:24 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:24.596+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:24.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:24 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:24.403+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:24.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:24 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:24.403+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:24.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:24 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:24.403+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:57.303548+0000 front 2024-03-09T00:13:57.303588+0000 (oldest deadline 2024-03-09T00:14:22.504224+0000) 2024-03-09T00:14:24.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:24 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:24.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:24.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:24 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:24.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:24.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:24 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:24.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:24.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:24 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:24.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:24.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:24 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:24.568+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:24.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:24 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:24.568+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:24.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:24 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:24.568+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:25.587 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:25.196+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:25.588 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:25.196+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:25.588 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:25.196+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:25.588 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:25.196+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:25.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:25.380+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:44.000598+0000 front 2024-03-09T00:13:44.000638+0000 (oldest deadline 2024-03-09T00:14:08.100659+0000) 2024-03-09T00:14:25.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:25.380+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.502692+0000 front 2024-03-09T00:13:51.502773+0000 (oldest deadline 2024-03-09T00:14:14.402657+0000) 2024-03-09T00:14:25.737 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6[60074]: 2024-03-09T00:14:25.380+0000 7f02e6110700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:57.303548+0000 front 2024-03-09T00:13:57.303588+0000 (oldest deadline 2024-03-09T00:14:22.504224+0000) 2024-03-09T00:14:25.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:25.596+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:25.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:25.596+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:25.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:25.596+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:25.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:25 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:25.596+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:25.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:25.590+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:25.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:25.590+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:25.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:25 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:25.590+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:25.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:25.386+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:25.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:25.386+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:25.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:25.386+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:25.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:25.609+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:25.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:25.609+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:25.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:25 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:25.609+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:26.608 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:26.171+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:26.608 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:26.171+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:26.608 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:26.171+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:26.609 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:26.171+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:26.840 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:26 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:26.583+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:26.840 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:26 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:26.583+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:26.840 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:26 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:26.583+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:26.840 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:26 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:26.583+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:26.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:26.610+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:26.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:26.610+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:26.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:26 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:26.610+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:26.882 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:26.636+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:26.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:26.636+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:26.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:26.636+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:26.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:26.424+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:26.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:26.424+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:26.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:26.424+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:26.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:26 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:26.424+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:27.106 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:26 smithi022 podman[113391]: 2024-03-09 00:14:26.842754826 +0000 UTC m=+5.874699564 container died a07cabf19e78fa275c231b2c96ef82fcc47092918aa39988d1eb405d89212927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:14:27.473 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:27 smithi022 podman[113391]: 2024-03-09 00:14:27.233656792 +0000 UTC m=+6.265601537 container cleanup a07cabf19e78fa275c231b2c96ef82fcc47092918aa39988d1eb405d89212927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0) 2024-03-09T00:14:27.474 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:27 smithi022 bash[113391]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6 2024-03-09T00:14:27.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:27.153+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:27.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:27.153+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:27.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:27.153+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:27.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:27.153+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:27.736 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:27 smithi022 podman[113409]: 2024-03-09 00:14:27.476760927 +0000 UTC m=+0.634765911 container remove a07cabf19e78fa275c231b2c96ef82fcc47092918aa39988d1eb405d89212927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:27.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:27.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:27.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:27.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:27.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:27.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:27.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:27 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:27.592+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:27.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:27.635+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:27.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:27.635+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:27.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:27.635+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:27.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:27.451+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:27.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:27.451+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:27.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:27.451+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:27.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:27 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:27.451+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:28.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:27.634+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:28.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:27.634+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:28.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:27.634+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:28.219 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:27 smithi022 podman[113508]: 2024-03-09 00:14:27.714108084 +0000 UTC m=+0.017894147 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:14:28.219 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:27 smithi022 podman[113508]: 2024-03-09 00:14:27.855598875 +0000 UTC m=+0.159384944 container create 941396662c44f5b954692480f91aec41ceaf8c45100dbb5467ac7805aaed9978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307) 2024-03-09T00:14:28.487 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:28 smithi022 podman[113508]: 2024-03-09 00:14:28.222173718 +0000 UTC m=+0.525959809 container init 941396662c44f5b954692480f91aec41ceaf8c45100dbb5467ac7805aaed9978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:28.487 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:28 smithi022 podman[113508]: 2024-03-09 00:14:28.22580674 +0000 UTC m=+0.529592809 container start 941396662c44f5b954692480f91aec41ceaf8c45100dbb5467ac7805aaed9978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True) 2024-03-09T00:14:28.487 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:28 smithi022 podman[113508]: 2024-03-09 00:14:28.330290693 +0000 UTC m=+0.634076768 container attach 941396662c44f5b954692480f91aec41ceaf8c45100dbb5467ac7805aaed9978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:14:28.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:28.155+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:28.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:28.155+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:28.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:28.155+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:28.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:28.155+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:28.880 INFO:journalctl@ceph.osd.6.smithi022.stdout:Mar 09 00:14:28 smithi022 podman[113508]: 2024-03-09 00:14:28.523397558 +0000 UTC m=+0.827183616 container died 941396662c44f5b954692480f91aec41ceaf8c45100dbb5467ac7805aaed9978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-09T00:14:28.881 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:28 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:28.624+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:28.881 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:28 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:28.624+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:28.881 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:28 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:28.624+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:28.881 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:28 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:28.624+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:28.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:28.657+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:28.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:28.657+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:28.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:28.657+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:28.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:28.433+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:28.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:28.433+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:28.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:28.433+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:28.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:28 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:28.433+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:29.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:28.616+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:29.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:28.616+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:29.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:28 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:28.616+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:29.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.6.service' 2024-03-09T00:14:29.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:14:29.258 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-09T00:14:29.259 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-09T00:14:29.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.7 2024-03-09T00:14:29.572 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:29 smithi062 systemd[1]: Stopping Ceph osd.7 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:14:29.573 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:29.150+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:29.573 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:29.150+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:29.573 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:29.150+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:29.573 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:29.150+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:29.784 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:29.623+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:29.784 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:29.623+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:29.784 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:29.623+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:29.785 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:29.438+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:29.785 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:29.438+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:29.785 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:29.438+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:29.785 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:29 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:29.438+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:29.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:29.574+0000 7f0445811700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:14:29.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:29.574+0000 7f0445811700 -1 osd.7 131 *** Got signal Terminated *** 2024-03-09T00:14:29.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:29.574+0000 7f0445811700 -1 osd.7 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:14:29.864 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:29.613+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:29.864 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:29.613+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:29.864 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:29 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:29.613+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:29.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:29.612+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:29.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:29.612+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:29.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:29.612+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:29.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:29 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:29.612+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:30.583 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:30.156+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:30.584 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:30.156+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:30.584 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:30.156+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:30.584 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:30.156+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:30.798 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:30.599+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:30.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:30.599+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:30.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:30.599+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:30.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:30.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:30.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:30.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:30.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:30.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:30.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:30 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:30.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:30.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:30.580+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:30.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:30.580+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:30.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:30.580+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:30.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:30.580+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.674536+0000 front 2024-03-09T00:14:08.674572+0000 (oldest deadline 2024-03-09T00:14:30.374747+0000) 2024-03-09T00:14:30.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:30.601+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:30.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:30.601+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:30.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:30.601+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:30.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:30 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:30.601+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:31.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:31.206+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:31.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:31.206+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:31.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:31.206+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:31.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:31.206+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:31.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:31.614+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:31.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:31.614+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:31.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:31.614+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:31.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:31.410+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:31.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:31.410+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:31.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:31.410+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:31.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:31 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:31.410+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:31.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:31 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:31.579+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:31.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:31 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:31.579+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:31.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:31 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:31.579+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:31.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:31 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:31.579+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:32.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:31.628+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:32.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:31.628+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:32.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:31.628+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:32.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:31 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:31.628+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.674536+0000 front 2024-03-09T00:14:08.674572+0000 (oldest deadline 2024-03-09T00:14:30.374747+0000) 2024-03-09T00:14:32.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:32.223+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:32.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:32.223+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:32.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:32.223+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:32.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:32.223+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:32.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:32.223+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:32.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:32.613+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:32.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:32.613+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:32.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:32.613+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:32.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:32.389+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:32.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:32.389+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:32.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:32.389+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:32.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:32 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:32.389+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:32.986 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:32.609+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:32.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:32.609+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:32.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:32.609+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:32.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:32.609+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:32.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:32 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:32.609+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:33.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:32.675+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:33.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:32.675+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:33.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:32.675+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:33.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:32 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:32.675+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.674536+0000 front 2024-03-09T00:14:08.674572+0000 (oldest deadline 2024-03-09T00:14:30.374747+0000) 2024-03-09T00:14:33.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:33.216+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:33.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:33.216+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:33.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:33.216+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:33.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:33.216+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:33.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:33.216+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:33.633 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:33.564+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:33.634 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:33.564+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:33.634 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:33.564+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:33.634 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:33.564+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:33.634 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:33.355+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:33.634 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:33.355+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:33.635 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:33.355+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:33.635 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:33 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:33.355+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:33.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:33.633+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:33.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:33.633+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:33.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:33.633+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:33.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:33.633+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:33.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:33 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:33.633+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:34.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:33.654+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:39.668525+0000 front 2024-03-09T00:13:39.668567+0000 (oldest deadline 2024-03-09T00:14:09.669186+0000) 2024-03-09T00:14:34.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:33.654+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.671346+0000 front 2024-03-09T00:13:50.670912+0000 (oldest deadline 2024-03-09T00:14:15.271626+0000) 2024-03-09T00:14:34.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:33.654+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:56.972681+0000 front 2024-03-09T00:13:56.972706+0000 (oldest deadline 2024-03-09T00:14:22.272750+0000) 2024-03-09T00:14:34.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:33 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7[54293]: 2024-03-09T00:14:33.654+0000 7f0441e05700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.674536+0000 front 2024-03-09T00:14:08.674572+0000 (oldest deadline 2024-03-09T00:14:30.374747+0000) 2024-03-09T00:14:34.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:34 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:34.233+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:34.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:34 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:34.233+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:34.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:34 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:34.233+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:34.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:34 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:34.233+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:34.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:34 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:34.233+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:34.633 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:34.554+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:34.633 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:34.554+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:34.634 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:34.554+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:34.634 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:34.554+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:34.634 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:34.357+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:34.634 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:34.357+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:34.634 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:34.357+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:34.634 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:34 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:34.357+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:34.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:34 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:34.653+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:34.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:34 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:34.653+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:34.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:34 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:34.653+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:34.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:34 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:34.653+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:34.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:34 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:34.653+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:35.347 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:35 smithi062 podman[75706]: 2024-03-09 00:14:35.056270006 +0000 UTC m=+5.598287405 container died fa4d34717fc834f9ad3663695b39be11a3cb69f02ad29fc4939b846858cac6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:35.348 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:35 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:35.250+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:35.348 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:35 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:35.250+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:35.348 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:35 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:35.250+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:35.349 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:35 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:35.250+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:35.349 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:35 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:35.250+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:35.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:35 smithi062 podman[75706]: 2024-03-09 00:14:35.574983686 +0000 UTC m=+6.117001077 container cleanup fa4d34717fc834f9ad3663695b39be11a3cb69f02ad29fc4939b846858cac6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:14:35.613 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:35 smithi062 bash[75706]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7 2024-03-09T00:14:35.886 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:35.575+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:35.887 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:35.575+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:35.887 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:35.575+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:35.887 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:35.575+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:35.888 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:35.386+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:35.888 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:35.386+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:35.888 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:35.386+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:35.888 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:35 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:35.386+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:35.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:35 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:35.625+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:35.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:35 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:35.625+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:35.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:35 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:35.625+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:35.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:35 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:35.625+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:35.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:35 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:35.625+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:36.113 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:35 smithi062 podman[75721]: 2024-03-09 00:14:35.834622833 +0000 UTC m=+0.780301461 container remove fa4d34717fc834f9ad3663695b39be11a3cb69f02ad29fc4939b846858cac6c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:36.577 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:36 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:36.291+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:36.577 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:36 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:36.291+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:36.577 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:36 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:36.291+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:36.577 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:36 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:36.291+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:36.578 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:36 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:36.291+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:36.578 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:36 smithi062 podman[75813]: 2024-03-09 00:14:36.157252259 +0000 UTC m=+0.018640177 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:14:36.578 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:36 smithi062 podman[75813]: 2024-03-09 00:14:36.268929662 +0000 UTC m=+0.130317587 container create 6387ad23c980e9f93f84b26e0f26b41d858e72e1c66dbdc567dad706191ac779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:14:36.578 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:36 smithi062 podman[75813]: 2024-03-09 00:14:36.488120684 +0000 UTC m=+0.349508618 container init 6387ad23c980e9f93f84b26e0f26b41d858e72e1c66dbdc567dad706191ac779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:14:36.578 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:36 smithi062 podman[75813]: 2024-03-09 00:14:36.491488539 +0000 UTC m=+0.352876465 container start 6387ad23c980e9f93f84b26e0f26b41d858e72e1c66dbdc567dad706191ac779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:36.578 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:36 smithi062 podman[75813]: 2024-03-09 00:14:36.579653687 +0000 UTC m=+0.441041639 container attach 6387ad23c980e9f93f84b26e0f26b41d858e72e1c66dbdc567dad706191ac779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-09T00:14:36.863 INFO:journalctl@ceph.osd.7.smithi062.stdout:Mar 09 00:14:36 smithi062 podman[75813]: 2024-03-09 00:14:36.79323889 +0000 UTC m=+0.654626818 container died 6387ad23c980e9f93f84b26e0f26b41d858e72e1c66dbdc567dad706191ac779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:14:36.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:36.541+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:36.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:36.541+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:36.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:36.541+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:36.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:36.541+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:36.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:36.392+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:36.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:36.392+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:36.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:36.392+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:36.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:36 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:36.392+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:36.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:36.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:36.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:36.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:36.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:36.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:36.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:36.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:36.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:36 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:36.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:37.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:37.316+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:37.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:37.316+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:37.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:37.316+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:37.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:37.316+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:37.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:37 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:37.316+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:37.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.7.service' 2024-03-09T00:14:37.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:14:37.463 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-09T00:14:37.463 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-09T00:14:37.464 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.8 2024-03-09T00:14:37.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:37 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:37.650+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:37.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:37 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:37.650+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:37.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:37 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:37.650+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:37.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:37 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:37.650+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:37.738 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:37 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:37.650+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:37.798 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:37.552+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:37.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:37.552+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:37.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:37.552+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:37.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:37.552+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:37.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:37.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:37.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:37.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:37.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:37.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:37.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:37.431+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:37.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 systemd[1]: Stopping Ceph osd.8 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:14:37.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:37.800+0000 7fcc05c63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:14:37.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:37.800+0000 7fcc05c63700 -1 osd.8 131 *** Got signal Terminated *** 2024-03-09T00:14:37.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:37 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:37.800+0000 7fcc05c63700 -1 osd.8 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:14:38.604 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:38 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:38.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:38.605 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:38 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:38.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:38.605 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:38 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:38.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:38.605 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:38 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:38.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:38.605 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:38 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:38.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:38.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:38.401+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:38.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:38.401+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:38.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:38.401+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:38.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:38.401+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:38.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:38.597+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:38.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:38.597+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:38.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:38.597+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:38.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:38.597+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:38.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:38 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:38.597+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:38.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:38.667+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:38.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:38.667+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:38.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:38.667+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:38.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:38.667+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:38.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:38 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:38.667+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:39.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:39.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:39.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:39.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:39.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:39.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:39 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:39.308+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:39.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:39.578+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:39.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:39.578+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:39.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:39.578+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:39.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:39.578+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:39.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:39.578+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:39.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:39.403+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:39.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:39.403+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:39.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:39.404+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:39.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:39 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:39.404+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:39.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:39.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:39.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:39.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:39.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:39.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:39.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:39.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:39.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:39.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:39.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:39 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:39.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:40.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:40 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:40.287+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:40 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:40.287+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:40 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:40.287+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:40 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:40.287+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:40 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:40.287+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:40 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:40.287+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:40.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:40.626+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:40.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:40.626+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:40.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:40.626+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:40.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:40.626+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:40.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:40.626+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:40.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:40.430+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:40.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:40.430+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:40.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:40.430+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:40.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:40.430+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:40.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:40.430+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:14.313915+0000 front 2024-03-09T00:14:14.313952+0000 (oldest deadline 2024-03-09T00:14:39.614085+0000) 2024-03-09T00:14:40.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:40.712+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:40.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:40.712+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:40.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:40.712+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:40.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:40.712+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:40.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:40.712+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:40.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:40 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:40.712+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:41.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:41 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:41.299+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:41.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:41 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:41.299+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:41.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:41 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:41.299+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:41.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:41 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:41.299+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:41.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:41 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:41.299+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:41.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:41 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:41.299+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:41.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:41.414+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:41.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:41.414+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:41.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:41.414+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:41.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:41.414+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:41.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:41.414+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:14.313915+0000 front 2024-03-09T00:14:14.313952+0000 (oldest deadline 2024-03-09T00:14:39.614085+0000) 2024-03-09T00:14:41.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:41.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:41.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:41.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:41.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:41.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:41.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:41.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:41.885 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:41 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:41.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:41.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:41.709+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:41.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:41.709+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:41.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:41.709+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:41.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:41.709+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:41.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:41.709+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:41.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:41 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:41.709+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:42.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:42 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:42.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:42.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:42 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:42.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:42.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:42 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:42.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:42.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:42 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:42.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:42.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:42 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:42.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:42.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:42 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:42.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:42.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:42.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:42.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:42.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:42.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:42.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:42.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:42.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:42.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:42.646+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:42.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:42.411+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:31.508629+0000 front 2024-03-09T00:13:31.508507+0000 (oldest deadline 2024-03-09T00:13:56.808604+0000) 2024-03-09T00:14:42.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:42.411+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.810622+0000 front 2024-03-09T00:13:43.810787+0000 (oldest deadline 2024-03-09T00:14:09.710598+0000) 2024-03-09T00:14:42.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:42.411+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:49.711172+0000 front 2024-03-09T00:13:49.711093+0000 (oldest deadline 2024-03-09T00:14:13.811212+0000) 2024-03-09T00:14:42.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:42.411+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:00.212723+0000 front 2024-03-09T00:14:00.212645+0000 (oldest deadline 2024-03-09T00:14:25.512539+0000) 2024-03-09T00:14:42.884 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:42 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8[55130]: 2024-03-09T00:14:42.411+0000 7fcc02257700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:14.313915+0000 front 2024-03-09T00:14:14.313952+0000 (oldest deadline 2024-03-09T00:14:39.614085+0000) 2024-03-09T00:14:42.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:42.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:42.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:42.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:42.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:42.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:42.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:42.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:42.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:42.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:42.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:42 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:42.668+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:43.546 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:43 smithi192 podman[75656]: 2024-03-09 00:14:43.265582761 +0000 UTC m=+5.605214171 container died 4ae33a334892058061c076a5890ff6d243a000d03171ae04a2257bcd69332309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-09T00:14:43.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:43 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:43.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:43.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:43 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:43.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:43.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:43 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:43.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:43.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:43 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:43.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:43.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:43 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:43.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:43.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:43 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:43.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:43.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:43 smithi192 podman[75656]: 2024-03-09 00:14:43.648236447 +0000 UTC m=+5.987867855 container cleanup 4ae33a334892058061c076a5890ff6d243a000d03171ae04a2257bcd69332309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-09T00:14:43.883 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:43 smithi192 bash[75656]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8 2024-03-09T00:14:43.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:43 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:43.665+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:43.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:43 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:43.665+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:43.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:43 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:43.665+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:43.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:43 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:43.665+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:43.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:43 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:43.665+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:43.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:43.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:43.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:43.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:43.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:43.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:43.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:43.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:43.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:43.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:43.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:43 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:43.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:44.234 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:43 smithi192 podman[75673]: 2024-03-09 00:14:43.941320382 +0000 UTC m=+0.672696966 container remove 4ae33a334892058061c076a5890ff6d243a000d03171ae04a2257bcd69332309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-09T00:14:44.500 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:44 smithi192 podman[75763]: 2024-03-09 00:14:44.140378917 +0000 UTC m=+0.017530829 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:14:44.500 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:44 smithi192 podman[75763]: 2024-03-09 00:14:44.27548747 +0000 UTC m=+0.152639373 container create 5c8de04fac9e741fe65bd5630253dae9946c15755a420f3f1b56035d2acf0c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2) 2024-03-09T00:14:44.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:44 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:44.333+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:44.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:44 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:44.333+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:44.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:44 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:44.333+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:44.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:44 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:44.333+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:44.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:44 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:44.333+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:44.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:44 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:44.333+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:44.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:44 smithi192 podman[75763]: 2024-03-09 00:14:44.503147904 +0000 UTC m=+0.380299820 container init 5c8de04fac9e741fe65bd5630253dae9946c15755a420f3f1b56035d2acf0c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:14:44.799 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:44 smithi192 podman[75763]: 2024-03-09 00:14:44.50672652 +0000 UTC m=+0.383878423 container start 5c8de04fac9e741fe65bd5630253dae9946c15755a420f3f1b56035d2acf0c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-09T00:14:44.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:44 smithi192 podman[75763]: 2024-03-09 00:14:44.603064362 +0000 UTC m=+0.480216279 container attach 5c8de04fac9e741fe65bd5630253dae9946c15755a420f3f1b56035d2acf0c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-09T00:14:44.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:44 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:44.677+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:44.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:44 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:44.677+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:44.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:44 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:44.677+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:44.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:44 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:44.677+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:44.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:44 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:44.677+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:44.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:44 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:44.615+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:44.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:44 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:44.615+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:44.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:44 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:44.615+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:44.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:44 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:44.615+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:44.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:44 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:44.615+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:44.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:44 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:44.615+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:45.133 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 09 00:14:44 smithi192 podman[75763]: 2024-03-09 00:14:44.802666346 +0000 UTC m=+0.679818255 container died 5c8de04fac9e741fe65bd5630253dae9946c15755a420f3f1b56035d2acf0c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True) 2024-03-09T00:14:45.482 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.8.service' 2024-03-09T00:14:45.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:14:45.524 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-09T00:14:45.524 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-09T00:14:45.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.9 2024-03-09T00:14:45.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:45 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:45.314+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:45.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:45 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:45.314+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:45.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:45 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:45.314+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:45.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:45 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:45.314+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:45.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:45 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:45.314+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:45.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:45 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:45.314+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:45.841 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.568+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:45.842 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.568+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:45.842 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.568+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:45.842 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.568+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:45.842 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.568+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:45.842 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.568+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:45.842 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 systemd[1]: Stopping Ceph osd.9 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:14:45.842 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.843+0000 7fc657ba1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:14:45.843 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.843+0000 7fc657ba1700 -1 osd.9 131 *** Got signal Terminated *** 2024-03-09T00:14:45.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:45 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:45.675+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:45.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:45 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:45.675+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:45.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:45 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:45.675+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:45.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:45 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:45.675+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:45.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:45 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:45.675+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:46.237 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:45 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:45.843+0000 7fc657ba1700 -1 osd.9 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:14:46.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:46.329+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:46.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:46.329+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:46.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:46.329+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:46.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:46.329+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:46.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:46.329+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:46.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:46 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:46.329+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:46.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:46 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:46.627+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:46.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:46 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:46.627+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:46.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:46 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:46.627+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:46.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:46 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:46.627+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:46.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:46 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:46.627+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:46.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:46 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:46.581+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:46.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:46 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:46.581+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:46.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:46 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:46.581+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:46.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:46 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:46.581+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:46.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:46 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:46.581+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:46.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:46 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:46.581+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:47.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:47 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:47.296+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:47.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:47 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:47.296+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:47.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:47 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:47.296+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:47.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:47 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:47.296+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:47.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:47 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:47.296+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:47.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:47 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:47.296+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:47.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:47.634+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:47.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:47.634+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:47.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:47.634+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:47.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:47.634+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:47.883 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:47.634+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:47.884 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:47 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:47.634+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:47.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:47.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:47.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:47.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:47.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:47.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:47.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:47.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:47.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:47.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:47.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:47 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:47.628+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:48.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:48 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:48.286+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:48.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:48 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:48.286+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:48.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:48 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:48.286+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:48.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:48 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:48.286+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:48.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:48 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:48.286+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:48.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:48 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:48.286+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:48.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:48 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:48.642+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:48.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:48 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:48.642+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:48.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:48 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:48.642+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:48.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:48 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:48.642+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:48.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:48 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:48.642+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:48.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:48 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:48.642+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:49.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:48 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:48.681+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:49.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:48 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:48.681+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:49.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:48 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:48.681+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:49.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:48 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:48.681+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:49.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:48 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:48.681+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:49.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:48 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:48.681+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:49.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:49 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:49.237+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:49.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:49 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:49.237+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:49.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:49 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:49.237+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:49.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:49 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:49.237+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:49.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:49 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:49.237+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:49.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:49 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:49.237+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:49.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:49 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:49.237+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:49.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:49 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:49.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:49.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:49 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:49.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:49.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:49 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:49.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:49.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:49 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:49.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:49.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:49 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:49.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:49.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:49 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:49.663+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:50.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:49 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:49.701+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:50.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:49 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:49.701+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:50.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:49 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:49.701+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:50.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:49 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:49.701+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:50.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:49 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:49.701+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:50.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:49 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:49.701+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:50.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:50 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:50.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:50.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:50 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:50.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:50.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:50 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:50.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:50.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:50 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:50.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:50.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:50 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:50.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:50.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:50 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:50.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:50.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:50 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:50.284+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:50.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:50 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:50.671+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.040048+0000 front 2024-03-09T00:13:35.040077+0000 (oldest deadline 2024-03-09T00:13:57.340236+0000) 2024-03-09T00:14:50.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:50 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:50.671+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.141837+0000 front 2024-03-09T00:13:43.141563+0000 (oldest deadline 2024-03-09T00:14:06.641632+0000) 2024-03-09T00:14:50.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:50 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:50.671+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:50.742906+0000 front 2024-03-09T00:13:50.742821+0000 (oldest deadline 2024-03-09T00:14:15.442990+0000) 2024-03-09T00:14:50.987 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:50 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:50.671+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:14:01.343996+0000 front 2024-03-09T00:14:01.343964+0000 (oldest deadline 2024-03-09T00:14:23.644103+0000) 2024-03-09T00:14:50.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:50 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:50.671+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.045543+0000 front 2024-03-09T00:14:07.045459+0000 (oldest deadline 2024-03-09T00:14:32.345507+0000) 2024-03-09T00:14:50.988 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:50 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9[66848]: 2024-03-09T00:14:50.671+0000 7fc654195700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.947210+0000 front 2024-03-09T00:14:13.947097+0000 (oldest deadline 2024-03-09T00:14:39.247128+0000) 2024-03-09T00:14:51.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:50.704+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:51.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:50.704+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:51.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:50.704+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:51.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:50.704+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:51.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:50.704+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:51.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:50.704+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:51.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:51.274+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:51.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:51.274+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:51.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:51.274+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:51.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:51.274+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:51.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:51.274+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:51.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:51.274+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:51.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:51 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:51.274+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:51.737 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:51 smithi022 podman[113616]: 2024-03-09 00:14:51.470670048 +0000 UTC m=+5.746979680 container died 50eee3980a2c18afb19722997716e2a490ef4727b42a778682666e0eeff1e63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-09T00:14:52.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:51 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:51.714+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:52.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:51 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:51.714+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:52.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:51 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:51.714+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:52.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:51 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:51.714+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:52.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:51 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:51.714+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:52.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:51 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:51.714+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:52.217 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:51 smithi022 podman[113616]: 2024-03-09 00:14:51.943849059 +0000 UTC m=+6.220158690 container cleanup 50eee3980a2c18afb19722997716e2a490ef4727b42a778682666e0eeff1e63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=) 2024-03-09T00:14:52.217 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:51 smithi022 bash[113616]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9 2024-03-09T00:14:52.487 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:52 smithi022 podman[113633]: 2024-03-09 00:14:52.220007708 +0000 UTC m=+0.748360150 container remove 50eee3980a2c18afb19722997716e2a490ef4727b42a778682666e0eeff1e63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-03-09T00:14:52.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:52 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:52.301+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:52.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:52 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:52.301+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:52.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:52 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:52.301+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:52.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:52 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:52.301+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:52.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:52 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:52.301+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:52.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:52 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:52.301+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:52.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:52 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:52.301+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:52.862 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:52 smithi022 podman[113724]: 2024-03-09 00:14:52.466283538 +0000 UTC m=+0.017518827 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:14:52.862 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:52 smithi022 podman[113724]: 2024-03-09 00:14:52.60431182 +0000 UTC m=+0.155547102 container create e3c9a23cd48889bd5da9ee1537036d9799951e18668be425e95b97a8ce501ad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-09T00:14:52.862 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:52 smithi022 podman[113724]: 2024-03-09 00:14:52.798449788 +0000 UTC m=+0.349685070 container init e3c9a23cd48889bd5da9ee1537036d9799951e18668be425e95b97a8ce501ad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:14:52.863 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:52 smithi022 podman[113724]: 2024-03-09 00:14:52.801499425 +0000 UTC m=+0.352734716 container start e3c9a23cd48889bd5da9ee1537036d9799951e18668be425e95b97a8ce501ad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-09T00:14:52.863 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:52 smithi022 podman[113724]: 2024-03-09 00:14:52.865043179 +0000 UTC m=+0.416278479 container attach e3c9a23cd48889bd5da9ee1537036d9799951e18668be425e95b97a8ce501ad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:14:53.121 INFO:journalctl@ceph.osd.9.smithi022.stdout:Mar 09 00:14:53 smithi022 podman[113724]: 2024-03-09 00:14:53.096290279 +0000 UTC m=+0.647525572 container died e3c9a23cd48889bd5da9ee1537036d9799951e18668be425e95b97a8ce501ad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:14:53.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:52 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:52.722+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:53.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:52 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:52.722+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:53.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:52 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:52.722+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:53.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:52 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:52.722+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:53.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:52 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:52.722+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:53.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:52 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:52.722+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:53.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:53.343+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:53.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:53.343+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:53.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:53.343+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:53.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:53.343+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:53.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:53.343+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:53.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:53.343+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:53.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:53.343+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:53.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.9.service' 2024-03-09T00:14:53.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:14:53.900 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-09T00:14:53.900 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-09T00:14:53.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.10 2024-03-09T00:14:54.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:53 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:53.755+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:54.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:53 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:53.755+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:54.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:53 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:53.755+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:54.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:53 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:53.755+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:54.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:53 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:53.755+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:54.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:53 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:53.755+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:54.234 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:53 smithi062 systemd[1]: Stopping Ceph osd.10 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:14:54.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.236+0000 7f88cc7ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:14:54.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.236+0000 7f88cc7ff700 -1 osd.10 131 *** Got signal Terminated *** 2024-03-09T00:14:54.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.236+0000 7f88cc7ff700 -1 osd.10 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:14:54.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:54.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:54.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:54.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:54.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:54.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:54.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:54 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:54.323+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:55.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:54.775+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:55.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:54.775+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:55.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:54.775+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:55.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:54.775+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:55.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:54.775+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:55.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:54.775+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:55.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:55 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:55.360+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:55.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:55 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:55.360+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:55.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:55 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:55.360+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:55.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:55 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:55.360+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:55.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:55 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:55.360+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:55.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:55 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:55.360+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:55.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:55 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:55.360+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:56.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:55 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:55.736+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:56.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:55 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:55.736+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:56.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:55 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:55.736+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:56.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:55 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:55.736+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:56.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:55 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:55.736+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:56.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:55 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:55.736+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:56.863 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:56 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:56.389+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:56.863 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:56 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:56.389+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:56.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:56 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:56.389+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:56.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:56 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:56.389+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:56.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:56 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:56.389+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:56.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:56 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:56.390+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:56.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:56 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:56.390+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:57.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:56 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:56.737+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:57.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:56 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:56.737+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:57.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:56 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:56.737+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:57.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:56 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:56.737+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:57.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:56 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:56.737+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:57.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:56 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:56.737+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:57.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:57.341+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:57.341+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:57.341+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:57.341+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:57.341+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:57.341+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:57 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:57.341+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:58.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:57.747+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:58.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:57.747+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:58.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:57.747+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:58.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:57.747+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:58.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:57.747+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:58.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:57 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:57.747+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:58.863 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:58.368+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:35.152717+0000 front 2024-03-09T00:13:35.152736+0000 (oldest deadline 2024-03-09T00:13:56.852774+0000) 2024-03-09T00:14:58.863 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:58.368+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-03-09T00:13:43.555923+0000 front 2024-03-09T00:13:43.555956+0000 (oldest deadline 2024-03-09T00:14:07.056063+0000) 2024-03-09T00:14:58.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:58.368+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:52.357298+0000 front 2024-03-09T00:13:52.357313+0000 (oldest deadline 2024-03-09T00:14:15.257341+0000) 2024-03-09T00:14:58.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:58.368+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.958411+0000 front 2024-03-09T00:13:59.958630+0000 (oldest deadline 2024-03-09T00:14:22.159038+0000) 2024-03-09T00:14:58.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:58.368+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:07.261492+0000 front 2024-03-09T00:14:07.261450+0000 (oldest deadline 2024-03-09T00:14:31.862282+0000) 2024-03-09T00:14:58.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:58.368+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:16.464177+0000 front 2024-03-09T00:14:16.464156+0000 (oldest deadline 2024-03-09T00:14:38.764209+0000) 2024-03-09T00:14:58.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:58 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10[60936]: 2024-03-09T00:14:58.368+0000 7f88c8df3700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:25.765939+0000 front 2024-03-09T00:14:25.765960+0000 (oldest deadline 2024-03-09T00:14:48.665782+0000) 2024-03-09T00:14:59.034 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:58.781+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:14:59.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:58.781+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:14:59.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:58.781+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:14:59.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:58.781+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:14:59.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:58.781+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:14:59.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:58.781+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:14:59.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:58 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:58.781+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:14:59.999 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:14:59 smithi062 podman[75929]: 2024-03-09 00:14:59.690684205 +0000 UTC m=+5.579574192 container died 2973e1683e539bc6173e513941b1be9c3cc3e74ac121fed696b9fb7471df7a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-09T00:15:00.000 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 podman[75929]: 2024-03-09 00:15:00.002604471 +0000 UTC m=+5.891494458 container cleanup 2973e1683e539bc6173e513941b1be9c3cc3e74ac121fed696b9fb7471df7a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-09T00:15:00.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:59.757+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:00.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:59.757+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:00.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:59.757+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:00.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:59.757+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:00.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:59.757+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:00.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:59.757+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:00.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:14:59 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:14:59.757+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:00.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 bash[75929]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10 2024-03-09T00:15:00.363 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 podman[75946]: 2024-03-09 00:15:00.203931653 +0000 UTC m=+0.510696524 container remove 2973e1683e539bc6173e513941b1be9c3cc3e74ac121fed696b9fb7471df7a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:15:00.888 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 podman[76038]: 2024-03-09 00:15:00.521022381 +0000 UTC m=+0.018303149 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:15:00.888 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 podman[76038]: 2024-03-09 00:15:00.621425901 +0000 UTC m=+0.118706650 container create 04ff3bbde3f15fcd1a9beebc741cf68db1865011979d6584f6d3fb270466dfec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-09T00:15:00.888 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 podman[76038]: 2024-03-09 00:15:00.890719457 +0000 UTC m=+0.388000226 container init 04ff3bbde3f15fcd1a9beebc741cf68db1865011979d6584f6d3fb270466dfec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-09T00:15:01.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:00.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:01.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:00.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:01.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:00.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:01.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:00.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:01.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:00.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:01.135 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:00.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:01.135 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:00 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:00.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:01.193 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 podman[76038]: 2024-03-09 00:15:00.893966349 +0000 UTC m=+0.391247094 container start 04ff3bbde3f15fcd1a9beebc741cf68db1865011979d6584f6d3fb270466dfec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-09T00:15:01.193 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:00 smithi062 podman[76038]: 2024-03-09 00:15:00.973903333 +0000 UTC m=+0.471184104 container attach 04ff3bbde3f15fcd1a9beebc741cf68db1865011979d6584f6d3fb270466dfec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-09T00:15:01.482 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:01 smithi062 podman[76038]: 2024-03-09 00:15:01.196249736 +0000 UTC m=+0.693530515 container died 04ff3bbde3f15fcd1a9beebc741cf68db1865011979d6584f6d3fb270466dfec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:15:01.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.10.service' 2024-03-09T00:15:01.739 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:01 smithi062 podman[76064]: 2024-03-09 00:15:01.693805269 +0000 UTC m=+0.492479882 container remove 04ff3bbde3f15fcd1a9beebc741cf68db1865011979d6584f6d3fb270466dfec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-09T00:15:01.739 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:01 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.10.service: Deactivated successfully. 2024-03-09T00:15:01.740 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:01 smithi062 systemd[1]: Stopped Ceph osd.10 for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-09T00:15:01.740 INFO:journalctl@ceph.osd.10.smithi062.stdout:Mar 09 00:15:01 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.10.service: Consumed 6min 25.892s CPU time. 2024-03-09T00:15:01.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:01.755 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-09T00:15:01.756 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-09T00:15:01.756 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.11 2024-03-09T00:15:02.091 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:01.716+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:02.091 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:01.716+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:02.091 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:01.716+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:02.091 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:01.716+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:02.092 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:01.716+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:02.092 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:01.716+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:02.092 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:01.716+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:02.092 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:01 smithi192 systemd[1]: Stopping Ceph osd.11 for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:02.383 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.092+0000 7f374ac93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:02.383 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.092+0000 7f374ac93700 -1 osd.11 131 *** Got signal Terminated *** 2024-03-09T00:15:02.383 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.092+0000 7f374ac93700 -1 osd.11 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-09T00:15:03.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.730+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:03.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.730+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:03.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.730+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:03.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.730+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:03.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.730+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:03.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.730+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:03.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:02 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:02.730+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:04.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:03.753+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:04.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:03.753+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:04.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:03.753+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:04.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:03.753+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:04.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:03.753+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:04.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:03.753+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:04.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:03 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:03.753+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:05.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:05.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:05.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:05.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:05.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:05.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:05.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:05.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:04 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:04.738+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-03-09T00:14:40.227554+0000 front 2024-03-09T00:14:40.227528+0000 (oldest deadline 2024-03-09T00:15:04.327524+0000) 2024-03-09T00:15:06.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:06.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:06.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:06.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:06.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:06.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:06.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:06.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:05 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:05.765+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-03-09T00:14:40.227554+0000 front 2024-03-09T00:14:40.227528+0000 (oldest deadline 2024-03-09T00:15:04.327524+0000) 2024-03-09T00:15:07.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2024-03-09T00:13:32.416735+0000 front 2024-03-09T00:13:32.416803+0000 (oldest deadline 2024-03-09T00:13:57.716811+0000) 2024-03-09T00:15:07.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-09T00:13:51.119732+0000 front 2024-03-09T00:13:51.119759+0000 (oldest deadline 2024-03-09T00:14:14.619898+0000) 2024-03-09T00:15:07.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2024-03-09T00:13:59.722583+0000 front 2024-03-09T00:13:59.722456+0000 (oldest deadline 2024-03-09T00:14:23.222459+0000) 2024-03-09T00:15:07.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-03-09T00:14:08.523622+0000 front 2024-03-09T00:14:08.523632+0000 (oldest deadline 2024-03-09T00:14:33.223735+0000) 2024-03-09T00:15:07.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-09T00:14:13.224267+0000 front 2024-03-09T00:14:13.224200+0000 (oldest deadline 2024-03-09T00:14:38.524316+0000) 2024-03-09T00:15:07.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2024-03-09T00:14:22.625405+0000 front 2024-03-09T00:14:22.625429+0000 (oldest deadline 2024-03-09T00:14:47.325544+0000) 2024-03-09T00:15:07.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-03-09T00:14:33.226705+0000 front 2024-03-09T00:14:33.226720+0000 (oldest deadline 2024-03-09T00:14:57.926708+0000) 2024-03-09T00:15:07.134 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:06 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11[62039]: 2024-03-09T00:15:06.760+0000 7f3747287700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-03-09T00:14:40.227554+0000 front 2024-03-09T00:14:40.227528+0000 (oldest deadline 2024-03-09T00:15:04.327524+0000) 2024-03-09T00:15:07.873 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:07 smithi192 podman[75880]: 2024-03-09 00:15:07.567330958 +0000 UTC m=+5.615441709 container died 25e450962a9cf2df68a70032732f359792b521f6497677c051b0e43b9c785814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-09T00:15:07.873 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:07 smithi192 podman[75880]: 2024-03-09 00:15:07.87563022 +0000 UTC m=+5.923740964 container cleanup 25e450962a9cf2df68a70032732f359792b521f6497677c051b0e43b9c785814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-03-09T00:15:08.132 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:07 smithi192 bash[75880]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11 2024-03-09T00:15:08.383 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:08 smithi192 podman[75899]: 2024-03-09 00:15:08.135230547 +0000 UTC m=+0.562489414 container remove 25e450962a9cf2df68a70032732f359792b521f6497677c051b0e43b9c785814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2) 2024-03-09T00:15:08.727 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:08 smithi192 podman[75990]: 2024-03-09 00:15:08.337832676 +0000 UTC m=+0.017584613 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-09T00:15:08.728 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:08 smithi192 podman[75990]: 2024-03-09 00:15:08.469340414 +0000 UTC m=+0.149092332 container create 108f3aa32ac4421589551ce43194ec6636a4f0aab4263ee8d00fb2b8d6e64d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:15:08.728 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:08 smithi192 podman[75990]: 2024-03-09 00:15:08.663255631 +0000 UTC m=+0.343007568 container init 108f3aa32ac4421589551ce43194ec6636a4f0aab4263ee8d00fb2b8d6e64d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11-deactivate, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-09T00:15:08.728 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:08 smithi192 podman[75990]: 2024-03-09 00:15:08.666509787 +0000 UTC m=+0.346261711 container start 108f3aa32ac4421589551ce43194ec6636a4f0aab4263ee8d00fb2b8d6e64d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-03-09T00:15:09.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:08 smithi192 podman[75990]: 2024-03-09 00:15:08.729684362 +0000 UTC m=+0.409436299 container attach 108f3aa32ac4421589551ce43194ec6636a4f0aab4263ee8d00fb2b8d6e64d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, ceph=True) 2024-03-09T00:15:09.133 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 09 00:15:08 smithi192 podman[75990]: 2024-03-09 00:15:08.957722429 +0000 UTC m=+0.637474362 container died 108f3aa32ac4421589551ce43194ec6636a4f0aab4263ee8d00fb2b8d6e64d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:15:09.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@osd.11.service' 2024-03-09T00:15:09.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:09.560 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-09T00:15:09.560 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-09T00:15:09.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.a 2024-03-09T00:15:09.935 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 09 00:15:09 smithi022 systemd[1]: Stopping Ceph mds.a for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:10.238 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 09 00:15:09 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a[73231]: 2024-03-09T00:15:09.936+0000 7f4813fef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:10.238 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 09 00:15:09 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a[73231]: 2024-03-09T00:15:09.936+0000 7f4813fef700 -1 mds.a *** got signal Terminated *** 2024-03-09T00:15:13.487 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 09 00:15:13 smithi022 podman[113847]: 2024-03-09 00:15:13.137522004 +0000 UTC m=+3.377609580 container died 8957338e3b2fbf3460a9ae78e95ae4469f0c13b3f7a9d30e8112a04d4c14f2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:15:13.877 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 09 00:15:13 smithi022 podman[113847]: 2024-03-09 00:15:13.628428158 +0000 UTC m=+3.868515730 container cleanup 8957338e3b2fbf3460a9ae78e95ae4469f0c13b3f7a9d30e8112a04d4c14f2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-09T00:15:13.877 INFO:journalctl@ceph.mds.a.smithi022.stdout:Mar 09 00:15:13 smithi022 bash[113847]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-a 2024-03-09T00:15:13.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.a.service' 2024-03-09T00:15:14.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:14.036 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-09T00:15:14.036 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-09T00:15:14.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.d 2024-03-09T00:15:14.237 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 09 00:15:14 smithi022 systemd[1]: Stopping Ceph mds.d for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:14.737 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 09 00:15:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-d[72277]: 2024-03-09T00:15:14.365+0000 7f1cc9a40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:14.737 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 09 00:15:14 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-d[72277]: 2024-03-09T00:15:14.365+0000 7f1cc9a40700 -1 mds.d *** got signal Terminated *** 2024-03-09T00:15:17.530 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 09 00:15:17 smithi022 podman[113968]: 2024-03-09 00:15:17.260521115 +0000 UTC m=+3.000767625 container died 859152d61f581c2f3b00ed421357eec502d316e7c9b9fc8c7f16c050beeb08ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-09T00:15:17.897 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 09 00:15:17 smithi022 podman[113968]: 2024-03-09 00:15:17.6151304 +0000 UTC m=+3.355376911 container cleanup 859152d61f581c2f3b00ed421357eec502d316e7c9b9fc8c7f16c050beeb08ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0) 2024-03-09T00:15:17.897 INFO:journalctl@ceph.mds.d.smithi022.stdout:Mar 09 00:15:17 smithi022 bash[113968]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-d 2024-03-09T00:15:18.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.d.service' 2024-03-09T00:15:18.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:18.069 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-09T00:15:18.069 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-09T00:15:18.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.g 2024-03-09T00:15:18.487 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 09 00:15:18 smithi022 systemd[1]: Stopping Ceph mds.g for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:18.487 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 09 00:15:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g[72883]: 2024-03-09T00:15:18.401+0000 7faef8034700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:18.488 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 09 00:15:18 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g[72883]: 2024-03-09T00:15:18.401+0000 7faef8034700 -1 mds.g *** got signal Terminated *** 2024-03-09T00:15:22.675 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 09 00:15:22 smithi022 podman[114089]: 2024-03-09 00:15:22.409740894 +0000 UTC m=+4.117202962 container died 56d084ddba189ea5ccdc032e8914c4dd80ff2499d4c1e6107eb7c346188fca92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-09T00:15:22.987 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 09 00:15:22 smithi022 podman[114089]: 2024-03-09 00:15:22.768486918 +0000 UTC m=+4.475948994 container cleanup 56d084ddba189ea5ccdc032e8914c4dd80ff2499d4c1e6107eb7c346188fca92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-03-09T00:15:22.987 INFO:journalctl@ceph.mds.g.smithi022.stdout:Mar 09 00:15:22 smithi022 bash[114089]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-g 2024-03-09T00:15:23.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.g.service' 2024-03-09T00:15:23.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:23.184 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-09T00:15:23.184 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-09T00:15:23.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.j 2024-03-09T00:15:23.737 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 09 00:15:23 smithi022 systemd[1]: Stopping Ceph mds.j for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:23.737 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 09 00:15:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j[71887]: 2024-03-09T00:15:23.504+0000 7f732ae23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:23.738 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 09 00:15:23 smithi022 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j[71887]: 2024-03-09T00:15:23.504+0000 7f732ae23700 -1 mds.j *** got signal Terminated *** 2024-03-09T00:15:26.662 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 09 00:15:26 smithi022 podman[114210]: 2024-03-09 00:15:26.377447385 +0000 UTC m=+2.977329109 container died 962b59335e90d2f7906ca3e5e693e6183659aaeaef57b8d1061976377a6db86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0) 2024-03-09T00:15:26.987 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 09 00:15:26 smithi022 podman[114210]: 2024-03-09 00:15:26.738469703 +0000 UTC m=+3.338351427 container cleanup 962b59335e90d2f7906ca3e5e693e6183659aaeaef57b8d1061976377a6db86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-09T00:15:26.987 INFO:journalctl@ceph.mds.j.smithi022.stdout:Mar 09 00:15:26 smithi022 bash[114210]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-j 2024-03-09T00:15:27.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.j.service' 2024-03-09T00:15:27.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:27.195 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-09T00:15:27.195 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-09T00:15:27.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.b 2024-03-09T00:15:27.518 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:15:27 smithi062 systemd[1]: Stopping Ceph mds.b for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:27.863 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:15:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b[66615]: 2024-03-09T00:15:27.519+0000 7ffad375a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:27.864 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:15:27 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b[66615]: 2024-03-09T00:15:27.519+0000 7ffad375a700 -1 mds.b *** got signal Terminated *** 2024-03-09T00:15:29.434 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:15:29 smithi062 podman[76144]: 2024-03-09 00:15:29.172971555 +0000 UTC m=+1.757398906 container died e0f4eae37122ade1691fac39b70c41e6d435863c6f11e7c19abe566993251058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-09T00:15:29.818 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:15:29 smithi062 podman[76144]: 2024-03-09 00:15:29.536315669 +0000 UTC m=+2.120743020 container cleanup e0f4eae37122ade1691fac39b70c41e6d435863c6f11e7c19abe566993251058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:15:29.818 INFO:journalctl@ceph.mds.b.smithi062.stdout:Mar 09 00:15:29 smithi062 bash[76144]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-b 2024-03-09T00:15:29.949 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.b.service' 2024-03-09T00:15:29.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:29.991 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-09T00:15:29.992 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-09T00:15:29.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.e 2024-03-09T00:15:30.363 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:15:30 smithi062 systemd[1]: Stopping Ceph mds.e for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:30.364 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:15:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e[65998]: 2024-03-09T00:15:30.331+0000 7fdfe5ed9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:30.364 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:15:30 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e[65998]: 2024-03-09T00:15:30.331+0000 7fdfe5ed9700 -1 mds.e *** got signal Terminated *** 2024-03-09T00:15:33.413 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:15:33 smithi062 podman[76276]: 2024-03-09 00:15:33.164610052 +0000 UTC m=+2.938241317 container died 898377f6b379ccf14c6f5d26b267b0708986cd2e65937b7b397d38ad64795d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-09T00:15:33.413 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:15:33 smithi062 podman[76276]: 2024-03-09 00:15:33.415342256 +0000 UTC m=+3.188973481 container cleanup 898377f6b379ccf14c6f5d26b267b0708986cd2e65937b7b397d38ad64795d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-09T00:15:33.823 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:15:33 smithi062 bash[76276]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e 2024-03-09T00:15:33.823 INFO:journalctl@ceph.mds.e.smithi062.stdout:Mar 09 00:15:33 smithi062 podman[76290]: 2024-03-09 00:15:33.625306247 +0000 UTC m=+0.453957411 container remove 898377f6b379ccf14c6f5d26b267b0708986cd2e65937b7b397d38ad64795d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD) 2024-03-09T00:15:33.838 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.e.service' 2024-03-09T00:15:33.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:33.878 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-09T00:15:33.879 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-09T00:15:33.879 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.h 2024-03-09T00:15:34.113 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:15:33 smithi062 systemd[1]: Stopping Ceph mds.h for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:34.614 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:15:34 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: 2024-03-09T00:15:34.193+0000 7f3601e20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:34.614 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:15:34 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h[70091]: 2024-03-09T00:15:34.193+0000 7f3601e20700 -1 mds.h *** got signal Terminated *** 2024-03-09T00:15:37.433 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:15:37 smithi062 podman[76397]: 2024-03-09 00:15:37.169300459 +0000 UTC m=+3.062850937 container died cb149f6106b746f619e02354969149b60a35442c50d6161c03010bdff4768b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-09T00:15:37.433 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:15:37 smithi062 podman[76397]: 2024-03-09 00:15:37.435795446 +0000 UTC m=+3.329345918 container cleanup cb149f6106b746f619e02354969149b60a35442c50d6161c03010bdff4768b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:15:37.701 INFO:journalctl@ceph.mds.h.smithi062.stdout:Mar 09 00:15:37 smithi062 bash[76397]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-h 2024-03-09T00:15:37.881 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.h.service' 2024-03-09T00:15:37.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:37.921 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-09T00:15:37.921 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-09T00:15:37.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.k 2024-03-09T00:15:38.254 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:38 smithi062 systemd[1]: Stopping Ceph mds.k for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:38.613 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:38 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k[65679]: 2024-03-09T00:15:38.255+0000 7f06508c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:38.614 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:38 smithi062 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k[65679]: 2024-03-09T00:15:38.255+0000 7f06508c6700 -1 mds.k *** got signal Terminated *** 2024-03-09T00:15:39.613 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:39 smithi062 podman[76519]: 2024-03-09 00:15:39.361616841 +0000 UTC m=+1.215806852 container died d0630ecc7c86da24f20a87c2905be96aab8cb4e53c3729d3c1c046b018513d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:15:39.913 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:39 smithi062 podman[76519]: 2024-03-09 00:15:39.723493109 +0000 UTC m=+1.577683104 container cleanup d0630ecc7c86da24f20a87c2905be96aab8cb4e53c3729d3c1c046b018513d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:15:39.913 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:39 smithi062 bash[76519]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k 2024-03-09T00:15:40.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.k.service' 2024-03-09T00:15:40.175 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:39 smithi062 podman[76533]: 2024-03-09 00:15:39.916039883 +0000 UTC m=+0.528671175 container remove d0630ecc7c86da24f20a87c2905be96aab8cb4e53c3729d3c1c046b018513d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0) 2024-03-09T00:15:40.175 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:40 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.k.service: Deactivated successfully. 2024-03-09T00:15:40.175 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:40 smithi062 systemd[1]: Stopped Ceph mds.k for 8c5a3016-dda5-11ee-95c6-87774f69a715. 2024-03-09T00:15:40.175 INFO:journalctl@ceph.mds.k.smithi062.stdout:Mar 09 00:15:40 smithi062 systemd[1]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.k.service: Consumed 1.135s CPU time. 2024-03-09T00:15:40.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:40.186 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-09T00:15:40.186 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-09T00:15:40.186 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.c 2024-03-09T00:15:40.472 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:15:40 smithi192 systemd[1]: Stopping Ceph mds.c for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:40.883 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:15:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c[67511]: 2024-03-09T00:15:40.473+0000 7f46e5c29700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:40.883 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:15:40 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c[67511]: 2024-03-09T00:15:40.473+0000 7f46e5c29700 -1 mds.c *** got signal Terminated *** 2024-03-09T00:15:44.788 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:15:44 smithi192 podman[76096]: 2024-03-09 00:15:44.48392514 +0000 UTC m=+4.103842296 container died 3fca2d1aea84d31d0ba1a4b182ca30be18e30805c2c4dd0ce26eb5c2b853d15e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-09T00:15:44.788 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:15:44 smithi192 podman[76096]: 2024-03-09 00:15:44.790606073 +0000 UTC m=+4.410523225 container cleanup 3fca2d1aea84d31d0ba1a4b182ca30be18e30805c2c4dd0ce26eb5c2b853d15e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-09T00:15:45.039 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 09 00:15:44 smithi192 bash[76096]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-c 2024-03-09T00:15:45.163 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.c.service' 2024-03-09T00:15:45.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:45.202 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-09T00:15:45.202 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-09T00:15:45.202 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.f 2024-03-09T00:15:45.383 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:15:45 smithi192 systemd[1]: Stopping Ceph mds.f for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:45.883 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:15:45 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f[67202]: 2024-03-09T00:15:45.518+0000 7f23d5fd6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:45.883 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:15:45 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f[67202]: 2024-03-09T00:15:45.518+0000 7f23d5fd6700 -1 mds.f *** got signal Terminated *** 2024-03-09T00:15:49.567 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:15:49 smithi192 podman[76231]: 2024-03-09 00:15:49.279904589 +0000 UTC m=+3.859693215 container died f9a3eec91580d83d5dbd19e3ed94563bc02d8adc42984978c41eef0615eadf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-09T00:15:49.883 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:15:49 smithi192 podman[76231]: 2024-03-09 00:15:49.660504444 +0000 UTC m=+4.240293061 container cleanup f9a3eec91580d83d5dbd19e3ed94563bc02d8adc42984978c41eef0615eadf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-09T00:15:49.883 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 09 00:15:49 smithi192 bash[76231]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-f 2024-03-09T00:15:50.077 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.f.service' 2024-03-09T00:15:50.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:50.113 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-09T00:15:50.113 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-09T00:15:50.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.i 2024-03-09T00:15:50.633 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:15:50 smithi192 systemd[1]: Stopping Ceph mds.i for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:50.634 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:15:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: 2024-03-09T00:15:50.438+0000 7fdd89c51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:50.634 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:15:50 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i[67823]: 2024-03-09T00:15:50.438+0000 7fdd89c51700 -1 mds.i *** got signal Terminated *** 2024-03-09T00:15:53.578 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:15:53 smithi192 podman[76353]: 2024-03-09 00:15:53.255512622 +0000 UTC m=+2.920207138 container died 60a42d821339e8b1ce3bda27dcbd3a16830afc0788578e8d235d2bd4a1d06575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux ) 2024-03-09T00:15:53.578 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:15:53 smithi192 podman[76353]: 2024-03-09 00:15:53.580735775 +0000 UTC m=+3.245430292 container cleanup 60a42d821339e8b1ce3bda27dcbd3a16830afc0788578e8d235d2bd4a1d06575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482) 2024-03-09T00:15:53.883 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:15:53 smithi192 bash[76353]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i 2024-03-09T00:15:53.883 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 09 00:15:53 smithi192 podman[76367]: 2024-03-09 00:15:53.782077512 +0000 UTC m=+0.523312972 container remove 60a42d821339e8b1ce3bda27dcbd3a16830afc0788578e8d235d2bd4a1d06575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-i, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-09T00:15:53.986 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.i.service' 2024-03-09T00:15:54.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:54.021 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-09T00:15:54.022 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-09T00:15:54.022 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.l 2024-03-09T00:15:54.332 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 09 00:15:54 smithi192 systemd[1]: Stopping Ceph mds.l for 8c5a3016-dda5-11ee-95c6-87774f69a715... 2024-03-09T00:15:54.633 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 09 00:15:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l[66876]: 2024-03-09T00:15:54.334+0000 7f7a16b91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-09T00:15:54.634 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 09 00:15:54 smithi192 ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l[66876]: 2024-03-09T00:15:54.334+0000 7f7a16b91700 -1 mds.l *** got signal Terminated *** 2024-03-09T00:15:56.633 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 09 00:15:56 smithi192 podman[76475]: 2024-03-09 00:15:56.383693181 +0000 UTC m=+2.146097213 container died 7f6a1d705b91e9193cf60eec538054032089502963576afb27c42df82d937c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-09T00:15:57.061 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 09 00:15:56 smithi192 podman[76475]: 2024-03-09 00:15:56.667938551 +0000 UTC m=+2.430342584 container cleanup 7f6a1d705b91e9193cf60eec538054032089502963576afb27c42df82d937c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-09T00:15:57.061 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 09 00:15:56 smithi192 bash[76475]: ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l 2024-03-09T00:15:57.061 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 09 00:15:56 smithi192 podman[76490]: 2024-03-09 00:15:56.860410658 +0000 UTC m=+0.480229295 container remove 7f6a1d705b91e9193cf60eec538054032089502963576afb27c42df82d937c59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-8c5a3016-dda5-11ee-95c6-87774f69a715-mds-l, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-09T00:15:57.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c5a3016-dda5-11ee-95c6-87774f69a715@mds.l.service' 2024-03-09T00:15:57.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-09T00:15:57.117 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-09T00:15:57.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-09T00:15:57.316 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-09T00:15:59.690 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-09T00:15:59.899 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-09T00:16:02.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-09T00:16:02.799 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-09T00:16:05.407 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-09T00:16:05.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-09T00:16:05.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-09T00:16:05.512 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-09T00:16:05.514 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi022/crash 2024-03-09T00:16:05.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash -- . 2024-03-09T00:16:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-09T00:16:05.563 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2024-03-09T00:16:05.565 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi062/crash 2024-03-09T00:16:05.566 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash -- . 2024-03-09T00:16:05.610 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-09T00:16:05.610 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-03-09T00:16:05.612 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi192/crash 2024-03-09T00:16:05.613 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash -- . 2024-03-09T00:16:05.654 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-09T00:16:05.655 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-03-09T00:16:05.656 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-09T00:16:05.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-09T00:16:05.694 INFO:tasks.cephadm:Compressing logs... 2024-03-09T00:16:05.694 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-09T00:16:05.737 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-09T00:16:05.739 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-09T00:16:05.763 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-09T00:16:05.763 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-09T00:16:05.763 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.6.log 2024-03-09T00:16:05.764 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.a.log 2024-03-09T00:16:05.765 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-09T00:16:05.765 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.g.log 2024-03-09T00:16:05.765 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.j.log 2024-03-09T00:16:05.767 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log: /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.3.log 2024-03-09T00:16:05.767 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.x.log 2024-03-09T00:16:05.767 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.0.log 2024-03-09T00:16:05.768 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.x.log: 90.0% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-09T00:16:05.771 INFO:teuthology.orchestra.run.smithi022.stderr: 94.7% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.g.log.gz 2024-03-09T00:16:05.771 INFO:teuthology.orchestra.run.smithi022.stderr: 94.6% 93.3% -- replaced with /var/log/ceph/cephadm.log.gz -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.j.log.gz 2024-03-09T00:16:05.771 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-09T00:16:05.774 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-09T00:16:05.775 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log 2024-03-09T00:16:05.775 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-09T00:16:05.775 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log: 83.8% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-09T00:16:05.775 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.9.log 2024-03-09T00:16:05.775 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.d.log 2024-03-09T00:16:05.777 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.9.log: /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log: 86.9% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-09T00:16:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-09T00:16:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-09T00:16:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.10.log 2024-03-09T00:16:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-09T00:16:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.b.log 2024-03-09T00:16:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.4.log 2024-03-09T00:16:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.1.log 2024-03-09T00:16:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.k.log 2024-03-09T00:16:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.h.log 2024-03-09T00:16:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.7.log 2024-03-09T00:16:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.e.log 2024-03-09T00:16:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.7.log: --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.y.log 2024-03-09T00:16:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-09T00:16:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log 2024-03-09T00:16:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log: 2024-03-09T00:16:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.b.log 2024-03-09T00:16:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.a.log 2024-03-09T00:16:05.787 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log: /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.b.log: 94.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.k.log.gz 2024-03-09T00:16:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-09T00:16:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-09T00:16:05.791 INFO:teuthology.orchestra.run.smithi062.stderr: 92.2% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.y.log.gz 2024-03-09T00:16:05.793 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.d.log.gz 2024-03-09T00:16:05.795 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-03-09T00:16:05.795 INFO:teuthology.orchestra.run.smithi192.stderr: /var/log/ceph/cephadm.log 2024-03-09T00:16:05.796 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.l.log 2024-03-09T00:16:05.802 INFO:teuthology.orchestra.run.smithi192.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-09T00:16:05.802 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.i.log 2024-03-09T00:16:05.802 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.11.log 2024-03-09T00:16:05.803 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.z.log 2024-03-09T00:16:05.803 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.8.log 2024-03-09T00:16:05.803 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.5.log 2024-03-09T00:16:05.803 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.f.log 2024-03-09T00:16:05.803 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-09T00:16:05.803 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log 2024-03-09T00:16:05.804 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.2.log 2024-03-09T00:16:05.804 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.c.log 2024-03-09T00:16:05.804 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-09T00:16:05.804 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.c.log 2024-03-09T00:16:05.804 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log: /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.c.log: 94.6% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.l.log.gz 2024-03-09T00:16:05.805 INFO:teuthology.orchestra.run.smithi192.stderr: 92.3% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.z.log.gz 2024-03-09T00:16:05.805 INFO:teuthology.orchestra.run.smithi022.stderr: 94.6% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.a.log.gz 2024-03-09T00:16:05.805 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-09T00:16:05.817 INFO:teuthology.orchestra.run.smithi062.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-09T00:16:05.899 INFO:teuthology.orchestra.run.smithi192.stderr: 83.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-09T00:16:06.004 INFO:teuthology.orchestra.run.smithi062.stderr: 83.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-09T00:16:06.006 INFO:teuthology.orchestra.run.smithi062.stderr: 90.1% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-09T00:16:06.008 INFO:teuthology.orchestra.run.smithi062.stderr: 86.9% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-09T00:16:06.010 INFO:teuthology.orchestra.run.smithi192.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-09T00:16:06.012 INFO:teuthology.orchestra.run.smithi192.stderr: 86.9% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-09T00:16:06.050 INFO:teuthology.orchestra.run.smithi192.stderr: 90.2% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-09T00:16:06.125 INFO:teuthology.orchestra.run.smithi192.stderr: 91.0% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-09T00:16:06.129 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-09T00:16:06.182 INFO:teuthology.orchestra.run.smithi062.stderr: 94.7% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.e.log.gz 2024-03-09T00:16:06.218 INFO:teuthology.orchestra.run.smithi022.stderr: 90.0% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mgr.x.log.gz 2024-03-09T00:16:06.462 INFO:teuthology.orchestra.run.smithi062.stderr: 91.9% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.b.log.gz 2024-03-09T00:16:08.650 INFO:teuthology.orchestra.run.smithi022.stderr: 91.6% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.a.log.gz 2024-03-09T00:16:15.940 INFO:teuthology.orchestra.run.smithi192.stderr: 94.7% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.f.log.gz 2024-03-09T00:16:22.891 INFO:teuthology.orchestra.run.smithi192.stderr: 91.9% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mon.c.log.gz 2024-03-09T00:16:23.726 INFO:teuthology.orchestra.run.smithi062.stderr: 95.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.b.log.gz 2024-03-09T00:16:24.402 INFO:teuthology.orchestra.run.smithi062.stderr: 94.9% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.h.log.gz 2024-03-09T00:16:46.263 INFO:teuthology.orchestra.run.smithi062.stderr: 91.7% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.1.log.gz 2024-03-09T00:16:48.348 INFO:teuthology.orchestra.run.smithi022.stderr: 91.7% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.0.log.gz 2024-03-09T00:16:52.629 INFO:teuthology.orchestra.run.smithi062.stderr: 91.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.10.log.gz 2024-03-09T00:16:56.136 INFO:teuthology.orchestra.run.smithi022.stderr: 91.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.3.log.gz 2024-03-09T00:16:57.469 INFO:teuthology.orchestra.run.smithi022.stderr: 91.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.6.log.gz 2024-03-09T00:17:02.754 INFO:teuthology.orchestra.run.smithi022.stderr: 91.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.9.log.gz 2024-03-09T00:17:02.756 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-03-09T00:17:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m57.006s 2024-03-09T00:17:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:user 3m18.758s 2024-03-09T00:17:02.756 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m5.732s 2024-03-09T00:17:03.531 INFO:teuthology.orchestra.run.smithi062.stderr: 91.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.7.log.gz 2024-03-09T00:17:15.821 INFO:teuthology.orchestra.run.smithi062.stderr: 91.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.4.log.gz 2024-03-09T00:17:15.823 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-09T00:17:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:real 1m10.068s 2024-03-09T00:17:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:user 3m48.406s 2024-03-09T00:17:15.823 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m6.624s 2024-03-09T00:19:52.088 INFO:teuthology.orchestra.run.smithi192.stderr: 95.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.c.log.gz 2024-03-09T00:21:04.165 INFO:teuthology.orchestra.run.smithi192.stderr: 91.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.8.log.gz 2024-03-09T00:21:05.449 INFO:teuthology.orchestra.run.smithi192.stderr: 91.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.11.log.gz 2024-03-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi192.stderr: 91.5% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.2.log.gz 2024-03-09T00:25:09.455 INFO:teuthology.orchestra.run.smithi192.stderr: 91.2% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-osd.5.log.gz 2024-03-09T00:30:50.163 INFO:teuthology.orchestra.run.smithi192.stderr: 93.4% -- replaced with /var/log/ceph/8c5a3016-dda5-11ee-95c6-87774f69a715/ceph-mds.i.log.gz 2024-03-09T00:30:50.249 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-03-09T00:30:50.250 INFO:teuthology.orchestra.run.smithi192.stderr:real 14m44.493s 2024-03-09T00:30:50.250 INFO:teuthology.orchestra.run.smithi192.stderr:user 9m22.076s 2024-03-09T00:30:50.250 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m23.169s 2024-03-09T00:30:50.250 INFO:tasks.cephadm:Archiving logs... 2024-03-09T00:30:50.251 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi022/log 2024-03-09T00:30:50.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-09T00:31:24.447 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi062/log 2024-03-09T00:31:24.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-09T00:31:59.930 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi192/log 2024-03-09T00:31:59.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-09T00:33:32.585 INFO:tasks.cephadm:Removing cluster... 2024-03-09T00:33:32.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 --force 2024-03-09T00:33:32.785 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-09T00:33:33.354 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 --force 2024-03-09T00:33:33.560 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-09T00:33:34.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c5a3016-dda5-11ee-95c6-87774f69a715 --force 2024-03-09T00:33:34.532 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 8c5a3016-dda5-11ee-95c6-87774f69a715 2024-03-09T00:33:35.248 INFO:tasks.cephadm:Removing cephadm ... 2024-03-09T00:33:35.248 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-09T00:33:35.267 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-09T00:33:35.285 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-09T00:33:35.301 INFO:tasks.cephadm:Teardown complete 2024-03-09T00:33:35.301 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-09T00:33:35.315 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-09T00:33:35.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-09T00:33:35.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-09T00:33:35.329 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-09T00:33:35.456 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-09T00:33:35.468 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-09T00:33:35.468 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-09T00:33:35.471 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-09T00:33:35.473 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-09T00:33:35.485 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-03-09T00:33:35.487 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-03-09T00:33:35.488 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-03-09T00:33:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-09T00:33:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-03-09T00:33:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 -67us[ -67us] +/- 147ms 2024-03-09T00:33:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 +40us[ +52us] +/- 53ms 2024-03-09T00:33:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -30us[ -30us] +/- 66ms 2024-03-09T00:33:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T00:33:35.526 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-09T00:33:35.527 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-03-09T00:33:35.527 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 -70us[ -55us] +/- 147ms 2024-03-09T00:33:35.527 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 +28us[ +43us] +/- 53ms 2024-03-09T00:33:35.527 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +16us[ +31us] +/- 66ms 2024-03-09T00:33:35.527 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T00:33:35.584 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-09T00:33:35.584 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-03-09T00:33:35.584 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 -82us[ -70us] +/- 147ms 2024-03-09T00:33:35.584 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 +33us[ +45us] +/- 53ms 2024-03-09T00:33:35.585 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -19us[-7379ns] +/- 66ms 2024-03-09T00:33:35.585 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T00:33:35.586 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-09T00:33:35.597 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-09T00:33:35.597 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-09T00:33:35.607 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-09T00:33:35.702 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-09T00:33:35.797 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-09T00:33:35.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-09T00:33:35.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T00:33:35.951 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-09T00:33:35.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T00:33:35.987 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-09T00:33:36.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T00:33:36.018 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-09T00:33:36.030 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-09T00:33:36.041 INFO:teuthology.task.internal:Duration was 3550.258525 seconds 2024-03-09T00:33:36.042 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-09T00:33:36.052 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-09T00:33:36.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-09T00:33:36.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-09T00:33:36.058 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-09T00:33:36.099 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-09T00:33:36.117 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-09T00:33:36.129 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-09T00:33:36.465 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-09T00:33:36.466 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-03-09T00:33:36.466 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-09T00:33:36.488 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-03-09T00:33:36.488 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-09T00:33:36.515 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-03-09T00:33:36.515 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-09T00:33:36.537 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-09T00:33:36.537 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-09T00:33:36.540 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-09T00:33:36.557 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-09T00:33:36.617 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-09T00:33:36.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-09T00:33:36.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-09T00:33:36.631 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-09T00:33:37.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-09T00:33:37.039 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-09T00:33:37.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-09T00:33:37.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-09T00:33:37.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-09T00:33:37.161 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-09T00:33:37.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-09T00:33:37.175 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-09T00:33:37.178 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-09T00:33:37.198 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-03-09T00:33:37.216 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-03-09T00:33:37.217 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-03-09T00:33:37.251 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-09T00:33:37.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T00:33:37.267 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-09T00:33:37.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T00:33:37.306 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-09T00:33:37.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T00:33:37.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-09T00:33:37.332 INFO:teuthology.task.internal:Transferring archived files... 2024-03-09T00:33:37.333 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi022 2024-03-09T00:33:37.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-09T00:33:37.378 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi062 2024-03-09T00:33:37.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-09T00:33:37.420 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315/remote/smithi192 2024-03-09T00:33:37.421 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-09T00:33:37.459 INFO:teuthology.task.internal:Removing archive directory... 2024-03-09T00:33:37.460 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-09T00:33:37.462 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-09T00:33:37.464 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-09T00:33:37.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-09T00:33:37.523 INFO:teuthology.task.internal:Not uploading archives. 2024-03-09T00:33:37.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-09T00:33:37.534 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-09T00:33:37.534 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-09T00:33:37.537 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-09T00:33:37.539 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-09T00:33:37.548 INFO:teuthology.orchestra.run.smithi022.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 9 00:33 /home/ubuntu/cephtest 2024-03-09T00:33:37.551 INFO:teuthology.orchestra.run.smithi062.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 9 00:33 /home/ubuntu/cephtest 2024-03-09T00:33:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 9 00:33 /home/ubuntu/cephtest 2024-03-09T00:33:37.564 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-09T00:33:37.576 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-09T00:33:37.602 INFO:teuthology.nuke:Checking targets against current locks 2024-03-09T00:33:37.629 DEBUG:teuthology.nuke:shortname: smithi022 2024-03-09T00:33:37.630 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-09T00:33:37.656 DEBUG:teuthology.nuke:shortname: smithi062 2024-03-09T00:33:37.657 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-09T00:33:37.661 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 23:24:19.797767', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKeRhSrqUtiBx0tIulRULBWf1qe+zwxyeRi5klsKSZkdMBEZs/XtFC9X3HHMlugG1+oRPCHTjsiwCVgWNxMqb2A='} 2024-03-09T00:33:37.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 23:24:19.794913', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFg0qYPoLRelSD2a2DmIYAAnA9i6+e1DVoyu9tiDKDKrY7RKGwofNxjRi5QJqKNR5JuKkk+rDs8fXTcjBwAo1W8='} 2024-03-09T00:33:37.692 DEBUG:teuthology.nuke:shortname: smithi192 2024-03-09T00:33:37.692 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-09T00:33:37.712 INFO:teuthology.orchestra.console.smithi022:Power off 2024-03-09T00:33:37.713 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T00:33:37.725 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 23:24:19.796982', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH4CG47NywenjtTdSmMJsbctV7eNJoUg0vtQhLUDv+tYVoGdahAsLLd1ncxmLzSVY1cYMZ8hjEzaehZk1EU3ZI8='} 2024-03-09T00:33:37.739 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-03-09T00:33:37.739 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:37.755 INFO:teuthology.orchestra.console.smithi062:Power off 2024-03-09T00:33:37.756 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T00:33:37.771 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-03-09T00:33:37.779 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-03-09T00:33:37.780 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:37.800 INFO:teuthology.orchestra.console.smithi192:Power off 2024-03-09T00:33:37.801 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T00:33:37.810 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-03-09T00:33:37.823 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-03-09T00:33:37.823 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:37.846 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-03-09T00:33:41.773 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:41.812 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:41.848 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:41.885 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-03-09T00:33:41.922 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-03-09T00:33:41.958 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-03-09T00:33:45.890 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:45.922 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:45.959 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:46.001 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-03-09T00:33:46.034 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-03-09T00:33:46.069 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-03-09T00:33:46.102 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-03-09T00:33:46.135 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-03-09T00:33:50.073 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T00:33:50.186 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-03-09T00:33:50.287 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-03-09T00:33:50.398 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3550.2585253715515 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi022 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c633aec26d6e4cef9d88206a3079132d status: fail success: false 2024-03-09T00:33:50.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T00:33:50.495 INFO:teuthology.run:FAIL